Mike Slinn
Mike Slinn

Command-Line AWS Utilities

Published 2021-03-22.

This article is categorized under AWS.

Here are some command-line utilities I have written for AWS. They are dependent on aws cli. You can download all of these utilities in tar format. Extract them into the current directory like this:

Shell
$ tar xf mslinn_aws.tar

awsCfInvalidate

Given a CloudFront distribution ID, invalidate the distribution.

#!/bin/bash

function help {
  printf "$1$(basename $0) - Invalidate the CloudFront distribution for the given ID.
If no distribution with the given ID exists, the empty string is returned and the return code is 2.
A message is printed asynchronously to the console when the invalidation completes.

Syntax: $(basename $0) distId

Syntax: awsCfS3Dist www.mslinn.com | $(basename $0)
"
  exit 1
}

function waitForInvalidation {
  echo "Waiting for invalidation $2 to complete."
  aws cloudfront wait invalidation-completed \
    --distribution-id "$1" \
    --id "$2"
  echo "Invalidation $2 has completed."
}


if [ "$1" == -h ]; then help; fi

if [ "$1" ]; then
  DIST_ID="$1"
  shift
elif [ ! -t 0 ]; then
  read -r DIST_ID
fi
if [ -z "$DIST_ID" ]; then help 'Error: No CloudFront distribution ID was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

JSON="$( aws cloudfront create-invalidation \
  --distribution-id "$DIST_ID" \
  --paths "/*"
)"

INVALIDATION_ID="$( jq -r .Invalidation.Id <<< "$JSON" )"
waitForInvalidation "$DIST_ID" "$INVALIDATION_ID" &

Example usages:

Shell
$ awsCfInvalidate E2P5S6OYKQNB6B
Waiting for invalidation IFOPKECU4YYHD to complete. 

... do other things ... 

$ Invalidation IFOPKECU4YYHD has completed. 
Shell
$ awsCfS3Dist www.mslinn.com | awsCfInvalidate
Waiting for invalidation IFOPKECU4YYHD to complete. 

... do other things ... 

$ Invalidation IFOPKECU4YYHD has completed. 

awsCfS3Dist

Given an S3 bucket name, return the CloudFront distribution JSON.

#!/bin/bash

function help {
  printf "$1$(basename $0) - Obtain the CloudFront distribution JSON for an S3 bucket.
If no S3 bucket with the given name exists, the empty string is returned and the return code is 2.

Syntax: $(basename $0) bucketName

Syntax: echo bucketName | $(basename $0)
"
  exit 1
}

if [ "$1" == -h ]; then help; fi

if [ "$1" ]; then
  BUCKET_NAME="$1"
  shift
elif [ ! -t 0 ]; then
  read -r BUCKET_NAME
fi
if [ -z "$BUCKET_NAME" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

if [ "$( aws s3api head-bucket --bucket $BUCKET_NAME 2> >(grep -i 'Not Found') )" ]; then
  >&2 echo "Error: Bucket $BUCKET_NAME does not exist."
  exit 2
fi

DIST_ID="$( awsCfS3DistId "$BUCKET_NAME" )"

if [ -z "$DIST_ID" ]; then exit 2; fi

aws cloudfront get-distribution-config --id "$DIST_ID"

Example usages:

Shell
$ awsCfS3Dist www.mslinn.com
{
  "ETag": "E1DIZUSLMOLXKP",
  "DistributionConfig": {
      "CallerReference": "1454487160038",
      "Aliases": {
          "Quantity": 2,
          "Items": [
              "www.mslinn.com",
              "mslinn.com"
          ]
      },
      "DefaultRootObject": "index.html",
      "Origins": {
          "Quantity": 1,
          "Items": [
              {
                  "Id": "S3-www.mslinn.com",
                  "DomainName": "www.mslinn.com.s3-website-us-east-1.amazonaws.com",
                  "OriginPath": "",
                  "CustomHeaders": {
                      "Quantity": 0
                  },
                  "CustomOriginConfig": {
                      "HTTPPort": 80,
                      "HTTPSPort": 443,
                      "OriginProtocolPolicy": "http-only",
                      "OriginSslProtocols": {
                          "Quantity": 3,
                          "Items": [
                              "TLSv1",
                              "TLSv1.1",
                              "TLSv1.2"
                          ]
                      },
                      "OriginReadTimeout": 30,
                      "OriginKeepaliveTimeout": 5
                  },
                  "ConnectionAttempts": 3,
                  "ConnectionTimeout": 10
              }
          ]
      },
      "OriginGroups": {
          "Quantity": 0
      },
      "DefaultCacheBehavior": {
          "TargetOriginId": "S3-www.mslinn.com",
          "TrustedSigners": {
              "Enabled": false,
              "Quantity": 0
          },
          "ViewerProtocolPolicy": "redirect-to-https",
          "AllowedMethods": {
              "Quantity": 2,
              "Items": [
                  "HEAD",
                  "GET"
              ],
              "CachedMethods": {
                  "Quantity": 2,
                  "Items": [
                      "HEAD",
                      "GET"
                  ]
              }
          },
          "SmoothStreaming": false,
          "Compress": true,
          "LambdaFunctionAssociations": {
              "Quantity": 0
          },
          "FieldLevelEncryptionId": "",
          "CachePolicyId": "658327ea-f89d-4fab-a63d-7e88639e58f6"
      },
      "CacheBehaviors": {
          "Quantity": 0
      },
      "CustomErrorResponses": {
          "Quantity": 2,
          "Items": [
              {
                  "ErrorCode": 403,
                  "ResponsePagePath": "",
                  "ResponseCode": "",
                  "ErrorCachingMinTTL": 60
              },
              {
                  "ErrorCode": 404,
                  "ResponsePagePath": "",
                  "ResponseCode": "",
                  "ErrorCachingMinTTL": 60
              }
          ]
      },
      "Comment": "",
      "Logging": {
          "Enabled": false,
          "IncludeCookies": false,
          "Bucket": "",
          "Prefix": ""
      },
      "PriceClass": "PriceClass_All",
      "Enabled": true,
      "ViewerCertificate": {
          "ACMCertificateArn": "arn:aws:acm:us-east-1:031372724784:certificate/2be42926-829c-4db9-be7d-a72e951256d4",
          "SSLSupportMethod": "sni-only",
          "MinimumProtocolVersion": "TLSv1",
          "Certificate": "arn:aws:acm:us-east-1:031372724784:certificate/2be42926-829c-4db9-be7d-a72e951256d4",
          "CertificateSource": "acm"
      },
      "Restrictions": {
          "GeoRestriction": {
              "RestrictionType": "none",
              "Quantity": 0
          }
      },
      "WebACLId": "",
      "HttpVersion": "http1.1",
      "IsIPV6Enabled": false
  }
} 
Shell
$ echo www.mslinn.com | awsCfS3Dist
{
  "ETag": "E1DIZUSLMOLXKP",
  "DistributionConfig": {
      "CallerReference": "1454487160038",
      "Aliases": {
          "Quantity": 2,
          "Items": [
              "www.mslinn.com",
              "mslinn.com"
          ]
      },
      "DefaultRootObject": "index.html",
      "Origins": {
          "Quantity": 1,
          "Items": [
              {
                  "Id": "S3-www.mslinn.com",
                  "DomainName": "www.mslinn.com.s3-website-us-east-1.amazonaws.com",
                  "OriginPath": "",
                  "CustomHeaders": {
                      "Quantity": 0
                  },
                  "CustomOriginConfig": {
                      "HTTPPort": 80,
                      "HTTPSPort": 443,
                      "OriginProtocolPolicy": "http-only",
                      "OriginSslProtocols": {
                          "Quantity": 3,
                          "Items": [
                              "TLSv1",
                              "TLSv1.1",
                              "TLSv1.2"
                          ]
                      },
                      "OriginReadTimeout": 30,
                      "OriginKeepaliveTimeout": 5
                  },
                  "ConnectionAttempts": 3,
                  "ConnectionTimeout": 10
              }
          ]
      },
      "OriginGroups": {
          "Quantity": 0
      },
      "DefaultCacheBehavior": {
          "TargetOriginId": "S3-www.mslinn.com",
          "TrustedSigners": {
              "Enabled": false,
              "Quantity": 0
          },
          "ViewerProtocolPolicy": "redirect-to-https",
          "AllowedMethods": {
              "Quantity": 2,
              "Items": [
                  "HEAD",
                  "GET"
              ],
              "CachedMethods": {
                  "Quantity": 2,
                  "Items": [
                      "HEAD",
                      "GET"
                  ]
              }
          },
          "SmoothStreaming": false,
          "Compress": true,
          "LambdaFunctionAssociations": {
              "Quantity": 0
          },
          "FieldLevelEncryptionId": "",
          "CachePolicyId": "658327ea-f89d-4fab-a63d-7e88639e58f6"
      },
      "CacheBehaviors": {
          "Quantity": 0
      },
      "CustomErrorResponses": {
          "Quantity": 2,
          "Items": [
              {
                  "ErrorCode": 403,
                  "ResponsePagePath": "",
                  "ResponseCode": "",
                  "ErrorCachingMinTTL": 60
              },
              {
                  "ErrorCode": 404,
                  "ResponsePagePath": "",
                  "ResponseCode": "",
                  "ErrorCachingMinTTL": 60
              }
          ]
      },
      "Comment": "",
      "Logging": {
          "Enabled": false,
          "IncludeCookies": false,
          "Bucket": "",
          "Prefix": ""
      },
      "PriceClass": "PriceClass_All",
      "Enabled": true,
      "ViewerCertificate": {
          "ACMCertificateArn": "arn:aws:acm:us-east-1:031372724784:certificate/2be42926-829c-4db9-be7d-a72e951256d4",
          "SSLSupportMethod": "sni-only",
          "MinimumProtocolVersion": "TLSv1",
          "Certificate": "arn:aws:acm:us-east-1:031372724784:certificate/2be42926-829c-4db9-be7d-a72e951256d4",
          "CertificateSource": "acm"
      },
      "Restrictions": {
          "GeoRestriction": {
              "RestrictionType": "none",
              "Quantity": 0
          }
      },
      "WebACLId": "",
      "HttpVersion": "http1.1",
      "IsIPV6Enabled": false
  }
} 

awsCfS3DistId

Given an S3 bucket name, return the CloudFront distribution ID.

#!/bin/bash

function help {
  printf "$1$(basename $0) - Obtain the CloudFront distribution ID for an S3 bucket.
If no S3 bucket with the given name exists, the empty string is returned and the return code is 2.

Syntax: $(basename $0) bucketName

Syntax: echo bucketName | $(basename $0)
"
  exit 1
}

if [ "$1" == -h ]; then help; fi

if [ "$1" ]; then
  BUCKET_NAME="$1"
  shift
elif [ ! -t 0 ]; then
  read -r BUCKET_NAME
fi
if [ -z "$BUCKET_NAME" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

if [ "$( aws s3api head-bucket --bucket $BUCKET_NAME 2> >(grep -i 'Not Found') )" ]; then
  >&2 echo "Error: Bucket $BUCKET_NAME does not exist."
  exit 2
fi

DIST_ID="$(
  aws cloudfront list-distributions \
  --query "DistributionList.Items[*].{id:Id,origin:Origins.Items[0].Id}[?origin=='S3-$BUCKET_NAME'].id" \
  --output text
)"

if [ -z "$DIST_ID" ]; then exit 2; fi
echo "$DIST_ID"

Example usages:

Shell
$ awsCfS3DistId www.mslinn.com
E2P5S6OYKQNB6B 
Shell
$ echo www.mslinn.com | awsCfS3DistId
E2P5S6OYKQNB6B 

awsCfS3MakeDist

Creates a CloudFront distribution for the given bucket name. Returns the new distribution's ID.

#!/bin/bash

function help {
  printf "$1$(basename $0) - Make a new CloudFront distribution for the given S3 bucket name.

Returns the new distribution's ID.

Syntax: $(basename $0) bucketName

Syntax: echo bucketName | $(basename $0)
"
  exit 1
}

function doesDistributionExist {
  DIST_ID="$( awsCfS3Dist "$BUCKET_NAME" )"
  if [ "$DIST_ID" ]; then echo true; fi
}

function createDist {
  read -r -d '' NEW_DIST_JSON <<EOF
{
  "CallerReference": "$BUCKET_NAME",
  "Aliases": {
    "Quantity": 0
  },
  "DefaultRootObject": "index.html",
  "Origins": {
    "Quantity": 1,
    "Items": [
      {
        "Id": "$BUCKET_NAME",
        "DomainName": "$BUCKET_NAME.s3.amazonaws.com",
        "S3OriginConfig": {
          "OriginAccessIdentity": ""
        }
      }
    ]
  },
  "DefaultCacheBehavior": {
    "TargetOriginId": "$BUCKET_NAME",
    "ForwardedValues": {
      "QueryString": true,
      "Cookies": {
        "Forward": "none"
      }
    },
    "TrustedSigners": {
      "Enabled": false,
      "Quantity": 0
    },
    "ViewerProtocolPolicy": "redirect-to-https",
    "MinTTL": 3600
  },
  "CacheBehaviors": {
    "Quantity": 0
  },
  "Comment": "",
  "Logging": {
    "Enabled": false,
    "IncludeCookies": true,
    "Bucket": "",
    "Prefix": ""
  },
  "PriceClass": "PriceClass_All",
  "Enabled": true
}
EOF

  NEW_DIST_RESULT_JSON = "$(
    aws cloudfront create-distribution --distribution-config "$NEW_DIST_JSON"
  )"

  DISTRIBUTION_ID="$( jq -r '.Distribution.Id' <<< "$NEW_DIST_RESULT_JSON" )"

  echo "$DISTRIBUTION_ID"
}


if [ "$1" == -h ]; then help; fi

if [ -t 0 ]; then
  if [ -z "$1" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi
  BUCKET_NAME="$1"
  shift
else
  read -r BUCKET_NAME
fi
if [ -z "$BUCKET_NAME" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

if [ "$( aws s3api head-bucket --bucket $BUCKET_NAME 2> >(grep -i 'Not Found') )" ]; then
  >&2 echo "Error: Bucket $BUCKET_NAME does not exist."
  exit 2
fi

if [ "$(doesDistributionExist)" ]; then
  >&2 echo "Error: a CloudFront distibution already exists for S3 bucket $BUCKET_NAME"
  exit 3
fi

createDist

Example usages:

Shell
$ awsCfS3MakeDist my_bucket
E2P5S6OYKQNB6B 
Shell
$ echo my_bucket | awsCfS3MakeDist
E2P5S6OYKQNB6B 

awsS3Mb

Make a new S3 bucket with the given name in the default AWS region. If the --public-read option is provided, set the ACL to public-read

#!/bin/bash

function help {
  printf "$1$(basename $0) - Make a new S3 bucket with the given name in the default AWS region.

Syntax: $(basename $0) bucketName [OPTIONS]

Syntax: echo bucketName | $(basename $0) [OPTIONS]

Options are:
  --public-read  Set bucket ACL to public-read
"
  exit 1
}

if [ "$1" == -h ]; then help; fi

if [ "$1" == "--public-read" ]; then
  ACL="public-read"
  shift
fi

if [ "$1" ]; then
  BUCKET_NAME="$1"
  shift
elif [ ! -t 0 ]; then
  read -r BUCKET_NAME
fi
if [ -z "$BUCKET_NAME" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

aws s3 mb s3://$BUCKET_NAME

if [ "$ACL" ]; then
  aws s3api put-bucket-acl --bucket $BUCKET_NAME --acl $ACL
fi

Example usages:

Shell
$ awsS3Mb my_bucket
Shell
$ awsS3Mb my_bucket --public-read
Shell
$ echo my_bucket | awsS3Mb --public-read

awsS3Website

Enable an S3 bucket to be a website.

#!/bin/bash

function help {
  printf "$1$(basename $0) - Enable an S3 bucket to be a website.

Syntax: $(basename $0) bucketName

Syntax: echo bucketName | $(basename $0)
"
  exit 1
}

if [ "$1" == -h ]; then help; fi

if [ "$1" ]; then
  BUCKET_NAME="$1"
  shift
elif [ ! -t 0 ]; then
  read -r BUCKET_NAME
fi
if [ -z "$BUCKET_NAME" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

aws s3 website s3://$BUCKET_NAME \
  --index-document index.html \
  --error-document error.html

Example usages:

Shell
$ awsS3Website my_bucket
Shell
$ echo my_bucket | awsS3Website