andy pai's tils

Configuring Amazon S3 for Use with Cloudflare DNS

Today I wanted to setup an S3 bucket to upload some static assets for images, javascript files and other random resources. I wanted the contents of the S3 bucket to be accessible with a custom domain (e.g. static.example.com). The process was a bit more confusing than I had imagined.

Create an S3 bucket

I originally thought I could name the S3 bucket whatever I liked but there is a strict convention that must be followed for the routing to work as expected.

If you want the S3 bucket to be accessible at static.example.com, the S3 bucket must be called static.example.com. All of this is possible to do with the AWS management console but I prefer using the aws CLI and the commands below were helpful for experimentation.

$ aws s3 mb s3://static.example.com --region us-west-1

Explanation:

• aws s3 is the AWS command-line interface for interacting with Amazon S3.
  • mb stands for "make bucket" and is used to create an S3 bucket.
  • s3://static.example.com specifies the name and location of the bucket to be created.
  • --region us-west-1 specifies the AWS region where the bucket will be created.

$ aws s3 sync s3://test.example.com s3://static.example.com

Explanation:

• aws s3 sync is used to synchronize the contents of two Amazon S3 buckets.
  • s3://test.example.com specifies the source S3 bucket named test.example.com.
  • s3://static.example.com specifies the destination S3 bucket named static.example.com.

$ aws s3 rb --force s3://test.example.com

Explanation:

• aws s3 rb is used to remove an S3 bucket.
  • --force flag is used to force the removal of the bucket even if it's not empty.
  • s3://test.example.com specifies the S3 bucket to remove.

Once the S3 bucket is set up, you need to update the bucket configuration to host a static website.

Update permissions

Under the Permissions tab for the bucket, you need to open public access under Block public access (bucket settings):

cf aws s3 public access

Upload a test document

Upload a test document to ensure files in the bucket can be accessed by anyone on the internet without additional authentication. I uploaded the US Constitution and got back this Object URL: https://s3.amazonaws.com/static.andypai.me/us_constitution.pdf

cf aws s3 object url

Update bucket policies to only allow Cloudflare IP addresses

To ensure that the bucket only responds to requests coming from the Cloudflare proxy, we need to set up permissions to allow Cloudflare to access the bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::www.example.com/*",
            "Condition": {
                "NotIpAddress": {
                    "aws:SourceIp": [
                      "103.21.244.0/22",
                      "103.22.200.0/22",
                      "103.31.4.0/22",
                      "104.16.0.0/13",
                      "104.24.0.0/14",
                      "108.162.192.0/18",
                      "131.0.72.0/22",
                      "141.101.64.0/18",
                      "162.158.0.0/15",
                      "172.64.0.0/13",
                      "173.245.48.0/20",
                      "188.114.96.0/20",
                      "190.93.240.0/20",
                      "197.234.240.0/22",
                      "198.41.128.0/17",
                      "2400:cb00::/32",
                      "2606:4700::/32",
                      "2803:f800::/32",
                      "2405:b500::/32",
                      "2405:8100::/32",
                      "2a06:98c0::/29",
                      "2c0f:f248::/32"
                    ]
                }
            }
        }
    ]
}

Note, after this policy is updated, the URL https://s3.amazonaws.com/static.andypai.me/us_constitution.pdf will no longer be valid.

<Error>
  <script/>
  <Code>AccessDenied</Code>
  <Message>Access Denied</Message>
  <RequestId>PRYMQBAM90BTZPZ7</RequestId>
  <HostId>/EswA6oLxWXWfx0LalRXG9naHnpPNk+ZziuzIpj3CYENCz3FwSNGcl32PZiLtXRUVadrz5+Y1SA=</HostId>
</Error>

Update Cloudflare DNS

Next, you need to create a new CNAME record for the subdomain in the Cloudflare dashboard which follows the format [subdomain].s3.amazonaws.com:

cf aws s3 cname target

I found this somewhat confusing because although the URL in S3 appeared as s3.amazonaws.com/static.andypai.me, turns out, the file can also be accessed from static.andypai.me.s3.amazonaws.com/us_constitution.pdf.

Once the CNAME DNS record is setup, the file can be accessed directly without the .s3.amazonaws.com suffix.

cf aws s3 us constituion

If all configurations are correct, attempting to access the file from https://s3.amazonaws.com/static.andypai.me/us_constitution.pdf will result in access denial.

Helpful Resources