Tiago Rodrigues

CloudFront on S3 and InvalidViewerCertificate

Tiago Rodrigues -

Continuing our iOS AWSary App project, we want to put a CDN in front of S3 to speed up content deliver as well as reduce S3 costs with cache at edge.

If you try to create CloudFront with terraform you will probably use the same region as the rest of your infrastructure, and if you are not on us-east-1 you will find something strange:

The specified SSL certificate doesn't exist, isn't in us-east-1 region, isn't valid, or doesn't include a valid certificate chain.

Well, to use CloudFront and with a custom domain, you will need to generate a certificate with ACM, and ACM allows you to generate certificates in all regions, but CloudFront and the rest of edge services, they will be deployed globally to all regions, but they are a edge case, they have the main location as us-east-1 US East (N. Virginia) Region.

This will require you to have your ACM certificates generated in us-east-1 as well, and lucly you will find some AWS documentation about this InvalidViewerCertificate error.

First things first, let's create on terraform two aws providers, one that we already had, and a new one with an alias = "us-east-1" that we will use for this specific resources:

provider "aws" {
  region  = local.region
  profile = "tig-awsary"
  default_tags {
    tags = {
      terraform     = "true"
      region        = local.region
      business-unit = "awsary"
      stage         = terraform.workspace
      repository    = "https://github.com/tigpt/AWSary"
    }
  }
}

provider "aws" {
  profile = "tig-awsary"
  alias   = "us-east-1"
  region  = "us-east-1"
  default_tags {
    tags = {
      terraform     = "true"
      region        = "us-east-1"
      business-unit = "awsary"
      stage         = terraform.workspace
      repository    = "https://github.com/tigpt/AWSary"
    }
  }
}

Now we can create our resources on us-east-1 without changing all our resources to this location, by adding this snipped to that resources:

  providers = {
    aws = aws.us-east-1
  }
  • CloudFront module - We create a CloudFront distribution with our custom domain as alias and respective origin, origin_access_identity and it's origin_access_control that will be only S3 for now.
  • Certificates - We need to create TLS certificates for this domain
  • Sub-domain - Then we update our Route53 with the new alias to have a custom domain on CloudFront
  • S3 Policy - We need to update as well permitions to S3 bucket, if you remember, we created it as private bucket and because of that, CloudFront is not able to get object from it, with this new permitions policy we will allow CloudFront to GetObject from S3 to answer requests, and finally attach this policy to the bucket.
module "cloudfront" {
  source = "terraform-aws-modules/cloudfront/aws"
  providers = {
    aws = aws.us-east-1
  }

  aliases = ["cdn.awsary.com"]

  comment             = "CloudFront for cdn.awsary.com"
  enabled             = true
  is_ipv6_enabled     = true
  price_class         = "PriceClass_All"
  retain_on_delete    = false
  wait_for_deployment = false

  create_origin_access_identity = true
  origin_access_identities = {
    s3_bucket_one = "CloudFront for cdn.awsary.com can access"
  }

  create_origin_access_control = true
  origin_access_control = {
    s3_oac = {
      description      = "CloudFront access to S3"
      origin_type      = "s3"
      signing_behavior = "always"
      signing_protocol = "sigv4"
    }
  }

  origin = {
    s3_oac = {
      domain_name           = module.s3_bucket.s3_bucket_bucket_regional_domain_name
      origin_access_control = "s3_oac"
    }
  }

  default_cache_behavior = {
    target_origin_id       = "s3_oac"
    viewer_protocol_policy = "allow-all"

    allowed_methods = ["GET", "HEAD", "OPTIONS"]
    cached_methods  = ["GET", "HEAD"]
    compress        = true
    query_string    = true

    # This is id for SecurityHeadersPolicy copied from https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-managed-response-headers-policies.html
    response_headers_policy_id = "67f7725c-6f97-4210-82d7-5512b31e9d03"
  }


  ordered_cache_behavior = [
    {
      path_pattern           = "*"
      target_origin_id       = "s3_oac"
      viewer_protocol_policy = "redirect-to-https"

      allowed_methods = ["GET", "HEAD", "OPTIONS"]
      cached_methods  = ["GET", "HEAD"]
      compress        = true
      query_string    = true
    }
  ]

  viewer_certificate = {
    acm_certificate_arn = module.acm.acm_certificate_arn
    ssl_support_method  = "sni-only"
  }
}


data "aws_route53_zone" "this" {
  name = local.domain_name
}

module "acm" {
  source  = "terraform-aws-modules/acm/aws"
  version = "~> 4.0"
  providers = {
    aws = aws.us-east-1
  }

  domain_name               = "awsary.com"
  zone_id                   = data.aws_route53_zone.this.id
  subject_alternative_names = ["cdn.awsary.com"]
}


data "aws_iam_policy_document" "s3_policy" {
  # Origin Access Identities
  statement {
    actions   = ["s3:GetObject"]
    resources = ["${module.s3_bucket.s3_bucket_arn}/*"]

    principals {
      type        = "AWS"
      identifiers = module.cloudfront.cloudfront_origin_access_identity_iam_arns
    }
  }

  # Origin Access Controls
  statement {
    actions   = ["s3:GetObject"]
    resources = ["${module.s3_bucket.s3_bucket_arn}/*"]

    principals {
      type        = "Service"
      identifiers = ["cloudfront.amazonaws.com"]
    }

    condition {
      test     = "StringEquals"
      variable = "aws:SourceArn"
      values   = [module.cloudfront.cloudfront_distribution_arn]
    }
  }
}

resource "aws_s3_bucket_policy" "bucket_policy" {
  bucket = module.s3_bucket.s3_bucket_id
  policy = data.aws_iam_policy_document.s3_policy.json
}

After all we can test access to the S3 bucket directly and it's blocked and will give us an 403 Forbidden:

curl -I https://s3-awsary-assets.s3.eu-west-1.amazonaws.com/blog/Arch_Amazon-CloudFront_64%405x.png

HTTP/1.1 403 Forbidden
x-amz-request-id: 4QDE9XSP7R6WWNAE
x-amz-id-2: BbZhTlt2DoKNE+ptB+jA3fszEYIhCk1aT75Ukucvya0bbFkYJLTGtqB/5xRrlfAa7Lpcm0VDrgA=
Content-Type: application/xml
Date: Sun, 26 Feb 2023 22:22:32 GMT
Server: AmazonS3

While accessign the same content from our newly created CloudFront will give us a 200 🥳🎉😅

curl -I https://cdn.awsary.com/blog/Arch_Amazon-CloudFront_64%405x.png

HTTP/2 200
content-type: image/png
content-length: 159035
date: Sun, 26 Feb 2023 22:23:01 GMT
last-modified: Sun, 26 Feb 2023 22:19:18 GMT
etag: "9f169178643a2f885f8697610dc677de"
x-amz-server-side-encryption: AES256
accept-ranges: bytes
server: AmazonS3
x-cache: RefreshHit from cloudfront
via: 1.1 e4fc537726e6de98f17edd9f0158561a.cloudfront.net (CloudFront)
x-amz-cf-pop: LIS50-C1
x-amz-cf-id: 4gkSGILVYL3LXKyDUZ2TIVCRCv0rQPnKjutZsmzZTLoLDpeTGY6guA==

Later on we will add logging and other origins for this CDN. Stay tunned for more on this project or jump in at GitHub to contribute to it: https://github.com/tigpt/AWSary