Skip to content

Instantly share code, notes, and snippets.

@nicolasdao

nicolasdao/aws_cloudformation_templates.md

Last active March 18, 2024 06:41
Show Gist options
  • Save nicolasdao/e99be20b68dc680a7df58377add41c51 to your computer and use it in GitHub Desktop.
Save nicolasdao/e99be20b68dc680a7df58377add41c51 to your computer and use it in GitHub Desktop.
Download ZIP
AWS CloudFormation templates. Keywords: aws cloudformation template
Raw
aws_cloudformation_templates.md

AWS CLOUDFORMATION TEMPLATES

Table of contents

DynamoDB

The following sample shows how to provision a new DynamoDB called my_new_table with a composite primary key made of a partition key (aka hash) and a sort key (aka range).

Resources:
  NumberTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: my_new_table
      AttributeDefinitions:
        - AttributeName: device_id
          AttributeType: N
        - AttributeName: timestamp
          AttributeType: S
      KeySchema:
        - AttributeName: device_id
          KeyType: HASH
        - AttributeName: timestamp
          KeyType: RANGE
      # StreamSpecification: # Enable stream. Doc: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-dynamodb-table-streamspecification.html
      #  StreamViewType: NEW_IMAGE
      ProvisionedThroughput:
        ReadCapacityUnits: 1
        WriteCapacityUnits: 1

EC2

Basic template

AWSTemplateFormatVersion: '2010-09-09'

Description:
  "Your web server description"

Parameters:
  Environment:
    Description: 'The operating environment'
    Type: String
    Default: test
    AllowedValues: [ test, prod ]
  NetworkStack:
    Description: 'Stack name of network stack.'
    Type: String

Resources:
  WebServerSG:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow RDP, HTTP and HTTPS connections 
      VpcId: 
        Fn::ImportValue: !Sub "${NetworkStack}-Vpc-VPC"
      SecurityGroupIngress:
        - Description: RDP IPv4
          IpProtocol: tcp
          FromPort: 3389
          ToPort: 3389
          CidrIp: 0.0.0.0/0
        - Description: HTTP IPv4
          IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
        - Description: HTTP IPv6
          IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIpv6: ::/0
        - Description: HTTPS IPv4
          IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: 0.0.0.0/0
        - Description: HTTPS IPv6
          IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIpv6: ::/0
  WebServer:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: t2.small
      ImageId: ami-07b03c86b9a8923fc
      KeyName: apk
      NetworkInterfaces:
        - AssociatePublicIpAddress: "true"
          DeviceIndex: "0"
          GroupSet: 
            - !Ref WebServerSG
          SubnetId: 
            Fn::ImportValue: !Sub "${NetworkStack}-Vpc-SubnetBPublic"
      Tags:
        - Key: Name
          Value: your-web-server-name

NOTICE: The tag name is used to actually name your server. If you don't add that tag, the server's name will be blank in the AWS console.

Elastic load balancer

Official doc at https://docs.aws.amazon.com/elasticloadbalancing/latest/application/application-load-balancers.html

There are two generations of load balancers:

  1. The legacy classic load balancer AWS::ElasticLoadBalancing::LoadBalancer.
  2. The latest load balancer AWS::ElasticLoadBalancingV2::LoadBalancer which can either be configured in application or network type.

To learn more about the ELB and learn how to choose the right load balancer, please refer to the ELB section in the Annex.

Classic load balancer

Doc coming soon...

Application load balancer

PREREQUISITE: This section assumes that your domain is maintained in AWS Route 53 in the same AWS account that provisions all the following resources.

This section demos how to provision an EC2 with two web APIs (one on port 8090 and the other on port 8095) behind an application load balancer over SSL that uses pathname and FQDN to determine how to route traffic. It requires a minimum of 12 components (yes 12!!! This is another great example of how over-complicated CloudFormation is):

  1. Security - SSL cert using AWS Certificate Manager.
  2. Security - Security group for the load balancer with ingress rules to allow traffic on port 80 and 443.
  3. Security - Security group for the EC2 instance with ingress rules to allow access from the load balancer's security group.
  4. EC2 - EC2 instance with the security group created in the previous step.
  5. ELB - Application load balancer with the security group created in step 2.
  6. ELB - Target group 01 containing the EC2 instance and where to route the traffic on that instance (e.g., specific port).
  7. ELB - Target group 02 containing the EC2 instance and where to route the traffic on that instance (e.g., specific port).
  8. ELB - HTTPS listener attached to the application load balancer, with a default target group, and configured with the SSL cert created in step 1 to capture HTTPS requests.
  9. ELB - HTTP listener attached to the application load balancer to capture HTTP requests and redirect them to HTTPS.
  10. ELB - Listener rule 01 attached to the HTTPS listener and configured with the rule to send traffic to a specific target group (e.g., target group 01).
  11. ELB - Listener rule 02 attached to the HTTPS listener and configured with the rule to send traffic to a specific target group (e.g., target group 02).
  12. DNS - A records in AWS Route 53 to configure the domain to the application load balancer using an alias.

Step 1 - Security

Create an SSL cert

The following snippet creates an SSL cert for two domains:

  • api.myapp.com
  • api.myapp.net

WARNINGS:

  • This resource will prevent your stack deployment to finish until you MANUALLY complete the DNS challenge in Route 53. The record details can be found by login into the AWS console and selecting the Certificate Manager. You should see the certificate in a Pending validation mode. Expand it to see the record details to set up in your DNS (i.e., Route 53).
  • The DomainValidationOptions must contain one item per domain. In the example below, there are three FQDN using two domains (myapp.com and myapp.net).
  • Though the SubjectAlternativeNames is optional, I've experienced issues when it is not specified. If you only have one FQDN to setup, add the domain name under SubjectAlternativeNames.
Resources:
  LoadBalancerSSLCert:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: api.myapp.com
      DomainValidationOptions:
        - DomainName: api.myapp.com
          ValidationDomain: myapp.com
        - DomainName: api.myapp.net
          ValidationDomain: myapp.net
      ValidationMethod: DNS
      SubjectAlternativeNames:
        - api.myapp.net
        - api.v1.myapp.net

Create two security groups

Those two security groups allows the load balancer to receive traffic from the internet and allows the EC2 instance to receive traffic from the load balancer.

Resources:
  LoadBalancerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allows HTTP and HTTPS connections 
      SecurityGroupIngress:
        - Description: HTTP IPv4
          IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
        - Description: HTTP IPv6
          IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIpv6: ::/0
        - Description: HTTPS IPv4
          IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: 0.0.0.0/0
        - Description: HTTPS IPv6
          IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIpv6: ::/0
  WebServerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow SSH and traffic from load balancer 
      SecurityGroupIngress:
        - Description: SSH traffic
          IpProtocol: ssh
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
        - Description: Allow traffic from load balancer
          IpProtocol: tcp
          FromPort: 8083
          ToPort: 8085
          SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup

Step 2 - EC2

Refer to the previous EC2 section. Use the security group created above to configure it.

Step 3 - Load balancer

Create an application load balancer

WARNING: The name of the LB must be alphanumerical, less than 32 characters and contain no spaces.

Resources:
  LoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties: 
      Name: your-lb-name
      Type: application
      IpAddressType: ipv4 # Valid values are: dualstack, ipv4. 'dualstack' means IPv4 and IPv4. 
      LoadBalancerAttributes: 
        - Key: routing.http2.enabled
          Value: true
      Scheme: internet-facing
      SecurityGroups: 
        - !Ref LoadBalancerSecurityGroup

Create target groups

WARNING: Do not set up the Name property. It will prevent to update that resource later.

Resources:
  Api01Target:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Protocol: HTTP
      Port: 80
      TargetType: instance
      Targets:
        - Id: !Ref WebServer
          Port: 8090
      HealthCheckEnabled: true
      HealthCheckProtocol: HTTP
      HealthCheckPath: /
      Matcher:
        HttpCode: 200
  Api02Target:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Protocol: HTTP
      Port: 80
      TargetType: instance
      Targets:
        - Id: !Ref WebServer
          Port: 8095
      HealthCheckEnabled: true
      HealthCheckProtocol: HTTP
      HealthCheckPath: /
      Matcher:
        HttpCode: 200

Create HTTP/S listeners

Contrary to what one may think, the application load balancer:

  • Does not listen to traffic, it just route it.
  • Is not configured with an SSL cert.

Those two concerns, listening to traffic and securing it with SSL are managed by a listener. The application load balancer needs at least one listener. The listener can be configured in different modes:

  • forward: That's the most common. It forwards the traffic, most likely the load balancer.
  • redirect: The most common use case for a redirect is to upgrade all HTTP requests to HTTPS.
  • authenticate-cognito
  • authenticate-oidc
  • fixed-response

A listener must define at least one DefaultActions. If that no Listener rules are created, that default action (example forward traffic to target group) is used.

The next configuration defines an HTTPS listener attached to the application load balancer configured with the SSL cert provisioned earlier and with a default action that forwards the traffic to the Api01Target. The second listener redirects all HTTP requests to HTTPS.

Resources:
  HTTPSListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      Certificates:
        - CertificateArn: !Ref LoadBalancerSSLCert
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref Api01Target
      LoadBalancerArn: !Ref LoadBalancer
      Port: 443
      Protocol: HTTPS
  HTTPlistener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
      - Type: redirect
        RedirectConfig:
          Protocol: HTTPS
          Port: '443'
          Host: '#{host}'
          Path: /#{path}
          Query: '#{query}'
          StatusCode: HTTP_301
      LoadBalancerArn: !Ref LoadBalancer
      Port: 80
      Protocol: HTTP

Create listeners rules

Listener rules are what allows to fine tune the routing in the listeners using the layer 7 of the OSI model. The next two listener rules are configured as follow:

  • Api01ListenerRule: If the FQDN is api.myapp.net or api.v1.myapp.net, then route the traffic to Api01Target.
  • Api02ListenerRule: If the request's path starts with /v2, then route the traffic to Api02Target.
Resources:
  Api01ListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties: 
      Actions: 
        - Type: forward
          TargetGroupArn: !Ref Api01Target
      Conditions:
        - Field: host-header
          HostHeaderConfig:
            Values:
              - api.myapp.net
              - api.v1.myapp.net
      ListenerArn: !Ref HTTPSListener
      Priority: 1
  Api02ListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties: 
      Actions: 
        - Type: forward
          TargetGroupArn: !Ref Api02Target
      Conditions:
        - Field: path-pattern
          PathPatternConfig:
            Values:
              - /v2
      ListenerArn: !Ref HTTPSListener
      Priority: 2

Step 4 - Configure the custom domain in Route 53

In step 1, we created an SSL certs for three FQDN (api.myapp.com, api.myapp.net, api.v1.myapp.net). In step 3, we created an HTTPS listener and configured it with that SSL cert so that the application load balancer can support HTTPS. In this step, we're going to configure the DNS in Route 53 to configure three custom domain for the aplication load balancer. This is done by adding three A records with an alias to the application load balancer.

Resources:
  Api01ARecord:
    Type: AWS::Route53::RecordSet
    Properties: 
      HostedZoneId: !FindInMap [EnvironmentMap, !Ref Environment, HostedZoneId]
      Type: A
      Name: api.myapp.com
      AliasTarget:
        DNSName: 
          Fn::GetAtt:
            - LoadBalancer
            - DNSName
        HostedZoneId: 
          Fn::GetAtt:
            - LoadBalancer
            - CanonicalHostedZoneID
  Api02ARecord:
    Type: AWS::Route53::RecordSet
    Properties: 
      HostedZoneId: !FindInMap [EnvironmentMap, !Ref Environment, HostedZoneId]
      Type: A
      Name: api.myapp.net
      AliasTarget:
        DNSName: 
          Fn::GetAtt:
            - LoadBalancer
            - DNSName
        HostedZoneId: 
          Fn::GetAtt:
            - LoadBalancer
            - CanonicalHostedZoneID
  Api03ARecord:
    Type: AWS::Route53::RecordSet
    Properties: 
      HostedZoneId: !FindInMap [EnvironmentMap, !Ref Environment, HostedZoneId]
      Type: A
      Name: api.v1.myapp.net
      AliasTarget:
        DNSName: 
          Fn::GetAtt:
            - LoadBalancer
            - DNSName
        HostedZoneId: 
          Fn::GetAtt:
            - LoadBalancer
            - CanonicalHostedZoneID

Network load balancer

Doc coming soon...

Security Group

Basic security groups

Resources:
  PublicHTTPSIPv4InboundRule:
    Type: AWS::EC2::SecurityGroupIngress
    Properties: 
      Description: Public HTTPS IPv4
      IpProtocol: tcp
      FromPort: 443
      ToPort: 443
      CidrIp: 0.0.0.0/0
      
  PublicHTTPSIPv6InboundRule:
    Type: AWS::EC2::SecurityGroupIngress
    Properties: 
      Description: Public HTTPS IPv6
      IpProtocol: tcp
      FromPort: 443
      ToPort: 443
      CidrIp: ::/0
      
  PrivateConnInboundRule:
    Type: AWS::EC2::SecurityGroupIngress
    Properties: 
      Description: Private TCP connection with EC2 on ports going from 8087 to 8090
      IpProtocol: tcp
      FromPort: 8087
      ToPort: 8090
      SourceSecurityGroupId: !Ref SomeEC2SecurityGroup

Adding a new rule to an existing security group

The following template shows how to grant a web server access to a DB. The process consists in adding a new ingress rule to the existing DB security group (DBSecurityGroup) that allows TCP access on port 1433 from the web server's security group (WebServerSecurityGroup).

Resources:
  NewInboundRule:
    Type: AWS::EC2::SecurityGroupIngress
    Properties: 
      Description: Allows access from Web Server to RDS database
      IpProtocol: tcp
      FromPort: 1433
      ToPort: 1433
      SourceSecurityGroupId: !Ref WebServerSecurityGroup
      GroupId: !Ref DBSecurityGroup

NOTE:

  • The value of !Ref WebServerSecurityGroup looks like sg-08437156bd7291234
  • The value of !Ref DBSecurityGroup looks like sg-0c258ew21bcbadef1

RDS

Aurora

service: your-service-name

custom:
  default-vpc: vpc-12345
  stage: staging
  db-pwd: 123456789
  db-name: your-db-name

provider:
  name: aws
  profile: hillridge
  runtime: nodejs8.10
  stage: test
  region: ap-southeast-2
  versionFunctions: false

resources:
  Resources:
    RDSRole:
      Type: AWS::IAM::Role
      Properties: 
        RoleName: hillridge-${self:custom.stage}-rds-role
        Policies:
          - PolicyName: LambdaInvocationPolicy
            PolicyDocument:
              Version: "2012-10-17"
              Statement: 
                - Effect: "Allow"
                  Action: 
                    - "lambda:InvokeFunction"
                    - "lambda:InvokeAsync"
                  Resource: "*"
        AssumeRolePolicyDocument:
          Version: "2012-10-17"
          Statement: 
            - Effect: "Allow"
              Principal:
                Service: "rds.amazonaws.com"
              Action: "sts:AssumeRole"

    MySQLParameterGroup:
      Type: "AWS::RDS::DBClusterParameterGroup"
      Properties: 
        Description: "Paramater group that set the aws_default_lambda_role key"
        Family: aurora-mysql5.7
        Parameters:
          aws_default_lambda_role: !GetAtt RDSRole.Arn
          max_allowed_packet: 10000000

    MySQLWildcardSecurityGroup:
      Type: AWS::EC2::SecurityGroup
      Properties:
        GroupName: hillridge-${self:custom.stage}-mysql-wildcard-security-group
        GroupDescription: This group allows inbound and outbound acccess from and to any IP
        SecurityGroupIngress:
          - CidrIp: 0.0.0.0/0
            IpProtocol: tcp
            FromPort: 3306
            ToPort: 3306
        SecurityGroupEgress:
          - CidrIp: 0.0.0.0/0
            IpProtocol: -1
        VpcId: ${self:custom.default-vpc}

    DatabaseCluster:
      Type: AWS::RDS::DBCluster
      Properties:
        DBClusterIdentifier: ${self:custom.db-name}-${self:custom.stage}-cluster
        DatabaseName: ${self:custom.db-name}-${self:custom.stage}
        Engine: aurora-mysql
        EngineVersion: 5.7.12
        EngineMode: provisioned
        DBClusterParameterGroupName: !Ref MySQLParameterGroup
        StorageEncrypted: true
        MasterUsername: admin
        MasterUserPassword: {self:custom.db-pwd}
        BackupRetentionPeriod: 7
        PreferredBackupWindow: 01:00-02:00
        PreferredMaintenanceWindow: mon:03:00-mon:04:00
        VpcSecurityGroupIds:
          - !Ref MySQLWildcardSecurityGroup

    DatabasePrimaryInstance:
      Type: AWS::RDS::DBInstance
      Properties:
        DBClusterIdentifier: !Ref DatabaseCluster
        DBInstanceClass: db.t2.small
        DBInstanceIdentifier: ${self:custom.db-name}-${self:custom.stage}-primary
        Engine: aurora-mysql
        PubliclyAccessible: true

Annex

ELB

Elastic Load Balancer comes in three flavors:

  1. Classic: That's the legacy AWS load balancer. It supports UDP, TCP, HTTP/S (via HTTP1) but not WebSocket. It operates at both layer 4 and 7 in the OSI (Open Systems Interconnection).
  2. Application: Default choice for a Web App. It only supports HTTP/S (via HTTP2) and WebSocket. It operates at the layer 7 in the OSI (Open Systems Interconnection).
  3. Network: Use it for system that must provide high-performance even with millions of connections per seconds. It supports only UDP, TCP, TLS. It operates at the layer 4 in the OSI (Open Systems Interconnection).

In most cases, you should use the Application or Network load balancer rather than the Classic. The Classic does not support the following features:

  • WebSocket (only for Application LB)
  • HTTP2 (only for Application LB)
  • Elastic IP (i.e., static IP in the Cloud that guarantees that the service's IP does not change even if the service is reprovisioned). Elastic IP is only available to for Network LBs.

If you have a standard Web App, most likely, you'll use the Application LB. The key features of that LB are:

  • WebSocket
  • HTTP2
  • Layer 7 in the OSI, which means you can route based on the HTTP header.

Reasons to use Application LB over Classic LB:

  • You need HTTP2 or WebSocket.

Reasons to use Application LB over Network LB:

  • You need HTTP/S or WebSocket.
  • You need to route based on the HTTP header.

Readons to use Network LB over Application LB:

  • You need static IP out-of-the-box (if you still need an Application LB with static IP, you need to setup extra services. To learn more about this, please refer to the article titled Using static IP addresses for Application Load Balancers).
  • You need to deal with million of TCP/UDP requests per seconds.
  • You need deal with TCP or UDP.

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment