Skip to content

Instantly share code, notes, and snippets.

@Gershon-A

Gershon-A/elasticcache.yaml

Created November 28, 2022 07:49
Show Gist options
  • Save Gershon-A/511ebf770139648f2270d3313e6731f6 to your computer and use it in GitHub Desktop.
Save Gershon-A/511ebf770139648f2270d3313e6731f6 to your computer and use it in GitHub Desktop.
Download ZIP
AWS Cloud Formation template for high availability (MultiAZ) ElasticCache Redis cluster, scheduled automatic backup, special user and user group for authorization, Encryption in transit (tls required), disk encryption
Raw
elasticcache.yaml
AWSTemplateFormatVersion: "2010-09-09"
Description: Setup ElasticCache.
Parameters:
PrimaryAvailabilityZone:
Description: Primary Availability Zone
Type: String
Default: "us-east-1a"
ReplicaAvailabilityZone1:
Description: Replica Availability Zone. We need it since we have more than 1 replicas.
Type: String
Default: "us-east-1b"
ReplicaAvailabilityZone2:
Description: Replica Availability Zone.We need it since we have more than 1 replicas.
Type: String
Default: "us-east-1c"
CacheEngine:
Type: String
Default: "redis"
CacheEngineVersion:
Type: String
Default: "7.0"
RedisPort:
Type: Number
Default: "6379"
usecase:
Description: What environment should be deployed
Type: String
AllowedValues:
- "prod"
- "stage"
- "dev"
Default: prod
SubnetIda:
Description: Choose which subnets the ElasticCache should be deployed to (select EKS PrivateA subnet)
Type: AWS::EC2::Subnet::Id
SubnetIdb:
Description: Choose which subnets the ElasticCache should be deployed to (select EKS PrivateB subnet)
Type: AWS::EC2::Subnet::Id
SubnetIdc:
Description: Choose which subnets the ElasticCache should be deployed to (select EKS PrivateC subnet)
Type: AWS::EC2::Subnet::Id
SecurityGroup:
Description: Select the Security Group to use for the ElasticCache cluster. (select EKS Security Group)
Type: AWS::EC2::SecurityGroup::Id
Mappings:
ElasticCacheNode:
CacheNodeType:
prod: cache.t3.medium
stage: cache.t3.small
dev: cache.t3.micro
ElasticCacheReplicaCount:
ReplicaCount:
prod: "2"
stage: "2"
dev: "2"
Resources:
CacheKMS:
Type: "AWS::KMS::Key"
Properties:
Description: Symmetric encryption KMS key for ElasticCache.
EnableKeyRotation: true
PendingWindowInDays: 20
KeyPolicy:
Version: 2012-10-17
Id: key-default-1
Statement:
- Sid: Enable IAM User Permissions
Effect: Allow
Principal:
AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
Action: "kms:*"
Resource: "*"
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}
- Key: Service
Value: ElastiCache
- Key: Env
Value: !Ref usecase
- Key: Region
Value: !Ref "AWS::Region"
CacheKMSAlias:
Type: "AWS::KMS::Alias"
Properties:
AliasName: !Sub alias/${AWS::StackName}-cache-${usecase}
TargetKeyId: !Ref CacheKMS
# Subnet group to control where the ElastiCache (Redis) gets placed
CacheSubnetGroup:
Type: AWS::ElastiCache::SubnetGroup
DependsOn: ElastiCacheLogGroup
Properties:
CacheSubnetGroupName: !Sub "${AWS::StackName}-subnetgroup"
Description: !Sub "${AWS::StackName}-ElastiCacheSubnetGroup"
SubnetIds:
- !Ref "SubnetIda"
- !Ref "SubnetIdb"
- !Ref "SubnetIdc"
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}
- Key: Service
Value: ElastiCache
- Key: Env
Value: !Ref usecase
- Key: Region
Value: !Ref "AWS::Region"
# The ElastiCache itself.
ReplicationGroup:
Type: AWS::ElastiCache::ReplicationGroup
# UpdateReplacePolicy: Retain
# DependsOn: "ParameterGroup"
Properties:
AutoMinorVersionUpgrade: true
CacheNodeType: !FindInMap [ElasticCacheNode, CacheNodeType, !Ref usecase]
CacheParameterGroupName: !Ref ParameterGroup
CacheSubnetGroupName: !Ref CacheSubnetGroup
Engine: !Ref CacheEngine
EngineVersion: !Ref CacheEngineVersion
MultiAZEnabled: true
TransitEncryptionEnabled: true
# Backup
PreferredMaintenanceWindow: sun:23:00-mon:01:30
SnapshotRetentionLimit: 7
SnapshotWindow: "03:30-05:30"
# used to encrypt the disk on the cluster
KmsKeyId: !GetAtt CacheKMS.KeyId
LogDeliveryConfigurations:
- DestinationDetails:
CloudWatchLogsDetails:
LogGroup: !Ref ElastiCacheLogGroup
DestinationType: cloudwatch-logs
LogFormat: json
LogType: engine-log
NodeGroupConfiguration:
- PrimaryAvailabilityZone: !Sub "${PrimaryAvailabilityZone}"
ReplicaAvailabilityZones:
- !Sub "${ReplicaAvailabilityZone1}"
- !Sub "${ReplicaAvailabilityZone2}"
ReplicaCount:
!FindInMap [ElasticCacheReplicaCount, ReplicaCount, !Ref usecase]
Port: !Ref RedisPort
ReplicationGroupDescription: !Sub "${AWS::StackName}-${usecase}-ElastiCache-ReplicationGroup"
ReplicationGroupId: !Sub "${AWS::StackName}-replicationgroup"
UserGroupIds:
- !Ref CacheUserGroup
SecurityGroupIds:
- !Ref SecurityGroup
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}
- Key: Service
Value: ElastiCache
- Key: Env
Value: !Ref usecase
- Key: Region
Value: !Ref "AWS::Region"
# The ElastiCache dedicated ParameterGroup
ParameterGroup:
Type: "AWS::ElastiCache::ParameterGroup"
DeletionPolicy: Retain
UpdateReplacePolicy: Retain
Properties:
CacheParameterGroupFamily: redis7
Description: !Sub "${AWS::StackName}-ParameterGroup"
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}
- Key: Service
Value: ElastiCache
- Key: Env
Value: !Ref usecase
- Key: Region
Value: !Ref "AWS::Region"
# Redis Engine logs
ElastiCacheLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub "/aws/redis/instance/${AWS::StackName}-${usecase}"
RetentionInDays: 7
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}
- Key: Service
Value: ElastiCache
- Key: Env
Value: !Ref usecase
- Key: Region
Value: !Ref "AWS::Region"
# ------------------------------------------------------------#
# Store ElastiCache User password in SecretsManager
# ------------------------------------------------------------#
CacheUserSecret:
Type: AWS::SecretsManager::Secret
Properties:
# Elastic cache authentication
Name: !Sub ${AWS::StackName}-${usecase}
Description: "Secret with dynamically generated password."
GenerateSecretString:
SecretStringTemplate: !Sub '{"username": "${AWS::StackName}-${usecase}"}'
GenerateStringKey: "password"
PasswordLength: 16
ExcludeCharacters: '"@/\&'
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}
- Key: Service
Value: ElastiCache
- Key: Env
Value: !Ref usecase
- Key: Region
Value: !Ref "AWS::Region"
# ------------------------------------------------------------#
# ElastiCache Users
# ------------------------------------------------------------#
ElastiCacheDefaultUser:
Type: AWS::ElastiCache::User
Properties:
AccessString: "off ~keys* -@all +get" # Since we must have user named "default", we create it but disabling access for it.
Engine: redis
NoPasswordRequired: false
Passwords:
- !Sub "{{resolve:secretsmanager:${CacheUserSecret}::password}}"
UserId: !Sub ${AWS::StackName}-default
UserName: "default"
# We creating the real "Admin" user
ElastiCacheUser:
Type: AWS::ElastiCache::User
DependsOn: ElastiCacheDefaultUser
Properties:
AccessString: "on ~* +@all" # Admin access. Allowing all.
Engine: redis
NoPasswordRequired: false
Passwords:
- !Sub "{{resolve:secretsmanager:${CacheUserSecret}::password}}"
UserId: !Sub "${AWS::StackName}-${usecase}"
UserName: !Sub "{{resolve:secretsmanager:${CacheUserSecret}::username}}"
CacheUserGroup:
DependsOn: ElastiCacheUser
Type: AWS::ElastiCache::UserGroup
Properties:
Engine: redis
UserGroupId: !Sub ${AWS::StackName}-${usecase}
UserIds:
- !Sub "${AWS::StackName}-default"
- !Sub "${AWS::StackName}-${usecase}"
# ------------------------------------------------------------#
# SSM Parameter Exports
# ------------------------------------------------------------#
parameterPrimaryEndpointAddress:
Type: AWS::SSM::Parameter
Properties:
Description: !Sub "${AWS::StackName} ${usecase} ElastiCache PrimaryEndPoint"
Name: !Sub "/${usecase}/infrastructure/redis-PrimaryEndPoint"
Type: String
Value: !GetAtt ReplicationGroup.PrimaryEndPoint.Address
Tags:
Name: !Sub ${AWS::StackName}
Service: ElastiCache
Env: !Ref usecase
Region: !Ref "AWS::Region"
parameterReaderEndPointAddress:
Type: AWS::SSM::Parameter
Properties:
Description: !Sub "${AWS::StackName} ${usecase} ElastiCache ReaderEndPoint"
Name: !Sub "/${usecase}/infrastructure/redis-ReaderEndPoint"
Type: String
Value: !GetAtt ReplicationGroup.ReaderEndPoint.Address
Tags:
Name: !Sub ${AWS::StackName}
Service: ElastiCache
Env: !Ref usecase
Region: !Ref "AWS::Region"
Outputs:
CachePrimaryEndpointAddress:
Value: !GetAtt ReplicationGroup.PrimaryEndPoint.Address
CacheReaderEndPointAddress:
Value: !GetAtt ReplicationGroup.ReaderEndPoint.Address
@Gershon-A
Copy link
Author

Gershon-A commented Nov 28, 2022 •
edited
Loading

AWS ElasticCache Cloudformation
This CloudFormation template will setup High availability (MultiAZ) ElasticCache Redis cluster, scheduled automatic backup, special user and user group.
One node will be used as the primary and two nodes as replica.
AWS Secret Manager will be used for store credentials for ElasticCache users and will be retrieved later by pod running on EKS cluster.
AWS Parameter Store will be used for store ElasticCache endpoints and will be retrieved later by pod running on the EKS cluster.
Aws KMS will be used for disk encryption.
ElasticCache subnet groups will be using the same private subnet groups as the EKS cluster.

  • Connection string Example:
redis-cli -h master.replicationgroup.xf6ddw.use1.cache.amazonaws.com -c --tls --user [Usernasme] --pass [Password] -p 6379

(to pass SSL verification , add --insecure)

  • Subnet Group
    Specify two or more subnets in the SubnetIds property.
    Note that if ElastiCache is multi-AZ, the AZs of the subnets specified here must be separated.
  CacheSubnetGroup:
    Type: AWS::ElastiCache::SubnetGroup
    DependsOn: ElastiCacheLogGroup
    Properties:
      CacheSubnetGroupName: !Sub '${AWS::StackName}-subnetgroup'
      Description: !Sub '${AWS::StackName}-ElastiCacheSubnetGroup'
      SubnetIds:
        - !Ref 'SubnetIda'
        - !Ref 'SubnetIdb'
        - !Ref 'SubnetIdc'
  CacheUserSecret:
    Type: AWS::SecretsManager::Secret
    Properties:
      # Elastic cache authentication
      Name: !Sub ${AWS::StackName}-${usecase}
      Description: "Secret with dynamically generated password."
      GenerateSecretString:
        SecretStringTemplate: !Sub '{"username": "${AWS::StackName}-${usecase}"}'
        GenerateStringKey: "password"
        PasswordLength: 16
        ExcludeCharacters: '"@/\&'

Then, we have to create ElastiCache default user with different ID and disabling it:

  ElastiCacheDefaultUser:
    Type: AWS::ElastiCache::User
    Properties:
      AccessString: "off ~keys* -@all +get" # Since we must have user named "default", we create it but disabling access for it.
      Engine: redis
      NoPasswordRequired: false
      Passwords:
        - !Sub "{{resolve:secretsmanager:${CacheUserSecret}::password}}"
      UserId: !Sub ${AWS::StackName}-default
      UserName: "default"

The real user with full access will looks like:

  ElastiCacheUser:
    Type: AWS::ElastiCache::User
    DependsOn: ElastiCacheDefaultUser
    Properties:
      AccessString: "on ~* +@all" # Admin access. Allowing all.
      Engine: redis
      NoPasswordRequired: false
      Passwords:
        - !Sub "{{resolve:secretsmanager:${CacheUserSecret}::password}}"
      UserId: !Sub "{{resolve:secretsmanager:${CacheUserSecret}::username}}"
      UserName: !Sub "{{resolve:secretsmanager:${CacheUserSecret}::username}}"

And finally, we have to attach both users to the group:

  CacheUserGroup:
    DependsOn: ElastiCacheUser
    Type: AWS::ElastiCache::UserGroup
    Properties:
      Engine: redis
      UserGroupId: !Sub ${AWS::StackName}-${usecase}
      UserIds:
        - !Sub ${AWS::StackName}-default
        - !Sub "{{resolve:secretsmanager:${CacheUserSecret}::username}}"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment