Skip to content

Instantly share code, notes, and snippets.

@mrseanryan

mrseanryan/create-ec2-gpu-llm.yaml

Last active February 1, 2024 09:39
Show Gist options
  • Save mrseanryan/9c114b206373bb52412e573e8c6b2716 to your computer and use it in GitHub Desktop.
Save mrseanryan/9c114b206373bb52412e573e8c6b2716 to your computer and use it in GitHub Desktop.
Download ZIP
CloudFront script to create an EC2 GPU instance hosting an LLM
Raw
create-ec2-gpu-llm.yaml
AWSTemplateFormatVersion: 2010-09-09
Resources:
LlmEc2KeyPair:
Type: AWS::EC2::KeyPair
Properties:
KeyName: !Join [ "-", [ "llmkeys", !Ref "AWS::StackName" ] ]
LlmSshSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Join [ "-", [ "llmsshsg", !Ref "AWS::StackName"]]
GroupDescription: !Join [ " ", [ "SSH Security Group for", !Ref "AWS::StackName" ] ]
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
LlmHostEc2Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: ami-0bd617643854c7908 # Deep Learning AMI - Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.0.1 (Ubuntu 20.04)
InstanceType: g4dn.xlarge # 16 GB GPU
Tags:
- Key: Name
Value: !Ref "AWS::StackName"
KeyName: !Ref LlmEc2KeyPair
SecurityGroupIds:
- !Ref LlmSshSecurityGroup
BlockDeviceMappings:
- DeviceName: /dev/sda1 # Thus must match the AMI, otherwise CloudFront will create a second volume instead of resizing the root volume!
Ebs:
VolumeType: gp3
VolumeSize: '250'
DeleteOnTermination: 'true'
Encrypted: 'false'
UserData:
"Fn::Base64":
!Sub |
#!/bin/bash -e
cd /home/ubuntu
if [ -f llm-installed-ok.txt ]; then
echo "Previous LLM installation detected - exiting (ok)"
exit 0
fi
echo "=== === Download a Basic NON interactive script to install text-generation-webui to host LLM on Ubuntu"
curl https://gist.githubusercontent.com/mrseanryan/f1f572f8a16c0dd37741bdb232bb4c61/raw > install-text-gen.sh
echo "=== === Download a script to auto-start text-generation-webui by setting up a new Ubuntu service"
curl https://gist.githubusercontent.com/mrseanryan/6bf26b78c6af43fe38d322533c8e10b5/raw > install-llm-service.sh
echo "=== === Install text-generation-webui"
chown ubuntu install-text-gen.sh
sudo -H -u ubuntu chmod +x install-text-gen.sh && sudo -H -u ubuntu ./install-text-gen.sh 16 # change 16 to match the size of the GPU card (see InstanceType)
echo "=== === Setup a custom Ubuntu service"
chmod +x install-llm-service.sh && ./install-llm-service.sh
touch llm-installed-ok.txt
echo "=== === [done]"
Outputs:
1getkey:
Value: !Join [ "", [ "aws ssm get-parameter --name /ec2/keypair/", !GetAtt LlmEc2KeyPair.KeyPairId, " --region eu-west-1 --with-decryption --query 'Parameter.Value' --output text > ", !Ref "AWS::StackName" ,".pem" ] ]
2deletekey:
Value: !Join [ "", [ "aws ssm delete-parameter --name /ec2/keypair/", !GetAtt LlmEc2KeyPair.KeyPairId, " --region eu-west-1" ] ]
3chmod:
Value: !Join [ "", [ "chmod go-rwx ", !Ref "AWS::StackName" ,".pem" ] ]
4ssh:
Value: !Join [ "", [ "ssh -i ", !Ref "AWS::StackName" ,".pem ubuntu@", !GetAtt LlmHostEc2Instance.PublicIp ] ]
5loginstall:
Value: !Join [ "", [ "tail /var/log/cloud-init-output.log -f" ] ]
6instanceid:
Value: !Join [ "", [ "aws ec2 describe-instances --filter Name=key-name,Values=", !Ref LlmEc2KeyPair, " --query Reservations[*].Instances[*].InstanceId | grep 'i-'" ] ]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment