Skip to content

Instantly share code, notes, and snippets.

@jhw

jhw/.BERT_SAGEMAKER.md

Last active May 20, 2024 12:12
Show Gist options
  • Save jhw/0ea1fc20bfa1334fc70ad2513dcdd61e to your computer and use it in GitHub Desktop.
Save jhw/0ea1fc20bfa1334fc70ad2513dcdd61e to your computer and use it in GitHub Desktop.
Download ZIP
Bert Word Alignment Sagemaker model
Raw
.gitignore
*.pyc
__pycache__
env
tmp
Raw
delete_stack.sh
#!/usr/bin/env bash
aws cloudformation delete-stack --stack-name $STACK_NAME
Raw
deploy_stack.sh
#!/usr/bin/env bash
aws cloudformation deploy --stack-name $STACK_NAME --template-file stack.yaml --capabilities CAPABILITY_NAMED_IAM --parameter-overrides RepoName=$REPO_NAME
Raw
list_events.sh
#!/usr/bin/env bash
aws cloudformation describe-stack-events --stack-name $STACK_NAME --query "StackEvents[].{\"1.Timestamp\":Timestamp,\"2.Id\":LogicalResourceId,\"3.Type\":ResourceType,\"4.Status\":ResourceStatus,\"5.Reason\":ResourceStatusReason}"
Raw
list_images.sh
#!/usr/bin/env bash
aws ecr list-images --repository-name "$REPO_NAME" --query 'imageIds[*]' --output table
Raw
list_outputs.sh
#!/usr/bin/env bash
aws cloudformation describe-stacks --stack-name $STACK_NAME --query 'Stacks[0].Outputs' --output table
Raw
list_resources.sh
#!/usr/bin/env bash
aws cloudformation describe-stack-resources --stack-name $STACK_NAME --query "StackResources[].{\"1.Timestamp\":Timestamp,\"2.LogicalId\":LogicalResourceId,\"3.PhysicalId\":PhysicalResourceId,\"4.Type\":ResourceType,\"5.Status\":ResourceStatus}"
Raw
list_stacks.sh
#!/usr/bin/env bash
aws cloudformation describe-stacks --query "Stacks[].{\"1.Name\":StackName,\"2.Status\":StackStatus}"
Raw
NOTES.md

logs 19/05/24

  • sagemaker -> inference -> endpoints -> [click endpoint name] -> logs
Raw
ping_app.py
import boto3, json, os, sys
def fetch_outputs(cf, stackname):
outputs = {}
for stack in cf.describe_stacks()["Stacks"]:
if (stack["StackName"].startswith(stackname) and
"Outputs" in stack):
for output in stack["Outputs"]:
outputs[output["OutputKey"]] = output["OutputValue"]
return outputs
def translate_text(text,
source_language_code = "ru",
target_language_code = "en"):
translate = boto3.client('translate')
result = translate.translate_text(
Text=text,
SourceLanguageCode=source_language_code,
TargetLanguageCode=target_language_code
)
return result['TranslatedText']
def columnise(text, n = 48):
return text[:n] if len(text) > n else text+" ".join(["" for i in range(n-len(text))])
if __name__ == "__main__":
try:
if "STACK_NAME" not in os.environ:
raise RuntimeError("STACK_NAME not defined")
stack_name = os.environ["STACK_NAME"]
if len(sys.argv) < 2:
raise RuntimeError("please enter Russian text")
src = sys.argv[1]
tgt = translate_text(src)
struct = {
"src": src,
"tgt": tgt,
}
cf, sagemaker = (boto3.client("cloudformation"),
boto3.client('sagemaker-runtime'))
outputs = fetch_outputs(cf, stack_name)
endpoint_name = outputs["SageMakerEndpointName"]
response = sagemaker.invoke_endpoint(
EndpointName = endpoint_name,
ContentType = 'application/json',
Accept = 'application/json',
Body = json.dumps(struct)
)
pairs = json.loads(response['Body'].read().decode())
for text, translation in pairs:
print (f"{columnise(text)}\t{columnise(translation)}")
except RuntimeError as error:
print ("Error: %s" % str(error))
Raw
requirements.txt
awscli
boto3
botocore
Raw
setenv.sh
#!/usr/bin/env bash
export AWS_PROFILE=woldeploy
export AWS_DEFAULT_OUTPUT=table
export AWS_REGION=eu-west-1
export AWS_ACCOUNT_ID=119552584133
export REPO_NAME=bert-word-alignment
export STACK_NAME=bwa-model-stack
Raw
stack.yaml
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
RepoName:
Type: String
ImageTag:
Type: String
Default: latest
ModelMode:
Type: String
Default: SingleModel
InstanceType:
Type: String
Default: 'ml.m5.large'
Resources:
SageMakerExecutionRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- 'sagemaker.amazonaws.com'
Action: 'sts:AssumeRole'
Policies:
- PolicyName: 'SageMakerFullAccessPolicy'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 'sagemaker:*'
- 'logs:*'
- 's3:*'
- 'ecr:*'
Resource: '*'
SageMakerModel:
Type: 'AWS::SageMaker::Model'
Properties:
ModelName: !Sub "${RepoName}-model"
ExecutionRoleArn: !GetAtt SageMakerExecutionRole.Arn
PrimaryContainer:
Image: !Sub '${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${RepoName}:${ImageTag}'
Mode: !Ref ModelMode
Environment: {}
SageMakerEndpointConfig:
Type: 'AWS::SageMaker::EndpointConfig'
Properties:
ProductionVariants:
- VariantName: 'AllTraffic'
ModelName: !GetAtt SageMakerModel.ModelName
InitialInstanceCount: 1
InstanceType: !Ref InstanceType
InitialVariantWeight: 1.0
SageMakerEndpoint:
Type: 'AWS::SageMaker::Endpoint'
Properties:
EndpointConfigName: !GetAtt SageMakerEndpointConfig.EndpointConfigName
Outputs:
SageMakerModelName:
Value: !GetAtt SageMakerModel.ModelName
SageMakerEndpointConfigName:
Value: !GetAtt SageMakerEndpointConfig.EndpointConfigName
SageMakerEndpointName:
Value: !GetAtt SageMakerEndpoint.EndpointName
Raw
TODO.md

short

done

  • change stack to output names
  • ping_app to lookup endpoint name
  • test removing depends
  • test removing hardcoded names
  • direct integration/invocation test
  • web url test
  • check model/endpoint still deploys
 (rapid) exec '/var/runtime/bootstrap' (cwd=/app, handler=)
  • list images
  • deploy stack
  • delete stack
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment