Given the highlights from the Alestic blog Feb-2018, now enhance the Algo VPN Ansible script to start a spot instance vs on demand instance.
Alestic blog highlights reason to enhance algo script
- Can run an instance like you normally do for on-demand instances. One new parameter can make it a Spot instance
- Spot price volatility has been significantly reduced
- No longer have to specify a bid price
- CloudWatch Events can now send a two-minute warning before a Spot instance is interrupted
- IAM policy to merge into existing Algo IAM policy permisions shown below
- ToDo: Edit the Ansible script to launch spot instances instead of standard on demand instances
First get the EC2 instance-id which was created by the Cloud Formation stack
stack_name="YOUR_STACK_NAME"
aws cloudformation list-stack-resources --stack-name=$stack_name > stackout.json
instance_id=$(grep -B0 -A1 EC2Instance stackout.json | grep -o i-.[^\"]*)
instance_name=$(aws ec2 describe-tags --region $region --filters "Name=resource-id,Values=$instance_id" "Name=key,Values=Name" --output text | cut -f5)
echo $instance_id: $instance_name
region="us-east-1"
- Create SNS topic to recieve notices. If the SNS topic and topic arn were created in the past, it is fine to create again with the same
sns_topic_name
as a quick way to know thesns_topic_arn
.
sns_topic_name=instance_state_change
sns_topic_arn=$(aws sns create-topic \
--region "$region" \
--name "$sns_topic_name" \
--output text \
--query 'TopicArn'
)
echo sns_topic_arn=$sns_topic_arn
-
One time task for email subscription and CloudWatch permissions. Skip to next step if this was done in the past.
- Subscribe email to the SNS notification:
email_address="YOUR@EMAIL.ADDRESS" aws sns subscribe \ --region "$region" \ --topic-arn "$sns_topic_arn" \ --protocol email \ --notification-endpoint "$email_address"
-
Check email to confirm that you want to subscribe that email address to the SNS topic.
-
Grant CloudWatch Events permission to post to the SNS topic:
aws sns set-topic-attributes \ --region "$region" \ --topic-arn "$sns_topic_arn" \ --attribute-name Policy \ --attribute-value '{ "Version": "2008-10-17", "Id": "cloudwatch-events-publish-to-sns-'"$sns_topic_name"'", "Statement": [{ "Effect": "Allow", "Principal": { "Service": "events.amazonaws.com" }, "Action": [ "SNS:Publish" ], "Resource": "'"$sns_topic_arn"'" }] }'
-
Create a CloudWatch Events Rule that filters for instance state changes for this specific instance.
- TODO: Modify the ansible stack formation steps to create this rule auotmatically. Ref: awslab example
rule_name_state_change="ec2-state-change-$instance_id"
rule_description_state_change="EC2 instance $instance_id state change"
event_pattern_state_change='{
"source": [
"aws.ec2"
],
"detail-type": [
"EC2 Instance State-change Notification"
],
"detail": {
"instance-id": [ "'"$instance_id"'" ]
}
}'
aws events put-rule \
--region "$region" \
--name "$rule_name_state_change" \
--description "$rule_description_state_change" \
--event-pattern "$event_pattern_state_change" \
--state "ENABLED"
- Set the target of CloudWatch Events rule to the above state_change SNS topic:
sns_target_state_change='[{
"Id": "target-sns-'"$sns_topic_name"'",
"Arn": "'"$sns_topic_arn"'",
"InputTransformer": {
"InputPathsMap": {
"detail-type" : "$.detail-type",
"instance-id" : "$.detail.instance-id",
"resources" : "$.resources",
"state" : "$.detail.state",
"time" : "$.time",
"region" : "$.region"
},
"InputTemplate":
"\"<detail-type>: At <time>, the status of EC2 instance <instance-id> in the AWS Region <region> has changed to <state>. <resources>\""
}
}]'
aws events put-targets \
--region "$region" \
--rule "$rule_name_state_change" \
--targets "$sns_target_state_change"
-
Create SNS topic to recieve notices. If the SNS topic and topic arn were created in the past, it is fine to create again with the same
sns_topic_name
as a quick way to know thesns_topic_arn
.sns_topic_name=spot-activity sns_topic_arn=$(aws sns create-topic \ --region "$region" \ --name "$sns_topic_name" \ --output text \ --query 'TopicArn' ) echo sns_topic_arn=$sns_topic_arn
-
One time task for email subscription and CloudWatch permissions. Skip to next step if this was done in the past.
- Subscribe email to the SNS notification:
email_address="YOUR@EMAIL.ADDRESS" aws sns subscribe \ --region "$region" \ --topic-arn "$sns_topic_arn" \ --protocol email \ --notification-endpoint "$email_address"
-
Check email to confirm that you want to subscribe that email address to the SNS topic.
-
Grant CloudWatch Events permission to post to the SNS topic:
aws sns set-topic-attributes \ --region "$region" \ --topic-arn "$sns_topic_arn" \ --attribute-name Policy \ --attribute-value '{ "Version": "2008-10-17", "Id": "cloudwatch-events-publish-to-sns-'"$sns_topic_name"'", "Statement": [{ "Effect": "Allow", "Principal": { "Service": "events.amazonaws.com" }, "Action": [ "SNS:Publish" ], "Resource": "'"$sns_topic_arn"'" }] }'
-
Create a CloudWatch Events Rule that filters for Spot instance interruption warnings for this specific instance.
- TODO: Modify the ansible stack formation steps to create this rule auotmatically. Ref: awslab example
rule_name_interrupted="ec2-spot-interruption-$instance_id"
rule_description_interrupted="EC2 Spot instance $instance_id interrupted"
event_pattern_interrupted='{
"source": [
"aws.ec2"
],
"detail-type": [
"EC2 Spot Instance Interruption Warning"
],
"detail": {
"instance-id": [ "'"$instance_id"'" ]
}
}'
aws events put-rule \
--region "$region" \
--name "$rule_name_interrupted" \
--description "$rule_description_interrupted" \
--event-pattern "$event_pattern_interrupted" \
--state "ENABLED"
- Set the target of CloudWatch Events rule to the above spot-interruption SNS topic:
sns_target_interrupted='[{
"Id": "target-sns-'"$sns_topic_name"'",
"Arn": "'"$sns_topic_arn"'",
"InputTransformer": {
"InputPathsMap": {
"title": "$.detail-type",
"source": "$.source",
"account": "$.account",
"time": "$.time",
"region": "$.region",
"instance": "$.detail.instance-id",
"action": "$.detail.instance-action"
},
"InputTemplate":
"\"<title>: <source> will <action> <instance> ('"$instance_name"') in <region> of <account> at <time>\""
}
}]'
aws events put-targets \
--region "$region" \
--rule "$rule_name_interrupted" \
--targets "$sns_target_interrupted"
- ref: Alestic blog Feb-2018
- ref: AWS new Spot abilities
- ref: Algo VPN