Skip to content

Instantly share code, notes, and snippets.

@jayco

jayco/pipeline.yml

Created November 10, 2020 07:10
Show Gist options
  • Save jayco/3524a02458e342c878dd64bf806d207c to your computer and use it in GitHub Desktop.
Save jayco/3524a02458e342c878dd64bf806d207c to your computer and use it in GitHub Desktop.
Download ZIP
Multi environment/region test, build and deploy
Raw
pipeline.yml
env:
DOCKER_IMAGE_TAG: "myservice-${BUILDKITE_BRANCH}-${BUILDKITE_COMMIT:0:7}"
steps:
- name: 'test :hammer:'
agents:
queue: aws-buildkite-agent
command: echo ".buildkite/test.sh"
- wait:
- env:
DOCKER_BUILD_NETWORK: myservice-build
name: 'Build & Push :docker:'
agents:
queue: aws-buildkite-agent
command: echo ".buildkite/builder.sh"
- wait:
- block: Start Test Database Migration
branches: "!master"
- env:
CONSUL_HOST: consul.us.test.dummy.com
name: ":postgres: schema migration - test environment us-east-1"
agents:
queue: aws-migrate-agent
region: us-east-1
environment: test
command: echo "/etc/buildkite-agent/buildkite-scripts/migrate.sh"
- wait:
- block: Start test deployment
branches: "!master"
- trigger: deploy-test-us-east-1
name: ":docker: Deploy myservice to test us-east-1"
build:
env:
DOCKER_IMAGE_TAG: $DOCKER_IMAGE_TAG
- wait:
- block: Start Production Database Migration
branches: master
- env:
CONSUL_HOST: consul.us.prod.dummy.com
name: ":postgres: schema migration - production us-east-1"
agents:
queue: aws-migrate-agent
region: us-east-1
environment: production
command: echo "/etc/buildkite-agent/buildkite-scripts/migrate.sh"
branches: master
- env:
CONSUL_HOST: consul.syd.prod.dummy.com
name: ":postgres: schema migration - production ap-southeast-2"
agents:
queue: aws-migrate-agent
region: ap-southeast-2
environment: production
command: echo "/etc/buildkite-agent/buildkite-scripts/migrate.sh"
branches: master
- env:
CONSUL_HOST: consul.dub.prod.dummy.com
name: ":postgres: schema migration for production eu-west-1"
agents:
queue: aws-migrate-agent
region: eu-west-1
environment: production
command: echo "/etc/buildkite-agent/buildkite-scripts/migrate.sh"
branches: master
- wait:
- name: ":docker: Deploy myservice to Production ap-southeast-2"
build:
env:
DOCKER_IMAGE_TAG: $DOCKER_IMAGE_TAG
trigger: deploy-myservice-ap-southeast-2-production
branches: master
- wait:
- name: ":docker: Deploy myservice to Production eu-west-1"
build:
env:
DOCKER_IMAGE_TAG: $DOCKER_IMAGE_TAG
trigger: deploy-myservice-eu-west-1-production
branches: master
- wait:
- name: ":docker: Deploy myservice to Production us-east-1"
build:
env:
DOCKER_IMAGE_TAG: $DOCKER_IMAGE_TAG
trigger: deploy-myservice-us-east-1-production
branches: master
@jayco
Copy link
Author

jayco commented Nov 10, 2020 •
edited
Loading

Concept

This pipeline serves as a conceptual example of steps to test, build, push and deploy across regions using a test and production environment.

Here is the break down of the steps:

  • Runs tests
  • Builds and pushes image to something like ECR
  • If on branch:
    • Manual unblock to migrate and deploy to test environment
  • If on master:
    • Migrate and deploy to the testing environment
    • Manual unblock to migrate and deploy to production in three regions US, AU and EU
  • Dedicated pipelines for deployment (quick painless rollbacks, see https://buildkite.com/docs/deployments#dedicated-deployment-pipelines)

Note: the use of agent queues and tags to target different agents across regions.

Video of the build - PR branch (not master, optional test deploys)

Screen Shot 2020-11-10 at 6 56 02 pm

Video of the build - (master, optional prod deploys)

Screen Shot 2020-11-10 at 7 06 29 pm

@namnd
Copy link

namnd commented Aug 30, 2023

Hi @jayco ,

Is this syntax still working, i.e.

agents:
      queue: aws-migrate-agent
      region: us-east-1

I have all steps using one region except for one step I want to use agent from another region. Is this still doable? Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment