Skip to content

Instantly share code, notes, and snippets.

@azizulwahid

azizulwahid/dt__prod_release_workflow.md

Last active August 13, 2024 03:28
Show Gist options
  • Save azizulwahid/6d871e39bf650cc93523933d9061bf51 to your computer and use it in GitHub Desktop.
Save azizulwahid/6d871e39bf650cc93523933d9061bf51 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
dt__prod_release_workflow.md
  1. Pause all running data ingestion jobs on Airflow
  1. Pause master and transport order job, and other specified pipelines.
  1. Restart Synapse
  1. Ask the team to approve and then merge the PRs on Datahub.
  1. Run Azure Data Pipeline and wait until completed.
  1. Trigger model job
  1. Un-pause transport order job.
Raw
dt__uat_release_workflow.md
  1. Update Airflow Variables AIRFLOW_SCHEDULE, append _1 to PAS2_DELTA_MASTER_SCHEDULE to stop scheduler
  2. Pause all running data ingestion jobs on Airflow
  1. Pause transport order job.
  1. Run Azure Data Pipeline and wait until completed.
  2. Remove _1 to PAS2_DELTA_MASTER_SCHEDULE_1
  3. Trigger model job
  1. Un-pause transport order job.
Note: Make sure https://airflow-uat.it.pickles.com.au/dags/pas2_delta_master_package is not running.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment