Skip to content

Instantly share code, notes, and snippets.

@Myrddraal

Myrddraal/Test-ChangesMadeInPath.ps1

Last active August 28, 2024 15:46
Show Gist options
  • Save Myrddraal/f5a84cf3242e3b3804fa727005ed2786 to your computer and use it in GitHub Desktop.
Save Myrddraal/f5a84cf3242e3b3804fa727005ed2786 to your computer and use it in GitHub Desktop.
Download ZIP
Skip infrastructure deployments if there are no changes to deploy (Azure Pipelines)
Raw
_Readme.md

Skip infrastructure deployments if there are no changes to deploy (Azure Pipelines)

⚠️ Update 2024: For anyone looking at this gist in 2024, we have moved on from using the Pipelines API to instead using tags. Each time we successfully deploy to an environment, we tag the commit with the environment name. When testing for changes, we then compare the current commit being deployed to the tagged commit for the same environment. This has proven to be a bit more robust than using the changes API. The principles are the same though. Let me know in a comment if it's not obvious how to adapt the script.

One devops strategy is to keep infrastructure state and code in a single repository, and deploy them together. It ensures your infrastructure is always in the correct state to match the version of your code.

The downside is that infrastructure deployments are slow, and generally don't change as frequently as code.

This gist shows how to skip an infrastructure deployment task if there are no changes in a particular sub-path of the repository.

It takes advantage of the pipelines API, which can provide a list of all commits since the last successful pipeline execution. This ensures that it works even when multiple commits are pushed at once, or when the build with the infrastructure change failed. The pipelines API does the hard work of working out which commits need checking.

Example of skipped infra deployment

Raw
azure-pipelines.yaml
resources:
repositories:
- repository: self
type: git
ref: main
stages:
- stage: Build
pool:
vmImage: ubuntu-latest
jobs:
- job: BuildJob
displayName: Build Job
steps:
- checkout: self
clean: true
- task: PowerShell@2
inputs:
filePath: "azure-pipelines/Test-ChangesMadeInPath.ps1"
arguments: >-
-authorisation "Bearer $(system.accesstoken)"
-pathFilter "azure-pipelines/deployment"
-buildId $(Build.BuildId)
-collectionUri $(System.CollectionUri)
-project $(System.TeamProject)
name: DetermineChangesMadeInDeploymentFolder
env:
SYSTEM_ACCESSTOKEN: $(system.accesstoken)
- task: DotNetCoreCLI@2
displayName: Build project
inputs:
projects: "**/*.csproj"
arguments: --output publish_output --configuration Release
- task: PublishBuildArtifacts@1
displayName: "Publish Artifact: drop"
- stage: Deploy
pool:
vmImage: ubuntu-latest
condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/main'))
jobs:
- deployment: DeployInfrastructure
condition: eq(stageDependencies.Build.BuildJob.outputs['DetermineChangesMadeInDeploymentFolder.filesUpdated'], 'True')
displayName: Deploy infrastructure
environment: "prod"
strategy:
runOnce:
deploy:
steps:
- template: deployment/YAML/deploy-infrastructure.yml
parameters:
environment: $(Environment.Name)
- deployment: DeployCode
displayName: Deploy code
dependsOn: DeployInfrastructure
condition: in(dependencies.DeployInfrastructure.result, 'Succeeded', 'Skipped')
environment: "prod"
strategy:
runOnce:
deploy:
steps:
- template: deployment/YAML/deploy-code.yml
parameters:
environment: $(Environment.Name)
Raw
Test-ChangesMadeInPath.ps1
[CmdletBinding()]
param (
$authorisation,
$pathFilter,
$collectionUri,
$project,
$buildId
)
$changesUrl = "$collectionUri/$project/_apis/build/builds/$buildId/changes?api-version=6.0"
$changesResponse = Invoke-RestMethod -Uri $changesUrl -Headers @{Authorization = $authorisation } -Method Get
$commits = @($changesResponse.value | ForEach-Object { $_.id })
Write-Host "##vso[task.setvariable variable=filesUpdated;isOutput=true]False"
Write-Host "Checking $($commits.Length) commits for changes matching path $pathFilter"
for ($j = 0; $j -lt $commits.Length; $j++) {
Write-Host "Checking commit: $($commits[$j]) with its parent"
$files = $(git diff "$($commits[$j])~" $commits[$j] --name-only)
Write-Host $files
if ($files -like "*$pathFilter/*") {
Write-Host "Found file matching path filter in commit $($commits[$j])"
Write-Host "##vso[task.setvariable variable=filesUpdated;isOutput=true]True"
break
}
}
@gprestes
Copy link

https://gist.github.com/Myrddraal/f5a84cf3242e3b3804fa727005ed2786#file-azure-pipelines-yaml-L23:
There is an extra ' in the end of this line.

@Myrddraal
Copy link
Author

https://gist.github.com/Myrddraal/f5a84cf3242e3b3804fa727005ed2786#file-azure-pipelines-yaml-L23: There is an extra ' in the end of this line.

Thanks, fixed!

@vatsalkgor
Copy link

Thanks for the gist. It's working great.
In case if anyone's facing fatal: ambiguous argument '<commit_hash>~': unknown revision or path not in the working tree., add below in the checkout step of the pipeline (reference - https://stackoverflow.com/questions/75055130/resolving-git-diff-error-in-azure-devops-pipeline)

- checkout: self
  clean: true
  fetchDepth: 0

@Elisabeth-Kury
Copy link

Hi Myrddraal, this sounds like a very interesting solution! As I'm quite new to Azure pipelines unfortunately for me it isn't too obvious how to adapt the script in order to replace the API interaction with tags. What would be the Environment.Name in this scenario?
Is it always the same for a certain repo? Then wouldn't I have multiple commits with the same tag?
Thanks in advance!

@DavidAtImpactCubed
Copy link

DavidAtImpactCubed commented Jul 30, 2024 •
edited
Loading

Hi Elisabeth,
(EDIT replying with my work account)
I'm afraid I don't have a huge amount of time to write this up properly, but hopefully the following will help:

Our Test-ChangesMadeInPath powershell script now looks like this:

[CmdletBinding()]
param (
  [String] $pathFilter,
  [String] $tag,
  [String] $stageToBeDeployed,
  [String] $defaultValue
)

Write-Host "##vso[task.setvariable variable=Deploy${stageToBeDeployed};isOutput=true]${defaultValue}";
Write-Host "Checking diff to tag ${tag} for changes. Setting variable name: Deploy${stageToBeDeployed} with default value: ${defaultValue}.";

if (git tag -l $tag) {
  $files = $(git diff $tag --name-only)
  $pathFilters = $pathFilter.Split(",")
  Write-Host ($files -replace ' ', "`r`n")
  for ($j = 0; $j -lt $pathFilters.Length; $j++) {
    if ($files -like "*$($pathFilters[$j])*") {
      Write-Host "Found file matching path filter $($pathFilters[$j]) for tag ${tag}";
      Write-Host "##vso[task.setvariable variable=Deploy${stageToBeDeployed};isOutput=true]true";
      Write-Host "##[warning]${stageToBeDeployed} will be deployed to environment ${tag}";
      return
    }
  }

  Write-Host "No files found matching path filters for tag ${tag}";
} else {
  Write-Host "Tag ${tag} does not exist, setting to deploy";
  Write-Host "##vso[task.setvariable variable=Deploy${stageToBeDeployed};isOutput=true]true";
  Write-Host "##[warning]${stageToBeDeployed} will be deployed to environment ${tag}";
  return
}

At the end of a successful deployment, we then do this:

  - script: |
      if [ $(git tag -l ${{parameters.environment}}) ]; then
        git tag -d ${{parameters.environment}}                      # delete the tag for most recent deployment
        git push origin :refs/tags/${{parameters.environment}}      # delete remotely
      fi

      git tag ${{parameters.environment}} $(Build.SourceVersion)  # make a new tag for most recent deployment locally
      git push origin ${{parameters.environment}}                 # push the new local tag to the remote
    workingDirectory: $(Build.SourcesDirectory)

The idea is that you can compare the current git commit to the tagged version to see what changes have happened since the last successful deployment. This has several advantages, not least that you can rollback by deploying a previous deployment, and it will correctly identify if files have changed between what is live and the version you are redeploying.

I'm thinking it probably makes sense to use branches rather than tags but tags are working for us right now and it hasn't been a priority to change.

Let me know if that gives you enough info to work it out!

@Elisabeth-Kury
Copy link

Hi David, thanks so much for your reply!
I'll try it out, will definitely help :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment