An exploration of how and why running our pipelines locally.
- Why ?
- How ?
- Expectation ?
Goal: compare what exists today (with or without tekton), and discuss what’s next ?
- Use the same “recipe” for building locally and on the CI
- Change the recipe at one place and that’s it
- Debug CI pipelines (?)
Use the same thing for inner-loop and outer-loop.
Local execution should be as close as possible to the CI execution (minus some environment and different secrets). Ideally the exact same command is ran.
- What is the role of CI/CD (and Tekton) in a developer’s local workflow? #145
- CI/CD and the development workflow from Christie
- It is up to us to define it ✍️.
- Tekton Mission
Be the industry-standard, cloud-native CI/CD platform components and ecosystem. […]
- Engineers who need CI/CD: (aka all software engineers!) […]
- Be able to use multiple Tekton conformant systems instead of being locked into one or being forced to build glue between multiple completely different systems
- Use an ecosystem of tools that know how to interact with Tekton components, e.g. IDE integrations, linting, CLIs, security and policy systems
- Engineers who need CI/CD: (aka all software engineers!) […]
How does it work today
Lost time is never found again 😼 … … or, you are on your own 😉, not possible
The easy way (with some limitations) : jenkinsfile-runner
The hard way : setup a local jenkins properly
If no Jenkinsfile
, you are on your own 🙃
$ gitlab-runner exec docker \
# --env CI_COMMIT_REF_SLUG="main" \
# […] \
{job}
- Same runner used on Gitlab CI (same binary)
- Relatively smart (guess most environment variables necessary)
nektos/act
— run your GitHub Actions locally 🚀
It does have some limitations.
$ act -j {job}
… or, *officially* you are on your own 😉, not possible
kind
- Docker Desktop
- Minikube
$ kubectl …
Does not build local working directory (yet)
buildkit-tekton
- Very young and experimental
- Not fully supports all Tekton features
- Does not build local working directory (yet)
$ tkn local run -f …
Dagger is a portable devkit for CI/CD. […] Benefits of using Dagger include:
Unify dev and CI environments. Write your pipeline once, Dagger will run it the same everywhere. Reduce CI lock-in. No more re-writing everything from scratch every 6 months. Rapid debugging of your pipelines. Dagger runs equally well on your local machine, and in CI.
dagger do …
dagger
seems the closest to that ideal.- Does it make sense for Tekton to “care” about inner-loop ?
- Does it make sense for the tekton community to “invest” ?
Note: Opinions matter for local execution but Tekton is not opiniated 😅.
- GitHub and Gitlab are 1 images for all the “steps”
- Dagger and Tekton are DAGs of containers (steps in Tekton)
- This is where we need to discuss, brainstorm, experiment, play 🏓