Pipeline currently provides extensibility by using different context images in pipeline steps. An image with certain utilities can be considered as a plugin. But we lack a centralized place to maintain available plugins, either provided by Rancher or community users. Also, the user experience for using different plugins can be improved so that users with poor experience in container/kubernetes can use existing util images more easily.
We use questions files to describe the usage of a pipeline plugin. An example for using drone s3 plugin:
name: s3
image: plugins/s3
command: drone-s3
categories:
- publish
- storage
questions:
- variable: S3_ENDPOINT
default: ""
label: "s3 endpoint"
description: "endpoint for the s3 connection"
type: string
required: true
- variable: S3_BUCKET
default: ""
label: "bucket"
description: "s3 bucket"
type: string
required: true
- variable: PLUGIN_SOURCE
default: ""
label: "upload files from source folder"
type: string
required: true
- variable: PLUGIN_TARGET
default: ""
label: "upload files to target folder"
type: string
required: true
- variable: AWS_ACCESS_KEY_ID
default: ""
label: "aws access key"
type: secret
required: true
- variable: AWS_SECRET_ACCESS_KEY
default: ""
label: "aws secret key"
type: secret
required: true
In pipeline project settings, add a configuration for plugin list:
It should be a list of question files for the plugins:
#pipeline-plugins.yaml
name: s3
image: xx
command: xx
questions: xx
---
name: xx
image: xx
command: xx
question: xx
---
...
When users add a pipeline step, they can choose the plugin step in the list:
As the result, the step configuration will be converted to the pipeline yaml format as:
...
steps:
- pluginConfig:
plugin: s3
# args in map type
args:
S3_ENDPOINT: https://play.minio.io:9000
S3_BUCKET: my-bucket-name
PLUGIN_SOURCE: public/**/*
PLUGIN_TARGET: /target/location
envFrom:
- sourceName: secret1
sourceKey: secretkey1
targetKey: AWS_ACCESS_KEY_ID
- sourceName: secret1
sourceKey: secretkey2
targetKey: AWS_SECRET_ACCESS_KEY
In this way, users can have customized/additional pipeline steps by simply changing the plugin questions yaml file.
- enhancements for questions.yml we are using for catalog
- Add name/image/command for Plugin support
- Add secret type question for referencing secrets in pipeline steps
- It would be nice to have reference field type like
reference[workload] #or other resources
for a question, so that users can pick a project workload for upgrade in dropdown in pipeline step configuration. But this is not a critical one.
- UI supports rendering plugin steps according to the questions file.
- Backend supports setting plugin questions file and plugin steps.