FileSet Pipeline Resource

Today in Tekton, tasks that consume files must operate on a typed Input resource, usually of Type Git or Storage. This leads to a few problems:

• Tasks must choose which to support, when both should be equivalent
• Pipelines of Tasks that operate on specific files must invent out-of-band mechanisms to specify which files to operate on

Ideally, Tasks that operate on Files would have a more generic type of resource that can be populated from various sources, and Tasks should not need to communicate about file paths or names out-of-band.

This proposal details a FileSet Pipeline Resource designed to achieve these goals.

API:

To create one

You can supply one of:

• rawData (string representing one file)
• configMapRef (just use all the files in a configmap)
• ResourceRef
  • pointing to a Git resource or a Storage resource (or maybe other types later)
  • And a set of glob strings (to select which files)

Other ideas for "sources":

• Container Image + file globs (we would extract them from an image in a registry)
• URL (download the contents at the URL)
• maybe more!

When passed as an Input to a Task

The files to operate on will be under: /workspace/$resourcename/ in an arbitrary directory structure. Only the files the task needs to operate on are present in this structure. The task should operate on everything in this directory recursively.

The task will also receieve an environment variable named: $SOMETHING (also available for interpolation) containing a list of filenames, separated by spaces.

The goal is for users to be able to write tasks with a step like:

kubectl apply -f ${something} and have that work correctly on a list of filenames.

When produced as an Output of a Task

The task is responsible for filling a directory at: /workspace/outputs/$resourcename with the files.