- Provide Fabric operators with a mechanism to build and launch chaincode outside of Docker
- Begin the process of decoupling and extracting chaincode build processing from the peer
Our current chaincode deployment model looks a lot like a PaaS; chaincode developers write code in the language of their choice and push it to the peer for execution. The peer, on demand, transforms the code into an executable package, launches the chaincode in an execution environment, and routes requests to the running chaincode.
Unfortunately, we have implemented this by hard coding the logic for packaging, building, and running chaincode inside Docker containers. In addition to making it difficult to experiment with new chaincode types, the implementation requires operators to grant the peer access to a Docker daemon. Since Docker daemon access enables the peer to perform a number of privileged actions, it's strongly discouraged in security conscious environments. This binding also prevents operators from using platforms like Kubernetes to host chaincode.
Let's begin by looking at some of the technology in Cloud Foundry that supports staging and execution of applications.
A buildpack is a software package that implements the process of transforming user provided source assets to executable code. The contract is simple, but flexible enough to support pure binaries, scripting languages, and complicated build processes that use autoconf and make.
The buildpack contract is comprised of three scripts: detect
, compile
, and release
. detect
is used to determine if a buildpack is can be used with source assets provided by a user, compile
is used to transform the assets into something that can be executed, and release
is used to provide the start command to the platform.
Cloud Foundry uses lifecycle implementations to manage applications. Like a buildpack, a lifecycle consists of three executables: builder
, launcher
, and healthcheck
. The builder
is responsible for preparing an application for execution, the launcher
is responsible for starting an application, and the healthcheck
is called regularly to monitor the launched application for liveness.
In Cloud Foundry, two implementations of lifecycle exist: docker
and buildpack
. The buildpack
lifecycle uses buildpack processing to build and launch an application while the docker
lifecycle is used extract metadata from an image and launch it in inside a garden container.
Our proposal is to adapt patterns used by Cloud Foundry for building and running applications to the chaincode model of the peer. These patterns will introduce seams that enable operators to build and launch chaincode with the technology of their choice and, eventually, will enable us to remove hard code chaincode platforms from the peer.
Combining aspects of the buildpack
and lifecycle
models in Cloud Foundry, the proposed peer chaincode contract consists of three process based entry points: detect
, build
, and launch
.
The purpose of detect
is to determine if the external builder should be used with the chaincode. The executable will be called by the peer with one argument: the path to a source directory containing the contents of the chaincode package. If detect
exits with a return code of 0
, the external builder will be used to build the chaincode. Any other exit code will indicate the peer's internal builder should be used.
Any modifications to the contents of the source directory may be discarded. Messages written to stderr
will be written to the peer's log.
When detect
has returned with a 0
return code, the build
executable will be called to transform the chaincode package into something that can be launched by launch
.
The executable will be called with two arguments: the first is the path to a source directory containing the contents of the chaincode package, and the second is the path to a build directory where the build
can cache information or hold build outputs. A successful build operation must result in a 0
return code from build
and any other return code will be treated as a failure.
When build
is successful, the contents of the build directory will be saved on the peer's file system; any modifications to the source may be discarded. Messages written to stderr
will be written to the peer's log.
A typical implementation of build might be to generate a build context from the input source, perform an unprivileged build using a tool like Google's kaniko
, and then populate the docker image reference to the build directory.
The purpose of launch
is to locate or start chaincode on behalf of the peer. The launch
executable is called with three arguments: the first is the path to a directory containing chaincode package source, the second is the path to a directory containing the contents of the build tree, and the third is the path to a directory containing chaincode.json
and any artifacts referenced by that file. This file will be used by the peer to provide metadata like the peer chaincode server address and assets such as generated TLS keys and certificates required for chaincode registration. It is up to the launcher to provide this information to the launched chaincode as appropriate.
The launch
operation is considered successful when it exits with a return code of 0
; any other return code is considered a failure. Prior to exiting, the launcher may write a JSON encoded message to the peer. This message is similar to chaincode.java
in structure but is used to inform the peer about the chaincode invocation contract. In particular, it can specify the address of the running chaincode and provide TLS keys and credentials necessary to connect to chaincode. When this information is made available, the peer will consider registration complete and invoke chaincode at the target address using the provided credentials. If this information is not provided, the legacy chaincode model is assumed and the peer will wait for the chaincode instance to register.
A failure to parse the JSON message provided from launch
to the peer will be treated as a launch failure. Fields that are unknown to the peer are ignored. Messages written to stderr
will be written to the peer's log.
A typical implementation of launch
might launch a container in a Kubernetes bare pod with the appropriate start command, environment and key material necessary to register with the peer's chaincode server. Once the pod object has been created, the launcher would return 0 with an empty JSON message of {}
to stdout
. A message that includes the ID of the new pod would be written to stderr
and captured in the peer's log. The peer would then wait for chaincode registration to complete.
An alternative implementation of launch
might retrieve the appropriate Kubernetes service definition and credentials to invoke the desired chaincode via some service, populate a message with the target information, write the message to stdout
, and exit with a return code of 0
. The peer would then parse the message and invoke the chaincode at the provided address with the provided credentials.
The peer configuration will be augmented to include a reference to a directory on the peer's local file system that contains the detect, build, and launch executables. These executables
When the feature is not configured, the peer will fallback to the existing implementation that is embedded within the peer.
The proposed launch mechanism does not provide any mechanism to include externally launched chaincode in the peer logs. If log aggregation is required, the peer and chaincode should be configured to direct logs to a common destination that can merge the events.
Jason noted that this does not address metadata extraction from the chaincode package such as CouchDB indices. A mechanism should be added to the build contract to support that.