A build process for GraalVM native images that can be deployed on AWS Lambda as Docker container images
Use zio-lambda to write your own Lambda function based on Scala and ZIO
- Checkout the example project for inspiration
- Make sure you enable the
GraalVMNativeImagePlugin
like the example project does and configure it with the appropriate flags to build the native image - You may most likely need to add some reflect configuration and this post from Inner Product shows a good way to include the configuration in the project so Graal native-image will find it
If you have graal native already installed with the correct libraries in place for linking then great, otherwise use one of the following Docker images tagged with graalvm
NOTE: If you want to use the latest version of GraalVM, you can adapt the following Dockerfile
FROM ghcr.io/graalvm/graalvm-ce:ol8-java17
ENV SBT_VERSION ${SBT_VERSION:-1.6.2}
ENV SCALA_VERSION ${SCALA_VERSION:-2.13.8}
ENV JAVA_OPTS -XX:+UseG1GC
ENV USER_ID ${USER_ID:-1001}
ENV GROUP_ID ${GROUP_ID:-1001}
# Install sbt
RUN \
curl -fsL "https://github.com/sbt/sbt/releases/download/v$SBT_VERSION/sbt-$SBT_VERSION.tgz" | tar xfz - -C /usr/share && \
chown -R root:root /usr/share/sbt && \
chmod -R 755 /usr/share/sbt && \
ln -s /usr/share/sbt/bin/sbt /usr/local/bin/sbt
# Install Scala
## Piping curl directly in tar
RUN \
case $SCALA_VERSION in \
"3"*) URL=https://github.com/lampepfl/dotty/releases/download/$SCALA_VERSION/scala3-$SCALA_VERSION.tar.gz SCALA_DIR=/usr/share/scala3-$SCALA_VERSION ;; \
*) URL=https://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz SCALA_DIR=/usr/share/scala-$SCALA_VERSION ;; \
esac && \
curl -fsL $URL | tar xfz - -C /usr/share && \
mv $SCALA_DIR /usr/share/scala && \
chown -R root:root /usr/share/scala && \
chmod -R 755 /usr/share/scala && \
ln -s /usr/share/scala/bin/* /usr/local/bin && \
case $SCALA_VERSION in \
"3"*) echo "@main def main = println(util.Properties.versionMsg)" > test.scala ;; \
*) echo "println(util.Properties.versionMsg)" > test.scala ;; \
esac && \
scala -nocompdaemon test.scala && rm test.scala
# Add and use user sbtuser
RUN groupadd --gid $GROUP_ID sbtuser && useradd --gid $GROUP_ID --uid $USER_ID sbtuser --shell /bin/bash
USER sbtuser
# Switch working directory
WORKDIR /home/sbtuser
# Prepare sbt (warm cache)
RUN \
sbt sbtVersion && \
mkdir -p project && \
echo "scalaVersion := \"${SCALA_VERSION}\"" > build.sbt && \
echo "sbt.version=${SBT_VERSION}" > project/build.properties && \
echo "// force sbt compiler-bridge download" > project/Dependencies.scala && \
echo "case object Temp" > Temp.scala && \
sbt compile && \
rm -r project && rm build.sbt && rm Temp.scala && rm -r target
# Link everything into root as well
# This allows users of this container to choose, whether they want to run the container as sbtuser (non-root) or as root
USER root
RUN \
ln -s /home/sbtuser/.cache /root/.cache && \
ln -s /home/sbtuser/.sbt /root/.sbt && \
if [ -d "/home/sbtuser/.ivy2" ]; then ln -s /home/sbtuser/.ivy2 /root/.ivy2; fi
# Switch working directory back to root
## Users wanting to use this container as non-root should combine the two following arguments
## -u sbtuser
## -w /home/sbtuser
WORKDIR /root
CMD sbt
Tag it as a builder:
docker build -t scala-graal-builder .
Use this builder image to build the application for Graal native binary via a multi-stage build. As an example, we will build the example project found in zio-lambda:
FROM scala-graal-builder:latest as builder
COPY . /build
WORKDIR /build
RUN gu install native-image && sbt graalvm-native-image:packageBin
FROM gcr.io/distroless/base
COPY --from=builder /build/lambda-example/target/graalvm-native-image /app
CMD ["/app/zio-lambda-example"]
docker build -t native-image-binary .
Upload this native-binary-image
image to the AWS ECR container registry so that AWS Lambda can access it.
Alternatively, you can greatly simplify the process and simply add the following setting for native package to do it all for you in build.sbt
:
.settings(
publish / skip := true,
name := "zio-lambda-example",
stdSettings("zio-lambda-example"),
assembly / assemblyJarName := "zio-lambda-example.jar",
GraalVMNativeImage / mainClass := Some("zio.lambda.example.SimpleHandler"),
// add the following
GraalVMNativeImage / containerBuildImage := GraalVMNativeImagePlugin
.generateContainerBuildImage(
"hseeberger/scala-sbt:graalvm-ce-21.3.0-java17_1.6.2_3.1.1"
)
.value,
...
Now you can just run sbt graalvm-native-image:packageBin
on your host machine and SBT native package will take care of the rest and automatically build the image inside of Docker.
You would still need to follow the steps from earlier if you want to package your native image binary into a container image that would need to be uploaded to ECR. However, we found that you can directly use this native binary directly since all GLIBC dependencies are linked properly and you can follow one of the options on the zio-lambda readme.
Here's how we create our AWS Lambda
Once we run sbt graalvm-native-image:packageBin
, we'll find the binary present under the graalvm-native-image
folder:
We'll just create the following bootstap file (which calls out to the binary) and place it in the same directory alongside the binary
#!/usr/bin/env bash
set -euo pipefail
./zio-lambda-example
Now we can zip both these files up:
> pwd
/home/cal/IdeaProjects/zio-lambda/lambda-example/target/graalvm-native-image
> zip upload.zip bootstrap zio-lambda-example
Now take upload.zip
and upload it to AWS Lambda and test your function
Following the steps from Package your runtime as a Docker image
You should now have a Docker image that contains the native image and invokes it. Take this image and push it to AWS ECR
pass=$(aws ecr get-login-password --region us-east-1)
docker login --username AWS --password $pass <your_AWS_ECR_REPO>
docker tag native-image-binary <your-particular-ecr-image-repository>:latest
docker push <your-particular-ecr-image-repository>:latest
Create a Lambda function and choose container image:
Please note that because you incur the overhead of your native binary residing within a Docker container, there is more overhead than the other approach of deploying the binary straight to AWS Lambda
Creating the lambda function with a container image: