Skip to content

Instantly share code, notes, and snippets.

@guy9

guy9/migrator-example-2.yml

Last active August 20, 2024 09:10
Show Gist options
  • Save guy9/68e1c8a6f3ff008d6d9f52c9e39260b0 to your computer and use it in GitHub Desktop.
Save guy9/68e1c8a6f3ff008d6d9f52c9e39260b0 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
migrator-example-2.yml
FROM alpine:3.20
ENV SPARK_VERSION=3.5.1 \
HADOOP_VERSION=3 \
SCALA_VERSION=2.13 \
SPARK_HOME="/spark"
RUN set -ex; \
apk add --no-cache openjdk11-jre bash rsync procps openssh coreutils; \
wget https://archive.apache.org/dist/spark/spark-${SPARK_VERSION}/spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}-scala${SCALA_VERSION}.tgz; \
tar --directory / -xvzf spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}-scala${SCALA_VERSION}.tgz; \
mv /spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}-scala${SCALA_VERSION} ${SPARK_HOME}; \
rm spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}-scala${SCALA_VERSION}.tgz
ENV PATH="${SPARK_HOME}/sbin:${SPARK_HOME}/bin:${PATH}"
WORKDIR ${SPARK_HOME}
COPY entrypoint.sh .
ENTRYPOINT ["/spark/entrypoint.sh"]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment