Skip to content

Instantly share code, notes, and snippets.

@Kalyan-Alamuru

Kalyan-Alamuru/docker-cdh-build-image

Created April 24, 2019 19:35
Show Gist options
  • Save Kalyan-Alamuru/e5e7a9ec0b398ca7e6e02120afdde4d3 to your computer and use it in GitHub Desktop.
Save Kalyan-Alamuru/e5e7a9ec0b398ca7e6e02120afdde4d3 to your computer and use it in GitHub Desktop.
Download ZIP
Cloudera Hadoop Docker Build
Raw
docker-cdh-build-image
FROM centos:centos6
ENV container docker
EXPOSE 22 7180
ADD docker-cdh-*.sh /tmp/
RUN chmod u+x /tmp/docker-cdh-*.sh
RUN /tmp/docker-cdh-build.sh
CMD ["/bin/bash"]
Raw
docker-cdh-build.sh
#! /usr/bin/env bash
echo 'root:cloudera' | chpasswd
echo ''
echo 'Installing host linux packages ...'
yum clean all
yum -y update
yum -y install openssh-server openssh-clients wget nano curl
echo ''
echo 'Installing cloudera yum repositories ...'
wget 'https://archive.cloudera.com/cm5/redhat/6/x86_64/cm/cloudera-manager.repo' -P /etc/yum.repos.d/
wget 'https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/cloudera-cdh5.repo' -P /etc/yum.repos.d/
wget 'https://archive.cloudera.com/kafka/redhat/6/x86_64/kafka/cloudera-kafka.repo' -P /etc/yum.repos.d/
rpm --import https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera
yum -y update
echo''
echo 'Installing cloudera manager server packages ...'
yum -y install cloudera-manager-server-db-2 cloudera-manager-daemons cloudera-manager-server cloudera-manager-agent
echo''
echo 'Installing cloudera hadoop services packages ...'
yum -y install oracle-j2sdk1.7 hadoop-hdfs-fuse hadoop-hdfs-nfs3 hadoop-httpfs hbase-solr hive-hbase hive-webhcat hue-beeswax hue-hbase hue-impala hue-plugins hue-rdbms hue-search hue-spark hue-sqoop hue-zookeeper impala impala-shell oozie search sentry solr-mapreduce spark-core spark-master spark-worker spark-history-server spark-python sqoop sqoop2 kafka kafka-server zookeeper
echo''
echo 'Initialising host environment for cloudera hadoop ...'
service cloudera-scm-server-db start
service cloudera-scm-server start
service cloudera-scm-agent start
mkdir /etc/yarn
mkdir /dfs/
chown hdfs:hdfs -R /dfs
chkconfig --level 2345 oozie off
chkconfig --level 2345 hadoop-httpfs off
cp /etc/sysctl.conf /etc/sysctl.conf.old
sed -i '$ a vm.swappiness=10' /etc/sysctl.conf
echo ''
echo 'Completed cloudera hadoop build!!!'
echo '- wait a few seconds for the remaining docker build steps to complete'
echo '- recommend save a copy of the new image'
Raw
docker-cdh-start.sh
#! /usr/bin/env bash
echo ''
echo 'Starting cloudera hadoop services ...'
service cloudera-scm-server-db start
service cloudera-scm-server start
service cloudera-scm-agent start
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment