-
Clone ce-kafka checkout 5.3.x and build
gradle && ./gradlew jar
-
Start ZK in ce-kafka repo
./bin/zookeeper-server-start.sh config/zookeeper.properties
-
Start Kafka (change logs to /tmp) in ce-kafka repo
./bin/kafka-server-start.sh config/server.properties
-
Start Connect in ce-kafka repo
./bin/connect-distributed.sh config/connect-distributed.properties
-
Clone schema-registry checkout 5.3.x and build
mvn clean package -DskipTests
-
Start schema registry in schema-registry repo
./bin/schema-registry-start config/schema-registry.properties
-
For KSQL, clone the KSQL repo checkout 5.3.x and build:
mvn clean package -DskipTests
-
Run KSQL Server and the CLI:
./bin/ksql-server-start -daemon config/ksql-server.properties ./bin/ksql
-
-
Clone blueway checkout 5.3.x and build
mvn clean package -DskipTests
-
Start control center from blueway repo
./bin/control-center-start config/control-center-dev.properties
-
Clone frontend-vault checkout 5.3.x and build
yarn in && yarn workspace @confluent/control-center start
Last active
June 28, 2019 17:49
-
-
Save joel-hamill/0bc6bd72e706258da49f9d1bf6c9ce71 to your computer and use it in GitHub Desktop.
Build Confluent Platform manually
Super helpful, thanks!
For KSQL, clone the KSQL repo and build:
mvn clean package -DskipTests
Run KSQL Server and the CLI:
./bin/ksql-server-start -daemon config/ksql-server.properties
./bin/ksql
Works on Ubuntu 19.04 Disco Dingo! I found that need to delete the Kafka log files and the Zookeeper metadata between sessions:
sudo rm -rf /var/lib/kafka/
sudo rm -rf /var/lib/zookeeper/
This path is set by the log.dir
property in the server.properties
file.
I hacked together a couple of quick scripts to start and stop the stack:
sudo ./start-hand-rolled-cp.sh
sudo ./stop-hand-rolled-cp.sh
TODO: start/stop C3.
The start-hand-rolled-cp.sh
script starts the cluster:
#!/bin/bash
# Change CP_ROOT to your Git root directory.
CP_ROOT="~/repos"
KAFKA_BIN=$CP_ROOT"/ce-kafka/bin"
SR_BIN=$CP_ROOT"/schema-registry/bin"
KSQL_BIN=$CP_ROOT"/ksql/bin"
KAFKA_ETC=$CP_ROOT"/ce-kafka/config"
SR_ETC=$CP_ROOT"/schema-registry/config"
KSQL_ETC=$CP_ROOT"/ksql/config"
echo $CP_ROOT
echo $KAFKA_BIN
echo $SR_BIN
echo $KSQL_BIN
echo "Removing old state"
# This directory is set by log.dir in the server.properties file and may vary by OS.
sudo rm -rf /var/lib/kafka/
sudo rm -rf /var/lib/zookeeper/
echo "Starting Zookeeper"
eval $KAFKA_BIN"/zookeeper-server-start "$KAFKA_ETC"/zookeeper.properties &"
echo "Starting Kafka"
eval $KAFKA_BIN"/kafka-server-start "$KAFKA_ETC"/server.properties &"
echo "Starting connect"
eval $KAFKA_BIN"/connect-distributed "$KAFKA_ETC"/connect-distributed.properties &"
echo "Starting Schema Registry"
eval $SR_BIN"/schema-registry-start "$SR_ETC"/schema-registry.properties &"
echo "KSQL Server"
eval $KSQL_BIN"/ksql-server-start "$KSQL_ETC"/ksql-server.properties &"
#eval $KSQL_BIN"/ksql"
~ ~
The stop-hand-rolled-cp.sh
script stops the cluster:
#!/bin/bash
# Change CP_ROOT to your Git root directory.
CP_ROOT="~/repos"
KAFKA_BIN=$CP_ROOT"/ce-kafka/bin"
SR_BIN=$CP_ROOT"/schema-registry/bin"
KSQL_BIN=$CP_ROOT"/ksql/bin"
KAFKA_ETC=$CP_ROOT"/ce-kafka/config"
SR_ETC=$CP_ROOT"/schema-registry/config"
KSQL_ETC=$CP_ROOT"/ksql/config"
echo $CP_ROOT
echo $KAFKA_BIN
echo $SR_BIN
echo $KSQL_BIN
eval $KAFKA_BIN"/zookeeper-server-stop"
eval $KAFKA_BIN"/kafka-server-stop"
eval $SR_BIN"/schema-registry-stop"
eval $KSQL_BIN"/ksql-server-stop"
# TODO: stop connect-distributed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Before step 9, run
yarn in
to pull in all dependencies.As developers commit frontend fixes, do a
git pull origin --rebase
in thefrontend-vault cp-5.3.x
branch for example and run step 9 again.