This guide helps you to setup the image registry configuration on a multi-node Hasura installation on GKE.
This is required in multi-node setups, because the sshd agent (which builds the docker images on git push) needs to push the image to an external image registry service, so that the image is available on all the nodes.
- gcloud CLI (https://cloud.google.com/sdk/install)
- kubectl (https://kubernetes.io/docs/tasks/tools/install-kubectl/)
First make sure are logged in: gcloud login
List all the projects:
gcloud projects list
And then set the correct project on which you have the Hasura cluster:
gcloud config set project <project-id>
Run the below steps:
# create a GCP service account; format of account is email address
SA_EMAIL=$(gcloud iam service-accounts --format='value(email)' create hasura-gcr-auth --display-name hasura-gcr-auth)
# create the json key file and associate it with the service account -> DON'T CHECK-IN TO VERSION CONTROL!
gcloud iam service-accounts keys create hasura-gcr-auth.json --iam-account=$SA_EMAIL
# get the project id
PROJECT=$(gcloud config list core/project --format='value(core.project)')
# add the IAM policy binding for the defined project and service account
gcloud projects add-iam-policy-binding $PROJECT --member serviceAccount:$SA_EMAIL --role roles/storage.admin
Make sure your kubectl is pointing to the right cluster. (Use kubectl config use-context <cluster-context>
to set it to the correct cluster).
- Put this in a script, in the same directory as above, and run:
#!/bin/bash
set -e
_auth="_json_key:$(cat hasura-gcr-auth.json)"
auth=$(echo $_auth | base64 -w 0)
cat <<EOF > _tmp_dockercfg.json
{
"https://gcr.io": {
"username": "_json_key",
"email": "a@example.com",
"password": "$(cat hasura-gcr-auth.json | sed 's/"/\\"/g' | sed -E ':a;N;$!ba;s/\r{0,1}\n/\\n/g')",
"auth": "$auth"
}
}
EOF
Then create a Kubernetes secret from the service account credentials file and above output.
kubectl create secret generic registry-creds \
--from-file=.dockercfg="_tmp_dockercfg.json" \
--type kubernetes.io/dockercfg
- Edit
controller-conf
:
kubectl -n hasura edit configmap controller-conf
- Add the following lines inside the
cluster.json
key under data. (Use the project-id from previous steps)
{
"prefix": "gcr.io/<project-id>",
"dockercfgSecret": "registry-creds"
}
Make sure the entry under cluster.json
key is a valid JSON string.
So the controller-conf should look something like this:
apiVersion: v1
data:
cluster.json: |
{
"namespaces": {"user":"default","hasura":"hasura"},
"gateway": {"selector":{"app":"gateway"},"ports":[{"targetPort":80,"protocol":"TCP","name":"http","port":80},{"targetPort":443,"pr
otocol":"TCP","name":"https","port":443},{"targetPort":22,"protocol":"TCP","name":"ssh","port":22}],"type":"LoadBalancer","loadBalancerI
P":"35.274.50.140"},
"postgres": {
"volume": {"name":"postgres","gcePersistentDisk":{"pdName":"postgres"}}
},
"sessionStore": {
"volume": {"name":"sessionstore","gcePersistentDisk":{"pdName":"sessionstore"}}
},
"filestore": {
"volume": {"name":"filestore","gcePersistentDisk":{"pdName":"filestore"}}
},
"volumes": {"postgres":{"name":"postgres","gcePersistentDisk":{"pdName":"postgres"}},"sessionstore":{"name":"sessionstore","gcePersistentDisk":{"pdName":"sessionstore"}},"filestore":{"name":"filestore","gcePersistentDisk":{"pdName":"filestore"}}},
"registry": {"prefix":"gcr.io/painkilling69","dockercfgSecret":"registry-creds"}
}
controller-conf.json: |
{
"controllerNamespace": "hasura",
...
If everything is well, you should be able to now push your microservices to Hasura and they will get built and pushed to your custom registry.
git push hasura master