Skip to content

Instantly share code, notes, and snippets.

@dofinn

dofinn/muo_nodedrainsetup.sh

Last active February 26, 2021 13:24
Show Gist options
  • Save dofinn/148cf09f48502809eca7cbf97160eeef to your computer and use it in GitHub Desktop.
Save dofinn/148cf09f48502809eca7cbf97160eeef to your computer and use it in GitHub Desktop.
Download ZIP
Script for node drain dev
Raw
muo_nodedrainsetup.sh
#!/bin/bash
# setup test app from e2e
oc new-project guestbook
# pull guestbook from e2e
curl https://raw.githubusercontent.com/openshift/osde2e/main/assets/workloads/e2e/guestbook/frontend-deployment.yaml | oc apply -f -
curl https://raw.githubusercontent.com/openshift/osde2e/main/assets/workloads/e2e/guestbook/frontend-service.yaml | oc apply -f -
curl https://raw.githubusercontent.com/openshift/osde2e/main/assets/workloads/e2e/guestbook/redis-master-deployment.yaml | oc apply -f -
curl https://raw.githubusercontent.com/openshift/osde2e/main/assets/workloads/e2e/guestbook/redis-master-service.yaml | oc apply -f -
curl https://raw.githubusercontent.com/openshift/osde2e/main/assets/workloads/e2e/guestbook/redis-slave-deployment.yaml | oc apply -f -
curl https://raw.githubusercontent.com/openshift/osde2e/main/assets/workloads/e2e/guestbook/redis-slave-service.yaml | oc apply -f -
# apply PDB
cat <<EOF | oc apply -f - -n guestbook
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
name: guestbook
spec:
maxUnavailable: 0
selector:
matchLabels:
app: guestbook
EOF
echo "patch PDB guestbook to cause failed draining PDB config"
oc patch -n guestbook deployment.apps/frontend -p '{"spec":{"replicas":1}}' --type=merge
# Add finalizer pod
cat <<EOF | oc apply -f - -n guestbook
apiVersion: v1
kind: Pod
metadata:
name: busybox-sleep
finalizers:
- ingress.openshift.io/canttouchthis
spec:
containers:
- name: busybox
image: busybox
args:
- sleep
- "1000000"
EOF
# Create upgradeconfig at same version as cluster
CURRENT_VERSION=$(oc get clusterversion version -o json | jq -r '.status.history[].version')
oc adm groups add-users osd-sre-cluster-admins $(oc whoami)
echo "sleep for 10"
sleep 10
cat <<EOF | oc apply -f - -n openshift-managed-upgrade-operator
apiVersion: upgrade.managed.openshift.io/v1alpha1
kind: UpgradeConfig
metadata:
name: example-upgrade-config
spec:
type: "OSD"
upgradeAt: "2020-01-01T00:00:00Z"
proceed: true
PDBForceDrainTimeout: 60
desired:
channel: "stable-4.4"
force: false
version: "${CURRENT_VERSION}"
EOF
echo "patch replica to 0 for local dev"
oc patch -n openshift-managed-upgrade-operator deployment.apps/managed-upgrade-operator -p '{"spec":{"replicas":0}}' --type=merge
echo "patch upgrade config for lower PDB timeout"
oc patch -n openshift-managed-upgrade-operator upgradeconfig example-upgrade-config -p '{"spec":{"PDBForceDrainTimeout":1}}' --type=merge
echo "Summary"
oc get all -n guestbook
oc get all -n openshift-managed-upgrade-operator
# swtich to MUO
oc project openshift-managed-upgrade-operator
# Some help
echo
echo "Update the SRE node drain manually"
echo oc edit cm managed-upgrade-operator-config -o yaml
oc get cm managed-upgrade-operator-config -o yaml | grep -w -A1 nodeDrain
echo
echo "Use this to terminate busybox if required"
echo oc patch pod -n guestbook busybox-sleep -p '{\"metadata\":{\"finalizers\":[]}}' --type=merge
echo "Use this to restore operator deployment"
echo oc patch -n openshift-managed-upgrade-operator deployment.apps/managed-upgrade-operator -p '{\"spec\":{\"replicas\":1}}' --type=merge
echo
echo "You can run the operator locally as seen below and tail the output log for targeted node to capture specific logs"
echo 'UPGRADING=true OPERATOR_NAMESPACE=openshift-managed-upgrade-operator make run |& tee /tmp/nodedrain.log'
echo 'tail -f /tmp/nodedrain.log | grep $NODE'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment