Skip to content

Instantly share code, notes, and snippets.

@kwokhou
Created September 7, 2019 02:40
Show Gist options
  • Save kwokhou/87d3e97b60daf5653e446090ae4f1783 to your computer and use it in GitHub Desktop.
Save kwokhou/87d3e97b60daf5653e446090ae4f1783 to your computer and use it in GitHub Desktop.
Fix kubernetes pods or namespace stuck at removing/terminating state
# Pre-requisite
# 1. change <my-namespace> to the namespace to alter finalizing
# 2. change <my-cluster-id> to the cluster id from rancher url (e.g. https://my-rancher-ip/c/<my-cluster-id>/nodes
# 3 Replace the following commands in a text editor and run
(NAMESPACE=<my-namespace>; CLUSTER=c-fm5bd; kubectl proxy & kubectl get namespace $NAMESPACE -o json |jq '.spec = {"finalizers":[]}' >temp.json; curl -k -H "Content-Type: application/json" -X PUT --data-binary @temp.json 127.0.0.1:8001/k8s/clusters/$CLUSTER/api/v1/namespaces/$NAMESPACE/finalize; )
# Source: https://github.com/rancher/rancher/issues/16477
@kwokhou
Copy link
Author

kwokhou commented Sep 12, 2019

fix stuck terminating pv by patching the finalizers to null

kubectl patch pv pvc-abcd1234abcd1234 -p '{"metadata":{"finalizers":null}}'

or for pod

kubectl patch pod db-74755f6698-8td72 -p '{"metadata":{"finalizers":null}}'

source: kubernetes/kubernetes#69697 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment