Skip to content

Instantly share code, notes, and snippets.

@lachie83
Last active August 23, 2018 07:36
Show Gist options
  • Save lachie83/bfe4a7d3c7a6d3430753 to your computer and use it in GitHub Desktop.
Save lachie83/bfe4a7d3c7a6d3430753 to your computer and use it in GitHub Desktop.
K8s NodeSelector usage
# List running nodes
docker run -it --rm --volumes-from data levenson/kubectl get nodes
NAME LABELS STATUS
10.161.34.62 kubernetes.io/hostname=10.161.34.62 Ready
10.161.34.65 kubernetes.io/hostname=10.161.34.65 Ready
10.161.34.70 kubernetes.io/hostname=10.161.34.70 Ready
# Label nodes with name key
docker run -it --rm --volumes-from data levenson/kubectl label nodes 10.161.34.62 name=node-1
docker run -it --rm --volumes-from data levenson/kubectl label nodes 10.161.34.65 name=node-2
docker run -it --rm --volumes-from data levenson/kubectl label nodes 10.161.34.70 name=node-3
# Confirm labels applied to nodes
docker run -it --rm --volumes-from data levenson/kubectl get nodes
NAME LABELS STATUS
10.161.34.62 kubernetes.io/hostname=10.161.34.62,name=node-1 Ready
10.161.34.65 kubernetes.io/hostname=10.161.34.65,name=node-2 Ready
10.161.34.70 kubernetes.io/hostname=10.161.34.70,name=node-3 Ready
# Create kafka-1 list,svc,pod.
docker run -it --rm --volumes-from data levenson/kubectl create -f /data/kafka-1.json
{
"kind": "List",
"apiVersion": "v1",
"id": "kafka",
"items":[
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "kafka-1",
"namespace": "realtime",
"labels": {
"name": "kafa-1"
}
},
"spec": {
"ports": [
{
"name": "service",
"port": 9092,
"targetPort": 9092
},
{
"name": "jmx",
"port": 7203,
"targetPort": 7203
}
],
"selector": {
"server-id": "1"
}
}
},
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "kafka-1",
"namespace": "realtime",
"labels": {
"name": "kafka",
"server-id": "1"
}
},
"spec": {
"containers": [
{
"name": "server",
"image": "levenson/k8s-kafka",
"env":[
{ "name": "KAFKA_SERVER_ID", "value": "1" },
{ "name": "KAFKA_CLIENT_1_SERVICE_HOST", "value": "kafka-1.realtime.svc.kubernetes.io" },
{ "name": "KAFKA_CLIENT_1_SERVICE_PORT", "value": "9093" },
{ "name": "ZK_CLIENT_1_SERVICE_HOST", "value": "zookeeper.realtime.svc.kubernetes.io" },
{ "name": "ZK_CLIENT_1_SERVICE_PORT", "value": "2181" }
],
"volumeMounts": [
{
"name": "kafka-1-data",
"mountPath": "/data/"
}
],
"ports":[
{
"containerPort": 9092
},
{
"containerPort": 7203
}
]
}
],
"nodeSelector": {
"name": "node-1"
},
"volumes": [
{
"name": "kafka-1-data",
"hostPath": {
"path": "/data/kafka-1/data"
}
}
]
}
}
]
}
# Confirm kafka-1 is running on node with label name=node-1 -- No it's running on node labelled name=node-2
docker run -it --rm --volumes-from data levenson/kubectl get pods kafka-1 -o wide --namespace=realtime
NAME READY STATUS RESTARTS AGE NODE
kafka-1 1/1 Running 0 1h 10.161.34.65
@lachie83
Copy link
Author

Updated nodeSelector from node to name key and recreated pod. Same issue.

docker run -it --rm --volumes-from data levenson/kubectl get pods kafka-1 -o wide --namespace=realtime
NAME READY STATUS RESTARTS AGE NODE
kafka-1 0/1 Running 0 9s 10.161.34.65

@lachie83
Copy link
Author

After moving nodeSelector under the Pod spec rather than the container spec

docker run -it --rm --volumes-from data levenson/kubectl --validate create -f /data/kafka-1.json
error validating data: couldn't find type: v1.List
docker run -it --rm --volumes-from data levenson/kubectl create -f /data/kafka-1.json
services/kafka-1
pods/kafka-1
docker run -it --rm --volumes-from data levenson/kubectl get pods kafka-1 -o wide --namespace=realtime -o wide
NAME      READY     STATUS    RESTARTS   AGE       NODE
kafka-1   0/1       Running   0          6s        10.161.34.65

@lachie83
Copy link
Author

Found the issue was in the context placement of the nodeSelector key. I isolated using --validate. This is now working as expected.

@satoshi75nakamoto
Copy link

@lachie83 awesome this is great news. Do the docs need to be updated?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment