Set topologyKey for podAntiAffinity rule to kubernetes.io/hostname #3189
Conversation
jandubois
force-pushed
the
CAP-1067-fix-pod-anti-affinity-rules
branch
from
02d51d4 to
e685e45
Compare
There was a problem hiding this comment.
Indeed, basing topology on OS seems… dumb.
Looking at the docs (scroll down a bit, there isn't a good anchor around), it looks like we may want to set it to empty instead (to mean some combination of hostname, zone, and region)?
I'm okay with just basing it on hostname, of course.
You would think that (and so did I), but it doesn't actually work: It was part of the original implementation in kube 1.7, but didn't work even then. kube 1.8 added validation that explicitly forbids this: kubernetes/kubernetes#49976 Pod scheduling in kube is still mostly an unsolved problem. Look at https://kubernetes.io/docs/concepts/scheduling/kube-scheduler/ for a list of all the things you can tweak, and maybe read https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/ for yet another feature to eventually reach resilient and balanced pod placement. Enter at your own risk... Read kubernetes/kubernetes#41598 if you ever wonder why the docs say:
Just some background on why I decided to set topology to hostname and called it a day... 😃 |
jandubois
force-pushed
the
CAP-1067-fix-pod-anti-affinity-rules
branch
from
e685e45 to
75f92ae
Compare
Also make it configurable via kube.podAntiAffinityTopologyKey. The original value of beta.kubernetes.io/os doesn't make any sense: it means to try to schedule each pod on a different OS. But our pods only work on Linux, and we don't expect to see any non-Linux nodes in the cluster anyways. [jsc#CAP-1067]
…mysql-proxy These seem to be copy&paste errors, or incomplete renames. All roles should have anti-affinity to themselves. [jsc#CAP-1067]
jandubois
force-pushed
the
CAP-1067-fix-pod-anti-affinity-rules
branch
from
75f92ae to
39c8fd1
Compare
Also make it configurable via kube.podAntiAffinityTopologyKey.
The original value of beta.kubernetes.io/os doesn't make any sense: it means to try to schedule each pod on a different OS. But our pods only work on Linux, and we don't expect to see any non-Linux nodes in the cluster anyways.
[jsc#CAP-1067]