Kubernetes v1.6.4 -> v1.7.0 update breaks statefulsets #48327
Description
/kind bug
What happened:
Updated from Kubernetes v1.6.4 environment to v1.7.0 environment with existing statefulsets. After seeing some errors regarding to scheduling I noticed repeated event:
8m 17s 32 statefulset Warning FailedUpdate update Pod affinity-test-0 in StatefulSet affinity-test failed error: Pod "affinity-test-0" is invalid: spec: Forbidden: pod updates may not change fields other than
spec.containers[*].image
,spec.initContainers[*].image
,spec.activeDeadlineSeconds
orspec.tolerations
(only additions to existing tolerations)
It seems that the statefulset is trying to update all its pods but with something that is immutable. Trying to figure out what and my current best guess is serviceAccount (which was not there previously when it was created).
What you expected to happen:
Delete and re-create the pods since it is changing immutable fields
How to reproduce it (as minimally and precisely as possible):
- create statefulset in 1.6.4 with some features missing (SA?)
- update to v1.7.0
I have logs from controller-manager but they are rather spammy at --v=8 so if you have something to look for - here is what I guessed should be relevant (private info removed): https://gist.github.com/CallMeFoxie/353971434e20a3048b368d3fc835ab76
Environment:
- Kubernetes version (use
kubectl version
): v1.6.4 -> v1.7.0 - Cloud provider or hardware configuration**: self-hosted
- Install tools: manual