Fix federation e2e tests by correctly managing cluster clients #28211
Conversation
ghost
added
priority/critical-urgent
k8s-github-robot
added
the
size/M
|
@nikhiljindal FYI after rebasing, it looks like this probably ran into "federation: Upgrading the groupversion to v1beta1 #28186". It would be great if you could take a look when you get in in the morning. |
|
Come to think of it,I was running a new test binary against an old cluster, so that's probably the problem. I'll rebuild and recheck in the morning. |
|
GCE e2e build/test passed for commit 945c745. |
erictune
added
the
cherry-pick-approved
|
LGTM. It is a bad smell that it was so easy to do this and not notice in any way until we added the debug logging. Factoring this stuff into the framework will help, but there is probably also a rule of thumb that modifying shared state variables is error-prone, and you need to think about if/how to reset them between runs, not just how to "set things up". |
mml
added
the
lgtm
|
@mml, yes, absolutely. A few assertions along the way (probably in BeforeEach and AfterEach) would also help. In this case the number of API clients was not equal to the number of clusters. |
|
Automatic merge from submit-queue |
…-federation-e2e Automatic merge from submit-queue Fix federation e2e tests by correctly managing cluster clients 1. The main fix: Correct overall BeforeEach() to create a new set of cluster clients, rather than just append to the set created by all previous tests. This was screwing up a lot of stuff in difficult to diagnose ways. 2. Add lots of debug logging. 3. Be better about cleaning up after each test. ``` SUCCESS! -- 6 Passed | 0 Failed :-) ``` cc @nikhiljindal @madhusudancs @mfanjie @colhom FYI (cherry picked from commit 02de009)
Automatic merge from submit-queue Batch update for 1.3 #28030: Revert "Federation e2e supports aws" #28026: Address outstanding review comments in #27999. #28034: Adding lock files for kubeconfig updating #28004: return nil from NewClientConfig instead of empty struct #28032: Increase pod CPU/memory for fluentd, dns and kube-proxy. #27208: Bump minimum API version for docker to 1.21 #28061: Remove extra double quotes in --federations. #28060: rkt: Fix the 'privileged' check when stage1 annotation is provided. #27996: Image GC logic should compensate for reserved blocks #28044: rkt: Bump required rkt version to 1.9.1. #28040: Tracked addition of federation, sed support in kube DNS #28043: Set grace period to 0 when deleting namespaces after the test. #28002: Fix startup type error in initializeCaches #28087: Hotfix: Fixup the hyperkube dns manifest from a breaking federation PR #28108: Fix initialization of volume controller caches. #28056: Increase kube-dns requirements on CoreOS. #28147: Fix error checks after cloning. #28159: Use : as seccomp security option operator for Docker 1.10 #28165: Refactored, expanded and fixed federated-services e2e tests. #28095: Kubelet should mark VolumeInUse before checking if it is Attached #28172: Build: Add KUBE_GCS_RELEASE_BUCKET_MIRROR option to push-ci-build.sh #28207: Bump cluster autoscaler to 0.2.2 #28160: Volume manager must verify containers terminated before deleting for ungracefully terminated pods #28211: Fix federation e2e tests by correctly managing cluster clients #27944: Fix pvc label selector validation error #28186: federation: Upgrading the groupversion to v1beta1
|
Commit found in the "release-1.3" branch appears to be this PR. Removing the "cherrypick-candidate" label. If this is an error find help to get your PR picked. |
cc @nikhiljindal @madhusudancs @mfanjie @colhom FYI