Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[k8s.io] ConfigMap updates should be reflected in volume [Conformance] {Kubernetes e2e suite} #30352

Closed
k8s-github-robot opened this issue Aug 10, 2016 · 19 comments
Closed

[k8s.io] ConfigMap updates should be reflected in volume [Conformance] {Kubernetes e2e suite} #30352

k8s-github-robot opened this issue Aug 10, 2016 · 19 comments
Assignees
@Random-Liu @kevin-wangzefeng
Labels
kind/flake Categorizes issue or PR as related to a flaky test. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Milestone
v1.5

Comments

@k8s-github-robot
Copy link

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/30222/kubernetes-pull-build-test-e2e-gce/52694/

Failed: [k8s.io] ConfigMap updates should be reflected in volume [Conformance] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/configmap.go:149
Timed out after 300.000s.
Expected
    <string>: content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1
    content of file "/etc/configmap-volume/data-1": value-1

to contain substring
    <string>: value-2
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/configmap.go:148

Happened on a presubmit run in #30222.
@k8s-github-robot k8s-github-robot added priority/backlog Higher priority than priority/awaiting-more-evidence. kind/flake Categorizes issue or PR as related to a flaky test. labels Aug 10, 2016
@k8s-github-robot k8s-github-robot mentioned this issue Aug 11, 2016
Closed
This was referenced Aug 23, 2016
Closed
Closed
@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-e2e-gke/14169/

Failed: [k8s.io] ConfigMap updates should be reflected in volume [Conformance] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:132
Aug 26 21:04:10.768: Couldn't delete ns "e2e-tests-configmap-zrwgz": an error on the server has prevented the request from succeeding (delete namespaces e2e-tests-configmap-zrwgz)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:308

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @Random-Liu @kevin-wangzefeng

This flaky-test issue would love to have more attention...

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-e2e-gke/14350/

Failed: [k8s.io] ConfigMap updates should be reflected in volume [Conformance] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/configmap.go:160
Aug 29 17:12:12.311: unable to create test configMap : an error on the server has prevented the request from succeeding (post configmaps)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/configmap.go:101

@k8s-github-robot k8s-github-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed priority/backlog Higher priority than priority/awaiting-more-evidence. labels Aug 30, 2016
@bprashanth
Copy link
Contributor

The first failure looks like a legit fixable test problem, pod-configmaps-ba77374d-5edf-11e6-a759-42010af0003f was started on e2e-gce-agent-pr-27-0-minion-group-vnn0. I think the problem is the mounttester pod we use in the test. It polls for 2m and exits with 0, and the pod has RestartPolicy: Never, so it enters Phase: Succeeded;

I0810 09:52:12.764261    3514 status_manager.go:421] Status for pod "pod-configmaps-ba77374d-5edf-11e6-a759-42010af0003f_e2e-tests-configmap-h6h4b(ba78801b-5edf-11e6-a6b9-42010af00002)" updated successfully: {status:{Phase:Succeeded Conditions:[{Type:Initialized Status:True LastProbeTime:{Time:{sec:0 nsec:0 loc:0x49adde0}} LastTransitionTime:{Time:{sec:63606419368 nsec:0 loc:0x49adde0}} Reason:PodCompleted Message:} {Type:Ready Status:False LastProbeTime:{Time:{sec:0 nsec:0 loc:0x49adde0}} LastTransitionTime:{Time:{sec:63606419515 nsec:0 loc:0x49adde0}} Reason:PodCompleted Message:} {Type:PodScheduled Status:True LastProbeTime:{Time:{sec:0 nsec:0 loc:0x49adde0}} LastTransitionTime:{Time:{sec:63606419368 nsec:0 loc:0x49adde0}} Reason: Message:}] Message: Reason: HostIP:10.240.0.3 PodIP:10.180.1.46 StartTime:0xc8214a1c00 InitContainerStatuses:[] ContainerStatuses:[{Name:configmap-volume-test State:{Waiting:<nil> Running:<nil> Terminated:0xc821bda150} LastTerminationState:{Waiting:<nil> Running:<nil> Terminated:<nil>} Ready:false RestartCount:0 Image:gcr.io/google_containers/mounttest:0.6 ImageID:docker://sha256:92cfc941baf57c8cac60b8e9e2444a1560839d3c75a8e5c7c6237f3ecb08c971 ContainerID:docker://3476629f45533f4a01ea60da9c19970ddf4716703c37bb1e8425f6469564bf43}]} version:3 podName:pod-configmaps-ba77374d-5edf-11e6-a759-42010af0003f podNamespace:e2e-tests-configmap-h6h4b}

But after that the e2e polls on container logs searching for the modified configmap value.
The pod is: https://github.com/kubernetes/kubernetes/tree/master/test/images/mount-tester, we should make it loop forever till test kills it.

The previous 2 failures seem pretty random:

  • The last failure is a GKE failure and it failed POST. The configmap in question is configmap-test-upd-5987b44b-6e46-11e6-a085-0242ac110005, suggest looking through GKE master logs. We should also retry in the test once the problem is understood.
  • The failure before that, the test passed but we failed to delete namespace
STEP: Destroying namespace "e2e-tests-configmap-zrwgz" for this suite.
Aug 26 21:04:10.768: INFO: Couldn't delete ns "e2e-tests-configmap-zrwgz": an error on the server has prevented the request from succeeding (delete namespaces e2e-tests-configmap-zrwgz)

@bprashanth bprashanth mentioned this issue Aug 30, 2016
Closed
@k8s-github-robot
Copy link
Author

[FLAKE-PING] @Random-Liu @kevin-wangzefeng

This flaky-test issue would love to have more attention.

1 similar comment
@k8s-github-robot
Copy link
Author

[FLAKE-PING] @Random-Liu @kevin-wangzefeng

This flaky-test issue would love to have more attention.

This was referenced Sep 8, 2016
Closed
Closed
@k8s-github-robot
Copy link
Author

[FLAKE-PING] @Random-Liu @kevin-wangzefeng

This flaky-test issue would love to have more attention.

4 similar comments
@k8s-github-robot
Copy link
Author

[FLAKE-PING] @Random-Liu @kevin-wangzefeng

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @Random-Liu @kevin-wangzefeng

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @Random-Liu @kevin-wangzefeng

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @Random-Liu @kevin-wangzefeng

This flaky-test issue would love to have more attention.

This was referenced Sep 27, 2016
Closed
Closed
Closed
Closed
@k8s-github-robot
Copy link
Author

[FLAKE-PING] @Random-Liu @kevin-wangzefeng

This flaky-test issue would love to have more attention.

1 similar comment
@k8s-github-robot
Copy link
Author

[FLAKE-PING] @Random-Liu @kevin-wangzefeng

This flaky-test issue would love to have more attention.

@k8s-github-robot k8s-github-robot mentioned this issue Oct 10, 2016
Closed
This was referenced Dec 2, 2016
Closed
Closed
Closed
Closed
Closed
Closed
This was referenced Dec 9, 2016
Closed
Closed
Closed
Closed
Closed
This was referenced Dec 17, 2016
Closed
Closed
@k8s-github-robot k8s-github-robot mentioned this issue Dec 26, 2016
Closed
This was referenced Jan 2, 2017
Closed
Closed
Closed
Closed
This was referenced Jan 11, 2017
Closed
Closed
Closed
This was referenced Jan 26, 2017
Closed
Closed
Closed
This was referenced Mar 4, 2017
Closed
Closed
This was referenced Mar 16, 2017
Closed
Closed
@k8s-github-robot k8s-github-robot mentioned this issue Apr 15, 2017
Closed
@k8s-github-robot k8s-github-robot mentioned this issue Apr 28, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Random-Liu Random-Liu

@kevin-wangzefeng kevin-wangzefeng

Labels
kind/flake Categorizes issue or PR as related to a flaky test. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Milestone
v1.5
Development

No branches or pull requests
6 participants
@calebamiles @Random-Liu @kevin-wangzefeng @saad-ali @bprashanth @k8s-github-robot