Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

node e2e flake: mirror pod failure #27779

Closed
caesarxuchao opened this issue Jun 21, 2016 · 9 comments
Closed

node e2e flake: mirror pod failure #27779

caesarxuchao opened this issue Jun 21, 2016 · 9 comments
Assignees
@yujuhong
Labels
kind/flake Categorizes issue or PR as related to a flaky test. sig/node Categorizes an issue or PR as relevant to SIG Node.

Comments

@caesarxuchao
Copy link
Member 

Summarizing 6 Failures:

[Fail] [k8s.io] MirrorPod when create a mirror pod  [BeforeEach] should be updated when static pod updated 
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e_node/mirror_pod_test.go:52

[Fail] [k8s.io] MirrorPod when create a mirror pod  [BeforeEach] should be recreated when mirror pod gracefully deleted 
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e_node/mirror_pod_test.go:52

[Fail] [k8s.io] MirrorPod when create a mirror pod  [BeforeEach] should be recreated when mirror pod forcibly deleted 
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e_node/mirror_pod_test.go:52

[Fail] [k8s.io] Kubelet when scheduling a busybox command in a pod [It] it should print the output to logs 
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e_node/kubelet_test.go:67

[Fail] [k8s.io] Kubelet when scheduling a read only busybox container [It] it should not write to root filesystem 
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e_node/kubelet_test.go:120

[Fail] [k8s.io] Kubelet metrics api when querying /stats/summary [It] it should report resource usage through the stats api 
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e_node/kubelet_test.go:212

Ran 20 of 23 Specs in 1126.176 seconds
FAIL! -- 14 Passed | 6 Failed | 0 Pending | 3 Skipped --- FAIL: TestE2eNode (1126.18s)
FAIL

log
@caesarxuchao caesarxuchao added the kind/flake Categorizes issue or PR as related to a flaky test. label Jun 21, 2016
@caesarxuchao caesarxuchao mentioned this issue Jun 21, 2016
Merged
@fejta fejta added the sig/node Categorizes an issue or PR as relevant to SIG Node. label Jun 21, 2016
@dchen1107 dchen1107 assigned yujuhong and unassigned dchen1107 Jun 21, 2016
@dchen1107
Copy link
Member

@yujuhong Can you take a look at this? Thanks!

@yujuhong
Copy link
Contributor

/cc @Random-Liu

@Random-Liu
Copy link
Member

@caesarxuchao The log link is dead. :(

@caesarxuchao
Copy link
Member Author

Copy-pasted in pastebin: http://pastebin.com/VpaEhj9m

@Random-Liu
Copy link
Member

@dchen1107 @yujuhong This is still a group of failure on coreos image. There is no docker or kernel log now, I guess it's #27498, but no evidence yet.

@yujuhong
Copy link
Contributor

Since we don't have other logs to triage the failure, let's close this issue. @Random-Liu, hanks for taking a look~

@Random-Liu
Copy link
Member

@yujuhong I found the kernel log, and found

Jun 21 04:50:24 localhost kernel: unregister_netdevice: waiting for lo to become free. Usage count = 2
Jun 21 04:50:24 localhost kernel: unregister_netdevice: waiting for vethb6dd68f to become free. Usage count = 1

Proved my guess. :)

@nikhiljindal nikhiljindal mentioned this issue Jun 25, 2016
Merged
@nikhiljindal
Copy link
Contributor

Some of these MirrorPod failures happened again: logs

@yujuhong
Copy link
Contributor

Some of these MirrorPod failures happened again: logs

There are multiple test failures (incl. but not limited to the mirror pod tests) and all on the coreos image. This seems similar to #27498

@caesarxuchao caesarxuchao mentioned this issue Jun 29, 2016
Merged
3 tasks
@erictune erictune mentioned this issue Jun 29, 2016
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yujuhong yujuhong

Labels
kind/flake Categorizes issue or PR as related to a flaky test. sig/node Categorizes an issue or PR as relevant to SIG Node.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@fejta @caesarxuchao @Random-Liu @dchen1107 @nikhiljindal @yujuhong