Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration test flake: build failed #30462

Closed
janetkuo opened this issue Aug 11, 2016 · 13 comments
Closed

Integration test flake: build failed #30462

janetkuo opened this issue Aug 11, 2016 · 13 comments
Assignees
@spxtr
Labels
area/test-infra kind/flake Categorizes issue or PR as related to a flaky test. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.

Comments

@janetkuo
Copy link
Member

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/28509/kubernetes-pull-test-unit-integration/38712/

ok      k8s.io/kubernetes/test/integration/metrics  0.467s
ok      k8s.io/kubernetes/test/integration/openshift    0.123s
ok      k8s.io/kubernetes/test/integration/persistentvolumes    60.227s
ok      k8s.io/kubernetes/test/integration/pods 4.069s
FAIL    k8s.io/kubernetes/test/integration/quota [build failed]
ok      k8s.io/kubernetes/test/integration/replicaset   170.725s
ok      k8s.io/kubernetes/test/integration/replicationcontroller    170.933s
ok      k8s.io/kubernetes/test/integration/scheduler    80.066s
FAIL    k8s.io/kubernetes/test/integration/secrets [build failed]
ok      k8s.io/kubernetes/test/integration/serviceaccount   24.485s
FAIL    k8s.io/kubernetes/test/integration/storageclasses [build failed]
+++ [0810 14:26:11] Saved JUnit XML test report to /workspace/artifacts/junit_v1,authorization.k8s.io-v1beta1,autoscaling-v1,batch-v1,apps-v1alpha1,policy-v1alpha1,extensions-v1beta1,rbac.authorization.k8s.io-v1alpha1,certificates-v1alpha1_20160810-142114.xml
Makefile:118: recipe for target 'test' failed
make[1]: *** [test] Error 1
+++ [0810 14:26:11] Cleaning up etcd
+++ [0810 14:26:11] Integration test cleanup complete
Makefile:126: recipe for target 'test-integration' failed
make: *** [test-integration] Error 1
Build step 'Execute shell' marked build as failure
[xUnit] [INFO] - Starting to record.
[xUnit] [INFO] - Processing JUnit
[xUnit] [INFO] - [JUnit] - 2 test report file(s) were found with the pattern '_artifacts/**.xml' relative to '/var/lib/jenkins/workspace/kubernetes-pull-test-unit-integration' for the testing framework 'JUnit'.
[xUnit] [INFO] - Check 'Failed Tests' threshold.
@janetkuo janetkuo added the kind/flake Categorizes issue or PR as related to a flaky test. label Aug 11, 2016
@janetkuo janetkuo mentioned this issue Aug 11, 2016
Merged
@caesarxuchao
Copy link
Member

https://console.cloud.google.com/storage/browser/kubernetes-jenkins/pr-logs/pull/30483/kubernetes-pull-test-unit-integration/39010/artifacts/

Another occurrence:

FAIL    k8s.io/kubernetes/test/integration/replicationcontroller [build failed]

@caesarxuchao caesarxuchao mentioned this issue Aug 11, 2016
Closed
@derekwaynecarr derekwaynecarr mentioned this issue Aug 13, 2016
Merged
This was referenced Aug 14, 2016
Merged
Merged
This was referenced Aug 15, 2016
Merged
Closed
Closed
Merged
@Random-Liu
Copy link
Member

This flakes so much, :(

@Q-Lee Q-Lee mentioned this issue Aug 16, 2016
Merged
@nikhiljindal nikhiljindal mentioned this issue Aug 16, 2016
Merged
@nikhiljindal
Copy link
Contributor

nikhiljindal commented Aug 16, 2016 •
edited
Loading

Happened again in: https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/30655/kubernetes-pull-test-unit-integration/39791/build-log.txt.

Looks like this is happening only on PR builder and not on post commit Jenkins (otherwise the bot would have filed an issue and nag it continuously)?

@janetkuo janetkuo added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. area/test-infra labels Aug 16, 2016
@mml
Copy link
Contributor

mml commented Aug 16, 2016 

FAIL    k8s.io/kubernetes/test/integration/metrics [build failed]

@mml mml mentioned this issue Aug 16, 2016
Merged
@mml mml added priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. and removed priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Aug 16, 2016
@mml mml assigned rmmh Aug 16, 2016
@mml
Copy link
Contributor

mml commented Aug 16, 2016

Assigning to test-infra oncall. @rmmh this certainly looks like an infra problem.

@mml
Copy link
Contributor

mml commented Aug 16, 2016

The "killed" line looks relevant. Don't see anything in artifacts/ explaining it, though.

+++ [0816 15:41:58] Running tests without code coverage
go build testmain: /usr/local/go/pkg/tool/linux_amd64/link: signal: killed
ok      k8s.io/kubernetes/test/integration/auth 1.382s
ok      k8s.io/kubernetes/test/integration/client       7.486s
ok      k8s.io/kubernetes/test/integration/configmap    1.744s
ok      k8s.io/kubernetes/test/integration/garbagecollector     226.899s
[restful] 2016/08/16 15:43:08 log.go:30: [restful/swagger] listing is available at https:///swaggerapi/
[restful] 2016/08/16 15:43:08 log.go:30: [restful/swagger] https:///swaggerui/ is mapped to folder /swagger-ui/
ok      k8s.io/kubernetes/test/integration/kubectl      2.528s
ok      k8s.io/kubernetes/test/integration/master       3.146s
FAIL    k8s.io/kubernetes/test/integration/metrics [build failed]
[...]

@caesarxuchao caesarxuchao mentioned this issue Aug 17, 2016
Merged
@liggitt liggitt mentioned this issue Aug 17, 2016
Merged
@dchen1107 dchen1107 mentioned this issue Aug 17, 2016
Merged
@bgrant0607 bgrant0607 mentioned this issue Aug 17, 2016
Merged
@derekwaynecarr derekwaynecarr mentioned this issue Aug 17, 2016
Merged
@smarterclayton
Copy link
Contributor

This is running out of memory building go code. We can address it by passing -p N to the go test / go build scripts that reduces the number of parallel builds happening during test. We set this in test-go, but probably not in integration test-go.

@smarterclayton smarterclayton mentioned this issue Aug 17, 2016
Merged
@liggitt liggitt mentioned this issue Aug 18, 2016
Merged
@saad-ali saad-ali mentioned this issue Aug 18, 2016
Merged
This was referenced Aug 18, 2016
Merged
Merged
@yujuhong yujuhong mentioned this issue Aug 19, 2016
Merged
@nikhiljindal
Copy link
Contributor

nikhiljindal commented Aug 19, 2016 •
edited
Loading

Its test/integration/pods this time: https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/30823/kubernetes-pull-test-unit-integration/40506/build-log.txt
Ref #30823

@dchen1107 dchen1107 mentioned this issue Aug 20, 2016
Merged
@fejta fejta assigned spxtr and unassigned rmmh Aug 22, 2016
@fejta
Copy link
Contributor

fejta commented Aug 22, 2016

@spxtr did switching off highcpu instances resolve this issue?

@janetkuo janetkuo mentioned this issue Aug 22, 2016
Closed
@spxtr
Copy link
Contributor

spxtr commented Aug 22, 2016

I'm not sure, but it looks related. That issue was #30407. I'll close this, please reopen if it reoccurs.

@spxtr spxtr closed this as completed Aug 22, 2016
@brendandburns
Copy link
Contributor

Re-opening since this re-occured here:
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/31607/kubernetes-pull-test-unit-integration/43802/

@brendandburns brendandburns reopened this Sep 9, 2016
@brendandburns brendandburns mentioned this issue Sep 9, 2016
Merged
@fejta
Copy link
Contributor

fejta commented Sep 9, 2016

This is not a flake. The commit does not compile (be sure to expand the skipped lines):

ok      k8s.io/kubernet# k8s.io/kubernetes/pkg/kubectl
pkg/kubectl/describe_test.go:610: undefined: extensions.StorageClass
pkg/kubectl/describe_test.go:733: undefined: extensions.StorageClass

@fejta fejta closed this as completed Sep 9, 2016
@mml mml reopened this Sep 9, 2016
@mml mml closed this as completed Sep 9, 2016
@sbezverk
Copy link
Contributor

Keeps failing: FAIL: pull-kubernetes-verify on a tiny change...
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/49598/pull-kubernetes-verify/41706/
Could you please check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spxtr spxtr

Labels
area/test-infra kind/flake Categorizes issue or PR as related to a flaky test. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests