Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration test is broken due to swagger-checker breakage #27205

Closed
dchen1107 opened this issue Jun 10, 2016 · 3 comments
Closed

Integration test is broken due to swagger-checker breakage #27205

dchen1107 opened this issue Jun 10, 2016 · 3 comments
Assignees
@dchen1107
Labels
area/test-infra kind/flake Categorizes issue or PR as related to a flaky test. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Milestone
v1.3

Comments

@dchen1107
Copy link
Member

Several prs are failed with integration tests, here is one of the examples: https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/27185/kubernetes-pull-test-unit-integration/30033/build-log.txt

E0610 08:14:59.030127   26791 etcd_watcher.go:448] failure to decode api object: empty data
from &client.Response{Action:"delete", Node:(*client.Node)(0xc85164a420), PrevNode:(*client.Node)(0xc85164a480), Index:0x85f} &client.Node{Key:"/registry", Dir:true, Value:"", Nodes:client.Nodes(nil), CreatedIndex:0x856, ModifiedIndex:0x861, Expiration:(*time.Time)(nil), TTL:0}
E0610 08:14:59.030263   26791 etcd_watcher.go:448] failure to decode api object: empty data
from &client.Response{Action:"delete", Node:(*client.Node)(0xc851731d40), PrevNode:(*client.Node)(0xc851731da0), Index:0x85d} &client.Node{Key:"/registry", Dir:true, Value:"", Nodes:client.Nodes(nil), CreatedIndex:0x856, ModifiedIndex:0x861, Expiration:(*time.Time)(nil), TTL:0}
E0610 08:14:59.030312   26791 etcd_watcher.go:448] failure to decode api object: empty data
from &client.Response{Action:"delete", Node:(*client.Node)(0xc8515da1e0), PrevNode:(*client.Node)(0xc8515da240), Index:0x85e} &client.Node{Key:"/registry", Dir:true, Value:"", Nodes:client.Nodes(nil), CreatedIndex:0x856, ModifiedIndex:0x861, Expiration:(*time.Time)(nil), TTL:0}
E0610 08:14:59.030860   26791 etcd_watcher.go:448] failure to decode api object: empty data
from &client.Response{Action:"delete", Node:(*client.Node)(0xc851ca5b60), PrevNode:(*client.Node)(0xc851ca5bc0), Index:0x85b} &client.Node{Key:"/registry", Dir:true, Value:"", Nodes:client.Nodes(nil), CreatedIndex:0x856, ModifiedIndex:0x861, Expiration:(*time.Time)(nil), TTL:0}
E0610 08:14:59.032954   26791 etcd_watcher.go:448] failure to decode api object: empty data
from &client.Response{Action:"delete", Node:(*client.Node)(0xc8517fcf60), PrevNode:(*client.Node)(0xc8517fcfc0), Index:0x85d} &client.Node{Key:"/registry", Dir:true, Value:"", Nodes:client.Nodes(nil), CreatedIndex:0x856, ModifiedIndex:0x861, Expiration:(*time.Time)(nil), TTL:0}
W0610 08:14:59.403640   26791 genericapiserver.go:259] Network range for service cluster IPs is unspecified. Defaulting to 10.0.0.0/24.
I0610 08:14:59.403686   26791 genericapiserver.go:288] Node port range unspecified. Defaulting to 30000-32767.
E0610 08:14:59.434914   26791 util.go:45] Metric for serviceaccount_controller already registered
E0610 08:14:59.434992   26791 util.go:45] Metric for serviceaccount_controller already registered
W0610 08:14:59.435442   26791 request.go:347] Field selector: v1 - serviceaccounts - metadata.name - default: need to check if this is versioned correctly.
W0610 08:14:59.436757   26791 request.go:347] Field selector: v1 - serviceaccounts - metadata.name - default: need to check if this is versioned correctly.
W0610 08:14:59.884849   26791 controller.go:277] Resetting endpoints for master service "kubernetes" to kind:"" apiVersion:"" 
E0610 08:15:06.430508   26791 handlers.go:54] Unable to authenticate the request due to an error: Token has been invalidated
E0610 08:15:06.830341   26791 handlers.go:54] Unable to authenticate the request due to an error: Token has been invalidated
E0610 08:15:07.230527   26791 handlers.go:54] Unable to authenticate the request due to an error: Token has been invalidated
E0610 08:15:07.630456   26791 handlers.go:54] Unable to authenticate the request due to an error: Token has been invalidated
FAIL    k8s.io/kubernetes/test/integration  352.445s

I marked this as p0 since several 1.3 prs with lgtm are blocked by this. @caesarxuchao I don't think this is CSI issue, but please help with triage and re-assign. Thanks! cc/ @lavalamp
@dchen1107 dchen1107 added priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. kind/flake Categorizes issue or PR as related to a flaky test. labels Jun 10, 2016
@dchen1107 dchen1107 added this to the v1.3 milestone Jun 10, 2016
@dchen1107
Copy link
Member Author

Ok, I am taking over this one since jenkins job kubernetes-test-go is broken due to the same failure, which will block the merge queue completely.

@dchen1107 dchen1107 assigned dchen1107 and unassigned caesarxuchao Jun 10, 2016
@dchen1107 dchen1107 added area/test-infra and removed sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. labels Jun 10, 2016
@smarterclayton smarterclayton mentioned this issue Jun 10, 2016
Merged
@dchen1107
Copy link
Member Author

There are two issues mixing together which cause kubernetes-test-go failed

  1. First issue same as the one I posted in the initial issue description, which decoding issue and invalidate token are red-herring. The failed tests are below:

--- FAIL: TestPersistentVolumeMultiPVs (10.16s)
persistent_volumes_test.go:254: volumes created
persistent_volumes_test.go:260: claim created
persistent_volumes_test.go:264: volume bound
persistent_volumes_test.go:266: claim bound
persistent_volumes_test.go:284: Bind mismatch! Expected pvc-2 capacity 50000000000 but got fake-pvc-25 capacity 5000000000
=== RUN TestPersistentVolumeMultiPVsPVCs
--- FAIL: TestPersistentVolumeMultiPVsPVCs (3.26s)
persistent_volumes_test.go:379: PVC "pvc-5" is not bound

This is dup of #27179
I will leave #27129 to address the issue.

  1. The second cause is a recent breakage caused by
    15:37:38 FAILED ./hack/../hack/verify-swagger-spec.sh 56s

I will resolve this one.

@dchen1107 dchen1107 changed the title Integration test is broken due to invalidated token Integration test is broken due to swagger-checker breakage Jun 10, 2016
@dchen1107 dchen1107 mentioned this issue Jun 10, 2016
Merged
@dchen1107
Copy link
Member Author

cc/ @david-mcmahon I think your commit cce9db3 completely breaks swagger-checker. Unfortunately there is no pr associated with it. Luckily only your commit cce9db3 is on the top of that. I manually reverted both of your commits to stop the bleeding for now.

@david-mcmahon david-mcmahon mentioned this issue Jun 11, 2016
Merged
@smarterclayton smarterclayton mentioned this issue Jun 11, 2016
Merged
@david-mcmahon david-mcmahon mentioned this issue Jun 11, 2016
Merged
@therc therc mentioned this issue Jun 11, 2016
Merged
@davidopp davidopp mentioned this issue Jun 12, 2016
Merged
@smarterclayton smarterclayton mentioned this issue Jun 13, 2016
Merged
@david-mcmahon david-mcmahon mentioned this issue Aug 12, 2016
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dchen1107 dchen1107

Labels
area/test-infra kind/flake Categorizes issue or PR as related to a flaky test. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Projects
None yet
Milestone
v1.3
Development

No branches or pull requests
2 participants
@caesarxuchao @dchen1107