Skip to content

[upgrade test failure] Restart [Disruptive] should restart all nodes and ensure all nodes and pods recover #50797

Closed
@ericchiang

Description

Opening this since this seems slightly different from #46651

/cc @kubernetes/sig-node-bugs

This tests has been consistently failing on a lot of the upgrade tests:

https://k8s-testgrid.appspot.com/master-upgrade#gke-cvm-1.7-gci-master-upgrade-master
https://k8s-testgrid.appspot.com/master-upgrade#gke-cvm-1.7-gci-master-upgrade-cluster
https://k8s-testgrid.appspot.com/master-upgrade#gke-gci-1.7-cvm-master-upgrade-master
https://k8s-testgrid.appspot.com/master-upgrade#gke-gci-1.7-cvm-master-upgrade-cluster
https://k8s-testgrid.appspot.com/master-upgrade#gke-gci-1.7-gci-master-upgrade-master
https://k8s-testgrid.appspot.com/master-upgrade#gke-gci-1.7-gci-master-upgrade-cluster

What's really weird is the error message most of them spit out

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/restart.go:92
Expected error:
    <*errors.errorString | 0xc422322770>: {
        s: "couldn't find -1 nodes within 20s; last error: expected to find -1 nodes but found only 3 (20.007524987s elapsed)",
    }
    couldn't find -1 nodes within 20s; last error: expected to find -1 nodes but found only 3 (20.007524987s elapsed)
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/restart.go:77

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-cvm-new-cvm-master-upgrade-cluster/128#k8sio-restart-disruptive-should-restart-all-nodes-and-ensure-all-nodes-and-pods-recover

Really hard to be clear who owns this tests, going to tag sig-node until there's further evidence otherwise.

cc @kubernetes/kubernetes-release-managers @mbohlool

Metadata

Labels

kind/bugCategorizes issue or PR as related to a bug.priority/critical-urgentHighest priority. Must be actively worked on as someone's top priority right now.sig/cluster-lifecycleCategorizes an issue or PR as relevant to SIG Cluster Lifecycle.sig/nodeCategorizes an issue or PR as relevant to SIG Node.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions