Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Possible e2e flake: Services [It] should be able to up and down services #23668

Closed
cjcullen opened this issue Mar 31, 2016 · 1 comment
Closed

Possible e2e flake: Services [It] should be able to up and down services #23668

cjcullen opened this issue Mar 31, 2016 · 1 comment

Comments

@cjcullen
Copy link
Member 

Failure [270.683 seconds]
[k8s.io] Services
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework.go:420
  should be able to up and down services [It]
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:276

  Expected error:
      <*errors.errorString | 0xc20904ddb0>: {
          s: "Only 2 pods started out of 3",
      }
      Only 2 pods started out of 3
  not to have occurred

Full log
Saw this on the PR builder for #23646. Might be the PR's fault. Filing this to remind myself to follow up.
@cjcullen cjcullen mentioned this issue Mar 31, 2016
Merged
@bprashanth
Copy link
Contributor

That looks like image pulling hang, the pod in question is service1-do4cj and it landed on node https://pantheon.corp.google.com/storage/browser/kubernetes-jenkins/pr-logs/pull/23646/kubernetes-pull-build-test-e2e-gce/34008/artifacts/e2e-gce-builder-3-0-minion-ki6z/?pli=1

I0331 01:39:19.438955    3456 server.go:609] Event(api.ObjectReference{Kind:"Pod", Namespace:"e2e-tests-services-7l7v3", Name:"service1-do4cj", UID:"628d1481-f6e1-11e5-9a1f-42010af00002", APIVersion:"v1", ResourceVersion:"1880", FieldPath:"spec.containers{service1}"}): type: 'Normal' reason: 'Pulling' pulling image 
...
I0331 01:43:39.364324    3456 server.go:609] Event(api.ObjectReference{Kind:"Pod", Namespace:"e2e-tests-services-7l7v3", Name:"service1-do4cj", UID:"628d1481-f6e1-11e5-9a1f-42010af00002", APIVersion:"v1", ResourceVersion:"1880", FieldPath:"spec.containers{service1}"}): type: 'Normal' reason: 'Pulled' Successfully pulled image "gcr.io/google_containers/serve_hostname:v1.4"

And of course I see the registry endpoint ping failures

v2 ping attempt failed with error: Get https://gcr.io/v2/: dial tcp: lookup gcr.io on 169.254.169.254:53: dial udp 169.254.169.254:53: network is unreachable
 v1 ping attempt failed with error: Get https://gcr.io/v1/_ping: dial tcp: lookup gcr.io on 169.254.169.254:53: dial udp 169.254.169.254:53: network is unreachable
I0331 01:38:44.179238    3456 server.go:609] Event(api.ObjectReference{Kind:"Pod", Namespace:"e2e-tests-kubectl-hf6bt", Name:"update-demo-nautilus-emehh", UID:"4acf79b7-f6e1-11e5-9a1f-42010af00002", APIVersion:"v1", ResourceVersion:"1026", FieldPath:""}): type: 'Warning' reason: 'FailedSync' Error syncing pod, skipping: failed to "StartContainer" for "update-demo" with ErrImagePull: "unable to ping registry endpoint https://gcr.io/v0/\nv2 ping attempt failed with error: Get https://gcr.io/v2/: dial tcp: lookup gcr.io on 169.254.169.254:53: dial udp 169.254.169.254:53: network is unreachable\n v1 ping attempt failed with error: Get https://gcr.io/v1/_ping: dial tcp: lookup gcr.io on 169.254.169.254:53: dial udp 169.254.169.254:53: network is unreachable"

So closing as dupe of #20550

@bprashanth bprashanth mentioned this issue Mar 31, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@a-robinson @cjcullen @bprashanth