[k8s.io] SchedulerPredicates [Serial] validates that taints-tolerations is respected if not matching {Kubernetes e2e suite} #33876

k8s-github-robot · 2016-10-01T10:12:45Z

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-e2e-gce-serial/2253/

Failed: [k8s.io] SchedulerPredicates [Serial] validates that taints-tolerations is respected if not matching {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:232
Expected error:
    <*errors.errorString | 0xc820b63fe0>: {
        s: "1 / 30 pods in namespace \"kube-system\" are NOT in the desired state in 5m0s\nPOD                     NODE                          PHASE   GRACE CONDITIONS\nkibana-logging-v1-j99la jenkins-e2e-minion-group-rx30 Running       [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:08:06 -0700 PDT  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:36:15 -0700 PDT ContainersNotReady containers with unready status: [kibana-logging]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:08:06 -0700 PDT  }]\n",
    }
    1 / 30 pods in namespace "kube-system" are NOT in the desired state in 5m0s
    POD                     NODE                          PHASE   GRACE CONDITIONS
    kibana-logging-v1-j99la jenkins-e2e-minion-group-rx30 Running       [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:08:06 -0700 PDT  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:36:15 -0700 PDT ContainersNotReady containers with unready status: [kibana-logging]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:08:06 -0700 PDT  }]

not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:225

Previous issues for this test: #27655

The text was updated successfully, but these errors were encountered:

k8s-github-robot · 2016-10-04T14:16:29Z

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-e2e-gce-serial/2274/

Failed: [k8s.io] SchedulerPredicates [Serial] validates that taints-tolerations is respected if not matching {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:1479
Not scheduled Pods: []api.Pod{api.Pod{TypeMeta:unversioned.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:api.ObjectMeta{Name:"still-no-tolerations", GenerateName:"", Namespace:"e2e-tests-sched-pred-pv88o", SelfLink:"/api/v1/namespaces/e2e-tests-sched-pred-pv88o/pods/still-no-tolerations", UID:"2de4417a-8a24-11e6-b2fe-42010af00002", ResourceVersion:"7335", Generation:0, CreationTimestamp:unversioned.Time{Time:time.Time{sec:63611176667, nsec:0, loc:(*time.Location)(0x50bbe40)}}, DeletionTimestamp:(*unversioned.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]api.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:api.PodSpec{Volumes:[]api.Volume{api.Volume{Name:"default-token-rlpqr", VolumeSource:api.VolumeSource{HostPath:(*api.HostPathVolumeSource)(nil), EmptyDir:(*api.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*api.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*api.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*api.GitRepoVolumeSource)(nil), Secret:(*api.SecretVolumeSource)(0xc820d8bd10), NFS:(*api.NFSVolumeSource)(nil), ISCSI:(*api.ISCSIVolumeSource)(nil), Glusterfs:(*api.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*api.PersistentVolumeClaimVolumeSource)(nil), RBD:(*api.RBDVolumeSource)(nil), Quobyte:(*api.QuobyteVolumeSource)(nil), FlexVolume:(*api.FlexVolumeSource)(nil), Cinder:(*api.CinderVolumeSource)(nil), CephFS:(*api.CephFSVolumeSource)(nil), Flocker:(*api.FlockerVolumeSource)(nil), DownwardAPI:(*api.DownwardAPIVolumeSource)(nil), FC:(*api.FCVolumeSource)(nil), AzureFile:(*api.AzureFileVolumeSource)(nil), ConfigMap:(*api.ConfigMapVolumeSource)(nil), VsphereVolume:(*api.VsphereVirtualDiskVolumeSource)(nil), AzureDisk:(*api.AzureDiskVolumeSource)(nil)}}}, InitContainers:[]api.Container(nil), Containers:[]api.Container{api.Container{Name:"still-no-tolerations", Image:"gcr.io/google_containers/pause-amd64:3.0", Command:[]string(nil), Args:[]string(nil), WorkingDir:"", Ports:[]api.ContainerPort(nil), Env:[]api.EnvVar(nil), Resources:api.ResourceRequirements{Limits:api.ResourceList(nil), Requests:api.ResourceList(nil)}, VolumeMounts:[]api.VolumeMount{api.VolumeMount{Name:"default-token-rlpqr", ReadOnly:true, MountPath:"/var/run/secrets/kubernetes.io/serviceaccount", SubPath:""}}, LivenessProbe:(*api.Probe)(nil), ReadinessProbe:(*api.Probe)(nil), Lifecycle:(*api.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", ImagePullPolicy:"IfNotPresent", SecurityContext:(*api.SecurityContext)(nil), Stdin:false, StdinOnce:false, TTY:false}}, RestartPolicy:"Always", TerminationGracePeriodSeconds:(*int64)(0xc821341508), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string{"kubernetes.io/e2e-label-key-2ddf36aa-8a24-11e6-af8a-0242ac110007":"testing-label-value"}, ServiceAccountName:"default", NodeName:"", SecurityContext:(*api.PodSecurityContext)(0xc820d9c180), ImagePullSecrets:[]api.LocalObjectReference(nil), Hostname:"", Subdomain:""}, Status:api.PodStatus{Phase:"Pending", Conditions:[]api.PodCondition{api.PodCondition{Type:"PodScheduled", Status:"False", LastProbeTime:unversioned.Time{Time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}}, LastTransitionTime:unversioned.Time{Time:time.Time{sec:63611176667, nsec:0, loc:(*time.Location)(0x50bbe40)}}, Reason:"Unschedulable", Message:""}}, Message:"", Reason:"", HostIP:"", PodIP:"", StartTime:(*unversioned.Time)(nil), InitContainerStatuses:[]api.ContainerStatus(nil), ContainerStatuses:[]api.ContainerStatus(nil)}}}
Expected
    <int>: 1
to equal
    <int>: 0
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:168

k8s-github-robot · 2016-10-06T11:13:23Z

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-e2e-gce-serial/2286/

Failed: [k8s.io] SchedulerPredicates [Serial] validates that taints-tolerations is respected if not matching {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:1479
Not scheduled Pods: []api.Pod{api.Pod{TypeMeta:unversioned.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:api.ObjectMeta{Name:"still-no-tolerations", GenerateName:"", Namespace:"e2e-tests-sched-pred-jvlpy", SelfLink:"/api/v1/namespaces/e2e-tests-sched-pred-jvlpy/pods/still-no-tolerations", UID:"0253ac66-8bad-11e6-8ef1-42010af00002", ResourceVersion:"22159", Generation:0, CreationTimestamp:unversioned.Time{Time:time.Time{sec:63611345386, nsec:0, loc:(*time.Location)(0x50761c0)}}, DeletionTimestamp:(*unversioned.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]api.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:api.PodSpec{Volumes:[]api.Volume{api.Volume{Name:"default-token-wqhpf", VolumeSource:api.VolumeSource{HostPath:(*api.HostPathVolumeSource)(nil), EmptyDir:(*api.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*api.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*api.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*api.GitRepoVolumeSource)(nil), Secret:(*api.SecretVolumeSource)(0xc8212c7320), NFS:(*api.NFSVolumeSource)(nil), ISCSI:(*api.ISCSIVolumeSource)(nil), Glusterfs:(*api.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*api.PersistentVolumeClaimVolumeSource)(nil), RBD:(*api.RBDVolumeSource)(nil), Quobyte:(*api.QuobyteVolumeSource)(nil), FlexVolume:(*api.FlexVolumeSource)(nil), Cinder:(*api.CinderVolumeSource)(nil), CephFS:(*api.CephFSVolumeSource)(nil), Flocker:(*api.FlockerVolumeSource)(nil), DownwardAPI:(*api.DownwardAPIVolumeSource)(nil), FC:(*api.FCVolumeSource)(nil), AzureFile:(*api.AzureFileVolumeSource)(nil), ConfigMap:(*api.ConfigMapVolumeSource)(nil), VsphereVolume:(*api.VsphereVirtualDiskVolumeSource)(nil), AzureDisk:(*api.AzureDiskVolumeSource)(nil)}}}, InitContainers:[]api.Container(nil), Containers:[]api.Container{api.Container{Name:"still-no-tolerations", Image:"gcr.io/google_containers/pause-amd64:3.0", Command:[]string(nil), Args:[]string(nil), WorkingDir:"", Ports:[]api.ContainerPort(nil), Env:[]api.EnvVar(nil), Resources:api.ResourceRequirements{Limits:api.ResourceList(nil), Requests:api.ResourceList(nil)}, VolumeMounts:[]api.VolumeMount{api.VolumeMount{Name:"default-token-wqhpf", ReadOnly:true, MountPath:"/var/run/secrets/kubernetes.io/serviceaccount", SubPath:""}}, LivenessProbe:(*api.Probe)(nil), ReadinessProbe:(*api.Probe)(nil), Lifecycle:(*api.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", ImagePullPolicy:"IfNotPresent", SecurityContext:(*api.SecurityContext)(nil), Stdin:false, StdinOnce:false, TTY:false}}, RestartPolicy:"Always", TerminationGracePeriodSeconds:(*int64)(0xc820db5178), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string{"kubernetes.io/e2e-label-key-02511586-8bad-11e6-9f3d-0242ac11000a":"testing-label-value"}, ServiceAccountName:"default", NodeName:"", SecurityContext:(*api.PodSecurityContext)(0xc820c03b00), ImagePullSecrets:[]api.LocalObjectReference(nil), Hostname:"", Subdomain:""}, Status:api.PodStatus{Phase:"Pending", Conditions:[]api.PodCondition{api.PodCondition{Type:"PodScheduled", Status:"False", LastProbeTime:unversioned.Time{Time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}}, LastTransitionTime:unversioned.Time{Time:time.Time{sec:63611345386, nsec:0, loc:(*time.Location)(0x50761c0)}}, Reason:"Unschedulable", Message:""}}, Message:"", Reason:"", HostIP:"", PodIP:"", StartTime:(*unversioned.Time)(nil), InitContainerStatuses:[]api.ContainerStatus(nil), ContainerStatuses:[]api.ContainerStatus(nil)}}}
Expected
    <int>: 1
to equal
    <int>: 0
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:168

k8s-github-robot · 2016-10-06T11:21:06Z

[FLAKE-PING] @derekwaynecarr

This flaky-test issue would love to have more attention.

derekwaynecarr · 2016-10-07T14:51:32Z

This looks like an error in bringing up a cluster and not this specific test.

derekwaynecarr · 2016-10-07T14:52:02Z

Hmm, actually, the last cases look like legitimiate flakes.

derekwaynecarr · 2016-10-07T14:52:11Z

@aveshagarwal -- can you investigate?

aveshagarwal · 2016-10-07T14:55:52Z

@derekwaynecarr sure.

aveshagarwal · 2016-10-07T18:12:29Z

This test case failed because one of the pods (although does not seem like this kibana pod is needed for this test) never became Running (Ready=true).

kibana-logging-v1-j99la  jenkins-e2e-minion-group-rx30  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:08:06 -0700 PDT  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:31:06 -0700 PDT ContainersNotReady containers with unready status: [kibana-logging]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:08:06 -0700 PDT  }]

Due to this, the actual test infact did not run at all and got out with the following error:

<*errors.errorString | 0xc820b63fe0>: {
          s: "1 / 30 pods in namespace \"kube-system\" are NOT in the desired state in 5m0s\nPOD                     NODE                          PHASE   GRACE CONDITIONS\nkibana-logging-v1-j99la jenkins-e2e-minion-group-rx30 Running       [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:08:06 -0700 PDT  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:36:15 -0700 PDT ContainersNotReady containers with unready status: [kibana-logging]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:08:06 -0700 PDT  }]\n",
      }
      1 / 30 pods in namespace "kube-system" are NOT in the desired state in 5m0s
      POD                     NODE                          PHASE   GRACE CONDITIONS
      kibana-logging-v1-j99la jenkins-e2e-minion-group-rx30 Running       [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:08:06 -0700 PDT  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:36:15 -0700 PDT ContainersNotReady containers with unready status: [kibana-logging]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2016-09-30 23:08:06 -0700 PDT  }]

aveshagarwal · 2016-10-07T19:10:19Z

It seems that kibana pod is one of the system pods that is needed to in Running (Read = true) for this test to go ahead and failure of the pod failed this test.

aveshagarwal · 2016-10-07T19:22:25Z

So for any test case in test/e2e/scheduler_predicates.go to even execute, BeforeEach should work without any error, otherwise the tests fail.

BeforeEach(
...
                err = framework.WaitForPodsRunningReady(c, api.NamespaceSystem, int32(systemPodsNo), framework.PodReadyBeforeTimeout, ignoreLabels)
                Expect(err).NotTo(HaveOccurred())
....
)

Automatic merge from submit-queue Fix confusing log messages While debugging #33876 , I noticed following confusing message: ``` The status of Pod kibana-logging-v1-j99la is Running, waiting for it to be either Running or Failed ```

k8s-github-robot · 2016-10-15T19:29:08Z

[FLAKE-PING] @aveshagarwal

This flaky-test issue would love to have more attention.

k8s-github-robot · 2016-10-23T19:36:52Z

[FLAKE-PING] @aveshagarwal

This flaky-test issue would love to have more attention.

k8s-github-robot assigned derekwaynecarr Oct 1, 2016

k8s-github-robot added kind/flake Categorizes issue or PR as related to a flaky test. priority/backlog Higher priority than priority/awaiting-more-evidence. labels Oct 1, 2016

k8s-github-robot mentioned this issue Oct 6, 2016

kubernetes-e2e-gci-gke-serial: broken test run #34087

Closed

k8s-github-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed priority/backlog Higher priority than priority/awaiting-more-evidence. labels Oct 6, 2016

k8s-github-robot mentioned this issue Oct 6, 2016

kubernetes-e2e-gci-gce-serial: broken test run #33647

Closed

derekwaynecarr assigned aveshagarwal and unassigned derekwaynecarr Oct 7, 2016

aveshagarwal mentioned this issue Oct 7, 2016

Fix confusing log messages #34346

Merged

This was referenced Oct 11, 2016

kubernetes-e2e-gce-serial: broken test run #34183

Closed

kubernetes-e2e-gke-serial: broken test run #31964

Closed

This was referenced Oct 15, 2016

kubernetes-e2e-gci-gke-serial: broken test run #34674

Closed

kubernetes-e2e-gci-gke-test: broken test run #34879

Closed

This was referenced Oct 25, 2016

kubernetes-e2e-gce-serial: broken test run #34679

Closed

kubernetes-e2e-gci-gke-subnet: broken test run #34902

Closed

kubernetes-e2e-gci-gke-pre-release: broken test run #34904

Closed

This was referenced Jan 12, 2017

ci-kubernetes-e2e-gci-gke-subnet: broken test run #39799

Closed

ci-kubernetes-e2e-gci-gke-prod: broken test run #39885

Closed

This was referenced Mar 11, 2017

ci-kubernetes-e2e-gci-gke-test: broken test run #40787

Closed

ci-kubernetes-soak-gke-test: broken test run #37898

Closed

ci-kubernetes-e2e-gci-gke-staging: broken test run #43037

Closed

ci-kubernetes-e2e-gce-serial: broken test run #43092

Closed

This was referenced Mar 21, 2017

ci-kubernetes-e2e-gci-gke-serial: broken test run #43434

Closed

ci-kubernetes-e2e-gke-serial: broken test run #43550

Closed

ci-kubernetes-e2e-gci-gke-prod: broken test run #43036

Closed

This was referenced Mar 30, 2017

ci-kubernetes-e2e-gke-test: broken test run #43549

Closed

ci-kubernetes-e2e-kops-aws-serial: broken test run #42602

Closed

k8s-github-robot mentioned this issue Apr 7, 2017

ci-kubernetes-e2e-gke-staging: broken test run #42792

Closed

This was referenced Apr 29, 2017

ci-kubernetes-e2e-gci-gke-test: broken test run #44699

Closed

ci-kubernetes-e2e-gke-serial: broken test run #45139

Closed

ci-kubernetes-e2e-gci-gke-subnet: broken test run #43283

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[k8s.io] SchedulerPredicates [Serial] validates that taints-tolerations is respected if not matching {Kubernetes e2e suite} #33876

[k8s.io] SchedulerPredicates [Serial] validates that taints-tolerations is respected if not matching {Kubernetes e2e suite} #33876

k8s-github-robot commented Oct 1, 2016

k8s-github-robot commented Oct 4, 2016

k8s-github-robot commented Oct 6, 2016

k8s-github-robot commented Oct 6, 2016

derekwaynecarr commented Oct 7, 2016

derekwaynecarr commented Oct 7, 2016

derekwaynecarr commented Oct 7, 2016

aveshagarwal commented Oct 7, 2016

aveshagarwal commented Oct 7, 2016

aveshagarwal commented Oct 7, 2016

aveshagarwal commented Oct 7, 2016

k8s-github-robot commented Oct 15, 2016

k8s-github-robot commented Oct 23, 2016

[k8s.io] SchedulerPredicates [Serial] validates that taints-tolerations is respected if not matching {Kubernetes e2e suite} #33876

[k8s.io] SchedulerPredicates [Serial] validates that taints-tolerations is respected if not matching {Kubernetes e2e suite} #33876

Comments

k8s-github-robot commented Oct 1, 2016

k8s-github-robot commented Oct 4, 2016

k8s-github-robot commented Oct 6, 2016

k8s-github-robot commented Oct 6, 2016

derekwaynecarr commented Oct 7, 2016

derekwaynecarr commented Oct 7, 2016

derekwaynecarr commented Oct 7, 2016

aveshagarwal commented Oct 7, 2016

aveshagarwal commented Oct 7, 2016

aveshagarwal commented Oct 7, 2016

aveshagarwal commented Oct 7, 2016

k8s-github-robot commented Oct 15, 2016

k8s-github-robot commented Oct 23, 2016