Skip to content

[Flake Test]: pull-kubernetes-integration failed #127830

Closed
@googs1025

Description

Which jobs are flaking?

pull-kubernetes-integration There is a probability of failure.
Not sure if this is a duplicate issue, I will close it if it is.

Which tests are flaking?

/test pull-kubernetes-integration

error log:

W1003 06:49:33.697366  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:49:33.699041  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
W1003 06:49:40.507612  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:49:40.509269  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
W1003 06:49:43.698684  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:49:43.700279  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
I1003 06:49:43.734211  113711 genericapiserver.go:711] [graceful-termination] apiserver is exiting
W1003 06:49:43.738577  113711 cacher.go:171] Terminating all watchers from cacher storageversions.internal.apiserver.k8s.io
W1003 06:49:43.739866  113711 cacher.go:171] Terminating all watchers from cacher leases.coordination.k8s.io
W1003 06:49:50.507595  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:49:50.509253  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
W1003 06:49:53.699094  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:49:53.700591  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
W1003 06:50:00.509613  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:50:00.511419  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
W1003 06:50:03.699618  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:50:03.701132  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
E1003 06:50:06.523440  113711 gc_controller.go:136] "Unhandled Error" err="lease apiserver-tcwjsvs6hjq4qbwzx5rvp5fsqe failed with: Get \"https://127.0.0.1:39011/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-tcwjsvs6hjq4qbwzx5rvp5fsqe\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
W1003 06:50:10.508149  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:50:10.509812  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
E1003 06:50:10.987647  113711 gc_controller.go:159] "Unhandled Error" err="storage version core.resourcequotas failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/core.resourcequotas\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:11.086271  113711 gc_controller.go:159] "Unhandled Error" err="storage version networking.k8s.io.ipaddresses failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/networking.k8s.io.ipaddresses\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:11.186977  113711 gc_controller.go:159] "Unhandled Error" err="storage version resource.k8s.io.podschedulingcontexts failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/resource.k8s.io.podschedulingcontexts\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:11.289258  113711 gc_controller.go:159] "Unhandled Error" err="storage version admissionregistration.k8s.io.validatingadmissionpolicies failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/admissionregistration.k8s.io.validatingadmissionpolicies\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:11.388759  113711 gc_controller.go:159] "Unhandled Error" err="storage version scheduling.k8s.io.priorityclasses failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/scheduling.k8s.io.priorityclasses\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:11.489218  113711 gc_controller.go:159] "Unhandled Error" err="storage version policy.poddisruptionbudgets failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/policy.poddisruptionbudgets\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:11.587109  113711 gc_controller.go:159] "Unhandled Error" err="storage version core.namespaces failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/core.namespaces\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:11.688151  113711 gc_controller.go:159] "Unhandled Error" err="storage version admissionregistration.k8s.io.mutatingwebhookconfigurations failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/admissionregistration.k8s.io.mutatingwebhookconfigurations\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:11.788929  113711 gc_controller.go:159] "Unhandled Error" err="storage version resource.k8s.io.resourceclaims failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/resource.k8s.io.resourceclaims\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:11.890817  113711 gc_controller.go:159] "Unhandled Error" err="storage version rbac.authorization.k8s.io.clusterrolebindings failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/rbac.authorization.k8s.io.clusterrolebindings\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:11.987173  113711 gc_controller.go:159] "Unhandled Error" err="storage version discovery.k8s.io.endpointslices failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/discovery.k8s.io.endpointslices\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:12.088048  113711 gc_controller.go:159] "Unhandled Error" err="storage version storage.k8s.io.volumeattributesclasses failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/storage.k8s.io.volumeattributesclasses\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:12.188119  113711 gc_controller.go:159] "Unhandled Error" err="storage version core.events failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/core.events\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:12.286678  113711 gc_controller.go:159] "Unhandled Error" err="storage version core.serviceaccounts failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/core.serviceaccounts\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:12.387715  113711 gc_controller.go:159] "Unhandled Error" err="storage version flowcontrol.apiserver.k8s.io.prioritylevelconfigurations failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/flowcontrol.apiserver.k8s.io.prioritylevelconfigurations\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:12.489864  113711 gc_controller.go:159] "Unhandled Error" err="storage version core.persistentvolumeclaims failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/core.persistentvolumeclaims\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:12.588692  113711 gc_controller.go:159] "Unhandled Error" err="storage version apiregistration.k8s.io.apiservices failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/apiregistration.k8s.io.apiservices\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:12.688268  113711 gc_controller.go:159] "Unhandled Error" err="storage version apps.deployments failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/apps.deployments\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:12.785431  113711 gc_controller.go:159] "Unhandled Error" err="storage version apiextensions.k8s.io.customresourcedefinitions failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/apiextensions.k8s.io.customresourcedefinitions\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
E1003 06:50:12.888139  113711 gc_controller.go:159] "Unhandled Error" err="storage version coordination.k8s.io.leasecandidates failed with: Get \"https://127.0.0.1:39011/apis/internal.apiserver.k8s.io/v1alpha1/storageversions/coordination.k8s.io.leasecandidates\": dial tcp 127.0.0.1:39011: connect: connection refused" logger="UnhandledError"
W1003 06:50:13.700224  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:50:13.701893  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
W1003 06:50:20.509471  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:50:20.511180  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
W1003 06:50:23.700556  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:50:23.702082  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
W1003 06:50:30.509257  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:50:30.510757  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
W1003 06:50:33.700744  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:50:33.702137  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
W1003 06:50:40.510423  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:50:40.511959  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
W1003 06:50:43.700980  113711 lease.go:265] Resetting endpoints for master service "kubernetes" to [127.0.0.1]
E1003 06:50:43.702537  113711 controller.go:163] "Unhandled Error" err="unable to sync kubernetes service: Endpoints \"kubernetes\" is invalid: subsets[0].addresses[0].ip: Invalid value: \"127.0.0.1\": may not be in the loopback range (127.0.0.0/8, ::1/128)" logger="UnhandledError"
I1003 06:50:43.753403  113711 peer_proxy_test.go:190] 
ServerB has retrieved jobs list of length 0 
    peer_proxy_test.go:191: 
        	Error Trace:	/home/prow/go/src/k8s.io/kubernetes/test/integration/apiserver/peerproxy/peer_proxy_test.go:191
        	Error:      	Received unexpected error:
        	            	timed out waiting for the condition
        	Test:       	TestPeerProxiedRequestToThirdServerAfterFirstDies
I1003 06:50:43.753562  113711 genericapiserver.go:546] "[graceful-termination] shutdown event" name="ShutdownInitiated"
I1003 06:50:43.753590  113711 genericapiserver.go:549] "[graceful-termination] shutdown event" name="AfterShutdownDelayDuration"
I1003 06:50:43.753595  113711 peer_endpoint_lease.go:175] Shutting down peer endpoint lease reconciler
I1003 06:50:43.755342  113711 controller.go:128] Shutting down kubernetes service endpoint reconciler
I1003 06:50:43.758636  113711 genericapiserver.go:691] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped"
I1003 06:50:43.758686  113711 genericapiserver.go:615] "[graceful-termination] shutdown event" name="NotAcceptingNewRequest"
I1003 06:50:43.758707  113711 genericapiserver.go:649] "[graceful-termination] not going to wait for active watch request(s) to drain"
I1003 06:50:43.758730  113711 controller.go:84] Shutting down OpenAPI AggregationController
I1003 06:50:43.758737  113711 remote_available_controller.go:427] Shutting down RemoteAvailability controller
I1003 06:50:43.758749  113711 genericapiserver.go:640] [graceful-termination] in-flight non long-running request(s) have drained
I1003 06:50:43.758759  113711 crd_finalizer.go:281] Shutting down CRDFinalizer
I1003 06:50:43.758778  113711 apiservice_controller.go:134] Shutting down APIServiceRegistrationController
I1003 06:50:43.758807  113711 autoregister_controller.go:168] Shutting down autoregister controller
I1003 06:50:43.758751  113711 customresource_discovery_controller.go:328] Shutting down DiscoveryController
I1003 06:50:43.758824  113711 local_available_controller.go:172] Shutting down LocalAvailability controller
I1003 06:50:43.758833  113711 controller.go:157] Shutting down quota evaluator
I1003 06:50:43.758837  113711 storage_flowcontrol.go:172] APF bootstrap ensurer is exiting
I1003 06:50:43.758842  113711 controller.go:132] Ending legacy_token_tracking_controller

Since when has it been flaking?

2024 10/4

Testgrid link

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/127726/pull-kubernetes-integration/1841725809274916864

Reason for failure (if possible)

No response

Anything else we need to know?

No response

Relevant SIG(s)

/sig

Metadata

Assignees

No one assigned

    Labels

    kind/flakeCategorizes issue or PR as related to a flaky test.needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.sig/api-machineryCategorizes an issue or PR as relevant to SIG API Machinery.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions