Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kubernetes 1.32: Informer with WatchClient fails to send events with Fakeclient #129408

Open
dilipmighty245 opened this issue Dec 26, 2024 · 2 comments
Open

kubernetes 1.32: Informer with WatchClient fails to send events with Fakeclient #129408

dilipmighty245 opened this issue Dec 26, 2024 · 2 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.

Comments

@dilipmighty245
Copy link

What happened?

I tried to start an informer with WatchClient enabled in client go and i see this error

W1220 19:44:49.417766   26071 reflector.go:1044] k8s.io/client-go@v1.32/tools/cache/reflector.go:243: awaiting required bookmark event for initial events stream, no events received for 10.000196792s

and the above error keeps coming every 10 seconds.

The informer factory is stuck at this line

xInformerFactory.WaitForCacheSync(ctx.Done())

What did you expect to happen?

Watclist feature to work and Events to propagate when a resource is created.

How can we reproduce it (as minimally and precisely as possible)?

package main

import (
	"context"
	"flag"
	"fmt"
	"os"
    "time"

	corev1 "k8s.io/api/core/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/runtime"
	utilruntime "k8s.io/apimachinery/pkg/util/runtime"
	"k8s.io/client-go/informers"
	"k8s.io/client-go/kubernetes/fake"
	"k8s.io/client-go/tools/cache"
	"k8s.io/klog/v2"
)

func init() {
	scheme := runtime.NewScheme()
	utilruntime.Must(corev1.AddToScheme(scheme))

}

func main() {
	os.Setenv("KUBE_FEATURE_WatchListClient", "true")
	client := fake.NewSimpleClientset()

	defer klog.Flush()
	flagSet := flag.NewFlagSet("test", flag.ExitOnError)
	klog.InitFlags(flagSet)
	_ = flagSet.Parse([]string{"--v", "6"})

	// Create an informer factory for the fake client
	informerFactory := informers.NewSharedInformerFactory(client, 0)

	// Get the Pod informer
	podInformer := informerFactory.Core().V1().Pods().Informer()

	// Add an event handler to the informer
	podInformer.AddEventHandler(&cache.ResourceEventHandlerFuncs{
		AddFunc: func(obj interface{}) {
			pod := obj.(*corev1.Pod)
			fmt.Printf("Pod added: %s/%s\n", pod.Namespace, pod.Name)
		},
		UpdateFunc: func(oldObj, newObj interface{}) {
			newPod := newObj.(*corev1.Pod)
			fmt.Printf("Pod updated: %s/%s\n", newPod.Namespace, newPod.Name)
		},
		DeleteFunc: func(obj interface{}) {
			pod := obj.(*corev1.Pod)
			fmt.Printf("Pod deleted: %s/%s\n", pod.Namespace, pod.Name)
		},
	})

	// Start the informer
	stopCh := make(chan struct{})
	defer close(stopCh)
	informerFactory.Start(stopCh)

	// Wait for cache sync
	if !cache.WaitForCacheSync(stopCh, podInformer.HasSynced) {
		fmt.Println("Failed to sync cache")
		return
	}
	// Use the client to create, update, or fetch resources
	pod := &corev1.Pod{
		ObjectMeta: metav1.ObjectMeta{
			Name:      "example-pod",
			Namespace: "default",
		},
		Spec: corev1.PodSpec{
			Containers: []corev1.Container{
				{
					Name:  "example-container",
					Image: "nginx",
				},
			},
		},
	}

	// Create a Pod
	_, err := client.CoreV1().Pods("default").Create(context.TODO(), pod, metav1.CreateOptions{})
	if err != nil {
		panic(err)
	}

	
	time.Sleep(time.Second * 5) 
	stopCh <- struct{}{}
}

go.mod file:

module myworks/watchlist-test

go 1.23.0

require (
	k8s.io/api v0.32.0
	k8s.io/apimachinery v0.32.0
	k8s.io/client-go v0.32.0
	k8s.io/klog/v2 v2.130.1
)

require (
	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
	github.com/emicklei/go-restful/v3 v3.11.0 // indirect
	github.com/fxamacker/cbor/v2 v2.7.0 // indirect
	github.com/go-logr/logr v1.4.2 // indirect
	github.com/go-openapi/jsonpointer v0.21.0 // indirect
	github.com/go-openapi/jsonreference v0.20.2 // indirect
	github.com/go-openapi/swag v0.23.0 // indirect
	github.com/gogo/protobuf v1.3.2 // indirect
	github.com/golang/protobuf v1.5.4 // indirect
	github.com/google/gnostic-models v0.6.8 // indirect
	github.com/google/go-cmp v0.6.0 // indirect
	github.com/google/gofuzz v1.2.0 // indirect
	github.com/google/uuid v1.6.0 // indirect
	github.com/josharian/intern v1.0.0 // indirect
	github.com/json-iterator/go v1.1.12 // indirect
	github.com/mailru/easyjson v0.7.7 // indirect
	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
	github.com/modern-go/reflect2 v1.0.2 // indirect
	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
	github.com/pkg/errors v0.9.1 // indirect
	github.com/x448/float16 v0.8.4 // indirect
	golang.org/x/net v0.30.0 // indirect
	golang.org/x/oauth2 v0.23.0 // indirect
	golang.org/x/sys v0.26.0 // indirect
	golang.org/x/term v0.25.0 // indirect
	golang.org/x/text v0.19.0 // indirect
	golang.org/x/time v0.7.0 // indirect
	google.golang.org/protobuf v1.35.1 // indirect
	gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
	gopkg.in/inf.v0 v0.9.1 // indirect
	gopkg.in/yaml.v3 v3.0.1 // indirect
	k8s.io/kube-openapi v0.0.0-20241105132330-32ad38e42d3f // indirect
	k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 // indirect
	sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect
	sigs.k8s.io/structured-merge-diff/v4 v4.4.2 // indirect
	sigs.k8s.io/yaml v1.4.0 // indirect
)

Error logs:

go run main.go
I1223 14:40:04.018354   35133 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false
I1223 14:40:04.018890   35133 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false
I1223 14:40:04.018897   35133 envvar.go:169] "Feature gate updated state" feature="WatchListClient" enabled=true
I1223 14:40:04.018901   35133 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false
I1223 14:40:04.018930   35133 reflector.go:313] Starting reflector *v1.Pod (0s) from pkg/mod/k8s.io/client-go@v0.32.0/tools/cache/reflector.go:251
I1223 14:40:04.018968   35133 reflector.go:349] Listing and watching *v1.Pod from pkg/mod/k8s.io/client-go@v0.32.0/tools/cache/reflector.go:251
W1223 14:40:14.020097   35133 reflector.go:1052] pkg/mod/k8s.io/client-go@v0.32.0/tools/cache/reflector.go:251: awaiting required bookmark event for initial events stream, no events received for 10.000426333s
W1223 14:40:24.019566   35133 reflector.go:1052] pkg/mod/k8s.io/client-go@v0.32.0/tools/cache/reflector.go:251: awaiting required bookmark event for initial events stream, no events received for 20.000542791s

Anything else we need to know?

change the env var to false and you see things are working.

os.Setenv("KUBE_FEATURE_WatchListClient", "false")

Logs:

go run main.go
I1223 14:45:47.374375   39780 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false
I1223 14:45:47.374853   39780 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false
I1223 14:45:47.374859   39780 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false
I1223 14:45:47.374864   39780 envvar.go:169] "Feature gate updated state" feature="WatchListClient" enabled=false
I1223 14:45:47.374889   39780 reflector.go:313] Starting reflector *v1.Pod (0s) from pkg/mod/k8s.io/client-go@v0.32.0/tools/cache/reflector.go:251
I1223 14:45:47.374927   39780 reflector.go:349] Listing and watching *v1.Pod from pkg/mod/k8s.io/client-go@v0.32.0/tools/cache/reflector.go:251
I1223 14:45:47.375349   39780 reflector.go:376] Caches populated for *v1.Pod from pkg/mod/k8s.io/client-go@v0.32.0/tools/cache/reflector.go:251
Pod added: default/example-pod

Kubernetes version

Client Version: v1.32.0

Cloud provider

Local

OS version

MacOS
Darwin Kernel Version 23.6.0

Install tools

golang

go version
go version go1.23.0 darwin/arm64

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)
@dilipmighty245 dilipmighty245 added the kind/bug Categorizes issue or PR as related to a bug. label Dec 26, 2024
@k8s-ci-robot k8s-ci-robot added the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Dec 26, 2024
@k8s-ci-robot
Copy link
Contributor

There are no sig labels on this issue. Please add an appropriate label by using one of the following commands:

  • /sig <group-name>
  • /wg <group-name>
  • /committee <group-name>

Please see the group list for a listing of the SIGs, working groups, and committees available.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Dec 26, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@dilipmighty245 dilipmighty245 changed the title kubernetes 1.32: Informer with WatchClient fails to send send events kubernetes 1.32: Informer with WatchClient fails to send send events with Fakeclient Dec 26, 2024
@dilipmighty245 dilipmighty245 changed the title kubernetes 1.32: Informer with WatchClient fails to send send events with Fakeclient kubernetes 1.32: Informer with WatchClient fails to send events with Fakeclient Dec 26, 2024
@kubernetes kubernetes deleted a comment Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@k8s-ci-robot @dilipmighty245