Huge amount of logs: kubelet_getters.go:300] "Path does not exist" path="/var/lib/kubelet/pods/86cba354-348c-4826-9837-df9c616a8862/volumes"

### What happened?

After we updated two of our clusters from 1.22 to v1.24.4 we noticed a huge amount of logs (tens of entries per second) in kubelet:

```
kubelet_getters.go:300] "Path does not exist" path="/var/lib/kubelet/pods/86cba354-348c-4826-9837-df9c616a8862/volumes"
```

Neither does the `volumes` subdir exist, nor the corresponding subdir for the pod (`"/var/lib/kubelet/pods/86cba354-348c-4826-9837-df9c616a8862` in this case).

Kubelet currently produces multiple gigabytes of logs each day since the upgrade.

### What did you expect to happen?

No log spam (at least not with verbosity=0).

### How can we reproduce it (as minimally and precisely as possible)?

Not really sure. We changed nothing in the setup. We just upgraded to v1.24.4. The cluster does not have any volume providers running. Just a 'vanilla' kubernetes (ubuntu 20.04, kubeadm, cillium, cri-o). 

### Anything else we need to know?

_No response_

### Kubernetes version

<details>

```console
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.4", GitCommit:"95ee5ab382d64cfe6c28967f36b53970b8374491", GitTreeState:"clean", BuildDate:"2022-08-17T18:54:23Z", GoVersion:"go1.18.5", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.4", GitCommit:"95ee5ab382d64cfe6c28967f36b53970b8374491", GitTreeState:"clean", BuildDate:"2022-08-17T18:47:37Z", GoVersion:"go1.18.5", Compiler:"gc", Platform:"linux/amd64"}
```

</details>


### Cloud provider

<details>
bare metal
</details>


### OS version

<details>

```console

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

$ uname -a
Linux k8s-master-p-1 5.15.0-46-generic #49~20.04.1-Ubuntu SMP Thu Aug 4 19:15:44 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
```

</details>


### Install tools

<details>
kubeadm
</details>


### Container runtime (CRI) and version (if applicable)

<details>
CRI-O

Version:          1.24.2
GitCommit:        bd548b04f78a30e1e9d7c17162714edd50edd6ca
GitTreeState:     clean
BuildDate:        2022-08-09T16:42:39Z
GoVersion:        go1.18.2
Compiler:         gc
Platform:         linux/amd64
Linkmode:         dynamic
BuildTags:        apparmor, exclude_graphdriver_devicemapper, containers_image_ostree_stub, seccomp
SeccompEnabled:   true
AppArmorEnabled:  true
</details>


### Related plugins (CNI, CSI, ...) and versions (if applicable)

<details>
cilium v1.12.0
</details>


Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Huge amount of logs: kubelet_getters.go:300] "Path does not exist" path="/var/lib/kubelet/pods/86cba354-348c-4826-9837-df9c616a8862/volumes" #112124

What happened?

What did you expect to happen?

How can we reproduce it (as minimally and precisely as possible)?

Anything else we need to know?

Kubernetes version

Cloud provider

OS version

Install tools

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development