Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LVM node can't create new PV #328

Open
styshoo opened this issue Sep 10, 2024 · 2 comments
Open

LVM node can't create new PV #328

styshoo opened this issue Sep 10, 2024 · 2 comments
Labels
question Further information is requested

Comments

@styshoo
Copy link

styshoo commented Sep 10, 2024

What steps did you take and what happened:
[A clear and concise description of what the bug is, and what commands you ran.]
The lvm csi runs correctly at first. But after a longtime running(maybe two months), I create a StatefulSet with lvm pvc, but the pv can't be created.

What did you expect to happen:
PV can be successfully created.

The output of the following commands will help us better understand what's going on:
(Pasting long output into a GitHub gist or other Pastebin is fine.)
describe pod:
image

  • kubectl get pods -n openebs
# kubectl get pod -n openebs 
NAME                                                 READY   STATUS    RESTARTS   AGE
kg-lvm-csi-lvm-localpv-controller-8487d5745c-4ln7x   5/5     Running   0          5d
kg-lvm-csi-lvm-localpv-node-54tmn                    2/2     Running   0          5d
  • kubectl get lvmvol -A -o yaml

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
In the csi node log, if a pv can be successfully created, the log should be as following:

I0910 07:42:51.015586       1 volume.go:85] Got add event for Vol pvc-24626e4e-6f29-4fbc-a59b-a4f9b51a6ae9
I0910 07:42:51.015605       1 volume.go:86] lvmvolume object to be enqueued by Add handler: 
I0910 07:42:51.015690       1 volume.go:55] Getting lvmvol object name:pvc-24626e4e-6f29-4fbc-a59b-a4f9b51a6ae9, ns from cache

However, in my environment, we only get the lines "Got add event for Vo" and "lvmvolume object to be enqueued by Add handler", without the third line.
After I reboot the csi node pod, everything is fine again. I can create pv successfully.

So, I doubt that there is something wrong with the workqueue, LVMVolume is put into the workqueue, but in the processNextWorkItem loop, nothing can be get from the queue.
But I'm not sure with that, is there a clue ?

Environment:

  • LVM Driver version: 1.5.0
  • Kubernetes version (use kubectl version):1.24.1
  • Kubernetes installer & version:
  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release):
@abhilashshetty04
Copy link
Contributor

Hi @styshoo , Can you provide more details on the workflow..

kubectl get sc <sc-name> 
kubectl get pvc <pvc-name>
kubectl get lvmvolume <pv-name> -n openebs

Do you mean after restarting lvm-node plugin lv gets created successfully?

@avishnu avishnu added the question Further information is requested label Sep 19, 2024
@dsharma-dc
Copy link
Contributor

dsharma-dc commented Dec 5, 2024

Please share above requested info, and complete logs of both lvm pods if available, in case this issue is reproducible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants