Can I prevent `/var/lib/rook` being created for all kubernetes node? #15011

lesomnus · 2024-11-15T06:02:12Z

lesomnus
Nov 15, 2024

It seems that a single Kubernetes cluster can have multiple Ceph clusters using the CephCluster CR, and each Ceph cluster stores its data in the dataDirHostPath on the host, correct?

Can I specify which nodes are used by the CephCluster? I have a small number of nodes, and I plan to use all of them, but I want to ensure I know exactly which nodes are being used by the CephCluster.
For example, if a new node joins the Kubernetes cluster someday, I don’t want it to automatically become a CephCluster node before I verify it.

Answered by zhucan

Nov 15, 2024

like this，update the cr of cephcluster munally and one by one.

  storage:
    nodes:
    - devices:
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e079400b-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e079400a-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e0794009-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e0794008-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64bd09-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df642495-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64be61-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sa…

View full answer

lesomnus · 2024-11-15T06:17:49Z

lesomnus
Nov 15, 2024
Author

Should I use placement.all.nodeAffinity? I already have storage.nodes so I thought that it only provison on the given nodes.

1 reply

zhucan Nov 15, 2024
Collaborator

Before you ceate the cephcluster, you can set the useAllNodes=false, it will not use the all nodes automatically.

zhucan · 2024-11-15T08:38:43Z

zhucan
Nov 15, 2024
Collaborator

like this，update the cr of cephcluster munally and one by one.

  storage:
    nodes:
    - devices:
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e079400b-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e079400a-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e0794009-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e0794008-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64bd09-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df642495-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64be61-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df647249-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64c5f9-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df649111-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64f209-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64d789-lun-0
      name: node04
      resources: {}
    - devices:
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e079b00b-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e079b00a-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e079b008-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e079b009-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df643391-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df647259-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64a4f1-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64ac31-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df627649-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df649149-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df6483e9-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64e139-lun-0
      name: node01
      resources: {}
    - devices:
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x530809b92719400b-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x530809b92719400a-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x530809b927194009-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x530809b927194008-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64d71d-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64a319-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df649f35-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64f0e5-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64d791-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64bdb1-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64c52d-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64dcdd-lun-0
      name: node05
      resources: {}
    - devices:
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bc600a-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bc6009-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bc6008-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bc600b-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64a8ed-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64b999-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64dbb1-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df649681-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64572d-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64b0bd-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64a8d9-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64de81-lun-0
      name: node09
      resources: {}
    - devices:
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bc900b-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bc900a-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bc9008-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bc9009-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df6475a1-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df647491-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df63c6cd-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64730d-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64aaf5-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64e035-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df644c55-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64dd45-lun-0
      name: node08
      resources: {}
    - devices:
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e078e00b-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e078e00a-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e078e009-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e078e008-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64d839-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64a129-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64afc1-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64aab5-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df652ebd-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64f235-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df647275-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64bb49-lun-0
      name: node03
      resources: {}
    - devices:
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df647899-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64c449-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64afb9-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64a6d1-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64f155-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64f125-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df63d229-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df6494cd-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bcb00a-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bcb009-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bcb008-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x578aa82101bcb00b-lun-0
      name: node07
      resources: {}
    - devices:
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x530809b90d9fb00b-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x530809b90d9fb00a-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x530809b90d9fb009-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x530809b90d9fb008-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64a99d-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df642cc5-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64917d-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df6498f5-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df648209-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df6499ed-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64dbc5-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df63d339-lun-0
      name: node02
      resources: {}
    - devices:
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e07f400b-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e07f4009-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e07f4008-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x574eac82e07f400a-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64a8a9-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64b01d-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64a17d-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df648599-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64c61d-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df64f255-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df647229-lun-0
      - name: /dev/disk/by-path/pci-0000:61:00.0-sas-0x5000cca2df6492ed-lun-0
      name: node06
      resources: {}
    useAllDevices: false

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can I prevent `/var/lib/rook` being created for all kubernetes node? #15011

{{title}}

Replies: 2 comments 1 reply

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Can I prevent /var/lib/rook being created for all kubernetes node? #15011

lesomnus Nov 15, 2024

Replies: 2 comments · 1 reply

lesomnus Nov 15, 2024 Author

zhucan Nov 15, 2024 Collaborator

zhucan Nov 15, 2024 Collaborator

Can I prevent `/var/lib/rook` being created for all kubernetes node? #15011

lesomnus
Nov 15, 2024

Replies: 2 comments 1 reply

lesomnus
Nov 15, 2024
Author

zhucan Nov 15, 2024
Collaborator

zhucan
Nov 15, 2024
Collaborator