Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StorageClass ceph-rbd-topology from import-external-cluster.sh has JSON Error #15224

Open
driftlessdev opened this issue Dec 29, 2024 · 1 comment
Open

StorageClass ceph-rbd-topology from import-external-cluster.sh has JSON Error #15224

driftlessdev opened this issue Dec 29, 2024 · 1 comment
Assignees
@parth-gr
Labels
bug

Comments

@driftlessdev
Copy link

Is this a bug report or feature request?

  • Bug Report

Deviation from expected behavior:

The topology storage class created by the import script has invalid JSON on the topologyConstrainedPools parameter. This is resulting in PVCs failing to be provisioned.

Storage class created from the import script.

allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  creationTimestamp: "2024-12-29T03:42:43Z"
  name: ceph-rbd-topology
  resourceVersion: "2977143"
  uid: 285c2823-de4a-4b7f-8fca-568cc4b0264d
parameters:
  clusterID: rook-ceph-external
  csi.storage.k8s.io/controller-expand-secret-name: rook-csi-rbd-provisioner
  csi.storage.k8s.io/controller-expand-secret-namespace: rook-ceph-external
  csi.storage.k8s.io/fstype: ext4
  csi.storage.k8s.io/node-stage-secret-name: rook-csi-rbd-node
  csi.storage.k8s.io/node-stage-secret-namespace: rook-ceph-external
  csi.storage.k8s.io/provisioner-secret-name: rook-csi-rbd-provisioner
  csi.storage.k8s.io/provisioner-secret-namespace: rook-ceph-external
  imageFeatures: layering
  imageFormat: "2"
  topologyConstrainedPools: |
    [
     {"poolName":"pool-zone1",
      "domainSegments":[
        {"domainLabel":"zone","value":"zone1"}]},
     {"poolName":"pool-zone2",
      "domainSegments":[
        {"domainLabel":"zone","value":"zone2"}]},
     {"poolName":"pool-zone3",
      "domainSegments":[
        {"domainLabel":"zone","value":"zone3"}]},
     {"poolName":"pool-zone4",
      "domainSegments":[
        {"domainLabel":"zone","value":"zone4"}]},
    ]
provisioner: rook-ceph.rbd.csi.ceph.com
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

Expected behavior:

Storage class should be operable with valid JSON. Working around issue with:

  1. kubectl get sc ceph-rbd-topology -o=yaml > crtSc.yaml
  2. Edit topologyConstrainedPools to remove the last comma
  3. kubectl replace -f crtSc.yaml --force

How to reproduce it (minimal and precise):

  1. Generate params using create-external-cluster-resources.py with the flags --topology-pools=pool-zone1,pool-zone2,pool-zone3,pool-zone4 --topology-failure-domain-label=zone --topology-failure-domain-values=zone1,zone2,zone3,zone4
  2. Use the output to create the storage class using import-external-cluster.sh
  3. Label nodes as appropriate with topology.kubernetes.io/zone
  4. Generate a PVC using the storage class of ceph-rbd-topology

Logs to submit:

  • Operator's log
I1229 19:14:25.693836       1 event.go:389] "Event occurred" object="cnpg-system/test-db-1" fieldPath="" kind="PersistentVolumeClaim" apiVersion="v1" type="Warning" reason="ProvisioningFailed" message=<
        failed to provision volume with StorageClass "ceph-rbd-topology": rpc error: code = InvalidArgument desc = failed to parse JSON encoded topology constrained pools parameter ([
         {"poolName":"pool-zone1",
          "domainSegments":[
            {"domainLabel":"zone","value":"zone1"}]},
         {"poolName":"pool-zone2",
          "domainSegments":[
            {"domainLabel":"zone","value":"zone2"}]},
         {"poolName":"pool-zone3",
          "domainSegments":[
            {"domainLabel":"zone","value":"zone3"}]},
         {"poolName":"pool-zone4",
          "domainSegments":[
            {"domainLabel":"zone","value":"zone4"}]},
        ]
        ): invalid character ']' looking for beginning of value

Environment:

  • OS (e.g. from /etc/os-release): Ubuntu 24.04.1 LTS
  • Kernel (e.g. uname -a): 6.8.0-51-generic
  • Cloud provider or hardware configuration: Proxmox VM
  • Rook version (use rook version inside of a Rook Pod): 1.16.0
  • Storage backend version (e.g. for ceph do ceph -v): 19.2.0
  • Kubernetes version (use kubectl version): 1.31.4+k3s1
  • Kubernetes cluster type (e.g. Tectonic, GKE, OpenShift): k3s, ceph in Proxmox 8.3.1
  • Storage backend status (e.g. for Ceph use ceph health in the Rook Ceph toolbox): HEALTH_WARN 4 pool(s) have no replicas configured
@parth-gr
Copy link
Member

parth-gr commented Jan 6, 2025

@driftlessdev can you share the bash output of the python script,

Last I tried the sc output looks like this #13821 (comment) with some more commos.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@parth-gr parth-gr

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@driftlessdev @parth-gr