Skip to content
This repository has been archived by the owner on Aug 14, 2021. It is now read-only.

Add openebs snapshot feature #37

Merged

Conversation

prateekpandey14
Copy link
Collaborator

@prateekpandey14 prateekpandey14 commented Mar 28, 2018

This PR will add OpenEBS as snapshot provider.

Below are the update to track down the features:

Start Snapshot Controller:

(assuming you have a running Kubernetes local cluster):

  • Note: Get Maya-apisrever address and export the address as env variable
    export MAPI_ADDR=http://172.18.0.5:5656

  • Start snapshot controller:

_output/bin/snapshot-controller  -kubeconfig=${HOME}/.kube/config
  • Start provisioner (assuming running Kubernetes local cluster):
 _output/bin/snapshot-provisioner  -kubeconfig=${HOME}/.kube/config

Prepare a PV to take snapshot. You can either use OpenEBS dynamic provisioned PVs or static PVs.

kubectl get pvc
NAME              STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
openebs-vol1-claim   Bound     pvc-f1c1fdf2-00d2-11e8-acdc-54e1ad0c1ccc   5G         RWO            openebs-percona   29s

Create a snapshot

Now we have PVC bound to a PV that contains some data. We want to take snapshot of this data so we can restore the data later.

  • Create a Snapshot for OpenEBS volume Resource
$ kubectl create -f examples/openebs/snapshot.yaml

Check VolumeSnapshot and VolumeSnapshotData are created

$ kubectl get volumesnapshot,volumesnapshotdata -o yaml
 apiVersion: v1
 items:
   - apiVersion: volumesnapshot.external-storage.k8s.io/v1
  kind: VolumeSnapshot
  metadata:
    clusterName: ""
    creationTimestamp: 2018-01-24T06:58:38Z
    generation: 0
    labels:
      SnapshotMetadata-PVName: pvc-f1c1fdf2-00d2-11e8-acdc-54e1ad0c1ccc
      SnapshotMetadata-Timestamp: "1516777187974315350"
    name: snapshot-demo
    namespace: default
    resourceVersion: "1345"
    selfLink: /apis/volumesnapshot.external-storage.k8s.io/v1/namespaces/default/volumesnapshots/fastfurious
    uid: 014ec851-00d4-11e8-acdc-54e1ad0c1ccc
  spec:
    persistentVolumeClaimName: demo-vol1-claim
    snapshotDataName: k8s-volume-snapshot-2a788036-00d4-11e8-9aa2-54e1ad0c1ccc
  status:
    conditions:
    - lastTransitionTime: 2018-01-24T06:59:48Z
      message: Snapshot created successfully
      reason: ""
      status: "True"
      type: Ready
    creationTimestamp: null
- apiVersion: volumesnapshot.external-storage.k8s.io/v1
  kind: VolumeSnapshotData
  metadata:
    clusterName: ""
    creationTimestamp: 2018-01-24T06:59:48Z
    name: k8s-volume-snapshot-2a788036-00d4-11e8-9aa2-54e1ad0c1ccc
    namespace: ""
    resourceVersion: "1344"
    selfLink: /apis/volumesnapshot.external-storage.k8s.io/v1/k8s-volume-snapshot-2a788036-00d4-11e8-9aa2-54e1ad0c1ccc
    uid: 2a789f5a-00d4-11e8-acdc-54e1ad0c1ccc
  spec:
    openebsVolume:
      snapshotId: pvc-f1c1fdf2-00d2-11e8-acdc-54e1ad0c1ccc_1516777187978793304
    persistentVolumeRef:
      kind: PersistentVolume
      name: pvc-f1c1fdf2-00d2-11e8-acdc-54e1ad0c1ccc
    volumeSnapshotRef:
      kind: VolumeSnapshot
      name: default/snapshot-demo
  status:
    conditions:
    - lastTransitionTime: null
      message: Snapshot created successfully
      reason: ""
      status: "True"
      type: Ready
    creationTimestamp: null
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

Restore Snapshot snapshot-demo as a new PV :

$ kubectl create -f examples/openebs/snapshot_claim.yaml

Delete the Snapshot:

$ kubectl delete volumesnapshot/snapshot-demo
volumesnapshot "snapshot-demo" deleted

Verify the volumesnapshot object:

$ kubectl get volumesnapshot -o yaml
apiVersion: v1
items: []
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

Signed-off-by: prateekpandey14 <prateekpandey14@gmail.com>
@prateekpandey14 prateekpandey14 force-pushed the Add-openebs-snapshot-feature branch from f028ac8 to 37b9fc1 Compare March 28, 2018 13:24
Signed-off-by: prateekpandey14 <prateekpandey14@gmail.com>
snapshot.alpha.kubernetes.io/snapshot: fastfurious
spec:
storageClassName: snapshot-promoter
accessModes: [ "ReadWriteOnce" ]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be read-only?

//Marshal serializes the value provided into a YAML document
yamlValue, _ := yaml.Marshal(snap)

glog.V(2).Infof("[DEBUG] snapshot Spec Created:\n%v\n", string(yamlValue))
Copy link

@kmova kmova Apr 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it a good practice to use [DEBUG]? It will be good to follow the conventions mentioned here for error levels. https://github.com/kubernetes/community/blob/master/contributors/devel/logging.md

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will clean this up , and set proper log levels

@kmova
Copy link

kmova commented Apr 2, 2018

@prateekpandey14 - can you add the e2e tests to the ci folder to do the following:

  • run the snapshot controllers as pods in the minikube
  • load the storage-class (given in the example) that uses the above snapshot provisioners
  • create a volume
  • create snapshot
  • promote snapshot
  • verify the data in the promoted snapshot is same as the one created previously.

1. Get the maya-apiserver endpoint(MAPI_ADDR) from maya-service
   and set as env variable, which needed for making API request.

2. Add the deployment yaml to deploy snapshot-controller and provisioner.

Signed-off-by: prateekpandey14 <prateekpandey14@gmail.com>
@prateekpandey14 prateekpandey14 force-pushed the Add-openebs-snapshot-feature branch from fe88c48 to 4401144 Compare April 5, 2018 10:44
Signed-off-by: prateekpandey14 <prateekpandey14@gmail.com>
@prateekpandey14 prateekpandey14 force-pushed the Add-openebs-snapshot-feature branch from a487c05 to a866cf7 Compare April 5, 2018 12:15
@prateekpandey14 prateekpandey14 force-pushed the Add-openebs-snapshot-feature branch 18 times, most recently from 7624de0 to 377d3ca Compare April 11, 2018 11:40
Copy link

@kmova kmova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please resolve the merge conflicts

@kmova
Copy link

kmova commented May 21, 2018

@prateekpandey14 I see a lot of redundant yaml files between CI and the examples. Can they be refactored to remove the redundancy? Also, if you are using the helm chart to install the OpenEBS maya-apiserver etc, can you remove the openebs-operator listed here?

@prateekpandey14 prateekpandey14 force-pushed the Add-openebs-snapshot-feature branch 4 times, most recently from cab106a to c7f8689 Compare May 22, 2018 18:09
@@ -0,0 +1,189 @@
# Define the Service Account
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to keep this file in sync with the actual openebs-operator.yaml? Is this fine really required? Why not install via helm?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With helm there is issue to get the OPENEBS_MAPI_SERVICE(its auto generated prefix name) env to get MAPI_ADDR , which is set during the openebs installation using helm( inside provisioner POD), and can not be imported in snapshot-controller and snapshot-provisioner pod

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

openebs-operator yaml file uses ci images of maya-apiserver, provisioner and jiva.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the following:

helm install stable/openebs --name ci --namespace openebs --set apiserver.imageTag="ci",jiva.replicas="1",jiva.imageTag="ci", provisioner.imageTag="ci"

The MAPI_ADDR can then be obtained using:

MAPI_SVC_ADDR=`kubectl get service -n openebs ci-openebs-apiservice -o json | grep clusterIP | awk -F\" '{print $4}'`

It will be good to avoid duplicating the operator.yaml files. Infact we should reduce the usage so much and then remove it. The operator-yaml should be auto-generated from helm chart.

Copy link
Collaborator Author

@prateekpandey14 prateekpandey14 May 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How we can export this inside a pod containers dynamically ?

@prateekpandey14 prateekpandey14 force-pushed the Add-openebs-snapshot-feature branch from 10c4319 to 3ed8f76 Compare May 22, 2018 19:13
Signed-off-by: prateekpandey14 <prateekpandey14@gmail.com>
@prateekpandey14 prateekpandey14 force-pushed the Add-openebs-snapshot-feature branch from 3ed8f76 to 154a776 Compare May 22, 2018 19:22
//}

//snapshotName := &tags["kubernetes.io/created-for/snapshot/name"]
snapshotName := createSnapshotName(pv.Name)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we pass the snapname here as well to generate a unique snapname?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the snapshot name as a part of unique snapshot name

volumeSpec.Metadata.Labels.Namespace = pvc.Namespace
volumeSpec.Metadata.Name = pvName

err := openebsVol.ListVolume(pvRefName, pvRefNamespace, &oldvolume)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Snapshot Provisioner should pass the source volume name to the maya-apiserver. The API server should have the intelligence to retrieve the required clone IP. This way the snapshot provisioner doesn't have to depend on the internal details of the volume types supported by the maya-apiserver.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The storage class name passed should be that of the source volume, so that the maya-apsierver can bring up the cloned volume with the same parameters as source volume. In case the cloned volume needs to override some values, they should be mentioned in the PVC.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a second thought, it might be better to have the maya-apiserver fetch the storage class associated with the source volume, while it is fetching the source controller IP.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added , now snapshot-provisioner will fetch the SC from the source volume.

Signed-off-by: prateekpandey14 <prateekpandey14@gmail.com>
1. Now snapshot provisioner will use StorageClass of source
   volume for promoting snapshot as a new PV.
2. Add unit test for snapshot API requests.
Signed-off-by: prateekpandey14 <prateekpandey14@gmail.com>
Copy link

@kmova kmova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an initial PR to support the workflow for snapshot creation. A new PR will be raised that will harden the workflow for error conditions and also the API between snapshotcontroller and maya-apiserver will change to pass the PV name instead of storageclass name.

@kmova kmova merged commit 68abdaf into openebs-archive:release Jun 1, 2018
prateekpandey14 added a commit to prateekpandey14/external-storage that referenced this pull request Aug 20, 2018
* Add OpenEBS Plugin for snapshot-controller and snapshot-provisioner
* Add e2e test for openebs-snapshot and creating a PV from snapshot
* Update README with snapshot clone/restore steps
* Add unit test for snapshot API requests.

Signed-off-by: prateekpandey14 <prateekpandey14@gmail.com>
@prateekpandey14 prateekpandey14 deleted the Add-openebs-snapshot-feature branch August 20, 2018 09:14
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Snaphsot feature for OpenEBS in K8s snapshot API
2 participants