[FEATURE] Import and export SPDK longhorn volumes to longhorn sparse file directory #4100
Description
Create a JSON-RPC command to import a directory of sparse files into a thinly provisioned SPDK logical volume store and a JSON-RPC command to export a logical volume in SPDK to a directory of sparse files.
You can make a directory of sparse files with longhorn by setting up with one replica:
https://github.com/longhorn/longhorn/wiki/How-to-run-longhorn-engine-without-containers
To make snapshots, you can mount the longhorn volume, write some data, then run ./longhorn-engine snapshot create <snapshot-name>
.
In longhorn-engine, sparse files are read with https://github.com/longhorn/sparse-tools and the fiemap ioctl.
https://www.kernel.org/doc/html/latest/filesystems/fiemap.html
In longhorn-engine, sparse files are created with ftruncate(2)
. Extents are created whenever data is written to an offset.
In SPDK, the logical volume is implemented as a blob. In lib/blob/blobstore.h
, there is struct spdk_blob_mut_data
. The clusters
member of struct spdk_blob_mut_data
is an array of addresses within the blobstore; a value of 0 for an element in the array implies that that cluster is zero and no cluster is allocated for it.
For SPDK, I have been creating an empty file on my computer. Then I register it as an aio block device. Then I create a logical volume store with that block device and create a logical volume in that logical volume store.
https://spdk.io/doc/bdev.html
https://spdk.io/doc/nvmf.html
https://spdk.io/doc/logical_volumes.html
Metadata
Assignees
Labels
Type
Projects
Status
Resolved
Status
Closed