Skip to content

[FEATURE] Import and export SPDK longhorn volumes to longhorn sparse file directory #4100

Closed
@keithalucas

Description

Create a JSON-RPC command to import a directory of sparse files into a thinly provisioned SPDK logical volume store and a JSON-RPC command to export a logical volume in SPDK to a directory of sparse files.

You can make a directory of sparse files with longhorn by setting up with one replica:
https://github.com/longhorn/longhorn/wiki/How-to-run-longhorn-engine-without-containers

To make snapshots, you can mount the longhorn volume, write some data, then run ./longhorn-engine snapshot create <snapshot-name>.

In longhorn-engine, sparse files are read with https://github.com/longhorn/sparse-tools and the fiemap ioctl.
https://www.kernel.org/doc/html/latest/filesystems/fiemap.html

In longhorn-engine, sparse files are created with ftruncate(2). Extents are created whenever data is written to an offset.

In SPDK, the logical volume is implemented as a blob. In lib/blob/blobstore.h, there is struct spdk_blob_mut_data. The clusters member of struct spdk_blob_mut_data is an array of addresses within the blobstore; a value of 0 for an element in the array implies that that cluster is zero and no cluster is allocated for it.

For SPDK, I have been creating an empty file on my computer. Then I register it as an aio block device. Then I create a logical volume store with that block device and create a logical volume in that logical volume store.

https://spdk.io/doc/bdev.html
https://spdk.io/doc/nvmf.html
https://spdk.io/doc/logical_volumes.html

Metadata

Assignees

Labels

Type

No type

Projects

  • Status

    Closed

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions