Add design doc for external orchestration

(of Ceph entities like pools and filesystems) Partially fixes: #1868 Signed-off-by: John Spray <john.spray@redhat.com>
rook · Sep 5, 2018 · 08bd82d · 08bd82d
1 parent 7121418
commit 08bd82d
Showing 1 changed file with 120 additions and 0 deletions.
diff --git a/design/external-management.md b/design/external-management.md
@@ -0,0 +1,120 @@
+# Enable external Ceph management tools
+
+## TL;DR
+
+Some tools want to use Rook to run containers, but not manage the logical
+Ceph resources like Pools.  We should make Rook's pool management optional.
+
+## Background
+
+Currently in Rook 0.8, creating and destroying a Filesystem (or ObjectStore)
+in a Ceph cluster also creates and destroys the associated Ceph filesystem
+and pools.
+
+The current design works well when the Ceph configuration is within the
+scope of what Rook can configure itself, and the user does not modify
+the Ceph configuration of pools out of band.
+
+## Limitations
+
+The current model is problematic in some cases:
+
+- A user wants to use Ceph functionality outside of Rook's subset, and
+  therefore create their pools by hand before asking Rook to run
+  the daemon containers for a filesystem.
+- A user externally modifies the configuration of a pool (such as the
+  number of replicas), they probably want that new configuration, rather than
+  for Rook to change it back to match the Rook Filesystem settings.
+- A risk-averse user wants to ensure that mistaken edits to their Rook config cannot
+  permanently erase Ceph pools (i.e. they want to only delete pools through
+  an imperative interface with confirmation prompts etc).
+
+## Proposal
+
+In FilesystemSpec and ObjectStoreSpec, group the metadata and
+data pool attributes into a new struct called FilesystemPoolsSpec.
+When this field is nil, Rook will not do any management of logical
+Ceph resources (Ceph pools and Ceph filesystems) for the filesystem.
+
+The FilesystemPoolsSpec may be initially non-nil, and later modified
+to be nil.  In this case, while Rook may have created the logical
+resources for the filesystem, it will not remove them when the Rook filesystem
+is removed.
+
+If FilesystemPoolsSpec is non-nil, it must have both data and metadata
+pools populated -- it is not permitted to have Rook manage metadata but
+not data pools, or vice versa.
+
+### Before
+
+```yaml
+apiVersion: ceph.rook.io/v1beta1
+kind: Filesystem
+metadata:
+  name: myfs
+  namespace: rook-ceph
+spec:
+  metadataPool:
+    replicated:
+      size: 3
+  dataPools:
+    - erasureCoded:
+       dataChunks: 2
+       codingChunks: 1
+  metadataServer:
+    activeCount: 1
+    activeStandby: true
+```
+
+### After (with Pools specified)
+
+```yaml
+apiVersion: ceph.rook.io/v1beta1
+kind: Filesystem
+metadata:
+  name: myfs
+  namespace: rook-ceph
+spec:
+  pools:
+      metadata:
+        replicated:
+          size: 3
+      data:
+        - erasureCoded:
+           dataChunks: 2
+           codingChunks: 1
+  metadataServer:
+    activeCount: 1
+    activeStandby: true
+```
+
+### After (no pools specified)
+
+In this example, the `pools` attribute is omitted.  Rook will not create
+any pools or a Ceph filesystem.  A filesystem named ``myfs`` should already
+exist in Ceph, otherwise Rook will not start any MDS pods.
+
+```yaml
+apiVersion: ceph.rook.io/v1beta1
+kind: Filesystem
+metadata:
+  name: myfs
+  namespace: rook-ceph
+spec:
+  metadataServer:
+    activeCount: 1
+    activeStandby: true
+```
+
+
+## Impact
+
+- Rook Operator: the updates to controller code consist of changes to
+adapt to the new CRD structure, and logic to skip logical resource
+management when FilesystemPoolsSpec or ObjectStorePoolsSpec are
+nil.
+
+- Migration: existing FilesystemSpec structures would have their MetadataPool
+and DataPools fields moved into a new FilesystemPoolsSpec instance.  The
+equivalent migration would be done on ObjectStoreSpec.
+