Objective

There is no mechanism to specify storage managed by vSphere to be consumed by a Pod. The purpose of this document is to describe a viable short term solution to address needs of customers running ESX and wanting to consume storage in K8S.

The reason this is short term for now is 2 fold:

• External: We need more clarity on the long term direction of storage management in kubernetes. The state of storage management is evolving and might undergo fundamental changes.
• Internal: We need more clarity on how to coexist and best surface capabilities part of ESX/vCenter/Photon Controller.

Proposal

Implement a K8S volume plugin based on the plugin framework for the last stable release in the forked repo. This will unblock customers wanting to try K8S + vSphere + Storage. Going forward based on the outcome of kubernetes#18333 the plugin will be updated.

• Plugin will be part of the k8s code.
• The base image authored by VMware used to install k8s on top of vSphere will include an additional daemon that will service control plane requests from the plugins [1]
• The ESX user space will require a VIB to be installed on all nodes to allow for control operations to be executed. [1]
• The creation and deletion of volumes needs to be done via a CLI tool installed on the master.
• Volumes can be mounted and detached from any of the nodes based on the description for the Pod.

Alternative

We ship a binary plugin and depend on FlexVolume framework (Experimental for now and only in the master, no stable release). This allows us to not be dependent on getting our code into the official repo.

Cons

• Existing customers of kubernetes running stable releases (at the time of writing this issue) will not have any mechanism to consume vSphere storage other than it being statically attached to nodes.
• Future directional changes by kubernetes could make the plugin work throwaway.

[1] The code external to the K8S plugin and is common with the docker plugin.