This is a tracking issue to cover the open issues and features necessary for running a significant 3rd party extension on top of a Kubernetes cluster. Depending on the complexity of an extension, some or all of these may be needed. At this time, the most complex extension is OpenShift and the items below these derive from those use cases, but I expect as other components that wish to extend or integrate with the cluster (logs, heapster, auditors, cluster integrations) are developed these will be useful.

The end result of this issue, as required features are implemented, should be a document describing how to build new cluster components that expose new capabilities on top of the core cluster services, what best practices should be followed, what design patterns to avoid, and to encode useful patterns into libraries and examples when someone wishes to "extend Kubernetes to do X". Some of the items discussed below may not fit into that document - they will be moved into other umbrella issues or their own as time goes on. Note, some of these overlap with regular application features, but are specifically listed here because in an ideal cluster components become aware of their environment and can use its APIs to leverage the flexibility the cluster manager provides.

Use cases (still WIP)

• Allow components running on the infrastructure...
  • To be highly available
  • To have persistent storage that can survive the death of individual nodes for data persistence
    • In small clusters, it is sufficient to allow some components access to the core etcd (suitably protected)
    • In larger clusters, it is preferable to allow components to run their own available data stores (overlaps with database use case)
  • To have new versions deployed predictably
  • To securely connect to the API server with high level authority (delegate cluster admin control to a component) and be configured with that authority automatically
  • To perform pass through or delegated authentication for an API consumer that has a token for accessing the core services
  • To set and configure security policy via API (or allow an admin to easily define additional policy)
• Allow core client tools to...
  • Discover REST resources provided by a plugin and operate on them in a generic fashion (simple CRUD operations) - discussed
  • Apply standard patterns of templatization, creation, and behavior to any resource that follows the core patterns discussed
  • Be easily reused by 3rd party client creators (kubectl, pkg/client, future UI) to build extensions of the core behavior - implemented
    • Design the kubectl commands so they can easily be tweaked to add new resources, or change the display and help text
    • Ensure reviewers are aware of the additional reqts on the kubectl code
    • Add unit tests to pkg/kubectl that verify generic kubectl commands do not depend on Kube specific globals
• Allow a cluster to be configured...
  • To use a different scheduler - implemented / example
  • To use a different authentication / identity provider - designed
  • To use a different policy engine for authorization - discussed
    • An endpoint that could be configured via cluster config to make authorization decisions, provided by OpenShift or other service
  • To use a different admission controller on pod (or service, or RC, etc) so that different decision processes may be possible - designed
  • So that an external plugin can contribute environment variables, volume configuration, or potentially security information into running pods without having to replace the scheduler or alter the code of the apiserver