forked from kubernetes/kubernetes
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request kubernetes#33704 from yifan-gu/rktlet_proposal
Automatic merge from submit-queue Proposal: Add a rktlet proposal in upstream. Mostly copied from docs/proposals/kubelet-rkt-runtime.md, plus roadmaps. /cc @kubernetes/sig-rktnetes @kubernetes/sig-node @philips @calebamiles @lucab @s-urbaniak @squeed
- Loading branch information
Showing
1 changed file
with
132 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| <!-- BEGIN MUNGE: UNVERSIONED_WARNING --> | ||
|
|
||
| <!-- BEGIN STRIP_FOR_RELEASE --> | ||
|
|
||
| <img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING" | ||
| width="25" height="25"> | ||
| <img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING" | ||
| width="25" height="25"> | ||
| <img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING" | ||
| width="25" height="25"> | ||
| <img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING" | ||
| width="25" height="25"> | ||
| <img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING" | ||
| width="25" height="25"> | ||
|
|
||
| <h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2> | ||
|
|
||
| If you are using a released version of Kubernetes, you should | ||
| refer to the docs that go with that version. | ||
|
|
||
| Documentation for other releases can be found at | ||
| [releases.k8s.io](http://releases.k8s.io). | ||
| </strong> | ||
| -- | ||
|
|
||
| <!-- END STRIP_FOR_RELEASE --> | ||
|
|
||
| <!-- END MUNGE: UNVERSIONED_WARNING --> | ||
|
|
||
| Next generation rkt runtime integration | ||
| ======================================= | ||
|
|
||
| Authors: Euan Kemp (@euank), Yifan Gu (@yifan-gu) | ||
|
|
||
| ## Abstract | ||
|
|
||
| This proposal describes the design and road path for integrating rkt with kubelet with the new container runtime interface. | ||
|
|
||
| ## Background | ||
|
|
||
| Currently, the Kubernetes project supports rkt as a container runtime via an implementation under [pkg/kubelet/rkt package](https://github.com/kubernetes/kubernetes/tree/v1.5.0-alpha.0/pkg/kubelet/rkt). | ||
|
|
||
| This implementation, for historical reasons, has required implementing a large amount of logic shared by the original Docker implementation. | ||
|
|
||
| In order to make additional container runtime integrations easier, more clearly defined, and more consistent, a new [Container Runtime Interface](https://github.com/kubernetes/kubernetes/blob/v1.5.0-alpha.0/pkg/kubelet/api/v1alpha1/runtime/api.proto) (CRI) is being designed. | ||
| The existing runtimes, in order to both prove the correctness of the interface and reduce maintenance burden, are incentivized to move to this interface. | ||
|
|
||
| This document proposes how the rkt runtime integration will transition to using the CRI. | ||
|
|
||
| ## Goals | ||
|
|
||
| ### Full-featured | ||
|
|
||
| The CRI integration must work as well as the existing integration in terms of features. | ||
|
|
||
| Until that's the case, the existing integration will continue to be maintained. | ||
|
|
||
| ### Easy to Deploy | ||
|
|
||
| The new integration should not be any more difficult to deploy and configure than the existing integration. | ||
|
|
||
| ### Easy to Develop | ||
|
|
||
| This iteration should be as easy to work and iterate on as the original one. | ||
|
|
||
| It will be available in an initial usable form quickly in order to validate the CRI. | ||
|
|
||
| ## Design | ||
|
|
||
| In order to fulfill the above goals, the rkt CRI integration will make the following choices: | ||
|
|
||
| ### Remain in-process with Kubelet | ||
|
|
||
| The current rkt container runtime integration is able to be deployed simply by deploying the kubelet binary. | ||
|
|
||
| This is, in no small part, to make it *Easy to Deploy*. | ||
|
|
||
| Remaining in-process also helps this integration not regress on performance, one axis of being *Full-Featured*. | ||
|
|
||
| ### Communicate through gRPC | ||
|
|
||
| Although the kubelet and rktlet will be compiled together, the runtime and kubelet will still communicate through gRPC interface for better API abstraction. | ||
|
|
||
| For the near short term, they will still talk through a unix socket before we implement a custom gRPC connection that skips the network stack. | ||
|
|
||
| ### Developed as a Separate Repository | ||
|
|
||
| Brian Grant's discussion on splitting the Kubernetes project into [separate repos](https://github.com/kubernetes/kubernetes/issues/24343) is a compelling argument for why it makes sense to split this work into a separate repo. | ||
|
|
||
| In order to be *Easy to Develop*, this iteration will be maintained as a separate repository, and re-vendored back in. | ||
|
|
||
| This choice will also allow better long-term growth in terms of better issue-management, testing pipelines, and so on. | ||
|
|
||
| Unfortunately, in the short term, it's possible that some aspects of this will also cause pain and it's very difficult to weight each side correctly. | ||
|
|
||
| ### Exec the rkt binary (initially) | ||
|
|
||
| While significant work on the rkt [api-service](https://coreos.com/rkt/docs/latest/subcommands/api-service.html) has been made, | ||
| it has also been a source of problems and additional complexity, | ||
| and was never transitioned to entirely. | ||
|
|
||
| In addition, the rkt cli has historically been the primary interface to the rkt runtime. | ||
|
|
||
| The initial integration will execute the rkt binary directly for app creation/start/stop/removal, as well as image pulling/removal. | ||
|
|
||
| The creation of pod sanbox is also done via rkt command line, but it will run under `systemd-run` so it's monitored by the init process. | ||
|
|
||
| In the future, some of these decisions are expected to be changed such that rkt is vendored as a library dependency for all operations, and other init systems will be supported as well. | ||
|
|
||
|
|
||
| ## Roadmap and Milestones | ||
|
|
||
| 1. rktlet integrate with kubelet to support basic pod/container lifecycle (pod creation, container creation/start/stop, pod stop/removal) [[Done]](https://github.com/kubernetes-incubator/rktlet/issues/9) | ||
| 2. rktlet integrate with kubelet to support more advanced features: | ||
| - Support kubelet networking, host network | ||
| - Support mount / volumes [[#33526]](https://github.com/kubernetes/kubernetes/issues/33526) | ||
| - Support exposing ports | ||
| - Support privileged containers | ||
| - Support selinux options [[#33139]](https://github.com/kubernetes/kubernetes/issues/33139) | ||
| - Support attach [[#29579]](https://github.com/kubernetes/kubernetes/issues/29579) | ||
| - Support exec [[#29579]](https://github.com/kubernetes/kubernetes/issues/29579) | ||
| - Support logging [[#33111]](https://github.com/kubernetes/kubernetes/pull/33111) | ||
|
|
||
| 3. rktlet integrate with kubelet, pass 100% e2e and node e2e tests, with nspawn stage1. | ||
| 4. rktlet integrate with kubelet, pass 100% e2e and node e2e tests, with kvm stage1. | ||
| 5. Revendor rktlet into `pkg/kubelet/rktshim`, and start deprecating the `pkg/kubelet/rkt` package. | ||
| 6. Eventually replace the current `pkg/kubelet/rkt` package. | ||
|
|
||
|
|
||
| <!-- BEGIN MUNGE: GENERATED_ANALYTICS --> | ||
| []() | ||
| <!-- END MUNGE: GENERATED_ANALYTICS --> |