Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize 'pause' container #15140

Closed
thockin opened this issue Oct 6, 2015 · 24 comments
Closed

Standardize 'pause' container #15140

thockin opened this issue Oct 6, 2015 · 24 comments
Labels
priority/backlog Higher priority than priority/awaiting-more-evidence. sig/node Categorizes an issue or PR as relevant to SIG Node.

Comments

@thockin
Copy link
Member

thockin commented Oct 6, 2015

#15138 demonstrates that our code has no less than THREE different tags of the pause image.

Some of those say 1.0, but our codebase only has 0.8.0 - can someone explain what is going on?

@vmarmol @vishh ?

Also, can someone remind me why we didn't do this as a C program?

@thockin thockin added the priority/backlog Higher priority than priority/awaiting-more-evidence. label Oct 6, 2015
@davidopp
Copy link
Member

davidopp commented Oct 6, 2015

Can we also document it? Some of the snippets in docs/ mention it but it's never explained anywhere.

@mattmoor
Copy link
Contributor

mattmoor commented Oct 6, 2015

FWIW, it isn't the only one. I see at least two versions of each of mounttest and nettest.

@mattmoor
Copy link
Contributor

mattmoor commented Oct 6, 2015

Actually, for pause there are 4 since at least some references don't specify a tag, which means latest... Others are: 0.8.0, 1.0, and go.

@a-robinson a-robinson added sig/node Categorizes an issue or PR as relevant to SIG Node. team/cluster labels Oct 6, 2015
@vmarmol
Copy link
Contributor

vmarmol commented Oct 6, 2015

The original is 0.8 which is also tagged as go. News to me that there is a 1.0 :P I think there are so many references since tests decided to use whatever tag they wanted.

As per the why Go, at the time we tried Go produced the smallest binary and didn't require us to ask people to have yet another dependency.

@mattmoor
Copy link
Contributor

mattmoor commented Oct 6, 2015

Actually, if you look in the GCR UI you'll see that 1.0 == latest == go. :)

@dchen1107
Copy link
Member

News to me that there is a 1.0 too!

In production, we are using and only using gcr.io/google_containers/pause:0.8.0, which is built and pushed through Makefile: ./build/pause/Makefile
Such image is also cached in ContainerVM image.

@vmarmol
Copy link
Contributor

vmarmol commented Oct 6, 2015

Sounds like someone made and pushed a 1.0 without updating the one we use in production :)

@thockin
Copy link
Member Author

thockin commented Oct 6, 2015

Does anyone have free time to either a) make the whole codebase use 0.8.0
or b) bump it to 2.0, push to beta.gcr, and update the codebase to use 2.0
(and container VM)?

I'm still grumpy about it not being C - I have seen the Go binary die.

On Tue, Oct 6, 2015 at 9:23 AM, Dawn Chen notifications@github.com wrote:

News to me that there is a 1.0 too!

In production, we are using and only using
gcr.io/google_containers/pause:0.8.0, which is built and pushed through
Makefile: ./build/pause/Makefile
Such image is also cached in ContainerVM image.


Reply to this email directly or view it on GitHub
#15140 (comment)
.

@vmarmol
Copy link
Contributor

vmarmol commented Oct 6, 2015

I had an assembly one! but you weren't a fan :D

On a more serious note, long-term it is probably better that it is in C. The Go runtime wakes up a few times a minute to do nothing. So we do see some CPU usage from pause (although minimal).

@bprashanth
Copy link
Contributor

Slightly OT, but we can just get rid of pause no?

The way CNI is currently the runtime is reponsible for setting up the new network namespace (and bind mounting a few things) but the plugin will add containers to it. I can surely see how having a library in kube that is responsible for creating namespaces with or without docker is useful. For netns, this would create and manage a point to point link with the host.

@dchen1107
Copy link
Member

@bprashanth I really want to get rid of pause. It complicates our container runtime implmentation, and resource management, introduces extra overhead, etc.
But when? cni has a 0.1 release a while back, but haven't heard from that for a while. Even today's rkt implementation is still rely on spawn extra systemd for namespace creation.

I measured last time, rkt's systemd for each POD uses similar amount of resource: cpu and memory.

@bprashanth
Copy link
Contributor

IMHO the win here is not reclaimed resources but the ability to manage our own networking and support the minimal runc model. We'd still need something to create that first net ns that everything else joins like you mentioned. I just like the idea of teaching kube about namespace joining so it can support bare bones container runtimes (this probably goes back to pods vs containers, though).

@thockin
Copy link
Member Author

thockin commented Oct 7, 2015

It will be some time before runc is ready for us to use, and we have more
than net namespaces to manage.

On Tue, Oct 6, 2015 at 11:47 AM, Prashanth B notifications@github.com
wrote:

IMHO the win here is not reclaimed resources but the ability to manage our
own networking and support the minimal runc model. We'd still need
something to create that first net ns that everything else joins like you
mentioned. I just like the idea of teaching kube about namespace joining so
it can support bare bones container runtimes (this probably goes back to
pods vs containers, though).


Reply to this email directly or view it on GitHub
#15140 (comment)
.

@thockin
Copy link
Member Author

thockin commented Oct 7, 2015 via email

@bprashanth
Copy link
Contributor

True. I just don't like being beholden (read: enslaved) to a third party container runtime :)

@luxas
Copy link
Member

luxas commented Mar 12, 2016

@thockin Could you kindly upload the C code you used to build your new pause image (if you have it now). I guess it wasn't too complicated.

The gcc compiler isn't a problem, we could just build pause in the kube-cross image.

And just for curiosity, how much are we using pause atm? I guess runc has evolved a bit since 7 Oct when this was discussed. runc spec is now 0.4, but cni is stuck at 0.1

@davidopp
Copy link
Member

Just to repeat my previous request -- can we please document the "pause" container? People ask about it pretty frequently and it's not documented anywhere (at least not in the docs in the docs repo).

@mattmoor
Copy link
Contributor

@davidopp In addition, it would be good to make the container name itself slightly more self-documenting. pause isn't a useful name at all (at least to me).

@thockin
Copy link
Member Author

thockin commented May 12, 2016

@uluyol there are still a bunch of places in the tree that reference :0.8 or or :1.0 or :2.0 - do you have 15 minutes to finish this cleanup? :D

k8s-github-robot pushed a commit that referenced this issue May 28, 2016
Automatic merge from submit-queue

Use pause image depending on the server's platform when testing

Removed all pause image constant strings, now the pause image is chosen by arch. Part of the effort of making e2e arch-agnostic.

The pause image name and version is also now only in two places, and it's documented to bump both
Also removed "amd64" constants in the code. Such constants should be replaced by `runtime.GOARCH` or by looking up the server platform

Fixes: #22876 and #15140
Makes it easier for: #25730
Related: #17981

This is for `v1.3`
@ixdy @thockin @vishh @kubernetes/sig-testing @andyzheng0831 @Pensu
@luxas
Copy link
Member

luxas commented May 29, 2016

This is fixed now with #23009 and #25972.

The pause image is now at version 3.0 and only in two places in the repo, which means it's easily updateable.

@luxas luxas closed this as completed May 29, 2016
@luxas
Copy link
Member

luxas commented May 29, 2016

@uluyol A side note, the memory footprint became roughly 50-60x smaller (from 1.8mb to 32kb), at least on my Raspberry Pi now.

I knew it would be smaller, it was @thockin's main point, but I didn't imagine it would decrease so greatly :)

@vishh
Copy link
Contributor

vishh commented Jun 1, 2016

cc @dchen1107 ^^

@davidopp
Copy link
Member

davidopp commented Jun 6, 2016

Another request to document pause container.

@luxas
Copy link
Member

luxas commented Jun 6, 2016

@davidopp Please see #26863
If you'd like more content in the pause section, feel free to comment on it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority/backlog Higher priority than priority/awaiting-more-evidence. sig/node Categorizes an issue or PR as relevant to SIG Node.
Projects
None yet
Development

No branches or pull requests

9 participants