Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit the potentially unbounded list of images in the node status #23355

Closed
yujuhong opened this issue Mar 22, 2016 · 6 comments
Closed

Limit the potentially unbounded list of images in the node status #23355

yujuhong opened this issue Mar 22, 2016 · 6 comments
Labels
priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/node Categorizes an issue or PR as relevant to SIG Node.

Comments

@yujuhong
Copy link
Contributor

Forked from #23323 (comment)

In short, kubelet reports the list of images on the node as part of the node status. The kubelet image GC kicks in only when the disk space hits a threshold, so the list can grow very long with many small images.
@lavalamp mentioned that this could be a problem for the apiserver, and we should handle it somehow.

A naive solution (off the top of my head): kubelet sorts the images by size and report only the top N images.

/cc @davidopp

@yujuhong yujuhong added sig/node Categorizes an issue or PR as relevant to SIG Node. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. labels Mar 22, 2016
@davidopp
Copy link
Member

If the reason we put the images in the node status was just for the scheduler, we probably don't need it at all. Scheduler can just watch which pods get bound to which nodes, and keep the mapping from nodes to images in its memory. Kubelet would just need to somehow report when it GCs images so scheduler can update its information. When scheduler restarts it would lose all the information, but putting pods on nodes that already have the image installed is a best-effort feature anyway.

If there was some additional reason we were reporting images that I have forgotten, then never mind.

@yujuhong
Copy link
Contributor Author

I believe it was added for image-affinity scheduling, and only that.

Kubelet would just need to somehow report when it GCs images so scheduler can update its information.

This'd make things more complicated than necessary. If we're aiming for best-effort only, scheduler can construct the list of images by looking at only running pods (which may be too limited?).

@dchen1107
Copy link
Member

@davidopp putting pods on nodes with images installed is the only use case so far. But some users want that feature support is not best-effort since they pre-cached the images to the node for speed up the container startup.

@davidopp
Copy link
Member

But some users want that feature support is not best-effort since they pre-cached the images to the node for speed up the container startup.

The suggestion in the first entry in this issue (sort and drop the smaller packages from the report) also makes the feature best-effort.

@yujuhong
Copy link
Contributor Author

The suggestion in the first entry in this issue (sort and drop the smaller packages from the report) also makes the feature best-effort.

Yes, I guess one argument will be that users usually pre-pull if image size is significant, and scheduler will disregard this "optimization* by looking at pod mappings only.

BTW, it was just a random idea i threw out...

@ghost ghost added the priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. label Mar 31, 2016
@yujuhong
Copy link
Contributor Author

Fixed by #25328

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/node Categorizes an issue or PR as relevant to SIG Node.
Projects
None yet
Development

No branches or pull requests

3 participants