Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update kubelet limits to be pods/core up to max-pods. #25762

Closed
timothysc opened this issue May 17, 2016 · 14 comments
Closed

Update kubelet limits to be pods/core up to max-pods. #25762

timothysc opened this issue May 17, 2016 · 14 comments
Labels
priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/node Categorizes an issue or PR as relevant to SIG Node.

Comments

@timothysc
Copy link
Member

Per discussions on @kubernetes/sig-scalability, we should update the advertised maximum pods to be a function of pods/core(default 10) up to a hard cap max-pods (currently 110).

xref: https://docs.google.com/a/bobsplanet.com/document/d/1hEpf25qifVWztaeZPFmjNiJvPo-5JX1z0LSvvVY5G2g/edit?usp=drive_web

/cc @kubernetes/sig-node @rrati

@timothysc timothysc changed the title Update kubelet limits to be pods/core up to some limit. Update kubelet limits to be pods/core up to max-pods. May 17, 2016
@ArtfulCoder ArtfulCoder added the sig/node Categorizes an issue or PR as relevant to SIG Node. label May 17, 2016
@wojtek-t
Copy link
Member

wojtek-t commented May 18, 2016

I agree, but this is a medium-term goal. For 1.3 we are going to leave limits as they were so far (i.e. pods/cluster, nodes/cluster, pods/node).

@wojtek-t wojtek-t modified the milestones: v1.3, next-candidate May 18, 2016
@timothysc
Copy link
Member Author

@wojtek-t we could probably create a patch that would default off but we-(Red Hat) could leverage. It would be no-op change for upstream, but our customers have much larger and denser nodes that they would like to leverage. Given this would be a 0-feature change for upstream, what is your opinion?

/cc @smarterclayton

@wojtek-t
Copy link
Member

Yeah - adding a "by-default" non-active limit sgtm.

@jeremyeder
Copy link

#23349

rrati pushed a commit to rrati/kubernetes that referenced this issue May 27, 2016
k8s-github-robot pushed a commit that referenced this issue May 28, 2016
Automatic merge from submit-queue

Added pods-per-core to kubelet. #25762

Added --pods-per-core to kubelet

#25762
@timothysc timothysc modified the milestones: v1.4, next-candidate Jun 17, 2016
@timothysc
Copy link
Member Author

Code is in, we'll just need to update defaults for 1.4

@goltermann
Copy link
Contributor

@timothysc @rrati @wojtek-t any more work needed for v1.4?

@timothysc
Copy link
Member Author

we've enabled flag over-rides in 1.3, but have not updated defaults for 1.4, so I'm moving out to 1.5.

I'd like to prioritize this as a 1.5 action item for @kubernetes/sig-node

@timothysc timothysc modified the milestones: v1.5, v1.4 Sep 6, 2016
@goltermann goltermann added priority/backlog Higher priority than priority/awaiting-more-evidence. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Sep 6, 2016
@goltermann goltermann removed the priority/backlog Higher priority than priority/awaiting-more-evidence. label Sep 6, 2016
@pskrzyns
Copy link

I was pointed that this is correct thread for my question so here I am.
Can we change existing density tests, so we don't have hard coded number of pods (should allow starting 95 pods per node) but instead have number of pods calculated basing on the number of cores (should allow starting 1 pod per core). Or should there be another set of tests for that ?

@timothysc
Copy link
Member Author

@pskrzyns no, b/c upstream has not defaulted to this change, but we have defaulted to this on our side.

/cc @yujuhong

@pskrzyns
Copy link

@timothysc is the changed version of those tests publicly available ?

@timothysc
Copy link
Member Author

We can probably share that data, we do a baseline overhead test @ 250 pods/node on our env.

But this will vary based upon the storage layer used by docker, and the docker version itself.

/cc @mffiedler

@dims
Copy link
Member

dims commented Nov 17, 2016

This needs to be triaged as a release-blocker or not for 1.5 @Random-Liu (or move to 1.6?) @timothysc

@yujuhong yujuhong modified the milestones: v1.6, v1.5 Nov 17, 2016
@yujuhong
Copy link
Contributor

Not sure what we can do for 1.5 at this moment. Moving it to 1.6

@dchen1107 dchen1107 removed this from the v1.6 milestone Mar 13, 2017
@timothysc
Copy link
Member Author

Knobs are all there sig-node can update defaults as needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/node Categorizes an issue or PR as relevant to SIG Node.
Projects
None yet
Development

No branches or pull requests

10 participants