Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kube-proxy requests 2x cpu shares of addons. #22022

Merged
merged 1 commit into from
Feb 26, 2016

Conversation

bprashanth
Copy link
Contributor

see #21820 (comment)
@kubernetes/goog-node 200m should be fine right?

@bprashanth
Copy link
Contributor Author

@kubernetes/goog-cluster

@k8s-github-robot
Copy link

Labelling this PR as size/XS

@k8s-github-robot k8s-github-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Feb 25, 2016
@@ -31,6 +31,9 @@ spec:
containers:
- name: kube-proxy
image: {{pillar['kube_docker_registry']}}/kube-proxy:{{pillar['kube-proxy_docker_tag']}}
resources:
requests:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should set limits ideally.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to curtail kube-proxy, I want it to get what it needs (plus I'm not 100% sure how much it will need).

But I guess it doesn't make sense giving unlimited cpu to the proxy if nothing can run on the node, and if it bumps into the limit it's because something else with higher priority came along, and it'll just be slow, unlike memory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

requests are OK for now I guess.
Depending on where we go with the QoS policies, this might make kube-proxy
have less priority. But given that it is a DaemonSet, I guess the policies
will be thwarted anyways.

On Thu, Feb 25, 2016 at 3:25 PM, Prashanth B notifications@github.com
wrote:

In cluster/saltbase/salt/kube-proxy/kube-proxy.manifest
#22022 (comment)
:

@@ -31,6 +31,9 @@ spec:
containers:

  • name: kube-proxy
    image: {{pillar['kube_docker_registry']}}/kube-proxy:{{pillar['kube-proxy_docker_tag']}}
    • resources:
    •  requests:
      

I don't want to curtail kube-proxy, I want it to get what it needs (plus
I'm not 100% sure how much it will need).

But I guess it doesn't make sense giving unlimited cpu to the proxy if
nothing can run on the node, and if it bumps into the limit it's because
something else with higher priority came along, and it'll just be slow,
unlike memory.


Reply to this email directly or view it on GitHub
https://github.com/kubernetes/kubernetes/pull/22022/files#r54182925.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah none of the master components set limits, for the same reason. If kube-proxy on a node is really slow it could impact a cluster spanning service, so I think we should leave it as request for now and revisit if required.

@bprashanth bprashanth force-pushed the proxy_cpu_request branch 2 times, most recently from 98acea3 to 1834834 Compare February 25, 2016 23:38
@mikedanese mikedanese assigned vishh and unassigned mikedanese Feb 25, 2016
@vishh vishh added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 25, 2016
@k8s-bot
Copy link

k8s-bot commented Feb 25, 2016

GCE e2e build/test passed for commit e34bbafd8962dcda8d7f12845e11607eccea4570.

@bprashanth bprashanth added the priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. label Feb 25, 2016
@k8s-bot
Copy link

k8s-bot commented Feb 26, 2016

GCE e2e build/test failed for commit 7d47d2d.

Please reference the list of currently known flakes when examining this failure. If you request a re-test, you must reference the issue describing the flake.

@bprashanth
Copy link
Contributor Author

@k8s-bot test this github issue: #22029

@k8s-bot
Copy link

k8s-bot commented Feb 26, 2016

GCE e2e build/test passed for commit 98acea32d0fd69242024073edb8f1ff269d3daff.

@k8s-bot
Copy link

k8s-bot commented Feb 26, 2016

GCE e2e build/test failed for commit 1834834269333cdd07bd2b0d80366e6ba116a473.

Please reference the list of currently known flakes when examining this failure. If you request a re-test, you must reference the issue describing the flake.

@k8s-bot
Copy link

k8s-bot commented Feb 26, 2016

GCE e2e build/test passed for commit 7d47d2d.

@bprashanth
Copy link
Contributor Author

@k8s-oncall this should help with services test flake, if the merge bot doesn't get to it overnight

@wojtek-t
Copy link
Member

Merging manually - this doesn't seem risky.

wojtek-t added a commit that referenced this pull request Feb 26, 2016
Kube-proxy requests 2x cpu shares of addons.
@wojtek-t wojtek-t merged commit 38957f8 into kubernetes:master Feb 26, 2016
@wojtek-t
Copy link
Member

Actually - it appeared that it broke kubemark, because we haven't had enough place to run hollow-nodes in the cluster. But this is already fixed. @gmarek

@thockin
Copy link
Member

thockin commented Feb 26, 2016

I am shocked that kube-proxy needs more CPU. It barely does anything. Do
we have a profile for it? Abhi - any clues?
On Feb 25, 2016 5:15 PM, "Prashanth B" notifications@github.com wrote:

see #21820 (comment)
#21820 (comment)
@kubernetes/goog-node https://github.com/orgs/kubernetes/teams/goog-node

200m should be fine right?

You can view, comment on, or merge this pull request online at:

#22022
Commit Summary

  • Kube-proxy requests 2x cpu shares of addons.

File Changes

Patch Links:


Reply to this email directly or view it on GitHub
#22022.

@ArtfulCoder
Copy link
Contributor

I am sure if @bprashanth actually profiled kube-proxy before making this PR ?
I think he is guessing that kube-proxy is cpu-starved and is therefore not processing events fast enough ?

@thockin
Copy link
Member

thockin commented Feb 29, 2016

I am sure this was not a wild guess, I am just surprised to believe it needs that much CPU

@thockin
Copy link
Member

thockin commented Feb 29, 2016

in other words - we need to figure out why. RAM I can buy. CPU, no way

@ArtfulCoder
Copy link
Contributor

Prashant can elaborate on how he reached the more-cpu-needed conclusion

Sent from my iPhone

On Feb 29, 2016, at 9:35 AM, Tim Hockin notifications@github.com wrote:

in other words - we need to figure out why. RAM I can buy. CPU, no way


Reply to this email directly or view it on GitHub.

@bprashanth
Copy link
Contributor Author

It doesn't need 200m, we need to figure out how much. It surely was starved, it was getting as many shares as pause, see: #21820 (comment). 200m is just a number > than the addons.

@thockin
Copy link
Member

thockin commented Feb 29, 2016

Sure, we can fine-tune it later

On Mon, Feb 29, 2016 at 10:00 AM, Prashanth B notifications@github.com
wrote:

It doesn't need 200m, we need to figure out how much. It surely was
starved, it was getting as many shares as pause, see: #21820 (comment)
#21820 (comment).
200m is just a number > than the addons.


Reply to this email directly or view it on GitHub
#22022 (comment)
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants