Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Change CPU sample sanitization in HPA. #68068

Merged
merged 2 commits into from
Aug 31, 2018
Merged

Change CPU sample sanitization in HPA. #68068

merged 2 commits into from
Aug 31, 2018

Conversation

krzysztof-jastrzebski
Copy link
Contributor

@krzysztof-jastrzebski krzysztof-jastrzebski commented Aug 30, 2018 •
edited
Loading

What this PR does / why we need it:
Change CPU sample sanitization in HPA.
Ignore samples if:
- Pod is beeing initalized - 5 minutes from start defined by flag
- pod is unready
- pod is ready but full window of metric hasn't been colected since
transition
- Pod is initialized - 5 minutes from start defined by flag:
- Pod has never been ready after initial readiness period.

Release notes:

Improve CPU sample sanitization in HPA by taking metric's freshness into account.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Aug 30, 2018
@k8s-ci-robot k8s-ci-robot requested review from liggitt and luxas August 30, 2018 13:16
@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/autoscaling Categorizes an issue or PR as relevant to SIG Autoscaling. labels Aug 30, 2018
@krzysztof-jastrzebski
Copy link
Contributor Author

/assign mwielgus

@krzysztof-jastrzebski
Copy link
Contributor Author

/assign DirectXMan12

@krzysztof-jastrzebski
Copy link
Contributor Author

/assign @DirectXMan12

@MaciekPytel
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Aug 30, 2018
DirectXMan12
DirectXMan12 suggested changes Aug 30, 2018
View reviewed changes
Copy link
Contributor

@DirectXMan12 DirectXMan12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of quick comments inline. Need to do a full scan later today.

cmd/kube-controller-manager/app/options/options.go
@@ -134,7 +135,7 @@ func NewKubeControllerManagerOptions() (*KubeControllerManagerOptions, error) {
HorizontalPodAutoscalerSyncPeriod: componentConfig.HPAController.HorizontalPodAutoscalerSyncPeriod,
HorizontalPodAutoscalerUpscaleForbiddenWindow: componentConfig.HPAController.HorizontalPodAutoscalerUpscaleForbiddenWindow,
HorizontalPodAutoscalerDownscaleForbiddenWindow: componentConfig.HPAController.HorizontalPodAutoscalerDownscaleForbiddenWindow,
HorizontalPodAutoscalerCPUTaintPeriod: componentConfig.HPAController.HorizontalPodAutoscalerCPUTaintPeriod,
HorizontalPodAutoscalerCPUInitializationPeriod: componentConfig.HPAController.HorizontalPodAutoscalerCPUInitializationPeriod,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this say CPU in it? Doesn't it apply to other metrics as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+mwielgus
Currently this is only for CPU and I belive it will be only used for CPU.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it got changed a PR or two ago. It used to be universal across all metrics, so we probably need to decide if we really want to change it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mwielgus

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We didn't have such flag before and we want to change behavior only for CPU as it is kind of special.

pkg/controller/podautoscaler/metrics/legacy_metrics_client.go Outdated Show resolved Hide resolved
@krzysztof-jastrzebski krzysztof-jastrzebski force-pushed the hpas2 branch 3 times, most recently from faa800c to b56352e Compare August 30, 2018 16:35
@krzysztof-jastrzebski
Copy link
Contributor Author

/test pull-kubernetes-integration

DirectXMan12
DirectXMan12 suggested changes Aug 30, 2018
View reviewed changes
Copy link
Contributor

@DirectXMan12 DirectXMan12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nit inline, otherwise fine modulo the conversation about where this applies

pkg/controller/podautoscaler/metrics/legacy_metrics_client_test.go Outdated
@@ -364,3 +404,7 @@ func testCollapseTimeSamples(t *testing.T) {
assert.InEpsilon(t, float64(75), val, 0.1, "collapsed sample value should be as expected")
assert.True(t, timestamp.Equal(now), "timestamp should be the current time (the newest)")
}

func calcTimestamp(t int) time.Time {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should have a way better name. These tests are already hard to follow (maybe call it offsetTimestampBy or something).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 30, 2018
@krzysztof-jastrzebski krzysztof-jastrzebski changed the title Hpas2 Change CPU sample sanitization in HPA. Aug 30, 2018
@krzysztof-jastrzebski
Copy link
Contributor Author

Is there anything else that I should fix apart calcTimestamp name?

@DirectXMan12
Copy link
Contributor

I think I'm fine with this otherwise.

@fedebongio
Copy link
Contributor

/cc @wenjiaswe please figure out why it's been incorrectly assigned to api-machinery

@krzysztof-jastrzebski
Change CPU sample sanitization in HPA.
5357bf9 
Ignore samples if:
- Pod is beeing initalized - 5 minutes from start defined by flag
    - pod is unready
    - pod is ready but full window of metric hasn't been colected since
    transition
- Pod is initialized - 5 minutes from start defined by flag:
    - Pod has never been ready after initial readiness period.
@krzysztof-jastrzebski
Copy link
Contributor Author

Please give me LGTM so I can collect approvals tomorrow.

@krzysztof-jastrzebski
Copy link
Contributor Author

/test pull-kubernetes-integration

@mwielgus mwielgus added this to the v1.12 milestone Aug 31, 2018
@mwielgus
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 31, 2018
@krzysztof-jastrzebski
Copy link
Contributor Author

/assign wojtek-t

@wojtek-t
Copy link
Member

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: krzysztof-jastrzebski, mwielgus, wojtek-t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 31, 2018
@k8s-github-robot
Copy link

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-github-robot
Copy link

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

@k8s-github-robot k8s-github-robot merged commit 2548fb0 into kubernetes:master Aug 31, 2018
@wenjiaswe wenjiaswe mentioned this pull request Aug 31, 2018
Closed
@krzysztof-jastrzebski krzysztof-jastrzebski mentioned this pull request Jun 14, 2019
Closed
@max-rocket-internet max-rocket-internet mentioned this pull request Jul 16, 2019
Closed
@snowplayfire snowplayfire mentioned this pull request Mar 17, 2021
Closed
@aude aude mentioned this pull request May 5, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DirectXMan12 DirectXMan12 DirectXMan12 requested changes

@liggitt liggitt Awaiting requested review from liggitt

@luxas luxas Awaiting requested review from luxas

Assignees

@DirectXMan12 DirectXMan12

@wojtek-t wojtek-t

@mwielgus mwielgus

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/autoscaling Categorizes an issue or PR as relevant to SIG Autoscaling. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
v1.12
Development

Successfully merging this pull request may close these issues.
8 participants
@krzysztof-jastrzebski @MaciekPytel @DirectXMan12 @fedebongio @mwielgus @wojtek-t @k8s-ci-robot @k8s-github-robot