Skip to content
This repository has been archived by the owner on Jun 26, 2024. It is now read-only.

Compare various KPIs to the threshold values and fail if any is crossed #1271

Merged
merged 1 commit into from
Oct 18, 2022
Merged

Compare various KPIs to the threshold values and fail if any is crossed #1271

merged 1 commit into from
Oct 18, 2022

Conversation

pmacik
Copy link
Contributor

@pmacik pmacik commented Oct 13, 2022 •
edited
Loading

Signed-off-by: Pavel Macík pavel.macik@gmail.com

Changes

This PR:

  • Introduces test-performance-thresholds Makefile targed that compares various vaules from kpi.yaml generated by make test-performance-collect-kpi to the threshold values defined in environment variables TEST_PERFORMANCE_AVG_MEMORY, TEST_PERFORMANCE_MAX_MEMORY, TEST_PERFORMANCE_AVG_CPU, TEST_PERFORMANCE_MAX_CPU. If any of the threshold is crossed it fails.
  • The test-performance-thresholds is to be executed as part of performance* OpenShift CI jobs ([rhd-sbo] Add periodic (weekly) jobs to run performance tests openshift/release#33192) (after the test-performance-collect-kpi step) and fails the job if any of the thresholds is crossed.

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

  • Docs
    included if any changes are user facing
  • Tests
    included if any functionality added or changed. For bugfixes please include tests that can catch regressions
  • All acceptance test scenarios included in the PR which verifies a bugfix or a requested feature reported by a non-member are tagged with @external-feedback tag.
  • Follows the commit message standard

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Oct 13, 2022

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@codecov
Copy link

codecov bot commented Oct 13, 2022 •
edited
Loading

Codecov Report

Merging #1271 (42a7ae1) into master (6cdb804) will not change coverage.
The diff coverage is n/a.

❗ Current head 42a7ae1 differs from pull request most recent head e7dfbdd. Consider uploading reports for the commit e7dfbdd to get more accurate results

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1271   +/-   ##
=======================================
  Coverage   58.06%   58.06%           
=======================================
  Files          33       33           
  Lines        2797     2797           
=======================================
  Hits         1624     1624           
  Misses       1018     1018           
  Partials      155      155

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6cdb804...e7dfbdd. Read the comment docs.

@pmacik
Copy link
Contributor Author

pmacik commented Oct 13, 2022

/test 4.10-performance-reduced
/test 4.11-performance-reduced
/test performance

1 similar comment
@pmacik
Copy link
Contributor Author

pmacik commented Oct 17, 2022

/test 4.10-performance-reduced
/test 4.11-performance-reduced
/test performance

@pmacik
Compare various KPIs to the threshold values and fail if any is exceeded
e7dfbdd 
Signed-off-by: Pavel Macík <pavel.macik@gmail.com>
@pmacik pmacik force-pushed the performance-thresholds branch from 42a7ae1 to e7dfbdd Compare October 18, 2022 04:26
@pmacik pmacik marked this pull request as ready for review October 18, 2022 05:08
@openshift-ci openshift-ci bot requested review from filariow and sadlerap October 18, 2022 05:09
@pmacik
Copy link
Contributor Author

pmacik commented Oct 18, 2022

/test 4.10-performance-reduced
/test 4.11-performance-reduced
/test performance

@pmacik pmacik mentioned this pull request Oct 18, 2022
Merged
@pmacik pmacik added the release/v1.3.x Used to mark PRs to be cherry-picked in release-v1.3.x branch label Oct 18, 2022
@pmacik pmacik changed the title Compare various KPIs to the threshold values and fail if any is exceeded Compare various KPIs to the threshold values and fail if any is crossed Oct 18, 2022
@baijum
Copy link
Member

baijum commented Oct 18, 2022

/lgtm

@openshift-ci openshift-ci bot added the lgtm label Oct 18, 2022
dperaza4dustbit
dperaza4dustbit approved these changes Oct 18, 2022
View reviewed changes
Copy link
Contributor

@dperaza4dustbit dperaza4dustbit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Oct 18, 2022

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@pmacik
Copy link
Contributor Author

pmacik commented Oct 18, 2022

/retest

@openshift-merge-robot openshift-merge-robot merged commit c6777ea into redhat-developer:master Oct 18, 2022
service-binding-operator-bot pushed a commit to service-binding-operator-bot/service-binding-operator that referenced this pull request Oct 18, 2022
@pmacik @web-flow
Compare various KPIs to the threshold values and fail if any is excee…
f4388cb 
…ded (redhat-developer#1271)

Signed-off-by: Pavel Macík <pavel.macik@gmail.com>

Signed-off-by: Pavel Macík <pavel.macik@gmail.com>
openshift-merge-robot pushed a commit that referenced this pull request Oct 19, 2022
@service-binding-operator-bot @pmacik
Compare various KPIs to the threshold values and fail if any is excee…
186b0ef 
…ded (#1271) (#1276)

Signed-off-by: Pavel Macík <pavel.macik@gmail.com>

Signed-off-by: Pavel Macík <pavel.macik@gmail.com>

Signed-off-by: Pavel Macík <pavel.macik@gmail.com>
Co-authored-by: Pavel Macík <pavel.macik@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@dperaza4dustbit dperaza4dustbit dperaza4dustbit approved these changes

@filariow filariow Awaiting requested review from filariow

@sadlerap sadlerap Awaiting requested review from sadlerap

Assignees

@baijum baijum

@dperaza4dustbit dperaza4dustbit

Labels
approved lgtm release/v1.3.x Used to mark PRs to be cherry-picked in release-v1.3.x branch
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@pmacik @baijum @dperaza4dustbit @openshift-merge-robot