Benchmarking in CI (SCP-2353) #3441
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add a benchmarking script that will be executed by buildkite and a GitHub action to trigger the buidkite pipeline running this script through `/benchmark` comments on PRs
gilligan
force-pushed
the
benchmarking-ci
branch
from
3dec377 to
4349229
Compare
Add comments explaining `cabal update` and the use of `jq`
gilligan
commented
Jun 28, 2021
| @@ -0,0 +1,31 @@ | |||
| #!/usr/bin/env bash | |||
There was a problem hiding this comment.
@michaelpj now that I extracted it i agree that it's nicer to have this in a separate file/script 👍
michaelpj
reviewed
Jun 28, 2021
| BRANCH="$1" | ||
| PR="$2" | ||
|
|
||
| echo "[trigger-buildkite-pipeline]: Triggering build for $PR_BRANCH" |
There was a problem hiding this comment.
Suggested change
| echo "[trigger-buildkite-pipeline]: Triggering build for $PR_BRANCH" | |
| echo "[trigger-buildkite-pipeline]: Triggering build for $BRANCH" |
michaelpj
approved these changes
Jun 28, 2021
- Move scripts to ./scripts folder - Extract curl/buildkite invocation to separate script
gilligan
force-pushed
the
benchmarking-ci
branch
from
1166d31 to
3d975bd
Compare
koslambrou
pushed a commit
that referenced
this pull request
Jun 29, 2021
* Benchmarking in CI (SCP-2353) Add a benchmarking script that will be executed by buildkite and a GitHub action to trigger the buidkite pipeline running this script through `/benchmark` comments on PRs * ci-plutus-benchmark.sh: Add some comments Add comments explaining `cabal update` and the use of `jq` * Refactor benchmark scripts - Move scripts to ./scripts folder - Extract curl/buildkite invocation to separate script
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add a benchmarking script that will be executed by buildkite and a
GitHub action to trigger the buidkite pipeline running this script
through
/benchmarkcomments on PRsDetails
The benchmark.yaml workflow below looks for
/benchmarkcomments which works nicely but it should be noted this only works with this code inmaster.I did test and experiment using a fork and everything seemed to work nicely eventually but it's of course possible that something will still go wrong as we can't really test this fully before it's actually merged (annoying, yes).
The comments added by the bot aren't super pretty but i guess that is something we can change down the road once it's been established that this actually works reliably. Comments have to be posted as JSON content which is a bit annoying and I just used
jqto escape the output generated by the benchmark comparison script.Pre-submit checklist:
Pre-merge checklist: