Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix block benchmarks #1767

Merged
merged 5 commits into from
Nov 2, 2023
Merged

fix block benchmarks #1767

merged 5 commits into from
Nov 2, 2023

Conversation

chessai
Copy link
Contributor

@chessai chessai commented Oct 30, 2023 •
edited
Loading

PR Goals

  • Get rid of unused code paths/code bloat (the 'CoinContractRequest' type was pretty much unused
  • Actually submit the amount of TXs the benchmarks claim they do (mempool setting was just broken before)

@chessai
Copy link
Contributor Author

chessai commented Oct 30, 2023 

❯ cabal run bench -- -m pattern "PactService"
benchmarking PactService/forkingBench
time                 177.8 ms   (176.2 ms .. 180.3 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 176.6 ms   (176.0 ms .. 177.4 ms)
std dev              1.027 ms   (605.2 μs .. 1.515 ms)
variance introduced by outliers: 12% (moderately inflated)

benchmarking PactService/doubleForkingBench
time                 353.9 ms   (349.5 ms .. 356.0 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 355.2 ms   (354.5 ms .. 355.6 ms)
std dev              714.7 μs   (19.47 μs .. 885.6 μs)
variance introduced by outliers: 19% (moderately inflated)

benchmarking PactService/block-new-validated[1]
time                 5.683 ms   (5.636 ms .. 5.743 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 5.694 ms   (5.676 ms .. 5.720 ms)
std dev              67.52 μs   (54.60 μs .. 91.50 μs)

benchmarking PactService/block-new-validated[10]
time                 34.26 ms   (34.03 ms .. 34.50 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 34.46 ms   (34.27 ms .. 34.66 ms)
std dev              417.3 μs   (279.6 μs .. 586.7 μs)

benchmarking PactService/block-new-validated[50]
time                 161.1 ms   (160.3 ms .. 162.0 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 162.0 ms   (161.5 ms .. 162.7 ms)
std dev              848.2 μs   (509.1 μs .. 1.207 ms)
variance introduced by outliers: 12% (moderately inflated)

benchmarking PactService/block-new-validated[100]
time                 318.8 ms   (312.9 ms .. 323.5 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 320.1 ms   (318.6 ms .. 321.4 ms)
std dev              1.878 ms   (1.484 ms .. 2.220 ms)
variance introduced by outliers: 16% (moderately inflated)

benchmarking PactService/block-new[0]
time                 610.5 μs   (595.0 μs .. 630.2 μs)
                     0.996 R²   (0.994 R² .. 0.999 R²)
mean                 628.7 μs   (620.7 μs .. 635.2 μs)
std dev              25.15 μs   (20.48 μs .. 30.61 μs)
variance introduced by outliers: 32% (moderately inflated)

benchmarking PactService/block-new[1]
time                 2.501 ms   (2.464 ms .. 2.526 ms)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 2.498 ms   (2.481 ms .. 2.516 ms)
std dev              52.09 μs   (44.04 μs .. 71.79 μs)

benchmarking PactService/block-new[10]
time                 17.31 ms   (17.20 ms .. 17.45 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 17.33 ms   (17.28 ms .. 17.39 ms)
std dev              146.0 μs   (121.2 μs .. 187.9 μs)

benchmarking PactService/block-new[50]
time                 82.25 ms   (81.60 ms .. 82.99 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 82.26 ms   (81.95 ms .. 82.54 ms)
std dev              495.4 μs   (396.7 μs .. 641.4 μs)

benchmarking PactService/block-new[100]
time                 164.7 ms   (163.7 ms .. 165.9 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 165.0 ms   (164.5 ms .. 165.4 ms)
std dev              654.2 μs   (469.5 μs .. 933.9 μs)
variance introduced by outliers: 12% (moderately inflated)

ghost
ghost approved these changes Oct 31, 2023
View reviewed changes
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Would be good to have a big picture idea of changes in the PR's description.

@chessai
Copy link
Contributor Author

chessai commented Nov 1, 2023

LGTM 👍

Would be good to have a big picture idea of changes in the PR's description.

Good point, I edited it!

@chessai chessai added this pull request to the merge queue Nov 2, 2023
Merged via the queue into master with commit 299126b Nov 2, 2023
34 of 39 checks passed
@ghost ghost deleted the chessai/fix-benchmarks branch November 2, 2023 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edmundnoble edmundnoble edmundnoble left review comments

@larskuhtz larskuhtz Awaiting requested review from larskuhtz

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@chessai @edmundnoble