This project is part of the @thi.ng/umbrella monorepo.
Benchmarking utilities w/ various statistics & formatters (CSV, Markdown etc.).
Though no public API change (only additions), since v2.0.0 this library
internally attempts to use high-res ES
BigInt
timestamps (in Node via
process.hrtime.bigint()).
If BigInt is not available in the target environment, timestamps are
still only sourced via Date.now().
STABLE - used in production
Search or submit any issues for this package
- @thi.ng/csv - Customizable, transducer-based CSV parser/object mapper and transformer
- @thi.ng/hiccup-markdown - Markdown parser & serializer from/to Hiccup format
yarn add @thi.ng/bench// ES module
<script type="module" src="https://unpkg.com/@thi.ng/bench?module" crossorigin></script>
// UMD
<script src="https://unpkg.com/@thi.ng/bench/lib/index.umd.js" crossorigin></script>Package sizes (gzipped, pre-treeshake): ESM: 1.37 KB / CJS: 1.47 KB / UMD: 1.47 KB
Several demos in this repo's /examples directory are using this package.
A selection:
| Screenshot | Description | Live demo | Source |
|---|---|---|---|
 |
Fork-join worker-based raymarch renderer | Demo | Source |
import { timed, bench, benchmark } from "@thi.ng/bench";
// test functions
const fib = (n) => n > 2 ? fib(n - 1) + fib(n - 2) : n > 0 ? 1 : 0;
const fib2 = (n) => {
const res = [0, 1];
for(let i = 2; i <= n; i++) {
res[i] = res[i - 1] + res[i - 2];
}
return res[n];
};
// measure single execution time
timed(() => fib(40));
// 714ms
// 102334155
timed(() => fib2(40));
// 0ms
// 102334155
// measure 1mil iterations (default)
bench(() => fib(10), 1e6);
// 395ms
// 55
bench(() => fib2(10), 1e6);
// 53ms
// 55The benchmark() function executes a number of warmup runs, before
executing the main measurement and producing a number of useful
statistics: mean, median, min/max, 1st/3rd quartile, standard deviation
(as percentage)...
See api.ts for configuration options.
Also see the formatting section below for other output options. This example uses the default format...
benchmark(() => fib(40), { title: "fib", iter: 10, warmup: 5 });
// benchmarking: fib
// warmup... 3707.17ms (5 runs)
// executing...
// total: 7333.72ms, runs: 10
// mean: 733.37ms, median: 733.79ms, range: [728.58..743.43]
// q1: 730.98ms, q3: 735.03ms
// sd: 0.54%
// also returns results:
// {
// title: "fib",
// iter: 10,
// total: 7333.72402,
// mean: 733.372402,
// median: 733.794194,
// min: 728.5808,
// max: 743.432538,
// q1: 730.980115,
// q3: 735.025314,
// sd: 0.542200865574415
// }Multiple benchmarks can be run sequentially as suite (also returns an array of all results):
b.suite(
[
{ title: "fib2(10)", fn: () => fib2(10) },
{ title: "fib2(20)", fn: () => fib2(20) },
{ title: "fib2(30)", fn: () => fib2(30) },
{ title: "fib2(40)", fn: () => fib2(40) },
],
{ iter: 10, size: 100000, warmup: 5, format: b.FORMAT_MD }
)
// | Title| Iter| Size| Total| Mean| Median| Min| Max| Q1| Q3| SD%|
// |------------------------|-------:|-------:|-----------:|-------:|-------:|-------:|-------:|-------:|-------:|-------:|
// | fib2(10)| 10| 100000| 54.34| 5.43| 5.15| 4.40| 8.14| 4.84| 6.67| 20.32|
// | fib2(20)| 10| 100000| 121.24| 12.12| 12.13| 11.73| 12.91| 11.93| 12.35| 2.61|
// | fib2(30)| 10| 100000| 152.98| 15.30| 14.51| 13.93| 20.77| 14.35| 16.35| 12.65|
// | fib2(40)| 10| 100000| 164.79| 16.48| 15.60| 15.01| 19.27| 15.42| 18.80| 9.34|Same table as actual Markdown:
| Title | Iter | Size | Total | Mean | Median | Min | Max | Q1 | Q3 | SD% |
|---|---|---|---|---|---|---|---|---|---|---|
| fib2(10) | 10 | 100000 | 54.34 | 5.43 | 5.15 | 4.40 | 8.14 | 4.84 | 6.67 | 20.32 |
| fib2(20) | 10 | 100000 | 121.24 | 12.12 | 12.13 | 11.73 | 12.91 | 11.93 | 12.35 | 2.61 |
| fib2(30) | 10 | 100000 | 152.98 | 15.30 | 14.51 | 13.93 | 20.77 | 14.35 | 16.35 | 12.65 |
| fib2(40) | 10 | 100000 | 164.79 | 16.48 | 15.60 | 15.01 | 19.27 | 15.42 | 18.80 | 9.34 |
The following output formatters are available. Custom formatters can be easily
defined (see source for examples). Formatters are configured via the format
option given to benchmark() or suite().
FORMAT_DEFAULT- default plain text formattingFORMAT_CSV- Comma-separated values (w/ column header)FORMAT_MD- Markdown table format
Karsten Schmidt
If this project contributes to an academic publication, please cite it as:
@misc{thing-bench,
title = "@thi.ng/bench",
author = "Karsten Schmidt",
note = "https://thi.ng/bench",
year = 2018
}© 2018 - 2021 Karsten Schmidt // Apache Software License 2.0