Reduce benchmark runtime on CI (bytecodealliance#3896)

After adding the `call`-oriented benchmark recently I just noticed that running benchmarks on CI is taking 30+ minutes which is not intended. Instead of running a full benchmark run on CI (which I believe we're not looking at anyway) instead only run the benchmarks for a single iteration to ensure they still work but otherwise don't collect statistics about them. Additionally cap the number of parallel instantiations to 16 to avoid running tons of tests for machines with lots of cpus.
sdeleuze · Mar 7, 2022 · 7c3dd33 · 7c3dd33
1 parent 7b0238f
commit 7c3dd33
Show file tree

Hide file tree

Showing 2 changed files with 3 additions and 3 deletions.
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -360,8 +360,8 @@ jobs:
     - run: rustup target add wasm32-wasi
     - name: Install Rust
       run: rustup update stable && rustup default stable
-    - run: cargo bench
-    - run: cargo bench --features uffd
+    - run: cargo test --benches --release
+    - run: cargo test --benches --release --features uffd
 
   # Verify that cranelift's code generation is deterministic
   meta_determinist_check:

diff --git a/benches/instantiation.rs b/benches/instantiation.rs
@@ -86,7 +86,7 @@ fn bench_parallel(c: &mut Criterion, path: &Path) {
             (engine, pre)
         });
 
-        for threads in 1..=num_cpus::get_physical() {
+        for threads in 1..=num_cpus::get_physical().min(16) {
             let name = format!(
                 "{}: with {} thread{}",
                 path.file_name().unwrap().to_str().unwrap(),