Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce Virtualizer test concurrency #4762

Merged
merged 1 commit into from
Sep 13, 2024
Merged

Reduce Virtualizer test concurrency #4762

merged 1 commit into from
Sep 13, 2024

Conversation

graynorton
Copy link
Contributor

Virtualizer tests recently started failing.

Some troubleshooting indicates that the failures occur when many test files are run concurrently. It's unclear why this wasn't a problem in the past; clearly, it's something in our tree of testing dependencies—maybe even an issue with the version of Chromium currently being installed by Playwright, given that we haven't knowingly changed any other deps.

In any case, reducing concurrency to 1 gets the tests passing again.

@changeset-bot changeset-bot
Copy link

changeset-bot bot commented Sep 13, 2024

🦋 Changeset detected

Latest commit: b2290b7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 13, 2024 •
edited
Loading

📊 Tachometer Benchmark Results

Summary

nop-update

  • this-change, tip-of-tree, previous-release: faster ✔ 0% - 7% (0.00ms - 0.81ms)
    this-change vs tip-of-tree

render

  • this-change: 45.58ms - 56.71ms
  • this-change, tip-of-tree, previous-release: unsure 🔍 -3% - +3% (-0.61ms - +0.50ms)
    this-change vs tip-of-tree
  • this-change, tip-of-tree, previous-release: unsure 🔍 -1% - +3% (-0.30ms - +1.16ms)
    this-change vs tip-of-tree
  • this-change, tip-of-tree, previous-release: unsure 🔍 -30% - +18% (-16.60ms - +10.06ms)
    this-change vs tip-of-tree

update

  • this-change: 472.66ms - 482.02ms
  • this-change, tip-of-tree, previous-release: unsure 🔍 -8% - +2% (-3.09ms - +0.89ms)
    this-change vs tip-of-tree
  • this-change, tip-of-tree, previous-release: unsure 🔍 -2% - +0% (-1.22ms - +0.34ms)
    this-change vs tip-of-tree
  • this-change, tip-of-tree, previous-release: unsure 🔍 -0% - +1% (-1.82ms - +6.34ms)
    this-change vs tip-of-tree

update-reflect

  • this-change: 471.01ms - 479.38ms
  • this-change, tip-of-tree, previous-release: unsure 🔍 -1% - +0% (-4.29ms - +2.30ms)
    this-change vs tip-of-tree

Results

this-change

render

VersionAvg timevs
45.58ms - 56.71ms-

update

VersionAvg timevs
472.66ms - 482.02ms-

update-reflect

VersionAvg timevs
471.01ms - 479.38ms-
this-change, tip-of-tree, previous-release

render

VersionAvg timevs this-change
vs tip-of-tree
tip-of-tree		vs previous-release
previous-release
this-change
18.22ms - 18.90ms-unsure 🔍
-3% - +3%
-0.61ms - +0.50ms		unsure 🔍
-3% - +2%
-0.60ms - +0.34ms
tip-of-tree
tip-of-tree		18.17ms - 19.05msunsure 🔍
-3% - +3%
-0.50ms - +0.61ms		-unsure 🔍
-3% - +2%
-0.62ms - +0.46ms
previous-release
previous-release		18.37ms - 19.00msunsure 🔍
-2% - +3%
-0.34ms - +0.60ms		unsure 🔍
-3% - +3%
-0.46ms - +0.62ms		-

update

VersionAvg timevs this-change
vs tip-of-tree
tip-of-tree		vs previous-release
previous-release
this-change
36.21ms - 38.64ms-unsure 🔍
-8% - +2%
-3.09ms - +0.89ms		unsure 🔍
-7% - +3%
-2.50ms - +1.23ms
tip-of-tree
tip-of-tree		36.95ms - 40.10msunsure 🔍
-2% - +8%
-0.89ms - +3.09ms		-unsure 🔍
-4% - +7%
-1.66ms - +2.58ms
previous-release
previous-release		36.65ms - 39.48msunsure 🔍
-3% - +7%
-1.23ms - +2.50ms		unsure 🔍
-7% - +4%
-2.58ms - +1.66ms		-

nop-update

VersionAvg timevs this-change
vs tip-of-tree
tip-of-tree		vs previous-release
previous-release
this-change
11.61ms - 12.15ms-faster ✔
0% - 7%
0.00ms - 0.81ms		unsure 🔍
-4% - +3%
-0.46ms - +0.40ms
tip-of-tree
tip-of-tree		11.99ms - 12.59msunsure 🔍
-0% - +7%
+0.00ms - +0.81ms		-unsure 🔍
-1% - +7%
-0.08ms - +0.83ms
previous-release
previous-release		11.58ms - 12.25msunsure 🔍
-3% - +4%
-0.40ms - +0.46ms		unsure 🔍
-7% - +1%
-0.83ms - +0.08ms		-
this-change, tip-of-tree, previous-release

render

VersionAvg timevs this-change
vs tip-of-tree
tip-of-tree		vs previous-release
previous-release
this-change
34.29ms - 35.65ms-unsure 🔍
-1% - +3%
-0.30ms - +1.16ms		unsure 🔍
-1% - +3%
-0.35ms - +1.09ms
tip-of-tree
tip-of-tree		34.28ms - 34.80msunsure 🔍
-3% - +1%
-1.16ms - +0.30ms		-unsure 🔍
-1% - +1%
-0.41ms - +0.29ms
previous-release
previous-release		34.37ms - 34.83msunsure 🔍
-3% - +1%
-1.09ms - +0.35ms		unsure 🔍
-1% - +1%
-0.29ms - +0.41ms		-

update

VersionAvg timevs this-change
vs tip-of-tree
tip-of-tree		vs previous-release
previous-release
this-change
68.00ms - 68.96ms-unsure 🔍
-2% - +0%
-1.22ms - +0.34ms		unsure 🔍
-1% - +1%
-0.58ms - +0.77ms
tip-of-tree
tip-of-tree		68.31ms - 69.54msunsure 🔍
-0% - +2%
-0.34ms - +1.22ms		-unsure 🔍
-0% - +2%
-0.24ms - +1.31ms
previous-release
previous-release		67.91ms - 68.86msunsure 🔍
-1% - +1%
-0.77ms - +0.58ms		unsure 🔍
-2% - +0%
-1.31ms - +0.24ms		-
this-change, tip-of-tree, previous-release

render

VersionAvg timevs this-change
vs tip-of-tree
tip-of-tree		vs previous-release
previous-release
this-change
41.20ms - 59.66ms-unsure 🔍
-30% - +18%
-16.60ms - +10.06ms		unsure 🔍
-32% - +14%
-18.47ms - +8.31ms
tip-of-tree
tip-of-tree		44.08ms - 63.32msunsure 🔍
-21% - +34%
-10.06ms - +16.60ms		-unsure 🔍
-27% - +21%
-15.48ms - +11.86ms
previous-release
previous-release		45.80ms - 65.22msunsure 🔍
-18% - +38%
-8.31ms - +18.47ms		unsure 🔍
-23% - +29%
-11.86ms - +15.48ms		-

update

VersionAvg timevs this-change
vs tip-of-tree
tip-of-tree		vs previous-release
previous-release
this-change
477.45ms - 483.20ms-unsure 🔍
-0% - +1%
-1.82ms - +6.34ms		unsure 🔍
-1% - +1%
-2.77ms - +5.20ms
tip-of-tree
tip-of-tree		475.18ms - 480.96msunsure 🔍
-1% - +0%
-6.34ms - +1.82ms		-unsure 🔍
-1% - +1%
-5.04ms - +2.95ms
previous-release
previous-release		476.36ms - 481.87msunsure 🔍
-1% - +1%
-5.20ms - +2.77ms		unsure 🔍
-1% - +1%
-2.95ms - +5.04ms		-

update-reflect

VersionAvg timevs this-change
vs tip-of-tree
tip-of-tree		vs previous-release
previous-release
this-change
504.57ms - 509.11ms-unsure 🔍
-1% - +0%
-4.29ms - +2.30ms		unsure 🔍
-1% - +1%
-3.60ms - +2.66ms
tip-of-tree
tip-of-tree		505.46ms - 510.22msunsure 🔍
-0% - +1%
-2.30ms - +4.29ms		-unsure 🔍
-1% - +1%
-2.69ms - +3.74ms
previous-release
previous-release		505.16ms - 509.47msunsure 🔍
-1% - +1%
-2.66ms - +3.60ms		unsure 🔍
-1% - +1%
-3.74ms - +2.69ms		-

tachometer-reporter-action v2 for Benchmarks

@github-actions GitHub Actions
Copy link
Contributor

The size of lit-html.js and lit-core.min.js are as expected.

@justinfagnani justinfagnani merged commit 42260c2 into main Sep 13, 2024
9 checks passed
@justinfagnani justinfagnani deleted the update-virtualizer-test-config branch September 13, 2024 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@justinfagnani justinfagnani justinfagnani approved these changes

@kevinpschaaf kevinpschaaf Awaiting requested review from kevinpschaaf kevinpschaaf is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@graynorton @justinfagnani