Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(turborepo): Create Cache Tests #5371

Merged
merged 3 commits into from
Jul 6, 2023

Conversation

NicholasLYang
Copy link
Contributor

Description

Tests for creating a cache. These do have different hashes than the Go ones, but I think that's fine since there's no expectation that tar construction or zstd compression is necessarily deterministic.

Testing Instructions

@vercel
Copy link

vercel bot commented Jun 23, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
examples-native-web 🔄 Building (Inspect) Visit Preview 💬 Add feedback Jul 6, 2023 9:46pm
10 Ignored Deployments
Name Status Preview Comments Updated (UTC)
examples-basic-web ⬜️ Ignored (Inspect) Visit Preview Jul 6, 2023 9:46pm
examples-cra-web ⬜️ Ignored (Inspect) Visit Preview Jul 6, 2023 9:46pm
examples-designsystem-docs ⬜️ Ignored (Inspect) Visit Preview Jul 6, 2023 9:46pm
examples-gatsby-web ⬜️ Ignored (Inspect) Visit Preview Jul 6, 2023 9:46pm
examples-kitchensink-blog ⬜️ Ignored (Inspect) Visit Preview Jul 6, 2023 9:46pm
examples-nonmonorepo ⬜️ Ignored (Inspect) Visit Preview Jul 6, 2023 9:46pm
examples-svelte-web ⬜️ Ignored (Inspect) Visit Preview Jul 6, 2023 9:46pm
examples-tailwind-web ⬜️ Ignored (Inspect) Visit Preview Jul 6, 2023 9:46pm
examples-vite-web ⬜️ Ignored (Inspect) Visit Preview Jul 6, 2023 9:46pm
turbo-site ⬜️ Ignored (Inspect) Visit Preview Jul 6, 2023 9:46pm

@NicholasLYang NicholasLYang force-pushed the nicholasyang/create-cache-tests branch from bc15cb5 to 2675f33 Compare June 23, 2023 15:37
@github-actions
Copy link
Contributor

github-actions bot commented Jun 23, 2023

🟢 CI successful 🟢

Thanks

@NicholasLYang NicholasLYang force-pushed the nicholasyang/create-cache-tests branch from 2675f33 to 84b08c2 Compare June 29, 2023 22:57
@NicholasLYang NicholasLYang force-pushed the nicholasyang/create-cache-tests branch 2 times, most recently from 6aa518f to 5da6ac7 Compare June 30, 2023 15:45
@NicholasLYang NicholasLYang force-pushed the nicholasyang/create-cache-tests branch 2 times, most recently from c3d0e8b to f205209 Compare June 30, 2023 20:40
Copy link
Contributor

@arlyon arlyon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened this and realised I had a pending comment which seems to be outdated.

crates/turborepo-cache/src/cache_archive/create.rs Outdated Show resolved Hide resolved
let mut encoder = zstd::Encoder::new(&mut buffer, 0)?.auto_finish();
encoder.write(b"hello world")?;
// Should finish encoding on drop
drop(encoder);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That feels odd to me but it makes sense. Coming from the hash / digest api I am surprised but I guess since the digest apis spit out data when you finalize rather than (I assume) write it as you go it makes sense to have 'implicit finalize' on drop.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we could require an explicit finalize, but that's too easy to miss imo, so it's probably better to just finalize on drop

@@ -33,8 +34,9 @@ impl CacheReader {

pub fn open(path: &AbsoluteSystemPathBuf) -> Result<Self, CacheError> {
let file = path.open()?;
let is_compressed = path.extension() == Some("zst");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably worth discussing here whether we want to check for the zstd magic number as well / instead / as a fallback? It'll be slower in the general case but then we are not dependent on the file name causing issues

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer to @gsoltis on that, but we should probably keep the same Go behavior until we're done with the port

@github-actions
Copy link
Contributor

github-actions bot commented Jul 6, 2023

✅ This change can build next-swc

@github-actions
Copy link
Contributor

github-actions bot commented Jul 6, 2023

Linux Benchmark for ecf5740

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 5854.20µs ± 32.25µs 5901.60µs ± 30.45µs +0.81%
bench_hmr_to_eval/Turbopack CSR/1000 modules 5558.31µs ± 44.12µs 5625.69µs ± 128.04µs +1.21%
bench_startup/Turbopack CSR/1000 modules 910.38ms ± 0.51ms 912.30ms ± 4.02ms +0.21%

@NicholasLYang NicholasLYang force-pushed the nicholasyang/create-cache-tests branch from bf3de20 to 0a8c365 Compare July 6, 2023 19:34
@github-actions
Copy link
Contributor

github-actions bot commented Jul 6, 2023

Linux Benchmark for 6070821

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 5767.18µs ± 23.93µs 5802.22µs ± 36.82µs +0.61%
bench_hmr_to_eval/Turbopack CSR/1000 modules 5514.74µs ± 22.83µs 5494.96µs ± 80.78µs -0.36%
bench_startup/Turbopack CSR/1000 modules 900.81ms ± 0.79ms 901.88ms ± 2.87ms +0.12%

@NicholasLYang NicholasLYang force-pushed the nicholasyang/create-cache-tests branch from 0a8c365 to 398f794 Compare July 6, 2023 21:46
@NicholasLYang NicholasLYang merged commit 0a92abc into main Jul 6, 2023
@NicholasLYang NicholasLYang deleted the nicholasyang/create-cache-tests branch July 6, 2023 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants