Open
Description
In addition to being affected by the memory-inefficiency of chunk uploading (#2291), sourcemap uploads are affected by even more severe memory usage issues.
From investigating a sourcemap upload with a memory profiler, I observed three large memory allocations which we should attempt to fix. Note that these tests were performed with --no-rewrite
(more info on this later):
- The first (and longest) allocation reads all of the sourcemaps into memory. This allocation lasts for pretty much the entire program run. We perform the allocation on this line, which we call when collecting sourcemaps for upload.
- We can probably address this allocation by memory-mapping the sourcemap files.
- However, we perform some processing/validation with the
sourcemap
crate. The crate's documentation states that it loads the entire sourcemap into memory, so we might end up still needing at least to allocate one sourcemap at a time into memory, even after addressing this problem.
- The second allocation occurs when we clone all of the in-memory sourcemaps in preparation for upload. See Prevent unnecessary memory allocation in sourcemaps uploading #2342. Fixed in ref(sourcemaps): Reduce sourcemap upload memory usage #2343.
- The final allocation occurs in
symbolic
when we validate that the sourcemap is valid UTF-8. Fix is here: ref(sourcebundle): Check UTF-8 validity memory efficiently symbolic#890 and build: Bumpsymbolic
to12.13.3
#2346
Lastly, without --no-rewrite
, memory usage seems to be even worse. We still need to investigate the case where --no-rewrite
is not passed to Sentry CLI.
Metadata
Assignees
Labels
No labels