perf(accountsdb): begin optimizing snapshot generation #290
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
InKryption
force-pushed
the
ink/faster-snapshot-gen
branch
2 times, most recently
from
8c4518c
to
afdedaf
Compare
|
Spurious CI failure: https://github.com/Syndica/sig/actions/runs/11112601615/job/30875047639?pr=290 (reran the job) |
InKryption
force-pushed
the
ink/faster-snapshot-gen
branch
9 times, most recently
from
df294e3
to
8c5b235
Compare
|
The change in 78c6bc8 yields a 1.75x speed increase in the accountsdb benchmark: |
InKryption
force-pushed
the
ink/faster-snapshot-gen
branch
from
8c5b235
to
e9c1e60
Compare
InKryption
changed the title
InKryption
force-pushed
the
ink/faster-snapshot-gen
branch
from
e9c1e60
to
78c6bc8
Compare
0xNineteen
requested changes
Oct 9, 2024
InKryption
force-pushed
the
ink/faster-snapshot-gen
branch
from
65270b7
to
0f8d5b8
Compare
* `validateLoadFromSnapshot`: now accepts a parameter struct which only accepts exactly the relevant data needed to validate the specified snapshot(s). * `generateFullSnapshotWithCompressor`: re-organized & simplified. * `generateIncrementalSnapshotWithCompressor`: heavily re-organized, and now returns the associated `BankIncrementalSnapshotPersistence` to represent the incremental snapshot data; this was done because all of the data in `IncSnapshotGenerationInfo` is redundant with the aforementioned struct, except for the incremental slot, which is already known by the caller, since they supply it as a primary function parameter.
Decouples the various behaviours of this function into standalone functions, in order to facilitate restructuring it for optimizations, whilst retaining the essence of this procedure.
Pull out the timer/duration, declare its explicit error set, and clarify doc comments.
InKryption
force-pushed
the
ink/faster-snapshot-gen
branch
from
aa49bf8
to
c1a2fff
Compare
Co-authored-by: x19 <100000306+0xNineteen@users.noreply.github.com>
Co-authored-by: x19 <100000306+0xNineteen@users.noreply.github.com>
InKryption
force-pushed
the
ink/faster-snapshot-gen
branch
from
c1a2fff
to
1f25569
Compare
Co-authored-by: x19 <100000306+0xNineteen@users.noreply.github.com>
InKryption
force-pushed
the
ink/faster-snapshot-gen
branch
from
cdfc0f6
to
cd7613a
Compare
0xNineteen
approved these changes
Oct 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #282.
There is likely room for more optimizations, but this sets a good foundation for that work, and I think keeping this down in scope and easy-ish at this level of review would be best - more work can be done in a follow up PR.