-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add code to api.RaftSnapshot to detect incomplete snapshots #12388
Conversation
…any that don't include the final SHA256SUMS.sealed file.
To test whether or not my implementation is a memory hog, I added the following to the end of OperatorRaftSnapshotSaveCommand.Run in operator_raft_snapshot_save.go:
I wrote 100k small keys to a raft cluster and took a snapshot with 1.8.1 and the above change:
With my branch and the above change:
(there's a bit of variability, but re-runs yielded similar numbers). The snapshot file was 11MB uncompressed, and I'm pretty sure if we held it all in memory those memory numbers would've gone up substantially. This confirms my understanding from code inspection of archive/tar, I just wanted to double check. |
Fixes #12168 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, just a question about error handling
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, love the injection of the error in the seal to trigger it.
An incomplete snapshot is one that doesn't include the final SHA256SUMS.sealed file.