Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added rocksdb with transactions using Yiyuanliu branch of rust-rocksdb. #173

Merged
merged 3 commits into from
Aug 5, 2022
Merged

Conversation

Mo0nbase
Copy link
Contributor

TODO:

  • Fix the lifetimes for the raw prefix search
  • Change the API and Database trait to support self referencing transactions
  • Use native rocksdb column families instead of prefixing (e.g. collecting signature shares for a given token and deleting the column later)
  • Perhaps create inherited traits for basic database -> transactional database -> transactional + column family database
  • Crate generic trait for expected "transaction" behaviors and functionality
  • Get rid of sled_impl and replace all references
  • Remove Batch.rs
  • Improve UT and add coverage for all expected trait functionality
    Note: transactions are soon to be merged into rust-rocksdb we are simply waiting on a couple changes for the next rocksdb you can track the issue here https://github.com/rust-rocksdb/rust-rocksdb/pull/565/files#. Obviously we will want to change to master as soon as possible.

@justinmoon justinmoon requested a review from maan2003 June 24, 2022 15:44
@maan2003
Copy link
Member

I think most of these TODOs should be addressed in separate PRs.

For lifetimes, feel free to add 98eae77 to the PR

@maan2003
Copy link
Member

On wasm side, I suggest making rocksdb optional dependency, and enabling it in minimint, ln-gateway and mint-client-cli

@Mo0nbase
Copy link
Contributor Author

Okay the build should be fixed although with the inclusion of C code GitHub wont be able to compile and test without clang tools (I'm not sure how that works). As for the TODO's I think I'll limit this PR to adding the remaining tests for the existing database trait and making rocksdb an optional dependency and enabling it in those files Maan2003 mentioned. The other changes such as the migration to transactions and the other TODO's will have separate PR's.

@maan2003
Copy link
Member

maan2003 commented Jun 28, 2022

notes for CI:

  • run cargo fmt
  • install clang in nix
  • make rocksdb optional dependency, so it is not used on wasm

@maan2003
Copy link
Member

feel free to include 9ebfa35 for wasm fixes

@Mo0nbase
Copy link
Contributor Author

Thank you @maan2003! The current problem is getting this to compile in CI issues with what looks like a vendored librocksdb-sys which is incompatible with the nixos filesystem.

minimint-api/Cargo.toml Outdated Show resolved Hide resolved
@Mo0nbase Mo0nbase marked this pull request as ready for review July 8, 2022 05:56
@elsirion
Copy link
Contributor

elsirion commented Jul 8, 2022

I added a commit to fix the builds in nix-shell. Just needs a fix for the flake based stuff now, I asked @wiredhikari and he had some ideas how to do it that need documentation, maybe you can coordinate with him.

@wiredhikari
Copy link
Member

You can try a nix flake update

@elsirion
Copy link
Contributor

elsirion commented Jul 8, 2022

Awesome that the tests finally run, you can probably clean up the git history a bit and squash some commits, then this should be ready for review!

@justinmoon
Copy link
Contributor

What's the status here?

@maan2003
Copy link
Member

I have rebased and squashed some commits in https://github.com/Maan2003/minimint/commits/db-rebase

@justinmoon
Copy link
Contributor

The rust-rocksdb PR was just approved, but not merged yet.

I think we should just rebase and merge this. Looks like ^^ will get in.

Copy link
Contributor

@elsirion elsirion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants