Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add resharding integration tests #5069

Merged
merged 6 commits into from
Jan 7, 2025
Merged

Add resharding integration tests #5069

merged 6 commits into from
Jan 7, 2025

Conversation

ffuugoo
Copy link
Contributor

@ffuugoo ffuugoo commented Sep 12, 2024

This PR (re)implements resharding integration tests, because existing ones are not applicable anymore, since we moved resharding driver into cluster-manager.

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

  1. Does your submission pass tests?
  2. Have you formatted your code locally using cargo +nightly fmt --all command prior to submission?
  3. Have you checked your code using cargo clippy --all --all-features command?

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

@ffuugoo ffuugoo marked this pull request as ready for review January 3, 2025 12:21
Copy link
Member

@timvisee timvisee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some remarks, happy to resolve them in a separate PR so we can merge this one.

abort_resharding,
])

# Finish resharding
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're also checking we cannot reapply, shall we also assert that we cannot finish resharding a second time?

tests/consensus_tests/test_resharding.py Show resolved Hide resolved
tests/consensus_tests/test_resharding.py Outdated Show resolved Hide resolved

peer_api_uris, _peer_dirs, _bootstrap_uri = start_cluster(tmp_path, 3, None, extra_env=env)
# TODO: Wait for/check... *what*? 🤔
Copy link
Member

@timvisee timvisee Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We cannot check this.

Maybe we want to assert the collection is properly deleted, I don't think we can do more. So lets remove the todo.

Comment on lines +125 to +126
@pytest.mark.parametrize("direction, peers", [("up", 3), ("down", 3)])
def test_resharding_forward(tmp_path: pathlib.Path, direction: Literal["up", "down"], peers: int):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can also parameterize the number of shards/replicas (if that isn't too expensive)

@ffuugoo ffuugoo merged commit ea25a5c into dev Jan 7, 2025
17 checks passed
@ffuugoo ffuugoo deleted the resharding-tests branch January 7, 2025 13:58
timvisee added a commit that referenced this pull request Jan 8, 2025
Co-authored-by: Tim Visée <tim+github@visee.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants