Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve rebalance flow #2372

Merged
merged 3 commits into from
Dec 22, 2024
Merged

improve rebalance flow #2372

merged 3 commits into from
Dec 22, 2024

Conversation

mensfeld
Copy link
Member

This PR moves the post-revoked data revocation closer to the revocation event and removes no longer needed (because of this change) diffing of post-revoked messages via lost_partitions. Because we do cleanup now right after the revocation, we don't have to compute lost partitions anymore. Aside from fixing the below bug in transactional flows it also improves the general flow.

close #2370

A lot of extra specs are added just to cover various cases.

@mensfeld mensfeld requested a review from nijikon December 21, 2024 20:48
@mensfeld mensfeld self-assigned this Dec 21, 2024
@mensfeld mensfeld added enhancement New features, updated functionalities and improvements transactions Things related to Transactions and Exactly-Once-Semantics labels Dec 22, 2024
@mensfeld mensfeld marked this pull request as ready for review December 22, 2024 16:18
@mensfeld mensfeld merged commit b773edb into master Dec 22, 2024
31 checks passed
@mensfeld mensfeld deleted the 2370 branch December 22, 2024 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New features, updated functionalities and improvements transactions Things related to Transactions and Exactly-Once-Semantics
Development

Successfully merging this pull request may close these issues.

Non cooperative-sticky transactional offset management can refetch reclaimed partitions
1 participant