Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[adapters] Fix deadlock around catalog handle. #2450

Merged
merged 1 commit into from
Sep 10, 2024
Merged

[adapters] Fix deadlock around catalog handle. #2450

merged 1 commit into from
Sep 10, 2024

Conversation

ryzhyk
Copy link
Contributor

@ryzhyk ryzhyk commented Sep 9, 2024

The circuit thread locked the catalog for much longer than needed, grabbing unrelated locks while holding the catalog lock and causing a deadlock. A simple fix would have been narrowing the critical section, but @gz pointed out that the catalog is only read after it is initially populated, so the mutex is not necessary. This commit refactors the code to eliminate the mutex around the catalog and enables previously disabled tests.

@ryzhyk ryzhyk requested a review from gz September 9, 2024 19:26
Copy link
Collaborator

@gz gz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for debugging and fixing the mess I made :(

@gz gz enabled auto-merge September 9, 2024 19:34
@gz gz added this pull request to the merge queue Sep 9, 2024
@blp
Copy link
Member

blp commented Sep 9, 2024

Best kind of mutex is no mutex!

@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 9, 2024
@gz gz added this pull request to the merge queue Sep 9, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Sep 10, 2024
The circuit thread locked the catalog for much longer than needed,
grabbing unrelated locks while holding the catalog lock and causing a
deadlock.  A simple fix would have been narrowing the critical section,
but @gz pointed out that the catalog is only read after it is initially
populated, so the mutex is not necessary.  This commit refactors the
code to eliminate the mutex around the catalog and enables previously
disabled tests.

Signed-off-by: Leonid Ryzhyk <leonid@feldera.com>
@ryzhyk ryzhyk added this pull request to the merge queue Sep 10, 2024
Merged via the queue into main with commit 4d8dbb2 Sep 10, 2024
5 checks passed
@ryzhyk ryzhyk deleted the catalog_race branch September 10, 2024 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants