-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hive partitioning: Fix preprocessing of CreateDirectories #9535
Conversation
Fixes duckdblabs/duckdb-internal#588 improving on duckdb#9473. Idea is that we iterate on all global partitions instead of iterating on the local ones.
This comment was marked as outdated.
This comment was marked as outdated.
Actually, not yet fixed in a slightly more involved case. |
Invariant would be that CreateDirectories will only be called while holding the global lock, and the same global lock will be held while iterating on what partitions needs to be created.
7366472
to
85256c3
Compare
Thanks! |
This comment was marked as abuse.
This comment was marked as abuse.
@l1t1: Thanks for double checking on this, but you mentioned what I think is an outdated link, could you give either https://github.com/duckdb/duckdb/actions/runs/6719943007?pr=9535 (note the link you cited ended with 6717806759) or nightly builds like https://github.com/duckdb/duckdb/actions/runs/6726710145 ? Thanks a lot |
This comment was marked as abuse.
This comment was marked as abuse.
Thanks for checking! |
Ran my test case and results look as expected. Thanks :-)
|
Fixes https://github.com/duckdblabs/duckdb-internal/issues/588 improving on #9473.
Idea is that we iterate on all global partitions instead of iterating on the local ones.
Thanks @l1t1 for providing the test case.