-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reorder semi and anti joins. #11815
Merged
Mytherin
merged 118 commits into
duckdb:feature
from
Tmonster:reorder_semi_anti_joins_easier_fix_refactor
Jun 5, 2024
Merged
Reorder semi and anti joins. #11815
Changes from 1 commit
Commits
Show all changes
118 commits
Select commit
Hold shift + click to select a range
18b7e51
I think we can reorder semi joins
Tmonster 77d503e
I have access to the left and right sets and the join type. Now I sho…
Tmonster 34ad120
can now estimate a semi join to be 20% of the left table
Tmonster d8f8f9f
have most everything working. some CEs not quite. Q05 tpch needs to b…
Tmonster 8e6b730
Merge remote-tracking branch 'upstream/main' into reorder-semi-and-an…
Tmonster 4ef7415
some debugging statements
Tmonster d0e4892
make format-fix
Tmonster 9eecc85
fix code where numerator relations were not properly being merged. Al…
Tmonster 2732612
add cross product join type
Tmonster bd1d164
fix join is reorderabe function. generate enums
Tmonster 8be872d
pausing point
Tmonster b9885f6
fix last join issue and better min/max stats help on distinct count
Tmonster e7eaf10
remove min max stats changesg
Tmonster 2183142
add back in removed test
Tmonster 7f49a57
pausing to work on adding benchmarks for ingestion
Tmonster 206109e
trying to fix this column lifetime analyzer bug
Tmonster 830cdec
you can also change the bindings filters return. but that's not a goo…
Tmonster dfab1b6
remove changing what bindings filters return
Tmonster 72cd1be
check tests one more time. still need to figure out logical as of joi…
Tmonster 990b296
Merge branch 'reorder-semi-and-anti-joins' of github.com:Tmonster/duc…
Tmonster 333f252
make format-fix
Tmonster 7d7690c
might be a better solution, lets see if debug passes
Tmonster 4414834
Merge remote-tracking branch 'upstream/main' into reorder_semi_anti_j…
Tmonster b22940f
remove debugging statements
Tmonster 8f8edbd
fix logic for cross product relations
Tmonster d651dcf
skip test that I will remove eventually
Tmonster 1fdb318
these ideas dont work tbh
Tmonster 8fc0048
so close, but cant figure it out yet
Tmonster 9827732
still need to track down this bug
Tmonster 3ee5e65
follow this path. Somewhere in the test visit_operator_expressions_in…
Tmonster 49548c1
I think we can reorder semi joins
Tmonster 4ad73e1
I have access to the left and right sets and the join type. Now I sho…
Tmonster f665ea7
can now estimate a semi join to be 20% of the left table
Tmonster 6c39fb1
have most everything working. some CEs not quite. Q05 tpch needs to b…
Tmonster c45bc52
some debugging statements
Tmonster c9bbc47
make format-fix
Tmonster 7f1ab5e
fix code where numerator relations were not properly being merged. Al…
Tmonster 57a21f2
add cross product join type
Tmonster 30983ec
fix join is reorderabe function. generate enums
Tmonster 818bceb
pausing point
Tmonster 8b250a7
fix last join issue and better min/max stats help on distinct count
Tmonster 168e1d0
remove min max stats changesg
Tmonster bf24aed
add back in removed test
Tmonster 7802160
pausing to work on adding benchmarks for ingestion
Tmonster be6f854
trying to fix this column lifetime analyzer bug
Tmonster 54d13eb
you can also change the bindings filters return. but that's not a goo…
Tmonster 23a1854
remove changing what bindings filters return
Tmonster d57c740
check tests one more time. still need to figure out logical as of joi…
Tmonster 3108292
make format-fix
Tmonster 5afbd5b
might be a better solution, lets see if debug passes
Tmonster 75f6d9d
remove debugging statements
Tmonster 1d0b442
fix logic for cross product relations
Tmonster 8012a31
skip test that I will remove eventually
Tmonster c8d88e0
these ideas dont work tbh
Tmonster 21817fe
so close, but cant figure it out yet
Tmonster fbd8162
still need to track down this bug
Tmonster 996fd64
follow this path. Somewhere in the test visit_operator_expressions_in…
Tmonster 14dcd16
skip failing test
Tmonster d9441ad
Merge branch 'feature' into reorder_semi_anti_joins_easier_fix
Tmonster 922fb49
no longer fails that test
Tmonster ee258b3
disable other test, see what else fails
Tmonster 9910429
make format-fix
Tmonster a08db68
disable other failing test, see what else fails
Tmonster 46d0114
basically everything works now. Just need to fix the cardinality esti…
Tmonster a26d06b
fix join order optimizer seg fault failure
Tmonster a3adc84
remove/clean up tests
Tmonster 5076c09
clean up some unused code
Tmonster 9ab4ca4
Merge branch 'reorder_semi_anti_joins_easier_fix' of github.com:Tmons…
Tmonster 5a00256
remove randomly failing test
Tmonster faa0755
more code clean up
Tmonster 71da791
make format-fix
Tmonster e4352af
comment out ensuring statististcs because arrow has not statistics
Tmonster 1cb53ad
make format-fix (again)
Tmonster 4d7b82e
rewrite mark join needs to be in more places
Tmonster e4d78f4
Merge remote-tracking branch 'upstream/feature' into reorder_semi_ant…
Tmonster 9e538dd
distinct count should be max of 1 and the reported distinct count
Tmonster d9df4df
removed unused variable
Tmonster e01ab55
clean up test fileg
Tmonster 790bd63
remove hashtag
Tmonster b6bb52a
clang tidy fixes #1
Tmonster c5a2a42
make format-fixes 2
Tmonster c9f56d1
pr comments 1
Tmonster 7dc4bcb
add more comments for readability
Tmonster b998e86
it points to unused memory is because we remove entries from the subg…
Tmonster 004f308
refactoring is a lot easier to think about if it is just a spanning tree
Tmonster 6c8644f
lots of refactor done, need to understand where query graph edges com…
Tmonster d53a5f7
Merge branch 'feature' into reorder_semi_anti_joins_easier_fix_refactor
Tmonster 3093b9a
got it all to compile and work finally
Tmonster 3be1487
fixes with join order optimizer
Tmonster 5bac735
why is q09 regressing?
Tmonster 50ed8cb
refactor should be done now
Tmonster f30f26c
more small fixes
Tmonster 28b9e76
remove break point
Tmonster 02fbf90
revert script changes, actually remove unneeded break point
Tmonster 01dc878
remove Print tdom info and other unused code
Tmonster a347cf7
Merge branch 'feature' into reorder_semi_anti_joins_easier_fix_refactor
Tmonster 7553dd4
fix regression and refactor function name
Tmonster 601ea05
remove break_here
Tmonster 75bcac6
fixed q95 again.
Tmonster 0d3c357
Merge remote-tracking branch 'upstream/feature' into reorder_semi_ant…
Tmonster 6bc5e0e
remove unused filter strength variable. add constant semi_anti filter…
Tmonster eab427f
Merge branch 'feature' into reorder_semi_anti_joins_easier_fix_refactor
Tmonster a8ad4d0
will this pass CI?
Tmonster 13e58e9
make format-fix
Tmonster fe6fdef
remove unused variable
Tmonster 3fe233f
Merge remote-tracking branch 'upstream/feature' into reorder_semi_ant…
Tmonster db16bac
make format-fix
Tmonster 4c5ef7d
Merge branch 'feature' into reorder_semi_anti_joins_easier_fix_refactor
Tmonster b3d7955
add explicit constructor
Tmonster 9a02adb
make format-fix
Tmonster 82043f3
change default semi anti selectivity to 5
Tmonster a7b60eb
add test to make sure push down is happening
Tmonster 72ec97a
make format-fix
Tmonster 3f039bc
CI should run again please
Tmonster f3ffde1
add patch for substrait for semi and anti joins
Tmonster fe56a6b
Merge remote-tracking branch 'upstream/feature' into reorder_semi_ant…
Tmonster 8fa3935
actually apply patches
Tmonster 336dafa
fix patch fileg
Tmonster File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
lots of refactor done, need to understand where query graph edges com…
…es from
- Loading branch information
commit 6c8644fa1ce7fd66edc908a504db41e97bdfed45
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Can this be a reference or an optional_ptr?