Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor to_arrow, fix row deltas for pivoted views #1519

Merged
merged 2 commits into from
Aug 31, 2021
Merged

Refactor to_arrow, fix row deltas for pivoted views #1519

merged 2 commits into from
Aug 31, 2021

Conversation

sc1f
Copy link
Contributor

@sc1f sc1f commented Aug 23, 2021

This PR makes refactors to to_arrow so that it better matches the behavior of to_format by skipping hidden sorts, fixing bugs that occurred when emitting deltas for 1 and 2-sided contexts, which in many situations were completely incorrect. Additionally, the unique aggregate now emits null for non-unique string values instead of the string "-", which is not semantically correct. A large number of tests have also been added to ensure correctness.

@sc1f sc1f added this to the 1.0.0 milestone Aug 23, 2021
@sc1f sc1f self-assigned this Aug 23, 2021
@sc1f sc1f added bug Concrete, reproducible bugs C++ labels Aug 23, 2021
WIP: fix deltas for 1 sided, col-only still broken

WIP
@sc1f sc1f changed the title Refactor to_arrow, fix row deltas for pivoted views with hidden sort Refactor to_arrow, fix row deltas for pivoted views Aug 27, 2021
@sc1f sc1f marked this pull request as ready for review August 30, 2021 21:23
Copy link
Member

@texodus texodus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thanks for the PR!

@texodus texodus merged commit 34a87e2 into master Aug 31, 2021
@texodus texodus deleted the 2-arrow branch August 31, 2021 01:54
@texodus texodus linked an issue Aug 31, 2021 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Concrete, reproducible bugs C++
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1-sided row deltas fail with hidden sorts applied
2 participants