Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improvements to Arrow updates and indexed columns #837

Merged
merged 13 commits into from
Dec 9, 2019
Merged

Improvements to Arrow updates and indexed columns #837

merged 13 commits into from
Dec 9, 2019

Conversation

sc1f
Copy link
Contributor

@sc1f sc1f commented Dec 7, 2019

  • Updates using Apache Arrow use Perspective table schema, thus allowing single-column and partial updates with Arrow datasets
  • Assert correct behavior for arrow updates on tables created from schema, where arrow int64 needs to supersede int32 columns defined in Perspective.
  • Rewrite and centralize arrow loading in Perspective.js test suite
  • Allow for null to be a valid value in columns used as index
  • Remove vestigial code in t_vocab, unify behavior of t_vocab across JS and Python runtimes
  • Add more arrow tests in Python, including for dictionary, file writer, and legacy-flagged arrows

@sc1f sc1f requested a review from texodus December 7, 2019 23:27
@sc1f sc1f requested a review from timkpaine December 7, 2019 23:27
@sc1f sc1f added C++ enhancement Feature requests or improvements internal Internal refactoring and code quality improvement JS Python labels Dec 7, 2019
@texodus
Copy link
Member

texodus commented Dec 9, 2019

Thanks for the PR! Very nice work and great test coverage!

@texodus texodus merged commit 359cd0b into master Dec 9, 2019
@texodus texodus deleted the arrow-dates branch December 9, 2019 05:40
@texodus texodus removed the enhancement Feature requests or improvements label Jan 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@timkpaine timkpaine timkpaine approved these changes

@texodus texodus Awaiting requested review from texodus

Assignees
No one assigned
Labels
C++ internal Internal refactoring and code quality improvement JS Python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@sc1f @texodus @timkpaine @finos-admin