Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tweak date/datetime inference, remove dependency on non-core Numpy/Pandas API #802

Merged
merged 10 commits into from
Nov 10, 2019
Merged

Tweak date/datetime inference, remove dependency on non-core Numpy/Pandas API #802

merged 10 commits into from
Nov 10, 2019

Conversation

sc1f
Copy link
Contributor

@sc1f sc1f commented Nov 8, 2019 •
edited
Loading

This PR fixes the inference of dates and datetimes for pandas DataFrames, specifically for DatetimeIndex of various frequencies. Instead of inferring the data type based on the datetime64 unit, we use the index's datetime frequency to decide whether to type the column as a date or a datetime.

Other fixes:

  • remove fillna from our dataframe processing flow
  • remove dependencies on masked_invalid and non top-level numpy/pandas API
  • enforce all column names as strings, and throw an error otherwise
  • add new tests + util.make_dataframe and util.make_series as test utilities

timkpaine
timkpaine previously requested changes Nov 8, 2019
View reviewed changes
python/perspective/perspective/table/_accessor.py Outdated Show resolved Hide resolved
@timkpaine timkpaine added enhancement Feature requests or improvements Python labels Nov 9, 2019
@texodus texodus dismissed timkpaine’s stale review November 10, 2019 03:05

Reviewed offline

@texodus texodus merged commit 3ec9b22 into master Nov 10, 2019
@texodus texodus deleted the numpy-fix branch November 10, 2019 03:05
@texodus texodus added internal Internal refactoring and code quality improvement and removed enhancement Feature requests or improvements labels Jan 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@timkpaine timkpaine timkpaine left review comments

Assignees
No one assigned
Labels
internal Internal refactoring and code quality improvement Python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@sc1f @timkpaine @texodus @finos-admin