Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleaner API through the Table abstraction, prepare for Python API #642

Merged
merged 12 commits into from
Jul 26, 2019

Conversation

sc1f
Copy link
Contributor

@sc1f sc1f commented Jul 3, 2019

This PR overhauls the internal workings of Perspective.js and the core engine by wrapping the pool and gnode in the Table class, which manages operations on the pool/gnode.

Previously, the C++ engine's notion of a "table" was an internal structure that wasn't fully expressive of what the Javascript notion of "table" was, notably the management of the pool/gnode structures that were essential to the JS conception of the table. The new abstraction introduced in this PR unifies the notions of a Table across the core and JS library, which means further API development should be clearer in terms of requirements on what a table is, does, etc.

Additionally, this PR prepares for the Python API by making the binding surface to the core engine more accessible and easily shared. Through the t_val and t_data_accessor class, we aim to increase code reusability between the Emscripten and the Python bindings.

When work starts on the Python API, we'll be able to write our custom implementations of t_val and t_data_accessor that uses Pybind, and simply plug it in to the existing methods defined in binding.h.

@timkpaine: I built this refactor on top of the Python changes I had made, and dropping the commit and rebasing it is super gnarly because all my commits are on top of it. The old Python code is still in Git, so replacing it for now (especially since we're going to rewrite Python starting in the next week) should be fine.

Some notable/specific changes:

  • t_table becomes t_data_table
  • t_val and t_data_accessor are typedef'd to emscripten::val in the WASM build
  • we no longer use raw pointers to pass between C++/JS
  • Removed column_metadata from table API as it was not being used
  • Made the bodies of the table API functions contain the logic previously in "underscore-marked" private functions, thus bringing the table inline with the patterns of the view API.

Memory semantics around the new Table structure should be equivalent to the current implementation of table/pool/gnode—preliminary benchmarks show no significant impact on performance for better or worse.

@timkpaine
Copy link
Member

please see the other PR and revert the python changes again

@timkpaine
Copy link
Member

Screen Shot 2019-07-02 at 11 29 57 PM

@timkpaine
Copy link
Member

its fine to just rm -rf python/ and then copy from master back in and commit on top. but don't let those changes get into master.

@sc1f
Copy link
Contributor Author

sc1f commented Jul 5, 2019

@timkpaine Python is on the same state as master, although it'll fail the build because the bindings have not been updated.

@sc1f
Copy link
Contributor Author

sc1f commented Jul 15, 2019

Rebased on top of latest master.

@texodus
Copy link
Member

texodus commented Jul 26, 2019

Looks good, thanks for the PR!

@texodus texodus merged commit d81703b into master Jul 26, 2019
@texodus texodus deleted the port-table branch July 26, 2019 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants