Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calling view.to_columns([options]) causes table to not be scrollable/stuck #1856

Closed
SergioTijerino opened this issue Jun 13, 2022 · 1 comment

Comments

@SergioTijerino
Copy link

Bug Report

Steps to Reproduce:

  1. Call view.to_columns([options]) an await for the result.

Expected Result:

While the table data is being parsed, we are able to play around with the table as normal.

Actual Result:

While the table data is being parsed, we are unable to scroll freely, changing to the settings view is impossible and it basically lags until it finishes.

Environment:

Application is built in Vue3 and error happens while running locally or with AWS CloudFront

@texodus
Copy link
Member

texodus commented Jun 13, 2022

Thanks for the report @SergioTijerino!

This isn't a bug, Perspective's wasm engine is not multi-threaded currently. While there are various threading PRs in flight right now e.g. #1825, in general expensive operations are going to interfere with performance when you invoke them on the engine <perspective-viewer> uses to render (e.g., <perspective-viewer> itself does not internally call to_columns() unbound generally). There are a few potential workarounds for you, depending on your situation:

  1. Avoid moving the entire dataset out if you can, using windowing options e.g. end_row, or queries with filter, columns or group_by options which reduce the resulting data set size (or reduce the number of string columns, which are most expensive). Most of Perspective's performance relies on having access to data directly in wasm memory - if you find you need to frequently pull the entirety of a large dataset in/out of Perspective, you are likely going to end up with a much slower solution than just leaving the dataset in JSON to begin with.
  2. I presume your dataset is very large else this wouldn't be an issue, so using to_arrow() or to_csv() will be much faster than anything that returns JSON structure as we can do the transform in pure wasm. Arrow specifically is fastest and >10x the encoding speed as JSON formats.
  3. While not multi-threaded, Perspective.js is multi-process. You can create as many Worker instances as you want which will run on their own threads, and then calling .to_columns() on one worker will not interfere with the worker used to create the Table passed to your <perspective-viewer>; but, you'll need to instantiate/duplicate the Table in both workers as they cannot share memory. to_arrow() can be helpful here to duplicate data between workers, and it is much faster than JSON encoding, so you could e.g. go from worker1 -> worker2 via arrow, then call to_columns() on worker2 (which presumably was pre-instantiated) which would then run in parallel.

@texodus texodus closed this as completed Jun 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants