Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial support for loading Apache Arrow data #11

Merged
merged 16 commits into from
Jan 26, 2018
Merged

Initial support for loading Apache Arrow data #11

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
c16c1c9
Initial support for Arrow (https://arrow.apache.org/)
nmichaud Jan 12, 2018
fb989ca
In place loading of arrow arrays into perspective
nmichaud Jan 12, 2018
5faf2f2
Don't compile with dynamic executions.
nmichaud Jan 20, 2018
f3469fb
Arrow uses ES6's Symbol, need polyfill for IE
nmichaud Jan 21, 2018
5d43c4d
Add arrow example
nmichaud Jan 21, 2018
7999cdf
Fixed name mangling issue in arrow build.
texodus Jan 22, 2018
f5f3843
Make arrow detection implicit
nmichaud Jan 22, 2018
a8d5264
Clean up differentiation of arrow/non-arrow data
nmichaud Jan 22, 2018
9f36b2e
Fixed webworker bug. Added unit test for Arrow constructor.
texodus Jan 24, 2018
7376186
Convert additional types to js vals
nmichaud Jan 26, 2018
97f0c25
Better dispatch on arrow datatypes for numeric data
nmichaud Jan 26, 2018
d40f820
Unpack arrow bitmap column into psp bool
nmichaud Jan 26, 2018
3be64db
Don't copy more than col->size() data from arrow
nmichaud Jan 26, 2018
235352d
Add test case with all supported arrow datatypes
nmichaud Jan 26, 2018
c29aaed
Refactor to support updates using arrow data
nmichaud Jan 26, 2018
7abd9f0
Rearrange scalar to val output
nmichaud Jan 26, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Better dispatch on arrow datatypes for numeric data
  • Loading branch information
@nmichaud
nmichaud committed Jan 26, 2018
commit 97f0c2582cf2855698db3bb1b91d4eec0cdf1084
36 changes: 16 additions & 20 deletions packages/perspective/src/js/perspective.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1045,29 +1045,25 @@ const perspective = {
types.push(__MODULE__.t_dtype.DTYPE_STR);
break;
case 'FloatingPoint':
switch (column.data.BYTES_PER_ELEMENT) {
case 4:
types.push(__MODULE__.t_dtype.DTYPE_FLOAT32);
break;
case 8:
types.push(__MODULE__.t_dtype.DTYPE_FLOAT64);
break;
if (column instanceof Arrow.Float64Vector) {
types.push(__MODULE__.t_dtype.DTYPE_FLOAT64);
}
else if (column instanceof Arrow.Float32Vector) {
types.push(__MODULE__.t_dtype.DTYPE_FLOAT32);
}
break;
case 'Int':
switch (column.data.BYTES_PER_ELEMENT) {
case 1:
types.push(__MODULE__.t_dtype.DTYPE_INT8);
break;
case 2:
types.push(__MODULE__.t_dtype.DTYPE_INT16);
break;
case 4:
types.push(__MODULE__.t_dtype.DTYPE_INT32);
break;
case 8:
types.push(__MODULE__.t_dtype.DTYPE_INT64);
break;
if (column instanceof Arrow.Int64Vector) {
types.push(__MODULE__.t_dtype.DTYPE_INT64);
}
else if (column instanceof Arrow.Int32Vector) {
types.push(__MODULE__.t_dtype.DTYPE_INT32);
}
else if (column instanceof Arrow.Int16Vector) {
types.push(__MODULE__.t_dtype.DTYPE_INT16);
}
else if (column instanceof Arrow.Int8Vector) {
types.push(__MODULE__.t_dtype.DTYPE_INT8);
}
break;
case 'Bool':
Expand Down