-
|
I'd like to be able to create a DataArray with dims, coordinates, etc. but no In case my problem can be solved in an easier way what I'd like is an xarray based solution to track metadata ( |
Beta Was this translation helpful? Give feedback.
Replies: 4 comments 16 replies
-
|
For minimal memory usage: Also see https://xarray-schema.readthedocs.io/en/latest/quickstart.html though it doesn't support coordinate values IIUC |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
From my basic tests it seems like the |
Beta Was this translation helpful? Give feedback.
-
|
No. |
Beta Was this translation helpful? Give feedback.
-
That makes sense, thanks. However, when I try to pickle a |
Beta Was this translation helpful? Give feedback.
-
|
You're purely in numpy land with that question - take it up with them :) |
Beta Was this translation helpful? Give feedback.
-
I don't think this is possible. You have to wrap some kind of array (though it can take up very little space as in Deepak's suggestion).
The dims of a DataArray are the dims of the Variable it contains, so no variable means no dims. And coordinates on a DataArray cannot have dims that the Variable does not have. You can however create an
As well as xarray-schema you might also be interested in how virtualizarr works - by creating a dummy array-like object to wrap, and deliberately avoiding creating any in-memory |
Beta Was this translation helpful? Give feedback.
-
|
That worked as expected, thanks! FWIW it looks like the new |
Beta Was this translation helpful? Give feedback.
-
That's cool! But in terms of compatibility with xarray all we care about is whether they follow the requirements listed here, which I'm guessing |
Beta Was this translation helpful? Give feedback.
-
|
(I'll mark this as answered for now) |
Beta Was this translation helpful? Give feedback.
-
|
#7280 yep. I'm writing a comprehensive answer now that I'll accept if that's okay. |
Beta Was this translation helpful? Give feedback.
-
|
for the sake of completeness (I know I'm kinda late), That process has been going on for a while, but I don't think the Array API or |
Beta Was this translation helpful? Give feedback.
-
|
Thanks to @TomNicholas and @dcherian for the many answers and suggestions, this comprehensive answer just puts all the information in one spot for future readers. AnswerIt's not directly possible to have and empty In my case a more appropriate solution was using and storing the |
Beta Was this translation helpful? Give feedback.
-
|
Random note: pickling So I end up pickling |
Beta Was this translation helpful? Give feedback.
Thanks to @TomNicholas and @dcherian for the many answers and suggestions, this comprehensive answer just puts all the information in one spot for future readers.
Answer
It's not directly possible to have and empty
.dataattribute, but using asparse.COOarray of the correct shape with no nonzeros results in negligible memory usage. At the time of writingscipy.sparse.coo_arraywas not supported as an input toDataArray()and gave an error (see #7280).It may also be possible to use a view of an array by setting
data=np.broadcast_to(np.array([0]), <your shape>)but when trying to save the resulting
xarray.DataArrayas a pickle this resulted in memory usage proportional to the size of the …