[Bug]: Panel has strange I/O issues #264

CodyCBakerPhD · 2023-02-15T02:56:01Z

What happened?

The new Panel feature has some strange behavior when run on certain DANDI files...

On most dandisets it seems to work just fine, such as 000409 (IBL) but others cause a certain failure that I usually indicates some kind of issue with file I/O using fsspec

It's worth noting that I can't reproduce any of these issues by manually calling nwb2widget on the same S3 path

Steps to Reproduce

A reproducible example seems to at least include 000402 (MICrONS) - just pick a file and attempt to view some content in it get the traceback below

Traceback

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File /opt/conda/lib/python3.10/site-packages/ipywidgets/widgets/widget.py:766, in Widget._handle_msg(self, msg)
    764         if 'buffer_paths' in data:
    765             _put_buffers(state, data['buffer_paths'], msg['buffers'])
--> 766         self.set_state(state)
    768 # Handle a state request.
    769 elif method == 'request_state':

File /opt/conda/lib/python3.10/site-packages/ipywidgets/widgets/widget.py:643, in Widget.set_state(self, sync_data)
    638         self._send(msg, buffers=echo_buffers)
    640 # The order of these context managers is important. Properties must
    641 # be locked when the hold_trait_notification context manager is
    642 # released and notifications are fired.
--> 643 with self._lock_property(**sync_data), self.hold_trait_notifications():
    644     for name in sync_data:
    645         if name in self.keys:

File /opt/conda/lib/python3.10/contextlib.py:142, in _GeneratorContextManager.__exit__(self, typ, value, traceback)
    140 if typ is None:
    141     try:
--> 142         next(self.gen)
    143     except StopIteration:
    144         return False

File /opt/conda/lib/python3.10/site-packages/traitlets/traitlets.py:1502, in HasTraits.hold_trait_notifications(self)
   1500 for changes in cache.values():
   1501     for change in changes:
-> 1502         self.notify_change(change)

File /opt/conda/lib/python3.10/site-packages/ipywidgets/widgets/widget.py:694, in Widget.notify_change(self, change)
    691     if name in self.keys and self._should_send_property(name, getattr(self, name)):
    692         # Send new state to front-end
    693         self.send_state(key=name)
--> 694 super().notify_change(change)

File /opt/conda/lib/python3.10/site-packages/traitlets/traitlets.py:1517, in HasTraits.notify_change(self, change)
   1515 def notify_change(self, change):
   1516     """Notify observers of a change event"""
-> 1517     return self._notify_observers(change)

File /opt/conda/lib/python3.10/site-packages/traitlets/traitlets.py:1564, in HasTraits._notify_observers(self, event)
   1561 elif isinstance(c, EventHandler) and c.name is not None:
   1562     c = getattr(self, c.name)
-> 1564 c(event)

File /opt/conda/lib/python3.10/site-packages/nwbwidgets/base.py:94, in dict2accordion.<locals>.on_selected_index(change)
     92 def on_selected_index(change):
     93     if change.new is not None and isinstance(change.owner.children[change.new], widgets.HTML):
---> 94         children[change.new] = nwb2widget(
     95             list(d.values())[change.new], neurodata_vis_spec=neurodata_vis_spec, **pass_kwargs
     96         )
     97         change.owner.children = children

File /opt/conda/lib/python3.10/site-packages/nwbwidgets/base.py:204, in nwb2widget(node, neurodata_vis_spec, **pass_kwargs)
    202             return lazy_tabs(spec, node)
    203         elif callable(spec):
--> 204             visualization = spec(node, neurodata_vis_spec=neurodata_vis_spec, **pass_kwargs)
    205             return vis2widget(visualization)
    206 out1 = widgets.Output()

File /opt/conda/lib/python3.10/site-packages/nwbwidgets/base.py:355, in show_multi_container_interface(node, neurodata_vis_spec)
    352 else:
    353     cls_conf = node.__clsconf__
--> 355 return dict2accordion(
    356     {x["attr"]: getattr(node, x["attr"]) for x in cls_conf},
    357     neurodata_vis_spec=neurodata_vis_spec,
    358 )

File /opt/conda/lib/python3.10/site-packages/nwbwidgets/base.py:83, in dict2accordion(d, neurodata_vis_spec, **pass_kwargs)
     81 def dict2accordion(d: dict, neurodata_vis_spec: dict, **pass_kwargs) -> widgets.Widget:
     82     if len(d) == 1:
---> 83         return nwb2widget(list(d.values())[0], neurodata_vis_spec=neurodata_vis_spec)
     84     children = [widgets.HTML("Rendering...") for _ in d]
     85     accordion = widgets.Accordion(children=children, selected_index=None)

File /opt/conda/lib/python3.10/site-packages/nwbwidgets/base.py:204, in nwb2widget(node, neurodata_vis_spec, **pass_kwargs)
    202             return lazy_tabs(spec, node)
    203         elif callable(spec):
--> 204             visualization = spec(node, neurodata_vis_spec=neurodata_vis_spec, **pass_kwargs)
    205             return vis2widget(visualization)
    206 out1 = widgets.Output()

File /opt/conda/lib/python3.10/site-packages/nwbwidgets/base.py:83, in dict2accordion(d, neurodata_vis_spec, **pass_kwargs)
     81 def dict2accordion(d: dict, neurodata_vis_spec: dict, **pass_kwargs) -> widgets.Widget:
     82     if len(d) == 1:
---> 83         return nwb2widget(list(d.values())[0], neurodata_vis_spec=neurodata_vis_spec)
     84     children = [widgets.HTML("Rendering...") for _ in d]
     85     accordion = widgets.Accordion(children=children, selected_index=None)

File /opt/conda/lib/python3.10/site-packages/nwbwidgets/base.py:204, in nwb2widget(node, neurodata_vis_spec, **pass_kwargs)
    202             return lazy_tabs(spec, node)
    203         elif callable(spec):
--> 204             visualization = spec(node, neurodata_vis_spec=neurodata_vis_spec, **pass_kwargs)
    205             return vis2widget(visualization)
    206 out1 = widgets.Output()

File /opt/conda/lib/python3.10/site-packages/nwbwidgets/behavior.py:90, in route_spatial_series(spatial_series, **kwargs)
     88 else:
     89     return widgets.HTML("Unsupported number of dimensions.")
---> 90 return lazy_tabs(dict_, spatial_series)

File /opt/conda/lib/python3.10/site-packages/nwbwidgets/base.py:124, in lazy_tabs(in_dict, node, style)
    105 """Creates a lazy tab object where multiple visualizations can be used for a single node and are generated on the
    106 fly
    107 
   (...)
    120 
    121 """
    122 tabs_spec = list(in_dict.items())
--> 124 children = [tabs_spec[0][1](node)] + [widgets.HTML("Rendering...") for _ in range(len(tabs_spec) - 1)]
    125 tab = style(children=children)
    126 [tab.set_title(i, label) for i, (label, _) in enumerate(tabs_spec)]

File /opt/conda/lib/python3.10/site-packages/nwbwidgets/timeseries.py:291, in AbstractTraceWidget.__init__(self, timeseries, foreign_time_window_controller, **kwargs)
    288 self.out_fig = None
    290 if foreign_time_window_controller is None:
--> 291     tmin = get_timeseries_mint(timeseries)
    292     tmax = get_timeseries_maxt(timeseries)
    293     self.time_window_controller = StartAndDurationController(tmax, tmin)

File /opt/conda/lib/python3.10/site-packages/nwbwidgets/utils/timeseries.py:75, in get_timeseries_mint(node)
     62 """
     63 Returns the minimum time of any TimeSeries
     64 
   (...)
     72 
     73 """
     74 if node.timestamps is not None:
---> 75     return node.timestamps[0]
     76 elif np.isnan(node.starting_time):
     77     return 0

File h5py/_objects.pyx:54, in h5py._objects.with_phil.wrapper()

File h5py/_objects.pyx:55, in h5py._objects.with_phil.wrapper()

File /opt/conda/lib/python3.10/site-packages/h5py/_hl/dataset.py:766, in Dataset.__getitem__(self, args, new_dtype)
    763 if new_dtype is None:
    764     new_dtype = getattr(self._local, 'astype', None)
--> 766 if self._fast_read_ok and (new_dtype is None):
    767     try:
    768         return self._fast_reader.read(args)

File /opt/conda/lib/python3.10/site-packages/h5py/_hl/base.py:534, in cached_property.__get__(self, obj, cls)
    531 if obj is None:
    532     return self
--> 534 value = obj.__dict__[self.func.__name__] = self.func(obj)
    535 return value

File /opt/conda/lib/python3.10/site-packages/h5py/_hl/dataset.py:745, in Dataset._fast_read_ok(self)
    741 @cached_property
    742 def _fast_read_ok(self):
    743     """Is this dataset suitable for simple reading"""
    744     return (
--> 745         self._extent_type == h5s.SIMPLE
    746         and isinstance(self.id.get_type(), (h5t.TypeIntegerID, h5t.TypeFloatID))
    747     )

File /opt/conda/lib/python3.10/site-packages/h5py/_hl/base.py:534, in cached_property.__get__(self, obj, cls)
    531 if obj is None:
    532     return self
--> 534 value = obj.__dict__[self.func.__name__] = self.func(obj)
    535 return value

File h5py/_objects.pyx:54, in h5py._objects.with_phil.wrapper()

File h5py/_objects.pyx:55, in h5py._objects.with_phil.wrapper()

File /opt/conda/lib/python3.10/site-packages/h5py/_hl/dataset.py:634, in Dataset._extent_type(self)
    630 @cached_property
    631 @with_phil
    632 def _extent_type(self):
    633     """Get extent type for this dataset - SIMPLE, SCALAR or NULL"""
--> 634     return self.id.get_space().get_simple_extent_type()

File h5py/_objects.pyx:54, in h5py._objects.with_phil.wrapper()

File h5py/_objects.pyx:55, in h5py._objects.with_phil.wrapper()

File h5py/h5d.pyx:347, in h5py.h5d.DatasetID.get_space()

ValueError: Invalid dataset identifier (invalid dataset identifier)

Operating System

Linux

Python Version

3.10

Package Versions

nwbwidgets==0.10.1

Code of Conduct

I agree to follow this project's Code of Conduct
Have you ensured this bug was not already reported?

The text was updated successfully, but these errors were encountered:

bendichter · 2023-02-15T15:37:06Z

s3 and fsspec use different protocols by default (s3 vs http). Maybe this is an issue with the http protocol for these files

bendichter · 2023-02-15T15:37:55Z

A possible solution might be to change fsspec to the s3 backend (which is slightly superior anyway)

CodyCBakerPhD · 2023-06-03T20:40:14Z

I can confirm after even more experimentation that this issue is nearly deterministic when using it to view local files instead of streaming from DANDI

bendichter · 2023-06-15T19:12:50Z

@CodyCBakerPhD what do you mean by this last message? Do you run into the same problems when you open files locally?

bendichter · 2023-06-15T21:25:19Z

I have not been able to reproduce this bug on my personal mac...

bendichter · 2023-06-15T21:34:27Z

Strangely, this works just fine on the hub:

import fsspec
import h5py
from pynwb import NWBHDF5IO
from nwbwidgets.utils.dandi import get_file_url

http = get_file_url("000535", file_path="sub-460654/sub-460654_ses-20190611T181840_behavior+ophys.nwb")

fsys = fsspec.filesystem("http")

f = fsys.open(http, "rb")
file = h5py.File(f)
io = NWBHDF5IO(file=file, load_namespaces=True)

nwbfile = io.read()

nwbfile.processing["ophys"].data_interfaces["DfOverF"].roi_response_series['RoiResponseSeries'].data[:,0]

bendichter · 2023-06-15T21:49:22Z

And works just fine outside of Panel:

from nwbwidgets import nwb2widget
nwb2widget(nwbfile)

CodyCBakerPhD · 2023-06-16T01:59:50Z

what do you mean by this last message? Do you run into the same problems when you open files locally?

@pauladkisson and I found reproduced the same issue while looking at some files on the DANDI Hub, but the files were not being streamed, they were locally available

I have not been able to reproduce this bug on my personal mac...

Even running the Panel in the same exact way on the same exact part of the same exact streamed asset? And same Widgets version?

Would be interesting to know if this is a DANDI Hub specific issue...

* fix bug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * close previous io * changelog --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Ben Dichter <ben.dichter@gmail.com>

CodyCBakerPhD added the bug Something isn't working label Feb 15, 2023

bendichter mentioned this issue Jun 15, 2023

[Bug]: #290

Closed

2 tasks

luiztauffer self-assigned this Jun 21, 2023

luiztauffer mentioned this issue Jun 26, 2023

fix #264 #295

Merged

bendichter closed this as completed in #295 Jun 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Panel has strange I/O issues #264

[Bug]: Panel has strange I/O issues #264

CodyCBakerPhD commented Feb 15, 2023

bendichter commented Feb 15, 2023

bendichter commented Feb 15, 2023

CodyCBakerPhD commented Jun 3, 2023

bendichter commented Jun 15, 2023

bendichter commented Jun 15, 2023

bendichter commented Jun 15, 2023

bendichter commented Jun 15, 2023

CodyCBakerPhD commented Jun 16, 2023 •

edited

Loading

[Bug]: Panel has strange I/O issues #264

[Bug]: Panel has strange I/O issues #264

Comments

CodyCBakerPhD commented Feb 15, 2023

What happened?

Steps to Reproduce

Traceback

Operating System

Python Version

Package Versions

Code of Conduct

bendichter commented Feb 15, 2023

bendichter commented Feb 15, 2023

CodyCBakerPhD commented Jun 3, 2023

bendichter commented Jun 15, 2023

bendichter commented Jun 15, 2023

bendichter commented Jun 15, 2023

bendichter commented Jun 15, 2023

CodyCBakerPhD commented Jun 16, 2023 • edited Loading

CodyCBakerPhD commented Jun 16, 2023 •

edited

Loading