When using ZarrFile.create_dataset to create a dataset directly at a subpath that has not been pre-created with create_group, the group is created without metadata. I.e., the .zgroup file containing {"zarr_format": 2} isn't written. As a consequence, the dataset cannot be opened.

Admittedly found with version 2.0.17, but the code changes don't look like 2.0.18 fixes this 😅

Desired behaviour

create_dataset should either write all metadata correctly, or error out in cases it cannot handle.

Reproduce

import z5py
import numpy
from pathlib import Path
print(z5py.__version__)  # 2.0.17
p = Path("C:\\Users\\me\\boop.zarr")
data = (numpy.random.rand(1, 10, 20, 1, 1) * 255).astype(numpy.uint8)
f = z5py.ZarrFile(p, "w")
f.create_dataset("data", data=data)
f.visititems(print)  # Try 1: works
f.create_dataset("volume/data", data=data)
f.visititems(print)  # Try 2: breaks

Contents:

• C:/Users/me/boop.zarr - /data, /volume, .zgroup
• C:/Users/me/boop.zarr/data - .array, 0.0.0.0.0, 0.0.1.0.0
• C:/Users/me/boop.zarr/volume - /data (n.b. no .zgroup)
• C:/Users/me/boop.zarr/volume/data - .array, 0.0.0.0.0, 0.0.1.0.0

Traceback

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\me\.conda\envs\ila\lib\site-packages\z5py\group.py", line 262, in visititems
    for name, obj in self.items():
  File "C:\Users\me\.conda\envs\ila\lib\_collections_abc.py", line 851, in __iter__
    yield (key, self._mapping[key])
  File "C:\Users\me\.conda\envs\ila\lib\site-packages\z5py\group.py", line 79, in __getitem__
    return Dataset._open_dataset(self, name.lstrip('/'))
  File "C:\Users\me\.conda\envs\ila\lib\site-packages\z5py\dataset.py", line 244, in _open_dataset
    ds = _z5py.open_dataset(ghandle, name)
RuntimeError: Invalid path: no metadata existing