issue_comments

Since we have eliminated xarray with this, you should be able to submit an issue to the h5py issue tracker while mentioning that this is probably a bug in libhdf5 since netcdf4 also fails with the same error (and you can also link this issue for more information)

great, good to know. Can you try this with h5py: ```python import h5py

N_TIMES = 48 with h5py.File("test.nc", mode="w") as f: time = f.create_dataset("time", (N_TIMES,), dtype="i") time[:] = 0

d1 = f.create_dataset("d1", (N_TIMES, 201, 201), dtype="f")
d1[:] = 0

```

okay, then does changing the dtype do anything? I.e. does this only happen with datetime64 / bytes, or do int / float / str also fail?

with this: python ds2 = ds.time.dt.strftime("%Y%m%d%H%M%S").str.encode("utf-8").to_dataset().assign(d1=ds.d1) but we don't really need to check if the first dataset already fails.

Now I'd probably check if it's just the size that makes it fail (i.e. remove "time" from ds and keep just d1 while maybe increasing it by one if it does not fail as-is), or if it depends on the dtype (i.e. replace set time_vals to np.arange(N_TIMES, dtype=int)).

which ones fail if you add the 3D variable?

if this crashes with both netcdf4 and h5netcdf this might be a bug in the libhdf5 library. If we can manage to reduce this to use just h5py (or netCDF4), it should be suitable for reporting on their issue tracker, and those libraries can then push it further to libhdf5 (otherwise, if you're up for investigating / debugging the C library, you could also report to libhdf5 directly).

As for the MCVE: I wonder if we can trim it a bit. Can you reproduce with ```python import xarray as xr import pandas as pd

N_TIMES = 48 time_vals = pd.date_range("2022-10-06", freq="20 min", periods=N_TIMES) ds = xr.Dataset({"time": ("T", time_vals)}) ds.to_netcdf(path="/my_s3_fs/test_netcdf.nc", format="NETCDF4", mode="w") or, if it is important to have bytes:python ds2 = ds.time.dt.strftime("%Y%m%d%H%M%S").str.encode("utf-8").to_dataset() ds2.to_netcdf(...) also it would be interesting to know if this happens only for data variables, or if coordinates have the same effect (use `ds2` instead of `ds2` if bytes are important):python ds3 = ds.set_coords("time") ds3.to_netcdf(...) ```

Thanks @d1mach . Could it be related to https://github.com/pydata/xarray/issues/7136 ?

That's quite an old version of xarray! Could we confirm it has similar results on a more recent version?