This is resolved in recent netcdf-c/netcdf4-python and works with recent Xarray.

Problem source identified in netcdf-c: https://github.com/Unidata/netcdf-c/issues/2159

Sounds good, but it seems like you're correct that it's a netcdf/netcdf4-python problem here, so I'll defer to others as to what the best changes to default settings would be to avoid the segfaults

@scottstanie I'll check my h5py/hdf5 settings. But I doubt that might be the difference. I've experienced that the trailing garbage is changing from run to run, sometimes disappearing.

ah sorry, didn't see the request for ncdump.

``` $ ncdump test_str_list.h5 netcdf test_str_list { dimensions: phony_dim_0 = 2 ; phony_dim_1 = 2 ; variables: string pairs(phony_dim_0, phony_dim_1) ; data:

pairs = "2020010120200201 ", NIL, "2020010120200301 ", NIL ; } `` Interesting that mypairsseems different than yours without the obvious trailing garbage. Also, when I run your first code snippet, I have different areas that are garbled, with bothNULLPADandnumpy_S` displaying garbage

``` netcdf test_str_list_attr {

// global attributes: string :NULLPAD = "20200101�<T��\007", "20200201", "20200101�=T��\007", "20200301" ; string :NULLTERM = "20200101", "20200201", "20200101", "20200301" ; string :numpy_S = "20200101", "20200201\1775T��\007", "20200101", "20200301�3T��\007" ; string :numpy_O = "20200101", "20200201", "20200101", "20200301" ; } ```

@scottstanie Here is the output of ncdump:

``` netcdf test_str_list { dimensions: phony_dim_0 = 2 ; phony_dim_1 = 2 ; variables: string pairs(phony_dim_0, phony_dim_1) ; data:

pairs = "2020010120200201�\f\033��U", NIL, "2020010120200301 ", NIL ; } ```

You see the trailing garbage. This is obviously a problem with netcdf-c/netcdf4-python, as it is not there with pure hdf5 (h5py/h5netcdf).

But, there is a difference with Attributes and Datasets:

```pathon import h5py import xarray as xr

with h5py.File("test_str_list_attr.h5", "w") as hf: sid = h5py.h5s.create_simple((2, 2), (2, 2)) tid1 = h5py.h5t.TypeID.copy(h5py.h5t.C_S1) tid1.set_size(8) tid1.set_strpad(h5py.h5t.STR_NULLPAD)

tid2 = h5py.h5t.TypeID.copy(h5py.h5t.C_S1)
tid2.set_size(9)
tid2.set_strpad(h5py.h5t.STR_NULLTERM)

blob = np.array([["20200101", "20200201"], ["20200101", "20200301"]]).astype("S")

# Attributes
aid = h5py.h5a.create(hf.id, b"NULLPAD", tid1, sid)
ret = aid.write(blob)

aid = h5py.h5a.create(hf.id, b"NULLTERM", tid2, sid)
ret = aid.write(blob)

hf.attrs["numpy_S"] = blob
hf.attrs["numpy_O"] = blob.astype("O")

!h5dump test_str_list_attr.h5 !ncdump test_str_list_attr.h5

with xr.load_dataset("test_str_list_attr.h5", engine="h5netcdf", phony_dims="sort") as ds: display(ds) with xr.load_dataset("test_str_list_attr.h5", engine="netcdf4") as ds: display(ds) with nc.Dataset("test_str_list_attr.h5") as ds: display(ds) display(ds.NULLTERM) display(ds.NULLPAD) display(ds.numpy_O) display(ds.numpy_S) ```

Output:

It's clearly seen, that the Datasets are correct in hdf5 dump, but somehow netcdf-c has issues with the string NULLPAD/NULLTERM. But at least there is no segfault with attributes. Othe than with Datasets/Variables:

```python import h5py import xarray as xr

with h5py.File("test_str_list_ds.h5", "w") as hf: blob = np.array([["20200101", "20200201"], ["20200101", "20200301"]]).astype("S")

# Datasets
sid = h5py.h5s.create_simple((2, 2), (2, 2))

tid3 = h5py.h5t.TypeID.copy(h5py.h5t.C_S1)
tid3.set_size(8)
tid3.set_strpad(h5py.h5t.STR_NULLPAD)

tid4 = h5py.h5t.TypeID.copy(h5py.h5t.C_S1)
tid4.set_size(9)
tid4.set_strpad(h5py.h5t.STR_NULLTERM)

aid = h5py.h5d.create(hf.id, b"NULLPAD", tid3, sid)
ret = aid.write(sid, h5py.h5s.ALL, blob)

aid = h5py.h5d.create(hf.id, b"NULLTERM", tid4, sid)
ret = aid.write(sid, h5py.h5s.ALL, blob)

hf["numpy_S"] = blob
hf["numpy_O"] = blob.astype("O")

!h5dump test_str_list_ds.h5 !ncdump test_str_list_ds.h5

with xr.load_dataset("test_str_list_ds.h5", engine="h5netcdf", phony_dims="sort") as ds: display(ds)

with xr.load_dataset("test_str_list_ds.h5", engine="netcdf4") as ds:

display(ds["numpy_O"])

with nc.Dataset("test_str_list_ds.h5") as ds:

display(ds)

#display("NULLTERM:", ds["NULLTERM"][:])

#display("NULLPAD:", ds["NULLPAD"][:])

display("numpy_O", ds["numpy_O"][:])

#display("numpy_S", ds["numpy_S"][:])

```
Output:

So here, netcdf-c/netcdf4-python will segfault for all variables beside numpy_O.

It looks like the only option to achieve this for datasets/variables is to use numpy opaque dtype.

sure! here it is:

bash $ h5dump test_str_list.h5 HDF5 "test_str_list.h5" { GROUP "/" { DATASET "pairs" { DATATYPE H5T_STRING { STRSIZE 8; STRPAD H5T_STR_NULLPAD; CSET H5T_CSET_ASCII; CTYPE H5T_C_S1; } DATASPACE SIMPLE { ( 2, 2 ) / ( 2, 2 ) } DATA { (0,0): "20200101", "20200201", (1,0): "20200101", "20200301" } } } }

(and just to include the specific traceback that hapened now, in case my versions are different from what I showed):

xr.show_versions

@scottstanie Could you please provide the output of h5dump test_str_list.h5? I've a hunch but want to be sure. Also, what is the output with ncdump?