issue_comments

Sounds good, but it seems like you're correct that it's a netcdf/netcdf4-python problem here, so I'll defer to others as to what the best changes to default settings would be to avoid the segfaults

ah sorry, didn't see the request for ncdump.

``` $ ncdump test_str_list.h5 netcdf test_str_list { dimensions: phony_dim_0 = 2 ; phony_dim_1 = 2 ; variables: string pairs(phony_dim_0, phony_dim_1) ; data:

pairs = "2020010120200201 ", NIL, "2020010120200301 ", NIL ; } `` Interesting that mypairsseems different than yours without the obvious trailing garbage. Also, when I run your first code snippet, I have different areas that are garbled, with bothNULLPADandnumpy_S` displaying garbage

``` netcdf test_str_list_attr {

// global attributes: string :NULLPAD = "20200101�<T��\007", "20200201", "20200101�=T��\007", "20200301" ; string :NULLTERM = "20200101", "20200201", "20200101", "20200301" ; string :numpy_S = "20200101", "20200201\1775T��\007", "20200101", "20200301�3T��\007" ; string :numpy_O = "20200101", "20200201", "20200101", "20200301" ; } ```

Sorry, should have included that version of the error as well. Both give errors.

Here's when I use engine=h5netcdf ``` Traceback (most recent call last): File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/file_manager.py", line 199, in _acquire_with_cache_info file = self._cache[self._key] File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/lru_cache.py", line 53, in getitem value = self._cache[key] KeyError: [<class 'h5netcdf.core.File'>, ('/home/scott/testdata.nc',), 'a', (('invalid_netcdf', None),)]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "testxr.py", line 35, in <module> da_new.to_dataset(name="new_testdata").to_netcdf("testdata.nc", engine="h5netcdf") File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/core/dataset.py", line 1900, in to_netcdf return to_netcdf( File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/api.py", line 1060, in to_netcdf store = store_open(target, mode, format, group, kwargs) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 178, in open return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 123, in init self.filename = find_root_and_group(self.ds)[0].filename File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf.py", line 189, in ds return self.acquire() File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf.py", line 181, in _acquire with self._manager.acquire_context(needs_lock) as root: File "/home/scott/miniconda3/envs/mapping/lib/python3.8/contextlib.py", line 113, in enter return next(self.gen) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/file_manager.py", line 187, in acquire_context file, cached = self._acquire_with_cache_info(needs_lock) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/file_manager.py", line 205, in _acquire_with_cache_info file = self._opener(*self._args, kwargs) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/h5netcdf/core.py", line 712, in init self._h5file = h5py.File(path, mode, **kwargs) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/h5py/_hl/files.py", line 406, in init fid = make_fid(name, mode, userblock_size, File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/h5py/_hl/files.py", line 179, in make_fid fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5f.pyx", line 108, in h5py.h5f.create OSError: Unable to create file (unable to lock file, errno = 11, error message = 'Resource temporarily unavailable') ```

And here's the original ```$ python testxr.py Traceback (most recent call last): File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/file_manager.py", line 199, in _acquire_with_cache_info file = self._cache[self._key] File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/lru_cache.py", line 53, in getitem value = self._cache[key] KeyError: [<class 'h5netcdf.core.File'>, ('/home/scott/testdata.nc',), 'a', (('invalid_netcdf', None),)]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "testxr.py", line 35, in <module> da_new.to_dataset(name="new_testdata").to_netcdf("testdata.nc", engine="h5netcdf") File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/core/dataset.py", line 1900, in to_netcdf return to_netcdf( File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/api.py", line 1060, in to_netcdf store = store_open(target, mode, format, group, kwargs) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 178, in open return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 123, in init self.filename = find_root_and_group(self.ds)[0].filename File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf.py", line 189, in ds return self.acquire() File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf.py", line 181, in _acquire with self._manager.acquire_context(needs_lock) as root: File "/home/scott/miniconda3/envs/mapping/lib/python3.8/contextlib.py", line 113, in enter return next(self.gen) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/file_manager.py", line 187, in acquire_context file, cached = self._acquire_with_cache_info(needs_lock) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/file_manager.py", line 205, in _acquire_with_cache_info file = self._opener(*self._args, kwargs) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/h5netcdf/core.py", line 712, in init self._h5file = h5py.File(path, mode, **kwargs) File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/h5py/_hl/files.py", line 406, in init fid = make_fid(name, mode, userblock_size, File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/h5py/_hl/files.py", line 179, in make_fid fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5f.pyx", line 108, in h5py.h5f.create OSError: Unable to create file (unable to lock file, errno = 11, error message = 'Resource temporarily unavailable') ```

xr.show_versions()

sure! here it is:

bash $ h5dump test_str_list.h5 HDF5 "test_str_list.h5" { GROUP "/" { DATASET "pairs" { DATATYPE H5T_STRING { STRSIZE 8; STRPAD H5T_STR_NULLPAD; CSET H5T_CSET_ASCII; CTYPE H5T_C_S1; } DATASPACE SIMPLE { ( 2, 2 ) / ( 2, 2 ) } DATA { (0,0): "20200101", "20200201", (1,0): "20200101", "20200301" } } } }

(and just to include the specific traceback that hapened now, in case my versions are different from what I showed):

xr.show_versions

As a temporary workout for my case, I'm just going to do

python In [3]: pf = (da.copy(True)).polyfit("z", deg=2, w=np.arange(nz)) In [5]: da.max(), da.mean() Out[5]: (<xarray.DataArray ()> array(0.99878237), <xarray.DataArray ()> array(0.50869358))

The thing I don't understand is that _to_temp_dataset seems to be trying to do a deep copy (based on the argument names)

https://github.com/pydata/xarray/blob/da99a5664df4f5013c2f6b0e758394bec5e0bc80/xarray/core/dataarray.py#L490-L491

But it is acting like a shallow copy:

python In [6]: pf = (da.copy(False)).polyfit("z", deg=2, w=np.arange(nz)) In [7]: da.max(), da.mean() Out[7]: (<xarray.DataArray ()> array(8.92217147), <xarray.DataArray ()> array(2.29014397))