issues

While working on a pull request (#5149) for #5146 I came across an inconsistency in allowed dimension names. If I try and create a DataArray with a non-string dimension, I get a TypeError:

```python console

import xarray as xr da = xr.DataArray(np.ones((5, 5)), dims=[1, "y"]) ... TypeError: dimension 1 is not a string ```

But creating it with a string and renaming it works:

```python console

da = xr.DataArray(np.ones((5, 5)), dims=["x", "y"]).rename(x=1) da <xarray.DataArray (1: 5, y: 5)> array([[1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.]]) Dimensions without coordinates: 1, y ```

I can create a dataset via this renaming, but trying to get the repr value fails as xarray.core.utils.SortedKeysDict tries to sort it and cannot compare the string dimension to the int dimension:

```python console

import xarray as xr ds = xr.Dataset({"test": xr.DataArray(np.ones((5, 5)), dims=["x", "y"]).rename(x=1)}) ds ... ~/software/external/xarray/xarray/core/formatting.py in dataset_repr(ds) 519 520 dims_start = pretty_print("Dimensions:", col_width) --> 521 summary.append("{}({})".format(dims_start, dim_summary(ds))) 522 523 if ds.coords:

~/software/external/xarray/xarray/core/formatting.py in dim_summary(obj) 422 423 def dim_summary(obj): --> 424 elements = [f"{k}: {v}" for k, v in obj.sizes.items()] 425 return ", ".join(elements) 426

~/software/external/xarray/xarray/core/formatting.py in <listcomp>(.0) 422 423 def dim_summary(obj): --> 424 elements = [f"{k}: {v}" for k, v in obj.sizes.items()] 425 return ", ".join(elements) 426

/usr/lib/python3.9/_collections_abc.py in iter(self) 847 848 def iter(self): --> 849 for key in self._mapping: 850 yield (key, self._mapping[key]) 851

~/software/external/xarray/xarray/core/utils.py in iter(self) 437 438 def iter(self) -> Iterator[K]: --> 439 return iter(self.mapping) 440 441 def len(self) -> int:

~/software/external/xarray/xarray/core/utils.py in iter(self) 504 def iter(self) -> Iterator[K]: 505 # see #4571 for the reason of the type ignore --> 506 return iter(sorted(self.mapping)) # type: ignore[type-var] 507 508 def len(self) -> int:

TypeError: '<' not supported between instances of 'str' and 'int' ``` The same thing happens if I call rename on the dataset rather than the array it is initialised with.

If the initialiser requires the dimension names to be strings, and other code (which includes the HTML formatter I was looking at when I found this) assume that they are, then rename and any other method which can alter dimension names should also enforce the string requirement.

Environment:

If I create a chunked lazily-computed array, writing it to disk with to_netcdf() computes and writes it with the threading and distributed schedulers, but not with the multiprocessing scheduler. The only reference I've found when searching for the exception message comes from this StackOverflow question.

MCVE Code Sample

```python import dask import numpy as np import xarray as xr

if name == "main": # Simple worker function. def inner(ds): if sum(ds.dims.values()) == 0: return ds return ds**2

# Some random data to work with.
ds = xr.Dataset(
        {"test": (("a", "b"), np.random.uniform(size=(1000, 1000)))},
        {"a": np.arange(1000), "b": np.arange(1000)}
)

# Chunk it and apply the worker to each chunk.
ds_chunked = ds.chunk({"a": 100, "b": 200})
ds_squared = ds_chunked.map_blocks(inner)

# Thread pool scheduler can compute while writing.
dask.config.set(scheduler="threads")
print("Writing thread pool test to disk.")
ds_squared.to_netcdf("test-threads.nc")

# Local cluster with distributed works too.
c = dask.distributed.Client()
dask.config.set(scheduler=c)
print("Writing local cluster test to disk.")
ds_squared.to_netcdf("test-localcluster.nc")

# Process pool scheduler can compute.
dask.config.set(scheduler="processes")
print("Computing with process pool scheduler.")
ds_squared.compute()

# But it cannot compute while writing.
print("Trying to write process pool test to disk.")
ds_squared.to_netcdf("test-process.nc")

```

Expected Output

Complete netCDF files should be created from all three schedulers.

Problem Description

The thread pool and distributed local cluster schedulers result in a complete output. The process pool scheduler fails when trying to write (note that test-process.nc is created with the header and coordinate information, but no actual data is written). The traceback is:

pytb Traceback (most recent call last): File "bug.py", line 54, in <module> ds_squared.to_netcdf("test-process.nc") File "/usr/lib/python3.8/site-packages/xarray/core/dataset.py", line 1535, in to_netcdf return to_netcdf( File "/usr/lib/python3.8/site-packages/xarray/backends/api.py", line 1097, in to_netcdf writes = writer.sync(compute=compute) File "/usr/lib/python3.8/site-packages/xarray/backends/common.py", line 198, in sync delayed_store = da.store( File "/usr/lib/python3.8/site-packages/dask/array/core.py", line 923, in store result.compute(**kwargs) File "/usr/lib/python3.8/site-packages/dask/base.py", line 165, in compute (result,) = compute(self, traverse=False, **kwargs) File "/usr/lib/python3.8/site-packages/dask/base.py", line 436, in compute results = schedule(dsk, keys, **kwargs) File "/usr/lib/python3.8/site-packages/dask/multiprocessing.py", line 212, in get result = get_async( File "/usr/lib/python3.8/site-packages/dask/local.py", line 494, in get_async fire_task() File "/usr/lib/python3.8/site-packages/dask/local.py", line 460, in fire_task dumps((dsk[key], data)), File "/usr/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 62, in dumps cp.dump(obj) File "/usr/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 538, in dump return Pickler.dump(self, obj) File "/usr/lib/python3.8/multiprocessing/synchronize.py", line 101, in __getstate__ context.assert_spawning(self) File "/usr/lib/python3.8/multiprocessing/context.py", line 363, in assert_spawning raise err RuntimeError: Lock objects should only be shared between processes through inheritance

With a bit of editing of the system multiprocessing module I was able to determine that the lock being reported by this exception was the first lock created. I then added a breakpoint to the Lock constructor to get a traceback of what was creating it:

| File | Line | Function |----------------------|------|------------------------- | core/dataset.py | 1535 | Dataset.to_netcdf | backends/api.py | 1071 | to_netcdf | backends/netCDF4_.py | 350 | open | backends/locks.py | 114 | get_write_lock | backends/locks.py | 39 | _get_multiprocessing_lock

This last function creates the offending multiprocessing.Lock() object. Note that there are six Locks constructed and so its possible that the later-created ones would also cause an issue.

The h5netcdf backend has the same problem with Lock. However the SciPy backend gives a NotImplementedError for this:

python ds_squared.to_netcdf("test-process.nc", engine="scipy")

pytb Traceback (most recent call last): File "bug.py", line 54, in <module> ds_squared.to_netcdf("test-process.nc", engine="scipy") File "/usr/lib/python3.8/site-packages/xarray/core/dataset.py", line 1535, in to_netcdf return to_netcdf( File "/usr/lib/python3.8/site-packages/xarray/backends/api.py", line 1056, in to_netcdf raise NotImplementedError( NotImplementedError: Writing netCDF files with the scipy backend is not currently supported with dask's multiprocessing scheduler

I'm not sure how simple it would be to get this working with the multiprocessing scheduler, or how vital it is given that the distributed scheduler works. If nothing else, it would be good to get the same NotImplementedError as with the SciPy backend.

Output of xr.show_versions()