What happened?

I'm testing my code with numpy 2.0 and current main xarray and dask and ran into a change that I guess is expected given the way xarray does things, but want to make sure as it could be unexpected for many users.

Doing DataArray.where with an integer array less than 64-bits and an integer as the new value will upcast the array to 64-bit integers (python's int). With old versions of numpy this would preserve the dtype of the array. As far as I can tell the relevant xarray code hasn't changed so this seems to be more about numpy making things more consistent.

The main problem seems to come down to:

https://github.com/pydata/xarray/blob/d933578ebdc4105a456bada4864f8ffffd7a2ced/xarray/core/duck_array_ops.py#L218

As this converts my scalar input int to a numpy array. If it didn't do this array conversion then numpy works as expected. See the MCVE for the xarray specific example, but here's the numpy equivalent:

```python import numpy as np

a = np.zeros((2, 2), dtype=np.uint16)

what I'm intending to do with my xarray data_arr.where(cond, 2)

np.where(a != 0, a, 2).dtype

dtype('uint16')

equivalent to what xarray does:

np.where(a != 0, a, np.asarray(2)).dtype

dtype('int64')

workaround, cast my scalar to a specific numpy type

np.where(a != 0, a, np.asarray(np.uint16(2))).dtype

dtype('uint16')

```

From a numpy point of view, the second where call makes sense that 2 arrays should be upcast to the same dtype so they can be combined. But from an xarray user point of view, I'm entering a scalar so I expect it to be the same as the first where call above.

What did you expect to happen?

See above.

Minimal Complete Verifiable Example

```Python import xarray as xr import numpy as np

data_arr = xr.DataArray(np.array([1, 2], dtype=np.uint16)) print(data_arr.where(data_arr == 2, 3).dtype)

int64

```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [X] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

No response

Anything else we need to know?

Numpy 1.x preserves the dtype.

```python In [1]: import numpy as np

In [2]: np.asarray(2).dtype Out[2]: dtype('int64')

In [3]: a = np.zeros((2, 2), dtype=np.uint16)

In [4]: np.where(a != 0, a, np.asarray(2)).dtype Out[4]: dtype('uint16')

In [5]: np.where(a != 0, a, np.asarray(np.uint16(2))).dtype Out[5]: dtype('uint16') ```

Environment

Is your feature request related to a problem?

I'd like mypy to understand that xarray DataArrays passed to numpy ufuncs have a return type of xarray DataArray.

```python import xarray as xr import numpy as np

def compute_relative_azimuth(sat_azi: xr.DataArray, sun_azi: xr.DataArray) -> xr.DataArray: abs_diff = np.absolute(sun_azi - sat_azi) ssadiff = np.minimum(abs_diff, 360 - abs_diff) return ssadiff

```

bash $ mypy ./xarray_mypy.py xarray_mypy.py:7: error: Incompatible return value type (got "ndarray[Any, dtype[Any]]", expected "DataArray") [return-value] Found 1 error in 1 file (checked 1 source file)

Describe the solution you'd like

I'm not sure if this is possible, if it is something xarray can fix, or something numpy needs to "fix". I'd like the above situation to "just work" without anything more than maybe some extra type-stub package.

Describe alternatives you've considered

Cast types or other type coercion or tell mypy to ignore the type issues for these numpy call.

Additional context

https://stackoverflow.com/questions/77369042/typing-when-passing-xarray-dataarray-objects-to-numpy-ufuncs

What happened:

While working on some tests that catch and check for warnings in my library I found that xarray with new versions of Python (I think this is the trigger) causes a ton of DeprecationWarnings on import:

python /home/davidh/miniconda3/envs/satpy_py39/lib/python3.9/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. duck_array_version = LooseVersion(duck_array_module.__version__)

What you expected to happen:

No warnings.

Minimal Complete Verifiable Example:

```python import warnings warnings.simplefilter("always")

import xarray as xr ```

Results in:

/home/davidh/miniconda3/envs/satpy_py39/lib/python3.9/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. duck_array_version = LooseVersion(duck_array_module.__version__) /home/davidh/miniconda3/envs/satpy_py39/lib/python3.9/site-packages/xarray/core/pycompat.py:37: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. duck_array_version = LooseVersion("0.0.0") /home/davidh/miniconda3/envs/satpy_py39/lib/python3.9/site-packages/xarray/core/pycompat.py:37: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. duck_array_version = LooseVersion("0.0.0") /home/davidh/miniconda3/envs/satpy_py39/lib/python3.9/site-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. other = LooseVersion(other) /home/davidh/miniconda3/envs/satpy_py39/lib/python3.9/site-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. other = LooseVersion(other) /home/davidh/miniconda3/envs/satpy_py39/lib/python3.9/site-packages/xarray/core/npcompat.py:82: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. if LooseVersion(np.__version__) >= "1.20.0": /home/davidh/miniconda3/envs/satpy_py39/lib/python3.9/site-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. other = LooseVersion(other) /home/davidh/miniconda3/envs/satpy_py39/lib/python3.9/site-packages/xarray/core/pdcompat.py:45: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. if LooseVersion(pd.__version__) < "0.25.0": /home/davidh/miniconda3/envs/satpy_py39/lib/python3.9/site-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. other = LooseVersion(other)

Anything else we need to know?:

Environment:

I've turned on testing my Satpy package with unstable or pre-releases of some of our dependencies including numpy and xarray. I've found one error so far where in previous versions of xarray it was possible to assign to the numpy array taken from a DataArray.

MCVE Code Sample

```python import numpy as np import dask.array as da import xarray as xr

data = np.arange(15, 301, 15).reshape(2, 10) data_arr = xr.DataArray(data, dims=('y', 'x'), attrs={'test': 'test'}) data_arr = data_arr.copy() data_arr = data_arr.expand_dims('bands') data_arr['bands'] = ['L'] n_arr = np.asarray(data_arr.data) n_arr[n_arr == 45] = 5

```

Which results in:

```

ValueError Traceback (most recent call last) <ipython-input-12-90dae37dd808> in <module> ----> 1 n_arr = np.asarray(data_arr.data); n_arr[n_arr == 45] = 5

ValueError: assignment destination is read-only ```

Expected Output

A writable array. No error.

Problem Description

If this is expected new behavior then so be it, but wanted to check with the xarray devs before I tried to work around it.

Output of xr.show_versions()

What happened?

The Satpy project has a CI environment that installs numpy, pandas, and xarray (and a couple other packages) from their unstable sources (nightly builds, github source, etc). In the last week or two this environment has started failing with various datetime64 issues. It all seems to be caused by some recent change in pandas, but I can't place exactly what the problem is nor the commit/PR that started it. It seems there are a couple datetime related PRs.

What did you expect to happen?

Datetime or datetime64 objects should be allowed to be in whatever units they need to be in (days or minutes or nanoseconds. It seems parts of xarray (or pandas) assume datetime64[ns] but a change in pandas is no longer doing this conversion automatically (from datetime64[X] to datetime64[ns].

Minimal Complete Verifiable Example

You should be able to take any environment with modern xarray and install dev pandas with:

python -m pip install --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple/ --trusted-host pypi.anaconda.org --no-deps --pre --upgrade pandas

Then run this snippet:

```Python import xarray as xr import numpy as np from xarray.coding.times import CFDatetimeCoder

a = xr.DataArray(np.arange(1.0), dims=("time",), coords={"time": [np.datetime64('2018-05-30T10:05:00')]})

CFDatetimeCoder().encode(a.coords["time"]) ```

I haven't been able to generate a higher-level MVCE yet, but I'm hoping this little snippet will make the issue obvious to someone familiar with xarray internals.

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

```

----> 1 CFDatetimeCoder().encode(a.coords["time"])

File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:676, in CFDatetimeCoder.encode(self, variable, name) 672 dims, data, attrs, encoding = unpack_for_encoding(variable) 673 if np.issubdtype(data.dtype, np.datetime64) or contains_cftime_datetimes( 674 variable 675 ): --> 676 (data, units, calendar) = encode_cf_datetime( 677 data, encoding.pop("units", None), encoding.pop("calendar", None) 678 ) 679 safe_setitem(attrs, "units", units, name=name) 680 safe_setitem(attrs, "calendar", calendar, name=name)

File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:612, in encode_cf_datetime(dates, units, calendar) 609 dates = np.asarray(dates) 611 if units is None: --> 612 units = infer_datetime_units(dates) 613 else: 614 units = _cleanup_netcdf_time_units(units)

File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:394, in infer_datetime_units(dates) 392 print("Formatting datetime object") 393 reference_date = dates[0] if len(dates) > 0 else "1970-01-01" --> 394 reference_date = format_cftime_datetime(reference_date) 395 unique_timedeltas = np.unique(np.diff(dates)) 396 units = _infer_time_units_from_diff(unique_timedeltas)

File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:405, in format_cftime_datetime(date) 400 def format_cftime_datetime(date): 401 """Converts a cftime.datetime object to a string with the format: 402 YYYY-MM-DD HH:MM:SS.UUUUUU 403 """ 404 return "{:04d}-{:02d}-{:02d} {:02d}:{:02d}:{:02d}.{:06d}".format( --> 405 date.year, 406 date.month, 407 date.day, 408 date.hour, 409 date.minute, 410 date.second, 411 date.microsecond, 412 )

AttributeError: 'numpy.datetime64' object has no attribute 'year' ```

Anything else we need to know?

No response

Environment

Problem description

This issue is to start the discussion for a feature that would be helpful to a lot of people. It may not necessarily be best to put it in xarray, but let's figure that out. I'll try to describe things below to the best of my knowledge. I'm typically thinking of raster/image data when it comes to this stuff, but it could probably be used for GIS-like point data.

Geographic data can be projected (uniform grid) or unprojected (nonuniform). Unprojected data typically has longitude and latitude values specified per-pixel. I don't think I've ever seen non-uniform data in a projected space. Projected data can be specified by a CRS (PROJ.4), a number of pixels (shape), and extents/bbox in CRS units (xmin, ymin, xmax, ymax). This could also be specified in different ways like origin (X, Y) and pixel size. Seeing as xarray already computes all coords data it makes sense for extents and array shape to be used. With this information provided in an xarray object any library could check for these properties and know where to place the data on a map.

So the question is: Should these properties be standardized in xarray Dataset/DataArray objects and how?

Related libraries and developers

• pyresample (me, @mraspaud, @pnuu)
• verde and gmt-python (@leouieda)
• metpy (@dopplershift)
• geo-xarray (@andrewdhicks)
• rasterio
• cartopy

I know @WeatherGod also showed interest on gitter.

Complications and things to consider

1. Other related coordinate systems like ECEF where coordinates are specified in three dimensions (X, Y, Z). Very useful for calculations like nearest neighbor of lon/lat points or for comparisons between two projected coordinate systems.
2. Specifying what coords arrays are the CRS coordinates or geographic coordinates in general.
3. If xarray should include these properties, where is the line drawn for what functionality xarray supports? Resampling/gridding, etc?
4. How is the CRS object represented? PROJ.4 string, PROJ.4 dict, existing libraries CRS object, new CRS object, pyproj.Proj object?
5. Affine versus geotransforms instead of extents: https://github.com/mapbox/rasterio/blob/master/docs/topics/migrating-to-v1.rst#affineaffine-vs-gdal-style-geotransforms
6. Similar to 4, I never mentioned "rotation" parameters which some users may want and are specified in the affine/geotransform.
7. Dynamically generated extents/affine objects so that slicing operations don't have to be handled specially.
8. Center of pixel coordinates versus outer edge of pixel coordinates.

What happened?

I have a case where a Dataset to be written to a NetCDF file has "ancillary_variables" that have a circular dependence. For example, variable A has .attrs["ancillary_variables"] that contains variable B, and B has .attrs["ancillary_variables"] that contains A.

What did you expect to happen?

Circular dependencies are detected and avoided. No maximum recursion error.

Minimal Complete Verifiable Example

```Python In [1]: import xarray as xr

In [2]: a = xr.DataArray(1.0, attrs={})

In [3]: b = xr.DataArray(2.0, attrs={})

In [4]: a.attrs["other"] = b

In [5]: b.attrs["other"] = a

In [6]: a_copy = a.copy(deep=True)

RecursionError Traceback (most recent call last) Cell In [6], line 1 ----> 1 a_copy = a.copy(deep=True)

File ~/miniconda3/envs/satpy_py310/lib/python3.10/site-packages/xarray/core/dataarray.py:1172, in DataArray.copy(self, deep, data) 1104 def copy(self: T_DataArray, deep: bool = True, data: Any = None) -> T_DataArray: 1105 """Returns a copy of this array. 1106 1107 If deep=True, a deep copy is made of the data array. (...) 1170 pandas.DataFrame.copy 1171 """ -> 1172 variable = self.variable.copy(deep=deep, data=data) 1173 indexes, index_vars = self.xindexes.copy_indexes(deep=deep) 1175 coords = {}

File ~/miniconda3/envs/satpy_py310/lib/python3.10/site-packages/xarray/core/variable.py:996, in Variable.copy(self, deep, data) 989 if self.shape != ndata.shape: 990 raise ValueError( 991 "Data shape {} must match shape of object {}".format( 992 ndata.shape, self.shape 993 ) 994 ) --> 996 attrs = copy.deepcopy(self._attrs) if deep else copy.copy(self._attrs) 997 encoding = copy.deepcopy(self._encoding) if deep else copy.copy(self._encoding) 999 # note: dims is already an immutable tuple

File ~/miniconda3/envs/satpy_py310/lib/python3.10/copy.py:146, in deepcopy(x, memo, _nil) 144 copier = _deepcopy_dispatch.get(cls) 145 if copier is not None: --> 146 y = copier(x, memo) 147 else: 148 if issubclass(cls, type):

File ~/miniconda3/envs/satpy_py310/lib/python3.10/copy.py:231, in _deepcopy_dict(x, memo, deepcopy) 229 memo[id(x)] = y 230 for key, value in x.items(): --> 231 y[deepcopy(key, memo)] = deepcopy(value, memo) 232 return y

File ~/miniconda3/envs/satpy_py310/lib/python3.10/copy.py:153, in deepcopy(x, memo, _nil) 151 copier = getattr(x, "deepcopy", None) 152 if copier is not None: --> 153 y = copier(memo) 154 else: 155 reductor = dispatch_table.get(cls)

File ~/miniconda3/envs/satpy_py310/lib/python3.10/site-packages/xarray/core/dataarray.py:1190, in DataArray.deepcopy(self, memo) 1187 def deepcopy(self: T_DataArray, memo=None) -> T_DataArray: 1188 # memo does nothing but is required for compatibility with 1189 # copy.deepcopy -> 1190 return self.copy(deep=True)

File ~/miniconda3/envs/satpy_py310/lib/python3.10/site-packages/xarray/core/dataarray.py:1172, in DataArray.copy(self, deep, data) 1104 def copy(self: T_DataArray, deep: bool = True, data: Any = None) -> T_DataArray: 1105 """Returns a copy of this array. 1106 1107 If deep=True, a deep copy is made of the data array. (...) 1170 pandas.DataFrame.copy 1171 """ -> 1172 variable = self.variable.copy(deep=deep, data=data) 1173 indexes, index_vars = self.xindexes.copy_indexes(deep=deep) 1175 coords = {}

File ~/miniconda3/envs/satpy_py310/lib/python3.10/site-packages/xarray/core/variable.py:996, in Variable.copy(self, deep, data) 989 if self.shape != ndata.shape: 990 raise ValueError( 991 "Data shape {} must match shape of object {}".format( 992 ndata.shape, self.shape 993 ) 994 ) --> 996 attrs = copy.deepcopy(self._attrs) if deep else copy.copy(self._attrs) 997 encoding = copy.deepcopy(self._encoding) if deep else copy.copy(self._encoding) 999 # note: dims is already an immutable tuple

File ~/miniconda3/envs/satpy_py310/lib/python3.10/copy.py:146, in deepcopy(x, memo, _nil) 144 copier = _deepcopy_dispatch.get(cls) 145 if copier is not None: --> 146 y = copier(x, memo) 147 else: 148 if issubclass(cls, type):

File ~/miniconda3/envs/satpy_py310/lib/python3.10/copy.py:231, in _deepcopy_dict(x, memo, deepcopy) 229 memo[id(x)] = y 230 for key, value in x.items(): --> 231 y[deepcopy(key, memo)] = deepcopy(value, memo) 232 return y

File ~/miniconda3/envs/satpy_py310/lib/python3.10/copy.py:153, in deepcopy(x, memo, _nil) 151 copier = getattr(x, "deepcopy", None) 152 if copier is not None: --> 153 y = copier(memo) 154 else: 155 reductor = dispatch_table.get(cls)

File ~/miniconda3/envs/satpy_py310/lib/python3.10/site-packages/xarray/core/dataarray.py:1190, in DataArray.deepcopy(self, memo) 1187 def deepcopy(self: T_DataArray, memo=None) -> T_DataArray: 1188 # memo does nothing but is required for compatibility with 1189 # copy.deepcopy -> 1190 return self.copy(deep=True)

[... skipping similar frames: DataArray.copy at line 1172 (495 times), DataArray.__deepcopy__ at line 1190 (494 times), _deepcopy_dict at line 231 (494 times), Variable.copy at line 996 (494 times), deepcopy at line 146 (494 times), deepcopy at line 153 (494 times)]

File ~/miniconda3/envs/satpy_py310/lib/python3.10/site-packages/xarray/core/variable.py:996, in Variable.copy(self, deep, data) 989 if self.shape != ndata.shape: 990 raise ValueError( 991 "Data shape {} must match shape of object {}".format( 992 ndata.shape, self.shape 993 ) 994 ) --> 996 attrs = copy.deepcopy(self._attrs) if deep else copy.copy(self._attrs) 997 encoding = copy.deepcopy(self._encoding) if deep else copy.copy(self._encoding) 999 # note: dims is already an immutable tuple

File ~/miniconda3/envs/satpy_py310/lib/python3.10/copy.py:146, in deepcopy(x, memo, _nil) 144 copier = _deepcopy_dispatch.get(cls) 145 if copier is not None: --> 146 y = copier(x, memo) 147 else: 148 if issubclass(cls, type):

File ~/miniconda3/envs/satpy_py310/lib/python3.10/copy.py:231, in _deepcopy_dict(x, memo, deepcopy) 229 memo[id(x)] = y 230 for key, value in x.items(): --> 231 y[deepcopy(key, memo)] = deepcopy(value, memo) 232 return y

File ~/miniconda3/envs/satpy_py310/lib/python3.10/copy.py:153, in deepcopy(x, memo, _nil) 151 copier = getattr(x, "deepcopy", None) 152 if copier is not None: --> 153 y = copier(memo) 154 else: 155 reductor = dispatch_table.get(cls)

File ~/miniconda3/envs/satpy_py310/lib/python3.10/site-packages/xarray/core/dataarray.py:1190, in DataArray.deepcopy(self, memo) 1187 def deepcopy(self: T_DataArray, memo=None) -> T_DataArray: 1188 # memo does nothing but is required for compatibility with 1189 # copy.deepcopy -> 1190 return self.copy(deep=True)

File ~/miniconda3/envs/satpy_py310/lib/python3.10/site-packages/xarray/core/dataarray.py:1172, in DataArray.copy(self, deep, data) 1104 def copy(self: T_DataArray, deep: bool = True, data: Any = None) -> T_DataArray: 1105 """Returns a copy of this array. 1106 1107 If deep=True, a deep copy is made of the data array. (...) 1170 pandas.DataFrame.copy 1171 """ -> 1172 variable = self.variable.copy(deep=deep, data=data) 1173 indexes, index_vars = self.xindexes.copy_indexes(deep=deep) 1175 coords = {}

File ~/miniconda3/envs/satpy_py310/lib/python3.10/site-packages/xarray/core/variable.py:985, in Variable.copy(self, deep, data) 982 ndata = indexing.MemoryCachedArray(ndata.array) 984 if deep: --> 985 ndata = copy.deepcopy(ndata) 987 else: 988 ndata = as_compatible_data(data)

File ~/miniconda3/envs/satpy_py310/lib/python3.10/copy.py:137, in deepcopy(x, memo, _nil) 134 if memo is None: 135 memo = {} --> 137 d = id(x) 138 y = memo.get(d, _nil) 139 if y is not _nil:

RecursionError: maximum recursion depth exceeded while calling a Python object ```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

I have at least one other issue related to the new xarray release but I'm still tracking it down. I think it is also related to the deep copy behavior change which was merged a day before the release so our CI didn't have time to test the "unstable" version of xarray.

Environment

What happened:

I have a couple CI environments that use unstable versions of xarray and dask among other libraries. A couple tests have been failing in different but similar ways. I was able to get one of them down to the below example. This happens with the dask master branch on either the xarray latest release or xarray master. Here are the commits I was using when doing unstable both.

Dask: ad01acc14088d03127dfec4f881cce959f3ac4d9 Xarray: eb7e112d45a9edebd8e5fb4f873e3e6adb18824a

I don't see this with dask's latest release and this is likely caused by a change in dask, but after seeing PRs like #4884 I'm wondering if this is something similar and requires a change in xarray.

The bottom line is that doing various things like my_data_arr.data.compute() or dask.compute(my_data_arr.data) produces:

ImplicitToExplicitIndexingAdapter(array=CopyOnWriteArray(array=LazilyOuterIndexedArray(array=<xarray.backends.rasterio_.RasterioArrayWrapper object at 0x7f257b382150>, key=BasicIndexer((slice(0, 2, 1), slice(0, 100, 1), slice(0, 100, 1))))))

What you expected to happen:

I would expect to get a numpy array back when computing the underlying dask array.

Minimal Complete Verifiable Example:

```python from PIL import Image import xarray as xr import numpy as np

create a test image

Image.fromarray(np.zeros((5, 5, 3), dtype=np.uint8)).save('test.png') r = xr.open_rasterio('test.png', chunks='auto') print(r.data.compute())

ImplicitToExplicitIndexingAdapter(array=CopyOnWriteArray(array=LazilyOuterIndexedArray(array=<xarray.backends.rasterio_.RasterioArrayWrapper object at 0x7f25e72bcc10>, key=BasicIndexer((slice(0, 2, 1), slice(0, 100, 1), slice(0, 100, 1))))))

```

Anything else we need to know?:

As mentioned above other weird things are happening when array wrappers seem to be involved but I haven't been able to make a small example of them.

Environment:

Code Sample, a copy-pastable example if possible

I'm working on a proof-of-concept for the geoxarray project where I'd like to store coordinate reference system (CRS) information in the coordinates of a DataArray or Dataset object. I'd like to avoid subclassing objects and instead depend completely on xarray accessors to implement any utilities I need.

I'm having trouble deciding what the best place is for this CRS information so that it benefits the user; .coords made the most sense. My hope was that adding two DataArrays together with two different crs coordinates would cause an error, but found out that since crs is not a dimension it doesn't get treated the same way; even when changing join method to 'exact'.

```python from pyproj import CRS import xarray as xr import dask.array as da

crs1 = CRS.from_string('+proj=lcc +datum=WGS84 +lon_0=-95 +lat_0=25 +lat_1=25') crs2 = CRS.from_string('+proj=lcc +datum=WGS84 +lon_0=-95 +lat_0=35 +lat_1=35')

a = xr.DataArray(da.zeros((5, 5), chunks=2), dims=('y', 'x'), coords={'y': da.arange(1, 6, chunks=3), 'x': da.arange(2, 7, chunks=3), 'crs': crs1, 'test': 1, 'test2': 2})

b = xr.DataArray(da.zeros((5, 5), chunks=2), dims=('y', 'x'), coords={'y': da.arange(1, 6, chunks=3), 'x': da.arange(2, 7, chunks=3), 'crs': crs2, 'test': 2, 'test2': 2})

a + b

Results in:

<xarray.DataArray 'zeros-e5723e7f9121b7ac546f61c19dabe786' (y: 5, x: 5)>

dask.array<shape=(5, 5), dtype=float64, chunksize=(2, 2)>

Coordinates:

* y (y) int64 1 2 3 4 5

* x (x) int64 2 3 4 5 6

test2 int64 2

```

In the above code I was hoping that because the crs coordinates are different (lat_0 and lat_1 are different and crs1 != crs2) that I could get it to raise an exception.

Any ideas for how I might be able to accomplish something like this? I'm not an expert on xarray/pandas indexes, but could this be another possible solution?

Edit: xr.merge with compat='no_conflicts' does detect this difference.

I have projects that use versioneer after finding out that xarray and dask used it. However, it seems the project is no longer actively maintained with the last commit happening almost 2 years ago:

https://github.com/warner/python-versioneer

There are two issues I've found with versioneer:

1. Wheels do not have a valid version number. They get 0+unknown unless an sdist is built first to populate the egg info. Building sdist is not something that is always done by CI environments like travis. This also happens if someone installs the package from github pip install git+https://github.com/pydata/xarray.git. At least these are my findings with my satpy library.
2. versioneer.py has some style issues: versioneer.py:941:-13592: W605 invalid escape sequence '\s' versioneer.py:941:-13408: W605 invalid escape sequence '\s' versioneer.py:941:-13228: W605 invalid escape sequence '\s' versioneer.py:941:-11196: W605 invalid escape sequence '\d' versioneer.py:941:-8162: W605 invalid escape sequence '\d' This was worked around in xarray by adding a noqa to the top of the module.

So my question is, do the xarray devs (or dask devs) see this as a problem? What are your plans for the future? Have you enjoyed using versioneer?

I guess I should probably CC @warner on this too in case they are still active on github.

Code Sample, a copy-pastable example if possible

```python In [1]: import dask.array as da; import xarray as xr; import numpy as np

In [2]: a = xr.DataArray(da.random.random((4, 6), chunks=2), attrs={'test': ['x1', 'y1']}, dims=('y', 'x'))

In [3]: b = xr.DataArray(da.random.random((4, 6), chunks=2), attrs={'test': ['x2', 'y2']}, dims=('y', 'x'))

In [4]: xr.concat([a, b], 'y')

ValueError Traceback (most recent call last) <ipython-input-4-c8b32db4cfb7> in <module>() ----> 1 xr.concat([a, b], 'y')

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/combine.py in concat(objs, dim, data_vars, coords, compat, positions, indexers, mode, concat_over) 119 raise TypeError('can only concatenate xarray Dataset and DataArray ' 120 'objects, got %s' % type(first_obj)) --> 121 return f(objs, dim, data_vars, coords, compat, positions) 122 123

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/combine.py in _dataarray_concat(arrays, dim, data_vars, coords, compat, positions) 337 338 ds = _dataset_concat(datasets, dim, data_vars, coords, compat, --> 339 positions) 340 return arrays[0]._from_temp_dataset(ds, name) 341

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/combine.py in _dataset_concat(datasets, dim, data_vars, coords, compat, positions) 303 if k in concat_over: 304 vars = ensure_common_dims([ds.variables[k] for ds in datasets]) --> 305 combined = concat_vars(vars, dim, positions) 306 insert_result_variable(k, combined) 307

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/variable.py in concat(variables, dim, positions, shortcut) 1772 return IndexVariable.concat(variables, dim, positions, shortcut) 1773 else: -> 1774 return Variable.concat(variables, dim, positions, shortcut) 1775 1776

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/variable.py in concat(cls, variables, dim, positions, shortcut) 1299 if var.dims != first_var.dims: 1300 raise ValueError('inconsistent dimensions') -> 1301 utils.remove_incompatible_items(attrs, var.attrs) 1302 1303 return cls(dims, data, attrs, encoding)

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/utils.py in remove_incompatible_items(first_dict, second_dict, compat) 157 if (k not in second_dict or 158 (k in second_dict and --> 159 not compat(first_dict[k], second_dict[k]))): 160 del first_dict[k] 161

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/utils.py in equivalent(first, second) 106 return ((first is second) or 107 (first == second) or --> 108 (pd.isnull(first) and pd.isnull(second))) 109 110

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() ```

Problem description

If two or more DataArrays are concatentated and they have list attributes that are not equal an exception is raised about arrays not being truth values.

Expected Output

I guess the expected result would be that the list attribute is not included in the resulting DataArray's attributes.

Output of xr.show_versions()

MCVE Code Sample

```python from dask.diagnostics import ProgressBar import xarray as xr import numpy as np import dask.array as da

a = xr.DataArray(da.zeros((10, 10), chunks=2), dims=('y', 'x'), coords={'y': np.arange(10), 'x': np.arange(10), 'lons': (('y', 'x'), da.zeros((10, 10), chunks=2))}) b = xr.DataArray(da.zeros((10, 10), chunks=2), dims=('y', 'x'), coords={'y': np.arange(10), 'x': np.arange(10), 'lons': (('y', 'x'), da.zeros((10, 10), chunks=2))})

with ProgressBar(): c = a + b

```

Output:

[########################################] | 100% Completed | 0.1s

Problem Description

Using arrays with 2D dask array coordinates results in the coordinates being computed for any binary operations (anything combining two or more DataArrays). I use ProgressBar in the above example to show when coordinates are being computed.

In my own work, when I learned that 2D dask coordinates were possible, I started adding longitude and latitude coordinates. These are rather large and can take a while to load/compute so I was surprised that simple operations (ex. a.fillna(b)) were causing things to be computed and taking a long time.

Is this computation by design or a possible bug?

Expected Output

No output from the ProgressBar, hoping that no coordinates would be computed/loaded.

Output of xr.show_versions()

MCVE Code Sample

1. Create test_accessor.py in current directory with:

```python import xarray as xr

@xr.register_dataarray_accessor('test') class TestDataArrayAccessor(object): def init(self, obj): self._obj = obj print("DataArray accessor created")

@xr.register_dataset_accessor('test') class TestDatasetAccessor(object): def init(self, obj): self._obj = obj print("Dataset accessor created") ```

1. Run the following code:

```python import xarray as xr import numpy as np import test_accesor

ds = xr.Dataset({'a': xr.DataArray(np.array([1, 2, 3])), 'b': xr.DataArray(np.array([4, 5, 6]))}) ds.test

Dataset accessor created <accessor created on first access - expected>

ds.test

<no output - expected>

ds['a'].test

DataArray accessor created <accessor created on first access - expected>

ds['a'].test

DataArray accessor created <accessor created on second access - unexpected>

var = ds['a'] var.test

DataArray accessor created <accessor created on direct instance access - expected/unexpected>

var.test

<no output - expected>

```

Expected Output

Based on the xarray accessor documentation I would have assumed that the accessor would stick around on the same DataArray object for the life of the data. My guess is that Dataset.__getitem__ is recreating the DataArray every time from the underlying Variable object which means ds['a'] is not ds['a']?

Problem Description

I'm currently working on an accessor for a new package called geoxarray to address the issues talked about in #2288. One of the cases I'm trying to handle is a NetCDF file with CR standard grid_mapping variables. This means that the easiest way to set a CRS object for all variables in a Dataset is to have the Dataset accessor use the accessor of every DataArray (to cache a _crs property). However, with the way things are working doing something like the below won't work:

python ds.geo.apply_grid_mapping() ds['Rad'].geo.crs # this is current `None` because the DataArray was recreated

Output of xr.show_versions()

Code Sample, a copy-pastable example if possible

I'm currently trying to figure out why some coordinates of a netcdf file are included in the resulting Dataset object and why some are not. I've tried looking at the open_dataset source code and think I'm kind of stuck.

To reproduce:

Download GOES-16 ABI L1b NetCDF file from (click the link to have google redirect you to the actual file):

https://storage.cloud.google.com/gcp-public-data-goes-16/ABI-L1b-RadC/2019/140/18/OR_ABI-L1b-RadC-M6C01_G16_s20191401801336_e20191401804109_c20191401804156.nc?_ga=2.44800740.-1513329882.1547344783

python import xarray as xr a = xr.open_dataset('OR_ABI-L1b-RadC-M6C01_G16_s20191401801336_e20191401804109_c20191401804156.nc') print(a.coords)

Results in:

In [5]: a['Rad'].coords Out[5]: Coordinates: t datetime64[ns] ... * y (y) float32 0.151844 0.151788 0.151732 ... -0.151788 -0.151844 * x (x) float32 -0.151844 -0.151788 -0.151732 ... 0.151788 0.151844 y_image float32 ... x_image float32 ...

Even though ncdump -h <file>.nc shows:

variables: short Rad(y, x) ; ... Rad:coordinates = "band_id band_wavelength t y x" ; ...

and:

a.coords

shows:

Coordinates: t datetime64[ns] ... * y (y) float32 0.151844 0.151788 ... -0.151844 * x (x) float32 -0.151844 -0.151788 ... 0.151844 y_image float32 ... x_image float32 ... band_id (band) int8 ... band_wavelength (band) float32 ... t_star_look (num_star_looks) datetime64[ns] ... band_wavelength_star_look (num_star_looks) float32 ...

Problem description

I would have expected the 'Rad' variable/DataArray to include the band_id and band_wavelength coordinate variables but not the x_image and y_image variables since they are not listed in the NetCDF variables coordinates attribute.

Can someone summarize the rules on how xarray came up with these coordinates? Or is this a bug?

Output of xr.show_versions()

Code Sample, a copy-pastable example if possible

```python import xarray as xr import dask.array as da from dask.diagnostics import ProgressBar a = xr.DataArray(da.zeros((3, 5, 5), chunks=(1, 2, 2)), dims=('bands', 'y', 'x'))

with ProgressBar(): b = a.transpose('y', 'x', 'bands')

dask does not show a progress bar due to no computation

with ProgressBar(): b = a.transpose('y', 'x', 'bands') b.compute()

dask computes the array (since we told it to) and we see a progress bar

```

Question

The documentation for transpose says that it is not lazy. Is this only in certain situations? By not lazy does it mean that when the data is computed that the transpose task will require all data to be loaded at once (one large chunk) or does it mean that the transpose operation will immediately compute the transposed array? My test above does not seem to compute the data when transpose is called.

Or is the documentation just outdated?