• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

This PR improves the typing of align. Before: the type of the inputs was reduced to the common superclass and the return type was the same. This often required casts or ignores when mixing classes (e.g. da, ds = xr.align(da, ds). Now: the return types are exactly the same as the input types if the number of passed arguments is <=5.

Only downside: it requires some ugly overloads with type ignores on align. Maybe someone knows how to type this better?

What is your issue?

The doctest is currently failing with

E UserWarning: h5py is running against HDF5 1.14.3 when it was built against 1.14.2, this may cause problems

• [x] Closes #8448
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

I guess we update manually for now?

• xref https://github.com/pydata/xarray/issues/8238

This is highly WIP and probably everything is broken right now... Just creating this now, so other people don't work on the same :) Feel free to continue here with me.

@pydata/xarray 1. what do we do with commonly used functions, is it ok to copy them? 2. Moving the typed ops requires a lot of functions to be added to NamedArray, is there a consensus of what we want to move? Is it basically everything? 3. Slowly the utils module is becomming a graveyard of stuff we dont want to put elsewhere, maybe we should at least move the typing stuff over to a types module.

What is your issue?

According to the array api standard, the shape property returns tuple[int | None, ...]. Currently we only support tuple[int, ...]

This will actually raise some errors if a duckarray actually returns some None. E.g. NamedArray.size will fail.

(On a side note: dask arrays actually use NaN instead of None for some reason.... Only advantage of this is that NamedArray.size will actually also return NaN instead of raising...)

Mainly deprecated "closed" argument in date_range and passing pd.MultiIndex directly to the constructor.

What is your issue?

Xarray raises a FutureWarning in its date_range, also observable in your tests. The precise warning is:

xarray/coding/cftime_offsets.py:1130: FutureWarning: Argument closed is deprecated in favor of inclusive.

You should discuss if you will adapt the new inclusive argument or add a workaround.

• [x] Closes #7312
• [ ] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~New functions/methods are listed in api.rst~

Turns out the default value of arguments is important for overloads, haha.

This PR adds the mypy option "warn_unused_ignores" which will raise an error if a # type: ignore is used where it is no longer necessary.

Should enable us to keep our types updated.

I am not sure if this will lead to many issues whenever e.g. numpy changes/improves their typing, so we might get errors whenever there is a new version. Maybe it is not that bad, or maybe we can also remove the option again and only do it manually from time to time?

• [x] Closes #7158

Technically does not close all fails, but if we close it, the CI will open a new issue anyway and the discussion is not relevant anymore :)

Not sure if that is the correct approach (to simply use a mypy flag) but lets see what people think about it.

tiny PR that disables coverage on types.py and all typing-only imports a.la. if TYPE_CHECKING: ...

• [x] Closes #6875
• [x] ~~Tests added~~
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~~New functions/methods are listed in api.rst~~

• [x] Closes #6931
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~~New functions/methods are listed in api.rst~~

Some typing on the way :)

This PR moves all DataArrayRolling and DatasetRolling tests to their own module.

See request https://github.com/pydata/xarray/pull/6744#issuecomment-1182169308

Is your feature request related to a problem?

Coarsen reductions are currently added dynamically which is not very useful for typing. This is a follow-up to @Illviljan in https://github.com/pydata/xarray/pull/6702#discussion_r900700532_

Same goes for Weighted. And similar for Rolling (not sure if it is exactly the same though?)

Describe the solution you'd like

Extend the generate_reductions script to include DataArrayCoarsen and DatasetCoarsen. Once finished: use type checking in all test_coarsen tests.

This is initial try to get type hints for str and dt accessors.

I think there is no way of accessing the class at class scope (or is there?), so I had to use plain "DataArray" as the generic type of the accessors. I think that is acceptable for now.

The hack of DatetimeAccessor vs TimedeltaAccessor in the CombinedDatetimelikeAccessor.__new__ is something static typing can handle, so at type-checking time all properties of both accessors are available. If someone has a better idea?

Maybe a common interface class for accessors would be also beneficial?

This PR adds a mypy badge to the README.

Also, nicer alt texts for all other badges.

What happened?

I tried to assign values to a dataset with a selector-dict where a variable is missing the dim from the selector-dict. This raises a ValueError.

What did you expect to happen?

I expect that assigning works the same as selecting and it will ignore the missing dims.

Minimal Complete Verifiable Example

```Python import xarray as xr

ds = xr.Dataset({"a": ("x", [1, 2, 3]), "b": ("y", [4, 5])})

ds[{"x": 1}]

this works and returns:

<xarray.Dataset>

Dimensions: (y: 2)

Dimensions without coordinates: y

Data variables:

a int64 2

b (y) int64 4 5

ds[{"x": 1}] = 1

this fails and raises a ValueError

ValueError: Variable 'b': indexer {'x': 1} not available

```

Relevant log output

```Python Traceback (most recent call last): File "xarray/core/dataset.py", line 1591, in _setitem_check var_k = var[key] File "xarray/core/dataarray.py", line 740, in getitem return self.isel(indexers=self._item_key_to_dict(key)) File "xarray/core/dataarray.py", line 1204, in isel variable = self._variable.isel(indexers, missing_dims=missing_dims) File "xarray/core/variable.py", line 1181, in isel indexers = drop_dims_from_indexers(indexers, self.dims, missing_dims) File "xarray/core/utils.py", line 834, in drop_dims_from_indexers raise ValueError( ValueError: Dimensions {'x'} do not exist. Expected one or more of ('y',)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "<stdin>", line 1, in <module> File "xarray/core/dataset.py", line 1521, in setitem value = self._setitem_check(key, value) File "xarray/core/dataset.py", line 1593, in _setitem_check raise ValueError( ValueError: Variable 'b': indexer {'x': 1} not available ```

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS

commit: None python: 3.9.1 (default, Jan 13 2021, 15:21:08) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4

xarray: 0.21.1 pandas: 1.4.0 numpy: 1.21.5 scipy: 1.7.3 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.5.1.1 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 49.2.1 pip: 22.0.3 conda: None pytest: 6.2.5 IPython: 8.0.0 sphinx: None

• [x] Closes #6600
• [x] Tests added

• [x] Closes #6597
• [x] Tests added

• [x] Closes #6554
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Additionally I have added new literal types for error handling (Only applied to functions related to isel such that mypy stops complaining).

What happened?

When grouping and reducing an array or dataset over a multi-index the coordinates that make up the multi-index get renamed to "{name_of_multiindex}_level_{i}".

It only works correctly when the Multiindex is a "homogenous grid", i.e. as obtained by stacking.

What did you expect to happen?

I expect that all coordinates keep their initial names.

Minimal Complete Verifiable Example

```Python import xarray as xr

this works:

d = xr.DataArray(range(4), dims="t", coords={"x": ("t", [0, 0, 1, 1]), "y": ("t", [0, 1, 0, 1])}) dd = d.set_index({"t": ["x", "y"]})

returns

<xarray.DataArray (t: 4)>

array([0, 1, 2, 3])

Coordinates:

* t (t) MultiIndex

- x (t) int64 0 0 1 1

- y (t) int64 0 1 0 1

dd.groupby("t").mean(...)

returns

<xarray.DataArray (t: 4)>

array([0., 1., 2., 3.])

Coordinates:

* t (t) MultiIndex

- x (t) int64 0 0 1 1

- y (t) int64 0 1 0 1

this does not work

d2 = xr.DataArray(range(6), dims="t", coords={"x": ("t", [0, 0, 1, 1, 0, 1]), "y": ("t", [0, 1, 0, 1, 0, 0])}) dd2 = d2.set_index({"t": ["x", "y"]})

returns

<xarray.DataArray (t: 6)>

array([0, 1, 2, 3, 4, 5])

Coordinates:

* t (t) MultiIndex

- x (t) int64 0 0 1 1 0 1

- y (t) int64 0 1 0 1 0 0

dd2.groupby("t").mean(...)

returns

<xarray.DataArray (t: 4)>

array([2. , 1. , 3.5, 3. ])

Coordinates:

* t (t) MultiIndex

- t_level_0 (t) int64 0 0 1 1

- t_level_1 (t) int64 0 1 0 1

```

Relevant log output

No response

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS

commit: None python: 3.9.1 (default, Jan 13 2021, 15:21:08) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4

xarray: 0.21.1 pandas: 1.4.0 numpy: 1.21.5 scipy: 1.7.3 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.5.1.1 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 49.2.1 pip: 22.0.3 conda: None pytest: 6.2.5 IPython: 8.0.0 sphinx: None