issues

This PR makes the typing of reduction methods (count, all, any, max, min, mean, prod, sum, std, var, median) for Dataset and DataArray consistent with the typing of most other methods of Dataset and DataArray after the great recent improvements by @headtr1ck in e.g. #6661.

• [x] Closes #4054
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

This draft PR is for soliciting feedback on an approach to adding typing to unary and binary operators for xarray classes. The approach involves generating a stub file using a simple script.

In order to untangle the various helper classes implementing arithmetic throughout the code base I have refactored the code, thereby getting rid of the injection of the operators after class definition. It now inserts the operators during construction of the subclasses leveraging Python's __init_subclass__ mechanism.

For this example python file:

the status quo gives this mypy output:

With this PR the output now becomes:

• [x] Passes isort . && black . && mypy . && flake8

Small patch to improve typing of OrderedSet: allowing more general input type in its constructor and .update method (Iterator instead of AbstractSet) helps to get rid of # type: ignore.

Side-effect: some performance gain in micro-benchmark (before and after): %timeit xr.core.utils.OrderedSet(range(10)) 3.46 µs ± 5.37 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each) 2.06 µs ± 6.39 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

• [x] Closes #4741
• [x] Passes isort . && black . && mypy . && flake8

• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

• make _attr_sources and _item_sources lazy

• make lookup of virtual dimension coordinates lazy
• make helper class more general so it can be used for lookup of both virtual dimension and level coordinates

I appreciate xarray's ability to use attribute style access ds.foo as an alternative to ds["foo"] as it requires less characters/keystrokes and has less 'visual clutter'.

A drawback is that it can be much slower as lookup time seems to display O(n) behaviour instead of O(1) with n being the number of variables in the dataset. For e.g. n=100 it is approximately 100 times slower than dictionary-style access:

# Dataset with many (100) variables
ds = xr.Dataset({f'var{v}': [] for v in range(100)})

%timeit ds['var0']
462 µs ± 1.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit ds.var0
47.1 ms ± 205 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

dir() and _ipython_key_completions_() which are used for e.g. tab completion in iPython are equally slow:

%timeit dir(ds)
47 ms ± 163 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

%timeit ds._ipython_key_completions_()
46.8 ms ± 210 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

I would like to see xarray having much better performance for attribute style access.

• [x] Closes #4644
• [x] Tests added
• [x] Passes isort . && black . && mypy . && flake8
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

What happened: I upgraded from xarray 0.15.1 to 0.16.2 and the astype method seems to have lost the order parameter.

```python In [1]: import xarray as xr

In [2]: xr.version Out[2]: '0.16.2'

In [3]: xr.DataArray([[1.0, 2.0], [3.0, 4.0]]).astype(dtype='d', order='F').values.strides

TypeError Traceback (most recent call last) <ipython-input-3-208ab49008ef> in <module> ----> 1 xr.DataArray([[1.0, 2.0], [3.0, 4.0]]).astype(dtype='d', order='F').values.strides

TypeError: astype() got an unexpected keyword argument 'order' ```

What you expected to happen: I was expecting to get the same result as with xarray 0.15.1:

```python In [1]: import xarray as xr

In [2]: xr.version Out[2]: '0.15.1'

In [3]: xr.DataArray([[1.0, 2.0], [3.0, 4.0]]).astype(dtype='d', order='F').values.strides Out[3]: (8, 16) ```

Anything else we need to know?: Looking at the documentation it seems it disappeared between 0.16.0 and 0.16.1. The documentation at http://xarray.pydata.org/en/v0.16.0/generated/xarray.DataArray.astype.html still has this snippet

order ({'C', 'F', 'A', 'K'}, optional) – Controls the memory layout order of the result. ‘C’ means C order, ‘F’ means Fortran order, ‘A’ means ‘F’ order if all the arrays are Fortran contiguous, ‘C’ order otherwise, and ‘K’ means as close to the order the array elements appear in memory as possible. Default is ‘K’.

(which was identical to the documentation from numpy.ndarray.astype at https://numpy.org/doc/stable/reference/generated/numpy.ndarray.astype.html)

while http://xarray.pydata.org/en/v0.16.1/generated/xarray.DataArray.astype.html seems to lack that part.