Yes, let's merge.

should we merge this now and sort the typing out in a new PR?

that's fine with me, but that would be a new (hard) dependency for python>=3.8 so I think we should make that a new PR

I would vote to have typing_extensions as a dependency if it's helpful. At one point xarray-beam couldn't use the very latest version, but as long as we're OK with a version ~6 months old then I think we should be fine having it. Agree with @Illviljan re workarounds not being fun!

I feel having to do these kinds of workarounds is not nice and sucks out any willpower of improving typing

Well, I can understand that: it's usually the same with other actively developed libraries.

My point was actually that the changes I applied shouldn't have changed the dependencies we currently have: typing_extensions was restricted to python < 3.8 before, so it was only installed on python=3.7 (unless it was installed using conda / from conda-forge, which doesn't support this kind of restriction).

I think it should stay required until typing has matured more, there's too much good stuff coming out each release not to make use of it immediately.

I feel having to do these kinds of workarounds is not nice and sucks out any willpower of improving typing: https://github.com/pydata/xarray/blob/e056cacdca55cc9d9118c830ca622ea965ebcdef/xarray/core/utils.py#L309-L336 (This Try Except is not necessary anymore if it stays a required dependency)

We also don't run mypy with minimum requirements so it's very likely we miss typing errors while going through these convoluted workaround paths.

Here's a few more examples from eariler PRs:

5678 - The necessary workaround if it's not required:

https://github.com/Illviljan/xarray/blob/c11aac2f00edf3584d1eec508fc850fb8f17e084/xarray/core/options.py#L4-L70+

5871 - Just a simple if check if it's required:

https://github.com/dcherian/xarray/blob/59f8586a747b1f0a7b5e42e0ee186a32a3a36819/xarray/core/_reductions.py#L10-L13

we're already using TypeGuard in xarray.core.utils, so I guess it has to stay as a optional dependency (i.e. I will add it back to py38-min-all-deps). I don't think we still need the hard dependency, though.

I think typing_extensions should remain available so we can immediately use the latest and greatest typing functions. For example TypeGuard https://www.python.org/dev/peps/pep-0647/ is a good one that I think we can make use of now. I've also been testing it in #5667.

what do we do about the typing_extensions and importlib-resources dependencies? Those are py37-only at the moment... I guess we don't need importlib-resources anymore, but what about typing_extensions? Is it a optional dependency?

Also, there's a mention of python=3.7 in dask_array_ops related to the method parameter of cumreduction. What is that about?

Edit: otherwise, this looks good to me

Unit Test Results

6 files           6 suites   51m 33s :stopwatch: 16 290 tests 14 565 :heavy_check_mark: 1 725 :zzz: 0 :x: 90 936 runs  82 752 :heavy_check_mark: 8 184 :zzz: 0 :x:

Results for commit 37eb7bda.

:recycle: This comment has been updated with latest results.

Hello @Illviljan! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers:

Comment last updated at 2021-11-06 19:40:44 UTC

Thanks, @mathause it seems indeed setuptools was the culprit, even for the weird dask errors as well. Now the tests are passing!

There is no setuptools=40.4 for python 3.8. The lowest version available for py38 is 41.6.0 (see below). conda/ mamba has just a very confusing error message that is the wrong way round. I think it starts with the newest build that satisfies the version and then goes back until it arrives at the py27 version which it cannot install (which it then reports)...

``` conda search conda-forge::setuptools=40.4.3 Loading channels: done

Name Version Build Channel

setuptools 40.4.3 py27_0 conda-forge
setuptools 40.4.3 py35_0 conda-forge
setuptools 40.4.3 py36_0 conda-forge
setuptools 40.4.3 py37_0 conda-forge ```

Oldest available version for python 3.8: ``` conda search conda-forge::setuptools[build=py38*] Loading channels: done

Name Version Build Channel

setuptools 41.6.0 py38_1 conda-forge
setuptools 42.0.0 py38_0 conda-forge
... setuptools 58.2.0 py38h578d9bd_0 conda-forge ```

I don't get these, we haven't supported 2.7 or 3.6 for a while now why doesn't this happen on main then?

ubuntu-latest py38-bare-minimum ``` Run mamba env update -f $CONDA_ENV_FILE mamba env update -f $CONDA_ENV_FILE shell: /usr/bin/bash -l {0} env: CONDA_ENV_FILE: ci/requirements/py38-bare-minimum.yml CONDA_PKGS_DIR: /home/runner/conda_pkgs_dir conda-forge/linux-64 Using cache conda-forge/noarch Using cache

Looking for: ['python=3.8', 'coveralls', 'pip', 'pytest', 'pytest-cov', 'pytest-env', 'pytest-xdist', 'numpy=1.17', 'pandas=1.0', 'setuptools=40.4']

Encountered problems while solving: - package setuptools-40.4.0-py27_0 requires python >=2.7,<2.8.0a0, but none of the providers can be installed

Error: Process completed with exit code 1. ```

ubuntu-latest py38-min-all-deps ``` Run mamba env update -f $CONDA_ENV_FILE mamba env update -f $CONDA_ENV_FILE shell: /usr/bin/bash -l {0} env: CONDA_ENV_FILE: ci/requirements/py38-min-all-deps.yml CONDA_PKGS_DIR: /home/runner/conda_pkgs_dir conda-forge/linux-64 Using cache conda-forge/noarch Using cache

Looking for: ['python=3.8', 'boto3=1.13', 'bottleneck=1.3', 'cartopy=0.17', 'cdms2=3.1', 'cfgrib=0.9', 'cftime=1.1', 'coveralls', 'dask=2.24', 'distributed=2.24', 'h5netcdf=0.8', 'h5py=2.10', 'hdf5=1.10', 'hypothesis', 'iris=2.4', 'lxml=4.5', 'matplotlib-base=3.2', 'nc-time-axis=1.2', 'netcdf4=1.5.3', 'numba=0.49', 'numpy=1.17', 'pandas=1.0', 'pint=0.15', 'pip', 'pseudonetcdf=3.1', 'pydap=3.2', 'pynio=1.5', 'pytest', 'pytest-cov', 'pytest-env', 'pytest-xdist', 'rasterio=1.1', 'scipy=1.4', 'seaborn=0.10', 'setuptools=40.4', 'sparse=0.8', 'toolz=0.10', 'zarr=2.4']

Encountered problems while solving: - package distributed-2.24.0-py36h9f0ad1d_0 requires python_abi 3.6. _cp36m, but none of the providers can be installed ```