issue_comments

My question is whether we can make xarray by default pull the equivalent of xarray[tutorial], and a separate xarray[minimal] for anyone who really wants to avoid deps...

I don't think this is possible currently with pip: https://github.com/pypa/setuptools/issues/1139

I think it can be done with conda, but typically by defining two separate packages, e.g., dask-core and dask. The seems a little annoying to setup and possibly surprising for users who expect names to match pip, though.

Is there a way of having pooch as a default but not required dependency? I'm not sure how python works with this — can we specify a minimal / required dependency set, but have pooch installed by default?

pip install xarray is always the minimal set of dependencies, but we can make something like pip install xarray[tutorial] work. Actually, pooch should be (but currently isn't) included in xarray[io]: https://github.com/pydata/xarray/blob/9e84d093102c9d7b98fd03d25db0636ff4aaf073/setup.cfg#L82

I've added a temporary work-around to provide better errors for the tutorial datasets into #5300, i.e., solution (3) from my previouso comment.

This is challenging to unit-test, so I'll just share my IPython session(s) from a new environment to show how it works: ``` (xarray-min) ~/dev/xarray better-no-plugins-errors $ ipython Python 3.9.4 (default, Apr 9 2021, 09:32:38) Type 'copyright', 'credits' or 'license' for more information IPython 7.22.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import xarray

In [2]: xarray.tutorial.open_dataset('air_temperature')

ModuleNotFoundError Traceback (most recent call last) ~/dev/xarray/xarray/tutorial.py in open_dataset(name, cache, cache_dir, engine, **kws) 113 try: --> 114 import pooch 115 except ImportError:

ModuleNotFoundError: No module named 'pooch'

During handling of the above exception, another exception occurred:

ImportError Traceback (most recent call last) <ipython-input-2-e970a0b41221> in <module> ----> 1 xarray.tutorial.open_dataset('air_temperature')

~/dev/xarray/xarray/tutorial.py in open_dataset(name, cache, cache_dir, engine, **kws) 114 import pooch 115 except ImportError: --> 116 raise ImportError("using the tutorial data requires pooch") 117 118 logger = pooch.get_logger()

ImportError: using the tutorial data requires pooch

In [3]: ! pip install pooch Collecting pooch Using cached pooch-1.3.0-py3-none-any.whl (51 kB) Requirement already satisfied: requests in /Users/shoyer/miniconda3/envs/xarray-min/lib/python3.9/site-packages (from pooch) (2.25.1) Requirement already satisfied: appdirs in /Users/shoyer/miniconda3/envs/xarray-min/lib/python3.9/site-packages (from pooch) (1.4.4) Requirement already satisfied: packaging in /Users/shoyer/miniconda3/envs/xarray-min/lib/python3.9/site-packages (from pooch) (20.9) Requirement already satisfied: pyparsing>=2.0.2 in /Users/shoyer/miniconda3/envs/xarray-min/lib/python3.9/site-packages (from packaging->pooch) (2.4.7) Requirement already satisfied: certifi>=2017.4.17 in /Users/shoyer/miniconda3/envs/xarray-min/lib/python3.9/site-packages (from requests->pooch) (2020.12.5) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/shoyer/miniconda3/envs/xarray-min/lib/python3.9/site-packages (from requests->pooch) (1.26.4) Requirement already satisfied: idna<3,>=2.5 in /Users/shoyer/miniconda3/envs/xarray-min/lib/python3.9/site-packages (from requests->pooch) (2.10) Requirement already satisfied: chardet<5,>=3.0.2 in /Users/shoyer/miniconda3/envs/xarray-min/lib/python3.9/site-packages (from requests->pooch) (4.0.0) Installing collected packages: pooch Successfully installed pooch-1.3.0

In [4]: xarray.tutorial.open_dataset('air_temperature')

ModuleNotFoundError Traceback (most recent call last) ~/dev/xarray/xarray/tutorial.py in _check_netcdf_engine_installed(name) 51 try: ---> 52 import scipy 53 except ImportError:

ModuleNotFoundError: No module named 'scipy'

During handling of the above exception, another exception occurred:

ModuleNotFoundError Traceback (most recent call last) ~/dev/xarray/xarray/tutorial.py in _check_netcdf_engine_installed(name) 54 try: ---> 55 import netCDF4 56 except ImportError:

ModuleNotFoundError: No module named 'netCDF4'

During handling of the above exception, another exception occurred:

ImportError Traceback (most recent call last) <ipython-input-4-e970a0b41221> in <module> ----> 1 xarray.tutorial.open_dataset('air_temperature')

~/dev/xarray/xarray/tutorial.py in open_dataset(name, cache, cache_dir, engine, **kws) 128 default_extension = ".nc" 129 if engine is None: --> 130 _check_netcdf_engine_installed(name) 131 path = path.with_suffix(default_extension) 132 elif path.suffix == ".grib":

~/dev/xarray/xarray/tutorial.py in _check_netcdf_engine_installed(name) 55 import netCDF4 56 except ImportError: ---> 57 raise ImportError( 58 f"opening tutorial dataset {name} requires either scipy or " 59 "netCDF4 to be installed."

ImportError: opening tutorial dataset air_temperature requires either scipy or netCDF4 to be installed.

In [5]: ! pip install netcdf4 Collecting netcdf4 Using cached netCDF4-1.5.6-cp39-cp39-macosx_10_9_x86_64.whl (4.0 MB) Requirement already satisfied: numpy>=1.9 in /Users/shoyer/miniconda3/envs/xarray-min/lib/python3.9/site-packages (from netcdf4) (1.20.1) Requirement already satisfied: cftime in /Users/shoyer/miniconda3/envs/xarray-min/lib/python3.9/site-packages (from netcdf4) (1.4.1) Installing collected packages: netcdf4 Successfully installed netcdf4-1.5.6

In [6]: xarray.tutorial.open_dataset('air_temperature')

ValueError Traceback (most recent call last) <ipython-input-6-e970a0b41221> in <module> ----> 1 xarray.tutorial.open_dataset('air_temperature')

~/dev/xarray/xarray/tutorial.py in open_dataset(name, cache, cache_dir, engine, kws) 138 # retrieve the file 139 filepath = pooch.retrieve(url=url, known_hash=None, path=cache_dir) --> 140 ds = _open_dataset(filepath, engine=engine, kws) 141 if not cache: 142 ds = ds.load()

~/dev/xarray/xarray/backends/api.py in open_dataset(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, backend_kwargs, args, *kwargs) 478 479 if engine is None: --> 480 engine = plugins.guess_engine(filename_or_obj) 481 482 backend = plugins.get_backend(engine)

~/dev/xarray/xarray/backends/plugins.py in guess_engine(store_spec) 118 ) 119 else: --> 120 raise ValueError( 121 "xarray is unable to open this file because it has no currently " 122 "installed IO backends. Xarray's read/write support requires "

ValueError: xarray is unable to open this file because it has no currently installed IO backends. Xarray's read/write support requires installing optional dependencies: http://xarray.pydata.org/en/stable/getting-started-guide/installing.html http://xarray.pydata.org/en/stable/user-guide/io.html

In [7]: Do you really want to exit ([y]/n)? y (xarray-min) ~/dev/xarray better-no-plugins-errors $ ipython Python 3.9.4 (default, Apr 9 2021, 09:32:38) Type 'copyright', 'credits' or 'license' for more information IPython 7.22.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import xarray

In [2]: xarray.tutorial.open_dataset('air_temperature') Out[2]: <xarray.Dataset> Dimensions: (lat: 25, lon: 53, time: 2920) Coordinates: * lat (lat) float32 75.0 72.5 70.0 67.5 65.0 ... 25.0 22.5 20.0 17.5 15.0 * lon (lon) float32 200.0 202.5 205.0 207.5 ... 322.5 325.0 327.5 330.0 * time (time) datetime64[ns] 2013-01-01 ... 2014-12-31T18:00:00 Data variables: air (time, lat, lon) float32 ... Attributes: Conventions: COARDS title: 4x daily NMC reanalysis (1948) description: Data is from NMC initialized reanalysis\n(4x/day). These a... platform: Model references: http://www.esrl.noaa.gov/psd/data/gridded/data.ncep.reanaly... ```

This isn't quite perfect -- the user needs to restart their session to actually load the data -- but hopefully this is much more obvious and roughly within user expectations.

CC @alexamici @aurghs

I would suggest three levels of fixes:

1. Better fall-back message when engine guessing fails: https://github.com/pydata/xarray/pull/5300
2. Suggesting specific dependencies to install (e.g., netCDF4) based on the provided file, from inside open_dataset.
3. Suggesting specific dependencies to install when using tutorial datasets.

If we do a nice job of (2) than maybe (3) will not be necessary. I'll open a new issue to discuss my proposed solution for (2).

Good point -- in this case specifying an engine explicitly cannot help, because the required engine is not installed.

This is indeed unfortunate, thanks for the report!

I believe xarray.tutorial.load_dataset("air_temperature") works -- at least in my testing -- if either scipy or netCDF4 is installed (along with Pooch).

But if you don't have either scipy or netCDF4 installed, you get the very unhelpful error message ValueError: cannot guess the engine, try passing one explicitly

At the very least, we should report which engines are installed -- and ideally make a specific recommendation like installing scipy or netCDF4.