issues: 1687297423
This data as json
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | active_lock_reason | draft | pull_request | body | reactions | performed_via_github_app | state_reason | repo | type |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1687297423 | I_kwDOAMm_X85kkh2P | 7792 | If "chunks=None" is set in open_mfdataset, it is changed to "chunks={}" before being passed to "_dataset_from_backend_dataset" | 297142 | open | 0 | 2 | 2023-04-27T17:45:01Z | 2023-04-27T19:22:59Z | NONE | What happened?Using the grib2io engine, but have to use on a system that currently doesn't allow dask to be installed. Looking through the code I think that setting "chunks=None" would work to not use dask, but on https://github.com/pydata/xarray/blob/0f4e99d036b0d6d76a3271e6191eacbc9922662f/xarray/backends/api.py#L968 "chunks=None" is converted to "chunks={}". This means that at this test https://github.com/pydata/xarray/blob/0f4e99d036b0d6d76a3271e6191eacbc9922662f/xarray/backends/api.py#L339 for "chunks is None" will never be true and the dask code path will always run. The example below uses the rasterio engine because I could open publicly available files from S3. The rasterio engine gives the same error as the grib2io engine. What did you expect to happen?Expected open_mfdataset to work without dask installed. Minimal Complete Verifiable Example```Python Have to create an environment that doesn't include dask. For example:conda create -n xarrayenv -c conda-forge xarray rioxarrayconda activate xarrayenvimport xarray as xr import os os.environ["AWS_NO_SIGN_REQUEST"] = "YES" ds = xr.open_mfdataset( [ "/vsis3/noaa-nbm-grib2-pds/blend.20230401/02/core/blend.t02z.core.f003.co.grib2", "/vsis3/noaa-nbm-grib2-pds/blend.20230401/02/core/blend.t02z.core.f004.co.grib2", ], engine="rasterio", chunks=None, ) Traceback (most recent call last):File "/home/tim/test.py", line 6, in <module>ds = xr.open_mfdataset(^^^^^^^^^^^^^^^^^^File "/home/tim/anaconda3/envs/xarray/lib/python3.11/site-packages/xarray/backends/api.py", line 982, in open_mfdatasetdatasets = [open_(p, **open_kwargs) for p in paths]^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "/home/tim/anaconda3/envs/xarray/lib/python3.11/site-packages/xarray/backends/api.py", line 982, in <listcomp>datasets = [open_(p, **open_kwargs) for p in paths]^^^^^^^^^^^^^^^^^^^^^^^File "/home/tim/anaconda3/envs/xarray/lib/python3.11/site-packages/xarray/backends/api.py", line 531, in open_datasetds = _dataset_from_backend_dataset(^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "/home/tim/anaconda3/envs/xarray/lib/python3.11/site-packages/xarray/backends/api.py", line 342, in _dataset_from_backend_datasetds = _chunk_ds(^^^^^^^^^^File "/home/tim/anaconda3/envs/xarray/lib/python3.11/site-packages/xarray/backends/api.py", line 302, in _chunk_dsfrom dask.base import tokenizeModuleNotFoundError: No module named 'dask'``` MVCE confirmation
Relevant log outputNo response Anything else we need to know?No response Environment
/home/tim/anaconda3/envs/xarrayenv/lib/python3.11/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
INSTALLED VERSIONS
------------------
commit: None
python: 3.11.3 | packaged by conda-forge | (main, Apr 6 2023, 08:57:19) [GCC 11.3.0]
python-bits: 64
OS: Linux
OS-release: 5.15.0-70-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: None
libnetcdf: None
xarray: 2023.4.2
pandas: 2.0.1
numpy: 1.24.3
scipy: 1.10.1
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: None
nc_time_axis: None
PseudoNetCDF: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 67.7.2
pip: 23.1.2
conda: None
pytest: None
mypy: None
IPython: None
sphinx: None
|
{ "url": "https://api.github.com/repos/pydata/xarray/issues/7792/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
13221727 | issue |