home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 1687297423

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1687297423 I_kwDOAMm_X85kkh2P 7792 If "chunks=None" is set in open_mfdataset, it is changed to "chunks={}" before being passed to "_dataset_from_backend_dataset" 297142 open 0     2 2023-04-27T17:45:01Z 2023-04-27T19:22:59Z   NONE      

What happened?

Using the grib2io engine, but have to use on a system that currently doesn't allow dask to be installed. Looking through the code I think that setting "chunks=None" would work to not use dask, but on https://github.com/pydata/xarray/blob/0f4e99d036b0d6d76a3271e6191eacbc9922662f/xarray/backends/api.py#L968 "chunks=None" is converted to "chunks={}".

This means that at this test https://github.com/pydata/xarray/blob/0f4e99d036b0d6d76a3271e6191eacbc9922662f/xarray/backends/api.py#L339 for "chunks is None" will never be true and the dask code path will always run.

The example below uses the rasterio engine because I could open publicly available files from S3. The rasterio engine gives the same error as the grib2io engine.

What did you expect to happen?

Expected open_mfdataset to work without dask installed.

Minimal Complete Verifiable Example

```Python

Have to create an environment that doesn't include dask. For example:

conda create -n xarrayenv -c conda-forge xarray rioxarray

conda activate xarrayenv

import xarray as xr import os

os.environ["AWS_NO_SIGN_REQUEST"] = "YES"

ds = xr.open_mfdataset( [ "/vsis3/noaa-nbm-grib2-pds/blend.20230401/02/core/blend.t02z.core.f003.co.grib2", "/vsis3/noaa-nbm-grib2-pds/blend.20230401/02/core/blend.t02z.core.f004.co.grib2", ], engine="rasterio", chunks=None, )

Traceback (most recent call last):

File "/home/tim/test.py", line 6, in <module>

ds = xr.open_mfdataset(

^^^^^^^^^^^^^^^^^^

File "/home/tim/anaconda3/envs/xarray/lib/python3.11/site-packages/xarray/backends/api.py", line 982, in open_mfdataset

datasets = [open_(p, **open_kwargs) for p in paths]

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/tim/anaconda3/envs/xarray/lib/python3.11/site-packages/xarray/backends/api.py", line 982, in <listcomp>

datasets = [open_(p, **open_kwargs) for p in paths]

^^^^^^^^^^^^^^^^^^^^^^^

File "/home/tim/anaconda3/envs/xarray/lib/python3.11/site-packages/xarray/backends/api.py", line 531, in open_dataset

ds = _dataset_from_backend_dataset(

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/tim/anaconda3/envs/xarray/lib/python3.11/site-packages/xarray/backends/api.py", line 342, in _dataset_from_backend_dataset

ds = _chunk_ds(

^^^^^^^^^^

File "/home/tim/anaconda3/envs/xarray/lib/python3.11/site-packages/xarray/backends/api.py", line 302, in _chunk_ds

from dask.base import tokenize

ModuleNotFoundError: No module named 'dask'

```

MVCE confirmation

  • [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • [X] Complete example — the example is self-contained, including all data and the text of any traceback.
  • [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

No response

Environment

/home/tim/anaconda3/envs/xarrayenv/lib/python3.11/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils. warnings.warn("Setuptools is replacing distutils.") INSTALLED VERSIONS ------------------ commit: None python: 3.11.3 | packaged by conda-forge | (main, Apr 6 2023, 08:57:19) [GCC 11.3.0] python-bits: 64 OS: Linux OS-release: 5.15.0-70-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: None libnetcdf: None xarray: 2023.4.2 pandas: 2.0.1 numpy: 1.24.3 scipy: 1.10.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nc_time_axis: None PseudoNetCDF: None iris: None bottleneck: None dask: None distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None flox: None numpy_groupies: None setuptools: 67.7.2 pip: 23.1.2 conda: None pytest: None mypy: None IPython: None sphinx: None
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/7792/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    13221727 issue

Links from other tables

  • 2 rows from issues_id in issues_labels
  • 2 rows from issue in issue_comments
Powered by Datasette · Queries took 0.713ms · About: xarray-datasette