home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 936313924

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
936313924 MDExOlB1bGxSZXF1ZXN0NjgzMDY3OTU5 5571 Rely on NEP-18 to dispatch to dask in duck_array_ops 35968931 closed 0     20 2021-07-03T19:24:33Z 2022-07-09T18:12:05Z 2021-09-29T17:48:40Z MEMBER   0 pydata/xarray/pulls/5571

Removes special-casing for dask in duck_array_ops.py, instead relying on NEP-18 to call it when the input is a dask array.

Probably actually don't need the _dask_or_eager_func() (now _module_func()) helper function at all, because all remaining instances look like pandas_isnull = _module_func("isnull", module=pd), which could just be pandas_isnull = pd.isnull.

Only problem is that I seem to have broken one (parameterized) test: test_duck_array_ops.py::test_min_count[True-True-None-sum-True-bool_-1] fails with

```python @pytest.mark.parametrize("dim_num", [1, 2]) @pytest.mark.parametrize("dtype", [float, int, np.float32, np.bool_]) @pytest.mark.parametrize("dask", [False, True]) @pytest.mark.parametrize("func", ["sum", "prod"]) @pytest.mark.parametrize("aggdim", [None, "x"]) @pytest.mark.parametrize("contains_nan", [True, False]) @pytest.mark.parametrize("skipna", [True, False, None]) def test_min_count(dim_num, dtype, dask, func, aggdim, contains_nan, skipna): if dask and not has_dask: pytest.skip("requires dask")

    da = construct_dataarray(dim_num, dtype, contains_nan=contains_nan, dask=dask)
    min_count = 3

    # If using Dask, the function call should be lazy.
    with raise_if_dask_computes():
      actual = getattr(da, func)(dim=aggdim, skipna=skipna, min_count=min_count)

/home/tegn500/Documents/Work/Code/xarray/xarray/tests/test_duck_array_ops.py:578:


/home/tegn500/Documents/Work/Code/xarray/xarray/core/common.py:56: in wrapped_func return self.reduce(func, dim, axis, skipna=skipna, kwargs) /home/tegn500/Documents/Work/Code/xarray/xarray/core/dataarray.py:2638: in reduce var = self.variable.reduce(func, dim, axis, keep_attrs, keepdims, kwargs) /home/tegn500/Documents/Work/Code/xarray/xarray/core/variable.py:1725: in reduce data = func(self.data, kwargs) /home/tegn500/Documents/Work/Code/xarray/xarray/core/duck_array_ops.py:328: in f return func(values, axis=axis, kwargs) /home/tegn500/Documents/Work/Code/xarray/xarray/core/nanops.py:106: in nansum a, mask = _replace_nan(a, 0) /home/tegn500/Documents/Work/Code/xarray/xarray/core/nanops.py:23: in _replace_nan mask = isnull(a) /home/tegn500/Documents/Work/Code/xarray/xarray/core/duck_array_ops.py:83: in isnull return pandas_isnull(data) /home/tegn500/Documents/Work/Code/xarray/xarray/core/duck_array_ops.py:40: in f return getattr(module, name)(args, kwargs) /home/tegn500/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/pandas/core/dtypes/missing.py:127: in isna return _isna(obj) /home/tegn500/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/pandas/core/dtypes/missing.py:166: in _isna return _isna_ndarraylike(np.asarray(obj), inf_as_na=inf_as_na) /home/tegn500/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/numpy/core/_asarray.py:102: in asarray return array(a, dtype, copy=False, order=order) /home/tegn500/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/dask/array/core.py:1502: in array x = self.compute() /home/tegn500/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/dask/base.py:285: in compute (result,) = compute(self, traverse=False, kwargs) /home/tegn500/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/dask/base.py:567: in compute results = schedule(dsk, keys, *kwargs)


self = <xarray.tests.CountingScheduler object at 0x7f0804db2310> dsk = {('xarray-<this-array>-29953318277423606f95b509ad1a9aa7', 0): array([False, False, False, False], dtype=object), ('xar...pe=object), ('xarray-<this-array>-29953318277423606f95b509ad1a9aa7', 3): array([nan, False, False, nan], dtype=object)} keys = [[('xarray-<this-array>-29953318277423606f95b509ad1a9aa7', 0), ('xarray-<this-array>-29953318277423606f95b509ad1a9aa7'...array-<this-array>-29953318277423606f95b509ad1a9aa7', 2), ('xarray-<this-array>-29953318277423606f95b509ad1a9aa7', 3)]] kwargs = {}

def __call__(self, dsk, keys, **kwargs):
    self.total_computes += 1
    if self.total_computes > self.max_computes:
      raise RuntimeError(
            "Too many computes. Total: %d > max: %d."
            % (self.total_computes, self.max_computes)
        )

E RuntimeError: Too many computes. Total: 1 > max: 0.

/home/tegn500/Documents/Work/Code/xarray/xarray/tests/init.py:118: RuntimeError ```

  • [x] Closes #5559
  • [x] Tests added
  • [x] Passes pre-commit run --all-files
  • [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/5571/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    13221727 pull

Links from other tables

  • 1 row from issues_id in issues_labels
  • 20 rows from issue in issue_comments
Powered by Datasette · Queries took 0.739ms · About: xarray-datasette