issues: 1368740629

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1368740629 PR_kwDOAMm_X84-uWtE 7019 Generalize handling of chunked array types 35968931 closed 0     30 2022-09-10T22:02:18Z 2023-07-24T20:40:29Z 2023-05-18T17:34:31Z MEMBER   0 pydata/xarray/pulls/7019

Initial attempt to get cubed working within xarray, as an alternative to dask.

  • [x] Closes #6807, at least for the case of cubed
  • [x] Tests added
  • [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
  • [x] New functions/methods are listed in api.rst
  • [x] Correct type hints

I've added a manager kwarg to the .chunk methods so you can do da.chunk(manager="cubed") to convert to a chunked cubed.CoreArray, with the default still being da.chunk(manager="dask"). (I couldn't think of a better name than "manager", as "backend" and "executor" are already taken.)

~~At the moment it should work except for an import error that I don't understand, see below.~~

Fro cubed to work at all with this PR we would also need: - [x] Cubed to expose the correct array type consistently https://github.com/tomwhite/cubed/issues/123 - [x] A cubed version of apply_gufunc https://github.com/tomwhite/cubed/pull/119 - implemented in https://github.com/tomwhite/cubed/pull/149 :partying_face:

To-dos for me on this PR: - [x] Re-route xarray.apply_ufunc through cubed.apply_gufunc instead of dask's apply_gufunc when appropriate, - [x] Add from_array_kwargs to opening functions, e.g. open_zarr, and open_dataset, - [x] Add from_array_kwargs to creation functions, such as full_like, - [x] Add store_kwargs as a way to propagate cubed-specific kwargs when saving to_zarr.

To complete this project more generally we should also: - [ ] Have cubed.apply_gufunc support multiple output arguments https://github.com/tomwhite/cubed/issues/152 - [x] Have a top-level cubed.unify_chunks to match dask.array.core.unify_chunks - [ ] Write a test suite for wrapping cubed arrays, which would be best done via #6894 - [ ] Generalise xarray.map_blocks to work on cubed arrays, ideally by first rewriting xarray's implementation of map_blocks to use dask.array.map_blocks

cc @tomwhite 

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/7019/reactions",
    "total_count": 4,
    "+1": 2,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 2,
    "eyes": 0
}
     13221727 pull

Links from other tables