issue_comments: 1469907455

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/pull/7019#issuecomment-1469907455 https://api.github.com/repos/pydata/xarray/issues/7019 1469907455 IC_kwDOAMm_X85XnQH_ 85085 2023-03-15T12:21:23Z 2023-03-15T12:21:23Z CONTRIBUTOR

I think it might make sense for me to remove the CubedManager class from this PR and instead put that & cubed+xarray tests into another repo. That keeps xarray's changes minimal, doesn't require putting cubed in any xarray CI envs, and hopefully allows us to merge the ChunkManager changes here earlier.

That sounds like a good plan to me.

Places dask is still explicitly imported in xarray

There are a few remaining places where I haven't generalised to remove specific import dask calls either because it won't be imported at runtime unless you ask for it, cubed doesn't implement the equivalent function, that function isn't in the array API standard, or because I'm not sure if the dask concept used generalises to other parallel frameworks.

  • [ ] open_mfdataset(..., parallel=True) - there is no cubed.delayed to wrap the open_dataset calls in,
  • [ ] Dataset.__dask_graph__ and all the other similar dask magic methods
  • [ ] dask_array_ops.rolling - uses functions from dask.array.overlap,
  • [ ] dask_array_ops.least_squares - uses dask.array.apply_along_axis and dask.array.linalg.lstsq,
  • [ ] dask_array_ops.push - uses dask.array.reductions.cumreduction

This is a useful list! I hope that we could close the gap for some of these over time.

I would like to get to the point where you can use xarray with a chunked array without ever importing dask. I think this PR gets very close, but that would be tricky to test because cubed depends on dask (so I can't just run the test suite without dask in the environment)

Agreed. I have opened https://github.com/tomwhite/cubed/issues/154 to make it possible to test without a Dask dependency.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   1368740629