issues: 488243328

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
488243328 MDExOlB1bGxSZXF1ZXN0MzEzMzA4NzE4 3276 map_blocks 2448579 closed 0     19 2019-09-02T15:32:42Z 2019-10-17T01:30:30Z 2019-10-10T23:44:19Z MEMBER   0 pydata/xarray/pulls/3276
  • [x] xref #3147, closes #3323
  • [x] Tests added
  • [x] Passes black . && mypy . && flake8
  • [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

This is a second attempt. It works with Datasets also in that it can pass Dataset chunks to the user's function. I haven't figured out how to reconstruct a Dataset from things returned from the user's function. For DataArrays, I currently assume that the user function returns chunks of the same shape as the provided chunk.

A few questions: 1. I wonder if we should introduce a meta xarray object that describes the final result that user expects back i.e. a template DataArray or Dataset with chunks, dtype etc. specified. We could have a utility function that makes it easy to create these templates from existing objects.

  1. If the user function doesn't return anything, (e.g. it writes to a file), how do I "compute" the graph? In this case, we could rename this function to apply_blocks and release that till we figure out point 1?

  2. Should we also add Dataset.to_delayed and DataArray.to_delayed? This would make use-cases like the one in point 2 easier.

ping @mrocklin 

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3276/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     13221727 pull

Links from other tables