id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 573972855,MDExOlB1bGxSZXF1ZXN0MzgyMzgyODA5,3818,map_blocks: Allow passing dask-backed objects in args,2448579,closed,0,,,7,2020-03-02T13:26:12Z,2020-06-11T18:22:42Z,2020-06-07T16:13:35Z,MEMBER,,0,pydata/xarray/pulls/3818," - [x] Tests added - [x] Passes `isort -rc . && black . && mypy . && flake8` - [x] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API It parses `args` and breaks any xarray objects into appropriate blocks before passing them on to the user function. e.g. ```python da1 = xr.DataArray( np.ones((10, 20)), dims=[""x"", ""y""], coords={""x"": np.arange(10), ""y"": np.arange(20)} ).chunk({""x"": 5, ""y"": 4}) da1 def sumda(da1, da2): #print(da1.shape) #print(da2.shape) return da1 - da2 da3 = (da1 + 1).isel(x=1, drop=True).rename({""y"": ""k""}) mapped = xr.map_blocks(sumda, da1, args=[da3]) xr.testing.assert_equal(da1-da3, mapped) # passes ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3818/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 573768194,MDExOlB1bGxSZXF1ZXN0MzgyMjE1OTYw,3816,Add template xarray object kwarg to map_blocks,2448579,closed,0,,,9,2020-03-02T07:20:19Z,2020-05-06T16:41:59Z,2020-05-06T16:41:54Z,MEMBER,,0,pydata/xarray/pulls/3816," - [x] Closes #3575 - [x] Tests added - [x] Passes `isort -rc . && black . && mypy . && flake8` - [x] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API This PR adds a `template` kwarg to `map_blocks` so that we can do more complicated things where automated inference of the template fails. `template` is expected to be an xarray object that looks like the result of the `map_blocks` computation. @jhamman To me, this seems a lot easier than defining a `dict` based schema. With dask variables, the memory cost shouldn't be high. It's easy to use standard xarray operations to make something that looks like the result dataset. Here's a notebook prototyping a `to_schema`/`from_schema` approach: https://gist.github.com/dcherian/130ba22d0fbadb616837deb914eaa67e#file-map_blocks_for_metsim_test-ipynb ```python ds = xr.tutorial.load_dataset(""air_temperature"").chunk({""lat"": 5}) template = ds.isel(lat=slice(1, None, 5)) result = xr.map_blocks(lambda x: x.isel(lat=[1]), ds) # fails result = xr.map_blocks(lambda x: x.isel(lat=[1]), ds, template=template) # yay xr.testing.assert_equal(result, template) # passes ``` Todo: - [x] add tests - [x] what if the function returns a variable not in the template? - [x] should we do anything with `template.attrs`? ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3816/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 573964634,MDExOlB1bGxSZXF1ZXN0MzgyMzc1OTcz,3817,map_blocks: allow user function to add new unindexed dimension.,2448579,closed,0,,,1,2020-03-02T13:12:25Z,2020-03-21T19:51:12Z,2020-03-21T19:51:07Z,MEMBER,,0,pydata/xarray/pulls/3817," - [x] Tests added - [x] Passes `isort -rc . && black . && mypy . && flake8` - [x] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API Small change that makes `map_blocks` apply functions that add new unindexed dimensions.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3817/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull