html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/pull/3584#issuecomment-573099009,https://api.github.com/repos/pydata/xarray/issues/3584,573099009,MDEyOklzc3VlQ29tbWVudDU3MzA5OTAwOQ==,2448579,2020-01-10T16:11:01Z,2020-01-10T16:11:01Z,MEMBER,Thanks @crusaderky ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,530657789
https://github.com/pydata/xarray/pull/3584#issuecomment-572197513,https://api.github.com/repos/pydata/xarray/issues/3584,572197513,MDEyOklzc3VlQ29tbWVudDU3MjE5NzUxMw==,2448579,2020-01-08T18:31:08Z,2020-01-08T18:31:08Z,MEMBER,gentle ping @crusaderky ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,530657789
https://github.com/pydata/xarray/pull/3584#issuecomment-561232232,https://api.github.com/repos/pydata/xarray/issues/3584,561232232,MDEyOklzc3VlQ29tbWVudDU2MTIzMjIzMg==,2448579,2019-12-03T15:57:16Z,2019-12-03T15:57:16Z,MEMBER,"The tests fail on `dask == 2.8.1` with this interesting bug. Here's a reproducible example.
``` python
import dask
import xarray as xr
ds = xr.Dataset({'x': (('y',), dask.array.ones(10, chunks=(3,)))})
mapped = ds.map_blocks(lambda x: x)
mapped.compute() # works
xr.testing.assert_equal(mapped, ds) # does not work
xr.testing.assert_equal(mapped, ds.compute()) # works
xr.testing.assert_equal(mapped.compute(), ds) # works
xr.testing.assert_equal(mapped.compute(), ds.compute()) # works
```
The traceback is
```
~/miniconda3/envs/dcpy/lib/python3.7/site-packages/dask/array/optimization.py in optimize(dsk, keys, fuse_keys, fast_functions, inline_functions_fast_functions, rename_fused_keys, **kwargs)
41 if isinstance(dsk, HighLevelGraph):
42 dsk = optimize_blockwise(dsk, keys=keys)
---> 43 dsk = fuse_roots(dsk, keys=keys)
44
45 # Low level task optimizations
~/miniconda3/envs/dcpy/lib/python3.7/site-packages/dask/blockwise.py in fuse_roots(graph, keys)
819 isinstance(layer, Blockwise)
820 and len(deps) > 1
--> 821 and not any(dependencies[dep] for dep in deps) # no need to fuse if 0 or 1
822 and all(len(dependents[dep]) == 1 for dep in deps)
823 ):
~/miniconda3/envs/dcpy/lib/python3.7/site-packages/dask/blockwise.py in (.0)
819 isinstance(layer, Blockwise)
820 and len(deps) > 1
--> 821 and not any(dependencies[dep] for dep in deps) # no need to fuse if 0 or 1
822 and all(len(dependents[dep]) == 1 for dep in deps)
823 ):
KeyError: 'lambda-6720ab0e3639d5c63fc06dfc66a3ce47-x'
```
This key is not in `dependencies`. From https://github.com/dask/dask/blob/67fb5363009c583c175cb577776a4f2f4e811410/dask/blockwise.py#L816-L826
``` python
for name, layer in graph.layers.items():
deps = graph.dependencies[name]
if (
isinstance(layer, Blockwise)
and len(deps) > 1
and not any(dependencies[dep] for dep in deps) # no need to fuse if 0 or 1
and all(len(dependents[dep]) == 1 for dep in deps)
):
new = toolz.merge(layer, *[layers[dep] for dep in deps])
new, _ = fuse(new, keys, ave_width=len(deps))
```
I'm not sure whether this is a bug in `fuse_roots`, `HighLevelGraph.from_collections` or in how `map_blocks` calls `HighLevelGraph.from_collections` here:
https://github.com/pydata/xarray/blob/69c85b85a1ef5d13b6ca51b6b655a1f719cc5abf/xarray/core/parallel.py#L315
cc @mrocklin
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,530657789