html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/3932#issuecomment-610167347,https://api.github.com/repos/pydata/xarray/issues/3932,610167347,MDEyOklzc3VlQ29tbWVudDYxMDE2NzM0Nw==,11750960,2020-04-07T04:32:12Z,2020-04-07T04:32:12Z,CONTRIBUTOR,I'll close this for now as there doesn't seem to be other ideas about this,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,593825520 https://github.com/pydata/xarray/issues/3932#issuecomment-609605285,https://api.github.com/repos/pydata/xarray/issues/3932,609605285,MDEyOklzc3VlQ29tbWVudDYwOTYwNTI4NQ==,11750960,2020-04-06T07:08:19Z,2020-04-06T07:08:19Z,CONTRIBUTOR,"This sounds like method 1 (with dask delayed) to me. There may be no faster option, thanks for giving it a thought @fujiisoup ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,593825520 https://github.com/pydata/xarray/issues/3932#issuecomment-609483638,https://api.github.com/repos/pydata/xarray/issues/3932,609483638,MDEyOklzc3VlQ29tbWVudDYwOTQ4MzYzOA==,6815844,2020-04-05T21:09:58Z,2020-04-05T21:09:58Z,MEMBER,"> An inspection of the dask dashboard indicates that the computation is not distributed among workers though. > How could I make sure this happens? Ah, I have no idea... Are you able to distribute the function `some_exp` without wrapping by xarray? Within my limited knowledge, it may be better to prepare another function that distributes `some_exp` over the workers and put this another function into `apply_ufunc`, but I am not 100% sure. Probably there is a better way...","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,593825520 https://github.com/pydata/xarray/issues/3932#issuecomment-609407162,https://api.github.com/repos/pydata/xarray/issues/3932,609407162,MDEyOklzc3VlQ29tbWVudDYwOTQwNzE2Mg==,11750960,2020-04-05T12:17:15Z,2020-04-05T12:17:47Z,CONTRIBUTOR,"thanks a lot @fujiisoup, your suggestion does help getting rid of the necessity to build the `ds['_y']` variable. Here is the updated `apply_ufunc` solution: ``` x = np.arange(10*100) y = np.arange(20*100) ds = xr.Dataset(coords={'x': x, 'y': y}) # ds = ds.chunk({'x': 1, 'y':1}) # does not change anything # let's say each experiment outputs 5 statistical diagnostics Nstats = 5 some_exp = lambda x, y: np.ones((Nstats,)) out = xr.apply_ufunc(some_exp, ds.x, ds.y, dask='parallelized', vectorize=True, output_dtypes=[float], output_sizes={'stats': Nstats}, output_core_dims=[['stats']]) ``` An inspection of the dask dashboard indicates that the computation is not distributed among workers though. How could I make sure this happens? ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,593825520 https://github.com/pydata/xarray/issues/3932#issuecomment-609407192,https://api.github.com/repos/pydata/xarray/issues/3932,609407192,MDEyOklzc3VlQ29tbWVudDYwOTQwNzE5Mg==,11750960,2020-04-05T12:17:26Z,2020-04-05T12:17:26Z,CONTRIBUTOR,sorry closed by accident,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,593825520 https://github.com/pydata/xarray/issues/3932#issuecomment-609094164,https://api.github.com/repos/pydata/xarray/issues/3932,609094164,MDEyOklzc3VlQ29tbWVudDYwOTA5NDE2NA==,6815844,2020-04-04T21:54:41Z,2020-04-04T21:54:56Z,MEMBER,"Is ```python xr.apply_ufunc(some_exp, ds.x, ds.y, dask='parallelized', output_dtypes=[float], output_sizes={'stats': Nstats}, output_core_dims=[['stats']], vectorize=True) ``` what you want? This gives ```python array([[[1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.], ... [1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.]]]) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 * y (y) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Dimensions without coordinates: stats In [26]: ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,593825520