issues: 1307523148
This data as json
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | active_lock_reason | draft | pull_request | body | reactions | performed_via_github_app | state_reason | repo | type |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1307523148 | I_kwDOAMm_X85N7zhM | 6803 | Passing a distributed.Future to the kwargs of apply_ufunc should resolve the future | 33886395 | closed | 0 | 10 | 2022-07-18T07:31:28Z | 2024-01-09T18:21:15Z | 2023-12-19T05:40:20Z | NONE | What is your issue?I am trying to scatter an large array and pass it as keyword argument to a function applied using Here is a minimal example to reproduce this issue ```python import dask.array as da import xarray as xr import numpy as np data = xr.DataArray(data=da.random.random((15, 15, 20)), coords={'x': range(15), 'y': range(15), 'z': range(20)}, dims=('x', 'y', 'z')) test = np.full((20,), 30) test_future = client.scatter(test, broadcast=True) def _copy_test(d, test=None): return test new_data_actual = xr.apply_ufunc( _copy_test, data, input_core_dims=[['z']], output_core_dims=[['new_z']], vectorize=True, dask='parallelized', output_dtypes="float64", kwargs={'test':test}, dask_gufunc_kwargs = {'output_sizes':{'new_z':20}} ) new_data_future = xr.apply_ufunc( _copy_test, data, input_core_dims=[['z']], output_core_dims=[['new_z']], vectorize=True, dask='parallelized', output_dtypes="float64", kwargs={'test':test_future}, dask_gufunc_kwargs = {'output_sizes':{'new_z':20}} ) data[0, 0].compute() [0.3034994 , 0.08172002, 0.34731092, ...]new_data_actual[0, 0].compute() [30.0, 30.0, 30.0, ...]new_data_future[0,0].compute() KilledWorker``` I tried different versions of this, going from explicitly calling Am I trying to do something completely silly? or is this an unexpected behavior? |
{ "url": "https://api.github.com/repos/pydata/xarray/issues/6803/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | 13221727 | issue |