issue_comments: 392217441

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/2186#issuecomment-392217441 https://api.github.com/repos/pydata/xarray/issues/2186 392217441 MDEyOklzc3VlQ29tbWVudDM5MjIxNzQ0MQ== 12929327 2018-05-26T00:03:59Z 2018-05-26T00:03:59Z NONE

I'm now wondering if this issue is in dask land, based on this issue: https://github.com/dask/dask/issues/3247

It has been suggested in other places to get around the memory accumulation by running each loop iteration in a forked process:

```python def worker(ds, k): print('accessing data') data = ds.datavar[k,:,:].values print('data acquired')

for k in range(ds.dims['t']): p = multiprocessing.Process(target=worker, args=(ds, k)) p.start() p.join() ``` But apparently one can't access dask-wrapped xarray datasets in subprocesses without a deadlock. I don't know enough about the internals to understand why.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   326533369