issue_comments: 1280759221

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/6803#issuecomment-1280759221 https://api.github.com/repos/pydata/xarray/issues/6803 1280759221 IC_kwDOAMm_X85MVtW1 33886395 2022-10-17T12:11:05Z 2022-10-17T12:11:05Z NONE

I'm not sure I understand the code above.

In my case I have an array of approximately 300k elements that each and every function call needs to have access. I can pass it as a kwargs in its numpy form, but once I scale up the calculation across a large dataset (many large chunks) such array gets replicated for every task pushing the scheduler out of memory.

That is why I tried to send the dataset to the cluster beforehand using scatter, but I cannot resolve the Future at the workers

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   1307523148