issue_comments: 381969631
This data as json
| html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | performed_via_github_app | issue |
|---|---|---|---|---|---|---|---|---|---|---|---|
| https://github.com/pydata/xarray/pull/1811#issuecomment-381969631 | https://api.github.com/repos/pydata/xarray/issues/1811 | 381969631 | MDEyOklzc3VlQ29tbWVudDM4MTk2OTYzMQ== | 1872600 | 2018-04-17T12:12:15Z | 2018-04-17T12:15:19Z | NONE | @jhamman , I'm trying to test Write National Water Model data to Zarrfrom dask.distributed import Client import pandas as pd import xarray as xr import s3fs import zarr if name == 'main':
root = 'http://tds.renci.org:8080/thredds/dodsC/nwm/forcing_short_range/' # OPenDAP
bucket_endpoint='https://iu.jetstream-cloud.org:8080'
``` and after 20 seconds or so, the process dies with this error: ```python-traceback /home/rsignell/my-conda-envs/zarr/lib/python3.6/site-packages/distributed/worker.py:742: UserWarning: Large object of size 1.23 MB detected in task graph: (<xarray.backends.zarr.ZarrStore object at 0x7f5d8 ... deedecefab224') Consider scattering large objects ahead of time with client.scatter to reduce scheduler burden and keep data on workers
% (format_bytes(len(b)), s)) ``` Do you have suggestions on how to modify my code? |
{
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
286542795 |