issue_comments: 381969631
This data as json
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | performed_via_github_app | issue |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/pydata/xarray/pull/1811#issuecomment-381969631 | https://api.github.com/repos/pydata/xarray/issues/1811 | 381969631 | MDEyOklzc3VlQ29tbWVudDM4MTk2OTYzMQ== | 1872600 | 2018-04-17T12:12:15Z | 2018-04-17T12:15:19Z | NONE | @jhamman , I'm trying to test Write National Water Model data to Zarrfrom dask.distributed import Client import pandas as pd import xarray as xr import s3fs import zarr if name == 'main':
root = 'http://tds.renci.org:8080/thredds/dodsC/nwm/forcing_short_range/' # OPenDAP
bucket_endpoint='https://iu.jetstream-cloud.org:8080'
``` and after 20 seconds or so, the process dies with this error: ```python-traceback /home/rsignell/my-conda-envs/zarr/lib/python3.6/site-packages/distributed/worker.py:742: UserWarning: Large object of size 1.23 MB detected in task graph: (<xarray.backends.zarr.ZarrStore object at 0x7f5d8 ... deedecefab224') Consider scattering large objects ahead of time with client.scatter to reduce scheduler burden and keep data on workers
% (format_bytes(len(b)), s)) ``` Do you have suggestions on how to modify my code? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
286542795 |