issue_comments

Table actions

2 rows where issue = 745801652 and user = 463809 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • amatsukawa · 2 ✖

issue 1

  • Serialization issue with distributed, h5netcdf, and fsspec (ImplicitToExplicitIndexingAdapter) · 2 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
871611079 https://github.com/pydata/xarray/issues/4591#issuecomment-871611079 https://api.github.com/repos/pydata/xarray/issues/4591 MDEyOklzc3VlQ29tbWVudDg3MTYxMTA3OQ== amatsukawa 463809 2021-06-30T17:53:54Z 2021-06-30T17:53:54Z CONTRIBUTOR

I am trying to use worker_client that is opening xarrays, submitting further compute, and then saving xarrays. Perhaps somehow related to that?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Serialization issue with distributed, h5netcdf, and fsspec (ImplicitToExplicitIndexingAdapter) 745801652
870152019 https://github.com/pydata/xarray/issues/4591#issuecomment-870152019 https://api.github.com/repos/pydata/xarray/issues/4591 MDEyOklzc3VlQ29tbWVudDg3MDE1MjAxOQ== amatsukawa 463809 2021-06-29T01:10:30Z 2021-06-29T01:14:58Z CONTRIBUTOR

This issue appears to be back in some form, with engine=zarr.

The code looks like this, using fsspec's mapper API to access Azure blob store: fs = fsspec.filesystem("az://...") ds = xr.open_dataset(fs.get_mapper(path), engine="zarr", chunks="auto"): ...

I have not tracked down a self-contained reproducer, as it only fails for one call but not others of a similar form. Reporting it while I dig into it further, in case you have any suggestions.

[2021-06-29 00:44:47] [2021-06-29 00:44:47 core.py:74 CRITICAL] Failed to Serialize [2021-06-29 00:44:47] Traceback (most recent call last): [2021-06-29 00:44:47] File "/deps/envs/deps/lib/python3.7/site-packages/distributed/protocol/core.py", line 70, in dumps [2021-06-29 00:44:47] frames[0] = msgpack.dumps(msg, default=_encode_default, use_bin_type=True) [2021-06-29 00:44:47] File "/deps/envs/deps/lib/python3.7/site-packages/msgpack/__init__.py", line 35, in packb [2021-06-29 00:44:47] return Packer(**kwargs).pack(o) [2021-06-29 00:44:47] File "msgpack/_packer.pyx", line 286, in msgpack._cmsgpack.Packer.pack [2021-06-29 00:44:47] File "msgpack/_packer.pyx", line 292, in msgpack._cmsgpack.Packer.pack [2021-06-29 00:44:47] File "msgpack/_packer.pyx", line 289, in msgpack._cmsgpack.Packer.pack [2021-06-29 00:44:47] File "msgpack/_packer.pyx", line 258, in msgpack._cmsgpack.Packer._pack [2021-06-29 00:44:47] File "msgpack/_packer.pyx", line 225, in msgpack._cmsgpack.Packer._pack [2021-06-29 00:44:47] File "msgpack/_packer.pyx", line 225, in msgpack._cmsgpack.Packer._pack [2021-06-29 00:44:47] File "msgpack/_packer.pyx", line 258, in msgpack._cmsgpack.Packer._pack [2021-06-29 00:44:47] File "msgpack/_packer.pyx", line 225, in msgpack._cmsgpack.Packer._pack [2021-06-29 00:44:47] File "msgpack/_packer.pyx", line 225, in msgpack._cmsgpack.Packer._pack [2021-06-29 00:44:47] File "msgpack/_packer.pyx", line 225, in msgpack._cmsgpack.Packer._pack [2021-06-29 00:44:47] File "msgpack/_packer.pyx", line 279, in msgpack._cmsgpack.Packer._pack [2021-06-29 00:44:47] File "/deps/envs/deps/lib/python3.7/site-packages/distributed/protocol/core.py", line 56, in _encode_default [2021-06-29 00:44:47] obj, serializers=serializers, on_error=on_error, context=context [2021-06-29 00:44:47] File "/deps/envs/deps/lib/python3.7/site-packages/distributed/protocol/serialize.py", line 422, in serialize_and_split [2021-06-29 00:44:47] header, frames = serialize(x, serializers, on_error, context) [2021-06-29 00:44:47] File "/deps/envs/deps/lib/python3.7/site-packages/distributed/protocol/serialize.py", line 256, in serialize [2021-06-29 00:44:47] iterate_collection=True, [2021-06-29 00:44:47] File "/deps/envs/deps/lib/python3.7/site-packages/distributed/protocol/serialize.py", line 348, in serialize [2021-06-29 00:44:47] raise TypeError(msg, str(x)[:10000]) [2021-06-29 00:44:47] TypeError: ('Could not serialize object of type ImplicitToExplicitIndexingAdapter.', 'ImplicitToExplicitIndexingAdapter(array=CopyOnWriteArray(array=LazilyIndexedArray(array=<xarray.backends.zarr.ZarrArrayWrapper object at 0x7f52dedbb690>, key=BasicIndexer((slice(None, None, None), slice(None, None, None))))))') [2021-06-29 00:44:47] [2021-06-29 00:44:47 utils.py:37 ERROR] ('Could not serialize object of type ImplicitToExplicitIndexingAdapter.', 'ImplicitToExplicitIndexingAdapter(array=CopyOnWriteArray(array=LazilyIndexedArray(array=<xarray.backends.zarr.ZarrArrayWrapper object at 0x7f52dedbb690>, key=BasicIndexer((slice(None, None, None), slice(None, None, None))))))')

pip list | grep 'dask\|distributed\|xarray\|zarr\|msgpack\|adlfs' adlfs 0.7.7 dask 2021.6.2 distributed 2021.6.2 msgpack 1.0.0 xarray 0.18.2 zarr 2.8.3

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Serialization issue with distributed, h5netcdf, and fsspec (ImplicitToExplicitIndexingAdapter) 745801652

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);