issue_comments: 1061502981

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/5563#issuecomment-1061502981 https://api.github.com/repos/pydata/xarray/issues/5563 1061502981 IC_kwDOAMm_X84_RUAF 5821660 2022-03-08T07:56:53Z 2022-03-08T07:56:53Z MEMBER

Automatic decoding of bytes was implemented in #477 to properly decode returned bytes for CF decoding. In the case of non-utf-8 this brakes as shown.

hdf5 (and with that netCDF4) only has a notion of ASCII and UTF-8 for encoding (see h5py docs, https://docs.h5py.org/en/stable/strings.html#encodings). So the example above creates a non-standard file.

The question is what should be returned in the non-standard case if the attribute contains non-utf-8 encoded bytes? We could catch the UnicodeDecodeError and return something else (what?). But that would open the door for breakages with decoding CF metadata. I'm not sure if that can be properly resolved within xarray.

Why are those attributes in non-utf-8 encoding? Legacy data?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   935607748