home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

6 rows where issue = 787947436 and user = 5821660 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • kmuehlbauer · 6 ✖

issue 1

  • h5netcdf fails to decode attribute coordinates. · 6 ✖

author_association 1

  • MEMBER 6
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1081745934 https://github.com/pydata/xarray/issues/4822#issuecomment-1081745934 https://api.github.com/repos/pydata/xarray/issues/4822 IC_kwDOAMm_X85AeiIO kmuehlbauer 5821660 2022-03-29T11:20:44Z 2022-03-29T11:20:44Z MEMBER

@yt87 This works for me with latest h5netcdf version. There have been several fixes regarding attribute reading. The issue might be closed after confirming.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  h5netcdf fails to decode attribute coordinates. 787947436
762725640 https://github.com/pydata/xarray/issues/4822#issuecomment-762725640 https://api.github.com/repos/pydata/xarray/issues/4822 MDEyOklzc3VlQ29tbWVudDc2MjcyNTY0MA== kmuehlbauer 5821660 2021-01-19T09:43:53Z 2021-01-19T09:43:53Z MEMBER

Possibly related issue: https://github.com/h5netcdf/h5netcdf/issues/27

Anyway, I think I've found the root-cause of the issue. So with netcdf4-python it is possible to specify writing of attributes as NC_STRING string attributes.

The following will create a netcdf file which is readable with current xarray and h5netcdf backend.

```python import netCDF4 as nc import xarray as xr rootgrp = nc.Dataset("test_nc4_attrs.nc", mode="w", format="NETCDF4")

rootgrp.set_ncstring_attrs(True)

x = rootgrp.createDimension("x", 1) y = rootgrp.createDimension("y", 1) foo = rootgrp.createVariable("foo", "i4", ("y", "x")) foo.coordinates = "y x" foo[0, 0] = 0 rootgrp.close() ds = xr.open_dataset("test_nc4_attrs.nc", engine="h5netcdf") print(ds) print(ds.foo) ds.close() ```

If you uncomment the one line, this creates netcdf file which breaks with the error @yt87 showed above. If you try to open the file with decode_cf=False and write it back using to_netcdf the same error occurs.

h5dump before uncomment ATTRIBUTE "coordinates" { DATATYPE H5T_STRING { STRSIZE 3; STRPAD H5T_STR_NULLTERM; CSET H5T_CSET_ASCII; CTYPE H5T_C_S1; } DATASPACE SCALAR DATA { (0): "y x" } }

h5dump after uncomment ATTRIBUTE "coordinates" { DATATYPE H5T_STRING { STRSIZE H5T_VARIABLE; STRPAD H5T_STR_NULLTERM; CSET H5T_CSET_UTF8; CTYPE H5T_C_S1; } DATASPACE SIMPLE { ( 1 ) / ( 1 ) } DATA { (0): "y x" } }

Another thing I found, that this only accounts for coordinate attributes. All other attributes are read correctly AFAIKT. So with this in mind I would consider that a bug. The assumption that attributes are always scalar doesn't hold true.

And one more thing, also roundtripping in netcdf4 does not keep the NC_STRING type:

```python import netCDF4 as nc import xarray as xr rootgrp = nc.Dataset("test_nc4_attrs.nc", mode="w", format="NETCDF4") rootgrp.set_ncstring_attrs(True) x = rootgrp.createDimension("x", 1) y = rootgrp.createDimension("y", 1) foo = rootgrp.createVariable("foo", "i4", ("y", "x")) rootgrp.coordinates = "z y x" foo.coordinates = "y x" foo[0, 0] = 0 rootgrp.close()

ds = xr.open_dataset("test_nc4_attrs.nc", engine="netcdf4") ds.to_netcdf("test_nc4_attrs_out.nc") ds.close() ```

``` ncdump test_nc4_attrs.nc

netcdf test_nc4_attrs { dimensions: x = 1 ; y = 1 ; variables: int foo(y, x) ; string foo:coordinates = "y x" ;

// global attributes: string :coordinates = "z y x" ; data:

foo = 0 ; } ```

``` ncdump test_nc4_attrs_out.nc

netcdf test_nc4_attrs_out { dimensions: y = 1 ; x = 1 ; variables: int foo(y, x) ; foo:coordinates = "y x" ; data:

foo = 0 ; } ```

So changes are needed in several places to check if the attribute is scalar or array, not only for coordinates attributes. Thoughts?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  h5netcdf fails to decode attribute coordinates. 787947436
762429899 https://github.com/pydata/xarray/issues/4822#issuecomment-762429899 https://api.github.com/repos/pydata/xarray/issues/4822 MDEyOklzc3VlQ29tbWVudDc2MjQyOTg5OQ== kmuehlbauer 5821660 2021-01-18T19:18:22Z 2021-01-18T19:18:22Z MEMBER

@yt87 Thanks for providing all this.

Could you please give -k 'nc4' as option to the call to ncgen and report back? I think this is a problem with data model.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  h5netcdf fails to decode attribute coordinates. 787947436
762400927 https://github.com/pydata/xarray/issues/4822#issuecomment-762400927 https://api.github.com/repos/pydata/xarray/issues/4822 MDEyOklzc3VlQ29tbWVudDc2MjQwMDkyNw== kmuehlbauer 5821660 2021-01-18T18:07:27Z 2021-01-18T18:07:27Z MEMBER

If you want a quick fix to read the file, add kwarg decode_cf=False in open_dataset. Then fix the coordinate attribute and call xr.decode_cf(ds).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  h5netcdf fails to decode attribute coordinates. 787947436
762396972 https://github.com/pydata/xarray/issues/4822#issuecomment-762396972 https://api.github.com/repos/pydata/xarray/issues/4822 MDEyOklzc3VlQ29tbWVudDc2MjM5Njk3Mg== kmuehlbauer 5821660 2021-01-18T17:58:07Z 2021-01-18T17:58:07Z MEMBER

Hmm, this worked in my case. How exactly did you create the netcdf file?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  h5netcdf fails to decode attribute coordinates. 787947436
762258597 https://github.com/pydata/xarray/issues/4822#issuecomment-762258597 https://api.github.com/repos/pydata/xarray/issues/4822 MDEyOklzc3VlQ29tbWVudDc2MjI1ODU5Nw== kmuehlbauer 5821660 2021-01-18T13:42:20Z 2021-01-18T13:42:20Z MEMBER

@yt87 AFAIK the CDL doesn't declare attribute types specifically. This should work:

``` netcdf x { dimensions: x = 1 ; y = 1 ; variables: int foo(y, x) ; foo:coordinates = "x y" ; data:

foo = 0 ; } ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  h5netcdf fails to decode attribute coordinates. 787947436

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 30.836ms · About: xarray-datasette