home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

10 rows where issue = 286072335 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 6

  • shoyer 2
  • milancurcic 2
  • gbromley 2
  • charlie-becker 2
  • fmaussion 1
  • raybellwaves 1

author_association 3

  • NONE 6
  • MEMBER 3
  • CONTRIBUTOR 1

issue 1

  • WRF output : cannot serialize variable · 10 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
486390856 https://github.com/pydata/xarray/issues/1809#issuecomment-486390856 https://api.github.com/repos/pydata/xarray/issues/1809 MDEyOklzc3VlQ29tbWVudDQ4NjM5MDg1Ng== milancurcic 4133310 2019-04-24T19:22:52Z 2019-04-24T19:24:27Z NONE

I can't seem recreate this with a minimal example, xarray roundtrips a NetCDF file with a coordinates attribute correctly:

```python from netCDF4 import Dataset import xarray as xr

with Dataset('test.nc', format='NETCDF4', mode='w') as nc: nc.createDimension('dim1', size=0) var = nc.createVariable('var1', 'f8', dimensions=('dim1')) var[:] = [1., 2., 3.] var.setncattr('coordinates', 'dim1')

xr.open_dataset('test.nc').to_netcdf('test2.nc') ```

There is something peculiar about how WRF handles the coordinates attribute, but I can't see anything off about it yet.

Interestingly, I can workaround the WRF coordinates issue by setting decode_coords=False in xarray.open_dataset(), for example, this works:

python xr.open_dataset('wrfout_d01_2019-04-16_15_00_00', decode_coords=False).to_netcdf('test.nc')

while this doesn't:

python xr.open_dataset('wrfout_d01_2019-04-16_15_00_00').to_netcdf('test.nc')

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WRF output : cannot serialize variable 286072335
485210441 https://github.com/pydata/xarray/issues/1809#issuecomment-485210441 https://api.github.com/repos/pydata/xarray/issues/1809 MDEyOklzc3VlQ29tbWVudDQ4NTIxMDQ0MQ== milancurcic 4133310 2019-04-21T01:09:04Z 2019-04-24T19:24:02Z NONE

I ran into this issue trying to roundtrip a WRF output file. It looks like xarray raises an error for any NetCDF file that has variables with a coordinates attribute:

python # These coordinates are saved according to CF conventions for var_name, coord_names in variable_coordinates.items(): attrs = variables[var_name].attrs if 'coordinates' in attrs: raise ValueError('cannot serialize coordinates because variable ' "%s already has an attribute 'coordinates'" % var_name) attrs['coordinates'] = ' '.join(map(str, coord_names))

~~Both this choice, and the proposed solution in this issue (delete all coordinates attributes), I don't understand.~~ Variables with a coordinates attribute are CF conforming, so xarray should be able to play along with this.

~~The solution that makes more sense to me is to raise a warning and overwrite or ignore the coordinates attribute, if the attribute is already present. Later step of the fix could even be a keyword argument to allow the user to choose whether to overwrite or ignore "conflicting" attributes.~~

Or perhaps I'm missing something obvious here... Let me know either way. I'd be happy to make a PR to patch this.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WRF output : cannot serialize variable 286072335
445955599 https://github.com/pydata/xarray/issues/1809#issuecomment-445955599 https://api.github.com/repos/pydata/xarray/issues/1809 MDEyOklzc3VlQ29tbWVudDQ0NTk1NTU5OQ== gbromley 5695215 2018-12-10T20:11:32Z 2018-12-10T20:11:32Z NONE

That's perfect. Thank you!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WRF output : cannot serialize variable 286072335
445921610 https://github.com/pydata/xarray/issues/1809#issuecomment-445921610 https://api.github.com/repos/pydata/xarray/issues/1809 MDEyOklzc3VlQ29tbWVudDQ0NTkyMTYxMA== shoyer 1217238 2018-12-10T18:31:31Z 2018-12-10T18:31:31Z MEMBER

@gbromley The best option is probably a loop, e.g., python def remove_problematic_attrs(ds): for variable in ds.variables.values(): if 'coordinates' in variable.attrs: del variable.attrs['coordinates']

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WRF output : cannot serialize variable 286072335
445920567 https://github.com/pydata/xarray/issues/1809#issuecomment-445920567 https://api.github.com/repos/pydata/xarray/issues/1809 MDEyOklzc3VlQ29tbWVudDQ0NTkyMDU2Nw== gbromley 5695215 2018-12-10T18:28:28Z 2018-12-10T18:28:28Z NONE

Hello, I realized I never followed up with this but am again running into issues. The real problem for me is that deleting coordinates on 30+ variables isn't really feasible. Is it possible to delete the coordinates attribute for all variables?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WRF output : cannot serialize variable 286072335
391152491 https://github.com/pydata/xarray/issues/1809#issuecomment-391152491 https://api.github.com/repos/pydata/xarray/issues/1809 MDEyOklzc3VlQ29tbWVudDM5MTE1MjQ5MQ== charlie-becker 31113893 2018-05-22T21:51:13Z 2018-05-22T21:51:13Z NONE

Thanks for the solution (that was my SO post as well). Deleting the attrs.['coordinates'] was a clean workaround. Here's the requested WRF meta. It was concatenated with NCO, but meta should be consistent.

WRF_meta.txt

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WRF output : cannot serialize variable 286072335
391144127 https://github.com/pydata/xarray/issues/1809#issuecomment-391144127 https://api.github.com/repos/pydata/xarray/issues/1809 MDEyOklzc3VlQ29tbWVudDM5MTE0NDEyNw== shoyer 1217238 2018-05-22T21:18:41Z 2018-05-22T21:18:41Z MEMBER

See https://stackoverflow.com/questions/50475453/xarray-cannot-serialize-coordinates/50475925.

It seems like WRF may often produce netCDF files that lead to this issue. If the problem can be reproduced by simply opening and resaving a netCDF file then we may want to revisit our logic in xarray, because we always want xarray.open_dataset(input_path).to_netcdf(output_path) to work. If this is the case, it would be great if someone could share metadata for such a problematic netCDF file (e.g., from ncdump -h).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WRF output : cannot serialize variable 286072335
391103609 https://github.com/pydata/xarray/issues/1809#issuecomment-391103609 https://api.github.com/repos/pydata/xarray/issues/1809 MDEyOklzc3VlQ29tbWVudDM5MTEwMzYwOQ== charlie-becker 31113893 2018-05-22T18:58:40Z 2018-05-22T18:58:40Z NONE

Did you ever find a solution to this?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WRF output : cannot serialize variable 286072335
355812453 https://github.com/pydata/xarray/issues/1809#issuecomment-355812453 https://api.github.com/repos/pydata/xarray/issues/1809 MDEyOklzc3VlQ29tbWVudDM1NTgxMjQ1Mw== fmaussion 10050469 2018-01-07T10:19:13Z 2018-01-07T10:19:13Z MEMBER

It looks like your variable has an attribute coordinates: try deleting it before writing the data.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WRF output : cannot serialize variable 286072335
355793319 https://github.com/pydata/xarray/issues/1809#issuecomment-355793319 https://api.github.com/repos/pydata/xarray/issues/1809 MDEyOklzc3VlQ29tbWVudDM1NTc5MzMxOQ== raybellwaves 17162724 2018-01-07T02:05:18Z 2018-01-07T02:05:18Z CONTRIBUTOR

Would you mind printing md first to show what it looks like?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WRF output : cannot serialize variable 286072335

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 12.419ms · About: xarray-datasette