home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

5 rows where issue = 429914958 sorted by updated_at descending

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 3

  • shoyer 2
  • yt87 2
  • dopplershift 1

author_association 3

  • MEMBER 2
  • NONE 2
  • CONTRIBUTOR 1

issue 1

  • xr.open_dataset(f1).to_netcdf(file2) is not idempotent · 5 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
499921424 https://github.com/pydata/xarray/issues/2871#issuecomment-499921424 https://api.github.com/repos/pydata/xarray/issues/2871 MDEyOklzc3VlQ29tbWVudDQ5OTkyMTQyNA== dopplershift 221526 2019-06-07T15:06:29Z 2019-06-07T15:06:29Z CONTRIBUTOR

Just to correct something here, missing_value is no longer considered deprecated in the current version of the standard: http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/cf-conventions.html

Near the bottom is a specific note about removing the deprecation.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  xr.open_dataset(f1).to_netcdf(file2) is not idempotent 429914958
481033093 https://github.com/pydata/xarray/issues/2871#issuecomment-481033093 https://api.github.com/repos/pydata/xarray/issues/2871 MDEyOklzc3VlQ29tbWVudDQ4MTAzMzA5Mw== yt87 40218891 2019-04-08T22:35:10Z 2019-04-08T22:35:10Z NONE

After rethinking the issue, I would drop it: one can simply pass ds.fromkeys(ds.data_vars.keys(), {}) as the encoding attribute. Going back to the original problem. The fix above is not enough, the SerializationWarning is still present. An alternative, provided that missing_value attribute is still considered deprecated: http://cfconventions.org/Data/cf-conventions/cf-conventions-1.1/build/cf-conventions.html#missing-data, would be to replace it by _FillValue on decoding: $ diff variables.py variables.py.orig 179,180d178 < if 'FillValue' not in encoding: < encoding['_FillValue'] = encoding.pop('missing_value')``

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  xr.open_dataset(f1).to_netcdf(file2) is not idempotent 429914958
480624904 https://github.com/pydata/xarray/issues/2871#issuecomment-480624904 https://api.github.com/repos/pydata/xarray/issues/2871 MDEyOklzc3VlQ29tbWVudDQ4MDYyNDkwNA== shoyer 1217238 2019-04-07T20:11:05Z 2019-04-07T20:11:05Z MEMBER

Indeed it works. Thanks. My quick fix:

$ diff variables.py variables.py.orig 152,155d151 < elif encoding.get('missing_value') is not None: < fill_value = pop_to(encoding, attrs, 'missing_value', name=name) < if not pd.isnull(fill_value): < data = duck_array_ops.fillna(data, fill_value) I also figured out how to write back floating point values: encoding=None means use existing values, so specifying encoding={'tmpk': {}} in to_netcdf() did the trick. Should there be an option for this? What you see on the screen is not what you get in the file.

Yes, this is a challenging design problem and the current state of affairs is definitely not ideal. If you have ideas about how to reveal the presence of an encoding dict associated with a variable without overwhelming the user with information those would be very welcome. The alternative would be to not keep around encoding at all, but I think users do appreciate that reading/writing a netCDF file generally returns the exact same thing.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  xr.open_dataset(f1).to_netcdf(file2) is not idempotent 429914958
480475645 https://github.com/pydata/xarray/issues/2871#issuecomment-480475645 https://api.github.com/repos/pydata/xarray/issues/2871 MDEyOklzc3VlQ29tbWVudDQ4MDQ3NTY0NQ== yt87 40218891 2019-04-06T05:24:52Z 2019-04-06T05:24:52Z NONE

Indeed it works. Thanks. My quick fix: $ diff variables.py variables.py.orig 152,155d151 < elif encoding.get('missing_value') is not None: < fill_value = pop_to(encoding, attrs, 'missing_value', name=name) < if not pd.isnull(fill_value): < data = duck_array_ops.fillna(data, fill_value) I also figured out how to write back floating point values: encoding=None means use existing values, so specifying encoding={'tmpk': {}} in to_netcdf() did the trick. Should there be an option for this? What you see on the screen is not what you get in the file.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  xr.open_dataset(f1).to_netcdf(file2) is not idempotent 429914958
480460266 https://github.com/pydata/xarray/issues/2871#issuecomment-480460266 https://api.github.com/repos/pydata/xarray/issues/2871 MDEyOklzc3VlQ29tbWVudDQ4MDQ2MDI2Ng== shoyer 1217238 2019-04-06T00:49:39Z 2019-04-06T00:49:39Z MEMBER

I think this could be fixed by making CFMaskCoder handle missing_value and _FillValue in a symmetric way in encoding/decoding: https://github.com/pydata/xarray/blob/23d54a890e6cfe1de420071b597c911164de4cb8/xarray/coding/variables.py#L142

Right now, it only uses missing_value in the decode() method.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  xr.open_dataset(f1).to_netcdf(file2) is not idempotent 429914958

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 4718.177ms · About: xarray-datasette
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows