issue_comments

Table actions

5 rows where issue = 324040111 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 2

issue 1

  • [REGRESSION] to_netcdf doesn't accept dtype=S1 encoding anymore · 5 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
390037392 https://github.com/pydata/xarray/issues/2149#issuecomment-390037392 https://api.github.com/repos/pydata/xarray/issues/2149 MDEyOklzc3VlQ29tbWVudDM5MDAzNzM5Mg== shoyer 1217238 2018-05-17T22:47:48Z 2018-05-17T22:47:48Z MEMBER

I just got off an airplane (without internet), but I'll try to look into this shortly.

I guess I shouldn't be surprised that https://github.com/pydata/xarray/commit/3c8935e537e6ec05a83dbe372bfe45d88308d665 was the source of this bug -- it was pretty big refactor :(.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   [REGRESSION] to_netcdf doesn't accept dtype=S1 encoding anymore 324040111
389954770 https://github.com/pydata/xarray/issues/2149#issuecomment-389954770 https://api.github.com/repos/pydata/xarray/issues/2149 MDEyOklzc3VlQ29tbWVudDM4OTk1NDc3MA== crusaderky 6213168 2018-05-17T17:57:07Z 2018-05-17T17:57:07Z MEMBER

@shoyer I'm struggling to understand your change - help would be appreciated.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   [REGRESSION] to_netcdf doesn't accept dtype=S1 encoding anymore 324040111
389947329 https://github.com/pydata/xarray/issues/2149#issuecomment-389947329 https://api.github.com/repos/pydata/xarray/issues/2149 MDEyOklzc3VlQ29tbWVudDM4OTk0NzMyOQ== crusaderky 6213168 2018-05-17T17:33:06Z 2018-05-17T17:33:37Z MEMBER

@shoyer found the culprit: https://github.com/pydata/xarray/commit/3c8935e537e6ec05a83dbe372bfe45d88308d665 Still need to understand why if you set the encoding directly on the variable it works though.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   [REGRESSION] to_netcdf doesn't accept dtype=S1 encoding anymore 324040111
389885258 https://github.com/pydata/xarray/issues/2149#issuecomment-389885258 https://api.github.com/repos/pydata/xarray/issues/2149 MDEyOklzc3VlQ29tbWVudDM4OTg4NTI1OA== crusaderky 6213168 2018-05-17T14:25:24Z 2018-05-17T14:25:24Z MEMBER
  1. this is limited to dtype=S1. There's auto tests implemented for dtype=f4 and dtype=int16 (https://github.com/pydata/xarray/blob/master/xarray/tests/test_backends.py#L729-L756), and those work fine. ds = xarray.Dataset({'x': [1, 2]}) ds.to_netcdf('test.nc', encoding={'x': {'dtype': 'f4'}})
  2. the bug can be worked around by setting the dtype on the variable directly - for which there is an automated test specific for S1 (https://github.com/pydata/xarray/blob/master/xarray/tests/test_backends.py#L333-L343): ds = xarray.Dataset({'x': ['foo', 'bar', 'baz']}) ds['x'].encoding['dtype'] = 'S1' ds.to_netcdf('test.nc')
 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   [REGRESSION] to_netcdf doesn't accept dtype=S1 encoding anymore 324040111
389884190 https://github.com/pydata/xarray/issues/2149#issuecomment-389884190 https://api.github.com/repos/pydata/xarray/issues/2149 MDEyOklzc3VlQ29tbWVudDM4OTg4NDE5MA== shoyer 1217238 2018-05-17T14:22:24Z 2018-05-17T14:22:24Z MEMBER

Yes, this would warrant an immediate bug fix release. On Thu, May 17, 2018 at 7:09 AM crusaderky notifications@github.com wrote:

In xarray 0.10.4, the dtype encoding in to_netcdf has stopped working, for all engines:

import xarray ds = xarray.Dataset({'x': ['foo', 'bar', 'baz']}) ds.to_netcdf('test.nc', encoding={'x': {'dtype': 'S1'}}) [...]

xarray/backends/netCDF4_.py in _extract_nc4_variable_encoding(variable, raise_on_invalid, lsd_okay, h5py_okay, backend, unlimited_dims) 196 if invalid: 197 raise ValueError('unexpected encoding parameters for %r backend: ' --> 198 ' %r' % (backend, invalid)) 199 else: 200 for k in list(encoding):

ValueError: unexpected encoding parameters for 'netCDF4' backend: ['dtype']

I'm still trying to figure out how the regression tests didn't pick it up and what change introduced it.

@shoyer https://github.com/shoyer I'm working on this as my top priority. Do you agree this is serious enough for an emergency re-release? (0.10.4.1 or 0.10.5, your choice)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/2149, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1o6DKXddpZGzWNVoVa3IYfXomAN4ks5tzYSMgaJpZM4UDJB8 . 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   [REGRESSION] to_netcdf doesn't accept dtype=S1 encoding anymore 324040111

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);