home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

7 rows where issue = 1416709246 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 4

  • kmuehlbauer 3
  • WillyChap 2
  • dopplershift 1
  • dcherian 1

author_association 3

  • MEMBER 4
  • NONE 2
  • CONTRIBUTOR 1

issue 1

  • Cannot Save NetCDF: Conflicting _FillValue and Missing_Value · 7 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1287124949 https://github.com/pydata/xarray/issues/7191#issuecomment-1287124949 https://api.github.com/repos/pydata/xarray/issues/7191 IC_kwDOAMm_X85Mt_fV dcherian 2448579 2022-10-21T15:30:10Z 2022-10-21T15:30:35Z MEMBER

I'm also not sure why _FillValue and missing_value should be required to have the same value.

Xarray has only one way to represent both concepts: np.nan. So when you decode you lose information on whether a value was a missing_value or _FillValue. Then when you encode, you have to pick either _FillValue or missing_value to represent the np.nan

to_netcdf will assign '_FillValue': nan.

This seems like a bug. Can you open a new issue with a minimum reproducible example that uses random data please?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Cannot Save NetCDF: Conflicting _FillValue and Missing_Value  1416709246
1286980037 https://github.com/pydata/xarray/issues/7191#issuecomment-1286980037 https://api.github.com/repos/pydata/xarray/issues/7191 IC_kwDOAMm_X85MtcHF WillyChap 14932329 2022-10-21T13:42:41Z 2022-10-21T13:42:41Z NONE

@kmuehlbauer So it looks like uwnd.encoding does contain _FillVale and missing_value which are contrasting. Setting them to equal values enables a save.

However, these variables were set to contrasting values after an xarray subset or save. See workflow below. It appears that to_netcdf will assign '_FillValue': nan. I don't know if this is behavior that requires a change, or if it is just a quirk that folks will have to work through.

python fn_E20c_VP=sorted(glob.glob('/Users/wchapman/Downloads/uvwndNOAA/uwnd*.nc')) DSorig = xr.open_dataset(fn_E20c_VP[0]) DSorig.uwnd.encoding {'zlib': True, 'szip': False, 'zstd': False, 'bzip2': False, 'blosc': False, 'shuffle': True, 'complevel': 2, 'fletcher32': False, 'contiguous': False, 'chunksizes': (1, 1, 73, 144), 'least_significant_digit': 1, 'source': '/Users/wchapman/Downloads/uvwndNOAA/uwnd.1950.nc', 'original_shape': (365, 17, 73, 144), 'dtype': dtype('float32'), 'missing_value': -9.96921e+36} Subset and Save: python fn_E20c_VP=sorted(glob.glob('/Users/wchapman/Downloads/uvwndNOAA/uwnd*.nc')) DS_subset = DSorig.sel(level=300) DS_subset.to_netcdf('/Users/wchapman/Downloads/test_behavior.nc') DS_subset.uwnd.encoding {'zlib': True, 'szip': False, 'zstd': False, 'bzip2': False, 'blosc': False, 'shuffle': True, 'complevel': 2, 'fletcher32': False, 'contiguous': False, 'chunksizes': (1, 1, 73, 144), 'least_significant_digit': 1, 'source': '/Users/wchapman/Downloads/uvwndNOAA/uwnd.1950.nc', 'original_shape': (365, 17, 73, 144), 'dtype': dtype('float32'), 'missing_value': -9.96921e+36} Re-open newly saved file and inspect :

python DS_subset_aftersave = xr.open_dataset('/Users/wchapman/Downloads/test_behavior.nc') DS_subset_aftersave.uwnd.encoding {'zlib': True, 'szip': False, 'zstd': False, 'bzip2': False, 'blosc': False, 'shuffle': True, 'complevel': 2, 'fletcher32': False, 'contiguous': False, 'chunksizes': (1, 73, 144), 'least_significant_digit': 1, 'source': '/Users/wchapman/Downloads/test_behavior.nc', 'original_shape': (365, 73, 144), 'dtype': dtype('float32'), 'missing_value': -9.96921e+36, '_FillValue': nan, 'coordinates': 'level'}

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Cannot Save NetCDF: Conflicting _FillValue and Missing_Value  1416709246
1286951591 https://github.com/pydata/xarray/issues/7191#issuecomment-1286951591 https://api.github.com/repos/pydata/xarray/issues/7191 IC_kwDOAMm_X85MtVKn kmuehlbauer 5821660 2022-10-21T13:18:59Z 2022-10-21T13:18:59Z MEMBER

@dopplershift equivalence is checked here, there is also the error origination from.

https://github.com/pydata/xarray/blob/3c98ec7d96cc4b46664850cc7a40af2bc184fea0/xarray/coding/variables.py#L153-L163

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Cannot Save NetCDF: Conflicting _FillValue and Missing_Value  1416709246
1285942035 https://github.com/pydata/xarray/issues/7191#issuecomment-1285942035 https://api.github.com/repos/pydata/xarray/issues/7191 IC_kwDOAMm_X85MpesT kmuehlbauer 5821660 2022-10-20T18:01:20Z 2022-10-20T18:01:20Z MEMBER

And on the variable? Please also have a look at the docs on to_netcdf regarding encoding-kwarg.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Cannot Save NetCDF: Conflicting _FillValue and Missing_Value  1416709246
1285892163 https://github.com/pydata/xarray/issues/7191#issuecomment-1285892163 https://api.github.com/repos/pydata/xarray/issues/7191 IC_kwDOAMm_X85MpShD WillyChap 14932329 2022-10-20T17:14:24Z 2022-10-20T17:14:24Z NONE

@kmuehlbauer it looks like encoding is empty.

```python print(DS_cera0.encoding)

{}

```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Cannot Save NetCDF: Conflicting _FillValue and Missing_Value  1416709246
1285866801 https://github.com/pydata/xarray/issues/7191#issuecomment-1285866801 https://api.github.com/repos/pydata/xarray/issues/7191 IC_kwDOAMm_X85MpMUx dopplershift 221526 2022-10-20T16:51:21Z 2022-10-20T16:51:21Z CONTRIBUTOR

I'm also not sure why _FillValue and missing_value should be required to have the same value.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Cannot Save NetCDF: Conflicting _FillValue and Missing_Value  1416709246
1285656024 https://github.com/pydata/xarray/issues/7191#issuecomment-1285656024 https://api.github.com/repos/pydata/xarray/issues/7191 IC_kwDOAMm_X85MoY3Y kmuehlbauer 5821660 2022-10-20T14:33:44Z 2022-10-20T14:34:07Z MEMBER

@WillyChap Could you inspect the contents of .encoding. IIRC that values are located there. Or you would need to specify the encoding kwarg in to_netcdf.

Disclaimer: This is from the top of my head, so mistakes are likely.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Cannot Save NetCDF: Conflicting _FillValue and Missing_Value  1416709246

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 12.922ms · About: xarray-datasette