issue_comments

Table actions

3 rows where issue = 963006707 and user = 1217238 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • shoyer · 3 ✖

issue 1

  • ENH: Add default fill values for decode_cf · 3 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
895508489 https://github.com/pydata/xarray/pull/5680#issuecomment-895508489 https://api.github.com/repos/pydata/xarray/issues/5680 IC_kwDOAMm_X841YGAJ shoyer 1217238 2021-08-09T20:11:24Z 2021-08-09T20:11:24Z MEMBER

To follow up, from a practical perspective, there are two problems with assuming that there are always "truly missing values" (case 2):

  1. It makes it impossible to represent the full range of values in a data type, e.g., 255 for uint8 now means "missing".
  2. Due to unfortunately limited options for representing missing data in NumPy, Xarray represents truly missing values in its data model with "NaN". This is more or less OK for floating point data, but means that integer data gets converted into floats. For example, uint8 would now get automatically converted into float32.

Both of these issues are problematic for faithful "round tripping" of Xarray data into netCDF and back. For this reason, Xarray needs an unambiguous way to know if a netCDF variable could contain semantically missing values. So far, we've used the presence of missing_value and _FillValue attributes for that.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   ENH: Add default fill values for decode_cf 963006707
895455163 https://github.com/pydata/xarray/pull/5680#issuecomment-895455163 https://api.github.com/repos/pydata/xarray/issues/5680 IC_kwDOAMm_X841X4-7 shoyer 1217238 2021-08-09T18:46:59Z 2021-08-09T18:46:59Z MEMBER

Right, so netCDF3 has a default value used for filling out variables before any data is written.

My concern is that there are two (overlapping) use-case for fill values:

  1. The default array value used for variables on disk, e.g., before they are written
  2. Truly missing values (with different semantics), which Xarray represents with NaN

Certainly these sometimes coincide, but that isn't necessarily the case.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   ENH: Add default fill values for decode_cf 963006707
895022291 https://github.com/pydata/xarray/pull/5680#issuecomment-895022291 https://api.github.com/repos/pydata/xarray/issues/5680 IC_kwDOAMm_X841WPTT shoyer 1217238 2021-08-09T07:54:22Z 2021-08-09T07:54:22Z MEMBER

Could you clarify where these default fill values come from?

Are they just an arbitrary choice by netCDF4-Python? Or are they part of some broader standard?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   ENH: Add default fill values for decode_cf 963006707

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);