issue_comments

Table actions

2 rows where author_association = "CONTRIBUTOR" and issue = 924676925 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

issue 1

  • Nan/ changed values in output when only reading data, saving and reading again · 2 ✖

author_association 1

  • CONTRIBUTOR · 2 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
863972083 https://github.com/pydata/xarray/issues/5490#issuecomment-863972083 https://api.github.com/repos/pydata/xarray/issues/5490 MDEyOklzc3VlQ29tbWVudDg2Mzk3MjA4Mw== d70-t 6574622 2021-06-18T11:32:38Z 2021-06-18T11:33:14Z CONTRIBUTOR

I've checked your example files. This is mostly related to the fact, that the original data is encoded as short and uses scale_factor and add_offset: python In [35]: ds_loc.q.encoding Out[35]: {'source': '/private/tmp/test_xarray/Minimal_test_data/2012_europe_9_130_131_132_133_135.nc', 'original_shape': (720, 26, 36, 41), 'dtype': dtype('int16'), 'missing_value': -32767, '_FillValue': -32767, 'scale_factor': 3.0672840096982675e-07, 'add_offset': 0.010050721147263318}

Probably the scaling and adding is carried out in float64, but then rounded down to float32. When storing the dataset back to netCDF, xarray re-uses the information from the encoding attribute and goes back to int16, possibly creating even more rounding errors. Reading the data back in is then not reproducible anymore.

Possibly related issues are #4826 and #3020

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Nan/ changed values in output when only reading data, saving and reading again 924676925
863945975 https://github.com/pydata/xarray/issues/5490#issuecomment-863945975 https://api.github.com/repos/pydata/xarray/issues/5490 MDEyOklzc3VlQ29tbWVudDg2Mzk0NTk3NQ== d70-t 6574622 2021-06-18T10:44:38Z 2021-06-18T10:44:38Z CONTRIBUTOR

Are your input files on (exactly) the same grid? If not, combining the files might introduce NaN to fill up missmatching cells. Furthemore, if you are working with NaNs, are you aware of: ```python In [1]: import numpy as np

In [2]: np.nan == np.nan Out[2]: False ``` Which is as it should be per IEEE 754.

When writing out the files to netCDF, do you accidentally convert from 64bit float to 32bit float?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Nan/ changed values in output when only reading data, saving and reading again 924676925

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);