issue_comments

Table actions

4 rows where issue = 292231408 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 2

author_association 2

issue 1

  • test decoding num_dates in float types · 4 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
362462705 https://github.com/pydata/xarray/pull/1863#issuecomment-362462705 https://api.github.com/repos/pydata/xarray/issues/1863 MDEyOklzc3VlQ29tbWVudDM2MjQ2MjcwNQ== shoyer 1217238 2018-02-02T02:01:43Z 2018-02-02T02:01:43Z MEMBER

Nice, thank you @j08lue for tracking this down and adding tests!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test decoding num_dates in float types 292231408
361091073 https://github.com/pydata/xarray/pull/1863#issuecomment-361091073 https://api.github.com/repos/pydata/xarray/issues/1863 MDEyOklzc3VlQ29tbWVudDM2MTA5MTA3Mw== j08lue 3404817 2018-01-28T19:59:23Z 2018-01-28T19:59:23Z CONTRIBUTOR

Store _NS_PER_TIME_DELTA values as int, then numpy will do the casting.

Haha, OK, this is actually what you implemented in https://github.com/pydata/xarray/commit/50b0a69a7aa0fb7ac3afb28e7bd971cf08055f99:

https://github.com/pydata/xarray/blob/50b0a69a7aa0fb7ac3afb28e7bd971cf08055f99/xarray/coding/times.py#L32-L37

So that is why it works now.

Case closed, I guess.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test decoding num_dates in float types 292231408
361089977 https://github.com/pydata/xarray/pull/1863#issuecomment-361089977 https://api.github.com/repos/pydata/xarray/issues/1863 MDEyOklzc3VlQ29tbWVudDM2MTA4OTk3Nw== j08lue 3404817 2018-01-28T19:45:49Z 2018-01-28T19:48:09Z CONTRIBUTOR

I believe the issue originates in these lines:

https://github.com/pydata/xarray/blob/ac854f081d4b57d292755d3aff1476f8e2e2da11/xarray/conventions.py#L174-L175

where we multiply the num_dates with some float value and then cast to int64.

If num_dates is float32, numpy keeps float32 when multiplying with e.g. 1e9 and that somehow introduces an error. Here is a stripped version of the above:

python flat_num_dates = np.arange(100).astype('float32') n = 1e9 roundtripped = (flat_num_dates * n).astype(np.int64) / n assert np.all(flat_num_dates == roundtripped)

By the way, the factor has to be large, like 1e9. E.g. 1e6 ('ms since ...') won't give the error.

The weird thing is that the corresponding code in the current master is identical:

https://github.com/pydata/xarray/blob/50b0a69a7aa0fb7ac3afb28e7bd971cf08055f99/xarray/coding/times.py#L151-L152

I will look into why the result is still different from v0.10.0.

Also, if this really is the origin of the error, there are two easy ways to avoid this:

  1. Cast flat_num_dates to float64: (flat_num_dates.astype(np.float64) * n).astype(np.int64)
  2. Store _NS_PER_TIME_DELTA values as int, then numpy will do the casting.
 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test decoding num_dates in float types 292231408
361089805 https://github.com/pydata/xarray/pull/1863#issuecomment-361089805 https://api.github.com/repos/pydata/xarray/issues/1863 MDEyOklzc3VlQ29tbWVudDM2MTA4OTgwNQ== j08lue 3404817 2018-01-28T19:43:21Z 2018-01-28T19:43:21Z CONTRIBUTOR

For comparison, I added the same test to the 0.10.0 version in https://github.com/j08lue/xarray/tree/0p10-f4-time-decode, where it fails:

``` pytest -v xarray\tests\test_conventions.py

...

          self.assertArrayEqual(expected, actual_cmp)

E AssertionError: E Arrays are not equal E E (mismatch 90.0%) E x: array([datetime.datetime(2000, 1, 1, 0, 0), E datetime.datetime(2000, 1, 2, 0, 0), E datetime.datetime(2000, 1, 3, 0, 0),... E y: array(['2000-01-01T00:00:00.000000', '2000-01-02T00:00:00.003211', E '2000-01-03T00:00:00.006422', '2000-01-04T00:00:00.001245', E '2000-01-05T00:00:00.012845', '2000-01-06T00:00:00.024444',... ```

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test decoding num_dates in float types 292231408

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);