home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

2 rows where author_association = "MEMBER" and issue = 374070147 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 2

  • shoyer 1
  • spencerkclark 1

issue 1

  • to_netcdf() fails because of datetime encoding · 2 ✖

author_association 1

  • MEMBER · 2 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
433235316 https://github.com/pydata/xarray/issues/2512#issuecomment-433235316 https://api.github.com/repos/pydata/xarray/issues/2512 MDEyOklzc3VlQ29tbWVudDQzMzIzNTMxNg== shoyer 1217238 2018-10-25T23:11:03Z 2018-10-25T23:11:03Z MEMBER

I agree, this is definitely a bug. We do have logic that is supposed to automatically convert datetime object arrays into datetime64, but for some reason it isn't being triggered here: https://github.com/pydata/xarray/blob/2a4691321a3c13baf0a5615f16740435621b153d/xarray/core/variable.py#L182-L184

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  to_netcdf() fails because of datetime encoding 374070147
433194914 https://github.com/pydata/xarray/issues/2512#issuecomment-433194914 https://api.github.com/repos/pydata/xarray/issues/2512 MDEyOklzc3VlQ29tbWVudDQzMzE5NDkxNA== spencerkclark 6628425 2018-10-25T20:30:49Z 2018-10-25T20:30:49Z MEMBER

I think this is a bug. The error message in particular makes things confusing. The problem stems from this line, which results in an array of dtype object (see the explanation below): python dts = np.array([dt + timedelta(days=x) for x in range(10)]) As a temporary workaround, if you keep dts as a list before passing it to the Dataset constructor, things will work as you expect: python dts = [dt + timedelta(days=x) for x in range(10)]


By default NumPy will cast a list of datetime.datetime objects to an array with dtype object: ``` In [4]: dts = np.array([dt + timedelta(days=x) for x in range(10)])

In [5]: dts Out[5]: array([datetime.datetime(1999, 1, 1, 0, 0), datetime.datetime(1999, 1, 2, 0, 0), datetime.datetime(1999, 1, 3, 0, 0), datetime.datetime(1999, 1, 4, 0, 0), datetime.datetime(1999, 1, 5, 0, 0), datetime.datetime(1999, 1, 6, 0, 0), datetime.datetime(1999, 1, 7, 0, 0), datetime.datetime(1999, 1, 8, 0, 0), datetime.datetime(1999, 1, 9, 0, 0), datetime.datetime(1999, 1, 10, 0, 0)], dtype=object) If you specify an array of dtype object as a coordinate, xarray currently has some logic that requires that it be cast to a generic `pandas.Index` with dtype object (and will preserve the `datetime.datetime` elements of the array). In [14]: da = xarray.DataArray(range(10), coords=[dts], dims=['time'])

In [15]: da Out[15]: <xarray.DataArray (time: 10)> array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) Coordinates: * time (time) object 1999-01-01 1999-01-02 1999-01-03 1999-01-04 ...

In [16]: da.indexes['time'] Out[16]: Index([1999-01-01 00:00:00, 1999-01-02 00:00:00, 1999-01-03 00:00:00, 1999-01-04 00:00:00, 1999-01-05 00:00:00, 1999-01-06 00:00:00, 1999-01-07 00:00:00, 1999-01-08 00:00:00, 1999-01-09 00:00:00, 1999-01-10 00:00:00], dtype='object', name='time') The code where this happens is here: https://github.com/pydata/xarray/blob/b2a377f8dd215a0e1d3cbab61e6ae930347f4063/xarray/core/utils.py#L69-L73 In practice, in the event that an object type array contains `datetime.datetime` objects, this should probably convert them to `np.datetime64` instead and return a `pandas.DatetimeIndex`. This would be more consistent with how xarray currently handles passing an object array of `datetime.datetime` objects as the data argument to a DataArray; there it converts things automatically: In [18]: xarray.DataArray(dts) Out[18]: <xarray.DataArray (dim_0: 10)> array(['1999-01-01T00:00:00.000000000', '1999-01-02T00:00:00.000000000', '1999-01-03T00:00:00.000000000', '1999-01-04T00:00:00.000000000', '1999-01-05T00:00:00.000000000', '1999-01-06T00:00:00.000000000', '1999-01-07T00:00:00.000000000', '1999-01-08T00:00:00.000000000', '1999-01-09T00:00:00.000000000', '1999-01-10T00:00:00.000000000'], dtype='datetime64[ns]') Dimensions without coordinates: dim_0 `` Xarray's logic to encode dates (i.e. what is used when saving datetime data to files) requires that dates are either of typenp.datetime64orcftime.datetime(datetime.datetimeobjects are not supported there). Ifdatetime.datetimearrays were automatically converted tonp.datetime64` arrays in all cases then this would not be an issue.

The error message actually results from the dates being converted to np.datetime64 after the datetime encoding logic is skipped (it happens in the Variable constructor here) and netCDF4 doesn't know how to deal with that data type, but I think the fundamental issue relates to the lines of code I referenced above.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  to_netcdf() fails because of datetime encoding 374070147

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 1029.548ms · About: xarray-datasette