issue_comments

Table actions

3 rows where issue = 1157163377 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

author_association 2

issue 1

  • 'numpy.datetime64' object has no attribute 'year' when writing to zarr or netcdf · 3 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1059832700 https://github.com/pydata/xarray/issues/6318#issuecomment-1059832700 https://api.github.com/repos/pydata/xarray/issues/6318 IC_kwDOAMm_X84_K8N8 spencerkclark 6628425 2022-03-05T21:19:17Z 2022-03-05T21:19:17Z MEMBER

I did a little more digging. I'm not a backend expert, but I think the issue can be distilled to the following. When passed an ordinary NumPy array as an input, xarray.Variable will automatically cast any np.datetime64 values to nanosecond precision: ``` In [3]: arr = np.array([np.datetime64("2000-01-01", "us")])

In [4]: xarray.Variable(["time"], arr) Out[4]: <xarray.Variable (time: 1)> array(['2000-01-01T00:00:00.000000000'], dtype='datetime64[ns]') However, if passed a `LazilyIndexedArray`, this casting will not occur (note the dtype is still `'datetime64[us]'`): In [5]: lazily_indexed_arr = xarray.core.indexing.LazilyIndexedArray(arr)

In [6]: xarray.Variable(["time"], lazily_indexed_arr) Out[6]: <xarray.Variable (time: 1)> array(['2000-01-01T00:00:00.000000'], dtype='datetime64[us]') `` This is based on [this code](https://github.com/pydata/xarray/blob/f42ac28629b7b2047f859f291e1d755c36f2e834/xarray/backends/zarr.py#L404-L419) withinxarray.backends.zarr.ZarrStore`.

The casting does not occur in the Variable constructor, because it requires the type of the array be np.ndarray -- see here. Regardless, even if we relaxed that, _possibly_convert_objects would raise an error, because it is not compatible with LazilyIndexedArray objects.

Is this something that we maybe need to address within the xarray zarr backend? As I understand it, zarr is a bit unusual compared to other storage formats we deal with in that it can store and open np.datetime64 data directly -- normally datetime data starts as some numeric type and is converted to datetime data after going through our decoding pipeline.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   'numpy.datetime64' object has no attribute 'year' when writing to zarr or netcdf 1157163377
1056924176 https://github.com/pydata/xarray/issues/6318#issuecomment-1056924176 https://api.github.com/repos/pydata/xarray/issues/6318 IC_kwDOAMm_X84-_2IQ spencerkclark 6628425 2022-03-02T13:20:23Z 2022-03-02T13:20:23Z MEMBER

To be honest I didn't know it was possible to open a Dataset and maintain np.datetime64[us] values. I feel like casting maybe should occur automatically -- we do this already in other contexts, e.g.: ``` In [3]: da = xr.DataArray([np.datetime64("2000-01-01", "us")], dims=["time"], name="time")

In [4]: da Out[4]: <xarray.DataArray 'time' (time: 1)> array(['2000-01-01T00:00:00.000000000'], dtype='datetime64[ns]') Dimensions without coordinates: time ``` I'll try and dig deeper into this in the next few days, but @keewis's workaround should be good in the meantime.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   'numpy.datetime64' object has no attribute 'year' when writing to zarr or netcdf 1157163377
1056917100 https://github.com/pydata/xarray/issues/6318#issuecomment-1056917100 https://api.github.com/repos/pydata/xarray/issues/6318 IC_kwDOAMm_X84-_0Zs rsignell-usgs 1872600 2022-03-02T13:13:24Z 2022-03-02T13:14:40Z NONE

While I was typing this, @keewis provided a workaround here: https://github.com/fsspec/kerchunk/issues/130#issuecomment-1056897730 ! Leaving this open until I know whether this is something best left for users to implement or something to be handled in xarray. #6318 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   'numpy.datetime64' object has no attribute 'year' when writing to zarr or netcdf 1157163377

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);