home / github / issue_comments

Menu
  • Search all tables
  • GraphQL API

issue_comments: 1059832700

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/6318#issuecomment-1059832700 https://api.github.com/repos/pydata/xarray/issues/6318 1059832700 IC_kwDOAMm_X84_K8N8 6628425 2022-03-05T21:19:17Z 2022-03-05T21:19:17Z MEMBER

I did a little more digging. I'm not a backend expert, but I think the issue can be distilled to the following. When passed an ordinary NumPy array as an input, xarray.Variable will automatically cast any np.datetime64 values to nanosecond precision: ``` In [3]: arr = np.array([np.datetime64("2000-01-01", "us")])

In [4]: xarray.Variable(["time"], arr) Out[4]: <xarray.Variable (time: 1)> array(['2000-01-01T00:00:00.000000000'], dtype='datetime64[ns]') However, if passed a `LazilyIndexedArray`, this casting will not occur (note the dtype is still `'datetime64[us]'`): In [5]: lazily_indexed_arr = xarray.core.indexing.LazilyIndexedArray(arr)

In [6]: xarray.Variable(["time"], lazily_indexed_arr) Out[6]: <xarray.Variable (time: 1)> array(['2000-01-01T00:00:00.000000'], dtype='datetime64[us]') `` This is based on [this code](https://github.com/pydata/xarray/blob/f42ac28629b7b2047f859f291e1d755c36f2e834/xarray/backends/zarr.py#L404-L419) withinxarray.backends.zarr.ZarrStore`.

The casting does not occur in the Variable constructor, because it requires the type of the array be np.ndarray -- see here. Regardless, even if we relaxed that, _possibly_convert_objects would raise an error, because it is not compatible with LazilyIndexedArray objects.

Is this something that we maybe need to address within the xarray zarr backend? As I understand it, zarr is a bit unusual compared to other storage formats we deal with in that it can store and open np.datetime64 data directly -- normally datetime data starts as some numeric type and is converted to datetime data after going through our decoding pipeline.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  1157163377
Powered by Datasette · Queries took 0.89ms · About: xarray-datasette