issues
1 row where repo = 13221727, type = "issue" and user = 14983768 sorted by updated_at descending
This data as json, CSV (advanced)
Suggested facets: created_at (date), updated_at (date), closed_at (date)
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at ▲ | closed_at | author_association | active_lock_reason | draft | pull_request | body | reactions | performed_via_github_app | state_reason | repo | type |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1685803922 | I_kwDOAMm_X85ke1OS | 7790 | Fill values in time arrays (numpy.datetime64) are lost in zarr | christine-e-smit 14983768 | closed | 0 | 25 | 2023-04-26T22:03:06Z | 2023-09-17T08:15:28Z | 2023-09-17T08:15:28Z | NONE | What happened?I have a time array of type numpy.datetime64 with fill values. When I save the dateset I created with xarray to zarr and then read that zarr store back out again with xarray, the fill values are lost. What did you expect to happen?I expected my fill values to still be in my time array when I read it back out of the zarr store. Minimal Complete Verifiable Example```Python import numpy as np import xarray as xr import zarr Create a numpy array of type np.datetime64 with one fill value and one datetime_fill_value = np.datetime64("NaT") time = np.array([time_fill_value,'2023-01-02'],dtype='M8[ns]') Create a dataset with this one arrayxr_time_array = xr.DataArray(data=time,dims=['time'],name='time') xr_ds = xr.Dataset(dict(time=xr_time_array)) print("******") print("Created with fill value (NaT)") print(xr_ds["time"]) Save the dataset to zarrlocation = "from_xarray.zarr" encoding = { "time":{"_FillValue":time_fill_value} } xr_ds.to_zarr(location,encoding=encoding) xr_read = xr.open_zarr(location) print("******") print("No fill value") print(xr_read["time"]) ``` MVCE confirmation
Relevant log outputNo response Anything else we need to know?When I look in the .zmetadata file generated for this zarr store, I see that the time array as been converted to float and there is a units attribute:
If I open the data using zarr, the actual values in the time array look fine, given the transformation: ```python z = zarr.open(location,'r') print(f'Fill value: {z["time"].fill_value}')
print(f'Data values: {z["time"][:]}')
Environment
INSTALLED VERSIONS
------------------
commit: None
python: 3.11.3 | packaged by conda-forge | (main, Apr 6 2023, 08:58:31) [Clang 14.0.6 ]
python-bits: 64
OS: Darwin
OS-release: 22.4.0
machine: arm64
processor: arm
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: None
libnetcdf: None
xarray: 2023.4.2
pandas: 2.0.1
numpy: 1.24.3
scipy: None
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: 2.14.2
cftime: None
nc_time_axis: None
PseudoNetCDF: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 67.7.2
pip: 23.1.2
conda: None
pytest: None
mypy: None
IPython: 8.12.0
sphinx: None
|
{ "url": "https://api.github.com/repos/pydata/xarray/issues/7790/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | xarray 13221727 | issue |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issues] ( [id] INTEGER PRIMARY KEY, [node_id] TEXT, [number] INTEGER, [title] TEXT, [user] INTEGER REFERENCES [users]([id]), [state] TEXT, [locked] INTEGER, [assignee] INTEGER REFERENCES [users]([id]), [milestone] INTEGER REFERENCES [milestones]([id]), [comments] INTEGER, [created_at] TEXT, [updated_at] TEXT, [closed_at] TEXT, [author_association] TEXT, [active_lock_reason] TEXT, [draft] INTEGER, [pull_request] TEXT, [body] TEXT, [reactions] TEXT, [performed_via_github_app] TEXT, [state_reason] TEXT, [repo] INTEGER REFERENCES [repos]([id]), [type] TEXT ); CREATE INDEX [idx_issues_repo] ON [issues] ([repo]); CREATE INDEX [idx_issues_milestone] ON [issues] ([milestone]); CREATE INDEX [idx_issues_assignee] ON [issues] ([assignee]); CREATE INDEX [idx_issues_user] ON [issues] ([user]);