issues: 1685803922
This data as json
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | active_lock_reason | draft | pull_request | body | reactions | performed_via_github_app | state_reason | repo | type |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1685803922 | I_kwDOAMm_X85ke1OS | 7790 | Fill values in time arrays (numpy.datetime64) are lost in zarr | 14983768 | closed | 0 | 25 | 2023-04-26T22:03:06Z | 2023-09-17T08:15:28Z | 2023-09-17T08:15:28Z | NONE | What happened?I have a time array of type numpy.datetime64 with fill values. When I save the dateset I created with xarray to zarr and then read that zarr store back out again with xarray, the fill values are lost. What did you expect to happen?I expected my fill values to still be in my time array when I read it back out of the zarr store. Minimal Complete Verifiable Example```Python import numpy as np import xarray as xr import zarr Create a numpy array of type np.datetime64 with one fill value and one datetime_fill_value = np.datetime64("NaT") time = np.array([time_fill_value,'2023-01-02'],dtype='M8[ns]') Create a dataset with this one arrayxr_time_array = xr.DataArray(data=time,dims=['time'],name='time') xr_ds = xr.Dataset(dict(time=xr_time_array)) print("******") print("Created with fill value (NaT)") print(xr_ds["time"]) Save the dataset to zarrlocation = "from_xarray.zarr" encoding = { "time":{"_FillValue":time_fill_value} } xr_ds.to_zarr(location,encoding=encoding) xr_read = xr.open_zarr(location) print("******") print("No fill value") print(xr_read["time"]) ``` MVCE confirmation
Relevant log outputNo response Anything else we need to know?When I look in the .zmetadata file generated for this zarr store, I see that the time array as been converted to float and there is a units attribute:
If I open the data using zarr, the actual values in the time array look fine, given the transformation: ```python z = zarr.open(location,'r') print(f'Fill value: {z["time"].fill_value}')
print(f'Data values: {z["time"][:]}')
Environment
INSTALLED VERSIONS
------------------
commit: None
python: 3.11.3 | packaged by conda-forge | (main, Apr 6 2023, 08:58:31) [Clang 14.0.6 ]
python-bits: 64
OS: Darwin
OS-release: 22.4.0
machine: arm64
processor: arm
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: None
libnetcdf: None
xarray: 2023.4.2
pandas: 2.0.1
numpy: 1.24.3
scipy: None
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: 2.14.2
cftime: None
nc_time_axis: None
PseudoNetCDF: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 67.7.2
pip: 23.1.2
conda: None
pytest: None
mypy: None
IPython: 8.12.0
sphinx: None
|
{ "url": "https://api.github.com/repos/pydata/xarray/issues/7790/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | 13221727 | issue |