id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 1643408278,I_kwDOAMm_X85h9GuW,7691,`nan` values appearing when saving and loading from `netCDF` due to encoding,42553970,closed,0,,,11,2023-03-28T07:58:21Z,2024-03-15T16:31:06Z,2024-03-15T16:31:05Z,NONE,,,,"### What happened? When writing to and reading my dataset from `netCDF` using `ds.to_netcdf()` and `xr.open_dataset(...)`, `xarray` creates `nan` values where previously number values (`float32`) where. The issue seems related to the `encoding` used for the original dataset, which causes the data to be stored as `short`. During loading, the stored values then collide with `_FillValue` leading to the numbers being interpreted as `nan`. ### What did you expect to happen? Values after saving & loading should be the same as before saving. ### Minimal Complete Verifiable Example ```Python We had a back-and-forth on SO about this, I hope it's fine to just refer to it here: https://stackoverflow.com/a/75806771/11318472 ``` ### MVCE confirmation - [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [ ] Complete example — the example is self-contained, including all data and the text of any traceback. - [ ] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [X] New issue — a search of GitHub Issues suggests this is not a duplicate. ### Relevant log output ```Python See the SO link above. ``` ### Anything else we need to know? I'm not sure whether this should be considered a bug or just a combination of conflicting features. My current workaround is resetting the `encoding` and letting `xarray` decide to store as `float` instead of `short` (cf. https://github.com/pydata/xarray/issues/7686). ### Environment
INSTALLED VERSIONS ------------------ commit: None python: 3.11.0 | packaged by conda-forge | (main, Oct 25 2022, 06:24:40) [GCC 10.4.0] python-bits: 64 OS: Linux OS-release: 5.15.90.1-microsoft-standard-WSL2 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: C.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.8.1 xarray: 2022.11.0 pandas: 1.5.2 numpy: 1.23.5 scipy: 1.10.0 netCDF4: 1.6.2 pydap: None h5netcdf: 1.1.0 h5py: 3.8.0 Nio: None zarr: 2.13.6 cftime: 1.6.2 nc_time_axis: None PseudoNetCDF: None rasterio: 1.3.3 cfgrib: None iris: None bottleneck: 1.3.5 dask: 2022.02.1 distributed: 2022.2.1 matplotlib: 3.6.2 cartopy: None seaborn: None numbagg: None fsspec: 2022.11.0 cupy: None pint: None sparse: None flox: None numpy_groupies: None setuptools: 65.5.1 pip: 22.3.1 conda: None pytest: 7.2.0 IPython: 8.11.0 sphinx: None
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7691/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue