html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/5223#issuecomment-831966730,https://api.github.com/repos/pydata/xarray/issues/5223,831966730,MDEyOklzc3VlQ29tbWVudDgzMTk2NjczMA==,6503378,2021-05-04T14:02:32Z,2021-05-04T14:02:32Z,NONE,"> I think that we can then either close this issue or move it to Xarray-simlab.
Ok. I'll prepare a better example, will try to come up with a suggestion and create an issue over there.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,868907284
https://github.com/pydata/xarray/issues/5223#issuecomment-831263359,https://api.github.com/repos/pydata/xarray/issues/5223,831263359,MDEyOklzc3VlQ29tbWVudDgzMTI2MzM1OQ==,6503378,2021-05-03T13:35:09Z,2021-05-03T13:36:32Z,NONE,"> If you reset the encoding in your notebook example, e.g., `ds_out_no_scale.p__var.encoding = {}` you will be able to save the Dataset to a netcdf4 file. Not sure why...
Yes, thank you @benbovy , that works for me as well as a work-around.
However .. (excuse my stubbornness):
I tried writing a simple array from ds -> zarr -> nc -> ds. Surprisingly everything works fine. Even the fill value is there (with and w/o mask_and_scale). It might be a xarray simlab issue and something goes wrong when writing/reading to/from zarr. Should I create an issue over there and close this one?
https://github.com/jvail/xarray-simlab/blob/test_encoding_netcdf/notebooks/ds_to_zarr_to_nc_to_ds.ipynb
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,868907284
https://github.com/pydata/xarray/issues/5223#issuecomment-829023263,https://api.github.com/repos/pydata/xarray/issues/5223,829023263,MDEyOklzc3VlQ29tbWVudDgyOTAyMzI2Mw==,6503378,2021-04-29T07:59:55Z,2021-05-02T09:03:40Z,NONE,"> @jvail could you provide a small reproducible example?
Sure, thank you. I'll put a notebook together
I hope this helps:
https://github.com/jvail/xarray-simlab/blob/test_encoding_netcdf/notebooks/test_encoding_netcdf.ipynb
---- edit
If I serialize to a dict after popping the `_FillValue` and then back it seems to work (both the datetime and NaT encoding) unless the datetime64 variable has at least one none 'Nat'.
`xr.Dataset.from_dict(ds.to_dict()).to_netcdf('nc', engine='netcdf4')`
Removing the entire encoding obj from the variable with dtype datetime64 seems to work as well. However, no idea, if that is a good idea.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,868907284
https://github.com/pydata/xarray/issues/5223#issuecomment-829013469,https://api.github.com/repos/pydata/xarray/issues/5223,829013469,MDEyOklzc3VlQ29tbWVudDgyOTAxMzQ2OQ==,6503378,2021-04-29T07:44:58Z,2021-04-29T07:44:58Z,NONE,"> I guess if I manually drop the `attrs._FillValue` writing netcdf should work.
Hm, now I get `ValueError: unsupported dtype for netCDF4 variable: datetime64[ns]` :) I'll see if I can figure out where the problem is. Maybe in zarr since I can write an identical Dataset to netcdf without error. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,868907284
https://github.com/pydata/xarray/issues/5223#issuecomment-828410285,https://api.github.com/repos/pydata/xarray/issues/5223,828410285,MDEyOklzc3VlQ29tbWVudDgyODQxMDI4NQ==,6503378,2021-04-28T12:23:02Z,2021-04-28T12:23:02Z,NONE,"> Opened #5226
Thank you @benbovy! I could not have pinned down the issue myself. I guess if I manually drop the `attrs._FillValue` writing netcdf should work.
Not sure if this should be closed or maybe moved to xarray-simlab?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,868907284
https://github.com/pydata/xarray/issues/5223#issuecomment-828171777,https://api.github.com/repos/pydata/xarray/issues/5223,828171777,MDEyOklzc3VlQ29tbWVudDgyODE3MTc3Nw==,6503378,2021-04-28T06:07:33Z,2021-04-28T06:07:56Z,NONE,"> Does putting it on `.encoding` solve the immediate issue?
The issue is gone if I have neither an `attrs._FillValue` nor `encoding._FillValue` or just `encoding._FillValue`. The issue comes with `attrs._FillValue`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,868907284
https://github.com/pydata/xarray/issues/5223#issuecomment-828159821,https://api.github.com/repos/pydata/xarray/issues/5223,828159821,MDEyOklzc3VlQ29tbWVudDgyODE1OTgyMQ==,6503378,2021-04-28T05:40:46Z,2021-04-28T05:40:46Z,NONE,"> Please could I ask once more — forgive me if I'm missing something but I did ask this a week ago and still don't understand [#5200 (comment)](https://github.com/pydata/xarray/discussions/5200#discussioncomment-638329):
>
> > Is there a specific reason _FillValue needs to be in the attrs? (I'm not a big netcdf user so there may be)
Sorry, overlooked that one. I'll dig in the simlab code.
> > Why is xarray not ignoring the _FillValue in attrs?
>
> It's trying to serialize it, as it would any other attrs.
Ok, that would make sense if '_FillValue' would just be an ordinary attr. But it seems it is not. If I set it to a string 'NaT' netcdf complains.
```
lib/python3.8/site-packages/scipy/io/netcdf.py in _get_encoded_fill_value(self)
1030 """"""
1031 if '_FillValue' in self._attributes:
-> 1032 fill_value = np.array(self._attributes['_FillValue'],
1033 dtype=self.data.dtype).tobytes()
1034 if len(fill_value) == self.itemsize():
ValueError: could not convert string to float: b'NaT'
```
If I set it to 0 I do not get an error. But then - after reloading the netcdf file - my NaT becomes `array(['1970-01-01T00:00:00.000000000'], dtype='datetime64[ns]')`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,868907284
https://github.com/pydata/xarray/issues/5223#issuecomment-828151911,https://api.github.com/repos/pydata/xarray/issues/5223,828151911,MDEyOklzc3VlQ29tbWVudDgyODE1MTkxMQ==,6503378,2021-04-28T05:20:50Z,2021-04-28T05:22:00Z,NONE,"Re-Hello and thank you,
hm, so the proper way (if necessary at all) would be to set
`da.encoding = {'_FillValue': np.datetime64('NAT')}`
rather than putting it in `attrs`?
Maybe I make a mistake in xarray-simlab and I should set the encoding explicitly and my confusion comes from having `_FillValue` in `attrs` and `encoding`. But - as far as I can see - in the xarray docs for writing netcdf the encoding `_FillValue` is mentioned.
Why is xarray not ignoring the `_FillValue` in `attrs`? ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,868907284