issue_comments

I think that we can then either close this issue or move it to Xarray-simlab.

Ok. I'll prepare a better example, will try to come up with a suggestion and create an issue over there.

If you reset the encoding in your notebook example, e.g., ds_out_no_scale.p__var.encoding = {} you will be able to save the Dataset to a netcdf4 file. Not sure why...

Yes, thank you @benbovy , that works for me as well as a work-around.

However .. (excuse my stubbornness):

I tried writing a simple array from ds -> zarr -> nc -> ds. Surprisingly everything works fine. Even the fill value is there (with and w/o mask_and_scale). It might be a xarray simlab issue and something goes wrong when writing/reading to/from zarr. Should I create an issue over there and close this one?

https://github.com/jvail/xarray-simlab/blob/test_encoding_netcdf/notebooks/ds_to_zarr_to_nc_to_ds.ipynb

@jvail could you provide a small reproducible example?

Sure, thank you. I'll put a notebook together

I hope this helps: https://github.com/jvail/xarray-simlab/blob/test_encoding_netcdf/notebooks/test_encoding_netcdf.ipynb

---- edit If I serialize to a dict after popping the _FillValue and then back it seems to work (both the datetime and NaT encoding) unless the datetime64 variable has at least one none 'Nat'.

xr.Dataset.from_dict(ds.to_dict()).to_netcdf('nc', engine='netcdf4')

Removing the entire encoding obj from the variable with dtype datetime64 seems to work as well. However, no idea, if that is a good idea.

I guess if I manually drop the attrs._FillValue writing netcdf should work.

Hm, now I get ValueError: unsupported dtype for netCDF4 variable: datetime64[ns] :) I'll see if I can figure out where the problem is. Maybe in zarr since I can write an identical Dataset to netcdf without error.

Opened #5226

Thank you @benbovy! I could not have pinned down the issue myself. I guess if I manually drop the attrs._FillValue writing netcdf should work.

Not sure if this should be closed or maybe moved to xarray-simlab?

Does putting it on .encoding solve the immediate issue?

The issue is gone if I have neither an attrs._FillValue nor encoding._FillValue or just encoding._FillValue. The issue comes with attrs._FillValue.

Please could I ask once more — forgive me if I'm missing something but I did ask this a week ago and still don't understand #5200 (comment):

Is there a specific reason _FillValue needs to be in the attrs? (I'm not a big netcdf user so there may be)

Sorry, overlooked that one. I'll dig in the simlab code.

Why is xarray not ignoring the _FillValue in attrs?

It's trying to serialize it, as it would any other attrs.

Ok, that would make sense if '_FillValue' would just be an ordinary attr. But it seems it is not. If I set it to a string 'NaT' netcdf complains.

``` lib/python3.8/site-packages/scipy/io/netcdf.py in _get_encoded_fill_value(self) 1030 """ 1031 if '_FillValue' in self._attributes: -> 1032 fill_value = np.array(self._attributes['_FillValue'], 1033 dtype=self.data.dtype).tobytes() 1034 if len(fill_value) == self.itemsize():

ValueError: could not convert string to float: b'NaT' ```

If I set it to 0 I do not get an error. But then - after reloading the netcdf file - my NaT becomes array(['1970-01-01T00:00:00.000000000'], dtype='datetime64[ns]')

Re-Hello and thank you,

hm, so the proper way (if necessary at all) would be to set

da.encoding = {'_FillValue': np.datetime64('NAT')}

rather than putting it in attrs?

Maybe I make a mistake in xarray-simlab and I should set the encoding explicitly and my confusion comes from having _FillValue in attrs and encoding. But - as far as I can see - in the xarray docs for writing netcdf the encoding _FillValue is mentioned. Why is xarray not ignoring the _FillValue in attrs?