issue_comments

Yes, this will be in the next release. (Which will hopefully be very soon!)

Sorry for the slow response, I have little time at the moment. The option invalid_netcdf=True is not yet in the latest release, is it? I get an TypeError. I would have to use a manually installed version of xarray to use it, right?

I opened an issue to discuss this in the CF convention issue tracker -- let's see what they think: https://github.com/cf-convention/cf-conventions/issues/204

I am in the exact same situation. @DerWeh with the current master you can do da.to_netcdf("complex.nc", engine="h5netcdf", invalid_netcdf=True) which works for me until there is engine="hdf5" or may be a method da.to_hdf()?

It might make sense to implement engine=“hdf5” as an alias for engine=“h5netcdf” with invalid_netcdf=True. It would certainly be a more ergonomic API.

I think the answer here is to use h5netcdf until a proper hdf5 backend is created.

It would be nice to add this to the documentation and mention h5netcdf more generally under https://xarray.pydata.org/en/stable/io.html . @DerWeh Are you up for sending in a PR?

I agree that including it in NetCDF is the 'most sane' approach. I don't really know how much work it is, expanding the standard.

To be honest, I don't really care about NetCDF, for me xarray is just an incredible good way to make code more stable and readable (though it still has several usability issues). In my community everyone uses HDF5 anyway, so dropping compatibility is no big issue. I just want a way to persist data as it is and conveniently load it for plotting and post processing.

I would still encourage you to push saving of complex data. In most fields people use complex data and it is hard to convince them that they benefit from this great library, if saving simple data takes complicated keyword arguments and annoys you with warnings compared to a simple np.savez on regular ndarrays.

My 2 cents:

For a proper resolution, I'd rather have the topic discussed with the NetCDF specs maintainers, so that NetCDF can just be expanded to support the same structure like HDF5. Once the format is standard, it would then be a trivial PR to h5netcdf to suppress the warning. We've already gone through the exact same process for compression algorithms other than gzip. Adding the functionality to the NetCDF C library and the python wrapper would be a completely different order of magnitude of work.

Another good alternative is to use h5netcdf forcing the malformation through, and just call the file .h5 instead of .nc 😉

If you really need to interact with (non-Python) people that are stuck on the NetCDF C library, and who for reasons I can't imagine can't switch to the HDF5 C library, I think writing two bespoke pre/postprocess functions in your code to add a dimension is the best approach.