id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 1933496257,I_kwDOAMm_X85zPs_B,8288,New library for handling complex numbers in netCDF,1486942,closed,0,,,7,2023-10-09T17:21:20Z,2024-03-25T13:16:40Z,2024-03-25T13:16:40Z,CONTRIBUTOR,,,,"### Is your feature request related to a problem? Complex numbers are not natively handled by netCDF, and currently only the `h5netcdf` engine in xarray can handle them. However, it produces files unreadable by current versions of netCDF. There are also [many different conventions](https://github.com/PlasmaFAIR/nc-complex/blob/main/README.md#conventions-for-complex-numbers), and `h5netcdf` can only read a few of them. ### Describe the solution you'd like I've written a [drop-in extension to netCDF](https://github.com/PlasmaFAIR/nc-complex) that seamlessly handles complex numbers, and can read many different conventions. The Python API is built on top of netCDF4-python, so only a very simple change to xarray is needed to use it: ```diff modified xarray/backends/api.py @@ -115,7 +115,10 @@ def _get_default_engine_gz() -> Literal[""scipy""]: def _get_default_engine_netcdf() -> Literal[""netcdf4"", ""scipy""]: engine: Literal[""netcdf4"", ""scipy""] try: - import netCDF4 # noqa: F401 + try: + import nc_complex # noqa: F401 + except ImportError: + import netCDF4 # noqa: F401 engine = ""netcdf4"" except ImportError: # pragma: no cover modified xarray/backends/netCDF4_.py @@ -327,7 +327,10 @@ class NetCDF4DataStore(WritableCFDataStore): def __init__( self, manager, group=None, mode=None, lock=NETCDF4_PYTHON_LOCK, autoclose=False ): - import netCDF4 + try: + import nc_complex as netCDF4 + except ImportError: + import netCDF4 if isinstance(manager, netCDF4.Dataset): if group is None: @@ -364,7 +367,10 @@ class NetCDF4DataStore(WritableCFDataStore): lock_maker=None, autoclose=False, ): - import netCDF4 + try: + import nc_complex as netCDF4 + except ImportError: + import netCDF4 if isinstance(filename, os.PathLike): filename = os.fspath(filename) ``` and then it works out of the box: ```python import numpy as np import xarray as xr complex_array = np.array([0 + 0j, 1 + 0j, 0 + 1j, 1 + 1j, 0.25 + 0.75j], dtype=""c16"") ds = xr.Dataset({""complex_data"": (""x"", complex_array)}) ds.to_netcdf(""test.nc"") with xr.open_dataset(""test.nc"") as df: print(df.complex_data) assert ds == df # # [5 values with dtype=complex128] # Dimensions without coordinates: x ``` If the library is built against the latest `main` of netCDF-C, then it also supports files written by `h5netcdf`. And while the package isn't on PyPI yet, [built wheels with this feature are available from CI jobs](https://github.com/PlasmaFAIR/nc-complex/actions/runs/6459506996). `nc_complex` is not quite ready yet, but I wanted to gauge interest for adding it as an optional replacement for `netCDF4` in xarray. ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8288/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue