id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 1933496257,I_kwDOAMm_X85zPs_B,8288,New library for handling complex numbers in netCDF,1486942,closed,0,,,7,2023-10-09T17:21:20Z,2024-03-25T13:16:40Z,2024-03-25T13:16:40Z,CONTRIBUTOR,,,,"### Is your feature request related to a problem? Complex numbers are not natively handled by netCDF, and currently only the `h5netcdf` engine in xarray can handle them. However, it produces files unreadable by current versions of netCDF. There are also [many different conventions](https://github.com/PlasmaFAIR/nc-complex/blob/main/README.md#conventions-for-complex-numbers), and `h5netcdf` can only read a few of them. ### Describe the solution you'd like I've written a [drop-in extension to netCDF](https://github.com/PlasmaFAIR/nc-complex) that seamlessly handles complex numbers, and can read many different conventions. The Python API is built on top of netCDF4-python, so only a very simple change to xarray is needed to use it: ```diff modified xarray/backends/api.py @@ -115,7 +115,10 @@ def _get_default_engine_gz() -> Literal[""scipy""]: def _get_default_engine_netcdf() -> Literal[""netcdf4"", ""scipy""]: engine: Literal[""netcdf4"", ""scipy""] try: - import netCDF4 # noqa: F401 + try: + import nc_complex # noqa: F401 + except ImportError: + import netCDF4 # noqa: F401 engine = ""netcdf4"" except ImportError: # pragma: no cover modified xarray/backends/netCDF4_.py @@ -327,7 +327,10 @@ class NetCDF4DataStore(WritableCFDataStore): def __init__( self, manager, group=None, mode=None, lock=NETCDF4_PYTHON_LOCK, autoclose=False ): - import netCDF4 + try: + import nc_complex as netCDF4 + except ImportError: + import netCDF4 if isinstance(manager, netCDF4.Dataset): if group is None: @@ -364,7 +367,10 @@ class NetCDF4DataStore(WritableCFDataStore): lock_maker=None, autoclose=False, ): - import netCDF4 + try: + import nc_complex as netCDF4 + except ImportError: + import netCDF4 if isinstance(filename, os.PathLike): filename = os.fspath(filename) ``` and then it works out of the box: ```python import numpy as np import xarray as xr complex_array = np.array([0 + 0j, 1 + 0j, 0 + 1j, 1 + 1j, 0.25 + 0.75j], dtype=""c16"") ds = xr.Dataset({""complex_data"": (""x"", complex_array)}) ds.to_netcdf(""test.nc"") with xr.open_dataset(""test.nc"") as df: print(df.complex_data) assert ds == df # # [5 values with dtype=complex128] # Dimensions without coordinates: x ``` If the library is built against the latest `main` of netCDF-C, then it also supports files written by `h5netcdf`. And while the package isn't on PyPI yet, [built wheels with this feature are available from CI jobs](https://github.com/PlasmaFAIR/nc-complex/actions/runs/6459506996). `nc_complex` is not quite ready yet, but I wanted to gauge interest for adding it as an optional replacement for `netCDF4` in xarray. ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8288/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1952720792,PR_kwDOAMm_X85dTIJ7,8338,"Enable subclassing the netCDF4 backend, changing the dataset class",1486942,closed,0,,,4,2023-10-19T17:09:22Z,2023-12-04T17:16:50Z,2023-12-04T17:16:49Z,CONTRIBUTOR,,0,pydata/xarray/pulls/8338,"See #8288 - [ ] Closes #xxxx - [ ] Tests added - [ ] User visible changes (including notable bug fixes) are documented in `whats-new.rst` - [ ] New functions/methods are listed in `api.rst` These changes allow me to make a pretty minimal subclass that just swaps out the `netCDF4.Dataset` class for my version: ```python from xarray.backends.netCDF4_ import NetCDF4BackendEntrypoint, NetCDF4DataStore from ._wrapper import Dataset # This is my netCDF extension class NcComplexDataStore(NetCDF4DataStore): @classmethod def DatasetClass(self) -> type: return Dataset class NcComplexBackendEntrypoint(NetCDF4BackendEntrypoint): DataStore = NcComplexDataStore ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8338/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 1949710368,PR_kwDOAMm_X85dI74W,8331,Docs: Add syntax for registering backends in `pyproject.toml`,1486942,closed,0,,,3,2023-10-18T13:06:55Z,2023-10-19T12:47:45Z,2023-10-19T12:47:34Z,CONTRIBUTOR,,0,pydata/xarray/pulls/8331,"Uses `sphinx-inline-tabs` for pretty and compact options. Also reword surrounding text a little for clarity. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8331/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 1807517529,PR_kwDOAMm_X85VqE9h,7995,Use variable name in all exceptions raised in `as_variable`,1486942,closed,0,,,5,2023-07-17T11:03:32Z,2023-07-17T15:33:50Z,2023-07-17T14:59:39Z,CONTRIBUTOR,,0,pydata/xarray/pulls/7995,"This is more consistent with the other exceptions raised in this function, and helps to more quickly diagnose issues when e.g. creating datasets. - [x] User visible changes (including notable bug fixes) are documented in `whats-new.rst` Is this a large enough change to warrant an entry in `whats-new`?","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7995/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull