home / github / issues

Menu
  • Search all tables
  • GraphQL API

issues: 1933496257

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1933496257 I_kwDOAMm_X85zPs_B 8288 New library for handling complex numbers in netCDF 1486942 closed 0     7 2023-10-09T17:21:20Z 2024-03-25T13:16:40Z 2024-03-25T13:16:40Z CONTRIBUTOR      

Is your feature request related to a problem?

Complex numbers are not natively handled by netCDF, and currently only the h5netcdf engine in xarray can handle them. However, it produces files unreadable by current versions of netCDF. There are also many different conventions, and h5netcdf can only read a few of them.

Describe the solution you'd like

I've written a drop-in extension to netCDF that seamlessly handles complex numbers, and can read many different conventions.

The Python API is built on top of netCDF4-python, so only a very simple change to xarray is needed to use it:

```diff modified xarray/backends/api.py @@ -115,7 +115,10 @@ def _get_default_engine_gz() -> Literal["scipy"]: def _get_default_engine_netcdf() -> Literal["netcdf4", "scipy"]: engine: Literal["netcdf4", "scipy"] try: - import netCDF4 # noqa: F401 + try: + import nc_complex # noqa: F401 + except ImportError: + import netCDF4 # noqa: F401

     engine = "netcdf4"
 except ImportError:  # pragma: no cover

modified xarray/backends/netCDF4_.py @@ -327,7 +327,10 @@ class NetCDF4DataStore(WritableCFDataStore): def init( self, manager, group=None, mode=None, lock=NETCDF4_PYTHON_LOCK, autoclose=False ): - import netCDF4 + try: + import nc_complex as netCDF4 + except ImportError: + import netCDF4

     if isinstance(manager, netCDF4.Dataset):
         if group is None:

@@ -364,7 +367,10 @@ class NetCDF4DataStore(WritableCFDataStore): lock_maker=None, autoclose=False, ): - import netCDF4 + try: + import nc_complex as netCDF4 + except ImportError: + import netCDF4

     if isinstance(filename, os.PathLike):
         filename = os.fspath(filename)

```

and then it works out of the box:

```python import numpy as np import xarray as xr

complex_array = np.array([0 + 0j, 1 + 0j, 0 + 1j, 1 + 1j, 0.25 + 0.75j], dtype="c16")

ds = xr.Dataset({"complex_data": ("x", complex_array)}) ds.to_netcdf("test.nc")

with xr.open_dataset("test.nc") as df: print(df.complex_data) assert ds == df

<xarray.DataArray 'complex_data' (x: 5)>

[5 values with dtype=complex128]

Dimensions without coordinates: x

```

If the library is built against the latest main of netCDF-C, then it also supports files written by h5netcdf. And while the package isn't on PyPI yet, built wheels with this feature are available from CI jobs.

nc_complex is not quite ready yet, but I wanted to gauge interest for adding it as an optional replacement for netCDF4 in xarray.

Describe alternatives you've considered

No response

Additional context

No response

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/8288/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 1 row from issues_id in issues_labels
  • 0 rows from issue in issue_comments
Powered by Datasette · Queries took 0.735ms · About: xarray-datasette