issue_comments

:heavy_plus_sign: 1

``` ❯ pip freeze |grep -Ei "xarray|numpy|netcdf" netCDF4==1.6.3 numpy==1.23.3 xarray==2023.4.2

❯ python -c "import xarray;import warnings;warnings.filterwarnings('error');import netCDF4"
Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/z3538708/projects/gitlab.com/sqc-eng/se/core/.venv-core/lib/python3.10/site-packages/netCDF4/init.py", line 3, in <module> from ._netCDF4 import * File "src/netCDF4/_netCDF4.pyx", line 1, in init netCDF4._netCDF4 RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 16 from C header, got 96 from PyObject

❯ python -c "import netCDF4;import xarray;import warnings;warnings.filterwarnings('error');import netCDF4" ❯ `` No errors when the order isnetCD4` import, before import xarray

weirdly enough, this: python -c "import warnings; warnings.filterwarnings('error'); import numpy; import netCDF4" does not error, haha.

I would suggest to open an issue in netcdf4, since it seems to happen when you import numpy before netcdf4.

I'm really not sure. It seems to happen with a large swath of versions from my recent search.

Also running from the python REPL, i don't see the warning. which makes me feel like numpy/cython/netcdf4 are trying to suppress the harmless warning.

https://github.com/cython/cython/blob/0.29.x/Cython/Utility/ImportExport.c#L365

If you are still motivated, you could try figuring out if it happens with older versions of numpy and netcdf as well...

one or the other.

So it happens when you import numpy before netCDF? Then I would claim that it is a netCDF problem, haha.

I think it is a numpy thing bash mamba create --name np numpy netcdf4 --channel conda-forge --override-channels conda activate np python -c "import numpy; import warnings; warnings.filterwarnings('error'); import netCDF4" python Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/mark/mambaforge/envs/np/lib/python3.11/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * File "src/netCDF4/_netCDF4.pyx", line 1, in init netCDF4._netCDF4 RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 16 from C header, got 96 from PyObject

mamba create --name xr numpy pandas xarray netcdf4 --channel conda-forge --override-channels conda activate xr python -c "import xarray; import warnings; warnings.filterwarnings('error'); import netCDF4"

Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/mark/mambaforge/envs/xr/lib/python3.11/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * File "src/netCDF4/_netCDF4.pyx", line 1, in init netCDF4._netCDF4 RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 16 from C header, got 96 from PyObject

Can we find a minimum environment that can reproduce this results? I.e. only numpy, pandas, xarray and netcdf4?

It is also reproducible on binder:

It seems that the binder uses conda-forge, which is why i'm commenting here.

It is really strange in the sense that xarray doesn't compile anything.

https://github.com/conda-forge/xarray-feedstock/blob/main/recipe/meta.yaml#L16

So it must be something that gets lazy loaded that triggers things.

It is also reproducible locally only xarray makes the difference, for the git bisect I used a fresh env.

Is it repeatable locally with the exact same package versions installed?

Yeah we first saw it on the CI, which moved it from bad local install (as pretty much all accepted StackOverflow answers also suggest) to

I have no idea what is going wrong ...

😅

I have run into this problem already several times and the problem was always a somehow broken numpy install. A reinstall usually fixed this for me. But I assume this is part of a CI pipeline or so?

We were/are also entirely baffled and have no clue what kind of black magic is going is going on with this 😅

It is true that the import behavior had changed in the linked PR (it is now lazy importing netCDF4) but I don't see how this can cause such problems.

Normally this warning indicates that something is wrong with your install, but since you can solve it by importing netcdf4 first, I have no idea what is going wrong ...