id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 1057335460,I_kwDOAMm_X84_Baik,6002,Abnormal process termination when using bottleneck function on xarray data after transposing and having a dimension with length 1,34062862,closed,0,,,5,2021-11-18T13:06:25Z,2021-11-19T13:20:28Z,2021-11-19T13:20:28Z,NONE,,,,"When running the following example the python interpreter exists abnormally (exit code -1073741819) The code that causes the exception is the maxnan() function in bottleneck. This error only occurs when: 1. The data contains a dimension with length 1 2. The data is transposed 3. bottleneck is used So I suspect that the transpose function changes the data somehow such that bottleneck can not handle it anymore. - Calling bn.maxnan on a normal ndarray works fine - Calling data.max() before transposing works fine - Running the example without using bottleneck (either by not installing it or by disabling it in OPTIONS) works fine - Running the example with len(time) > 1 works fine **What happened**: Process finished with exit code -1073741819 (0xC0000005) **What you expected to happen**: Script should run without crashing **Minimal Complete Verifiable Example**: ```python from collections import OrderedDict import numpy as np import xarray as xr xr.show_versions() n_time = 1 # 1 : Fails, 2 : everything is fine from xarray.core.options import OPTIONS OPTIONS[""use_bottleneck""] = True # Set to False for work-around # Build some dataset dirs = np.linspace(0,360, num=121) freqs = np.linspace(0,4,num=192) spec_data = np.random.random(size=(n_time,192,121)) dims = ('time', 'freq', 'dir') coords = OrderedDict() coords['time'] = range(n_time) coords['freq'] = freqs coords['dir'] = dirs xdata = xr.DataArray( data=spec_data, coords=coords, dims=dims, name='Spec name', ).to_dataset() xdata = xdata.transpose(..., ""freq"") # remove this line and the script will run tm = xdata.max() print('Done') ``` **Anything else we need to know?**: Uhm... it was really hard to dig this deep? :-) **Environment**: Tested with python 3.8 and 3.9 on win64. Only required packages are xarray and bottleneck. For example: ```yml name: ws dependencies: - python=3.9 - xarray channels: - defaults - conda-forge ``` note: xarray requires pandas which installs bottleneck. To install without bottleneck pin pandas to 1.2.4
Output of xr.show_versions() INSTALLED VERSIONS ------------------ commit: None python: 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: AMD64 Family 23 Model 96 Stepping 1, AuthenticAMD byteorder: little LC_ALL: None LANG: None LOCALE: ('English_United States', '1252') libhdf5: None libnetcdf: None xarray: 0.20.1 pandas: 1.2.4 numpy: 1.21.4 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: None distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 59.1.1 pip: 21.3.1 conda: None pytest: None IPython: None sphinx: None
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6002/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1057082683,I_kwDOAMm_X84_Ac07,6001,Crash when calling max() after transposing a dataset in combination with numba,34062862,closed,0,,,4,2021-11-18T08:39:56Z,2021-11-18T20:17:56Z,2021-11-18T09:21:18Z,NONE,,,,"I have a piece of code runs fine on an conda environment with just xarray. But when I add **numba** then the code crashes **What happened**: Abnormal process termination (eg Process finished with exit code -1073741819 (0xC0000005)) **What you expected to happen**: should calculate the max of the data (same as before transposing) **Minimal Complete Verifiable Example**: ```python from collections import OrderedDict import numpy as np import xarray as xr # Build some dataset dirs = np.linspace(0,360, num=121) freqs = np.linspace(0,4,num=192) spec_data = np.random.random(size=(192,121)) data = [spec_data] dims = ('time', 'freq', 'dir') coords = OrderedDict() coords['time'] = [0,] coords['freq'] = freqs coords['dir'] = dirs print('constructing data-array') xdata = xr.DataArray( data=data, coords=coords, dims=dims, name='Spec name', ).to_dataset() print('transposing data-array') print('getting max') print(xdata.max()) # works fine tdata = xdata.transpose(..., ""freq"") print('getting max') print(tdata.max()) # <==== Process finished with exit code -1073741819 (0xC0000005) print('done!') ``` **Anything else we need to know?**: Running on windows 10 x64 Python 3.10 but also occurs with 3.8 and 3.9 (did not test any others) **Environment**: When I create an environment with ONLY xarray then everything works as expected: This environment WORKS
Output of xr.show_versions() INSTALLED VERSIONS ------------------ commit: None python: 3.10.0 | packaged by conda-forge | (default, Oct 12 2021, 21:17:52) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: AMD64 Family 23 Model 96 Stepping 1, AuthenticAMD byteorder: little LC_ALL: None LANG: None LOCALE: English_United States.1252 libhdf5: None libnetcdf: None xarray: 0.18.0 pandas: 1.3.4 numpy: 1.21.4 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None pint: None setuptools: 59.1.1 pip: 21.3.1 conda: None pytest: None IPython: None sphinx: None
But when I add numba then it fails:
Output of INSTALLED VERSIONS ------------------ commit: None python: 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: AMD64 Family 23 Model 96 Stepping 1, AuthenticAMD byteorder: little LC_ALL: None LANG: None LOCALE: ('English_United States', '1252') libhdf5: None libnetcdf: None xarray: 0.20.1 pandas: 1.3.4 numpy: 1.20.3 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: None distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 59.1.1 pip: 21.3.1 conda: None pytest: None IPython: None sphinx: None --- conda output --- # packages in environment at c:\python\miniconda3\envs\ws: # # Name Version Build Channel blas 1.0 mkl bottleneck 1.3.2 py39h7cc1a96_1 ca-certificates 2021.10.26 haa95532_2 importlib-metadata 4.8.2 py39hcbf5309_0 conda-forge importlib_metadata 4.8.2 hd8ed1ab_0 conda-forge intel-openmp 2021.4.0 haa95532_3556 llvmlite 0.37.0 py39h23ce68f_1 mkl 2021.4.0 haa95532_640 mkl-service 2.4.0 py39h2bbff1b_0 mkl_fft 1.3.1 py39h277e83a_0 mkl_random 1.2.2 py39hf11a4ad_0 numba 0.54.1 py39hf11a4ad_0 numexpr 2.7.3 py39hb80d3ca_1 numpy 1.20.3 py39ha4e8547_0 numpy-base 1.20.3 py39hc2deb75_0 openssl 1.1.1l h2bbff1b_0 pandas 1.3.4 py39h6214cd6_0 pip 21.3.1 pyhd8ed1ab_0 conda-forge python 3.9.7 h6244533_1 python-dateutil 2.8.2 pyhd3eb1b0_0 python_abi 3.9 2_cp39 conda-forge pytz 2021.3 pyhd3eb1b0_0 setuptools 59.1.1 py39hcbf5309_0 conda-forge six 1.16.0 pyhd3eb1b0_0 sqlite 3.36.0 h2bbff1b_0 tbb 2021.4.0 h59b6b97_0 typing_extensions 4.0.0 pyha770c72_0 conda-forge tzdata 2021e hda174b7_0 ucrt 10.0.20348.0 h57928b3_0 conda-forge vc 14.2 h21ff451_1 vs2015_runtime 14.29.30037 h902a5da_5 conda-forge wheel 0.37.0 pyhd3eb1b0_1 xarray 0.20.1 pyhd8ed1ab_0 conda-forge zipp 3.6.0 pyhd3eb1b0_0 zlib 1.2.11 h62dcd97_4
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6001/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 537936090,MDU6SXNzdWU1Mzc5MzYwOTA=,3622,custom interpolation,34062862,closed,0,,,2,2019-12-14T16:43:16Z,2020-01-17T08:06:05Z,2020-01-17T08:06:04Z,NONE,,,,"I need to interpolate (wave) forces on a ship between headings and/or frequencies. heading and frequency are both dimensions. force is a phase + amplitude pair. What I would normally do is linear interpolation of the force amplitude and linear interpolation of the unwrapped phase. Interpolation of the amplitude works fine, but interpolation of the phase is troublesome because I can not unwrap the phases in two dimensions (heading and frequency) at the same time. A solution that I can think of is storing the phase as a complex number, interpolate that, and then get the phase of the interpolated value. But calculating the angle (phase) from the interpolated values would be slow and it feels like a workaround rather than a good solution. It would be great if I could pass a custom interpolation function to the interpolate method to use instead of scipy.interpolate.interp1d. But as far as I can see this is not (yet) an option. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3622/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue