issue_comments

The scipy backend has shown to be a good alternative as of now, if not I'll write a work around though. Thanks for your help !

Marking this as closed since the issue will be fixed in the next netcdf4-python release. Feel free to submit a PR with a work around for xarray if using the scipy backend is not a good alternative for you.

This is pretty clearly a netCDF4-Python bug, which I have reported upstream: https://github.com/Unidata/netcdf4-python/issues/743

If desired, we probably could pretty easily add a work around for this behavior (e.g., to create an empty numpy array of the appropriate shape) to our netCDF4-python wrapper: https://github.com/pydata/xarray/blob/9d09c1659741dafb1fadeed49c81f9e90a548b07/xarray/backends/netCDF4_.py#L50

This might be specific to string variables (character arrays) and netCDF4. My guess is that netCDF4 may not handle indexing on zero-dimensional arrays properly.

Reproduced also in my environment.

Selected repr of your dataset is python <xarray.Dataset> Dimensions: (N_CALIB: 1, N_HISTORY: 0, N_LEVELS: 1007, N_PARAM: 3, N_PROF: 33) Dimensions without coordinates: N_CALIB, N_HISTORY, N_LEVELS, N_PARAM, N_PROF Data variables: ... SCIENTIFIC_CALIB_DATE (N_PROF, N_CALIB, N_PARAM) object b'20150514141619' ... HISTORY_INSTITUTION (N_HISTORY, N_PROF) object ... It seems the data variable in concern, such as ds['HISTORY_STEP'], is 2-dimensional but one of the dimensions N_HISTORY is size 0.

~~Currently, xarray does not handle such zero-length n-dimensional arrays. Maybe we need to drop such variables when reading?~~

EDIT: This was wrong. xarray should handle 0-size nd-array.

@shoyer, @fmaussion, any idea?

I tried with xarray v0.9.6 but it raises the same error. Maybe also related to #1719 .

But is this really the issue? Because this works:

```python import numpy as np import xarray as xr

a = np.zeros((0, 2)) da = xr.DataArray(a) da.to_netcdf('test.nc')

xr.open_dataarray('test.nc') Out[16]: <xarray.DataArray (dim_0: 0, dim_1: 2)> array([], shape=(0, 2), dtype=float64) Dimensions without coordinates: dim_0, dim_1 ```

Currently, xarray does not handle such zero-length n-dimensional arrays. Maybe we need to drop such variables when reading?

Rather than drop a variable the user could read with no problem under xarray 0.9.6, I propose we examine our new indexing wrappers and figure out how to fix this bug.

Sorry guys, just found out that the issue is still going on with some of the variables in the dataset:

Works ok for temperature TEMP for instance: python ds = xr.open_dataset(argofile, autoclose=True, decode_cf=True) ds['TEMP'] Out[89]: <xarray.DataArray 'TEMP' (N_PROF: 338, N_LEVELS: 51)> array([[ 27.393 , 27.392 , 27.393 , ..., 3.597 , 3.34 , nan], [ 27.57 , 27.572001, 27.570999, ..., 3.543 , 3.265 , nan], [ 28.094999, 28.091999, 28.096001, ..., 3.544 , 3.287 , nan], ..., [ 27.157 , 27.156 , 27.159 , ..., 3.318 , nan, nan], [ 27.608999, 27.610001, 27.608999, ..., 3.419 , nan, nan], [ 27.569 , 27.566999, 27.561001, ..., 3.422 , nan, nan]]) Dimensions without coordinates: N_PROF, N_LEVELS Attributes: long_name: Sea temperature in-situ ITS-90 scale standard_name: sea_water_temperature units: degree_Celsius valid_min: -2.5 valid_max: 40.0 C_format: %9.3f FORTRAN_format: F9.3 resolution: 0.001

but for the variable "HISTORY_STEP", I get the error: python ds['HISTORY_STEP'] Out[90]: Traceback (most recent call last): File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/IPython/core/formatters.py", line 190, in catch_format_error r = method(self, *args, **kwargs) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/IPython/core/formatters.py", line 672, in __call__ printer.pretty(obj) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/IPython/lib/pretty.py", line 383, in pretty return _default_pprint(obj, self, cycle) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/IPython/lib/pretty.py", line 503, in _default_pprint _repr_pprint(obj, p, cycle) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/IPython/lib/pretty.py", line 701, in _repr_pprint output = repr(obj) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/common.py", line 100, in __repr__ return formatting.array_repr(self) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/formatting.py", line 393, in array_repr summary.append(short_array_repr(arr.values)) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/dataarray.py", line 412, in values return self.variable.values File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/variable.py", line 396, in values return _as_array_or_item(self._data) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/variable.py", line 217, in _as_array_or_item data = np.asarray(data) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/numpy/core/numeric.py", line 482, in asarray return array(a, dtype, copy=False, order=order) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/indexing.py", line 557, in __array__ self._ensure_cached() File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/indexing.py", line 554, in _ensure_cached self.array = NumpyIndexingAdapter(np.asarray(self.array)) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/numpy/core/numeric.py", line 482, in asarray return array(a, dtype, copy=False, order=order) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/indexing.py", line 538, in __array__ return np.asarray(self.array, dtype=dtype) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/numpy/core/numeric.py", line 482, in asarray return array(a, dtype, copy=False, order=order) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/indexing.py", line 505, in __array__ return np.asarray(array[self.key], dtype=None) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/conventions.py", line 388, in __getitem__ return mask_and_scale(self.array[key], self.fill_value, File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/conventions.py", line 498, in __getitem__ return char_to_bytes(self.array[key]) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/conventions.py", line 640, in char_to_bytes arr = np.array(arr, copy=False, order='C') File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/indexing.py", line 505, in __array__ return np.asarray(array[self.key], dtype=None) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/backends/netCDF4_.py", line 72, in __getitem__ raise IndexError(msg) IndexError: The indexing operation you are attempting to perform is not valid on netCDF4.Variable object. Try loading your data into memory first by calling .load(). Original traceback: Traceback (most recent call last): File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/backends/netCDF4_.py", line 61, in __getitem__ data = getitem(self.get_array(), key) File "netCDF4/_netCDF4.pyx", line 3961, in netCDF4._netCDF4.Variable.__getitem__ File "netCDF4/_netCDF4.pyx", line 4796, in netCDF4._netCDF4.Variable._get IndexError

The new state of the versions: ```python INSTALLED VERSIONS

commit: None python: 2.7.12.final.0 python-bits: 64 OS: Darwin OS-release: 16.7.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: None.None xarray: 0.10.0 pandas: 0.21.0 numpy: 1.11.3 scipy: 0.18.1 netCDF4: 1.3.1 h5netcdf: 0.3.1 Nio: None bottleneck: 1.2.0 cyordereddict: 1.0.0 dask: 0.16.0 matplotlib: 1.5.3 cartopy: 0.15.1 seaborn: 0.7.1 setuptools: 36.5.0 pip: 9.0.1 conda: None pytest: None IPython: 5.2.2 sphinx: 1.5.2 ```

~It's good to see that the release-candidate process actually works!~ 😭

It's good to see that the release-candidate process actually works! 😉

Ok, upgrading to 0.10.0 solve the issue ! Thanks Should have tried this in the 1st place

Yes, please upgrade to 0.10.0 and report back. I cannot reproduce your issue with this release version.

Guillaume--this is very troubling! I use xarray frequently on ARGO netCDF files. It would be a shame if we broke something related to reading them. It sounds like this could be related to the indexing changes in #1676.

Since rc1, xarray 0.10.0 has gone through another release candidate (rc2) and is now in full release. Can you try upgrading to the just-released 0.10.0 and verify whether the problem is still present?