issues: 275744315

This data as json

id	node_id	number	title	user	state	locked	assignee	milestone	comments	created_at	updated_at	closed_at	author_association	active_lock_reason	draft	pull_request	body	reactions	performed_via_github_app	state_reason	repo	type
275744315	MDU6SXNzdWUyNzU3NDQzMTU=	1732	IndexError when printing dataset from an Argo file	1956032	closed	0			14	2017-11-21T15:04:16Z	2017-11-27T08:21:15Z	2017-11-25T19:49:24Z	CONTRIBUTOR				Working with a netcdf Argo data file, I encountered the following error: ```python Sample data file here: https://storage.googleapis.com/myargo/sample/4902076_prof.nc argofile = '4902076_prof.nc' ds = xr.open_dataset(argofile) print ds [...full trace below...] Out[]: IndexError: The indexing operation you are attempting to perform is not valid on netCDF4.Variable object. Try loading your data into memory first by calling .load(). Original traceback: Traceback (most recent call last): File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/backends/netCDF4_.py", line 62, in getitem data = getitem(self.get_array(), key) File "netCDF4/_netCDF4.pyx", line 3961, in netCDF4._netCDF4.Variable.getitem File "netCDF4/_netCDF4.pyx", line 4796, in netCDF4._netCDF4.Variable._get IndexError ``` The error remains the same even if I try to load the data as suggested in the error message. However, I can keep working with the dataset and access variable. This only affects the printing of the ds object. I can't get to determine where in my package updating workflow this really pop out. It used to work very fined up to xarray version 0.9.5. Here I'm using 0.10.0rc1 (see full version details below). It is worth noting that using the 'scipy' engine solves the issue ! `python ds = xr.open_dataset(argofile, engine='scipy') print ds Out[48]: <xarray.Dataset> Dimensions: (N_CALIB: 1, N_HISTORY: 0, N_LEVELS: 1007, N_PARAM: 3, N_PROF: 33) Dimensions without coordinates: N_CALIB, N_HISTORY, N_LEVELS, N_PARAM, N_PROF Data variables: PROFILE_PRES_QC (N_PROF) object ... DATA_TYPE object ... JULD (N_PROF) datetime64[ns] . [...]` I suspect a compatibility issue somewhere with netCDF4 Any ideas ? thanks Exact trace: python print ds Traceback (most recent call last): File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2881, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "<ipython-input-31-63698f3c943a>", line 1, in <module> print ds File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/formatting.py", line 64, in __repr__ return ensure_valid_repr(self.__unicode__()) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/dataset.py", line 1050, in __unicode__ return formatting.dataset_repr(self) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/formatting.py", line 426, in dataset_repr summary.append(data_vars_repr(ds.data_vars, col_width=col_width)) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/formatting.py", line 297, in _mapping_repr summary += [summarizer(k, v, col_width) for k, v in mapping.items()] File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/formatting.py", line 236, in summarize_datavar return summarize_variable(name, var.variable, col_width, show_values) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/formatting.py", line 212, in summarize_variable elif isinstance(var.data, dask_array_type): File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/variable.py", line 308, in data return self.values File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/variable.py", line 369, in values return _as_array_or_item(self._data) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/variable.py", line 225, in _as_array_or_item data = np.asarray(data) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/numpy/core/numeric.py", line 482, in asarray return array(a, dtype, copy=False, order=order) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/indexing.py", line 412, in __array__ self._ensure_cached() File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/indexing.py", line 409, in _ensure_cached self.array = np.asarray(self.array) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/numpy/core/numeric.py", line 482, in asarray return array(a, dtype, copy=False, order=order) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/indexing.py", line 393, in __array__ return np.asarray(self.array, dtype=dtype) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/numpy/core/numeric.py", line 482, in asarray return array(a, dtype, copy=False, order=order) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/indexing.py", line 360, in __array__ return np.asarray(array[self.key], dtype=None) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/conventions.py", line 384, in __getitem__ return mask_and_scale(self.array[key], self.fill_value, File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/conventions.py", line 493, in __getitem__ return char_to_bytes(self.array[key]) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/conventions.py", line 635, in char_to_bytes arr = np.array(arr, copy=False, order='C') File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/core/indexing.py", line 360, in __array__ return np.asarray(array[self.key], dtype=None) File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/backends/netCDF4_.py", line 73, in __getitem__ raise IndexError(msg) IndexError: The indexing operation you are attempting to perform is not valid on netCDF4.Variable object. Try loading your data into memory first by calling .load(). Original traceback: Traceback (most recent call last): File "/Users/gmaze/anaconda/envs/obidam/lib/python2.7/site-packages/xarray/backends/netCDF4_.py", line 62, in __getitem__ data = getitem(self.get_array(), key) File "netCDF4/_netCDF4.pyx", line 3961, in netCDF4._netCDF4.Variable.__getitem__ File "netCDF4/_netCDF4.pyx", line 4796, in netCDF4._netCDF4.Variable._get IndexError Expected Output: python In [26]: print ds <xarray.Dataset> Dimensions: (N_CALIB: 1, N_HISTORY: 0, N_LEVELS: 1007, N_PARAM: 3, N_PROF: 33) Dimensions without coordinates: N_CALIB, N_HISTORY, N_LEVELS, N_PARAM, N_PROF Data variables: DATA_TYPE object 'Argo profile ' FORMAT_VERSION object '3.1 ' HANDBOOK_VERSION object '1.2 ' REFERENCE_DATE_TIME object '19500101000000' DATE_CREATION object '20150402105716' DATE_UPDATE object '20160127150022' PLATFORM_NUMBER (N_PROF) object '4902076 ' '4902076 ' ... PROJECT_NAME (N_PROF) object 'US ARGO PROJECT ' ... PI_NAME (N_PROF) object 'GREGORY C. JOHNSON ' ... STATION_PARAMETERS (N_PROF, N_PARAM) object 'PRES ' ... CYCLE_NUMBER (N_PROF) float64 1.0 2.0 4.0 5.0 6.0 7.0 ... DIRECTION (N_PROF) object 'A' 'A' 'A' 'A' 'A' 'A' ... DATA_CENTRE (N_PROF) object 'AO' 'AO' 'AO' 'AO' 'AO' ... DC_REFERENCE (N_PROF) object '5448_0469_001 ' ... DATA_STATE_INDICATOR (N_PROF) object '2B ' '2B ' '2B ' ... DATA_MODE (N_PROF) object 'A' 'A' 'A' 'A' 'A' 'A' ... PLATFORM_TYPE (N_PROF) object 'NAVISIR ' ... FLOAT_SERIAL_NO (N_PROF) object '0469 ' ... FIRMWARE_VERSION (N_PROF) object '011514 ' ... WMO_INST_TYPE (N_PROF) object '863 ' '863 ' '863 ' ... JULD (N_PROF) datetime64[ns] 2015-03-03T08:56:39.984000 ... JULD_QC (N_PROF) object '1' '1' '1' '1' '1' '1' ... JULD_LOCATION (N_PROF) datetime64[ns] 2015-03-03T09:12:09.993600 ... LATITUDE (N_PROF) float64 20.26 20.11 21.53 22.0 ... LONGITUDE (N_PROF) float64 -121.5 -121.5 -121.1 ... POSITION_QC (N_PROF) object '1' '1' '1' '1' '1' '1' ... POSITIONING_SYSTEM (N_PROF) object 'GPS ' 'GPS ' ... PROFILE_PRES_QC (N_PROF) object 'A' 'A' 'A' 'A' 'A' 'A' ... PROFILE_TEMP_QC (N_PROF) object 'A' 'A' 'A' 'A' 'A' 'A' ... PROFILE_PSAL_QC (N_PROF) object 'A' 'A' 'A' 'A' 'A' 'A' ... VERTICAL_SAMPLING_SCHEME (N_PROF) object 'Primary sampling: averaged [] ' ... CONFIG_MISSION_NUMBER (N_PROF) float64 1.0 2.0 4.0 5.0 6.0 7.0 ... PRES (N_PROF, N_LEVELS) float64 4.0 6.0 8.0 ... PRES_QC (N_PROF, N_LEVELS) object '1' '1' '1' '1' ... PRES_ADJUSTED (N_PROF, N_LEVELS) float64 3.73 5.73 7.73 ... PRES_ADJUSTED_QC (N_PROF, N_LEVELS) object '1' '1' '1' '1' ... PRES_ADJUSTED_ERROR (N_PROF, N_LEVELS) float64 nan nan nan nan ... TEMP (N_PROF, N_LEVELS) float64 22.24 22.24 ... TEMP_QC (N_PROF, N_LEVELS) object '1' '1' '1' '1' ... TEMP_ADJUSTED (N_PROF, N_LEVELS) float64 22.24 22.24 ... TEMP_ADJUSTED_QC (N_PROF, N_LEVELS) object '1' '1' '1' '1' ... TEMP_ADJUSTED_ERROR (N_PROF, N_LEVELS) float64 nan nan nan nan ... PSAL (N_PROF, N_LEVELS) float64 34.45 34.45 ... PSAL_QC (N_PROF, N_LEVELS) object '1' '1' '1' '1' ... PSAL_ADJUSTED (N_PROF, N_LEVELS) float64 34.45 34.45 ... PSAL_ADJUSTED_QC (N_PROF, N_LEVELS) object '1' '1' '1' '1' ... PSAL_ADJUSTED_ERROR (N_PROF, N_LEVELS) float64 nan nan nan nan ... PARAMETER (N_PROF, N_CALIB, N_PARAM) object 'PRES ' ... SCIENTIFIC_CALIB_EQUATION (N_PROF, N_CALIB, N_PARAM) object 'PRES_ADJUSTED = PRES - surface_pressure ' ... SCIENTIFIC_CALIB_COEFFICIENT (N_PROF, N_CALIB, N_PARAM) object 'surface_pressure=0.27 dbar ' ... SCIENTIFIC_CALIB_COMMENT (N_PROF, N_CALIB, N_PARAM) object 'Pressure adjusted at real time based on most recent valid surface pressure ' ... SCIENTIFIC_CALIB_DATE (N_PROF, N_CALIB, N_PARAM) object '20150514141619' ... HISTORY_INSTITUTION (N_HISTORY, N_PROF) object HISTORY_STEP (N_HISTORY, N_PROF) object HISTORY_SOFTWARE (N_HISTORY, N_PROF) object HISTORY_SOFTWARE_RELEASE (N_HISTORY, N_PROF) object HISTORY_REFERENCE (N_HISTORY, N_PROF) object HISTORY_DATE (N_HISTORY, N_PROF) object HISTORY_ACTION (N_HISTORY, N_PROF) object HISTORY_PARAMETER (N_HISTORY, N_PROF) object HISTORY_START_PRES (N_HISTORY, N_PROF) float64 HISTORY_STOP_PRES (N_HISTORY, N_PROF) float64 HISTORY_PREVIOUS_VALUE (N_HISTORY, N_PROF) float64 HISTORY_QCTEST (N_HISTORY, N_PROF) object Attributes: title: Argo float vertical profile institution: Coriolis GDAC source: Argo float history: 2016-01-27T15:00:22Z creation references: http://www.argodatamgt.org/Documentation user_manual_version: 3.1 Conventions: Argo-3.1 CF-1.6 featureType: trajectoryProfile Output of `xr.show_versions()` INSTALLED VERSIONS ------------------ commit: None python: 2.7.12.final.0 python-bits: 64 OS: Darwin OS-release: 16.7.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: None.None xarray: 0.10.0rc1 pandas: 0.21.0 numpy: 1.11.3 scipy: 0.18.1 netCDF4: 1.3.1 h5netcdf: 0.3.1 Nio: None bottleneck: 1.2.0 cyordereddict: 1.0.0 dask: 0.16.0 matplotlib: 1.5.3 cartopy: 0.15.1 seaborn: 0.7.1 setuptools: 36.5.0 pip: 9.0.1 conda: None pytest: None IPython: 5.2.2 sphinx: 1.5.2	{ "url": "https://api.github.com/repos/pydata/xarray/issues/1732/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 }		completed	13221727	issue

Links from other tables

0 rows from issues_id in issues_labels
14 rows from issue in issue_comments