issue_comments

@shoyer Thanks. Confirmed. Closing.

It looks like this may already be fixed upstream. As suggested in https://github.com/ipython/ipython/issues/10521, updating to Jedi 0.11 seems to fix the issue.

Thanks @shoyer . I think I should close the issue in jedi.

See here for the IPython issue: https://github.com/ipython/ipython/issues/10720

It seems jedi's bug.

A minimal working example is 1. Save following as example.py ```python import traceback

class ExampleClass(object): def init(self): pass

@property
def value(self):
    print('value called')
    print(traceback.print_exc())

2. In IPython, dopython In [1]: import example

In [2]: ex = example.ExampleClass()

In [3]: ex.val -> press [Tab] value called Traceback (most recent call last): None KeyError: (('value',), frozenset()) File "/home/keisukefujii/anaconda3/envs/tensorflow/lib/python3.5/site-packages/jedi/cache.py", line 119, in wrapper return dct[key] File "/home/keisukefujii/anaconda3/envs/tensorflow/lib/python3.5/site-packages/jedi/cache.py", line 46, in wrapper return getattr(self, name) value called None AttributeError: 'CompiledName' object has no attribute '_infer' Traceback (most recent call last): ```

I've been investigating a bit, and actually in the example above the data may be loaded + duplicated!

in Dataset._item_sources (this line), the item {d: self[d] for d in self.dims} computes the DataArray, ds['x'], which has the same (huge) size than myvar.

Still don't know exactly why the data of myvar is loaded (maybe something indeed related to the use in IPython 6.x of a more advanced completion system based on jedi).

I found this happens with IPython=6.1.0, but does not with IPython=5.1.0. Maybe it is the upstream issue.

But if we can avoid this within xarray, it would be great,

We have a unit test test_lazy_load in test_dataset.py that is probably a good place to start on the test case for this.

Thanks. This is reproduced also in my environment. It's strange ... Tagged as a bug.

Sorry my example above was wrong. In the example below myvar is not a coordinate, but it still triggers data loading when pressing tab generally (not only trying to auto-complete myvar).

``` In [1]: import xarray as xr

In [2]: ds = xr.open_dataset('test.nc')

In [3]: ds Out[3]: <xarray.Dataset> Dimensions: (x: 1000000000) Dimensions without coordinates: x Data variables: myvar (x) float64 ...

In [4]: ds['myvar'] Out[4]: <xarray.DataArray 'myvar' (x: 1000000000)> [1000000000 values with dtype=float64] Dimensions without coordinates: x

In [5]: ds['myvar'].variable._in_memory Out[5]: False

In [6]: ds.anything # pressed [Tab] and then [Enter]

AttributeError Traceback (most recent call last) <ipython-input-6-5e7144cb0cd3> in <module>() ----> 1 ds.anything

~/GitRepos/xarray_dev/xarray/core/common.py in getattr(self, name) 174 return source[name] 175 raise AttributeError("%r object has no attribute %r" % --> 176 (type(self).name, name)) 177 178 def setattr(self, name, value):

AttributeError: 'Dataset' object has no attribute 'anything'

In [7]: ds['myvar'].variable._in_memory Out[7]: True ```

Unless I miss something obvious (it's late), this is weird.

Your xr.Dataset consists only from a single coordinate variable python <xarray.Dataset> Dimensions: (myvar: 100000000) Coordinates: * myvar (myvar) float64 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 ... Data variables: *empty* (This behavior is somehow confusing...)

In xarray, coordinate variables are always loaded into memory.

The following example works well

```python In [1]: import xarray as xr ...: import numpy as np ...: ds = xr.Dataset({'myvar': (('x', 'y'), np.arange(100000000).reshape(1000 ...: , 100000))}, ...: coords={'x': np.arange(1000)}) ...: ds.to_netcdf('test.nc') ...:

In [2]: ds_on_disk = xr.open_dataset('test.nc') ...: ds_on_disk['myvar'].variable._in_memory ...: Out[2]: False

In [3]: ds_on_disk.my # -> [Tab] In [3]: ds_on_disk.myvar.variable._in_memory ...: Out[3]: False

In [4]: ds_on_disk['x'].variable._in_memory Out[4]: True ```