What happened?

Very possibly this is user-error, forgive me if so.

I'm trying to transition some code from the previous assignment of MultiIndexes, to the new world. Here's an MCVE:

What did you expect to happen?

No response

Minimal Complete Verifiable Example

```Python da = xr.tutorial.open_dataset("air_temperature")['air']

old code, works, but with a warning

da.expand_dims('foo').assign_coords(foo=(pd.MultiIndex.from_tuples([(1,2)])))

<ipython-input-25-f09b7f52bb42>:1: FutureWarning: the pandas.MultiIndex object(s) passed as 'foo' coordinate(s) or data variable(s) will no longer be implicitly promoted and wrapped into multiple indexed coordinates in the future (i.e., one coordinate for each multi-index level + one dimension coordinate). If you want to keep this behavior, you need to first wrap it explicitly using mindex_coords = xarray.Coordinates.from_pandas_multiindex(mindex_obj, 'dim') and pass it as coordinates, e.g., xarray.Dataset(coords=mindex_coords), dataset.assign_coords(mindex_coords) or dataarray.assign_coords(mindex_coords). da.expand_dims('foo').assign_coords(foo=(pd.MultiIndex.from_tuples([(1,2)]))) Out[25]: <xarray.DataArray 'air' (foo: 1, time: 2920, lat: 25, lon: 53)> array([[[[241.2 , 242.5 , 243.5 , ..., 232.79999, 235.5 , 238.59999], ... [297.69 , 298.09 , 298.09 , ..., 296.49 , 296.19 , 295.69 ]]]], dtype=float32) Coordinates: * lat (lat) float32 75.0 72.5 70.0 67.5 65.0 ... 22.5 20.0 17.5 15.0 * lon (lon) float32 200.0 202.5 205.0 207.5 ... 325.0 327.5 330.0 * time (time) datetime64[ns] 2013-01-01 ... 2014-12-31T18:00:00 * foo (foo) object MultiIndex * foo_level_0 (foo) int64 1 * foo_level_1 (foo) int64 2

new code — seems to get confused between the number of values in the index — 1 — and the number of levels — 3 including the parent:

da.expand_dims('foo').assign_coords(foo=xr.Coordinates.from_pandas_multiindex(pd.MultiIndex.from_tuples([(1,2)]), dim='foo'))

ValueError Traceback (most recent call last) Cell In[26], line 1 ----> 1 da.expand_dims('foo').assign_coords(foo=xr.Coordinates.from_pandas_multiindex(pd.MultiIndex.from_tuples([(1,2)]), dim='foo'))

File ~/workspace/xarray/xarray/core/common.py:621, in DataWithCoords.assign_coords(self, coords, **coords_kwargs) 618 else: 619 results = self._calc_assign_results(coords_combined) --> 621 data.coords.update(results) 622 return data

File ~/workspace/xarray/xarray/core/coordinates.py:566, in Coordinates.update(self, other) 560 # special case for PandasMultiIndex: updating only its dimension coordinate 561 # is still allowed but depreciated. 562 # It is the only case where we need to actually drop coordinates here (multi-index levels) 563 # TODO: remove when removing PandasMultiIndex's dimension coordinate. 564 self._drop_coords(self._names - coords_to_align._names) --> 566 self._update_coords(coords, indexes)

File ~/workspace/xarray/xarray/core/coordinates.py:834, in DataArrayCoordinates._update_coords(self, coords, indexes) 832 coords_plus_data = coords.copy() 833 coords_plus_data[_THIS_ARRAY] = self._data.variable --> 834 dims = calculate_dimensions(coords_plus_data) 835 if not set(dims) <= set(self.dims): 836 raise ValueError( 837 "cannot add coordinates with new dimensions to a DataArray" 838 )

File ~/workspace/xarray/xarray/core/variable.py:3014, in calculate_dimensions(variables) 3012 last_used[dim] = k 3013 elif dims[dim] != size: -> 3014 raise ValueError( 3015 f"conflicting sizes for dimension {dim!r}: " 3016 f"length {size} on {k!r} and length {dims[dim]} on {last_used!r}" 3017 ) 3018 return dims

ValueError: conflicting sizes for dimension 'foo': length 1 on <this-array> and length 3 on {'lat': 'lat', 'lon': 'lon', 'time': 'time', 'foo': 'foo'} ```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [X] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

No response

Anything else we need to know?

No response

Environment

What is your issue?

It looks like the doc build is failing on a pandas docstring:

``` /home/docs/checkouts/readthedocs.org/user_builds/xray/conda/8156/lib/python3.10/site-packages/cartopy/io/init.py:241: DownloadWarning: Downloading: https://naturalearth.s3.amazonaws.com/50m_physical/ne_50m_coastline.zip warnings.warn(f'Downloading: {url}', DownloadWarning) reading sources... [ 99%] user-guide/reshaping reading sources... [ 99%] user-guide/terminology reading sources... [ 99%] user-guide/time-series reading sources... [ 99%] user-guide/weather-climate reading sources... [100%] whats-new

/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/8156/lib/python3.10/site-packages/pandas/core/indexes/base.py:docstring of pandas.core.indexes.base.Index.join:14: WARNING: Inline literal start-string without end-string. /home/docs/checkouts/readthedocs.org/user_builds/xray/conda/8156/lib/python3.10/site-packages/pandas/core/indexes/base.py:docstring of pandas.core.indexes.base.Index.join:15: WARNING: Inline literal start-string without end-string. looking for now-outdated files... none found ```

(also including the cartopy warning in case that's relevant)

Is this expected? Is anyone familiar enough with the doc build to know whether we can disable warnings from 3rd party modules?

What is your issue?

I can't work out what is causing this, and can't repro locally, though I've tried to ensure the same things are installed.

The bisect is: - Passes: https://github.com/pydata/xarray/runs/6233389985?check_suite_focus=true - Fails: https://github.com/pydata/xarray/runs/6237267544?check_suite_focus=true

Probably we have to skip it in the meantime, which is a shame

Is there a better way of locking the dependency versions so we can rule that out? I generally don't use conda, and poetry is great at this. Is there a conda equivalent?

Is your feature request related to a problem?

Currently https://github.com/pydata/xarray/blob/main/HOW_TO_RELEASE.md has a few steps around the stable branch

Describe the solution you'd like

In our dev call, we discussed the possibility of using main in place of stable and removing the stable branch

IIRC there's something we can do on RTD to make that replacement. (If anyone knows to hand, comment here; otherwise I can search for it).

Is there anything else we need to do apart from RTD?

Describe alternatives you've considered

No response

Additional context

No response

Branching from https://github.com/pydata/xarray/issues/3770

Here's the proposal: allow lambdas on methods where the primary argument is a single xarray object, and interpret lambas as though they'd be supplied in a pipe method followed by the current method. Taking the example from the linked issue:

```python

In [1]: import xarray as xr

In [2]: import numpy as np

In [3]: da = xr.DataArray(np.random.rand(2,3))

In [4]: da.where(da > 0.5)
Out[4]: <xarray.DataArray (dim_0: 2, dim_1: 3)> array([[ nan, 0.71442406, nan], [0.55748705, nan, nan]]) Dimensions without coordinates: dim_0, dim_1

this should be equivalent (currently not valid)

In [5]: da.where(lambda x: x > 0.5)

the longer version (currently works)

In [5]: da.pipe(lambda x: x.where(x > 0.5))

```

Others I miss from pandas: assign, and loc. I haven't gone through the list though assume there are others; we don't have to agree 100% on the list before starting with the most obvious ones, assuming we're in agreement with the principle.

Any ideas why __hdf5libversion__ wouldn't be available? Shall I put a try / except around it?

```python In [4]: import xarray as xr

In [5]: xr.show_versions()

ModuleNotFoundError Traceback (most recent call last) /usr/local/lib/python3.7/site-packages/xarray/util/print_versions.py in netcdf_and_hdf5_versions() 64 try: ---> 65 import netCDF4 66 libhdf5_version = netCDF4.hdf5libversion

ModuleNotFoundError: No module named 'netCDF4'

During handling of the above exception, another exception occurred:

AttributeError Traceback (most recent call last) <ipython-input-5-6f391305f2fe> in <module> ----> 1 xr.show_versions()

/usr/local/lib/python3.7/site-packages/xarray/util/print_versions.py in show_versions(file) 78 sys_info = get_sys_info() 79 ---> 80 sys_info.extend(netcdf_and_hdf5_versions()) 81 82 deps = [

/usr/local/lib/python3.7/site-packages/xarray/util/print_versions.py in netcdf_and_hdf5_versions() 69 try: 70 import h5py ---> 71 libhdf5_version = h5py.hdf5libversion 72 except ImportError: 73 pass

AttributeError: module 'h5py' has no attribute 'hdf5libversion' ```

I check I'm on the latest h5py: pip install h5py -U Thu Jul 18 16:47:29 2019 Requirement already up-to-date: h5py in ./Library/Python/3.7/lib/python/site-packages (2.9.0) Requirement already satisfied, skipping upgrade: numpy>=1.7 in /usr/local/lib/python3.7/site-packages (from h5py) (1.16.4) Requirement already satisfied, skipping upgrade: six in ./Library/Python/3.7/lib/python/site-packages (from h5py) (1.12.0)

This is extremely minor, I generally wouldn't report it.

We're using classes inherited from Dataset more & more - this works really well for classes with a lot of array-like properties that can be aligned, and allows @property to lazily compute some calcs. (any feedback on this approach very welcome)

The top of the repr is incorrect

python <xarray.SecurityMeasure> Dimensions: (date: 6647, security: 285, sub_measure: 4) Coordinates: ...

Could just be the qualified name.

```python

In [1]: import xarray as xr

In [3]: ds=xr.Dataset({'a':xr.DataArray([1,2,3])})

In [4]: xr.Dataset(ds) /Users/maximilian/drive/workspace/xarray/xarray/core/dataset.py:373: FutureWarning: iteration over an xarray.Dataset will change in xarray v0.11 to only include data variables, not coordinates. Iterate over the Dataset.variables property instead to preserve existing behavior in a forwards compatible manner. both_data_and_coords = [k for k in data_vars if k in coords] Out[4]: <xarray.Dataset> Dimensions: (dim_0: 3) Dimensions without coordinates: dim_0 Data variables: a (dim_0) int64 1 2 3 ```

Problem description

Currently we run both_data_and_coords = [k for k in data_vars if k in coords], where data_vars is the first arg to the constructor. So, when data_vars is a Dataset, we'll raise a warning.

More importantly: this raises the question of how we should handle xr.Dataset(ds) Currently we retain the coords, but discard attrs. That seems arbitrary? I'd have to check, but when we change __iter__ to only reference data_vars, we may also discard coords

I don't have a strong view. If you think that Foo(foo) should always return foo (which is often the case in pandas), then we should retain the full object. If you think that foo is supplied to the data_vars arg, then retaining only the data_vars seems reasonable

Expected Output

Output of xr.show_versions()

I originally posted this on SO, as I thought it was a user question rather than a library issue. But after working on it more today, I'm not so sure.

I'm trying to do a 'join' in xarray, but using a non-index coordinate rather than a shared dim.

I have a Dataset indexed on 'a' with a coord on 'b', and a DataArray indexed on 'b':

```python In [17]: ds=xr.Dataset(dict(a=(('x'),np.random.rand(10))), coords=dict(b=(('x'),list(range(10)))))

In [18]: ds Out[18]: <xarray.Dataset> Dimensions: (x: 10) Coordinates: b (x) int64 0 1 2 3 4 5 6 7 8 9 Dimensions without coordinates: x Data variables: a (x) float64 0.3634 0.2132 0.6945 0.5359 0.1053 0.07045 0.5945 ...

In [19]: da=xr.DataArray(np.random.rand(10), dims=('b',), coords=dict(b=(('b'),list(range(10)))))

In [20]: da Out[20]: <xarray.DataArray (b: 10)> array([0.796987, 0.275992, 0.747882, 0.240374, 0.435143, 0.285271, 0.753582, 0.556038, 0.365889, 0.434844]) Coordinates: * b (b) int64 0 1 2 3 4 5 6 7 8 9

```

Can I add da onto my dataset, by joining on ds.b equalling da.b? The result would be:

python <xarray.Dataset> Dimensions: (x: 10) Coordinates: b (x) int64 0 1 2 3 4 5 6 7 8 9 Dimensions without coordinates: x Data variables: a (x) float64 0.3634 0.2132 0.6945 0.5359 0.1053 0.07045 0.5945 ... da (x) float64 0.796987, 0.275992, 0.747882, 0.240374, 0.435143 ...

(for completeness - the data isn't current in the correct position)

http://xray.readthedocs.org/en/stable/generated/xray.DataArray.count.html

name doesn't seem to be working correctly:

Dimension(s) over which to apply {name}.

This is the repr for a DataArray with 7 coordinates and 5 dimensions. Do we agree this should be shorter?

``` python <xray.DataArray (a: 2, b: 2, c: 5, d: 6771, e: 58)> array([[[[[ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], ..., [ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]]],


    [[[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]]]],



   [[[[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]]],


    [[[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]],

     [[ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      ..., 
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan],
      [ nan,  nan,  nan, ...,  nan,  nan,  nan]]]]])

Coordinates: ... ```

For an item to be a value in a Dataset, it either needs to be array-like, or pass lib.is_scalar.

I think this is probably too strict - I'd propose anything can be a value, and it only gets treated as an array if it looks like one - thoughts?

Here's the code that checks the values: https://github.com/pydata/xarray/blob/master/xarray/core/variable.py#L52

``` python In [13]: class B(object): pass ....:

In [14]: xr.Dataset({'a':B()})

ValueError Traceback (most recent call last) <ipython-input-14-b2eeb6de19ea> in <module>() ----> 1 xr.Dataset({'a':B()})

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/dataset.py in init(self, data_vars, coords, attrs, compat, **kwargs) 207 coords = set() 208 if data_vars is not None or coords is not None: --> 209 self._set_init_vars_and_dims(data_vars, coords, compat) 210 if attrs is not None: 211 self.attrs = attrs

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/dataset.py in _set_init_vars_and_dims(self, vars, coords, compat) 258 aligned = align_variables(variables) 259 new_variables, new_coord_names = expand_variables(aligned, --> 260 compat=compat) 261 262 new_coord_names.update(coords)

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/merge.py in expand_variables(raw_variables, old_variables, compat) 75 add_variable(dim, coord.variable) 76 var = var.variable ---> 77 add_variable(name, var) 78 79 return new_variables, new_coord_names

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/merge.py in add_variable(name, var) 55 56 def add_variable(name, var): ---> 57 var = _as_dataset_variable(name, var) 58 if name not in variables: 59 variables[name] = var

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/merge.py in _as_dataset_variable(name, var) 9 """ 10 try: ---> 11 var = as_variable(var, key=name) 12 except TypeError: 13 raise TypeError('variables must be given by arrays or a tuple of '

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/variable.py in as_variable(obj, key, strict, copy) 55 obj = Variable(obj.name, obj) 56 elif key is not None: ---> 57 obj = Variable(key, obj) 58 else: 59 raise TypeError('cannot infer Variable dimensions')

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/variable.py in init(self, dims, data, attrs, encoding, fastpath) 211 """ 212 self._data = as_compatible_data(data, fastpath=fastpath) --> 213 self._dims = self._parse_dimensions(dims) 214 self._attrs = None 215 self._encoding = None

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/xarray/core/variable.py in _parse_dimensions(self, dims) 319 raise ValueError('dimensions %s must have the same length as the ' 320 'number of data dimensions, ndim=%s' --> 321 % (dims, self.ndim)) 322 return dims 323

ValueError: dimensions ('a',) must have the same length as the number of data dimensions, ndim=0 ```

Inspired by: https://github.com/pydata/xarray/pull/844. This is a more general case for variable selection.

We could allow a selection of variables with a callable, similar (but not the same) as pandas' implementation:

``` python In [5]: ds=xr.Dataset({'a': (('x', 'y'), np.random.rand(10,2))}) Out[4]: <xarray.Dataset> Dimensions: (x: 10, y: 2) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 * y (y) int64 0 1 Data variables: a (x, y) float64 0.5819 0.1214 0.2645 0.9053 0.6968 0.1608 0.3199 ...

In [9]: ds['a'].attrs['clean'] = True

potentially:

In [10]: ds[lambda x: x.attrs['clean']]

... would return ds['a']

```

This would mean functions wouldn't be able to be dataset keys - I don't think that's a big issue, but could arise with callable classes, for example.

Another option would be a .filter / .select method or indexer.

What do we think about switching to py.test?

Current tests pass with py.test. Then we can use its features for new tests, while the old ones will continue working.

We have a pandas DataFrame which is not aligned on an xray Dataset:

``` python In [34]:

da = xray.DataArray( np.random.rand(5,2), coords=( ('date', pd.date_range(start='2000', periods=5)), ('company', list('ab')), ) ) da Out[34]: <xray.DataArray (date: 5, company: 2)> array([[ 0.82168647, 0.93097023], [ 0.34928855, 0.23245631], [ 0.32857461, 0.12554705], [ 0.44983381, 0.27182767], [ 0.31063147, 0.52894834]]) Coordinates: * date (date) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 ... * company (company) |S1 'a' 'b' In [35]:

ds = xray.Dataset({'returns': da}) ds Out[35]: <xray.Dataset> Dimensions: (company: 2, date: 5) Coordinates: * date (date) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 ... * company (company) |S1 'a' 'b' Data variables: returns (date, company) float64 0.8217 0.931 0.3493 0.2325 0.3286 ... In [36]:

df=da.to_pandas() df Out[36]: company a b date
2000-01-01 0.821686 0.930970 2000-01-02 0.349289 0.232456 2000-01-03 0.328575 0.125547 2000-01-04 0.449834 0.271828 2000-01-05 0.310631 0.528948 In [41]:

rank rank = df.rank() rank Out[41]: company a b date
2000-01-01 5 5 2000-01-02 3 2 2000-01-03 2 1 2000-01-04 4 3 2000-01-05 1 4 In [42]:

rank=rank.reindex(columns=list('ba')) rank Out[42]: company b a date
2000-01-01 5 5 2000-01-02 2 3 2000-01-03 1 2 2000-01-04 3 4 2000-01-05 4 1 ```

When we add it to a Dataset, it ignores the index on the columns:

``` python In [49]:

ds['rank'] = (('date','company'),rank) ds['rank'].to_pandas() Out[49]: company a b date
2000-01-01 5 5 2000-01-02 2 3 2000-01-03 1 2 2000-01-04 3 4 2000-01-05 4 1 ```

And adding the DataFrame without supplying dims doesn't work. One solution, is to construct a DataArray out of the pandas object:

``` python In [45]:

ds['rank'] = xray.DataArray(rank) ds Out[45]: <xray.Dataset> Dimensions: (company: 2, date: 5) Coordinates: * date (date) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 ... * company (company) object 'a' 'b' Data variables: returns (date, company) float64 0.8217 0.931 0.3493 0.2325 0.3286 ... rank (date, company) float64 5.0 5.0 3.0 2.0 2.0 1.0 4.0 3.0 1.0 4.0 ```

Possible additions to make this easier: - Align pandas objects that are passed in, when dims are supplied - Allow adding pandas objects to Datasets with labelled axes without supplying dims, and align those (similar to wrapping them in a DataArray constructor)

What are your thoughts?