However, other_sel in _yield_binary_applied looks like:

python <xarray.DataArray (x: 10)> array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 dayofyear int64 1

https://github.com/pydata/xarray/blob/f821fe20595c3700375ccecebf88e01a61444777/xarray/core/groupby.py#L484

This is still relevant. Now raises

python ValueError: 'x' not present in all datasets and coords='different'. Either add 'x' to datasets where it is missing or specify coords='minimal'. I think the issue may be that _dummy_copy does not include the x-dimension: python xr.core.groupby._dummy_copy(d2.groupby('time.dayofyear').mean('time')) yields python <xarray.DataArray ()> array(nan)

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

Clues about what's going on from pdb: ``` In [23]: import pdb; pdb.pm()

/Users/shoyer/dev/xarray/xarray/core/utils.py(290)getitem() -> return self.mapping[key] (Pdb) u /Users/shoyer/dev/xarray/xarray/core/combine.py(165)<genexpr>() -> for ds in datasets[1:]) (Pdb) u /Users/shoyer/dev/xarray/xarray/core/combine.py(165)differs() -> for ds in datasets[1:]) (Pdb) u /Users/shoyer/dev/xarray/xarray/core/combine.py(168)<genexpr>() -> if k not in concat_over and differs(k)) (Pdb) u /Users/shoyer/dev/xarray/xarray/core/combine.py(167)process_subset_opt() -> concat_new = set(k for k in getattr(datasets[0], subset) (Pdb) u /Users/shoyer/dev/xarray/xarray/core/combine.py(192)_calc_concat_over() -> concat_over.update(process_subset_opt(coords, 'coords')) (Pdb) u /Users/shoyer/dev/xarray/xarray/core/combine.py(212)_dataset_concat() -> concat_over = _calc_concat_over(datasets, dim, data_vars, coords) (Pdb) datasets[-5:] (<xarray.Dataset> Dimensions: (time: 32, x: 10) Coordinates: * time (time) datetime64[ns] 1979-12-28 1980-12-27 1981-12-28 ... * x (x) int64 0 1 2 3 4 5 6 7 8 9 dayofyear int64 362 Data variables: <this-array> (time, x) float64 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ..., <xarray.Dataset> Dimensions: (time: 32, x: 10) Coordinates: * time (time) datetime64[ns] 1979-12-29 1980-12-28 1981-12-29 ... * x (x) int64 0 1 2 3 4 5 6 7 8 9 dayofyear int64 363 Data variables: <this-array> (time, x) float64 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ..., <xarray.Dataset> Dimensions: (time: 32, x: 10) Coordinates: * time (time) datetime64[ns] 1979-12-30 1980-12-29 1981-12-30 ... * x (x) int64 0 1 2 3 4 5 6 7 8 9 dayofyear int64 364 Data variables: <this-array> (time, x) float64 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ..., <xarray.Dataset> Dimensions: (time: 32, x: 10) Coordinates: * time (time) datetime64[ns] 1979-12-31 1980-12-30 1981-12-31 ... * x (x) int64 0 1 2 3 4 5 6 7 8 9 dayofyear int64 365 Data variables: <this-array> (time, x) float64 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ..., <xarray.Dataset> Dimensions: (time: 8) Coordinates: * time (time) datetime64[ns] 1980-12-31 1984-12-31 1988-12-31 ... Data variables: <this-array> (time) float64 nan nan nan nan nan nan nan nan) (Pdb) ```

Yes, we have some logic that is supposed to handle missing groups in groupby, but apparently it's not working properly in this case: https://github.com/pydata/xarray/blob/bcd608101133c0cb84c74d341d22edef71ef4818/xarray/core/groupby.py#L331-L333

I don't know if you tried this yet, but if you changed the length to 365 and keep it with non-leap year, it still errors out so I guess the root issue is with how time.dayofyear uses 366 days?

``` import xarray as xr import numpy as np import pandas as pd

d1 = xr.DataArray(np.zeros(12000), [('time', pd.date_range('2004-01-01', freq='D', periods=12000))]) d2 = xr.DataArray(np.zeros((365, 10)), {'time': pd.date_range('1979-01-01', freq='D', periods=365), 'x': ('x', np.arange(10))}, dims=['time', 'x'])

d1.groupby('time.month') * d2.groupby('time.month').mean('time') print('this works')

no work

d1.groupby('time.dayofyear') * d2.groupby('time.dayofyear').mean('time') print('this doesn\'t work') ```

This seems to be related somehow to the fact that 1979 isn't a leap year, so we have a missing value to fill in for 366. For example, it works if we switch to 2004 for d2: ``` In [14]: d2 = xr.DataArray(np.zeros((366, 10)), {'time': pd.date_range('2004-01-01', freq='D', periods=366), 'x': ('x', np.ara ...: nge(10))}, dims=['time', 'x'])

In [15]: d1.groupby('time.dayofyear') - d2.groupby('time.dayofyear').mean('time') Out[15]: <xarray.DataArray (time: 12000, x: 10)> array([[ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], ..., [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.]]) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 * time (time) datetime64[ns] 1979-01-01 1979-01-02 1979-01-03 ... dayofyear (time) int64 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ... ```

@ahuang11 Thanks, I can reproduce on the development version of xarray.

Thanks for your quick response! From conda list xarray 0.9.6 <pip>

If you swap the length it errors out. ``` import xarray as xr import numpy as np import pandas as pd

d1 = xr.DataArray(np.zeros(12000), [('time', pd.date_range('1979-01-01', freq='D', periods=12000))]) d2 = xr.DataArray(np.zeros((366, 10)), {'time': pd.date_range('1979-01-01', freq='D', periods=366), 'x': ('x', np.arange(10))}, dims=['time', 'x'])

d1.groupby('time.month') - d2.groupby('time.month').mean('time') print('this works')

no work

d1.groupby('time.dayofyear') - d2.groupby('time.dayofyear').mean('time') print('this doesn\'t work') ```

``` <xarray.DataArray (time: 12000, x: 10)> array([[ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], ..., [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.]]) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 * time (time) datetime64[ns] 1979-01-01 1979-01-02 1979-01-03 ... month (time) int64 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ... this works

KeyError Traceback (most recent call last) <ipython-input-24-4c92f88d0a14> in <module>() 10 11 # no work ---> 12 d1.groupby('time.dayofyear') - d2.groupby('time.dayofyear').mean('time') 13 print('this doesn\'t work')

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/groupby.py in func(self, other) 316 g = f if not reflexive else lambda x, y: f(y, x) 317 applied = self._yield_binary_applied(g, other) --> 318 combined = self._combine(applied) 319 return combined 320 return func

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/groupby.py in _combine(self, applied, shortcut) 532 combined = self._concat_shortcut(applied, dim, positions) 533 else: --> 534 combined = concat(applied, dim) 535 combined = _maybe_reorder(combined, dim, positions) 536

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in concat(objs, dim, data_vars, coords, compat, positions, indexers, mode, concat_over) 118 raise TypeError('can only concatenate xarray Dataset and DataArray ' 119 'objects, got %s' % type(first_obj)) --> 120 return f(objs, dim, data_vars, coords, compat, positions) 121 122

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in _dataarray_concat(arrays, dim, data_vars, coords, compat, positions) 304 305 ds = _dataset_concat(datasets, dim, data_vars, coords, compat, --> 306 positions) 307 return arrays[0]._from_temp_dataset(ds, name) 308

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in _dataset_concat(datasets, dim, data_vars, coords, compat, positions) 210 datasets = align(*datasets, join='outer', copy=False, exclude=[dim]) 211 --> 212 concat_over = _calc_concat_over(datasets, dim, data_vars, coords) 213 214 def insert_result_variable(k, v):

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in _calc_concat_over(datasets, dim, data_vars, coords) 190 if dim in v.dims) 191 concat_over.update(process_subset_opt(data_vars, 'data_vars')) --> 192 concat_over.update(process_subset_opt(coords, 'coords')) 193 if dim in datasets[0]: 194 concat_over.add(dim)

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in process_subset_opt(opt, subset) 165 for ds in datasets[1:]) 166 # all nonindexes that are not the same in each dataset --> 167 concat_new = set(k for k in getattr(datasets[0], subset) 168 if k not in concat_over and differs(k)) 169 elif opt == 'all':

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in <genexpr>(.0) 166 # all nonindexes that are not the same in each dataset 167 concat_new = set(k for k in getattr(datasets[0], subset) --> 168 if k not in concat_over and differs(k)) 169 elif opt == 'all': 170 concat_new = (set(getattr(datasets[0], subset)) -

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in differs(vname) 163 v = datasets[0].variables[vname] 164 return any(not ds.variables[vname].equals(v) --> 165 for ds in datasets[1:]) 166 # all nonindexes that are not the same in each dataset 167 concat_new = set(k for k in getattr(datasets[0], subset)

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in <genexpr>(.0) 163 v = datasets[0].variables[vname] 164 return any(not ds.variables[vname].equals(v) --> 165 for ds in datasets[1:]) 166 # all nonindexes that are not the same in each dataset 167 concat_new = set(k for k in getattr(datasets[0], subset)

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/utils.py in getitem(self, key) 288 289 def getitem(self, key): --> 290 return self.mapping[key] 291 292 def iter(self):

KeyError: 'x' ```

I attempted to reproduce but have not been successful: In [94]: d1 = xr.DataArray(np.zeros(366), [('time', pd.date_range('1979-01-01', freq='D', periods=366))]) In [95]: d2 = xr.DataArray(np.zeros((1000, 10)), {'time': pd.date_range('1979-01-01', freq='D', periods=1000), 'x': ('x', np.arange(10))}, dims=['time', 'x']) In [96]: d1.groupby('time.dayofyear') - d2.groupby('time.dayofyear').mean('time') Out[96]: <xarray.DataArray (time: 366, x: 10)> array([[ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], ..., [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.]]) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 * time (time) datetime64[ns] 1979-01-01 1979-01-02 1979-01-03 ... dayofyear (time) int64 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ...