html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/1527#issuecomment-700603268,https://api.github.com/repos/pydata/xarray/issues/1527,700603268,MDEyOklzc3VlQ29tbWVudDcwMDYwMzI2OA==,10194086,2020-09-29T10:05:42Z,2020-09-29T10:05:42Z,MEMBER,"However, `other_sel` in `_yield_binary_applied` looks like: ```python array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 dayofyear int64 1 ``` https://github.com/pydata/xarray/blob/f821fe20595c3700375ccecebf88e01a61444777/xarray/core/groupby.py#L484","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,253107677 https://github.com/pydata/xarray/issues/1527#issuecomment-700600387,https://api.github.com/repos/pydata/xarray/issues/1527,700600387,MDEyOklzc3VlQ29tbWVudDcwMDYwMDM4Nw==,10194086,2020-09-29T10:00:14Z,2020-09-29T10:00:14Z,MEMBER,"This is still relevant. Now raises ```python ValueError: 'x' not present in all datasets and coords='different'. Either add 'x' to datasets where it is missing or specify coords='minimal'. ``` I think the issue may be that `_dummy_copy` does not include the x-dimension: ```python xr.core.groupby._dummy_copy(d2.groupby('time.dayofyear').mean('time')) ``` yields ```python array(nan) ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,253107677 https://github.com/pydata/xarray/issues/1527#issuecomment-700462185,https://api.github.com/repos/pydata/xarray/issues/1527,700462185,MDEyOklzc3VlQ29tbWVudDcwMDQ2MjE4NQ==,26384082,2020-09-29T05:52:54Z,2020-09-29T05:52:54Z,NONE,"In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity If this issue remains relevant, please comment here or remove the `stale` label; otherwise it will be marked as closed automatically ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,253107677 https://github.com/pydata/xarray/issues/1527#issuecomment-325164022,https://api.github.com/repos/pydata/xarray/issues/1527,325164022,MDEyOklzc3VlQ29tbWVudDMyNTE2NDAyMg==,1217238,2017-08-26T21:54:27Z,2017-08-26T21:54:27Z,MEMBER,"Clues about what's going on from pdb: ``` In [23]: import pdb; pdb.pm() > /Users/shoyer/dev/xarray/xarray/core/utils.py(290)__getitem__() -> return self.mapping[key] (Pdb) u > /Users/shoyer/dev/xarray/xarray/core/combine.py(165)() -> for ds in datasets[1:]) (Pdb) u > /Users/shoyer/dev/xarray/xarray/core/combine.py(165)differs() -> for ds in datasets[1:]) (Pdb) u > /Users/shoyer/dev/xarray/xarray/core/combine.py(168)() -> if k not in concat_over and differs(k)) (Pdb) u > /Users/shoyer/dev/xarray/xarray/core/combine.py(167)process_subset_opt() -> concat_new = set(k for k in getattr(datasets[0], subset) (Pdb) u > /Users/shoyer/dev/xarray/xarray/core/combine.py(192)_calc_concat_over() -> concat_over.update(process_subset_opt(coords, 'coords')) (Pdb) u > /Users/shoyer/dev/xarray/xarray/core/combine.py(212)_dataset_concat() -> concat_over = _calc_concat_over(datasets, dim, data_vars, coords) (Pdb) datasets[-5:] ( Dimensions: (time: 32, x: 10) Coordinates: * time (time) datetime64[ns] 1979-12-28 1980-12-27 1981-12-28 ... * x (x) int64 0 1 2 3 4 5 6 7 8 9 dayofyear int64 362 Data variables: (time, x) float64 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ..., Dimensions: (time: 32, x: 10) Coordinates: * time (time) datetime64[ns] 1979-12-29 1980-12-28 1981-12-29 ... * x (x) int64 0 1 2 3 4 5 6 7 8 9 dayofyear int64 363 Data variables: (time, x) float64 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ..., Dimensions: (time: 32, x: 10) Coordinates: * time (time) datetime64[ns] 1979-12-30 1980-12-29 1981-12-30 ... * x (x) int64 0 1 2 3 4 5 6 7 8 9 dayofyear int64 364 Data variables: (time, x) float64 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ..., Dimensions: (time: 32, x: 10) Coordinates: * time (time) datetime64[ns] 1979-12-31 1980-12-30 1981-12-31 ... * x (x) int64 0 1 2 3 4 5 6 7 8 9 dayofyear int64 365 Data variables: (time, x) float64 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ..., Dimensions: (time: 8) Coordinates: * time (time) datetime64[ns] 1980-12-31 1984-12-31 1988-12-31 ... Data variables: (time) float64 nan nan nan nan nan nan nan nan) (Pdb) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,253107677 https://github.com/pydata/xarray/issues/1527#issuecomment-325163820,https://api.github.com/repos/pydata/xarray/issues/1527,325163820,MDEyOklzc3VlQ29tbWVudDMyNTE2MzgyMA==,1217238,2017-08-26T21:50:08Z,2017-08-26T21:50:08Z,MEMBER,"Yes, we have some logic that is supposed to handle missing groups in groupby, but apparently it's not working properly in this case: https://github.com/pydata/xarray/blob/bcd608101133c0cb84c74d341d22edef71ef4818/xarray/core/groupby.py#L331-L333","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,253107677 https://github.com/pydata/xarray/issues/1527#issuecomment-325163311,https://api.github.com/repos/pydata/xarray/issues/1527,325163311,MDEyOklzc3VlQ29tbWVudDMyNTE2MzMxMQ==,15331990,2017-08-26T21:38:35Z,2017-08-26T21:38:35Z,CONTRIBUTOR,"I don't know if you tried this yet, but if you changed the length to 365 and keep it with non-leap year, it still errors out so I guess the root issue is with how time.dayofyear uses 366 days? ``` import xarray as xr import numpy as np import pandas as pd d1 = xr.DataArray(np.zeros(12000), [('time', pd.date_range('2004-01-01', freq='D', periods=12000))]) d2 = xr.DataArray(np.zeros((365, 10)), {'time': pd.date_range('1979-01-01', freq='D', periods=365), 'x': ('x', np.arange(10))}, dims=['time', 'x']) d1.groupby('time.month') * d2.groupby('time.month').mean('time') print('this works') # no work d1.groupby('time.dayofyear') * d2.groupby('time.dayofyear').mean('time') print('this doesn\'t work') ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,253107677 https://github.com/pydata/xarray/issues/1527#issuecomment-325152586,https://api.github.com/repos/pydata/xarray/issues/1527,325152586,MDEyOklzc3VlQ29tbWVudDMyNTE1MjU4Ng==,1217238,2017-08-26T18:02:35Z,2017-08-26T18:02:35Z,MEMBER,"This seems to be related somehow to the fact that 1979 isn't a leap year, so we have a missing value to fill in for 366. For example, it works if we switch to 2004 for `d2`: ``` In [14]: d2 = xr.DataArray(np.zeros((366, 10)), {'time': pd.date_range('2004-01-01', freq='D', periods=366), 'x': ('x', np.ara ...: nge(10))}, dims=['time', 'x']) In [15]: d1.groupby('time.dayofyear') - d2.groupby('time.dayofyear').mean('time') Out[15]: array([[ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], ..., [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.]]) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 * time (time) datetime64[ns] 1979-01-01 1979-01-02 1979-01-03 ... dayofyear (time) int64 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ... ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,253107677 https://github.com/pydata/xarray/issues/1527#issuecomment-325152245,https://api.github.com/repos/pydata/xarray/issues/1527,325152245,MDEyOklzc3VlQ29tbWVudDMyNTE1MjI0NQ==,1217238,2017-08-26T17:55:30Z,2017-08-26T17:55:30Z,MEMBER,"@ahuang11 Thanks, I can reproduce on the development version of xarray.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,253107677 https://github.com/pydata/xarray/issues/1527#issuecomment-325149596,https://api.github.com/repos/pydata/xarray/issues/1527,325149596,MDEyOklzc3VlQ29tbWVudDMyNTE0OTU5Ng==,15331990,2017-08-26T17:28:28Z,2017-08-26T17:32:00Z,CONTRIBUTOR,"Thanks for your quick response! From `conda list` `xarray 0.9.6 ` If you swap the length it errors out. ``` import xarray as xr import numpy as np import pandas as pd d1 = xr.DataArray(np.zeros(12000), [('time', pd.date_range('1979-01-01', freq='D', periods=12000))]) d2 = xr.DataArray(np.zeros((366, 10)), {'time': pd.date_range('1979-01-01', freq='D', periods=366), 'x': ('x', np.arange(10))}, dims=['time', 'x']) d1.groupby('time.month') - d2.groupby('time.month').mean('time') print('this works') # no work d1.groupby('time.dayofyear') - d2.groupby('time.dayofyear').mean('time') print('this doesn\'t work') ``` ``` array([[ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], ..., [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.]]) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 * time (time) datetime64[ns] 1979-01-01 1979-01-02 1979-01-03 ... month (time) int64 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ... this works --------------------------------------------------------------------------- KeyError Traceback (most recent call last) in () 10 11 # no work ---> 12 d1.groupby('time.dayofyear') - d2.groupby('time.dayofyear').mean('time') 13 print('this doesn\'t work') /data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/groupby.py in func(self, other) 316 g = f if not reflexive else lambda x, y: f(y, x) 317 applied = self._yield_binary_applied(g, other) --> 318 combined = self._combine(applied) 319 return combined 320 return func /data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/groupby.py in _combine(self, applied, shortcut) 532 combined = self._concat_shortcut(applied, dim, positions) 533 else: --> 534 combined = concat(applied, dim) 535 combined = _maybe_reorder(combined, dim, positions) 536 /data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in concat(objs, dim, data_vars, coords, compat, positions, indexers, mode, concat_over) 118 raise TypeError('can only concatenate xarray Dataset and DataArray ' 119 'objects, got %s' % type(first_obj)) --> 120 return f(objs, dim, data_vars, coords, compat, positions) 121 122 /data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in _dataarray_concat(arrays, dim, data_vars, coords, compat, positions) 304 305 ds = _dataset_concat(datasets, dim, data_vars, coords, compat, --> 306 positions) 307 return arrays[0]._from_temp_dataset(ds, name) 308 /data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in _dataset_concat(datasets, dim, data_vars, coords, compat, positions) 210 datasets = align(*datasets, join='outer', copy=False, exclude=[dim]) 211 --> 212 concat_over = _calc_concat_over(datasets, dim, data_vars, coords) 213 214 def insert_result_variable(k, v): /data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in _calc_concat_over(datasets, dim, data_vars, coords) 190 if dim in v.dims) 191 concat_over.update(process_subset_opt(data_vars, 'data_vars')) --> 192 concat_over.update(process_subset_opt(coords, 'coords')) 193 if dim in datasets[0]: 194 concat_over.add(dim) /data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in process_subset_opt(opt, subset) 165 for ds in datasets[1:]) 166 # all nonindexes that are not the same in each dataset --> 167 concat_new = set(k for k in getattr(datasets[0], subset) 168 if k not in concat_over and differs(k)) 169 elif opt == 'all': /data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in (.0) 166 # all nonindexes that are not the same in each dataset 167 concat_new = set(k for k in getattr(datasets[0], subset) --> 168 if k not in concat_over and differs(k)) 169 elif opt == 'all': 170 concat_new = (set(getattr(datasets[0], subset)) - /data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in differs(vname) 163 v = datasets[0].variables[vname] 164 return any(not ds.variables[vname].equals(v) --> 165 for ds in datasets[1:]) 166 # all nonindexes that are not the same in each dataset 167 concat_new = set(k for k in getattr(datasets[0], subset) /data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in (.0) 163 v = datasets[0].variables[vname] 164 return any(not ds.variables[vname].equals(v) --> 165 for ds in datasets[1:]) 166 # all nonindexes that are not the same in each dataset 167 concat_new = set(k for k in getattr(datasets[0], subset) /data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/utils.py in __getitem__(self, key) 288 289 def __getitem__(self, key): --> 290 return self.mapping[key] 291 292 def __iter__(self): KeyError: 'x' ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,253107677 https://github.com/pydata/xarray/issues/1527#issuecomment-325147198,https://api.github.com/repos/pydata/xarray/issues/1527,325147198,MDEyOklzc3VlQ29tbWVudDMyNTE0NzE5OA==,1217238,2017-08-26T17:03:18Z,2017-08-26T17:03:18Z,MEMBER,"I attempted to reproduce but have not been successful: ``` In [94]: d1 = xr.DataArray(np.zeros(366), [('time', pd.date_range('1979-01-01', freq='D', periods=366))]) In [95]: d2 = xr.DataArray(np.zeros((1000, 10)), {'time': pd.date_range('1979-01-01', freq='D', periods=1000), 'x': ('x', np.arange(10))}, dims=['time', 'x']) In [96]: d1.groupby('time.dayofyear') - d2.groupby('time.dayofyear').mean('time') Out[96]: array([[ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], ..., [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.]]) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 * time (time) datetime64[ns] 1979-01-01 1979-01-02 1979-01-03 ... dayofyear (time) int64 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ... ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,253107677