html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/2215#issuecomment-396953390,https://api.github.com/repos/pydata/xarray/issues/2215,396953390,MDEyOklzc3VlQ29tbWVudDM5Njk1MzM5MA==,25231875,2018-06-13T14:16:19Z,2018-06-13T14:17:46Z,NONE,"@shoyer That did it. Under pandas 0.22, the DataArrays in `aligned` are all NaNs. I updated to pandas 0.23, and the non-NaN values were there as expected. To double-check, I downgraded to 0.22 again and got all NaNs again. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,329438885 https://github.com/pydata/xarray/issues/2215#issuecomment-394758682,https://api.github.com/repos/pydata/xarray/issues/2215,394758682,MDEyOklzc3VlQ29tbWVudDM5NDc1ODY4Mg==,25231875,2018-06-05T15:42:15Z,2018-06-05T16:23:12Z,NONE,"I found a way to reproduce the error. One of the MuliIndex levels on the DataArrays has NaNs in it. If I remove that level, the correct values appear in the result. Should the presence of that MultiIndex level cause this behavior? ``` import string import numpy as np import xarray as xr dims = (""x"", ""y"") shape = (10, 5) das = [] for j in (0, 1): data = np.full(shape, np.nan, dtype=""float64"") for i in range(shape[0]): data[i, i % shape[1]] = float(i) coords_d = { ""ints"": (""x"", range(j*shape[0], (j+1)*shape[0])), ""nans"": (""x"", np.array([np.nan] * shape[0], dtype=""float64"")), ""lower"": (""y"", list(string.ascii_lowercase[:shape[1]])) } da = xr.DataArray(data=data, dims=dims, coords=coords_d) da.set_index(append=True, inplace=True, x=[""ints"", ""nans""], y=[""lower""]) das.append(da) nonzeros_raw = [np.nonzero(~np.isnan(da)) for da in das] print(""nonzeros_raw: "") print(nonzeros_raw) aligned = xr.align(*das, join=""outer"") nonzeros_aligned = [np.nonzero(~np.isnan(da)) for da in aligned] print(""nonzeros_aligned: "") print(nonzeros_aligned) assert nonzeros_raw[0].shape == nonzeros_aligned[0].shape ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,329438885 https://github.com/pydata/xarray/issues/2215#issuecomment-394769300,https://api.github.com/repos/pydata/xarray/issues/2215,394769300,MDEyOklzc3VlQ29tbWVudDM5NDc2OTMwMA==,25231875,2018-06-05T16:12:30Z,2018-06-05T16:12:30Z,NONE,"This is what I would expect to see returned by align(): ``` ( array([[ 0., nan, nan, nan, nan], [nan, 1., nan, nan, nan], [nan, nan, 2., nan, nan], [nan, nan, nan, 3., nan], [nan, nan, nan, nan, 4.], [ 5., nan, nan, nan, nan], [nan, 6., nan, nan, nan], [nan, nan, 7., nan, nan], [nan, nan, nan, 8., nan], [nan, nan, nan, nan, 9.], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan]]) Coordinates: * x (x) MultiIndex - ints (x) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 - nans (x) object nan nan nan nan nan nan nan nan nan nan nan nan nan ... * y (y) object 'a' 'b' 'c' 'd' 'e', array([[nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [ 0., nan, nan, nan, nan], [nan, 1., nan, nan, nan], [nan, nan, 2., nan, nan], [nan, nan, nan, 3., nan], [nan, nan, nan, nan, 4.], [ 5., nan, nan, nan, nan], [nan, 6., nan, nan, nan], [nan, nan, 7., nan, nan], [nan, nan, nan, 8., nan], [nan, nan, nan, nan, 9.]]) Coordinates: * x (x) MultiIndex - ints (x) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 - nans (x) object nan nan nan nan nan nan nan nan nan nan nan nan nan ... * y (y) object 'a' 'b' 'c' 'd' 'e') ``` I see something very similar, but with the `nans` level removed, if I do this: `xr.align(*[da.reset_index(""nans"", drop=True) for da in das], join=""outer"")`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,329438885 https://github.com/pydata/xarray/issues/2215#issuecomment-394765054,https://api.github.com/repos/pydata/xarray/issues/2215,394765054,MDEyOklzc3VlQ29tbWVudDM5NDc2NTA1NA==,25231875,2018-06-05T15:59:57Z,2018-06-05T15:59:57Z,NONE,"For clarity, here are the prints of the arrays before and after alignment: Before alignment: ``` [ array([[ 0., nan, nan, nan, nan], [nan, 1., nan, nan, nan], [nan, nan, 2., nan, nan], [nan, nan, nan, 3., nan], [nan, nan, nan, nan, 4.], [ 5., nan, nan, nan, nan], [nan, 6., nan, nan, nan], [nan, nan, 7., nan, nan], [nan, nan, nan, 8., nan], [nan, nan, nan, nan, 9.]]) Coordinates: * x (x) MultiIndex - ints (x) int64 0 1 2 3 4 5 6 7 8 9 - nans (x) float64 nan nan nan nan nan nan nan nan nan nan * y (y) object 'a' 'b' 'c' 'd' 'e', array([[ 0., nan, nan, nan, nan], [nan, 1., nan, nan, nan], [nan, nan, 2., nan, nan], [nan, nan, nan, 3., nan], [nan, nan, nan, nan, 4.], [ 5., nan, nan, nan, nan], [nan, 6., nan, nan, nan], [nan, nan, 7., nan, nan], [nan, nan, nan, 8., nan], [nan, nan, nan, nan, 9.]]) Coordinates: * x (x) MultiIndex - ints (x) int64 10 11 12 13 14 15 16 17 18 19 - nans (x) float64 nan nan nan nan nan nan nan nan nan nan * y (y) object 'a' 'b' 'c' 'd' 'e'] ``` After alignment: ``` ( array([[nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan]]) Coordinates: * x (x) MultiIndex - ints (x) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 - nans (x) object nan nan nan nan nan nan nan nan nan nan nan nan nan ... * y (y) object 'a' 'b' 'c' 'd' 'e', array([[nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan], [nan, nan, nan, nan, nan]]) Coordinates: * x (x) MultiIndex - ints (x) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 - nans (x) object nan nan nan nan nan nan nan nan nan nan nan nan nan ... * y (y) object 'a' 'b' 'c' 'd' 'e') ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,329438885 https://github.com/pydata/xarray/issues/2215#issuecomment-394762522,https://api.github.com/repos/pydata/xarray/issues/2215,394762522,MDEyOklzc3VlQ29tbWVudDM5NDc2MjUyMg==,25231875,2018-06-05T15:52:57Z,2018-06-05T15:52:57Z,NONE,"Since the align is an outer join, I would expect all the non-NaN values in the original DataArrays to also appear in the aligned DataArrays. Perhaps I am misinterpreting the behavior of `join=""outer""`. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,329438885