html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/pull/5731#issuecomment-978098589,https://api.github.com/repos/pydata/xarray/issues/5731,978098589,IC_kwDOAMm_X846TJmd,1217238,2021-11-24T17:48:52Z,2021-11-24T17:48:52Z,MEMBER,I have a follow-up PR that simplifies the internal logic a little bit more: https://github.com/pydata/xarray/pull/6025,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-978081711,https://api.github.com/repos/pydata/xarray/issues/5731,978081711,IC_kwDOAMm_X846TFev,2448579,2021-11-24T17:25:17Z,2021-11-24T17:25:17Z,MEMBER,Thanks @Gijom ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-977725089,https://api.github.com/repos/pydata/xarray/issues/5731,977725089,IC_kwDOAMm_X846Ruah,9466648,2021-11-24T10:08:04Z,2021-11-24T10:08:04Z,CONTRIBUTOR,Sorry for the unresponsiveness. I confirm that using the `asfloat` conversion also works for me. Should I close the issue or should I let you do it ?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-977464974,https://api.github.com/repos/pydata/xarray/issues/5731,977464974,IC_kwDOAMm_X846Qu6O,2448579,2021-11-24T03:18:36Z,2021-11-24T03:18:36Z,MEMBER,"> so it probably makes sense to merge it now and save the other bug fix for a later issue? It was easy .Should be good to go if tests pass.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-977243782,https://api.github.com/repos/pydata/xarray/issues/5731,977243782,IC_kwDOAMm_X846P46G,1217238,2021-11-23T22:47:30Z,2021-11-23T22:47:30Z,MEMBER,"Checking in here -- it seems like this PR surfaced an existing bug (incorrect calculation of correlation when using dask), at the same time as it corrected a different bug, which is removing the hard dependency on dask for calculating correlation? If I'm understanding correctly, this is still a net win (fixing one bug), so it probably makes sense to merge it now and save the other bug fix for a later issue?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-908040499,https://api.github.com/repos/pydata/xarray/issues/5731,908040499,IC_kwDOAMm_X842H5kz,41898282,2021-08-30T05:26:55Z,2021-08-31T15:59:38Z,CONTRIBUTOR,"## Unit Test Results          6 files           6 suites   49m 53s :stopwatch: 16 246 tests 14 512 :heavy_check_mark: 1 734 :zzz: 0 :x: 90 672 runs  82 496 :heavy_check_mark: 8 176 :zzz: 0 :x: Results for commit 3dde39b1. :recycle: This comment has been updated with latest results.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-909335676,https://api.github.com/repos/pydata/xarray/issues/5731,909335676,IC_kwDOAMm_X842M1x8,2448579,2021-08-31T15:19:57Z,2021-08-31T15:19:57Z,MEMBER,"Ouch here's the problem: `da_a` has dtype int if it has no NaNs. this dtype could change but need not change after the masking. Importantly `skipna=False` by default for `int` dtypes (since they can't represent NaNs). The fix is to specify `skipna=True` explicitly in the `.mean` calls. Alternatively we could stick an `.astype(float)` in `_get_valid_values`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-909314125,https://api.github.com/repos/pydata/xarray/issues/5731,909314125,IC_kwDOAMm_X842MwhN,2448579,2021-08-31T14:54:51Z,2021-08-31T14:54:51Z,MEMBER,"oh never mind: I can reproduce with ``` da_a = xr.DataArray([[1, 2], [2, 1]], dims=[""x"", ""time""]) da_b = xr.DataArray([[1, 2], [1, np.nan]], dims=[""x"", ""time""]) dim = None xr.testing.assert_equal( xr.corr(da_a, da_b, dim=dim), xr.corr(da_a.chunk(), da_b.chunk(), dim=dim) ) xr.testing.assert_equal( xr.corr(da_a, da_b, dim=dim), xr.corr(da_a, da_b.chunk(), dim=dim) ) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-909311207,https://api.github.com/repos/pydata/xarray/issues/5731,909311207,IC_kwDOAMm_X842Mvzn,2448579,2021-08-31T14:51:43Z,2021-08-31T14:52:06Z,MEMBER,"Thanks @Gijom, I can't reproduce. Can you post the output of `xr.show_versions()`? Here's mine
``` INSTALLED VERSIONS ------------------ commit: None python: 3.8.10 | packaged by conda-forge | (default, May 10 2021, 22:58:09) [Clang 11.1.0 ] python-bits: 64 OS: Darwin OS-release: 19.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.19.0 pandas: 1.3.1 numpy: 1.21.1 scipy: 1.5.3 netCDF4: 1.5.6 pydap: None h5netcdf: 0.11.0 h5py: 3.3.0 Nio: None zarr: 2.8.3 cftime: 1.5.0 nc_time_axis: 1.3.1 PseudoNetCDF: None rasterio: None cfgrib: None iris: 3.0.4 bottleneck: 1.3.2 dask: 2021.07.2 distributed: 2021.07.2 matplotlib: 3.4.2 cartopy: 0.19.0.post1 seaborn: 0.11.1 numbagg: None pint: 0.17 setuptools: 49.6.0.post20210108 pip: 21.2.3 conda: 4.10.3 pytest: 6.2.4 IPython: 7.26.0 sphinx: 4.1.2 ```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-904403621,https://api.github.com/repos/pydata/xarray/issues/5731,904403621,IC_kwDOAMm_X8416Bql,9466648,2021-08-24T07:43:28Z,2021-08-31T14:41:01Z,CONTRIBUTOR,"I included the proposed changes, thanks for the help. However the new test I implemented does not go through and I fail to understand why. Would you be able to check ? The idea of the test is to ensure that lazy computations give the same results than normal ones. I tracked down the problem to line 1377 of `computation.py`: ```python demeaned_da_a = da_a - da_a.mean(dim=dim) # <-- this one returns nan upon computation (.compute()) demeaned_da_b = da_b - da_b.mean(dim=dim) # <-- this one returns the correct value although the input is masked with nans in the same way ``` The values before the `mean` call are: ``` da_a.compute() = (x: 2, time: 2)> array([[ 1., 2.], [ 2., nan]]) da_b.compute() = (x: 2, time: 2)> array([[ 1., 2.], [ 1., nan]]) ``` And for the means (dim is None): ``` da_a.mean(dim=dim).compute()= ()> array(nan) da_b.mean(dim=dim).compute()= ()> array(1.33333333) ``` For non lazy computations everything seems fine.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-909300955,https://api.github.com/repos/pydata/xarray/issues/5731,909300955,IC_kwDOAMm_X842MtTb,9466648,2021-08-31T14:40:24Z,2021-08-31T14:40:24Z,CONTRIBUTOR,"I was referring to: https://github.com/pydata/xarray/pull/5731#issuecomment-904403621 Otherwise I am working on the conflict with the main branch. Should be done soon. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-909297283,https://api.github.com/repos/pydata/xarray/issues/5731,909297283,IC_kwDOAMm_X842MsaD,2448579,2021-08-31T14:36:19Z,2021-08-31T14:36:19Z,MEMBER,Thanks @Gijom can you post the error for those tests please.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-909095076,https://api.github.com/repos/pydata/xarray/issues/5731,909095076,IC_kwDOAMm_X842L7Ck,9466648,2021-08-31T10:10:15Z,2021-08-31T10:10:15Z,CONTRIBUTOR,"@keewis indeed this was the problem. I removed it and corrected one test which did not have the @require_dask decorator. There is still a test error with the test arrays 5 and 6: the lazy array version do not return the same value than the usual corr. I consider this a different bug and just removed the tests with a TODO in the code. The last commit pass the tests on my machine and I added the whats-new text to version 0.19.1.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-908039831,https://api.github.com/repos/pydata/xarray/issues/5731,908039831,IC_kwDOAMm_X842H5aX,14808389,2021-08-30T05:24:57Z,2021-08-30T05:24:57Z,MEMBER,"> The already existing test function `test_corr` does not have a `@requires_dask` decorator. But it did not fail as it should have. Not sure why. the reason for this is that `dask` is imported using `importorskip` at the top of the module: https://github.com/pydata/xarray/blob/a96154a85e330ce5aec6c34e49997833c1ae0ba2/xarray/tests/test_computation.py#L27 Since most tests already have `requires_dask` I think it should be fine to just remove the `importandskip` call.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-903840548,https://api.github.com/repos/pydata/xarray/issues/5731,903840548,IC_kwDOAMm_X84134Mk,2448579,2021-08-23T14:46:28Z,2021-08-23T14:46:47Z,MEMBER,"Thanks @Gijom! > he already existing test function test_corr does not have a @requires_dask decorator. But it did not fail as it should have. Not sure why. I can't tell either. It would be good to track this down. > I am not sure of the whats-new.rst format and I thus did not add it. Should I just add a bullet item on the top of the file ? Yes just copy an existing entry and modify it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237 https://github.com/pydata/xarray/pull/5731#issuecomment-903574187,https://api.github.com/repos/pydata/xarray/issues/5731,903574187,IC_kwDOAMm_X84123Kr,9466648,2021-08-23T08:57:36Z,2021-08-23T08:57:36Z,CONTRIBUTOR,"Concerning the tests I added a test to check that lazy correlations have identical results to non-lazy correlations. But this is not directly related to the bug fix. Indeed there is a need to have a test that checks if the non-lazy correlation works without dask installed. The already existing test function `test_corr` does not have a `@requires_dask` decorator. But it did not fail as it should have. Not sure why. About documenting the changes, I am not sure of the whats-new.rst format and I thus did not add it. Should I just add a bullet item on the top of the file ?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,976790237