issue_comments

Table actions

10 rows where author_association = "MEMBER" and issue = 976790237 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 3

issue 1

  • Make xr.corr and xr.map_blocks work without dask · 10 ✖

author_association 1

  • MEMBER · 10 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
978098589 https://github.com/pydata/xarray/pull/5731#issuecomment-978098589 https://api.github.com/repos/pydata/xarray/issues/5731 IC_kwDOAMm_X846TJmd shoyer 1217238 2021-11-24T17:48:52Z 2021-11-24T17:48:52Z MEMBER

I have a follow-up PR that simplifies the internal logic a little bit more: https://github.com/pydata/xarray/pull/6025

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Make xr.corr and xr.map_blocks work without dask 976790237
978081711 https://github.com/pydata/xarray/pull/5731#issuecomment-978081711 https://api.github.com/repos/pydata/xarray/issues/5731 IC_kwDOAMm_X846TFev dcherian 2448579 2021-11-24T17:25:17Z 2021-11-24T17:25:17Z MEMBER

Thanks @Gijom 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Make xr.corr and xr.map_blocks work without dask 976790237
977464974 https://github.com/pydata/xarray/pull/5731#issuecomment-977464974 https://api.github.com/repos/pydata/xarray/issues/5731 IC_kwDOAMm_X846Qu6O dcherian 2448579 2021-11-24T03:18:36Z 2021-11-24T03:18:36Z MEMBER

so it probably makes sense to merge it now and save the other bug fix for a later issue?

It was easy .Should be good to go if tests pass.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Make xr.corr and xr.map_blocks work without dask 976790237
977243782 https://github.com/pydata/xarray/pull/5731#issuecomment-977243782 https://api.github.com/repos/pydata/xarray/issues/5731 IC_kwDOAMm_X846P46G shoyer 1217238 2021-11-23T22:47:30Z 2021-11-23T22:47:30Z MEMBER

Checking in here -- it seems like this PR surfaced an existing bug (incorrect calculation of correlation when using dask), at the same time as it corrected a different bug, which is removing the hard dependency on dask for calculating correlation?

If I'm understanding correctly, this is still a net win (fixing one bug), so it probably makes sense to merge it now and save the other bug fix for a later issue?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Make xr.corr and xr.map_blocks work without dask 976790237
909335676 https://github.com/pydata/xarray/pull/5731#issuecomment-909335676 https://api.github.com/repos/pydata/xarray/issues/5731 IC_kwDOAMm_X842M1x8 dcherian 2448579 2021-08-31T15:19:57Z 2021-08-31T15:19:57Z MEMBER

Ouch here's the problem: da_a has dtype int if it has no NaNs. this dtype could change but need not change after the masking.

Importantly skipna=False by default for int dtypes (since they can't represent NaNs).

The fix is to specify skipna=True explicitly in the .mean calls.

Alternatively we could stick an .astype(float) in _get_valid_values

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Make xr.corr and xr.map_blocks work without dask 976790237
909314125 https://github.com/pydata/xarray/pull/5731#issuecomment-909314125 https://api.github.com/repos/pydata/xarray/issues/5731 IC_kwDOAMm_X842MwhN dcherian 2448579 2021-08-31T14:54:51Z 2021-08-31T14:54:51Z MEMBER

oh never mind: I can reproduce with ``` da_a = xr.DataArray([[1, 2], [2, 1]], dims=["x", "time"]) da_b = xr.DataArray([[1, 2], [1, np.nan]], dims=["x", "time"])

dim = None

xr.testing.assert_equal( xr.corr(da_a, da_b, dim=dim), xr.corr(da_a.chunk(), da_b.chunk(), dim=dim) ) xr.testing.assert_equal( xr.corr(da_a, da_b, dim=dim), xr.corr(da_a, da_b.chunk(), dim=dim) ) ```

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Make xr.corr and xr.map_blocks work without dask 976790237
909311207 https://github.com/pydata/xarray/pull/5731#issuecomment-909311207 https://api.github.com/repos/pydata/xarray/issues/5731 IC_kwDOAMm_X842Mvzn dcherian 2448579 2021-08-31T14:51:43Z 2021-08-31T14:52:06Z MEMBER

Thanks @Gijom, I can't reproduce. Can you post the output of xr.show_versions()?

Here's mine

``` INSTALLED VERSIONS ------------------ commit: None python: 3.8.10 | packaged by conda-forge | (default, May 10 2021, 22:58:09) [Clang 11.1.0 ] python-bits: 64 OS: Darwin OS-release: 19.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.19.0 pandas: 1.3.1 numpy: 1.21.1 scipy: 1.5.3 netCDF4: 1.5.6 pydap: None h5netcdf: 0.11.0 h5py: 3.3.0 Nio: None zarr: 2.8.3 cftime: 1.5.0 nc_time_axis: 1.3.1 PseudoNetCDF: None rasterio: None cfgrib: None iris: 3.0.4 bottleneck: 1.3.2 dask: 2021.07.2 distributed: 2021.07.2 matplotlib: 3.4.2 cartopy: 0.19.0.post1 seaborn: 0.11.1 numbagg: None pint: 0.17 setuptools: 49.6.0.post20210108 pip: 21.2.3 conda: 4.10.3 pytest: 6.2.4 IPython: 7.26.0 sphinx: 4.1.2 ``` 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Make xr.corr and xr.map_blocks work without dask 976790237
909297283 https://github.com/pydata/xarray/pull/5731#issuecomment-909297283 https://api.github.com/repos/pydata/xarray/issues/5731 IC_kwDOAMm_X842MsaD dcherian 2448579 2021-08-31T14:36:19Z 2021-08-31T14:36:19Z MEMBER

Thanks @Gijom can you post the error for those tests please.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Make xr.corr and xr.map_blocks work without dask 976790237
908039831 https://github.com/pydata/xarray/pull/5731#issuecomment-908039831 https://api.github.com/repos/pydata/xarray/issues/5731 IC_kwDOAMm_X842H5aX keewis 14808389 2021-08-30T05:24:57Z 2021-08-30T05:24:57Z MEMBER

The already existing test function test_corr does not have a @requires_dask decorator. But it did not fail as it should have. Not sure why.

the reason for this is that dask is imported using importorskip at the top of the module: https://github.com/pydata/xarray/blob/a96154a85e330ce5aec6c34e49997833c1ae0ba2/xarray/tests/test_computation.py#L27 Since most tests already have requires_dask I think it should be fine to just remove the importandskip call.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Make xr.corr and xr.map_blocks work without dask 976790237
903840548 https://github.com/pydata/xarray/pull/5731#issuecomment-903840548 https://api.github.com/repos/pydata/xarray/issues/5731 IC_kwDOAMm_X84134Mk dcherian 2448579 2021-08-23T14:46:28Z 2021-08-23T14:46:47Z MEMBER

Thanks @Gijom!

he already existing test function test_corr does not have a @requires_dask decorator. But it did not fail as it should have. Not sure why.

I can't tell either. It would be good to track this down.

I am not sure of the whats-new.rst format and I thus did not add it. Should I just add a bullet item on the top of the file ?

Yes just copy an existing entry and modify it.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Make xr.corr and xr.map_blocks work without dask 976790237

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);