home / github / issues

Menu
  • Search all tables
  • GraphQL API

issues: 1120583442

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1120583442 I_kwDOAMm_X85Cyr8S 6230 [PERFORMANCE]: `isin` on `CFTimeIndex`-backed `Coordinate` slow 12237157 open 0     5 2022-02-01T12:04:02Z 2022-02-07T23:40:48Z   CONTRIBUTOR      

Is your feature request related to a problem?

I want to do coord1.isin.coord2 and it is quite slow when coords are large and of object type CFTimeIndex.

```python import xarray as xr import numpy as np

n=1000 coord1 = xr.cftime_range(start='2000', freq='MS', periods=n) coord2 = xr.cftime_range(start='2000', freq='3MS', periods=n)

cftimeindex: very fast

%timeit coord1.isin(coord2) # 743 µs ± 1.33 µs

np.isin on index.asi8

%timeit np.isin(coord1.asi8,coord2.asi8) # 7.83 ms ± 14.1 µs

da = xr.DataArray(np.random.random((n,n)),dims=['a','b'],coords={'a':coord1,'b':coord2})

when xr.DataArray coordinate slow

%timeit da.a.isin(da.b) # 94.9 ms ± 959 µs

when converting xr.DataArray coordinate back to index slow

%timeit np.isin(da.a.to_index(), da.b.to_index()) # 97.4 ms ± 819 µs

when converting xr.DataArray coordinate back to index asi

%timeit np.isin(da.a.to_index().asi8, da.b.to_index().asi8) # 7.89 ms ± 15.2 µs ```

Describe the solution you'd like

faster coord1.isin.coord2 by default. could we re-route here, e.g. to the alternative?

conversion from coordinate to_index() is costly I guess

Describe alternatives you've considered

np.isin(coord1.to_index().asi8, coord2.to_index().asi8 brings me nice speedups in https://github.com/pangeo-data/climpred/pull/724

Additional context

unsure whether this issue should go here on in cftime

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/6230/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    13221727 issue

Links from other tables

  • 3 rows from issues_id in issues_labels
  • 5 rows from issue in issue_comments
Powered by Datasette · Queries took 159.523ms · About: xarray-datasette