home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

4 rows where author_association = "MEMBER", issue = 307318224 and user = 1217238 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • shoyer · 4 ✖

issue 1

  • Slicing DataArray can take longer than not slicing · 4 ✖

author_association 1

  • MEMBER · 4 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
460881018 https://github.com/pydata/xarray/issues/2004#issuecomment-460881018 https://api.github.com/repos/pydata/xarray/issues/2004 MDEyOklzc3VlQ29tbWVudDQ2MDg4MTAxOA== shoyer 1217238 2019-02-06T02:32:46Z 2019-02-06T02:32:46Z MEMBER

The performance difference here does indeed to have been fixed with netCDF-C 4.6.2 (but see also https://github.com/pydata/xarray/issues/2747)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Slicing DataArray can take longer than not slicing 307318224
375067743 https://github.com/pydata/xarray/issues/2004#issuecomment-375067743 https://api.github.com/repos/pydata/xarray/issues/2004 MDEyOklzc3VlQ29tbWVudDM3NTA2Nzc0Mw== shoyer 1217238 2018-03-21T19:29:51Z 2018-03-21T19:29:51Z MEMBER

H5py is doing all the hard work for this in h5netcdf. On Wed, Mar 21, 2018 at 11:51 AM Benjamin Root notifications@github.com wrote:

Ah, nevermind, I see that our examples only had one greater-than-one stride

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/2004#issuecomment-375056363, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1g1ciNap4E9K2_dPKrol8ocz3DvLks5tgqEWgaJpZM4S0lM- .

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Slicing DataArray can take longer than not slicing 307318224
375020977 https://github.com/pydata/xarray/issues/2004#issuecomment-375020977 https://api.github.com/repos/pydata/xarray/issues/2004 MDEyOklzc3VlQ29tbWVudDM3NTAyMDk3Nw== shoyer 1217238 2018-03-21T17:08:15Z 2018-03-21T17:08:15Z MEMBER

The culprit appears to be netCDF4-python and/or netCDF-C: ``` f = netCDF4.Dataset('test.nc')

%time f['xarray_dataarray_variable'][:, ::10]

CPU times: user 313 ms, sys: 1.23 s, total: 1.54 s

```

When I try doing the same operation with h5netcdf, it runs very quickly: ```python reopened = xr.open_dataarray('test.nc', engine='h5netcdf')

%time reopened[::1, ::10].compute()

CPU times: user 6.11 ms, sys: 3.63 ms, total: 9.74 ms

```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Slicing DataArray can take longer than not slicing 307318224
375010010 https://github.com/pydata/xarray/issues/2004#issuecomment-375010010 https://api.github.com/repos/pydata/xarray/issues/2004 MDEyOklzc3VlQ29tbWVudDM3NTAxMDAxMA== shoyer 1217238 2018-03-21T16:38:59Z 2018-03-21T16:38:59Z MEMBER

Here's a simpler case that gets at the essence of the problem: ```python import xarray as xr import numpy as np

source = xr.DataArray(np.zeros((100, 12000)), dims=['time', 'x']) source.to_netcdf('test.nc', format='NETCDF4') reopened = xr.open_dataarray('test.nc')

%time reopened[::1, ::1].compute()

CPU times: user 1.35 ms, sys: 6.77 ms, total: 8.12 ms

%time reopened[::1, ::10].compute()

CPU times: user 371 ms, sys: 1.33 s, total: 1.7 s

```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Slicing DataArray can take longer than not slicing 307318224

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 183.974ms · About: xarray-datasette