issue_comments

Table actions

3 rows where issue = 291332965 and user = 2443309 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: updated_at (date)

user 1

  • jhamman · 3 ✖

issue 1

  • Drop coordinates on loading large dataset. · 3 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
364498649 https://github.com/pydata/xarray/issues/1854#issuecomment-364498649 https://api.github.com/repos/pydata/xarray/issues/1854 MDEyOklzc3VlQ29tbWVudDM2NDQ5ODY0OQ== jhamman 2443309 2018-02-09T17:18:53Z 2018-02-09T17:18:53Z MEMBER

@rabernat - good points.

@jamesstidard - perhaps you can a single files ncdump using the ncdump -h -s filename.nc syntax. That should tell us how the file is chunked on disk.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Drop coordinates on loading large dataset. 291332965
364491330 https://github.com/pydata/xarray/issues/1854#issuecomment-364491330 https://api.github.com/repos/pydata/xarray/issues/1854 MDEyOklzc3VlQ29tbWVudDM2NDQ5MTMzMA== jhamman 2443309 2018-02-09T16:53:57Z 2018-02-09T16:53:57Z MEMBER

@jamesstidard - let's see how the distributed scheduler plays:

```Python from distributed import Client client = Client()

ds = xr.open_mfdataset('path/to/ncs/*.nc', chunks={'latitude': 50, 'longitude': 50}) recs = ds.sel(latitude=10, longitude=10).to_dataframe().to_records() ``` Also, it would be worth updating distributed before you use its scheduler.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Drop coordinates on loading large dataset. 291332965
364478761 https://github.com/pydata/xarray/issues/1854#issuecomment-364478761 https://api.github.com/repos/pydata/xarray/issues/1854 MDEyOklzc3VlQ29tbWVudDM2NDQ3ODc2MQ== jhamman 2443309 2018-02-09T16:12:22Z 2018-02-09T16:12:22Z MEMBER

@jamesstidard - it would be good to know a few more details here:

  • what dask scheduler you're using (you might also try with the distributed scheduler)
  • what versions of dask/distributed/etc you're using
  • how using a smaller chunk size in space (latitude and longitude) impacts performance
 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Drop coordinates on loading large dataset. 291332965

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);