issue_comments

Table actions

3 rows where issue = 267628781 and user = 703554 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • alimanfoo · 3 ✖

issue 1

  • Low memory/out-of-core index? · 3 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
338786761 https://github.com/pydata/xarray/issues/1650#issuecomment-338786761 https://api.github.com/repos/pydata/xarray/issues/1650 MDEyOklzc3VlQ29tbWVudDMzODc4Njc2MQ== alimanfoo 703554 2017-10-23T20:29:41Z 2017-10-23T20:29:41Z CONTRIBUTOR

Index API sounds good.

Also I was just looking at dask.dataframe indexing, there .loc is implemented using information about index values at the boundaries of each partition (chunk). Not sure xarray should use same strategy for chunked datasets, but is another approach to avoid loading indexes into memory. 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Low memory/out-of-core index? 267628781
338687376 https://github.com/pydata/xarray/issues/1650#issuecomment-338687376 https://api.github.com/repos/pydata/xarray/issues/1650 MDEyOklzc3VlQ29tbWVudDMzODY4NzM3Ng== alimanfoo 703554 2017-10-23T14:58:59Z 2017-10-23T14:58:59Z CONTRIBUTOR

It looks like #1017 is about having no index at all. I want indexes, but I want to avoid loading all coordinate values into memory.

On Mon, Oct 23, 2017 at 1:47 PM, Fabien Maussion notifications@github.com wrote:

Has anyone considered implementing an index for monotonic data that does not require loading all values into main memory?

But this is already the case? #1017 https://github.com/pydata/xarray/pull/1017

With on file datasets I think it is sufficient to drop_variables when opening the dataset in order not to parse the coordinates:

ds = xr.open_dataset(f, drop_variables=['lon', 'lat'])

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/1650#issuecomment-338647540, or mute the thread https://github.com/notifications/unsubscribe-auth/AAq8QsbZ81N2pKybO1sFHVHK0KTk1aELks5svIrJgaJpZM4QCq62 .

-- Alistair Miles Head of Epidemiological Informatics Centre for Genomics and Global Health http://cggh.org Big Data Institute Building Old Road Campus Roosevelt Drive Oxford OX3 7LF United Kingdom Phone: +44 (0)1865 743596 Email: alimanfoo@googlemail.com Web: http://a http://purl.org/net/alimanlimanfoo.github.io/ Twitter: https://twitter.com/alimanfoo

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Low memory/out-of-core index? 267628781
338627454 https://github.com/pydata/xarray/issues/1650#issuecomment-338627454 https://api.github.com/repos/pydata/xarray/issues/1650 MDEyOklzc3VlQ29tbWVudDMzODYyNzQ1NA== alimanfoo 703554 2017-10-23T11:19:30Z 2017-10-23T11:19:30Z CONTRIBUTOR

Just to add a further thought, which is that the upper levels of the binary search tree could be be cached to get faster performance for repeated searches.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Low memory/out-of-core index? 267628781

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);