issue_comments

Table actions

7 rows where author_association = "MEMBER" and issue = 647804004 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 2

issue 1

  • Xarray open_mfdataset with engine Zarr · 7 ✖

author_association 1

  • MEMBER · 7 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
696783734 https://github.com/pydata/xarray/pull/4187#issuecomment-696783734 https://api.github.com/repos/pydata/xarray/issues/4187 MDEyOklzc3VlQ29tbWVudDY5Njc4MzczNA== dcherian 2448579 2020-09-22T15:08:36Z 2020-09-22T15:08:36Z MEMBER

Thanks @weiji14 and @Mikejmnez . This is a great contribution.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Xarray open_mfdataset with engine Zarr 647804004
696518519 https://github.com/pydata/xarray/pull/4187#issuecomment-696518519 https://api.github.com/repos/pydata/xarray/issues/4187 MDEyOklzc3VlQ29tbWVudDY5NjUxODUxOQ== shoyer 1217238 2020-09-22T05:40:57Z 2020-09-22T05:40:57Z MEMBER

Thanks @weiji14 and @Mikejmnez for your contribution!

 
{
    "total_count": 3,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 3,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Xarray open_mfdataset with engine Zarr 647804004
690885702 https://github.com/pydata/xarray/pull/4187#issuecomment-690885702 https://api.github.com/repos/pydata/xarray/issues/4187 MDEyOklzc3VlQ29tbWVudDY5MDg4NTcwMg== shoyer 1217238 2020-09-11T05:35:29Z 2020-09-11T05:35:29Z MEMBER

Just to note down a few things:

  1. The deprecated "auto_chunk" kwarg was removed
  2. open_zarr uses chunks="auto" by default, whereas open_dataset uses chunks=None (see my comment inline)

The different default chunk behaviour (point 2) is worth raising, and it might be best to postpone the deprecation of open_zarr until the next release, so that there's time to discuss what we want the default setting to be (None or auto).

These all sound good to me!

I agree that we shouldn't change the default behavior for open_dataset, and should keep open_zarr around for now -- there is no urgent need to deprecate it.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Xarray open_mfdataset with engine Zarr 647804004
652482831 https://github.com/pydata/xarray/pull/4187#issuecomment-652482831 https://api.github.com/repos/pydata/xarray/issues/4187 MDEyOklzc3VlQ29tbWVudDY1MjQ4MjgzMQ== dcherian 2448579 2020-07-01T15:20:39Z 2020-07-01T15:20:39Z MEMBER

I wonder if it's really worth deprecating open_zarr(). open_dataset(..., engine='zarr') is a bit more verbose, especially with backend_kwargs to pass optional arguments. It seems pretty harmless to keep open_zarr() around, especially if it's just an alias for open_datraset(engine='zarr').

I agree. I think we should keep open_zarr around.

As reminder (because it took me a while to remember!), one goal with this refactor is to have open_mfdataset work with all backends (including zarr and rasterio) by specifying the engine kwarg.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Xarray open_mfdataset with engine Zarr 647804004
652206442 https://github.com/pydata/xarray/pull/4187#issuecomment-652206442 https://api.github.com/repos/pydata/xarray/issues/4187 MDEyOklzc3VlQ29tbWVudDY1MjIwNjQ0Mg== shoyer 1217238 2020-07-01T05:49:51Z 2020-07-01T05:50:31Z MEMBER

As for the annoyance of needing to write backend_kwargs={"consolidated": True}, I wonder if we could detect this automatically by checking for the existence of a .zmetadata key? This would add a small amount of overhead (one file access) but this probably would not be prohibitively expensive.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Xarray open_mfdataset with engine Zarr 647804004
652204634 https://github.com/pydata/xarray/pull/4187#issuecomment-652204634 https://api.github.com/repos/pydata/xarray/issues/4187 MDEyOklzc3VlQ29tbWVudDY1MjIwNDYzNA== shoyer 1217238 2020-07-01T05:44:20Z 2020-07-01T05:44:32Z MEMBER

We could also automatically detect zarr stores in open_dataset without requiring engine='zarr' if:

  1. the argument inherits from collections.abc.Mapping, and
  2. it contains a key '.zgroup', corresponding to zarr metadata.
 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Xarray open_mfdataset with engine Zarr 647804004
652202093 https://github.com/pydata/xarray/pull/4187#issuecomment-652202093 https://api.github.com/repos/pydata/xarray/issues/4187 MDEyOklzc3VlQ29tbWVudDY1MjIwMjA5Mw== shoyer 1217238 2020-07-01T05:36:00Z 2020-07-01T05:36:00Z MEMBER

I wonder if it's really worth deprecating open_zarr(). open_dataset(..., engine='zarr') is a bit more verbose, especially with backend_kwargs to pass optional arguments. It seems pretty harmless to keep open_zarr() around, especially if it's just an alias for open_datraset(engine='zarr').

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Xarray open_mfdataset with engine Zarr 647804004

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);