issue_comments
13 rows where author_association = "MEMBER" and issue = 372848074 sorted by updated_at descending
This data as json, CSV (advanced)
Suggested facets: created_at (date), updated_at (date)
issue 1
- open_mfdataset usage and limitations. · 13 ✖
id | html_url | issue_url | node_id | user | created_at | updated_at ▲ | author_association | body | reactions | performed_via_github_app | issue |
---|---|---|---|---|---|---|---|---|---|---|---|
768470505 | https://github.com/pydata/xarray/issues/2501#issuecomment-768470505 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDc2ODQ3MDUwNQ== | dcherian 2448579 | 2021-01-27T18:06:16Z | 2021-01-27T18:06:16Z | MEMBER | I think this is stale now. See https://xarray.pydata.org/en/stable/io.html#reading-multi-file-datasets for latest guidance on reading such datasets. Please open a new issue if you are still having trouble with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
510217080 | https://github.com/pydata/xarray/issues/2501#issuecomment-510217080 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDUxMDIxNzA4MA== | TomAugspurger 1312546 | 2019-07-10T20:30:41Z | 2019-07-10T20:30:41Z | MEMBER | Yep, that’s my suspicion as well. I’m still plugging away at it. Currently the pausing logic isn’t quite working well.
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
510169853 | https://github.com/pydata/xarray/issues/2501#issuecomment-510169853 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDUxMDE2OTg1Mw== | rabernat 1197350 | 2019-07-10T18:10:37Z | 2019-07-10T18:10:37Z | MEMBER | I believe that the memory issue is basically the same as https://github.com/dask/distributed/issues/2602. The graphs look like: Reading and rechunking increase memory consumption. Writing relieves it. In Rich's case, the workers just load too much data before they write it. Eventually they run out of memory. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
510167911 | https://github.com/pydata/xarray/issues/2501#issuecomment-510167911 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDUxMDE2NzkxMQ== | TomAugspurger 1312546 | 2019-07-10T18:05:07Z | 2019-07-10T18:05:07Z | MEMBER | Great, thanks. I’ll look into the memory issue when writing. We may already have an issue for it.
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
509346055 | https://github.com/pydata/xarray/issues/2501#issuecomment-509346055 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDUwOTM0NjA1NQ== | TomAugspurger 1312546 | 2019-07-08T18:46:58Z | 2019-07-08T18:46:58Z | MEMBER | @rsignell-usgs very helpful, thanks. I'd noticed that there was a pause after the open_dataset tasks finish, indicating that either the scheduler or (more likely) the client was doing work rather than the cluster. Most likely @rabernat's guess
is correct. Verifying all that now, and looking into if / how that can be done on the workers. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
509307081 | https://github.com/pydata/xarray/issues/2501#issuecomment-509307081 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDUwOTMwNzA4MQ== | TomAugspurger 1312546 | 2019-07-08T16:57:15Z | 2019-07-08T16:57:15Z | MEMBER | I'm looking into it today. Can you clarify
by "process" do you mean a dask worker process, or just the main python process executing the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
506497180 | https://github.com/pydata/xarray/issues/2501#issuecomment-506497180 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDUwNjQ5NzE4MA== | TomAugspurger 1312546 | 2019-06-27T20:24:26Z | 2019-06-27T20:24:26Z | MEMBER |
Good to know! FYI, https://github.com/pydata/xarray/issues/2501#issuecomment-506478508 was user error (I can access it, but need to specify the us-east-1 region). Taking a look now. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
506482057 | https://github.com/pydata/xarray/issues/2501#issuecomment-506482057 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDUwNjQ4MjA1Nw== | rabernat 1197350 | 2019-06-27T19:36:51Z | 2019-06-27T19:36:51Z | MEMBER | @rsignell-usgs Can you post the xarray repr of two sample files post pre-processing function? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
506481845 | https://github.com/pydata/xarray/issues/2501#issuecomment-506481845 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDUwNjQ4MTg0NQ== | rabernat 1197350 | 2019-06-27T19:36:11Z | 2019-06-27T19:36:11Z | MEMBER |
The datasets in our cloud datastore are designed explicitly to avoid this problem! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
506478508 | https://github.com/pydata/xarray/issues/2501#issuecomment-506478508 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDUwNjQ3ODUwOA== | TomAugspurger 1312546 | 2019-06-27T19:25:05Z | 2019-06-27T19:25:05Z | MEMBER | Thanks, will take a look this afternoon. Are there any datasets on https://pangeo-data.github.io/pangeo-datastore/ that would exhibit this poor behavior? I may not have access to the bucket (or I'm misusing
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
503641038 | https://github.com/pydata/xarray/issues/2501#issuecomment-503641038 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDUwMzY0MTAzOA== | rabernat 1197350 | 2019-06-19T16:48:29Z | 2019-06-19T16:48:29Z | MEMBER | Try writing a preprocessor function that drops all coordinates
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
432342306 | https://github.com/pydata/xarray/issues/2501#issuecomment-432342306 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDQzMjM0MjMwNg== | rabernat 1197350 | 2018-10-23T17:27:50Z | 2018-10-23T17:27:50Z | MEMBER | ^ I'm assuming you're in a notebook. If not, call |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 | |
432342180 | https://github.com/pydata/xarray/issues/2501#issuecomment-432342180 | https://api.github.com/repos/pydata/xarray/issues/2501 | MDEyOklzc3VlQ29tbWVudDQzMjM0MjE4MA== | rabernat 1197350 | 2018-10-23T17:27:30Z | 2018-10-23T17:27:30Z | MEMBER | In To help us help you debug, please provide more information about the files your are opening. Specifically, please call |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
open_mfdataset usage and limitations. 372848074 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [performed_via_github_app] TEXT, [issue] INTEGER REFERENCES [issues]([id]) ); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
user 3