home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

9 rows where author_association = "MEMBER" and issue = 173773358 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 3

  • shoyer 4
  • jhamman 4
  • rabernat 1

issue 1

  • Creating unlimited dimensions with xarray.Dataset.to_netcdf · 9 ✖

author_association 1

  • MEMBER · 9 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
269510467 https://github.com/pydata/xarray/issues/992#issuecomment-269510467 https://api.github.com/repos/pydata/xarray/issues/992 MDEyOklzc3VlQ29tbWVudDI2OTUxMDQ2Nw== jhamman 2443309 2016-12-28T17:29:57Z 2016-12-28T17:29:57Z MEMBER

For those of you who are interested in this feature, I'd appreciate your feedback on #1170.

{
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 1,
    "rocket": 0,
    "eyes": 0
}
  Creating unlimited dimensions with xarray.Dataset.to_netcdf 173773358
267185985 https://github.com/pydata/xarray/issues/992#issuecomment-267185985 https://api.github.com/repos/pydata/xarray/issues/992 MDEyOklzc3VlQ29tbWVudDI2NzE4NTk4NQ== jhamman 2443309 2016-12-14T23:09:50Z 2016-12-14T23:09:50Z MEMBER

All - I'm actively working on this. I have an initial implementation and will get it cleaned up here in the next few days. Stay tuned.

{
    "total_count": 4,
    "+1": 4,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Creating unlimited dimensions with xarray.Dataset.to_netcdf 173773358
258877559 https://github.com/pydata/xarray/issues/992#issuecomment-258877559 https://api.github.com/repos/pydata/xarray/issues/992 MDEyOklzc3VlQ29tbWVudDI1ODg3NzU1OQ== shoyer 1217238 2016-11-07T16:05:04Z 2016-11-07T16:05:04Z MEMBER

Agreed, it's awkward to have this information on variables.

I was somewhat opposed to adding more state to the Dataset object but it seems like the necessary solution here. I'm not sure we need it in the Dataset constructor though -- could just have encoding as an attribute you need to modify. Honestly, could probably do the same for DataArray.encoding -- it's pretty low level.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Creating unlimited dimensions with xarray.Dataset.to_netcdf 173773358
258834200 https://github.com/pydata/xarray/issues/992#issuecomment-258834200 https://api.github.com/repos/pydata/xarray/issues/992 MDEyOklzc3VlQ29tbWVudDI1ODgzNDIwMA== jhamman 2443309 2016-11-07T13:24:20Z 2016-11-07T13:24:20Z MEMBER

@shoyer -

I played around with this a bit yesterday. I have two implementation questions: - for a dataset with multiple variables, putting the dimension encoding on the Variable easily leads to conflicts. For example, one variable's encoding may say that dimension x is unlimited while another doesn't. How should we handle these conflicts. - are we apposed to a encoding attribute on the dataset object?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Creating unlimited dimensions with xarray.Dataset.to_netcdf 173773358
243546130 https://github.com/pydata/xarray/issues/992#issuecomment-243546130 https://api.github.com/repos/pydata/xarray/issues/992 MDEyOklzc3VlQ29tbWVudDI0MzU0NjEzMA== shoyer 1217238 2016-08-30T19:07:19Z 2016-08-30T19:07:19Z MEMBER

However, when the dataset is indexed/subset/resampled along the unlimited dimension, it would make sense that its state is dropped. But that would require a lot of ifs and buts, so I suggest we leave that aside for now.

This is exactly how Variable.encoding currently works: any operation that creates a new variable from the original variable drops the encoding.

If we put this encoding information on the variable corresponding to the dimension, any time you save a Dataset using that exact same dimension variable, it would be saved as unlimited size. So if you only modify other dimensions (e.g., with resampling or indexing), the unlimited dimension would indeed persist, as you desire.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Creating unlimited dimensions with xarray.Dataset.to_netcdf 173773358
243287795 https://github.com/pydata/xarray/issues/992#issuecomment-243287795 https://api.github.com/repos/pydata/xarray/issues/992 MDEyOklzc3VlQ29tbWVudDI0MzI4Nzc5NQ== shoyer 1217238 2016-08-29T23:24:21Z 2016-08-29T23:24:21Z MEMBER

Yes, we could put this in encoding if we want to preserve through reading/writing files. NetCDF4 supports multiple unlimited dimensions. Netcdf3 does not. On Mon, Aug 29, 2016 at 1:56 PM Jonas notifications@github.com wrote:

OK, I'd be up for taking a shot at it.

Since it is per-variable and specific to netCDF, I guess the perfect place to add this is in the encoding dictionary that you can pass to to_netcdf https://github.com/pydata/xarray/blob/606e1d9c7efd72e10b530a688d6ef870e8ec1843/xarray/backends/api.py#L316, right? Maybe as key unlimited? E.g.

ds.to_netcdf(encoding={'time': dict(unlimited=True)})

I need to look up whether netCDF allows for defining more than one unlimited dimension, otherwise that must throw an error.

And then it is just about passing None as length to CreateDimension, at least in netCDF4 and scipy.io.netcdf. But I did not look into how xarray handles that under the hood.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/992#issuecomment-243253510, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1mTPcRqf3JI-yRk4wnIQ_xgKE3Wvks5qk0eXgaJpZM4JveCA .

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Creating unlimited dimensions with xarray.Dataset.to_netcdf 173773358
243177100 https://github.com/pydata/xarray/issues/992#issuecomment-243177100 https://api.github.com/repos/pydata/xarray/issues/992 MDEyOklzc3VlQ29tbWVudDI0MzE3NzEwMA== jhamman 2443309 2016-08-29T16:31:45Z 2016-08-29T16:31:45Z MEMBER

xref: https://github.com/pydata/xarray/issues/678

I'm also a +1 on this.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Creating unlimited dimensions with xarray.Dataset.to_netcdf 173773358
243168308 https://github.com/pydata/xarray/issues/992#issuecomment-243168308 https://api.github.com/repos/pydata/xarray/issues/992 MDEyOklzc3VlQ29tbWVudDI0MzE2ODMwOA== rabernat 1197350 2016-08-29T16:00:08Z 2016-08-29T16:00:08Z MEMBER

👍 from me. In the broader netCDF ecosystem, it can be pretty important to have record dimensions for various reasons, appending being the main one.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Creating unlimited dimensions with xarray.Dataset.to_netcdf 173773358
243167445 https://github.com/pydata/xarray/issues/992#issuecomment-243167445 https://api.github.com/repos/pydata/xarray/issues/992 MDEyOklzc3VlQ29tbWVudDI0MzE2NzQ0NQ== shoyer 1217238 2016-08-29T15:57:27Z 2016-08-29T15:57:27Z MEMBER

Currently it's not supported, but yes, we could absolutely add it as an option. I would be happy to add this functionality if someone makes a pull request. This won't be very useful for editing the file with xarray of course because we don't support editing netcdf files without making a complete copy.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Creating unlimited dimensions with xarray.Dataset.to_netcdf 173773358

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 16.357ms · About: xarray-datasette