issue_comments

Table actions

3 rows where author_association = "MEMBER", issue = 207021356 and user = 1217238 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

These facets timed out: author_association, issue

user 1

  • shoyer · 3 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
279846111 https://github.com/pydata/xarray/issues/1262#issuecomment-279846111 https://api.github.com/repos/pydata/xarray/issues/1262 MDEyOklzc3VlQ29tbWVudDI3OTg0NjExMQ== shoyer 1217238 2017-02-14T21:46:09Z 2017-02-14T21:46:09Z MEMBER

CC @pydata/xarray in case anyone else has opinions here

Benefits in favor would be that I suspect XArray already has mechanisms for coercion and such and it would reduce the number of total libraries.

We really don't have much existing machinery. Two things we have that might be useful:

  • a couple of mixin classes for easily defining custom array types. This could be a nice building block, but it's self-contained and only a few dozen lines of code.
  • some existing code for function dispatch to either numpy or dask.array. This is quite messy, somewhat xarray-specific and not worth copying.

Fewer libraries is definitely nice, but I see this as more of a secondary rather than primary goal.

More broadly, doing this project right will need strong separation of concerns from xarray's handling of labeled arrays. So there's not a huge amount to be gained by doing it in the same repository.

Argument against is that XArray is currently only focused on indexed and labeled arrays, and possible it doesn't want to deal with the dtype mess.

I would love to see this project be successful and integrated with xarray. But better dtypes is tangental to our current focus, and project maintenance is already stretched pretty thin -- there's still a lot of core functionality to build out for manipulation of labeled arrays.

So I'm not comfortable with building this in xarray at this time. But I would be happy to revisit this decision when you have a design document, prototype and someone committed to developing and maintaining the module.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Logical DTypes 207021356
279509836 https://github.com/pydata/xarray/issues/1262#issuecomment-279509836 https://api.github.com/repos/pydata/xarray/issues/1262 MDEyOklzc3VlQ29tbWVudDI3OTUwOTgzNg== shoyer 1217238 2017-02-13T20:18:42Z 2017-02-13T20:18:42Z MEMBER

One major API design challenge to solve with such a package (unresolved in NumPy) is how to handle dtype-specific methods/properties, e.g., year, month and day properties for a custom datetime dtype, or a .keys() methods for a structured dtype (https://github.com/numpy/numpy/pull/8615).

Fitting these into a generic NDArray type is not very natural. So perhaps the solution is to use subclasses (fixed for each dtype) with some very strict design constraints (e.g., only add new methods/properties, don't override functionality). The contract would still be that the dtype defines all valid extension points for overriding functionality.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Logical DTypes 207021356
279505323 https://github.com/pydata/xarray/issues/1262#issuecomment-279505323 https://api.github.com/repos/pydata/xarray/issues/1262 MDEyOklzc3VlQ29tbWVudDI3OTUwNTMyMw== shoyer 1217238 2017-02-13T20:00:53Z 2017-02-13T20:09:43Z MEMBER

So question: Is it sensible to add logical dtype information to XArray?

Sure, this would pretty sensible, especially if there is a nice story for wrapping upstream libraries providing alternate physical arrays such as dask.array and bolt (cc @freeman-lab).

There are certainly plenty of use-cases. A few more examples that would be particularly relevant for xarray:

Can this be done with only moderate effort and maintenance costs to the XArray project?

If we have a well defined interface that defines the right operations, my guess is indeed "yes, probably". See https://github.com/bolt-project/bolt/issues/58 for a list of operations worth considering wrapping (obviously some of these, like arithmetic, are not needed for all dtypes).

If the answer is "yes, probably", then what is the right way to go about this?

I think it should start as a separate package to ensure a cleanly separated interface and because there are definitely other clients than xarray. We can quickly add it as an optional dependency to xarray for testing purposes.

I'm excited about this, but I'm unlikely to have much time available to work on this directly.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Logical DTypes 207021356

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);