issue_comments

Table actions

6 rows where issue = 305327479 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 3

author_association 2

issue 1

  • open_mfdataset() on a single file drops the concat_dim · 6 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
380137124 https://github.com/pydata/xarray/issues/1988#issuecomment-380137124 https://api.github.com/repos/pydata/xarray/issues/1988 MDEyOklzc3VlQ29tbWVudDM4MDEzNzEyNA== WeatherGod 291576 2018-04-10T15:12:05Z 2018-04-10T15:12:05Z CONTRIBUTOR

Yup... looks like that did the trick (for auto_combine and open_mfdataset). I even have a simple test to demonstrate it. PR coming shortly.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset() on a single file drops the concat_dim 305327479
379939574 https://github.com/pydata/xarray/issues/1988#issuecomment-379939574 https://api.github.com/repos/pydata/xarray/issues/1988 MDEyOklzc3VlQ29tbWVudDM3OTkzOTU3NA== WeatherGod 291576 2018-04-10T00:55:48Z 2018-04-10T00:55:48Z CONTRIBUTOR

I'll give it a go tomorrow. My work has gotten to this point now, and I have some unit tests that happen to exercise this edge case.

On a somewhat related note, would a allow_missing feature be welcomed in open_mfdataset()? I have written up some code that expects a concat_dim, and a list of filenames. It will then pass to open_mfdataset() only the files (and corresponding concat_dim values) that exists, and then calls reindex() with the original concat_dim to have a nan-filled slab where-ever there was a missing file.

Any interest?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset() on a single file drops the concat_dim 305327479
379904046 https://github.com/pydata/xarray/issues/1988#issuecomment-379904046 https://api.github.com/repos/pydata/xarray/issues/1988 MDEyOklzc3VlQ29tbWVudDM3OTkwNDA0Ng== shoyer 1217238 2018-04-09T21:45:53Z 2018-04-09T21:46:01Z MEMBER

@WeatherGod Possibly! As usual, tests are the hard part :)

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset() on a single file drops the concat_dim 305327479
379901414 https://github.com/pydata/xarray/issues/1988#issuecomment-379901414 https://api.github.com/repos/pydata/xarray/issues/1988 MDEyOklzc3VlQ29tbWVudDM3OTkwMTQxNA== WeatherGod 291576 2018-04-09T21:35:11Z 2018-04-09T21:35:11Z CONTRIBUTOR

Could the fix be as simple as if len(datasets) == 1 and dim is None:?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset() on a single file drops the concat_dim 305327479
375558323 https://github.com/pydata/xarray/issues/1988#issuecomment-375558323 https://api.github.com/repos/pydata/xarray/issues/1988 MDEyOklzc3VlQ29tbWVudDM3NTU1ODMyMw== shoyer 1217238 2018-03-23T06:32:17Z 2018-03-23T06:32:17Z MEMBER

Yes, this seems like a bug. open_mfdataset() should always concatenate if a dim argument is provided explicitly.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset() on a single file drops the concat_dim 305327479
375548064 https://github.com/pydata/xarray/issues/1988#issuecomment-375548064 https://api.github.com/repos/pydata/xarray/issues/1988 MDEyOklzc3VlQ29tbWVudDM3NTU0ODA2NA== jhamman 2443309 2018-03-23T05:18:59Z 2018-03-23T05:18:59Z MEMBER

This does seem inconsistent to me. Not that the behavior of concat already works with a single object:

```Python In [1]: import xarray as xr

In [2]: da = xr.DataArray([1, 2], dims='x', name='foo')

In [3]: xr.concat([da], dim='y') Out[3]: <xarray.DataArray 'foo' (y: 1, x: 2)> array([[1, 2]]) Dimensions without coordinates: y, x ```

The offending line is here:

https://github.com/pydata/xarray/blob/9261601f89c0d3cfc54db16718c82399d95266bd/xarray/core/combine.py#L342-L344

Based on this, I'm surprised my little example with concat works the way it does. In either event, it would be great if someone could spend some time normalizing the behavior here. 

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset() on a single file drops the concat_dim 305327479

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);