issue_comments

Table actions

8 rows where issue = 131972229 sorted by updated_at descending

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

author_association 2

issue 1

  • IO fails with complex datatypes · 8 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
457567124 https://github.com/pydata/xarray/issues/748#issuecomment-457567124 https://api.github.com/repos/pydata/xarray/issues/748 MDEyOklzc3VlQ29tbWVudDQ1NzU2NzEyNA== stale[bot] 26384082 2019-01-25T13:07:03Z 2019-01-25T13:07:03Z NONE

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity If this issue remains relevant, please comment here; otherwise it will be marked as closed automatically

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   IO fails with complex datatypes 131972229
257117471 https://github.com/pydata/xarray/issues/748#issuecomment-257117471 https://api.github.com/repos/pydata/xarray/issues/748 MDEyOklzc3VlQ29tbWVudDI1NzExNzQ3MQ== alexamici 226037 2016-10-29T21:37:19Z 2016-10-29T21:37:19Z MEMBER

I reinstalled a bunch of stuff and I can confirm my example above works for me now using engine='h5netcdf'. It still fails with the default engine.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   IO fails with complex datatypes 131972229
249390682 https://github.com/pydata/xarray/issues/748#issuecomment-249390682 https://api.github.com/repos/pydata/xarray/issues/748 MDEyOklzc3VlQ29tbWVudDI0OTM5MDY4Mg== shoyer 1217238 2016-09-24T22:16:17Z 2016-09-24T22:16:29Z MEMBER

@alexamici Something else must have gone wrong here -- your example works for me:

``` In [9]: dataset = xr.Dataset({'data': np.zeros((2,), dtype='complex128')})

In [10]: dataset Out[10]: <xarray.Dataset> Dimensions: (data: 2) Coordinates: * data (data) complex128 0j 0j Data variables: empty

In [11]: dataset.to_netcdf('tmp.nc', engine='h5netcdf') ```

Maybe you need to update to a newer version of h5py? (Just a guess)

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   IO fails with complex datatypes 131972229
249353905 https://github.com/pydata/xarray/issues/748#issuecomment-249353905 https://api.github.com/repos/pydata/xarray/issues/748 MDEyOklzc3VlQ29tbWVudDI0OTM1MzkwNQ== alexamici 226037 2016-09-24T08:54:37Z 2016-09-24T08:54:49Z MEMBER

I add to this issue as it refers to general IO failures. Saving a Dataset that contains DataArray with complex data-type fails on all netCDF4 engines:

dataset = xr.Dataset({'data': np.zeros((2,), dtype='complex128')}) dataset.to_netcdf('tmp.nc', engine='netcdf4') [...] TypeError: illegal primitive data type, must be one of dict_keys(['i8', 'f4', 'f8', 'i2', 'u2', 'i1', 'i4', 'S1', 'u8', 'u1', 'u4']), got complex128

and

dataset.to_netcdf('tmp.nc', engine='h5netcdf') [...] TypeError: Object dtype dtype('O') has no native HDF5 equivalent

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   IO fails with complex datatypes 131972229
182947295 https://github.com/pydata/xarray/issues/748#issuecomment-182947295 https://api.github.com/repos/pydata/xarray/issues/748 MDEyOklzc3VlQ29tbWVudDE4Mjk0NzI5NQ== arsenovic 1228240 2016-02-11T16:38:04Z 2016-02-11T16:38:04Z NONE

excellent. thanks for looking into it, this work around is sufficient for my needs fyi. 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   IO fails with complex datatypes 131972229
182943474 https://github.com/pydata/xarray/issues/748#issuecomment-182943474 https://api.github.com/repos/pydata/xarray/issues/748 MDEyOklzc3VlQ29tbWVudDE4Mjk0MzQ3NA== shoyer 1217238 2016-02-11T16:28:04Z 2016-02-11T16:28:04Z MEMBER

OK, I can reproduce this. The issue seems to be that xarray uses the netCDF4 package as the default backend for reading netCDF4/HDF5 files, and netCDF4 seems to silently drop these variables with a compound dtype, even though allegedly it supports them.

The work around is to use engine='h5netcdf' explicitly when opening these netCDF files:

``` In [35]: ntwk_opened = xr.open_dataset('ring_slot.nc', engine='h5netcdf')

In [36]: ntwk_opened Out[36]: <xarray.Dataset> Dimensions: (f: 201, m: 2, n: 2) Coordinates: * f (f) float64 75.0 75.17 75.35 75.53 75.7 75.88 76.05 76.22 76.4 ... * m (m) int64 0 1 * n (n) int64 0 1 Data variables: s (f, m, n) complex128 (-0.503723180993+0.457844804761j) ... z0 (f, m) complex128 (50+0j) (50+0j) (50+0j) (50+0j) (50+0j) ... Attributes: name: ring slot ```

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   IO fails with complex datatypes 131972229
181490631 https://github.com/pydata/xarray/issues/748#issuecomment-181490631 https://api.github.com/repos/pydata/xarray/issues/748 MDEyOklzc3VlQ29tbWVudDE4MTQ5MDYzMQ== arsenovic 1228240 2016-02-08T17:36:51Z 2016-02-08T17:36:51Z NONE

2.5.0-np19py27_3 from conda

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   IO fails with complex datatypes 131972229
181438176 https://github.com/pydata/xarray/issues/748#issuecomment-181438176 https://api.github.com/repos/pydata/xarray/issues/748 MDEyOklzc3VlQ29tbWVudDE4MTQzODE3Ng== shoyer 1217238 2016-02-08T15:57:02Z 2016-02-08T15:57:02Z MEMBER

This looks bad! I was pretty sure we had tested this with h5netcdf. I'll take a look later... what version of h5py are you using?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   IO fails with complex datatypes 131972229

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);