home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

9 rows where issue = 480512400 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 4

  • aidanheerdegen 4
  • shoyer 3
  • DocOtak 1
  • brianpm 1

author_association 3

  • CONTRIBUTOR 5
  • MEMBER 3
  • NONE 1

issue 1

  • decode_cf called on mfdataset throws error: 'Array' object has no attribute 'tolist' · 9 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
521524233 https://github.com/pydata/xarray/issues/3215#issuecomment-521524233 https://api.github.com/repos/pydata/xarray/issues/3215 MDEyOklzc3VlQ29tbWVudDUyMTUyNDIzMw== aidanheerdegen 6063709 2019-08-15T05:56:18Z 2019-08-15T05:57:40Z CONTRIBUTOR

Confirmed that NUMPY_EXPERIMENTAL_ARRAY_FUNCTION=0 fixes this issue for me.

Have submitted a PR with your suggested fix https://github.com/pydata/xarray/pull/3220

Confirmed the submitted code fixes my issue.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  decode_cf called on mfdataset throws error: 'Array' object has no attribute 'tolist' 480512400
521504317 https://github.com/pydata/xarray/issues/3215#issuecomment-521504317 https://api.github.com/repos/pydata/xarray/issues/3215 MDEyOklzc3VlQ29tbWVudDUyMTUwNDMxNw== aidanheerdegen 6063709 2019-08-15T03:55:00Z 2019-08-15T03:55:00Z CONTRIBUTOR

Thanks for the explanation.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  decode_cf called on mfdataset throws error: 'Array' object has no attribute 'tolist' 480512400
521503585 https://github.com/pydata/xarray/issues/3215#issuecomment-521503585 https://api.github.com/repos/pydata/xarray/issues/3215 MDEyOklzc3VlQ29tbWVudDUyMTUwMzU4NQ== shoyer 1217238 2019-08-15T03:50:00Z 2019-08-15T03:50:00Z MEMBER

I still don't understand the different code paths between decode_cf=True and decode_cf=False + explicit call to xarray.decode_cf()

In the first case, CF conventions decoding is done on xarray's internal lazy BackendArray objects.

In the second case, CF conventions decoding is done on dask arrays. There are a few cases where this can be slower / require loading more data to look at a small slice of array data.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  decode_cf called on mfdataset throws error: 'Array' object has no attribute 'tolist' 480512400
521502195 https://github.com/pydata/xarray/issues/3215#issuecomment-521502195 https://api.github.com/repos/pydata/xarray/issues/3215 MDEyOklzc3VlQ29tbWVudDUyMTUwMjE5NQ== aidanheerdegen 6063709 2019-08-15T03:40:27Z 2019-08-15T03:40:27Z CONTRIBUTOR

Thanks @shoyer.

I still don't understand the different code paths between decode_cf=True and decode_cf=False + explicit call to xarray.decode_cf()

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  decode_cf called on mfdataset throws error: 'Array' object has no attribute 'tolist' 480512400
521500878 https://github.com/pydata/xarray/issues/3215#issuecomment-521500878 https://api.github.com/repos/pydata/xarray/issues/3215 MDEyOklzc3VlQ29tbWVudDUyMTUwMDg3OA== shoyer 1217238 2019-08-15T03:32:02Z 2019-08-15T03:32:02Z MEMBER

A short term work around is to set the environment variable NUMPY_EXPERIMENTAL_ARRAY_FUNCTION=0 before importing NumPy.

{
    "total_count": 2,
    "+1": 2,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  decode_cf called on mfdataset throws error: 'Array' object has no attribute 'tolist' 480512400
521492717 https://github.com/pydata/xarray/issues/3215#issuecomment-521492717 https://api.github.com/repos/pydata/xarray/issues/3215 MDEyOklzc3VlQ29tbWVudDUyMTQ5MjcxNw== shoyer 1217238 2019-08-15T02:40:10Z 2019-08-15T02:40:10Z MEMBER

This is definitely due to NumPy 1.17 / __array_function__, which means that np.ravel now calls out to a dask function rather than coercing to NumPy.

I think the right fix is to add explicit casting to NumPy, e.g., np.ravel(np.asarray(array[indexer]))

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  decode_cf called on mfdataset throws error: 'Array' object has no attribute 'tolist' 480512400
521456728 https://github.com/pydata/xarray/issues/3215#issuecomment-521456728 https://api.github.com/repos/pydata/xarray/issues/3215 MDEyOklzc3VlQ29tbWVudDUyMTQ1NjcyOA== aidanheerdegen 6063709 2019-08-14T23:28:35Z 2019-08-14T23:28:35Z CONTRIBUTOR

Seems that this is by design, from here

Dask Array doesn’t implement operations like tolist that would be very inefficient for larger datasets. Likewise, it is very inefficient to iterate over a Dask array with for loops

So dask has never had a tolist method, so in one case the object is a dask array, but not in the other case.

I still don't understand why it fails when decode_cf is called separately. Suggests there is a different code path as all underlying packages are identical.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  decode_cf called on mfdataset throws error: 'Array' object has no attribute 'tolist' 480512400
521349841 https://github.com/pydata/xarray/issues/3215#issuecomment-521349841 https://api.github.com/repos/pydata/xarray/issues/3215 MDEyOklzc3VlQ29tbWVudDUyMTM0OTg0MQ== DocOtak 868027 2019-08-14T17:51:06Z 2019-08-14T17:51:06Z CONTRIBUTOR

I think this is being thrown by dask, here is an even more minimal example: ```python

import dask as da da.array.from_array([]).tolist() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'Array' object has no attribute 'tolist' ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  decode_cf called on mfdataset throws error: 'Array' object has no attribute 'tolist' 480512400
521344406 https://github.com/pydata/xarray/issues/3215#issuecomment-521344406 https://api.github.com/repos/pydata/xarray/issues/3215 MDEyOklzc3VlQ29tbWVudDUyMTM0NDQwNg== brianpm 5471157 2019-08-14T17:36:12Z 2019-08-14T17:36:12Z NONE

I'm having the same issue with xarray 0.12.3, numpy 1.17.0, python 3.7.3.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  decode_cf called on mfdataset throws error: 'Array' object has no attribute 'tolist' 480512400

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 794.622ms · About: xarray-datasette