home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

13 rows where issue = 630062936 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 6

  • kmpaul 4
  • dcherian 2
  • fujiisoup 2
  • keewis 2
  • jukent 2
  • max-sixty 1

author_association 2

  • MEMBER 7
  • CONTRIBUTOR 6

issue 1

  • coarsen deletes attrs on original object · 13 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
689770846 https://github.com/pydata/xarray/issues/4120#issuecomment-689770846 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY4OTc3MDg0Ng== max-sixty 5635139 2020-09-09T19:30:42Z 2020-09-09T19:30:42Z MEMBER

Also, I think _replace is a bit confusing: the name seems to imply a inplace operation, but it returns a new object without changing the original object.

I think this mea culpa IIRC... I would be very open to consolidating this and .copy into something that returns a (generally shallow) copy of the object with some optional replacements. (The general approach is better than calling a constructor with every attribute, even if the names are not great)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
678425306 https://github.com/pydata/xarray/issues/4120#issuecomment-678425306 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3ODQyNTMwNg== keewis 14808389 2020-08-21T18:20:28Z 2020-08-21T18:20:28Z MEMBER

I just checked, and I think the issue can indeed be solved by replacing variable = self with variable = self.copy() (deep=False is the default, so no need to explicitly set it).

Also, I think _replace is a bit confusing: the name seems to imply a inplace operation, but it returns a new object without changing the original object.

{
    "total_count": 2,
    "+1": 2,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
678072493 https://github.com/pydata/xarray/issues/4120#issuecomment-678072493 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3ODA3MjQ5Mw== fujiisoup 6815844 2020-08-21T06:42:45Z 2020-08-21T06:42:45Z MEMBER

My last post was wrong.

I think this part overwrites the attrs,

https://github.com/pydata/xarray/blob/43a2a4bdf3a492d89aae9f2c5b0867932ff51cef/xarray/core/variable.py#L2028 https://github.com/pydata/xarray/blob/43a2a4bdf3a492d89aae9f2c5b0867932ff51cef/xarray/core/variable.py#L2073-L2076

The first line should be replaced by variable = self.copy(deep=False)

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
677945186 https://github.com/pydata/xarray/issues/4120#issuecomment-677945186 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3Nzk0NTE4Ng== fujiisoup 6815844 2020-08-20T22:51:21Z 2020-08-21T06:37:33Z MEMBER

~~These lines are suspicious. Maybe we should copy attrs here not geting its reference.~~

https://github.com/pydata/xarray/blob/43a2a4bdf3a492d89aae9f2c5b0867932ff51cef/xarray/core/rolling.py#L498

https://github.com/pydata/xarray/blob/43a2a4bdf3a492d89aae9f2c5b0867932ff51cef/xarray/core/rolling.py#L498

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
677939698 https://github.com/pydata/xarray/issues/4120#issuecomment-677939698 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3NzkzOTY5OA== jukent 46687291 2020-08-20T22:32:01Z 2020-08-20T22:53:14Z CONTRIBUTOR

My doubts on this are because self._replace are used elsewhere in the code.

Changing from _replace to copy causes the tests on coarsen to fail, I am looking more into this now.


It seems that a condition of copy is that the data shapes match (variable.py line 947).

If I run ds = xr.tutorial.load_dataset("air_temperature") ds.air.coarsen(lat=5) there is no problem, but once I add a .mean() to the end ds = xr.tutorial.load_dataset("air_temperature") ds.air.coarsen(lat=5).mean()

The error is ValueError: Data shape (2920, 5, 53) must match shape of object (2920, 25, 53)

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
677931591 https://github.com/pydata/xarray/issues/4120#issuecomment-677931591 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3NzkzMTU5MQ== dcherian 2448579 2020-08-20T22:08:01Z 2020-08-20T22:08:48Z MEMBER

~I think you are right. _replace calls the constructor with self.variable, ..., fastpath=True which does a direct assignment to DataArray._variable; so you have a new DataArray but with the same underlying data.~ EDIT: Ignore I was looking at dataarray.py; not variable.py

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
677929143 https://github.com/pydata/xarray/issues/4120#issuecomment-677929143 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3NzkyOTE0Mw== keewis 14808389 2020-08-20T22:01:37Z 2020-08-20T22:01:37Z MEMBER

That could be the issue. If it is, you should be able to fix that using python return self.copy(data=func(reshaped, axis=axes, **kwargs))

{
    "total_count": 2,
    "+1": 2,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
677926857 https://github.com/pydata/xarray/issues/4120#issuecomment-677926857 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3NzkyNjg1Nw== jukent 46687291 2020-08-20T21:56:07Z 2020-08-20T21:59:48Z CONTRIBUTOR

@dcherian Could this be because the return of variable.py/coarsen or is it likely happening earlier in the fx?

return self._replace(data=func(reshaped, axis=axes, **kwargs))

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
673869137 https://github.com/pydata/xarray/issues/4120#issuecomment-673869137 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3Mzg2OTEzNw== kmpaul 11411331 2020-08-14T03:55:32Z 2020-08-14T03:55:32Z CONTRIBUTOR

Yeah. That's true. I did overlook that. Thanks!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
673761464 https://github.com/pydata/xarray/issues/4120#issuecomment-673761464 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3Mzc2MTQ2NA== dcherian 2448579 2020-08-13T23:38:18Z 2020-08-13T23:38:18Z MEMBER

Thanks @kmpaul

These are two issues. But the third one is that the original dataset ds is being modified inplace by ds.coarsen. This should never happen.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
673748046 https://github.com/pydata/xarray/issues/4120#issuecomment-673748046 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3Mzc0ODA0Ng== kmpaul 11411331 2020-08-13T22:50:48Z 2020-08-13T22:50:48Z CONTRIBUTOR

Also, while I was walking through the logic in this problem, I found that in the _reduce_method functions of the DataArrayCoarsen and DatasetCoarsen classes, the kwargs variable is being shadowed:

https://github.com/pydata/xarray/blob/df7b2eae3a26c1e86bd5f1dd7dab9cc8c4e53914/xarray/core/rolling.py#L692-L701

So, you can see that the skipna option is just being ignored. That's another pretty easy fix, but it will change existing behavior. It might be prudent to look to see if there are any known bug reports referencing the skipna parameter being ignored.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
673746511 https://github.com/pydata/xarray/issues/4120#issuecomment-673746511 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3Mzc0NjUxMQ== kmpaul 11411331 2020-08-13T22:45:40Z 2020-08-13T22:45:40Z CONTRIBUTOR

@dcherian @jukent: After a little walk through the code, I think the problem is in xarray/core/variable.py. If you look at the first part of the coarsen function in this file:

https://github.com/pydata/xarray/blob/df7b2eae3a26c1e86bd5f1dd7dab9cc8c4e53914/xarray/core/variable.py#L1945-L1953

you will see that **kwargs is not being pass into the _coarsen_reshape function:

https://github.com/pydata/xarray/blob/df7b2eae3a26c1e86bd5f1dd7dab9cc8c4e53914/xarray/core/variable.py#L1961

And down at the bottom of that function:

https://github.com/pydata/xarray/blob/df7b2eae3a26c1e86bd5f1dd7dab9cc8c4e53914/xarray/core/variable.py#L2024-L2025

it retrieves the keep_attrs parameter from global options, and doesn't check for a passed-in argument.

@jukent, why don't you draft a PR to fix this problem?

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936
673729936 https://github.com/pydata/xarray/issues/4120#issuecomment-673729936 https://api.github.com/repos/pydata/xarray/issues/4120 MDEyOklzc3VlQ29tbWVudDY3MzcyOTkzNg== kmpaul 11411331 2020-08-13T21:56:14Z 2020-08-13T21:56:46Z CONTRIBUTOR

After some testing, I discovered that:

```python

Your code here

import xarray as xr

xr.set_options(keep_attrs=True) # NOTE GLOBAL OPTION!!!!

ds = xr.tutorial.load_dataset("air_temperature") ds2 = xr.tutorial.load_dataset("air_temperature")

xr.testing.assert_identical(ds, ds2) # passes ds.coarsen(lat=5).mean() xr.testing.assert_identical(ds, ds2) # passes ```

makes your example pass. So, it seems that somewhere along the chain of functions the keep_attrs parameter is being lost or modified.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  coarsen deletes attrs on original object 630062936

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 11.966ms · About: xarray-datasette