issue_comments

Table actions

17 rows where issue = 392361367 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

author_association 3

issue 1

  • Adding mask to open_rasterio · 17 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
867648515 https://github.com/pydata/xarray/pull/2618#issuecomment-867648515 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDg2NzY0ODUxNQ== jsignell 4806877 2021-06-24T13:44:33Z 2021-06-24T13:44:33Z CONTRIBUTOR

I'm fine with this being closed. TBH I had totally forgotten about it :grimacing: 

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
867459036 https://github.com/pydata/xarray/pull/2618#issuecomment-867459036 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDg2NzQ1OTAzNg== keewis 14808389 2021-06-24T08:48:24Z 2021-06-24T08:48:24Z MEMBER

this has been unintentionally closed when renaming the branch to merge into to main, but given that we consider deprecating open_rasterio in favor of the rasterio engine provided by rioxarray (which seems to already support the mask kwarg) that's probably not a mistake. If you disagree feel free to reopen this PR.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
550467547 https://github.com/pydata/xarray/pull/2618#issuecomment-550467547 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDU1MDQ2NzU0Nw== snowman2 8699967 2019-11-06T19:37:19Z 2019-11-06T19:37:19Z CONTRIBUTOR

@jsignell, the masked kwarg is in rioxarray if you need it: https://corteva.github.io/rioxarray/html/examples/clip_geom.html

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
448394246 https://github.com/pydata/xarray/pull/2618#issuecomment-448394246 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ0ODM5NDI0Ng== pep8speaks 24736507 2018-12-18T22:24:07Z 2019-04-26T15:27:42Z NONE

Hello @jsignell! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 3138:80: E501 line too long (81 > 79 characters)

Comment last updated at 2019-04-26 15:27:42 UTC
 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
487059783 https://github.com/pydata/xarray/pull/2618#issuecomment-487059783 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ4NzA1OTc4Mw== snowman2 8699967 2019-04-26T13:38:13Z 2019-04-26T13:38:13Z CONTRIBUTOR

Good question, I was thinking adding tests.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
487052582 https://github.com/pydata/xarray/pull/2618#issuecomment-487052582 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ4NzA1MjU4Mg== jsignell 4806877 2019-04-26T13:15:40Z 2019-04-26T13:15:40Z CONTRIBUTOR

I think it would be good to write the dataset to a netcdf, and read it back in with xarray.open_dataset to ensure everything encodes/decides properly.

Do you mean write a test to do this, or just check it locally?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
487050467 https://github.com/pydata/xarray/pull/2618#issuecomment-487050467 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ4NzA1MDQ2Nw== snowman2 8699967 2019-04-26T13:08:39Z 2019-04-26T13:08:39Z CONTRIBUTOR

I think it would be good to write the dataset to a netcdf, and read it back in with xarray.open_dataset to ensure everything encodes/decides properly.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
487046879 https://github.com/pydata/xarray/pull/2618#issuecomment-487046879 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ4NzA0Njg3OQ== jsignell 4806877 2019-04-26T12:56:10Z 2019-04-26T12:56:10Z CONTRIBUTOR

Yeah I have some time to finish it up. Looking back at it now it seems like masked would be a better kwarg for open_rasterio. Is there anything else that should be changed?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
487003189 https://github.com/pydata/xarray/pull/2618#issuecomment-487003189 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ4NzAwMzE4OQ== fmaussion 10050469 2019-04-26T10:02:26Z 2019-04-26T10:02:26Z MEMBER

I'm sorry this has stalled now - I think this is a nice addition.

@jsignell : would you have time to finish this up? 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
449070341 https://github.com/pydata/xarray/pull/2618#issuecomment-449070341 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ0OTA3MDM0MQ== shoyer 1217238 2018-12-20T17:13:19Z 2018-12-20T17:13:19Z MEMBER

xarray will autoconvert a masked array into a numpy array with NaNs. It probably doesn't make a difference where this happens.

On Thu, Dec 20, 2018 at 9:04 AM Fabien Maussion notifications@github.com wrote:

Do we really want a np.masked_array or do we just want an array with np.Nans in it? I prefer nans, but I guess the downside is type conversion for ints?

Masked arrays come with huge performance drops, I don't think we should use these.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/2618#issuecomment-449067569, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1uztASAzUF-7fNanXXJvRWxELml3ks5u68MegaJpZM4ZZMJn . 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
449067569 https://github.com/pydata/xarray/pull/2618#issuecomment-449067569 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ0OTA2NzU2OQ== fmaussion 10050469 2018-12-20T17:04:30Z 2018-12-20T17:04:30Z MEMBER

Do we really want a np.masked_array or do we just want an array with np.Nans in it? I prefer nans, but I guess the downside is type conversion for ints?

Masked arrays come with huge performance drops, I don't think we should use these.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
448641329 https://github.com/pydata/xarray/pull/2618#issuecomment-448641329 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ0ODY0MTMyOQ== jsignell 4806877 2018-12-19T15:45:04Z 2018-12-19T15:45:04Z CONTRIBUTOR

I did as @snowman2 suggested but filled the mask with nans at the end. Since it is now using rasterio directly this should address @fmaussion's concern about nodata (which I think was ending up in the nodatavals attr anyways). I also moved the nodatavals from attrs to encoding when mask==True, but I didn't set it to true by default since I didn't want to break backwards compatibility. 

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
448633729 https://github.com/pydata/xarray/pull/2618#issuecomment-448633729 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ0ODYzMzcyOQ== jsignell 4806877 2018-12-19T15:24:28Z 2018-12-19T15:24:28Z CONTRIBUTOR

Hmm. Do we really want a np.masked_array or do we just want an array with np.Nans in it? I prefer nans, but I guess the downside is type conversion for ints?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
448629146 https://github.com/pydata/xarray/pull/2618#issuecomment-448629146 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ0ODYyOTE0Ng== jsignell 4806877 2018-12-19T15:11:49Z 2018-12-19T15:11:49Z CONTRIBUTOR

I was wondering if we should just pass the mask option to rasterio like that. I'll try it out.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
448628008 https://github.com/pydata/xarray/pull/2618#issuecomment-448628008 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ0ODYyODAwOA== snowman2 8699967 2018-12-19T15:08:34Z 2018-12-19T15:09:01Z CONTRIBUTOR

I think the solution would be to add a masked argument and property to the class RasterioArrayWrapper here.

And pass in the (masked=masked) argument here

That would pass riods.read( ..., masked=self.masked) when reading data from rasterio here

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
448542055 https://github.com/pydata/xarray/pull/2618#issuecomment-448542055 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ0ODU0MjA1NQ== fmaussion 10050469 2018-12-19T10:15:33Z 2018-12-19T10:15:33Z MEMBER

I wonder if nodatavals is the only attribute for masking - in the doc (https://rasterio.readthedocs.io/en/latest/topics/masks.html) they also mention nodata, which in my experience is also used

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367
448402521 https://github.com/pydata/xarray/pull/2618#issuecomment-448402521 https://api.github.com/repos/pydata/xarray/issues/2618 MDEyOklzc3VlQ29tbWVudDQ0ODQwMjUyMQ== snowman2 8699967 2018-12-18T22:54:44Z 2018-12-18T22:54:44Z CONTRIBUTOR

@jsignell, this looks like it would work - I like the implementation. Some things I would recommend to add is checks for when the nodata value varies across bands (maybe skip the mask and raise a warning) and when the nodata value is already NaN. Also, you should probably set the nodatavals to NaN if masked to resemble the behavior of xarray elsewhere.

I think @fmaussion would be a good reviewer for this PR.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Adding mask to open_rasterio 392361367

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);