home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

3 rows where author_association = "CONTRIBUTOR" and issue = 140063713 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • IamJeffG 3

issue 1

  • ENH: Optional Read-Only RasterIO backend · 3 ✖

author_association 1

  • CONTRIBUTOR · 3 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
211362112 https://github.com/pydata/xarray/issues/790#issuecomment-211362112 https://api.github.com/repos/pydata/xarray/issues/790 MDEyOklzc3VlQ29tbWVudDIxMTM2MjExMg== IamJeffG 2002703 2016-04-18T12:40:31Z 2016-04-18T12:40:31Z CONTRIBUTOR

I didn't realize you'd already been integrating xarray with rasterio already. Is that library open source?

Alas no, and in fact I don't even have access anymore; it's with my last employer. I believe we'll benefit from the fresh look as you guys are already doing! Mostly I aimed to point out the value of exposing the affine transform, in addition to the data & coordinate variables.

:+1: Agreed to keep warping explicit, and not deal with it in this issue.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  ENH: Optional Read-Only RasterIO backend 140063713
211045128 https://github.com/pydata/xarray/issues/790#issuecomment-211045128 https://api.github.com/repos/pydata/xarray/issues/790 MDEyOklzc3VlQ29tbWVudDIxMTA0NTEyOA== IamJeffG 2002703 2016-04-17T15:38:03Z 2016-04-17T15:38:03Z CONTRIBUTOR

In a past life I made side library that wraps rasterio's API to take and return xarray.DataArrays. It provides IO/clip/warp/rasterize operations on DataArrays, which themselves are annotated with the CRS and affine transforms as attributes.

My most common use case was reading disparate rasters and aligning them to the same grid: 1. Use rasterio to load separate spatial rasters over roughly the same area; let's say one is 30-meter satellite and one is 3-meter agricultural yield. Often I'll immediately wrap them in an xarray.DataArray and persist the CRS and affine transform as attributes. 2. Clip the fine-resolution yield array to my area of interest. I can either use rasterio.read(window=(...)) when reading or xarray.sel(x=slice(...), y=slice(...)) post-hoc. 3. I want to overlay the two arrays in a single xarray.Dataset. They may or may not have the same projection but definitely do not the same grid. I'll leave the fine-resolution data untouched to avoid needless resampling, but instead upsample and re-align the coarse-resolution satellite array to match the affine transform of the clipped fine-resolution array. Behind-the-scenes this looks like:

# agriculture and satellite are both 2D DataArrays. satellite_10m = np.zeros(agriculture.shape, dtype=satellite.dtype) rasterio.warp.transform( satellite.values, satellite_10m, src_transform=satellite.attrs['transform'], src_crs=satellite.attrs['crs'], dst_transform=agriculture.attrs['transform'], dst_crs=agriculture.attrs['crs']) # This forces realization of the dataset in memory. I don't do much out-of-core. 4. Stack them into the same Dataset, sharing the transform and coordinate variables from the fine-resolution array. 5. Now use xarray to do cool compuations on the aligned datasets.

Reprojecting or clipping after reading xarray, like I do, goes against @perrygeo's recommendation. So maybe my example is moot, but I really like being able to do this programmatically in python, not CLI.

Even if xarray's new rasterio backend only provides a reader (and not rasterio.warp or rasterio.features functions), Step 3 shows it's very useful if, in addition to the data, the reader will expose the CRS and affine transform objects to the client.

However, if you both expose the transform and realize the coordinate variables, it's possible for them to diverge as the single source of truth. In my above workflow, anytime I clip (step 2) or warp (step 3) data, my side library needed to manually re-set that DataArray's transform and coordinate variables. (This is surely out of scope for rasterio or xarray!)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  ENH: Optional Read-Only RasterIO backend 140063713
211044118 https://github.com/pydata/xarray/issues/790#issuecomment-211044118 https://api.github.com/repos/pydata/xarray/issues/790 MDEyOklzc3VlQ29tbWVudDIxMTA0NDExOA== IamJeffG 2002703 2016-04-17T15:28:41Z 2016-04-17T15:28:41Z CONTRIBUTOR

@jhamman Small observation from your notebook: It's not good to assume a negative y-step size. Rarely, I will come across a dataset that breaks convention with a positive y coordinate, meaning the first pixel is the lower-left corner, but at least the dataset is self-consistent. Rasterio works beautifully even with these black sheep, so we don't want an xarray reader to force the assumption.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  ENH: Optional Read-Only RasterIO backend 140063713

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 13.536ms · About: xarray-datasette