home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

10 rows where issue = 344058811 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 3

  • djhoese 4
  • snowman2 4
  • fmaussion 2

author_association 2

  • CONTRIBUTOR 8
  • MEMBER 2

issue 1

  • Proposal: Update rasterio backend to store CRS/nodata information in standard locations. · 10 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
428177967 https://github.com/pydata/xarray/issues/2308#issuecomment-428177967 https://api.github.com/repos/pydata/xarray/issues/2308 MDEyOklzc3VlQ29tbWVudDQyODE3Nzk2Nw== snowman2 8699967 2018-10-09T12:46:54Z 2018-10-09T12:46:54Z CONTRIBUTOR

Feel free to re-open if this becomes something of interest. Thanks!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Proposal: Update rasterio backend to store CRS/nodata information in standard locations. 344058811
408016722 https://github.com/pydata/xarray/issues/2308#issuecomment-408016722 https://api.github.com/repos/pydata/xarray/issues/2308 MDEyOklzc3VlQ29tbWVudDQwODAxNjcyMg== fmaussion 10050469 2018-07-26T08:19:29Z 2018-07-26T08:19:29Z MEMBER

I'm going to leave this decision to @shoyer and @jhamman !

If we go forward with this, we have a couple of possibilities:

  1. add crs to a coordinate variable and keep the attribute too (not so nice because doubled)
  2. add crs to a coordinate variable and deprecate the crs attribute

Same with _FillValue.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Proposal: Update rasterio backend to store CRS/nodata information in standard locations. 344058811
407862762 https://github.com/pydata/xarray/issues/2308#issuecomment-407862762 https://api.github.com/repos/pydata/xarray/issues/2308 MDEyOklzc3VlQ29tbWVudDQwNzg2Mjc2Mg== snowman2 8699967 2018-07-25T19:07:27Z 2018-07-25T19:07:27Z CONTRIBUTOR

@fmaussion, I guess the main reason for the proposed change is to be able to read in the raster using open_rasterio() and be able to generate a netCDF using to_netcdf() that would be compatible with the rasterio model when using rasterio.open().

Also, since: "xarray.Dataset is an in-memory representation of a netCDF file", to me it makes more sense to store the data in a standard netCDF location that is consistent regardless of the backend. In doing so, other parts of xarray and libraries build off of xarray won't be required to find the same data in multiple locations.

Thanks for taking the time to consider this idea. I have used it quite a bit and it has made the datasets compatible across the various GIS tools I use (rasterio, GDAL, QGIS). It has definitely made my life easier and so I wanted to share what I have learned. But, it is just an idea. I really appreciate the work of the xarray developers and whatever y'all decide is fine with me.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Proposal: Update rasterio backend to store CRS/nodata information in standard locations. 344058811
407739349 https://github.com/pydata/xarray/issues/2308#issuecomment-407739349 https://api.github.com/repos/pydata/xarray/issues/2308 MDEyOklzc3VlQ29tbWVudDQwNzczOTM0OQ== djhoese 1828519 2018-07-25T12:35:52Z 2018-07-25T12:35:52Z CONTRIBUTOR

@fmaussion I completely agree except now that all of this is being brought up I see why it may have been better to put the 'crs' in the coordinates of the DataArray returned by open_rasterio. Since two DataArrays in different projections are not and should not be considered "mergeable". But I can also see how this walks the line of special handling of a data format by trying to interpret things in a certain way, but that line is already pretty blurry in the case of reading rasterio compatible datasets.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Proposal: Update rasterio backend to store CRS/nodata information in standard locations. 344058811
407684501 https://github.com/pydata/xarray/issues/2308#issuecomment-407684501 https://api.github.com/repos/pydata/xarray/issues/2308 MDEyOklzc3VlQ29tbWVudDQwNzY4NDUwMQ== fmaussion 10050469 2018-07-25T08:58:31Z 2018-07-25T08:58:31Z MEMBER

Again, I'm yet to be convinced that this logic should live in xarray, regardless if netCDF or rasterio is used as a backend. The job of xarray is to read the rasterio files in a "xarray way" (e.g. in order to leverage dask) and exposing the file attributes to the user, ideally with semantics/attribute names which are as close as possible as the underlying data model (rasterio).

All the logic you describe could live in a dedicated library which would become a wrapper around xarray's open_* functions, and converting them to the format you describe.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Proposal: Update rasterio backend to store CRS/nodata information in standard locations. 344058811
407613401 https://github.com/pydata/xarray/issues/2308#issuecomment-407613401 https://api.github.com/repos/pydata/xarray/issues/2308 MDEyOklzc3VlQ29tbWVudDQwNzYxMzQwMQ== djhoese 1828519 2018-07-25T02:30:38Z 2018-07-25T02:30:38Z CONTRIBUTOR

This is the output of using xarray with a standard GOES-16 ABI L1B data file:

```python In [2]: import xarray as xr

In [3]: nc = xr.open_dataset('OR_ABI-L1b-RadF-M3C01_G16_s20181741200454_e20181741211221_c20181741211264.nc')

In [4]: nc.data_vars['Rad'] Out[4]: <xarray.DataArray 'Rad' (y: 10848, x: 10848)> array([[nan, nan, nan, ..., nan, nan, nan], [nan, nan, nan, ..., nan, nan, nan], [nan, nan, nan, ..., nan, nan, nan], ..., [nan, nan, nan, ..., nan, nan, nan], [nan, nan, nan, ..., nan, nan, nan], [nan, nan, nan, ..., nan, nan, nan]], dtype=float32) Coordinates: t datetime64[ns] ... * y (y) float32 0.151858 0.15183 0.151802 0.151774 0.151746 ... * x (x) float32 -0.151858 -0.15183 -0.151802 -0.151774 -0.151746 ... y_image float32 ... x_image float32 ... Attributes: long_name: ABI L1b Radiances standard_name: toa_outgoing_radiance_per_unit_wavelength sensor_band_bit_depth: 10 valid_range: [ 0 1022] units: W m-2 sr-1 um-1 resolution: y: 0.000028 rad x: 0.000028 rad grid_mapping: goes_imager_projection cell_methods: t: point area: point ancillary_variables: DQF

In [5]: nc.data_vars['goes_imager_projection'] Out[5]: <xarray.DataArray 'goes_imager_projection' ()> array(-2147483647, dtype=int32) Coordinates: t datetime64[ns] ... y_image float32 ... x_image float32 ... Attributes: long_name: GOES-R ABI fixed grid projection grid_mapping_name: geostationary perspective_point_height: 35786023.0 semi_major_axis: 6378137.0 semi_minor_axis: 6356752.31414 inverse_flattening: 298.2572221 latitude_of_projection_origin: 0.0 longitude_of_projection_origin: -75.0 sweep_angle_axis: x ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Proposal: Update rasterio backend to store CRS/nodata information in standard locations. 344058811
407610465 https://github.com/pydata/xarray/issues/2308#issuecomment-407610465 https://api.github.com/repos/pydata/xarray/issues/2308 MDEyOklzc3VlQ29tbWVudDQwNzYxMDQ2NQ== snowman2 8699967 2018-07-25T02:12:16Z 2018-07-25T02:12:16Z CONTRIBUTOR

That is strange. Was the variable put in data_vars and not in coords?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Proposal: Update rasterio backend to store CRS/nodata information in standard locations. 344058811
407560681 https://github.com/pydata/xarray/issues/2308#issuecomment-407560681 https://api.github.com/repos/pydata/xarray/issues/2308 MDEyOklzc3VlQ29tbWVudDQwNzU2MDY4MQ== djhoese 1828519 2018-07-24T21:38:34Z 2018-07-24T21:38:34Z CONTRIBUTOR

I wouldn't expect it to add crs if there wasn't a grid_mapping specified, but if it was then I would. In a simple test where I did xr.open_dataset('my_nc.nc') which has a grid_mapping attribute, xarray does nothing special to create a crs or other named coordinate variable referencing the associated grid_mapping variable.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Proposal: Update rasterio backend to store CRS/nodata information in standard locations. 344058811
407541054 https://github.com/pydata/xarray/issues/2308#issuecomment-407541054 https://api.github.com/repos/pydata/xarray/issues/2308 MDEyOklzc3VlQ29tbWVudDQwNzU0MTA1NA== snowman2 8699967 2018-07-24T20:28:28Z 2018-07-24T20:28:28Z CONTRIBUTOR

I don't think so. The crs should already exist on the dataset beforehand and I think it may be a bit too much to expect it to add it to the dataset in that function as there are some that don't have it and not all xarray datasets are geospatially located.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Proposal: Update rasterio backend to store CRS/nodata information in standard locations. 344058811
407522819 https://github.com/pydata/xarray/issues/2308#issuecomment-407522819 https://api.github.com/repos/pydata/xarray/issues/2308 MDEyOklzc3VlQ29tbWVudDQwNzUyMjgxOQ== djhoese 1828519 2018-07-24T19:23:51Z 2018-07-24T19:23:51Z CONTRIBUTOR

@snowman2 This should mean that open_dataset should handle CRS specially too, right? Currently it doesn't seem to do anything special for the coordinate variable pointed to by grid_mapping.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Proposal: Update rasterio backend to store CRS/nodata information in standard locations. 344058811

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 1271.267ms · About: xarray-datasette