home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

12 rows where issue = 477081946 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 7

  • weiji14 4
  • shoyer 2
  • jhamman 2
  • naught101 1
  • kmuehlbauer 1
  • keewis 1
  • gabriel-abrahao 1

author_association 3

  • MEMBER 6
  • CONTRIBUTOR 5
  • NONE 1

issue 1

  • open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 · 12 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
799657214 https://github.com/pydata/xarray/issues/3185#issuecomment-799657214 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDc5OTY1NzIxNA== keewis 14808389 2021-03-15T18:38:58Z 2021-03-15T18:38:58Z MEMBER

yes, I would think so. Thanks for the reminder, @gabriel-abrahao.

The original issue was that libnetcdf picked up by netCDF4 didn't match the version installed by the netCDF4 wheels, which is not really xarray's fault.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946
799635182 https://github.com/pydata/xarray/issues/3185#issuecomment-799635182 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDc5OTYzNTE4Mg== gabriel-abrahao 30908904 2021-03-15T18:10:28Z 2021-03-15T18:10:28Z CONTRIBUTOR

Maybe this can be closed now, since #5021 changed the docs?

@naught101 Looks like those coordinates are now in the order as declared in the affine package (since rasterio >= 1.0). See also: https://rasterio.readthedocs.io/en/latest/topics/migrating-to-v1.html.

All coordinates are created correctly, so from my point of view, only the docs here have to be changed.

python from affine import Affine da = xr.open_rasterio('path_to_file.tif') transform = Affine(*da.attrs['transform']) nx, ny = da.sizes['x'], da.sizes['y'] x, y = np.meshgrid(np.arange(nx)+0.5, np.arange(ny)+0.5) * transform

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946
574215734 https://github.com/pydata/xarray/issues/3185#issuecomment-574215734 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDU3NDIxNTczNA== kmuehlbauer 5821660 2020-01-14T15:02:38Z 2020-01-14T15:02:38Z MEMBER

@naught101 Looks like those coordinates are now in the order as declared in the affine package (since rasterio >= 1.0). See also: https://rasterio.readthedocs.io/en/latest/topics/migrating-to-v1.html.

All coordinates are created correctly, so from my point of view, only the docs here have to be changed.

python from affine import Affine da = xr.open_rasterio('path_to_file.tif') transform = Affine(*da.attrs['transform']) nx, ny = da.sizes['x'], da.sizes['y'] x, y = np.meshgrid(np.arange(nx)+0.5, np.arange(ny)+0.5) * transform

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946
570993776 https://github.com/pydata/xarray/issues/3185#issuecomment-570993776 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDU3MDk5Mzc3Ng== naught101 167164 2020-01-06T03:54:03Z 2020-01-06T03:54:03Z NONE

I'm seeing a problem with geotiffs that I think might be related:

sh $ gdalinfo /home/nedcr/cr/software/ana/lib/geo/tests/data/Urandangi_MGA55.tif Driver: GTiff/GeoTIFF Files: /home/nedcr/cr/software/ana/lib/geo/tests/data/Urandangi_MGA55.tif /home/nedcr/cr/software/ana/lib/geo/tests/data/Urandangi_MGA55.tif.aux.xml Size is 10, 10 Coordinate System is: PROJCS["GDA94 / MGA zone 55", GEOGCS["GDA94", DATUM["Geocentric_Datum_of_Australia_1994", SPHEROID["GRS 1980",6378137,298.257222101, AUTHORITY["EPSG","7019"]], TOWGS84[0,0,0,0,0,0,0], AUTHORITY["EPSG","6283"]], PRIMEM["Greenwich",0, AUTHORITY["EPSG","8901"]], UNIT["degree",0.0174532925199433, AUTHORITY["EPSG","9122"]], AUTHORITY["EPSG","4283"]], PROJECTION["Transverse_Mercator"], PARAMETER["latitude_of_origin",0], PARAMETER["central_meridian",147], PARAMETER["scale_factor",0.9996], PARAMETER["false_easting",500000], PARAMETER["false_northing",10000000], UNIT["metre",1, AUTHORITY["EPSG","9001"]], AXIS["Easting",EAST], AXIS["Northing",NORTH], AUTHORITY["EPSG","28355"]] Origin = (-406507.954209543997422,7588834.152862589806318) Pixel Size = (263.500000000000000,-265.500000000000000) Metadata: AREA_OR_POINT=Area Image Structure Metadata: INTERLEAVE=BAND Corner Coordinates: Upper Left ( -406507.954, 7588834.153) (138d16' 6.99"E, 21d34'24.95"S) Lower Left ( -406507.954, 7586179.153) (138d16' 1.84"E, 21d35'50.30"S) Upper Right ( -403872.954, 7588834.153) (138d17'37.56"E, 21d34'29.73"S) Lower Right ( -403872.954, 7586179.153) (138d17'32.42"E, 21d35'55.09"S) Center ( -405190.454, 7587506.653) (138d16'49.70"E, 21d35'10.02"S) Band 1 Block=10x10 Type=Float32, ColorInterp=Gray Min=0.069 Max=8.066 Minimum=0.069, Maximum=8.066, Mean=2.556, StdDev=1.749 NoData Value=-3.40282306073709653e+38 Metadata: STATISTICS_MAXIMUM=8.06591796875 STATISTICS_MEAN=2.5563781395387 STATISTICS_MINIMUM=0.068740844726562 STATISTICS_STDDEV=1.7493082797107 STATISTICS_VALID_PERCENT=89

```python import xarray as xr from osgeo.gdal import Open

ras = Open('/home/nedcr/cr/software/ana/lib/geo/tests/data/Urandangi_MGA55.tif') ras.GetGeoTransform()

(-406507.954209544, 263.5, 0.0, 7588834.15286259, 0.0, -265.5)

ds = xr.open_rasterio('/home/nedcr/cr/software/ana/lib/geo/tests/data/Urandangi_MGA55.tif') ds.transform

(263.5, 0.0, -406507.954209544, 0.0, -265.5, 7588834.15286259)

```

The transform in the xarray dataset is transposed, and not really useful anymore.

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.0.0-37-lowlatency machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_AU.UTF-8 LOCALE: en_AU.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.2 xarray: 0.14.1 pandas: 0.25.1 numpy: 1.16.4 scipy: 1.3.0 netCDF4: 1.5.1.2 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.0.3.4 nc_time_axis: None PseudoNetCDF: None rasterio: 1.0.22 cfgrib: None iris: None bottleneck: None dask: 1.2.0 distributed: 1.27.1 matplotlib: 3.0.3 cartopy: 0.17.0 seaborn: None numbagg: None setuptools: 40.8.0 pip: 19.0.3 conda: None pytest: 4.3.1 IPython: 7.3.0 sphinx: 2.1.2
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946
520693149 https://github.com/pydata/xarray/issues/3185#issuecomment-520693149 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDUyMDY5MzE0OQ== weiji14 23487320 2019-08-13T05:24:06Z 2019-08-13T05:26:03Z CONTRIBUTOR

xr.show_versions() gets its libnetcdf version from netCDF4 (specially netCDF4.__netcdf4libversion__). So I'm guessing that somehow netCDF4 is picking up libnetcdf from somewhere else -- maybe you pip installed it? It might be worth trying another fresh conda environment...

I'm not sure where it's picking up the libnetcdf 4.6.3 version from, but I found your comment at https://github.com/pydata/xarray/issues/2535#issuecomment-445944261 and think it might indeed be an incompatibility issue with rasterio and netCDF4 binary wheels (do rasterio wheels include netcdf binaries?). Probably somewhat related to https://github.com/mapbox/rasterio/issues/1574 too.

Managed to get things to work by combining the workaround in this Pull Request and StackOverflow post, basically having pip compile the netcdf python package from source instead of using the wheel:

bash HDF5_DIR=$CONDA_PREFIX pip install --no-binary netCDF4 netCDF4==1.4.2 where $CONDA_PREFIX is the path to the conda environment e.g. /home/jovyan/.conda/envs/name-of-env. I've tested my MCVE code sample above and it works up to the latest netCDF4==1.5.1.2 version!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946
520680549 https://github.com/pydata/xarray/issues/3185#issuecomment-520680549 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDUyMDY4MDU0OQ== shoyer 1217238 2019-08-13T04:05:18Z 2019-08-13T04:05:18Z MEMBER

The clean xarray-tests conda environment that works with netcdf==1.5.1.2 has libnetcdf: 4.6.2, but for some strange reason, running xr.show_versions() on my setup shows libnetcdf: 4.6.3 even though conda list | grep libnetcdf shows that I've installed libnetcdf 4.6.2 h056eaf5_1002 conda-forge.

xr.show_versions() gets its libnetcdf version from netCDF4 (specially netCDF4.__netcdf4libversion__). So I'm guessing that somehow netCDF4 is picking up libnetcdf from somewhere else -- maybe you pip installed it? It might be worth trying another fresh conda environment...

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946
520658129 https://github.com/pydata/xarray/issues/3185#issuecomment-520658129 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDUyMDY1ODEyOQ== weiji14 23487320 2019-08-13T01:54:54Z 2019-08-13T01:54:54Z CONTRIBUTOR

Yes, there's https://gdal.org/drivers/raster/netcdf.html :smile: I've done a bit more debugging (having temporarily isolated salem from my script) and am still having issues with my setup.

The clean xarray-tests conda environment that works with netcdf==1.5.1.2 has libnetcdf: 4.6.2, but for some strange reason, running xr.show_versions() on my setup shows libnetcdf: 4.6.3 even though conda list | grep libnetcdf shows that I've installed libnetcdf 4.6.2 h056eaf5_1002 conda-forge.

Not sure if this libnetcdf 4.6.3 version is the problem, but it stands out the most (to me at least) when looking at the diff between my setup and the clean one. Is there a way to check the order in which xarray looks for the netcdf binaries as I feel it might be a PATH related issue. Also not sure if this issue fits here in xarray or somewhere else now...

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946
520234456 https://github.com/pydata/xarray/issues/3185#issuecomment-520234456 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDUyMDIzNDQ1Ng== jhamman 2443309 2019-08-11T14:53:10Z 2019-08-11T14:53:10Z MEMBER

GDAL can open so many formats (https://gdal.org/drivers/raster/index.html). We've really only tested against TIFF like formats though. I think the assumption will need to be is that rasterio will provide standard metadata for each of these drivers.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946
520186738 https://github.com/pydata/xarray/issues/3185#issuecomment-520186738 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDUyMDE4NjczOA== shoyer 1217238 2019-08-10T23:16:53Z 2019-08-10T23:16:53Z MEMBER

I guess GDAL can also read netCDF files :).

Indeed, we did not anticipate people using open_rasterio this way.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946
518930975 https://github.com/pydata/xarray/issues/3185#issuecomment-518930975 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDUxODkzMDk3NQ== weiji14 23487320 2019-08-07T04:05:19Z 2019-08-07T04:05:19Z CONTRIBUTOR

Hold on, the coordinates seems to be parsed out correctly from the netCDF file (even with netCDF==1.5.1.2) when I have a clean conda installation created following the instructions at https://xarray.pydata.org/en/latest/contributing.html#creating-a-python-environment.

I've isolated the issue and think the problem arises when I also import salem (an xarray accessor)... Will try to narrow this down before I close this issue.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946
518860992 https://github.com/pydata/xarray/issues/3185#issuecomment-518860992 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDUxODg2MDk5Mg== weiji14 23487320 2019-08-06T22:02:44Z 2019-08-06T22:02:44Z CONTRIBUTOR

Well open_rasterio did have an "experimental" warning on it in the docs :laughing:, but it was really nice having it work on GeoTIFFs and NetCDF files. I've forked the repo and will try to debug the situation a bit more. If anyone who's worked on that part of the codebase before has any pointers on what might be the cause of this issue / where to start that would be great.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946
518690320 https://github.com/pydata/xarray/issues/3185#issuecomment-518690320 https://api.github.com/repos/pydata/xarray/issues/3185 MDEyOklzc3VlQ29tbWVudDUxODY5MDMyMA== jhamman 2443309 2019-08-06T14:20:29Z 2019-08-06T14:20:29Z MEMBER

Thanks @weiji14 for opening this issue. When we implemented open_rasterio, I don't think we intended this to be used for netCDF data so I'm surprised it works (worked) at all. I think we'll need someone to dig into the xarray implementation a bit to see if we can pull out the correct coordinates (metadata) using rasterio.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  open_rasterio does not read coordinates from netCDF file properly with netCDF4>=1.4.2 477081946

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 17.694ms · About: xarray-datasette