home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

8 rows where issue = 1622197017 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 2

  • kmuehlbauer 5
  • RY4GIT 3

author_association 2

  • MEMBER 5
  • NONE 3

issue 1

  • Y-axis flipped when reading data with Xarray · 8 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1470494062 https://github.com/pydata/xarray/issues/7621#issuecomment-1470494062 https://api.github.com/repos/pydata/xarray/issues/7621 IC_kwDOAMm_X85XpfVu RY4GIT 52061672 2023-03-15T17:53:22Z 2023-03-15T17:53:36Z NONE

@kmuehlbauer

The code I provided above (with the fn_NASA_Earthdata_download file) works for you too, but with fn_NSIDC_output it does not?

You are right. I've gone made updates to both the issue description (i.e., the first post) and the Jupyter notebook based on our discussion. So, there might be an issue with the data or the rasterio engine, but for now, we'll use the NetCDF engine for NSIDC data, and hope that someone figures out the problem. Also, thanks for your suggestion on SO; it's much appreciated!

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Y-axis flipped when reading data with Xarray 1622197017
1469467986 https://github.com/pydata/xarray/issues/7621#issuecomment-1469467986 https://api.github.com/repos/pydata/xarray/issues/7621 IC_kwDOAMm_X85Xlk1S kmuehlbauer 5821660 2023-03-15T07:15:47Z 2023-03-15T07:15:47Z MEMBER

@RY4GIT Glad you can use the code. And good there's a way to use the data as is without tampering.

Regarding the different rasterio-approaches, just to be on the same page. The code I provided above (with the fn_NASA_Earthdata_download file) works for you too, but with fn_NSIDC_output it does not? Out of the box, the second also doesn't work for me. Didn't check this.

I've no immediate idea, what's going on in that case. The only thing I can think of is, that in the conversion process something has gone wrong with the georeferencing and/or the image origin. I hope you can figure that out, eventually.

Here for reference my package versions:

INSTALLED VERSIONS ------------------ commit: None python: 3.10.5 | packaged by conda-forge | (main, Jun 14 2022, 07:06:46) [GCC 10.3.0] python-bits: 64 OS: Linux OS-release: 5.14.21-150400.24.46-default machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: de_DE.UTF-8 LOCALE: ('de_DE', 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.9.1 xarray: 2023.2.1.dev22+g49ae0f8d.d20230310 pandas: 1.4.2 numpy: 1.23.5 scipy: 1.9.1 netCDF4: 1.6.2 pydap: None h5netcdf: 1.1.0 h5py: 3.8.0 Nio: None zarr: 2.12.0 cftime: 1.5.1.1 nc_time_axis: 1.4.1 PseudoNetCDF: None rasterio: 1.3.6 cfgrib: None iris: 3.2.1 bottleneck: 1.3.5 dask: 2022.10.0 distributed: 2022.10.0 matplotlib: 3.6.3 cartopy: 0.21.1 seaborn: None numbagg: None fsspec: 2023.1.0 cupy: None pint: 0.17 sparse: None flox: None numpy_groupies: None setuptools: 59.2.0 pip: 21.3.1 conda: None pytest: 7.1.2 mypy: None IPython: 8.2.0 sphinx: None rioxarray: 0.13.3 gdal: 3.6.2 pyproj: 3.4.1 PROJ: 9.1.1
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Y-axis flipped when reading data with Xarray 1622197017
1469408924 https://github.com/pydata/xarray/issues/7621#issuecomment-1469408924 https://api.github.com/repos/pydata/xarray/issues/7621 IC_kwDOAMm_X85XlWac RY4GIT 52061672 2023-03-15T06:19:10Z 2023-03-15T06:19:10Z NONE

The post on SO is actually what I wrote. I didn't feel right about messing around with y coordinates, so I came here to figure out what was happening.

Thank you so much for taking the time out of your busy schedule to download the data and test it.

The Xarray Netcdf engine solution worked on my end as well. It was really helpful in exploring what works best for this specific data, where latitude and longitude are stored in different locations. I appreciate your code and will use it for my future processing.

As for the rasterio solution, the output is still flipped along the Y axis. I updated both the rasterio and xarray packages to the latest versions, but the results were still the same. I used the same lines of code that you provided, but with a different file (instead of direct NASA output, I used NSIDC output, as the filename was specified as a variable, fn_NSIDC_output).

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Y-axis flipped when reading data with Xarray 1622197017
1468375177 https://github.com/pydata/xarray/issues/7621#issuecomment-1468375177 https://api.github.com/repos/pydata/xarray/issues/7621 IC_kwDOAMm_X85XhaCJ kmuehlbauer 5821660 2023-03-14T15:58:20Z 2023-03-14T15:58:20Z MEMBER

@RY4GIT I forgot to add my package versions. I'll add them the next day. But I'm pretty sure to have the most recent one's installed in a Python 3.10 conda-forge environment.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Y-axis flipped when reading data with Xarray 1622197017
1467552710 https://github.com/pydata/xarray/issues/7621#issuecomment-1467552710 https://api.github.com/repos/pydata/xarray/issues/7621 IC_kwDOAMm_X85XeRPG kmuehlbauer 5821660 2023-03-14T07:40:28Z 2023-03-14T07:41:24Z MEMBER

For the rasterio-approach:

python ds_NASA_download_rasterio = xr.open_dataset(os.path.join(input_path, fn_NASA_Earthdata_download), engine='rasterio') ds_NASA_download_rasterio = ds_NASA_download_rasterio.set_coords(["cell_lat", "cell_lon"]) ds_NASA_download_rasterio.Geophysical_Data_precipitation_total_surface_flux[0].plot(y="cell_lat", x="cell_lon")

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Y-axis flipped when reading data with Xarray 1622197017
1467520091 https://github.com/pydata/xarray/issues/7621#issuecomment-1467520091 https://api.github.com/repos/pydata/xarray/issues/7621 IC_kwDOAMm_X85XeJRb kmuehlbauer 5821660 2023-03-14T07:21:30Z 2023-03-14T07:35:37Z MEMBER

First, I've found this on SO: https://gis.stackexchange.com/questions/454543/fixing-the-flipped-inverted-y-axis-in-the-xarray-with-rasterio

For your data reading method 1, it works for me like this:

```python

load root group with coordinates

ds_NSIDC_root = xr.open_dataset(os.path.join(input_path, fn_NSIDC_output), group="/", engine='netcdf4')

load data from Geophysical_Data group

ds_NSIDC_precip = xr.open_dataset(os.path.join(input_path, fn_NSIDC_output), group="Geophysical_Data", engine='netcdf4')

merge groups

ds_NSIDC = xr.merge([ds_NSIDC_root, ds_NSIDC_precip])

plot

ds_NSIDC.precipitation_total_surface_flux.plot()

the above is essentially the same as

ds_NSIDC.precipitation_total_surface_flux.plot(x="x", y="y")

```

This can be expanded to use the cell_lon/cell_lat:

```

set lat/lon as coords

ds_NSIDC = ds_NSIDC.set_coords(["cell_lon", "cell_lat"]) ds_NSIDC.precipitation_total_surface_flux.plot(x="cell_lon", y="cell_lat") ```

Update: This should also work like above when directly using NASA Earth Data.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Y-axis flipped when reading data with Xarray 1622197017
1467200120 https://github.com/pydata/xarray/issues/7621#issuecomment-1467200120 https://api.github.com/repos/pydata/xarray/issues/7621 IC_kwDOAMm_X85Xc7J4 RY4GIT 52061672 2023-03-14T01:20:10Z 2023-03-14T01:21:40Z NONE

Thank you for checking the code, @kmuehlbauer. However, I am still experiencing issues even after fixing it.

Regarding the coordinates: - The y coordinate starts from the maximum values and decreases as you move down the array. - The latitude values increase with increasing y-coordinate values. - i.e., The latitude corresponding to the maximum y coordinate is around 85, and the one corresponding to the minimum y coordinate is around -85.

Regarding the precipitation data: - The structure is the same as the cell_lat/cell_lon array. - The y coordinate starts from the maximum values and decreases as you move down the array. - If I plot the precipitation with the xy coordinate, it looks like this, indicating that the data is still inverted. Please refer to these images for further clarification: Note that the unit is read incorrectly as degrees_north and degrees_east; please ignore it.

Here is the updated code:

``` lons = ds_NSIDC_output_rasterio.cell_lon.load() lons_array = lons[0][0]

lats = ds_NSIDC_output_rasterio.cell_lat.load() lats_array = np.arange(lats[0][0][0], lats[0][-1][0], -1*(lats.max().values-lats.min().values)/ds_NSIDC_output.y.size)

ds_NSIDC_output_rasterio_xymanual = xr.DataArray( data=ds_NSIDC_output_rasterio.precipitation_total_surface_flux.sel(band=1).values, dims = ["y","x"], coords = dict( y = lats_array, x = lons_array.values ) ) ds_NSIDC_output_rasterio_xymanual.plot() ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Y-axis flipped when reading data with Xarray 1622197017
1466920659 https://github.com/pydata/xarray/issues/7621#issuecomment-1466920659 https://api.github.com/repos/pydata/xarray/issues/7621 IC_kwDOAMm_X85Xb27T kmuehlbauer 5821660 2023-03-13T20:36:54Z 2023-03-13T20:36:54Z MEMBER

@RY4GIT You would need to assign cell_lat/cell_lon as coordinates of the dataset and then use x="cell_lon" and y="cell_lat" in the call to .plot().

Plotting without that, will use the dimension coordinates (x,y) and there might be a difference in origin (upper vs. lower).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Y-axis flipped when reading data with Xarray 1622197017

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 16.531ms · About: xarray-datasette