issue_comments

Table actions

4 rows where issue = 1268956107 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 2

author_association 2

issue 1

  • When a raster is modified, it cannot be read correctly. · 4 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1154420620 https://github.com/pydata/xarray/issues/6693#issuecomment-1154420620 https://api.github.com/repos/pydata/xarray/issues/6693 IC_kwDOAMm_X85Ezw-M JiangtaoLiud 27909361 2022-06-13T20:43:33Z 2022-06-13T20:43:33Z NONE

I forgot to add lock=False. It works now. Thanks!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   When a raster is modified, it cannot be read correctly. 1268956107
1153654151 https://github.com/pydata/xarray/issues/6693#issuecomment-1153654151 https://api.github.com/repos/pydata/xarray/issues/6693 IC_kwDOAMm_X85Ew12H keewis 14808389 2022-06-13T08:53:32Z 2022-06-13T20:33:48Z MEMBER

If you open the changed file in a new session you get the expected result, which indicates that this issue is a due to file caching. If you call python xr.backends.file_manager.FILE_CACHE.clear() in between the xr.open_dataset calls I can confirm that the calls are loaded as expected (don't use that in your code, though, that is a implementation detail).

If you actually use the dataset (e.g. by calling .load()), the file cache seems to reset and the subsequent open also has the expected shape.

The same happens if you pass lock=False to open_dataset, so if you can make sure to have closed the file (ds.close()) before the next open you should be able to use that.

In general I think it would be best to avoid rewriting files (whether that is by modifying or by deleting the file and writing it again), but I'm not an expert on backends so there might be a way to handle that.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   When a raster is modified, it cannot be read correctly. 1268956107
1154397714 https://github.com/pydata/xarray/issues/6693#issuecomment-1154397714 https://api.github.com/repos/pydata/xarray/issues/6693 IC_kwDOAMm_X85EzrYS keewis 14808389 2022-06-13T20:29:58Z 2022-06-13T20:32:22Z MEMBER

can you post what exactly you tried that didn't work? For me this works: python In [1]: import rasterio ...: import xarray as xr ...: import numpy as np ...: import os ...: ...: data = np.arange(10 * 10.0).reshape(10, 10) ...: inp_rs = "demo.tif" ...: ...: ...: def generate_raster(data): ...: with rasterio.open( ...: inp_rs, ...: "w", ...: driver="GTiff", ...: height=data.shape[0], ...: width=data.shape[1], ...: count=1, ...: dtype=data.dtype, ...: crs="+proj=latlong", ...: ) as dst: ...: dst.write(data, 1) ...: ...: ...: generate_raster(data) ...: ...: rs1 = rasterio.open(inp_rs) ...: print("rasterio :", rs1.shape) ...: rs2 = xr.open_dataset(inp_rs, engine="rasterio", lock=False) ...: print("xarray: ", rs2.band_data.shape) ...: ...: # modify it ...: os.remove(inp_rs) ...: data_new = np.arange(9 * 9.0).reshape(9, 9) ...: generate_raster(data_new) ...: ...: rs1 = rasterio.open(inp_rs) ...: print("rasterio :", rs1.shape) ...: rs2 = xr.open_dataset(inp_rs, engine="rasterio", lock=False) ...: print("xarray: ", rs2.band_data.shape) xarray: (1, 10, 10) rasterio : (10, 10) xarray: (1, 9, 9) rasterio : (9, 9)

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   When a raster is modified, it cannot be read correctly. 1268956107
1154276167 https://github.com/pydata/xarray/issues/6693#issuecomment-1154276167 https://api.github.com/repos/pydata/xarray/issues/6693 IC_kwDOAMm_X85EzNtH JiangtaoLiud 27909361 2022-06-13T19:06:04Z 2022-06-13T19:06:04Z NONE

No, it doesn't work. It is still the wrong result.

Before modifying this raster, I used rs1.close() rs2.close() To be more precise, I deleted the old raster first and then generated a new one later.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   When a raster is modified, it cannot be read correctly. 1268956107

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);