issue_comments

Table actions

3 rows where issue = 1030811490 and user = 7237617 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • porterdf · 3 ✖

issue 1

  • problem appending to zarr on GCS when using json token · 3 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
968176008 https://github.com/pydata/xarray/issues/5878#issuecomment-968176008 https://api.github.com/repos/pydata/xarray/issues/5878 IC_kwDOAMm_X845tTGI porterdf 7237617 2021-11-13T23:43:17Z 2021-11-13T23:44:27Z NONE

Update: my local notebook accessing the public bucket does see the appended zarr store exactly as expected, while the 2i2c-hosted notebook still is not (been well over 3600s).

Also, I do as @jkingslake does above and set the cache_timeout=0. From GCSFs docs Set cache_timeout <= 0 for no caching, seems like the functionality we desire, yet I continue to only see the un-appended zarr 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   problem appending to zarr on GCS when using json token  1030811490
967408017 https://github.com/pydata/xarray/issues/5878#issuecomment-967408017 https://api.github.com/repos/pydata/xarray/issues/5878 IC_kwDOAMm_X845qXmR porterdf 7237617 2021-11-12T19:40:46Z 2021-11-13T23:25:53Z NONE

Right now, it shows the shape is [6], as expected after the appending. However, if you read the file immediately after appending (within the 3600s max-age), you will get the cached copy. The cached copy will still be of shape [3]--it won't know about the append.

Ignorant question: is this cache relevant to client (Jupyter) side or server (GCS) side? It has been well over 3600s and I'm still not seeing the appended zarr when reading it in using Xarray.

To test this hypothesis, you would need to disable caching on the bucket. Do you have privileges to do that?

I tried to do this last night but did not have permission myself. Perhaps @jkingslake does?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   problem appending to zarr on GCS when using json token  1030811490
967340995 https://github.com/pydata/xarray/issues/5878#issuecomment-967340995 https://api.github.com/repos/pydata/xarray/issues/5878 IC_kwDOAMm_X845qHPD porterdf 7237617 2021-11-12T18:52:01Z 2021-11-12T18:58:52Z NONE

Thanks for pointing out this cache feature @rabernat. I had no idea - makes sense in general but slows down testing if no known about! Anyway for my case, when appending the second Zarr store to the first, the Zarr's size (using gsutil du) does indeed double. I'm new to cloud storage, but my hunch is that this suggests it was appended?

Can you post the full stack trace of the error you get when you try to append?

In my instance, there is no error, only this returned: <xarray.backends.zarr.ZarrStore at 0x7f662d31f3a0>

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   problem appending to zarr on GCS when using json token  1030811490

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);