home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 966758709

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/5878#issuecomment-966758709 https://api.github.com/repos/pydata/xarray/issues/5878 966758709 IC_kwDOAMm_X845n5E1 48723181 2021-11-12T02:05:04Z 2021-11-12T02:05:04Z NONE

Thanks for taking a look @rabernat.

The code below writes a new zarr and checks immediately if it's there using gcsfs. It seems to appear within a few seconds.

Is this what you meant?

``` %%time import fsspec import xarray as xr import json import gcsfs

define a mapper to the ldeo-glaciology bucket. - needs a token

with open('../secrets/ldeo-glaciology-bc97b12df06b.json') as token_file: token = json.load(token_file)

get a mapper with fsspec for a new zarr

mapper = fsspec.get_mapper('gs://ldeo-glaciology/append_test/test11', mode='w', token=token)

check what files are in there

fs = gcsfs.GCSFileSystem(project='pangeo-integration-te-3eea', mode='ab', cache_timeout = 0) print('Files in the test directory before writing:') filesBefore = fs.ls('gs://ldeo-glaciology/append_test/') print(*filesBefore,sep='\n')

define a simple datasets

ds0 = xr.Dataset({'temperature': (['time'], [50, 51, 52])}, coords={'time': [1, 2, 3]})

write the simple dataset to zarr

ds0.to_zarr(mapper)

check to see if the new file is there

print('Files in the test directory after writing:') filesAfter = fs.ls('gs://ldeo-glaciology/append_test/') print(*filesAfter,sep='\n') Output: Files in the test directory before writing: ldeo-glaciology/append_test/test1 ldeo-glaciology/append_test/test10 ldeo-glaciology/append_test/test2 ldeo-glaciology/append_test/test3 ldeo-glaciology/append_test/test4 ldeo-glaciology/append_test/test5 ldeo-glaciology/append_test/test6 ldeo-glaciology/append_test/test7 ldeo-glaciology/append_test/test8 ldeo-glaciology/append_test/test9 Files in the test directory after writing: ldeo-glaciology/append_test/test1 ldeo-glaciology/append_test/test10 ldeo-glaciology/append_test/test11 ldeo-glaciology/append_test/test2 ldeo-glaciology/append_test/test3 ldeo-glaciology/append_test/test4 ldeo-glaciology/append_test/test5 ldeo-glaciology/append_test/test6 ldeo-glaciology/append_test/test7 ldeo-glaciology/append_test/test8 ldeo-glaciology/append_test/test9 CPU times: user 130 ms, sys: 16.5 ms, total: 146 ms Wall time: 2.19 s ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  1030811490
Powered by Datasette · Queries took 0.66ms · About: xarray-datasette