html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/6033#issuecomment-1017189009,https://api.github.com/repos/pydata/xarray/issues/6033,1017189009,IC_kwDOAMm_X848oRKR,2443309,2022-01-20T07:25:28Z,2022-01-20T19:59:22Z,MEMBER,"It is worth mentioning that, specifically when using Zarr with fsspec, you have multiple layers of caching available. 

1. You can ask fsspec to cache locally:

```python
path = 's3://hrrrzarr/sfc/20211124/20211124_00z_fcst.zarr/surface/PRES'
ds = xr.open_zarr('simplecache::'+path)
```
(more details on configuration: https://filesystem-spec.readthedocs.io/en/latest/features.html#caching-files-locally)

2. You can ask Zarr to cache chunks as they are read:

```python
mapper = fsspec.get_mapper(path)
store = LRUStoreCache(mapper, max_size=1e9)
ds = xr.open_zarr(store)
```

(more details on configuration here: https://zarr.readthedocs.io/en/stable/api/storage.html#zarr.storage.LRUStoreCache) 

3. Configure a more complex mapper/cache using 3rd party mappers (i.e. [Zict](https://zict.readthedocs.io/en/latest/))

perhaps @martindurant has more to add here?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1064837571
https://github.com/pydata/xarray/issues/6033#issuecomment-981064185,https://api.github.com/repos/pydata/xarray/issues/6033,981064185,IC_kwDOAMm_X846edn5,14371165,2021-11-28T10:59:05Z,2021-11-28T10:59:05Z,MEMBER,If you think the data would fit in memory maybe #5704 would be enough?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1064837571
https://github.com/pydata/xarray/issues/6033#issuecomment-980501470,https://api.github.com/repos/pydata/xarray/issues/6033,980501470,IC_kwDOAMm_X846cUPe,5635139,2021-11-27T04:43:31Z,2021-11-27T04:43:31Z,MEMBER,"> Is there a way to check what is and isn't downloaded?

What is the time difference between the approach you've tried vs. before anything is downloaded? ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1064837571
https://github.com/pydata/xarray/issues/6033#issuecomment-980459050,https://api.github.com/repos/pydata/xarray/issues/6033,980459050,IC_kwDOAMm_X846cJ4q,5635139,2021-11-26T22:41:32Z,2021-11-26T22:41:32Z,MEMBER,"Thanks @adair-kovac .

To what extent is this the time to download the data? How big is the dataset? What's the absolute difference for a very small dataset? Or for a large dataset including the time to download the data first?

The threading issue may be threading contention, or it could be the main thread waiting for another thread to complete the download (others will know more here).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1064837571