html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/4100#issuecomment-1102540862,https://api.github.com/repos/pydata/xarray/issues/4100,1102540862,IC_kwDOAMm_X85Bt3A-,26384082,2022-04-19T11:43:48Z,2022-04-19T11:43:48Z,NONE,"In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity

If this issue remains relevant, please comment here or remove the `stale` label; otherwise it will be marked as closed automatically
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,626042217
https://github.com/pydata/xarray/issues/4100#issuecomment-635586295,https://api.github.com/repos/pydata/xarray/issues/4100,635586295,MDEyOklzc3VlQ29tbWVudDYzNTU4NjI5NQ==,1217238,2020-05-28T20:21:44Z,2020-05-28T20:21:44Z,MEMBER,"Take a look here: https://portal.hdfgroup.org/display/knowledge/Questions+about+thread-safety+and+concurrent+access

I haven't actually tried compiling in thread-safe mode myself","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,626042217
https://github.com/pydata/xarray/issues/4100#issuecomment-635564640,https://api.github.com/repos/pydata/xarray/issues/4100,635564640,MDEyOklzc3VlQ29tbWVudDYzNTU2NDY0MA==,433803,2020-05-28T20:00:00Z,2020-05-28T20:00:00Z,NONE,"@shoyer could you tell me more about what it means to compile HDF5 in ""thread safe"" mode?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,626042217
https://github.com/pydata/xarray/issues/4100#issuecomment-635560237,https://api.github.com/repos/pydata/xarray/issues/4100,635560237,MDEyOklzc3VlQ29tbWVudDYzNTU2MDIzNw==,433803,2020-05-28T19:50:56Z,2020-05-28T19:50:56Z,NONE,"Hey @shoyer, thanks very much for the quick response and for suggesting possible workarounds!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,626042217
https://github.com/pydata/xarray/issues/4100#issuecomment-635102475,https://api.github.com/repos/pydata/xarray/issues/4100,635102475,MDEyOklzc3VlQ29tbWVudDYzNTEwMjQ3NQ==,1217238,2020-05-28T05:03:30Z,2020-05-28T05:03:30Z,MEMBER,"There are also a few work-arounds you might consider in the meantime here:

1. If you're reading netCDF4 files, HDF5 can be compiled in ""thread safe"" mode (which just adds its own global lock).
2. If you're reading netCDF3 files, the ""scipy"" backend is thread safe.
3. Other file formats like ""zarr"" don't have this issue at all, and more gracefully scale to very large datasets.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,626042217
https://github.com/pydata/xarray/issues/4100#issuecomment-635099870,https://api.github.com/repos/pydata/xarray/issues/4100,635099870,MDEyOklzc3VlQ29tbWVudDYzNTA5OTg3MA==,1217238,2020-05-28T04:55:01Z,2020-05-28T04:55:01Z,MEMBER,"Thanks for the clear report!

I know we use backend-specific locks by default when opening netCDF files, so I was initially puzzled by this. But now that I've looked back over the implementation, this makes sense.

We currently only guarantee thread safety when reading data *after* files have been opened. For example, you could write something like:
```python
    dataset = xr.open_dataset(SAVED_FILE_NAME, engine=""netcdf4"")
    threads = [
        threading.Thread(target=lambda: do_something_with_xarray(dataset))
        for _ in range(N_THREADS)
    ]
```

For many use-cases (e.g., in dask), this is a sufficient form of parallelism, because xarray's file opening is lazy and only needs to read metadata, not array values.

It would indeed be nice if `open_dataset()` itself were thread safe. Mostly I think this could be achieved by making use of the existing `lock` attribute found on NetCDF4DataStore and most other DataStore classes.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,626042217