html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/7574#issuecomment-1466263309,https://api.github.com/repos/pydata/xarray/issues/7574,1466263309,IC_kwDOAMm_X85XZWcN,5821660,2023-03-13T14:37:40Z,2023-03-13T14:37:40Z,MEMBER,"Thanks @martindurant for looking into this. I'll try to go to the bottom of this, if I find the time.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1605108888
https://github.com/pydata/xarray/issues/7574#issuecomment-1463576675,https://api.github.com/repos/pydata/xarray/issues/7574,1463576675,IC_kwDOAMm_X85XPGhj,5821660,2023-03-10T10:12:27Z,2023-03-10T10:12:27Z,MEMBER,"The difference between `parallel=False` vs `parallel=True` is how the cache is setup:
- `parallel=False`, the cache-key looks like:
`[, (,), 'r', (('decode_vlen_strings', True), ('invalid_netcdf', None)), '50fa07a0-7db1-420d-b2e1-5ad4c1328be4']`
- `parallel=True`, the cache-key looks like:
`[, ((b'FHIB\x00\x1dn""\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u[...cutted...],), 'r', (('decode_vlen_strings', True), ('invalid_netcdf', None)), 'be975f9d-710d-4c2e-9e03-606b6f8f0810']`
So this has something to do, when and how the cache is initialized. In the first case we've got the file-like object, but in the second case we've got the binary string. @martindurant do you have some idea what's going on here and where to look for fixing this?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1605108888
https://github.com/pydata/xarray/issues/7574#issuecomment-1463528885,https://api.github.com/repos/pydata/xarray/issues/7574,1463528885,IC_kwDOAMm_X85XO621,5821660,2023-03-10T09:36:38Z,2023-03-10T09:36:38Z,MEMBER,"@Berhinj The issue is with `parallel=True`. The following works as expected:
```python
import fsspec
import xarray as xr
paths = [
's3://noaa-goes16/ABI-L2-LSTC/2022/185/03/OR_ABI-L2-LSTC-M6_G16_s20221850301180_e20221850303553_c20221850305091.nc',
's3://noaa-goes16/ABI-L2-LSTC/2022/185/02/OR_ABI-L2-LSTC-M6_G16_s20221850201180_e20221850203553_c20221850205142.nc'
]
fs = fsspec.filesystem('s3', anon=True)
flist = [fs.open(path, mode=""rb"") for path in paths]
# single dataset
ds = xr.open_dataset(flist[0], engine=""h5netcdf"")
print(ds)
# multiple datasets, ATT: parallel=False works
ts = xr.open_mfdataset(
flist,
engine=""h5netcdf"",
combine=""nested"",
concat_dim=""t"",
parallel=False
)
print(ts)
```
I've not digged further, but it looks like it has some issues with the file-object...
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1605108888