html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/3815#issuecomment-604207177,https://api.github.com/repos/pydata/xarray/issues/3815,604207177,MDEyOklzc3VlQ29tbWVudDYwNDIwNzE3Nw==,3019665,2020-03-26T03:26:02Z,2020-03-26T03:26:02Z,NONE,Sure an upstream issue would be welcome. Thanks for unpacking that further Mark 😀,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-604181264,https://api.github.com/repos/pydata/xarray/issues/3815,604181264,MDEyOklzc3VlQ29tbWVudDYwNDE4MTI2NA==,90008,2020-03-26T01:49:45Z,2020-03-26T01:49:45Z,CONTRIBUTOR,"And actually, zarr provides a `data` argument in `create_dataset` that actually encounters the same bug ```python import zarr import numpy as np name = 'hello' data = np.array('world', dtype=' I've avoided attrs since they often get ""lost"" in computation, and don't get dragged along as rigorously as coordinates. Yeah, we're gradually fixing this. You can set a config to keep attrs by default, or pass `keep_attrs=True` in most methods. We might still be missing a couple of methods; would appreciate a bug report for any of those.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-603921577,https://api.github.com/repos/pydata/xarray/issues/3815,603921577,MDEyOklzc3VlQ29tbWVudDYwMzkyMTU3Nw==,90008,2020-03-25T15:54:37Z,2020-03-25T15:54:37Z,CONTRIBUTOR,"Hmm, interesting! I've avoided attrs since they often get ""lost"" in computation, and don't get dragged along as rigorously as coordinates. I do have some real coordinates that are stored as strings. Thanks for the quickfeedback. Here is the reproducing code without using context managers (which auto clsoe things you know) ```python import xarray as xr import zarr x = xr.Dataset() x['hello'] = 'world' x with zarr.ZipStore('test_store.zip', mode='w') as store: x.to_zarr(store) read_store = zarr.ZipStore('test_store.zip', mode='r') x_read = xr.open_zarr(read_store).compute() # The error will happen before this line is executed # read_store.close() ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-603650213,https://api.github.com/repos/pydata/xarray/issues/3815,603650213,MDEyOklzc3VlQ29tbWVudDYwMzY1MDIxMw==,5635139,2020-03-25T05:36:00Z,2020-03-25T05:36:00Z,MEMBER,"I had a quick debug but didn't completely resolve it. It works fine if you add it as an attribute; do you mean to be adding a single string as a data variable? ```python In [8]: import xarray as xr ...: import zarr ...: x = xr.Dataset() ...: x.attrs['hello'] = 'world' # changed ...: x ...: with zarr.ZipStore('test_store.zip', mode='w') as store: ...: x.to_zarr(store) ...: with zarr.ZipStore('test_store.zip', mode='r') as store: ...: x_read = xr.open_zarr(store).compute() ...: ``` Debugging is non-trivial because the ZipFile is closed by the time I could run `%debug`. It's not impossible but would take more than a quick look in a repl. That said, I do think this looks likely to be a zarr issue given it depends on the storage format that zarr is using. It's possible xarray is calling it incorrectly but I'd think it's less likely. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-603646754,https://api.github.com/repos/pydata/xarray/issues/3815,603646754,MDEyOklzc3VlQ29tbWVudDYwMzY0Njc1NA==,5635139,2020-03-25T05:21:53Z,2020-03-25T05:21:53Z,MEMBER,"@hmaarrfk my apologies, I misread your issue and it was always reproducible. I had thought it was referencing a file `test_store.zip` when actually it was writing it, mea culpa.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-603608958,https://api.github.com/repos/pydata/xarray/issues/3815,603608958,MDEyOklzc3VlQ29tbWVudDYwMzYwODk1OA==,90008,2020-03-25T02:45:12Z,2020-03-25T02:45:12Z,CONTRIBUTOR,I will have to try the debugging things you mentionned some later time :/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-603608822,https://api.github.com/repos/pydata/xarray/issues/3815,603608822,MDEyOklzc3VlQ29tbWVudDYwMzYwODgyMg==,90008,2020-03-25T02:44:40Z,2020-03-25T02:44:40Z,CONTRIBUTOR,Not sure if the builds in https://github.com/pydata/xarray/pull/3888 help reproduce things or not?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-603601762,https://api.github.com/repos/pydata/xarray/issues/3815,603601762,MDEyOklzc3VlQ29tbWVudDYwMzYwMTc2Mg==,90008,2020-03-25T02:16:29Z,2020-03-25T02:16:29Z,CONTRIBUTOR,"hmm i didn't realize this. I""m running from conda-forge + linux. Let me try on your CIs.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-603569789,https://api.github.com/repos/pydata/xarray/issues/3815,603569789,MDEyOklzc3VlQ29tbWVudDYwMzU2OTc4OQ==,5635139,2020-03-25T00:13:16Z,2020-03-25T00:13:16Z,MEMBER,"You did supply the traceback! But it's not reproducible. And consistent with https://github.com/pydata/xarray/issues/3831#issuecomment-595403356, it's difficult to attribute the break on examples like these. You could try debugging into the `return array[key.tuple]` call, which is the last one in xarray, and identifying whether xarray is making a bad call or zarr is incorrectly failing?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-603556048,https://api.github.com/repos/pydata/xarray/issues/3815,603556048,MDEyOklzc3VlQ29tbWVudDYwMzU1NjA0OA==,90008,2020-03-24T23:24:53Z,2020-03-24T23:24:53Z,CONTRIBUTOR,See the zipstore example in my first comment,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-603555953,https://api.github.com/repos/pydata/xarray/issues/3815,603555953,MDEyOklzc3VlQ29tbWVudDYwMzU1NTk1Mw==,90008,2020-03-24T23:24:34Z,2020-03-24T23:24:34Z,CONTRIBUTOR,"I thought I provided it, but in either case, here is my traceback
```python In [3]: import xarray as xr ...: import zarr ...: x = xr.Dataset() ...: x['hello'] = 'world' ...: x ...: with zarr.ZipStore('test_store.zip', mode='w') as store: ...: x.to_zarr(store) ...: with zarr.ZipStore('test_store.zip', mode='r') as store: ...: x_read = xr.open_zarr(store).compute() ...: --------------------------------------------------------------------------- BadZipFile Traceback (most recent call last) in 7 x.to_zarr(store) 8 with zarr.ZipStore('test_store.zip', mode='r') as store: ----> 9 x_read = xr.open_zarr(store).compute() 10 ~/miniconda3/envs/mcam_dev/lib/python3.7/site-packages/xarray/core/dataset.py in compute(self, **kwargs) 805 """""" 806 new = self.copy(deep=False) --> 807 return new.load(**kwargs) 808 809 def _persist_inplace(self, **kwargs) -> ""Dataset"": ~/miniconda3/envs/mcam_dev/lib/python3.7/site-packages/xarray/core/dataset.py in load(self, **kwargs) 657 for k, v in self.variables.items(): 658 if k not in lazy_data: --> 659 v.load() 660 661 return self ~/miniconda3/envs/mcam_dev/lib/python3.7/site-packages/xarray/core/variable.py in load(self, **kwargs) 373 self._data = as_compatible_data(self._data.compute(**kwargs)) 374 elif not hasattr(self._data, ""__array_function__""): --> 375 self._data = np.asarray(self._data) 376 return self 377 ~/miniconda3/envs/mcam_dev/lib/python3.7/site-packages/numpy/core/_asarray.py in asarray(a, dtype, order) 83 84 """""" ---> 85 return array(a, dtype, copy=False, order=order) 86 87 ~/miniconda3/envs/mcam_dev/lib/python3.7/site-packages/xarray/core/indexing.py in __array__(self, dtype) 555 def __array__(self, dtype=None): 556 array = as_indexable(self.array) --> 557 return np.asarray(array[self.key], dtype=None) 558 559 def transpose(self, order): ~/miniconda3/envs/mcam_dev/lib/python3.7/site-packages/xarray/backends/zarr.py in __getitem__(self, key) 47 array = self.get_array() 48 if isinstance(key, indexing.BasicIndexer): ---> 49 return array[key.tuple] 50 elif isinstance(key, indexing.VectorizedIndexer): 51 return array.vindex[ ~/miniconda3/envs/mcam_dev/lib/python3.7/site-packages/zarr/core.py in __getitem__(self, selection) 570 571 fields, selection = pop_fields(selection) --> 572 return self.get_basic_selection(selection, fields=fields) 573 574 def get_basic_selection(self, selection=Ellipsis, out=None, fields=None): ~/miniconda3/envs/mcam_dev/lib/python3.7/site-packages/zarr/core.py in get_basic_selection(self, selection, out, fields) 693 if self._shape == (): 694 return self._get_basic_selection_zd(selection=selection, out=out, --> 695 fields=fields) 696 else: 697 return self._get_basic_selection_nd(selection=selection, out=out, ~/miniconda3/envs/mcam_dev/lib/python3.7/site-packages/zarr/core.py in _get_basic_selection_zd(self, selection, out, fields) 709 # obtain encoded data for chunk 710 ckey = self._chunk_key((0,)) --> 711 cdata = self.chunk_store[ckey] 712 713 except KeyError: ~/miniconda3/envs/mcam_dev/lib/python3.7/site-packages/zarr/storage.py in __getitem__(self, key) 1249 with self.mutex: 1250 with self.zf.open(key) as f: # will raise KeyError -> 1251 return f.read() 1252 1253 def __setitem__(self, key, value): ~/miniconda3/envs/mcam_dev/lib/python3.7/zipfile.py in read(self, n) 914 self._offset = 0 915 while not self._eof: --> 916 buf += self._read1(self.MAX_N) 917 return buf 918 ~/miniconda3/envs/mcam_dev/lib/python3.7/zipfile.py in _read1(self, n) 1018 if self._left <= 0: 1019 self._eof = True -> 1020 self._update_crc(data) 1021 return data 1022 ~/miniconda3/envs/mcam_dev/lib/python3.7/zipfile.py in _update_crc(self, newdata) 946 # Check the CRC if we're at the end of the file 947 if self._eof and self._running_crc != self._expected_crc: --> 948 raise BadZipFile(""Bad CRC-32 for file %r"" % self.name) 949 950 def read1(self, n): BadZipFile: Bad CRC-32 for file 'hello/0' ```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-603295542,https://api.github.com/repos/pydata/xarray/issues/3815,603295542,MDEyOklzc3VlQ29tbWVudDYwMzI5NTU0Mg==,5635139,2020-03-24T15:10:21Z,2020-03-24T15:10:21Z,MEMBER,Do you have a reproducible example? Otherwise I'm not sure how to help resolve...,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-603190621,https://api.github.com/repos/pydata/xarray/issues/3815,603190621,MDEyOklzc3VlQ29tbWVudDYwMzE5MDYyMQ==,90008,2020-03-24T11:41:37Z,2020-03-24T11:41:37Z,CONTRIBUTOR,"My guess is that that xarray might be trying to write to the store character by character??? Otherwise, not too sure. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844 https://github.com/pydata/xarray/issues/3815#issuecomment-602271821,https://api.github.com/repos/pydata/xarray/issues/3815,602271821,MDEyOklzc3VlQ29tbWVudDYwMjI3MTgyMQ==,5635139,2020-03-22T20:57:02Z,2020-03-22T20:57:02Z,MEMBER,"Hi @hmaarrfk , I don't know this area of code well but attempting to ensure that we resolve the question. Should this be an xarray or a zarr issue? Is there something you think xarray is doing incorrectly here?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,573577844