html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/1770#issuecomment-453799889,https://api.github.com/repos/pydata/xarray/issues/1770,453799889,MDEyOklzc3VlQ29tbWVudDQ1Mzc5OTg4OQ==,2443309,2019-01-13T03:52:46Z,2019-01-13T03:52:46Z,MEMBER,"Closing. I think our fixes in xarray and zarr last winter addressed most of the problems here. If others feel differently, please reopen.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,280626621 https://github.com/pydata/xarray/issues/1770#issuecomment-396828029,https://api.github.com/repos/pydata/xarray/issues/1770,396828029,MDEyOklzc3VlQ29tbWVudDM5NjgyODAyOQ==,3019665,2018-06-13T06:27:36Z,2018-06-13T06:27:36Z,NONE,Is this still an issue?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,280626621 https://github.com/pydata/xarray/issues/1770#issuecomment-351106809,https://api.github.com/repos/pydata/xarray/issues/1770,351106809,MDEyOklzc3VlQ29tbWVudDM1MTEwNjgwOQ==,306380,2017-12-12T16:33:00Z,2017-12-12T16:33:00Z,MEMBER,"https://github.com/dask/gcsfs/pull/49 ```python import gcsfs fs = gcsfs.GCSFileSystem(project='pangeo-181919') gcsmap = gcsfs.mapping.GCSMap('pangeo-data/test997', gcs=fs, check=True, create=True) import dask.array as dsa shape = (30, 50, 1080, 2160) chunkshape = (1, 1, 1080, 2160) ar = dsa.random.random(shape, chunks=chunkshape) import zarr za = zarr.create(ar.shape, chunks=chunkshape, dtype=ar.dtype, store=gcsmap) In [2]: import cloudpickle In [3]: %time len(cloudpickle.dumps(gcsmap)) CPU times: user 0 ns, sys: 0 ns, total: 0 ns Wall time: 560 µs Out[3]: 213 ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,280626621 https://github.com/pydata/xarray/issues/1770#issuecomment-351106449,https://api.github.com/repos/pydata/xarray/issues/1770,351106449,MDEyOklzc3VlQ29tbWVudDM1MTEwNjQ0OQ==,6042212,2017-12-12T16:31:55Z,2017-12-12T16:31:55Z,CONTRIBUTOR,"Yes, `dirs` exists to prevent the need to query the server for file-lists multiple times. There is an outstanding issue to move to prefix/delimited listing as with s3fs, rather than fetch the complete listing for a bucket. If all the paths are known beforehand, as might be the case for zarr, then it may be of no use at all - but actually I'm not sure then why it would have been populated.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,280626621 https://github.com/pydata/xarray/issues/1770#issuecomment-351104996,https://api.github.com/repos/pydata/xarray/issues/1770,351104996,MDEyOklzc3VlQ29tbWVudDM1MTEwNDk5Ng==,306380,2017-12-12T16:27:27Z,2017-12-12T16:27:27Z,MEMBER,t,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,280626621 https://github.com/pydata/xarray/issues/1770#issuecomment-351104967,https://api.github.com/repos/pydata/xarray/issues/1770,351104967,MDEyOklzc3VlQ29tbWVudDM1MTEwNDk2Nw==,306380,2017-12-12T16:27:23Z,2017-12-12T16:27:23Z,MEMBER,"It looks like serializing `GCSFileSystem.dirs` can be quite expensive. I *think* that this is just here for caching and efficiency, is this correct?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,280626621 https://github.com/pydata/xarray/issues/1770#issuecomment-351101352,https://api.github.com/repos/pydata/xarray/issues/1770,351101352,MDEyOklzc3VlQ29tbWVudDM1MTEwMTM1Mg==,306380,2017-12-12T16:17:11Z,2017-12-12T16:17:11Z,MEMBER,"Ah, we can just serialize the `.gcs` object and leave the rest to the `GCSFileSystem`. Perhaps the `MutableMapping` collections class defines get/setstate differently. I'll play aorund.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,280626621 https://github.com/pydata/xarray/issues/1770#issuecomment-351100872,https://api.github.com/repos/pydata/xarray/issues/1770,351100872,MDEyOklzc3VlQ29tbWVudDM1MTEwMDg3Mg==,6042212,2017-12-12T16:15:43Z,2017-12-12T16:15:43Z,CONTRIBUTOR,"I am puzzled that serializing the mapping is pulling the data. GCSMap does not have get/set_state, but the only attributes are the GCSFileSystem and path. Perhaps the `__getitem__` gets called? As for the GCSFileSystem, it stores the token with a renewable token, which lives indefinitely, and the refresh API is called upon deserialization. There should probably be a check in `_call` to ensure that the token hasn't expired.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,280626621 https://github.com/pydata/xarray/issues/1770#issuecomment-351098929,https://api.github.com/repos/pydata/xarray/issues/1770,351098929,MDEyOklzc3VlQ29tbWVudDM1MTA5ODkyOQ==,306380,2017-12-12T16:09:37Z,2017-12-12T16:09:37Z,MEMBER,"When pickling the GCS mapping it looks like we're actually pulling down all of the data within it (Zarr has already placed some metadata) instead of serializing the connection information. @martindurant what information should we safely be passing around when serializing? These tasks would need to remain valid for longer than the standard hour-long short-lived-token.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,280626621 https://github.com/pydata/xarray/issues/1770#issuecomment-350387325,https://api.github.com/repos/pydata/xarray/issues/1770,350387325,MDEyOklzc3VlQ29tbWVudDM1MDM4NzMyNQ==,306380,2017-12-08T22:24:35Z,2017-12-09T21:32:00Z,MEMBER,"There threading locks in your profile is likely due to using the dask threaded scheduler. I recommend using the single threaded scheduler when profiling. Dask.get","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,280626621 https://github.com/pydata/xarray/issues/1770#issuecomment-350381719,https://api.github.com/repos/pydata/xarray/issues/1770,350381719,MDEyOklzc3VlQ29tbWVudDM1MDM4MTcxOQ==,1197350,2017-12-08T21:54:01Z,2017-12-08T21:54:01Z,MEMBER,"does `dask.array.store(..., lock=None)` do the same thing as `dask.array.store(..., lock=False)`?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,280626621