home / github / issues

Menu
  • Search all tables
  • GraphQL API

issues: 865206283

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
865206283 MDU6SXNzdWU4NjUyMDYyODM= 5210 Probable error using zarr process synchronizer 25071375 closed 0     1 2021-04-22T17:05:10Z 2023-10-14T20:36:18Z 2023-10-14T20:36:18Z CONTRIBUTOR      

Hi I was trying to use Xarray open_zarr with the Zarr ProcessSynchronizer class and it produces a set of errors, I don't know if those errors are produced because I don't understand the logic of the ProcessSynchronizer or is a simple bug. I have a small code which reproduces the problems, basically, if I put a different path in the Zarr ProcessSynchronizer class all the error disappear but it creates a new folder.

```python import xarray import zarr import numpy as np

arr = xarray.DataArray( data=np.array([ [1, 2, 7, 4, 5], [np.nan, 3, 5, 5, 6], [3, 3, np.nan, 5, 6], [np.nan, 3, 10, 5, 6], [np.nan, 7, 8, 5, 6], ], dtype=float), dims=['index', 'columns'], coords={'index': [0, 1, 2, 3, 4], 'columns': [0, 1, 2, 3, 4]}, )

If the synchronizer is created using another path the code will work without any error but it creates a new folder,

that is the correct way to use the process synchronizer?

synchronizer = zarr.ProcessSynchronizer('dummy_array.sync')

Using the original path produce a set of weird problems

synchronizer = zarr.ProcessSynchronizer('dummy_array')

Executing the commented code I obtain: PermissionError: [WinError 5].

arr.to_dataset(name='data').to_zarr('dummy_array', mode='w', synchronizer=synchronizer, compute=True)

arr.to_dataset(name='data').to_zarr('dummy_array', mode='w', compute=True)

If this section of the code is uncommented It will throw a different error when xarray.open_zarr being executed

a = zarr.open_array('dummy_array/data', synchronizer=synchronizer, mode='r')

PermissionError: [Errno 13] Permission denied

xarray.open_zarr('dummy_array', synchronizer=synchronizer) ```

Output of <tt>xr.show_versions()</tt> INSTALLED VERSIONS ------------------ commit: None python: 3.8.5 (default, Sep 3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 165 Stepping 2, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: es_ES.cp1252 libhdf5: 1.10.4 libnetcdf: None xarray: 0.17.0 pandas: 1.1.3 numpy: 1.19.2 scipy: 1.5.2 netCDF4: None pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: 2.7.1 cftime: None nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.30.0 distributed: 2.30.1 matplotlib: 3.3.2 cartopy: None seaborn: 0.11.0 numbagg: None pint: None setuptools: 50.3.1.post20201107 pip: 21.0.1 conda: 4.10.0 pytest: 6.2.3 IPython: 7.19.0 sphinx: 3.2.1
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/5210/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 1 row from issues_id in issues_labels
  • 0 rows from issue in issue_comments
Powered by Datasette · Queries took 75.587ms · About: xarray-datasette