issue_comments

Table actions

3 rows where author_association = "NONE" and issue = 830507003 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 2

issue 1

  • Saving zarr to remote location lower cases all data_vars · 3 ✖

author_association 1

  • NONE · 3 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
860879523 https://github.com/pydata/xarray/issues/5028#issuecomment-860879523 https://api.github.com/repos/pydata/xarray/issues/5028 MDEyOklzc3VlQ29tbWVudDg2MDg3OTUyMw== joshmoore 88113 2021-06-14T17:58:36Z 2021-06-14T17:58:36Z NONE

A heads up that https://github.com/zarr-developers/zarr-python/pull/755 reverses the default behavior of FSStore and so as of Zarr 2.8.3 this might "just work"?

 
{
    "total_count": 2,
    "+1": 2,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Saving zarr to remote location lower cases all data_vars 830507003
799393304 https://github.com/pydata/xarray/issues/5028#issuecomment-799393304 https://api.github.com/repos/pydata/xarray/issues/5028 MDEyOklzc3VlQ29tbWVudDc5OTM5MzMwNA== fwrite 4426586 2021-03-15T12:50:33Z 2021-03-15T13:05:40Z NONE

The issue appears to be in zarr.storage.FSStore, and is also discussed in zarr#649.

Recently, a storage_options argument was added to open_zarr(). This argument might be used to provide the extra option storage_options={'normalize_keys': False} to Zarr.

Coming back to my last example: python zarr.open_consolidated('http://localhost:8080/', storage_options={'normalize_keys': False})['Group'].Variable[0,0,0] Works.

Work-around, instantiate the store first: python xr.open_zarr(store=zarr.storage.FSStore('http://localhost:8080/', normalize_keys=False), group='Group', consolidated=True).Variable[0,0,0].compute() # = 1

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Saving zarr to remote location lower cases all data_vars 830507003
799381383 https://github.com/pydata/xarray/issues/5028#issuecomment-799381383 https://api.github.com/repos/pydata/xarray/issues/5028 MDEyOklzc3VlQ29tbWVudDc5OTM4MTM4Mw== fwrite 4426586 2021-03-15T12:31:03Z 2021-03-15T12:31:03Z NONE

Interestingly, the same applies to locally created and remotely opened data sets. The variables are recognized correctly from the consolidated metadata, but on opening are made lower case.

A quick example:

```python import xarray as xr, numpy as np x, y, t = np.arange(500), np.arange(500), np.arange(0,500,6)

xr.Dataset( data_vars={'Group/Variable': (('t', 'y', 'x'), np.ones((t.size, x.size, y.size)))}, coords={'x': x, 'y': y, 'Time': t} ).to_zarr('/tmp/zarr2058', consolidated=True) Locally, everything is fine (every value should be 1).python xr.open_zarr('/tmp/zarr2058', group='Group').Variable[0,0,0].compute() # = 1 ```

However, if we access the data via http, the values are not read correctly.

To test, start a local webserver: bash python3 -m http.server -d /tmp/zarr2058 8080

Open our array, the group and variable are properly cased (and derived from .zmetadata): python xr.open_zarr('http://localhost:8080/', group='Group', consolidated=True) However, access to one of the variables will yield a lower case request to the server, and output nan for any array position: python xr.open_zarr('http://localhost:8080/', group='Group', consolidated=True).Variable[0,0,0].compute() # = nan The terminal will read the requested url, all in lower case: "GET /group/variable/0.0.0 HTTP/1.1" 404

However, the problem appears to be in Zarr, not (only) in xarray: python zarr.open_group('http://localhost:8080/')['Group'] Behaves exactly the same.

Environment:

Output of <tt>xr.show_versions()</tt> INSTALLED VERSIONS ------------------ commit: None python: 3.7.7 (default, Mar 23 2020, 22:36:06) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.8.0-44-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.1 xarray: 0.15.0 pandas: 1.1.3 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.4.2 pydap: None h5netcdf: 0.10.0 h5py: 2.10.0 Nio: None zarr: 2.6.1 cftime: 1.4.1 nc_time_axis: None PseudoNetCDF: None rasterio: 1.1.0 cfgrib: None iris: None bottleneck: None dask: 2.13.0 distributed: 2.30.1 matplotlib: 3.3.2 cartopy: None seaborn: None numbagg: None setuptools: 52.0.0.post20210125 pip: 21.0.1 conda: None pytest: None IPython: 7.13.0 sphinx: None 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Saving zarr to remote location lower cases all data_vars 830507003

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);