home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 1588516592

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1588516592 PR_kwDOAMm_X85KLKUD 7540 added 'storage_transformers' to valid_encodings 7460993 open 0     4 2023-02-16T23:29:44Z 2023-03-26T20:02:40Z   FIRST_TIME_CONTRIBUTOR   0 pydata/xarray/pulls/7540

This PR adds "storage_transformers" to the valid encodings of a variable, thus allowing Zarr stores to be written using the new sharding storage transformers.

Example of usage:

```python import xarray as xr import numpy as np from zarr._storage.v3_storage_transformers import ShardingStorageTransformer from zarr._storage.v3 import DirectoryStoreV3

Dummy dataset

da = xr.DataArray(np.random.randn(1000, 1000), dims=("x", "y")) ds = xr.Dataset(dict(dummy_var=da)) ds = ds.chunk({"x": 100, "y": 100})

Sharded store and sharding transformer

store = DirectoryStoreV3("/home/user/dummy-sharded-store.zarr") transformer = ShardingStorageTransformer("indexed", chunks_per_shard=(5, 1))

Write sharded store

ds.to_zarr( store, encoding={"dummy_var": {"storage_transformers": [transformer]}}, zarr_version=3, ) ```

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/7540/reactions",
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 1
}
    13221727 pull

Links from other tables

  • 4 rows from issues_id in issues_labels
  • 4 rows from issue in issue_comments
Powered by Datasette · Queries took 0.959ms · About: xarray-datasette