home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 360590825

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/pull/1793#issuecomment-360590825 https://api.github.com/repos/pydata/xarray/issues/1793 360590825 MDEyOklzc3VlQ29tbWVudDM2MDU5MDgyNQ== 3019665 2018-01-25T20:29:58Z 2018-01-25T20:29:58Z NONE

Yep, using dask.array.store regularly with the distributed scheduler both on our cluster and in a local Docker image for testing. Am using Zarr Arrays as the targets for store to write to. Basically rechunk the data to match the chunking selected for the Zarr Array and then write out in parallel lock-free.

Our cluster uses NFS for things like one's home directory. So these are accessible across nodes. Also there are other types of storage available that are a bit faster and still remain accessible across nodes. So these work pretty well.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  283388962
Powered by Datasette · Queries took 0.553ms · About: xarray-datasette