home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 1047608434

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1047608434 I_kwDOAMm_X84-cTxy 5954 Writeable backends via entrypoints 1197350 open 0     7 2021-11-08T15:47:12Z 2021-11-09T16:28:59Z   MEMBER      

The backend refactor has gone a long way towards making it easier to implement custom backend readers via entry points. However, it is still not clear how to implement a writeable backend from a third party package as an entry point. Some of the reasons for this are:

  • While our reading function (open_dataset) has a generic name, our writing functions (Dataset.to_netcdf / Dataset.to_zarr) are still format specific. (Related to https://github.com/pydata/xarray/issues/3638). I propose we introduce a generic Dataset.to method and deprecate the others.
  • The BackendEntrypoint base class does not have a writing method, just open_dataset: https://github.com/pydata/xarray/blob/e0deb9cf0a5cd5c9e3db033fd13f075added9c1e/xarray/backends/common.py#L356-L370 (Related to https://github.com/pydata/xarray/issues/1970)
  • As a result, writing is implemented ad-hoc for each backend.
  • This makes it impossible for a third-party package to to implement writing.

We should fix this situation! Here are the steps I would take.

  • [ ] Decide on the desired API for writeable backends.
  • [ ] Formalize this in the BackendEntrypoint base class.
  • [ ] Refactor the existing writeable backends (netcdf4-python, h5netcdf, scipy, Zarr) to use this API
  • [ ] Maybe deprecate to_zarr and to_netcdf (or at least refactor to make a shallow call to a generic method)
  • [ ] Encourage third party implementors to try it (e.g. TileDB)
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/5954/reactions",
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 1
}
    13221727 issue

Links from other tables

  • 2 rows from issues_id in issues_labels
  • 7 rows from issue in issue_comments
Powered by Datasette · Queries took 0.839ms · About: xarray-datasette