issues: 302077805

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
302077805 MDU6SXNzdWUzMDIwNzc4MDU= 1961 Extend xarray with custom "coordinate wrappers" 4160723 closed 0     10 2018-03-04T11:26:15Z 2022-09-19T08:47:45Z 2022-09-19T08:47:44Z MEMBER      

Recent and ongoing developments in xarray turn DataArray and Dataset more and more into data wrappers that are extensible at (almost) every level:

  • domain-specific methods (accessors)
  • io backends (netcdf, raster, zarr, etc.) via an abstract DataStore interface
  • array backends (numpy, dask, sparse) via multidispatch or hooks (#1938)
  • soon custom indexes? (kd-tree, out-of-core indexes... #1603, #1650, #475)

Regarding the latter, I’m thinking about the idea of extending xarray at an even more abstract level, i.e., the possibility of adding / registering "coordinate wrappers" to DataArray or Dataset objects. Basically, it would correspond to adding any object that allows to do some operation based on one or several coordinates ~~(I haven’t found any better name than "coordinate agent" to describe that)~~.

EDIT: "coordinate agents" may not be quite right here, I changed that to "coordinate wrappers")

Indexes are a specific case of coordinate wrappers that serve the purpose of indexing. This is built in xarray.

While indexing is enough in 80% of cases, I see a couple of use cases where other coordinate wrappers (built outside of xarray) would be nice to have:

  • Grids. For example, xgcm implements operations (interp, diff) on physical axes that may each include several coordinates, depending on the position of the coordinate labels on the axis (center, left…). Other grids define their topology using a greater number of coordinates (e.g., ugrid). Storing regridding weights might be another use case?
  • Clocks. For example, xarray-simlab use one or several coordinates to define the timeline of a computational simulation.

In those examples we usually rely on coordinate attributes and/or classes that encapsulate xarray objects to implement the specific features that we need. While it works, it has limitations and I think it can be improved.

Custom coordinate wrappers would be a way of extending xarray that is very consistent with other current (or considered) extension mechanisms.

This is still a very vague idea and I’m sure that there are lots of details that can be discussed (serialization, etc.).

But before going further, I’d like to know your thoughts @pydata/xarray. Do you think it is a silly idea? Do you have in mind other use cases where custom coordinate wrappers would be useful?

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/1961/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed 13221727 issue

Links from other tables