issues: 1479121713

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1479121713 I_kwDOAMm_X85YKZsx 7363 expand dimension by re-allocating larger arrays with more space "at the end of the corresponding dimension", block copying previously existing data, and autofill newly created entry by a default value (note: alternative to reindex, but much faster for extending large arrays along, for example, the time dimension) 8382834 closed 0     29 2022-12-06T13:33:12Z 2022-12-07T13:15:38Z 2022-12-07T13:07:10Z CONTRIBUTOR      

Is your feature request related to a problem?

I regularly need to extend some netCDF files along the time axis (to add some newer data segments for example), and I do not know of a "good" method for this in xarray. For examle, SO recommends the following which is a bit heavy: https://stackoverflow.com/questions/34987972/expand-dimensions-xarray . The function with the natural name for this task, https://docs.xarray.dev/en/stable/generated/xarray.DataArray.expand_dims.html , adds an axis rather than extend an existing axis.

I have done a bit of looking around, I hope I do not miss any resources.

Describe the solution you'd like

I would like to be able to, given a dataset in xarray (in my case the underlying data are netCDF, but I guess this makes no difference):

  • issue a simple command / run a single function, that would
  • add / extend by a number of entries on a dimension, making the dimension grow "by its end" (I know there is not really a dimension array, but you see what I mean: grow the dimension by making it longer, from its end)
  • add the corresponding number of entries in the right way on all data variables that use this dimension, making the data variables grow "by their end"
  • fill the new entries of the data variables that have just been extended with a default value (would be great if this could be user specifiable)

Describe alternatives you've considered

As of today, I think (if I am wrong please let me know / suggest better :) ) that the best solution is to simply:

  • extract the data from the existing xarray dataset into numpy arrays
  • extend the data variables as numpy arrays by hand
  • generate a new xarray dataset from the extended numpy arrays and copy the metadata from the initial xarray

I do not know if an "effective, xarray native" solution would do exactly this in a small python wrapper, or if there is a more effective / natural way to do this in "xarray native" code.

Additional context

I have a netCDF-CF file with a time dimension that I need to regularly grow as more data become available. For internal reasons I want to keep a single netCDF file and make it grow, rather than end up with a multitude of small segments and multi file netCDF dataset.

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/7363/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed 13221727 issue

Links from other tables