issues: 401874795

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
401874795 MDU6SXNzdWU0MDE4NzQ3OTU= 2697 read ncml files to create multifile datasets 1197350 closed 0     18 2019-01-22T17:33:08Z 2023-05-29T13:41:38Z 2023-05-29T13:41:38Z MEMBER      

This issue was motivated by a recent conversation with @jdha regarding how they are preparing inputs for regional ocean models. They are currently using ncml with netcdf-java to consolidate and homogenize diverse data sources. But this approach doesn't play well with the xarray / dask stack.

ncml is standard developed by Unidata for use with their netCDF-java library:

NcML is an XML representation of netCDF metadata, (approximately) the header information one gets from a netCDF file with the "ncdump -h" command.

In addition to describing individual netCDF files, ncml can be used to annotate modifications to netCDF metadata (attributes, dimension names, etc.) and also to aggregate multiple files into a single logical dataset. This is what such an aggregation over an existing dimension looks like in ncml:

xml <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"> <aggregation dimName="time" type="joinExisting"> <netcdf location="jan.nc" /> <netcdf location="feb.nc" /> </aggregation> </netcdf>

Obviously this maps very well to xarray's concat operation. Similar aggregations can be defined that map to merge operations.

I think it would be great if we could support the ncml spec in xarray, allowing us to write code like

python ds = xr.open_ncml('file.ncml')

This idea has been discussed before in #893. Perhaps it's time has finally come. 

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/2697/reactions",
    "total_count": 7,
    "+1": 7,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed 13221727 issue

Links from other tables