issue_comments

Note that current CF-1.9 now specifies how to read coordinates from any group in file (see spec).

If it is not already one of the goals of the new DataTree structure #4665 it would probably best be added. cc @TomNicholas @aurghs

I think it would be totally reasonable and consistent with the netcdf data model to read dimension coordinates from parent groups when they are missing on the requested group.

On Fri, Apr 10, 2020 at 10:39 AM Lucas Morton notifications@github.com wrote:

I'm currently working around this by loading the root group & the branch group with two separate calls and then merging the resulting datasets. It's ugly b/c I have to manually associate the 'phony_dim_x' dimensions from one group with the other.

Maybe I can find the time during quarantine to make an attempt at resolving #1092 https://github.com/pydata/xarray/issues/1092, which I think would facilitate resolving this issue as well.

Another option would be to allow the group kwarg to be a tuple of group names, and load_dataset could yield a (flat) Dataset including both the root and the branch variables.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/1982#issuecomment-612137754, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJJFVR25HJVJOFNSKOV6Q3RL5KTRANCNFSM4EUZSFYA .

I'm currently working around this by loading the root group & the branch group with two separate calls and then merging the resulting datasets. It's ugly b/c I have to manually associate the 'phony_dim_x' dimensions from one group with the other.

Maybe I can find the time during quarantine to make an attempt at resolving #1092, which I think would facilitate resolving this issue as well.

Another option would be to allow the group kwarg to be a tuple of group names, and load_dataset could yield a (flat) Dataset including both the root and the branch variables.

I think it could logically make sense to recursively check parent groups for coordinates referenced by variables in the opened group.

From an implementation perspective, this might be a little tricky. The current backend interface isn't really setup for this. Currently, we open all variables, and do all CF convention decoding afterwards. This would require going back to the dataset to open more variables.

How about searching in the parent group until the root group is reached? As far as I can see it, this cannot cause any conflict. It would be really awesome to have this functionality. I really like to idea of nicely organizing NetCDF files while still having the full functionality of xarray at hand.

On 12 Nov 2018, at 18:55, Stephan Hoyer notifications@github.com wrote:

When reading a sub group from a netCDF file with dimensions defined in the root group, the dimensions are not read from the root group. This contradicts the netCDF documentation, which states that dimensions are scoped such that they can be seen by all sub groups.

To be clear, xarray does properly read "dimensions" in parent groups. (This is actually ensured by libraries like netCDF4-Python.)

What xarray doesn't do is read "coordinates" from parent groups. As far as I can tell, this isn't part of either the netCDF4 data model or CF conventions. This might be a usability improvement but the right way to do isn't dictated by the specs.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/1982#issuecomment-437972944, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGu6bZgyAIR4mLV-kuxGg4uF9r6T41Pks5uubYUgaJpZM4SmZFw.

When reading a sub group from a netCDF file with dimensions defined in the root group, the dimensions are not read from the root group. This contradicts the netCDF documentation, which states that dimensions are scoped such that they can be seen by all sub groups.

To be clear, xarray does properly read "dimensions" in parent groups. (This is actually ensured by libraries like netCDF4-Python.)

What xarray doesn't do is read "coordinates" from parent groups. As far as I can tell, this isn't part of either the netCDF4 data model or CF conventions. This might be a usability improvement but the right way to do isn't dictated by the specs.

The problem occurs with plotting as well. See the StackOverflow question that I posted a few days ago.

https://stackoverflow.com/questions/53196437/missing-axis-values-in-plotting-of-netcdf-variable-in-group-with-xarray

but it should respect the scope of netCDF dimensions

Just so we're all using the same terminology, you are actually referring to "coordinates", not dimensions. In your example above, tss is getting the correct dimension (time2) but the corresponding coordinate (time2) is not loaded. In netCDF/xarray dimensions are just names for individual axes.

The fundamental issue is that we don't have any machinery in xarray to look outside of a single netCDF group when opening a dataset. That is the common piece to #1092.

It looks to me like #1092 is about a Dataset-like object which can contain groups and sub-groups. Here we have a simpler issue: the Dataset can still be a flat object containing a single group, but it should respect the scope of netCDF dimensions. This means that any dimensions which are mentioned but not visible in the group being written should be searched for and copied (linked?) from a parent group, up to and including the root group if the dimensions reside there.

This seems to be a duplicate of #1092. The short answer is that we don't have a first class solution to working with groups in netCDF files.