issue_comments

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

Still relevant.

Issue still present in recent xarray.

Even if version information is (not yet) available, this would be very useful. Currently, when packaging xarray for downstream distributions, it is a very burdensome task to piece together which optional dependencies there are in the first place. I understand that dealing with the backlog makes this a rather tedious task, but making it a requirement going forward to add such dependency information whenever a dependency is changed or added would be required in any case.

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

I think this is still relevant.

you might be able to achieve what you want by using swap_dims

Yes, that is a nice workaround (I used another, less convenient one). But I would say that the current (lack of) functionality is a bug: the documentation talks about coordinates, not about dimensions. Moreover, it seems like something one would want to support. I do not know how complicated this would be to implement, but if reasonably feasible, I would request to keep this issue open.

Is this a dupe of #1431 ?

Yes. My example and explanation are a bit clearer, so I do not know which would be best to close.

In any case, I hope this is considered for fixing separately from some big refactoring of the MultiIndex machinery, which seems to have stalled. This seems like a corner case that might not be too big, but I may be mistaken.

I'd also like to see better support for compound types, writing them for starters. I'll collect some information here:

• In the code @tfurf linked to (_nc4_values_and_dtype), an elif needs to be added to catch structured dtypes. I think they have kind == 'V'.

• dtype.builtin can be used to detect whether we are indeed dealing with a structured type. Namely dtype.builtin must be 0.

• The structured type must fist be added to the netCDF4.Dataset using its method createCompoundType. This must be done recursively, with the deepest levels first.

• The netCDF variable is created in prepare_variable, which calls _nc4_values_and_dtype. There, via self.ds we also have access to the netCDF4 Dataset to be used for the creation of the as mentioned above. However, is self.ds really the Dataset, or some NetCDF4.Group? In any case _nc4_values_and_dtype and its use in prepare_variable needs to be refactored, because we need access to the underlying netCDF4 Dataset.

Is there anything I've missed? Can someone shed light on whether self.ds in prepare_variable can be assumed to the underlying netCDF4 Dataset?

I'm late to the discussion and may be repeating some things essentially already said, but I'd still like to add a further voice.

@shoyer said on 8 Nov 2016:

I am reluctant to add the additional complexity of groups directly into the xarray.Dataset data model. For example, how do groups get updated when you slice, aggregate or concatenate datasets? The rules for coordinates are already pretty complex.

If you prepend the paths to all the names (of dimensions, coordinate variables, and variables) and use the resulting strings as names, don't you just get a collection that would fit right in a xarray.Dataset? (Perhaps I'm just repeating what @lamorton said on 28 Mar 2017.) My feeling is that the only thing missing would be coordinate variables defined in a group closer to the root of the hierarchy, but that needs to be dealt with anyway if you want to read from netCDF4 files correctly (see my #1888). My guess would be that having groups inherit all the dimensions/coordinates of their parent that they do not redefine should be the way to go.

My use case is data from a single metmast over time. There are various instruments measuring all kinds of variables of which 10-minute statistics are recorded. I use groups to keep an overview. (I use something like /[wind|air|prec]/<device>/<variable>, /<device>/<variable>, or /<device>/<variable>/statistic as a hierarchy.) Slicing along the time axis for the whole hierarchy would make perfect sense.

@shoyer said on 30 Mar 2017:

We would possibly want to make another Dataset subclass for the sub-datasets to ensure that their variables are linked to the parent, e.g., xarray.LinkedDataset. This would enable ds['flux']['poloidal'] = subset.

But I'm also not convinced this is actually worth the trouble given how easy it is to write ds['flux', 'poloidal'].

I would prefer the former option, as it more clearly shows the hierarchical nature. If also copying the netCDF4-path-separator-convention, then ds['flux/poloidal'] is shorter than ds['flux', 'poloidal']. (Allowing Dataset or DataArray to have names that include '/' would be dangerous anyway in view of netCDF4 serialization.)