issues: 1216517115

What happened?

When loading a NetCDF4 dataset from a file (at least using the 'h5netcdf' engine) I get a xarray.Dataset where each DataArray wraps a numpy.ndarray with the OWNDATA flag set to False. This is counter-intuitive as the high-level user has not knowingly run anything that would create a view/slice to get a second array sharing memory with the "first" array.

This is of course a rather minor issue, but it annoyed me when I was making tools to keep track of which arrays in my dataset were using much RAM, because I had made an option to only show memory usage for the primary/base array, not for possible extra array views that reuse the same memory. With this option enabled, the reported issue however prevents me from getting useful information (just shows nearly zero memory usage) when inspecting a Dataset loaded from NetCDF4-file instead of a "freshly" created or deep-copied Dataset.

What did you expect to happen?

I would prefer the OWNDATA flag to be True as they are from the lower-level h5py-reading of the HDF5-file. After some debugging this means avoiding to do things like array = array[:, ...] at various places in the multiple layers of wrappers that are involved in the dataset-loading, as that would create and return a new ndarray-instance "sharing" (not owning) memory with the original (while no user-accessible reference to the original ndarray that technically "owns" the memory seems to be retained).

See the minimal code example below. It and my patch were made using xarray 0.20.1 with h5netcdf 0.12.0 but looking it looks like the relevant parts of xarray/core/indexing.py are still the same.

Minimal Complete Verifiable Example

Python import xarray as xr import numpy as np ds = xr.Dataset() ds['stuff'] = xr.DataArray(np.random.randn(2), dims='x') path = 'demo.nc' ds.to_netcdf(path, engine='h5netcdf', format='netCDF4', invalid_netcdf=True) ds2 = xr.load_dataset(path, engine='h5netcdf') print(ds2['stuff'].values.flags['OWNDATA']) # initially False, True after patching

Relevant log output

No response

Anything else we need to know?

I patched two parts of xarray/core/indexing.py to solves the issue: xarray core indexing.diff.txt

Testing in other situations will of course be needed to make sure this doesn't disturb anything else, but I hope the general idea would be useful even if some more condition might be needed for when to take the shortcut of returning the original numpy.ndarray rather than a view of it.

Environment