issues

Part of #1846: add a property-based test for reading & writing netCDF4 files.

This is the first time I've played with Hypothesis, but it seems to be working - e.g. I got an error with float16, and the netCDF docs show that 16-bit floats are not a supported data type.

However:

• This currently only tests a dataset with a single variable - it could be extended to multiple variables if that's useful.
• It looks like netCDF4 should support unicode characters, but it failed when I didn't have max_codepoint=255 in there. I don't know if that's an expected limitation I'm not aware of, or a bug somewhere. But I thought I'd make the test pass for now.

It's documented that when you .unstack() a DataArray, "New dimensions will be added at the end."

I'm using .unstack() kind of like np.reshape(), to split one dimension into two. From this perspective, it's surprising that new dimensions go at the end, rather than in place of the old dimension. This also means that either the new array is no longer C-contiguous, or the .unstack() operation has had to copy all the data to rearrange it. It appears I'm not the only one to think this: the code sample in #2419 (about adding a reshape method) does the same workaround that I do: unstack and then carefully reorder the dimensions.

First, I'd like to understand: why the end? It seems like there's an obvious place to put unstacked dimensions (ab, c -> a, b, c), but presumably there's some reason it doesn't work that way.

Then, is there anything one can do with xarray today to 'unstack' a dimension while keeping the order, short of using .transpose() afterwards to recreate the desired order? And if not, is there any potential to add a feature for that?

I guess this might touch on missing data: unstack() can create and handle 'gaps', whereas reshape() can't. If so, is there some more limited operation that could only unstack a perfectly regular multiindex - where it's guaranteed to separate into 2 dimensions without creating gaps?

Part of #1846: test roundtripping between xarray DataArray & Dataset and pandas Series & DataFrame.

I haven't particularly tried to hunt down corner cases (e.g. dataframes with 0 columns), in favour of adding tests that currently pass. But these tests probably form a useful platform if you do want to ensure corner cases like that behave nicely - just modify the limits and see what fails.

This isn't entirely finished, but I need to stop working on it for a bit, and I think enough of it is ready to be reviewed. The core code is passing its tests; the remaining failures are all in talking to the Scipy and netCDF4 backends. I also have PRs open against Scipy (scipy/scipy#3617) and netCDF4 (Unidata/netcdf4-python#252) to fix bugs I've encountered there.

Particular issues that came up: - There were quite a few circular imports. For now, I've fudged these to work rather than trying to reorganise the code. - isinstance(x, int) doesn't reliably catch numpy integer types - see e.g. numpy/numpy#2951. I changed several such cases to isinstance(x, (int, np.integer)).