issue_comments

+1 for a check_dim_order option to .equals, assert_equal that can be disabled. (Ideally I think the default would be not to check dim order, but that ship has sailed now).

Or failing that, it would at least be nice to have xarray.testing.assert_equal_modulo_dim_order etc. When writing tests I usually don't care about dimension order and it's frustrating to have to manually do e.g. xarray.testing.assert_allclose(a, b.transpose(a.dims)).

As pointed out, most of the xarray API is dimension-order-invariant and so it's odd to have no supported way to do comparisons in a dimension-order-invariant way.

@jbusecke I agree with your point of view that "xarray-style" comparison is more practical in a scientific workflow. Especially if dimension order is irrelevant for most (all?) xarray operations, ignoring the order for assert_allclose / assert_equal too makes sense and is consistent.

However, it might be also dangerous / harmful if the workflow includes data conversion between labeled vs. unlabelled formats. There's a risk of checking for equality with xarray, then later converting to numpy and assuming that arrays are equal without feeling the need to check again. If dimension sizes are the same this might lead to very subtle bugs.

Since it is easy to ignore or forget about default values, having a check_dim_order option that defaults to True is probably safer IMHO, although slightly less convenient. No strong views, though.

@dcherian I like your idea but I'm not sure what's best between your code snippet and checking equality of aligned dimensions datasets only if non-dimension-aligned are not equal.

what the default comparison behaviour should be

I don't think we can change this because it's very backwards incompatible and affects tests in downstream packages.

But :+1: to adding a flag allowing users to opt out of dimension order checking.

I think we definitely need a flag, because we have two conflicting use cases: 1) Developers who use asserts in test suites and might well care about differences like python xr.testing.assert_allclose(ds1.data, ds2.data) # ok np.testing.assert_allclose(ds1.data.values, ds2.data.values) # fails 2) Users for whom xarray allows to forget about dimension order, and would expect transposed data to be equal.

The easiest way to cover both is to have an option to switch between them.

In my opinion the only question is what the default comparison behaviour should be. I like the idea of adding a check_dim_order kwarg to assert_equal, assert_allclose, and assert_identical, but have the flag default to False for the first two functions and True for the last one.

Strictly speaking, the values are different I guess. However I think this error would be clearer if it said that the dimension order was different but the values are equal once the dimensions are transposed.

I guess this comes down a bit to a philosophical question related to @benbovy s comment above. You can either make this operation be similar to the numpy equivalent (with some more xarray specific checks) or it can check whether the values at a certain combo of labels are the same/close.

The latter would be the way I think about data in xarray as a user. To me the removal of axis logic (via labels) is one of the biggest draws for myself, but importantly I also pitch this as one of the big reasons to switch to xarray for beginners.

I would argue that a 'strict' (numpy style) comparision is less practical in a scientific workflow and we do have the numpy implementation to achieve that functionality. So I would ultimately argue that xarray should check closeness between values at certain label positions by default.

However, this might be very opinionated on my end, and a better error message would already be a massive improvement.

I have a related question

``` python import xarray as xr

da = xr.DataArray([[1, 1, 1], [2, 2, 2]], dims=("x", "y"))

xr.testing.assert_identical(da, da.transpose()) AssertionError: Left and right DataArray objects are not identical Differing dimensions: (x: 2, y: 3) != (y: 3, x: 2) Differing values: L array([[1, 1, 1], [2, 2, 2]]) R array([[1, 2], [1, 2], [1, 2]]) ```

Strictly speaking, the values are different I guess. However I think this error would be clearer if it said that the dimension order was different but the values are equal once the dimensions are transposed.

I.e. we could

python if set(a.dims) == set(b.dims): a = a.transpose(b.dims) # check values and raise if actually different else: # current behaviour

Is this a good idea?

What about a check_dim_order option? Also, it would be useful if information about non-matching dimension order was shown more explicitly in the assertion error message.

This sounds good to me. We should also have a check_attrs kwarg since that's another thing that only identical checks.

Is there any operation in xarray where dimension order matters? If yes, I'm not sure if it's a good idea to allow transposed dimension pass assert_allclose or assert_equal. But even otherwise, I'd find it a bit weird for a "numpy equivalent" function to have:

python xr.testing.assert_allclose(ds1.data, ds2.data) # ok np.testing.assert_allclose(ds1.data.values, ds2.data.values) # fails

What about a check_dim_order option? Also, it would be useful if information about non-matching dimension order was shown more explicitly in the assertion error message.

I agree this is another station on the journey between "byte identical" and "practically similar", which we've discovered over the past couple of years... :)

I would lean towards having it fail for assert_identical, with no view on all_close; definitely open to it coercing transpose.