issue_comments

if we can get the strategies from https://github.com/pydata/xarray/pull/6908 to shrink well

I think they already do shrink well. Each of them has corresponding unit tests, and none of those tests fail due to hypothesis timeouts.

I also agree that both the strategies as well as the base classes should eventually live in xarray.testing, but I'd probably keep them in xarray.tests for now (just so we can experiment a bit more).

So I may actually have overexcitedly already jumped the gun and made a PR to move strategies to xarray.testing.strategies last night :sweat_smile: #6908 We might just want to wait to merge this before merging that though anyway.

Note, however, that it's probably best not to separate 2 and 3 because there might be issues with the base class design we don't notice without trying to actually use them.

Good point.

Q: Shouldn't the base classes live in xarray.testing rather than xarray.tests?

Another Q on a similar note: Are we planning to eventually publicly expose the (awesome btw) strategies that you've built here @keewis ? They could be very useful for testing other parts of xarray.

We could also make this PR much more incremental by splitting it into 2, or even 3 separate PRs: 1) strategies, to live somewhere like xarray.testing.strategies and eventually be made public 2) duck array base classes to inherit from, to live somewhere like xarray.testing.duckarrays 3) specific tests for pint/sparse, to live in our own test suite for now but moved out eventually.

The advantage of that would be that (1) & (2) can move forwards without requiring all the tests in (3) to pass.

Q: Shouldn't the base classes live in xarray.testing rather than xarray.tests?

Then the test_units.py and test_sparse.py live in xarray.tests (until they eventually get moved to another repo), but import the base classes from xarray.tests?

See https://github.com/pydata/xarray/issues/6894 for general discussion of the general plans

I'm watching the progress of this PR with bated breath! I literally want to be able to test 3 different array libraries right now: pint, cubed, and awkward. :exploding_head:

Thinking about the complexity of testing like this in general I have a bunch of follow-up questions for future PRs though, e.g:

1) Shouldn't we start with some very simple tests that first check if the correct properties are defined on the wrapped array class, i.e. shape, dtype etc? 2) What's the easiest way to expose a framework of tests like this to the user, so they can import them and run them on any duck array type? 3) Docs on how to use these public tests? 4) Can we possibly allow the user to run tests on data they created? I'm thinking in particular of creating ragged awkward arrays and then testing reduce etc. on them.

Should I therefore make a separate issue to specifically track how we test (and expose test frameworks for) duck array wrapping? (So we can discuss these questions there?)

Can we xfail these new tests and merge for now? As long as tests/test_sparse.py still passes, there are no issues, correct?

Yeah we now have another array type to consider testing, so I'm also in favour of merging now with passing tests for pint, and un-xfailing tests for other array types (i.e. sparse) in a later PR.