html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/6894#issuecomment-1255644395,https://api.github.com/repos/pydata/xarray/issues/6894,1255644395,IC_kwDOAMm_X85K15zr,35968931,2022-09-22T23:06:21Z,2022-09-22T23:06:21Z,MEMBER,"Looks like these 

> https://data-apis.org/array-api-tests/.

use these

> [experimental array api strategies](https://hypothesis.readthedocs.io/en/latest/numpy.html#array-api)

>  Would that be useful here?

I think they are complementary. In **theory** if xarray supports the array API standard and a library passes all the data array API tests, then it **should** also pass all of xarray's tests (rendering the latter uneccessary). But in practice I think the tests here would still be useful, if only for the possible case of libraries that don't fully meet the API standard yet would still work fine in xarray.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1332231863
https://github.com/pydata/xarray/issues/6894#issuecomment-1255626863,https://api.github.com/repos/pydata/xarray/issues/6894,1255626863,IC_kwDOAMm_X85K11hv,2443309,2022-09-22T22:35:58Z,2022-09-22T22:35:58Z,MEMBER,@asmeurer recently pointed me to https://data-apis.org/array-api-tests/. Would that be useful here?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1332231863
https://github.com/pydata/xarray/issues/6894#issuecomment-1216451559,https://api.github.com/repos/pydata/xarray/issues/6894,1216451559,IC_kwDOAMm_X85IgZPn,14808389,2022-08-16T10:25:43Z,2022-08-16T10:25:43Z,MEMBER,there's also the [experimental array api strategies](https://hypothesis.readthedocs.io/en/latest/numpy.html#array-api) built into hypothesis,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1332231863
https://github.com/pydata/xarray/issues/6894#issuecomment-1210187154,https://api.github.com/repos/pydata/xarray/issues/6894,1210187154,IC_kwDOAMm_X85IIf2S,35968931,2022-08-10T05:42:34Z,2022-08-10T05:42:34Z,MEMBER,Another thing that might be useful is the [hypothesis strategies](https://github.com/data-apis/array-api-tests/blob/master/array_api_tests/hypothesis_helpers.py) in the test suite for the array API consortium standard (cc @keewis).,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1332231863
https://github.com/pydata/xarray/issues/6894#issuecomment-1209758257,https://api.github.com/repos/pydata/xarray/issues/6894,1209758257,IC_kwDOAMm_X85IG3Ix,35968931,2022-08-09T18:53:58Z,2022-08-09T18:53:58Z,MEMBER,"> Typing duck array is also a little challenging I find

Thanks @Illviljan - I was literally just thinking about that [here](https://github.com/pydata/xarray/pull/6903/files#r941697640).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1332231863
https://github.com/pydata/xarray/issues/6894#issuecomment-1209756650,https://api.github.com/repos/pydata/xarray/issues/6894,1209756650,IC_kwDOAMm_X85IG2vq,14371165,2022-08-09T18:52:13Z,2022-08-09T18:52:13Z,MEMBER,"Typing duck array is also a little challenging I find, we pretty much only do `Any` at the moment. I found some nice references and discussions that might be interesting for this:
https://github.com/pmeier/array-protocol
https://github.com/data-apis/array-api/issues/229","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1332231863
https://github.com/pydata/xarray/issues/6894#issuecomment-1209519085,https://api.github.com/repos/pydata/xarray/issues/6894,1209519085,IC_kwDOAMm_X85IF8vt,35968931,2022-08-09T15:17:12Z,2022-08-09T15:17:12Z,MEMBER,"> you should already be able to specify a hypothesis strategy to create e.g. a random awkward array

Sounds good!

> or maybe it should be just `check`?

Yes just `check` probably.

> However, maybe we should just not use parametrize but instead define separate functions for each reduce operation?

But then the user writing the test code would have to write one of their own tests per xarray method wouldn't they? I think we should avoid putting that much work on them if we can. I think your current approach seems fine so far...

> the pint tests use old versions of the conversion functions from pint-xarray

That's basically technical debt, so we should make a point to remove them from xarray eventually.

> the tests for pint seem to increase the total test coverage of xarray https://github.com/pydata/xarray/pull/5692#issuecomment-1040002844. I guess that just means we'd have to improve the rest of the testsuite?

So long as @benbovy (or someone) writes new tests to cover the bugs that were revealed then this is fine.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1332231863
https://github.com/pydata/xarray/issues/6894#issuecomment-1209144356,https://api.github.com/repos/pydata/xarray/issues/6894,1209144356,IC_kwDOAMm_X85IEhQk,14808389,2022-08-09T09:33:07Z,2022-08-09T09:33:07Z,MEMBER,"with the implementation in #4972 you should already be able to specify a hypothesis strategy to create e.g. a random awkward array. Same with `dask` or other parallel computing frameworks: if you can construct a hypothesis strategy for them the testing framework should be able to use that. `check_reduce` (or maybe it should be just `check`?) should allow customizing the comparison (or actually, that's the entire test code at the moment), so putting `compute` (or `todense` / `get`) calls should be easy.

For setup and teardown I think we could use `pytest` fixtures (and apply them automatically to each function). However, maybe we should just not use parametrize but instead define separate functions for each reduce operation? Then it would be possible to override that manually. As far as I remember I chose not to do that because tests that only delegate to `super().test_function()` just are not great design – if we can think of a way to do that while avoiding those kinds of test redefinitions I'd be happy with that (and then we could get rid of the `apply_marks` function which is a ugly hack of `pytest` internals).

I agree that moving the array library tests to dedicated repositories makes a lot sense (for example, the `pint` tests use old versions of the conversion functions from `pint-xarray`), but note that at the moment the tests for `pint` seem to increase the total test coverage of `xarray` [a bit](https://github.com/pydata/xarray/pull/5692#issuecomment-1040002844). I guess that just means we'd have to improve the rest of the testsuite?","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1332231863