html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/3701#issuecomment-575737878,https://api.github.com/repos/pydata/xarray/issues/3701,575737878,MDEyOklzc3VlQ29tbWVudDU3NTczNzg3OA==,14808389,2020-01-17T18:17:59Z,2020-01-17T18:18:08Z,MEMBER,here's the `dask` issue about `np.percentile`'s missing `axis` parameter: dask/dask#2824,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,551344538
https://github.com/pydata/xarray/issues/3701#issuecomment-575736943,https://api.github.com/repos/pydata/xarray/issues/3701,575736943,MDEyOklzc3VlQ29tbWVudDU3NTczNjk0Mw==,2448579,2020-01-17T18:15:26Z,2020-01-17T18:15:26Z,MEMBER,"yes. you'll need to have 1 chunk along the dimension being reduced (here 'time').
The reason is that there is no approximate quantile algorithm implemented in dask. So this implementation is just calling `np.nanpercentile` on each block of the array. If you're reducing along time, each block must have the entire time axis for this to work.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,551344538
https://github.com/pydata/xarray/issues/3701#issuecomment-575671803,https://api.github.com/repos/pydata/xarray/issues/3701,575671803,MDEyOklzc3VlQ29tbWVudDU3NTY3MTgwMw==,14808389,2020-01-17T15:27:39Z,2020-01-17T15:27:39Z,MEMBER,"that would work if we would be working with variables, but `Dataset.quantile` does a lot more that I didn't want to copy.
> We should issue a release soon though.
I agree","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,551344538
https://github.com/pydata/xarray/issues/3701#issuecomment-575670529,https://api.github.com/repos/pydata/xarray/issues/3701,575670529,MDEyOklzc3VlQ29tbWVudDU3NTY3MDUyOQ==,2448579,2020-01-17T15:24:48Z,2020-01-17T15:24:48Z,MEMBER,"> seems to make quantile work, but it might be a bad idea to monkeypatch. T
Personally I would add this `quantile` function and then do `quantile(ds, q=0.99, dim=""SOMETHING"")`. We should issue a release soon though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,551344538
https://github.com/pydata/xarray/issues/3701#issuecomment-575599184,https://api.github.com/repos/pydata/xarray/issues/3701,575599184,MDEyOklzc3VlQ29tbWVudDU3NTU5OTE4NA==,14808389,2020-01-17T12:01:18Z,2020-01-17T12:23:57Z,MEMBER,"sorry, my mistake. I've been testing on master which has #3559 merged and thus works with dask arrays. Unless someone has a better idea, I'd suggest copying that approach for now: using `apply_ufunc` to apply `np.nanpercentile`. ~~Ideally, that would change the `quantile` call to~~
```python
percentiles = quantile(ds, q=percentile / 100, dim=""time"")
```
~~I'll try to write a working `quantile` function.~~
Edit: see below, with that you should be able to use
```python
ds.quantile(...)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,551344538
https://github.com/pydata/xarray/issues/3701#issuecomment-575605008,https://api.github.com/repos/pydata/xarray/issues/3701,575605008,MDEyOklzc3VlQ29tbWVudDU3NTYwNTAwOA==,14808389,2020-01-17T12:21:30Z,2020-01-17T12:22:25Z,MEMBER,"This
```python
def quantile(self, q, dim=None, interpolation=""linear"", keep_attrs=None):
from xarray.core.computation import apply_ufunc
from xarray.core import utils
scalar = utils.is_scalar(q)
q = np.atleast_1d(np.asarray(q, dtype=np.float64))
if dim is None:
dim = self.dims
if utils.is_scalar(dim):
dim = [dim]
def _wrapper(npa, **kwargs):
# move quantile axis to end. required for apply_ufunc
return np.moveaxis(np.nanpercentile(npa, **kwargs), 0, -1)
axis = np.arange(-1, -1 * len(dim) - 1, -1)
result = apply_ufunc(
_wrapper,
self,
input_core_dims=[dim],
exclude_dims=set(dim),
output_core_dims=[[""quantile""]],
output_dtypes=[np.float64],
output_sizes={""quantile"": len(q)},
dask=""parallelized"",
kwargs={""q"": q * 100, ""axis"": axis, ""interpolation"": interpolation},
)
# for backward compatibility
result = result.transpose(""quantile"", ...)
if scalar:
result = result.squeeze(""quantile"")
if keep_attrs:
result.attrs = self._attrs
return result
xr.Variable.quantile = quantile
```
seems to make `quantile` work, but it might be a bad idea to monkeypatch. Thoughts, @dcherian?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,551344538
https://github.com/pydata/xarray/issues/3701#issuecomment-575593711,https://api.github.com/repos/pydata/xarray/issues/3701,575593711,MDEyOklzc3VlQ29tbWVudDU3NTU5MzcxMQ==,14808389,2020-01-17T11:42:38Z,2020-01-17T11:49:51Z,MEMBER,"you should be able to just use `quantile`. I called `compute` to demonstrate it returns the same values as `np.percentile`, but you don't need it with `quantile`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,551344538
https://github.com/pydata/xarray/issues/3701#issuecomment-575590320,https://api.github.com/repos/pydata/xarray/issues/3701,575590320,MDEyOklzc3VlQ29tbWVudDU3NTU5MDMyMA==,14808389,2020-01-17T11:31:27Z,2020-01-17T11:31:27Z,MEMBER,"the reason for this is that the `percentile` implementation of `dask` arrays does not support `axis`. I think you can achieve the same thing by either using `compute()` (which is probably not desirable) or by using `quantile`:
```python
In [13]: # Get the percentiles at each lat/lon:
...: ds = ds.compute()
...: percentiles = ds.reduce(np.percentile, q=percentile, dim='time')
...: quantiles = ds.quantile(q=percentile / 100, dim=""time"")
...: xr.testing.assert_identical(percentiles.drop_vars(""height""), quantiles.drop_vars(""quantile""))
```
the only difference is that the unused coordinate `height` disappears and a `quantile` dimension is added.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,551344538