home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

14 rows where issue = 29136905 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 6

  • shoyer 6
  • jcmgray 4
  • joemcglinchy 1
  • mathause 1
  • stale[bot] 1
  • HiperMaximus 1

author_association 3

  • MEMBER 7
  • CONTRIBUTOR 4
  • NONE 3

issue 1

  • Implement DataArray.idxmax() · 14 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
598493015 https://github.com/pydata/xarray/issues/60#issuecomment-598493015 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDU5ODQ5MzAxNQ== shoyer 1217238 2020-03-13T00:43:48Z 2020-03-13T00:43:48Z MEMBER

idxmax() should return the coordinate labels, not integer positions, corresponding to the max.

e.g., xr.DataArray([1, 3, 2], dims=['x'], coords={'x': [10, 20, 30]}).argmax() should return 20 (but probably inside an xarray.DataArray)

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
598487450 https://github.com/pydata/xarray/issues/60#issuecomment-598487450 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDU5ODQ4NzQ1MA== mathause 10194086 2020-03-13T00:16:32Z 2020-03-13T00:16:32Z MEMBER

How would idxmax be different to argmax? E.g.

python import xarray as xr xr.DataArray([1, 3, 2]).argmax() Could this be closed?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
532354509 https://github.com/pydata/xarray/issues/60#issuecomment-532354509 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDUzMjM1NDUwOQ== joemcglinchy 4762214 2019-09-17T18:54:40Z 2019-09-17T18:54:40Z NONE

I got around this with some (masked) numpy operations. perhaps it is useful? I was seeing the np.argmax results on entries with all NaN evaluate to zero, which was not useful since the axis I was computing argmax across had valid entries if the result was 0 (think 0-index month, i.e., January, within a year). So I did this instead:

```

test_arr is some array with some nodata value, and is of dims [channels, rows, columns]

nodata = -32768 ma = np.ma.masked_equal(test_arr, nodata)

use np.any to get a mask of rows/columns which have all masked entries

spec_axis = 0 all_na_mask = np.any(ma, axis=spec_axis)

get the argmax across specified axis

argm = np.argmax(test_arr, axis=spec_axis) argm = np.ma.masked_less(argm, -np.inf) argm.mask = ~all_na_mask ```

big piece here is modifying the mask directly and making sure that is correct. numpy docs advise against this approach but it seems to be giving me what I want.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
527576309 https://github.com/pydata/xarray/issues/60#issuecomment-527576309 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDUyNzU3NjMwOQ== HiperMaximus 45774781 2019-09-03T18:15:10Z 2019-09-03T18:15:10Z NONE

this is still very relevant

{
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 1,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
457059732 https://github.com/pydata/xarray/issues/60#issuecomment-457059732 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDQ1NzA1OTczMg== shoyer 1217238 2019-01-24T04:05:17Z 2019-01-24T04:05:17Z MEMBER

This is still relevant

{
    "total_count": 5,
    "+1": 5,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
457052566 https://github.com/pydata/xarray/issues/60#issuecomment-457052566 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDQ1NzA1MjU2Ng== stale[bot] 26384082 2019-01-24T03:22:02Z 2019-01-24T03:22:02Z NONE

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity If this issue remains relevant, please comment here; otherwise it will be marked as closed automatically

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
276543337 https://github.com/pydata/xarray/issues/60#issuecomment-276543337 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDI3NjU0MzMzNw== shoyer 1217238 2017-02-01T01:01:27Z 2017-02-01T01:01:27Z MEMBER

Would using obj.fillna(0) not mess with argmax if for instance all the data is negative? Could fill with the min value instead?

Indeed, fillna(0) won't work right. For what I was thinking of, we could use the three argument version of where (#576) here, e.g., obj.where(allna, 0). But fillna with the min value could also work -- that's actually exactly how np.nanargmax works.

Ah yes true. I was slightly anticipating e.g. filling with NaT if the dim was time-like, though time types are not something I am familiar with.

Yes, ideally we would detect the dtype and find an appropriate fill or minimum value, similar to _maybe_promote. The argument to fillna would either be a scalar (for a DataArray)` or a dict (for a Dataset).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
276540506 https://github.com/pydata/xarray/issues/60#issuecomment-276540506 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDI3NjU0MDUwNg== jcmgray 8982598 2017-02-01T00:43:52Z 2017-02-01T00:43:52Z CONTRIBUTOR

Would using obj.fillna(0) not mess with argmax if for instance all the data is negative? Could fill with the min value instead?

Ah yes true. I was slightly anticipating e.g. filling with NaT if the dim was time-like, though time types are not something I am familiar with.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
276538303 https://github.com/pydata/xarray/issues/60#issuecomment-276538303 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDI3NjUzODMwMw== shoyer 1217238 2017-02-01T00:30:32Z 2017-02-01T00:30:32Z MEMBER

Yes, that looks pretty reasonable. Two minor concerns: - obj.fillna(-np.inf) converts all dtypes to float. It would be better to stick to obj.fillna(0), though integers can't have NaNs anyways. - I'm pretty sure .fillna(np.nan) is a no-op, filling in NaNs with NaN.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
276537615 https://github.com/pydata/xarray/issues/60#issuecomment-276537615 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDI3NjUzNzYxNQ== jcmgray 8982598 2017-02-01T00:26:24Z 2017-02-01T00:26:24Z CONTRIBUTOR

Ah yes both ways are working now, thanks. Just had a little play around with timings, and this seems like a reasonably quick way to achieve correct NaN behaviour:

```python def xr_idxmax(obj, dim): sig = ([(dim,), (dim,)], [()]) kwargs = {'axis': -1}

allna = obj.isnull().all(dim)

return apply_ufunc(gufunc_idxmax, obj.fillna(-np.inf), obj[dim],
                   signature=sig, kwargs=kwargs,
                   dask_array='allowed').where(~allna).fillna(np.nan)

```

i.e. originally replace all NaN values with -Inf, use the usual argmax, and remask the all-NaN values afterwards.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
276235524 https://github.com/pydata/xarray/issues/60#issuecomment-276235524 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDI3NjIzNTUyNA== shoyer 1217238 2017-01-31T00:21:35Z 2017-01-31T00:21:35Z MEMBER

take is numpy function that only handles scalar or 1d arguments: https://docs.scipy.org/doc/numpy/reference/generated/numpy.take.html#numpy.take

I just merged #1237 -- see if it works with that.

multiple maxes is presumably fine as long as user is aware it just takes the first.

Yeah, that's not a problem here, only for the where based implementation.

However, nanargmax is probably the actual desired function here, but looks like it will raise on all-nan slices. Would dropping these and then re-aligning be too much overhead?

This behavior for nanargmax is unfortunate. The "right" behavior for xarray is probably to use NaN or NaT to mark the index in such locations but numpy makes this tricky. I think this could be achieved, though, with some mix of where, isnull and other vectorized operations. Basically you need to replace all NaN slices with some placeholder value before calculating nanargmax, and then use the locations of all NaN slices again to replace the results of nanargmax with the appropriate fill value.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
276232678 https://github.com/pydata/xarray/issues/60#issuecomment-276232678 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDI3NjIzMjY3OA== jcmgray 8982598 2017-01-31T00:06:02Z 2017-01-31T00:06:02Z CONTRIBUTOR

So I thought take was just the functional equivalent of fancy indexing - I spotted it in the dask api and assumed it would work but having tried it does indeed just raise a 'not implemented error'. Just as a note, with the map_blocks approach above take is working for some cases where x[inds, ] is not -- related to #1237?

Regarding edge cases: multiple maxes is presumably fine as long as user is aware it just takes the first. However, nanargmax is probably the actual desired function here, but looks like it will raise on all-nan slices. Would dropping these and then re-aligning be too much overhead?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
275960531 https://github.com/pydata/xarray/issues/60#issuecomment-275960531 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDI3NTk2MDUzMQ== shoyer 1217238 2017-01-30T00:54:09Z 2017-01-30T17:30:48Z MEMBER

See http://stackoverflow.com/questions/40179593/how-to-get-the-coordinates-of-the-maximum-in-xarray for examples of how to do this with the current version of xarray. @MaximilianR's answer using where is pretty clean, but maybe not the most efficient or exactly what we want. (I think it breaks in a few edge cases, such as if the max value appears multiple times, or the array is all NaN.)

@jcmgray Your proposal looks pretty close to me. But to handle higher dimension arrays, instead of take(y, indx), I think you need to NumPy style fancy indexing, y[indx,]. That doesn't work with dask, so you'll need to write a function function that uses dask.array.map_blocks when necessary.

I think something like the following would work: ```python def _index_from_1d_array(array, indices): return array[indices,]

def gufunc_idxmax(x, y, axis=None): # note: y is always a numpy.ndarray, because IndexVariable objects # always have their data loaded into memory indx = argmax(x, axis) func = functools.partial(_index_from_1d_array, y)

if isinstance(array, dask_array_type):
    import dask.array as da
    return da.map_blocks(func, indx, dtype=indx.dtype)
else:
    return func(indx)

```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905
275778443 https://github.com/pydata/xarray/issues/60#issuecomment-275778443 https://api.github.com/repos/pydata/xarray/issues/60 MDEyOklzc3VlQ29tbWVudDI3NTc3ODQ0Mw== jcmgray 8982598 2017-01-27T21:24:31Z 2017-01-27T21:24:31Z CONTRIBUTOR

Just as I am interested in having this functionality, and the new apply_ufunc is available, would something along these lines suffice?

```python from wherever import argmax, take # numpy or dask

def gufunc_idxmax(x, y, axis=None): indx = argmax(x, axis) return take(y, indx)

def idxmax(obj, dim): sig = ([(dim,), (dim,)], [()]) kwargs = {'axis': -1} return apply_ufunc(gufunc_idxmin, obj, obj[dim], signature=sig, kwargs=kwargs, dask_array='allowed') ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement DataArray.idxmax() 29136905

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 960.428ms · About: xarray-datasette