Thanks for following up! It's been a while so I don't remember; feel free to disregard what I said.

Actually would xr.DataArray.str.format work? e.g. xr.DataArray([0, 1, 2]).str would raise an error right since it's int type?

Dont understand this xarray/core/dataarray.py:4605: error: Argument 1 to "isel" of "DataArray" has incompatible type "Dict[str, Any]"; expected "Optional[Mapping[Hashable, Any]]" [arg-type] Found 1 error in 1 file (checked 140 source files)

self.isel(indexes)

Sure.

What's the reasoning for a single dimension?

On Wed, May 12, 2021, 11:34 AM Maximilian Roos @.***> wrote:

@ahuang11 https://github.com/ahuang11 we discussed this on the core team call. People are excited to merge this, and appreciate you bearing with the changes.

I suggested that we narrow this even further to one dimension and merge, so we can benefit from this now and consider additions from there. Would that be OK with you?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/5239#issuecomment-839925301, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADU7FFSS2IQ5QP7BYG4OCWLTNKUXXANCNFSM4353OVNA .

I failed to commit properly so see https://github.com/pydata/xarray/pull/5239 where I only do drop duplicates for dims

I can take a look this weekend. If narrow, could simply rollback to this commit, make minor adjustments and merge. https://github.com/pydata/xarray/pull/5089/commits/28aa96ab13db72bfa6ad8b156c2c720b49ec9a04

But I personally prefer full so it'd be nice if we could come to a consensus on how to handle it~

@ahuang11 IIUC, this is only using .stack where it needs to actually stack the array, is that correct? So a list of dims is passed (rather than non-dim coords), then it's not stacking.

I agree with @shoyer that we could do it in a single isel in the basic case. One option is to have a fast path for non-dim coords only, and call isel once with those.

Yes correct. I am not feeling well at the moment so I probably won't get to this today, but feel free to make commits!

Not sure if there's a more elegant way of implementing this.

Oh I just saw the edits with keeping the dims. I guess that would work.

I prefer drop duplicate values to be under the unique() PR; maybe could be renamed as drop_duplicate_values().

Also I think preserving existing dimensions is more powerful than flattening the dimensions.

On Sun, Apr 4, 2021, 11:01 PM Stephan Hoyer @.***> wrote:

From an API perspective, I think the name drop_duplicates() would be fine. I would guess that handling arbitrary variables in a Dataset would not be any harder than handling only coordinates?

One thing that is a little puzzling to me is how deduplicating across multiple dimensions is handled. It looks like this function preserves existing dimensions, but inserts NA is the arrays would be ragged? This seems a little strange to me. I think it could make more sense to "flatten" all dimensions in the contained variables into a new dimension when dropping duplicates.

This would require specifying the name for the new dimension(s), but perhaps that could work by switching to the de-duplicated variable name? For example, ds.drop_duplicates('valid') on the example in the PR description would result in a "valid" coordinate/dimension of length 3.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/5089#issuecomment-813168052, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADU7FFWCT2NXOR2AYNLGVQDTHEYYFANCNFSM4Z6ZAMUA .

What if we added coordinates/dims to it and it returns a stacked dimension if multiple dims? ``` def unique(da): da_stack = da.stack({'tmp_dim': da.dims}) _, index = np.unique(da_stack.values, return_index=True) return da_stack.isel({'tmp_dim': index})

da = xr.DataArray([[[0, 1, 1], [2, 3, 4], [4, 5, 6]], [[7, 8, 9], [10, 11, 12], [13, 14, 15]]], coords={'lat': [0, 1, 2], 'lon': [4, 5, 6], 'time': [7, 8]}, dims=['time', 'lat', 'lon']) unique(da) # would be da.unique() `` Then users can useda.unique().unstack()` if they like.

Thanks for the PR @ahuang11 !

I think the method could be really useful. Does anyone else have thoughts?

One important decision is whether this should operate on dimensioned coords or all coords (or even any array?). My guess would be that we could start with dimensioned coords given those are the most likely use case, and we could extent to non-dimensioned coords later.

(here's a glossary as the terms can get confusing: http://xarray.pydata.org/en/stable/terminology.html)

~~Let's start with just dims for now.~~

Okay, since I had some time, I decided to do coords too.

Not sure how to fix this: ```

xarray/core/dataset.py:7111: error: Keywords must be strings Found 1 error in 1 file (checked 138 source files) ```

Yes I want to drop the padding (remove the nans from the beginning)

On Thu, Feb 18, 2021, 3:44 PM Deepak Cherian notifications@github.com wrote:

Do you want to avoid the padding?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/4588#issuecomment-781654195, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADU7FFVBUJGELDFGWQ6TARDS7WC23ANCNFSM4TX2CAEQ .

Sure.

On Sat, Jan 2, 2021, 6:38 AM Daniel Mesejo-León notifications@github.com wrote:

Hey! Can I work on this?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/4647#issuecomment-753468760, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADU7FFVUS2TTAA6MRGX5G73SX4HSRANCNFSM4UMKLBWQ .

Maybe a simple fix would be to replace np.unique with pd.unique since it's ordered? ``` Hash table-based unique. Uniques are returned in order of appearance. This does NOT sort.

Significantly faster than numpy.unique. Includes NA values. ```

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.unique.html

ds.transpose('not_existing_dim', 'lat', 'lon', 'time', ...)

IMO this should raise an error too

I actually like it handling non_existing_dims automatically; maybe could be keyword though: ds.transpose('not_existing_dim', 'lat', 'lon', 'time', ..., errors='ignore')

I think this is fixed in the latest master.

``` import xarray as xr import numpy as np import cartopy.crs as ccrs

da = xr.tutorial.open_dataset('air_temperature')['air']

p = da.isel(time=[0, 1]).plot( transform=ccrs.PlateCarree(), col='time', subplot_kws={'projection': ccrs.Orthographic(-80, 35)} )

for ax in p.axes.flat: ax.coastlines() ax.gridlines() ```

p = da.isel(time=0).plot( transform=ccrs.PlateCarree(), subplot_kws={'projection': ccrs.Orthographic(-80, 35)} )

However if you try iterating over the axes, it will crash because p is quadmesh type not a facetgrid type so I was wondering whether it'd be good to have an keyword like force_facetgrid_type=True which uses the FacetGrid class to plot a single map so that the user can keep the same code for single axes and facet axes? ``` p = da.isel(time=0).plot( transform=ccrs.PlateCarree(), subplot_kws={'projection': ccrs.Orthographic(-80, 35)} )

for ax in p.axes.flat: ax.coastlines() ax.gridlines() ```

If I were to make a PR, where would this method reside? Would it be under dataset.py and dataarray.py? Also, would I simply call np.select inside the method, and if so, how would I add support for dask?

My minimal example atm: ``` import xarray as xr import numpy as np import hvplot.xarray

ds = xr.tutorial.open_dataset('air_temperature').isel(time=0)

ds['air_cats'] = ( ('lat', 'lon'), np.select([ds['air'].values >= 273.15, ds['air'].values < 273.15], ['above freezing', 'below freezing']) ) ds.hvplot('lon', 'lat', hover_cols=['air_cats']) ```

Neat solution! Thanks!

No, not from me at least.

Would also like to see this move forward and would be happy to help if needed.

Cool; that's good to know that I can set a title on the top left and top right side together. Thanks.

This looks awesome; it would be really nice to have built-in xarray animation support!

I'm personally more excited about 2D animations and am happy to implement that once all the common infrastructure is in.

Also wanted to point out while that's progressing along, hvplot supports 2D animations. https://hvplot.pyviz.org/ ``` import xarray as xr import hvplot.xarray import holoviews as hv hv.extension('matplotlib')

ds = xr.tutorial.open_dataset('air_temperature').isel(time=slice(0, 15)) hmap = ds.hvplot('lon', 'lat', dynamic=False).opts(fig_size=300, clim=(230, 300)) hv.save(hmap, 'anim.html', fmt='scrubber')

---

from IPython.core.display import display, HTML display(HTML('anim.html')) ```

Right, it would return a 1D numpy or dask array.

I suppose I'm used to simply typing pd.Series().unique() rather than np.unique(pd.Series()).

I use it in for loops primarily. for season in da['time.season'].unique(): vs for season in np.unique(da['time.season'].data):

Cool, didn't know the function had different kwargs from the method. Thanks!

Thanks for the quick replies! Is there interest in making this a built-in function? If so, I can help contribute a PR.

Also wondering about a way to wrap logic to that mapping.

Like below 0, replace with -1, between 0 and 10, replace with 5, and above 10, replace with 15 which is possible with three np.place statements I think, but have to think in backwards logic with ds.where().

I guess I'm thinking about more complex cases such as changing 0 -> 50, 1 -> 29, 2 -> 10

ds = xr.Dataset({'test': [0, 1, 2]}) ds.where((ds != 1) & (ds != 2), 50)

Thoughts on simplifying this?

Option 2 sounds good; I'll try putting together a pull request sometime, hopefully within a week!

Just wanted to bump this! Would make my hack neater to interpolate across the prime meridian.

I don't know if you tried this yet, but if you changed the length to 365 and keep it with non-leap year, it still errors out so I guess the root issue is with how time.dayofyear uses 366 days?

``` import xarray as xr import numpy as np import pandas as pd

d1 = xr.DataArray(np.zeros(12000), [('time', pd.date_range('2004-01-01', freq='D', periods=12000))]) d2 = xr.DataArray(np.zeros((365, 10)), {'time': pd.date_range('1979-01-01', freq='D', periods=365), 'x': ('x', np.arange(10))}, dims=['time', 'x'])

d1.groupby('time.month') * d2.groupby('time.month').mean('time') print('this works')

no work

d1.groupby('time.dayofyear') * d2.groupby('time.dayofyear').mean('time') print('this doesn\'t work') ```

Thanks for your quick response! From conda list xarray 0.9.6 <pip>

If you swap the length it errors out. ``` import xarray as xr import numpy as np import pandas as pd

d1 = xr.DataArray(np.zeros(12000), [('time', pd.date_range('1979-01-01', freq='D', periods=12000))]) d2 = xr.DataArray(np.zeros((366, 10)), {'time': pd.date_range('1979-01-01', freq='D', periods=366), 'x': ('x', np.arange(10))}, dims=['time', 'x'])

d1.groupby('time.month') - d2.groupby('time.month').mean('time') print('this works')

no work

d1.groupby('time.dayofyear') - d2.groupby('time.dayofyear').mean('time') print('this doesn\'t work') ```

``` <xarray.DataArray (time: 12000, x: 10)> array([[ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], ..., [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.], [ 0., 0., 0., ..., 0., 0., 0.]]) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 * time (time) datetime64[ns] 1979-01-01 1979-01-02 1979-01-03 ... month (time) int64 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ... this works

KeyError Traceback (most recent call last) <ipython-input-24-4c92f88d0a14> in <module>() 10 11 # no work ---> 12 d1.groupby('time.dayofyear') - d2.groupby('time.dayofyear').mean('time') 13 print('this doesn\'t work')

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/groupby.py in func(self, other) 316 g = f if not reflexive else lambda x, y: f(y, x) 317 applied = self._yield_binary_applied(g, other) --> 318 combined = self._combine(applied) 319 return combined 320 return func

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/groupby.py in _combine(self, applied, shortcut) 532 combined = self._concat_shortcut(applied, dim, positions) 533 else: --> 534 combined = concat(applied, dim) 535 combined = _maybe_reorder(combined, dim, positions) 536

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in concat(objs, dim, data_vars, coords, compat, positions, indexers, mode, concat_over) 118 raise TypeError('can only concatenate xarray Dataset and DataArray ' 119 'objects, got %s' % type(first_obj)) --> 120 return f(objs, dim, data_vars, coords, compat, positions) 121 122

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in _dataarray_concat(arrays, dim, data_vars, coords, compat, positions) 304 305 ds = _dataset_concat(datasets, dim, data_vars, coords, compat, --> 306 positions) 307 return arrays[0]._from_temp_dataset(ds, name) 308

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in _dataset_concat(datasets, dim, data_vars, coords, compat, positions) 210 datasets = align(*datasets, join='outer', copy=False, exclude=[dim]) 211 --> 212 concat_over = _calc_concat_over(datasets, dim, data_vars, coords) 213 214 def insert_result_variable(k, v):

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in _calc_concat_over(datasets, dim, data_vars, coords) 190 if dim in v.dims) 191 concat_over.update(process_subset_opt(data_vars, 'data_vars')) --> 192 concat_over.update(process_subset_opt(coords, 'coords')) 193 if dim in datasets[0]: 194 concat_over.add(dim)

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in process_subset_opt(opt, subset) 165 for ds in datasets[1:]) 166 # all nonindexes that are not the same in each dataset --> 167 concat_new = set(k for k in getattr(datasets[0], subset) 168 if k not in concat_over and differs(k)) 169 elif opt == 'all':

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in <genexpr>(.0) 166 # all nonindexes that are not the same in each dataset 167 concat_new = set(k for k in getattr(datasets[0], subset) --> 168 if k not in concat_over and differs(k)) 169 elif opt == 'all': 170 concat_new = (set(getattr(datasets[0], subset)) -

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in differs(vname) 163 v = datasets[0].variables[vname] 164 return any(not ds.variables[vname].equals(v) --> 165 for ds in datasets[1:]) 166 # all nonindexes that are not the same in each dataset 167 concat_new = set(k for k in getattr(datasets[0], subset)

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/combine.py in <genexpr>(.0) 163 v = datasets[0].variables[vname] 164 return any(not ds.variables[vname].equals(v) --> 165 for ds in datasets[1:]) 166 # all nonindexes that are not the same in each dataset 167 concat_new = set(k for k in getattr(datasets[0], subset)

/data/keeling/a/ahuang11/anaconda3/lib/python3.6/site-packages/xarray/core/utils.py in getitem(self, key) 288 289 def getitem(self, key): --> 290 return self.mapping[key] 291 292 def iter(self):

KeyError: 'x' ```