issue_comments

@spencerkclark yeah I was using a different platform, updated my xarray version and this works now, thanks!

This seems like an xarray version issue; the other argument was not added to where until version 0.10.0. Are you running this script on a different platform than before? I'm surprised your original code, https://github.com/pydata/xarray/issues/2450#issuecomment-426346137, didn't raise the same error.

@spencerkclark ```python Traceback (most recent call last): File "/var/spool/torque/mom_priv/jobs/63347.rdf-xcat.SC", line 22, in <module> a = ml50_shear[0].where(ml50_shear[0] >= 11, other = 0)

NotImplementedError: The optional argument 'other' has not yet been implemented ```

Gonna try with just the first of the four

👍 let me know how it goes.

@spencerkclark Oh you're right, I didn't think about that - it returns 4

Gonna try with just the first of the four

Does complete_shear happen to return more than one DataArray?

@spencerkclark ah sorry I missed line 5 - I'm not looking to get the mean, I want to make sure the unusual values are still represented. That might be why mine went wrong, it generally only works when there's just one time. Anyway, here's what I tried: ```python import xarray as xr from function_codes import sfcwinds, complete_shear, winds import os pathway = #my pathway os.chdir(pathway)

sfc_winds = sfcwinds("sfc") ml50_winds = winds("ml50") ml50_shear = complete_shear(sfc_winds, ml50_winds)

a = ml50_shear.where(ml50_shear >= 11, other = 0) b = a.where(a == 0, other = 1) pa = (b.sum('time') / b.sizes['time']) * 100 print(pa) ```

@am-thyst could you show all the code that you used to try it out? If I run all the lines (from 1-8 in https://github.com/pydata/xarray/issues/2450#issuecomment-426340226 and https://github.com/pydata/xarray/issues/2450#issuecomment-426353764) in my IPython console things work for me.

@spencerkclark following your method: python AttributeError: 'tuple' object has no attribute 'where'

@spencerkclark Thanks a lot, I'll give it a try 👍

@am-thyst thanks for the extra details -- try out my method -- I think it does the same thing. Well perhaps you'll just need to multiply the result by 100 at the end and use >= instead of > for the threshold condition :)

Your approach can also be vectorized in such a way that you don't need to manually iterate over the individual times: ``` In [6]: b = da.where(da >= threshold, other=0)

In [7]: c = b.where(b == 0, other=1)

In [8]: (c.sum('time') / c.sizes['time']) * 100 Out[8]: <xarray.DataArray (x: 2, y: 3)> array([[ 25., 50., 25.], [ 25., 50., 50.]]) Dimensions without coordinates: x, y `` Effectivelyda >= threshold` in my earlier example is shorthand for lines 6 and 7 from the example above (and taking the mean is the same as summing over the dimension and dividing by the number of elements along that dimension).

@spencerkclark python a = da.loc[date[0]] #the initial time, obviously this part doesn't work but this is the methodology b = a.where(a >= threshold, other = 0) #keeps values that satisfy the threshold, sets the rest to 0 c = b.where(b == 0, other = 1) #sets all values that satisfy the threshold to 1 new_array = zero_array + c #zero_array is just an array of the same shape without time, set to 0 runs = runs + 1 Loops through this whole thing for every time (changing the loc to date[1], date[2], etc.), then: python percentage = (new_array / runs) * 100

This is not my complete code but you should get the idea, as I said it works fine with only 1 time per day

I'm not sure if I'm understanding the calculation correctly; is something like the following what you are looking for? ``` In [1]: import numpy as np; import pandas as pd; import xarray as xr

In [2]: times = pd.date_range('1979', periods=4, freq='12H')

In [3]: da = xr.DataArray(np.random.rand(4, 2, 3), coords={'time': times}, dims=['time', 'x', 'y'])

In [4]: threshold = 0.5

In [5]: (da > threshold).mean('time') Out[5]: <xarray.DataArray (x: 2, y: 3)> array([[ 0.25, 0.5 , 0.25], [ 0.25, 0.5 , 0.5 ]]) Dimensions without coordinates: x, y ```

@spencerkclark I'm creating a 2D array of the percentage of time each lon/lat coordinate exceeds a defined threshold. I did this fine before, but I was only using 1 time from each day

@am-thyst could you provide more details on what sort of operation you are trying to do with each time slice? Typically this kind of process can be better handled in a vectorized way (i.e. without manually iterating over the times).

@spencerkclark thanks for this, using '1979-01-01T00' works which is a relief. I am trying to use the method of date[number] because I have in excess of 4,000 times to go through. Is there a way to get it to work? I figured adding the %H would work, but obviously not!

Otherwise, at least I know I can revise my code to state the datetime explicitly.

Thanks @am-thyst, this happens to be expected behavior. For a dimension indexed by a DatetimeIndex (e.g. your 'time' dimension), label-based indexing with strings uses so-called "partial string indexing". This behavior in xarray is inherited from pandas. Essentially what it means is that if you provide a string to a label-based indexer (like .loc), then it will return values from all times that would begin with the string you provided if expressed in string form. So, in your example, if you provide only the year, month, and day (e.g. '1979-01-01') it will return values from '1979-01-01' and 1979-01-01T12:00:00, because both string representations start with '1979-01-01'.

As you inferred, if you add detail to the string you provide, you can refine the selection. That said, you need to be careful of the format you use (I think the reason you get a KeyError in your second example is that pandas does not know how to interpret the string you provided). While pandas has some flexibility in how it parses datetime-strings I might recommend sticking to the ISO 8601 format, which should work in all cases. For example to select just the values associated with the first date in your array you could do something like this: ``` In [1]: import numpy as np; import pandas as pd; import xarray as xr

In [2]: times = pd.date_range('1979', periods=8, freq='12H')

In [3]: da = xr.DataArray(np.arange(8), coords=[times], dims=['time'])

In [4]: da Out[4]: <xarray.DataArray (time: 8)> array([0, 1, 2, 3, 4, 5, 6, 7]) Coordinates: * time (time) datetime64[ns] 1979-01-01 ... 1979-01-04T12:00:00

In [5]: da.loc['1979-01-01T00'] Out[5]: <xarray.DataArray ()> array(0) Coordinates: time datetime64[ns] 1979-01-01 One other thing to note, in general, is that you can pass datetimes themselves to label-based indexers if you want to select specific dates; this way you don't need to go through the intermediate step of converting a date to a string: In [6]: da.loc[datetime(1979, 1, 1)] Out[6]: <xarray.DataArray ()> array(0) Coordinates: time datetime64[ns] 1979-01-01 ```