issue_comments

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

NumPy doesn't support timezones, but pandas does. This puts things in a slightly tricky position for xarray.

We do manage to get things to work for pandas dtypes stored in indexes, in most cases. Given that our resampling behavior also relies on pandas, I think we should be able to get this work, probably by tweaking our PandasIndexAdapter, as @darothen notes.

It's borderline whether this is a new bug or feature, but this would certainly be nice to fix if possible, so I'm marking this as "Contributions welcome".

Did some digging.

Note here that the dtypes of time1 and time2 are different; the first is a datetime64[ns] but the second is a datetime64[ns, UTC]. For the sake of illustration, I'm going to change the timezone to EST. If we print time2, we get something that looks like this:

``` python

time2 DatetimeIndex(['2000-01-01 00:00:00-05:00', '2000-01-01 01:00:00-05:00', '2000-01-01 02:00:00-05:00', '2000-01-01 03:00:00-05:00', '2000-01-01 04:00:00-05:00', '2000-01-01 05:00:00-05:00', '2000-01-01 06:00:00-05:00', '2000-01-01 07:00:00-05:00', '2000-01-01 08:00:00-05:00', '2000-01-01 09:00:00-05:00', ... '2000-12-30 14:00:00-05:00', '2000-12-30 15:00:00-05:00', '2000-12-30 16:00:00-05:00', '2000-12-30 17:00:00-05:00', '2000-12-30 18:00:00-05:00', '2000-12-30 19:00:00-05:00', '2000-12-30 20:00:00-05:00', '2000-12-30 21:00:00-05:00', '2000-12-30 22:00:00-05:00', '2000-12-30 23:00:00-05:00'], dtype='datetime64[ns, EST]', length=8760, freq='H') ```

But, if we directly print its values, we get something slightly different:

``` python

time2.values array(['2000-01-01T05:00:00.000000000', '2000-01-01T06:00:00.000000000', '2000-01-01T07:00:00.000000000', ..., '2000-12-31T02:00:00.000000000', '2000-12-31T03:00:00.000000000', '2000-12-31T04:00:00.000000000'], dtype='datetime64[ns]') ```

The difference is that the timezone delta has been automatically added in terms of hours to each value in time2. This brings up something to note: if you construct your Dataset using time1.values and time2.values, there is no problem:

python import pandas as pd import xarray as xr time1 = pd.date_range('2000-01-01', freq='H', periods=365 * 24) #timezone naïve time2 = pd.date_range('2000-01-01', freq='H', periods=365 * 24, tz='UTC') #timezone aware ds1 = xr.Dataset({'foo': ('time', np.arange(365 * 24)), 'time': time1.values}) ds2 = xr.Dataset({'foo': ('time', np.arange(365 * 24)), 'time': time2.values}) ds1.resample('3H', 'time', how='mean') # works fine ds2.resample('3H', 'time', how='mean') # works fine

Both time1 and time2 are instances of pd.DatetimeIndex which are subclasses of pd.Index. When xarray tries to turn them into Variables, it ultimately uses a PandasIndexAdapter to decode the contents of time1 and time2, and this is where the trouble happens. The PandasIndexAdapter tries to safely cast the dtype of the array it is passed, which works just fine for time1. But for some weird reason, numpy doesn't recognize its own datetime dtypes when they have timezone information. That is, this will work:

``` python

np.dtype('datetime64[ns]') dtype('<M8[ns]') ``` But this won't:

``` python

np.dtype('datetime64[ns, UTC]') TypeError: Invalid datetime unit in metadata string "[ns, UC]" ```

But also, the type of time2.dtype is a pandas.types.dtypes.DatetimeTZDtype, which NumPy doesn't know what to do with (it doesn't know how to map that type to its own datetime64).

So what happens is that the resulting Variable which defines the time coordinate on your ds2 has an array with the correct values, but is explicitly told to have the dtype object. When the array is decoded, then, bad things happen.

One solution would be to catch this potential glitch in either is_valid_numpy_dtype() or the PandasIndexAdapter constructor. Alternatively, we could eagerly coerce arrays with type pandas.types.dtypes.DatetimeTZDtype into numpy-compliant types at some earlier point.