issue_comments

Thanks for the super fast fix. I have confirmed this fixes #5677

@aidanheerdegen I tested your example with my upstream changes (Unidata/cftime#252) and it now works, so this should be fixed with the next release of cftime. I'll go ahead and close this issue. Thanks for your help debugging this!

A colleague suggested it might be some sort of pickling issue, passing the generated object back to the main thread, but it was just speculation and I had no idea how to test that.

Yes, it must be something of that sort. Here's perhaps an even more minimal example: ```

import cftime; import distributed header, frames = distributed.protocol.serialize(cftime.DatetimeNoLeap(2000, 1, 1)) distributed.protocol.deserialize(header, frames) cftime.datetime(2000, 1, 1, 0, 0, 0, 0, calendar='noleap', has_year_zero=True) Further, removing `distributed` from the mix, we can show this just using `pickle`: import pickle serialized = pickle.dumps(cftime.DatetimeNoLeap(2000, 1, 1)) deserialized = pickle.loads(serialized) deserialized cftime.datetime(2000, 1, 1, 0, 0, 0, 0, calendar='noleap', has_year_zero=True) ``` I'll make an issue in cftime.

A colleague suggested it might be some sort of pickling issue, passing the generated object back to the main thread, but it was just speculation and I had no idea how to test that.

Thanks again for the prompt response @spencerkclark. Yes your MCVE is more (less?) M than mine. Thanks.

Perhaps I shouldn't have started a new issue, but it seemed the specific problem with .sel was just a knock on effect from this cftime issue.

I should have said in #5677 that as far as I could tell I was using cftime=1.5.0.

Thanks for the updated report! Could you kindly share the full error traceback?

Sorry, see below python Traceback (most recent call last): File "/g/data/v45/aph502/helpdesk/fromgithub/20210804-Navid/mcve.py", line 28, in <module> assert (index_microseconds == xr.open_mfdataset('2???.nc', parallel=True).xindexes['time'].array.asi8).all() File "/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.07/lib/python3.9/site-packages/xarray/coding/cftimeindex.py", line 683, in asi8 [ File "/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.07/lib/python3.9/site-packages/xarray/coding/cftimeindex.py", line 684, in <listcomp> _total_microseconds(exact_cftime_datetime_difference(epoch, date)) File "/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.07/lib/python3.9/site-packages/xarray/core/resample_cftime.py", line 358, in exact_cftime_datetime_difference seconds = b.replace(microsecond=0) - a.replace(microsecond=0) File "src/cftime/_cftime.pyx", line 1369, in cftime._cftime.datetime.__sub__ TypeError: cannot compute the time difference between dates with different calendars

@aidanheerdegen thanks a lot for the minimal example -- I'm able to reproduce it now -- this is indeed a confusing bug! I think this is a symptom of the same problem I identified in #5677: the dates in your dataset are being decoded to cftime.datetime objects instead of cftime.DatetimeNoLeap objects. This causes problems downstream in a variety of places in xarray, because xarray currently infers the date and calendar type of the index by checking the type of the dates it contains.

The question is: why is this happening? I initially thought this could only happen if you were using cftime version 1.4.0, but that is clearly not true. It is interesting that this only comes up when using a distributed client and parallel=True in open_mfdataset, while with other more basic approaches the dates are decoded properly to cftime.DatetimeNoLeap objects. I think a more minimal example of this issue may be the following: ```

import cftime; import dask; import distributed cftime.version '1.5.0' dask.version '2021.07.2' distributed.version '2021.07.2' client = distributed.Client() cftime.num2date([0, 1, 2], units="days since 2000-01-01", calendar="noleap") array([cftime.DatetimeNoLeap(2000, 1, 1, 0, 0, 0, 0, has_year_zero=True), cftime.DatetimeNoLeap(2000, 1, 2, 0, 0, 0, 0, has_year_zero=True), cftime.DatetimeNoLeap(2000, 1, 3, 0, 0, 0, 0, has_year_zero=True)], dtype=object) delayed_num2date = dask.delayed(cftime.num2date) delayed_num2date([0, 1, 2], units="days since 2000-01-01", calendar="noleap").compute() array([cftime.datetime(2000, 1, 1, 0, 0, 0, 0, calendar='noleap', has_year_zero=True), cftime.datetime(2000, 1, 2, 0, 0, 0, 0, calendar='noleap', has_year_zero=True), cftime.datetime(2000, 1, 3, 0, 0, 0, 0, calendar='noleap', has_year_zero=True)], dtype=object) `` Note that when usingdelayedin conjunction with adistributed.Clientthe dates are decoded tocftime.datetimeobjects instead ofcftime.DatetimeNoLeap` objects. This fairly clearly demonstrates that this is an upstream issue -- likely in cftime -- but I need to dig a little more to determine exactly what the issue is.

Thanks for the updated report! Could you kindly share the full error traceback?