html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/5686#issuecomment-895596762,https://api.github.com/repos/pydata/xarray/issues/5686,895596762,IC_kwDOAMm_X841Ybja,6628425,2021-08-09T22:36:53Z,2021-08-09T22:36:53Z,MEMBER,"@aidanheerdegen I tested your example with my upstream changes (Unidata/cftime#252) and it now works, so this should be fixed with the next release of cftime. I'll go ahead and close this issue. Thanks for your help debugging this!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,963688125 https://github.com/pydata/xarray/issues/5686#issuecomment-895162608,https://api.github.com/repos/pydata/xarray/issues/5686,895162608,IC_kwDOAMm_X841Wxjw,6628425,2021-08-09T11:57:39Z,2021-08-09T11:57:39Z,MEMBER,"> A colleague suggested it might be some sort of pickling issue, passing the generated object back to the main thread, but it was just speculation and I had no idea how to test that. Yes, it must be something of that sort. Here's perhaps an even more minimal example: ``` >>> import cftime; import distributed >>> header, frames = distributed.protocol.serialize(cftime.DatetimeNoLeap(2000, 1, 1)) >>> distributed.protocol.deserialize(header, frames) cftime.datetime(2000, 1, 1, 0, 0, 0, 0, calendar='noleap', has_year_zero=True) ``` Further, removing `distributed` from the mix, we can show this just using `pickle`: ``` >>> import pickle >>> serialized = pickle.dumps(cftime.DatetimeNoLeap(2000, 1, 1)) >>> deserialized = pickle.loads(serialized) >>> deserialized cftime.datetime(2000, 1, 1, 0, 0, 0, 0, calendar='noleap', has_year_zero=True) ``` I'll make an issue in cftime.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,963688125 https://github.com/pydata/xarray/issues/5686#issuecomment-895074385,https://api.github.com/repos/pydata/xarray/issues/5686,895074385,IC_kwDOAMm_X841WcBR,6628425,2021-08-09T09:24:30Z,2021-08-09T09:24:30Z,MEMBER,"@aidanheerdegen thanks a lot for the minimal example -- I'm able to reproduce it now -- this is indeed a confusing bug! I think this is a symptom of the same problem I identified in #5677: the dates in your dataset are being decoded to `cftime.datetime` objects instead of `cftime.DatetimeNoLeap` objects. This causes problems downstream in a variety of places in xarray, because xarray currently infers the date and calendar type of the index by checking the `type` of the dates it contains. The question is: why is this happening? I initially thought this could only happen if you were using cftime version 1.4.0, but that is clearly not true. It is interesting that this only comes up when using a `distributed` client and `parallel=True` in `open_mfdataset`, while with other more basic approaches the dates are decoded properly to `cftime.DatetimeNoLeap` objects. I think a more minimal example of this issue may be the following: ``` >>> import cftime; import dask; import distributed >>> cftime.__version__ '1.5.0' >>> dask.__version__ '2021.07.2' >>> distributed.__version__ '2021.07.2' >>> client = distributed.Client() >>> cftime.num2date([0, 1, 2], units=""days since 2000-01-01"", calendar=""noleap"") array([cftime.DatetimeNoLeap(2000, 1, 1, 0, 0, 0, 0, has_year_zero=True), cftime.DatetimeNoLeap(2000, 1, 2, 0, 0, 0, 0, has_year_zero=True), cftime.DatetimeNoLeap(2000, 1, 3, 0, 0, 0, 0, has_year_zero=True)], dtype=object) >>> delayed_num2date = dask.delayed(cftime.num2date) >>> delayed_num2date([0, 1, 2], units=""days since 2000-01-01"", calendar=""noleap"").compute() array([cftime.datetime(2000, 1, 1, 0, 0, 0, 0, calendar='noleap', has_year_zero=True), cftime.datetime(2000, 1, 2, 0, 0, 0, 0, calendar='noleap', has_year_zero=True), cftime.datetime(2000, 1, 3, 0, 0, 0, 0, calendar='noleap', has_year_zero=True)], dtype=object) ``` Note that when using `delayed` in conjunction with a `distributed.Client` the dates are decoded to `cftime.datetime` objects instead of `cftime.DatetimeNoLeap` objects. This fairly clearly demonstrates that this is an upstream issue -- likely in cftime -- but I need to dig a little more to determine exactly what the issue is.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,963688125 https://github.com/pydata/xarray/issues/5686#issuecomment-895018706,https://api.github.com/repos/pydata/xarray/issues/5686,895018706,IC_kwDOAMm_X841WObS,1217238,2021-08-09T07:46:39Z,2021-08-09T07:46:39Z,MEMBER,Thanks for the updated report! Could you kindly share the full error traceback?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,963688125