issue_comments: 895074385

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/5686#issuecomment-895074385 https://api.github.com/repos/pydata/xarray/issues/5686 895074385 IC_kwDOAMm_X841WcBR 6628425 2021-08-09T09:24:30Z 2021-08-09T09:24:30Z MEMBER

@aidanheerdegen thanks a lot for the minimal example -- I'm able to reproduce it now -- this is indeed a confusing bug! I think this is a symptom of the same problem I identified in #5677: the dates in your dataset are being decoded to cftime.datetime objects instead of cftime.DatetimeNoLeap objects. This causes problems downstream in a variety of places in xarray, because xarray currently infers the date and calendar type of the index by checking the type of the dates it contains.

The question is: why is this happening? I initially thought this could only happen if you were using cftime version 1.4.0, but that is clearly not true. It is interesting that this only comes up when using a distributed client and parallel=True in open_mfdataset, while with other more basic approaches the dates are decoded properly to cftime.DatetimeNoLeap objects. I think a more minimal example of this issue may be the following: ```

import cftime; import dask; import distributed cftime.version '1.5.0' dask.version '2021.07.2' distributed.version '2021.07.2' client = distributed.Client() cftime.num2date([0, 1, 2], units="days since 2000-01-01", calendar="noleap") array([cftime.DatetimeNoLeap(2000, 1, 1, 0, 0, 0, 0, has_year_zero=True), cftime.DatetimeNoLeap(2000, 1, 2, 0, 0, 0, 0, has_year_zero=True), cftime.DatetimeNoLeap(2000, 1, 3, 0, 0, 0, 0, has_year_zero=True)], dtype=object) delayed_num2date = dask.delayed(cftime.num2date) delayed_num2date([0, 1, 2], units="days since 2000-01-01", calendar="noleap").compute() array([cftime.datetime(2000, 1, 1, 0, 0, 0, 0, calendar='noleap', has_year_zero=True), cftime.datetime(2000, 1, 2, 0, 0, 0, 0, calendar='noleap', has_year_zero=True), cftime.datetime(2000, 1, 3, 0, 0, 0, 0, calendar='noleap', has_year_zero=True)], dtype=object) `` Note that when usingdelayedin conjunction with adistributed.Clientthe dates are decoded tocftime.datetimeobjects instead ofcftime.DatetimeNoLeap` objects. This fairly clearly demonstrates that this is an upstream issue -- likely in cftime -- but I need to dig a little more to determine exactly what the issue is.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   963688125