html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/pull/7399#issuecomment-1362895889,https://api.github.com/repos/pydata/xarray/issues/7399,1362895889,IC_kwDOAMm_X85RPCQR,20629530,2022-12-22T14:20:36Z,2022-12-22T14:20:36Z,CONTRIBUTOR,"This makes complete sense to me, thanks! I'm quite surprised I didn't get this issue before in my scripts, sorry about that!","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1507817863 https://github.com/pydata/xarray/issues/4348#issuecomment-1353551624,https://api.github.com/repos/pydata/xarray/issues/4348,1353551624,IC_kwDOAMm_X85QrY8I,20629530,2022-12-15T18:47:38Z,2022-12-15T18:47:38Z,CONTRIBUTOR,"This is happenning to me too. For the record, I need to use `engine='pydap'` because it supports authentification to access private thredds data programmatically (by passing `session`), while `netCDF4` does not. I pointed to this issue in the related (but currently badly named and stale) issue upstream.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,681291824 https://github.com/pydata/xarray/pull/7361#issuecomment-1346807178,https://api.github.com/repos/pydata/xarray/issues/7361,1346807178,IC_kwDOAMm_X85QRqWK,20629530,2022-12-12T16:11:29Z,2022-12-12T16:11:29Z,CONTRIBUTOR,"Hmm, I don't think so! Thanks for the heads up.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1477931237 https://github.com/pydata/xarray/issues/6946#issuecomment-1240904315,https://api.github.com/repos/pydata/xarray/issues/6946,1240904315,IC_kwDOAMm_X85J9rJ7,20629530,2022-09-08T15:51:15Z,2022-09-08T15:51:27Z,CONTRIBUTOR,"I found another problem that this issue causes : the dataset can't be written to netCDF and the error message suggests exactly what has already been done: ```python import xarray as xr ds = xr.tutorial.open_dataset('air_temperature') ds = ds.stack(spatial=['lon', 'lat']) ds = ds.reset_index('spatial', drop=True) ds.to_netcdf('test.nc') ``` raises: ``` NotImplementedError: variable 'lat' is a MultiIndex, which cannot yet be serialized to netCDF files (https://github.com/pydata/xarray/issues/1077). Use reset_index() to convert MultiIndex levels into coordinate variables instead. ``` @benbovy I'm guessing your PR will fix this, but I'm raising the issue to be sure. ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1347026292 https://github.com/pydata/xarray/issues/4205#issuecomment-1176645772,https://api.github.com/repos/pydata/xarray/issues/4205,1176645772,IC_kwDOAMm_X85GIjCM,20629530,2022-07-06T20:00:09Z,2022-07-06T20:00:32Z,CONTRIBUTOR,"I have the same problem in xarray 2022.3.0. The issue is that this creates unnecessary dask tasks in the graph and some operations acting on the coordinates unexpectedly trigger dask computations. ""Unexpected"" because the coordinates at the beginning of the process where not chunked. So computation that was expected to happen in the main thread (or not happen at all) is now happenning in the dask workers. An example: ```python3 import numpy as np import xarray as xr from dask.diagnostics import ProgressBar # A 2D variable da = xr.DataArray( np.ones((12, 10)), dims=('x', 'y'), coords={'x': np.arange(12), 'y': np.arange(10)} ) # A 1D variable sharing a dim with da db = xr.DataArray( np.ones((12,)), dims=('x'), coords={'x': np.arange(12)} ) # A non-dimension coordinate cx = xr.DataArray(np.zeros((12,)), dims=('x',), coords={'x': np.arange(12)}) # Assign it to da and db da = da.assign_coords(cx=cx) db = db.assign_coords(cx=cx) # We need to chunk along y da = da.chunk({'y': 1}) # Notice how `cx` is now a dask array, even if it is a 1D coordinate and does not have 'Y' as a dimension. print(da) # This triggers a dask computation with ProgressBar(): da - db ``` The reason my example triggers dask is that xarray ensure the coordinates are aligned and equal (I think?). Anyway, I didn't expect it. Personally, I think the `chunk` method shouldn't apply to the coordinates at all, no matter their dimensions. They're coordinate so we expect to be able to read them easily when aligning/comparing dataset. Dask is to be used with the ""real"" data only. Does this vision fit the one from the devs? I feel this ""skip"" could be easily implemented.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,651945063 https://github.com/pydata/xarray/issues/6741#issuecomment-1171576721,https://api.github.com/repos/pydata/xarray/issues/6741,1171576721,IC_kwDOAMm_X85F1NeR,20629530,2022-06-30T19:08:28Z,2022-06-30T19:08:28Z,CONTRIBUTOR,"To add details, I suspect this commit as the culprit: https://github.com/headtr1ck/xarray/commit/cd2f2b247cf6e1c68c7db6e92100898272fb0e1a It removes a direct import of the `core.resample` submodule in `core.dataarray` and moves it to the ""Type checking"" imports. Thus, importing xarray (`import xarray as xr`), does not trigger an explicit import of `resample` anymore.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1290524064 https://github.com/pydata/xarray/pull/5734#issuecomment-1124203598,https://api.github.com/repos/pydata/xarray/issues/5734,1124203598,IC_kwDOAMm_X85DAfxO,20629530,2022-05-11T19:24:56Z,2022-05-11T19:25:11Z,CONTRIBUTOR,"> Presumably you haven't found any bugs? Not for now,, but I haven't done much and it wasn't very complex. > You can pass method as .mean(..., method=...) Yes! I saw that, but I did search a bit. In fact, my idea was triggered partly because I'm lazy. My suggestion was to do something alike what xarray does with `keep_attrs` : a global/context option that changes the default value of `method`, so it doesn't need to be explicitly added everywhere. > Note you can only blockwise if all members of a group are in a single block. Oh! I didn't catch that. My test data was already appropriately chunked. > Right now, I'd just like this to get merged :) Haha, I understand. My idea, if pertinent, could be done in another PR. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,978356586 https://github.com/pydata/xarray/pull/5734#issuecomment-1124142026,https://api.github.com/repos/pydata/xarray/issues/5734,1124142026,IC_kwDOAMm_X85DAQvK,20629530,2022-05-11T18:18:17Z,2022-05-11T18:18:17Z,CONTRIBUTOR,"Hi @dcherian and @andersy005, thanks a lot for this PR and for flox. I'm currently trying this with our climate indices library (xclim) and it works well! I get a clear improvement in performance, In xclim's indicators, all groupby operations are time resampling ops and the best performance is achieved with `method='blockwise'`, by far. Is there a plan to let users control the default method through something like `xr.set_options` ? Like, `use_flox` could accept False, True or any method name? (The default still being `map-reduce`) ? What do you think? ","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,978356586 https://github.com/pydata/xarray/issues/5155#issuecomment-953076586,https://api.github.com/repos/pydata/xarray/issues/5155,953076586,IC_kwDOAMm_X844zstq,20629530,2021-10-27T16:06:01Z,2021-10-27T16:06:15Z,CONTRIBUTOR,"I agree that it kinda crowds the xarray API. My first idea was to implement these calendar utilities in `cf-xarray`, but I was redirected here. Seems to me that if that move is made, it's the most logical place for all CF[time] related things.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,857947050 https://github.com/pydata/xarray/pull/5233#issuecomment-941292524,https://api.github.com/repos/pydata/xarray/issues/5233,941292524,IC_kwDOAMm_X844Gvvs,20629530,2021-10-12T18:52:15Z,2021-10-12T18:52:15Z,CONTRIBUTOR,"Here we are! Also, I don't think the test failure comes from my changes.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,870312451 https://github.com/pydata/xarray/pull/5233#issuecomment-931424397,https://api.github.com/repos/pydata/xarray/issues/5233,931424397,IC_kwDOAMm_X843hGiN,20629530,2021-09-30T15:25:23Z,2021-09-30T15:25:23Z,CONTRIBUTOR,"@spencerkclark Done! Note that we have this issue : Ouranosinc/xclim#841 open on our side. It's for a third method for converting 360_day calendars. Based on something used in the litterature, it randomizes the days to insert/drop when converting. My plan is to implement it in xclim soonish, then if it works I could bring it up here and open a new PR.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,870312451 https://github.com/pydata/xarray/pull/5233#issuecomment-909508817,https://api.github.com/repos/pydata/xarray/issues/5233,909508817,IC_kwDOAMm_X842NgDR,20629530,2021-08-31T18:49:39Z,2021-08-31T18:49:39Z,CONTRIBUTOR,"@spencerkclark After more than 2 weeks, I had time to address your comments!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,870312451 https://github.com/pydata/xarray/pull/5233#issuecomment-909375096,https://api.github.com/repos/pydata/xarray/issues/5233,909375096,IC_kwDOAMm_X842M_Z4,20629530,2021-08-31T16:07:28Z,2021-08-31T16:07:28Z,CONTRIBUTOR,"@spencerkclark I came back to this today, merged upstream/main and now I can't commit the merge because `xarray/core/_typed_ops.pyi` is not passing the `mypy` pre-commit hook. Is this a problem on my side? Have you ever seen this? It's strange cause in `setup.cfg` only *.py files are supposed to be checked by mypy, but I guess `pre-commit` doesn't respect that? `pre-commit run --all` does work though... May be @max-sixty has an idea, since he recently pushed a PR playing with this file and typing... The errors :
``` xarray/core/_typed_ops.pyi:32: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:33: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:34: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:35: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:36: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:37: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:38: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:39: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:40: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:41: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:42: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:43: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:44: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:45: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:46: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:47: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:48: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:49: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:50: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:51: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:52: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:53: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:54: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:55: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:56: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:57: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:60: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:61: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:62: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:63: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:64: error: The erased type of self ""xarray.core.dataset.Dataset"" is not a supertype of its class ""xarray.core._typed_ops.DatasetOpsMixin"" [misc] xarray/core/_typed_ops.pyi:65: error: The erased type of self ""xarray.core.dataset.Dataset"" is not a supertype of its class ""xarray.core._typed_ops.DatasetOpsMixin"" [misc] xarray/core/_typed_ops.pyi:66: error: The erased type of self ""xarray.core.dataset.Dataset"" is not a supertype of its class ""xarray.core._typed_ops.DatasetOpsMixin"" [misc] xarray/core/_typed_ops.pyi:67: error: The erased type of self ""xarray.core.dataset.Dataset"" is not a supertype of its class ""xarray.core._typed_ops.DatasetOpsMixin"" [misc] xarray/core/_typed_ops.pyi:77: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:83: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:89: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:95: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:101: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:107: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:113: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:119: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:125: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:131: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:137: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:143: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:149: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:155: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:161: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:167: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:173: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:179: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:185: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:191: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:197: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:203: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:209: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:215: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:221: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:227: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:230: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:231: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:232: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:233: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:234: error: The erased type of self ""xarray.core.dataarray.DataArray"" is not a supertype of its class ""xarray.core._typed_ops.DataArrayOpsMixin"" [misc] xarray/core/_typed_ops.pyi:235: error: The erased type of self ""xarray.core.dataarray.DataArray"" is not a supertype of its class ""xarray.core._typed_ops.DataArrayOpsMixin"" [misc] xarray/core/_typed_ops.pyi:236: error: The erased type of self ""xarray.core.dataarray.DataArray"" is not a supertype of its class ""xarray.core._typed_ops.DataArrayOpsMixin"" [misc] xarray/core/_typed_ops.pyi:237: error: The erased type of self ""xarray.core.dataarray.DataArray"" is not a supertype of its class ""xarray.core._typed_ops.DataArrayOpsMixin"" [misc] xarray/core/_typed_ops.pyi:247: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:253: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:259: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:265: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:271: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:277: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:283: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:289: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:295: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:301: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:307: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:313: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:319: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:325: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:331: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:337: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:343: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:349: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:355: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:361: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:367: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:373: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:379: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:385: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:391: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:397: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:400: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:401: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:402: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:403: error: Self argument missing for a non-static method (or an invalid type for self) [misc] xarray/core/_typed_ops.pyi:404: error: The erased type of self ""xarray.core.variable.Variable"" is not a supertype of its class ""xarray.core._typed_ops.VariableOpsMixin"" [misc] xarray/core/_typed_ops.pyi:405: error: The erased type of self ""xarray.core.variable.Variable"" is not a supertype of its class ""xarray.core._typed_ops.VariableOpsMixin"" [misc] xarray/core/_typed_ops.pyi:406: error: The erased type of self ""xarray.core.variable.Variable"" is not a supertype of its class ""xarray.core._typed_ops.VariableOpsMixin"" [misc] xarray/core/_typed_ops.pyi:407: error: The erased type of self ""xarray.core.variable.Variable"" is not a supertype of its class ""xarray.core._typed_ops.VariableOpsMixin"" [misc] Found 102 errors in 1 file (checked 3 source files) ```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,870312451 https://github.com/pydata/xarray/issues/5629#issuecomment-886905635,https://api.github.com/repos/pydata/xarray/issues/5629,886905635,IC_kwDOAMm_X8403Rsj,20629530,2021-07-26T17:52:42Z,2021-07-26T17:52:42Z,CONTRIBUTOR,"> @aulemahal what do you think about how easy it might be to change polyfit to use apply_ufunc? Should be easy! The code was written without apply_ufunc for a few reasons, one being that using `apply_ufunc+vectorize` was slower than the `apply_along_axis` trick it currently uses. But, with the issue raised here, that might not be true...","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",,950882492 https://github.com/pydata/xarray/issues/5551#issuecomment-870748510,https://api.github.com/repos/pydata/xarray/issues/5551,870748510,MDEyOklzc3VlQ29tbWVudDg3MDc0ODUxMA==,20629530,2021-06-29T16:35:02Z,2021-06-29T16:39:59Z,CONTRIBUTOR,"EDIT: I hit enter a bit too soon... I think I found the very source of the problem, maybe this could be fixed in cftime: ```python import pickle ds.time # same No-Leap coordinate as in the MWE above # Items are instances of `cftime.DatetimeNoLeap` pickle.loads(pickle.dumps(ds.time)) # Items are instances of `cftime.datetime`, calendar information is now only an attribute, `get_clean_interp_index` will fail ``` However, as xarray has the possiblity to contain cftime coordinates where the calendar is an attribute, I guess `get_clean_interp_index` needs to be fixed?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,932898075 https://github.com/pydata/xarray/pull/5402#issuecomment-853900455,https://api.github.com/repos/pydata/xarray/issues/5402,853900455,MDEyOklzc3VlQ29tbWVudDg1MzkwMDQ1NQ==,20629530,2021-06-03T14:11:16Z,2021-06-03T14:11:16Z,CONTRIBUTOR,"Agreed that this is limited and there's need for a more general solution! However, this problem is quite tricky... The best I see is what I think you suggested in the other thread : special workaround with re-casting when the operation involves "" x, zy data_t = data.transpose(2, 0, 1) # Dask array with shape (30, 10, 20), the same as `reordered` in `Variable._stack_once`. new_data = data_t.reshape((30, -1), merge_chunks=True) # True is the default, this is the same call as in xarray new_data.chunks # ((30,), (20, 20, 20, 20, 20, 20, 20, 20, 20, 20)) new_data = data_t.reshape((30, -1), merge_chunks=False) new_data.shape # I'm printing shape because chunks is too large, but see the bug: # (30, 6000) # instead of (30, 200)!!! # Doesn't happen when we do not transpose. So let's reshape data as z, y, x -> zy, x new_data = data.reshape((-1, 30), merge_chunks=True) new_data.chunks # ((5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5), (30,)) # Chunks were not merged? But this is the output expected by paigem. new_data = data.reshape((-1, 30), merge_chunks=False) new_data.chunks # ((5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5), (30,)) # That's what I expected with merge_chunks=False. ``` For `polyfit` itself, the `apply_along_axis` call could be changed to a `apply_ufunc` with `vectorize=True`, I think this would avoid the problem and behave the same on the user's side. Would need some refactoring. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,732910109 https://github.com/pydata/xarray/issues/5155#issuecomment-819570900,https://api.github.com/repos/pydata/xarray/issues/5155,819570900,MDEyOklzc3VlQ29tbWVudDgxOTU3MDkwMA==,20629530,2021-04-14T14:40:14Z,2021-04-14T14:40:14Z,CONTRIBUTOR,"@aaronspring Oh, I didn't think of that trick for 1, thanks! But again, this fails with numpy-backed time coordinates. With the definition of a ""default"" calendar, there could be a more general way.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,857947050 https://github.com/pydata/xarray/issues/5010#issuecomment-793057577,https://api.github.com/repos/pydata/xarray/issues/5010,793057577,MDEyOklzc3VlQ29tbWVudDc5MzA1NzU3Nw==,20629530,2021-03-08T20:35:01Z,2021-03-08T20:35:01Z,CONTRIBUTOR,Dang. How did I miss that?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,824917345 https://github.com/pydata/xarray/issues/4853#issuecomment-771120951,https://api.github.com/repos/pydata/xarray/issues/4853,771120951,MDEyOklzc3VlQ29tbWVudDc3MTEyMDk1MQ==,20629530,2021-02-01T20:03:04Z,2021-02-01T20:03:04Z,CONTRIBUTOR,"Woups, the bug seems more important than I first thought. It happens when reading files: MWE: ```python import xarray as xr da = xr.DataArray( [0, 1, 2], dims=('time',), coords={'time': xr.cftime_range('2000-01-01', freq='D', periods=3, calendar='noleap')}, name='test' ) da.to_netcdf('test.nc') da2 = xr.open_dataset('test.nc') da2.indexes['time'] ``` Fails with an error similar to the example above. Also failing on : ```python da2.sel(time='2000') ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,798592803 https://github.com/pydata/xarray/issues/4463#issuecomment-700076657,https://api.github.com/repos/pydata/xarray/issues/4463,700076657,MDEyOklzc3VlQ29tbWVudDcwMDA3NjY1Nw==,20629530,2020-09-28T15:22:23Z,2020-09-28T15:22:23Z,CONTRIBUTOR,"Oh! I should have thought of this. The error message is indeed unclear... I'll close the issue as the problem is solved, but I'll note that: ```python3 da.sel(y=dy, x=dx, method='nearest') ``` does work, without the need to explicitly pass the shared dimension `t`. This is why I expected that `interp` would work the same.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,709272776 https://github.com/pydata/xarray/issues/4058#issuecomment-698462962,https://api.github.com/repos/pydata/xarray/issues/4058,698462962,MDEyOklzc3VlQ29tbWVudDY5ODQ2Mjk2Mg==,20629530,2020-09-24T16:50:10Z,2020-09-24T16:50:10Z,CONTRIBUTOR,"It doesn't work with 0.16.0, but does with later versions (just tried with the current master). Thanks!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,617704483 https://github.com/pydata/xarray/issues/4375#issuecomment-696845767,https://api.github.com/repos/pydata/xarray/issues/4375,696845767,MDEyOklzc3VlQ29tbWVudDY5Njg0NTc2Nw==,20629530,2020-09-22T16:50:26Z,2020-09-22T16:50:26Z,CONTRIBUTOR,"I did encounter this when writing up `polyfit`! I decided to discard that for the time being, but I believe changing `get_clean_interp_index` would the best way to fix this. One would also need to modify `polyfit` sligthly, though. Here: https://github.com/pydata/xarray/blob/9c85dd5f792805bea319f01f08ee51b83bde0f3b/xarray/core/dataset.py#L6064-L6075 When the non-fitted dimensions are stacked together, `dim` should be renamed to the real dimension of the variable. I will not have time to make a PR, but it should be quite simple, if anyone has time!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,685739084 https://github.com/pydata/xarray/pull/4193#issuecomment-675046525,https://api.github.com/repos/pydata/xarray/issues/4193,675046525,MDEyOklzc3VlQ29tbWVudDY3NTA0NjUyNQ==,20629530,2020-08-17T18:42:51Z,2020-08-17T18:42:51Z,CONTRIBUTOR,@mathause Fixed the tests to catch the warning and added a case for the specific of the issue : too many nans causing a rank deficiency. ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,650044968 https://github.com/pydata/xarray/pull/4193#issuecomment-674153884,https://api.github.com/repos/pydata/xarray/issues/4193,674153884,MDEyOklzc3VlQ29tbWVudDY3NDE1Mzg4NA==,20629530,2020-08-14T16:23:12Z,2020-08-14T16:23:35Z,CONTRIBUTOR,"So while adding tests, I realized there were more bugs concerning deficient rank matrices and `full=True` output. I believe I fixed all I could find. However, I was not able to reduce the number of warnings for the case using dask : the computation occurs outside of the `catch_warnings()` context... Also, there is a bug in the `dask.array.linalg.lstsq` output (see issue dask/dask#6516), so if `full=True` we must use `skipna=True` with dask. Same when rank != order (deficient rank), `dask.array.linalg.lstsq` will fail, so we work around by using our (slower) nan-skipping method.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,650044968 https://github.com/pydata/xarray/issues/4300#issuecomment-668264540,https://api.github.com/repos/pydata/xarray/issues/4300,668264540,MDEyOklzc3VlQ29tbWVudDY2ODI2NDU0MA==,20629530,2020-08-03T22:05:25Z,2020-08-03T22:05:25Z,CONTRIBUTOR,"My comments Q.1 : For now `xr.apply_ufunc` does not accept core dimensions to be chunked along, which would be kinda a sad for curve fitting. However, dask's least square method does, which is part of the reason why I used it in polyfit. On the other hand, scipy's least-squares procedure is not simple. Curve fitting is quite complex and rewriting all the code to use dask might be a project too ambitious, and surely out-of-scope for xarray... Q.3 : For simple directly declared function, inspect does a good job, but it can get tricky with wrapped functions, which might arise in more complex workflows. Could we have a `params` arg that takes in a list of names?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,671609109 https://github.com/pydata/xarray/pull/4033#issuecomment-634215961,https://api.github.com/repos/pydata/xarray/issues/4033,634215961,MDEyOklzc3VlQ29tbWVudDYzNDIxNTk2MQ==,20629530,2020-05-26T19:01:36Z,2020-05-26T19:01:36Z,CONTRIBUTOR,@spencerkclark Here you go! Tests added for all cases you mentioned.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,612846594 https://github.com/pydata/xarray/pull/4033#issuecomment-628857894,https://api.github.com/repos/pydata/xarray/issues/4033,628857894,MDEyOklzc3VlQ29tbWVudDYyODg1Nzg5NA==,20629530,2020-05-14T20:02:34Z,2020-05-14T20:02:34Z,CONTRIBUTOR,"@spencerkclark Thanks for the feedback! I rewrote the `_CFTimeFrequencyInferer` class independently of pandas private objects. I removed outputs that wouldn't match offsets defined in xarray: weekly frequencies, week-of-month (WOM) and all business-related anchors. It wouldn't be difficult to add them later on if need be and if `cftime_offsets.py` defines them. Note that `infer_freq` is quite independent of the rest of xarray, this choice was purely so that the behavior matches the rest of xr. I did leave the sub-seconds frequencies (L for ms and U for µs). I expect the performance of `xr.infer_freq` to be slightly worst than pandas' since they use optimized and compiled code whereas I only used common numpy and xarray stuff. However, I assume that this part of the workflow is far from being the bottleneck of most xarray usecases... Will write up some docs soon.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,612846594 https://github.com/pydata/xarray/pull/3733#issuecomment-603976716,https://api.github.com/repos/pydata/xarray/issues/3733,603976716,MDEyOklzc3VlQ29tbWVudDYwMzk3NjcxNg==,20629530,2020-03-25T17:26:59Z,2020-03-25T17:26:59Z,CONTRIBUTOR,"Yay! Many thanks, this will greatly improve our projects.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 1, ""eyes"": 0}",,557627188 https://github.com/pydata/xarray/pull/3733#issuecomment-603901324,https://api.github.com/repos/pydata/xarray/issues/3733,603901324,MDEyOklzc3VlQ29tbWVudDYwMzkwMTMyNA==,20629530,2020-03-25T15:19:53Z,2020-03-25T15:19:53Z,CONTRIBUTOR,Ping. It be green.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,557627188 https://github.com/pydata/xarray/issues/3349#issuecomment-581584469,https://api.github.com/repos/pydata/xarray/issues/3349,581584469,MDEyOklzc3VlQ29tbWVudDU4MTU4NDQ2OQ==,20629530,2020-02-03T19:43:53Z,2020-02-03T19:43:53Z,CONTRIBUTOR,"I pushed a new PR trying to implement polyfit in xarray, #3733. It is still work in progress, but I would like the opinion on those who participated in this thread. Considering all options discussed in the thread, I chose an implementation that seemed to give the best performance and generality (skipping NaN values), but it duplicates a lot of code from `numpy.polyfit`. Main question: - Should xarray's implementation really replicate the behaviour of numpy's? A lot of extra code could be removed if we'd say we only want to compute and return the residuals and the coefficients. All the other variables are a few lines of code away for the user that really wants them, and they don't need the power of xarray and dask anyway. I'm guessing @huard @dcherian @rabernat and @shoyer might have comments.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,499477363 https://github.com/pydata/xarray/pull/3733#issuecomment-580369935,https://api.github.com/repos/pydata/xarray/issues/3733,580369935,MDEyOklzc3VlQ29tbWVudDU4MDM2OTkzNQ==,20629530,2020-01-30T17:39:54Z,2020-01-30T22:23:31Z,CONTRIBUTOR,"Oh `dask=""allowed""`, that's true! For the coeficients and residuals, I can easily do this stacking and unstacking and return only one variable. So, `apply_ufunc` could be used. However, it would not be very useful right now, considering they way I implemented the least-squares fit directly. Meaning: the left hand side computation and scaling and the not-fitted-dimensions stacking would be done on each call of `numpy.polyfit` if I did call it directly. Right now, we save some of this overhead by doing it once before iterating on 1d axes (which is anyway needed with `skipna=True`).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,557627188