{"database": "github", "table": "issues", "is_view": false, "human_description_en": "where comments = 14, state = \"open\" and user = 35968931 sorted by updated_at descending", "rows": [[1940536602, "I_kwDOAMm_X85zqj0a", 8298, "cftime.DatetimeNoLeap incorrectly decoded from netCDF file", 35968931, "open", 0, null, null, 14, "2023-10-12T18:13:53Z", "2024-01-08T01:01:53Z", null, "MEMBER", null, null, null, "### What happened?\r\n\r\nI have been given a netCDF file (I think it's netCDF3) which when I open it does not decode the time variable in the way I expected it to. The time coordinate created is a numpy object array\r\n\r\n![Screenshot from 2023-10-12 14-10-29](https://github.com/pydata/xarray/assets/35968931/78c9de00-ed80-481a-9849-d64faa2bb408)\r\n\r\n### What did you expect to happen?\r\n\r\nI expected it to automatically create a coordinate backed by a `CFTimeIndex` object, not a `CFTimeIndex` object wrapped inside another array type.\r\n\r\n### Minimal Complete Verifiable Example\r\n\r\nThe original problematic file is 455MB (I can share it if necessary), but I can create a small netCDF file that displays the same issue.\r\n\r\n```python\r\nimport cftime\r\n\r\ntime_values = [cftime.DatetimeNoLeap(347, 2, 1, 0, 0, 0, 0, has_year_zero=True)]\r\ntime_ds = xr.Dataset(coords={'time': (['time'], time_values)})\r\nprint(time_ds)\r\ntime_ds.to_netcdf('time_mwe.nc')\r\n```\r\n```\r\n<xarray.Dataset>\r\nDimensions:  (time: 1)\r\nCoordinates:\r\n  * time     (time) object 0347-02-01 00:00:00\r\nData variables:\r\n    *empty*\r\n```\r\n```python\r\nds = xr.open_dataset('time_mwe.nc', engine='netcdf4', decode_times=True, use_cftime=True)\r\nprint(ds)\r\n```\r\n```\r\n<xarray.Dataset>\r\nDimensions:  (time: 1)\r\nCoordinates:\r\n  * time     (time) object 0347-02-01 00:00:00\r\nData variables:\r\n    *empty*\r\n```\r\n\r\n\r\n### MVCE confirmation\r\n\r\n- [X] Minimal example \u2014 the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.\r\n- [X] Complete example \u2014 the example is self-contained, including all data and the text of any traceback.\r\n- [X] Verifiable example \u2014 the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result.\r\n- [X] New issue \u2014 a search of GitHub Issues suggests this is not a duplicate.\r\n- [X] Recent environment \u2014 the issue occurs with the latest version of xarray and its dependencies.\r\n\r\n### Relevant log output\r\n\r\n_No response_\r\n\r\n### Anything else we need to know?\r\n\r\n_No response_\r\n\r\n### Environment\r\n\r\n```\r\ncftime 1.6.2\r\nnetcdf4 1.6.4\r\nxarray 2023.8.0\r\n```", "{\"url\": \"https://api.github.com/repos/pydata/xarray/issues/8298/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", null, null, 13221727, "issue"], [1372035441, "I_kwDOAMm_X85Rx5lx", 7031, "Periodic Boundary Index", 35968931, "open", 0, null, null, 14, "2022-09-13T21:39:40Z", "2022-09-16T10:50:10Z", null, "MEMBER", null, null, null, "### What is your issue?\r\n\r\nI would like to create a `PeriodicBoundaryIndex` using the Explicit Indexes refactor. I want to do it first in 1D, then 2D, then maybe ND.\r\n\r\nI'm thinking this would be useful for:\r\n1) Geoscientists with periodic longitudes\r\n2) Any scientists with periodic domains\r\n3) Road-testing the refactor + how easy the documentation is to follow.\r\n\r\nEventually I think perhaps this index should live in xarray itself? As it's domain-agnostic, doesn't introduce extra dependencies, and could be a conceptually simple example of a custom index.\r\n\r\nI had a first go, using the [`benbovy:add-set-xindex-and-drop-indexes`](https://github.com/pydata/xarray/pull/6971) branch, and reading the [in-progress docs page](https://github.com/pydata/xarray/blob/2f9a4b35de8b393ebd0370eb98ba6e81dcaa7cf0/doc/internals/how-to-create-custom-index.rst). I got a bit stuck early on though.\r\n\r\n@benbovy here's what I have so far:\r\n\r\n```python\r\nimport numpy as np\r\nimport pandas as pd\r\nimport xarray as xr\r\nfrom xarray.core.variable import Variable\r\nfrom xarray.core.indexes import PandasIndex, is_scalar\r\n\r\nfrom typing import Union, Mapping, Any\r\n\r\n\r\nclass PeriodicBoundaryIndex(PandasIndex):\r\n    \"\"\"\r\n    An index representing any 1D periodic numberline.\r\n    \r\n    Implementation subclasses a normal xarray PandasIndex object but intercepts indexer queries.\r\n    \"\"\"\r\n        \r\n    def _periodic_subset(self, indxr: Union[int, slice, np.ndarray]) -> pd.Index:\r\n        \"\"\"Equivalent of __getitem__ for a pd.Index, but respects periodicity.\"\"\"\r\n        \r\n        length = len(self)\r\n        \r\n        if isinstance(indxr, int):\r\n            return self.index[indxr % length]\r\n        elif isinstance(indxr, slice):\r\n            raise NotImplementedError()\r\n        elif isinstance(indxr, np.ndarray):\r\n            raise NotImplementedError()\r\n        else:\r\n            raise TypeError    \r\n    \r\n    def isel(\r\n        self, indexers: Mapping[Any, Union[int, slice, np.ndarray, Variable]]\r\n    ) -> Union[\"PeriodicBoundaryIndex\", None]:\r\n\r\n        print(\"isel called\")\r\n\r\n        indxr = indexers[self.dim]\r\n        if isinstance(indxr, Variable):\r\n            if indxr.dims != (self.dim,):\r\n                # can't preserve a index if result has new dimensions\r\n                return None\r\n            else:\r\n                indxr = indxr.data\r\n        if not isinstance(indxr, slice) and is_scalar(indxr):\r\n            # scalar indexer: drop index\r\n            return None\r\n\r\n        subsetted_index = self._periodic_subset[indxr]\r\n        return self._replace(subsetted_index)\r\n```\r\n\r\n```python\r\nairtemps = xr.tutorial.open_dataset(\"air_temperature\")['air']\r\n\r\nda = airtemps.drop_indexes(\"lon\")\r\n\r\nworld = da.set_xindex(\"lon\", index_cls=PeriodicBoundaryIndex)\r\n```\r\n\r\n\r\nNow selecting a value with isel inside the range works fine, giving the same result same as without my custom index. (The length of the example dataset along `lon` is `53`.)\r\n\r\n```python\r\nworld.isel(lon=45)\r\n```\r\n\r\n```\r\nisel called\r\n<xarray.DataArray 'air' (time: 2920, lat: 25)>\r\n...\r\n```\r\n\r\nBut indexing with a `lon` value outside the range of the index data gives an `IndexError`, seemingly without consulting my new index object. It didn't even print `\"isel called\"` :confused:  What should I have implemented that I didn't implement?\r\n\r\n\r\n```python\r\nworld.isel(lon=55)\r\n```\r\n\r\n```python\r\n---------------------------------------------------------------------------\r\nIndexError                                Traceback (most recent call last)\r\nInput In [35], in <cell line: 1>()\r\n----> 1 world.isel(lon=55)\r\n\r\nFile ~/Documents/Work/Code/xarray/xarray/core/dataarray.py:1297, in DataArray.isel(self, indexers, drop, missing_dims, **indexers_kwargs)\r\n   1292     return self._from_temp_dataset(ds)\r\n   1294 # Much faster algorithm for when all indexers are ints, slices, one-dimensional\r\n   1295 # lists, or zero or one-dimensional np.ndarray's\r\n-> 1297 variable = self._variable.isel(indexers, missing_dims=missing_dims)\r\n   1298 indexes, index_variables = isel_indexes(self.xindexes, indexers)\r\n   1300 coords = {}\r\n\r\nFile ~/Documents/Work/Code/xarray/xarray/core/variable.py:1233, in Variable.isel(self, indexers, missing_dims, **indexers_kwargs)\r\n   1230 indexers = drop_dims_from_indexers(indexers, self.dims, missing_dims)\r\n   1232 key = tuple(indexers.get(dim, slice(None)) for dim in self.dims)\r\n-> 1233 return self[key]\r\n\r\nFile ~/Documents/Work/Code/xarray/xarray/core/variable.py:793, in Variable.__getitem__(self, key)\r\n    780 \"\"\"Return a new Variable object whose contents are consistent with\r\n    781 getting the provided key from the underlying data.\r\n    782 \r\n   (...)\r\n    790 array `x.values` directly.\r\n    791 \"\"\"\r\n    792 dims, indexer, new_order = self._broadcast_indexes(key)\r\n--> 793 data = as_indexable(self._data)[indexer]\r\n    794 if new_order:\r\n    795     data = np.moveaxis(data, range(len(new_order)), new_order)\r\n\r\nFile ~/Documents/Work/Code/xarray/xarray/core/indexing.py:657, in MemoryCachedArray.__getitem__(self, key)\r\n    656 def __getitem__(self, key):\r\n--> 657     return type(self)(_wrap_numpy_scalars(self.array[key]))\r\n\r\nFile ~/Documents/Work/Code/xarray/xarray/core/indexing.py:626, in CopyOnWriteArray.__getitem__(self, key)\r\n    625 def __getitem__(self, key):\r\n--> 626     return type(self)(_wrap_numpy_scalars(self.array[key]))\r\n\r\nFile ~/Documents/Work/Code/xarray/xarray/core/indexing.py:533, in LazilyIndexedArray.__getitem__(self, indexer)\r\n    531     array = LazilyVectorizedIndexedArray(self.array, self.key)\r\n    532     return array[indexer]\r\n--> 533 return type(self)(self.array, self._updated_key(indexer))\r\n\r\nFile ~/Documents/Work/Code/xarray/xarray/core/indexing.py:505, in LazilyIndexedArray._updated_key(self, new_key)\r\n    503         full_key.append(k)\r\n    504     else:\r\n--> 505         full_key.append(_index_indexer_1d(k, next(iter_new_key), size))\r\n    506 full_key = tuple(full_key)\r\n    508 if all(isinstance(k, integer_types + (slice,)) for k in full_key):\r\n\r\nFile ~/Documents/Work/Code/xarray/xarray/core/indexing.py:278, in _index_indexer_1d(old_indexer, applied_indexer, size)\r\n    276         indexer = slice_slice(old_indexer, applied_indexer, size)\r\n    277     else:\r\n--> 278         indexer = _expand_slice(old_indexer, size)[applied_indexer]\r\n    279 else:\r\n    280     indexer = old_indexer[applied_indexer]\r\n\r\nIndexError: index 55 is out of bounds for axis 0 with size 53\r\n```", "{\"url\": \"https://api.github.com/repos/pydata/xarray/issues/7031/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", null, null, 13221727, "issue"]], "truncated": false, "filtered_table_rows_count": 2, "expanded_columns": [], "expandable_columns": [[{"column": "repo", "other_table": "repos", "other_column": "id"}, "name"], [{"column": "milestone", "other_table": "milestones", "other_column": "id"}, "title"], [{"column": "assignee", "other_table": "users", "other_column": "id"}, "login"], [{"column": "user", "other_table": "users", "other_column": "id"}, "login"]], "columns": ["id", "node_id", "number", "title", "user", "state", "locked", "assignee", "milestone", "comments", "created_at", "updated_at", "closed_at", "author_association", "active_lock_reason", "draft", "pull_request", "body", "reactions", "performed_via_github_app", "state_reason", "repo", "type"], "primary_keys": ["id"], "units": {}, "query": {"sql": "select id, node_id, number, title, user, state, locked, assignee, milestone, comments, created_at, updated_at, closed_at, author_association, active_lock_reason, draft, pull_request, body, reactions, performed_via_github_app, state_reason, repo, type from issues where \"comments\" = :p0 and \"state\" = :p1 and \"user\" = :p2 order by updated_at desc limit 101", "params": {"p0": "14", "p1": "open", "p2": "35968931"}}, "facet_results": {"state": {"name": "state", "type": "column", "hideable": false, "toggle_url": "/github/issues.json?comments=14&state=open&user=35968931", "results": [{"value": "open", "label": "open", "count": 2, "toggle_url": "http://xarray-datasette.fly.dev/github/issues.json?comments=14&user=35968931", "selected": true}], "truncated": false}, "repo": {"name": "repo", "type": "column", "hideable": false, "toggle_url": "/github/issues.json?comments=14&state=open&user=35968931", "results": [{"value": 13221727, "label": "xarray", "count": 2, "toggle_url": "http://xarray-datasette.fly.dev/github/issues.json?comments=14&state=open&user=35968931&repo=13221727", "selected": false}], "truncated": false}, "type": {"name": "type", "type": "column", "hideable": false, "toggle_url": "/github/issues.json?comments=14&state=open&user=35968931", "results": [{"value": "issue", "label": "issue", "count": 2, "toggle_url": "http://xarray-datasette.fly.dev/github/issues.json?comments=14&state=open&user=35968931&type=issue", "selected": false}], "truncated": false}}, "suggested_facets": [{"name": "created_at", "type": "date", "toggle_url": "http://xarray-datasette.fly.dev/github/issues.json?comments=14&state=open&user=35968931&_facet_date=created_at"}, {"name": "updated_at", "type": "date", "toggle_url": "http://xarray-datasette.fly.dev/github/issues.json?comments=14&state=open&user=35968931&_facet_date=updated_at"}], "next": null, "next_url": null, "private": false, "allow_execute_sql": true, "query_ms": 31.736004166305065}