id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type
1966675016,I_kwDOAMm_X851ORRI,8388,Type annotation compatibility with numpy ufuncs,1828519,closed,0,,,4,2023-10-28T17:25:11Z,2023-11-02T12:44:50Z,2023-11-02T12:44:50Z,CONTRIBUTOR,,,,"### Is your feature request related to a problem?
I'd like mypy to understand that xarray DataArrays passed to numpy ufuncs have a return type of xarray DataArray.
```python
import xarray as xr
import numpy as np
def compute_relative_azimuth(sat_azi: xr.DataArray, sun_azi: xr.DataArray) -> xr.DataArray:
abs_diff = np.absolute(sun_azi - sat_azi)
ssadiff = np.minimum(abs_diff, 360 - abs_diff)
return ssadiff
```
```bash
$ mypy ./xarray_mypy.py
xarray_mypy.py:7: error: Incompatible return value type (got ""ndarray[Any, dtype[Any]]"", expected ""DataArray"") [return-value]
Found 1 error in 1 file (checked 1 source file)
```
### Describe the solution you'd like
I'm not sure if this is possible, if it is something xarray can fix, or something numpy needs to ""fix"". I'd like the above situation to ""just work"" without anything more than maybe some extra type-stub package.
### Describe alternatives you've considered
Cast types or other type coercion or tell mypy to ignore the type issues for these numpy call.
### Additional context
https://stackoverflow.com/questions/77369042/typing-when-passing-xarray-dataarray-objects-to-numpy-ufuncs","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8388/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue
1419602897,I_kwDOAMm_X85UnWvR,7197,Unstable pandas causes CF datetime64 issues,1828519,closed,0,,,4,2022-10-23T02:30:39Z,2022-10-26T16:00:35Z,2022-10-26T16:00:35Z,CONTRIBUTOR,,,,"### What happened?
The Satpy project has a CI environment that installs numpy, pandas, and xarray (and a couple other packages) from their unstable sources (nightly builds, github source, etc). In the last week or two this environment has started failing with various datetime64 issues. It all seems to be caused by some recent change in pandas, but I can't place exactly what the problem is nor the commit/PR that started it. It seems there are a couple datetime related PRs.
### What did you expect to happen?
Datetime or datetime64 objects should be allowed to be in whatever units they need to be in (days or minutes or nanoseconds. It seems parts of xarray (or pandas) assume `datetime64[ns]` but a change in pandas is no longer doing this conversion automatically (from `datetime64[X]` to `datetime64[ns]`.
### Minimal Complete Verifiable Example
You should be able to take any environment with modern xarray and install dev pandas with:
```
python -m pip install --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple/ --trusted-host pypi.anaconda.org --no-deps --pre --upgrade pandas
```
Then run this snippet:
```Python
import xarray as xr
import numpy as np
from xarray.coding.times import CFDatetimeCoder
a = xr.DataArray(np.arange(1.0), dims=(""time"",), coords={""time"": [np.datetime64('2018-05-30T10:05:00')]})
CFDatetimeCoder().encode(a.coords[""time""])
```
I haven't been able to generate a higher-level MVCE yet, but I'm hoping this little snippet will make the issue obvious to someone familiar with xarray internals.
### MVCE confirmation
- [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- [X] Complete example — the example is self-contained, including all data and the text of any traceback.
- [X] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result.
- [X] New issue — a search of GitHub Issues suggests this is not a duplicate.
### Relevant log output
```
----> 1 CFDatetimeCoder().encode(a.coords[""time""])
File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:676, in CFDatetimeCoder.encode(self, variable, name)
672 dims, data, attrs, encoding = unpack_for_encoding(variable)
673 if np.issubdtype(data.dtype, np.datetime64) or contains_cftime_datetimes(
674 variable
675 ):
--> 676 (data, units, calendar) = encode_cf_datetime(
677 data, encoding.pop(""units"", None), encoding.pop(""calendar"", None)
678 )
679 safe_setitem(attrs, ""units"", units, name=name)
680 safe_setitem(attrs, ""calendar"", calendar, name=name)
File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:612, in encode_cf_datetime(dates, units, calendar)
609 dates = np.asarray(dates)
611 if units is None:
--> 612 units = infer_datetime_units(dates)
613 else:
614 units = _cleanup_netcdf_time_units(units)
File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:394, in infer_datetime_units(dates)
392 print(""Formatting datetime object"")
393 reference_date = dates[0] if len(dates) > 0 else ""1970-01-01""
--> 394 reference_date = format_cftime_datetime(reference_date)
395 unique_timedeltas = np.unique(np.diff(dates))
396 units = _infer_time_units_from_diff(unique_timedeltas)
File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:405, in format_cftime_datetime(date)
400 def format_cftime_datetime(date):
401 """"""Converts a cftime.datetime object to a string with the format:
402 YYYY-MM-DD HH:MM:SS.UUUUUU
403 """"""
404 return ""{:04d}-{:02d}-{:02d} {:02d}:{:02d}:{:02d}.{:06d}"".format(
--> 405 date.year,
406 date.month,
407 date.day,
408 date.hour,
409 date.minute,
410 date.second,
411 date.microsecond,
412 )
AttributeError: 'numpy.datetime64' object has no attribute 'year'
```
### Anything else we need to know?
_No response_
### Environment
```
INSTALLED VERSIONS
------------------
commit: None
python: 3.9.12 | packaged by conda-forge | (main, Mar 24 2022, 23:25:59)
[GCC 10.3.0]
python-bits: 64
OS: Linux
OS-release: 5.19.0-76051900-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.12.2
libnetcdf: 4.8.1
xarray: 2022.10.0
pandas: 2.0.0.dev0+422.g6c46013c54
numpy: 1.23.4
scipy: 1.10.0.dev0+1848.f114d8b
netCDF4: 1.6.0
pydap: None
h5netcdf: 1.0.0
h5py: 3.7.0
Nio: None
zarr: 2.13.0a3.dev5
cftime: 1.6.2
nc_time_axis: None
PseudoNetCDF: None
rasterio: 1.4dev
cfgrib: None
iris: None
bottleneck: 1.3.5
dask: 2022.10.0+6.gc8dc3955
distributed: None
matplotlib: 3.7.0.dev473+gc450aa7baf
cartopy: 0.20.3
seaborn: None
numbagg: None
fsspec: 2022.5.0
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 65.3.0
pip: 22.2.2
conda: None
pytest: 7.1.1
IPython: 8.2.0
sphinx: 5.0.0
```
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7197/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue
812692450,MDU6SXNzdWU4MTI2OTI0NTA=,4934,ImplicitToExplicitIndexingAdapter being returned with dask unstable version,1828519,closed,0,,,4,2021-02-20T19:41:12Z,2021-03-09T22:51:13Z,2021-03-09T22:51:13Z,CONTRIBUTOR,,,,"**What happened**:
I have a couple CI environments that use unstable versions of xarray and dask among other libraries. A couple tests have been failing in different but similar ways. I was able to get one of them down to the below example. This happens with the dask master branch on either the xarray latest release or xarray master. Here are the commits I was using when doing unstable both.
Dask: `ad01acc14088d03127dfec4f881cce959f3ac4d9`
Xarray: `eb7e112d45a9edebd8e5fb4f873e3e6adb18824a`
I don't see this with dask's latest release and this is likely caused by a change in dask, but after seeing PRs like #4884 I'm wondering if this is something similar and requires a change in xarray.
The bottom line is that doing various things like `my_data_arr.data.compute()` or `dask.compute(my_data_arr.data)` produces:
```
ImplicitToExplicitIndexingAdapter(array=CopyOnWriteArray(array=LazilyOuterIndexedArray(array=, key=BasicIndexer((slice(0, 2, 1), slice(0, 100, 1), slice(0, 100, 1))))))
```
**What you expected to happen**:
I would expect to get a numpy array back when computing the underlying dask array.
**Minimal Complete Verifiable Example**:
```python
from PIL import Image
import xarray as xr
import numpy as np
# create a test image
Image.fromarray(np.zeros((5, 5, 3), dtype=np.uint8)).save('test.png')
r = xr.open_rasterio('test.png', chunks='auto')
print(r.data.compute())
# ImplicitToExplicitIndexingAdapter(array=CopyOnWriteArray(array=LazilyOuterIndexedArray(array=, key=BasicIndexer((slice(0, 2, 1), slice(0, 100, 1), slice(0, 100, 1))))))
```
**Anything else we need to know?**:
As mentioned above other weird things are happening when array wrappers seem to be involved but I haven't been able to make a small example of them.
**Environment**:
Output of xr.show_versions()
INSTALLED VERSIONS
------------------
commit: None
python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 22:33:48)
[GCC 7.3.0]
python-bits: 64
OS: Linux
OS-release: 5.8.0-7642-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
libhdf5: 1.10.5
libnetcdf: 4.7.3
xarray: 0.16.3.dev132+geb7e112d
pandas: 1.2.2
numpy: 1.20.1
scipy: 1.6.1
netCDF4: 1.5.3
pydap: None
h5netcdf: None
h5py: 2.10.0
Nio: None
zarr: 2.6.2.dev42
cftime: 1.4.1
nc_time_axis: None
PseudoNetCDF: None
rasterio: 1.3dev
cfgrib: None
iris: None
bottleneck: 1.4.0.dev0+117.gf2bc792
dask: 2021.02.0+21.gad01acc1
distributed: 2021.02.0+7.g383ea032
matplotlib: 3.4.0rc1
cartopy: 0.17.0
seaborn: None
numbagg: None
pint: None
setuptools: 45.2.0.post20200209
pip: 20.0.2
conda: None
pytest: 5.3.5
IPython: 7.12.0
sphinx: 2.4.3