issue_comments

I am closing this. Feel free to re-open/ or open a new issue.

Yes float64 should cause less imprecision. You can convert using astype:

```python import numpy as np import xarray as xr

da = xr.DataArray(np.array([1, 2], dtype=np.float32))

da = da.astype(float) ```

As for the other problems I think you are better of asking the people over at rioxarray. However, you should first gather all the steps you did to convert the data as code. This way it is easier to see what you are actually doing.

xarray handles nan values and ignores them per default - so you don't need to remove them. For example: ```python import numpy as np import xarray as xr

da = xr.DataArray([1, 2, 3, np.nan]) da.mean() `` If you have precision problems - that might be because you havefloat32` values.

I don't know what goes wrong with your lon values - that is an issue in the reprojection. You could convert them to 0...360 by using

```python

lon_dim = "x" new_lon = np.mod(da[lon_dim], 360) da = da.assign_coords({lon_dim: new_lon}) da.reindex({lon_dim : np.sort(da[lon_dim])})

```

Yes but there are less - so as mentioned it removes all columns/ rows with only nans, if there is at least one non-nan value the row is kept.

What is the reason that you want to get rid of the nan values?

Can you try notnull instead of isnull - I often get the boolean array wrong in where:

python da = ds['z'] da = da.where(da.notnull(), drop=True)

What is the reason that you want to get rid of the nan values?

The reason they come back is that are needed to fill the grid again. The dataframe is 1D but the dataarray is 2D.

What you can try is to use where:

python da = ds['z'] da = da.where(da.isnull(), drop=True) but it will only drop the values if the entire row/ column is nan.

cc @spencerkclark @znicholls

I am closing this as an upstream issue - feel free to reopen if you disagree.

Thanks for looking into this!

Of course - I added it above.

I think this is 'fixed' since cartopy 0.20 (SciTools/cartopy#1646) and matplotlib 3.3 (matplotlib/matplotlib#18398) both of which are >= our minimum version requirements. So yes, I think it could be remove. However, that should not stop us from merging this PR I'd say.

I probably got this from a pandas issue/ pull request - but I can't remember where exactly - sorry...

Nice! (at one point we can consider removing infer_intervals as this is now also done in matplotlib and cartopy).

I think that's in the tests themselves

https://github.com/pydata/xarray/blob/6531b57f8c5cb7f3c564ff895c2e4b6573bb5521/xarray/tests/test_coding_times.py#L778

This should merge on green - thanks for your contribution @pierreloicq!

I think the problem is that there is no way to pin optional dependecies in conda. I understand your frustration but it's not practical to explicitly test for all of this in the code.

Thanks for the PR. Should that be summed over? The test failure should be unrelated.

Thanks for the PR. However, does that actually make a difference? To me it looks like _contains_cftime_datetimes also only considers one element of the array.

https://github.com/pydata/xarray/blob/b4515582ffc8b7f63632bfccd109d19889d00384/xarray/core/common.py#L1779-L1780

Good point. Combining two FacetGrids is a good use case. My intuition would be to allow passing exactly the correct number of axes and not the figure. But then I have not heard from subfigure before now and this may also be a valid option.

python f, axs = plt.subplots(2, 4) da1.plot(col='season', ax=axs[0, :]) da2.plot(col='season', ax=axs[1, :])

I think the usual workflow is to create the figure with gridspec and then update the created subplots and figure. Is there something that is not possible if done this way around?

Thanks @itcarroll and welcome to xarray!

Looks good. #7027 may be somewhat related.

You installed dask with pip so the following should get you there

bash python -m pip install --upgrade dask

However, your environment is a mix of packages installed with conda and pip - it's generally recommended to install everything with the same installer. Thus my suggestion would be to create a new environment and install everything with conda. You may also want to take a look at mamba - mamba is a drop-in replacement for conda that is (currently and for most cases) faster than conda, and thus more enjoyable to work with.

Thanks @mroeschke

Thanks @slevang!

@ChukwuwikeCephas I am closing this issue - feel free to reopen (or open a new one) if you have further questions.

NameError: name 'xr' is not defined

I had not realized that you did not import xarray yourself but this is done via plotly. It's usually done as import xarray as xr. So xr.show_versions() would also output some version info. But I could also get the info from your conda list output.

From this I can see the version of dask and xarray you have installed: dask 2021.1.0 pypi_0 pypi xarray 2022.6.0 pyhd8ed1ab_1 conda-forge

However, xarray version 2022.06 requires a newer dask version you have installed.

https://github.com/pydata/xarray/blob/4ad706fc4ef102c525555d55b20bc7ccc72d7045/ci/requirements/min-all-deps.yml#L18

So you can try to update your dask version.

Is that the whole output of xr.show_versions()?

Can you paste the output of conda list or pip freeze? Thanks!

bash mamba repoquery whoneeds netcdf4

but that requires an already solved environment

aiobotocore pins botocore as the versions must match (see aio-libs/aiobotocore#983). I opened #7326 to suppress the warning in our tests.

Thanks for the context @keewis - I like your suggestion & updated the PR. AFAIK we have not changed the ignored values in a very long time. I'll merge once the tests ran through.

Close/ reopen

I suspect there are floating point inaccuracies which can sometimes trip up xarray. But I would also not expect differences when the coords are created with the same piece of code.

Unfortunately your test does not work - it will first align and then do the subtraction... Any of the following should work:

python (d0.y - d1.y).shape == d0.shape d0.y.values == d1.y.values xr.align(d0, d1, join="exact")

If this worked earlier

Looks good - thanks! Should auto-merge on green.

Could it be that the coordinates of the individual arrays are not exactly the same?

For context: the group / groupstrconfusion may stem from our (quite) old docs, where the type hints were not rendered nicely: https://docs.xarray.dev/en/v0.12.3/generated/xarray.open_dataset.html

Pandas has a find_stack_level function (https://github.com/pandas-dev/pandas/blob/0106c26529900bad0561efb9c9180f7f016365b0/pandas/util/_exceptions.py#L28). But I am not sure if this would fix the problem?

Good catch - this should merge once all tests are green. Thanks @marekninja - welcome to xarray!

Close/ reopen to check if the docs now pass.

Thanks @keewis

Yes I mean the external ones. I agree that it is a good idea to disallow internal warnings (sometimes PRs introduce them without realizing...). And of course I also agree that we should act when then deprecation warnings are being thrown and not when the stuff is actually removed.

I am a bit skeptical - we test a large range of versions and the kinds of warnings we get can be quite different. I fear that we get too many false positives and end up wrapping everything with filterwarnings("ignore").

Thanks for the report. There was a recent release (2022.09) with many Index related bugfixes (including MuliIndex) - could you test this?

Nice work!

You'll have to uninstall bottleneck then - the use_bottleneck option was added more recently.

Do you have bottleneck installed? Could you try xr.set_options(use_bottleneck=False)?

See pydata/bottleneck#379

Indeed, this looks incomplete. As always - PR very welcome 😁

Thanks @arfriedman!

Close/ reopen

I'd opt for (1) & a comment on why it needs a separate module.

Maybe: python len(list(ax.get_shared_x_axes())) != 0

It would be nice to be able to preserve the MultiIndex with sel (e.g. ds.sel(one=["a"]) but if it makes the behavior inconsistent it is no good either...

Does this warrant a test? But good to go anyways.

For xr.dot it's also different:

python xr.dot(a, b, dims=None) # reduce over common dimensions xr.dot(a, b, dims=...) # reduce over all dimensions

Thanks!

Thanks - I missed #6992 & it indeed fixes the problems I had. I added this to the list of fixed issues in #6992.

Similarly, the following results in a wrong repr:

python import xarray as xr ds = xr.Dataset(data_vars={"foo": 1}, coords={"a": ("x", [1, 2, 3]), "b": ("x", ['a', 'b', 'c'])}) ds.set_index(z=['a', 'b']).reset_index("z", drop=True)

<xarray.Dataset> Dimensions: (z: 3) Coordinates: a (z) int64 1 2 3 b (z) <U1 'a' 'b' 'c' Data variables: *empty*

foo is missing.

@benbovy

Thanks for the report & I agree that this should lead to the same but the code paths are indeed different - but I have not looked in to the actual root cause. Could be that this is also not super thoroughly tested (and used!):

https://github.com/pydata/xarray/blob/b018442c8dfa3e71ec35e294de69e2011949afec/xarray/core/rolling.py#L289

https://github.com/pydata/xarray/blob/b018442c8dfa3e71ec35e294de69e2011949afec/xarray/core/rolling.py#L721

B.t.w. a copy-pastable example would be appreciated.

Interesting - did not think that this would work. Potential follow up - make all imports of dask lazy (#6726).

Let's see if that works now - should auto-merge on green.

Thanks for your contribution and welcome to xarray!

Thanks!

cc @dcherian

Ah - I probably don't have flox installed - that could be the reason.

Thanks @headtr1ck!

I cannot reproduce this (on master but I don't think we touched this since 2022.06):

python import xarray as xr import cf_xarray air = xr.tutorial.open_dataset("air_temperature") air.resample(time="MS").mean(dim="time").lat Returns <xarray.DataArray 'lat' (lat: 25)> array([75. , 72.5, 70. , 67.5, 65. , 62.5, 60. , 57.5, 55. , 52.5, 50. , 47.5, 45. , 42.5, 40. , 37.5, 35. , 32.5, 30. , 27.5, 25. , 22.5, 20. , 17.5, 15. ], dtype=float32) Coordinates: * lat (lat) float32 75.0 72.5 70.0 67.5 65.0 ... 25.0 22.5 20.0 17.5 15.0 Attributes: standard_name: latitude long_name: Latitude units: degrees_north axis: Y Would be interesting to get to the bottom of this.

(Funny enough - the attrs are not even dropped for r.mean(keep_attrs=False).lat.)

Ok, thanks, that solves the problem!

Great to hear!

Ok, I see there is another bugfix in v0.21.0 (#6118) - thus you will need at least this version for your example to work.

What version of xarray are you using? I think this should have been fixed in #5187 (i.e. in v0.18.0).

Edit: quickly checked and the newest version gets this right.

With #6971 on the way I guess we can close this.

Thanks @headtr1ck!

Closing in favour of #6984

Ok merging this. I think we got some to fix it if this ever causes trouble for a future version of python.

cc @headtr1ck

I could not get rid of RuntimeWarning: All-NaN slice encountered for tests with dask. Does anyone know why is that? pytest.mark.filterwarnings does not seem to capture them...

Yes, because we remove them in our assert* checks. Would be fixed with #6212 using somewhat non-public API. We should probably do it anyway...

Thanks!

Jup, that's always the tradeoff - #1613 discusses a similar case.

A smaller repro:

```python import numpy as np import xarray as xr

xr.DataArray(np.arange(5), dims="x", coords={"x": [2, 1, 0, 3, 5]})

da.sel(x=slice(2, 3))

```

Returns: <xarray.DataArray (x: 4)> array([0, 1, 2, 3]) Coordinates: * x (x) int64 2 4 5 3

python da.sel(x=[2, 3]) Returns <xarray.DataArray (x: 2)> array([0, 3]) Coordinates: * x (x) int64 2 3

Yes, good point. What might be possible is to warn if selecting with a slice and the index is not monotonic increasing or decreasing.

Good point, that can be surprising. I think a warning could be added.

python if not set(self.obj.dims) & set(self.weights.dims): warnings.warn("")

However, I wonder if that would be too noisy, e.g. if there is a scalar on the Dataset:

python xr.Dataset({"test_array": test_array, "scalar": 1}).weighted(weights2).mean()

(could potentially skip scalars in the check).

Edit:

https://github.com/pydata/xarray/blob/790a444b11c244fd2d33e2d2484a590f8fc000ff/xarray/core/weighted.py#L192

may not be the right place, as it checks it on the Dataset level (and not on the DataArray level).

Should be uncontroversial - so I am merging this as well.

I am merging this - will revert this if pydap publishes a new version on pypi.

Turns out pydap 3.3 (which fixes this problem) is released on conda but not on pypi - I opened an issue on this pydap/pydap#268. However, I have no idea if & when they will fix this - so I suggest to merge this anyway & I can undo this change once it is fixed upstream.

Turns out pydap 3.3 (which fixes this problem) is released on conda but not on pypi - I opened an issue on this pydap/pydap#268 & also opened #6965 to work around this in the meantime

Thanks! This should merge on green. Also, I see that this is your first PR here - welcome to xarray!

I think we could not install pydap if for python versions 3.10 and up:

diff -pydap +pydap;python_version<"3.10" in https://github.com/pydata/xarray/blob/3d3b236df3f8f5982760e68e064eb0db2aacc4a2/setup.cfg#L87

It does work if the array keeps the size:

python data.resample(index="M").apply(lambda x: x.values)

As a workaround you could allow your function to consume a dummy axis. Or you could pass dim as ...

python data.resample(index="M").reduce(lambda x, axis: 1) # workaround 1 data.resample(index="M").reduce(lambda x: 1, dim=...) # workaround 2

(reduce only passes axis if dim is not None but groupby passes the group_dim per default.

apply seems to expect that the applied function returns a DataArray. As an alternative could you use data.resample(index="M").reduce(np.median)?

This looks good - but from staring at the code it seems to me the logic is the same as before?

Thanks for noticing.

Any objections to merge this as is?

I also just realized, that we made many arguments keyword-only without deprecation (e.g. xr.Dataset.mean) - but that should not stop us from using this - there are still a lot of instances where it could be helpful.

I just realized that there is a bug in the decorator for "keyword-only arguments without a default". I wonder if I should fix this or just disallow this pattern, i.e. the following does currently not work properly:

python @_deprecate_positional_args("v0.1.0") def func(a, *, b): pass

but I am not sure how helpful it is.

EDIT: It now raises a TypeError for this case.

Are the functions you are considering using this functions that never had keyword arguments before? When I wrote a similar decorator before i had an explicit list of arguments that were allowed to be converted.

No, I want to consider arguments that have default values.

```python def mean(dim, skipna=None, keep_attrs=None): pass

@_deprecate_positional_args("v0.1.0") def mean(dim, *, skipna=None, keep_attrs=None): pass ```

Or do I missunderstand your question?

Do you think that you can get it to work with static typing?

I think functools.wraps also attaches the annotations?

I just opened #6910 - feedback welcome.

These decorators are kinda fun to write and are quite taylored to a certain release philosophy. It might be warranted to just write your own ;)

I am glad that's fine for you.

I just realized that @hmaarrfk's decorator works similarly, but it also has some bells and whistles we probably don't need..?

I just discovered the scikit-learns decorator to deprecate positional args - see https://github.com/scikit-learn/scikit-learn/pull/13311 - and now have a preference for this one. It seems the simplest and does not need any input as it reads the keyword-only args from the signature (could potentially add the version of the deprecation).

python @_deprecate_positional_args def f1(a, b, *, c=1, d=1): pass

and at the end of the deprecation period you only need to do

diff -@_deprecate_positional_args def f1(a, b, *, c=1, d=1): pass

I just had another look at this using bash python -X importtime -c "import llvmlite" 2> import.log and tuna for the visualization.

• pseudoNETCDF adds quite some overhead, but I think only few people have this installed (could be made faster, but not sure if worth it)
• llmvlite (required by numba) seems the last dependency relying on pkg_resources but this is fixed in the new version which should be out soonish
• dask recently merged a PR that avoids a slow import dask/dask/pull/9230 (from which we should profit)

This should bring it down a bit by another 0.25 s, but I agree it would be nice to have it even lower.

Yes good point - just calling compute may be the better solution.

Thanks for your report. I think the issue is that dask cannot correctly infer the dtype of the result - or at least not it's length (maybe because it does not do value-based casting? not sure).

As a workaround you could do an intermediate cast to an object:

python dac.astype(object).str.cat(dac, sep='--').astype("U").compute()

Thanks @DimitriPapadopoulos - for reference #6316.

Thanks @jsignell!

I renamed the fixtures back to da & ds to make the PR smaller & will merge once the checks are green...