With the new changes, this will now conflict with #1128, though easy to solve.

Travis succeeds, though lots of failures under environments with allowed failure. They look unrelated to me, but I find it hard to tell. Appveyor doesn't seem to run the quantities tests, so I guess the requirements there are missing too. Where would I add requirements for Appveyor?

You are right. There seem to be quite a number of varying requirements. Should I add it to all of them? Also, I'm not very well versed in Travis-CI .yml: What repo are the requirements served from? I think python-quantities is in debian. Or should I just go for pip? There hasn't been a release in a long time, github master has progressed quite a bit, but the released version should be compatible too.

Unfortunately, I was unable to come up with a good regression test. Interactive testing confirms that the fix is working (no iteration is performed, and the runtime of the example given in #1121 went down from ~1s to 0.3 us).

I think I found it (#1122). I guess whenever a non-scalar assignment is made (as in result[:] = [value]), something like np.asanyarray is performed on the new value. Luckily, numpy is perfectly happy with indexing of a 0d array with a 0d index (i.e. an empty tuple). Thinking about it, result[0] = value would probably have worked too.

Well, xarrays are way too useful not to nest them, even if that involves the scary dtype=object :-). Thanks for pointing me in the right direction, I'll try to find a fix.

This fix handles the case ds['somecoord'].rename({'somecoord':'newcoord'}). It does not apply to ds['somecoord'].rename('newcoord'), since the documentation of DataArray,rename does not mention coordinates at all if a string is given as argument. Nevertheless, one could argue the the two should be equivalent, in which case the latter would have to be fixed as well.

As for the consistency concern, I wouldn't have expected that to be a big issue. I'd argue that most functions mapping np.ndarray -> np.ndarray will not mind receiving a DataArray instead. On the other hand, functions mapping DataArray -> np.ndarray would seldom prefer to receive the raw np.ndarray. So I see no use to the raw parameter (but then again, I do not know pandas and their use-case), such that my hypotetical DataArray.apply and the existing DataArray.pipe are essentially the same.

Aha! For my use-case, DataArray.pipe is perfectly fine, I just didn't know about it. I have to admit that I know nothing about pandas. Before I learned about xarray, pandas was not interesting to me at all. My datasets are often high-dimensional which does not work well with pandas' orientation towards (one-dimensional) collections of observations. In that sense, I could rather relabel this issue (or create a new one) as a documentation problem. The API reference does not indicate the existence of DataArray.pipe at all (only Dataset.pipe, even though that one mentions it works on DataArrays too). Also, there could possibly be a see-also link to pipe from apply. Shall I have a go at a PR?

Really? I get a ValueError: Indexers must be 1 dimensional (xarray/core/dataset.py:1031 in isel_points(self, dim, **indexers) when I try. That is xarray 0.8.2, in fact from my fork recently cloned (~2-3 weeks ago), where I changed one or two asarray to asanyarray to work with units. Was there a recent change in this area? EDIT: xarray/core/dataset.py looks very similar also here on master, and there are quite a few lines hinting that really only 1D indexers are supported.

Without following the discussion in detail, what is the status here? In particular, I would like to do pointwise selection on multiple 1D coordinates using multidimensional indexer arrays. I can do this with the current isel_points: 1. construct the multidimensional indexers 2. flatten them 3. create a corresponding MultiIndex 4. apply the flattened indexers using isel_points, and assign the multi-index as the new dimension 5. use unstack on the newly created dimension The first three points can be somewhat simplified by instead putting all of the multidimensional indexer into a Dataset and then stack it to create consistent flat versions and their multi-index.

Given this conceptually easy but somewhat tedious procedure, couldn't that be something that could quite easily be implemented into the current isel_points? Would a PR along that direction have a chance of being accepted?

Or another way to put it: While typical metadata/attributes are only relevant if you eventually read them (which is where you will notice if they were lost on the way), units are different: They work silently behind the scene at all times, even if you do not explicitly look for them. You want an addition to fail if units don't match, without having to explicitly first test if the operands have units. So what should the ufunc_hook do if it finds two Variables that don't seem to carry units, raise an exception? Most probably not, as that would prevent to use xarray at the same time without units. So if the units are lost on the way, you might never notice, but end up with wrong data. To me, that is just not unlikely enough to happen given the damage it can do (e.g. the time it takes to find out what's going on once you realise you get wrong data).

So for now, I'm hunting for np.asarray.

988 would certainly allow to me to implement unit functionality on xarray, probably by leveraging an existing units package.

What I don't like with that approach is the fact that I essentially end up with a separate distinct implementation of units. I am afraid that I will either have to re-implement many of the helpers that I wrote to work with physical quantities to be xarray aware. Furthermore, one important aspect of units packages is that it prevents you from doing conversion mistakes. But that only works as long as you don't forget to carry the units with you. Having units just as attributes to xarray makes it as simple as forgetting to read the attributes when accessing the data to lose the units. The units inside xarray approach would have the advantage that whenever you end up accessing the data inside xarray, you automatically have the units with you. From a conceptual point of view, the units are really an integral part of the data, so they should sit right there with the data. Whenever you do something with the data, you have to deal with the units. That is true no matter if it is implemented as an attribute handler or directly on the data array. My fear is, attributes leave the impression of "optional" metadata which are too easily lost. E.g. xarray doesn't call it's ufunc_hook for some operation where it should, and you silently lose units. My hope is that with nested arrays that carry units, you would instead fail verbosely. Of course, np.concatenate is precisely one of these cases where unit packages struggle with to get their hook in (and where units on dtypes would help). So they fight the same problem. Nonetheless, these problems are known and solved as well as possible in the units packages, but in xarray, one would have to deal with them all over again.

+1 for units support. I agree, parametrised dtypes would be the preferred solution, but I don't want to wait that long (I would be willing to contribute to that end, but I'm afraid that would exceed my knowledge of numpy).

I have never used dask. I understand that the support for dask arrays is a central feature for xarray. However, the way I see it, if one would put a (unit-aware) ndarray subclass into an xarray, then units should work out of the box. As you discussed, this seems not so easy to make work together with dask (particularly in a generic way). However, shouldn't that be an issue that the dask community anyway has to solve (i.e.: currently there is no way to use any units package together with dask, right)? In that sense, allowing such arrays inside xarrays would force users to choose between dask and units, which is something they have to do anyway. But for a big part of users, that would be a very quick way to units!

Or am I missing something here? I'll just try to monkeypatch xarray to that end, and see how far I get...