issue_comments

Thanks @DangoMelon ! I know this PR was a long road, but it's a material improvement to the ergonomics of xarray. We'd enjoy having you as a contributor in the future!

Test failure is the same as on master, ref https://github.com/pydata/xarray/issues/3265

Great, looking good @DangoMelon - I merged master to resolve a conflict - then we can get this in!

Great, will merge later unless other comments. Will be good to get this in!

Great - will merge unless anyone has final comments?

Yes, I think it would make sense to add an option to is_scalar() to indicate whether or not 0-d arrays should be considered "scalars"

On Thu, Aug 8, 2019 at 6:44 AM Gerardo Rivera notifications@github.com wrote:

That's a good point. I think in this case, given that it's passed to an arg expected an array, we should raise on 0d.

I was expecting to rely on the current implementation of is_scalar to do the type checking since I'm moving _check_data_shape above as_compatible_data to do something like this

if utils.is_scalar(data) and coords is not None:

Otherwise everything would be filter out since as_compatible_data returns a 0d given a scalar value.

https://github.com/pydata/xarray/blob/8d46bf09f20e022baca98b4050584d93c0ea118b/xarray/core/variable.py#L195-L196

I can only imagine copying is_scalar but removing getattr(value, 'ndim', None) == 0 to filter out the 0d to only do the duplication on scalars.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/3159?email_source=notifications&email_token=AAJJFVQCR5GRKIH4BR5UIGLQDQPLPA5CNFSM4IGMSZ7KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD33U55A#issuecomment-519524084, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJJFVSSFNOMIWO2TDXWSJLQDQPLPANCNFSM4IGMSZ7A .

Should I treat them like multi-dimensional arrays or leave the current behavior for consistency with the snippet above?

That's a good point. I think in this case, given that it's passed to an arg expected an array, we should raise on 0d. I realize that's a bit inconsistent with treating them as scalars elsewhere.

Happy to be outvoted if others disagree

If the default value is NaN, we could reuse xarray's pre-existing sentinel value for NA: https://github.com/pydata/xarray/blob/55593a8bcaf2edb79034507990eac9c55b41a07d/xarray/core/dtypes.py#L8

1. Fill the unspecified array with np.nan, to indicate invalid values.

👍

If users want an empty DataArray, make them omit the argument entirely, e.g., xr.DataArray(coord=coords, dims=dims).

👍

...so for clarity, no need to expose the sentinel values externally, they're an internal implementation that then fills np.nans

• Use a scalar array

This is the case that I'm not sure we want to support.

I think the rule we want is something like "scalar values are repeated automatically," but 0-dimensional arrays are kind of a strange case -- are they really scalars or multi-dimensional arrays? My inclination is to treat these like multi-dimensional arrays, in which case we should raise an error to avoid hiding errors.

In particular, one thing that an xarray user might expect, but which I think don't want to support, is full broadcasting of multi-dimensional arrays to match the shape of coordinates.

• Use None to get an empty array

Rather than using None, I would suggest using a custom sentinel value. Somebody might actually want an array full of all None values! If users want an empty DataArray, make them omit the argument entirely, e.g., xr.DataArray(coord=coords, dims=dims).

The way we do this in xarray is with a ReprObject, e.g., see here for apply_ufunc: https://github.com/pydata/xarray/blob/1757dffac2fa493d7b9a074b84cf8c830a706688/xarray/core/computation.py#L26 https://github.com/pydata/xarray/blob/1757dffac2fa493d7b9a074b84cf8c830a706688/xarray/core/computation.py#L692

There is also the question of what values should be inside such an empty array. Here I think there are roughly two options: 1. Fill the unspecified array with np.nan, to indicate invalid values. 2. Just use np.empty, which means the array can be filled with arbitrary invalid data.

It looks like you've currently implemented option (2), but again I'm not sure that is the most sensible default behavior for xarray. The performance gains from not filling in array values with a constant are typically very small (writing constant values into memory is very fast). Pandas also seems to use NaN as the default value: ```

pandas.Series(index=[1, 2]) 1 NaN 2 NaN dtype: float64 ```

Great @DangoMelon , that looks superb!

Could you run those cases you listed above in test functions? You can copy and paste those lines, and then compare the resulting object to one constructed with the full array (let me know if this is unclear and I can give more guidance).

Could you also construct a test case using @shoyer 's example which fails? Again lmk if you need any guidance in how to do that.

In the final case "Use None", is that correct? Where do all the values come from? (Maybe I'm missing something basic?)

And then I think we can merge, pending any other feedback! Thanks - this will be a valuable addition to xarray.

Hi @DangoMelon, no stress, I know git can be a pain.

I don't remember your previous effort so I'm not sure what the previous version looked like. Is it materially different from what currently exists? The current code looks relevant and @shoyer has reviewed & commented on it.

If you have overwritten your previous commits, you can generally get them back by using git reflog. There's plenty of documentation on that online, let us know if we can be helpful in the specific case though