html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/3381#issuecomment-541179896,https://api.github.com/repos/pydata/xarray/issues/3381,541179896,MDEyOklzc3VlQ29tbWVudDU0MTE3OTg5Ng==,1217238,2019-10-11T18:44:02Z,2019-10-11T18:44:02Z,MEMBER,"OK, thanks for clarify with that example.
I think we can track down the issue to the result of `b.sum(dim='foo')`:
```
>>> b.data
>>> b.sum(dim='foo').data
```
The fill value here is actually arbitrary, since the array is entirely dense. If this fill value were still `nan`, the later operation combining these arrays would work.
That said, sparse is making a reasonable choice here: `nansum()` applied to an array with all values given by `nan` is `0`. Unless sparse wants to add special logic for handling arrays with different sparsities, I don't know how they could change this.
Options for dealing with this:
- Use `.mean()` instead of `.sum()`.
- Explicitly convert `c` into a dense array before combining it, e.g., `d = xr.concat([b, c.copy(data=c.data.todense())], dim='foo')` (syntax could be better). But this currently errors with: `ValueError: All arrays must be instances of SparseArray.` from sparse. Maybe sparse's concatenate could be updated to handle the mixed ndarray/sparse case?
- Add some ergonomic way to explicitly override `fill_value` on sparse data in xarray, e.g., `b.sum(dim='foo').with_fill_value(np.nan)`.
- In principle, `b.sum(dim='foo', min_count=1)` could return a sparse array with `fill_value=nan`, but currently it doesn't.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,503711327
https://github.com/pydata/xarray/issues/3381#issuecomment-541138033,https://api.github.com/repos/pydata/xarray/issues/3381,541138033,MDEyOklzc3VlQ29tbWVudDU0MTEzODAzMw==,1217238,2019-10-11T16:42:11Z,2019-10-11T16:42:11Z,MEMBER,"Sparse only lets you combine arrays with different fill values if the result would also have a fixed value. That's why you can multiply or add but not concatenate, e.g.,
```
assert x.fill_value == 1 and y.fill_value == 2
assert (x + y).fill_value == 3
assert (x * y).fill_value == 2
np.stack([x, y]) # error, would need a mixture of different fill values to represent
```
Multiple fill values simply aren't representable by sparse's data model.
I think you could work this by wrapping the sparse arrays in dask first.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,503711327