home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 541179896

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/3381#issuecomment-541179896 https://api.github.com/repos/pydata/xarray/issues/3381 541179896 MDEyOklzc3VlQ29tbWVudDU0MTE3OTg5Ng== 1217238 2019-10-11T18:44:02Z 2019-10-11T18:44:02Z MEMBER

OK, thanks for clarify with that example.

I think we can track down the issue to the result of b.sum(dim='foo'): ```

b.data <COO: shape=(3, 6), dtype=float64, nnz=18, fill_value=nan> b.sum(dim='foo').data <COO: shape=(6,), dtype=float64, nnz=6, fill_value=0.0> ```

The fill value here is actually arbitrary, since the array is entirely dense. If this fill value were still nan, the later operation combining these arrays would work.

That said, sparse is making a reasonable choice here: nansum() applied to an array with all values given by nan is 0. Unless sparse wants to add special logic for handling arrays with different sparsities, I don't know how they could change this.

Options for dealing with this: - Use .mean() instead of .sum(). - Explicitly convert c into a dense array before combining it, e.g., d = xr.concat([b, c.copy(data=c.data.todense())], dim='foo') (syntax could be better). But this currently errors with: ValueError: All arrays must be instances of SparseArray. from sparse. Maybe sparse's concatenate could be updated to handle the mixed ndarray/sparse case? - Add some ergonomic way to explicitly override fill_value on sparse data in xarray, e.g., b.sum(dim='foo').with_fill_value(np.nan). - In principle, b.sum(dim='foo', min_count=1) could return a sparse array with fill_value=nan, but currently it doesn't.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  503711327
Powered by Datasette · Queries took 0.569ms · About: xarray-datasette