home / github / issues

Menu
  • Search all tables
  • GraphQL API

issues: 1180334986

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1180334986 I_kwDOAMm_X85GWnuK 6411 Better dask support in polyval 2448579 closed 0     0 2022-03-25T04:35:48Z 2022-05-05T20:17:07Z 2022-05-05T20:17:07Z MEMBER      

Is your feature request related to a problem?

polyval does not handle dask inputs well.

```python nt = 8772 // 4 ny = 489 nx = 655

chunks like the data is stored on disk

small in time, big in space

because the chunk sizes are -1 along lat, lon;

reshaping this array to (time, latlon) prior to fitting is pretty cheap

chunks = (8, -1, -1)

da = xr.DataArray( dask.array.random.random((nt, ny, nx), chunks=chunks), dims=("ocean_time", "eta_rho", "xi_rho"), )

dim = "ocean_time" deg = 1

p = da.polyfit(dim="ocean_time", deg=1, skipna=False)

create a chunked version of the "ocean_time" dimension

chunked_dim = xr.DataArray( dask.array.from_array(da[dim].data, chunks=da.chunksizes[dim]), dims=dim, name=dim ) xr.polyval(chunked_dim, p.polyfit_coefficients) ```

Describe the solution you'd like

Here's a partial solution. It does not handle datetime inputs (polyval handles this using get_clean_interp_index which computes dask inputs). But I've replaced the call to np.vander and used xr.dot.

```python def polyval(coord, coeffs, degree_dim="degree"): x = coord.data

deg_coord = coeffs[degree_dim]
N = int(deg_coord.max()) + 1

lhs = xr.DataArray(
    np.stack([x ** (N - 1 - i) for i in range(N)], axis=1),
    dims=(coord.name, degree_dim),
    coords={coord.name: coord, degree_dim: np.arange(deg_coord.max() + 1)[::-1]},
)
return xr.dot(lhs, coeffs, dims=degree_dim)

polyval(chunked_dim, p.polyfit_coefficients) ```

This looks like what I expected

cc @aulemahal

Describe alternatives you've considered

No response

Additional context

No response

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/6411/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 2 rows from issues_id in issues_labels
  • 0 rows from issue in issue_comments
Powered by Datasette · Queries took 1.376ms · About: xarray-datasette