home / github / issue_comments

Menu
  • Search all tables
  • GraphQL API

issue_comments: 565671981

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/3620#issuecomment-565671981 https://api.github.com/repos/pydata/xarray/issues/3620 565671981 MDEyOklzc3VlQ29tbWVudDU2NTY3MTk4MQ== 2448579 2019-12-14T02:18:15Z 2019-12-14T02:19:17Z MEMBER

I should have said "discrete lazily evaluated form (which we support through dask)". I think we already have what you want in principle (caveats at the end).

Here's an example: ``` python import dask import numpy as np import xarray as xr

xr.set_options(display_style="html")

def arbitrary_function(dataset): return dataset["a"] * dataset["wavelength"] * dataset.attrs["wcs_param"]

ds = xr.Dataset()

construct a dask array.

In practice this could represent an on-disk dataset,

with data reads only occurring when necessary

ds["a"] = xr.DataArray(dask.array.ones((10,)), dims=["wavelength"], coords={"wavelength": np.arange(10)})

some coordinate system parameter

ds.attrs["wcs_param"] = 1.0

complicated pixel to world function

no compute happens since we are working with dask arrays

so this is quite cheap.

ds.coords["azimuth"] = arbitrary_function(ds) ds ```

So you can carry around your coordinate system parameters in the .attrs dictionary and the non-dimensional coordinate azimuth is only evaluated when needed e.g. when plotting ``` python

Both 'a' and 'azimuth' are computed now, since actual values are required to plot

ds.a.plot(x="azimuth") ```

In practice, there are a few limitations. @djhoese and @snowman2 may have useful perspective here.

  1. xarray tends to compute "non-dimensional coordinates" more than necessary. The more egregious examples have been fixed (#3068, #3311, #3454, #3453) but there may still be some places where fixes are needed (#3588).
  2. there's some discussion about carrying around Earth-specific coordinate system parameters here: https://github.com/pydata/xarray/issues/2288; https://github.com/pydata/xarray/issues/2996.

Additional info: 1. https://docs.dask.org/en/latest/array.html 2. https://xarray.pydata.org/en/stable/dask.html 3. https://blog.dask.org/2019/06/20/load-image-data

PS: If it helps, I'd be happy to chat over skype for a half hour getting you oriented with how we do things.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  537772490
Powered by Datasette · Queries took 0.632ms · About: xarray-datasette