issue_comments

Table actions

2 rows where issue = 537772490 and user = 2448579 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • dcherian · 2 ✖

issue 1

  • Idea: functionally-derived non-dimensional coordinates · 2 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
565671981 https://github.com/pydata/xarray/issues/3620#issuecomment-565671981 https://api.github.com/repos/pydata/xarray/issues/3620 MDEyOklzc3VlQ29tbWVudDU2NTY3MTk4MQ== dcherian 2448579 2019-12-14T02:18:15Z 2019-12-14T02:19:17Z MEMBER

I should have said "discrete lazily evaluated form (which we support through dask)". I think we already have what you want in principle (caveats at the end).

Here's an example: ``` python import dask import numpy as np import xarray as xr

xr.set_options(display_style="html")

def arbitrary_function(dataset): return dataset["a"] * dataset["wavelength"] * dataset.attrs["wcs_param"]

ds = xr.Dataset()

construct a dask array.

In practice this could represent an on-disk dataset,

with data reads only occurring when necessary

ds["a"] = xr.DataArray(dask.array.ones((10,)), dims=["wavelength"], coords={"wavelength": np.arange(10)})

some coordinate system parameter

ds.attrs["wcs_param"] = 1.0

complicated pixel to world function

no compute happens since we are working with dask arrays

so this is quite cheap.

ds.coords["azimuth"] = arbitrary_function(ds) ds ```

So you can carry around your coordinate system parameters in the .attrs dictionary and the non-dimensional coordinate azimuth is only evaluated when needed e.g. when plotting ``` python

Both 'a' and 'azimuth' are computed now, since actual values are required to plot

ds.a.plot(x="azimuth") ```

In practice, there are a few limitations. @djhoese and @snowman2 may have useful perspective here.

  1. xarray tends to compute "non-dimensional coordinates" more than necessary. The more egregious examples have been fixed (#3068, #3311, #3454, #3453) but there may still be some places where fixes are needed (#3588).
  2. there's some discussion about carrying around Earth-specific coordinate system parameters here: https://github.com/pydata/xarray/issues/2288; https://github.com/pydata/xarray/issues/2996.

Additional info: 1. https://docs.dask.org/en/latest/array.html 2. https://xarray.pydata.org/en/stable/dask.html 3. https://blog.dask.org/2019/06/20/load-image-data

PS: If it helps, I'd be happy to chat over skype for a half hour getting you oriented with how we do things.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Idea: functionally-derived non-dimensional coordinates 537772490
565625792 https://github.com/pydata/xarray/issues/3620#issuecomment-565625792 https://api.github.com/repos/pydata/xarray/issues/3620 MDEyOklzc3VlQ29tbWVudDU2NTYyNTc5Mg== dcherian 2448579 2019-12-13T22:05:07Z 2019-12-13T22:05:07Z MEMBER

It would also be good to hear about "sub-pixel metadata" → this seems to be the main reason why you want to carry around the analytic rather than the discrete evaluated form (which we basically support through dask). Is that right or am I missing something? 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Idea: functionally-derived non-dimensional coordinates 537772490

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);