issues

Table actions

3 rows where "created_at" is on date 2020-03-02 and user = 2448579 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date), closed_at (date)

type 1

state 1

repo 1

id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
573972855 MDExOlB1bGxSZXF1ZXN0MzgyMzgyODA5 3818 map_blocks: Allow passing dask-backed objects in args dcherian 2448579 closed 0     7 2020-03-02T13:26:12Z 2020-06-11T18:22:42Z 2020-06-07T16:13:35Z MEMBER   0 pydata/xarray/pulls/3818
  • [x] Tests added
  • [x] Passes isort -rc . && black . && mypy . && flake8
  • [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

It parses args and breaks any xarray objects into appropriate blocks before passing them on to the user function.

e.g. ```python da1 = xr.DataArray( np.ones((10, 20)), dims=["x", "y"], coords={"x": np.arange(10), "y": np.arange(20)} ).chunk({"x": 5, "y": 4}) da1

def sumda(da1, da2): #print(da1.shape) #print(da2.shape) return da1 - da2

da3 = (da1 + 1).isel(x=1, drop=True).rename({"y": "k"}) mapped = xr.map_blocks(sumda, da1, args=[da3]) xr.testing.assert_equal(da1-da3, mapped) # passes ```

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3818/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 pull
573768194 MDExOlB1bGxSZXF1ZXN0MzgyMjE1OTYw 3816 Add template xarray object kwarg to map_blocks dcherian 2448579 closed 0     9 2020-03-02T07:20:19Z 2020-05-06T16:41:59Z 2020-05-06T16:41:54Z MEMBER   0 pydata/xarray/pulls/3816
  • [x] Closes #3575
  • [x] Tests added
  • [x] Passes isort -rc . && black . && mypy . && flake8
  • [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

This PR adds a template kwarg to map_blocks so that we can do more complicated things where automated inference of the template fails. template is expected to be an xarray object that looks like the result of the map_blocks computation.

@jhamman To me, this seems a lot easier than defining a dict based schema. With dask variables, the memory cost shouldn't be high. It's easy to use standard xarray operations to make something that looks like the result dataset. Here's a notebook prototyping a to_schema/from_schema approach: https://gist.github.com/dcherian/130ba22d0fbadb616837deb914eaa67e#file-map_blocks_for_metsim_test-ipynb

python ds = xr.tutorial.load_dataset("air_temperature").chunk({"lat": 5}) template = ds.isel(lat=slice(1, None, 5)) result = xr.map_blocks(lambda x: x.isel(lat=[1]), ds) # fails result = xr.map_blocks(lambda x: x.isel(lat=[1]), ds, template=template) # yay xr.testing.assert_equal(result, template) # passes Todo: - [x] add tests - [x] what if the function returns a variable not in the template? - [x] should we do anything with template.attrs?

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3816/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 pull
573964634 MDExOlB1bGxSZXF1ZXN0MzgyMzc1OTcz 3817 map_blocks: allow user function to add new unindexed dimension. dcherian 2448579 closed 0     1 2020-03-02T13:12:25Z 2020-03-21T19:51:12Z 2020-03-21T19:51:07Z MEMBER   0 pydata/xarray/pulls/3817
  • [x] Tests added
  • [x] Passes isort -rc . && black . && mypy . && flake8
  • [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Small change that makes map_blocks apply functions that add new unindexed dimensions.

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3817/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 pull

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);