issues

Table actions

4 rows where comments = 11, repo = 13221727 and user = 35968931 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date), closed_at (date)

type 2

state 2

repo 1

  • xarray · 4 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
2119537681 PR_kwDOAMm_X85mE7Im 8711 Opt out of auto creating index variables TomNicholas 35968931 closed 0     11 2024-02-05T22:04:36Z 2024-03-26T13:55:16Z 2024-03-26T13:50:14Z MEMBER   0 pydata/xarray/pulls/8711

Tries fixing #8704 by cherry-picking from #8124 as @benbovy suggested in https://github.com/pydata/xarray/issues/8704#issuecomment-1926868422

  • [x] Closes #8704
  • [x] Tests added
  • [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
  • [ ] ~~New functions/methods are listed in api.rst~~
 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/8711/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 pull
1223270563 PR_kwDOAMm_X843L_J2 6566 New inline_array kwarg for open_dataset TomNicholas 35968931 closed 0     11 2022-05-02T19:39:07Z 2022-05-11T22:12:24Z 2022-05-11T20:26:43Z MEMBER   0 pydata/xarray/pulls/6566

Exposes the inline_array kwarg of dask.array.from_array in xr.open_dataset, and ds/da/variable.chunk.

What setting this to True does is inline the array into the opening/chunking task, which avoids an an extra array object at the start of the task graph. That's useful because the presence of that single common task connecting otherwise independent parts of the graph can confuse the graph optimizer.

With open_dataset(..., inline_array=False):

With open_dataset(..., inline_array=True):

In our case (xGCM) this is important because once inlined the optimizer understands that all the remaining parts of the graph are embarrasingly-parallel, and realizes that it can fuze all our chunk-wise padding tasks into one padding task per chunk.

I think this option could help in any case where someone is opening data from a Zarr store (the reason we had this opener task) or a netCDF file.

The value of the kwarg should be kept optional because in theory inlining is a tradeoff between fewer tasks and more memory use, but I think there might be a case for setting the default to be True?

Questions: 1) How should I test this? 2) Should it default to False or True? 3) inline_array or inline? (inline_array doesn't really make sense for open_dataset, which creates multiple arrays)

  • [x] Closes #1895
  • [x] Tests added
  • [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

@rabernat @jbusecke 

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/6566/reactions",
    "total_count": 3,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 1,
    "confused": 0,
    "heart": 0,
    "rocket": 2,
    "eyes": 0
}
     xarray 13221727 pull
1042652334 I_kwDOAMm_X84-JZyu 5927 Release frequency TomNicholas 35968931 open 0     11 2021-11-02T17:53:57Z 2021-11-05T17:12:42Z   MEMBER      

In issuing the last 2 xarray releases, I've noticed a pattern, that goes something like this: 1) We don't have a release for 3+ months, for no particular reason. 2) Someone realises they want a release, to fix a bug or make a new feature available. 3) That person announces that they would like a release. 4) Lots of people (myself especially) suggest all sorts of unfinished issues that they think could or should go into that next release. 5) The dev team end up spending the better part of a week trying to finish up all of these miscellaneous PRs. 6) Finally it is deemed "ready" in some fairly arbitrary way. 7) The release is made manually using the "16 easy steps". 8) No-one wants to think about releasing again for another 3 months...

Frequency

I mentioned this to @rabernat and he suggested that we should be releasing much more frequently.

If we released more regularly then we wouldn't have this effect of "oh and we should try to squeeze XYZ into this release".

I think the majority of the time xarray's CI is passing, and even when it's not it's only 1 tiny fix away from passing. That means that we in theory could release the main branch at practically any time, and it would be perfectly stable for users. (I personally exclusively use the most recent version of main.)

I also don't know of any downside to releasing very regularly (other than that someone has to issue the release).

How about we try to release after each of the bi-weekly dev calls? We could make it an official part of the call to end by saying: - "any reason why we can't release right now?" - "no, CI is passing" - "okay [person] volunteers to click the button right after this meeting"

That would immediately increase our release frequency by up to 6x.

Automation

Can we automate any more steps of our release process? As far as I can tell the only steps that really need human intervention are - "write the release summary" and - "check that all the automated stuff went as expected".

We could potentially still automate - "add new section to the whats-new.rst", - "update the stable branch", - "update the active version of the docs" (maybe?), and - "email various mailing lists".

@pydata/xarray thoughts?

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/5927/reactions",
    "total_count": 4,
    "+1": 4,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 issue
400678645 MDExOlB1bGxSZXF1ZXN0MjQ1ODA4Nzg3 2690 Add create_test_data to public testing API TomNicholas 35968931 closed 0     11 2019-01-18T11:08:01Z 2021-06-24T08:51:36Z 2021-06-23T16:14:28Z MEMBER   0 pydata/xarray/pulls/2690
  • [x] Closes #2686 and #1839
  • [x] Fully documented, including whats-new.rst for all changes and api.rst for new API
 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/2690/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 pull

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);