html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/4574#issuecomment-727068025,https://api.github.com/repos/pydata/xarray/issues/4574,727068025,MDEyOklzc3VlQ29tbWVudDcyNzA2ODAyNQ==,14808389,2020-11-13T22:32:25Z,2020-11-13T22:49:49Z,MEMBER,"yeah, that makes the CI somewhat less useful. `allow_failure` is used for silencing the error code of `pytest`: https://github.com/pydata/xarray/blob/dd9fe2a8a414ddefa3b04b934163c9ccc628c5c7/ci/azure/unit-tests.yml#L14-L18

Not sure if that would make a difference, but maybe we should try using [`continueOnError`](https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema&viewFallbackFrom=vsts#job) instead?

Edit: we could also try to print a warning if the CI fails:
```yaml
- bash: |
    $(environment_variables) pytest \
    --junitxml=junit/test-results.xml \
    --cov=xarray \
    --cov-report=xml \
    $(pytest_extra_flags) \
    || ([ ""$ALLOW_FAILURE"" = ""true"" ] && echo -e ""\043#vso[task.logissue type=warning;] ignored CI failure!!"")
```
see [docs](https://docs.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=bash) for the format of these messages. It seems they are supposed to be visible in the status report of github.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,741115905
https://github.com/pydata/xarray/issues/4574#issuecomment-725701318,https://api.github.com/repos/pydata/xarray/issues/4574,725701318,MDEyOklzc3VlQ29tbWVudDcyNTcwMTMxOA==,14808389,2020-11-11T22:40:52Z,2020-11-11T22:40:52Z,MEMBER,"I think github actions have the option to send notifications for failed CI, so those who have that enabled would have to open an issue. Ideally, however, the failed action would automatically open a issue with a generic title (something like ""nightly upstream-dev CI failed"") and the build log (maybe trimmed to just the error logs?). Not sure if that is possible right now, though.

When thinking about this after we ended the call, I realized we would actually be losing a bit of coverage: what happens if we remove the upstream-dev CI for PRs and a PR introduces changes incompatible with upstream-dev? We would definitely catch that using the nightly CI, but only *after* merging.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,741115905