Skip to content

ci: standardize success()/failure()/cancelled() idioms in ci.yml - #1071

Draft
kbattocchi wants to merge 1 commit into
mainfrom
kebatt/fix-merge-lkg-ci-condition
Draft

kbattocchi wants to merge 1 commit into
mainfrom
kebatt/fix-merge-lkg-ci-condition

Conversation

@kbattocchi

@kbattocchi kbattocchi commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Problem

merge-lkg was incorrectly running (and failing) when an upstream test matrix entry failed (see, e.g., this run for PR #1067), even though its direct dependency generate-lkg was correctly skipped (LKG regeneration only runs on schedule/workflow_dispatch, not PRs).

Root cause: job-level success()/failure() walk the entire transitive ancestor graph of a job, not just its direct needs. A failure several jobs upstream can leak through a needs job that was itself cleanly skipped.

What changed

  • Fixed merge-lkg to gate on its direct dependency's result instead of the ad hoc success() || failure().
  • Audited every other success()/failure()/always()/cancelled() usage in ci.yml and standardized on two documented idioms:
    • Job-level: !cancelled() && (contains(needs.*.result, 'success') || contains(needs.*.result, 'failure')) — run if at least one direct dependency actually ran, regardless of transitive ancestors. This is an existence check, not an "all succeeded" check (documented explicitly in the file).
    • Step-level: always() && contains(fromJSON('["success", "failure"]'), steps.<id>.outcome) — run cleanup steps regardless of outcome, but only if the referenced step actually executed.
  • Fixed a second latent bug: coverage-report could be silently skipped on notebooks-only PRs, because success()/failure() can't distinguish "a sibling dependency was skipped" from "everything failed".
  • Removed a vacuous (success()||failure()) check on store-reqs-per-env, which has no needs: for it to evaluate.
  • Added a top-of-file comment documenting both patterns, their rationale, and their exact semantics, so future contributors have a consistent pattern to copy instead of reinventing ad hoc combinations.

No behavioral changes are intended for the common case (PRs where everything upstream succeeds); this only changes behavior when something upstream is skipped or fails.

@kbattocchi
kbattocchi force-pushed the kebatt/fix-merge-lkg-ci-condition branch from 43ee713 to ffcc183 Compare October 7, 2026 17:51
@kbattocchi
kbattocchi force-pushed the kebatt/fix-merge-lkg-ci-condition branch 3 times, most recently from 190730b to 9a37f95 Compare October 15, 2026 19:54
@kbattocchi

Copy link
Copy Markdown
Member Author

Reopening to retrigger CI (previous push's pull_request event appears to have been dropped due to concurrency-group contention).

@kbattocchi kbattocchi closed this Oct 16, 2026
@kbattocchi kbattocchi reopened this Oct 16, 2026
@kbattocchi
kbattocchi force-pushed the kebatt/fix-merge-lkg-ci-condition branch from 9a37f95 to 28124d3 Compare October 16, 2026 14:34
Jobs that should keep going after an upstream failure or skip now use a
single job-level pattern that names each direct dependency:

  if: ${{ !cancelled() && (needs.a.result != 'skipped' || ...) }}

This replaces `success() || failure()` and job-level `needs.*.result`
checks, both of which also consider transitive ancestors (see
actions/runner#1540). That caused merge-lkg to run (and fail) on PRs
and on re-runs even though its only dependency was skipped. Since the
decision lives entirely in the job's `if:`, no per-step conditions are
needed. generate-lkg also regains `!cancelled()`, so nightly LKG
generation is no longer skipped after an upstream test failure.

The recurring patterns and their rationale are documented at the top of
the file.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
kbattocchi force-pushed the kebatt/fix-merge-lkg-ci-condition branch from 717e1a6 to 1eb91c6 Compare October 19, 2026 17:42
@kbattocchi
kbattocchi added this pull request to stack #1075 October 19, 2026 17:53

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant