Repository navigation
Fail CI retry chains on the job that actually failed - #59570
Open
GijsWeterings wants to merge 1 commit into
Open
GijsWeterings wants to merge 1 commit into
GijsWeterings wants to merge 1 commit into
Conversation
|
@GijsWeterings has exported this pull request. If you are a Meta employee, you can view the originating Diff in D128043004. |
GijsWeterings
force-pushed
the
export-D128043004
branch
3 times, most recently
from
October 28, 2026 13:25
4e79d81 to
e682a84
Compare
Summary:
The E2E and Fantom retry chains reported their final failure through a separate `report` job. The `test` job always succeeded (its test step used `continue-on-error: true`), and `report` read its `status` output and exited 1 on the last attempt. This caused three problems:
- The red check was a five-second status job, not the job that ran the tests, so finding the real failure meant opening a green job and looking for the step with a warning icon.
- "Re-run failed jobs" re-ran only `report`. It read the previous attempt's cached `status` output and failed again without running any tests.
- The Debug and Release matrix legs wrote the same `status` output, and GitHub keeps only the last value written to a given output name. If one flavor failed and the other passed and finished later, `status` came out empty, so no retry ran and the failure was dropped silently.
This change:
- Removes the `report` jobs. The test step runs as two copies of the same step, gated on `inputs.fail-on-error`: earlier attempts use `continue-on-error: true`, so they stay green and record the failure in the `status` output, and the final attempt runs without it, so the test step itself fails the job. The two copies share their `with:` block through a YAML anchor. (A single step with `continue-on-error: ${{ !inputs.fail-on-error }}` doesn't work: GitHub evaluated it as true on the final attempt, so the failure stayed hidden. Step `if:` conditions read `inputs` correctly.)
- Has each matrix leg write a flavor-specific output (`debug_status` / `release_status`). GitHub combines outputs with different names across matrix legs. The reusable workflow's `status` output is `debug_status || release_status`, or `jobs.test.outputs.status` for the single-job Fantom workflow.
- Also triggers the retry jobs in `test-all.yml` on `needs.<attempt>.result == 'failure'`. The iOS `report` jobs counted a failed `build` or `test` job as a failure, and this keeps that behavior for every workflow.
The flavor is lowercased with `tr` rather than `${FLAVOR,,}` because the macOS runners' default `bash` may be 3.2. A flavor with no matching output fails the step with an error instead of writing an output nothing reads; a red `test` job still triggers the retry through `needs.<attempt>.result`.
Changelog: [Internal]
Differential Revision: D128043004
GijsWeterings
force-pushed
the
export-D128043004
branch
3 times, most recently
from
October 28, 2026 15:59
043a531 to
7da2f7a
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
The E2E and Fantom retry chains reported their final failure through a separate
reportjob. Thetestjob always succeeded (its test step usedcontinue-on-error: true), andreportread itsstatusoutput and exited 1 on the last attempt. This caused three problems:report. It read the previous attempt's cachedstatusoutput and failed again without running any tests.statusoutput, and GitHub keeps only the last value written to a given output name. If one flavor failed and the other passed and finished later,statuscame out empty, so no retry ran and the failure was dropped silently.This change:
reportjobs. The test step runs as two copies of the same step, gated oninputs.fail-on-error: earlier attempts usecontinue-on-error: true, so they stay green and record the failure in thestatusoutput, and the final attempt runs without it, so the test step itself fails the job. The two copies share theirwith:block through a YAML anchor. (A single step withcontinue-on-error: ${{ !inputs.fail-on-error }}doesn't work: GitHub evaluated it as true on the final attempt, so the failure stayed hidden. Stepif:conditions readinputscorrectly.)debug_status/release_status). GitHub combines outputs with different names across matrix legs. The reusable workflow'sstatusoutput isdebug_status || release_status, orjobs.test.outputs.statusfor the single-job Fantom workflow.test-all.ymlonneeds.<attempt>.result == 'failure'. The iOSreportjobs counted a failedbuildortestjob as a failure, and this keeps that behavior for every workflow.The flavor is lowercased with
trrather than${FLAVOR,,}because the macOS runners' defaultbashmay be 3.2. A flavor with no matching output fails the step with an error instead of writing an output nothing reads; a redtestjob still triggers the retry throughneeds.<attempt>.result.Changelog: [Internal]
Differential Revision: D128043004