Skip to content

fix(renovate): move concurrency to job level - #35

Merged
simonostendorf merged 1 commit into
mainfrom
fix/renovate-concurrency
Nov 13, 2026
Merged

simonostendorf merged 1 commit into
mainfrom
fix/renovate-concurrency

Conversation

@simonostendorf

@simonostendorf simonostendorf commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

The finding, stated first because the symptom suggests the opposite

The anti-loop guard is correct and never executes. It is not broken and nothing about it changes here.

A workflow-level concurrency group is resolved when the event arrives, before any job's if: runs. So a run the guard was always going to skip gets queued anyway, and can then be cancelled while still pending. A cancelled run reports as a failed check on whatever pull request triggered it.

What was observed

One Renovate run rebasing several pull requests fires a burst of pull_request: edited events, every one of them sender-filtered by the guard. In a caller repository:

success    pull_request  20:02:44
cancelled  pull_request  20:02:48
cancelled  pull_request  20:03:23
...                                 ← ten of these
skipped    pull_request  20:04:25   ← the guard, once the burst subsided

Six Renovate pull requests were left with a red check that had nothing to do with their contents. The two skipped runs at the end are the guard working normally, which is what makes the cancelled ones look like a different problem.

The change

concurrency: moves from workflow level to job level, inside jobs.renovate. Same group, same cancel-in-progress: false. The if: is untouched.

At job level the if: is evaluated first, so a job that skips never joins the group and there is nothing to cancel.

On the evidence for that ordering

Worth being precise, because the whole fix rests on it.

GitHub's documentation does not state outright that a job's if: is evaluated before it joins its own concurrency group. What it does state is that jobs.<job_id>.concurrency may use the needs, strategy and matrix contexts, which workflow-level concurrency cannot. That asymmetry only makes sense if job-level concurrency is resolved when the job is about to start, after its dependency graph and condition are known.

The behavioural claim is corroborated by two independent sources rather than inferred from that alone: a GitHub community report describing exactly this cancellation pattern and its resolution, and a Gitea pull request titled "evaluate job-level if: before concurrency check", written specifically to match GitHub's real semantics, whose summary notes that skipped jobs no longer occupy concurrency slots.

So: confirmed by cross-implementation evidence, not by a verbatim documentation sentence. Flagging that rather than overstating it.

Alternatives rejected

A per-pull-request group. It stops the pile-up, but Renovate processes the whole repository on every run, not just the triggering pull request. Two runs admitted to different groups are two concurrent Renovate processes against one repository, risking conflicting rebases and force-pushes. The anti-loop guard does nothing about this: it decides whether one run's job executes, never whether two already-admitted runs may run at once. That trades a cosmetic failure for a correctness problem.

Dropping the pull_request: edited trigger from callers. It removes the rebase checkbox, which is a mechanism people use, and it would mean editing every caller rather than fixing the cause here.

What still holds

  • A scheduled run and a manual dispatch still cannot run concurrently for the same repository: same group key, same serialisation.
  • A bot-triggered edit is reported skipped, not cancelled or failure.
  • A human ticking the rebase checkbox still triggers a real run.
  • The anti-loop guard's comment block is updated to say where the ordering guarantee now lives, rather than left describing the old shape.

The other reusable workflows

Checked all of them. Only this one declared workflow-level concurrency. The others declare none.

Three caller workflows in this repository do declare it, and none has the identical defect: two already group per pull request or ref, and the release workflow groups by workflow name but its triggers cannot produce a burst across many pull requests, since one release pull request exists at a time. Left untouched and flagged rather than changed.

Verification

task lint:actions and task lint:yaml clean. task check passes in full, including the shell test suite.

The workflow itself was not exercised. A reusable workflow cannot be triggered standalone, so the fix is proven only once a caller runs it. The first Renovate burst after this merges is the real test: the runs it would previously have cancelled should report skipped.

Blast radius

Every repository in the organisation consumes this workflow, so this changes all of them at once. That is the intent: the defect is in the shared file, and patching symptoms per repository would leave the cause in place.


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

A workflow level concurrency group queues every triggering event before
any job's if runs. One Renovate run rebasing several pull requests
fires a burst of pull_request edited events, all of them ones the
anti-loop guard would have skipped. They pile into that queue, get
cancelled while merely pending, and report as a failed check on each
rebased pull request.

The guard is correct and never executes. That is the whole defect, and
the opposite of what the symptom suggests.

At job level the if runs first, so a run that was always going to be
skipped is reported skipped rather than cancelled.

A per pull request group was rejected: Renovate processes the whole
repository on every run, so two runs admitted to different groups are
two concurrent Renovate processes against one repository, which the
guard does nothing to prevent. Dropping the pull_request trigger was
rejected because it costs the rebase checkbox.

Signed-off-by: Simon Ostendorf <github@simon-ostendorf.de>
@simonostendorf

Copy link
Copy Markdown
Member Author

Verdict: approve

Judgement on the evidence question

The ordering claim is strong enough to ship, though not from a verbatim GitHub sentence, exactly as the PR says.

Verified independently:

  • GitHub's own docs confirm the cited asymmetry: jobs.<job_id>.concurrency may read needs, strategy and matrix; concurrency: at workflow level may only read github, inputs and vars (docs.github.com/en/actions/using-jobs/using-concurrency). A job-level group key can only be resolved once that job's matrix cell and dependency graph exist, which is logically downstream of its own if:.
  • The Gitea PR exists and says what the author claims: fix(actions): evaluate job-level if: before concurrency check go-gitea/gitea#39437, "evaluate job-level if: before concurrency check", explicitly framed as matching GitHub Actions' real semantics ("Gitea decides all jobs' if: expressions... so a skipped job no longer takes part in job concurrency or holds a max-parallel slot"). That is a second engineering team reverse-engineering GitHub's real behaviour for compatibility, independent of this PR's author.
  • GitHub's docs separately confirm the mechanism the bug depends on: "any existing pending job or workflow in the same concurrency group will be canceled" when a new run joins, verified directly, not assumed.
  • I could not locate a GitHub community thread that is a verbatim match for "if-skip at workflow level cancels a pending run." I found adjacent ones (orgs/community#77942, stuck required checks from a burst of pull_request events under workflow-level concurrency; orgs/community#32376, confirming only the newest pending run survives in a group) that corroborate pieces of the mechanism but not the exact described report. Flagging this as unverified rather than confirmed.
  • I did not find a counter-example or a documentation sentence contradicting the ordering claim.

Net: two of the three cited sources check out as described; the third (the community report) is plausible but not independently confirmed. The change is also safe on the downside: it cannot make a genuinely-triggered run behave differently (same group key, same cancel-in-progress: false, if: byte-for-byte unchanged), it only changes what happens to a run the guard was always going to skip. Worst case if the ordering claim turned out wrong is "no improvement," not a regression. That asymmetry is itself a reason this is shippable even given the residual documentation gap.

Findings

Blocker: none.

Should: none.

Nit:

  • .github/workflows/release.yaml:11-13: the PR body's claim "none has the identical defect... cannot produce a burst across many pull requests, since one release pull request exists at a time" is true as stated but slightly undersells the residual shape. release.yaml still pairs a filtering if: (label name must be in the allowed list) with bare workflow-level concurrency: { group: ${{ github.workflow }} }, no PR/ref scoping. A non-matching labeled/unlabeled event on the one open release PR would still join the group before if: runs and could cancel a genuinely pending release run, or itself report a spurious cancelled/failed check, same mechanism, just bounded to one PR instead of many. Low likelihood today (no auto-labeler in this repository; release-me only applies the five release labels), so not a blocker for this PR, but worth its own follow-up issue rather than only "flagged."
  • PR body, "On the evidence for that ordering" section: the specific community-report citation could not be reproduced by search (see judgement above). Not a reason to hold the PR, since the two other sources plus the mechanism GitHub documents independently are sufficient, but the claim as written reads more confirmed than what I could verify.

Other checks

  • Anti-loop if: block (lines 142-151): byte-identical to the pre-change condition, confirmed from the diff. Properties hold: schedule/dispatch still share one group and serialise; a bot-sender edit still evaluates if: false and reports skipped; a human's rebase-checkbox edit still satisfies if: and still joins the same group as before.
  • Comment blocks: the "concurrency comment below" cross-reference at line 90 was correctly updated from "above" to "below" to match the new position. No stale comment describing the old workflow-level shape found.
  • Other reusable workflows (reusable-codeql.yaml, reusable-dco.yaml, reusable-license.yaml, reusable-renovate-config.yaml): grepped, confirmed none declares concurrency: at all, matching the PR body.
  • Caller workflows: checks.yaml groups per github.ref (cancel-in-progress only for pull_request), dco.yaml groups per PR number (cancel-in-progress true), both PR/ref-scoped, confirmed to differ from the bug's bare-workflow-name shape. release.yaml is the one that shares the bare-name shape; see the nit above.
  • Renovate's "whole repository per run" premise behind rejecting the per-pull-request group: RENOVATE_AUTODISCOVER: 'true' and the file's own header comment confirm autodiscovery drives this, consistent with Renovate's well-known architecture (one run processes every matching branch/PR of the discovered repository together). I did not independently fetch Renovate's own docs to re-confirm this in isolation; treating it as consistent with established tool behaviour rather than freshly verified here.

Verification run locally (worktree, head acaff115)

  • task lint:actions, clean, exit 0.
  • task lint:yaml, clean, exit 0.
  • task check, full pass: actionlint, yamllint, markdownlint, shellcheck, license, gitleaks (no leaks), shell test suite (18/18), exit 0.
  • gh pr checks 35, all 12 applicable checks pass; Call reusable-renovate / Self-hosted Renovate reports skipping, consistent with this PR's own trigger not matching the anti-loop guard. This does not exercise the fix itself: a reusable workflow cannot be triggered standalone, and this skip happened with nothing else pending in the group, so it is not a test of the cancellation scenario either way.

Not verified / out of scope

  • The fix itself is unexercised, as the PR body already states: no caller run here can reproduce the burst-of-skipped-runs scenario. The real test is the first Renovate rebase burst after merge, the runs that previously cancelled should report skipped instead.
  • Renovate's own documentation on autodiscover's per-repository batching was not independently fetched this session (see above).

Baseline feedback

None.

@simonostendorf

Copy link
Copy Markdown
Member Author

Correcting my own evidence claim in the description, and recording the argument that actually settles this.

One of the three cited sources does not hold up. The review could not locate the GitHub community report of this exact cancellation pattern; adjacent discussions support parts of the mechanism but not a match. I should not have cited it as corroboration without reproducing it. The remaining two stand: GitHub's documented context asymmetry for job-level concurrency, and the Gitea pull request written explicitly to match GitHub's semantics, which is real and says what I said it says.

The stronger argument is one I did not make, and it is the review's: this change is asymmetric-safe. The group key, the cancel-in-progress setting and the if: are byte-identical, so nothing can change for a run that genuinely triggers. If the ordering claim is wrong, the worst case is no improvement, never a regression. That is why this is worth merging on imperfect evidence, rather than the citations being conclusive.

The real test remains the first Renovate burst after merge: runs that would previously have been cancelled should report skipped. I will check that rather than assume it.

Noted the nit on release.yaml. It shares the bare workflow-name group paired with a filtering if:, so it carries a smaller single-pull-request version of the same shape. No auto-labeler exists in this repository today, so nothing triggers it, and I have deliberately left it alone rather than widening this change's blast radius. Worth its own issue if a labeler ever appears.

@simonostendorf
simonostendorf merged commit 43ed50c into main Nov 13, 2026
13 checks passed
@simonostendorf
simonostendorf deleted the fix/renovate-concurrency branch November 13, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant