Skip to content

ci(repo): run the release gate on pull requests, scoped to the change - #78

Merged
kiro-systemf[bot] merged 2 commits into
mainfrom
xs/release-gate-on-prs
Oct 9, 2026
Merged

kiro-systemf[bot] merged 2 commits into
mainfrom
xs/release-gate-on-prs

Conversation

@systemfsoftware-maker

@systemfsoftware-maker systemfsoftware-maker commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Capability PRs never ran the mutation gate. #77 merged src/graph with 45 live mutants (31 Survived, 14 NoCoverage), and main's release gate (run 37922084024 on 3742392) found them only after the merge. This PR runs the same release gate on every pull request, scoped to what the pull request changes, so a capability PR's own run is the proof.

What the pull-request run mutates

The plan job lists the files the pull request changes: git diff --name-only HEAD^1 HEAD on the merge commit, which is the change the merge would make to its base. scripts/stryker-plan-gate.ts turns that list into a scope (scopeOf):

The change touches Mutated
the gate: release-gate.yml, .github/actions/**, scripts/stryker-plan-gate.ts, scripts/deno.json/deno.lock, stryker.shared.ts, root package.json, pnpm-lock.yaml, pnpm-workspace.yaml, flake.nix/flake.lock every package's whole declared set
only files of a package's declared mutate list just those files
any other file of a package: a test, a fixture, its Stryker or Vitest config, its manifest, a source file outside the set that package's whole declared set
nothing in a mutating package nothing; the plan is empty and verdict passes

The scope rule does not sort files into tests and modules by name (CONST-T12). Any change to a package file outside the declared set counts as a possible test of the set. The rule can therefore mutate more than a change needs, but never less. A source file outside the set may be one the mutated files import, and changing it can change their mutants' outcomes. That is also why such a change mutates the whole set.

The scope reaches Stryker as MUTATION_SCOPE, a comma list of project directories (whole set) and repo-relative files. stryker.shared.ts intersects it with each package's declared list. It resolves the package from the working directory, which is where Stryker loads the config and resolves mutate for both stryker plan and each shard child. When the variable is unset (a local run, or a push or dispatch run with no diff), the config keeps the whole declared set. On main the scope is every mutating project, so main's run is unchanged.

Gate and verdict

  • gatePlan takes the changed list. It still refuses an in-scope package that the plan scheduled nothing for, and a malformed stryker.config.ts (no mutate list) is a new refusal. A change that touches no mutating package passes with no plan and logs 0 mutants for <pkg>: the change touches none of its files. A push run with an empty plan and no ledger entry is still refused as vacuous.
  • The verdict job now needs: [plan, mutation] and runs if: !cancelled(). Its first step fails unless the plan job succeeded and, when there are shards, every shard succeeded. Before this change, a failed plan skipped mutation and so skipped verdict, and a skipped job reports as success. With no shards, the job passes and says that no mutated file is in scope.
  • Pull-request runs skip the incremental cache, so every scoped mutant runs fresh against the PR's tests. A newer push to the same pull request cancels the older run. pull_request has no branch filter, so a stacked layer whose base is another branch is gated too.

This PR's own run

This PR changes the gate, so its run mutates the whole xstate set. The run must report exactly main's 45 graph survivors and nothing else. That proves the PR-scoped gate catches what main's gate caught. The PR stacked on top, #79, kills the survivors. Its gate run 37941873978 on 39a4780 (A plus B) passed: 0 Survived and 0 NoCoverage (Killed 203, Timeout 10, CompileError 416, Ignored 2). This PR's own run 37935773972 on 42a4aed failed on exactly main's 45 src/graph mutants and on nothing else.

Not changed

  • No ruleset or branch-protection setting. The verdict check (verdict · merged report) runs and reports on every pull request, and the conductor enforces it at merge time.
  • No Stryker threshold, no mutate list, no disable comments.
  • CONST-E9: this PR edits the gate on the conductor's ruling 217, which owns the instrument. It does not grade its own work. The graph fixes that the gate grades are in B.

Gates (local, on 23cfd00)

  • deno test scripts/stryker-plan-gate.test.ts: 15 passed. That is the 8 existing tests plus 7 for the scope rule and the scoped gate.
  • pnpm test:sandbox: 8 passed (10 steps), including the 5 release-gate sandbox proofs that read this workflow.
  • deno check, deno lint and tsc -p tsconfig.node.json: exit 0. dprint check is clean.
  • Smoke run of the real repo's discover step plus the real packages/xstate/stryker.config.ts under Node:
changed projects MUTATION_SCOPE config mutate
src/graph/adjacency.ts, src/graph/graph.ts, a changeset packages/xstate the two files the two files
src/graph/adjacency.ts + tests/graph.conformance.test.ts packages/xstate packages/xstate all 11
src/stateUtils.ts packages/xstate packages/xstate all 11
docs/notes.md (none) (empty) not loaded
release-gate.yml packages/xstate packages/xstate all 11
no diff file (push) packages/xstate packages/xstate all 11 (unset: all 11)

Capability PRs never ran the mutation gate, so #77 merged src/graph with
45 live mutants (31 Survived, 14 NoCoverage) that main's gate then
caught. The release gate now also runs on pull_request.

The plan job lists the files the pull request changes (the merge commit
against its first parent), and stryker-plan-gate.ts decides the scope:
the declared files the change touches; a package's whole declared set
when the change touches any other file in that package (a test, a
fixture, a config, a source file outside the set); every package's
whole set when it changes the gate (this workflow, the gate script,
stryker.shared.ts, the lockfile, the Nix toolchain). The scope reaches
Stryker as MUTATION_SCOPE, which stryker.shared.ts intersects with each
package's declared mutate list; unset (local, main without a diff), the
whole set is mutated. The gate refuses an in-scope package the plan
scheduled nothing for, as before, and passes a change that touches no
mutated package with no plan.

The verdict job now runs whenever the workflow was not cancelled and
fails unless the plan, and every shard when there are shards, finished,
so a skipped mutation job can no longer read as a pass. Pull-request
runs skip the incremental cache so every scoped mutant runs fresh, and
a newer push cancels the older run.
@systemfsoftware-maker systemfsoftware-maker changed the title xs/release gate on prs ci(repo): run the release gate on pull requests, scoped to the change Oct 9, 2026
@systemfsoftware-maker
systemfsoftware-maker added this pull request to stack #80 October 9, 2026 12:30
When a shard child fails, the shard runner prints only the first 1024
characters of the child's stderr, and those are startup INFO lines. The
child's error envelope goes to its stdout, which the runner discards. #79's
gate runs 37930507588 and 37933520982 both failed with exit 3 and left
nothing that names the cause.

Every package's Stryker config now writes stryker.log at info level. When
the mutation job fails, a follow-up step re-runs each of the shard's
project runs directly, with the same arguments the shard runner passes,
keeping stdout, stderr and the exit code under reports/diagnostics/. The
job then uploads that directory and every stryker.log as
mutation-diagnostics-shard-N. A green job runs neither step.

@kiro-systemf kiro-systemf Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified: fresh review PASS 8/8 on 42a4aed; 10 checks green except the verdict, which fails only on main's 45 src/graph mutants (#79 fixes them, run 37941873978 green on the stacked head); 0 threads; hunt clean.

@kiro-systemf
kiro-systemf Bot merged commit 6abd188 into main Oct 9, 2026
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant