Skip to content

feat(repo): wire dry-run module evidence into closure analysis - #223

Merged
kiro-systemf[bot] merged 7 commits into
mainfrom
stryker/closure-evidence-wire
Oct 7, 2026
Merged

kiro-systemf[bot] merged 7 commits into
mainfrom
stryker/closure-evidence-wire

Conversation

@systemfsoftware-maker

@systemfsoftware-maker systemfsoftware-maker commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Layer 3 of 3. This wires the dry run's testFileModules into every closure analysis and persists it in the incremental report. With it, the last 3 open stryker-js test closures close (src/conclude-run.ts, src/mutation-reporting.service.ts, src/run/instrument.ts). Those were open only because they reach the computed import(moduleName) in src/drivers/config.ts.

Wiring

  • Persist. dryRunCoverage.testFileModules stores original project paths, mapped from the sandbox the same way as globalTestInputs. A reused dry run carries it forward unchanged.
    • A sandbox-only value with no project original is dropped. That is the runner's generated setup module; otherwise it would open every closure.
    • Values outside the sandbox (data: URLs, store realpaths) pass through verbatim, so a data: target still opens the closure.
  • Feed. Evidence goes into both closureAnalysisOf calls in the dry-run cell: the prior coverage before the reuse decision, and the fresh evidence after a run. It also goes into the incremental-reuse analysis and the plan path.
  • Soundness, by induction. A key computed from the evidence of run N matches at run N+1 only if every module the test loaded at N is byte-identical. In that case the test loads the same modules again. A changed observation can only refuse reuse, never reuse a stale verdict. Known boundary, same as today's fs reads: a mutant that changes a computed specifier may load a module the dry run never saw.

Proof (real dry runs with the layer-1 runner, not hand-fed evidence)

tests/incremental-reuse.integration.test.ts. A test file statically imports src/left.mjs and, in its body, runs await load(new URL('../src/plugin.mjs', import.meta.url).href).

  • Editing the dynamic target src/plugin.mjs re-runs left's covered mutants.
  • Editing an unrelated src/unrelated.mjs keeps them.
  • A runtime data: target keeps the closure open, so the unrelated edit re-runs them.
  • The fix(repo): key a module-level mutant on its whole-suite closure digest #219 open-closure scenario used an uncalled import(name). With evidence, an uncalled import loads nothing, so it closes. The scenario now imports a data: URL at runtime, which keeps the soundness case meaningful.

Failing-before: "A runtime module a test file loads keeps the closure closed around an unreached edit". With layer 2's engine sources in a scratch worktree, it got kept: false, reran: true against expected kept: true, reran: false (1 failed, 13 passed). 14 of 14 pass here.

Projected effect (no mutation run; main CI measures after release)

Evidence was collected exactly as the runner collects it, reading the worker module map in each test file's afterAll during a plain vitest run of the package's mutation test set: 184 files, 1081 tests. Closures were computed with this branch's analyzeImportClosure. Mutants come from the merged report of main run 37574914218 (2624 reusable mutants), and the projection covers one-file edits of each of the 243 source files under main's keys (#216 plus #219).

open closures re-runs per one-file edit (mean / median / max) mean recorded cost
main today 3 / 177 273 / 258 / 478 19.2 min
this stack 0 / 177 225 / 258 / 478 15.7 min

The median holds at 258 because #219's whole-suite key re-runs every static mutant whenever any test reaches the edited file. That is most files; 42 of 243 edits now keep everything. With no open closures left, the held reach key (stryker/static-reach-closure) projects 165 / 211 / 431 at 12.0 min.

Validation

  • Full stryker-js suite: 231 files, 1250 tests. Vitest-runner: 23 tests. Plugin-interface: 270 tests.
  • pnpm check:ci: 111 of 112. The only failure is stryker-js#api:check, whose single diff is the machine-dependent Node_2 as Node alias. The committed report keeps the rendering that CI passes on main.
  • guard:projects reports Verdict-Semantics: unchanged, and both changeset checks exit 0.
  • Changeset: minor.

@systemfsoftware-maker
systemfsoftware-maker added this pull request to stack #224 October 7, 2026 06:51
The vitest runner collects Vitest's per-worker module execution record at the end of each dry-run test file and returns it as testFileModules, keyed by the absolute test file path with node: builtins dropped. The complete DryRunResult now carries the optional field so an incremental run can close a test file's closure from the modules it actually evaluated

Verdict-Semantics: unchanged
ImportClosureInput gains an optional observedModules record, and a
computed dynamic import no longer leaves a test file's closure open once
the runner reports the modules that test file actually evaluated.
Observed modules that name existing files join the closure as extra roots
scanned and hashed like statically reached files; a key that is not an
absolute path to an existing file, a parse error, or an unresolved static
specifier still opens it. Nothing supplies the evidence yet, so the layer
is inert in production

Verdict-Semantics: unchanged
The dry run records the modules each test file evaluated, mapped from the sandbox back to the project, and a reused dry-run coverage carries the record forward. Every import-closure analysis reads it, so an incremental run closes a test file whose only openness is a computed dynamic import. Evidence is taken only from modules that name an existing project file, dropping the runner's generated sandbox setup module which would otherwise keep every closure open

Verdict-Semantics: unchanged
Base automatically changed from stryker/closure-evidence-closure to main October 7, 2026 09:07

@kiro-systemf kiro-systemf Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Layer 3: wires dry-run module evidence into closures; unresolved evidence keeps closures open.

@kiro-systemf
kiro-systemf Bot merged commit 4f2a659 into main Oct 7, 2026
16 of 22 checks passed
@kiro-systemf
kiro-systemf Bot deleted the stryker/closure-evidence-wire branch October 7, 2026 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant