Skip to content

ci(ci): build the mutated projects once and restore the build in every shard - #226

Merged
kiro-systemf[bot] merged 1 commit into
mainfrom
stryker/shard-setup-cache
Oct 7, 2026
Merged

kiro-systemf[bot] merged 1 commit into
mainfrom
stryker/shard-setup-cache

Conversation

@systemfsoftware-maker

@systemfsoftware-maker systemfsoftware-maker commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Mutation shards now build once instead of rebuilding the same projects 20 times. A build job runs alongside plan and saves the turbo build cache, keyed on the content of the lockfile and the sources. Each shard restores that cache, so its turbo run build replays from cache instead of compiling.

Before

In main run 37602076567, every shard built the same 11 turbo tasks with Cached: 0 cached, 11 total. That took 22.9 s on shard 11, 35.7 s on shard 5 and 38.1 s on shard 2: about 20 × 30 s ≈ 10 job-minutes per run. Of a shard's ~60 s of setup, the remaining 25 s is checkout, pnpm and node setup, and pnpm install.

Design decisions

  • Key and correctness. The key is mutation-build-<os>-hashFiles(pnpm-lock.yaml, turbo.json, packages/**, test/e2e-core/**), hashed before install so node_modules and dist stay out of it. Restores fall back to the previous run's cache by prefix. Turbo re-hashes every task's inputs when it reads the cache, so a stale or partial restore can only cost a rebuild, never wrong dist.
  • Bounded cache. Before saving, the build job deletes restored entries that this build's --summarize hashes don't name. Without that, each save would carry every older run's entries.
  • Fail once. Shards needs: [plan, build]. A broken build fails one job, not 20.
  • The shard's own build step stays. On a cache miss it builds, as it does today.
  • pnpm install is not cached. It takes 6–8 s with the pnpm store already restored by setup-node. Restoring a node_modules archive with workspace symlinks would cost about the same and adds a second source of truth.

Borrowed: Bazel's action cache, which keys on the content of inputs, not the commit, and recomputes on a miss. Rejected: building in the plan job. Shards can't start until that job finishes, so the build would land on the critical path.

Wall-clock trade

Shards now wait for max(plan, build) instead of plan. Plan took 36 s in 37602076567. Build is about 25 s of setup plus the tasks this push changed: near 0 s when nothing changed, about 40 s for a lockfile change. Each shard saves about 30 s.

  • Typical push: a net gain on the critical path of 10–30 s.
  • Lockfile change: roughly a wash on wall-clock.
  • Every run: about 10 job-minutes saved.

Main Mutation runs will measure this; the projection is not a result.

Validation

  • Workflow structure: I parsed it with Bun's YAML parser. Jobs are plan, build, mutation, report; mutation needs [plan, build], and the build key is wired through job outputs.
  • Prune step: I extracted it from the workflow and ran it against a real turbo cache plus 3 planted stale files. The 33 files of the 11 live tasks were kept and the 3 stale ones deleted. Turbo then reported Cached: 11 cached, 11 total from the pruned directory, and 9 cached, 9 total for a stryker-js-only filter.
  • Repo checks: pnpm check:ci (112/112), dprint check and guard:projects exit 0.
  • No failing-before test: this is a workflow-only change, and no repo test covers workflow files. The before/after evidence is the main Mutation runs.

.github/workflows/ is read-only for agents under AGENTS.md. This edit is authorized by Kiro's Conductor direction of 2026-10-07 (cycle 111), as #213 was by the decision of 2026-10-06.

…y shard

Every shard rebuilt the same 11 turbo tasks from scratch (0 cached, 23-38 s in run 37602076567). A build job now runs beside plan, restores the last run's turbo cache, builds, keeps only the entries it used and saves under a lockfile and source content key; shards restore that key so their build replays from cache

Verdict-Semantics: unchanged
@systemfsoftware-maker systemfsoftware-maker changed the title stryker/shard setup cache ci(ci): build the mutated projects once and restore the build in every shard Oct 7, 2026

@kiro-systemf kiro-systemf Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build once beside plan; shards restore the turbo cache keyed on lockfile+sources. Workflow-only, no gates weakened.

@kiro-systemf
kiro-systemf Bot merged commit db180fc into main Oct 7, 2026
8 checks passed
@kiro-systemf
kiro-systemf Bot deleted the stryker/shard-setup-cache branch October 7, 2026 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant