Repository navigation
[go] keep CI build caches warm - #1541
Merged
Merged
Conversation
Verification and test jobs share an immutable dependency-keyed cache. Small verification builds can freeze that archive without the artifacts needed by the full suites, so later jobs repeat downloads and compilation. Separate shared module downloads from workload-specific build caches. Include workspace and module metadata in dependency keys, and refresh build archives after Go, SQL, fixture, and CI input changes. Retain exact toolchain, runner, database version, and race scopes when restoring older builds. Let only the primary full suite publish module downloads. Use the documented setup action across Go jobs and port fixture helpers. Check out sources before setup so local action loading and dependency hashing work. Include the action in filtered workflows and the existing monthly Dependabot coverage.
bgentry
force-pushed
the
bg/improve-go-ci-cache
branch
from
October 25, 2026 13:59
c2c274f to
e2b499e
Compare
brandur
approved these changes
Oct 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Go verification, benchmark, and test jobs currently share an immutable setup-go cache. In the baseline CI run, the Postgres race job restores a 9 MB archive, downloads dependencies again, and cannot save its new builds because the cache key already exists. CLI, lint, and submodule checks also install Go before checkout, so dependency-cache lookup reports missing files.
Centralize the policy in a documented “Set up Go for CI” composite action. Share module downloads across Go versions and workloads, with the primary full suite as the Linux writer and the CLI as the only Windows workload. Give build archives separate runner, toolchain, workload, database-version, and race scopes. Hash all modules' manifests/checksums, workspace metadata, and relevant source/configuration inputs so immutable archives can refresh. Restore compatible earlier builds as a fallback. Race and standard artifacts can safely coexist in Go's own cache; separate archives keep both workloads warm. Database-version scopes also prevent reusing test results against a different external server.
Use the action for Go CI and the Go helpers in port workflows, after checkout. Comments explain the inputs, key components, restore order, and why only cache paths need the job environment for nested post-job saves. Existing monthly Dependabot scans cover local actions, including underlying major action updates; filtered workflows now run when this action changes. Test concurrency, selection, and timeouts remain unchanged.
Measured on hosted runners using the same Go/test sources, Go 1.27.3, and Ubuntu image 20261012.334.1: compare the baseline with the warm rerun. Whole-job durations include runner/service setup and cache restore/post steps.
The initial race cache fill takes 3m21s, with a 2m46s test step, and successfully saves both archives. Its warm rerun has 53 cached package results and no dependency downloads. These are warm, unchanged-source results: changed packages and their dependents still rebuild/rerun. The benchmark step executes its benchmarks on every run, so its improvement demonstrates compilation/download reuse separately from cached test results.
The Linux module archive is about 182 MiB compressed, stored once rather than copied into each build archive. Representative build archives are about 94–104 MiB. More workload/source-specific archives consume additional cache capacity and remain subject to GitHub's normal eviction policy.
The Java workflow failure is caused by stale Maven Central prefix metadata restored from its immutable dependency cache. PR #1542 refreshes only that auto-discovered index, preserving dependency artifacts, repository filters, and the strict warning policy. It reproduces the failure and verifies the fix separately from this Go caching change.