Repository navigation
✨ feat(ci): cross-file consistency gates and Go 1.27.2 toolchain - #463
Merged
Merged
Conversation
…evcontainer image go.mod is now 1.26.9 and the Dockerfiles ship 1.27.2, not 1.26.6/1.27.1. The devcontainer pins golang:1.26.9 and no change covered it, so CFC-1 now includes it. Adds the spec-loop state and task prompt.
hack/test/consistency_go_toolchain_test.sh fails when go.mod's go directive, the Dockerfile golang images and the devcontainer image disagree, when go.mod gains a toolchain line, when a CI Go setup uses a go-version literal or does not read go.mod, when no golang image is found, and when renovate.json lacks the go/golang group rule (or places it before the rules it must override) or the go directive bump. --self-test mutates copies of the tree, one invariant each, plus a no-op control. Red on this tree by design: go.mod says 1.26.9 while both Dockerfiles ship golang:1.27.2 and Renovate has no group rule. The next commits fix that.
Version-only. Both Dockerfiles already build with golang:1.27.2, so the shipped binary was built by a Go that tests and govulncheck (which run go.mod's version) never saw. The devcontainer image uses the same golang:1.27.2 index digest as the Dockerfiles. No toolchain line; go mod tidy leaves go.sum unchanged.
The gomod manager does not bump the go directive by default, and the dockerfile and devcontainer managers put the golang image in other branches, so Renovate moved the image alone and go.mod fell behind what ships. One rule bumps the root go.mod's go directive (rangeStrategy bump); a second groups depNames go and golang into 'go toolchain'. It sits after the 'go minor and patch dependencies' and 'docker images' rules because a later rule's groupName wins. A local Renovate 41.173.1 lookup dry-run put go.mod, both Dockerfiles and the devcontainer image in renovate/go-toolchain; with the group rule first, go.mod and the Dockerfiles fell back to their old groups.
Read compact devcontainer JSON and fail when the devcontainer has no golang image; treat untagged or digest-only FROM golang as not comparable; accept quoted setup-go uses; require the go directive bump rule to name dep go; fail when a later Renovate rule can pull go or golang into another group. Each gap has a self-test mutant.
A later packageRule's groupName wins, so any group rule after it (a catch-all, a custom.regex group, a matchPackageNames filter) could pull the go directive or a golang image out of the group. As the last rule that sets a groupName it cannot be overridden, and the guard can check that one position instead of reasoning about every matcher. A Renovate 41.173.1 lookup dry-run still puts go.mod, both Dockerfiles and the devcontainer in renovate/go-toolchain.
Join Dockerfile backslash continuations before reading FROM; reject golang:${VAR}; read quoted YAML keys on setup-go steps and go-version literals; ignore JSONC comment lines in devcontainer.json; require the go directive bump rule not to filter to another package; and fail when any rule after the go toolchain group sets another groupName, since Renovate's last match wins. Each has a mutant or a green control in --self-test.
The 1.27.2 bump broke golangci-lint v2.13.1 and govulncheck v1.1.4, which CFC-2 as written could not see: it only compared the tool go directive against go.mod. Operator ruling 2026-10-15.
v2.13.1 carries golang.org/x/tools v0.49.0, which cannot read Go 1.27 export data ("export data version 5 is greater than maximum supported version 4"), so task lint failed with typecheck errors once go.mod moved to 1.27.2. v2.14.0 (x/tools v0.50.0) with the logcheck plugin custom build reports 0 issues. Makefile and hack/tooling/.custom-gcl.yml move together.
v1.15.0 fails under Go 1.27 (golang.org/x/tools loader: "package context without types was imported"), so task lint's arch-lint step went red after the go.mod bump to 1.27.2; it passes under go1.26.9. v1.19.0 reports no warnings on the same .go-arch-lint.yml, and its graph command still takes --type, --include-vendors and --out.
v1.1.4 carries golang.org/x/tools v0.29.0, whose SSA builder panics (unexpected expr: *ast.KeyValueExpr) in about 3 of 5 runs under Go 1.27.2, so task vulncheck flaked after the go.mod bump. v1.9.0 ran clean 5 of 5 on the same tree.
Add hack/test/consistency_tool_pins_test.sh (CFC-2, CFC-6). It resolves every go run/go install pin through go mod download -json, fails when a pin needs a newer Go than go.mod or cannot be resolved, and smoke-runs golangci-lint (the repo's custom build), govulncheck and go-arch-lint once against the module under go.mod's Go, so an analysis tool too old for the Go (as v2.13.1/v1.1.4/v1.15.0 were for 1.27.2) reds in one named step. --self-test mutates copies of the tracked tree, including a file:// proxy fixture that needs go 1.99.0. Not wired into CI until task 6.1.
Every temp dir now lives under one WORKROOT made in the parent shell, so the EXIT trap removes copies and tool builds (new_tmp ran in command substitutions and the list never reached the trap). A golangci-lint pinned outside the Makefile is built and smoke-run at its own version, and the Makefile build must report the pinned version. Every pin token on a go run/install line is scanned, whatever flags precede it, and Makefile recipes are scanned too. Four new self-test mutants; review records for round 1 added.
The tool-pin scan joins backslash continuations and drops quotes before looking for pins, so `go install \` + `pkg@v` on the next line and `pkg@"v1.2.3"` are checked instead of silently skipped. The EXIT trap's module-cache cleanup can no longer turn a green run red. Two new self-test mutants; round-2 review records, G2 evidence and the task 2.1 tick.
GOTOOLCHAIN=local would test whatever Go the host has; the guard runs exactly the go.mod version, which is what the requirement means. Also switches the loop task prompt to the duo review sizing.
Task 3.1. Default render covers every role.yaml triple; tenant mode misses 36, 30 of them cluster-scoped and named by the spec. The other six (deployments, certificates get/list/watch) are documented as left to the team, which the spec does not name: task 3.2 waits on that decision.
… check team-operator.md section 3 leaves workload get/list/watch to each team in tenant mode, so CFC-3 lists those triples as reasoned exceptions instead of requiring the chart Role to grant them. Operator ruling 2026-10-15.
consistency_rbac_test.sh expands config/rbac/role.yaml and the chart's rendered manager roles into (apiGroup, resource, verb) triples and fails on every role.yaml triple the chart does not grant, in the default and the tenantMode render. Exceptions carry their mode on each line, so a tenant-only exception cannot excuse a default-mode gap; each needs a reason, tenant lines a checked class (cluster-scoped or team-workload), and a stale line fails. No real drift: the 36 tenant gaps are the cluster-scoped and team-granted workload reads the CFC-3 ruling lists.
Round-1 review of the RBAC consistency guard. The default render now counts only ClusterRoles, so the namespaced leader-election Role can no longer stand in for a cluster-wide grant; the tenant render counts only Roles, and a Role never covers a cluster-scoped resource. team-workload exceptions are limited to the workload kinds the CFC-3 ruling names, so a dropped reconciler read cannot be filed as team workload. The tenant Role's selfsubjectaccessreviews rule grants nothing, so it becomes a reasoned cluster-scoped exception. Records the round-1 review.
Round-2 review of the RBAC consistency guard. A tenant exception whose reason is only its class token now fails. The self-test isolates the read-only clause of team-workload (a write on a team-workload kind), adds a cluster-scoped subresource listed in a Role, and proves that a wildcard group and resource cover. Records the round-2 review and closes task 3.2 with gaps: the spec does not name the selfsubjectaccessreviews exception, and CI wiring waits for task 6.1.
…scoped exceptions and record G3
hack/ci/go-patch-lag.sh compares go.mod's go directive with the newest stable release of its minor in the go.dev listing (include=all, since the default listing only carries the two newest minors). Behind is exit 1; an unreachable or unparsable listing, a minor the listing lacks, a go.mod ahead of it, or a non-canonical or oversized number is exit 2, so the lookup fails closed (bash arithmetic would otherwise turn a leading-zero or overflowing patch into a false green). A newer minor is only a note. curl is bounded inside the job timeout. .github/workflows/go-patch-lag.yaml runs it weekly and on dispatch with contents: read and no secrets. hack/test/go_patch_lag_test.sh pins the verdicts on a fixture generated from the live listing and the workflow's shape; --self-test proves each check can fail. Task 4.1 of cross-file-consistency-gates (CFC-5).
kollect_access_cache_total, kollect_export_shard_warn_total, kollect_namespace_fingerprint_cache_total, kollect_static_ref_resolution_total and kollect_watch_map_list_errors_total are registered in internal/metrics but were missing from the operator metrics page. Also stop calling metrics_catalog.go a source of truth for registration: it documents names, metrics.go and aggregation*.go register them.
The registerTo test never called Register(), so pointing Register at another registry stayed green. After Register(), re-registering each checked collector in controller-runtime's metrics.Registry must return AlreadyRegisteredError. TestRegister and the new test share one sync.Once, since a second Register() in a process panics.
A variable "tag" is not safe either: FROM localhost:${B} with B=5000/golang:1.99.0 names another repository, and so can debian:${T}. Every $ in a FROM image now fails, except golang:${V}, which the literal-version check already fails. The debian:${DEB_TAG} green control becomes a mutant.
The variable-image rule exempted golang:-prefixed images and left them to the
literal-version check. That check strips the digest and the tag suffix, so
golang:1.27.2@${D} and golang:1.27.2-${X} passed with the variable stripped.
Any $ in a FROM image now fails, golang included; two mutants cover both
spellings and the golang:${GO_VERSION} mutant now reds on the variable rule.
…Docker The FROM parser joined every line of a backslash continuation, so a comment line inside it became the image word and FROM \ + # note + golang:1.99.0 passed. A comment ending in a backslash also swallowed the next line, hiding its FROM. Comment and blank lines are now skipped inside a continuation and a comment never continues, as BuildKit parses them (checked with docker buildx --check). Three mutants cover the spellings.
Evidence for 8.1 and 8.2 (red-before/green-after mutants, BuildKit parse probe), ticks 8.1-8.3, and the Verification table statuses from G1-G8. The first scheduled CFC-5 run stays post-merge and the PR-head lint row not-run.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
The five probe scripts under the change's evidence/ (G3 RBAC probe and gate mutants, G5 gate mutants, G8 parser and BuildKit probes) were analysed by SonarCloud as new code: 44 of 47 uncovered lines and two taint findings came from the G3 Python probe, which turned the quality gate red although the branch adds no uncovered production code. They are records of how a claim was checked, never run by CI. Each one now lives verbatim in the evidence Markdown that cites it, as a fenced block with the command line it was run with and its input revision; the recorded output is unchanged. Sonar does not analyse Markdown, so no exclusion list grows.
Fails when a git-tracked file under openspec/changes/**/evidence/, archived changes included, is not .md, and names every offender. A scan that reads nothing (no openspec/, no evidence left, or a root that is not the top of its work tree) fails rather than passing vacuously. --self-test copies the tracked openspec/ tree into a fresh git work tree per case and asserts each mutant reds with its own message: a .py in a live change's evidence, a .sh in an archived one, a nested probe, a second offender, a .md.sh suffix, no openspec/, no evidence left and both bad roots; a no-op copy and a log outside evidence/ stay green.
Plain and --self-test run as separate steps next to the cross-file consistency gates, so the lint job (a required context that also runs on docs-only PRs) reds when non-Markdown evidence lands under openspec/ and when the guard can no longer fail.
A tracked openspec/ file deleted from the working tree made tar abort the self-test with only its own error. It now fails with a message saying nothing was tested and why.
The CWS-6 loop only checks guards that CI already invokes, so deleting both evidence-guard steps from ci.yaml passed every guard. Listing it with the CFC-6 guards makes the workflow-security test red when either mode is no longer run by a required job.
The self-test staged each copy with plain `git add -A`, so a global core.excludesFile ignoring *.log or *.txt kept a control or mutant from landing and the self-test failed on a clean tree. Staging with -f makes the copy independent of the host's ignore rules.
A self-test mutant deletes both evidence-guard steps from ci.yaml and expects the CFC-6 rule to red, so dropping the guard from the list is caught too. The CWS-6 header names the guard, and the failure message says "CFC-6: guard" now that the list holds more than consistency gates.
G9.md holds the verification matrix, sensors and both review rounds for moving the evidence probes into Markdown and guarding it. The inlined probes now say they ran from the change directory, with the file then under evidence/, and that a re-run belongs in scratch space, never back under evidence/ where the guard reds it.
|
This was referenced Oct 16, 2026
konih
added a commit
that referenced
this pull request
Oct 16, 2026
…wnership decisions Re-measured against main after cross-file-consistency-gates (#463): govulncheck v1.9.0 at Taskfile.yml:204, gitleaks env at ci.yaml:164, golangci-lint v2.14.0, a second dead Renovate manager (shellcheck), and git-cliff publishing only per-asset .sha512 checksums. Pin ownership A: golangci-lint and govulncheck floors and the Makefile/.custom-gcl.yml agreement move out of this change; CFC-2 owns them. Guard shape C: the checks live in consistency_toolchain_pins_test.sh, enrolled by CFC-6, and dev_mise_pin_drift_test.sh is folded into it mutant for mutant. Every behaviour task now follows the task that can fail on it, and manager liveness is defined only after the probe.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
hack/test/consistency_*/ patch-lag guards plus an openspec evidence guard, each with a--self-test, wired intolintandtask check;go.modbumped to Go 1.27.2 with the analysis tools that break under it; a new required jobtool-pins-selftest; a weeklygo-patch-lagworkflow.role.yaml, metric names) drifted silently; the shipped binary was built with Go 1.27.2 while tests and govulncheck ran on 1.26.9.Closes #441.
go.modsaidgo 1.26.9while both Dockerfiles built withgolang:1.27.2, so no test or vulnerability scan ran under the toolchain that produced the shipped image. This PR makes the module's Go equal to the image's, then adds guards so this drift, and three similar ones, fail CI.Most important things
golang.org/x/toolscannot analyse a newer Go: golangci-lint v2.13.1 → v2.14.0, go-arch-lint v1.15.0 → v1.19.0, govulncheck v1.1.4 → v1.9.0. The go-arch-lint bump was the same failure, version-only, and was not part of the operator's original ruling.tool-pins-selftest(45 min timeout, listed inhack/release/verify-eligibility.sh). It is separate fromlintbecause the tool-pins self-test builds every pinned tool and takes about 5 min locally. It has to be added to theprotect-mainruleset after this merges, because before the merge no PR on main can report it.$in aFROMimage. No variable image is parsed around,golang:included. A comment line inside aFROM \continuation is skipped the way BuildKit skips it.hack/test/testdata/consistency-rbac-exceptions.txtholds 37 tenant-mode exceptions, each mode-scoped with a closed reason class. Workload reads are exempt per team-operator.md §3 (operator ruling A, 2026-10-15). There is no drift in default mode.openspec/changes/**/evidence/were analysed by SonarCloud as uncovered new code and failed its quality gate. They are now inlined into the evidence.mdfiles, and a new guard,hack/test/openspec_evidence_markdown_only_test.sh(plain and--self-test, run inlint), keeps that directory Markdown-only.sonar.exclusionsstays as it was.docs/operator-manual/metrics.md(docs-only commit).Risk
lintandtool-pins-selftestare required.tool-pins-selftestdownloads modules and the vuln DB, so a network stall shows up as a timeout.go-patch-lagturns red on the first Monday after a Go patch release, which is intended.internal/metrics/metrics.go(registration made testable) and its tests; the binary now builds and is tested on 1.27.2 everywhere.tool-pins-selftestfrom protect-main (ruleset 18802410), or every PR waits on a check that never runs.localhost:5000/golang:xis not recognised as golang), a heredocFROM ${B}false red (fails closed), and collectors whose names are built by concatenation or Sprintf being invisible to the metrics literal scan (there are none today).Files to look at first
.github/workflows/ci.yaml: 12 new lint steps plus the new jobtool-pins-selftest.hack/test/ci_workflow_security_test.sh: new CFC-6 rule that fails on an unwired guard. This changes what green means for CI wiring.hack/test/openspec_evidence_markdown_only_test.sh: new guard that fails on any non-Markdown file under openspec evidence.hack/test/testdata/consistency-rbac-exceptions.txt: 37 tenant exceptions (trust boundary: RBAC)..github/workflows/go-patch-lag.yaml+hack/ci/go-patch-lag.sh: new scheduled sensor with network access,contents: read, fails closed.internal/metrics/metrics_test.go:TestRegisternow usesregisterOnce, because a secondRegisterpanics.hack/test/task_check_test.sh: one mutant re-anchored.hack/test/consistency_go_toolchain_test.sh: thegolang:${GO_VERSION}mutant sentinel changed.go.mod,Dockerfile*,.devcontainer/devcontainer.json,Makefile,Taskfile.yml,hack/tooling/.custom-gcl.yml,renovate.json: the Go 1.27.2 bump, tool pins, and a Renovate group so thegodirective and the golang images move in one PR.Review this commit-by-commit, not as one flattened diff. Each guard landed test-first with its review fixes as separate commits, and the 99-file diff is mostly OpenSpec evidence under
openspec/changes/.Testing protocol
task lint,task test(under 1.27.2),task vulncheck,task spec:validate,task lint:markdown, each guard plain and with--self-test.gitleaks detectoverorigin/main..HEADclean.lint(all twelve new steps appear in its log) andtool-pins-selftest(10-20 min expected).gh pr merge <n> --rebase --delete-branch.tool-pins-selftest(integration_id 15368) to ruleset 18802410, then read it back and diff against the backup. Only that context may differ.gh workflow run go-patch-lag.yaml --ref main; it must finishsuccesswhile go.mod is on the current 1.27 patch. A red run with "lookup failed" means go.dev was unreachable, not lag.Optional reading — the long version
Nothing above depends on this. Read it if you want the full story, the road not taken, or the deployment fine print. Skip it if the top half already told you what you needed.
The six requirements. CFC-1 Go equality (go.mod, Dockerfiles, devcontainer, Renovate group). CFC-2 tool pins: each
go run/go installpin builds under exactly go.mod's Go (GOTOOLCHAIN=go<version>), and each analysis tool runs once under it. CFC-3 chart RBAC coversconfig/rbac/role.yamlin default and tenant mode. CFC-4 registered metric names match the docs andprometheusrule.yaml, and a Go test compares the live registry withregisteredMetricNames. CFC-5 is the weekly patch-lag sensor. CFC-6 requires every guard to have a negative self-test and to be wired in CI.Review history. Spec set reviewed before #437. Tasks G1-G8 each had independent review legs (two rounds where applicable). The branch review ran two rounds; round 2 findings C1 and C2 were fixed in G8 with mutants, and the rest were deferred to BACKLOG. Full records are in
openspec/changes/cross-file-consistency-gates/loop.md,reviews/andevidence/G1.md-G8.md.Rejected alternatives. Running the tool-pins self-test inside
lintwould add about 5 min to the job every PR waits on. Atoolchainline ingo.modwas rejected because the equality guard is simpler when there is one version. A key-by-keyvalues.yamlvs schema gate would be vacuous (additionalProperties: true).Out of scope here. Archiving the change waits for a follow-up evidence PR once the first scheduled
go-patch-lagrun exists (task 6.2). Reporting go-patch-lag red runs belongs to change 7.