From b14e999aeaead9e22459435307c61272e8b7e6e2 Mon Sep 17 00:00:00 2001 From: Konrad Heimel Date: Wed, 14 Oct 2026 23:59:54 +0200 Subject: [PATCH 01/69] :page_facing_up: docs(spec): re-measure cross-file-consistency-gates and cover the devcontainer image go.mod is now 1.26.9 and the Dockerfiles ship 1.27.2, not 1.26.6/1.27.1. The devcontainer pins golang:1.26.9 and no change covered it, so CFC-1 now includes it. Adds the spec-loop state and task prompt. --- .../cross-file-consistency-gates/loop.md | 66 +++++++++++++++++++ .../cross-file-consistency-gates/proposal.md | 13 ++-- .../specs/cross-file-consistency/spec.md | 7 +- .../task-prompt.md | 52 +++++++++++++++ .../cross-file-consistency-gates/tasks.md | 4 +- 5 files changed, 131 insertions(+), 11 deletions(-) create mode 100644 openspec/changes/cross-file-consistency-gates/loop.md create mode 100644 openspec/changes/cross-file-consistency-gates/task-prompt.md diff --git a/openspec/changes/cross-file-consistency-gates/loop.md b/openspec/changes/cross-file-consistency-gates/loop.md new file mode 100644 index 00000000..99d0791a --- /dev/null +++ b/openspec/changes/cross-file-consistency-gates/loop.md @@ -0,0 +1,66 @@ +# Loop — cross-file-consistency-gates (spec-loop run state) + +Repo: `/home/koni/Projects/PlatformRelay/worktrees/kollect/cross-file-consistency-gates` · Branch: `feat/cross-file-consistency-gates` · Base: `origin/main` @ `0c1456b5d` · Started: 2026-10-14 · Reviewers: codex (`codex exec --sandbox read-only`) + `claude/sonnet` via `fanout.sh --no-unify` (no opencode on this host); `claude/opus` at gates +Next: L — dispatch task 1.1 +Budget: claude review legs 0/12 (B reserve 3) · active hours 0/8 (sessions: 2026-10-14–) · source: default + +OpenSpec repo (no `.specify/`): the feature dir is `openspec/changes/cross-file-consistency-gates/` +(proposal.md, specs/cross-file-consistency/spec.md, tasks.md). Landing order position 4 of 10; +1–3 merged (#450, #452, #453). Issue #441. + +## Stages +- [x] 0 orient — proposal / spec delta / tasks present, nothing implemented (all unticked); branch and base recorded +- [x] P plan + tasks — skipped: present · spec-set fix committed with this file (see Triage) +- [x] R spec-set review — done by other means before #437 merged: commits `a0aba9880` (independent Opus review), `10bc634ed` (round two), `50580a3ad` (cold adversarial), `1745b60ba` (post-merge status). Not repeated. Re-measure on 2026-10-14 found 3 WARNINGs (Triage), no CRITICAL; mechanical fixes applied +- [ ] L task loop — see *Tasks* +- [ ] B branch review +- [ ] Hand-off + +## Fitness functions +| Characteristic | Command | Kind | Baseline | Per task / at B | +|---|---|---|---|---| +| CI guard meta-tests | `bash hack/test/_test.sh` and `--self-test` (lint job) | triggered | green suite | per task (touched + new guards) | +| Shell hygiene | `task lint:shell` | triggered | no warnings | per task | +| Workflow supply chain | zizmor gate via `hack/test/ci_workflow_security_test.sh` | triggered | 0 high findings | per task touching `.github/` | +| Go build/vet | `go build ./... && go vet ./...` | triggered | clean | per task touching Go/go.mod | +| Lint incl. arch | `task lint` (golangci v2.13.1 + go-arch-lint) | triggered | green | task 1.3 and at B | +| OpenSpec strict | `task spec:validate` | triggered | 13/13 | per spec-set edit | +| Full local gate | `task check` | holistic | green on base | at B | +| Coverage floor | `task coverage` (COVERAGE_MIN 90) | holistic | ≥90% | at B (Go bump) | +| Vulns | `task vulncheck` | holistic | recorded at 1.3 | task 1.3, at B | + +Holistic run at B: pending + +## Triage +| Stage | Finding (one line) | Sev | Models | Disposition | Where it went / why | Needs user? | +|---|---|---|---|---|---|---| +| R | proposal quotes stale versions (1.26.6 / 1.27.1); now go.mod 1.26.9, Dockerfiles 1.27.2 | WARNING | re-measure | fix | proposal.md updated | no | +| R | `.devcontainer/devcontainer.json` pins `golang:1.26.9`; CFC-1 scanned only Dockerfiles, so it drifts after the bump | WARNING | re-measure | fix | CFC-1 + scenario + tasks 1.2/1.3 include the devcontainer image | no | +| R | `lint` is not a `protect-main` required check (required: preflight, test, kind-smoke, Analyze (Go)); CFC-6 "required job" depends on change 1's operator step 5.1 | WARNING | re-measure | defer | owner task (ruleset edit); 6.1 closes with that gap | yes (operator) | + +## Tasks +| Task | Verdict | Review (legs, rounds, Claude?) | Gaps / decision request | +|---|---|---|---| + +## Known red +| Test (file:name) | Story | Written in | Cleared in | +|---|---|---|---| + +## Test changes +| Test (file:name) | Written in | Changed in | Evidence it was wrong | +|---|---|---|---| + +## Lessons +- (none yet) + +## Where a human should look first + +## Proposed harness changes +| Lesson | Seen in | Proposed change | +|---|---|---| + +## Owner tasks (skipped by the loop) +| Task | Command sheet | +|---|---| +| ruleset | add `lint` (and per change 1: `vulncheck`, `workflow-security`, `dependency-review`) to `protect-main` required checks | +| 6.2 | merge with the `post-merge` row open; archive in a follow-up evidence PR after the first scheduled `go-patch-lag` run | diff --git a/openspec/changes/cross-file-consistency-gates/proposal.md b/openspec/changes/cross-file-consistency-gates/proposal.md index d9c14540..70f1f7b3 100644 --- a/openspec/changes/cross-file-consistency-gates/proposal.md +++ b/openspec/changes/cross-file-consistency-gates/proposal.md @@ -4,8 +4,9 @@ Some facts live in two files by necessity, and nothing compares them. Probed on this tree: -- `go.mod` says `go 1.26.6`; `Dockerfile:2` and `Dockerfile.pipeline:4` build with a newer - `golang:` image. CI tests and runs govulncheck with the go.mod toolchain (`go-version-file: +- `go.mod` says `go 1.26.9`; `Dockerfile:2` and `Dockerfile.pipeline:4` build with + `golang:1.27.2` (re-measured 2026-10-14; the proposal was written at 1.26.6 vs 1.27.1). + `.devcontainer/devcontainer.json` pins `golang:1.26.9`, so it drifts the other way after a bump. CI tests and runs govulncheck with the go.mod toolchain (`go-version-file: go.mod`; `Taskfile.yml:14` sets `GOTOOLCHAIN` from it), so the shipped binary is built by a toolchain no test ran under and govulncheck never scanned. Nothing notices. - Tools run through `go run @` or `go install` (go-arch-lint, govulncheck, @@ -27,9 +28,9 @@ taken. ## What Changes -- Go: `go.mod` is bumped to the Go version the Dockerfiles ship (`go 1.27.1`, no separate - `toolchain` line); a guard keeps the Dockerfiles and `go.mod` equal; Renovate groups the `go` - directive with the golang image so they move in one PR. +- Go: `go.mod` is bumped to the Go version the Dockerfiles ship (`go 1.27.2` at re-measure, no + separate `toolchain` line); a guard keeps the Dockerfiles, the devcontainer image and `go.mod` + equal; Renovate groups the `go` directive with the golang images so they move in one PR. - A guard builds each pinned `go run`/`go install` tool under `GOTOOLCHAIN=local` with the go.mod Go (reading each tool's Go directive from `go mod download -json`). - A scheduled sensor goes red when `go.mod`'s Go is behind the latest patch of its minor. @@ -77,7 +78,7 @@ newer one). The sensor's red runs are reported once change 7 lists it. ## Assumptions -- Go 1.27.1 builds the module at current dependency versions; probe `go build ./... && go vet +- Go 1.27.2 builds the module at current dependency versions; probe `go build ./... && go vet ./...` and the test suite before the bump (task 1.1). - `go mod download -json @` returns the module's `GoMod` file path, whose `go` line gives the tool's required Go. Source: `go help mod download`; probe in task 3.1. diff --git a/openspec/changes/cross-file-consistency-gates/specs/cross-file-consistency/spec.md b/openspec/changes/cross-file-consistency-gates/specs/cross-file-consistency/spec.md index b05f2559..466eccd1 100644 --- a/openspec/changes/cross-file-consistency-gates/specs/cross-file-consistency/spec.md +++ b/openspec/changes/cross-file-consistency-gates/specs/cross-file-consistency/spec.md @@ -13,8 +13,9 @@ service account (`templates/clusterrole.yaml`, `templates/role.yaml` and ### Requirement: CFC-1 The shipped binary is built with the tested toolchain -The Go version in the `FROM golang:` line of every `Dockerfile*` that builds Go SHALL -equal the `go` directive of `go.mod`. `go.mod` SHALL carry no separate `toolchain` line, and every +The Go version in the `FROM golang:` line of every `Dockerfile*` that builds Go, and in +the `golang:` image of `.devcontainer/devcontainer.json`, SHALL equal the `go` directive +of `go.mod`. `go.mod` SHALL carry no separate `toolchain` line, and every Go setup in CI SHALL use `go-version-file: go.mod`. Renovate SHALL move the `go` directive and the golang image tag in one group. @@ -25,7 +26,7 @@ golang image tag in one group. #### Scenario: Image differs from go.mod -- **WHEN** a Dockerfile names a newer or older Go than `go.mod` +- **WHEN** a Dockerfile or the devcontainer image names a newer or older Go than `go.mod` - **THEN** the check SHALL fail and name the file and both versions, because tests and govulncheck run the go.mod toolchain and never see the one that ships #### Scenario: Lone image PR diff --git a/openspec/changes/cross-file-consistency-gates/task-prompt.md b/openspec/changes/cross-file-consistency-gates/task-prompt.md new file mode 100644 index 00000000..7a772104 --- /dev/null +++ b/openspec/changes/cross-file-consistency-gates/task-prompt.md @@ -0,0 +1,52 @@ +You are closing ONE Spec Kit task in a spec-loop run. State lives in files only; you carry nothing in from any earlier session. + +Repo: /home/koni/Projects/PlatformRelay/worktrees/kollect/cross-file-consistency-gates (branch `feat/cross-file-consistency-gates`, must be clean when you start; check with `git status`). +Feature directory: openspec/changes/cross-file-consistency-gates (OpenSpec change: proposal.md, specs/cross-file-consistency/spec.md = the spec, tasks.md; no plan.md/constitution — repo rules are AGENTS.md and docs/development/spec-workflow.md). +Records directory for review registers: openspec/changes/cross-file-consistency-gates + +Do this: invoke the `close-task` skill (Skill tool, name `close-task`, args `__TID__`) and follow it end to end for task __TID__. Its helper scripts are at ~/.claude/skills/close-task/scripts/task.sh and sensors.sh; run `task.sh show __TID__` from the repo root first and read only the slice it gives you plus the ids it cites. + +Per-task review (this replaces the reviewer dispatch in close-task's REVIEW step). Record the pre-task sha before you change anything. Size the review to the diff: +- evidence or documentation only → `--legs "GLM-5.3:diff"`; +- anything else → `--legs "GLM-5.3:diff,DeepSeek-V4.1-Flash:diff"`; +- append `,claude/opus:diff` (with `--claude-budget 3`) ONLY when Claude review legs left for this task (see dispatch note) is above 0 and: the free register says BLOCK; fewer than two free legs produced a verdict (`legs.log` shows `exit=truncated` or non-zero) on a non-doc diff; or the diff touches secrets handling, hieradata, Terraform, manifests, CI or a protected path. +Command: + ~/.claude/skills/fanout-review/scripts/fanout.sh --repo /home/koni/Projects/PlatformRelay/worktrees/kollect/cross-file-consistency-gates --target "task __TID__: ; verification matrix in openspec/changes/cross-file-consistency-gates/evidence/__TID__.md; diff ..HEAD" --legs "" --out openspec/changes/cross-file-consistency-gates/reviews/L-tasks/__TID__[/round-N] +Without opencode, the orchestrator names the legs instead (e.g. `codex exec --sandbox read-only - < > leg-codex-diff.md` plus `fanout.sh --legs "claude/sonnet:diff" --no-unify`); write the brief with the Write tool and pass it on stdin. +Read only register.md; verify every CRITICAL and every single-model WARNING against the files yourself before acting; map CLEAN (or CONCERNS whose findings did not survive verification) → APPROVE, a verified finding → REQUEST_CHANGES, BLOCK → BLOCK. Record every gate run (command, sha, exit, result) in the evidence BEFORE writing the review brief, so reviewers do not spend round 1 on missing records. A second round happens only if round 1 found a verified BEHAVIOUR defect (evidence or wording findings: fix, no new round); it goes in a fresh `round-2/` directory and is the last: after it, fix only verified behavioural defects (no further review), record everything else as nits or gaps, each rejected with a reason or deferred to a named place, and close. If fanout.sh exits non-zero or register.md has no `## Unified verdict` line, do not wait: read the leg files yourself (each has its own `## Verdict`), say so in the state file, and continue. If fanout.sh cannot run at all, say so in the state file and use a fresh general-purpose subagent with ~/.claude/skills/close-task/reference/review-brief.md instead. Say in the state file which legs ran and whether a Claude leg ran and why. + +Fitness functions: the loop state file (openspec/changes/cross-file-consistency-gates/loop.md, section *Fitness functions*) lists the repository's architectural fitness functions; the rows marked per-task run in your `task` tier on the final tree. If your diff adds an import edge between components, a dependency, a package or component, a function over the complexity threshold, or touches determinism-sensitive code, either the existing functions pass unchanged or you extend one in this task and say why in the evidence. Never widen an allow-list, exclusion list or coverage baseline to get green: that is an architecture decision, report BLOCKED with the decision request. + +Test budget: order the steps as focused tests, the fast check, the mutation gate, the cold review, fixes, the mutation gate on the fix lines only, then ONE full suite on the final code tree (red, or a fix beyond wording, goes back through one review round) (review findings otherwise force a second full run: two ~7-minute runs on anvil 017 P2); after an evidence- or docs-only commit, the fast check only. Do not create or edit files in the repo while the full suite runs (untracked-file guards fail it). Mutation proof through the repository's changed-lines mutation gate when it has one; hand-written mutants only where it cannot reach, as a representative sample — and then restore from the FIRST saved copy, drop stale `__pycache__`, and judge only the target test's own failure line. If your change touches code that only a deselected test tier or a scheduled CI job runs (golden, e2e, live markers), run that tier locally before closing; if it cannot run here, close CLOSED-WITH-GAPS naming the tier (the orchestrator dispatches CI for it). + +Shell: if the session is worktree-isolated, keep each Bash command plain — no loops over hosts, no `$(…)` or variables feeding git/gh/codex, no heredoc whose text mentions git, no long jq/python filters inline; write scripts and briefs with the Write tool and run them as files. Refused commands cost a round trip each. + +Lessons from earlier tasks: see loop.md *Lessons* (render at dispatch) + +If a trigger fires but no Claude review legs are left, run no Claude leg and close CLOSED-WITH-GAPS naming the review you skipped. + +Test tasks (the task text says it writes tests, or sits under "Tests for User Story N"): follow close-task's *Test tasks* rule. Write the cases the task names from spec.md and the task text, not from plan.md's design rationale. Close only on a verified red: each new test fails for its stated reason, command and failure line in the evidence; list them under `## Known red`. A compile error or missing symbol is not that red: a stub with the signature plan.md fixes (body: not implemented) is in scope; report BLOCKED only if plan.md does not fix the signature. A test that already passes is a finding against the test unless you established that the outcome already holds; then say so and close. Never bypass a commit hook that refuses red tests: report BLOCKED. +Known red for this feature (excluded from "green" at close, nothing else is): see loop.md *Known red* +Implementation tasks: tests written by an earlier test task of this feature are fixed points. Change one only through close-task's TEST class with evidence citing spec.md or the constitution (a spec-versus-plan conflict is SPECIFICATION: BLOCKED), list each change under `## Test changes` in the evidence, and name it in your report. Report every known-red test you turned green. + +Task kinds: if the task text says "Owner session" or "Owner gate", or needs a host, account, push, pull request or decision you do not have, do NOT attempt that part. Whole task → stop at SELECT and report `OWNER` with the command sheet you would hand the owner written to openspec/changes/cross-file-consistency-gates/evidence/__TID__.md. Part → do the agent part, write the owner part as a command sheet in the same file, close CLOSED-WITH-GAPS listing it. + +Hard rules: +- Never read, print, decrypt or copy a secret value; reference entries and secret names only. No .env, kubeconfig or token files; Renovate/App secrets are referenced by name only. +- Do not push, add or change any git remote, or open a pull request. Local commits on `feat/cross-file-consistency-gates` only, one logical commit including the state file, Conventional Commit style with the repo convention `:gitmoji: (): ` (ASCII shortcode first, e.g. `:white_check_mark: test(ci): ...`, `:wrench: build: ...`, `:arrow_up: chore(deps): ...`); NO Co-Authored-By or any agent/model/tool attribution anywhere in the message. +- Do not edit CLAUDE.md, AGENTS.md, the constitution, .specify/ or .claude/; propose instead. +- Terse, plain English. Tag every claim in evidence as verified (you ran or read it, with date) or believed. +- Evidence: the close-task state file is openspec/changes/cross-file-consistency-gates/evidence/__TID__.md; if the task names another evidence file, write that too and link it. No key, token, kubeconfig, whole log or address that is not needed. +- Commit the review `register.md` and `leg-*.md` files with the task when records live in the repo; never the `*.err` transcripts. +- Tick the task in tasks.md exactly as close-task says. + +Final report, at most 20 lines: task · verdict (CLOSED / CLOSED-WITH-GAPS / BLOCKED / OWNER) · matrix summary · sensors run · corrections · review verdict, legs and register path, Claude legs used · findings rejected or deferred (one line each: finding · reject/defer · reason or destination) · known red added or cleared · test changes · learnings (class, plus each reusable lesson in one line, for the next task's prompt) · commit sha · next action or decision request. Nothing else. + +Repo specifics (kollect): +- Reviews on this host: NO opencode. Per-task legs: leg 1 = `codex exec --sandbox read-only - < > /leg-codex-diff.md` (run from the repo root; brief written with the Write tool, built from ~/.claude/skills/close-task/reference/review-brief.md plus the diff range); leg 2 = `~/.claude/skills/fanout-review/scripts/fanout.sh --repo --target "..." --legs "claude/sonnet:diff" --no-unify --out `. Docs/evidence-only diff: codex leg alone. Add `claude/opus:diff` only on the triggers above (CI workflow changes count as CI). There is no unified register: read both leg files' `## Verdict` yourself. +- Task ids in tasks.md are `N.M` numbers; you close the whole group named in the dispatch, ticking each line. One logical commit per sub-task where the task says "its own commit" (e.g. the go.mod bump is version-only, real drift fixes are separate). +- Guards follow `hack/test/dev_mise_pin_drift_test.sh` and `hack/test/ci_workflow_security_test.sh`: plain mode checks the real tree, `--self-test` corrupts a throwaway copy (mktemp) per mutant and asserts non-zero, plus a no-op copy that must pass. Run `task lint:shell` on new scripts. +- `task check` runs every hack/test guard by glob; do not run it per task (it is the stage-B holistic row). Do run the guards you touched, both modes. +- Gotchas: helm-unittest `contains` compares whole elements; "no tests to run" exits 0; `format:check`/`docs:verify` can be red on a clean main in a working copy (verify with stash before blaming your change). +- Nothing private ships: before committing run `git diff --cached | rg -i 'rewe|mkurator|titiseven|prod165|atlassian'` (must be empty) and `gitleaks protect --staged`. +- Evidence file per group: /evidence/G.md; review records in /reviews/L-tasks/G/. diff --git a/openspec/changes/cross-file-consistency-gates/tasks.md b/openspec/changes/cross-file-consistency-gates/tasks.md index 04896ab0..7fbb51fd 100644 --- a/openspec/changes/cross-file-consistency-gates/tasks.md +++ b/openspec/changes/cross-file-consistency-gates/tasks.md @@ -3,8 +3,8 @@ ## 1. Go equality and bump (CFC-1, CFC-6) - [ ] 1.1 Probe: `go build ./... && go vet ./...` and `task test` under the target Go; probe whether Renovate bumps the `go` directive and which group rule wins (the golang-image group must come after the existing "go minor and patch dependencies" and "docker images" rules); probe CodeQL's Go extractor on the target Go (`codeql.yaml`) -- [ ] 1.2 Write `hack/test/consistency_go_toolchain_test.sh` (plain and `--self-test`) with fixtures (equal, image newer, image older, disagree, toolchain line, literal, none found, group rule missing); watch the newer/older cases fail on the assertion -- [ ] 1.3 Bump `go.mod` to the version the Dockerfiles ship (version-only commit, no `toolchain` line); add the Renovate group; guard passes; `task test`, `task lint`, `task vulncheck` recorded +- [ ] 1.2 Write `hack/test/consistency_go_toolchain_test.sh` (plain and `--self-test`) with fixtures (equal, image newer, image older, devcontainer differs, disagree, toolchain line, literal, none found, group rule missing); watch the newer/older cases fail on the assertion +- [ ] 1.3 Bump `go.mod` (and the devcontainer image) to the version the Dockerfiles ship (version-only commit, no `toolchain` line); add the Renovate group; guard passes; `task test`, `task lint`, `task vulncheck` recorded ## 2. Tool pins (CFC-2, CFC-6) From c2b744ac039e3a24e0cc32188d44d676f14d3a8d Mon Sep 17 00:00:00 2001 From: Konrad Heimel Date: Thu, 15 Oct 2026 00:15:55 +0200 Subject: [PATCH 02/69] :white_check_mark: test(ci): add Go toolchain consistency guard (CFC-1) hack/test/consistency_go_toolchain_test.sh fails when go.mod's go directive, the Dockerfile golang images and the devcontainer image disagree, when go.mod gains a toolchain line, when a CI Go setup uses a go-version literal or does not read go.mod, when no golang image is found, and when renovate.json lacks the go/golang group rule (or places it before the rules it must override) or the go directive bump. --self-test mutates copies of the tree, one invariant each, plus a no-op control. Red on this tree by design: go.mod says 1.26.9 while both Dockerfiles ship golang:1.27.2 and Renovate has no group rule. The next commits fix that. --- hack/test/consistency_go_toolchain_test.sh | 379 +++++++++++++++++++++ 1 file changed, 379 insertions(+) create mode 100755 hack/test/consistency_go_toolchain_test.sh diff --git a/hack/test/consistency_go_toolchain_test.sh b/hack/test/consistency_go_toolchain_test.sh new file mode 100755 index 00000000..e9372b1b --- /dev/null +++ b/hack/test/consistency_go_toolchain_test.sh @@ -0,0 +1,379 @@ +#!/usr/bin/env bash +# CFC-1 (openspec change cross-file-consistency-gates): the shipped binary is built with the +# tested toolchain. +# +# Tests, govulncheck and every CI Go setup run the Go that go.mod's `go` directive names +# (actions/setup-go reads it through `go-version-file: go.mod`). The release image is built by +# the `FROM golang:` line of each Dockerfile, and contributors use the golang image of +# .devcontainer/devcontainer.json. Nothing made those agree: Renovate bumps the golang image +# and, by default, never the `go` directive, so the image ran ahead and the binary that shipped +# was built by a Go no test or vulnerability scan had seen. This gate fails when they differ. +# +# What it locks: +# * every golang image (Dockerfile* FROM lines, devcontainer image) equals go.mod's `go`; +# * the images agree with each other; +# * go.mod has a `go` directive and no `toolchain` line (a toolchain line is a second source +# of truth that setup-go and the images would not follow); +# * no workflow or composite action carries a `go-version:` literal, and every +# actions/setup-go step reads `go-version-file: go.mod`; +# * renovate.json moves the `go` directive and the golang image in ONE group: the gomod +# manager is told to bump the directive (rangeStrategy bump, depType golang -- it does not +# by default), and the group rule comes AFTER the "go minor and patch dependencies" and +# "docker images" rules, because a later packageRule overrides an earlier one's groupName. +# Verified with a local Renovate lookup dry-run on 2026-10-14 (evidence/G1.md); +# * an empty scan (no golang image found) is a failure, never a pass. +# +# Usage: +# hack/test/consistency_go_toolchain_test.sh # check the repository tree +# hack/test/consistency_go_toolchain_test.sh --self-test # mutate copies, assert each check fails +# hack/test/consistency_go_toolchain_test.sh # check an arbitrary tree root +set -euo pipefail + +fail() { + echo "FAIL: $*" >&2 + exit 1 +} + +pass() { + echo "ok - $*" +} + +SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")" + +MODE="check" +if [[ "${1:-}" == "--self-test" ]]; then + MODE="self-test" + shift +fi + +ROOT="${1:-}" +if [[ -z "${ROOT}" ]]; then + ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +fi +ROOT="${ROOT%/}" + +command -v jq >/dev/null 2>&1 || fail "jq not found; install jq to run this test" + +# Every Dockerfile* under the root, relative paths, sorted. Vendored, generated and +# local-only trees are pruned; a Dockerfile with no golang FROM line is simply not a Go build. +dockerfiles() { + local root="$1" + (cd "${root}" && find . \( -name .git -o -name node_modules -o -name references -o -name bin \ + -o -name .task -o -name dist -o -name vendor -o -name worktrees \) -prune -o \ + -type f -name 'Dockerfile*' -print | sed 's|^\./||' | sort) +} + +# " " for each golang image reference: Dockerfile FROM lines and the devcontainer +# `image`. The tag is everything after `golang:` up to a digest or whitespace. +golang_images() { + local root="$1" f + while IFS= read -r f; do + [[ -n "${f}" ]] || continue + awk -v f="${f}" ' + /^[[:space:]]*#/ { next } + toupper($1) == "FROM" { + for (i = 2; i <= NF; i++) { + if ($i ~ /^--/) continue + img = $i + sub(/^(docker\.io\/)?(library\/)?/, "", img) + if (img ~ /^golang:/) { sub(/^golang:/, "", img); sub(/@.*$/, "", img); print f, img } + break + } + } + ' "${root}/${f}" + done < <(dockerfiles "${root}") + + f=".devcontainer/devcontainer.json" + if [[ -f "${root}/${f}" ]]; then + sed -n -E 's/^[[:space:]]*"image"[[:space:]]*:[[:space:]]*"(docker\.io\/)?(library\/)?golang:([^"@]*)[^"]*".*$/\3/p' \ + "${root}/${f}" | while IFS= read -r tag; do echo "${f} ${tag}"; done + fi +} + +# Every CI YAML that can set up Go: workflows and composite actions. +ci_files() { + local root="$1" dir + for dir in "${root}/.github/workflows" "${root}/.github/actions"; do + [[ -d "${dir}" ]] || continue + find "${dir}" -type f \( -name '*.yml' -o -name '*.yaml' \) + done | sort +} + +# ":" of every actions/setup-go step whose with: block has no +# `go-version-file: go.mod` before the next step starts. +setup_go_without_gomod() { + local root="$1" file + while IFS= read -r file; do + awk -v f="${file#"${root}"/}" ' + function close_step() { if (open && !ok) print f ":" at; open = 0 } + /^[[:space:]]*#/ { next } + /^[[:space:]]*-[[:space:]]/ { close_step() } + /uses:[[:space:]]*actions\/setup-go@/ { open = 1; ok = 0; at = NR; next } + open && /^[[:space:]]*go-version-file:[[:space:]]*["\047]?go\.mod["\047]?[[:space:]]*(#.*)?$/ { ok = 1 } + END { close_step() } + ' "${file}" + done < <(ci_files "${root}") +} + +count_setup_go() { + local root="$1" file n=0 c + while IFS= read -r file; do + c="$(grep -Ec '^[^#]*uses:[[:space:]]*actions/setup-go@' "${file}" || true)" + n=$((n + c)) + done < <(ci_files "${root}") + echo "${n}" +} + +# The whole contract for one tree. Prints one FAIL line per broken invariant, returns non-zero +# when any is broken. +check_go_toolchain() { + local root="$1" broken=0 + local gomod="${root}/go.mod" renovate="${root}/renovate.json" + local go_version="" images path tag ver distinct lowest line file n + + # --- go.mod ------------------------------------------------------------------------------- + if [[ -f "${gomod}" ]]; then + go_version="$(sed -n -E 's/^go[[:space:]]+([0-9][0-9.]*)[[:space:]]*$/\1/p' "${gomod}" | head -1)" + fi + if [[ -z "${go_version}" ]]; then + echo "FAIL: go.mod has no 'go ' directive -- there is nothing for the images to agree with" >&2 + broken=1 + fi + if [[ -f "${gomod}" ]] && grep -Eq '^[[:space:]]*toolchain[[:space:]]' "${gomod}"; then + echo "FAIL: go.mod carries a toolchain line ($(grep -E '^[[:space:]]*toolchain[[:space:]]' "${gomod}" | head -1)) -- a second Go version that setup-go and the images do not follow; keep the go directive as the one source" >&2 + broken=1 + fi + + # --- golang images ------------------------------------------------------------------------ + images="$(golang_images "${root}")" + if [[ -z "${images}" ]]; then + echo "FAIL: no golang image found in any Dockerfile* FROM line or in .devcontainer/devcontainer.json -- the scan set is empty, so the comparison would pass without comparing anything" >&2 + broken=1 + else + while read -r path tag; do + ver="${tag%%-*}" + if [[ ! "${ver}" =~ ^[0-9]+\.[0-9]+(\.[0-9]+)?$ ]]; then + echo "FAIL: ${path}: golang:${tag} is not a literal Go version, so it cannot be compared with go.mod" >&2 + broken=1 + continue + fi + [[ -n "${go_version}" ]] || continue + [[ "${ver}" == "${go_version}" ]] && continue + lowest="$(printf '%s\n%s\n' "${ver}" "${go_version}" | sort -V | head -1)" + if [[ "${lowest}" == "${go_version}" ]]; then + echo "FAIL: ${path}: golang:${ver} is newer than go.mod's go ${go_version} -- the binary would ship built by a Go that tests and govulncheck never ran" >&2 + else + echo "FAIL: ${path}: golang:${ver} is older than go.mod's go ${go_version} -- the binary would ship built by a Go that tests and govulncheck never ran" >&2 + fi + broken=1 + done <<<"${images}" + + distinct="$(awk '{ v = $2; sub(/-.*/, "", v); print v }' <<<"${images}" | sort -u -V | tr '\n' ' ')" + distinct="${distinct% }" + if [[ "${distinct}" == *" "* ]]; then + echo "FAIL: the golang images disagree with each other: ${distinct}" >&2 + broken=1 + fi + fi + + # --- CI Go setups ------------------------------------------------------------------------- + while IFS= read -r file; do + while IFS= read -r line; do + [[ -n "${line}" ]] || continue + echo "FAIL: ${file#"${root}"/}: go-version: literal ($(sed -E 's/^[[:space:]]+//' <<<"${line#*:}")) -- CI must read the Go version from go.mod via go-version-file" >&2 + broken=1 + done < <(grep -nE '^[[:space:]]*-?[[:space:]]*go-version:' "${file}" || true) + done < <(ci_files "${root}") + + n="$(count_setup_go "${root}")" + if [[ "${n}" -eq 0 ]]; then + echo "FAIL: no actions/setup-go step found under .github -- the CI half of this check would pass without checking anything" >&2 + broken=1 + fi + while IFS= read -r line; do + [[ -n "${line}" ]] || continue + echo "FAIL: ${line}: actions/setup-go step does not read go-version-file: go.mod" >&2 + broken=1 + done < <(setup_go_without_gomod "${root}") + + # --- Renovate ----------------------------------------------------------------------------- + if [[ ! -f "${renovate}" ]] || ! jq -e '.packageRules | type == "array"' "${renovate}" >/dev/null 2>&1; then + echo "FAIL: renovate.json is missing, unparsable or has no packageRules -- nothing moves go.mod and the golang image together" >&2 + return 1 + fi + + local group_idx gomod_idx docker_idx + group_idx="$(jq '[.packageRules | to_entries[] + | select((.value.matchDepNames // []) | (index("go") != null and index("golang") != null)) + | select(.value.groupName // "" | length > 0) | .key] | last // -1' "${renovate}")" + gomod_idx="$(jq '[.packageRules | to_entries[] + | select(.value.groupName == "go minor and patch dependencies") | .key] | last // -1' "${renovate}")" + docker_idx="$(jq '[.packageRules | to_entries[] + | select(.value.groupName == "docker images") | .key] | last // -1' "${renovate}")" + + if [[ "${group_idx}" -lt 0 ]]; then + echo "FAIL: renovate.json has no packageRule that groups depNames go and golang -- Renovate opens a lone golang image PR and go.mod stays behind" >&2 + broken=1 + else + if [[ "${gomod_idx}" -gt "${group_idx}" ]]; then + echo "FAIL: renovate.json's go/golang group rule (packageRules[${group_idx}]) comes before the \"go minor and patch dependencies\" rule (packageRules[${gomod_idx}]); the later rule wins and pulls the go directive into its own group" >&2 + broken=1 + fi + if [[ "${docker_idx}" -gt "${group_idx}" ]]; then + echo "FAIL: renovate.json's go/golang group rule (packageRules[${group_idx}]) comes before the \"docker images\" rule (packageRules[${docker_idx}]); the later rule wins and pulls the golang image into its own group" >&2 + broken=1 + fi + fi + + if ! jq -e '[.packageRules[] + | select((.matchDepTypes // []) | index("golang") != null) + | select(.rangeStrategy == "bump") + | select((.matchManagers // ["gomod"]) | index("gomod") != null) + | select((.matchFileNames // ["go.mod"]) | index("go.mod") != null)] | length > 0' \ + "${renovate}" >/dev/null; then + echo "FAIL: renovate.json has no packageRule that sets rangeStrategy bump for the gomod go directive (matchDepTypes golang) -- Renovate does not bump the go directive by default, so the group would carry only the image" >&2 + broken=1 + fi + + return "${broken}" +} + +if [[ "${MODE}" == "check" ]]; then + if check_go_toolchain "${ROOT}"; then + pass "go.mod, every golang image, the CI Go setups and the Renovate go group agree" + exit 0 + fi + fail "the Go toolchain is stated inconsistently (listed above) -- the binary that ships would be built by a Go the tests never ran" +fi + +# --------------------------------------------------------------------------- +# Self-test (CFC-6): copy the files the gate reads, break one invariant per copy, and assert +# the gate reds WITH THE MESSAGE of the assertion the mutant targets -- a non-zero exit alone +# is not evidence. One unmutated copy must pass. +# --------------------------------------------------------------------------- +SCRATCH="" +trap 'rm -rf ${SCRATCH} 2>/dev/null' EXIT + +make_copy() { + local dst="$1" f + mkdir -p "${dst}/.github" + cp "${ROOT}/go.mod" "${ROOT}/renovate.json" "${dst}/" + for f in "${ROOT}"/Dockerfile*; do + [[ -f "${f}" ]] && cp "${f}" "${dst}/" + done + [[ -d "${ROOT}/.devcontainer" ]] && cp -R "${ROOT}/.devcontainer" "${dst}/.devcontainer" + cp -R "${ROOT}/.github/workflows" "${dst}/.github/workflows" + [[ -d "${ROOT}/.github/actions" ]] && cp -R "${ROOT}/.github/actions" "${dst}/.github/actions" + return 0 +} + +new_copy() { + local copy + copy="$(mktemp -d)" + SCRATCH="${SCRATCH} ${copy}" + make_copy "${copy}" + echo "${copy}" +} + +# mutant_rejected