Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Loop — cross-file-consistency-gates (spec-loop run state)

Repo: `/home/koni/Projects/PlatformRelay/worktrees/kollect/cross-file-consistency-gates` · Branch: `feat/cross-file-consistency-gates` · Base: `origin/main` @ `0c1456b5d` · Started: 2026-10-14 · Reviewers: astra + claude (duo) — `/spec-loop-duo` from G3 on (operator, 2026-10-15); G1–G2 ran the earlier codex + claude/sonnet two-leg per-task sizing
Next: integrator — push G9, PR #463 checks by headSha (SonarCloud must go green), rebase-merge, post-merge steps
Next: none
Budget: claude review legs 11/12 (cap lifted by operator goal 2026-10-16) (B reserve 3) · active hours 0/8 (sessions: 2026-10-14–) · source: default

OpenSpec repo (no `.specify/`): the feature dir is `openspec/changes/cross-file-consistency-gates/`
Expand All @@ -15,7 +15,7 @@ OpenSpec repo (no `.specify/`): the feature dir is `openspec/changes/cross-file-
- [x] R spec-set review — done by other means before #437 merged: commits `a0aba9880` (independent Opus review), `10bc634ed` (round two), `50580a3ad` (cold adversarial), `1745b60ba` (post-merge status). Not repeated. Re-measure on 2026-10-14 found 3 WARNINGs (Triage), no CRITICAL; mechanical fixes applied
- [x] L task loop — G1–G6 closed (all CLOSED-WITH-GAPS); 6.2 owner/post-merge
- [x] B branch review — round 1 (`reviews/B-branch/round-1/`, b18354a39): astra:adversarial BLOCK, claude/opus:spec CONCERNS; holistic: task check 2/18 red (lint:markdown = branch, demo_04 = host kubectl), coverage 91.3% = base, vulncheck 0. Triage below; fixes → G7; round 2 (`reviews/B-branch/round-2/`, aa2241523): astra BLOCK, opus CONCERNS; task check 17/18 (lint:markdown = orchestrator regression); race ./internal/... green; A2/A4/O3 closed, H1 regressed, A1 partial → G8. Rounds: 2/2
- [ ] Hand-off
- [x] Hand-off — PR #463 merged (rebase, merge 4e8c7b40d); protect-main ruleset += `tool-pins-selftest`; #441 closed; post-merge rows filled and change archived in a follow-up evidence PR

## Fitness functions

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
## 6. Wiring and land

- [x] 6.1 Run all guards in `lint` (plain and `--self-test` as separate steps) and, by the glob, in `task check` — closed 2026-10-15, evidence: evidence/G6.md
- [ ] 6.2 Merge with the `post-merge` row open; archive in a follow-up evidence PR once it passes
- [x] 6.2 Merge with the `post-merge` row open; archive in a follow-up evidence PR once it passes — closed 2026-10-16: merged in PR #463 (merge 4e8c7b40d); post-merge rows filled below; archived in this follow-up PR

## 7. Stage-B fixes (round 1)

Expand All @@ -52,6 +52,6 @@
| CFC-3 | `consistency_rbac_test.sh` with one triple removed; tenant render | fails naming the triple; tenant exceptions listed | pass | evidence/G3.md (self-test `triple removed`; tenant probe), G7.md |
| CFC-4 | `consistency_metrics_test.sh` cases | each fails or passes as specified | pass | evidence/G5.md, G7.md (registry test) |
| CFC-5 | formatter tests | behind and lookup failure red | pass | evidence/G4.md |
| CFC-5 | first scheduled run | green on current go.mod | post-merge | follow-up evidence PR |
| CFC-5 | first scheduled run | green on current go.mod | pass | workflow_dispatch run 38833372718 on main@4e8c7b40d: success. The first scheduled weekly run has not happened yet; the dispatched run exercises the same workflow and script |
| CFC-6 | each script's self-test plus no-op copy, by exit status | mutants red, no-op green | pass | evidence/G1.md–G5.md (per guard), G6.md (CI wiring), G7.md, G8.md |
| all | `lint` job on the PR head; independent review | green, all guards in the log; APPROVE; both revisions recorded | not-run | PR head CI |
| all | `lint` job on the PR head; independent review | green, all guards in the log; APPROVE; both revisions recorded | pass | PR #463 head 361a00e6e: all 44 non-skipped checks green, including lint, tool-pins-selftest and SonarCloud; main post-merge CI by headSha 4e8c7b40d: all workflows success; reviews in loop.md (G1–G9, B rounds 1–2) |
171 changes: 171 additions & 0 deletions openspec/specs/cross-file-consistency/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
# cross-file-consistency Specification

## Purpose
Defines facts that kollect states in more than one file and the gates that fail when those
statements disagree, so drift is found by CI and not by a user or an outage.

## Requirements

### Requirement: CFC-1 The shipped binary is built with the tested toolchain

The Go version in the `FROM golang:<version>` line of every `Dockerfile*` that builds Go, and in
the `golang:<version>` image of `.devcontainer/devcontainer.json`, SHALL equal the `go` directive
of `go.mod`. `go.mod` SHALL carry no separate `toolchain` line, and every
Go setup in CI SHALL use `go-version-file: go.mod`. Renovate SHALL move the `go` directive and the
golang image tag in one group.

#### Scenario: Equal

- **WHEN** `go.mod` and every Dockerfile name the same Go version
- **THEN** the check passes

#### Scenario: Image differs from go.mod

- **WHEN** a Dockerfile or the devcontainer image names a newer or older Go than `go.mod`
- **THEN** the check SHALL fail and name the file and both versions, because tests and govulncheck run the go.mod toolchain and never see the one that ships

#### Scenario: Lone image PR

- **WHEN** Renovate opens a PR bumping only the golang image (group rule missing, mis-ordered after a more general rule, or the manager does not bump the `go` directive)
- **THEN** the check turns red on that PR, and the guard also checks the group rule's presence and order

#### Scenario: Toolchain line or literal

- **WHEN** `go.mod` gains a `toolchain` line or a workflow adds a Go version literal
- **THEN** the check SHALL fail

#### Scenario: No Go image found

- **WHEN** the scan finds no `golang:` line
- **THEN** the check SHALL fail; an empty scan never passes

### Requirement: CFC-2 Pinned tools build with the repository's Go

Every tool pinned through `go run <pkg>@<version>` or `go install` SHALL build under
exactly the Go of `go.mod` (`GOTOOLCHAIN=go<version>`, never `auto` and never whatever Go the
host has); the check reads each tool's required Go from
`go mod download -json` and fails when it exceeds `go.mod`'s. Each pinned analysis tool SHALL also
run once against the module under that Go and exit cleanly.

#### Scenario: Tool needs a newer Go

- **WHEN** a pinned tool's `go` directive is newer than `go.mod`'s
- **THEN** the check SHALL fail and name the tool, version and both Go versions

#### Scenario: Analysis tool too old for go.mod's Go

- **WHEN** a pinned analysis tool (golangci-lint, govulncheck, go-arch-lint) is older than `go.mod`'s Go, so it cannot load or analyse the module (its `golang.org/x/tools` predates that Go)
- **THEN** the check SHALL fail and name the tool and version, because a smoke run of each analysis tool under `go.mod`'s Go exits non-zero or panics; one clean run is the pass condition

#### Scenario: Tool lookup fails

- **WHEN** `go mod download` cannot resolve a pin (network or typo)
- **THEN** the check SHALL fail; an unresolved pin is never a pass

#### Scenario: Transition after a Go bump

- **WHEN** `go.mod`'s Go is raised
- **THEN** every tool pin is rechecked in the same PR

### Requirement: CFC-3 The chart grants at least what the kustomize role grants

Every (apiGroup, resource, verb) triple in `config/rbac/role.yaml` SHALL be granted by the manager
roles rendered from `charts/kollect` with default values, unless listed with a reason in
`hack/test/testdata/consistency-rbac-exceptions.txt`.

#### Scenario: Chart matches

- **WHEN** the rendered roles cover every triple of `role.yaml`
- **THEN** the check passes

#### Scenario: A kubebuilder marker adds a permission the chart lacks

- **WHEN** a controller gains `+kubebuilder:rbac` for a new resource and `make manifests` updates `role.yaml`, but the chart templates are not changed
- **THEN** the check SHALL fail and name the missing triple

#### Scenario: Exception without a reason

- **WHEN** an exception line has no reason text
- **THEN** the check SHALL fail

#### Scenario: Tenant mode

- **WHEN** the chart is rendered with `tenantMode=true`
- **THEN** namespaced triples are covered by the rendered namespaced `Role`; cluster-scoped triples (namespaces, tokenreviews, subjectaccessreviews, selfsubjectaccessreviews, `kollectcluster*`) appear as listed exceptions with the reason that tenant mode drops cluster scope; namespaced
workload-collection reads (today `apps/deployments` and `cert-manager.io/certificates`, get/list/watch)
appear as listed exceptions with the reason that tenant mode leaves workload RBAC to the team
(`docs/examples/team-operator.md` §3); any other namespaced triple missing from the `Role` fails

### Requirement: CFC-4 Metric names agree between code, docs and chart

Every `kollect_*` metric name in `docs/operator-manual/metrics.md` and in
`charts/kollect/templates/prometheusrule.yaml` (histogram suffixes `_bucket`, `_sum`, `_count`
stripped) SHALL be registered in the non-test source of `internal/metrics` other than
`metrics_catalog.go`, every registered metric SHALL appear in `metrics.md`, and
`registeredMetricNames` SHALL equal that source set.

#### Scenario: Alert on a renamed metric

- **WHEN** a metric is renamed in code and `prometheusrule.yaml` still queries the old name
- **THEN** the check SHALL fail and name the rule file and the unknown metric

#### Scenario: New metric without docs

- **WHEN** a metric is registered and absent from `metrics.md`
- **THEN** the check SHALL fail and name the metric

#### Scenario: Catalog is not registration

- **WHEN** a name appears only in `metrics_catalog.go`
- **THEN** it does not count as registered

#### Scenario: Mirror list not updated

- **WHEN** a metric is added to `metrics.go` but not to `registeredMetricNames`
- **THEN** the check SHALL fail

#### Scenario: Histogram suffix

- **WHEN** a rule queries `kollect_export_duration_seconds_bucket`
- **THEN** it is accepted as the registered histogram

#### Scenario: Empty scan

- **WHEN** the source scan returns no names
- **THEN** the check SHALL fail

### Requirement: CFC-5 Falling behind the latest Go patch is a red signal

A scheduled workflow SHALL fail when the `go` directive of `go.mod` is behind the latest released
patch of its minor, and SHALL fail when the latest patch cannot be determined.

#### Scenario: Behind

- **WHEN** `go.mod` names patch N and the latest release of that minor is N+1
- **THEN** the scheduled run is red and says so (the CI-failure reporter, once present, files an issue)

#### Scenario: Current

- **WHEN** `go.mod` equals the latest patch
- **THEN** the run is green

#### Scenario: Lookup fails

- **WHEN** the release listing is unreachable or unparsable
- **THEN** the run SHALL fail, not pass

#### Scenario: Newer minor exists

- **WHEN** a newer minor than `go.mod`'s exists
- **THEN** the sensor does not fail on that alone

### Requirement: CFC-6 Each consistency gate is proven able to fail

Each check SHALL contain a self-test that corrupts a copy of the tree to break its invariant and
asserts a non-zero exit, plus a no-op copy that must pass, and SHALL run on `pull_request` in a
required job, plain and `--self-test` as separate steps.

#### Scenario: Mutant survives

- **WHEN** corrupting the copy does not make the check exit non-zero
- **THEN** the self-test SHALL fail
Loading