konih · Oct 25, 2026 · Oct 25, 2026 · Oct 25, 2026 · Oct 25, 2026
@@ -0,0 +1,190 @@
+# Evidence — ci-workflow-hardening (post-merge rows)
+
+Recorded 2026-10-25 against `PlatformRelay/Kollect` after #450 merged (2026-10-06,
+`3ee21266`). Every row below cites real GitHub Actions runs; all calls are read-only except
+the one throwaway probe PR in the CWS-3 section, which was closed unmerged and its branch
+deleted.
+
+## CWS-1 — `workflow-security` green on real PR heads
+
+```bash
+gh pr checks 494 -R PlatformRelay/Kollect | grep workflow-security
+gh pr checks 495 -R PlatformRelay/Kollect | grep workflow-security
+gh api repos/PlatformRelay/Kollect/actions/runs/<id> -q '[.id,.event,.head_sha,.conclusion]|@tsv'
+```
+
+| PR | PR head | CI run | event | `workflow-security` |
+| --- | --- | --- | --- | --- |
+| #494 | `13c70fb568d98f9cf8f2684fa5e21ef261d61c91` | 40223387529 | `pull_request` | pass (job 121259583355) |
+| #495 | `6e18b5abee9dcb5b84bc3ef4c7052697825b9d6b` | 40227161572 | `pull_request` | pass (job 121270924131) |
+
+Each run's `head_sha` equals the PR head, so the job ran on the PR head, not on a merge
+commit. The red side (template-injection fixture, no-`--offline` and skip-switch mutants) is
+in `evidence.md`.
+
+**Result: pass.**
+
+## CWS-3 / task 4.2 — `dependency-review` red on a disallowed licence and on a vulnerable module
+
+```bash
+gh api --allow-escape-sequences repos/PlatformRelay/Kollect/actions/jobs/<job>/logs \
+  | sed -e 's/\x1b\[[0-9;]*m//g' | grep -E 'incompatible|vulnerab|License:|GHSA|##\[error\]'
+```
+
+### Disallowed licence (real PRs, no probe needed)
+
+| PR | Run / job | Dependency flagged | Log line |
+| --- | --- | --- | --- |
+| #470 | 39170622155 / 117802707914 | `go-task/setup-task@a00fbb05…` (Action) | `License: GPL-3.0` → `Dependency review detected incompatible licenses.` |
+| #489 (Dependabot, open) | 39987119032 / 120529075026 | `pillow@13.0.0` (docs) | `License: MIT-CMU` → `Dependency review detected incompatible licenses.` |
+| #457 (Dependabot) | 37910176591 / 113753166938 | `cairosvg@2.9.1` (docs) | `License: LGPL-3.0-or-later` |
+| #456 | 37907543188 / 113744549550 | `golang.org/x/net@0.60.0` | `BSD-3-Clause AND LicenseRef-scancode-google-patent-license-golang` |
+
+In every one of these runs the Vulnerabilities group reports `did not detect any vulnerable
+packages`, so the red is the licence policy alone. Each of the last three was resolved with a
+package-scoped `allow-dependencies-licenses` exception (with a reason in `ci.yaml`), not by
+widening `allow-licenses`; #489 is still red today.
+
+### Vulnerable module (throwaway probe)
+
+No real PR had gone red on a vulnerability, so one throwaway draft PR was opened:
+
+- PR #496, branch `probe/dependency-review-vulnerable`, one commit `a9c1adca` that pins
+  `golang.org/x/net` back from v0.60.0 to v0.17.0 in `go.mod` (no other change; x/net's licence
+  is already excepted, so only the vulnerability check can fire).
+- Run 40231428226 (`pull_request`, head `a9c1adcadb04083a79d6022a15276913b43760c2`),
+  `dependency-review` job 121283803587: **fail**.
+
+```text
+go.mod » golang.org/x/net@0.17.0 – net/http, x/net/http2: close connections when receiving too many headers (moderate severity)
+  ↪ https://github.com/advisories/GHSA-4v7x-pqxf-cx7m
+go.mod » golang.org/x/net@0.17.0 – HTTP Proxy bypass using IPv6 Zone IDs in golang.org/x/net (moderate severity)
+  ↪ https://github.com/advisories/GHSA-qxp5-gwg8-xv66
+go.mod » golang.org/x/net@0.17.0 – golang.org/x/net vulnerable to Cross-site Scripting (moderate severity)
+  ↪ https://github.com/advisories/GHSA-vvgc-356p-c3xw
+go.mod » golang.org/x/net@0.17.0 – Go Net HTML parser is vulnerable to denial of service (moderate severity)
+  ↪ https://github.com/advisories/GHSA-5cv4-jp36-h3mw
+##[error]Dependency review detected vulnerable packages.
+```
+
+PR #496 was closed without merging and its branch deleted (`gh pr close 496 --delete-branch`).
+
+**Result: pass** — red on a disallowed licence (#470, #489) and on a vulnerable module (#496).
+
+## CWS-4 — a newer push cancels the PR's in-flight run
+
+`ci.yaml` groups by `${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}`
+with `cancel-in-progress: ${{ github.event_name == 'pull_request' }}`.
+
+```bash
+gh run list -R PlatformRelay/Kollect --workflow ci.yaml --event pull_request \
+  --json databaseId,headBranch,headSha,conclusion,createdAt
+gh api repos/PlatformRelay/Kollect/check-runs/121263420724/annotations -q '.[].message'
+```
+
+PR #495 (`docs/archive-synced-debounce-condition`), three pushes:
+
+| Run | Head | Started | Ended | Conclusion |
+| --- | --- | --- | --- | --- |
+| 40224677510 | `f8a96158` | 19:09:29Z | 19:20:50Z | cancelled |
+| 40225572307 | `1f5a0487` | 19:20:34Z | 19:40:59Z | cancelled |
+| 40227161572 | `6e18b5ab` | 19:40:41Z | — | success |
+
+Each run ends seconds after the next push's run starts. The cancelled job in run 40224677510
+carries the annotation `Canceling since a higher priority waiting request for CI-495 exists`,
+naming the PR-keyed group.
+
+**Result: pass.**
+
+## CWS-4 — quick successive merges to `main` each get their own run
+
+The five merges of 2026-10-25 landed within about two and a half hours (17:29 to 20:05
+UTC). Filtered by `headSha` (not `--commit`, which can return zero rows):
+
+```bash
+gh run list -R PlatformRelay/Kollect --workflow ci.yaml --branch main --limit 60 \
+  --json databaseId,headSha,event,conclusion
+gh run list -R PlatformRelay/Kollect --workflow e2e-smoke.yaml --branch main --limit 40 \
+  --json databaseId,headSha,event,conclusion
+```
+
+| Merge | Committed (UTC) | `CI` run | `e2e-smoke` run |
+| --- | --- | --- | --- |
+| `d2ba2edbb` | 17:29:34 | 40216722422 push success | 40216722466 push success |
+| `556f70d1f` | 18:42:33 | 40222532916 push success | 40222532873 push success |
+| `44f7a240a` | 19:14:25 | 40225080731 push success | 40225080828 push success |
+| `1353f7772` | 19:38:09 | 40226966266 push success | 40226966351 push success |
+| `463c9401f` | 20:05:28 | 40229120503 push success | 40229120473 push success |
+
+Every SHA has exactly one push run of each workflow and none was cancelled.
+
+**Result: pass.**
+
+## CWS-6 / task 5.1 — required checks in `protect-main`
+
+```bash
+gh api repos/PlatformRelay/Kollect/rulesets/18802410 \
+  -q '.rules[]|select(.type=="required_status_checks")|.parameters.required_status_checks[].context'
+gh api repos/PlatformRelay/Kollect/rulesets/18802410/history/<version_id> -q '.updated_at'
+```
+
+Ten required contexts today: `preflight`, `test`, `kind-smoke`, `Analyze (Go)`, `lint`,
+`workflow-security`, `dependency-review`, `tool-pins-selftest`, `vulncheck`, `breaking-marker`.
+The ruleset history shows when the four came in: `lint`, `workflow-security` and
+`dependency-review` in version 53247538 (2026-10-15), `vulncheck` in version 53474785
+(2026-10-16).
+
+**Ruleset half: pass.**
+
+### Skipped required job counts as passing — not proven
+
+Which required jobs can be skipped on a PR:
+
+- `test` and `kind-smoke` are `if: always()` aggregators; `lint`, `workflow-security`,
+  `tool-pins-selftest`, `preflight`, `breaking-marker` and `Analyze (Go)` have no job-level
+  `if:`; `dependency-review` is `if: github.event_name == 'pull_request'`, so it always runs on
+  a PR.
+- `vulncheck` is `if: needs.changes.outputs.code == 'true'`, so it **is skipped on every
+  docs-only PR**.
+
+Observed on real PRs (`gh pr checks <n>`, and GraphQL `isRequired(pullRequestNumber:)`):
+
+- #489 (Dependabot, docs-only, open): `vulncheck=SKIPPED` while required (run 39987119032).
+  The PR is also blocked by the red `dependency-review`, so its merge state cannot show
+  whether the skipped `vulncheck` alone would block.
+- #462 (docs-only): `vulncheck` skipped and the merge (`0c1456b5`, 2026-10-14) passed the
+  ruleset (rule suite 4517903794, `pass`), but `vulncheck` only became required on
+  2026-10-16, so that merge is no proof either.
+- A scan of every PR merged from 2026-07-17 (first ruleset version with required checks) to
+  2026-10-25 found no other merged PR with a skipped check that was required at merge time.
+
+So the case is real (docs-only PRs skip a required job), but no observation yet shows GitHub
+treating that skipped job as passing. The probe stays open until a docs-only PR with
+otherwise green checks reaches a mergeable state (or does not).
+
+**Result: open.** Task 5.1 stays unticked.
+
+## Task 5.2 — `require_extra_approval_for_unattributed_changes`
+
+```bash
+gh api repos/PlatformRelay/Kollect/rulesets/18802410 \
+  -q '.rules[]|select(.type=="pull_request")|.parameters'
+gh api "repos/PlatformRelay/Kollect/rulesets/rule-suites?ref=refs/heads/main&time_period=month"
+```
+
+The `pull_request` rule has `require_extra_approval_for_unattributed_changes: true`, with
+`required_approving_review_count: 0`, `require_last_push_approval: false` and
+`allowed_merge_methods: ["rebase"]`.
+
+It has not affected bot PRs in practice. Each of these merged with zero reviews and its push to
+`main` evaluated as `pass` (not `bypass`) in the rule-suite log:
+
+| PR | Author of the commits | Merge | Rule suite |
+| --- | --- | --- | --- |
+| #475 (Renovate, Go toolchain digest) | `renovate-bot <renovate@whitesourcesoftware.com>` | `cf448c4a` | 4618177030 pass |
+| #476 (Renovate, openspec 1.14.3) | `renovate-bot <renovate@whitesourcesoftware.com>` | `1353f777` | 4713611399 pass |
+| #457 (Dependabot, cairosvg) | `dependabot[bot]`, plus one maintainer commit | `3f17512f` | 4460537403 pass |
+
+Dependabot PRs also get the normal check set (#489's list includes every required context).
+
+**Result: pass** — recorded: `true`; no observed effect on Renovate or Dependabot PRs.
@@ -20,16 +20,18 @@
 ## 4. dependency-review (CWS-3)
 
 - [x] 4.1 Add the `dependency-review` job (PR only, default threshold, `allow-licenses` list derived from `go-licenses` or the current dependency set, with reasons for each per-module exception)
-- [ ] 4.2 Probe on a throwaway PR with a known-vulnerable module and one disallowed licence: job fails both times (needs two real throwaway PRs against the repo; the config-level mutants are covered by the meta-test's CWS-3 mutants)
+- [x] 4.2 Probe on a throwaway PR with a known-vulnerable module and one disallowed licence: job fails both times (needs two real throwaway PRs against the repo; the config-level mutants are covered by the meta-test's CWS-3 mutants) — closed 2026-10-25, evidence: evidence/post-merge.md
 
 ## 5. Operator-owned (hard prerequisite of the later changes)
 
 - [ ] 5.1 (operator) Probe that a skipped required job counts as passing, then make `lint`, `vulncheck`, `workflow-security` and `dependency-review` required checks in `protect-main`
-- [ ] 5.2 (operator) Read the ruleset JSON for `require_extra_approval_for_unattributed_changes` and record whether it affects bot PRs
+  - 2026-10-25: the four are required (ruleset versions 53247538 and 53474785). The skip probe is still open: `vulncheck` is skipped on every docs-only PR, and no docs-only PR has yet reached a state that shows whether the skipped job counts as passing (#489 is also red on `dependency-review`). Evidence: evidence/post-merge.md
+- [x] 5.2 (operator) Read the ruleset JSON for `require_extra_approval_for_unattributed_changes` and record whether it affects bot PRs — closed 2026-10-25, evidence: evidence/post-merge.md (`true`; Renovate #475/#476 and Dependabot #457 merged with rule suite `pass`)
 
 ## 6. Land
 
 - [ ] 6.1 Merge with the `post-merge` rows below open; the archive commit lands in a small follow-up evidence PR once they pass
+  - 2026-10-25: merged as #450; every post-merge row passes except the CWS-6 skip probe (task 5.1), so the archive waits for it
 
 ## zizmor threshold decision (measured 2026-10-05, zizmor 1.30.1)
 
@@ -55,13 +57,13 @@ findings are all cosmetic:
 
 | Req | Check | Expected | Status | Evidence |
 | --- | --- | --- | --- | --- |
-| CWS-1 | `workflow-security` on the PR head; template-injection fixture; meta-test mutants (no `--offline`, skip switch) | green on tree, red on fixture and mutants | mutants red locally (12 mutants); PR-head run is post-merge | evidence/evidence.md |
+| CWS-1 | `workflow-security` on the PR head; template-injection fixture; meta-test mutants (no `--offline`, skip switch) | green on tree, red on fixture and mutants | pass: mutants red locally (12 mutants); green on PR heads #494 (run 40223387529) and #495 (run 40227161572) | evidence/evidence.md, evidence/post-merge.md |
 | CWS-2 | meta-test bare-suppression mutant; review of each suppression | red on mutant; each justified | red on mutant; zero suppressions exist (`rules: {}`) | evidence/evidence.md |
-| CWS-3 | `dependency-review` on throwaway PRs (vulnerable module, disallowed licence); meta-test mutants | job red twice; mutants red | mutants red; throwaway-PR probe post-merge (task 4.2) | evidence/evidence.md |
-| CWS-4 | meta-test group check (ref-keyed, unprefixed mutants); two pushes to a PR | first run cancelled | mutants red locally; two-push cancellation is live post-merge | evidence/evidence.md |
-| CWS-4 | three quick merges to main each get a run | each SHA has a run | post-merge | follow-up evidence PR |
+| CWS-3 | `dependency-review` on throwaway PRs (vulnerable module, disallowed licence); meta-test mutants | job red twice; mutants red | pass: mutants red; licence red on #470 (run 39170622155) and #489 (run 39987119032); vulnerability red on throwaway #496 (run 40231428226) | evidence/evidence.md, evidence/post-merge.md |
+| CWS-4 | meta-test group check (ref-keyed, unprefixed mutants); two pushes to a PR | first run cancelled | pass: mutants red locally; #495 runs 40224677510 and 40225572307 cancelled by the next push | evidence/evidence.md, evidence/post-merge.md |
+| CWS-4 | three quick merges to main each get a run | each SHA has a run | pass: five 2026-10-25 merges, one push run each | evidence/post-merge.md |
 | CWS-5 | meta-test with `push` removed (mutant) | mutant red | red locally | evidence/evidence.md |
 | CWS-6 | meta-test: guard in non-required job, unpinned mode | red | red locally (19 pre-existing orphans wired into `lint`: 4 e2e-side + 15 docs-side) | evidence/evidence.md |
-| CWS-6 | ruleset lists the four jobs as required | listed | post-merge | operator, task 5.1 |
+| CWS-6 | ruleset lists the four jobs as required | listed; skipped required job counts as passing | listed (10 contexts); skip probe open | evidence/post-merge.md, task 5.1 |
 | CWS-7 | self-test mutants plus no-op control, by exit status | each mutant red, no-op green | green locally (34 mutants + no-op) | evidence/evidence.md |
 | all | independent review; CI on the PR head | APPROVE, green; reviewed and archive revisions recorded | not-run | this PR |
@@ -0,0 +1,60 @@
+# backend-pool Specification
+
+## Purpose
+Defines the lifecycle of a pooled sink backend when its sink goes away: deleting a family sink
+(`KollectSnapshotSink`, `KollectDatabaseSink`, `KollectEventSink`) evicts and closes the backend
+at once, a backend whose build was in flight at the delete is discarded rather than pooled,
+eviction is best-effort and idempotent, and the idle TTL stays only as the backstop.
+
+## Requirements
+
+### Requirement: BEP-1 Deleting a family sink evicts its pooled backend immediately
+
+When a `KollectSnapshotSink`, `KollectDatabaseSink` or `KollectEventSink` is deleted, the process
+pool SHALL drop the cached backend entry for that sink's object UID and Close it, instead of
+holding it for the idle TTL; when the delete event's object carries no UID, the eviction SHALL
+fall back to the sink's namespace/name key.
+
+#### Scenario: Deleted sink's pooled entry is evicted and closed
+
+- **WHEN** a sink's backend is pooled (an export ran) and the sink object is then deleted
+- **THEN** the pool no longer holds an entry for that sink's UID and the pooled backend's Close has run
+
+#### Scenario: Live sinks are untouched
+
+- **WHEN** a sink that was never deleted is reconciled or its spec changes
+- **THEN** its pooled entry is not evicted by the delete hook (spec-hash changes are handled by the existing pool swap on acquire)
+
+### Requirement: BEP-1.2 A backend built during the delete is discarded, not pooled
+
+A backend whose build was still in flight when the delete event landed SHALL be discarded instead
+of pooled for the deleted sink (delete-tombstone), so eviction cannot be undone by the re-store
+race.
+
+#### Scenario: In-flight build cannot re-pool after eviction
+
+- **WHEN** an acquire-build for a sink is in flight and the sink is deleted before the build's store lands
+- **THEN** the built backend is discarded, not pooled
+
+### Requirement: BEP-1.3 Eviction is best-effort and idempotent
+
+Eviction is best-effort and idempotent: it SHALL NOT re-export, retract, reconcile or clean up
+anything, and SHALL NOT fail when no entry is pooled. An in-flight export that already holds the
+evicted backend may fail against it; the sink no longer exists and no new acquire SHALL rebuild an
+entry for that sink's UID.
+
+#### Scenario: Eviction without a pooled entry is a no-op
+
+- **WHEN** a sink is deleted while its backend is not pooled (never exported, or already TTL-pruned)
+- **THEN** nothing fails and nothing is logged as an error
+
+### Requirement: BEP-2 The TTL remains the backstop, not the contract
+
+The idle-entry TTL SHALL keep its meaning (bound the pool and age out entries for sinks that
+stopped being reconciled for any reason). The delete hook SHALL NOT change the TTL, the pooling
+of live sinks, or the acquire-time spec-hash swap.
+
+#### Scenario: TTL still ages out stale entries
+
+- **WHEN** an entry is idle longer than `backendPoolTTL`
+- **THEN** the next acquire prunes it, as today