diff --git a/config/rulesets/README.adoc b/config/rulesets/README.adoc index 3d31b879..d8e3e296 100644 --- a/config/rulesets/README.adoc +++ b/config/rulesets/README.adoc @@ -6,7 +6,13 @@ baseline; `Optimus-Extras.json` carries the extras for repos on the Optimus prof `gates-only.json` / `gates.json` describe which workflow files become required contexts; `immutable-tags.json` is the tag canon. -== Nothing here is applied automatically +`branch-floor.json` and `tag-floor.json` are the *base protection floor* โ€” deliberately +**not** named `base*.json`, because `base.json` is the richer tier and the collision would +mislead every future reader. The floor is `deletion` + `non_fast_forward` with +`bypass_actors: []`; branch target scopes to `~DEFAULT_BRANCH`, tag target to `~ALL`. +`gcrypt-vault-class.txt` is the D50 exclusion class for that floor (see below). + +== The floor IS applied by a committed script; the richer tiers still are not `scripts/apply-tag-ruleset-canon.sh` converges *only* the tag ruleset, and its own header records the consequence: after the 2026-09-11 deployment wave the branch-side remediation was @@ -14,9 +20,51 @@ records the consequence: after the 2026-09-11 deployment wave the branch-side re `config/rulesets/`". That gap is why `Optimus-Extras.json` could keep four constraints ยง7.3 retired (`code_coverage`, `code_quality`, `required_deployments`, `copilot_code_review`) and re-add them on the next manual convergence, on 120-162 active rulesets estate-wide. The file is -now trimmed; the propagation mechanism is still missing by design, and this note is the +now trimmed; that propagation gap is still open for the richer tiers, and this note is the reminder that a template fix without an applier is a half fix. +`scripts/apply-branch-gates.sh` (required-status-check gates) and +`scripts/apply-protection-floor.sh` (the base floor) close the gap for their own scopes. +Both default to **report-only**; writes need `--apply`, never the inverse. + +=== Reading a floor requires `bypass_actors`, not just the rule list + +A ruleset's rules say what it forbids; `bypass_actors` says whom it forbids it to. A floor +read without the second column is not a floor. Measured 2026-09-23: `metadatastician`'s +`EstateTagging` carries every floor rule and **three actors that bypass it `always`**, and +`EstateBranching` carries **nine** bypass actors โ€” so "add the missing field to +`EstateBranching`" would have shipped a floor nine actors could walk through. The cure is a +**standalone** zero-bypass ruleset alongside (`Branch-Floor` 23868655, `Tag-Floor` 23868851), +never an edit to a richer ruleset: a floor that lives inside a richer ruleset dies the day +that ruleset is disabled, as 375 rulesets were on 2026-09-22. + +=== Org cover is a distinct state from repo cover + +`GET /repos/{o}/{r}/rulesets` returns org-inherited rulesets too, discriminated by +`.source_type`. They put rules **in force** but cannot be written per repo, so the applier +keeps them in a separate cover set and reports `ORG-INHERITED` rather than +`COVERED-BY-RICHER`. Conflating the two in either direction is a real defect: counting org +rulesets without unioning their rule types made `ORG-INHERITED` unreachable and reported 67 +already-covered repos as needing a write. An **absent** `.source_type` is a REFUSAL, never a +default to the writable arm. + +=== A half cover is not a cover + +`EstateBranching` carries `deletion` and not `non_fast_forward`. A repo covered only by it is +`WOULD-CREATE`, not covered. The regression suite pins this with a fixture of exactly that +shape. + +=== ๐Ÿšจ The gcrypt vault class must be excluded, by explicit list + +`hyperpolymath/dev-notes-vault` and `hyperpolymath/memory-vault` carry a deliberately +`deletion`-only `Gcrypt-Vault-Guard` (**D50**): git-remote-gcrypt **force-pushes on every +sync**, so `non_fast_forward` would silently stop the hourly backup at the next timer fire. +Any logic reading *"has `deletion`, lacks `non_fast_forward` โ‡’ complete the floor"* writes +exactly that rule. Membership is the explicit list in `gcrypt-vault-class.txt` and **never a +name regex** โ€” `reasonably-good-token-vault` and `befunge93-vault-cracker` match `/vault/` +and genuinely need the floor. A missing class file is a hard refusal, because an absent +exclusion list is indistinguishable from an empty one. + `scripts/plan-ruleset-constraint-repair.rb` is the sanctioned planner: it emits a reviewable PUT body for exactly those retired constraints, optionally dropping one integration's bypass once GitHub has rejected that app as no longer installed. It never calls GitHub. diff --git a/config/rulesets/branch-floor.json b/config/rulesets/branch-floor.json new file mode 100644 index 00000000..465908ac --- /dev/null +++ b/config/rulesets/branch-floor.json @@ -0,0 +1,16 @@ +{ + "name": "Branch-Floor", + "target": "branch", + "enforcement": "active", + "bypass_actors": [], + "conditions": { + "ref_name": { + "include": ["~DEFAULT_BRANCH"], + "exclude": [] + } + }, + "rules": [ + { "type": "deletion" }, + { "type": "non_fast_forward" } + ] +} diff --git a/config/rulesets/gcrypt-vault-class.txt b/config/rulesets/gcrypt-vault-class.txt new file mode 100644 index 00000000..ceb37657 --- /dev/null +++ b/config/rulesets/gcrypt-vault-class.txt @@ -0,0 +1,17 @@ +# The gcrypt-vault exclusion class -- owner ruling D50, 2026-09-14T20:58Z. +# +# A git-remote-gcrypt vault FORCE-PUSHES ON EVERY SYNC: the remote ref is a rolling +# pointer at the newest encrypted pack and is never a fast-forward of the previous one. +# Applying `non_fast_forward` to a member does not harden it -- it silently stops the +# hourly backup at the next timer fire. +# +# Members carry `Gcrypt-Vault-Guard` instead: `deletion` only, on `~ALL` (a gcrypt vault +# has exactly ONE remote ref and it is `master`, so a `main`-scoped rule is a fake gate). +# +# โš  MEMBERSHIP IS THIS LIST, NEVER A NAME MATCH. `reasonably-good-token-vault` and +# `befunge93-vault-cracker` both match /vault/ and are NOT members -- they are ordinary +# source repos that genuinely need the floor. +# +# Class record: dev-notes/estate-management/gcrypt-vault-protection-class-2026-09-14/ +hyperpolymath/dev-notes-vault +hyperpolymath/memory-vault diff --git a/config/rulesets/tag-floor.json b/config/rulesets/tag-floor.json new file mode 100644 index 00000000..5b2029dd --- /dev/null +++ b/config/rulesets/tag-floor.json @@ -0,0 +1,16 @@ +{ + "name": "Tag-Floor", + "target": "tag", + "enforcement": "active", + "bypass_actors": [], + "conditions": { + "ref_name": { + "include": ["~ALL"], + "exclude": [] + } + }, + "rules": [ + { "type": "deletion" }, + { "type": "non_fast_forward" } + ] +} diff --git a/scripts/apply-protection-floor.sh b/scripts/apply-protection-floor.sh new file mode 100755 index 00000000..a3c3a557 --- /dev/null +++ b/scripts/apply-protection-floor.sh @@ -0,0 +1,375 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# apply-protection-floor.sh -- establish the BASE PROTECTION FLOOR on a repo that has none. +# +# The floor is owner ruling D94: `deletion` + `non_fast_forward`, and nothing else. +# * branch target -> `~DEFAULT_BRANCH` (config/rulesets/branch-floor.json) +# * tag target -> `~ALL` (config/rulesets/tag-floor.json) +# Bypass is the empty list, per D96. +# +# WHY A THIRD APPLIER, BESIDE apply-branch-gates.sh AND apply-tag-ruleset-canon.sh +# apply-branch-gates.sh says so itself: "It never CREATES a ruleset. A repo with no +# active branch ruleset is reported NORULESET. Creating branch protection where none +# exists is a policy act, not a gate-fill." That policy act is now ruled (D94-D96), so +# it gets its own script and the gate-filler stays a gate-filler. +# +# WHY THE FLOOR IS EXACTLY TWO RULES +# It needs no per-repo derivation and no CI check to be satisfiable, so it can never be +# a vacuous gate and it cannot block a single PR. It stops precisely two irreversible +# accidents: deleting the default branch, and force-pushing over it. +# +# WHAT IT DELIBERATELY DOES NOT DO +# * It never EDITS an existing ruleset. Rulesets are additive (D95); the floor is POSTed +# alongside. A PUT would replace the whole object, and that is how an applier silently +# revives what a human switched off (the defect repaired in #1030). +# * It never re-enables a disabled ruleset. Disabled is a decision, not drift. +# * It never writes an ORG-INHERITED ruleset. `PUT` to a repo path for an org ruleset +# 404s; the discriminator is `.source_type`, and an ABSENT discriminator REFUSES +# rather than defaulting to the writable arm. +# * It never touches a member of the gcrypt-vault class. See below -- this one is not a +# nicety, it is the difference between a hardened repo and a dead backup. +# +# ๐Ÿšจ THE GCRYPT-VAULT EXCLUSION (owner ruling D50) +# git-remote-gcrypt FORCE-PUSHES ON EVERY SYNC. `non_fast_forward` on a vault does not +# harden it; it stops the hourly backup, silently, at the next timer fire. Vaults carry +# a deliberate `deletion`-only guard, which means naive floor logic -- "has deletion, +# lacks non_fast_forward, therefore complete the floor" -- writes exactly the fatal rule. +# Membership is the EXPLICIT LIST in config/rulesets/gcrypt-vault-class.txt and NEVER a +# name match: two estate repos match /vault/ and are not members. +# A missing class file is a REFUSAL, not an empty exclusion set. +# +# USAGE +# scripts/apply-protection-floor.sh --repos # report only (default) +# scripts/apply-protection-floor.sh --repos --apply # actually create +# scripts/apply-protection-floor.sh --repos --target tag +# Optional: --floor-even-if-covered also floor repos whose cover comes from a RICHER +# ruleset (see COVERED-BY-RICHER below). Off by +# default: that is a policy call, not a gap-fill. +# +# STATES (TSV: repo state detail) +# CONVERGED the exact floor already exists -- nothing to do +# COVERED-BY-RICHER both rules are in force, but from a richer ruleset. NOT the same +# as converged: that cover vanishes the moment the richer ruleset is +# disabled, which is how 230 repos lost protection on 2026-09-22. +# WOULD-CREATE report mode; --apply would POST the floor here +# CREATED POSTed and verified in force +# WROTE-UNVERIFIED POSTed, but the verifying read did not come back. Never assume. +# EXCLUDED-D50 gcrypt vault; no write, ever +# ARCHIVED archived repo; ruleset POST 403s. Skipped, not failed. +# PLAN-EXCLUDED 403/422 from the rulesets endpoint (private repo / plan limit). +# Counted as neither covered nor failed -- this is the honest +# denominator for "repos this tooling cannot reach". +# ORG-INHERITED covered by an org ruleset; cure once at the org, never per repo +# AMBIGUOUS more than one active repo-level floor-shaped ruleset. Fail closed. +# REFUSED a ruleset carried no `.source_type`. Fail closed. +# UNKNOWN a read was throttled or errored. SKIPPED, never recorded as clean. +# +# A THROTTLED READ IS SKIPPED, NEVER RECORDED. A junk row is indistinguishable from an +# honest one, and resume logic keyed on "repo already present" excludes it forever. +set -uo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +CONF="$REPO_ROOT/config/rulesets" +VAULT_CLASS="$CONF/gcrypt-vault-class.txt" + +APPLY=0 +TARGET="branch" +REPOS_FILE="" +FLOOR_EVEN_IF_COVERED=0 + +TMPDIR_ERR="$(mktemp -t protfloor-err.XXXXXX)" +# Org rulesets are IDENTICAL across every repo in the org, so their bodies are fetched +# once and cached by ruleset id rather than re-read per repo. +ORG_CACHE="$(mktemp -d -t protfloor-org.XXXXXX)" +# Remove the temporary error file and organization-ruleset cache. +cleanup() { rm -f "$TMPDIR_ERR"; rm -rf "$ORG_CACHE"; } +trap cleanup EXIT + +# Print a fatal error and exit with the script's refusal status. +die() { printf 'FATAL: %s\n' "$*" >&2; exit 2; } +# Emit one tab-separated repository status row. +report() { printf '%s\t%s\t%s\n' "$1" "$2" "${3:-}"; } + +while [ $# -gt 0 ]; do + case "$1" in + --apply) APPLY=1 ;; + --target) TARGET="${2:-}"; shift ;; + --repos) REPOS_FILE="${2:-}"; shift ;; + --floor-even-if-covered) FLOOR_EVEN_IF_COVERED=1 ;; + -h|--help) sed -n '2,60p' "${BASH_SOURCE[0]}"; exit 0 ;; + *) die "unknown argument: $1" ;; + esac + shift +done + +case "$TARGET" in + branch) CANON="$CONF/branch-floor.json"; WANT_INCLUDE='["~DEFAULT_BRANCH"]' ;; + tag) CANON="$CONF/tag-floor.json"; WANT_INCLUDE='["~ALL"]' ;; + *) die "--target must be 'branch' or 'tag', got '$TARGET'" ;; +esac + +[ -r "$CANON" ] || die "canon file missing: $CANON" + +# FAIL CLOSED. A missing class file must never read as "no vaults to protect". +[ -r "$VAULT_CLASS" ] || die "gcrypt-vault class file missing: $VAULT_CLASS -- refusing to run, because an absent exclusion list is indistinguishable from an empty one and D50 members would be written" + +[ -n "$REPOS_FILE" ] || die "--repos is required (one owner/repo per line)" +[ -r "$REPOS_FILE" ] || die "repo list not readable: $REPOS_FILE" + +# The floor's rule types, derived FROM THE CANON -- never typed a second time here, so the +# script and the file can never disagree. +FLOOR_TYPES="$(jq -r '[.rules[].type] | sort | join(",")' "$CANON")" +[ -n "$FLOOR_TYPES" ] || die "canon $CANON declares no rules" + +# Body actually POSTed. `name` is kept (a POST creates, so the name is ours to set). +CANON_BODY="$(jq -c . "$CANON")" + +VAULTS="$(command grep -vE '^[[:space:]]*(#|$)' "$VAULT_CLASS" | tr -d ' \t')" +[ -n "$VAULTS" ] || die "gcrypt-vault class file lists no members -- refusing; D50 names two" + +TARGETS="$(command grep -vE '^[[:space:]]*(#|$)' "$REPOS_FILE" | tr -d ' \t' | sort -u)" +[ -n "$TARGETS" ] || die "refusing to report a clean sweep over nothing: $REPOS_FILE yielded no repos" + +# A THROTTLED READ IS NOT A PLAN EXCLUSION. GitHub answers a primary rate limit, a +# secondary limit and an abuse trip all with 403 -- the same status a private repo on a +# plan without rulesets returns. Only the body text separates them, so throttling must +# be classified FIRST: matching *403* alone records a throttled repo as PLAN-EXCLUDED +# ("private repo / plan limit"), which silently UNDER-REPORTS the protection gap. +# Return success when an API error message identifies throttling. +is_throttled() { + case "$(printf '%s' "$1" | tr '[:upper:]' '[:lower:]')" in + *"rate limit"*|*"rate-limit"*|*"abuse"*|*"retry-after"*|*"http 429"*) return 0 ;; + *) return 1 ;; + esac +} + +# A throttle is a property of the CREDENTIAL, not of the repo: once the hourly window is +# spent every remaining repo fails identically, so grinding on turns a 267-repo sweep into +# 267 UNKNOWN rows -- measured twice on 2026-09-23. Stop once the wall is CONFIRMED, but +# tolerate a single secondary-limit blip: the count is CONSECUTIVE and any read that +# succeeds clears it. +THROTTLE_LIMIT=3 +throttled=0 +# Report a throttled operation and exit with status 3 at the consecutive-throttle limit. +note_throttled() { # repo operation + throttled=$((throttled + 1)) + repo_throttled=1 + report "$1" "UNKNOWN" "$2 throttled; skipped rather than recorded" + if [ "$throttled" -ge "$THROTTLE_LIMIT" ]; then + report "-" "ABORTED" "$THROTTLE_LIMIT consecutive throttled reads: the quota window is spent, so every remaining repo would report UNKNOWN. Re-run after the reset -- trust the X-RateLimit-Reset header, not gh api rate_limit, which has reported 5000 remaining against a header saying 0." + exit 3 + fi +} + +# Return success when the repository is an explicitly listed gcrypt vault. +is_vault() { + printf '%s\n' "$VAULTS" | command grep -qxF "$1" +} + +# --------------------------------------------------------------------------- +printf 'repo\tstate\tdetail\n' + +printf '%s\n' "$TARGETS" | while IFS= read -r repo; do + [ -n "$repo" ] || continue + + # The streak counts consecutive REPOS, not consecutive CALLS. At the wall a repo's GETs + # can still succeed while only its POST is refused -- 87 of the 111 UNKNOWN rows in the + # 2026-09-23 run were exactly that shape -- so resetting on any successful read pins the + # count at 1 forever and the guard never fires. Reset only when the PREVIOUS repo + # completed without being throttled at all. + if [ "${repo_throttled:-0}" -eq 0 ]; then throttled=0; fi + repo_throttled=0 + + # 1. D50 FIRST, before any read. A vault must not even be a candidate. + if is_vault "$repo"; then + report "$repo" "EXCLUDED-D50" "gcrypt vault: force-pushes every sync; non_fast_forward would stop the backup" + continue + fi + + # 2. Archived repos 403 on a ruleset write while every GET succeeds. + if ! meta="$(gh api "repos/$repo" 2>"$TMPDIR_ERR")"; then + err="$(cat "$TMPDIR_ERR" 2>/dev/null)" + if is_throttled "$err"; then note_throttled "$repo" "repos/$repo"; continue; fi + report "$repo" "UNKNOWN" "repos/$repo read failed: ${err%%$'\n'*}" + continue + fi + [ -n "$meta" ] || { report "$repo" "UNKNOWN" "repos/$repo returned empty"; continue; } + if [ "$(printf '%s' "$meta" | jq -r '.archived')" = "true" ]; then + report "$repo" "ARCHIVED" "ruleset POST 403s on an archived repo; unarchive/write/re-archive is a separate, explicit act" + continue + fi + default_branch="$(printf '%s' "$meta" | jq -r '.default_branch // empty')" + + # 3. List rulesets. A 403 is ambiguous: throttle first, THEN the private-repo / plan arm. + if ! listing="$(gh api "repos/$repo/rulesets" 2>"$TMPDIR_ERR")"; then + err="$(cat "$TMPDIR_ERR" 2>/dev/null)" + if is_throttled "$err"; then + note_throttled "$repo" "rulesets list" + continue + fi + case "$err" in + *403*|*422*|*"upgrade"*) report "$repo" "PLAN-EXCLUDED" "rulesets endpoint refused: ${err%%$'\n'*}" ;; + *) report "$repo" "UNKNOWN" "rulesets list failed: ${err%%$'\n'*}" ;; + esac + continue + fi + [ -n "$listing" ] || { report "$repo" "UNKNOWN" "rulesets list returned empty"; continue; } + + # 4. An entry with no `.source_type` is a REFUSAL, not a default to the writable arm. + if printf '%s' "$listing" | jq -e 'any(.[]?; has("source_type") | not)' >/dev/null 2>&1; then + report "$repo" "REFUSED" "a ruleset carried no .source_type; cannot tell repo-level from org-inherited, failing closed" + continue + fi + + org_n="$(printf '%s' "$listing" | jq "[.[]? | select(.source_type==\"Organization\" and .target==\"$TARGET\" and .enforcement==\"active\")] | length")" + repo_ids="$(printf '%s' "$listing" | jq -r ".[]? | select(.source_type==\"Repository\" and .target==\"$TARGET\" and .enforcement==\"active\") | .id")" + org_ids="$(printf '%s' "$listing" | jq -r ".[]? | select(.source_type==\"Organization\" and .target==\"$TARGET\" and .enforcement==\"active\") | .id")" + + # 5. Walk the active repo-level rulesets of this target and classify. + exact_n=0; exact_ids=""; union="" + if [ -n "$repo_ids" ]; then + while IFS= read -r rid; do + [ -n "$rid" ] || continue + body="$(gh api "repos/$repo/rulesets/$rid" 2>/dev/null)" || { body=""; } + if [ -z "$body" ]; then + exact_n=-1 # sentinel: a read we could not complete + break + fi + types="$(printf '%s' "$body" | jq -r '[.rules[].type] | sort | join(",")')" + inc="$(printf '%s' "$body" | jq -c '.conditions.ref_name.include')" + exc="$(printf '%s' "$body" | jq -c '[.conditions.ref_name.exclude[]?] | length')" + byp="$(printf '%s' "$body" | jq -c '[.bypass_actors[]?] | length')" + if [ "$exc" = "0" ] && printf '%s' "$body" | jq -e --argjson w "$WANT_INCLUDE" \ + 'any(.conditions.ref_name.include[]?; . == $w[0])' >/dev/null 2>&1; then + union="$union,$types" + fi + if [ "$types" = "$FLOOR_TYPES" ] && [ "$inc" = "$WANT_INCLUDE" ] && [ "$exc" = "0" ] && [ "$byp" = "0" ]; then + exact_n=$((exact_n + 1)); exact_ids="$exact_ids $rid" + fi + done <"$TMPDIR_ERR")" && + printf '%s' "$org_body" | jq -e '.rules | type == "array"' >/dev/null 2>&1; then + printf '%s' "$org_body" > "$cache" + else + err="$(cat "$TMPDIR_ERR" 2>/dev/null)" + if [ ! -s "$cache" ]; then + if tmpb="$(gh api "repos/$repo/rulesets/$rid" 2>"$TMPDIR_ERR")" \ + && printf '%s' "$tmpb" | jq -e '.rules | type == "array"' >/dev/null 2>&1; then + printf '%s' "$tmpb" > "$cache" + elif is_throttled "$(cat "$TMPDIR_ERR" 2>/dev/null)"; then + note_throttled "$repo" "org ruleset $rid"; org_read_ok=0; break + fi + fi + if [ ! -s "$cache" ]; then org_read_ok=0; break; fi + if jq -e --argjson w "$WANT_INCLUDE" \ + '([.conditions.ref_name.exclude[]?] | length) == 0 and + any(.conditions.ref_name.include[]?; . == $w[0])' "$cache" >/dev/null 2>&1; then + union_org="$union_org,$(jq -r '[.rules[].type] | sort | join(",")' "$cache")" + fi + b="$(jq -r '[.bypass_actors[]?] | length' "$cache")" + [ "$b" -gt "$org_byp_max" ] && org_byp_max="$b" + done <"$TMPDIR_ERR")"; then + err="$(cat "$TMPDIR_ERR" 2>/dev/null)" + if is_throttled "$err"; then note_throttled "$repo" "ruleset POST"; continue; fi + report "$repo" "UNKNOWN" "POST failed: ${err%%$'\n'*}" + continue + fi + new_id="$(printf '%s' "$created" | jq -r '.id // empty')" + + # 8. VERIFY BY AN INDEPENDENT READ. A ruleset write has returned 200 with an empty body + # and not applied -- never trust the write's own response. + if [ "$TARGET" = "branch" ] && [ -n "$default_branch" ]; then + eff="$(gh api "repos/$repo/rules/branches/$default_branch" 2>/dev/null)" || eff="" + if [ -z "$eff" ]; then + report "$repo" "WROTE-UNVERIFIED" "POSTed id=$new_id; effective-rules read did not return" + continue + fi + missing="" + printf '%s\n' "$FLOOR_TYPES" | tr ',' '\n' | while IFS= read -r t; do + [ -n "$t" ] || continue + printf '%s' "$eff" | jq -e --arg t "$t" 'any(.[]?; .type==$t)' >/dev/null 2>&1 || exit 7 + done || missing="yes" + if [ -n "$missing" ]; then + report "$repo" "WROTE-UNVERIFIED" "POSTed id=$new_id but the effective rules on $default_branch do not show both floor types" + continue + fi + report "$repo" "CREATED" "id=$new_id; verified effective on $default_branch" + else + back="$(gh api "repos/$repo/rulesets/$new_id" 2>/dev/null)" || back="" + got="$(printf '%s' "$back" | jq -r '[.rules[].type] | sort | join(",")' 2>/dev/null)" + if [ "$got" = "$FLOOR_TYPES" ]; then + report "$repo" "CREATED" "id=$new_id; verified by re-read" + else + report "$repo" "WROTE-UNVERIFIED" "POSTed id=$new_id; re-read returned '${got:-}'" + fi + fi +done diff --git a/scripts/tests/protection-floor-test.sh b/scripts/tests/protection-floor-test.sh new file mode 100755 index 00000000..9d4c2edd --- /dev/null +++ b/scripts/tests/protection-floor-test.sh @@ -0,0 +1,459 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# Regression suite for scripts/apply-protection-floor.sh +# +# House conventions, shared with scripts/tests/branch-gates-apply-test.sh: +# * a `gh` shim maps an API path to a fixture by KEY=$(tr '/?&=' '____') +# * A MISSING FIXTURE IS A FREE ASSERTION that the path is never queried: the shim +# exits 1, so any code reaching for an unplanned endpoint fails loudly. +# * every write is appended to $GH_FIX/PUTS.log, so "wrote nothing" is checkable. +# +# The suite ends by KILLING FOUR MUTANTS. A green suite against the real script proves +# only that it agrees with itself; each mutant reintroduces one specific defect and the +# suite must go red for exactly the right reason. +set -uo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT="$(cd "$HERE/../.." && pwd)" +SUT="$ROOT/scripts/apply-protection-floor.sh" +[ -r "$SUT" ] || { echo "FATAL: script under test missing: $SUT"; exit 2; } + +PASS=0; FAIL=0 +# Record a passing assertion and print its description. +ok() { PASS=$((PASS+1)); printf ' ok %s\n' "$1"; } +# Record a failing assertion and print its expected and actual values. +bad() { FAIL=$((FAIL+1)); printf ' FAIL %s\n expected: %s\n actual: %s\n' "$1" "$2" "$3"; } +# Compare expected and actual values, then record the assertion result. +check(){ if [ "$2" = "$3" ]; then ok "$1"; else bad "$1" "$2" "$3"; fi; } + +WORK="$(mktemp -d -t protfloor-test.XXXXXX)" +trap 'rm -rf "$WORK"' EXIT +BIN="$WORK/bin"; FIX="$WORK/fix"; mkdir -p "$BIN" "$FIX" + +cat > "$BIN/gh" <<'SHIM' +#!/usr/bin/env bash +# fixture shim: repos/x/y -> $GH_FIX/repos_x_y +method="GET"; path=""; stdin_body="" +while [ $# -gt 0 ]; do + case "$1" in + api) ;; + --method) method="$2"; shift ;; + --input) stdin_body="$(cat)"; shift ;; + -*) ;; + *) [ -z "$path" ] && path="$1" ;; + esac + shift +done +KEY="$(printf '%s' "$path" | tr '/?&=' '____')" +if [ "$method" != "GET" ]; then + printf '%s\t%s\t%s\n' "$method" "$path" "$stdin_body" >> "$GH_FIX/PUTS.log" + F="$GH_FIX/${method}_${KEY}" + [ -r "$F" ] || { echo "no fixture for $method $path" >&2; exit 1; } + # A write can be throttled while every GET still succeeds -- the dominant wall shape. + if [ -r "$F.rc" ]; then cat "$F" >&2; exit "$(cat "$F.rc")"; fi + cat "$F"; exit 0 +fi +F="$GH_FIX/$KEY" +[ -r "$F" ] || { echo "no fixture for $path" >&2; exit 1; } +if [ -r "$F.rc" ]; then cat "$F" >&2; exit "$(cat "$F.rc")"; fi +cat "$F" +SHIM +chmod 755 "$BIN/gh" + +# Run a script with the fixture-backed GitHub CLI shim. +run() { GH_FIX="$FIX" PATH="$BIN:$PATH" bash "$1" "${@:2}" 2>/dev/null; } +# Extract one repository's state from tab-separated report output. +state() { printf '%s\n' "$1" | awk -F'\t' -v r="$2" '$1==r{print $2}'; } +# Count the report rows for one repository. +rows() { printf '%s\n' "$1" | awk -F'\t' -v r="$2" '$1==r{n++} END{print n+0}'; } +# Recreate the fixture directory and initialize an empty write log. +reset_fix() { rm -rf "$FIX"; mkdir -p "$FIX"; : > "$FIX/PUTS.log"; } + +# Write repository metadata from a name, default branch, and archived flag. +mkrepo() { # name default-branch archived + printf '{"archived":%s,"default_branch":"%s"}\n' "$3" "$2" > "$FIX/repos_$(printf '%s' "$1" | tr '/' '_')" +} + +REPOS="$WORK/repos.txt" +# A SECOND target list for the backoff section. The applier sorts its targets, so the +# three throttled repos are named to sort FIRST and plain-repo to sort after them: +# the assertion is that the sweep never reaches it. +THR="$WORK/throttle-repos.txt" +cat > "$THR" <<'EOF' +hyperpolymath/aaa-throttle-1 +hyperpolymath/aaa-throttle-2 +hyperpolymath/aaa-throttle-3 +hyperpolymath/plain-repo +EOF + +# A THIRD list: the wall shape that actually happened. Both GETs succeed and only the +# POST is refused (87 of the 111 UNKNOWN rows on 2026-09-23), so a streak that resets on +# any successful read never reaches the limit and the backoff is decorative. +THRP="$WORK/post-throttle-repos.txt" +cat > "$THRP" <<'EOF' +hyperpolymath/aab-postthrottle-1 +hyperpolymath/aab-postthrottle-2 +hyperpolymath/aab-postthrottle-3 +hyperpolymath/plain-repo +EOF +ORG_EDGE="$WORK/org-edge-repos.txt" +cat > "$ORG_EDGE" <<'EOF' +metadatastician/org-cache-invalid-a +metadatastician/org-cache-retry-b +metadatastician/org-release-only +metadatastician/org-throttled +EOF +cat > "$REPOS" <<'EOF' +hyperpolymath/memory-vault +hyperpolymath/plain-repo +hyperpolymath/converged-repo +hyperpolymath/richer-repo +hyperpolymath/bypassed-twin +hyperpolymath/archived-repo +hyperpolymath/private-repo +hyperpolymath/throttled-repo +hyperpolymath/secondary-throttled-repo +hyperpolymath/nosourcetype-repo +metadatastician/org-covered-repo +metadatastician/org-halffloor-repo +EOF + +# Rebuild fixtures, optionally making excluded repositories fully writable. +build_fixtures() { # $1 = "with-vault" to give the vault a complete, writable fixture set + reset_fix + # By default memory-vault gets NO fixtures at all. The D50 guard must return before any + # read, so every missing fixture here asserts that nothing was queried. + # With "with-vault" the vault is made fully writable, so a mutant that removes the guard + # produces a REAL POST rather than dying on a missing fixture -- otherwise the mutant's + # red would measure the shim, not the guard. + if [ "${1:-}" = "with-vault" ]; then + mkrepo hyperpolymath/memory-vault main false + echo '[]' > "$FIX/repos_hyperpolymath_memory-vault_rulesets" + echo '{"id":9099}' > "$FIX/POST_repos_hyperpolymath_memory-vault_rulesets" + echo '[{"type":"deletion"},{"type":"non_fast_forward"}]' \ + > "$FIX/repos_hyperpolymath_memory-vault_rules_branches_main" + fi + + mkrepo hyperpolymath/plain-repo main false + echo '[]' > "$FIX/repos_hyperpolymath_plain-repo_rulesets" + echo '{"id":9001}' > "$FIX/POST_repos_hyperpolymath_plain-repo_rulesets" + echo '[{"type":"deletion"},{"type":"non_fast_forward"}]' \ + > "$FIX/repos_hyperpolymath_plain-repo_rules_branches_main" + + mkrepo hyperpolymath/converged-repo main false + echo '[{"id":10,"source_type":"Repository","target":"branch","enforcement":"active"}]' \ + > "$FIX/repos_hyperpolymath_converged-repo_rulesets" + cat > "$FIX/repos_hyperpolymath_converged-repo_rulesets_10" <<'J' +{"id":10,"rules":[{"type":"deletion"},{"type":"non_fast_forward"}], + "conditions":{"ref_name":{"include":["~DEFAULT_BRANCH"],"exclude":[]}},"bypass_actors":[]} +J + + mkrepo hyperpolymath/richer-repo main false + echo '[{"id":20,"source_type":"Repository","target":"branch","enforcement":"active"}]' \ + > "$FIX/repos_hyperpolymath_richer-repo_rulesets" + cat > "$FIX/repos_hyperpolymath_richer-repo_rulesets_20" <<'J' +{"id":20,"rules":[{"type":"deletion"},{"type":"non_fast_forward"},{"type":"required_signatures"}], + "conditions":{"ref_name":{"include":["~DEFAULT_BRANCH"],"exclude":[]}},"bypass_actors":[]} +J + + # same rule set and same include as the floor, but WITH a bypass actor. + mkrepo hyperpolymath/bypassed-twin main false + echo '[{"id":30,"source_type":"Repository","target":"branch","enforcement":"active"}]' \ + > "$FIX/repos_hyperpolymath_bypassed-twin_rulesets" + cat > "$FIX/repos_hyperpolymath_bypassed-twin_rulesets_30" <<'J' +{"id":30,"rules":[{"type":"deletion"},{"type":"non_fast_forward"}], + "conditions":{"ref_name":{"include":["~DEFAULT_BRANCH"],"exclude":[]}}, + "bypass_actors":[{"actor_id":5,"actor_type":"RepositoryRole","bypass_mode":"always"}]} +J + + mkrepo hyperpolymath/archived-repo main true + # By default NO rulesets fixture: the archived guard must return before any such read. + # In mutant mode the repo is made fully writable, so a mutant that removes the guard + # produces a REAL POST instead of dying on a missing fixture. + if [ "${1:-}" = "with-vault" ]; then + echo '[]' > "$FIX/repos_hyperpolymath_archived-repo_rulesets" + echo '{"id":9098}' > "$FIX/POST_repos_hyperpolymath_archived-repo_rulesets" + echo '[{"type":"deletion"},{"type":"non_fast_forward"}]' \ + > "$FIX/repos_hyperpolymath_archived-repo_rules_branches_main" + fi + + mkrepo hyperpolymath/private-repo main false + printf 'HTTP 403: Upgrade to GitHub Pro or make this repository public\n' \ + > "$FIX/repos_hyperpolymath_private-repo_rulesets" + echo 1 > "$FIX/repos_hyperpolymath_private-repo_rulesets.rc" + + # A THROTTLE ALSO ANSWERS 403, with the SAME status as the plan refusal above. + # These two exist so the discriminator cannot go back to matching *403* alone: + # that recorded a throttled repo as PLAN-EXCLUDED and under-reported the gap. + mkrepo hyperpolymath/throttled-repo main false + printf 'HTTP 403: API rate limit exceeded for user ID 12345. (https://api.github.com/repos/hyperpolymath/throttled-repo/rulesets)\n' \ + > "$FIX/repos_hyperpolymath_throttled-repo_rulesets" + echo 1 > "$FIX/repos_hyperpolymath_throttled-repo_rulesets.rc" + + mkrepo hyperpolymath/secondary-throttled-repo main false + printf 'HTTP 403: You have exceeded a secondary rate limit. Please wait a few minutes before you try again.\n' \ + > "$FIX/repos_hyperpolymath_secondary-throttled-repo_rulesets" + echo 1 > "$FIX/repos_hyperpolymath_secondary-throttled-repo_rulesets.rc" + + # The wall shows on the FIRST read of a repo, before any ruleset endpoint is touched. + mkrepo hyperpolymath/aaa-throttle-1 main false + mkrepo hyperpolymath/aaa-throttle-2 main false + mkrepo hyperpolymath/aaa-throttle-3 main false + printf 'HTTP 403: API rate limit exceeded for user ID 12345.\n' > "$FIX/repos_hyperpolymath_aaa-throttle-1" + printf 'HTTP 403: API rate limit exceeded for user ID 12345.\n' > "$FIX/repos_hyperpolymath_aaa-throttle-2" + printf 'HTTP 403: API rate limit exceeded for user ID 12345.\n' > "$FIX/repos_hyperpolymath_aaa-throttle-3" + echo 1 > "$FIX/repos_hyperpolymath_aaa-throttle-1.rc" + echo 1 > "$FIX/repos_hyperpolymath_aaa-throttle-2.rc" + echo 1 > "$FIX/repos_hyperpolymath_aaa-throttle-3.rc" + + # The POST-throttle triple: full, healthy GET fixtures, and only the write refused. + mkrepo hyperpolymath/aab-postthrottle-1 main false + mkrepo hyperpolymath/aab-postthrottle-2 main false + mkrepo hyperpolymath/aab-postthrottle-3 main false + echo '[]' > "$FIX/repos_hyperpolymath_aab-postthrottle-1_rulesets" + echo '[]' > "$FIX/repos_hyperpolymath_aab-postthrottle-2_rulesets" + echo '[]' > "$FIX/repos_hyperpolymath_aab-postthrottle-3_rulesets" + printf 'HTTP 403: You have exceeded a secondary rate limit.\n' > "$FIX/POST_repos_hyperpolymath_aab-postthrottle-1_rulesets" + printf 'HTTP 403: You have exceeded a secondary rate limit.\n' > "$FIX/POST_repos_hyperpolymath_aab-postthrottle-2_rulesets" + printf 'HTTP 403: You have exceeded a secondary rate limit.\n' > "$FIX/POST_repos_hyperpolymath_aab-postthrottle-3_rulesets" + echo 1 > "$FIX/POST_repos_hyperpolymath_aab-postthrottle-1_rulesets.rc" + echo 1 > "$FIX/POST_repos_hyperpolymath_aab-postthrottle-2_rulesets.rc" + echo 1 > "$FIX/POST_repos_hyperpolymath_aab-postthrottle-3_rulesets.rc" + + mkrepo hyperpolymath/nosourcetype-repo main false + echo '[{"id":40,"target":"branch","enforcement":"active"}]' \ + > "$FIX/repos_hyperpolymath_nosourcetype-repo_rulesets" + + # An ORG-inherited ruleset that CARRIES the whole floor. It is not writable per repo, + # so the only correct answer is ORG-INHERITED -- never a per-repo duplicate. + mkrepo metadatastician/org-covered-repo main false + echo '[{"id":60,"source_type":"Organization","target":"branch","enforcement":"active"}]' \ + > "$FIX/repos_metadatastician_org-covered-repo_rulesets" + cat > "$FIX/repos_metadatastician_org-covered-repo_rulesets_60" <<'J' +{"id":60,"rules":[{"type":"deletion"},{"type":"non_fast_forward"},{"type":"required_signatures"}], + "conditions":{"ref_name":{"include":["~DEFAULT_BRANCH"],"exclude":[]}},"bypass_actors":[]} +J + # Writable on purpose: a mutant that drops the org union must produce a REAL duplicate + # POST here, not die on a missing fixture (the false-green trap this suite already hit). + echo '{"id":9060}' > "$FIX/POST_repos_metadatastician_org-covered-repo_rulesets" + echo '[{"type":"deletion"},{"type":"non_fast_forward"}]' \ + > "$FIX/repos_metadatastician_org-covered-repo_rules_branches_main" + + # The real EstateBranching shape: an org ruleset carrying HALF the floor (deletion, no + # non_fast_forward). A half cover is NOT a cover; this repo must still be WOULD-CREATE. + mkrepo metadatastician/org-halffloor-repo main false + echo '[{"id":61,"source_type":"Organization","target":"branch","enforcement":"active"}]' \ + > "$FIX/repos_metadatastician_org-halffloor-repo_rulesets" + cat > "$FIX/repos_metadatastician_org-halffloor-repo_rulesets_61" <<'J' +{"id":61,"rules":[{"type":"deletion"},{"type":"pull_request"}], + "conditions":{"ref_name":{"include":["~DEFAULT_BRANCH"],"exclude":[]}},"bypass_actors":[{"actor_id":1}]} +J + echo '{"id":9061}' > "$FIX/POST_repos_metadatastician_org-halffloor-repo_rulesets" + echo '[{"type":"deletion"},{"type":"non_fast_forward"}]' \ + > "$FIX/repos_metadatastician_org-halffloor-repo_rules_branches_main" + + # A successful response without a rules array must not poison the org cache. The next + # repo inherits the same org ruleset id and must retry the read rather than reuse it. + mkrepo metadatastician/org-cache-invalid-a main false + mkrepo metadatastician/org-cache-retry-b main false + echo '[{"id":62,"source_type":"Organization","target":"branch","enforcement":"active"}]' \ + > "$FIX/repos_metadatastician_org-cache-invalid-a_rulesets" + cp "$FIX/repos_metadatastician_org-cache-invalid-a_rulesets" \ + "$FIX/repos_metadatastician_org-cache-retry-b_rulesets" + echo '{"id":62,"message":"rules temporarily unavailable"}' \ + > "$FIX/repos_metadatastician_org-cache-invalid-a_rulesets_62" + cat > "$FIX/repos_metadatastician_org-cache-retry-b_rulesets_62" <<'J' +{"id":62,"rules":[{"type":"deletion"},{"type":"non_fast_forward"}], + "conditions":{"ref_name":{"include":["~DEFAULT_BRANCH"],"exclude":[]}},"bypass_actors":[]} +J + + # Both floor rules on release branches do not cover the default branch. + mkrepo metadatastician/org-release-only main false + echo '[{"id":63,"source_type":"Organization","target":"branch","enforcement":"active"}]' \ + > "$FIX/repos_metadatastician_org-release-only_rulesets" + cat > "$FIX/repos_metadatastician_org-release-only_rulesets_63" <<'J' +{"id":63,"rules":[{"type":"deletion"},{"type":"non_fast_forward"}], + "conditions":{"ref_name":{"include":["refs/heads/release/*"],"exclude":[]}},"bypass_actors":[]} +J + + # A throttled org-body read is classified by note_throttled and reported once. + mkrepo metadatastician/org-throttled main false + echo '[{"id":64,"source_type":"Organization","target":"branch","enforcement":"active"}]' \ + > "$FIX/repos_metadatastician_org-throttled_rulesets" + printf 'HTTP 403: API rate limit exceeded for user ID 12345.\n' \ + > "$FIX/repos_metadatastician_org-throttled_rulesets_64" + echo 1 > "$FIX/repos_metadatastician_org-throttled_rulesets_64.rc" +} + +echo "== report mode (no --apply) ==" +build_fixtures +OUT="$(run "$SUT" --repos "$REPOS")" +check "vault is EXCLUDED-D50" "EXCLUDED-D50" "$(state "$OUT" hyperpolymath/memory-vault)" +check "bare repo is WOULD-CREATE" "WOULD-CREATE" "$(state "$OUT" hyperpolymath/plain-repo)" +check "exact floor is CONVERGED" "CONVERGED" "$(state "$OUT" hyperpolymath/converged-repo)" +check "richer cover is COVERED-BY-RICHER" "COVERED-BY-RICHER" "$(state "$OUT" hyperpolymath/richer-repo)" +check "bypassed twin is NOT converged" "COVERED-BY-RICHER" "$(state "$OUT" hyperpolymath/bypassed-twin)" +check "archived is ARCHIVED" "ARCHIVED" "$(state "$OUT" hyperpolymath/archived-repo)" +check "403 is PLAN-EXCLUDED" "PLAN-EXCLUDED" "$(state "$OUT" hyperpolymath/private-repo)" +# The plan arm above is the NEGATIVE CONTROL: it proves the throttle arm below did not +# simply swallow every 403. A throttled read is UNKNOWN -- skipped, never recorded. +check "rate-limit 403 is UNKNOWN" "UNKNOWN" "$(state "$OUT" hyperpolymath/throttled-repo)" +check "secondary-limit 403 is UNKNOWN" "UNKNOWN" "$(state "$OUT" hyperpolymath/secondary-throttled-repo)" +check "no source_type is REFUSED" "REFUSED" "$(state "$OUT" hyperpolymath/nosourcetype-repo)" +check "complete org cover is ORG-INHERITED" "ORG-INHERITED" "$(state "$OUT" metadatastician/org-covered-repo)" +check "HALF org cover is not a cover" "WOULD-CREATE" "$(state "$OUT" metadatastician/org-halffloor-repo)" +check "report mode writes nothing" "0" "$(wc -l < "$FIX/PUTS.log" | tr -d ' ')" + +echo "== org cache and scope edges ==" +build_fixtures +OUTE="$(run "$SUT" --repos "$ORG_EDGE")" +check "invalid org body is UNKNOWN" "UNKNOWN" "$(state "$OUTE" metadatastician/org-cache-invalid-a)" +check "invalid org body is not cached" "ORG-INHERITED" "$(state "$OUTE" metadatastician/org-cache-retry-b)" +check "release-only org floor does not cover default" "WOULD-CREATE" "$(state "$OUTE" metadatastician/org-release-only)" +check "throttled org body is UNKNOWN" "UNKNOWN" "$(state "$OUTE" metadatastician/org-throttled)" +check "throttled org body is reported once" "1" "$(rows "$OUTE" metadatastician/org-throttled)" + +echo "== apply mode ==" +build_fixtures +OUT="$(run "$SUT" --repos "$REPOS" --apply)" +check "bare repo is CREATED" "CREATED" "$(state "$OUT" hyperpolymath/plain-repo)" +check "half org cover still CREATED" "CREATED" "$(state "$OUT" metadatastician/org-halffloor-repo)" +# Exactly two repos lack a floor in force: the bare one and the HALF-org-covered one. +# Every other fixture must be left alone, so the count is an assertion in both directions. +check "exactly two writes" "2" "$(wc -l < "$FIX/PUTS.log" | tr -d ' ')" +check "every write is a POST" "POST" "$(cut -f1 "$FIX/PUTS.log" | sort -u)" +check "one write targets plain-repo" "1" "$(command grep -c 'repos/hyperpolymath/plain-repo/rulesets' "$FIX/PUTS.log")" +check "one write targets half-org repo" "1" "$(command grep -c 'repos/metadatastician/org-halffloor-repo/rulesets' "$FIX/PUTS.log")" +POSTED="$(command grep 'plain-repo' "$FIX/PUTS.log" | cut -f3 | jq -S -c .)" +CANONJ="$(jq -S -c . "$ROOT/config/rulesets/branch-floor.json")" +check "POST body equals the canon file" "$CANONJ" "$POSTED" +check "posted bypass_actors is empty" "0" "$(printf '%s' "$POSTED" | jq '.bypass_actors | length')" +check "no write touched the vault" "0" "$(command grep -c 'memory-vault' "$FIX/PUTS.log")" +check "no write touched the archived" "0" "$(command grep -c 'archived-repo' "$FIX/PUTS.log")" + +echo "== throttle backoff ==" +# A throttle is not a per-repo property. Three consecutive throttled reads mean the +# window is spent, so the sweep must ABORT: grinding on filed 267 repos as UNKNOWN twice +# on 2026-09-23, which READS as \"measured and unknowable\" when it means \"never looked\". +build_fixtures +OUTT="$(run "$SUT" --repos "$THR" --apply)" +check "third consecutive throttle aborts" "ABORTED" "$(state "$OUTT" -)" +check "the repo beyond the wall is not reported" "" "$(state "$OUTT" hyperpolymath/plain-repo)" +check "an aborted sweep writes nothing" "0" "$(wc -l < "$FIX/PUTS.log" | tr -d ' ')" + +# The same wall, POST-side. The shim logs a write attempt BEFORE it consults the +# fixture, so "wrote nothing" cannot be a line count here: assert instead that the repo +# beyond the wall was never reached at all. +build_fixtures +OUTP="$(run "$SUT" --repos "$THRP" --apply)" +check "third consecutive POST throttle aborts" "ABORTED" "$(state "$OUTP" -)" +check "no repo beyond a POST wall is reported" "" "$(state "$OUTP" hyperpolymath/plain-repo)" +check "no write reached the repo beyond the wall" "0" "$(command grep -c 'plain-repo' "$FIX/PUTS.log")" + +echo "== refusals ==" +build_fixtures +OUT2="$(GH_FIX="$FIX" PATH="$BIN:$PATH" bash "$SUT" --repos /dev/null 2>&1)" +case "$OUT2" in *"clean sweep over nothing"*) ok "empty repo list is refused";; + *) bad "empty repo list is refused" "refusal" "$OUT2";; esac + +MISSING="$WORK/noclass"; mkdir -p "$MISSING/scripts" "$MISSING/config/rulesets" +cp "$SUT" "$MISSING/scripts/" +cp "$ROOT/config/rulesets/branch-floor.json" "$ROOT/config/rulesets/tag-floor.json" "$MISSING/config/rulesets/" +# deliberately do NOT copy gcrypt-vault-class.txt +OUT3="$(GH_FIX="$FIX" PATH="$BIN:$PATH" bash "$MISSING/scripts/apply-protection-floor.sh" --repos "$REPOS" 2>&1)" +case "$OUT3" in *"class file missing"*) ok "missing vault class file is a refusal";; + *) bad "missing vault class file is a refusal" "refusal" "$OUT3";; esac + +echo "== mutants (each MUST make the suite go red, for the RIGHT reason) ==" +# ๐Ÿชค A mutant written to a temp dir resolves REPO_ROOT to that dir, cannot find the canon +# file, and dies with FATAL before ANY guard runs -- so all four "reds" would measure a +# broken path rather than the defect. The mutant therefore lives in the real scripts/ dir, +# and every mutant run is asserted to have produced real output first. +MUTROOT="$WORK/mutroot"; mkdir -p "$MUTROOT/scripts" "$MUTROOT/config/rulesets" +cp "$ROOT"/config/rulesets/*.json "$ROOT/config/rulesets/gcrypt-vault-class.txt" "$MUTROOT/config/rulesets/" +MUT="$MUTROOT/scripts/apply-protection-floor.sh" + +# Run a named mutation and assert the expected state change or write. +mutant() { # name sed-expr assertion-kind(wrote|state) arg [repos-file] + local name="$1" expr="$2" kind="$3" arg="$4" repos="${5:-$REPOS}" o got + sed "$expr" "$SUT" > "$MUT" + if ! bash -n "$MUT" 2>/dev/null; then + bad "mutant '$name'" "parses" "parse error -- red would measure the parser"; return + fi + if cmp -s "$MUT" "$SUT"; then + bad "mutant '$name'" "sed changes the script" "sed matched nothing -- the mutant is the original"; return + fi + build_fixtures with-vault + o="$(GH_FIX="$FIX" PATH="$BIN:$PATH" bash "$MUT" --repos "$repos" --apply 2>/dev/null)" + # the mutant must still RUN; a FATAL would make every check vacuous + if [ "$(printf '%s\n' "$o" | wc -l)" -lt 3 ]; then + bad "mutant '$name'" "runs and reports" "produced no report -- it died early, red is meaningless"; return + fi + case "$kind" in + wrote) got="$(command grep -c "$arg" "$FIX/PUTS.log")" + if [ "$got" -gt 0 ]; then ok "mutant '$name' dies (now POSTs to $arg)" + else bad "mutant '$name' DIES" "a POST to $arg" "no such write -- MUTANT SURVIVED"; fi ;; + state) got="$(state "$o" "${arg%%=*}")" + if [ "$got" = "${arg#*=}" ]; then ok "mutant '$name' dies (${arg%%=*} -> $got)" + else bad "mutant '$name' DIES" "${arg#*=}" "$got -- MUTANT SURVIVED"; fi ;; + esac +} + +# Baseline: with the vault fully writable, the REAL script must still write nothing to it. +build_fixtures with-vault +OUTV="$(run "$SUT" --repos "$REPOS" --apply)" +check "vault stays EXCLUDED even when writable" "EXCLUDED-D50" "$(state "$OUTV" hyperpolymath/memory-vault)" +check "vault receives no POST when writable" "0" "$(command grep -c 'memory-vault' "$FIX/PUTS.log")" +check "archived stays ARCHIVED when writable" "ARCHIVED" "$(state "$OUTV" hyperpolymath/archived-repo)" +check "archived receives no POST when writable" "0" "$(command grep -c 'archived-repo' "$FIX/PUTS.log")" +check "org-covered stays ORG-INHERITED" "ORG-INHERITED" "$(state "$OUTV" metadatastician/org-covered-repo)" +check "org-covered receives no duplicate POST" "0" "$(command grep -c 'org-covered-repo' "$FIX/PUTS.log")" + +mutant "D50 vault guard removed" \ + 's/^ if is_vault "\$repo"; then$/ if false; then/' \ + wrote "memory-vault" + +# The pending-fix defect: any 403 mapped to PLAN-EXCLUDED, so a rate-limit refusal was +# filed as "private repo / plan limit". It writes nothing either way, so the tell is the +# STATE, not a POST -- a mutant that silences the throttle arm must flip it back. +mutant "throttle arm removed from the 403 split" \ + 's/^ if is_throttled "\$err"; then$/ if false; then/' \ + state "hyperpolymath/throttled-repo=PLAN-EXCLUDED" + +mutant "archived guard removed" \ + 's/^ if \[ "\$(printf .%s. "\$meta" | jq -r ..archived.)" = "true" \]; then$/ if false; then/' \ + wrote "archived-repo" + +mutant "bypass dropped from the shape test" \ + 's/ \&\& \[ "\$byp" = "0" \]//' \ + state "hyperpolymath/bypassed-twin=CONVERGED" + +# Removing the converged early-return does NOT reach a write: the covered-by-richer check +# catches it next. That second line of defence is the point, so this mutant is asserted on +# the STATE it corrupts, not on a POST that correctly never happens. +# The bug this suite was extended for: org rulesets were COUNTED (org_n) but their rule +# types never entered the cover set, so ORG-INHERITED was unreachable and 67 org-covered +# repos reported WOULD-CREATE. Under --apply that is 67 duplicate rulesets. +mutant "org cover dropped from the union" \ + 's/",\$union,\$union_org,"/",$union,"/' \ + wrote "org-covered-repo" + +mutant "converged early-return removed" \ + 's/^ if \[ "\$exact_n" -eq 1 \]; then$/ if false; then/' \ + state "hyperpolymath/converged-repo=COVERED-BY-RICHER" + +# Without the abort the sweep grinds through the whole list against a spent window, so the +# repo beyond the wall is REACHED and written -- the 267-UNKNOWN failure, in miniature. +mutant "throttle backoff removed" \ + 's/^ exit 3$/ throttled=0/' \ + wrote "plain-repo" "$THR" + +# The reset-placement defect, verbatim: clearing the streak on any successful read means +# a repo whose GETs pass and whose POST is refused never accumulates one. Silencing +# repo_throttled reinstates exactly that, and the sweep grinds past the wall. +mutant "throttle streak counts calls, not repos" \ + 's/^ repo_throttled=1$/ :/' \ + wrote "plain-repo" "$THRP" + +printf '\n%d passed, %d failed\n' "$PASS" "$FAIL" +[ "$FAIL" -eq 0 ]