Skip to content

docs(backlog): record that the live-API drift detector is invisible from the file it guards (BACKLOG #1450) - #886

Open
wshallwshall wants to merge 2 commits into
mainfrom
worktree-agent-abacc752824987711
Open

docs(backlog): record that the live-API drift detector is invisible from the file it guards (BACKLOG #1450)#886
wshallwshall wants to merge 2 commits into
mainfrom
worktree-agent-abacc752824987711

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

One commit, one file: the docs/BACKLOG.md row for BACKLOG #1450. Nothing else is touched.

What changed, and what deliberately did not

I was briefed to reconcile .github/required-contexts.txt and its pin to the server, and to file an item recording that the guard "is structurally incapable of catching a server-side change". Both halves turned out to be already-owned or wrong, and this PR is what survived the measurement.

The reconciliation is PR 884, opened at 23:55:18Z while I was mid-edit. The collision registry flagged it, I read the branch, and I stood down. 884 removes the retired context, moves the pin 14 to 13, and also fixes docs/CI.md, tests/negative_controls.toml, tests/test_merge_gate_controls.py, tests/test_security_posture.py and codeql.yml — six files I never touched. Shipping a competing edit to the same lines of a file 60 open PRs already conflict over would have cost the Lander a hand-merge to gain nothing. No file in 884's diff appears in this PR. I had made those edits; I reverted them.

The item's premise was wrong, and the corrected version is the more interesting finding. scripts/ci/check_required_contexts_drift.py exists, reads the live server over the public /branches/{branch} endpoint, set-compares rather than count-compares, fails closed, and runs as the job the required-contexts file matches the server. A live-API guard is built. What is true is that neither .github/required-contexts.txt nor tests/test_required_contexts.py names it — so a reader asking "what would catch the server moving?" reads both and correctly concludes: nothing. That happened to two readers in one night, one of whom wrote my brief. That is the defect #1450 records.

Measurements, each with a positive control

Taken 2026-09-04 between 23:24Z and 23:56Z. A broken instrument returns the same empty output as a clean repo, so every count below has a control beside it.

Question Reading
gh api .../branches/main/protection contexts 13, no a reviewer has read this
gh api .../branches/main (public) contexts 13, identical set — the two endpoints agreed
git show origin/main:.github/required-contexts.txt 241 lines; control grep -c "CI gate" = 4 (non-zero, so the read is live); reviewer line = 2 occurrences
tests/test_required_contexts.py pin assert len(contexts) == 14
continue-on-error in required-workflow-state.yml 0; control grep -c runs-on = 2. It goes red on a finding
Is the drift job required? No — not among the 13
Has it failed on this drift? Not yet, and pending is not a verdict. Last verdict success at 23:25:00Z, read per job, not per run. Later runs queued with no conclusion

That 23:25:00Z green bounds the retirement: the server still held 14 then and held 13 by 23:52Z. The change landed inside that half hour.

I used MSYS_NO_PATHCONV=1 on every git show origin/main:..., per the trap in my brief.

Ruling on the live-API test arm: do not build it

I was asked to assess whether a pytest arm could read the live API and red on drift. Recommendation is against, on three grounds, all recorded in the item.

  1. tests/test_required_contexts.py runs inside a required test leg. The drift script fails closed by design — right for a scheduled advisory job, wrong for a required one. A transient API failure would red a required check on every open PR at once, for a reason unrelated to any of them.
  2. The suite also runs offline, on fork PRs, and on developer machines. An arm that skips when it cannot reach the network is worse than none: a skip renders as a pass.
  3. The instrument already exists. A second implementation of one question is what the drift script's own docstring gives as its reason for not folding into its sibling.

The gap is routing, not detection. Three things sit between a server-side change and a reader: the job blocks nothing, its pull_request: trigger is paths-filtered so it does not run on an ordinary PR at all, and what remains is a 07:00 UTC cron on main whose red notifies nobody — the same shape as #1402 and #1406. The cheap fix is naming the drift job in the two files that hide it. Promoting it to required is an owner decision and I do not recommend it without reconsidering the fail-closed behaviour first.

One thing PR 884 leaves stale, named so nobody re-measures it

.github/required-contexts.txt lines 18-24 still read "THE LIST BELOW IS SET-EQUAL TO THE SERVER -- fourteen contexts ... read from the API at 2026-08-31 20:57 CDT". 884 edits only the block near line 202, so on that branch the file will name thirteen contexts under a header asserting fourteen. Set-equality is the single assertion this file exists to carry. It wants the 2026-09-04 date and the thirteen-context reading, and the fix is that paragraph alone. I did not put it here because it cannot land coherently ahead of 884's removal of the context line. It is written into #1450 so it survives 884's author being gone.

Checks

Interpreter provenance, proving I read my own worktree and not the primary:

messagefoundry: C:\Users\Scott\Code\MessageFoundry\.claude\worktrees\agent-abacc752824987711\messagefoundry\__init__.py
sys.path[0]: ''
Check Result
pytest tests/test_required_contexts.py 9 passed. It passes against the unreconciled file, which is the Console's core reading confirmed: this suite cannot see the server move
pytest tests/test_backlog_status_check.py 32 passed
scripts/docs/backlog_status_check.py OK, 666 items, each declaring exactly one status
scripts/docs/backlog_citation_check.py OK, exit 0 (2 pre-existing advisory warnings, neither mine)
scripts/docs/dangling_citation_check.py exit 0; none of the 14 flagged numbers are mine
ruff check . All checks passed
ruff format --check . 1241 files already formatted
pre-commit on commit ledger gate, forbidden-content, control-characters, secrets all Passed
Glyph audit of the new item only U+1F6A7 (the sanctioned _OPEN banner) and U+00B7 (house-style middle dot)

Skipped: the full suite, per my brief — many live sessions and tests/test_connscale_smoke.py hard-codes a 24-port block (#1014). No pytest output was piped to head or tail before exit.

Legs only a hosted runner reports, which no Builder sees: the three test matrix legs, CI gate's roll-up, and every security scanner. This is a Markdown-only change to one file, so I expect none of them to be load-bearing here, but they still need reading after my process exits.

Assumptions, stated because I could not ask

  • I did not touch branch protection. Read only, per scope.
  • I did not re-add the reviewer context anywhere and do not argue the ruling. The item records the consequence as fact.
  • I left the three stale enforce_admins sites alone, as that file's own decision directs.
  • I did not add the reviewed label or arm auto-merge.
  • I judged that standing down on the reconciliation beat shipping a duplicate. If the Console would rather have 884 closed and a superset shipped here, that is a re-cut brief, not a fix to this PR.

BACKLOG #1450 is claimed at scripts/coord/claim.ps1 -Take 1450; release it on merge.

🤖 Generated with Claude Code

…rom the file it guards (BACKLOG #1450)

The detector exists. scripts/ci/check_required_contexts_drift.py reads the
public branches endpoint and set-compares it to the checked-in file, as the
job `the required-contexts file matches the server`. Neither
.github/required-contexts.txt nor tests/test_required_contexts.py names it,
so two readers in one night read those files and concluded the drift was
undetectable. One of them briefed this item as a structural impossibility.

Measured 2026-09-04, each reading with a positive control: server 13
contexts on both endpoints, file 14, pin 14. The drift job carries no
continue-on-error, is not a required context, and is paths-filtered on
pull_request, so a server-side change reaches only its 07:00 UTC cron on
main, which notifies nobody.

Recommends AGAINST a live-API arm in the pytest suite: that suite sits
inside a required leg and runs offline and on fork PRs, so a fail-closed
network call there would red every PR on an API blip. The gap is routing,
not detection.

The reconciliation itself is PR 884; this Builder stood down on it after the
collision registry flagged the overlap. No file in that diff is touched
here. The one thing 884 leaves stale is named in the item.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

MANAGER NOTE, per the Manager playbook section 6: I commissioned this diff, so this is a record of what I checked and what I judged, not an independent review.

NO reviewed LABEL ON THIS PR, DELIBERATELY. The reviewer context was retired from branch protection at about 23:33Z and is no longer among the required set, so a label here would assert nothing and block nothing. The review-gate.yml workflow still exists and still goes red without a label, so expect a red check named for it. It gates nothing; do not chase it.

WHAT I CHECKED ON THE DIFF:

  • Pictographs, using a codepoint predicate with a positive control over the banner alphabet. One added, U+1F6A7, the in-progress banner, which is correct for a newly filed item. A grep with \x{} escapes ERRORS on large codepoints and its silence is not a pass, which is why the control is there.
  • Whether a banner was flipped to a closed glyph, which is the Lander's act on merge. It was not.
  • Required-context counts. My hygiene check flagged four lines and I examined each rather than passing them.

ON THE COUNTS, because the repository rule says never to write the required-context count into a document and this item does state numbers. My judgment is that these are admissible, and the reasoning is that they are dated MEASUREMENTS rather than standing CLAIMS: each sits in a table beside the command that produced it, so a later reader can re-run it instead of trusting it. The rule exists because a bare count in prose goes stale silently and is then quoted as current. A row reading "server 13, by gh api .../branches/main/protection" cannot be quoted that way without also carrying its instrument.

An item whose entire subject is a drift between two counts cannot avoid naming them. What it can avoid is asserting one as the present state, and this does not.

Worth noting the same discipline applied elsewhere in this item: the finding about PR 884's stale header is deliberately written as "lines 18 to 24" rather than as a count, with the author's own note that it is "named rather than numbered so it does not need re-measuring". That is the correct instinct and I would rather record it than let it pass unremarked.

A peer retracted a review label earlier today over exactly this defect in its live form: a pull request that wrote the count into workflow comments, where it was both standing and already stale.

…main stale (BACKLOG #1450)

The item said the guard's paths-filtered pull_request trigger means it
does not run on an ordinary PR, so a server-side change is invisible to
every merge path. The first half is right and the conclusion is wrong.

A PR that touches those paths DOES fire it. Measured on PR 884: run
created 2026-09-04T23:55:21Z, event=pull_request.

What actually stops it reporting the drift is the checkout. The job runs
actions/checkout with no ref:, so on a pull_request event it reads the
file as that PR has it and compares against the live server. 884 compares
13 against 13 and passes. The run validates a fix; it cannot report that
main is stale.

"No pull request triggers it" is falsifiable and false. "No pull-request
run can see the drift on main" is true and stronger.

Caught by the Lander on review of the ledger entry.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant