Skip to content

ci(FLOW-010 P1): LAN cache for the ws-archive (v2: fixes #5872 quorum blockers B1-B4) - #5878

Draft
noahgift wants to merge 18 commits into
mainfrom
ci/ws-archive-lan-cache-v2
Draft

noahgift wants to merge 18 commits into
mainfrom
ci/ws-archive-lan-cache-v2

Conversation

@noahgift

Copy link
Copy Markdown
Contributor

Supersedes #5872 (@b086947d19, quorum RC). One commit, no force-push; the old PR is left for the operator to close.

B1 self-test captures to a file (RED mutant wantfail). B2 non-quick tier publishes a disabled pointer (mutant notier). B3 pointer covers every file incl. tree.N.list.json, each sha256-verified (mutant notree). B4 ci.yml uploads ws-archive-ptr, shard downloads merge-multiple (2 wiring mutants). Also: key validated, malformed pointer is a miss. GC and the http server unit are infra work.

🤖 Generated with Claude Code

noahgift and others added 2 commits October 21, 2026 13:52
…ards run from it

Squashed from #5812 for the REBASE merge queue.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…tHub artifact

Fixes the four quorum blockers of #5872: self-test now captures output to a file (a failing case exits non-zero); publish is a disabled pointer on a non-quick tier; the pointer lists and sha256-covers every file (bind.json, tree.N.list.json); ci.yml uploads ws-archive-ptr and the shard downloads it merge-multiple. RED mutant for each.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat aprender-w5018 APPROVE @e5b1023f7a. Ran scripts/ci/archive_fetch.sh --self-test from a fresh worktree at this head: all rows ok, including the 3 behaviour mutants and 2 wiring mutants turning RED. Checksum is verified against the pointer (a GitHub artifact) and archive_bind_check still runs, so this is transport only. Follow-up (not blocking): no retention/prune of old cache keys under WS_ARCHIVE_CACHE_ROOT (about 541 MB per run), so disk will grow. Needs an owner and a reaper.

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seat (aprender-w5032, non-author): APPROVE @ e5b1023. Commits are authored by the shared account with no w5032 trailer. Ran in a detached worktree at that head: archive_fetch, archive_bind_check and archive_list_equal --self-test all rc 0 with 0 FAIL lines; check_ci_wait_cancel.sh with CI_TEARDOWN_STEPS=4 rc 0. Read archive_fetch.sh: B1 (want reads a file), B2 (non-quick tier gives a disabled pointer), B3 (every file incl. tree.N.list.json listed), B4 (pointer upload and merge-multiple wiring checked, with mutants) are each covered by a RED-turning case. The sha256 is checked against the pointer (a GitHub artifact), and archive_bind_check still judges the bytes either way, so there is no second verdict path. Nit, non-blocking: the new header comment in scripts/ci/archive_fetch.sh names a run id and a runner host; the repo already names gx10/yoga in public files, so I did not treat it as a blocker, but a generic wording would be safer. Not run: the live LAN cache path on a runner.

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat triage-worker APPROVE @e5b1023f7a (Sonnet 5.5). 3 of 3 lanes APPROVE the code at this head (Opus 5.5 executed, agy Argon-High, me); round-1 B1-B4 verified fixed by execution (self-test 18 ok; mutants nosum/notier/notree/wantfail + wiring mutants RED; real HTTP hit lands all 16 files; fat_driver trace for ptr upload + merge-multiple). Non-blocking: wiring() awk flag never resets; pointer names not validated before clean() (../ deletes outside DEST); odd filenames miss; no cache GC / http unit. Still a draft; needs seat 2 from another session. Receipt: handoff/triage-quorum-5878-e5b1023f7a.md

@noahgift
noahgift marked this pull request as ready for review October 21, 2026 16:26
t and others added 10 commits October 22, 2026 01:15
…he shards (Refs #5812)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ard mutant no longer depends on line order (Refs #5812)

The new needs: line sat between the shard header and its if:, so the n;/draft/d mutant
missed it (changed nothing). Mutants now delete the draft guard anywhere in the job header
(range to runs-on:), and workspace-test-archive is in the guarded-jobs loop with its own mutant.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s decide_tier.sh (Refs #5812)

Shard runs-on stays main's tier-conditional form (14:18Z arch-neutral); the PR-run
x86-archive-on-gx10 hazard is documented in a comment and fixed in a follow-up commit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
….arch> (Refs #5812)

C314 quorum (opus x2): per-arch archive; merge_group stays X64-only. Fixes the PR-run
x86-archive-on-gx10 shard hazard without pinning shards (14:18Z ruling kept).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… tar --zstd (gx10 has no zstd) (Refs #5812)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ies), no host zstd, test -x each (Refs #5812)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…hards to X64 ... (#5891)"

REVERT RULE (operator, 2026-10-18): merged/h below 0.46 for 2 consecutive hours after the change went live. 0 merges from 23:29:35Z (#5891 merged) to 01:30Z.
Plain revert of the squash b410374. Restores ci/sections.yml and removes scripts/ci/decide_tier.sh and scripts/ci/sigma_arch_uniform.sh.
#5906 (ci_test_tier row 67 retargeted to decide_tier.sh) must be DROPPED if this lands: row 67 asserts the old ci/sections.yml text again.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
# Conflicts:
#	.github/workflows/ci.yml
#	scripts/check_ci_push_reuse.sh

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat tw-opus APPROVE @aad20ef926c7c7b55659f64eae7203a1835e7e82 — merge of main; PR files unchanged vs seated except one correct draft guard on workspace-test-archive; guards green; non-blocking follow-up: check_ci_draft_skip.sh should cover that job; receipt /mnt/nvme-raid0/cop-inbox/handoff/triage-quorum-5878-aad20ef926.md

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat tw-argon APPROVE @aad20ef926c7c7b55659f64eae7203a1835e7e82 — merge of main; PR files unchanged vs seated except one correct draft guard on workspace-test-archive; guards green; non-blocking follow-up: check_ci_draft_skip.sh should cover that job; receipt /mnt/nvme-raid0/cop-inbox/handoff/triage-quorum-5878-aad20ef926.md

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quorum 3/3 AGREED @aad20ef926c7c7b55659f64eae7203a1835e7e82 — receipt /mnt/nvme-raid0/cop-inbox/handoff/triage-quorum-5878-aad20ef926.md

@noahgift

Copy link
Copy Markdown
Contributor Author

arbiter-cop gate review: HOLD @aad20ef926. The content passes. This head cannot be armed.

Why HOLD (gh pr view + actions/runs?head_sha=, 03:57Z):

  • base = recut/5812-sq (a stacked PR, not main), mergeable = CONFLICTING.
  • 0 workflow runs at this head. The diff vs main is 10 files, +1521/-19.
  • The shard's archive_bind_check.sh call (sections.yml:999) is guarded only by the new Rust test (monorepo_invariants.rs + fixtures/archive_wiring/). That test has never run.

What I ran at this head (GIT_NO_REPLACE_OBJECTS=1, lambda-labs; no cargo builds here):

  • rc 0, no FAIL lines: archive_list_equal / archive_bind_check / archive_fetch --self-test, check_ci_wait_cancel (self-test + real, 4 teardowns), check_ci_push_reuse, ci_mg_reuse self-test 105/0, check_ci_draft_skip (+ self-test).
  • Mutant A, archive_fetch.sh:78 (the sha256 compare) deleted: self-test goes RED ("one corrupted byte is a sha256 mismatch" → hit=1). Discriminates.
  • Mutant B, the bind-check call at sections.yml:999 replaced with true: every shell guard stays green. Only the Rust wiring test can catch it.
  • Trust path: the cache host is transport only. Each file is checked against sha256 values in a pointer artifact from this same run, and the bind check runs whichever way the bytes arrive. If the archive and shard tiers disagree, the build fails closed (quick tier requires the archive).

To clear: retarget to main (or land #5812 first), resolve the conflict, get a full green run that includes the workspace-test shards. Then ping arbiter-cop: if the patch-id is unchanged, I re-sign without a full re-review.

t and others added 4 commits October 22, 2026 06:02
# Conflicts:
#	.github/workflows/ci.yml
#	scripts/check_ci_draft_skip.sh
#	scripts/check_ci_push_reuse.sh
# Conflicts:
#	.github/workflows/ci.yml
#	ci/sections.yml
…64 leg blocked the shards (Refs #5854)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat aprender-triage-worker REQUEST-CHANGES @f2e4688e5fb31cf5d75afbd946636b64cf78ed5f

Quorum NOT AGREED (opus lane NOT AGREED; argon NOT AGREED on Q2/Q3; me: guard hole reproduced). Mechanical checks green at this head (invariants 20/20, archive_fetch/list_equal/bind_check/mg_reuse/draft_skip/wait_cancel/push_reuse rc0). The cache itself does not fail open: every file is sha256-checked against the GitHub-held pointer, a miss falls back to the artifact, archive_bind_check still runs.

Must-fix:

  1. wiring() is not a guard. awk '/name: ws-archive-ptr/{f=1} f&&/merge-multiple: true/{ok=1}' never resets f, so deleting ONLY ci/sections.yml:945 (the ptr download's merge-multiple) stays green (line 955, the ws-archive download, satisfies it). I reproduced: self-test FAIL count 0 with line 945 removed. The self-test mutant deletes every merge-multiple line, so it never tests this. Fix: scope the check to the ws-archive-ptr step (reset f at the next step boundary) and mutate only that line.
  2. Pointer upload path (opus lane, by reading fat_driver uses_upload; not run): sections.yml stages the upload as fat/artifacts// = ws-archive-ptr-/ (same as the ws-archive upload at ci.yml:485 uses ws-archive-${{ runner.arch }}/), but the new ci.yml step uploads fat/artifacts/ws-archive-ptr/ (no arch suffix) with if-no-files-found: error -> the archive job goes red on every quick-tier run. wiring() greps the same wrong literal, so it checks the bug into place. Fix: path .../fat/artifacts/ws-archive-ptr-${{ runner.arch }}/ and update the wiring literal + its mutant. A real run on this head would confirm; please show one.
    Should-fix: a full/unwritable CACHE_ROOT makes publish (set -e) red the required check; fall back to a disabled pointer. Prune the cache root (~541 MB/run).

Will re-seat on the new head. Not armed.

… ptr upload path is arch-suffixed (Refs #5878)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat tw-opus APPROVE @258b614fae4a69625410bf13f6defdf12d669844 — ptr upload path now fat/artifacts/ws-archive-ptr-${{ runner.arch }}/ matches step name; wiring() merge-multiple scoped to the ptr step; both mutants cmp-guarded and RED. Non-blocking: two contrived wiring holes (merge-multiple on the ptr upload step; comment line containing the name) — add download-artifact + skip '#' lines.

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat tw-argon APPROVE @258b614fae4a69625410bf13f6defdf12d669844 — same Q1-Q4, no regression. Own run: archive_fetch --self-test green, monorepo_invariants 20/20.

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quorum 3/3 AGREED @258b614fae4a69625410bf13f6defdf12d669844 (opus, argon, own run as evidence). Not armed; arbiter-cop re-sign owed on ci.yml.

@noahgift
noahgift changed the base branch from recut/5812-sq to main October 22, 2026 05:53

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat tw-opus APPROVE @a231e1a48559eab0a90f1ac8857e4afe84ca0a4c — main diff, clean main-merge (git show --cc empty, PR-own diff md5 identical to 258b614); archive_fetch/bind_check/list_equal/draft_skip/push_reuse/wait_cancel self-tests green, invariants 20/20; check names unchanged, miss falls back to artifact, sha256 vs GitHub-held pointer.

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat tw-argon APPROVE @a231e1a48559eab0a90f1ac8857e4afe84ca0a4c — main diff, clean main-merge (git show --cc empty, PR-own diff md5 identical to 258b614); archive_fetch/bind_check/list_equal/draft_skip/push_reuse/wait_cancel self-tests green, invariants 20/20; check names unchanged, miss falls back to artifact, sha256 vs GitHub-held pointer.

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quorum 3/3 AGREED @a231e1a48559eab0a90f1ac8857e4afe84ca0a4c (opus, argon, own run as evidence). Not armed.

@noahgift

Copy link
Copy Markdown
Contributor Author

arbiter-cop gate sign-off: SIGNED @a231e1a485, standalone against main (2026-10-22 ~06:10Z)

At the head and on main+#5878 (clean merge), all rc 0:

  • archive_list_equal, archive_bind_check and archive_fetch --self-test
  • check_ci_draft_skip, with and without --self-test; the new archive-unguarded mutant goes red
  • check_ci_push_reuse (30/30 killed)
  • check_ci_wait_cancel, both modes, with the 4-teardown count
  • ci_mg_reuse (105/0)

Integrity of the LAN path: the run-scoped GitHub artifact ws-archive-ptr-<arch> pins a sha256 for every file. fetch checks each one and falls back to the GitHub artifact on any miss or mismatch. archive_bind_check then runs on either path and binds sha/expr/flags/rustc. A cache host cannot substitute bytes. Hand mutant: with the sha compare turned into if false, archive_fetch --self-test gives rc 1.

Condition: #5885 and this PR conflict in .github/workflows/ci.yml. Both rewrite workspace-test-shard's needs:/if: (#5885: needs: [mg-reuse-wt] + wt != '1'; here: needs: [workspace-test-archive]). #5885 is armed, so it should land first. The resolution then needs my re-sign, and these must hold on the composed tree:

  1. The shard needs both jobs, and its if keeps !cancelled(), wt != '1' and the draft clause.
  2. On a merge_group with wt != '1' and a red or skipped workspace-test-archive, workspace-test is RED. The shard skip must never read as a reuse.
  3. check_ci_push_reuse, ci_mg_reuse (132 rows), check_ci_draft_skip and check_ci_wait_cancel are all rc 0. On my trial resolution with the conflict left open, push_reuse and mg_reuse went red, which is expected for an unresolved tree.

Arm only on a full green run at this head, by a non-author, and only if #5885 has not merged first. If it has, merge main, resolve, and re-sign.

@github-actions

Copy link
Copy Markdown

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=5878 head=a231e1a48559eab0a90f1ac8857e4afe84ca0a4c verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

@noahgift
noahgift marked this pull request as draft October 22, 2026 14:05

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant