Skip to content

test(replay): freeze the v1-v5 replay corpus as differential oracles - #124

Merged
hoklims merged 1 commit into
mainfrom
test/r0-replay-corpus
Oct 27, 2026
Merged

hoklims merged 1 commit into
mainfrom
test/r0-replay-corpus

Conversation

@hoklims

@hoklims hoklims commented Oct 26, 2026

Copy link
Copy Markdown
Owner

Summary

Freezes the R0 replay corpus required by #84: real evidence manifests v1 to v5 and real qualification receipts, each produced by the published version that introduced its contract. Under ADR 0002 they are the differential oracles that every later implementation, Rust included, must replay with the same decision, digests and output.

  • tests/replay-corpus/corpus.json: 21 entries.
    • evidence-v1 to evidence-v5: manifests 1.0.0 to 5.0.0, produced by assertledger@1.0.0, 1.1.0, 1.2.0, 1.5.0 and 2.1.0.
    • qualification-{1.5.0,2.1.0}-{node-test,propagation}: receipt 1.0.0, all QUALIFIED.
    • 12 evidence-v4-container-* entries: the container-backend manifests of benchmarks/container-runtime/2026-10-11, read in place.
  • tests/replay-corpus/documents/: the 9 produced documents, byte for byte.
  • tests/replay-corpus/inputs/<id>/: the requests and fixture sources they were produced from.
  • tests/replay-corpus.test.ts: a completeness test plus one test per entry. Each entry test checks:
    • the sha256 of the document;
    • the recorded schema version, decision and digests;
    • that dist/cli.js replay - or qualification-replay - exits 0, with empty stderr and byte-identical stdout.
  • biome.json, tsconfig.json and .gitattributes: Biome, TypeScript and Git normalization skip the corpus.

At freeze time, every entry was replayed by every published version from its producer onwards, with identical results. CI replays only the current build. replayedBy records that matrix.

Proof

Classifier: proof-integrity-review triggered, with two categories.

  • test_oracles: all changed paths.
  • verification_routing: the 5 fixture package.json files.

Epoch sha256:3ec7f9eea5b86ea94ee9ffb20249287236ec29acc0590bde5bc9c3da313cf736 (base 54a952f, head cbd43c6).

Negative witnesses ran in a disposable worktree at cbd43c6. Each mutant was red through ERR_ASSERTION on the designated test, then green after restoration.

Claim Mutant Designated test Red Green
A frozen document cannot change by one byte {" → { " in evidence-v1.manifest.json replay evidence-v1 sha256 assertion 1/1
A resealed document is refused, even with consistent corpus metadata forged decisionDigest in the 1.5.0 propagation receipt, with sha256 and decisionDigest updated in corpus.json qualification-replay qualification-1.5.0-propagation exit 4 (valid: false) ≠ 0 1/1
The current build still replays v3 as its producer did src/core/index.ts: v3 added to repositoryRequired, then rebuilt replay evidence-v3 exit 4 (valid: false) ≠ 0 1/1
The corpus keeps a manifest for every version v1 to v5 evidence-v2 entry and document removed completeness test deepStrictEqual on versions 1/1

verification_routing.no_route_change: the 5 fixture package.json files have no scripts and no workspaces. They are referenced by no workflow, by no pnpm-workspace.yaml entry and by no root script, and pnpm test only globs tests/*.test.ts.

Local gate: pnpm check on cbd43c6, with 991/991 tests passing (format, lint, schemas, conformance and typecheck included).

Limits

  • The cross-version replay matrix ran once, at freeze time, from the npm packages; CI does not reinstall historical versions.
  • All documents were produced on Windows x64 (Node 22.23.2, Bun 1.4.2), in trusted-local mode (v1 and v3 have no container path).
  • The v3 to v5 manifests seal the resolved Bun executable path, which is a machine path. It is evidence, not an input.
  • No independent proof auditor was run; approval and merge remain the maintainer's.

Test plan

  • pnpm check (991/991)
  • Four negative witnesses red then green, bound to the epoch
  • CI green on the PR

Refs #84

🤖 Generated with Claude Code

Real evidence manifests v1 to v5 and qualification receipts, each
produced by the published version that introduced its contract, with
the 12 container v4 benchmark manifests read in place. The test checks
the frozen bytes, the producer's decision and digests, and the current
build's replay output byte for byte.

Refs #84

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hoklims hoklims added this to the R0 milestone Oct 26, 2026
@hoklims hoklims added the area:ts TypeScript implementation label Oct 26, 2026
@hoklims hoklims self-assigned this Oct 26, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T19:40:54.736466Z cbd43c6 PR opened
🔒 Security Review ✅ Completed 2026-10-26T19:42:21.559693Z cbd43c6 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@hoklims
hoklims merged commit 0a7de13 into main Oct 27, 2026
13 checks passed
@hoklims
hoklims deleted the test/r0-replay-corpus branch October 27, 2026 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:ts TypeScript implementation

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant