Skip to content

test(oracles): freeze canonical JSON edge cases as differential oracles - #128

Merged
hoklims merged 3 commits into
mainfrom
test/r0-edge-case-oracles
Oct 27, 2026
Merged

test(oracles): freeze canonical JSON edge cases as differential oracles#128
hoklims merged 3 commits into
mainfrom
test/r0-edge-case-oracles

Conversation

@hoklims

@hoklims hoklims commented Oct 27, 2026

Copy link
Copy Markdown
Owner

Summary

First slice of the R0 edge-case oracles (#84): canonical JSON and its digest, which R3 moves to Rust first (replay and core <op>, byte-for-byte differential).

tests/edge-oracles/canonical-json.json freezes 17 JSON input texts with the exact canonical text and sha256: digest the TypeScript implementation produces, or refused: true. The cases sit where another implementation most easily diverges:

  • key order by UTF-16 code unit: canonicalize sorts keys with JS <, so 😀 (surrogate D83D) sorts before ｡ (FF61). UTF-8 byte or code point order, the default in Rust, gives the reverse;
  • ECMAScript number formatting: -0 → 0, 1e21 → 1e+21, 1e20 → 100000000000000000000, 1e-7, rounding past 2^53, subnormal and largest finite double;
  • string escaping: lowercase \u control escapes, raw /, U+2028 and U+2029, raw non-ASCII hashed as UTF-8, lone surrogates escaped (never replaced by U+FFFD);
  • parse and refusal: a duplicate key keeps its last value; 1e400 and -1e400 are refused.

tests/edge-oracles.test.ts replays every case and checks:

  • the canonical text, the digest, and idempotence;
  • the digest against an independent node:crypto SHA-256 of the canonical UTF-8 bytes (the core hashes with its own SHA-256);
  • a refusal for refused inputs;
  • that the eight divergence traps are still present.

Refusals are recorded as refused: true, without the TypeScript message, so the oracle does not depend on one language.

Proof

Negative witnesses in a disposable worktree at 8959ada, each red on ERR_ASSERTION in the designated test, then green after restore:

Witness Mutant Red test
key-order-utf8-bytes compareOrdinal → Buffer.compare of UTF-8 bytes key-order-utf16-code-units
negative-zero-kept -0 serialized as -0 number-negative-zero
lone-surrogate-replaced strings passed through toWellFormed() string-lone-surrogates
non-finite-accepted non-finite numbers written as null number-overflow-refused
trap-case-dropped fixture: string-lone-surrogates removed keep every divergence trap…
  • pnpm check: green at 8959ada: typecheck, lint, build, 1013/1013 tests (run alone; an earlier run interrupted alongside the witnesses showed spawn and named-pipe errors in four unrelated suites, none of them reproduced)
  • Proof epoch: base 7c623ae, candidate 8959ada, subject digest sha256:3683ae18…, classifier category test_oracles (2 paths).

Limits

  • This slice covers canonical JSON and digests only. The scanner (tokens, UTF-16 positions) and closure edge cases that E7 (E7 — Scanner Rust équivalent au scanner TS #116) measures come next; the R0 box stays unchecked until then.
  • Expected values are the current TypeScript output, checked by hand against the ECMAScript JSON.stringify rules for the cases above, not by a second implementation.
  • No independent proof auditor ran.

Test plan

  • pnpm check
  • Five negative witnesses red then green
  • CI green

Refs #84

🤖 Generated with Claude Code

@hoklims @claude
test(oracles): freeze canonical JSON edge cases as differential oracles
8959ada 
Freezes JSON inputs with the exact canonical text and digest the
TypeScript implementation produces, or its refusal, where another
implementation most easily diverges: key order by UTF-16 code unit,
ECMAScript number formatting, string escaping, lone surrogates and
non-finite numbers. The digest is also checked against an independent
SHA-256 of the canonical UTF-8 bytes.

Refs #84

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hoklims hoklims added this to the R0 milestone Oct 27, 2026
@hoklims hoklims added the area:ts TypeScript implementation label Oct 27, 2026
@hoklims hoklims self-assigned this Oct 27, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •
edited
Loading

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T21:08:46.592186Z 8ec841a New commits
🔒 Security Review ✅ Completed 2026-10-27T21:09:26.707453Z 8ec841a New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

This was referenced Oct 27, 2026
Merged
Closed
@hoklims
hoklims enabled auto-merge (squash) October 27, 2026 20:34
hoklims added a commit that referenced this pull request Oct 27, 2026
@hoklims @claude
test(oracles): freeze scanner, specifier and closure edge cases (#129)
ba5d8eb 
## Summary

Second slice of the R0 edge-case oracles (#84): the JavaScript scanner,
module specifier discovery and the test closure, which E7 (#116) and R5a
(#92) move to Rust. The first slice, canonical JSON and digests, is
#128.

Three fixtures in `tests/edge-oracles/` freeze each input with the exact
output the TypeScript implementation produces, or its refusal code:

- **`javascript-tokens.json`, 18 cases.** Each token is `[kind, start,
end, text]`, plus `value` for identifier and literal kinds, the only
ones consumers read. The cases cover:
  - UTF-16 positions;
  - a BOM that is skipped while a shebang after it is not;
  - conflict markers, whose second side is skipped too;
  - Unicode spaces;
- regex slashes read as division, so a quote inside a regex opens a
string;
  - the empty `#` token being skipped;
- normalized numeric values (`0x1F` → `31`, `0xFFn` → `0xffn`, `017` →
`15`);
  - decoded and invalid escapes;
  - nested templates.
- **`module-specifiers.json`, 16 cases.** Every static and dynamic form,
`require`, Bun's `mock.module`, computed specifiers, escapes, `from`
across lines, a semicolon ending the search, and an import hidden behind
a regex quote.
- **`test-closure.json`, 12 scenarios.** Each describes an in-memory
repository:
- the order of resolution candidates (an exact `c.js` wins over `c.ts`);
- path keys lowercased with final-sigma context (`ΑΣ/x.js` reaches
`ας/x.ts`, while a per-character lowercase misses it);
  - the inventory spelling kept;
  - UTF-16 path order;
  - unbounded edges sorted by file, then reason;
  - packages, JSX runtime and runner modules;
  - links, both stopped and followed;
- tsconfig paths with comments, a bunfig preload, package `imports`, a
workspace package;
- two refusals, `TEST_CLOSURE_PACKAGE_IMPORT_UNSUPPORTED` and
`TEST_CLOSURE_TSCONFIG_EXTENDS_UNSUPPORTED`.

`tests/scanner-closure-oracles.test.ts` replays every case and checks:

- that each token text equals the source slice its UTF-16 offsets
delimit;
- that the required divergence traps of each fixture are still present.

Kinds are recorded by name, not by the numeric value of the TypeScript
enum. Refusals are recorded by their product code.

## Proof

Ten negative witnesses run in a disposable worktree at `721330a`. Each
one turns red on `ERR_ASSERTION` in the designated test, then green
after restore:

| Witness | Mutant | Red test |
| --- | --- | --- |
| utf8-byte-positions | token offsets in UTF-8 bytes |
`utf16-astral-positions` |
| numeric-value-raw | numeric value = raw text |
`numeric-literal-values` |
| template-not-rescanned | no template tail rescan | `template-nesting`
|
| any-module-call | any `x.module(...)` is an edge | `mock-module` |
| concatenated-import-literal | `import("a" + x)` keeps `"a"` |
`dynamic-concatenation` |
| per-character-lowercase | path key lowercased per character |
`path-keys-and-order` |
| utf8-path-order | closure files sorted by UTF-8 bytes |
`path-keys-and-order` |
| ts-source-before-exact | `.ts` source tried before the exact `.js` |
`resolution-candidates` |
| runner-modules-resolved | runner modules resolved to packages |
`packages-and-runner-modules` |
| trap-case-dropped | fixture: `regex-hides-import` removed | `keep
every divergence trap…` |

- `pnpm check`: green at `721330a`: format, lint, schemas, conformance,
typecheck, 1044/1044 tests (run alone)
- Proof epoch: base `7c623ae`, candidate `721330a`, subject digest
`sha256:8955fbb6…`, classifier category `test_oracles` (4 paths).

## Limits

- Expected values are the current TypeScript output. Several of them
freeze known limits of a context-free scanner rather than desirable
behavior: regex, JSX, and HTML-like comments. A Rust scanner has to
reproduce them, or a later decision has to change both implementations.
- Not frozen: a specifier that is not in NFC (`./café.js`) makes
`computeTestClosure` throw `TypeError: Ambiguous Unicode path segment
must use NFC` instead of a `TEST_CLOSURE_*` refusal. It is left out
until #130 decides the intended behavior.
- The `node:` builtin list comes from the running Node version. These
cases only use builtins that are stable across versions.
- No independent proof auditor ran.

## Test plan

- [x] `pnpm check`
- [x] Ten negative witnesses red then green
- [ ] CI green

Refs #84, #116, #92

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@hoklims
hoklims merged commit 6c140e2 into main Oct 27, 2026
13 checks passed
@hoklims
hoklims deleted the test/r0-edge-case-oracles branch October 27, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@hoklims hoklims

Labels

area:ts TypeScript implementation

Projects

AssertLedger
Status: No status

Milestone

R0

Development

Successfully merging this pull request may close these issues.

1 participant

@hoklims