Skip to content

test(surface): anchor CLI and MCP tests on the built package - #126

Merged
hoklims merged 1 commit into
mainfrom
test/r0-public-surface-anchoring
Oct 27, 2026
Merged

hoklims merged 1 commit into
mainfrom
test/r0-public-surface-anchoring

Conversation

@hoklims

@hoklims hoklims commented Oct 27, 2026

Copy link
Copy Markdown
Owner

Summary

Re-anchors the CLI and MCP tests on the public process boundary (ADR 0003: in 3.0.0 the public contract is CLI + MCP + schemas), so a later implementation of the same commands and tools can be swapped in under them.

  • tests/support/public-surface.ts:
    • runPublicCli runs dist/cli.js as a separate process, with the call shape of the in-process runCli;
    • connectPublicMcp starts assertledger mcp [--root] [--allow-unsafe-execution] over stdio;
    • listedInputSchemas reads the schemas published by tools/list, the public equivalent of the SDK-internal toolInputSchemaJson.
  • 22 test files moved (wholly or partly) from runCli / createAssertLedgerServer + InMemoryTransport to these helpers. qualification-mcp now expects the mechanism digest of the built modules, since that is what the stdio server runs.
  • tests/public-surface-anchoring.test.ts: ratchet that fails when a test imports src/cli or src/mcp/ outside an exact allowlist, and when an allowlist entry no longer imports one. Allowlist, each with its reason:
    • bun-preflight-diagnostics: mocks node:child_process inside the CLI process;
    • developer-experience: injects setup/demo dependencies and checks source-run refusals (its other calls now go through the built CLI);
    • naming-migration: pins the createTestForgeServer alias of the JS API removed in 3.0.0.

Internal tests (engine, core, SDK) are unchanged.

Proof

Negative witnesses in a disposable worktree at 924d039 (dist built there), each red on ERR_ASSERTION in the designated test, then green after restore:

Witness Mutant Red test
unlisted-source-import diagnostics.test.ts: import "../src/cli.js" added tests reach the CLI and MCP server only… (unlisted)
stale-allowlist-entry naming-migration.test.ts: src/mcp import → dist/mcp tests reach the CLI and MCP server only… (stale)
built-cli-fault dist/cli.js only: usage text assertledger → assertledgr supports help and version aliases…
built-mcp-fault dist/mcp/index.js only: assertledger_doctor registered under another name keeps runtime MCP opt-in…

The last two mutate only the built package, with sources untouched: the migrated tests observe what ships.

  • pnpm check: 996 tests pass, 0 fail.
  • Proof epoch: base 7c623ae, candidate 924d039, subject digest sha256:5f5f3a5e…, classifier category test_oracles (24 paths).

Limits

  • Success-path coverage per command and tool is not extended here: the goldens have success paths for 5 CLI commands and 5 MCP tools only.
  • integration.test.ts keeps one server root per connection: the CLI accepts a single --root.
  • No independent proof auditor ran.

Test plan

  • pnpm check
  • Four negative witnesses red then green
  • CI green

Refs #84

🤖 Generated with Claude Code

Tests that exercise the CLI or the MCP server now drive dist/cli.js as a
separate process and the MCP server over stdio (ADR 0003), through
tests/support/public-surface.ts. A ratchet fails when a test imports
src/cli or src/mcp outside an exact, motivated allowlist, or when an
allowlist entry goes stale.

Refs #84

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hoklims hoklims added this to the R0 milestone Oct 27, 2026
@hoklims hoklims added the area:ts TypeScript implementation label Oct 27, 2026
@hoklims hoklims self-assigned this Oct 27, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-27T16:24:41.537570Z 924d039 PR opened
🔒 Security Review ✅ Completed 2026-10-27T16:26:22.037648Z 924d039 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@hoklims
hoklims merged commit d65ed03 into main Oct 27, 2026
13 checks passed
@hoklims
hoklims deleted the test/r0-public-surface-anchoring branch October 27, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:ts TypeScript implementation

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant