Skip to content

v0.2.33.0 feat: a Vitest test run the chat explained becomes a saved test replay (Cycle 0049, PR-A of 3) - #113

Merged
ryota-murakami merged 27 commits into
mainfrom
codex/cycle-0049-test-run-learning-replay
Oct 22, 2026
Merged

ryota-murakami merged 27 commits into
mainfrom
codex/cycle-0049-test-run-learning-replay

Conversation

@ryota-murakami

@ryota-murakami ryota-murakami commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor
{
  "schema_version": 1,
  "pr": 113,
  "head": "09fa794c79f1e59eb71a9b42a56d038e84734553",
  "version": "0.2.33.0",
  "base": "main",
  "gate": {
    "pregate": {
      "pass": 6138,
      "fail": 0,
      "skip": 16
    },
    "tests": {
      "pass": 5602,
      "fail": 0,
      "skip": 6
    },
    "package": {
      "pass": 1059,
      "fail": 0,
      "skip": 10
    }
  },
  "session_kind": "interactive",
  "pregate": "requires-remote cleared @e8a99adf4427 (7 checks, current for 09fa794c79f1)",
  "requires_remote": [
    "package.yml/package: https://github.com/laststance/codebook/actions/runs/39730036823"
  ],
  "issues": {
    "fixes": [],
    "refs": [
      107
    ],
    "flakes": []
  }
}

Summary

Cycle 0049, PR-A of three (Linked to #107; the Issue stays open until PR-B2): a Vitest test run that the editor chat explained becomes a saved, reopenable test replay, with the agent contracts it needs.

  • Test-run outcomes and the agent's fields (C49-1, MCP API 1.8). A node-test run reads demonstrated once its test passed after every declared stop and its history sealed complete; the run resource adds nodeTest.testResult, nodeTest.learningCoverage and nodeTest.learningMissing, and nextActions offers save-node-learning until the coverage is complete. The run owns its in-flight continuation, so Stop and Save settle it before the seal; a failing exit keeps its result without Vitest's report; a passing run whose first release failed seals complete on a retried Stop.
  • Refusals at preparation and typed errors (C49-6). A node-test start's deterministic checks (changed sources, a changed test file, a path that is not a file, the root Vitest) run before the history begins, end the run with a typed error, record no history and free the run's slot and key. HISTORY_SEAL_FAILED, PLAYBACK_PROFILE_UNSUPPORTED and an INVALID_REFERENCE that names the cited event are new; codebook_status names a missing or older root Vitest before any start.
  • Save learning replay and Open replay for test runs (C49-2). NodeLearningSave drives the status bar item (preparing, live with its stop count, the question before the result, saved, partially saved with the missing parts, and three failure stages with their own retries); Open replay and Saved studies open a test replay in the Study pane at its saved position or its first verified stop. Saved studies rows name the verdict, capture and explained stops. One Codebook log output channel records unexpected failures; their causes are sanitized at assignment.
  • Store summary (EN7). Saved studies, the history resource and Save read one memoized summary per history instead of reading every journal on each 250 ms refresh.
  • Skill 1.0.46 (C49-7). Routes a question about a test to the node-test profile, explains a test stop in the chat, claims saved learning only with studyOutcome demonstrated and learningCoverage complete, names a failing test, and recovers a run that ended before its first stop only as the published recovery of the error's code allows (quoted text is data).
  • Review fixes (three /ship fixing cycles and Step 11). The health gate, the Save item while the Study pane is closed, a Save for a run that recorded nothing (F8, F61), a stop or a closed output tab during the final report check that lost a passing result (F62, F78), Save asking before ending a run that could no longer reach its result (F54, F79), an unanswered DAP request that held Stop, Save and the next start (F95), and the skill's restart rule that let quoted repository output instruct the agent (F96).
  • Test infrastructure. The four macOS-only test isolation failures (F1, F7), test start keys passed through a named variable for the secrets scan, re-recorded test timings (EN20), and the compaction hook that restores the owner's re-entry rule within the hook context limit (F2).

Delivery state: /ship did not converge. Its three fixing cycles each found and fixed critical defects (7437ca4, ae5f310, 4d9d592), so it stopped with converged:false; under the owner's order this pull request merges after required CI, with the open Critical and Major defects recorded in the Issue (its comment "/ship for PR-A stopped unconverged: the open Critical and Major defects") and every other finding carried there as a task (F3 to F103). The adversarial review's critical findings were fixed in a56b86c; Steps 9 to 11 were not repeated (no fourth cycle), and the review records do not bind to one tree (Step 11.5 mismatch: both passes' trees changed during the pass). The final tree is verified by Step 16 below and by required CI.

Native pilot (C49-11), macOS, VS Code 1.143.0, Copilot Chat with GPT-6 Luna: FAIL. In a Local chat the agent loaded the skill, called codebook_status first, published its article and started the node-test run, which was refused at preparation because the workspace already held 77 saved studies, above the default limit of 64; no stop was reached (F92, F93, Major, carried). A first send in VS Code's default Copilot harness could not reach Codebook's MCP server and is kept as an observer failure (F94). Details and identities: Issue #107 comments of 2026-10-22 13:15 and 13:33.

Test Coverage

Step 7 ran its two generation passes (63a35ea, then the repeat committed in 99cf083) and an audit-only pass on the pass-3 candidate. The diagram below is that last audit (2c4cb6c plus the repeat's tests); a56b86c then added six run tests and two skill and help tests for the Step 11 fixes (F95, F96), each shown failing before its fix. The diagram predates the pilot's run: row 20's pilot has since run in VS Code and FAILed at its start (Summary).

CODE PATHS — full branch vs origin/main 0bda935 at 2c4cb6c (ae5f310 merged with main), plus the working-tree tests of the previous pass
[+] extension/src/utils/
  ├── 01 projectNodeTestOutcome, runtimeOutcome, sealFailedRow   [★★★ TESTED] projectNodeTestOutcome.test.ts (320-combination table against an independent oracle)
  ├── 02 checkNodeTestFiles refusals, readVitestPrerequisite, sanitizeCause family, describe* catalogs, listMissing*, formatCodebookDiagnostic   [★★★ TESTED]
  └── 03 checkNodeTestFiles additional stop: its field, stop number, source-before-test order   [GAP] E1 (carried F18)
[+] extension/src/nodeLearningSave.ts
  ├── 04 save stages and retries: Save after the result, Retry stop, Retry read, Open retained history   [★★★ TESTED] nodeLearningSave.test.ts
  ├── 05 recordedNothing (7437ca4): no item and the Save refused for a refused run, a run without a history writer, a history sealed with no entry   [★★★ TESTED] nodeLearningSave.test.ts:478
  ├── 06 saveOnce after the stop (ae5f310, F61): a Save now whose stop sealed no entry ends as for a run that recorded nothing: the refusal, no receipt, no notice, no item   [★★★ TESTED] nodeLearningSave.test.ts:519
  │      flow: Codebook: Save Learning Replay → Save now while the run prepares with an empty journal → "Codebook could not save learning: …" (extension.ts:2129)
  │      Regression proof (informational; test written by fixing cycle 2) — red before the fix: yes (check removed in a scratch build: actual 'saved') · base control: unavailable (NodeLearningSave is not on main: collection error) · passes after fix: yes
  │      only this test kills the removed check (S1): the other 340 tests of 6 Save-related files pass; the real seal's zero count is row 30's test; the state deletion (S3) and the branch order (S4) are equivalent mutants
  └── 07 guards: another run's Save in flight, a failed seal shown before any click, an unverified summary at Retry, the read-back failure's Output entry   [GAP] E2 (carried F19)
[+] extension/src/ownedNodeTestRun.ts, revealOutput (ae5f310)
  ├── 08 release arm at the final result (F62, EN5): a stop during Vitest's report check keeps the passing result   [★★  TESTED] ownedNodeTestRun.test.ts:1305; service side nodeTestRunLifecycle.test.ts:306 (the stop waits, the run seals complete)
  │      Regression proof (informational; test written by fixing cycle 2) — red before the fix: yes (guard reverted in a scratch build: actual 'Owned Node test output terminal was closed.', expected 0; an inverted guard is red too) · base control: not run (on main the same stop could lose the result too, its release removing the report while the check read it: a fix test for a defect main shared, read from main's code) · passes after fix: yes
  │      ★★, not ★★★: the release reaches the terminal first only by microtask order; with 3 extra awaits in stop() before its release the reverted fix passes the test (1 or 2 extra awaits: red; 3 or 5: green) — Q1 below
  ├── 09 live arm: a live run brings its output tab back after each captured stop and after its result   [GAP] G1 (new): reveal dropped (R2) survives 69 tests in ownedNodeTestRun, nodeTestOutputTerminal and environment; guard widened to any final continuation (R3) survives the 54 of ownedNodeTestRun
  └──    release arm at a captured stop: unreachable, not counted (release() aborts the run at once, ownedNodeTestRun.ts:905 and :923; captureStop's last assertCurrent(), :665, throws on an aborted run and nothing is awaited before the reveal at :689); R9 survives as expected
[+] extension/src/service.ts, ownedNodeTestRun.ts
  ├── 10 passing run, failing exit, continuation ownership, seal, release retry, exit facts without a report (OwnedNodeTestRun.complete)   [★★★ TESTED]
  ├── 11 refusal at preparation: run error, no history, slot and start key freed   [★★★ TESTED]
  ├── 12 refusal at preparation: historyOutcome and recording reasons, nextActions   [GAP] E3 (carried F20)
  ├── 13 nodeRecordingReason (previous rows 10 and 11): the partial arm, the arms that read undefined, stopped before its history began, still preparing, seal failed   [★★  TESTED] nodeTestRunLifecycle.test.ts, nodeTestService.test.ts
  ├── 14 coverage cache none, partial, complete; run and history resources agree   [★★★ TESTED]
  ├── 15 material saved after the result and before Stop, counted at the seal   [GAP] E4 (carried F21)
  └── 16 HISTORY_SEAL_FAILED (test and browser), repeated stop, isError answer, playback refusal   [★★★ TESTED]
[+] extension/src/nodeTestHistoryStore.ts
  ├── 17 summary memo, list, read bound, changed material   [★★★ TESTED]
  ├── 18 memo invalidation when a writer ends without a seal   [GAP] E5 (carried F22)
  └── 19 step refusal INVALID_REFERENCE; citesEvent, describeCitationClaim   [★★  TESTED] nodeTestHistoryStore.test.ts (the other kinds: E10, carried F27)
[+] protocol.ts, resources.ts, errors.ts, constants.ts, SKILL.md 1.0.46
  └── 20 help, descriptions, instructions, API 1.8, skill pins   [★★★ TESTED] [→EVAL] the pilot C49-11 (NOT RUN, owner step D9)
[+] extension/src/environment.ts, extension.ts
  ├── 21 node-test profile names an older or missing root Vitest   [★★  TESTED] (ready path and read-error catch: carried F67)
  └── 22 Output channel and diagnostics sinks   [★★  TESTED] extension.test.ts
[+] extension/src/studyWorkspace.ts
  ├── 23 Open replay routing, landing, refusal, load failure, EN19, Saved rows, interim Application sentence   [★★★ TESTED]
  ├── 24 refreshPresentation (previous rows 22 and 23): render, publish without render (EN9), the publish arm's catalog and status, Save item with the pane closed   [★★  TESTED] studyWorkspaceRemoval.test.ts, studyWorkspaceTestReplay.test.ts (pane open but not rendering: R2, carried F76)
  └── 25 openNodeHistory, readNodeHistoryOpening (previous rows 24 and 25): journal read failure, supersession by a newer request and a changed revision, unreadable learner decision, learning material, reader position   [★★  TESTED] (reader closed mid-open: R1, carried F75)

USER FLOWS
  ├── 26 live Save: modal question, Save now, partial item, notice, Open replay   [★★★ TESTED] [→E2E] studyWorkspaceTestReplay.test.ts:522; native proof: N49 and the pilot
  ├── 27 Close browser replay answer opens the test replay   [★★★ TESTED] studyWorkspaceTestReplay.test.ts
  ├── 28 stale Save click for another run   [GAP] E6 (carried F23)
  ├── 29 Save item of a live test run while the Study pane is closed (a chat-driven run)   [★★  TESTED] studyWorkspaceTestReplay.test.ts
  └── 30 launch failed after its history began: Preparing, then no item, no replay   [★★★ TESTED] studyWorkspaceTestReplay.test.ts:720
  (ae5f310's two flows sit on rows 06 and 08: a Save now during an empty preparation; a stop during the final report check)

COVERAGE: 23/30 paths with any test (77%) | value-weighted 23/30 (77%) | Code paths: 19/25 | User flows: 4/5
QUALITY: ★★★:15 ★★:8 ★:0 unrated:0 | GAPS: 7 (6 carried E1–E6, 1 new G1; 0 E2E, 0 eval)
Legend: ★★★ behavior + edge + error | ★★ happy path, or an edge with a named weakness | ★ smoke | [→E2E] needs native proof | [→EVAL] needs an LLM eval
Rows: the previous 30 kept; 10+11, 22+23 and 24+25 merged (each at the weaker rating, ★★); ae5f310 adds 06, 08 and 09.

Tests: 377 → 377 test files (+0 new; this pass wrote no test; the merge of origin/main added staticHealth.test.mjs before it)
Test Coverage Audit: 30 new code paths. 23 covered (77% any test, 77% value-weighted). 0 tests generated (audit-only: both generation passes used).

REMAINING GAPS
New (ae5f310):
G1 extension/src/tests/ownedNodeTestRun.test.ts, new test "a live test run brings its output tab back after each captured stop and after its result": the harness's OutputTerminal stand-in counts reveals (a reveals() accessor); nodeRunHarness({ declaredStops: 2 }); after prepare() 1, after advance() 2, after complete() 3, exit code 0.
   Value: protects=a live test run brings its output tab back after each captured stop and after its result (the live arm of revealOutput); fails_when=revealOutput skips the reveal for a live run (a guard widened, e.g. to any final continuation) or drops it, so the Debug Console stays in front; why_new=the harness's terminal stand-in counts no reveal; the EN5 test proves only that none follows the release (mutants R2 and R3 survive all 54 tests); seam=none; no_claim=VS Code's panel focus (the terminal is a stand-in that counts reveals)
   Probe on a scratch copy of the compiled test (not the repository): green on HEAD; red with the reveal dropped and with the guard widened; green with captureStop unguarded (the unreachable arm) and with the fix reverted (the EN5 test catches that one).
Test quality (path covered, not counted as a gap):
Q1 extension/src/tests/ownedNodeTestRun.test.ts:1305, extended: await waitUntil(() => harness.calls.includes('stop-debugging'), 'the release', 1_000) between harness.run.stop() and finishRead().
   Value: protects=a passing result kept when a stop's release disposed the output terminal before Vitest's report check ended, whatever stop() awaits first; fails_when=the end of the report check reveals the disposed terminal (the F62 defect) after a stop() that awaits 3 or more times before its release; why_new=the current test reaches the release first by microtask order only: with 3 extra awaits in stop() the reverted fix passes it (probe F3); seam=none; no_claim=the service's seal (nodeTestRunLifecycle.test.ts:306) and VS Code's terminal
   Probe: green on HEAD and on HEAD with 5 extra awaits in stop(); red on the reverted fix, also with 5 extra awaits. A sibling of carried F38 (timing-dependent tests).
Carried, not re-proposed (Issue #107; value cards in the earlier diagrams): E1–E6 = F18–F23, E7–E10 = F24–F27, F67–F70, R1 = F75, R2 = F76. ae5f310 changes none of their meanings: its check catches only an empty journal; a Save now after Vitest journaled output seals a history of output entries (every entry type counts, nodeTestHistoryStore.ts:1216), which still takes the partial path: F70's preparing × save-now cell, unchanged.

Tests: 5,499 → 5,608 recorded in extension/test-timings.json (7a161b4).
Coverage: 77% value-weighted (77% including 0 weakly covered paths).
Test value: 9 tests written by the coverage audit (5 in pass 1, 4 in the repeat), 9 rejected by the authoring gate, 1 existing test extended, 0 paths weakly covered.
Remaining gaps, carried as Issue #107 tasks: E1 to E6 (F18 to F23), G1 (F80, a live run's output tab revealed after each stop and the result), Q1 (F81, the F62 test's microtask-order reliance), R1 and R2 (F75, F76).

Tests ran: pregate lanes 6154
Tests ran: tests count unavailable (the runner reports 5,608)
Tests ran: package count unavailable (the runner reports 1,069)

Pre-Landing Review

Three /ship fixing cycles; each pass ran the core checklist, six specialists (testing, maintainability, security, performance, simplification, state-machines with gstack-state-matrix) and the red team:

Pass Candidate Findings Fixed (each with a test that failed before)
1 d9ef3fbf9425 48 (3 critical) 7437ca4: the health gate, the Save item while the Study pane is closed, Save for a run that recorded nothing (F8)
2 d2260d5876fe 67 (12 critical: untested Save-matrix cells and one wrong row) ae5f310: F61 (Save now during an empty preparation), F62 (a stop during the report check lost a passing result)
3 118a65d44b34 57 (15 critical: untested cells and three wrong rows) 4d9d592: F78 (a closed output tab lost a passing result), F79 and F54 (Save asked before ending a run that could no longer reach its result)

The untested Save-matrix cells and every informational finding are carried as Issue #107 tasks (F9 to F91); the open Critical and Major ones are listed in the Issue's comment "/ship for PR-A stopped unconverged: the open Critical and Major defects". CYCLES 3 of 3: converged:false.

Gate edits: none detected (78 listed, 29 read; RH-1 ×2 justified: a fixture string and a symlink-capability skip).

Outside review (Step 11, Codex gpt-6-astra): adversarial completed (one High: the unanswered continuation, fixed in a56b86c as F95; one Medium carried as F102); structured codex review --base origin/main completed, GATE: PASS (one P2, the same as F102). Claude adversarial (native) completed: finding 1, a trust-boundary bypass, fixed in a56b86c as F96; findings 2 to 7 carried as F97 to F101 and the widened F74.

Exploratory QA

Report-only exploratory QA ran with each review pass (reports ship-c49-pra-20261022-070623, ship-c49-pra-p2-20261022-101044 and ship-c49-pra-p3-20261022-115612 in the gstack project store):

  • Smoke probes of the passing run, the lifecycle edges, the browser seal channel and the Save stages: pass in all three passes.
  • Plan checks: format:check, typecheck, lint, dupes, dead-code and health:static exit 0; pnpm test 5,598 tests with 0 failed (pass 3 on replay: one load-dependent wikiUpdateFlow flake, F82, passed alone 3 of 3); pnpm test:package 1,028 tests with 0 failed (the evidence tool's capture reads incomplete because of its own process-group kill; a direct run exits 0); pnpm health exit 0.
  • Native pilot (C49-11, macOS): see the Summary; FAIL, findings F92 to F94 carried.

Design Review

SCOPE_FRONTEND=false at Step 9 (no frontend product surface beyond webview strings), so design review (lite) and the Design specialist were skipped.

Eval Results

No eval suite in this repository; the bundled skill's sentences are pinned by script and unit tests, and the native pilot above is the only model run.

Scope Drift

Scope Check: DRIFT DETECTED (minor) and REQUIREMENTS MISSING. Delivered beyond PR-A's plan: the compaction hook fix F2 (7b1ada1), the macOS test isolation fixes F1 and F7, the EN9 refresh guard (planned for PR-B2) and describeNodeTestReplay (planned for PR-B1). Missing: the pilot's passing run and C49-12 (the pilot FAILed at its start), AC13 cards on 31 changed existing tests (F17) and two Open replay tests (F16).

Plan Completion

Plan version 7 (Issue #107, approved at the /autoplan final gate), audited three times in Step 8. The last audit: 36 items, 25 done, 1 changed (the summary memo keys on file size, mtime and inode), 5 partial (F9; F63; the redaction gaps F10, F15, F48, F49, F73, F51, F31, F58; F16; F17), 2 not done then (the version, now 0.2.33.0; the pilot, now run and FAILed), 3 unverifiable at code level (the Open replay arrangement, Cursor's managed skill copy, C49-12: native proof stays with N49 in PR-B2). Deferred to PR-B1 and PR-B2 by the plan: C49-3, C49-4, C49-5, C49-8, C49-9 and the Study-section half of C49-6.

Linked Spec

Linked to #107 (partial delivery — not auto-closing). Spec archive 20261021-224930-55890-cycle-0049-a-vitest-test-run-the-editor-chat-explained-is-sa.md, filed 2026-10-21T13:49:30Z. Remaining work: PR-B1 (the recorded test output in the Application pane, the Study section and the summary's selected step) and PR-B2 (the recorded source, transitions, timeline and the native round N49 in both editors), with the carried tasks. Close #107 manually after follow-up lands (after PR-B2).

Issues

Refs #107

Verification Results

Step 8.1's obligations, executed at Step 9 (pass 3) and again at Step 16 on the final tree: AC11's commands (format:check, typecheck, lint, test:package, dupes, dead-code, health, health:static, pnpm test): PASS; AC14 (browser outcomes, save and replay unchanged, by pnpm test): PASS; EN20 (timings re-recorded): PASS (7a161b4); AC15 (the pilot recorded before the merge): PASS as a record, its result FAIL; AC10 (N49): NOT RUN, owed by PR-B2. 0 BLOCKED.

Documentation

Status: current. No documentation edits and no blockers: every document Cycle 0049's plan owes this pull request (Design 12 and AC12: the CHANGELOG entry, the README status, the extension README's test-replay states with Quick start and Terms, the chat models in AGENTS.md, the version) is present and agrees with the code. The gaps below are non-blocking documentation debt.

Audited scope. Branch codex/cycle-0049-test-run-learning-replay, base f59c06c9a233, head 7a161b4b14b8: 75 committed files plus 6 unstaged files (77 distinct paths; nothing staged or untracked), read as the working-tree bytes the candidate describes (its 1,094 index entries and 80 blob hashes equal the tree). Read in full: README.md, AGENTS.md, CLAUDE.md, PRODUCT_GOAL.md, extension/README.md and extension/skills/codebook/SKILL.md; the approved PNG was opened and its identity checked (1,878,402 bytes, SHA-256 f189ee82541a72ed981e3f7a0e21bd35a19b6601df1c7208671117a335caf2aa, 1024 x 1536). The 0.2.33.0 entry of CHANGELOG.md and VERSION were read for the consistency pass only (protected, not edited); the plan's Design 12 and AC12 and the pilot result were read in Issue #107. Checked against source or by live run: the status-bar, Saved studies and Application pane strings and the preparation order (checks run before the history begins); the API 1.8 outcomes and nodeTest fields; the codes HISTORY_SEAL_FAILED, PLAYBACK_PROFILE_UNSUPPORTED and INVALID_REFERENCE; the Output › Codebook entry and the <storage>, <workspace> and ~ cause words; 16 tools and 6 resources; the Vitest 4 minimum; every version literal (0.2.33.0 and 0.2.33, API 1.8, skill 1.0.46); the compaction hook (its output is 6,279 characters with the rule at character 209, and node --test .codex/restore-product-context.test.mjs passes 6 of 6); and the package workflow commands. The 202 links in the seven documents have 0 broken (one command:… in a historical CHANGELOG entry is example text), 35 commit- and tag-pinned GitHub links resolve (9 only on GitHub: the local clone lacks commit 273660c49b), and the issue and PR states README.md names are right (#105 and #90 closed, PRs #106 and #91 merged, #107 open). Banned terms: none (the repository keeps no list). Not present: ARCHITECTURE.md, CONTRIBUTING.md, TODOS.md and authored templates; the two other doc-like files are frozen test fixtures, not documentation.

Edits. None. The unstaged hunks already in extension/README.md (Local session, Preparing wording, Output › Codebook paragraph, skill paragraph, API 1.8 sentences, the codebook-0.2.33.vsix path) were checked and kept as found.

Documentation health:

README.md                           Current (status, tracking, 0.2.33.0, API 1.8 and skill 1.0.46 agree with CHANGELOG and code)
AGENTS.md                           Current (re-entry rule, varied-questions rule and chat models present; hook paragraph matches .codex/ and .claude/settings.json)
CLAUDE.md                           Current (unchanged)
PRODUCT_GOAL.md                     Current (unchanged; goal version 10; design target identity verified)
docs/designs/assets/codebook-product-spec-v9.png  Current (identity verified; opened and inspected)
extension/README.md                 Current (test-replay states, Quick start, Terms, API 1.8 text and package path verified against source)
extension/skills/codebook/SKILL.md  Current (1.0.46 "Test runs" paragraph matches the tool descriptions, node-test help and error recovery)
CHANGELOG.md                        Current (read only, protected; the 0.2.33.0 entry agrees with the code; Step 5 not run)
TODOS.md                            Skipped (does not exist; the cycle rule forbids one)

Documentation coverage (public surface of the release diff):

Entity Reference How-to Tutorial Explanation
Test replay: save, open, Saved studies rows yes (extension/README "Test replays", README.md, CHANGELOG) yes ("Quick start: a test replay") no partial (PRODUCT_GOAL.md replay contract; README.md status)
MCP API 1.8 fields (nodeTest.testResult, learningCoverage, learningMissing, save-node-learning) yes (extension/README, node-test help, CHANGELOG) yes (node-test help walkthrough; SKILL.md "Test runs") no no
Codes HISTORY_SEAL_FAILED, PLAYBACK_PROFILE_UNSUPPORTED, INVALID_REFERENCE yes (codebook://help/errors, extension/README, CHANGELOG) partial (recovery in codebook://help/errors; no row in the README recovery table) no no
Output › Codebook channel yes (extension/README paragraph, CHANGELOG) no (no steps to open the channel or copy an entry) no no
Root Vitest named by codebook_status partial (README names the Vitest 4 minimum; only CHANGELOG and the node-test help say status reports it) yes (Quick start step 1) no no
VS Code Local-session requirement yes (extension/README setup step 3 and Quick start step 2, CHANGELOG) partial (SKILL.md line 13 still sends the learner to Set Up AI Chat; the Copilot image-transfer text says "new Copilot Chat") no no

Critical gaps (no coverage at all): none. Common gaps, non-blocking and carried under the cycle rule (nothing was written to fill them, and no docs-debt label is suggested: the repository has none and AGENTS.md carries debt in the cycle Issue): the two new codes have no row in the README recovery table; the Output › Codebook channel has no how-to; and extension/README.md omits four details that the CHANGELOG entry names (the node-history resource reports the same three nodeTest fields; the start's check list and that a refusal frees the run's slot and key; codebook_status names a missing or older root Vitest; no stored format changes). CodeRabbit's CHANGELOG instruction may report that last group. The skill's Local-session gap is Issue #107's F94, and the full saved-study library that refused the pilot's start is F92 and F93.

Diagram drift: none. The only diagrams are the two mermaid blocks in PRODUCT_GOAL.md (the product flow and the replay sequence example); this diff renames, splits or removes none of their entities.

Outside this diff, not edited: AGENTS.md ("commands run from the repository root") and extension/README.md ("From the repository root: pnpm --dir extension ...") give commands that fail on a Mac whose Corepack default pnpm is not the pinned 12.4.2: run from the root, pnpm --dir extension ... stops with ERR_PNPM_BAD_PM_VERSION (12.14.0 against 12.4.2), while pnpm run inside extension/ selects 12.4.2 (re-run in this audit).

Test plan

  • gstack-pregate --allow-repo-commands (Step 5, rerun on the final head 09fa794): preflight passes with two warnings already triaged (5 MEDIUM non-credential secrets-scan matches on hashes, ids and one test comment; one stale API 1.7 label in a test comment, downgraded by the policy); lanes 6,154 tests in 248 files, exit 0; one requires-remote obligation (package.yml/package on Linux, .codex/restore-product-context.test.mjs) cleared with this pull request's CI run.
  • cd extension && pnpm test: 5,608 tests, 5,602 pass, 0 fail, 6 skipped.
  • cd extension && pnpm test:package: 1,069 tests, 1,059 pass, 0 fail, 10 skipped.
  • pnpm build, pnpm package, pnpm package:check: exit 0 (codebook-0.2.33.vsix, 1,901,964 bytes, SHA-256 43145400133464264b7af02c2f8f749cbc11a8afab4f1acba0fd725b7a91766c).
  • pnpm format:check, typecheck, lint, health:static, dupes, dead-code: exit 0.
  • pnpm health with the pinned Node 24.21.0: exit 0, no finding beyond the measured baseline (its suite: 5,608 tests, 0 failed; coverage of 520 source files). A first run under Node 24.22.1 was refused with exit 2 ("the producers differ from the reference"), as the gate's pins require.
  • Required CI on the exact head, then the exact-HEAD gate and a squash merge with --match-head-commit.

🤖 Generated with Claude Code

ryota-murakami and others added 27 commits October 21, 2026 23:43
The compaction hook printed all of AGENTS.md (39,738 bytes). Claude Code
injects hook context above 10,000 characters only as a 2,000-character
preview, so the restore order did not reach the agent whole. The hook now
prints the owner's rule first (PRODUCT_GOAL.md in full and the approved PNG,
then the cycle Issue and AGENTS.md), the scope sentence and the Mandatory
entry and Owner directions sections verbatim (about 6,000 characters), and
keeps the rule when those sections outgrow the limit.

AGENTS.md carries the owner's rules of 2026-10-21: the re-entry rule after
every compaction, varied questions with learning fitted to the question, and
the development chats (GPT6 Luna in Copilot Chat, Grok 4.6 high in Cursor).

Issue #107 (task F2).
…utilities for test replays

- projectNodeTestOutcome: Design 1's outcome table for node-test runs, total over its inputs
- listMissingNodeLearning / listMissingNodeExplanations: structured missing parts and learningCoverage
- describeNodeTestResult / nodeTestResultKind: the verdict and nodeTest.testResult
- describeNodeLearningSave / describeNodeLearningPart: the Save String catalog
- sanitizeCause / cutCauseForDisplay: causes without storage, workspace or home paths, bounded on display
- readVitestPrerequisite / checkNodeTestFiles / resolveOwnedRepositoryFile: the deterministic node-test checks
  shared by preparation and the launch, with the Error contract's typed refusals
- prepareNodeTestLaunch reuses the shared checks; its tests pin the typed refusals
- API_VERSION 1.8; UI_CAUSE_MAX_CHARS, NODE_HISTORY_SUMMARY_READ_CONCURRENCY and Vitest package bounds
…at preparation

- state() projects node-test outcomes through projectNodeTestOutcome (Design 1): a passing run with every
  declared stop and a complete seal reads demonstrated; recording finalized with the sealed journal's hash;
  recordingAvailable/replayAvailable; nodeTest.testResult, learningCoverage and learningMissing from the
  run's coverage cache (EN2); nextActions save-node-learning
- Lifecycle (EN4-EN6): the run owns its in-flight continuation and Stop settles it before the seal; the
  seal's complete comes from the test's own facts; a release error is cleared by a successful retry;
  OwnedNodeTestRun.complete keeps the exit facts of a failing run without a report and removes the report
  only after the check settled; hasRunEnded counts a completed test
- Refusals at preparation (EN1): the deterministic file checks run before the history begins; a refused
  run records nothing and frees its slot and start key ('refused-at-preparation')
- codebook_stop answers HISTORY_SEAL_FAILED with isError and the run state after a failed seal (test
  history or browser recording finish); codebook_playback refuses a test replay with
  PLAYBACK_PROFILE_UNSUPPORTED before the presentation-idle check
- Store: finish() returns the sealed journal's hash; list() is a memoized summary projection read at most
  four at a time; step refusals are typed INVALID_REFERENCE naming the step and the event (X19)
- Status names a missing or older root Vitest before any start (X9, EN11)
- Help, instructions and descriptions carry the API 1.8 sentences, the example, walkthrough, results,
  refusals and limits; the removed 'unavailable' sentences are gone; neutral testTitles example
- Causes of a failed seal are sanitized at assignment and reported to the diagnostics sink (EN12, X5)
- EN9 display ownership: while a test replay is selected the refresh publishes the catalog and status only
- Tests pinning API 1.7, the old outcomes, the old help and the old step refusal are updated
…port only after its check

The previous commit described this change but did not include it (its edit script was not run).
- OwnedNodeTestRun.complete returns a non-zero exit's or a signal's facts with reportFailure beside them;
  an exit 0 without a confirming report still rejects; the release awaits the check before removing the
  private report folder
- Tests: a signal and a failing exit without a report keep their result, an exit 0 without one returns
  none, and a stop during the report check removes the report only after the read
…mary and help (PR-A)

- nodeTestRunLifecycle: refusals at preparation free slot and key (17 refused, 18th admitted); Stop
  during an advance, during the report check and during the result write settles before the seal; a
  signal and an exit 0 without a report; a release retried after a failure seals complete; coverage
  grows from the cache without store reads and the history resource agrees; HISTORY_SEAL_FAILED with
  a sanitized cause, a repeated stop and one Output entry; the MCP stop answer; no seal failure for a
  run without a history; PLAYBACK_PROFILE_UNSUPPORTED before the presentation check
- nodeTestHistorySummary: a second list reads no journal, a forged journal is read again, a changed
  material is counted without the journal, at most four reads at a time
- nodeTestHelp: the example parses once filled, the walkthrough's order, refusals, Vitest limit,
  failing tests and replay contents, no development-target title, API 1.8 instructions and descriptions
- environment: an older root Vitest makes the node-test profile name it before any start
…udies rows and the Output channel

- NodeLearningSave (EN14): the status item's states for a test run (preparing, live with the stop count,
  finished, saving, saved, partially saved, stop rejected, read after the seal failed, seal failed), the
  question before the result, the composed partial-save notification, Retry per stage and receipts by
  material revision; StudyWorkspace only wires its item and commands to it
- Open replay routes a test replay to openNodeHistoryFromHost: the Study pane opens when closed, a browser
  replay is closed first by the learner's choice, a failed open raises the load-failure notification and
  an Output entry; the pane's node-history message and the host open share openNodeHistory under one
  request counter (EN19), landing on the first verified stop without a saved position (DS1)
- Saved studies rows read 'Test replay · <verdict> · capture … · k of m stops explained'; the Application
  pane shows the interim sentence; the Study pane's 'replay unavailable' sentences are gone
- One 'Codebook' LogOutputChannel receives the service's and the workspace's diagnostics
- Workspace tests that start at a history's first event pin that position; label pins updated
…ows and Output entries

- studyWorkspaceTestReplay: Open replay in a closed Study pane lands on the first verified stop, a saved
  position wins, an unverified stop lands on the first event, a browser replay refuses with one
  notification, an unreadable history raises the load-failure notification and an Output entry, and a
  pane open during a host open wins with a double click showing one replay
- studyWorkspaceRemoval: refreshes after the background selection changed leave a selected test replay's
  source and debugger on its stop (red without the EN9 guard)
- describeNodeTestReplay: Saved studies rows, the interim Application sentence, the Output entry format
…ed learning only with complete coverage

A request about a test, or about library behavior a test exercises, uses the
node-test profile and its help; a test stop is explained in chat, never
through codebook_explain, whose paragraphs now name the browser profile; an
error before the first stop is a refusal at preparation; saved learning is
claimed only with studyOutcome demonstrated and nodeTest.learningCoverage
complete; a failing test is named as such; the learner opens the test replay
from the status bar. Seven pins read the bundled skill (six fail on 1.0.45).
CHANGELOG 0.2.33.0 says what a test replay is, how to get one and what API 1.8
changes for node-test runs; README's implementation status and tracking name
Cycle 0049 and Cycle 0048; the extension README's Save learning replay section
gains the test replay's states, a five-step quick start and a Terms list, its
cycle-records link points to the active cycle Issue, and its API sections say 1.8.
…ned history

codebook_stop answers HISTORY_SEAL_FAILED when a browser run's recording cannot
be finished at its stop, the same for a repeated stop with no second finish,
and nothing for a study the learner removed (each red without its guard);
Open retained history opens a test history whose seal failed as an interrupted
prefix on its recorded stop.
…nd-in

a1e47eb pushed the extension's storage folder into the service's
extraStorageRoots array and created the Output channel through
vscode.window without a guard: a service without that array threw before the
server started, and a host without a window API threw at activation. The
extension now assigns extraStorageRoots, as it assigns the service's other
collaborators, and creates the channel only where the window API exists.
Red before: 20 tests of editorChatOnboarding and extension (one of which then
hung its file until it was ended).
…s rule)

The examples of requireNodeTestTitles and verifyVitestTestResult quoted the
development target's test titles; they now use the help's neutral scheduler
example (Design 6).
…ntry count kept at the seal

pnpm dupes found the material read with its citation check twice in the store
summary (readSummary, summaryWithMaterial): both now call
readCitedLearningMaterial. pnpm dead-code did not see the run's outcome read
NodeTestHistoryWriter.entryCount through the record's optional writer: the seal
now keeps the sealed journal's entry count on the record, which only a sealed
history's outcome reads.
…test title says API 1.8

The pre-gate's credential scan read the example's key: 'explain-delayed-job-1'
as a secret-named assignment and cannot be downgraded; the start key is the
agent's own choice, so the example now shows '<a new key for this start>' like
its other placeholders (the example still parses as a start). The configured
discovery test asserts API 1.8 and its title now says so (pre-gate literals).
…now names the system it describes

Three boot-identity tests describe a system without a kernel boot id (Windows,
or the Linux their start inputs name) but read the boot for the system the test
ran on: on macOS readBootIdentity then reads the real kern.bootsessionuuid and
the record carries an id the test does not expect. They now pass the platform
they describe to readBootIdentity and currentStorageOwner. The recording-store
lock test refused unlink only for '<tmpdir>/writer.lock', while the store names
its lock under the directory's real path (/private/var on macOS), so the refusal
never applied; the test now builds the lock path from realpath(directory).
Red before on macOS (4 of 47 in the three files), green after; Linux unchanged.
No product code changes. Unblocks the /ship pre-gate on macOS, which refuses
publication while a lane fails (PREGATE_LANE_FAILED, not downgradable).
…itized stack, Close browser replay, the live Save flow) and re-recorded test timings (EN20)
… the Study pane is closed, a run that recorded nothing offers no Save (F8), and the new branching moves out of existing functions so pnpm health passes again
…macOS-only failures were test isolation; stand-ins are named by the temporary folder's real path, and a reaped-later zombie is not counted as a running process
…e pre-gate's secrets scan read the key literals as credentials)
…al report check keeps the passing result (the output tab is not revealed once the release began), and a Save now while the run still prepares ends as for a run that recorded nothing
…the test output tab after the test exited keeps its passing result (no reveal once the run is aborted), and Save asks before ending a test run only while the run can still reach its result (not after a rejected stop or a failed run)
…recover refusals only by their code

/ship Step 11 (adversarial review), the last review of PR-A's unconverged ship:

- Codex P1: the final continuation awaited its `continue` (and the pause
  check's stack read) without the run's cancellation, so an adapter that
  never answered held the continuation, and the release, Stop, Save and the
  next start waited for it for good. Every DAP request a continuation awaits
  (the entry, next-stop and final `continue`, the paused stack and frame
  reads, the stop's scopes) now goes through `awaitResponse` with the run's
  abort signal and its 20 s bound. Six tests hold one request each in the
  harness adapter; all six failed before ("still waiting for the adapter").
- Claude adversarial finding 1 (a trust-boundary bypass): the skill and the
  node-test help told the agent to "follow that error's fix", while a
  preparation error can quote up to 2 KiB of the repository's Vitest output.
  The agent now recovers only as codebook://help/errors gives the recovery
  for the error's code, starts again only where that recovery allows it, and
  treats quoted text as data; a learner's stop (RUNTIME_FAILED) is no longer
  restarted. Both pinning tests failed before.
…s, the seal failure's reason, a replay's unreadable side files, the refresh's publish arm, a launch that failed after its history began)
Cycle 0049, PR-A: a test run becomes a test replay. VERSION 0.2.33.0,
extension/package.json and EXTENSION_VERSION 0.2.33 with the package test's
expected version; the CHANGELOG entry reconciled with the pilot's result,
the scope of sanitized causes, the Preparing condition and skill 1.0.46's
restart rule; extension/README.md names every behavior the entry names
(the Local chat in VS Code, the Output entry, the skill's rules, the API 1.8
sentences) and the 0.2.33 package path.

Co-Authored-By: Claude <noreply@anthropic.com>
@ryota-murakami
ryota-murakami merged commit 34ac324 into main Oct 22, 2026
26 checks passed
@ryota-murakami
ryota-murakami deleted the codex/cycle-0049-test-run-learning-replay branch October 22, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cycle 0049: a Vitest test run the editor chat explained is saved and replayed as a learning replay in both editors

1 participant