diff --git a/CHANGELOG.md b/CHANGELOG.md index 54ba7aa..e9663b0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/). ### Added +- Pace and frame the cast of `rune proof run` for a reader: a rule per capability, a boxed title and kind line per scene, typed commands, a tick or cross per step, `--pause` between scenes, a tally. +- Read `# text` lines in a console fence as comments: dim in the cast, recorded in the transcript with the command lines, compared by `--check`. - Add proof frontmatter (`type: proof`, `change`, `head`, `recorded`, `transcript`, `scenes`) that `rune graph export` reads. - Emit `rune:Proof` with a proves edge per proven scene, only while `proof.txt` matches the recorded transcript. - Add proof README validation to `rune spec validate`: a malformed field or an instruction scene without a model fails the run and names the field. @@ -209,6 +211,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/). ### Fixed +- Number a console fence error by the line of the README file, not of the body under the frontmatter. - Fix `rune run --clean` for codex: the clean `config.toml` keeps every `[model_providers.*]` table with its `auth` and names the one custom provider as the route. - Fix `rune run grok@grok --clean-harness-state` exiting 127 through the `harness-run` shim: the clean run names the real `~/.grok/bin` in `HARNESS_REAL_BIN_DIR` before it moves `HOME`. - Fix the identity tests to expect the versioned display names the synced `author-identity.py` derives (`Claude Fable 5.2`, not `Claude`). diff --git a/docs/changes/proof-scene-layout/adr.md b/docs/changes/proof-scene-layout/adr.md new file mode 100644 index 0000000..74a9555 --- /dev/null +++ b/docs/changes/proof-scene-layout/adr.md @@ -0,0 +1,51 @@ +--- +title: "A proof cast is paced and framed for a reader" +description: "rune proof run writes the cast on a presentation timeline with one frame vocabulary per scene, so a person can read a GIF and a page shows the same scene a transcript records" +type: adr +category: architecture +tags: + - cli + - proof + - cast +status: proposed +created: 2026-09-27 +updated: 2026-09-27 +author: "@N4M3Z" +project: cli +related: + - "DECK-0015 Behavior Proof per Change" + - "DECK-0019 A Proof Plays as a Cast Where a Person Reads It" +responsible: ["@N4M3Z"] +accountable: ["@N4M3Z"] +consulted: ["claude-fable-5-1", "gpt-6-astra"] +informed: [] +upstream: + - "https://docs.asciinema.org/manual/asciicast/v2/" +change: proof-scene-layout +--- + +# A proof cast is paced and framed for a reader + +## Context and Problem Statement + +A proof has two readers. The transcript binds the claim and is read by `rune proof run --check` and by the graph. The cast is read by a person, on a page or as a GIF, and the person decides whether to sign. `rune proof run` writes the cast at real offsets, so a proof of fast commands is one frame of output with no typed command, and the deck's bash driver holds the layout the owner picked. Which timeline and which frame vocabulary should the cast carry? + +## Considered Options + +1. Real time. The cast shows what the clock saw. Honest about duration, unreadable for fast commands, and different from the driver's layout. +2. Presentation time with the driver's layout. The cast writes fixed pauses and types each command, one frame vocabulary for a proof and for a session recording, and the transcript alone binds the claim. +3. Post-process the real-time cast in the deck. The layout lives in a second tool, every proof needs two steps, and a page can show a cast the run never wrote. + +## Decision Outcome + +Option 2. The cast MUST be paced for a reader: a fixed pause after each scene (`--pause`, default 3 s), a shorter one after a header or a comment, and each command typed word by word behind a prompt. Each scene MUST open with a boxed `N/T · title` and a kind line naming the requirement, each capability with a rule naming it and its scene count, and the cast MUST close with a tally. Each step MUST end with one tick that names the exit status and the match, or a red cross with the reason. A fence comment `# text` MUST show dim and italic before its command and MUST be part of the transcript. The transcript stays the record. The cast's offsets are a reading aid and never evidence of duration. + +Option 1 keeps the reader out. Option 3 splits the layout from the run and lets a page play a cast that differs from the transcript. + +### Consequences + +- [+] One frame vocabulary for a proof, a session recording, and the page's scene list, in the layout the owner already chose. +- [+] A GIF of a fast proof is readable without the page. +- [-] The cast no longer tells how long a command took. A proof that wants duration on record puts it in the output. +- [-] Every recorded cast re-records to gain the layout, in the deck and in the cli. +- [-] Comments in a fence join the transcript, so adding one to a recorded proof changes its digest and needs a re-run. diff --git a/docs/changes/proof-scene-layout/proposal.md b/docs/changes/proof-scene-layout/proposal.md new file mode 100644 index 0000000..61f9e95 --- /dev/null +++ b/docs/changes/proof-scene-layout/proposal.md @@ -0,0 +1,31 @@ +--- +adr: docs/changes/proof-scene-layout/adr.md +status: proposed +decisions: ["A proof cast is paced and framed for a reader"] +--- + +# Proof scene layout + +## Why + +`rune proof run` writes `proof.cast` from the captured output at real offsets. Twenty-three scenes of a fast command sit inside the first 0.2 seconds. No typed command appears, and the only marker between scenes is a blue rule with a `# Scenario:` line that carries the whole scenario key. A person who opens the GIF sees one frame of output and cannot tell which command produced it. The deck's fallback driver `prove.sh` has the layout the owner picked on 2026-09-25: a capability rule, a boxed `N/T · title`, a kind line, dim comments, a typed command behind a prompt, the output, one green tick per expectation, and a closing tally. That layout exists only in bash, so a proof recorded by rune and one recorded by the driver look different. + +## What Changes + +- The cast `rune proof run` writes is paced for a reader, not by the clock: a fixed pause after each scene (`--pause`, default 3 s), a shorter one after a header or a comment. Each command is typed word by word behind a prompt. +- Each scene opens with a boxed `N/T · title` and a kind line that names the requirement. Each capability opens with a rule that carries its name and scene count. The cast closes with a tally. +- A fence line `# text` before a command is a comment. The cast shows it dim and italic, and the transcript records it with the command lines. +- Each step ends with one tick that names the exit status and that the output matched; a failed step ends with a red cross and the first line of the reason. +- The transcript `proof.txt` and its digest change only by the comment lines a README adds. + +## Capabilities + +### New Capabilities + +- proof-scene-layout + +## Impact + +- `src/cli/proof/fence.rs` (comment lines), `src/cli/proof/runner.rs` (per-step verdicts), `src/cli/proof/mod.rs` (the cast writer and pacing), `src/cli/mod.rs` (`--pause`). +- Every recorded `proof.cast` re-records to gain the layout. The transcripts hold unless a README adds comments. +- The deck's AcceptanceTesting skill drops `prove.sh` as the layout's home once this merges. The page tooling renders both proof and session casts in one frame vocabulary. diff --git a/docs/changes/proof-scene-layout/specs/proof-scene-layout/spec.md b/docs/changes/proof-scene-layout/specs/proof-scene-layout/spec.md new file mode 100644 index 0000000..0b9cc98 --- /dev/null +++ b/docs/changes/proof-scene-layout/specs/proof-scene-layout/spec.md @@ -0,0 +1,40 @@ +# proof-scene-layout + +## Purpose + +How the cast `rune proof run` writes reads for a person: the presentation timeline, the frame vocabulary per scene, and the fence comments that explain a step. + +## ADDED Requirements + +### Requirement: The Cast Is Paced For A Reader + +The run MUST write `proof.cast` on a presentation timeline rather than the clock: a rule per capability with its name and scene count, a boxed `N/T · title` and a kind line naming the requirement per scene, each command typed word by word behind a prompt with its input lines, its output, then one green tick naming the exit status and the match, or a red cross with the reason. A fixed pause (`--pause`, default 3 s) separates the scenes and a tally closes the cast. + +#### Scenario: Scenes are paced for a reader + +- **WHEN** a proof of three scenes whose commands finish within a millisecond runs with the default pause +- **THEN** the cast's last event sits past nine seconds, each scene's box and tick are separate events, and no two scenes share a second + +#### Scenario: Owner shortens the pause + +- **WHEN** the run is started with `--pause 0.5` +- **THEN** consecutive scene boxes in the cast sit about half a second apart plus the scene's own typing and output time + +#### Scenario: Failed step shows its reason in the cast + +- **WHEN** a step's output disagrees with the fence +- **THEN** the cast shows the command, its output, and a red cross line that starts with `unproven:` and states the difference, and the tally counts the scene as unproven + +### Requirement: A Fence Comment Explains The Step + +A line `# text` inside a console fence, before a command, MUST be a comment: the runner shows it dim and italic in the cast before the command it precedes, records it in the transcript with that command's lines, and `--check` compares it like a command line. A comment after a command's input lines belongs to the next command. A comment with no command after it is a fence error that names the line. + +#### Scenario: Comment before a command + +- **WHEN** a fence holds `# the guard refuses rm -rf` above `$ rune hook run --harness claude --native-event PreToolUse` +- **THEN** the cast shows the comment dim before the typed command and `proof.txt` holds the comment line above the command line + +#### Scenario: Comment with no command + +- **WHEN** a fence ends with a comment line and no command after it +- **THEN** the run fails before any scene runs and names the line of the dangling comment diff --git a/docs/changes/proof-scene-layout/tasks.md b/docs/changes/proof-scene-layout/tasks.md new file mode 100644 index 0000000..f685f5c --- /dev/null +++ b/docs/changes/proof-scene-layout/tasks.md @@ -0,0 +1,18 @@ +# Tasks + +## 1. Fence comments + +- [ ] 1.1 `src/cli/proof/fence.rs`: `# text` lines before a command parse as that step's comments, join `command_lines`, a dangling comment is a fence error naming its line +- [ ] 1.2 Tests: a comment before a command, a comment after input lines belongs to the next command, a dangling comment + +## 2. Cast layout and pacing + +- [ ] 2.1 `src/cli/proof/cast.rs`: the presentation timeline (`Cast` with pauses), the capability rule, the scene box and kind line, the dim comment, the typed command and input lines, the output, the tick and the cross, the tally +- [ ] 2.2 `src/cli/proof/runner.rs`: per-step outcomes (command lines, output, exit, match or reason) for the cast writer +- [ ] 2.3 `src/cli/proof/mod.rs` and `src/cli/mod.rs`: `--pause ` on `rune proof run`, the run assembles the cast through the writer +- [ ] 2.4 Tests: three fast scenes sit past nine seconds with the default pause, `--pause 0.5` spaces the boxes by about half a second, a failed step's cross and the tally, the transcript unchanged by the layout + +## 3. Proof and records + +- [ ] 3.1 Re-record `docs/proofs/harness-hook-dispatch` and this change's own proof with the layout +- [ ] 3.2 CHANGELOG entries, the deck's AcceptanceTesting skill points at `rune proof run` for the layout diff --git a/docs/proofs/proof-scene-layout/README.md b/docs/proofs/proof-scene-layout/README.md new file mode 100644 index 0000000..5ad15a3 --- /dev/null +++ b/docs/proofs/proof-scene-layout/README.md @@ -0,0 +1,112 @@ +--- +type: proof +change: proof-scene-layout +head: 275680d7a3efd4752c679f42d529c078895fec9f +recorded: 2026-10-16 +transcript: f5a7cb262873035586a08e60c10c2a11a43a675dcf8e985b61b0cb51d3aa9207 +scenes: +- scenario: proof-scene-layout#the-cast-is-paced-for-a-reader/scenes-are-paced-for-a-reader + kind: check +- scenario: proof-scene-layout#the-cast-is-paced-for-a-reader/owner-shortens-the-pause + kind: check +- scenario: proof-scene-layout#the-cast-is-paced-for-a-reader/failed-step-shows-its-reason-in-the-cast + kind: check +- scenario: proof-scene-layout#a-fence-comment-explains-the-step/comment-before-a-command + kind: check +- scenario: proof-scene-layout#a-fence-comment-explains-the-step/comment-with-no-command + kind: check +--- + +# Proof: proof-scene-layout + +One scene per scenario, in closure order. The scenes copy a proof fixture under `tests/fixtures/proof/`, run `rune proof run` on the copy, and read the cast it wrote back with a short Python script fed through the fence. + +## proof-scene-layout#the-cast-is-paced-for-a-reader/scenes-are-paced-for-a-reader + +```console +# three scenes whose commands finish within a millisecond, default pause +$ sh -c 'cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/paced" repo && rune proof run sample-change-one --source repo >/dev/null' +$ python3 - +< import json +< events = [json.loads(l) for l in open("repo/docs/proofs/sample-change-one/proof.cast").readlines()[1:]] +< boxes = [t for t, _, d in events if "\u250c" in d] +< ticks = [t for t, _, d in events if "\u2713" in d] +< print("last event past nine seconds:", events[-1][0] > 9) +< print("boxes:", len(boxes), "ticks:", len(ticks)) +< print("seconds shared by two scenes:", len(boxes) - len({int(t) for t in boxes})) +last event past nine seconds: True +boxes: 3 ticks: 3 +seconds shared by two scenes: 0 +``` + +## proof-scene-layout#the-cast-is-paced-for-a-reader/owner-shortens-the-pause + +```console +# the same proof at --pause 0.5 and at the default; the box gap shrinks by 2.5 s +$ sh -c 'cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/paced" slow && cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/paced" fast && rune proof run sample-change-one --source slow >/dev/null && rune proof run sample-change-one --pause 0.5 --source fast >/dev/null' +$ python3 - +< import json +< def gap(path): +< events = [json.loads(l) for l in open(path).readlines()[1:]] +< boxes = [t for t, _, d in events if "\u250c" in d] +< return boxes[1] - boxes[0] +< slow = gap("slow/docs/proofs/sample-change-one/proof.cast") +< fast = gap("fast/docs/proofs/sample-change-one/proof.cast") +< print("gap difference:", round(slow - fast, 2)) +< print("fast gap under three seconds:", fast < 3) +gap difference: 2.5 +fast gap under three seconds: True +``` + +## proof-scene-layout#the-cast-is-paced-for-a-reader/failed-step-shows-its-reason-in-the-cast + +```console +# the first scene expects rune 0.5.0 and the run is newer +$ sh -c 'cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/mismatch" repo && rune proof run sample-change-one --source repo' +? 1 +sample-capability-one#thing-holds/version-prints ... unproven: output of `rune --version` differs: +- rune 0.5.0 ([..]) ++ rune [..] +sample-capability-one#thing-holds/missing-tag-fails ... ok (check) +2 scenes, 1 proven, 1 unproven; transcript [..] +$ python3 - +< import json, re +< text = "".join(json.loads(l)[2] for l in open("repo/docs/proofs/sample-change-one/proof.cast").readlines()[1:]) +< plain = re.sub(r"\x1b\[[0-9;]*m", "", text).replace("\r", "") +< start = plain.index("rune --version") +< print(plain[start:start + 40].splitlines()[0]) +< print([l for l in plain.splitlines() if l.startswith("\u2717")][0]) +< print(plain.splitlines()[-1]) +rune --version +✗ unproven: output of `rune --version` differs: +2 scenes · 1 proven · 1 unproven +``` + +## proof-scene-layout#a-fence-comment-explains-the-step/comment-before-a-command + +```console +# the third scene of the fixture carries a comment above its command +$ sh -c 'cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/paced" repo && rune proof run sample-change-one --source repo >/dev/null' +$ grep -n -B1 -A1 '^# the shell' repo/docs/proofs/sample-change-one/proof.txt +[..]-kind: check +[..]:# the shell prints what it is given +[..]-$ printf 'one\n' +$ python3 - +< import json, re +< text = "".join(json.loads(l)[2] for l in open("repo/docs/proofs/sample-change-one/proof.cast").readlines()[1:]) +< plain = re.sub(r"\x1b\[[0-9;]*m", "", text).replace("\r", "") +< lines = plain.splitlines() +< at = lines.index("# the shell prints what it is given") +< print("comment:", lines[at]) +< print("then:", lines[at + 1]) +comment: # the shell prints what it is given +then: ❯ printf 'one\n' +``` + +## proof-scene-layout#a-fence-comment-explains-the-step/comment-with-no-command + +```console +$ sh -c 'cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/dangling-comment" repo && rune proof run sample-change-one --source repo' +? 2 +fatal: [..]README.md: line 20: a comment `# text` comes before the command it explains +``` diff --git a/docs/proofs/proof-scene-layout/proof.cast b/docs/proofs/proof-scene-layout/proof.cast new file mode 100644 index 0000000..5d1b979 --- /dev/null +++ b/docs/proofs/proof-scene-layout/proof.cast @@ -0,0 +1,267 @@ +{"version":2,"width":100,"height":30,"timestamp":1792143216,"env":{"SHELL":"/bin/sh","TERM":"xterm-256color"}} +[0.0,"o","\r\n\u001b[1;36m── proof-scene-layout · 5 scenes ───────────────────────────────────────────────────────────────────\u001b[0m\r\n"] +[1.2,"o","\r\n\u001b[2m┌─────────────────────────────────────┐\u001b[0m\r\n\u001b[2m│\u001b[0m \u001b[1m1/5 · Scenes are paced for a reader\u001b[0m \u001b[2m│\u001b[0m\r\n\u001b[2m└─────────────────────────────────────┘\u001b[0m\r\n\u001b[35mcheck\u001b[0m \u001b[2mthe cast is paced for a reader\u001b[0m\r\n"] +[2.4,"o","\u001b[2;3m# three scenes whose commands finish within a millisecond, default pause\u001b[0m\r\n"] +[3.6,"o","\u001b[1;32m❯\u001b[0m "] +[3.6,"o","sh"] +[3.66,"o"," "] +[3.66,"o","-c"] +[3.72,"o"," "] +[3.72,"o","'cp"] +[3.78,"o"," "] +[3.78,"o","-Rf"] +[3.84,"o"," "] +[3.84,"o","\"$RUNE_PROOF_ROOT/tests/fixtures/proof/paced\""] +[3.9,"o"," "] +[3.9,"o","repo"] +[3.96,"o"," "] +[3.96,"o","&&"] +[4.02,"o"," "] +[4.02,"o","rune"] +[4.08,"o"," "] +[4.08,"o","proof"] +[4.14,"o"," "] +[4.14,"o","run"] +[4.2,"o"," "] +[4.2,"o","sample-change-one"] +[4.26,"o"," "] +[4.26,"o","--source"] +[4.32,"o"," "] +[4.32,"o","repo"] +[4.38,"o"," "] +[4.38,"o",">/dev/null'"] +[4.44,"o","\r\n"] +[4.99,"o","\u001b[32m✓\u001b[0m \u001b[2mexit 0\u001b[0m\r\n"] +[5.39,"o","\u001b[1;32m❯\u001b[0m "] +[5.39,"o","python3"] +[5.45,"o"," "] +[5.45,"o","-"] +[5.51,"o","\r\n"] +[5.51,"o","\u001b[2m< import json\u001b[0m\r\n"] +[5.57,"o","\u001b[2m< events = [json.loads(l) for l in open(\"repo/docs/proofs/sample-change-one/proof.cast\").readlines()[1:]]\u001b[0m\r\n"] +[5.63,"o","\u001b[2m< boxes = [t for t, _, d in events if \"\\u250c\" in d]\u001b[0m\r\n"] +[5.69,"o","\u001b[2m< ticks = [t for t, _, d in events if \"\\u2713\" in d]\u001b[0m\r\n"] +[5.75,"o","\u001b[2m< print(\"last event past nine seconds:\", events[-1][0] > 9)\u001b[0m\r\n"] +[5.81,"o","\u001b[2m< print(\"boxes:\", len(boxes), \"ticks:\", len(ticks))\u001b[0m\r\n"] +[5.87,"o","\u001b[2m< print(\"seconds shared by two scenes:\", len(boxes) - len({int(t) for t in boxes}))\u001b[0m\r\n"] +[6.08,"o","last event past nine seconds: True\r\nboxes: 3 ticks: 3\r\nseconds shared by two scenes: 0\r\n"] +[6.48,"o","\u001b[32m✓\u001b[0m \u001b[2mexit 0 · output matches\u001b[0m\r\n"] +[9.88,"o","\r\n\u001b[2m┌────────────────────────────────┐\u001b[0m\r\n\u001b[2m│\u001b[0m \u001b[1m2/5 · Owner shortens the pause\u001b[0m \u001b[2m│\u001b[0m\r\n\u001b[2m└────────────────────────────────┘\u001b[0m\r\n\u001b[35mcheck\u001b[0m \u001b[2mthe cast is paced for a reader\u001b[0m\r\n"] +[11.08,"o","\u001b[2;3m# the same proof at --pause 0.5 and at the default; the box gap shrinks by 2.5 s\u001b[0m\r\n"] +[12.28,"o","\u001b[1;32m❯\u001b[0m "] +[12.28,"o","sh"] +[12.34,"o"," "] +[12.34,"o","-c"] +[12.4,"o"," "] +[12.4,"o","'cp"] +[12.46,"o"," "] +[12.46,"o","-Rf"] +[12.52,"o"," "] +[12.52,"o","\"$RUNE_PROOF_ROOT/tests/fixtures/proof/paced\""] +[12.58,"o"," "] +[12.58,"o","slow"] +[12.64,"o"," "] +[12.64,"o","&&"] +[12.7,"o"," "] +[12.7,"o","cp"] +[12.76,"o"," "] +[12.76,"o","-Rf"] +[12.82,"o"," "] +[12.82,"o","\"$RUNE_PROOF_ROOT/tests/fixtures/proof/paced\""] +[12.88,"o"," "] +[12.88,"o","fast"] +[12.94,"o"," "] +[12.94,"o","&&"] +[13.0,"o"," "] +[13.0,"o","rune"] +[13.06,"o"," "] +[13.06,"o","proof"] +[13.12,"o"," "] +[13.12,"o","run"] +[13.18,"o"," "] +[13.18,"o","sample-change-one"] +[13.24,"o"," "] +[13.24,"o","--source"] +[13.3,"o"," "] +[13.3,"o","slow"] +[13.36,"o"," "] +[13.36,"o",">/dev/null"] +[13.42,"o"," "] +[13.42,"o","&&"] +[13.48,"o"," "] +[13.48,"o","rune"] +[13.54,"o"," "] +[13.54,"o","proof"] +[13.6,"o"," "] +[13.6,"o","run"] +[13.66,"o"," "] +[13.66,"o","sample-change-one"] +[13.72,"o"," "] +[13.72,"o","--pause"] +[13.78,"o"," "] +[13.78,"o","0.5"] +[13.84,"o"," "] +[13.84,"o","--source"] +[13.9,"o"," "] +[13.9,"o","fast"] +[13.96,"o"," "] +[13.96,"o",">/dev/null'"] +[14.02,"o","\r\n"] +[14.57,"o","\u001b[32m✓\u001b[0m \u001b[2mexit 0\u001b[0m\r\n"] +[14.97,"o","\u001b[1;32m❯\u001b[0m "] +[14.97,"o","python3"] +[15.03,"o"," "] +[15.03,"o","-"] +[15.09,"o","\r\n"] +[15.09,"o","\u001b[2m< import json\u001b[0m\r\n"] +[15.15,"o","\u001b[2m< def gap(path):\u001b[0m\r\n"] +[15.21,"o","\u001b[2m< events = [json.loads(l) for l in open(path).readlines()[1:]]\u001b[0m\r\n"] +[15.27,"o","\u001b[2m< boxes = [t for t, _, d in events if \"\\u250c\" in d]\u001b[0m\r\n"] +[15.33,"o","\u001b[2m< return boxes[1] - boxes[0]\u001b[0m\r\n"] +[15.39,"o","\u001b[2m< slow = gap(\"slow/docs/proofs/sample-change-one/proof.cast\")\u001b[0m\r\n"] +[15.45,"o","\u001b[2m< fast = gap(\"fast/docs/proofs/sample-change-one/proof.cast\")\u001b[0m\r\n"] +[15.51,"o","\u001b[2m< print(\"gap difference:\", round(slow - fast, 2))\u001b[0m\r\n"] +[15.57,"o","\u001b[2m< print(\"fast gap under three seconds:\", fast < 3)\u001b[0m\r\n"] +[15.78,"o","gap difference: 2.5\r\nfast gap under three seconds: True\r\n"] +[16.18,"o","\u001b[32m✓\u001b[0m \u001b[2mexit 0 · output matches\u001b[0m\r\n"] +[19.58,"o","\r\n\u001b[2m┌────────────────────────────────────────────────┐\u001b[0m\r\n\u001b[2m│\u001b[0m \u001b[1m3/5 · Failed step shows its reason in the cast\u001b[0m \u001b[2m│\u001b[0m\r\n\u001b[2m└────────────────────────────────────────────────┘\u001b[0m\r\n\u001b[35mcheck\u001b[0m \u001b[2mthe cast is paced for a reader\u001b[0m\r\n"] +[20.78,"o","\u001b[2;3m# the first scene expects rune 0.5.0 and the run is newer\u001b[0m\r\n"] +[21.98,"o","\u001b[1;32m❯\u001b[0m "] +[21.98,"o","sh"] +[22.04,"o"," "] +[22.04,"o","-c"] +[22.1,"o"," "] +[22.1,"o","'cp"] +[22.16,"o"," "] +[22.16,"o","-Rf"] +[22.22,"o"," "] +[22.22,"o","\"$RUNE_PROOF_ROOT/tests/fixtures/proof/mismatch\""] +[22.28,"o"," "] +[22.28,"o","repo"] +[22.34,"o"," "] +[22.34,"o","&&"] +[22.4,"o"," "] +[22.4,"o","rune"] +[22.46,"o"," "] +[22.46,"o","proof"] +[22.52,"o"," "] +[22.52,"o","run"] +[22.58,"o"," "] +[22.58,"o","sample-change-one"] +[22.64,"o"," "] +[22.64,"o","--source"] +[22.7,"o"," "] +[22.7,"o","repo'"] +[22.76,"o","\r\n"] +[22.91,"o","sample-capability-one#thing-holds/version-prints ... unproven: output of `rune --version` differs:\r\n- rune 0.5.0 ([..])\r\n+ rune 0.6.0 (275680d7) built 2026-10-16T09:26:52Z\r\nsample-capability-one#thing-holds/missing-tag-fails ... ok (check)\r\n2 scenes, 1 proven, 1 unproven; transcript 0b12cfe5a004\r\n"] +[23.31,"o","\u001b[32m✓\u001b[0m \u001b[2mexit 1 as expected · output matches\u001b[0m\r\n"] +[23.71,"o","\u001b[1;32m❯\u001b[0m "] +[23.71,"o","python3"] +[23.77,"o"," "] +[23.77,"o","-"] +[23.83,"o","\r\n"] +[23.83,"o","\u001b[2m< import json, re\u001b[0m\r\n"] +[23.89,"o","\u001b[2m< text = \"\".join(json.loads(l)[2] for l in open(\"repo/docs/proofs/sample-change-one/proof.cast\").readlines()[1:])\u001b[0m\r\n"] +[23.95,"o","\u001b[2m< plain = re.sub(r\"\\x1b\\[[0-9;]*m\", \"\", text).replace(\"\\r\", \"\")\u001b[0m\r\n"] +[24.01,"o","\u001b[2m< start = plain.index(\"rune --version\")\u001b[0m\r\n"] +[24.07,"o","\u001b[2m< print(plain[start:start + 40].splitlines()[0])\u001b[0m\r\n"] +[24.13,"o","\u001b[2m< print([l for l in plain.splitlines() if l.startswith(\"\\u2717\")][0])\u001b[0m\r\n"] +[24.19,"o","\u001b[2m< print(plain.splitlines()[-1])\u001b[0m\r\n"] +[24.4,"o","rune --version\r\n✗ unproven: output of `rune --version` differs:\r\n2 scenes · 1 proven · 1 unproven\r\n"] +[24.8,"o","\u001b[32m✓\u001b[0m \u001b[2mexit 0 · output matches\u001b[0m\r\n"] +[28.2,"o","\r\n\u001b[2m┌────────────────────────────────┐\u001b[0m\r\n\u001b[2m│\u001b[0m \u001b[1m4/5 · Comment before a command\u001b[0m \u001b[2m│\u001b[0m\r\n\u001b[2m└────────────────────────────────┘\u001b[0m\r\n\u001b[35mcheck\u001b[0m \u001b[2ma fence comment explains the step\u001b[0m\r\n"] +[29.4,"o","\u001b[2;3m# the third scene of the fixture carries a comment above its command\u001b[0m\r\n"] +[30.6,"o","\u001b[1;32m❯\u001b[0m "] +[30.6,"o","sh"] +[30.66,"o"," "] +[30.66,"o","-c"] +[30.72,"o"," "] +[30.72,"o","'cp"] +[30.78,"o"," "] +[30.78,"o","-Rf"] +[30.84,"o"," "] +[30.84,"o","\"$RUNE_PROOF_ROOT/tests/fixtures/proof/paced\""] +[30.9,"o"," "] +[30.9,"o","repo"] +[30.96,"o"," "] +[30.96,"o","&&"] +[31.02,"o"," "] +[31.02,"o","rune"] +[31.08,"o"," "] +[31.08,"o","proof"] +[31.14,"o"," "] +[31.14,"o","run"] +[31.2,"o"," "] +[31.2,"o","sample-change-one"] +[31.26,"o"," "] +[31.26,"o","--source"] +[31.32,"o"," "] +[31.32,"o","repo"] +[31.38,"o"," "] +[31.38,"o",">/dev/null'"] +[31.44,"o","\r\n"] +[31.99,"o","\u001b[32m✓\u001b[0m \u001b[2mexit 0\u001b[0m\r\n"] +[32.39,"o","\u001b[1;32m❯\u001b[0m "] +[32.39,"o","grep"] +[32.45,"o"," "] +[32.45,"o","-n"] +[32.51,"o"," "] +[32.51,"o","-B1"] +[32.57,"o"," "] +[32.57,"o","-A1"] +[32.63,"o"," "] +[32.63,"o","'^#"] +[32.69,"o"," "] +[32.69,"o","the"] +[32.75,"o"," "] +[32.75,"o","shell'"] +[32.81,"o"," "] +[32.81,"o","repo/docs/proofs/sample-change-one/proof.txt"] +[32.87,"o","\r\n"] +[33.02,"o","17-kind: check\r\n18:# the shell prints what it is given\r\n19-$ printf 'one\\n'\r\n"] +[33.42,"o","\u001b[32m✓\u001b[0m \u001b[2mexit 0 · output matches\u001b[0m\r\n"] +[33.82,"o","\u001b[1;32m❯\u001b[0m "] +[33.82,"o","python3"] +[33.88,"o"," "] +[33.88,"o","-"] +[33.94,"o","\r\n"] +[33.94,"o","\u001b[2m< import json, re\u001b[0m\r\n"] +[34.0,"o","\u001b[2m< text = \"\".join(json.loads(l)[2] for l in open(\"repo/docs/proofs/sample-change-one/proof.cast\").readlines()[1:])\u001b[0m\r\n"] +[34.06,"o","\u001b[2m< plain = re.sub(r\"\\x1b\\[[0-9;]*m\", \"\", text).replace(\"\\r\", \"\")\u001b[0m\r\n"] +[34.12,"o","\u001b[2m< lines = plain.splitlines()\u001b[0m\r\n"] +[34.18,"o","\u001b[2m< at = lines.index(\"# the shell prints what it is given\")\u001b[0m\r\n"] +[34.24,"o","\u001b[2m< print(\"comment:\", lines[at])\u001b[0m\r\n"] +[34.3,"o","\u001b[2m< print(\"then:\", lines[at + 1])\u001b[0m\r\n"] +[34.51,"o","comment: # the shell prints what it is given\r\nthen: ❯ printf 'one\\n'\r\n"] +[34.91,"o","\u001b[32m✓\u001b[0m \u001b[2mexit 0 · output matches\u001b[0m\r\n"] +[38.31,"o","\r\n\u001b[2m┌───────────────────────────────┐\u001b[0m\r\n\u001b[2m│\u001b[0m \u001b[1m5/5 · Comment with no command\u001b[0m \u001b[2m│\u001b[0m\r\n\u001b[2m└───────────────────────────────┘\u001b[0m\r\n\u001b[35mcheck\u001b[0m \u001b[2ma fence comment explains the step\u001b[0m\r\n"] +[39.51,"o","\u001b[1;32m❯\u001b[0m "] +[39.51,"o","sh"] +[39.57,"o"," "] +[39.57,"o","-c"] +[39.63,"o"," "] +[39.63,"o","'cp"] +[39.69,"o"," "] +[39.69,"o","-Rf"] +[39.75,"o"," "] +[39.75,"o","\"$RUNE_PROOF_ROOT/tests/fixtures/proof/dangling-comment\""] +[39.81,"o"," "] +[39.81,"o","repo"] +[39.87,"o"," "] +[39.87,"o","&&"] +[39.93,"o"," "] +[39.93,"o","rune"] +[39.99,"o"," "] +[39.99,"o","proof"] +[40.05,"o"," "] +[40.05,"o","run"] +[40.11,"o"," "] +[40.11,"o","sample-change-one"] +[40.17,"o"," "] +[40.17,"o","--source"] +[40.23,"o"," "] +[40.23,"o","repo'"] +[40.29,"o","\r\n"] +[40.44,"o","fatal: /private/tmp/claude-501/rune-proof-10465-proof-scene-layout-a-fence-comment-explains-the-step-comment-with-no-command-17544270-18-0/repo/docs/proofs/sample-change-one/README.md: line 20: a comment `# text` comes before the command it explains\r\n"] +[40.84,"o","\u001b[32m✓\u001b[0m \u001b[2mexit 2 as expected · output matches\u001b[0m\r\n"] +[44.24,"o","\r\n\u001b[1;32m5 scenes · 5 proven · 0 unproven\u001b[0m\r\n"] diff --git a/docs/proofs/proof-scene-layout/proof.gif b/docs/proofs/proof-scene-layout/proof.gif new file mode 100644 index 0000000..215cdbf Binary files /dev/null and b/docs/proofs/proof-scene-layout/proof.gif differ diff --git a/docs/proofs/proof-scene-layout/proof.txt b/docs/proofs/proof-scene-layout/proof.txt new file mode 100644 index 0000000..7857554 --- /dev/null +++ b/docs/proofs/proof-scene-layout/proof.txt @@ -0,0 +1,78 @@ +## proof-scene-layout#the-cast-is-paced-for-a-reader/scenes-are-paced-for-a-reader +kind: check +# three scenes whose commands finish within a millisecond, default pause +$ sh -c 'cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/paced" repo && rune proof run sample-change-one --source repo >/dev/null' +$ python3 - +< import json +< events = [json.loads(l) for l in open("repo/docs/proofs/sample-change-one/proof.cast").readlines()[1:]] +< boxes = [t for t, _, d in events if "\u250c" in d] +< ticks = [t for t, _, d in events if "\u2713" in d] +< print("last event past nine seconds:", events[-1][0] > 9) +< print("boxes:", len(boxes), "ticks:", len(ticks)) +< print("seconds shared by two scenes:", len(boxes) - len({int(t) for t in boxes})) + last event past nine seconds: True + boxes: 3 ticks: 3 + seconds shared by two scenes: 0 + +## proof-scene-layout#the-cast-is-paced-for-a-reader/owner-shortens-the-pause +kind: check +# the same proof at --pause 0.5 and at the default; the box gap shrinks by 2.5 s +$ sh -c 'cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/paced" slow && cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/paced" fast && rune proof run sample-change-one --source slow >/dev/null && rune proof run sample-change-one --pause 0.5 --source fast >/dev/null' +$ python3 - +< import json +< def gap(path): +< events = [json.loads(l) for l in open(path).readlines()[1:]] +< boxes = [t for t, _, d in events if "\u250c" in d] +< return boxes[1] - boxes[0] +< slow = gap("slow/docs/proofs/sample-change-one/proof.cast") +< fast = gap("fast/docs/proofs/sample-change-one/proof.cast") +< print("gap difference:", round(slow - fast, 2)) +< print("fast gap under three seconds:", fast < 3) + gap difference: 2.5 + fast gap under three seconds: True + +## proof-scene-layout#the-cast-is-paced-for-a-reader/failed-step-shows-its-reason-in-the-cast +kind: check +# the first scene expects rune 0.5.0 and the run is newer +$ sh -c 'cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/mismatch" repo && rune proof run sample-change-one --source repo' + sample-capability-one#thing-holds/version-prints ... unproven: output of `rune --version` differs: + - rune 0.5.0 ([..]) + + rune 0.6.0 (275680d7) built 2026-10-16T09:26:52Z + sample-capability-one#thing-holds/missing-tag-fails ... ok (check) + 2 scenes, 1 proven, 1 unproven; transcript 0b12cfe5a004 +$ python3 - +< import json, re +< text = "".join(json.loads(l)[2] for l in open("repo/docs/proofs/sample-change-one/proof.cast").readlines()[1:]) +< plain = re.sub(r"\x1b\[[0-9;]*m", "", text).replace("\r", "") +< start = plain.index("rune --version") +< print(plain[start:start + 40].splitlines()[0]) +< print([l for l in plain.splitlines() if l.startswith("\u2717")][0]) +< print(plain.splitlines()[-1]) + rune --version + ✗ unproven: output of `rune --version` differs: + 2 scenes · 1 proven · 1 unproven + +## proof-scene-layout#a-fence-comment-explains-the-step/comment-before-a-command +kind: check +# the third scene of the fixture carries a comment above its command +$ sh -c 'cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/paced" repo && rune proof run sample-change-one --source repo >/dev/null' +$ grep -n -B1 -A1 '^# the shell' repo/docs/proofs/sample-change-one/proof.txt + 17-kind: check + 18:# the shell prints what it is given + 19-$ printf 'one\n' +$ python3 - +< import json, re +< text = "".join(json.loads(l)[2] for l in open("repo/docs/proofs/sample-change-one/proof.cast").readlines()[1:]) +< plain = re.sub(r"\x1b\[[0-9;]*m", "", text).replace("\r", "") +< lines = plain.splitlines() +< at = lines.index("# the shell prints what it is given") +< print("comment:", lines[at]) +< print("then:", lines[at + 1]) + comment: # the shell prints what it is given + then: ❯ printf 'one\n' + +## proof-scene-layout#a-fence-comment-explains-the-step/comment-with-no-command +kind: check +$ sh -c 'cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/dangling-comment" repo && rune proof run sample-change-one --source repo' + fatal: /private/tmp/claude-501/rune-proof-10465-proof-scene-layout-a-fence-comment-explains-the-step-comment-with-no-command-17544270-18-0/repo/docs/proofs/sample-change-one/README.md: line 20: a comment `# text` comes before the command it explains + diff --git a/docs/proofs/review-and-proof/README.md b/docs/proofs/review-and-proof/README.md index 797644b..b2108c2 100644 --- a/docs/proofs/review-and-proof/README.md +++ b/docs/proofs/review-and-proof/README.md @@ -166,7 +166,7 @@ $ sh -c 'grep -c "^< " "$RUNE_PROOF_ROOT/docs/proofs/review-and-proof/README.md" ```console $ sh -c 'cp -Rf "$RUNE_PROOF_ROOT/tests/fixtures/proof/input-first" repo && rune proof run sample-change-one --source repo' ? 2 -fatal: [..]README.md: line 14: an input line `< text` comes after the command it feeds +fatal: [..]README.md: line 25: an input line `< text` comes after the command it feeds ``` ## proof-scene-runner#every-scene-starts-clean/scenes-do-not-share-a-directory diff --git a/src/cli/proof/cast.rs b/src/cli/proof/cast.rs new file mode 100644 index 0000000..2b5fb99 --- /dev/null +++ b/src/cli/proof/cast.rs @@ -0,0 +1,361 @@ +//! The cast a person watches: an asciinema v2 stream on a presentation +//! timeline. The clock that ran the scenes plays no part; each element +//! advances the playhead by a fixed amount so a proof of fast commands +//! still reads scene by scene. The layout is the one the deck's driver +//! prints: a rule per capability, a boxed `N/T · title` and a kind line +//! per scene, dim comments, a typed command behind a prompt, its input +//! lines, the output, one tick or cross per step, and a closing tally. + +use std::fmt::Write as _; +use std::fs; +use std::path::Path; + +use rune::error::Error; + +pub const WIDTH: usize = 100; +const HEIGHT: usize = 30; + +const RESET: &str = "\x1b[0m"; +const PROMPT: &str = "\x1b[1;32m"; +const COMMENT: &str = "\x1b[2;3m"; +const CAPABILITY: &str = "\x1b[1;36m"; +const BOX: &str = "\x1b[2m"; +const TITLE: &str = "\x1b[1m"; +const KIND: &str = "\x1b[35m"; +const DETAIL: &str = "\x1b[2m"; +const PASS: &str = "\x1b[32m"; +const FAIL: &str = "\x1b[1;31m"; +const NOTE: &str = "\x1b[1;36m"; + +/// Seconds the playhead rests after a header, a box, or a comment. +const AFTER_HEADER: f64 = 1.2; +/// Seconds between typed words. +const PER_WORD: f64 = 0.06; +/// Seconds between a command and its output. +const BEFORE_OUTPUT: f64 = 0.15; +/// Seconds an output stays before its tick. +const BEFORE_TICK: f64 = 0.4; + +/// The events so far and where the playhead is. +pub struct Cast { + events: Vec<(f64, String)>, + at: f64, + /// Seconds after each scene, `--pause`. + pause: f64, +} + +impl Cast { + #[must_use] + pub fn new(pause: f64) -> Self { + Self { + events: Vec::new(), + at: 0.0, + pause: pause.max(0.0), + } + } + + /// The offset of the last event, in seconds. + #[cfg(test)] + pub fn length(&self) -> f64 { + self.events.last().map_or(0.0, |e| e.0) + } + + #[cfg(test)] + pub fn events(&self) -> &[(f64, String)] { + &self.events + } + + fn emit(&mut self, text: impl Into) { + self.events.push((self.at, text.into())); + } + + fn wait(&mut self, seconds: f64) { + self.at += seconds; + } + + /// A rule that opens the scenes of one capability. + pub fn capability(&mut self, name: &str, scenes: usize) { + let text = format!( + " {name} · {scenes} scene{} ", + if scenes == 1 { "" } else { "s" } + ); + let rule = "\u{2500}".repeat(WIDTH.saturating_sub(2 + text.chars().count())); + self.emit(format!( + "\n{CAPABILITY}\u{2500}\u{2500}{text}{rule}{RESET}\n" + )); + self.wait(AFTER_HEADER); + } + + /// The box that opens a scene and the kind line under it. + pub fn scene(&mut self, index: usize, total: usize, key: &str, kind: &str) { + let (requirement, title) = words(key); + let heading = format!("{index}/{total} \u{b7} {title}"); + let rule = "\u{2500}".repeat(heading.chars().count() + 2); + let mut text = String::new(); + let _ = writeln!(text, "\n{BOX}\u{250c}{rule}\u{2510}{RESET}"); + let _ = writeln!( + text, + "{BOX}\u{2502}{RESET} {TITLE}{heading}{RESET} {BOX}\u{2502}{RESET}" + ); + let _ = writeln!(text, "{BOX}\u{2514}{rule}\u{2518}{RESET}"); + let _ = writeln!(text, "{KIND}{kind}{RESET} {DETAIL}{requirement}{RESET}"); + self.emit(text); + self.wait(AFTER_HEADER); + } + + /// A fence comment, dim and italic. + pub fn comment(&mut self, text: &str) { + self.emit(format!("{COMMENT}# {text}{RESET}\n")); + self.wait(AFTER_HEADER); + } + + /// A command typed word by word behind a prompt. `lines` are the + /// fence's `$ `, `> `, and `< ` lines as written; continuation lines + /// are typed on their own line and input lines appear whole, dim. + pub fn command(&mut self, lines: &[String]) { + self.emit(format!("{PROMPT}\u{276f}{RESET} ")); + for line in lines { + if let Some(input) = + line.strip_prefix("< ") + .or(if line == "<" { Some("") } else { None }) + { + self.emit(format!("{DETAIL}< {input}{RESET}\n")); + self.wait(PER_WORD); + continue; + } + let text = line + .strip_prefix("$ ") + .or_else(|| line.strip_prefix("> ").map(|_| line.as_str())) + .unwrap_or(line); + if line.starts_with("> ") { + self.emit(format!("{DETAIL}>{RESET} ")); + } + let shown = text.strip_prefix("> ").unwrap_or(text); + let mut first = true; + for word in shown.split(' ') { + if !first { + self.emit(" "); + } + first = false; + self.emit(word.to_string()); + self.wait(PER_WORD); + } + self.emit("\n"); + } + self.wait(BEFORE_OUTPUT); + } + + /// What the command printed, as captured. + pub fn output(&mut self, text: &str) { + if !text.is_empty() { + let mut shown = text.to_string(); + if !shown.ends_with('\n') { + shown.push('\n'); + } + self.emit(shown); + } + self.wait(BEFORE_TICK); + } + + /// The step matched: exit status and output as the fence expects. + pub fn tick(&mut self, label: &str) { + self.emit(format!("{PASS}\u{2713}{RESET} {DETAIL}{label}{RESET}\n")); + self.wait(BEFORE_TICK); + } + + /// The step failed, so the scene is unproven. + pub fn cross(&mut self, reason: &str) { + let first = reason.lines().next().unwrap_or_default(); + self.emit(format!("{FAIL}\u{2717} unproven: {first}{RESET}\n")); + self.wait(BEFORE_TICK); + } + + /// A line about the scene that is neither a tick nor a cross, such as + /// a recorded instruction. + pub fn note(&mut self, text: &str) { + self.emit(format!("{NOTE}\u{25cf} {text}{RESET}\n")); + self.wait(BEFORE_TICK); + } + + /// The pause that closes a scene. + pub fn end_scene(&mut self) { + self.wait(self.pause); + } + + /// The tally that closes the cast. + pub fn tally(&mut self, total: usize, proven: usize) { + let unproven = total.saturating_sub(proven); + let colour = if unproven == 0 { PROMPT } else { FAIL }; + self.emit(format!( + "\n{colour}{total} scene{} \u{b7} {proven} proven \u{b7} {unproven} unproven{RESET}\n", + if total == 1 { "" } else { "s" } + )); + self.wait(AFTER_HEADER); + } + + /// Write the stream. The header is written by hand so `version` + /// leads, as asciinema writes it. + pub fn write(&self, path: &Path) -> Result<(), Error> { + let mut cast = String::new(); + let _ = writeln!( + cast, + "{{\"version\":2,\"width\":{WIDTH},\"height\":{HEIGHT},\"timestamp\":{},\"env\":{{\"SHELL\":\"/bin/sh\",\"TERM\":\"xterm-256color\"}}}}", + chrono::Utc::now().timestamp() + ); + for (offset, text) in &self.events { + let event = serde_json::json!([ + (offset * 1000.0).round() / 1000.0, + "o", + text.replace('\n', "\r\n") + ]); + let _ = writeln!(cast, "{event}"); + } + fs::write(path, cast).map_err(|error| Error::io(error.to_string())) + } +} + +/// The requirement and scenario slugs of a key as words, the scenario +/// with its first letter raised. +fn words(key: &str) -> (String, String) { + let rest = key.split_once('#').map_or(key, |(_, rest)| rest); + let (requirement, scenario) = rest.split_once('/').unwrap_or(("", rest)); + let mut title = scenario.replace('-', " "); + if let Some(first) = title.get(..1) { + title = first.to_uppercase() + &title[1..]; + } + (requirement.replace('-', " "), title) +} + +/// The label of a passing step. +#[must_use] +pub fn tick_label(exit: Option, expected_output: bool) -> String { + let exit = match exit { + Some(0) => "exit 0".to_string(), + Some(code) => format!("exit {code} as expected"), + None => "ended by signal as expected".to_string(), + }; + if expected_output { + format!("{exit} \u{b7} output matches") + } else { + exit + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn plain(cast: &Cast) -> String { + let mut text = String::new(); + for (_, chunk) in cast.events() { + text.push_str(chunk); + } + let mut out = String::new(); + let mut chars = text.chars().peekable(); + while let Some(c) = chars.next() { + if c == '\x1b' { + for next in chars.by_ref() { + if next == 'm' { + break; + } + } + } else { + out.push(c); + } + } + out + } + + fn three_scenes(pause: f64) -> Cast { + let mut cast = Cast::new(pause); + cast.capability("cap", 3); + for i in 1..=3 { + cast.scene(i, 3, "cap#req-name/scene-name", "check"); + cast.command(&["$ true".to_string()]); + cast.output(""); + cast.tick("exit 0"); + cast.end_scene(); + } + cast.tally(3, 3); + cast + } + + #[test] + fn three_fast_scenes_are_paced_past_nine_seconds() { + let cast = three_scenes(3.0); + assert!(cast.length() > 9.0, "{}", cast.length()); + let boxes: Vec = cast + .events() + .iter() + .filter(|(_, t)| t.contains('\u{250c}')) + .map(|(at, _)| *at) + .collect(); + assert_eq!(boxes.len(), 3); + assert!(boxes[1] - boxes[0] > 3.0); + assert!( + (boxes[1] - boxes[0]).abs() >= 1.0, + "two scenes in one second" + ); + } + + #[test] + fn a_short_pause_spaces_the_boxes_by_about_half_a_second_plus_the_scene() { + let long = three_scenes(3.0); + let short = three_scenes(0.5); + let gap = |cast: &Cast| { + let boxes: Vec = cast + .events() + .iter() + .filter(|(_, t)| t.contains('\u{250c}')) + .map(|(at, _)| *at) + .collect(); + boxes[1] - boxes[0] + }; + assert!((gap(&long) - gap(&short) - 2.5).abs() < 0.01); + } + + #[test] + fn the_layout_carries_box_kind_comment_prompt_tick_and_tally() { + let mut cast = Cast::new(1.0); + cast.capability("harness-hook-dispatch", 1); + cast.scene( + 7, + 23, + "harness-hook-dispatch#the-dispatcher-normalizes-and-answers/denial-encoded-for-claude", + "check", + ); + cast.comment("the guard refuses rm -rf"); + cast.command(&[ + "$ rune hook run --harness claude".to_string(), + "> --native-event PreToolUse".to_string(), + "< {\"tool\":\"Bash\"}".to_string(), + ]); + cast.output("{\"deny\":true}\n"); + cast.tick("exit 0 · output matches"); + cast.end_scene(); + cast.tally(23, 22); + let text = plain(&cast); + assert!(text.contains("── harness-hook-dispatch · 1 scene ─")); + assert!(text.contains("│ 7/23 · Denial encoded for claude │")); + assert!(text.contains("check the dispatcher normalizes and answers")); + assert!(text.contains("# the guard refuses rm -rf\n❯ rune hook run --harness claude\n> --native-event PreToolUse\n< {\"tool\":\"Bash\"}\n{\"deny\":true}\n✓ exit 0 · output matches")); + assert!(text.ends_with("23 scenes · 22 proven · 1 unproven\n")); + } + + #[test] + fn a_cross_carries_the_first_line_of_the_reason() { + let mut cast = Cast::new(0.0); + cast.cross("output of `ls` differs:\n- a\n+ b"); + assert_eq!(plain(&cast), "✗ unproven: output of `ls` differs:\n"); + } + + #[test] + fn the_stream_is_asciicast_v2() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("proof.cast"); + three_scenes(3.0).write(&path).unwrap(); + let text = fs::read_to_string(&path).unwrap(); + assert!(text.starts_with("{\"version\":2,\"width\":100,\"height\":30,")); + assert!(text.lines().nth(1).unwrap().starts_with("[0.0,\"o\",")); + } +} diff --git a/src/cli/proof/fence.rs b/src/cli/proof/fence.rs index fb20000..b9127f6 100644 --- a/src/cli/proof/fence.rs +++ b/src/cli/proof/fence.rs @@ -48,8 +48,10 @@ pub struct Step { pub stdin: Option, /// Expected output with `[..]` and `...` elisions, without a trailing newline. pub expected: String, - /// The `$ `, `> `, and `< ` lines as written, for the transcript. + /// The `# `, `$ `, `> `, and `< ` lines as written, for the transcript. pub command_lines: Vec, + /// The `# text` comments above the command, for the cast. + pub comments: Vec, } /// A scene: the scenario key of its heading and its steps. @@ -153,11 +155,28 @@ pub fn scenes(body: &str) -> Result, FenceError> { fn fence(lines: &[&str], start: usize) -> Result<(Vec, usize), FenceError> { let mut steps = Vec::new(); let mut i = start; + // Comments gathered for the next command, with the line of the first. + let mut comments: Vec = Vec::new(); + let mut comment_line = 0; while i < lines.len() { let line = lines[i]; if line.trim_start().starts_with("```") { + if !comments.is_empty() { + return Err(error( + comment_line, + "a comment `# text` comes before the command it explains", + )); + } return Ok((steps, i + 1)); } + if let Some(text) = line.strip_prefix("# ") { + if comments.is_empty() { + comment_line = i + 1; + } + comments.push(text.trim_end().to_string()); + i += 1; + continue; + } let Some(raw) = line.strip_prefix("$ ") else { if line.trim().is_empty() { i += 1; @@ -172,7 +191,10 @@ fn fence(lines: &[&str], start: usize) -> Result<(Vec, usize), FenceError> return Err(error(i + 1, format!("expected `$ command`, got `{line}`"))); }; let command_start = i + 1; - let mut command_lines = vec![line.to_string()]; + let mut command_lines: Vec = + comments.iter().map(|text| format!("# {text}")).collect(); + command_lines.push(line.to_string()); + let comments_here = std::mem::take(&mut comments); let mut command_text = raw.trim().to_string(); i += 1; while i < lines.len() { @@ -185,21 +207,7 @@ fn fence(lines: &[&str], start: usize) -> Result<(Vec, usize), FenceError> i += 1; } let words = shell_words(&command_text).map_err(|message| error(command_start, message))?; - let mut stdin: Option = None; - while i < lines.len() { - let text = if lines[i] == "<" { - "" - } else if let Some(text) = lines[i].strip_prefix("< ") { - text - } else { - break; - }; - command_lines.push(lines[i].to_string()); - let input = stdin.get_or_insert_with(String::new); - input.push_str(text); - input.push('\n'); - i += 1; - } + let stdin = input_lines(lines, &mut i, &mut command_lines); let mut status = Status::Success; if i < lines.len() && let Some(raw) = lines[i].strip_prefix("? ") @@ -208,8 +216,11 @@ fn fence(lines: &[&str], start: usize) -> Result<(Vec, usize), FenceError> i += 1; } let mut expected = String::new(); + // A `# ` line ends the expected output: it is the next command's + // comment. An output line that starts so is written `[..]`. while i < lines.len() && !lines[i].starts_with("$ ") + && !lines[i].starts_with("# ") && !lines[i].trim_start().starts_with("```") { expected.push_str(lines[i]); @@ -219,19 +230,7 @@ fn fence(lines: &[&str], start: usize) -> Result<(Vec, usize), FenceError> if expected.ends_with('\n') { expected.pop(); } - let mut env = BTreeMap::new(); - let mut argv = Vec::new(); - for word in words { - if argv.is_empty() - && let Some((key, value)) = word.split_once('=') - && !key.is_empty() - && key.chars().all(|c| c.is_ascii_alphanumeric() || c == '_') - { - env.insert(key.to_string(), value.to_string()); - } else { - argv.push(word); - } - } + let (env, argv) = env_and_argv(words); if argv.is_empty() { return Err(error(command_start, "a step names no program")); } @@ -242,11 +241,52 @@ fn fence(lines: &[&str], start: usize) -> Result<(Vec, usize), FenceError> stdin, expected, command_lines, + comments: comments_here, }); } Err(error(lines.len(), "console fence is not closed")) } +/// The `< text` lines under a command, joined with newlines, and copied +/// to the command lines as written. `i` moves past them. +fn input_lines(lines: &[&str], i: &mut usize, command_lines: &mut Vec) -> Option { + let mut stdin: Option = None; + while *i < lines.len() { + let text = if lines[*i] == "<" { + "" + } else if let Some(text) = lines[*i].strip_prefix("< ") { + text + } else { + break; + }; + command_lines.push(lines[*i].to_string()); + let input = stdin.get_or_insert_with(String::new); + input.push_str(text); + input.push('\n'); + *i += 1; + } + stdin +} + +/// Leading `KEY=value` words are the step's environment; the rest is the +/// program and its arguments. +fn env_and_argv(words: Vec) -> (BTreeMap, Vec) { + let mut env = BTreeMap::new(); + let mut argv = Vec::new(); + for word in words { + if argv.is_empty() + && let Some((key, value)) = word.split_once('=') + && !key.is_empty() + && key.chars().all(|c| c.is_ascii_alphanumeric() || c == '_') + { + env.insert(key.to_string(), value.to_string()); + } else { + argv.push(word); + } + } + (env, argv) +} + /// Split a command line into words the way a POSIX shell does: whitespace /// separates, single quotes take everything literally, double quotes /// group and let a backslash escape only `$`, `` ` ``, `"`, and `\`, and a @@ -324,6 +364,36 @@ mod tests { const README: &str = "# Scenes\n\n## cap#req/one\n\n```console\n$ rune --version\nrune 0.6.0 ([..]) built [..]\n\n```\n\n## cap#req/two\n\n```console\n$ FOO=bar rune sign list\n? 2\nerror: the following required arguments were not provided:\n...\n$ printf 'a b' \"c\"\n> --flag\na b c\n```\n\n## cap#req/empty\n\n```console\n```\n\n## Not a key\n\ntext\n"; + #[test] + fn a_comment_belongs_to_the_command_below_it() { + let body = "## cap#req/one\n\n```console\n# the guard refuses the delete\n$ rune hook run\n< {\"a\":1}\n# next\n$ true\n```\n"; + let scenes = scenes(body).expect("parse"); + let steps = &scenes[0].steps; + assert_eq!( + steps[0].comments, + vec!["the guard refuses the delete".to_string()] + ); + assert_eq!( + steps[0].command_lines, + vec![ + "# the guard refuses the delete", + "$ rune hook run", + "< {\"a\":1}" + ] + ); + assert_eq!(steps[0].expected, ""); + assert_eq!(steps[1].comments, vec!["next".to_string()]); + assert_eq!(steps[1].command_lines, vec!["# next", "$ true"]); + } + + #[test] + fn a_comment_with_no_command_after_it_is_an_error() { + let body = "## cap#req/one\n\n```console\n$ true\n# dangling\n```\n"; + let error = scenes(body).unwrap_err(); + assert_eq!(error.line, 5); + assert!(error.message.contains("before the command it explains")); + } + #[test] fn scenes_and_steps_parse_in_order() { let scenes = scenes(README).expect("parse"); diff --git a/src/cli/proof/mod.rs b/src/cli/proof/mod.rs index 866c98d..2aefde5 100644 --- a/src/cli/proof/mod.rs +++ b/src/cli/proof/mod.rs @@ -5,7 +5,6 @@ use std::fmt::Write as _; use std::fs; use std::path::{Path, PathBuf}; use std::process::Command; -use std::time::Instant; use clap::Subcommand; use rune::error::Error; @@ -14,6 +13,7 @@ use rune::proof::{self, Frontmatter, Kind, Proof, Scene as SceneRecord}; use super::closure; +mod cast; mod fence; mod matcher; mod runner; @@ -38,6 +38,9 @@ pub enum ProofAction { /// Record one instruction scene through `rune run ` #[arg(long, value_name = "SCENARIO")] instruction: Option, + /// Seconds the cast rests after each scene + #[arg(long, default_value_t = 3.0, value_name = "SECONDS")] + pause: f64, /// Deck or rune source root. Defaults to the current directory #[arg(long, default_value = ".")] source: String, @@ -51,6 +54,7 @@ pub fn execute(action: &ProofAction) -> Result { change, check, instruction, + pause, source, } => { let root = canonical(source)?; @@ -61,7 +65,7 @@ pub fn execute(action: &ProofAction) -> Result { if let Some(key) = instruction { return record_instruction(&root, &proof, key); } - run(&root, &proof) + run(&root, &proof, *pause) } } } @@ -132,80 +136,95 @@ fn scaffold(root: &Path, change: &str) -> Result { Ok(0) } -fn run(root: &Path, proof: &Proof) -> Result { +fn run(root: &Path, proof: &Proof, pause: f64) -> Result { let text = fs::read_to_string(&proof.readme).map_err(|error| Error::io(error.to_string()))?; let (_, body) = split_frontmatter(&text) .ok_or_else(|| Error::io(format!("{} has no frontmatter", proof.readme.display())))?; - let scenes = fence::scenes(body) - .map_err(|error| Error::io(format!("{}: {error}", proof.readme.display())))?; + // A fence error names the line of the file, not of the body under + // the frontmatter. + let offset = text.lines().count() - body.lines().count(); + let scenes = fence::scenes(body).map_err(|error| { + Error::io(format!( + "{}: line {}: {}", + proof.readme.display(), + error.line + offset, + error.message + )) + })?; // The head is read before any scene runs, so a scene cannot move it. let (head, head_note) = head_commit(root); if let Some(note) = head_note { println!("{note}"); } let previous = proof.transcript().unwrap_or_default(); - let started = Instant::now(); let mut frontmatter = proof.frontmatter.clone(); let total = frontmatter.scenes.len(); let mut transcript = String::new(); - let mut cast: Vec<(f64, String)> = Vec::new(); + let mut cast = cast::Cast::new(pause); let mut failed = 0usize; + let mut capability = String::new(); for (index, record) in frontmatter.scenes.iter_mut().enumerate() { let key = &record.scenario; - cast.push(( - started.elapsed().as_secs_f64(), - scene_header(index + 1, total, key), - )); + open_capability(&mut cast, &proof.frontmatter, key, &mut capability); + // The kind the box shows is the one the run settles on below, so + // an unproven scene is announced as what it was declared to be. if record.kind == Kind::Instruction { + cast.scene(index + 1, total, key, record.kind.as_str()); // A recorded answer is kept; an unrecorded instruction gets no // section, so the graph shows it unproven until it is run. - let verdict = if let Some(section) = previous_section(&previous, key) { + if let Some(section) = previous_section(&previous, key) { transcript.push_str(§ion); println!("{key} ... instruction (recorded)"); - Verdict::Kept("instruction, recorded") + cast.note("instruction, recorded"); } else { println!("{key} ... instruction (no section yet; run --instruction)"); - Verdict::Unproven("instruction not yet recorded".to_string()) - }; - cast.push((started.elapsed().as_secs_f64(), verdict.line())); + cast.cross("instruction not yet recorded"); + } + cast.end_scene(); continue; } let Some(scene) = scenes.iter().find(|s| s.key == *key) else { record.kind = Kind::Unproven; println!("{key} ... unproven (no scene in the README)"); - cast.push(( - started.elapsed().as_secs_f64(), - Verdict::Unproven("no scene in the README".to_string()).line(), - )); + cast.scene(index + 1, total, key, record.kind.as_str()); + cast.cross("no scene in the README"); + cast.end_scene(); failed += 1; continue; }; - let outcome = runner::run_scene(scene, root, started); - cast.extend(outcome.chunks.iter().cloned()); + let outcome = runner::run_scene(scene, root); + let kind = if outcome.passed && record.kind == Kind::Unproven { + Kind::Check + } else if outcome.passed { + record.kind + } else { + Kind::Unproven + }; + cast.scene(index + 1, total, key, kind.as_str()); + show_steps(&mut cast, &outcome.steps); if outcome.passed { - if record.kind == Kind::Unproven { - record.kind = Kind::Check; - } + record.kind = kind; push_section(&mut transcript, key, record.kind, &outcome.section); println!("{key} ... ok ({})", record.kind.as_str()); - cast.push(( - started.elapsed().as_secs_f64(), - Verdict::Ok(record.kind.as_str()).line(), - )); } else { record.kind = Kind::Unproven; let reason = outcome.failure.unwrap_or_default(); println!("{key} ... unproven: {}", reason.trim_end()); - cast.push(( - started.elapsed().as_secs_f64(), - Verdict::Unproven(reason.lines().next().unwrap_or_default().to_string()).line(), - )); + if outcome.steps.last().is_none_or(|s| s.verdict.is_ok()) { + // The scene failed before any step ran. + cast.cross(&reason); + } failed += 1; } + cast.end_scene(); } + cast.tally( + total, + total - failed - unrecorded_instructions(&frontmatter, &transcript), + ); fs::write(proof.dir.join("proof.txt"), &transcript) .map_err(|error| Error::io(error.to_string()))?; - write_cast(&proof.dir.join("proof.cast"), &cast)?; + cast.write(&proof.dir.join("proof.cast"))?; frontmatter.transcript = sha256(transcript.as_bytes()); frontmatter.recorded = today(); frontmatter.head = head; @@ -224,38 +243,6 @@ fn run(root: &Path, proof: &Proof) -> Result { Ok(i32::from(failed > 0)) } -/// How a scene ended, as the cast shows it. -enum Verdict<'a> { - Ok(&'a str), - Kept(&'a str), - Unproven(String), -} - -impl Verdict<'_> { - fn line(&self) -> String { - match self { - Self::Ok(kind) => format!("\x1b[1;32m\u{2714} ok ({kind})\x1b[0m\n\n"), - Self::Kept(what) => format!("\x1b[1;36m\u{25cf} {what}\x1b[0m\n\n"), - Self::Unproven(reason) => format!("\x1b[1;31m\u{2718} unproven: {reason}\x1b[0m\n\n"), - } - } -} - -/// The scene header in the cast: a rule, then `# Scenario: ` with -/// its index and key, so a player lists the scenes and a reader sees where -/// one ends and the next begins. The title is the scenario slug as words. -fn scene_header(index: usize, total: usize, key: &str) -> String { - let slug = key.rsplit('/').next().unwrap_or(key); - let mut title = slug.replace('-', " "); - if let Some(first) = title.get(..1) { - title = first.to_uppercase() + &title[1..]; - } - let rule = "\u{2501}".repeat(100); - format!( - "\x1b[1;34m{rule}\x1b[0m\n\x1b[1m# Scenario: {title}\x1b[0m \x1b[2m({index}/{total}, {key})\x1b[0m\n" - ) -} - /// Verify a proof without running it: the transcript hashes to the /// record, every proven scene has a section whose kind and command lines /// match the README, and the head is an ancestor of the current commit. @@ -312,6 +299,7 @@ fn check(root: &Path, proof: &Proof) -> i32 { line.starts_with("$ ") || line.starts_with("> ") || line.starts_with("< ") + || line.starts_with("# ") || *line == "<" }) .collect(); @@ -484,24 +472,53 @@ fn write_readme(readme: &Path, frontmatter: &Frontmatter, body: &str) -> Result< .map_err(|error| Error::io(error.to_string())) } -/// An asciinema v2 cast of the captured output, one event per chunk. -fn write_cast(path: &Path, chunks: &[(f64, String)]) -> Result<(), Error> { - let mut cast = String::new(); - // Written by hand so `version` leads, as asciinema writes it. - let _ = writeln!( - cast, - "{{\"version\":2,\"width\":100,\"height\":30,\"timestamp\":{},\"env\":{{\"SHELL\":\"/bin/sh\",\"TERM\":\"xterm-256color\"}}}}", - chrono::Utc::now().timestamp() - ); - for (offset, text) in chunks { - let event = serde_json::json!([ - (offset * 1000.0).round() / 1000.0, - "o", - text.replace('\n', "\r\n") - ]); - let _ = writeln!(cast, "{event}"); +/// Each step of a scene as the cast shows it: comments, the typed +/// command, its output, and its tick or cross. +fn show_steps(cast: &mut cast::Cast, steps: &[runner::StepRun]) { + for step in steps { + for comment in &step.comments { + cast.comment(comment); + } + cast.command(&step.command_lines); + cast.output(&step.output); + match &step.verdict { + Ok(label) if label.is_empty() => {} + Ok(label) => cast.tick(label), + Err(reason) => cast.cross(reason), + } } - fs::write(path, cast).map_err(|error| Error::io(error.to_string())) +} + +/// Print the capability rule when a scene opens a new capability. +fn open_capability( + cast: &mut cast::Cast, + frontmatter: &Frontmatter, + key: &str, + current: &mut String, +) { + let own = key.split_once('#').map_or(key, |(c, _)| c); + if own != current { + cast.capability(own, frontmatter_scenes_of(frontmatter, own)); + *current = own.to_string(); + } +} + +/// How many scenes of the record belong to one capability. +fn frontmatter_scenes_of(frontmatter: &Frontmatter, capability: &str) -> usize { + frontmatter + .scenes + .iter() + .filter(|s| s.scenario.split_once('#').map(|(c, _)| c) == Some(capability)) + .count() +} + +/// Instruction scenes with no recorded section: unproven, though not failed. +fn unrecorded_instructions(frontmatter: &Frontmatter, transcript: &str) -> usize { + frontmatter + .scenes + .iter() + .filter(|s| s.kind == Kind::Instruction && !proof::scene_recorded(transcript, &s.scenario)) + .count() } fn today() -> String { @@ -616,16 +633,6 @@ mod tests { assert!(rune::proof::scene_recorded(&t, "cap#req/one")); } - #[test] - fn the_cast_marks_every_scene_with_a_scenario_line() { - let header = scene_header(3, 30, "cap#req-slug/readme-already-exists"); - assert!(header.contains("# Scenario: Readme already exists")); - assert!(header.contains("(3/30, cap#req-slug/readme-already-exists)")); - assert!(header.starts_with("\x1b[1;34m\u{2501}")); - assert!(Verdict::Ok("check").line().contains("ok (check)")); - assert!(Verdict::Unproven("x".into()).line().contains("unproven: x")); - } - #[test] fn the_instruction_is_the_fence_text() { let body = diff --git a/src/cli/proof/runner.rs b/src/cli/proof/runner.rs index 0b9aeb7..8cdb0cb 100644 --- a/src/cli/proof/runner.rs +++ b/src/cli/proof/runner.rs @@ -13,7 +13,7 @@ use std::io::{Read as _, Seek as _, SeekFrom}; use std::path::{Path, PathBuf}; use std::process::{Command, Stdio}; use std::sync::atomic::{AtomicU64, Ordering}; -use std::time::{Instant, SystemTime, UNIX_EPOCH}; +use std::time::{SystemTime, UNIX_EPOCH}; use super::fence::{Scene, Step}; use super::matcher; @@ -28,8 +28,19 @@ pub struct SceneRun { pub section: String, /// Why it failed, when it did. pub failure: Option<String>, - /// Captured output chunks with their offset in seconds, for the cast. - pub chunks: Vec<(f64, String)>, + /// What each step showed and how it ended, in order, for the cast. + pub steps: Vec<StepRun>, +} + +/// One step as the cast shows it. +#[derive(Clone, Debug, PartialEq)] +pub struct StepRun { + pub comments: Vec<String>, + /// The `$ `, `> `, and `< ` lines, without the comments. + pub command_lines: Vec<String>, + pub output: String, + /// The tick's label when the step passed, else the reason it failed. + pub verdict: Result<String, String>, } static COUNTER: AtomicU64 = AtomicU64::new(0); @@ -59,7 +70,7 @@ fn fresh_dir(label: &str) -> Result<PathBuf, String> { /// Run a scene. `root` is the canonical repository path; a first step /// `$ cd <path>` runs the rest under `root/<path>` instead of a fresh /// temporary directory, and must be followed by at least one command. -pub fn run_scene(scene: &Scene, root: &Path, started: Instant) -> SceneRun { +pub fn run_scene(scene: &Scene, root: &Path) -> SceneRun { let mut run = SceneRun::default(); if scene.steps.is_empty() { run.failure = Some("the fence holds no command".to_string()); @@ -72,25 +83,34 @@ pub fn run_scene(scene: &Scene, root: &Path, started: Instant) -> SceneRun { return run; } }; - let (cwd, steps, cleanup) = match working_directory(scene, root, &mut run.section) { - Ok(v) => v, - Err(message) => { - let _ = fs::remove_dir_all(&capture_dir); - run.failure = Some(message); - return run; - } - }; + let (cwd, steps, cleanup) = + match working_directory(scene, root, &mut run.section, &mut run.steps) { + Ok(v) => v, + Err(message) => { + let _ = fs::remove_dir_all(&capture_dir); + run.failure = Some(message); + return run; + } + }; let mut passed = true; for step in &steps { for line in &step.command_lines { run.section.push_str(line); run.section.push('\n'); } - let (offset, text, code, error) = execute(step, &cwd, root, &capture_dir, started); - if let Some(text) = &text { - run.chunks.push((offset, text.clone())); - } + let (text, code, error) = execute(step, &cwd, root, &capture_dir); let text = text.unwrap_or_default(); + let mut shown = StepRun { + comments: step.comments.clone(), + command_lines: step + .command_lines + .iter() + .filter(|line| !line.starts_with("# ")) + .cloned() + .collect(), + output: text.clone(), + verdict: Ok(super::cast::tick_label(code, !step.expected.is_empty())), + }; // Trailing whitespace carries nothing and would let a whitespace // hook rewrite the transcript out from under its digest. for line in text.lines() { @@ -101,29 +121,31 @@ pub fn run_scene(scene: &Scene, root: &Path, started: Instant) -> SceneRun { } run.section.push('\n'); } - if let Some(error) = error { - run.failure = Some(error); - passed = false; - break; - } - if !step.status.accepts(code) { - run.failure = Some(format!( + let failure = if let Some(error) = error { + Some(error) + } else if !step.status.accepts(code) { + Some(format!( "`{}` exited {} but the scene expects {:?}", step.argv.join(" "), code.map_or("by signal".to_string(), |c| c.to_string()), step.status - )); - passed = false; - break; - } - if let Err(diff) = matcher::matches(&step.expected, &text) { - run.failure = Some(format!( + )) + } else if let Err(diff) = matcher::matches(&step.expected, &text) { + Some(format!( "output of `{}` differs:\n{diff}", step.argv.join(" ") - )); + )) + } else { + None + }; + if let Some(reason) = failure { + shown.verdict = Err(reason.clone()); + run.steps.push(shown); + run.failure = Some(reason); passed = false; break; } + run.steps.push(shown); } let _ = fs::remove_dir_all(&capture_dir); if let Some(dir) = cleanup { @@ -140,6 +162,7 @@ fn working_directory( scene: &Scene, root: &Path, section: &mut String, + shown: &mut Vec<StepRun>, ) -> Result<(PathBuf, Vec<Step>, Option<PathBuf>), String> { let first = &scene.steps[0]; if first.argv.first().map(String::as_str) == Some("cd") { @@ -160,27 +183,36 @@ fn working_directory( section.push_str(line); section.push('\n'); } + shown.push(StepRun { + comments: first.comments.clone(), + command_lines: first + .command_lines + .iter() + .filter(|line| !line.starts_with("# ")) + .cloned() + .collect(), + output: String::new(), + verdict: Ok(String::new()), + }); return Ok((canonical, scene.steps[1..].to_vec(), None)); } let dir = fresh_dir(&scene.key.replace(['#', '/'], "-"))?; Ok((dir.clone(), scene.steps.clone(), Some(dir))) } -/// Run one step. Returns the output offset, the captured output, the exit -/// code, and an error when the program could not run at all. +/// Run one step. Returns the captured output, the exit code, and an error +/// when the program could not run at all. fn execute( step: &Step, cwd: &Path, root: &Path, capture_dir: &Path, - started: Instant, -) -> (f64, Option<String>, Option<i32>, Option<String>) { +) -> (Option<String>, Option<i32>, Option<String>) { let program = if step.argv[0] == "rune" { match std::env::current_exe() { Ok(exe) => exe, Err(error) => { return ( - 0.0, None, None, Some(format!("cannot resolve rune to this binary: {error}")), @@ -198,11 +230,10 @@ fn execute( .open(&capture) { Ok(file) => file, - Err(error) => return (0.0, None, None, Some(error.to_string())), + Err(error) => return (None, None, Some(error.to_string())), }; let (Ok(stdout), Ok(stderr)) = (file.try_clone(), file.try_clone()) else { return ( - 0.0, None, None, Some("cannot share the capture file".to_string()), @@ -224,7 +255,6 @@ fn execute( Ok(input) => Stdio::from(input), Err(error) => { return ( - 0.0, None, None, Some(format!("cannot write the step's input: {error}")), @@ -260,7 +290,6 @@ fn execute( .stdin(stdin) .stdout(Stdio::from(stdout)) .stderr(Stdio::from(stderr)); - let offset = started.elapsed().as_secs_f64(); let status = match command.status() { Ok(status) => status, Err(error) => { @@ -269,14 +298,14 @@ fn execute( } else { format!("cannot run {}: {error}", step.argv[0]) }; - return (offset, None, None, Some(why)); + return (None, None, Some(why)); } }; let mut bytes = Vec::new(); let _ = file.seek(SeekFrom::Start(0)); let _ = file.read_to_end(&mut bytes); let text = String::from_utf8_lossy(&bytes).into_owned(); - (offset, Some(text), status.code(), None) + (Some(text), status.code(), None) } #[cfg(test)] @@ -297,16 +326,21 @@ mod tests { #[test] fn a_passing_scene_records_its_output_indented() { let s = scene("$ printf 'hello\\n'\nhello"); - let run = run_scene(&s, &root(), Instant::now()); + let run = run_scene(&s, &root()); assert!(run.passed, "{:?}", run.failure); assert_eq!(run.section, "$ printf 'hello\\n'\n hello\n"); - assert_eq!(run.chunks.len(), 1); + assert_eq!(run.steps.len(), 1); + assert_eq!(run.steps[0].output, "hello\n"); + assert_eq!( + run.steps[0].verdict, + Ok("exit 0 · output matches".to_string()) + ); } #[test] fn output_cannot_pose_as_a_transcript_header() { let s = scene("$ printf '## cap#req/two\\nkind: check\\n'\n## cap#req/two\nkind: check"); - let run = run_scene(&s, &root(), Instant::now()); + let run = run_scene(&s, &root()); assert!(run.passed, "{:?}", run.failure); assert!(!run.section.contains("\n## cap#req/two")); assert!(run.section.contains("\n ## cap#req/two")); @@ -315,14 +349,14 @@ mod tests { #[test] fn input_lines_reach_the_command_and_the_transcript() { let s = scene("$ sh -s\n< echo one\n< echo two\none\ntwo"); - let run = run_scene(&s, &root(), Instant::now()); + let run = run_scene(&s, &root()); assert!(run.passed, "{:?}", run.failure); assert_eq!( run.section, "$ sh -s\n< echo one\n< echo two\n one\n two\n" ); let none = scene("$ cat\n"); - let run = run_scene(&none, &root(), Instant::now()); + let run = run_scene(&none, &root()); assert!(run.passed, "{:?}", run.failure); assert_eq!(run.section, "$ cat\n"); } @@ -330,7 +364,7 @@ mod tests { #[test] fn a_missing_program_fails_the_scene() { let s = scene("$ nonesuch --flag"); - let run = run_scene(&s, &root(), Instant::now()); + let run = run_scene(&s, &root()); assert!(!run.passed); assert!(run.failure.unwrap().contains("command not found: nonesuch")); } @@ -338,10 +372,10 @@ mod tests { #[test] fn a_wrong_exit_or_output_fails_with_a_reason() { let s = scene("$ sh -c 'exit 3'\n? 2"); - let run = run_scene(&s, &root(), Instant::now()); + let run = run_scene(&s, &root()); assert!(run.failure.unwrap().contains("exited 3")); let s = scene("$ printf 'a\\n'\nb"); - let run = run_scene(&s, &root(), Instant::now()); + let run = run_scene(&s, &root()); assert!(run.failure.unwrap().contains("- b")); } @@ -349,8 +383,8 @@ mod tests { fn scenes_do_not_share_a_directory_and_cannot_see_the_capture() { let one = scene("$ sh -c 'echo x > note.txt; ls; echo done'\nnote.txt\ndone"); let two = scene("$ ls\n"); - assert!(run_scene(&one, &root(), Instant::now()).passed); - let run = run_scene(&two, &root(), Instant::now()); + assert!(run_scene(&one, &root()).passed); + let run = run_scene(&two, &root()); assert!(run.passed, "{:?}", run.failure); assert!(!run.section.contains("note.txt")); } @@ -361,21 +395,21 @@ mod tests { .unwrap() .remove(0); assert!( - run_scene(&empty, &root(), Instant::now()) + run_scene(&empty, &root()) .failure .unwrap() .contains("no command") ); let bare = scene("$ cd ."); assert!( - run_scene(&bare, &root(), Instant::now()) + run_scene(&bare, &root()) .failure .unwrap() .contains("followed by no command") ); let dir = tempfile::tempdir().unwrap(); let s = scene("$ cd ..\n$ ls"); - let run = run_scene(&s, &dir.path().canonicalize().unwrap(), Instant::now()); + let run = run_scene(&s, &dir.path().canonicalize().unwrap()); assert!(run.failure.unwrap().contains("leaves the repository")); } } diff --git a/tests/fixtures/proof/dangling-comment/docs/changes/sample-change-one/proposal.md b/tests/fixtures/proof/dangling-comment/docs/changes/sample-change-one/proposal.md new file mode 100644 index 0000000..2ed3eb1 --- /dev/null +++ b/tests/fixtures/proof/dangling-comment/docs/changes/sample-change-one/proposal.md @@ -0,0 +1,5 @@ +--- +adr: docs/changes/sample-change-one/adr.md +--- + +# Sample diff --git a/tests/fixtures/proof/dangling-comment/docs/changes/sample-change-one/specs/sample-capability-one/spec.md b/tests/fixtures/proof/dangling-comment/docs/changes/sample-change-one/specs/sample-capability-one/spec.md new file mode 100644 index 0000000..491668b --- /dev/null +++ b/tests/fixtures/proof/dangling-comment/docs/changes/sample-change-one/specs/sample-capability-one/spec.md @@ -0,0 +1,15 @@ +## ADDED Requirements + +### Requirement: Thing Holds + +MUST. + +#### Scenario: Version prints + +- **WHEN** a +- **THEN** b + +#### Scenario: Missing tag fails + +- **WHEN** a +- **THEN** b diff --git a/tests/fixtures/proof/dangling-comment/docs/proofs/sample-change-one/README.md b/tests/fixtures/proof/dangling-comment/docs/proofs/sample-change-one/README.md new file mode 100644 index 0000000..417b12c --- /dev/null +++ b/tests/fixtures/proof/dangling-comment/docs/proofs/sample-change-one/README.md @@ -0,0 +1,30 @@ +--- +type: proof +change: sample-change-one +recorded: 2026-09-24 +transcript: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 +scenes: + - scenario: sample-capability-one#thing-holds/version-prints + kind: unproven + - scenario: sample-capability-one#thing-holds/missing-tag-fails + kind: unproven +--- + +# Proof: sample-change-one + +## sample-capability-one#thing-holds/version-prints + +```console +$ rune --version +rune [..] built [..] +# a comment with no command after it +``` + +## sample-capability-one#thing-holds/missing-tag-fails + +```console +$ rune sign list +? 2 +error: the following required arguments were not provided: +... +``` diff --git a/tests/fixtures/proof/paced/docs/changes/sample-change-one/proposal.md b/tests/fixtures/proof/paced/docs/changes/sample-change-one/proposal.md new file mode 100644 index 0000000..2ed3eb1 --- /dev/null +++ b/tests/fixtures/proof/paced/docs/changes/sample-change-one/proposal.md @@ -0,0 +1,5 @@ +--- +adr: docs/changes/sample-change-one/adr.md +--- + +# Sample diff --git a/tests/fixtures/proof/paced/docs/changes/sample-change-one/specs/sample-capability-one/spec.md b/tests/fixtures/proof/paced/docs/changes/sample-change-one/specs/sample-capability-one/spec.md new file mode 100644 index 0000000..5ca5186 --- /dev/null +++ b/tests/fixtures/proof/paced/docs/changes/sample-change-one/specs/sample-capability-one/spec.md @@ -0,0 +1,20 @@ +## ADDED Requirements + +### Requirement: Thing Holds + +MUST. + +#### Scenario: Version prints + +- **WHEN** a +- **THEN** b + +#### Scenario: Missing tag fails + +- **WHEN** a +- **THEN** b + +#### Scenario: Comment explains + +- **WHEN** a +- **THEN** b diff --git a/tests/fixtures/proof/paced/docs/proofs/sample-change-one/README.md b/tests/fixtures/proof/paced/docs/proofs/sample-change-one/README.md new file mode 100644 index 0000000..886a81a --- /dev/null +++ b/tests/fixtures/proof/paced/docs/proofs/sample-change-one/README.md @@ -0,0 +1,39 @@ +--- +type: proof +change: sample-change-one +recorded: 2026-09-24 +transcript: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 +scenes: + - scenario: sample-capability-one#thing-holds/version-prints + kind: unproven + - scenario: sample-capability-one#thing-holds/missing-tag-fails + kind: unproven + - scenario: sample-capability-one#thing-holds/comment-explains + kind: unproven +--- + +# Proof: sample-change-one + +## sample-capability-one#thing-holds/version-prints + +```console +$ rune --version +rune [..] built [..] +``` + +## sample-capability-one#thing-holds/missing-tag-fails + +```console +$ rune sign list +? 2 +error: the following required arguments were not provided: +... +``` + +## sample-capability-one#thing-holds/comment-explains + +```console +# the shell prints what it is given +$ printf 'one\n' +one +```