feat(revise): verifiable evidence: quote and spot checks, one verdict per line, who ran a review

README.md

@@ -783,7 +783,8 @@ session-start line or `work` says something unexpected, these files are what it
 | `revision/claims.md`, `revision/merges.txt`, `revision/sweep/` | `revise`'s sweep at `prepare` step 2: the rules and documented claims it judges, `scripts/window_merges.py`'s printout with `--doc`, and one `diff-probe` block per merged pull request (`pr-NNNN.md`) | `revise`, for the report's Practice section, through `scripts/block_tally.py` | Replaced at each revision |
 | `revision/drift.txt` | `revise` `prepare` step 2, through `scripts/drift_checks.py`: the scripted claims' `claim` and `break` lines | `prepare` steps 2 and 6, for "Practice" and `ticket/` and `doc/` proposals | Rewritten at each revision |
 | `revision/state.json` | `revise` `prepare`, through `scripts/revise_state.py`: each stage done or skipped, with why | `prepare` step 10, which writes the report only once every stage is recorded | `revise_state.py start`, at the next `prepare` |
-| `revision/notes/` | The driver, before `revise` `prepare`: meeting or chat notes, pasted | `revise`, read as asks at `prepare` step 5 | Not cleared by the tools |
+| `revision/notes/` | The driver, before `revise` `prepare`: meeting or chat notes, pasted | `revise`, read as asks at `prepare` step 5, and by `scripts/quote_check.py` for the report's pasted quotes | Not cleared by the tools |
+| `revision/spot-check.md` | `revise` `prepare` step 2: one line per guidance item `scripts/spot_check.py` sampled, right or wrong | `scripts/spot_check.py tally`, for the report's Head | Rewritten at each revision |
 | `revision/tickets/` | `revise` `apply`: each accepted `ticket/` proposal as a prompt (`<n>-<slug>.md`) | `ticket-draft`, given a prompt's path, one at a time | Replaced at each `apply` |
 | `revision/feedback/` | `revise`: a `skill-feedback` draft per procedure finding, from `prepare` step 2 and `correct` step 2 | `skill-feedback`, given a draft's path | Not cleared by the tools |
 

agents/diff-probe.md

@@ -69,7 +69,9 @@ One line per claim id you were given, in the order given:
 
 Settle each verdict before you write its line: weigh the evidence, then write the one word it
 supports. A line's reason never argues another verdict than its word; when the evidence leaves
-you between two, write the one it leans to and say in the reason what made it close.
+you between two, write the one it leans to and say in the reason what made it close. A line
+whose place or reason names a second verdict, such as `judged as follows` after `breaks`, fails
+the block, and you are run again.
 
 ## The answer
 

agents/source-reader.md

@@ -134,8 +134,10 @@ Epic candidates: <each: key, summary, the milestone it serves>
 ```
 
 **`page <id | path>`**: `confluence_get_page`. Returns `## page <id> — <title>`, a line `Version: <n> ·
-Updated: <date>`, and one paragraph: what the page decides, and which code or process it
-constrains.
+Updated: <date>`, one paragraph: what the page decides, and which code or process it
+constrains, then one or two lines `Excerpt: "<the page's words>"`, each one run of the page's
+text, verbatim, at most 40 words, that bears out the paragraph, so a reader can find it on the
+page. A paragraph no excerpt bears out goes to `Questions:` instead.
 
 **A file in place of a key or an id.** `ticket` and `page` take a file path the caller names
 instead: build the block from the file's text, with `Source: file <path>` as its first line, and

docs/contracts.md

@@ -466,6 +466,71 @@ Writers: the driver, in harvest.md's "Clusters". Readers: `scripts/guidance_clus
 
 Pinned by: tests/test_guidance_clusters.py::test_prints_each_cluster_with_its_people_prs_and_quotes
 
+## The report's quotes
+
+Version: 1
+
+Fields: every line of a revision report, `ratchet-YYYY-MM.md`, that opens `said:` once its indent
+is stripped, a drafted entry's under `entry:` included: `said: "<words>" — <login>, <url>`, a
+comment's, or `said: "<words>" — <name>, pasted <file>`, a note in `.review/revision/notes/`.
+`scripts/quote_check.py <report> --comments <comment-list.json ...> --notes <dir> [--held
+<ledger>]` finds each: 1 to 40 words, in the own words of the comment with that `url` in the
+comment lists, whose `author` is `<login>`, as "The guidance items" finds an item's quote, or in
+the text of a note inside `--notes`, every line kept, never starting or ending inside a word. A
+line holding `said: "` anywhere that is not such a line of its own, a quote nested in a list item,
+a table or a blockquote, or a `said:` line in another case, is a fault. A URL's pull request is
+its repository and number, whatever page of it the URL names; a quote from a pull request no
+comment in the lists shares is a fault unless the ledger given with `--held` carries both its
+words and its URL under its `## Candidates below the criteria` heading, however that section lays
+them out: a held candidate's quote from before the log kept captures, which is unchecked. It prints `quotes: <n> · from
+comments: <n> · from notes: <n> · unchecked: <n>`, then `unchecked · line <n> · <url>: a held
+candidate's, no comment list given holds its pull request` per unchecked quote, and exits 0, or one `refused · line <n> · <why>` per fault, and `refused · <path> · <why>` per
+input it cannot read, and exits 1. An ellipsis or a changed word is a fault: a quote is one run of
+the comment, verbatim.
+
+Writers: `revise` `prepare` step 10, the report as `outline.md` gives it. Readers:
+`scripts/quote_check.py`, run by `prepare` step 10 before the report is offered for commit.
+
+Pinned by: tests/test_quote_check.py::test_a_quote_absent_from_its_comment_fails_the_report
+
+## The spot check
+
+Version: 1
+
+Fields: `scripts/spot_check.py sample --seed <text> [--n 10] <slice ...>` prints, for `--n` items
+picked at random from the guidance slices given (every item when there are fewer, the same ones for
+the same seed and slices), `spot · <dir>/<slice>#<n> · pr-NNNN · <url> · <kind> · <generality> ·
+"<quote>" · <principle>`, `<dir>` the slice's directory name, `<slice>` its name without `.json`,
+`<n>` the item's place in it from 1. `.review/revision/spot-check.md` holds one line per sampled
+item, `<id> · right` or `<id> · wrong · <what>`. `spot_check.py tally` with the same seed and slices
+and `--checked <that file>` prints `spot check: <n> items · <n> wrong · error rate <percent>%`, then
+`wrong · <id> · <what>` per wrong item, or `spot check: 0 items` with none. Both exit 1 with one
+`refused · <where> · <why>` line per fault: a slice that cannot be read, `--n` below 1, and for
+`tally` a line of neither form, an id not sampled, or a sampled id with no line or two.
+
+Writers: `revise` `prepare` step 2, `.review/revision/spot-check.md`. Readers:
+`scripts/spot_check.py tally`, whose `spot check:` line is the report's Head line.
+
+Pinned by: tests/test_spot_check.py::test_the_tally_counts_wrong_items_into_an_error_rate
+
+## The window's cost
+
+Version: 1
+
+Fields: `scripts/window_cost.py --threads <dir>` reads every `pr-NNNN/` dump under --threads, a
+harvest's or a load's, a pull request with no record included: its `pr.json` and `body.md`. It
+prints `cost: open-to-merge median <d>d, longest <d>d · <n> merged without a build · <n> pull
+requests`, durations in days to one decimal from `created_at` to `merged_at`, a pull request
+merged without a build one whose body has no Already checked block, as `scripts/record.py` reads
+it; or `cost: no merged pull request`. It exits 1 with one `refused · <path> · <why>` line per
+fault: a `pr.json` missing or with no readable `created_at` or `merged_at`, a merge before its
+opening, a dump with no `body.md`, or a `--threads` that is not a directory.
+
+Writers: `scripts/window_cost.py`. Readers: `revise` `prepare` step 1, whose line is the report's
+Head `cost:` line.
+
+Pinned by: tests/test_window_cost.py::test_the_median_and_longest_open_to_merge_and_the_unbuilt_count
+
 ## The judgments
 
 Version: 1
@@ -740,10 +805,10 @@ Pinned by: tests/test_profile.py::test_a_github_tracker_maps_at_most_one_label_t
 
 ## The record
 
-Version: 14
+Version: 15
 
 Fields: `<log>/pr-NNNN.yaml`, `<log>` the records directory `revise`'s "The standard's repository"
-names, first line `contract: 14`. This section is the one
+names, first line `contract: 15`. This section is the one
 place the fields are defined; the records directory's own README says how the revision
 reads them.
 
@@ -794,6 +859,9 @@ reads them.
   reviewed heads.
 - `findings:`, every cited finding: `key` (the full finding key `review-contract.md` defines),
   `severity`, `at` (the file and line, or the file alone at file level), `thread` (the review thread the verdict was read from),
+  `reviewer` (the login of the account that posted the finding's comment, whose token ran the
+  review; the attribution line names it too when the poster is the person, but the record reads the
+  comment's author, not that line),
   `verdict` (agree, disagree, discuss, unanswered, unparsed or contested), with `judged: true` and
   `basis:` when it was judged from a reply's own words, and `done: true` when that reply said the
   change was made, or `basis:` alone when the judge could not tell, `reason` on a
@@ -807,11 +875,11 @@ reads them.
   first line; its replies are not read, since the severity decides how they read.
   `scripts/ratchet_counts.py` counts it as a firing in its `unlabeled` column, not with the
   unanswered.
-- `questions:`, every cited question: `q`, `key`, `at`, `thread`, and `a`, the answer, or
+- `questions:`, every cited question: `q`, `key`, `at`, `thread`, `reviewer` as on a finding, and `a`, the answer, or
   `unanswered: true`.
 - `threads:`, a pointer to every other review thread: a person's, a bot's, or a finding posted
   with no key. Each carries `thread`; `by`, its opener, `bot` for a bot or `review` for an uncited
-  finding, which also carries its `severity`, `unparsed` with `verdict: unparsed` and `line:` as
+  finding, which also carries `reviewer` as on a finding and its `severity`, `unparsed` with `verdict: unparsed` and `line:` as
   on a finding; `authorship` of its opening comment, as
   "Authorship" below defines it; `at`; `opened`, the day; `resolved` and `outdated`, GitHub's flags; `replies`,
   the count from people; `verdict`, an uncited finding's read as a finding's, otherwise the pull
@@ -849,6 +917,8 @@ Version 13: `authorship` reads the signatures under "Authorship" as well as the
 and the standard's phrases, so a reply reworded from the attribution line, and an agent's own
 signature, read `ai-drafted` where earlier versions read `human`; a thread the author opened that
 way reads `verdict: none`, where it read `unanswered` or `unparsed`.
+Version 15 added `reviewer` on every finding, cited question and uncited finding, so a finding is
+attributable to the person who ran the review; a record of an earlier version has none.
 
 A harvested record reaches the log through `scripts/archive_records.py`, which refuses to
 replace a record whose findings, counted per rule, the harvested one falls short of, unless told
@@ -1109,7 +1179,7 @@ Pinned by: tests/test_revise_state.py::test_the_report_waits_until_every_stage_i
 
 ## The analysis blocks
 
-Version: 1
+Version: 2
 
 Fields: `.review/harvest/analysis/pr-NNNN.md`, one `thread-analyst` report, its blank lines
 ignored. Either the one line `record mismatch: <ids>`, or the heading `## pr-NNNN — threads read`
@@ -1129,8 +1199,9 @@ which is not the record's verdict; `procedure · pr-NNNN · <the Procedure: text
 pr-NNNN · <the Not read: text>` for each not `none`; and `mismatch · pr-NNNN · <ids>`. It exits 2
 with a `CANNOT RUN` line, printing nothing, on a named block missing, a pull request named twice or
 not as `pr-NNNN`, an empty block, a heading missing or naming another pull request, a `record
-mismatch:` line with more after it, a line of no form above, or a `Procedure:`, `Not read:` or
-`Read:` line missing or repeated.
+mismatch:` line with more after it, a line of no form above, an ask line with a second `·
+outcome:` or `· diff:`, which is two verdicts in one line (version 2), or a `Procedure:`, `Not
+read:` or `Read:` line missing or repeated.
 
 Writers: `thread-analyst`, at the path harvest.md's "Analysts" gives it. Readers:
 `scripts/block_tally.py`, run by harvest.md's "Analysts"; `revise` `prepare` step 2 and `onboard`'s
@@ -1140,7 +1211,7 @@ Pinned by: tests/test_block_tally.py::test_the_analysis_counts_asks_by_who_sourc
 
 ## The sweep blocks
 
-Version: 2
+Version: 3
 
 Fields: `.review/revision/sweep/pr-NNNN.md`, one `diff-probe` block, its blank lines ignored: the
 heading `## pr-NNNN · <sha, 7 characters>` naming the file's pull request; one line per claim,
@@ -1158,8 +1229,14 @@ rules first, `claim · <id> · judged <n> · follows <n> · breaks <n> · not to
 exits 2 with a `CANNOT RUN` line, printing nothing, on a named block, the claims file or a named
 merges file missing, a `### C-n` heading not in its form, an `every change` claim's document with
 no `doc` line in the merges file, a pull request named twice or not as `pr-NNNN`, a heading missing or naming another pull request, a
-line of no form above, a claim judged twice, a block with no line for a `C-n` claim it owes, or a `Not
-read:` line missing or repeated.
+line of no form above, a line holding two verdicts, a claim judged twice, a block with no line for a `C-n` claim it owes, or a `Not
+read:` line missing or repeated. A line holds two verdicts when its place or its reason has a `·`
+field that is `follows`, `breaks`, `not touched` or `not judged` alone, or opens with one as a
+label, `breaks:` or `breaks —`, other than its own verdict, or reads `judged as` one of them, the word ending at a space, punctuation other than `/`
+or `-`, a `.` not followed by a letter, or the line's end; a field that uses the word in a
+sentence, `follows the old habit`, or a path, `breaks/x.py`, is not one; the quoted line, which is
+code, is not read for it (version 3). Such a block is rerun, never
+counted.
 
 
 A claim whose heading ends ` · script: <check> [<arg> ...]` is judged by `scripts/drift_checks.py`,

docs/design.md

@@ -67,6 +67,8 @@ Libera's standard.
 
 **The month is read as it merges, and the revision reads the log.** Harvesting a month in one `prepare` started about sixty agents in one session, past Claude Code's limit on agents in a session, and a failure late in the run cost everything read before it. So the driver runs `revise capture` weekly: it harvests the whole window again, one `pr-reader`, so every record takes the replies posted since the last capture, and sends only the pull requests new or changed since then to the analysts and the extractor, keeping each pull request's harvest beside its record in the standard's log; `prepare` restores the window from there, capturing this repository only when its git shows a merge the log lacks, and each other covered repository always, one `pr-reader` that reads nothing unchanged, since no git here shows what merged there today. The log, committed, not a local directory, holds it, so a capture survives a new machine, a new driver and the clone `prepare` makes; the days each capture searched are written down, so `prepare` can tell a quiet week from one nobody captured, and checks git's merges against the captures by number, since GitHub's search is the only list a capture has. Nothing runs at merge itself: a GitHub Action, or a scheduled agent, would need a model's credentials and a person's answers at the archive's refusal and the commit, and a weekly capture spreads the same work. (`skills/revise/capture.md`; `docs/contracts.md`, "The capture".)
 
+**The report's evidence is checked, not trusted.** A revision once carried an analyst's paraphrase where a person's words belonged, counted a probe line that gave two verdicts as a break, recorded 42 findings with no one named as having run the review, and, in its write-up, eight figures nobody re-derived. Each is a place where a model's output reached the group unchecked. So every quote in the report is a `said:` line a script finds in the comment it cites; a line holding a second verdict fails its block, which is rerun rather than counted; each finding records the login that posted it; every figure is copied from a script's printout; and ten extracted items, picked at random, are checked against their comments by the driver, who did not extract them, with the error rate in the head, so the group can see how far to trust the extraction. A schema in JSON for the probes was weighed against tightening their line forms: a fixed line form a script parses already is a schema, and refusing a second verdict closes the gap with no new format. (`skills/revise/prepare.md` steps 2 and 10; `docs/contracts.md`, "The report's quotes", "The spot check" and "The sweep blocks".)
+
 **The revision reads what merged, not only what was said.** A rule's firings come only from findings, so a merged change that breaks a rule nobody flagged, or a documented practice that most pull requests skip, never reached the report. The sweep reads the merged pull requests' diffs, commits and descriptions against the rules whose `scope:` they touch and the claims the repository's documents make, from git alone, so it runs with no server: a script judges the claims a script can, such as a template's sections and a TODO's ticket, on every pull request opened after the document, and an agent the rest on at most five. It is a sibling of `thread-analyst`, `diff-probe`, rather than more work for it, because it runs on pull requests that had no threads and leaves the analyst's output unchanged; and it is not the full reviewer, which costs too much per pull request. Its results are reported and counted by no Dial until two revisions have shown how often they agree with the group, the same report-first start the probe had. What the sweep and the threads find still in the code becomes a `ticket/` proposal, and a document the merged work has left behind a `doc/` proposal, marked like a rule; an accepted ticket goes to `ticket-draft` one at a time, so Jira is still written only through its write plan. The report opens with an overview of every proposal by priority and impact in place of a fixed cap: a cap hid proposals the group might have wanted, while an ordered list lets the meeting stop when its time is up and leaves the rest blank, deferred. (`skills/revise/prepare.md`, steps 2, 6 and 10; `agents/diff-probe.md`; `skills/revise/apply.md`, step 3.)
 
 **A verdict is what a reply says, read from its word first.** A reply under a finding says Agree, Disagree and a reason, or Discuss. When its first line opens with the word, `scripts/record.py` reads it, for free and the same every time. When it does not, `pr-reader` judges what the reply says, and the record marks the verdict judged with a one-line basis: people answer in their own words, and a marker they must remember to type turned clear agreements into unanswered threads that sent the author back to `pr-fix`. The judge sees only replies with no word after the last word in the thread, and the script refuses a judgment on anything else, so a judgment never overrides a later word, the author's judged reply decides as their word would, and a person who disagrees with a judgment settles it by replying with the word. A reply the judge cannot read stays open and is walked in `pr-fix`; a finding nobody answers is recorded as unanswered and counts for nothing, and a resolved thread is not a verdict. The reason under a Disagree is the one part of a record that cannot be reconstructed later. (`pr-review`, "Whose reply counts"; `docs/contracts.md`, "The judgments".)