Checks an AI-written clinical note against the consultation transcript, and reports omissions, changed facts and unsupported statements, each with the passage that supports the flag.
Status: pre-alpha. Rules mode (no model) runs; full mode arrives milestone by milestone; see
docs/build-plan.md. Nothing here is ready for use yet.
Requires Python 3.11 or later. From a clone of this repository:
python -m pip install -e .
scribecheck demoThe demo runs offline in rules mode on three bundled synthetic consultations. Rules mode needs
no model or API key; its findings are warnings and never fail a build. Check your own files with
scribecheck check -t transcript.json -n note.md, or a folder of them with
scribecheck batch cases/. scribecheck init writes a starter config file.
- It checks whether the note is faithful to the transcript, not whether the transcript is correct. A misheard drug name copied faithfully into the note will pass.
- It is an evaluation tool for testing scribe systems, not a sign-off check on individual patients' notes in live care.
- v1 supports English only.
- It runs on your machine. Rules mode makes no network calls; full mode talks only to the model endpoint you configure.
- There is no telemetry, analytics or update checking.
The omission-checking method follows Fox S, Markham L, Lail R, Karotsieris M,
LLM Judges Verify Presence, Not Absence: Omission Blindness in AI Clinical Notes and What
Recovers It, arXiv:2608.31016 (2026), which builds on
MED-OMIT (Schumacher et al., 2023). Evaluation uses OmissionBench by Composo (data CC BY 4.0,
harness MIT), which is built partly on PriMock57 and ACI-Bench. The prompts in
src/scribecheck/prompts/route_one/ are copied unmodified from that harness (MIT, Copyright (c)
2026 Composo Limited; see the folder's LICENSE and README.md), and the severity rubric in
scribecheck's own audit prompt is theirs.
Drug names come from RxNorm Current Prescribable Content (release of 5 October 2026). This product uses publicly available data from the U.S. National Library of Medicine (NLM), National Institutes of Health, Department of Health and Human Services; NLM is not responsible for the product and does not endorse or recommend this or any other product. Symptom terms come from the Symptom Ontology (SYMP, CC0 1.0).
python -m venv .venv && source .venv/bin/activate # Windows: .venv\Scripts\activate
python -m pip install -e ".[dev]"
python -m pytest -q
python -m ruff check . && python -m ruff format --check .
python -m mypyInstructions for AI coding agents are in CLAUDE.md.
Apache-2.0. See LICENSE.