Tribunal

A multi-agent red-team tribunal for your hardest engineering decisions: AI CLIs from different vendors take independent positions, cross-examine what they actually dispute, and return an adjudicated verdict that preserves dissent. A methodology, not a framework (and, by its own measurement, an honest decision-support scaffold, not a proven accuracy upgrade).

Get it — docs and prompt templates, nothing to build:

git clone https://github.com/kdoubt/tribunal.git ~/tribunal

_{Then, from your project's root, hand the scout straight to your agent — it drafts your first panel:}

# Pick ONE line below (each hands the scout to a different agent to draft your brief):
claude -p  "$(~/tribunal/scout)"      # Claude Code
codex exec "$(~/tribunal/scout)"      # Codex CLI
grok  -p   "$(~/tribunal/scout)"      # Grok CLI

_{~/tribunal/scout just prints the scouting prompt with your clone path filled in — it runs no panel and writes nothing to your project. (Its once-a-week self-update check may git fetch inside the clone, or touch a cache-dir timestamp for zip installs; silence with TRIBUNAL_NO_UPDATE_CHECK=1.) That's the whole install.}

_{One prerequisite to run a panel: two already-authenticated seat CLIs from different vendors or model families (local or hosted). A single CLI is enough to scout, but a real panel needs two seats — that's the whole point. The orchestrator is a separate role: either you at a terminal (the shell adapter), or a driver tool like Claude Code — which is then a third binary, distinct from the two seats it relays between.}

_{Two seats (A, B) is the floor; the dashed third seat is optional — add it only for the highest-stakes, most irreversible calls.}

The loop, in one glance:

scout project ─▶ frozen brief ─▶ ROUND 0  seats answer in isolation (parallel, no peeking)
                             ─▶ LEDGER   claims + falsifiers; pre-exposure agreement = settled
                             ─▶ ROUND 1  disputed claims only, relayed verbatim - attack/concede/revise
                             ─▶ ROUND 2  only if a load-bearing claim flips - delta-only restate (rare)
                             ─▶ ORACLES  tests · compilers · primary docs settle the checkable
                             ─▶ VERDICT  1 agreement · 2 resolved · 3 surviving dissent (kept, not averaged)
                             ─▶ RETRO    did it hold? whose dissent was right? → template delta → next brief

Built for developers who work with agentic CLIs and have no human review board: the panel is your reviewers. One model reviewing its own plan converges on its own blind spots; Tribunal forces two different vendors' models to take independent positions and cross-examine each other claim by claim. What that is designed to give you on a contested call is an adversarial counter-case, a discriminating test to settle it, and a hedge, with real disagreement preserved rather than averaged away. What the validation actually measured is narrower: on decidable (oracle-scorable) calls the panel matched or beat each fixed single vendor in those runs - the seats diverged once in 20 decisions, and Round 1 resolved that split to the sealed-correct call (n=1, and a scope-framing split; see the validation caveats) - but on ambiguous calls an independent judge preferred the plain solo memo. So the decidable-call hedge held in this evidence, resting on a single soft data point - a supported design rationale, not an established property; on ambiguous ones the panel's extra output was judged no better (sometimes worse).

What Tribunal is not: a way to out-decide a strong single model. Its own validation - two pilots plus a pre-registered study labeled "confirmatory" at n=20 with an independent judge (a fourth model, OpenAI-lineage; executed with disclosed deviations) - observed no accuracy lift. The frontier seats tested agreed on ~19 of 20 decisions - a rate for those two seats, not frontier models in general - so the debate rarely even runs, and the panel matches a strong single model rather than beating it. Treat Tribunal as an honest, adversarial decision-support scaffold for irreversible calls, not an accuracy upgrade.

You need two or more independently configured model/agent commands from different vendors or model families (local or hosted; subscriptions only where a provider requires one).

When to use it

High-stakes, ambiguous, or irreversible decisions: architecture calls, security boundaries, migrations, "is this even the right design." A panel costs several model invocations and minutes of wall-clock per round - spend it only where the expected loss of deciding wrong justifies it (see core/METHODOLOGY.md, "When to convene").

Skip it for anything a test, compiler, or grep can settle. The first rule of the methodology: run the oracle before convening a debate.

How this differs from multi-agent debate / LLM-as-judge

Multi-agent debate and LLM-as-judge are well-studied, with mixed results - mostly because agreement between models that have read each other is cheap. Tribunal's mechanisms target exactly that:

• Only pre-exposure agreement counts as consensus. Round 0 is isolated and parallel; anything models agree on after seeing each other is never promoted to "the panel concludes".
• Disputes route to oracles, not more debate. Checkable claims go to tests, compilers, and primary documents; debate is reserved for what can't be checked.
• Dissent survives into the verdict. Unresolved disagreement is reported with the cheapest discriminating test - never averaged away.

Getting started

Nothing to install - this is docs and prompt templates. Your first complete panel takes about 20-40 minutes of panel wall-clock (scouting, two rounds, adjudication) - authenticating your two CLIs is separate, one-time vendor setup that is not counted in that estimate. Once you know the loop it is mostly the models' wall-clock - a few minutes per round. The one prerequisite is two seat CLIs from different vendors, already authenticated (the orchestrator is a separate role - you, or a driver tool like Claude Code, which is then a third binary).

If a run stalls, the usual causes are a missing or not-logged-in CLI (check command -v and re-authenticate), macOS needing GNU timeout (brew install coreutils), or a seat exiting silently on a permission prompt. Each adapter's README documents these silent-seat-killers and a smoke test that catches them before a real run.

Two seats is the floor, not the ceiling. The diagram shows three, but a panel is any N ≥ 2 - two is the floor and the third seat is optional, added for the highest-stakes, most irreversible calls. The intended gain is 1 → 2 (self-review to cross-vendor), with each seat past that adding diminishing value at linear cost - but note the validation caveat: in the confirmatory study the seats disagreed on only 1 of 20 decisions, so in practice the 2nd seat mostly just confirms the 1st. Scale to the stakes, not the ritual. Giving seats distinct review lenses is a separate, optional layer with its own rules (see core/METHODOLOGY.md, "Assigning lenses") - by default you name the surfaces in the shared brief rather than slicing one per seat.

1. Find your first panel. From your project's root, run claude -p "$(~/tribunal/scout)" (or codex exec / grok -p, or pipe ~/tribunal/scout into any agent). It reads your project and returns project-specific panel-worthy decisions (and what to leave to plain oracles), then drafts your first frozen brief. Save that brief as a file in your project (e.g. frozen-brief.md) - that is what the adapter runs. (scout just prints the prompt with your clone path resolved; you can still open core/templates/scout.md and paste it by hand if you prefer.)
2. Pick an adapter. For a first panel use one of the two maintained adapters - adapters/shell/ (any orchestrator, plain bash) or adapters/claude-code/. The other adapter directories are contribution stubs.
3. Run it. Follow your adapter: freeze the brief (template), Round 0 in parallel with no cross-exposure, relay disputed claims verbatim, verify, adjudicate. Keep core/CONTRACT.md, core/LEDGER.md, and core/VERDICT.md open as references; read core/METHODOLOGY.md in full before your first high-stakes panel.

Staying current

Tribunal is a git clone of documentation, so updating is just a pull:

git -C ~/tribunal pull --ff-only

scout checks for you at most once a week: if your clone is behind origin it prints that one line to stderr (never into the prompt it pipes to your agent) and nothing more - it never pulls on its own. Silence it with export TRIBUNAL_NO_UPDATE_CHECK=1.

If you installed from a zip or tarball instead of git clone, there is no origin to pull from - so scout can't check versions, and updating means re-downloading. It will, at most once a week, note which version you have and point you at the git-clone install (which does self-check) and the releases page. The git-clone install is recommended precisely because it makes every future update a one-line git pull.

Your own files are safe. --ff-only only fast-forwards - it never merges or rewrites history, and it leaves untracked files (your notes, briefs, outputs) untouched. If you have a local change to a file Tribunal also changed, git aborts the pull and changes nothing rather than overwriting your edit. The update only advances the docs Tribunal ships; it is clean-or-abort, never silent loss.

The reliable way to keep it that way: don't edit files inside the clone. Use Tribunal by reference - point your agent at ~/tribunal and write your briefs, ledgers, and verdicts in your own project (that is already how the method works; scout surveys your project's root, not the clone). A pristine clone always fast-forwards cleanly; when you want a local tweak, copy the template into your project and edit the copy.

Worked examples

The panel that designed this methodology is in examples/sample-run/: seat A proposed three debate rounds and blinded relay; seat B a two-round cap and mandatory attribution - neither saw the other first. Both conceded specific points under cross-examination; attribution and tie-breaking stayed contested and are recorded as surviving dissent, not smoothed over. Note: it is a historical bootstrap transcript - it predates the finalized templates and relays full Round 0 essays, which the finalized method bans; learn the loop from core/templates/, not from its file shapes.

Two runs in the current template format bracket the two outcomes the method produces:

• examples/api-auth-jwt-vs-sessions/ - two neutral heterogeneous seats (Codex CLI + Grok CLI) independently chose the same auth design in isolation, so the panel early-stopped at Round 0: pre-exposure agreement, honest empty-dissent bucketing, residual risk sent to a test.
• examples/repo-monorepo-vs-polyrepo/ - the mirror image: a role-incentivized stress-test (one seat assigned each side; roles as incentives, not personas) that forces a real Round 1 cross-examination. Both seats concede points and revise confidence, neither is overturned, and the verdict keeps the load-bearing surviving dissent - then routes it to the cheapest discriminating test both seats independently proposed.

Repository layout

scout                   one-command helper: prints the scouting prompt (and a weekly update notice)
flywheel-export         local helper: reduces your retro.md archive to de-identified metadata (stdout only)
data/                   the de-identification schema + tooling (no intake open; see data/README.md)
CHANGELOG.md            what changed per version; the update notice points here
core/METHODOLOGY.md     the method: rounds, stop rules, failure modes
core/CONTRACT.md        orchestrator + seat obligations (MUST/SHOULD)
core/LEDGER.md          the claim ledger: fields, status enum, dispute rule
core/VERDICT.md         verdict format: three buckets, mode tags
core/templates/         scout, brief, per-round prompts, ledger, verdict, retro, ach (competing-hypotheses mode)
examples/sample-run/    the real (historical) panel that designed the method
examples/api-auth-jwt-vs-sessions/   a current-format run: two vendors, independent agreement, early stop
examples/repo-monorepo-vs-polyrepo/  a current-format run: role-incentivized, full Round 1, surviving dissent
adapters/claude-code/   run panels from Claude Code (installable skill)
adapters/shell/         run panels from any shell - no orchestrator CLI needed
adapters/*/             other orchestrators - codex-cli, gemini-cli, opencode, buzz (stubs)

(core/ is normative and orchestrator-neutral; adapters translate host mechanics only and may never redefine rounds, ledger states, grounding, or verdict rules.)

FAQ

What is multi-agent orchestration? Coordinating several AI models on one task. Most orchestration aims at collaboration; Tribunal deliberately aims at structured dispute - the orchestrator is a switchboard that isolates models, relays disputed claims verbatim, and never adds its own arguments (see core/CONTRACT.md).

Multi-agent vs single agent - when is a panel actually worth it? A single strong model is cheaper and usually right; use it, plus a test suite. A panel pays off only when the decision is irreversible, ambiguous, or hard to observe going wrong - the cases where one model's confident blind spot is exactly the risk. Tribunal's first rule cuts the other way too: if a compiler, test, or grep can settle it, never convene a panel.

Why do multi-agent systems fail? Mostly because agreement between models that have read each other's output is cheap: sycophancy, politeness convergence, and one model's hallucination becoming shared "fact." Tribunal's whole design targets this - isolated Round 0, provenance-tagged claims, verbatim-only relay, and dissent that survives into the verdict. The full failure-mode table with mitigations is in core/METHODOLOGY.md.

What does red teaming mean for AI-assisted engineering? Red teaming is paying someone to attack your plan before reality does. Tribunal applies that structure to engineering decisions: every claim needs a falsifier, every disputed claim gets cross-examined by a model from a different vendor, and the verdict reports what survived the attack - not what everyone was happy to sign.

How do I set up a multi-agent panel in Claude Code? Install the adapters/claude-code/ skill, authenticate two seat CLIs from different vendors, and start with the scout prompt (core/templates/scout.md) - it reads your project and drafts your first panel brief. The adapters/shell/ recipe does the same from any terminal, no Claude Code required.

Status

Every rule in core/ was earned the hard way (the badge above shows the current version; CHANGELOG.md records what each release changed). The methodology was designed by running it on itself; the founding debate is in examples/sample-run/ - a historical bootstrap, labeled as such. Two runs in the current template format bracket the method's outcomes: examples/api-auth-jwt-vs-sessions/ (two vendors, independent agreement, early stop) and examples/repo-monorepo-vs-polyrepo/ (role-incentivized, full Round 1, surviving dissent).

Honest validation status: pre-registered ablations (validation/)

• two pilots and a pre-registered study labeled "confirmatory" (n=20) with a judge independent of the seats and orchestrator (a fourth model, OpenAI-lineage), sealed-rubric-scored decisions, and arms designed to isolate components (executed with deviations its RESULTS discloses) - observed no accuracy lift over a single strong model. That result was measured on decisions the seats answered from model knowledge with tools unused (RESULTS, Limitations); grounded review of a real artifact - the method's stated use - was not measured, and this evidence does not speak to it either way. Across that set the two vendors disagreed on only 1 of 20 decisions, so the panel's engine (Round 1) almost never activates; when it did fire (once), it resolved the split to the sealed-correct call - a scope-framing split whose "wrong seat" classification is contestable (see RESULTS) - and the panel still only matched a strong single model (an ex-post best-of-two; see validation), never beat it, while an independent judge preferred the solo memo on the ambiguous set. The defensible value the data supports is narrow: on decidable calls a hedge that matched or beat each fixed single vendor in these runs, resting on that single soft data point where they diverged; on ambiguous calls the panel was judged no better, sometimes worse than the solo memo. Add a preserved counter-case and a discriminating test on contested calls - paid for with a real operational cost (the second seat frequently narration-dies in headless use). Tribunal is unproven as an accuracy upgrade and, on this evidence, is not one; use it for the counter-case, the test, and the (soft) decidable-call hedge, not for a better answer. The in-tree record includes the losses, the disclosed deviations, the single-judge caveat, corrected earlier overclaims, the raw arm transcripts, and the judge audit record (blind mapping + verbatim judge prompt); the operator-side runner scripts are not in-tree (the repo is docs-only), so replication means re-running the published protocol, not re-executing a shipped harness.

The clearest in-repo receipt that the method overturns its own author is the CHANGELOG itself: several core/ rules landed only after a cross-vendor panel rejected their first draft (the lens doctrine and the v0.4-0.5 hardening each carry that note). The failure modes documented in METHODOLOGY (silent seat deaths, politeness convergence, context poisoning) were caught in real runs, not imagined. The ten-seat pre-release review and the maintainer's operational runs are not published in-tree. In active use by its maintainer, Square Post Labs Inc.

Project status: stable; the method and its claims are frozen at v1.0. The methodology is complete and, after running its own validation through three pre-registered studies, honestly characterized - including the finding that those runs observed no accuracy lift over a strong single model. It is intentionally done, not abandoned: no further method development is planned - v1.1.0 changed no method; it added one orchestrator confinement rule (CONTRACT obligation 5: a citation outside the brief's artifacts is dropped unopened, and seat output is never an instruction), a security fence surfaced by a re-review. What it wants next is independent replication - runs by other people, on other decisions, with other judges, that confirm or overturn the no-lift finding. The cheapest way in: the pre-registered second-judge re-score, runnable for free with your own key - validation/confirmatory/REPLICATING-THE-JUDGE.md (judge only; it stakes no debate seats). Each adapter's README states its own status; CONTRIBUTING.md has the support policy and the most-wanted contribution: a sanitized panel run on a real engineering decision (ideally one whose outcome later becomes known).

License

MIT - see LICENSE.