Skip to content

About

Nightwatch — an agentic framework for calibrated, falsifiable agent systems: compiled gates, Brier calibration, doctrine trip breakers, two-tier execution harness (wake leases + clone swarm)

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Nightwatch — an agentic framework for building calibrated, falsifiable agent systems

Nightwatch is a design pattern for multi-agent systems that treats every agent output as a falsifiable claim. A coordinator dispatches bounded work quanta to specialist agents and synthesizes fixed-size outputs; judgment is pre-registered as executable gates rather than exercised as discretion at runtime; and every probabilistic claim is scored against outcomes so miscalibrated components are flagged by rule, not by intuition.

The framework patterns

  • Coordinator fan-out. Fetch → dispatch one bounded quantum per specialist → barrier → synthesize fixed-size outputs. No sequential chains. The barrier is mechanical: a fan-out latch (bin/runner_join.py) records each runner's output hash, and synthesis over a partial house is refused, not improvised.
  • Compiled mechanical gates. Judgment is pre-registered as code and executed as math. The machine argues through falsification, not discretion.
  • Calibration loop. Every probabilistic claim is Brier-scored at resolution against decay-weighted history; drift is flagged by rule. Closed-loop delta records (bin/build_deltas.py) turn each scored prediction into a structured post-mortem against a JSON schema.
  • Doctrine trip breakers. The system's own rules carry kill conditions and trip automatically. Kill conditions themselves are mechanical evaluators (bin/kill_condition_evaluators.py) that must be shown to both trip and clear on synthetic data before the first real row arrives.
  • Shared ledger, allowlisted. Every scoring reader goes through one loader (bin/ledger_loader.py) with a desk allowlist: a new specialist is excluded from scoring until explicitly admitted — fail-closed.
  • Behavioral version control. Prompt and prose amendments deploy to a shadow run first, are snapshotted before and after, and restore byte-identically (bin/amend_prompt.py). A prompt is code; it versions like code.
  • Agent foundry. New specialists are bred from a charter template, each owning one differentiation axis, quarantined until scored output earns trust.
  • Shared data plane. Fetch once per cycle into shared caches; per-agent re-fetching is a spec violation.
  • Two-tier execution. A cloud agent head holds all judgment; a local execution tier on a consumer workstation does the bounded, verifiable work — wake-on-demand under reference-counted leases, an activity-aware sleep decider, and a hash-pinned clone swarm for parallel collection. The local tier holds no credentials and no commit path: if it goes dark, cycles get slower, never wronger. Its outputs carry provenance tags and pass a mechanical gate, a sealed canary set, cryptographic pins, and a circuit breaker before anything downstream may read them — and even then only through an exception digest, never raw. See docs/execution-tier.md.
  • Scoped remote channel. A second host can write through the single booking path without general access: an SSH forced command (bin/book_trade_channel.py) validates the requested argv token-by-token against the booking CLI's allowlist and refuses everything else.
  • Stream watchdog. The price stream is supervised in layers (stream_watch.py, bin/stream_watchdog.py): a silent writer or a stalled venue trips a file that blocks new bookings while exits and settlements stay live.

Reference implementation: prediction-market research engine

The framework's reference instantiation is a prediction-market research engine: a coordinator dispatches bounded candidate evaluations to desk agents (model-driven, sentiment-fade, contrarian, meta red-team, short-horizon), each returning a fixed decision block. Every candidate must clear a compiled gate stack before booking: quarter-Kelly position sizing with a 5-point expected-value band, a Bayesian lower-confidence-bound entry gate (the claimed edge is shrunk toward the price by the desk's calibration history, then Wilson-penalized), pre-registered one-shot traps, driver-concentration caps, and price-unit safety. Fills are depth-capped and breadth-first; entries fail closed without a live depth snapshot. Settlement is ledger-terminal: closures rewrite the book row itself to closed, with a ghost-ledger reconciliation for rows settled while technically open.

Repo layout

bin/            compiled gates and machinery: EV/LCB gates, traps, trip
                breakers, circuit breaker, Brier scoring, calibration,
                fill engine, bar registry, ledger loader, delta builder,
                fan-out latch, prompt version control, remote channel,
                and the local-tier integrity suite (mechanical gate,
                canary, pins, drift metrics, exception digest, provenance)
fixtures/       test fixtures: sample market metadata, driver taxonomy,
                the sealed canary set, and an example pins file
schemas/        JSON schemas for delta records and verdicts
docs/           architecture documentation, incl. the execution tier and
                the exception-digest contract
harness/        workstation harness reference code (lease manager with the
                wake-need bridge; site configuration by environment)
kelly.py        quarter-Kelly sizing (with the LCB gate as an advisory mode)
optimizer.py    candidate optimizer
book_trade.py   the single booking path — all gates enforced here
stream_watch.py stream-health derivation for the price feed
test_*.py       executable specifications for the booking path
bin/test_*.py   executable specifications for the gates and the tier suite
run_tests.sh    one-command runner: every suite against sandboxed state

Quick-start

./run_tests.sh                          # all 30 suites, sandboxed state
python3 -m unittest test_book_trade     # booking gate stack: 49 tests
python3 test_traps.py                   # pre-registered trap semantics
python3 test_mikiri.py                  # confidence-bounded entry gate
python3 test_deltas.py                  # closed-loop delta builder
python3 bin/test_ledger_loader.py       # allowlisted ledger loader
python3 bin/test_book_trade_channel.py  # forced-command validator refusals
python3 bin/proof_bayesian_lcb_gate.py  # LCB gate: every clause trips+clears
python3 kelly.py --side yes --price 76 --p 0.85 --bankroll 500

Every suite stages its own fixtures through testutil.py: subprocesses run with a sandboxed HOME, a sandboxed state dir (NIGHTWATCH_STATE_DIR), planted poller quotes, and a stub order-book snapshot (NIGHTWATCH_BOOK_STUB) — no suite touches live state or the network. The stub book matters: entries walk a depth snapshot and fail closed without one, so the harness plants a deep two-sided book at each test's price instead of letting the fill path reach for a live venue. In production the same code fetches the live book. State lives in state/ (beside the scripts, or wherever NIGHTWATCH_STATE_DIR points); the booking path's --help lists its subcommands (book, exit, settle, ledger, audit, kill, resume).

Requires Python 3.10+, standard library only.

What's not here

This is a public edition derived from a larger private system. The booking path, gates, calibration, fill engine, delta builder, remote channel, stream watchdog, local-tier workers and integrity suite, and lease harness in this repo are complete and runnable; the surrounding operations are not included: the hourly cycle worker and its audit steps, the venue gateway client, the local-inference client the tier's workers shell out to (its prompt/schema pins ship as fixtures; the workers degrade to an extractive CPU fallback without it), the doctrine and spec documents the code cites as policy, the desk charters, and the swarm launcher and merge tooling described in docs/execution-tier.md (§3 there is design documentation; the lease manager in harness/ is the shipped half of that story). References in comments to those components describe the private deployment, not files you failed to find.

Honesty

This is paper-trading simulator infrastructure. There are no live positions, no API keys, and no credentials anywhere in this repository. It is strategy research code, not financial advice.

This repo is a public mirror of the private working system; its history is squashed to a single commit at publish.

About

Nightwatch — an agentic framework for calibrated, falsifiable agent systems: compiled gates, Brier calibration, doctrine trip breakers, two-tier execution harness (wake leases + clone swarm)

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages