diff --git a/README.md b/README.md index f4645ad2..6f7f6965 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ berth status # who holds the card, who ## How it works -

How berth works. Clients ask for the card: agents such as Hermes and ComfyUI workflows through the berth gateway, one endpoint that starts engines on demand; an engine under berth run; a game; and a VM through berth vfio run. berthd queues them by priority, drains the holder at its boundary (saving KV caches or parking it in RAM), waits for the driver to show the memory free, then grants the lease. gpu0 serves strata while gpu1 is passed through to a VM. Below, a timeline of one 24 GiB card: llama-swap, strata, a game, a VM passage and ComfyUI take turns and never overlap; markers 1 to 4 show one handoff.

+

How berth works. Clients ask for the card: agents such as Hermes and ComfyUI workflows through the berth gateway, one endpoint, with one URL per card, that starts engines on demand; an engine under berth run; a game or a training run under berth train, which locks its cards; and a VM through berth vfio run. berthd queues them by priority, drains the holder at its boundary (saving KV caches or parking it in RAM), waits for the driver to show the memory free, then grants the lease. gpu0 serves strata while gpu1 is passed through to a VM. Below, a timeline of one 24 GiB card: llama-swap, strata, a game, a VM passage and ComfyUI take turns and never overlap; markers 1 to 4 show one handoff.

Every engine asks berthd for the card: through `berth run`, the gateway, ComfyUI's API or a game claim. When another engine holds it, Berth: @@ -256,8 +256,8 @@ which `berth service status` prints. ## Preview -The first two screenshots show the harbor web UI with its built-in demo data, on the CARD tab and on -the Engines tab in advanced mode. The third shows the CARD tab of a live node on an iPhone. +The screenshots show the harbor web UI with its built-in demo data: the CARD tab, the Engines tab in advanced +mode, and the CARD tab at phone width. [The harbor and telemetry](docs/harbor.md) explains every panel and shows how to add the harbor to an iPhone's Home Screen. @@ -265,7 +265,7 @@ iPhone's Home Screen.

Berth harbor Engines tab, advanced mode: decode throughput per engine, the token ledger, the per-model leaderboard, storage and residency

-

Berth harbor on an iPhone: the CARD tab with the ticket counter, collisions avoided, the collision tiles and the decode and TPOT KPIs, with CARD, ENGINES and ABOUT in a bottom bar

+

Berth harbor at phone width: the CARD tab with the ticket counter, collisions avoided, the collision tiles and the decode and TPOT KPIs, with CARD, SWITCH, ENGINES and ABOUT in a bottom bar

## Install @@ -331,6 +331,25 @@ request to the reply, lease included (2026-10-23): Strata's numbers and the method are in [Benchmarks](docs/benchmarks.md#swap-numbers). +## A month on the reference workstation + +berthd counts every handoff it makes, and the harbor's collisions board shows the counts. These are the counts +from one RTX 4090 workstation between 2026-10-12 and 2026-11-13, running llama-swap, NInfer, Strata, ComfyUI, +coding agents and games: + +| What Berth did | Count | +|---|---| +| Requests that waited for the card instead of loading on top of the engine holding it | 1,136 | +| Engines drained at a reply or job boundary to make room | 901 | +| VRAM handed from one engine to the next | 23.9 TiB | +| Processes holding VRAM without a lease, reported and left running | 573 | +| Swaps between two engines of equal priority that the ranking stopped | 57 | +| Releases that stalled and were caught before the card stayed full | 4 | +| Requests refused after their wait ran out | 24 | + +In the last 7 days, 23 engines and jobs held the card for 143 lease-hours. Leases that share the card count their +own hours each. + ## Roadmap The phases, the shipped releases and what comes next are in the [roadmap](docs/roadmap.md). diff --git a/assets/how-it-works.svg b/assets/how-it-works.svg index 59797e9a..0eeaa64f 100644 --- a/assets/how-it-works.svg +++ b/assets/how-it-works.svg @@ -1,6 +1,6 @@ How berth works -Clients ask berth for the GPU. Agents such as Hermes and ComfyUI workflows go through the berth gateway, one endpoint that starts engines on demand; engines under berth run, games and a VM passage (berth vfio run) ask directly. berthd queues them by priority, drains the holder at its boundary (saving KV caches or parking it in RAM), waits for the driver to show the memory free, then grants the lease. gpu0 serves strata while gpu1 is passed through to a VM. A timeline of one 24 GiB card shows llama-swap, strata, a game, a VM passage and ComfyUI taking turns with no overlap. +Clients ask berth for the GPU. Agents such as Hermes and ComfyUI workflows go through the berth gateway, one endpoint (and one per card) that starts engines on demand; engines under berth run, games, training runs under berth train and a VM passage (berth vfio run) ask directly. A training run locks its cards, and every other request there is refused at once. berthd queues them by priority, drains the holder at its boundary (saving KV caches or parking it in RAM), waits for the driver to show the memory free, then grants the lease. gpu0 serves strata while gpu1 is passed through to a VM. A timeline of one 24 GiB card shows llama-swap, strata, a game, a VM passage and ComfyUI taking turns with no overlap. @@ -26,7 +26,7 @@ WHO ASKS berth gateway -:9799 · one endpoint +:9799 · /cards/gpuN Agents · Hermes · chat apps POST /v1/chat/completions @@ -44,8 +44,8 @@ -A game launches -claim → hold the card +A game or a training run +claim or berth train diff --git a/assets/preview-harbor.png b/assets/preview-harbor.png index 9f01a8c6..7e855ae1 100644 Binary files a/assets/preview-harbor.png and b/assets/preview-harbor.png differ diff --git a/assets/preview-mobile.png b/assets/preview-mobile.png index 3045166f..a2d45d79 100644 Binary files a/assets/preview-mobile.png and b/assets/preview-mobile.png differ diff --git a/assets/preview-requests.png b/assets/preview-requests.png index 99dcd446..6a865997 100644 Binary files a/assets/preview-requests.png and b/assets/preview-requests.png differ diff --git a/assets/social-preview.png b/assets/social-preview.png new file mode 100644 index 00000000..6d373f8a Binary files /dev/null and b/assets/social-preview.png differ diff --git a/docs/harbor.md b/docs/harbor.md index 8ab7defc..fe008574 100644 --- a/docs/harbor.md +++ b/docs/harbor.md @@ -2,10 +2,9 @@ ## Preview -The harbor UI (basic mode, served from the built-in demo fixture - every number on the screen -comes from `demo.json`). The tide shows the card's memory by holder over six hours; the -hatched stretch is a stowaway the fixture replays: a model that ran without a lease after a -daemon restart, reported and left alone. +The harbor UI in basic mode, served from the built-in demo fixture. Every number on the screen comes from +`demo.json`. The tide shows the card's memory by holder over the last hour. The hatched stretch is a stowaway the +fixture replays: a model that ran without a lease after a daemon restart, which Berth reported and left alone.

Berth harbor: the CARD tab with ticket counter, request KPIs, the tide of card memory by holder, and the model timeline

@@ -15,10 +14,10 @@ decode throughput, the token ledger, the per-model leaderboard, storage and resi

Berth harbor Engines tab, advanced mode: decode throughput per engine, the token ledger, the per-model leaderboard, storage and residency

-On a phone, the harbor fits the screen: the tabs move to a bar at the bottom and the panels -stack. This is the CARD tab on an iPhone, read from a live node on the reference workstation. +On a phone, the harbor fits the screen. The tabs move to a bar at the bottom and the panels stack. This is the +CARD tab at phone width, from the same demo fixture. -

Berth harbor on an iPhone: the CARD tab with the ticket counter, collisions avoided, the collision tiles and the decode and TPOT KPIs, with CARD, ENGINES and ABOUT in a bottom bar

+

Berth harbor at phone width: the CARD tab with the ticket counter, collisions avoided, the collision tiles and the decode and TPOT KPIs, with CARD, SWITCH, ENGINES and ABOUT in a bottom bar

### The harbor on your iPhone's Home Screen