Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 23 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ berth status # who holds the card, who

## How it works

<p align="center"><img src="assets/how-it-works.svg" alt="How berth works. Clients ask for the card: agents such as Hermes and ComfyUI workflows through the berth gateway, one endpoint that starts engines on demand; an engine under berth run; a game; and a VM through berth vfio run. berthd queues them by priority, drains the holder at its boundary (saving KV caches or parking it in RAM), waits for the driver to show the memory free, then grants the lease. gpu0 serves strata while gpu1 is passed through to a VM. Below, a timeline of one 24 GiB card: llama-swap, strata, a game, a VM passage and ComfyUI take turns and never overlap; markers 1 to 4 show one handoff." width="100%"></p>
<p align="center"><img src="assets/how-it-works.svg" alt="How berth works. Clients ask for the card: agents such as Hermes and ComfyUI workflows through the berth gateway, one endpoint, with one URL per card, that starts engines on demand; an engine under berth run; a game or a training run under berth train, which locks its cards; and a VM through berth vfio run. berthd queues them by priority, drains the holder at its boundary (saving KV caches or parking it in RAM), waits for the driver to show the memory free, then grants the lease. gpu0 serves strata while gpu1 is passed through to a VM. Below, a timeline of one 24 GiB card: llama-swap, strata, a game, a VM passage and ComfyUI take turns and never overlap; markers 1 to 4 show one handoff." width="100%"></p>

Every engine asks berthd for the card: through `berth run`, the gateway, ComfyUI's API or a game
claim. When another engine holds it, Berth:
Expand Down Expand Up @@ -256,16 +256,16 @@ which `berth service status` prints.

## Preview

The first two screenshots show the harbor web UI with its built-in demo data, on the CARD tab and on
the Engines tab in advanced mode. The third shows the CARD tab of a live node on an iPhone.
The screenshots show the harbor web UI with its built-in demo data: the CARD tab, the Engines tab in advanced
mode, and the CARD tab at phone width.
[The harbor and telemetry](docs/harbor.md) explains every panel and shows how to add the harbor to an
iPhone's Home Screen.

<p align="center"><img src="assets/preview-harbor.png" alt="Berth harbor: the CARD tab with ticket counter, request KPIs, the tide of card memory by holder, and the model timeline" width="900"></p>

<p align="center"><img src="assets/preview-requests.png" alt="Berth harbor Engines tab, advanced mode: decode throughput per engine, the token ledger, the per-model leaderboard, storage and residency" width="900"></p>

<p align="center"><img src="assets/preview-mobile.png" alt="Berth harbor on an iPhone: the CARD tab with the ticket counter, collisions avoided, the collision tiles and the decode and TPOT KPIs, with CARD, ENGINES and ABOUT in a bottom bar" width="300"></p>
<p align="center"><img src="assets/preview-mobile.png" alt="Berth harbor at phone width: the CARD tab with the ticket counter, collisions avoided, the collision tiles and the decode and TPOT KPIs, with CARD, SWITCH, ENGINES and ABOUT in a bottom bar" width="300"></p>

## Install

Expand Down Expand Up @@ -331,6 +331,25 @@ request to the reply, lease included (2026-10-23):

Strata's numbers and the method are in [Benchmarks](docs/benchmarks.md#swap-numbers).

## A month on the reference workstation

berthd counts every handoff it makes, and the harbor's collisions board shows the counts. These are the counts
from one RTX 4090 workstation between 2026-10-12 and 2026-11-13, running llama-swap, NInfer, Strata, ComfyUI,
coding agents and games:

| What Berth did | Count |
|---|---|
| Requests that waited for the card instead of loading on top of the engine holding it | 1,136 |
| Engines drained at a reply or job boundary to make room | 901 |
| VRAM handed from one engine to the next | 23.9 TiB |
| Processes holding VRAM without a lease, reported and left running | 573 |
| Swaps between two engines of equal priority that the ranking stopped | 57 |
| Releases that stalled and were caught before the card stayed full | 4 |
| Requests refused after their wait ran out | 24 |

In the last 7 days, 23 engines and jobs held the card for 143 lease-hours. Leases that share the card count their
own hours each.

## Roadmap

The phases, the shipped releases and what comes next are in the [roadmap](docs/roadmap.md).
Expand Down
8 changes: 4 additions & 4 deletions assets/how-it-works.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/preview-harbor.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/preview-mobile.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/preview-requests.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/social-preview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
13 changes: 6 additions & 7 deletions docs/harbor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@

## Preview

The harbor UI (basic mode, served from the built-in demo fixture - every number on the screen
comes from `demo.json`). The tide shows the card's memory by holder over six hours; the
hatched stretch is a stowaway the fixture replays: a model that ran without a lease after a
daemon restart, reported and left alone.
The harbor UI in basic mode, served from the built-in demo fixture. Every number on the screen comes from
`demo.json`. The tide shows the card's memory by holder over the last hour. The hatched stretch is a stowaway the
fixture replays: a model that ran without a lease after a daemon restart, which Berth reported and left alone.

<p align="center"><img src="https://raw.githubusercontent.com/T-Crypt/berth/main/assets/preview-harbor.png" alt="Berth harbor: the CARD tab with ticket counter, request KPIs, the tide of card memory by holder, and the model timeline" width="900"></p>

Expand All @@ -15,10 +14,10 @@ decode throughput, the token ledger, the per-model leaderboard, storage and resi

<p align="center"><img src="https://raw.githubusercontent.com/T-Crypt/berth/main/assets/preview-requests.png" alt="Berth harbor Engines tab, advanced mode: decode throughput per engine, the token ledger, the per-model leaderboard, storage and residency" width="900"></p>

On a phone, the harbor fits the screen: the tabs move to a bar at the bottom and the panels
stack. This is the CARD tab on an iPhone, read from a live node on the reference workstation.
On a phone, the harbor fits the screen. The tabs move to a bar at the bottom and the panels stack. This is the
CARD tab at phone width, from the same demo fixture.

<p align="center"><img src="https://raw.githubusercontent.com/T-Crypt/berth/main/assets/preview-mobile.png" alt="Berth harbor on an iPhone: the CARD tab with the ticket counter, collisions avoided, the collision tiles and the decode and TPOT KPIs, with CARD, ENGINES and ABOUT in a bottom bar" width="300"></p>
<p align="center"><img src="https://raw.githubusercontent.com/T-Crypt/berth/main/assets/preview-mobile.png" alt="Berth harbor at phone width: the CARD tab with the ticket counter, collisions avoided, the collision tiles and the decode and TPOT KPIs, with CARD, SWITCH, ENGINES and ABOUT in a bottom bar" width="300"></p>

### The harbor on your iPhone's Home Screen

Expand Down
Loading