Skip to content

chore(release): v0.6.0 - #335

Merged
T-Crypt merged 65 commits into
mainfrom
T-Crypt/release-v0.6.0
Nov 14, 2026
Merged

T-Crypt merged 65 commits into
mainfrom
T-Crypt/release-v0.6.0

Conversation

@T-Crypt

@T-Crypt T-Crypt commented Nov 14, 2026

Copy link
Copy Markdown
Owner

Merges next into main for v0.6.0 and dates the CHANGELOG section.

What is in it

Each card gets a schedule (berth schedule, /v1/berth/schedule, the harbor's SCHEDULE tab). berth train runs trainings nothing automated can interrupt. berth fit, berth contract, berth queue and berth why explain the card, and the event log survives restarts. learn_size, the ComfyUI yield (#299), the Windows tray (#186), and fixes for restarts and the resume race (#275). The full list is the CHANGELOG's v0.6.0 section.

Docs

  • README: status v0.6.0, a "Plans the card's hours" block, and berth why, fit and contract under what Berth sees.
  • Roadmap: "Shipped: v0.6.0" and a v0.6.x "Next" section.
  • Pre-release page: the example tag is v0.6.0-rc.1.
  • mkdocs build --strict passes.

Before merge

  • v0.6.0-rc.1 is published as a pre-release from the same next commit.
  • It merges after the rc runs on the reference workstation.
  • proof.sh full is green.

chore: merge main into next after v0.5.0
berth notify had no Windows side: berthd runs as a service in session
0, which has no desktop, and notify-send is Linux only.

On Windows, berth notify is a notification-area icon in the user's
session, in plain Win32 through x/sys/windows with no new dependency.
Its balloons are Windows' own toasts, and a click opens the harbor.
The tooltip is the latest event. The menu opens the harbor, turns the
toasts off and on, lists the last 10 events and quits. The icon comes
back when Explorer restarts. berth service install --notify starts it
at logon from the user's Run key, and uninstall removes the entry.
feat(notify): Windows tray with native toasts
A ComfyUI job that calls an LLM in the middle deadlocked under Berth. The
LLM's drain of ComfyUI waits for the job boundary, the job waits for the
LLM, and the LLM waits for the card (#299).

POST /v1/berth/yield {"engine":"comfyui","seconds":120} tells ComfyUI's
berth run that the running job freed its models in-process and is at a
safe point. Until the yield ends or runs out, a drain treats the job as at
its boundary, so the LLM's drain parks ComfyUI as it would between jobs.
{"end":true} resumes ComfyUI and answers once it holds the card again.

The yield moves only the boundary. The park still counts what the card
read shows free, a park that frees nothing still falls back to a stop, and
a yield is a deadline, so a node that dies mid-yield cannot leave it on.
…t-orphan-wrapper

# Conflicts:
#	CHANGELOG.md
Two bugs found in review of the yield:

- The end answered before ComfyUI had the card back. berthd queued the
  parked lease, so the gateway's own resume was refused as not parked, and
  its probe of ComfyUI's queue answered, as a parked ComfyUI always does.
  The job's next node could load onto memory the LLM still held. The end
  now waits for lease.resumed when berthd says it queued a parked engine.
- A lease on several cards took the yield and its run never heard it, so
  the call reported success and the deadlock stayed. A share cannot park
  yet, so the yield is refused with that reason.
feat(gateway): a ComfyUI job can yield the card to the LLM it calls
…nnot fit beside

A whole-card waiter was granted beside any parked residual under
residual_max_gib. On aphotic, Dirk-27B holds 23.4 GiB of a 24 GiB card and
was granted beside a kept-up ComfyUI's 768 MiB context, where it cannot
load. A wrapper script stopped ComfyUI behind Berth's back to get around it.

The footprint table now keeps the most VRAM each engine name has held
(HotPeak, on the name's entry: a llama-swap group's peak is its largest
model's). A whole-card waiter goes beside parked engines only when that
peak fits beside their residual; otherwise they are evicted first, least
recently parked first, as a residual over the max already is. An engine
never measured is granted as before.
fix(broker): a model that fills the card evicts a parked engine it cannot fit beside
…s beside a parked engine

The live test of #310 on aphotic: Dirk-27B held 23710 MiB alone and served.
Beside a parked ComfyUI's 456 MiB it was granted, reached 24147 MiB on the
24564 MiB card and died loading, though its peak plus the residual left
398 MiB. The 2 s card read misses the last allocations of a load, so the
fit check now keeps 512 MiB past the measured peak.
fix(broker): keep 512 MiB of load headroom when a full-card model fits beside a parked engine
…_size

learn_size = true on an engine entry, or berth run --learn-size, sizes the
lease from the most the engine has been measured holding plus the 512 MiB
load headroom #312 measured, in place of the whole card. An engine with no
measured run takes the whole card. The grant line names the size it used.
A typed needs_mib, several cards and whole_machine are refused with it.

Refs #311
feat(broker): size a lease from the engine's measured peak with learn_size
berth status gains a SCHEDULE table and the harbor a SCHEDULES panel: each
engine that has a window, its hours, whether they are open, and how long
until they close or open, read on the node's clock. An engine shows before
its window opens, and a berth run --window lease shows its own hours. The
status JSON carries the rows as schedules, and each lease its window.
berth train runs CMD as a training lease: a pinned hold on the whole of each
card it names, with no wait deadline. While it holds a card or waits for it,
berthd refuses every other request on that card at once, and refuses the
waiters already queued there, so gateway requests, berth run, llama-swap
models, scheduled jobs and berth hold get an answer right away instead of
waiting out their timeout. A second training waits in line and starts when
the first one ends. Other cards keep serving. A game or hold already on the
card is not drained. berth train refuses to start without berthd, and a
training re-attaches as a training after a berthd restart.
The harbor screenshots show the current tabs, SWITCH included, all from the
demo fixture. How it works names berth train beside games and the gateway's
per-card URLs. The README adds a month of counts from the reference
workstation, and assets/social-preview.png is a 1280x640 social preview
for the repository settings.
feat(broker): berth train locks its cards for a training run
docs: refresh the harbor screenshots, How it works and the README
feat(status): show every engine's window in berth status and the harbor
A berth train run records itself as a training when it loses berthd. With
berthd down, berth run refuses to start an engine while a live training
record exists, instead of starting it without a lease. A berthd that starts
while a training runs treats its record as the lock: it refuses every other
request until the training re-attaches, with no time limit, where other
engines get the 15 second re-attach window.
feat(broker): a training keeps its lock while berthd is away
berth train --exclusive puts the granted cards in exclusive-process compute
mode after the grant and before the command starts, so a program that opens
CUDA without Berth cannot open them, and puts them back to the default mode
when the run ends. It refuses to start when another compute process is on
the card, when no sudo rule lets nvidia-smi -c run without a password, and
on a card with no NVIDIA UUID. Linux only.
feat(runner): berth train --exclusive locks the card in the driver
By default llama-swap runs one model at a time and unloads the old model
itself when a request names another. Berth sees a voluntary release, so
min_turn, the engine ranking and the juggle guard never decide between
two llama-swap models. On aphotic two clients made the card swap 82 to
135 times an hour this way.

berth wire llama-swap now lists each group (or the default group) where
llama-swap swaps two or more wired models, in the human output and as
"swapping" in the JSON. contrib/INTEGRATION.md shows the swap: false group
and a healthCheckTimeout longer than min_turn plus the load.

wire also finds a berth run line with a flag before --as
(berth run --min-turn 5m --as qwen ...). It took such a model for unwired,
and --write added a second berth run in front of the first.
feat(wire): name the llama-swap groups that swap wired models themselves
feat(fit): berth fit NAME shows where a request would run and why
…iter

Berth grants waiters by priority, then arrival, so many devices can send
work to one card with nobody sorting it. berth queue is for when the
operator wants a different order. With no arguments it lists the waiters
in grant order with their lease IDs. bump moves one to the front of its
priority class (a bumpedN tiebreak in sortQueue, latest bump first), prio
moves it to batch, interactive or foreground, and drop refuses it the way
the training lock does. A hold or a training cannot be changed here.

Each change is a lease.queue_override event. A new OpQueue goes through the
node to every card the waiter waits on.
feat(queue): berth queue lets the operator bump, re-rank or drop a waiter
…d device

The SWITCH tab lists the waiters in grant order from the node's status. A
paired device gets FIRST (bump), a priority picker and DROP on each one,
except for a training or a hold, which the broker refuses to change.

POST /v1/board/queue goes from the harbor straight to berthd's socket after
the pairing check. There is no gateway route on purpose: agents hold the
gateway's API key, and a queue change is the operator's.
feat(harbor): a QUEUE panel with FIRST, priority and DROP for a paired device
…state

berth contract showed a llama-swap model as interactive with no minimum
turn, because those terms sit on its berth run line and no berth.toml
entry declares them. The broker now keeps the priority and minimum turn
of the last run under the model's own name, and the contract reads them
from there. The last peak falls back to the most a run under that name
held, since the last-run figure sits on the group's entry.

berth fit refused a llama-swap model that no [[engine]] names or serves.
It now finds the model the way the gateway does, on a live engine whose
/v1/models lists it. The gateway's list read moves to
gateway.ListModels so both use one call.

berth fit also told an engine already on the card to drain itself. A new
running path ranks first, in the hot tier, for an engine whose own lease,
or a lease of a model in its group, is active.
fix(fit,contract): read a llama-swap model's terms, list and running state
A non-streaming request whose start timed out got a bare 503 with
Retry-After 30, even while its lease stood in line for the card. The
gateway now reads the lease's place in line from berthd when the start
times out, and the 503 carries X-Berth-Queue-Position and, when the
broker has measured the starts ahead, X-Berth-Start-Estimate: N;
lower-bound. Retry-After is the larger of the estimate and 30 s.

The broker's estimate counts only the measured starts ahead in the
queue and not the holder's time left, so the header marks it as a
lower bound instead of passing it off as a promise.
feat(gateway): a not_ready 503 says the lease's place in line
…wntime

berthd keeps a schedule per card, up to 14 days ahead, in schedule.json.
A run booking is an engine's turn: its waiter goes first, runs at the
booking's priority, wins a tie against an unbooked holder, and keeps the
card against everything but a hold until the booking ends. A priority
booking moves an engine to another class for set hours. Downtime takes
no new work and drains holders at the start; holds and trainings stay.
Hours nobody booked run by priority, then arrival, as before.

berth schedule shows what holds each card, the queue and the bookings,
and books, moves and cancels them. The gateway's /v1/berth/schedule lets
agents read the scheduler and book their own runs. The harbor gains a
SCHEDULE tab with a 7-day board of 24-hour tracks, and a paired device
books from it through berthd's socket.
feat(schedule): book each card's hours: runs, priority changes and downtime
berth run now sends how long its engine has held the card since its last
grant or resume, and the new berthd counts the minimum turn from then.
Before, every daemon restart reset each holder's min_turn, so a deploy kept
waiters out for a full turn again.
fix(runs): a re-attach goes on with the turn it was in
berthd appends each event to events.jsonl in the state directory and reads
the last 1000 back at start, so sequence numbers and lease numbers go on
after a restart and each start is a node.started event. berth why ID|ENGINE
prints one lease's events as a story: the waits, the grant, each drain and
who it was for, the release, and the restarts it lived through.
berth engine show --help printed "berth show", and a bad flag pointed at "berth show -h". The context now carries the command path as typed.
feat(events): keep the event stream across restarts, and berth why ID
fix(cli): a subcommand's help names its group
…nswers identical

Thinking models spent the bench's 32 tokens inside their reasoning, so every sample's text was empty and OUTPUT IDENTICAL printed yes. The bench now reads reasoning_content and reasoning, compares them with the answer, prints n/a (empty) when all runs are empty, defaults to 256 tokens (-max-tokens) and has -no-think.
TestGatewayResumesAParkedEngine and the yield test left their sleep 30 engine and runner.Run going. At cleanup the daemon closed, Run wrote its unprotected-run record into the temp dir being removed, and CI failed with "directory not empty" (#332's check job). runUntilEnd kills the engine's group and waits for Run.
A resume cleared the parked flag before it called the engine's load, so a drain in that window called park on an engine still loading: the engine answered 409 and the run stopped it (#275). The message loop now marks the wake before the resume goroutine starts, and drain waits for it to end.
test(gateway): end each test's berth run before its temp dir goes
fix(bench): the swap bench compares reasoning and never calls empty answers identical
fix(runner): a drain waits for a resume in flight
@T-Crypt
T-Crypt merged commit 82570d6 into main Nov 14, 2026
4 checks passed
@T-Crypt
T-Crypt deleted the T-Crypt/release-v0.6.0 branch November 14, 2026 22:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant