Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CLAUDE.md

Large diffs are not rendered by default.

27 changes: 17 additions & 10 deletions docs/api-reference.md

Large diffs are not rendered by default.

10 changes: 9 additions & 1 deletion docs/architecture-invariants.md
Original file line number Diff line number Diff line change
Expand Up @@ -720,11 +720,19 @@ Either way the actual launch (`runCustomModelEntry`) routes through `run()` itse

**Remove** stops the child, undoes the connection FIRST (a refused write of the endpoint store or of the settings rejects before any file is deleted), restores the flag only when this feature switched it on AND no other endpoint is saved, then deletes the runtime and the private home and, when asked, the models (`removeLocalModelFiles()`, which refuses any path outside the root). ⚠️ While a live session uses the endpoint the answer is 409 `sessions-open` with the session names (`sessionNamesUsingEndpoint()`, read at the moment of the request). ⚠️ The coding agent stays: what its installer wrote is not ours to remove.

**The library** (one model in use, several on disk). `LocalModelStateFile.library` lists every model that PASSED the check through Codeman: `{ id, label, context, digest, bytes?, cli?, verifiedAt? }`, no path. The service upserts the entry as a fact when `verify` passes (where it stores the digest); the reducer only carries the list, through every event but `remove`. An `installed` file an earlier build wrote has no list: `libraryOf()` then counts its one model, and an install or a switch from it writes that model into the list before the state's `model` member becomes the new job's. ⚠️ ONE model is in use: the endpoint record still lists exactly one model, the runtime still holds one (`OLLAMA_MAX_LOADED_MODELS=1`), and `launchRefusalMessage()` still starts only the state's model with its stored digest. Never widen the record to "every checked model": the port has no login, and two sessions on two models would make the runtime swap 20 GB on every request.

⚠️ Three verbs, each a row of the tables and never a special case. **Add**: `install` is also listed for `installed` in `INSTALL_TRANSITIONS`; the job is the same five steps, and the earlier model stays (the `replacedModelId` rule, `replacedModelFor()`, never names a model of the library: only an unchecked or partial download is deleted as a leftover). **Switch**: the event `use` (`installed`, `paused`, `failed`) starts a job of `USE_JOB_STEPS` (`verify`, `connect`) for a library entry, carrying its stored digest in `model.digest`; `stepVerify` compares that digest BEFORE it loads anything and again after the check, and a different or missing one fails with `model-changed`. A switch has no `model` step, so the leftovers of the job it replaced are deleted at the start of `verify` (`dropReplacedModel()`). Its coding agent is the entry's own when that is installed now, else the state's (`recheckCliInstalled()`), else a 400: a job that stopped while it installed another agent leaves the state naming one that is not there. **Delete** (`deleteLibraryModel()`) moves no state, so it is not an event; whether a state allows it is the `delete-model` action of its `INSTALL_STATE_TABLE` row. It takes a model NAME (the request schema and the service both hold it to `LOCAL_MODEL_ID_PATTERN`), requires that name in the server's own listing (the library or the runtime's tags), goes through the runtime's `/api/delete` only (never a file delete, never a path), and trusts only the runtime's answer that the model is gone, never an empty listing. ⚠️ It refuses the model of the installation or of the current job (`model-active`), a running job, and a runtime that is down or does not answer (`runtime-not-running`).

⚠️ `install` from `installed` and `use` are refused with 409 `sessions-open` while live sessions use the endpoint, like Stop: both change which model the runtime holds and often restart it for another context. ⚠️ While a job that started from `installed` is unfinished (`installing`, `paused`, `failed`) the launch check refuses every start with `job-unfinished` (a code only a refused start says, like `runtime-not-running`), because it wants the state `installed`; the view then offers `use` on the other checked models, which is the way back. A switch is never offered for the model of the stopped job itself: Resume goes on with that one. ⚠️ Both verbs undo a Stop (`undoStop()`, `runtime.enabled` back to true) once their job started: the job runs the runtime, and a model that is in use and running must also start with the server again.

**The view's two listings.** `install.library` (`buildLibraryRows()`, pure) joins the state's entries with what the RUNNING runtime lists (`/api/tags`): `ready`, `changed` (another digest under the name), `missing` (no longer listed) for a checked model, `unverified` for anything else the runtime lists (a model an earlier Remove kept, the model of a job that has not passed its check, one another program pulled through the unauthenticated port), which can be deleted and never switched to. A name that is not a plain `name:tag` is listed with no action. `install.store` is `measureModelStore()`: the sizes of the regular files directly in `models/blobs`, nothing followed, nothing outside the root. ⚠️ Neither is asked for on the one-second poll: `tagsCache` (5 s) and `storeCache` (30 s) are refreshed in the background, only a figure that was never read is waited for (300 ms at most), a runtime that does not answer keeps the last list (or every checked model would read as `missing` for a moment), and every verb and finished step that changes the store throws both away (`forgetStore()`). ⚠️ In `idle` there is no runtime to ask, so the list is empty and only `store` says that an earlier Remove kept models; `remove` is listed there only while it holds something, and `DELETE /api/local-model { removeModel: true }` is what frees it. `Candidate.library` (`active`, `ready`) is added by the ROUTE (`markLibraryCandidates()`), never by `recommend()`, which stays pure and knows nothing of what is installed.

**Developer tools**: `checkDeveloperTools()` runs ONLY `/usr/bin/xcode-select -p`, bounded, on darwin. ⚠️ Never `git`, `python3` or `xcrun` from the server: on a Mac without the tools each opens a system window. The view carries `developerTools`, and the window prints the one sentence when it is `missing`.

**The endpoint store's writer** (`src/custom-model-hosts-writer.ts`): `mutateCustomModelHosts(fn)` is the shape of `mutateSettingsFile()`: one promise chain, a random temp name, a refusal (409) to write over a file that does not parse. ⚠️ EVERY writer of `custom-model-hosts.json` goes through it: the four endpoint routes, the sweep and the connect step (`test/custom-model-hosts-writer.test.ts` fails a `writeCustomModelHosts()` call anywhere else in `src/`). The routes used to read, change and write on their own through one shared temp name, so two creates sent together lost a record or failed, and a file that did not parse was read as `[]` and replaced by one record, with every other endpoint's API key gone. ⚠️ A route's check runs INSIDE the callback, and discover splices onto the record as it is after the probe, not the copy read before it.

Tests: `test/local-model-install-job.test.ts` (the tables and the reducer, every state with every event), `test/local-model-install-service.test.ts`, `test/local-model-install-e2e.test.ts` (the real modules over the fake runtime), `test/local-model-state-store.test.ts`, `test/local-model-connect.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/cli-install-runner.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`, and the runtime's own `test/local-model-runtime.test.ts`, `test/local-model-ollama-client.test.ts`. The fake runtime is `test/fixtures/fake-ollama.ts`.
Tests: `test/local-model-install-job.test.ts` (the tables and the reducer, every state with every event, the library rows), `test/local-model-install-service.test.ts` (the three library verbs and the two caches included), `test/local-model-install-e2e.test.ts` (the real modules over the fake runtime: install A, add B, switch back, delete B), `test/local-model-state-store.test.ts`, `test/local-model-connect.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/cli-install-runner.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`, and the runtime's own `test/local-model-runtime.test.ts`, `test/local-model-ollama-client.test.ts`. The fake runtime is `test/fixtures/fake-ollama.ts`.

### Approvals Inbox

Expand Down
94 changes: 82 additions & 12 deletions docs/local-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ Codeman can set up a coding model that runs on the Mac the server runs on: no ac
no API key, and the session's prompts and files go to a model on this Mac rather than a
cloud provider (the coding agent itself may still check for updates). One button on the welcome screen opens a window that
shows what fits this Mac, installs the choice and ends on a coding agent session that
runs on it.
runs on it. The same window then lists the models that are on this Mac with what each takes
on disk, adds another one, switches between them and deletes one
([The models on this Mac](#the-models-on-this-mac)).

**Experimental.** The routes and their payloads may change in any release
([versioning-policy.md](versioning-policy.md), "Experimental local model routes"). Apple
Expand Down Expand Up @@ -103,7 +105,9 @@ for a row it does not know. Its answer for each memory size is then pinned in
## What an install does, in order

The person picks a model, presses "Install on this Mac" and confirms the sizes once. The
server then runs one job of five steps. The window polls `GET /api/local-model` once a
server then runs one job of five steps. The same job adds a second model to an installed one
("Add a model"): the runtime is there already, so it downloads the model alone, and the
earlier model stays on disk. The window polls `GET /api/local-model` once a
second; the job runs in the server, so the window can be closed.

| Step | What it does |
Expand Down Expand Up @@ -244,10 +248,13 @@ Switching the setting on also starts that five-minute sweep for every other save
existing custom model launch. Before a coding agent is STARTED on that endpoint the server
checks, in this order:

- no job that changes the model in use is running or stopped: until an install or a switch
ended, nothing is started ("The model on this Mac is being changed and is not ready");
- the runtime child Codeman started is alive (whatever else answers on the port is not
ours);
- the digest the runtime reports for the installed model is the one stored at install time;
- the model asked for is the one that was installed and checked.
- the digest the runtime reports for the model in use is the one stored when it was checked;
- the model asked for is the model in use. Another model that is on this Mac is not started
on the endpoint: switch to it in the window first.

That check runs wherever a new agent process comes to be on the endpoint: a session created
with the model, a model applied to a running session (which restarts its agent), the attach
Expand All @@ -273,22 +280,76 @@ answer back from a file it writes in its own folder. The skill never installs a
answers a confirmation, the runtime still answers one request at a time, and while a worker it
opened is live, Stop and Remove are refused until it is closed.

## The models on this Mac

The model store can hold several models. **One of them is the model in use**: the one the
endpoint record `local-ollama` lists, the runtime is started for and a session can be
started on. The runtime holds one model in memory at a time (`OLLAMA_MAX_LOADED_MODELS=1`),
so this is what the Mac can really do, and it keeps every rule above: one record, one
model, one stored digest per model.

The installed screen lists what is on this Mac, the model in use first:

| A row says | It means |
| ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| In use | The model sessions start on |
| Ready | Codeman downloaded and checked it. **Use** switches to it |
| Not checked | The runtime lists it and Codeman never checked it: a model an earlier Remove kept, or one another program pulled through the runtime's port. It can only be deleted |
| Changed | It was checked, and the runtime now reports another digest under its name. It is never started; delete it and install it again |
| Missing | It was checked, and the runtime no longer lists it. **Delete** drops the entry |

Each row shows the context the model was checked with and its size on disk, as the runtime
reports it. Under the list is what all downloads take together: the size of the files in
`local-model/models/blobs`, so layers two models share are counted once and an interrupted
download is included. The window asks the runtime for the list at most every five seconds
and measures the store at most every thirty, never on every poll.

- **Add a model** opens the chooser. A model that is not on this Mac is installed like the
first one, with its sizes confirmed; when the job ends it is the model in use, and the
earlier model stays in the list. A row that is on this Mac already says so and is switched
to, without a download and without a confirm screen.
- **Use** switches to a checked model. It is a job of two steps, `verify` and `connect`:
the runtime is restarted with the context the model was checked with, the model is loaded
and checked again (the tiny chat, the GPU, the tool probe), its digest must still be the
stored one (else the switch fails with `model-changed` before the model is loaded), and
the endpoint record is rewritten. The coding agent is the one the model was checked with
when it is still installed, else the one of the installation.
- **Delete** removes one model through the runtime's own delete, after one question in the
row. The request carries the model's name, never a path, and the server deletes only a
name of its own listing. The model in use cannot be deleted: switch first, or Remove. It
needs the running runtime, so a stopped model shows no Delete until Start.

Adding a model and switching are refused while a session uses the model ("Sessions are
still using this model"), like Stop: their coding agents were started with the other model's
name, and the runtime would swap models on every request.

While a job that started from an installed model runs, or after it stopped (paused or
failed), no session is started: the checks above want a finished installation, and the
refusal says so (`job-unfinished`). The window
then lists the checked models with **Use**, which is the way back to the earlier one. Going
back deletes what the stopped job left of a model that never passed its check, the way a new
install does; a checked model is never deleted that way.

## Stop, Start, Remove

- **Stop** stops the runtime and keeps it from starting at the next boot. **Start** undoes
that.
that, and so does a switch to another model or the install of one: the job starts the
runtime, and the model it ends on is in use.
- **Remove** stops the runtime, removes the endpoint record, switches the setting off again
only when this feature switched it on and no other endpoint is saved, and deletes the
runtime and its private home. The downloaded model is deleted only when asked. The coding
agent stays. While a live session uses the model, Remove answers with the session names
instead.
runtime and its private home. The downloaded models (all of them) are deleted only when
asked. The coding agent stays. While a live session uses the model, Remove answers with
the session names instead.
- **Models a Remove kept** are not lost space: with nothing installed the window says how
much they still take and offers "Delete them", and after a new install they show in the
list as "Not checked", each with its own Delete.

## Where everything is

All under the data directory (`~/.codeman`, or the instance's own):

```
local-model.json the state file (mode 0600)
local-model.json the state file (mode 0600), with the list of checked models
local-model/
runtime/ollama-0.40.15.tgz.partial the runtime archive while it downloads
runtime/ollama-0.40.15/ the unpacked runtime (about 579 MB)
Expand All @@ -313,9 +374,12 @@ in App Settings.
| ------------ | ------------------------------------------------------------- | ---------------- | -------------------------------------- |
| `idle` | stays idle | refused | install |
| `installing` | becomes `paused` at the same step | becomes `paused` | cancel |
| `paused` | stays paused | refused | resume, install, remove |
| `failed` | stays failed, same error | refused | resume, install, remove |
| `installed` | stays installed; the runtime is started unless it was stopped | refused | start a session, stop or start, remove |
| `paused` | stays paused | refused | resume, install, use, delete a model, remove |
| `failed` | stays failed, same error | refused | resume, install, use, delete a model, remove |
| `installed` | stays installed; the runtime is started unless it was stopped | refused | start a session, stop or start, install (add a model), use, delete a model, remove |

`use` and "delete a model" are listed only while a row of the list offers them, and in
`idle` "remove" is offered only while the model store still holds something.

One job at a time. A job a server restart interrupted comes back paused and never continues
without a click. Cancel interrupts the step in flight (the download, the pull, the request,
Expand Down Expand Up @@ -357,6 +421,12 @@ or `python3`.
on a Mac. "Recommended" appears only for the pair we ran, on the Mac we ran it on.
- **The model stays in memory for ten minutes** after the last request, then the runtime
unloads it.
- **One model runs at a time.** Several can be on disk; the runtime holds one, and a switch
restarts it and loads the other model (seconds to a minute). Close the sessions on the
model first: the window refuses the switch while one is open.
- **The disk figures are the runtime's.** A row's size is what the runtime reports for the
model, and the total is the size of its blob files. Neither was compared with Finder on a
Mac yet.

## Keeping the catalogue current

Expand Down
5 changes: 5 additions & 0 deletions docs/wiki/Custom-Model-Endpoints.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,11 @@ Good to know:
but it is changed and removed in the local model window, not here.
- Codeman starts the runtime with the server and stops it with it. **Stop** keeps it from
starting next time.
- More than one model can be on the Mac, and one of them runs at a time. Once a model is
installed the window lists the models with what each takes on disk: **Add a model**
downloads another one, **Use** switches to a model that is already there (nothing is
downloaded; close the sessions on the model first), and **Delete** removes one, after one
question. The model in use cannot be deleted: switch first, or use **Remove**.
- **Remove** deletes the runtime and the endpoint, and the model too if you tick that. The
coding agent stays installed.
- If the model on disk is no longer the one that was installed, Codeman refuses to start a
Expand Down
Loading
Loading