Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -1766,7 +1766,7 @@ A probe that fails is an answer, not an error: a server that is not on macOS, or
- `verdict`: `ok`, `limited` or `none`. `limited` is a statement about the Mac: no model with 12 billion parameters or more fits its memory, so what it can run handles small edits and questions and does not replace a cloud model. It is judged on every model that fits, before the free disk space is looked at and whatever the pick is. With `ok` the pick can still be a small model (a larger one may be slow, have no speed figure or have no room on the disk); `notes` then says so.
- `reason`, present only with `none`: `not-macos`, `intel`, `memory`, `disk`, `macos` (too old) or `no-harness` (no agent CLI that can be connected to a local model is installed or installable). `memory` has three sentences: nothing fits at all (slot `memory`, the size the Mac is sold as), a model fits but only below the context length an agent CLI needs (slots `budget` and `need`, so a Mac that misses by a little reads as that), and the memory size could not be read (no slot).
- `headline` and `notes[]`: `{ text, vars? }`. `text` is an English sentence that may hold `{placeholder}` slots, and `vars` carries the slot values as text. Notes, in order: the pick is a small model although larger ones fit (verdict `ok` only); a model that fits the memory was left out because its download would not leave 10 GiB free on the disk (slots `model` and `missing`: the best-ranked such model and the GB to free up); nobody tested the pick on a Mac like this one.
- `pick` (absent with `none`) and `alternatives[]` (at most four, best first; empty with `none`), each one model at one context length with one agent CLI. The order: a pair tested on a Mac like this one first, then speed (`ok` before `unknown` before `slow`), then the model's rank, the agent CLI's rank and the larger context. The memory fit decides whether a row is listed and what it says (`fit`, `tag`), it does not sort, so a model is offered at the largest context it fits at. A model whose download would leave less than 10 GiB free on the disk is not listed (an unknown free disk space excludes nothing):
- `pick` (absent with `none`) and `alternatives[]` (every other model of the catalogue that fits, best first; empty with `none`), each one model at one context length with one agent CLI. An install is accepted only for a row of this list. The order: a pair tested on a Mac like this one first, then speed (`ok` before `unknown` before `slow`), then the model's rank, the agent CLI's rank and the larger context. The memory fit decides whether a row is listed and what it says (`fit`, `tag`), it does not sort, so a model is offered at the largest context it fits at. A model whose download would leave less than 10 GiB free on the disk is not listed (an unknown free disk space excludes nothing):
- `modelId` (`name:tag`), `modelLabel`, `license`, `paramsB` (parameters in billions), `context` (tokens);
- `cli`, `cliLabel`, `cliInstalled` (`false`: the CLI is not on this machine yet but can be installed without administrator rights), `slowFirstTurn?`;
- `needBytes` and `needGb` (memory the model needs at that context, at its PEAK: the weights, what stays held per token of context, what the runner holds beside both, and what it takes for a moment while a long prompt is processed), `needSource` (`measured`: the figures were measured for this model on a Mac; `estimated`: they are another model's), `downloadBytes` and `downloadGb`, `budgetShare` (need divided by `budgetBytes`, rounded to three decimals);
Expand Down
77 changes: 50 additions & 27 deletions docs/local-models.md

Large diffs are not rendered by default.

224 changes: 199 additions & 25 deletions src/config/local-model-catalog.ts

Large diffs are not rendered by default.

16 changes: 11 additions & 5 deletions src/local-model/recommend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,10 @@
* gated, installed or installable, and accepting the model id.
* - Ranking, lexicographic: tested on a Mac like this before not tested (a measurement beats
* a formula); ok before unknown before slow; the higher model rank; the higher harness
* rank; the larger context. The pick is the first candidate, the alternatives are the next
* four MODELS, each at its own best context and harness, so a model is always offered at
* the largest context it fits at.
* rank; the larger context. The pick is the first candidate, the alternatives are EVERY
* other model that fits (`MAX_ALTERNATIVES` is above what the catalogue holds), each at its
* own best context and harness, so a model is always offered at the largest context it
* fits at.
* - Disk: a model whose download would leave less than 10 GiB free is not offered. It is not
* dropped in silence: the best-ranked model left out that way is named in `noteDiskDropped`
* with what is missing.
Expand Down Expand Up @@ -109,8 +110,13 @@ export const SLOW_BELOW_TOKENS_PER_SECOND = 10;
export const LIMITED_BELOW_PARAMS_B = 12;
/** What must stay free on the disk after the model download. */
export const DISK_HEADROOM_BYTES = 10 * GIB;
/** How many other models are listed under the pick. */
export const MAX_ALTERNATIVES = 4;
/**
* How many other models are listed under the pick, at most. Above what the stock catalogue
* holds (11 models: 10 beside the pick), so a Mac is shown every model that fits it and the
* install accepts every one of them. Still a bound: test/local-model-catalog.test.ts fails for
* a catalogue that outgrows it, so nothing drops out of the list unseen.
*/
export const MAX_ALTERNATIVES = 11;

/** A memory size of at least this many GB (as sold) gets 3/4 of its memory as device limit, below it 2/3. */
const LARGE_LIMIT_FROM_GB = 36;
Expand Down
8 changes: 4 additions & 4 deletions src/types/local-model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -577,10 +577,10 @@ export interface Recommendation {
/** Present exactly when `verdict` is `ok` or `limited`. */
pick?: Candidate;
/**
* The next models that fit and have room on the disk (at most four), best first: tested
* before not tested, then ok before unknown before slow speed, then the higher model rank.
* Each is at the largest context it fits at, with the best harness that can run it there.
* Empty when `verdict` is `none`.
* Every other model that fits and has room on the disk (`MAX_ALTERNATIVES` is above what the
* catalogue holds), best first: tested before not tested, then ok before unknown before
* slow speed, then the higher model rank. Each is at the largest context it fits at, with
* the best harness that can run it there. Empty when `verdict` is `none`.
*/
alternatives: Candidate[];
/** `DeviceBudget.budgetBytes`. 0 when no budget is computed: not macOS, or a Mac without Apple silicon. */
Expand Down
Loading
Loading