Skip to content

Rig pages: decode by prompt length (#130 step 4, curves) - #145

Merged
jackwsmth merged 2 commits into
mainfrom
issue-130-context-curves
Nov 18, 2026
Merged

jackwsmth merged 2 commits into
mainfrom
issue-130-context-curves

Conversation

@jackwsmth

Copy link
Copy Markdown
Collaborator

The last piece of the light step 4 in #130. Stacked on #144 (which is stacked on #143), so merge those first. Frontend only, no migration.

  • `lib/streams.ts` groups results by an axis. `streamSeries` (concurrency) behaves as before. The new `contextSeries` takes single-request results with `inputTokens` recorded, grouped by model, quant, runtime, build and part, and keeps the best rate per length. It leaves out speculative runs, so they can't pose as faster points, and legacy `contextLength`, which meant the prompt to some submitters and the window to others.
  • `StreamsChart` draws either axis. Prompt length uses a log scale, so 512 / 4k / 16k / 32k sit evenly apart.
  • The rig page gets a Decode tok/s by prompt length section below the concurrency one, shown only when a setup has two or more lengths.
  • Mock seed: a four-point run at 512 / 4k / 16k / 32k, so the section shows in mock mode.

No prod rig has two prompt lengths for one setup yet, so the section appears once someone submits a sweep (for example a `points` file from #144).

Tests: `streams.test.mjs` covers `contextSeries` (best per length, speculation and multi-request excluded, hidden and legacy ignored, a single length draws nothing). `results:test` 100/100, `agents:test` 34/34, typecheck and build pass. Screenshots were checked in mock mode at desktop and phone widths.

No second table: a run is results sharing a run_id.

- Migration 20261118120000: model_repo, model_file, model_sha256,
  decode_tps_min, decode_tps_max (browser-writable, reset verification)
  and run_id (set only by PR ingestion and the API batch route).
- Result files may hold a points list: one result per point, sharing the
  file's other fields and a run id. The parser reads each point as a full
  result and names shared problems once.
- POST /results/batch takes sameRun to put its items in one run.
- Submit form, result card, schema, README, agent docs and prompt.
streams.ts groups by an axis: concurrency (as before) or prompt length.
The prompt-length curve takes single-request results with inputTokens,
best per length, and leaves out speculative runs. StreamsChart draws
either axis; prompt length uses a log scale. Mock seed gains a four-point
run so the curve shows in mock mode.
@vercel

vercel Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
intelinside Ready Ready Preview Nov 18, 2026 2:45pm UTC

Request Review

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
No submission files to import.

Site / sign up · Workflow details and retry

@github-actions

Copy link
Copy Markdown

✅ The catalog validates: 85 hardware, 54 models, 29 quants, 13 runtimes.

Once merged, the entry is synced to the database automatically and shows up on the site with the next deploy. Format notes are in frontend/src/catalog/README.md.

@jackwsmth
jackwsmth merged commit e04940f into main Nov 18, 2026
7 checks passed

This branch was successfully deployed

1 active deployment
Preview — 3cbe4b50 Deployed Nov 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant