Skip to content

Runs and details on results (#130 step 4, light version) - #144

Merged
jackwsmth merged 1 commit into
mainfrom
issue-130-runs
Nov 18, 2026
Merged

jackwsmth merged 1 commit into
mainfrom
issue-130-runs

Conversation

@jackwsmth

Copy link
Copy Markdown
Collaborator

Step 4 of #130, the light version agreed on: runs go on top of the results table instead of a second table. Stacked on #143 (the protocol), so merge that one first. Until then, its commit also shows here.

What changes

Database (migration `20261118120000_result_runs`)

  • New optional columns: `model_repo`, `model_file`, `model_sha256` (lowercase hex), `decode_tps_min` and `decode_tps_max` (checked against `decode_tps`). The browser can write them, and editing one resets verification, the same as other measured fields.
  • `run_id uuid`, shared by the results of one run. It is not granted to `authenticated`. Only two paths set it: PR ingestion, and a definer helper (`private.assign_result_run`) that only the API dispatcher can execute. Both use a server-generated id, so nobody can attach a result to someone else's run.
  • `private.pr_result_imports.run_id`, so a rerun reports every result id.
  • `ingest_pr_results` and `agent_dispatch` are redefined. The diff against `20261113190000` is just the points loop, the new columns and `same_run`.

Result files

  • A file can carry `points` (1–64). Each point holds only the fields that change between points (`decodeTps`, min/max, `promptTps`, `ttftMs`, `inputTokens`, `outputTokens`, `concurrency`, `decodeScope`), and everything else is written once at the top. Each point becomes its own result, so each ranks in its own setup.
  • The parser reads each point as a full result. A problem every point shares is named once; any other problem names its point. It also rejects measured numbers at the top level, unknown keys inside a point, and duplicate points.
  • The PR comment lists every result a points file created.

API: `POST /results/batch` accepts `sameRun: true`. Results carry `runId`, and the five detail fields round-trip.

UI: the submit form has slowest/fastest run fields and model repository/file/SHA-256 fields. The result card shows the range under the headline number and the model file under Model. The legacy PR preview prefills from the first point and says how many points the file holds.

Docs: README (field table, One run, several points, and the protocol now uses the fields instead of `notes`), schema.json, agent API intro and agent prompt.

Tests

  • New `result-runs.test.mjs`: the parser, ingestion of points (dry run, import, rerun), browser grants (`run_id` is refused), constraints, and the verification reset.
  • New API test: `sameRun` shares an id, batches without it get none, and `runId` is rejected in a request body.
  • `results:test` 99/99, `agents:test` 34/34, typecheck, build and the agent docs check all pass locally.

Rollout

After merge: `supabase db push` for `20261118120000`. No backfill is needed; existing rows keep null. The rig-page curve that uses `run_id` and `inputTokens` comes in a follow-up PR.

No second table: a run is results sharing a run_id.

- Migration 20261118120000: model_repo, model_file, model_sha256,
  decode_tps_min, decode_tps_max (browser-writable, reset verification)
  and run_id (set only by PR ingestion and the API batch route).
- Result files may hold a points list: one result per point, sharing the
  file's other fields and a run id. The parser reads each point as a full
  result and names shared problems once.
- POST /results/batch takes sameRun to put its items in one run.
- Submit form, result card, schema, README, agent docs and prompt.
@vercel

vercel Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
intelinside Ready Ready Preview Nov 18, 2026 2:43pm UTC

Request Review

@github-actions

Copy link
Copy Markdown

✅ The catalog validates: 85 hardware, 54 models, 29 quants, 13 runtimes.

Once merged, the entry is synced to the database automatically and shows up on the site with the next deploy. Format notes are in frontend/src/catalog/README.md.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
No submission files to import.

Site / sign up · Workflow details and retry

@jackwsmth
jackwsmth merged commit e1e6976 into main Nov 18, 2026
7 checks passed

This branch was successfully deployed

1 active deployment
Preview — 5a5df874 Deployed Nov 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant