Skip to content

Cache: WIP (1/5) Add content-addressed local inference caches - #121

Draft
ErlisLushtaku wants to merge 29 commits into
rebase/inference-usage-on-meta-evalfrom
cache-on-118/01-local-store
Draft

Cache: WIP (1/5) Add content-addressed local inference caches#121
ErlisLushtaku wants to merge 29 commits into
rebase/inference-usage-on-meta-evalfrom
cache-on-118/01-local-store

Conversation

@ErlisLushtaku

@ErlisLushtaku ErlisLushtaku commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Description

Note

WIP: replacement cache stack on top of #118. Not final and may still change.

Adds content-addressed local SQLite caches for model completions and judge outputs.

Cache layout:

{store_root}/completions/{task}/{provider}/{model}/{descriptor_hash}/
├── metadata.json
└── completions.db

{store_root}/judgements/{task}/{provider}/{model}/{descriptor_hash}/
├── metadata.json
└── judgements.db

metadata.json stores the validated model descriptor. The databases use these schemas:

completions(
  input_hash TEXT PRIMARY KEY,
  input_text TEXT NOT NULL,
  completion TEXT NOT NULL,
  benchmark TEXT NOT NULL,
  instruction_id TEXT NOT NULL,
  model TEXT NOT NULL,
  pushed_by TEXT NOT NULL,
  pushed_at TEXT NOT NULL,
  run_id TEXT NOT NULL
)

judgements(
  input_hash TEXT PRIMARY KEY,
  judge_input TEXT NOT NULL,
  judge_completion TEXT NOT NULL,
  benchmark TEXT NOT NULL,
  instruction_id TEXT NOT NULL,
  model_a TEXT NOT NULL,
  model_b TEXT NOT NULL,
  judge TEXT NOT NULL,
  top_logprobs TEXT,
  orientation TEXT,
  pushed_by TEXT NOT NULL,
  pushed_at TEXT NOT NULL,
  run_id TEXT NOT NULL
)

top_logprobs is new relative to the earlier draft so judgements can restore the first-token logprobs used by #118 parsers.

This is stacked on #118.

kargibora and others added 29 commits September 4, 2026 13:41
Move private scoring helpers to their configured metrics and remove redundant state and transformations. Clarify parse-failure reporting without changing metric policy.

Replace Elo-gap hash priorities with seeded permutations as approved. Exact Elo-gap subsets may change for the same seed; preserve shared hard/soft samples, nested budgets, ID types, and the judged-panel sampler.
Keep completions and judgements in separate role namespaces with descriptor-only metadata and typed row associations for later inspection.
@ErlisLushtaku ErlisLushtaku changed the title Cache: WIP (1/6) Add content-addressed local inference caches Cache: WIP (1/5) Add content-addressed local inference caches Sep 9, 2026
@kargibora
kargibora force-pushed the rebase/inference-usage-on-meta-eval branch from 58d6791 to d1ca639 Compare September 10, 2026 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants