Skip to content

Cache: WIP (4/5) Adopt inference caching in task runners - #124

Draft
ErlisLushtaku wants to merge 1 commit into
cache-on-118/03-providersfrom
cache-on-118/04-runners
Draft

Cache: WIP (4/5) Adopt inference caching in task runners#124
ErlisLushtaku wants to merge 1 commit into
cache-on-118/03-providersfrom
cache-on-118/04-runners

Conversation

@ErlisLushtaku

@ErlisLushtaku ErlisLushtaku commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Description

Note

WIP: replacement cache stack on top of #118. Not final and may still change.

Adopts the inference cache in the task runners through --run.store_root.

  • Pairwise, meta-eval, MT-Bench, and Elo generation and judging share the do_inference cache.
  • Elo temperature calibration uses the same judgement cache.
  • Cache rows get instruction, model pair, and orientation metadata at the direct inference call.
  • Dataset-provided completions stay as direct inputs when they do not run inference.
  • Removes ignore_cache, cache_function_dataframe, and the old cache tokens.

This is stacked on #123.

Wire store_root through generation and judging paths and drop the legacy dataframe cache helpers.
@ErlisLushtaku
ErlisLushtaku force-pushed the cache-on-118/03-providers branch from df52a6d to e28ee43 Compare September 9, 2026 13:01
@ErlisLushtaku
ErlisLushtaku force-pushed the cache-on-118/04-runners branch from 269ccdf to 4f5d127 Compare September 9, 2026 13:01
@ErlisLushtaku ErlisLushtaku changed the title Cache: WIP (4/6) Adopt inference caching in task runners Cache: WIP (4/5) Adopt inference caching in task runners Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant