Repository navigation

server/llama: honor per-request speculative.n_max, per-sequence snapshot budget, honest rollback refusal (multi-slot decode with MTP) - #208

Open

alenizy361 wants to merge 15 commits into

PrismML-Eng:prismfrom

alenizy361:server-spec-multislot

Open

server/llama: honor per-request speculative.n_max, per-sequence snapshot budget, honest rollback refusal (multi-slot decode with MTP)#208
alenizy361 wants to merge 15 commits into
PrismML-Eng:prismfrom
alenizy361:server-spec-multislot

Commits on Sep 19, 2026
- gdn: in-place cache-writing GDN op for CPU recurrent decode (K == 1 gate)
  
  alenizy361
  and
  claude
  committed
  8a27b21
- gdn: take the in-place path in MTP speculative mode too (K = n_rs_seq + 1)
  
  alenizy361
  and
  claude
  committed
  40e9265
- tests: byte-for-byte check of the cache-writing GDN op vs the legacy paths
  
  alenizy361
  and
  claude
  committed
  8c1b37a
- memory-recurrent: lazily zeroed anonymous-mmap RS buffer on the CPU host
  
  alenizy361
  and
  claude
  committed
  c875ea4
- server: honor the per-request speculative.n_max (0 = no drafting for the request)
  
  alenizy361
  and
  claude
  committed
  89f8a27
- llama: per-sequence rollback snapshot budget (llama_memory_seq_rs_snapshots)
  
  alenizy361
  and
  claude
  committed
  44e4b09
- gdn: derive the snapshot plane count K of a ubatch from the seqs' budgets
  
  alenizy361
  and
  claude
  committed
  7240566
- server: give an opted-out slot a rollback snapshot budget of 0
  
  alenizy361
  and
  claude
  committed
  fd9a0af
- tests: gdn in-place op with a K = 1 rollback read and a T = 1 cross-K check
  
  alenizy361
  and
  claude
  committed
  e4be22f
- server: document the per-request speculative.n_max
  
  alenizy361
  and
  claude
  committed
  80164f7
- memory-recurrent: only roll back onto planes the seq's last ubatch wrote
  
  alenizy361
  and
  claude
  committed
  b6d3cb7
- llama: document the recurrent rollback invariant on llama_memory_seq_rm
  
  alenizy361
  and
  claude
  committed
  0fd8b1d
- tests: roll back only onto planes the last ubatch wrote in test-recurrent-state-rollback
  
  alenizy361
  and
  claude
  committed
  cbb4c55
- examples: rs-rollback decodes the drafts as one verify ubatch so the rollback stays inside it
  
  alenizy361
  and
  claude
  committed
  2d2c3c3
- speculative: no rollback snapshot planes in the draft context
  
  alenizy361
  and
  claude
  committed
  3ed7c43

Commits

Commits on Sep 19, 2026