You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
pd: a request larger than the page pool blocks the queue and spins a core #169
submit checks only max_context. If pages < max_context/page_size, a request that needs more pages than the pool can never be admitted, FIFO admission stops everything behind it, and the owner loop spins because the batcher never goes idle.
Evidence and reproduction
opt_prime/opt_prime/accel/pd/batching.py:177-179, pd_batching.py:363-365, paged.py:281. Ran: pages=2 with a 48-token request; neither it nor a small request behind it completed.
Environment
optimus-rm @253be20; Python 3.10, torch 2.10.0+cu128, transformers 5.1.0; RNGD+A100 host (CPU / one A100 for the checks).
Scope
PagedKV construction.
Done when
An undersized pool is rejected at construction; a test covers it.
Activity
fix: reject a KV page pool smaller than one max_context request (#169) (
gspark-etri commentedon Oct 23, 2026
Fixed by #186, squash-merged into optimus-rm.