Skip to content

pd: a request larger than the page pool blocks the queue and spins a core #169

Closed

Description

@gspark-etri

Problem

submit checks only max_context. If pages < max_context/page_size, a request that needs more pages than the pool can never be admitted, FIFO admission stops everything behind it, and the owner loop spins because the batcher never goes idle.

Evidence and reproduction

opt_prime/opt_prime/accel/pd/batching.py:177-179, pd_batching.py:363-365, paged.py:281. Ran: pages=2 with a 48-token request; neither it nor a small request behind it completed.

Environment

optimus-rm @253be20; Python 3.10, torch 2.10.0+cu128, transformers 5.1.0; RNGD+A100 host (CPU / one A100 for the checks).

Scope

PagedKV construction.

Done when

An undersized pool is rejected at construction; a test covers it.

Owner

@gspark-etri

Activity

  1. gspark-etri commented on Oct 23, 2026

    @gspark-etri
    CollaboratorAuthor

    Fixed by #186, squash-merged into optimus-rm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions