platforms: NPU graph cache compiles a new graph for every prefill length above the largest bucket #171

Problem

_bucket returns n itself when T exceeds the largest bucket (256 RNGD, 64 ATOM). Prefill rows reach run() unchunked, so every new prompt length compiles a graph (one per expert in baked mode) and _graphs grows without bound.

Evidence and reproduction

opt_prime/opt_prime/accel/platforms/npu_graph.py:77-80,241-269. Ran with a stub backend: lengths 300-303 gave 4 compiles in inputs mode and 32 in baked mode.

Environment

optimus-rm @253be20; Python 3.10, torch 2.10.0+cu128, transformers 5.1.0; RNGD+A100 host (CPU / one A100 for the checks).

Scope

npu_graph.py.

Done when

run() chunks rows to the largest bucket; a test feeds T above it and sees no new compiles.

Owner

@gspark-etri