Skip to content

feat: connect four-model RNGD producers to live ATOM HTTP PD - #235

Draft
gspark-etri wants to merge 8 commits into
exp/227-semantic-matrixfrom
feat/234-live-model-pd
Draft

gspark-etri wants to merge 8 commits into
exp/227-semantic-matrixfrom
feat/234-live-model-pd

Conversation

@gspark-etri

@gspark-etri gspark-etri commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Background

Offline RNGD KV export checks did not prove that requests could trigger actual prefill and continue through the ATOM HTTP scheduler. The existing worker was Qwen-specific; Gemma was a one-shot sweep and KONI additionally needed a single-block scheduler path.

Changes

  • Reuse callable Gemma computation in both its sweep and the live worker; add the validated partitioned KONI adapter and an explicit experimental long-Qwen factory.
  • Keep the existing SubprocessPrefillWorker shared-memory/JSON protocol. Factories are trusted local CLI code, never request data. Infer Llama head_dim only when absent; preserve explicit Qwen/Gemma geometry.
  • Add actual worker and HTTP validators using input/EOS fixtures and native token baselines. The HTTP validator never loads saved KV and covers serial, four queued clients, cancellation/disconnection and reuse.

Verification

  • Implementation3855050/74d0d3b; long evidence04dab7c; core scheduler dependencies feat: support fresh hybrid KV imports in ATOM PD scheduler #232 (1e54786) and feat: support cache-off ATOM PD with one KV block #237 (df02223), SIGPIPE-safe HTTP fix: avoid native SIGPIPE handlers on HTTP disconnect #215 (9e442cd), corrected Gemma artifact experiment: isolate Gemma native global-prefill causality discrepancy #225/upstream#827.
  • Four models at short, medium and long lengths on real RNGD->ATOM:14normal/concurrent/reuse responses per model/condition match native tokens through EOS,168total;192remote prefills including cancellation requests,0local fallbacks. Final lane ownership clean. Gemma/Qwen B1/context9216/k3; KONI uses its existing B1/context9216/k1 artifact. Prompt lengths23–143,1526–1643 and6026–6428; compiled widths128/512/2048/8192. Concurrency queues on B1, not hardware batch4.
  • Gemma callable refactor also matches4/4 original full K/V/logit exports; actual worker5/5 K/V/token matches plus oversized-request recovery.
  • CPU adapter/geometry/layer/single-block-helper suite28passed plus5subtests; scoped Ruff/diff/sensitive checks pass. Separate scheduler suite in#237:43passed. Failed initial geometry/factory/PD-capacity checks are retained in experiment records.
  • SDKs: producer furiosa-torch2026.4.0, torch2.10.0+cpu, transformers5.19.0; decoder optimum-rbln0.11.0.post1, rebel-compiler0.11.0, torch2.11.0+cpu, transformers5.8.1. Pinned model revisions, exact commands and JSON in serving_matrix/experiments and evidence234.

Compatibility and limits

Draft, part of#234. Stacked on#228 and includes the existing#200/#199 worker dependency. Shared SDK installations and communication libraries are unchanged; candidate core/HTTP modules are loaded process-locally for hardware tests. Full repository sensitive scan retains seven unchanged baseline documentation findings; changed files are clean.

Short/medium/long fresh PD is demonstrated for all four models. Prefill-time hardware cancellation/worker-failure recovery, cache restoration, hardware batch>1, cross-host live RNGD-to-GPU, actual transfer/compute overlap and controlled latency/throughput remain open. GPU-to-GPU live HTTP and numerical controls are recorded separately under#240/#241. Recorded latencies are observations, not speedup evidence. Native-output parity does not certify answer quality; Gemma numerical mismatch#233 remains unresolved.

Related Issue

Part of#234; related#199/#201/#227/#231/#233/#236. No overall-goal completion claim.

@gspark-etri gspark-etri changed the title feat: connect callable Gemma prefill to the live subprocess worker feat: connect four-model RNGD producers to live ATOM HTTP PD Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant