Skip to content

feat: support fresh hybrid KV imports in ATOM PD scheduler #232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gspark-etri wants to merge 3 commits into from
Closed

feat: support fresh hybrid KV imports in ATOM PD scheduler #232

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c76e03a
feat: support fresh hybrid KV imports in ATOM PD scheduler
gspark-etri Oct 23, 2026
1e54786
fix: reject malformed remote KV without poisoning ATOM lanes
gspark-etri Oct 23, 2026
69bca73
test: verify long hybrid PD imports through ATOM HTTP
gspark-etri Oct 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions opt_prime/examples/pd/http_matrix/STATUS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,11 @@ State: review; full goal remains active. GPU evidence remains as recorded. ATOM
## Production EOS correction (#229)

The common serve_batched factory now unions model generation EOS and tokenizer turn-end EOS. KONI config128001/tokenizer128009 previously continued into role delimiters; real GPU HTTP negative control ends length, fixed/stream/reuse end stop, ignore_eos remains length.22related CPU tests pass after3failed factory regressions. GPU and ATOM hardware both pass5/5 EOS/override/stream/reuse checks; NPU uses existing B1/context9216/k1 artifact. See serving-eos229 record/evidence229.

## Fresh hybrid PD (#231)

Gemma full-prefix KV imports now run through ATOM PD scheduling: 33 CPU tests and 18 real HTTP normal/reuse responses pass, with 20 remote / 0 local prefills including cancellations. Producer data is recorded RNGD KV replay. Live prefill, overlap, hybrid cache restoration and hardware batch >1 remain unverified. SIGPIPE log issue still depends on #214/#215. See hybrid-pd231 experiment and evidence231.

Geometry recovery follow-up: malformed remote heads/KV lengths fail only their request; real device import failures still poison the scheduler.37related tests pass. Hardware evidence remains c76e03a.

Long hardware follow-up on1e54786:6423–6428token inputs,10normal/reuse HTTP responses match nativeATOM;12remote/0local prefills including cancellation. Recorded RNGD KV replay, arithmetic wrong on both native/imported paths; no quality or live-transfer claim.
3 changes: 3 additions & 0 deletions opt_prime/examples/pd/http_matrix/evidence231/events.jsonl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"id": "2e762726f70745af91068ca493f9cdb6", "ts": "2026-10-23T21:56:32+09:00", "task": "Fresh hybrid PD scheduler (aicomp #231)", "result": "33 CPU tests pass; Gemma ATOM HTTP replay18 responses exact,20 remote/0 local prefills; cancellation and disconnect reuse pass. Known SIGPIPE logs remain #214.", "next": "Publish scoped PR; integrate live producer and measure overlap after long-context validation.", "state": "active", "done": false, "blocker": "", "metrics": {}, "params": {"model": "Gemma3-1B", "batch": "1", "worker": "recorded_rngd_kv_replay"}, "artifacts": ["opt_prime/examples/pd/http_matrix/evidence231/gemma-http-final.json"], "experiments": ["opt_prime/examples/pd/http_matrix/experiments/2026-10-23-hybrid-pd231.md"], "labels": ["experiment"], "review": true, "todos": [], "note": "", "agent": "codex", "host": "RNGD+ATOM host", "branch": "main", "commit": "351d39d-dirty"}
{"id": "c5fa00d2033c4c358b49d3d43194e104", "ts": "2026-10-23T22:01:44+09:00", "task": "Fresh hybrid PD scheduler (aicomp #231)", "result": "Reproduced wrong-head/KV-length import poisoning (2failed); request-scoped validation fix passes37tests, real device failures still poison. PR232 follow-up.", "next": "Update PR232 and finish live long-context queue before live heterogeneous PD.", "state": "active", "done": false, "blocker": "", "metrics": {}, "params": {"validation": "cpu_recovery"}, "artifacts": [], "experiments": ["opt_prime/examples/pd/http_matrix/experiments/2026-10-23-hybrid-pd231.md"], "labels": ["bug"], "review": true, "todos": [], "note": "", "agent": "codex", "host": "validation host", "branch": "main", "commit": "351d39d-dirty"}
{"id": "c0b7d30efdee4c99b1cae96743f5f6a0", "ts": "2026-10-23T22:12:20+09:00", "task": "EOS-aware four-model validation (aicomp #227)", "result": "Gemma6.4K GPU3/4,ATOM4/4; same equations GPU BF16/FP32 choose3 while RNGD2; growing layer KV error tracked#233. Long PD HTTP replay10responses/12remote verified on1e54786. Qwen4B PID131672 compiling; idleATOM released.", "next": "Finish Qwen4B semantic GPU/ATOM checks; isolate Gemma per-layer execution drift#233 and integrate live producers.", "state": "active", "done": false, "blocker": "", "metrics": {}, "params": {"model": "Gemma3-1B", "width": "8192"}, "artifacts": [], "experiments": ["opt_prime/examples/pd/http_matrix/experiments/2026-10-23-hybrid-pd231.md"], "labels": ["analysis"], "review": true, "todos": [], "note": "", "agent": "codex", "host": "validation host", "branch": "main", "commit": "351d39d-dirty"}
Loading