Skip to content

test: validate four ATOM models over HTTP with device and host cache - #216

Draft
gspark-etri wants to merge 2 commits into
exp/211-http-matrixfrom
exp/211-atom-http
Draft

gspark-etri wants to merge 2 commits into
exp/211-http-matrixfrom
exp/211-atom-http

Conversation

@gspark-etri

Copy link
Copy Markdown
Collaborator

Background

The GPU HTTP matrix did not establish ATOM request lifecycle or cache behavior. Extend the same production-factory harness to the four selected models: Qwen3-0.6B, Qwen3-4B, KONI-8B and Gemma3-1B.

Changes

Add an isolated ATOM backend option, artifact KV-block configuration and explicit host-cache checks. Publish measured results and limitations. Stacked on #213; engine prerequisite #210. The two fixed evidence files include the independent HTTP disconnect fix in #215.

Verification

  • ATOM cache-off: all four models pass concurrency4/8, explicit cancellation, disconnect/drain/reuse, invalid-input and readiness checks; 48/48 concurrent outputs match each model's serial baseline.
  • Both Qwen models: device-cache and host-cache cases each pass12/12 concurrent equality. Device-only prefix hits19584; host runs prefix hits33792 and host hits16128 (0.6B)/8064 (4B).
  • Inputs128/2048/8192, output32, one B1 lane, context9216. Clients are queued, not hardware batching. Synthetic fixed-output fixtures, not semantic-quality tests.
  • Engine21f2c87, harness16f58f7 extended by3ea0493; ATOM SDK0.11.0, optimum-rbln0.11.0.post1, torch2.11.0+cpu. Full commands/revisions/method in opt_prime/examples/pd/http_matrix/experiments/2026-10-23-atom-http211.md and JSON in evidence/.
  • Ruff, Python compile, diff and sensitive-data checks pass. Hardware results exercise the new branches.

Compatibility and limits

GPU remains the default. ATOM precision comes from its artifact. Gemma uses corrected causal-prefill graphs and #210; hybrid prefix/host/PD restoration remains unsupported. KONI uses one KV block because the three-block graph does not compile. No general throughput claim from single bursts, no complete RNGD/heterogeneous validation, and no real compute/transfer overlap claim.

Related Issue

Related to #211, #201, #207 and #208. Full matrix remains ongoing.

AI-assisted implementation and verification with Codex; maintainer review and merge required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant