Repository navigation
test: validate four-model live GPU PD and numerical controls - #243
Closed
gspark-etri wants to merge 1 commit into
Closed
gspark-etri wants to merge 1 commit into
gspark-etri wants to merge 1 commit into
Conversation
This was referenced Oct 23, 2026
gspark-etri
added a commit
that referenced
this pull request
Oct 30, 2026
…248, #263) (#286) Add opt_prime/examples/pd/http_matrix with the reusable parts of the HTTP work from #243, #252 and #264: - validate.py: HTTP lifecycle validation through serve_batched.build - benchmark.py, benchmark_server.py, atom_server.py: warm native/dp2/pd HTTP benchmark with token parity checks and external-server configs - heldout_quality.py and frozen fixtures for four models (fixtures/) - tests moved to opt_prime/tests so CI collects them Adjustments: atom_server.py module overrides are optional (default to the in-tree modules); heldout_quality.py reads .json.gz fixtures and accepts --model for the request name; the Qwen3-0.6B fixture records the Hub name instead of a local cache path (manifest hash updated; input IDs and per-case hashes unchanged). Run outputs are git-ignored. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Landed on optimus-rm via #286. Closing; the branch is archived locally (experiment records and evidence included). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Standalone GPU and offline KV handoff checks did not establish live GPU PD serving across four models, long inputs and concurrent clients. Producer cache/chunk defaults also differed from native execution.
Changes
Extend the existing HTTP validator with explicit decoder/producer devices, producer cache/chunk policy, actual producer metadata and remote-prefill counter checks. Add a trusted local HTTP-candidate option loaded before consumers. Preserve the failed import-order run and all original numerical divergences alongside successful controls.
Verification
Two A10080GB PCIe; Torch2.13.0+cu130, Transformers5.15.0. Core369c8ff, factory controlsfa753e1 (#242), HTTP9e442cd. Qwen3-0.6B,Qwen3-4B,KONI8B,Gemma3-1B;128/2048/8192input tokens,32outputs ignoringEOS,decode slots4,concurrency4/8; cancellation/disconnect/reuse and invalid-input checks.
Compatibility and limits
Stacked on #242.
passedmeans lifecycle/count checks passed; token equality is reported independently. No general model-quality, speedup or compute/transfer-overlap claim. Timing observations are not an apples-to-apples performance comparison: default producer cache/chunk settings differ from native. Cross-host live RNGD-to-GPU and remaining broader-goal acceptance stay open. Four unchanged baseline sensitive-scan findings are outside this change.Related Issue
Closes #240; uses #241/#242 and relates to #234.