[gspark-etri]

Background

The HTTP validation, benchmark and held-out quality scripts for PD serving were developed on several experiment branches (#243, #252, #264) together with evidence, status files and forked runtime modules. This PR lands only the reusable harness on optimus-rm (which already has the serving flags from #281), so it can be run against the in-tree serving stack and its pure-Python parts are covered by CI.

Changes

New directory opt_prime/examples/pd/http_matrix/:

Path	Source
validate.py	PR #243 head (exp/240-gpu-live-pd), opt_prime/examples/pd/http_matrix/validate.py, unchanged
benchmark.py, benchmark_server.py	PR #252 head (exp/248-heterogeneous-http), same paths, unchanged
atom_server.py	PR #252 head, same path; --batching, --http-module, --pd-module are now optional (default: the in-tree modules) instead of required
opt_prime/tests/test_benchmark.py	PR #252 http_matrix/test_benchmark.py, moved; imports benchmark.py by file path so CI collects it; the CLI test writes into tmp_path
heldout_quality.py	PR #264 head (exp/263-heldout-quality), same path; run now reads .json.gz fixtures directly and takes --model to set the request model name (input IDs and hashes unchanged)
opt_prime/tests/test_heldout_quality.py	PR #264, same path; added a manifest/hash check of the frozen fixtures and a gzip + --model runner test
fixtures/quality263-{gemma,koni,qwen06b,qwen4b}-fixtures.json.gz, fixtures/fixture-manifest.json	PR #264 http_matrix/evidence263/, moved. The Qwen3-0.6B file's model field held a local HF cache path; it now reads Qwen/Qwen3-0.6B, and the manifest SHA-256 for that file is updated (ed8adc8f... -> 5fd8133a...). Token IDs and per-case input_sha256 are unchanged.
README.md	New, short: what each script does and how to run it (no results)
.gitignore	Ignores evidence*/, *.json (except fixtures/fixture-manifest.json), *.jsonl, *.log

Not included: all evidence*/ outputs, STATUS.md, experiments/, .research/, diagnose_*, trace_*, forced_prefix_*, live_gpu_faults.py, atom_pipeline.py, and any forked copy of accel/serve/llmd.py.

Every import was checked against optimus-rm (4dd30cf): opt_prime.accel.serve.llmd.serve_stage, opt_prime.accel.serve.batched.{BatchedOwner,BatchedService}, opt_prime.accel.pd.pd_batching.SubprocessPrefillWorker, opt_prime.accel.platforms.atom.engine.AtomRblnEngine and examples/pd/serve_batched.build exist, and the SimpleNamespace in validate.py covers every attribute build reads. optimum-rbln (ATOM) and transformers are imported lazily inside main() where they are only needed at run time.

Verification

cd opt_prime && TORCH_DEVICE_BACKEND_AUTOLOAD=0 CUDA_VISIBLE_DEVICES=1 PYTHONPATH=$PWD \
  python3 -m pytest -q tests/test_benchmark.py tests/test_heldout_quality.py
15 passed in 1.12s

• python3 examples/pd/http_matrix/atom_server.py --help runs on a host without ATOM.
• Fixture hashes: SHA-256 of each decompressed fixture matches fixture-manifest.json, and every case's input_sha256 matches its token IDs (checked by the new test).
• No GPU/NPU runs in this PR (the scripts are moved, not changed in behaviour); full suite: CI.

Compatibility and limits

• Examples and tests only; no library code changes.
• validate.py, benchmark.py need CUDA GPUs (benchmark.py --mode pd/dp2 assumes cuda:0 and cuda:1); atom_server.py needs optimum-rbln and exactly one visible /dev/rbln* device.
• Models load with local_files_only=True.
• The fixtures record the model path used when they were frozen; the stage server rejects a mismatched model, so run needs --model <served name>.
• heldout_quality.py measures structured-answer adherence on 36 synthetic cases per model, not general quality.

Related Issue

Closes #240, Closes #263
Refs #211, #248
Supersedes the harness parts of #213, #216, #243, #251, #252, #264 (those PRs are left open and unchanged).

🤖 Generated with Claude Code