exp: validate four-model GPU HTTP lifecycle and prefix-cache precision

opt_prime/examples/pd/http_matrix/.research/events.jsonl

@@ -0,0 +1 @@
+{"id": "b61f99ae005a4258aa7f44d8edeca28d", "ts": "2026-10-23T18:08:44+09:00", "task": "Four-model HTTP matrix (aicomp #211)", "result": "All4 GPU models pass HTTP concurrency4/8, cancel/disconnect/drain/reuse checks. No-cacheBF16 and warm-cacheFP32 each48/48 concurrent outputs exact. Warm-cacheBF16 has31/48 raw equality; actualQwenHTTP tie explains observed branch. Warm-state cancellation isolates KONI cold-cache difference.", "next": "Publish GPUHTTP evidence; continue ATOM HTTP and RNGD routes. Preserve numerical/quality and true-overlap limitations.", "state": "active", "done": false, "blocker": "", "metrics": {}, "params": {"models": "4", "slots": "4", "concurrency": "4,8", "conditions": "bf16_no_cache,bf16_cache,fp32_cache"}, "artifacts": ["evidence/summary.json"], "experiments": ["experiments/2026-10-23-http-matrix211.md"], "labels": ["benchmark"], "review": true, "todos": [], "note": "", "agent": "codex", "host": "GPU host", "branch": "main", "commit": "351d39d-dirty"}

opt_prime/examples/pd/http_matrix/README.md

@@ -0,0 +1,11 @@
+# Four-model GPU HTTP validation (#211/#212)
+
+Run `validate.py --model MODEL --output result.json` under the engine environment with a free CUDA device. `--prefix-cache` enables the separate cache case; `--dtype float32` provides the numerical control. Offline immutable model snapshots are required. The default configuration is four paged execution slots, context9216, page64, prefill chunk256, inputs128/2048/8192 and32 fixed output tokens. This exercises the real HTTP service factory and sockets, not a mocked handler.
+
+`passed` covers HTTP lifecycle and completion counts. It does **not** assert equality of concurrent versus serial output: that is stored per response as `baseline_equal`. Cancellation compares warm-state output immediately before/after, and records cold-baseline equality separately. Cache-enabled runs require positive observed cache hits. Metrics are sampled after drain.
+
+`trace_prefix_http.py` uses the same CLI and records actual2K prefill boundaries and initial logits. `diagnose_prefix.py --model MODEL --output result.json` teacher-forces histories in the direct batcher. The direct probe did not reproduce the HTTP argmax change, so it must not replace actual HTTP evidence. An additional uncommitted allocation-mode probe changed only inference_mode to no_grad; its result is retained as a negative diagnostic in evidence.
+
+See experiments/2026-10-23-http-matrix211.md and evidence/summary.json. Evidence includes the initial failed KONI cold-baseline cancellation assertion. Earlier runs preceded the warm-state cancellation control and some metadata additions; their commands/configurations are recorded in the experiment. Public copies normalize local model-directory labels in metrics only; raw measurements remain in the research workspace.
+
+This GPU experiment does not complete ATOM/RNGD HTTP or PD coverage, long-context RNGD support, hybrid NPU cache restoration, broader semantic quality or true compute/transfer overlap.

opt_prime/examples/pd/http_matrix/STATUS.md

@@ -0,0 +1,3 @@
+# HTTP model matrix
+
+State: review; goal remains active. Four GPU models pass no-cache lifecycle checks and48/48 concurrent equality; BF16 warm cache has raw numerical differences. FP32 warm cache passes lifecycle and48/48 concurrent equality. Actual Qwen HTTP trace identifies a BF16 logit tie. Next: ATOM HTTP configurations and missing RNGD routes; broader semantic inputs and cache/host-tier ownership checks.

opt_prime/examples/pd/http_matrix/diagnose_prefix.py

@@ -0,0 +1,143 @@
+"""Teacher-force equal histories while varying prefix-cache history and precision (#212)."""
+
+import argparse
+import json
+from pathlib import Path
+
+import torch
+from opt_prime.accel.pd import PDRequest, SamplingParams, batching
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+p = argparse.ArgumentParser(description=__doc__)
+p.add_argument("--model", required=True)
+p.add_argument("--output", required=True)
+a = p.parse_args()
+torch.set_num_threads(4)
+torch.backends.cuda.matmul.allow_tf32 = False
+tok = AutoTokenizer.from_pretrained(a.model, local_files_only=True)
+head = [tok.bos_token_id] if tok.bos_token_id is not None else []
+fill = tok.encode(
+    "This is background context. The red box contains apples. The blue box contains books.\n",
+    add_special_tokens=False,
+)
+tail = tok.encode(
+    "\nQuestion: What is the capital of France?\nAnswer:", add_special_tokens=False
+)
+inputs = {
+    n: head
+    + (fill * ((n - len(head) - len(tail) + len(fill) - 1) // len(fill)))[
+        : n - len(head) - len(tail)
+    ]
+    + tail
+    for n in (128, 2048, 8192)
+}
+model = (
+    AutoModelForCausalLM.from_pretrained(
+        a.model, local_files_only=True, dtype=torch.bfloat16
+    )
+    .cuda()
+    .eval()
+)
+results = []
+original = batching.select_token
+try:
+    with torch.inference_mode():
+        for dtype in ("bfloat16", "float32"):
+            model.to(getattr(torch, dtype))
+            oracle = None
+            reference = None
+            for mode, history in [
+                ("cold", []),
+                ("partial", [128]),
+                ("warm", [128, 2048, 8192]),
+                ("warm_concurrent", [128, 2048, 8192]),
+            ]:
+                b = batching.ContinuousBatcher(
+                    model,
+                    slots=4,
+                    max_context=9216,
+                    kv="paged",
+                    page_size=64,
+                    prefill_chunk=256,
+                    prefix_cache=True,
+                )
+                for i, n in enumerate(history):
+                    f = b.submit(PDRequest(f"prime-{i}", tuple(inputs[n]), 32))
+                    b.run_until_idle()
+                    f.result()
+                vectors = []
+                predicted = []
+
+                def sample(
+                    logits,
+                    params,
+                    position,
+                    vectors=vectors,
+                    predicted=predicted,
+                    oracle=oracle,
+                ):
+                    if params.seed != 212:
+                        return original(logits, params, position)
+                    vectors.append(logits.float().cpu().clone())
+                    predicted.append(int(logits.argmax()))
+                    return (
+                        original(logits, params, position)
+                        if oracle is None
+                        else oracle[len(vectors) - 1]
+                    )
+
+                batching.select_token = sample
+                before = b.kv.hit_tokens
+                f = b.submit(
+                    PDRequest(
+                        "target", tuple(inputs[2048]), 32, SamplingParams(seed=212)
+                    )
+                )
+                if mode == "warm_concurrent":
+                    for j, n in enumerate([128, 8192, 128]):
+                        b.submit(PDRequest(f"companion-{j}", tuple(inputs[n]), 32))
+                b.run_until_idle()
+                tokens = f.result()
+                batching.select_token = original
+                values = torch.stack(vectors)
+                assert values.shape[0] == 32 and torch.isfinite(values).all()
+                if oracle is None:
+                    oracle = tokens
+                    reference = values
+                delta = values - reference
+                different = [
+                    i for i, (u, v) in enumerate(zip(predicted, oracle)) if u != v
+                ]
+                rows = []
+                for i in different:
+                    tops = [z[i].topk(3) for z in (reference, values)]
+                    rows.append(
+                        {
+                            "position": i,
+                            "top_ids": [x.indices.tolist() for x in tops],
+                            "top_logits": [x.values.tolist() for x in tops],
+                        }
+                    )
+                result = {
+                    "dtype": dtype,
+                    "oracle": oracle,
+                    "mode": mode,
+                    "cache_hit_tokens": b.kv.hit_tokens - before,
+                    "argmax_disagreements": different,
+                    "max_abs_logit_difference": float(delta.abs().max()),
+                    "relative_rms": float(
+                        delta.square().mean().sqrt() / reference.square().mean().sqrt()
+                    ),
+                    "divergence_details": rows,
+                }
+                results.append(result)
+                print(
+                    json.dumps(
+                        {k: v for k, v in result.items() if k != "divergence_details"}
+                    ),
+                    flush=True,
+                )
+                del b
+finally:
+    batching.select_token = original
+Path(a.output).write_text(json.dumps(results, indent=2) + "\n")