Calibration expansion, stage 2: shape lattice, Qwen3.5-35B-A3B class, HybridEP fix, per-table certificates

[bradhilton]

Stacked on #844 (stage 1). Stage 2 of the expanded calibration campaign: the shape-lattice recipe, the first new certified execution class (Qwen3.5-35B-A3B, GDN + MoE), two product fixes found by the campaign, and the per-table certificate machinery. Dense controls are still running and will land as follow-up commits or a follow-up PR.

Campaign machinery

• dev/trainer_rank_cost_calibration_lattice.sky.yaml: one model over an explicit lattice of (TP, CP, EP[, ETP]) shapes on a 4×H200 node, one torchrun per shape, EP/ETP pinned (the provider defaults EP to the visible GPU count for MoE models), HybridEP built in setup, faulthandler on. Both Ellavox corpora mounted.
• One manifest and one certificate per calibrated table (dev/trainer_rank_cost_calibration_{manifest,certificate}_<table>.json); the manifest names the fixed recipes and/or lattice launches (model, shapes, cell specs) it was run from, and the manifest test expands them into cell keys (EP/ETP suffixes, full-model layer counts) that must equal the manifest. The dense files are renamed; the dense certificate is regenerated against the renamed manifest with the same table hash.
• Harness rows record HybridEP row high-water and buffer capacity per run; a forced layout with no match under a memory-admission split rung is recorded as an admission failure instead of crashing the cell.
• Fitter: candidates are fitted in canonical (cell, label) order (the regret coordinate search is order-dependent and equivalent optima exist, so a refit from the certificate must see the same order); integerize repairs the rounded table on the integer grid only when rounding costs a ranking (it did for the 35B class; the dense table is unchanged); --exclude-cells @manifest takes exactly the manifest's listed exclusions; per-model, per-shape and per-family gate reports.

Product fixes

• HybridEP buffer sizing (_configure_hybridep): sized from the near-balanced extent only; cost-aware CP plans can put more rows on one rank (a 2-history Ellavox group at CP4/EP4 planned 12,546 rows against a 7,168-row buffer and failed). Now sized for the busiest rank's planned rows, identical on every rank. Validated on GPU by re-running that cell.
• Filed, not fixed here: TrainerRank memory admission on Qwen3.5-35B-A3B (MoE) at CP1: cold forward hard-OOMs in the expert grouped GEMM, then over-refusal after one observation #848 (MoE memory admission at CP1: cold OOM in the expert grouped GEMM, then over-refusal) and Qwen3.5-35B-A3B at EP1 with CP>1 segfaults in Transformer Engine's grouped GEMM on real-data routing (likely empty local experts) #851 (EP1 with CP>1 on real-data routing segfaults in Transformer Engine's grouped GEMM, most likely empty local experts).

New class: gdn-moe-h2048-h200-bf16 (Qwen3.5-35B-A3B)

Lattice separating CP from EP: EP1 at CP1/2/4 and TP2, EP2 at CP2/CP4/TP2, EP4 at CP4, on the six synthetic families and the eight Ellavox groups. 88 lattice cells, 15 excluded with reasons (#848 CP1 cells, #851 CP4/EP1 Ellavox cells), 73 certified cells, 3,658 within-cell pairs, 13 odd-Ellavox held-out cells.

	pairwise	median regret	p95	max
all 73 cells (integer table)	97.3%	0%	2.4%	4.9%
held-out 13	96.0%	0%	0.9%	0.9%
per shape (worst)	0.935 (CP4/EP2)	0%		4.9% (CP2)

The same ten terms fitted to their own table suffice; no expert-parallel term was needed (per-shape gates pass on EP2/EP4). The version-1 fallback this class used before loses up to 35% on the large Ellavox groups at CP2/EP2, and the dense table applied to this class loses up to 14%.

Dense controls: measured, not certified

All three dense controls measured cleanly (70 of 70 cells each, no admission failures) but none passes the gates with the ten terms, and every failure is at CP4 on real data, always driven by the same few Ellavox groups:

Class	pairwise (all)	max regret	failing shape	cause
Qwen3-1.7B (hidden 2,048)	90.9%	26%	CP4 (acc 0.82), TP2×CP2	CP plan
Qwen3-8B (hidden 4,096)	97.0%	14%	CP4 (acc 0.87), one clear miss	CP plan
Qwen3-14B (hidden 5,120)	99.6%	17%	CP4, one cell; every other shape ≤ 1%	CP plan

Root cause: layouts with near-identical features produce very different context-parallel plans, e.g. depth_one on one Ellavox group at CP4 needs four exchange waves with rank loads 7168/3584/1536/790 while uniform_depth_2 needs two waves with 6656/2560/2048/1751, a 25% time difference the O(segments) features cannot express. The dense certificate had no attention-plus-real-data cells at CP4; a targeted check of the certified Qwen3-4B geometry on those groups at TP1 × CP4 shows the shipped dense table has a 35% regret cell there (the version-1 score reaches 15% on another group; neither is adequate). This is documented as a known limitation in the design brief and README.

Offline, adding CP-plan-derived features (exchange waves × layers, maximum rank load × layers, derived by running the CP planner per candidate at about 25–40 ms each on CPU) makes the 1.7B class pass every gate (98.1%, max 2.9%) and lifts the 8B held-out gates to passing; a slightly richer per-wave cost model is likely needed to pass every class. That is a planner design change (a CP-plan-aware re-rank of the top candidates after the cheap score) to decide with Codex, not something to slip into this PR. The three dense-control manifests and certificates are parked outside the tree until then; those classes keep the version-1 score. Qwen3.5-27B is still running and will need a follow-up launch for the shapes cut by the pod deadline.

Dense attention classes: certified at their passing shapes (per review)

Following Codex's review of the CP4 blind spot, the three attention controls ship as shape-restricted tables and the dense table withholds one pair:

Table	Geometry	Admitted shapes	Cells	Pairwise	Max regret
dense-attn-h2048-h200-bf16	Qwen3-1.7B	TP1×CP1, TP1×CP2, TP2×CP1	42	96.5%	1.0%
dense-attn-h4096-h200-bf16	Qwen3-8B	TP1×CP1, TP1×CP2, TP2×CP1	42	99.9%	1.1%
dense-attn-h5120-h200-bf16	Qwen3-14B	+ TP2×CP2	56	99.9%	1.0%

• TP1×CP4 (and TP2×CP2 for the two smaller classes) is measured but excluded with the reason in each manifest; those shapes keep version 1 until the CP-aware re-ranker is certified.
• CalibratedTable.withheld: measured, fitted, not admitted pairs. The dense table withholds Qwen3-4B × TP1×CP4 (35% worst cell vs version 1's 15%); its coefficients and certificate hash are unchanged, and the manifest records the 16-cell real-data blind-spot launch as excluded evidence. The certificate test binds withheld pairs to the manifest.
• Planned next (Codex-recommended): a two-stage selector — cheap ten-term shortlist (k swept offline, incumbent included) plus a separate CP-plan re-ranker (exchange-wave count, critical-path work per layer), certified for recall, ranking, non-regression and planning economics, enabled per class/shape. The 8B 11.7% residual cell (Ellavox-qwen3 g6 at CP4) is being re-measured with 24 alternating rounds before any feature is added.

Two-stage selection at CP4 and the CP search speedup

• CP assignment search 4.4× faster (_evaluate_plans): the search's plan evaluator prices a batch of candidate moves in one vectorized numpy pass, results identical to the previous evaluator on all 446 real calibration candidates (the old evaluator is the reference in tests/unit/test_context_parallel_plan_evaluation.py). Median 18.3 ms → 4.1 ms per packed row, also on the production plan-build path of every CP>1 micro-batch.
• Re-ranked shapes: ReRanker + CalibratedTable.reranked_shapes; on TP1×CP4 the attention tables' ten-term score shortlists 3 layouts (+ depth-one anchor) and the plan structure (remote waves, max rank tokens via summarize_prefix_tree_plan) selects. Certified per class (certificate reranker block, recomputed by the certificate test):

Class	two-stage regret all / held-out	clear winners shortlisted	planning cost vs saving (mean % of cell)
Qwen3-1.7B	2.6% / 0.7%	6/6	2.6% vs 2.9%
Qwen3-8B	0.8% / 0.8%	4/4	1.6% vs 1.8%
Qwen3-14B	4.1% / 0.4%	7/7	1.0% vs 1.2%

• The 8B "residual" was real (24-round paired re-measurement: +66.6 ms, CI 62.8–70.4, 24/24 rounds): an extra remote wave costs ~1.9 ms/layer there, and the CP planner's own estimate prefers it — filed as CP planner underestimates the per-wave overhead of remote attention waves (prefers extra waves that measure 0.6–2.4 ms/layer slower) #854.
• GPU validation of the production two-stage selection (timed automatic rows, 14 CP4 cells per class, 4× H200; the harness-recorded plan structure and version-1 scores matched the certificates' features on all 124 candidates per class):

Class	production regret median / worst	worst cell	certificate's predicted worst
Qwen3-1.7B	0.07% / 3.1%	Ellavox g6 (a refinement layout outside the measured family)	2.6%
Qwen3-8B	−0.2% / 0.7%	synthetic g8	0.8%
Qwen3-14B	0.03% / 5.7%	Ellavox g6: the best layout is outside the shortlist (certified irrecoverable 3.5%); the pick equals the cheap-only and version-1 picks (5.3%), so no regression	4.1%

Before this change the same cells cost the shipped/version-1 selection up to 26% (1.7B), 14% (8B) and 17% (14B).

Fourth class: attn-moe-h2048-h200-bf16 (Qwen3-30B-A3B)

112 cells over the EP-deconfounding lattice (3 CP1 cells fail the #848 memory admission; no #851 segfaults on this model). The eight-round timings of CP2 (EP1/EP2) and CP4 (EP1/EP4) carried sporadic 10–20 s stalls on ~4% of rounds, so those shapes were re-measured with sixteen rounds (stalls persist at ~4% but shrink to ~2×; medians hold). Admitted directly at TP1×CP1, TP1×CP2 (EP1/EP2), TP2×CP1 (EP1/EP2) (67 cells, 96.3% pairwise, max 2.2%) and at TP1×CP4 with EP1/EP2/EP4 through the re-ranker (42 cells, max 2.1%, every clear winner shortlisted, planning 1.1% of cell time vs 1.4% saved); version 1 lost up to 13% at CP4 on this class.

Fifth class: dense-gdn-h5120-h200-bf16 (Qwen3.5-27B)

70 cells over TP1×CP1/2/4, TP2×CP1, TP2×CP2 (4 CP1/CP2 cells incomplete under the #848 memory admission, 1 TP2×CP2 cell lost to an NCCL timeout). Clean timings; the ten terms rank this GDN class at CP4 (like the 35B GDN-MoE class), so no re-ranker: admitted at CP1, CP2, CP4, TP2 (52 cells; 97.0% pairwise, max regret 3.4%, held-out 1.8%). TP2×CP2 fails its gates marginally (p95 5.1%) and stays on version 1. Version 1 was worst here: on the largest Ellavox group at CP4 it picked a layout 112% slower than the best (4,084 ms vs 1,927 ms).