cuda: EXL3 projections behind qlinear (#593)

[BHCC2025]

EXL3 projections on the native CUDA engine, behind qlinear: the EXL3 item of #593's step 4, claimed in #593. An operator-only change in the shape of #681: no model uses it yet. Routed experts follow in a stacked PR.

What it adds

• kernels/cuda/exl3.cu, exl3.cuh: EXL3 trellis tiles as checkpoints store them ([K/16, N/16, 16 * bits] int16), every codebook core/exl3_format.zig defines (3inst, mcg, mul1), whole widths 1 to 8. Hand-written, on the structure of the EXLR8 CUDA kernels already in the tree.
• cuda/exl3_linear.zig: the weight view (trellis, suh, svh, width, codebook) and its kernels; cuda/qlinear.zig takes it as Weight.exl3, refused with FormatNotLoaded when the module is not loaded, like the other formats.
• tf-cuda-test exl3 [DIR].

Arithmetic

One arithmetic for decode and prompt rows, the one docs/recipes/exl3.md (Prompts) describes for the Python engine's EXL3 path:

1. The input rotation: x * suh, then the 128-point Hadamard transform in a fixed order of fp32 adds, times 2^-4 (exact), rounded once to fp16. fp16 keeps three more mantissa bits than the bf16 activations it comes from; the 2^-4 keeps the transform's growth well inside fp16's range.
2. Each 16x16 tile decodes once, straight into the B fragments of two mma.m16n8k16 (a tile's value order is theirs), with fp32 sums. mul1 values come from one fused fp16 multiply-add of an exact sum, the single rounding the format specifies.
3. The output transform, svh and 2^-3, one rounding to bf16.

Decode rows go 1 to 16 a launch, their A fragments read from rows the rotation writes in fragment order. A matrix with fewer than 96 column blocks splits its input blocks into a number of parts fixed by its shape, and the last part to finish adds the parts in part order. Prompt rows go 64 a block through shared memory. Rows never mix, so each row's bits are the same at every row count on its path.

Tests (GB10)

tf-cuda-test exl3 /path/to/GLM-5.3-Flash-exl3-4.05bpw: 12 pass, 0 fail, 0 skip.

• All 65,536 states of each codebook equal exl3_format.codebookValue bit for bit.
• Random tiles at every width (1-8) and codebook (24 pairs) unpack to exl3_format.unpack's W_q bit for bit.
• FP64 references from exl3_format.dequantize, each value within a bound derived from the kernels' rounding order (fp16 inputs, fp32 sums, bf16 out), worst column and aggregate error reported: 11 synthetic matrices (widths 1-8, all three codebooks, split and unsplit), 98,304 values, worst 0.14 of the bound, worst column 2.8e-3 of the largest output.
• Decode windows of 1 to 16 rows, and prompts of 2 to 300 rows (across 64-row blocks and 256-row chunks), byte-equal to each row alone, through qlinear.
• With DIR, seven real matrices of turboderp's GLM-5.3-Flash EXL3 4.05bpw (mul1; K=4, 5 and 6): 21,504 FP64 values, worst 0.031 of the bound.

Also zig build test test-cuda-host passes, zig fmt --check is clean, and tools/zig/lean_check.py reports 539 problems, the same 539 as 0efbd447.

Timings (GB10, the same matrices)

One row and 16 rows are decode calls; 512 rows is a prompt. ExLlamaV3's own LinearEXL3 on the same tensors and GPU in the right-hand columns, for scale (its decode rounds outputs to fp16).

Matrix	Shape, bits	1 row	16 rows	512 rows	ExLlamaV3 1 / 16 / 512
layer 3 routed gate_proj	4096 x 2048, 4	16.5 us	22.3 us	2.14M rows/s	10.6 us / 18.7 us / 2.28M
layer 3 routed down_proj	2048 x 4096, 4	16.5 us	21.2 us	2.83M rows/s	10.4 us / 18.7 us / 2.16M
layer 0 down_proj	12288 x 4096, 5	145 us	170 us	484K rows/s	155 us / 138 us / 348K
layer 3 shared up_proj	4096 x 2048, 6	16.6 us	24.2 us	2.04M rows/s	15.6 us / 22.8 us / 1.81M
layer 3 q_b_proj	1536 x 16384, 6	42.9 us	49.7 us	1.03M rows/s	22.1 us / 37.0 us / 613K
layer 3 o_proj	16384 x 4096, 6	239 us	259 us	363K rows/s	232 us / 232 us / 271K
lm_head	4096 x 154880, 6	2.15 ms	2.21 ms	24.9K rows/s	2.07 ms / 2.15 ms / 27.4K

The small matrices sit in L2 when timed back to back, so their single-row times are mostly the two launches (rotation, projection); a captured decode graph removes most of that.

Environment

TensorFold 0efbd447 (1.0.8) plus this commit; Zig 0.17.0; NVIDIA GB10 (sm_121), driver 580.178.04, CUDA 13.0.88; Ubuntu 24.04.5, kernel 6.17. Checkpoint: turboderp/GLM-5.3-Flash-exl3, 4.05bpw, revision 2a30229e. zig build fatbins -Dsm=86,120,121 builds the new kernels for every release target.

Not run

• No model serves EXL3 on the native CUDA engine yet, so there is no served decode, bench_concurrent, prefill_cold or drafted-versus-plain receipt; those come with a family (GLM-5.3-Flash and full GLM-5.3 on the native CUDA engine (tracker) #593 step 3).
• sm_86 and sm_120 were compiled, not run.
• Metal is untouched.
• Half-integer mul1 widths are not supported (a Matrix takes whole widths); no checkpoint we have uses them.

[SvangenStudios]

A receipt from a second GB10, on a different checkpoint: Mia-AiLab/GLM-5.3-Flash-EXL3-4bpw-TensorFold @ 078455ff. Its quantization config specifies EXL3, 4 bits, codebook mcg, scope glm53_routed_experts_only.

Setup. GB10 sm_121, driver 580.178.04, kernel 7.0.0-1019-nvidia, Zig 0.17.0, CUDA 13.0. Tested #741 at 92b8cbf8, which contains this PR's 7b6dc024, built with zig build -Dnvcc=… -Dsm=121. A GLM server was running on the same GPU throughout. These are single-run kernel measurements, not a controlled performance comparison.

tf-cuda-test exl3 without DIR: four synthetic PASS checks — bit-exact codebooks, 24 width/codebook pairs, 11 matrices at a maximum 0.1424 of the error bound, and decode/prompt rows byte-equal to the row alone on their respective paths.

With DIR: the first two real matrices pass with mcg, whereas your projection receipt used mul1:

• layers.3.mlp.experts.0.gate_proj, K=4096 N=2048: within FP64 bounds on decode and prompt; 1 row 18.2 µs (230 GB/s), 16 rows 22.7 µs, a 512-row prompt 1,975,626 rows/s.
• layers.3.mlp.experts.0.down_proj, K=2048 N=4096: within the bounds on both paths; 1 row 18.3 µs (229 GB/s), 16 rows 22.7 µs, a 512-row prompt 2,563,568 rows/s.

The test then stops with FAIL exl3: TensorNotInIndex at layers.0.mlp.down_proj: real() looks up its .trellis entry, but this checkpoint has only .weight for that projection. The remaining selected projections — layer 3's mlp.shared_experts.up_proj, self_attn.q_b_proj, self_attn.o_proj, and lm_head — also have .weight and no .trellis.

Both real EXL3 matrices reached before the failure passed; the remaining selected matrices were not tested. Would it make sense to report known non-EXL3 projections as skips, while still failing for missing expected tensors and requiring at least one real EXL3 matrix to be checked?

[BHCC2025]

Thanks for the second GB10 and the mcg receipt, the first run of this path on a codebook other than mul1. Agreed on the skips: 76ac286 makes real() report a selected projection the checkpoint stores as plain .weight as SKIP, still fail with TensorNotInIndex when neither .trellis nor .weight is in the index, and fail when none of the selected matrices is EXL3. On GB10: turboderp's 4.05 bpw 7/7 PASS (unchanged); the same index with three projections as .weight, 3 SKIP + 4 PASS; with all seven as .weight, 7 SKIP then FAIL.

[SvangenStudios]

Confirmed on the same mcg checkpoint (078455ff) and GB10, built from #741's head 2639af8c with -Dsm=121. tf-cuda-test exl3 DIR now exits 0:

• the four synthetic checks PASS;
• layers.3.mlp.experts.0.gate_proj and down_proj PASS within the FP64 bounds on decode and prompt;
• layers.0.mlp.down_proj, shared_experts.up_proj, self_attn.q_b_proj, self_attn.o_proj and lm_head SKIP as .weight;
• PASS 2 real matrices: 6144 FP64 values, max 0.0266 of the bound.

[ashhart]

Thank you, @BHCC2025, we will carry the EXL3 projection work with your credit into the one qlinear EXL3 module and equal-bit comparison; the landing order and remaining checks are in our reply on #837.

Posted by Ash's AI Assistant Synthena

[ashhart]

Thank you, @BHCC2025, for the EXL3 qlinear projections and checkpoint tests.

We merged #740 for 1.1.0 in cd4f084, retaining your commits. This merge leaves the tree unchanged because the shared projection work and plain .weight test handling already arrived through #806.

The common implementation comes through #837, with #806's tiles available explicitly and EXL3 supported by qlinear's FP32 decode and prompt outputs.

Posted by Ash's AI Assistant Synthena