Repository navigation
Zig CUDA family for full GLM-5.3 (glm_moe_dsa) at TP4: plan and questions #552
Copy link
Copy link
Closed as duplicate
Closed as duplicate
Copy link
Description
We run four DGX Sparks (GB10, sm_121) on a 100G switch and would like to help bring full GLM-5.3 to the Zig engine, the port described in #402 and #310. We want to agree on the shape and the gates before writing code.
What we found in the tree (v1.0.2 plus main at 78ee0f4):
- The Zig GLM family reads GLM-5.3-Flash only and runs on Metal only.
- The native CUDA registry holds Nemotron only.
- We found no EXL3 reader or kernel under zig/src or zig/kernels. The full-model checkpoint that fits four Sparks is EXL3 at 2.75 bits per weight.
- cuda/nccl.zig has no send/recv, all-to-all or reduce-scatter, and the one-rank test is the only use we found. Would a TF_NCCL_LIB override for hosts without a system NCCL be welcome?
Proposal: a new glm_moe_dsa CUDA family at TP4 (four ranks), using the design of the Python engine in #159 as the reference: whole-tile EXL3 slices, replicated MLA latent, every expert split by width, rank-order fp32 sums. Measured numbers from the Python engine on four Sparks are the performance bar. Work would land as small independent pull requests:
- Multi-rank CUDA communication for the Zig engine (scope to be agreed against Zig CUDA ports: shared layers below
lanes.Backend, and one place to coordinate them #548). - An EXL3 reader and dequant kernel, with a bit-exact test.
- EXL3 linear and grouped-expert kernels with rank slices.
- One layer against an oracle, then layer subsets, then the full model with lanes and prefill.
Each step is gated on drafted equal to no-draft, resume equal to fresh, and concurrent equal to solo, plus an FP64 reference for every projection class whose reduction order changes.
Questions:
- Is a Zig glm_moe_dsa family wanted, and how does it relate to the Flash port in Native CUDA: GLM-5.3-Flash EXL3 with DFlash2 and TP/EP/DCP #509 and to Full GLM-5.3 (
glm_moe_dsa) on three DGX Sparks, as layers of the GLM-5.3-Flash CUDA engine: how should it sit beside #159? #402 and GLM-5.3-Flash on CUDA over three ranks (--tp 3): how it should sit beside #159, the communicator interface and #243 #310? Who is already doing what, so we do not duplicate? - Which multi-rank path should CUDA use: NCCL, or the RDMA Channel through cluster/?
- What exactness reference is required: only the internal gates above, or also bit equality with the Python engine's results?
- Is a hand-ported .cu EXL3 path acceptable, or do you prefer captured Triton kernels as AOT cubins? (The MLA, indexer, router and norms are Triton in the Python engine; the BF16 projections need a reference too.)
- Is bf16 KV the baseline?
- May CI use a tiny synthetic checkpoint?
We have an independent four-Spark exactness run of #159 and can share the receipts if useful.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
ashhart commentedon Oct 10, 2026
Thank you, @BHCC2025, for asking before writing code, and for the four-Spark offer. Your six questions are answered in #593, which now tracks all GLM work on native CUDA. Full GLM-5.3 at TP=4 is step 5 there, after the Flash family's shared pieces. The multi-rank PR in step 2 would be a good first claim if you'd like it. Closing in favour of #593.
glm: one rank's engine, its lane backend and rank 0 leading the other…