Skip to content

Zig CUDA family for full GLM-5.3 (glm_moe_dsa) at TP4: plan and questions #552

Closed as duplicate

Description

@BHCC2025

We run four DGX Sparks (GB10, sm_121) on a 100G switch and would like to help bring full GLM-5.3 to the Zig engine, the port described in #402 and #310. We want to agree on the shape and the gates before writing code.

What we found in the tree (v1.0.2 plus main at 78ee0f4):

  • The Zig GLM family reads GLM-5.3-Flash only and runs on Metal only.
  • The native CUDA registry holds Nemotron only.
  • We found no EXL3 reader or kernel under zig/src or zig/kernels. The full-model checkpoint that fits four Sparks is EXL3 at 2.75 bits per weight.
  • cuda/nccl.zig has no send/recv, all-to-all or reduce-scatter, and the one-rank test is the only use we found. Would a TF_NCCL_LIB override for hosts without a system NCCL be welcome?

Proposal: a new glm_moe_dsa CUDA family at TP4 (four ranks), using the design of the Python engine in #159 as the reference: whole-tile EXL3 slices, replicated MLA latent, every expert split by width, rank-order fp32 sums. Measured numbers from the Python engine on four Sparks are the performance bar. Work would land as small independent pull requests:

  1. Multi-rank CUDA communication for the Zig engine (scope to be agreed against Zig CUDA ports: shared layers below lanes.Backend, and one place to coordinate them #548).
  2. An EXL3 reader and dequant kernel, with a bit-exact test.
  3. EXL3 linear and grouped-expert kernels with rank slices.
  4. One layer against an oracle, then layer subsets, then the full model with lanes and prefill.
    Each step is gated on drafted equal to no-draft, resume equal to fresh, and concurrent equal to solo, plus an FP64 reference for every projection class whose reduction order changes.

Questions:

  1. Is a Zig glm_moe_dsa family wanted, and how does it relate to the Flash port in Native CUDA: GLM-5.3-Flash EXL3 with DFlash2 and TP/EP/DCP #509 and to Full GLM-5.3 (glm_moe_dsa) on three DGX Sparks, as layers of the GLM-5.3-Flash CUDA engine: how should it sit beside #159? #402 and GLM-5.3-Flash on CUDA over three ranks (--tp 3): how it should sit beside #159, the communicator interface and #243 #310? Who is already doing what, so we do not duplicate?
  2. Which multi-rank path should CUDA use: NCCL, or the RDMA Channel through cluster/?
  3. What exactness reference is required: only the internal gates above, or also bit equality with the Python engine's results?
  4. Is a hand-ported .cu EXL3 path acceptable, or do you prefer captured Triton kernels as AOT cubins? (The MLA, indexer, router and norms are Triton in the Python engine; the BF16 projections need a reference too.)
  5. Is bf16 KV the baseline?
  6. May CI use a tiny synthetic checkpoint?

We have an independent four-Spark exactness run of #159 and can share the receipts if useful.

Activity

  1. ashhart commented on Oct 10, 2026

    @ashhart
    Owner

    Thank you, @BHCC2025, for asking before writing code, and for the four-Spark offer. Your six questions are answered in #593, which now tracks all GLM work on native CUDA. Full GLM-5.3 at TP=4 is step 5 there, after the Flash family's shared pieces. The multi-rank PR in step 2 would be a good first claim if you'd like it. Closing in favour of #593.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions