[Bizuayeu]

What this changes

Three commits for NVIDIA's ModelOpt NVFP4 checkpoints, the format nvidia/GLM-5.3-Flash-NVFP4 ships in:

• modelopt.admit() checks config.json's quantization_config (top level, else text_config's) for
  quant_method modelopt, quant_algo NVFP4 and 4-bit float weights in groups of 16, and refuses anything else by
  name. Which linears are NVFP4 is left to the tensors, not the block's ignore list: the pinned revision stores the
  MTP layer's experts in BF16 without listing them there.
• nvfp4.zig reads a linear as stored: e2m1 codes two to a byte [n, k/2], an e4m3 scale per 16 inputs
  [n, k/16], and weight_scale_2 as an fp32 scalar. dims() checks the tensors fit together, Linear.read() maps
  them and marks input_scale used (W4A16 doesn't read it, and a leftover tensor would refuse the checkpoint).
  Its test decodes the linear it read with core.quant's fp64 reference (core/quant: FP8 and NVFP4 values and float64 references in one place #730); the third commit drops the f32
  row() the first version had, so the file adds no e2m1 or e4m3 decoding of its own.

The fixtures are that revision's config.json and the checkpoint's whole header (names, dtypes and shapes, gzip'd,
425 KB): every NVFP4 linear reads, and the MTP layer stays BF16.

Receipt

• Environment: TensorFold 1.0.9 (c36b536d) plus this change; one GB10 (DGX Spark, sm_121, CUDA 13.3).
• Exactness, decode and prompt speed: not applicable (reading only).
• Tests run:
  • zig build test: 470 passed, 5 skipped, 0 failed; the five new tests (admit, refusals by name, dims, a linear
    against core.quant's reference, every linear of the checkpoint's header) need this change.
  • zig build test-cuda-host: 434 passed, 0 failed. zig fmt --check: clean. lean_check: 538, as main.
  • At the third commit (fd2b0c00), from an empty cache: zig fmt --check clean, zig build test and
    zig build test-cuda-host 0 failed, lean_check 534.
• Not run, and why: Metal (no Metal path reads NVFP4 yet).

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
• No precision traded for speed. If the bits change, the description says which sums change and why.
• Prompt processing is no slower than the last release.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[jschmied]

Overlap note: #730 adds the same NVFP4 values in core/quant/nvfp4.zig (group, e2m1()) and the e4m3 decode in core/quant/fp8.zig (e4m3()), with float64 references. Whichever lands second can import the other's instead of keeping two tables. The checkpoint admission and reading tensors as stored don't overlap.

[Bizuayeu]

@jschmied, agreed. #758's e4m3 now reads 0x7f and 0xff as ±480 as #730's does, so the two decoders agree. If #730 lands first, #758's nvfp4.zig drops its own group, e2m1 table and e4m3 and imports core.quant.nvfp4.group, e2m1() and core.quant.fp8.e4m3(); if #758 lands first, #730 can import ours the same way.

#757 also adds to zig/tests/cuda/quant_ref_tests.zig (a function for the fp32 outputs and a call inside lanes); it doesn't touch the helpers #730 moves, so either order rebases with a small change.

[jschmied]

@Bizuayeu, your first version was the right one: e4m3fn reads 0x7f and 0xff as NaN, and so does __nv_cvt_fp8_to_halfraw in the CUDA kernels. #730 now does the same (298e812), so the two decoders agree with that line put back.

[Bizuayeu]

@jschmied, thanks. #758 is back to that version: e4m3 reads 0x7f and 0xff as NaN again, as #730's does at 298e812. The import plan above stands in either order.

[Bizuayeu]

@jschmied, as planned: rebased onto 1.0.9, and with #730 in, nvfp4.zig now takes group, e2m1() and e4m3() from core.quant (as f32, which every e2m1 and e4m3 value is exactly) and drops its own table, decoder and e4m3 test; the reference row is the same product in the same order. Host tests 434/434, zig build test 470 passed, 5 skipped.

[Bizuayeu]

Pushed fd2b0c00: the test now decodes the linear with core.quant's reference (#730), and the f32 row() is gone (only the test called it). The description and the receipt say so.

[ashhart]

Thank you, @Bizuayeu. Admitting NVIDIA's ModelOpt NVFP4 checkpoints from config.json's quantization block, and letting the tensors say which linears are NVFP4, is the right call for a revision whose MTP layer is BF16 without being listed. Pinning the whole header as a fixture makes the reader easy to trust.

Both commits ship in TensorFold 1.0.12 under your name (ae9205078, 27d4bb8c0), with our 99f5f01af after them: it wraps the new lines, and the MIXED_PRECISION test's comment says the format is refused for now, as your commit message does. Your follow-up fd2b0c001, which decodes the test's rows with core/quant's reference instead of a private f32 decoder, is merged too (a6fce1b89), so this pull request shows as merged.

All five of your tests pass on macOS and on a GB10 under Linux. We decoded headers.json.gz before it went public: 147,661 tensor names with their dtypes and shapes, nothing else.

GLM-5.3-Flash on CUDA (#804) is the first caller.

Posted by Ash's AI Assistant Synthena