Address post-merge review of the chat JSONL blend and the Super VL QAD tutorial

CHANGELOG.rst

@@ -27,6 +27,7 @@ Changelog
 - Add ``--vllm_fakequant_export`` to ``examples/megatron_bridge/export_quantized_megatron_to_hf.py`` to evaluate PTQ / QAD checkpoints in vLLM for formats it has no kernels for (e.g. NVFP4 on Hopper, GGML IQ): it folds the weight quantizers into BF16 weights and saves through Megatron-Bridge at any TP / EP / PP, LLMs and VLMs, with the activation and KV-cache quantizers in sidecars that ``examples/vllm_serve/vllm_serve_fakequant.py`` loads automatically. Also allow ``--ep_size`` > 1 for checkpoints built with ``--no_moe_grouped_gemm``.
 - GGML GPTQ checkpoints from ``examples/megatron_bridge`` keep their calibrated codes when the Megatron checkpoint is reloaded for QAD or export, and layerwise calibration now runs on Megatron vision-language models.
 - The Megatron unified HF export (``export_mcore_gpt_to_hf``) now copies the source checkpoint's non-model files (tokenizer, processor, remote code, chat templates, ...) into the export unchanged, from local disk only. The ``examples/megatron_bridge`` scripts make a Hugging Face Hub model local first with ``modelopt.torch.export.ensure_local_checkpoint``.
+- Add an end-to-end W4A4 NVFP4 PTQ and QAD tutorial for NVIDIA Nemotron 3.5 Super VL, covering chat-data blend preparation, assistant-only distillation, Hugging Face export and vLLM serving. See `examples/megatron_bridge/tutorials/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16/README.md <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/megatron_bridge/tutorials/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16/>`_ for details.
 
 *Misc*
 

examples/megatron_bridge/README.md

@@ -18,7 +18,7 @@ This directory contains examples of using Model Optimizer with the [NeMo Megatro
 This README is the getting-started guide, with each workflow shown on small models (mostly Qwen3-8B).
 
 > [!TIP]
-> For complete end-to-end workflows on larger models, see the [tutorials](tutorials/README.md): [Nemotron-3-Nano-30B-A3B](tutorials/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/README.md) pruning + distillation + quantization, [Qwen3.6-35B-A3B](tutorials/Qwen3.6-35B-A3B/README.md) W4A4 NVFP4 + QAD, and [NemotronH omni VL](tutorials/NemotronH-Omni-W4A4-QAD/README.md) W4A4 + QAD.
+> For complete end-to-end workflows on larger models, see the [tutorials](tutorials/README.md): [Nemotron-3-Nano-30B-A3B](tutorials/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/README.md) pruning + distillation + quantization, [Qwen3.6-35B-A3B](tutorials/Qwen3.6-35B-A3B/README.md) W4A4 NVFP4 + QAD, and [NVIDIA Nemotron 3.5 Super VL](tutorials/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16/README.md) W4A4 + QAD.
 
 ## Pre-Requisites
 

examples/megatron_bridge/tutorials/NemotronH-Omni-W4A4-QAD/README.md → examples/megatron_bridge/tutorials/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16/README.md

@@ -1,6 +1,6 @@
-# NemotronH Omni VL: W4A4 NVFP4 PTQ + QAD + Deployment
+# NVIDIA Nemotron 3.5 Super VL: W4A4 NVFP4 PTQ + QAD + Deployment
 
-End-to-end W4A4 quantization of a NemotronH omni vision-language checkpoint (architecture `NemotronH_Omni_Reasoning_V3`), a Mamba-Transformer MoE language model with a vision tower: mixed NVFP4/FP8 post-training quantization → Quantization-Aware Distillation (QAD) → Hugging Face export → vLLM serving. This document covers:
+End-to-end W4A4 quantization of [NVIDIA Nemotron 3.5 Super VL](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16) (architecture `NemotronH_Omni_Reasoning_V3`), a Mamba-Transformer MoE language model with a vision tower: mixed NVFP4/FP8 post-training quantization → Quantization-Aware Distillation (QAD) → Hugging Face export → vLLM serving. This document covers:
 
 1. **[Data Preparation](#1-data-preparation)**: the chat-format QAD blend, from `data_blend.yaml`
 2. **[Quantization](#2-quantization)**: W4A4 PTQ with `examples/megatron_bridge/quantize.py`
@@ -10,9 +10,11 @@ End-to-end W4A4 quantization of a NemotronH omni vision-language checkpoint (arc
 
 Only the language model (including its MTP head) is quantized; the vision tower and projector stay BF16 and are copied from the source checkpoint.
 
+The PTQ recipe is the same as the one used for the released [`nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4) checkpoint. The calibration data and the QAD training data are all public Hugging Face datasets. The evaluation results below differ from the official model card scores.
+
 ## Results
 
-All checkpoints were served with the same vLLM settings (see [Section 5](#5-evaluation-and-serving)); the quantized ones with an FP8 KV cache. BF16 is the mean of two runs.
+All checkpoints were served with the same vLLM settings (see [Section 5](#5-evaluation-and-serving)), MMLU-ProX at `--max-model-len 32768` for all three; the quantized ones with an FP8 KV cache.
 
 | Benchmark | BF16 (teacher) | W4A4 PTQ | W4A4 QAD (600 iters) |
 | --- | --- | --- | --- |
@@ -43,7 +45,8 @@ PTQ and export launch with `torchrun`; QAD runs on 8 nodes with the `sbatch` scr
 QAD trains on a weighted blend of 11 text-only chat sources, defined in [`data_blend.yaml`](data_blend.yaml) and computing the loss on assistant tokens only. Set `output_dir` in that file, then write the blend as chat JSONL with the [chat JSONL mode of `prepare_megatron_data_blend`](../../../dataset/MEGATRON_DATA_PREP.md#chat-jsonl-blends-for-assistant-only-loss):
 
 ```bash
-python -m modelopt.torch.utils.plugins.prepare_megatron_data_blend --config data_blend.yaml
+python -m modelopt.torch.utils.plugins.prepare_megatron_data_blend \
+    --config /opt/Model-Optimizer/examples/megatron_bridge/tutorials/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16/data_blend.yaml
 ```
 
 This writes `train.jsonl` (35,800 conversations) and `validation.jsonl` (1,000) to `output_dir`. Run it in the container: Megatron-Bridge's text-only chat filter selects the rows.
@@ -72,14 +75,14 @@ PTQ runs on **one 4-GPU GB300 node** with 512 calibration samples.
 
 ```bash
 torchrun --nproc_per_node 4 /opt/Model-Optimizer/examples/megatron_bridge/quantize.py \
-    --hf_model_name_or_path <nemotron_h_omni-checkpoint> \
+    --hf_model_name_or_path nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 \
     --trust_remote_code \
     --recipe models/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16/ptq/nvfp4-mse \
     --tp_size 1 --ep_size 4 --pp_size 1 \
     --calib_dataset_name cnn_nemotron_v2_mix \
     --calib_num_samples 512 \
     --calib_batch_size 1 \
-    --export_megatron_path /path/to/nemotron_h_omni_w4a4_megatron \
+    --export_megatron_path /path/to/nemotron_3_5_super_vl_w4a4_megatron \
     --skip_generate
 ```
 
@@ -108,9 +111,9 @@ We used **8 nodes × 4 GB300 (32 GPUs)** with context parallelism 4 and expert p
 export MASTER_ADDR=$(scontrol show hostnames "$SLURM_JOB_NODELIST" | head -n 1) MASTER_PORT=29500
 srun bash -c 'export RANK=$SLURM_PROCID WORLD_SIZE=$SLURM_NTASKS LOCAL_RANK=$SLURM_LOCALID
 python -u /opt/Model-Optimizer/examples/megatron_bridge/distill.py \
-    --teacher_hf_path <nemotron_h_omni-checkpoint> \
-    --student_hf_path <nemotron_h_omni-checkpoint> \
-    --student_megatron_path /path/to/nemotron_h_omni_w4a4_megatron \
+    --teacher_hf_path nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 \
+    --student_hf_path nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 \
+    --student_megatron_path /path/to/nemotron_3_5_super_vl_w4a4_megatron \
     --trust_remote_code \
     --tp_size 1 --pp_size 1 --cp_size 4 --ep_size 32 \
     --sft --sft_hf_dataset /path/to/qad_blend/train.jsonl \
@@ -145,10 +148,10 @@ Export the QAD checkpoint to a Hugging Face checkpoint on one 4-GPU node (pipeli
 
 ```bash
 torchrun --nproc_per_node 4 /opt/Model-Optimizer/examples/megatron_bridge/export_quantized_megatron_to_hf.py \
-    --hf_model_name_or_path <nemotron_h_omni-checkpoint> \
+    --hf_model_name_or_path nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 \
     --trust_remote_code \
     --megatron_path /path/to/qad_output/checkpoints \
-    --export_unified_hf_path /path/to/nemotron_h_omni_w4a4_hf \
+    --export_unified_hf_path /path/to/nemotron_3_5_super_vl_w4a4_hf \
     --pp_size 4
 ```
 
@@ -162,14 +165,23 @@ torchrun --nproc_per_node 4 /opt/Model-Optimizer/examples/megatron_bridge/export
 
 ### 5. Evaluation and Serving
 
-Serve with vLLM on one 4-GPU GB300 node, TP=4 with expert parallelism. Quantized checkpoints use these engine arguments; the BF16 baseline uses `--kv-cache-dtype auto`.
+Serve with vLLM on one 4-GPU GB300 node, TP=4 with expert parallelism. The results above used:
 
-| Setting | Value |
-| --- | --- |
-| `--kv-cache-dtype` | `fp8` |
-| `--max-model-len` | 262144 |
-| `--mamba-ssm-cache-dtype` | `float16` |
-| `--mamba-conv-cache-dtype` | `float32` |
-| Mamba cache rounding | stochastic rounding, 5 Philox rounds |
-| Reasoning parser | `nemotron_v3` |
-| Chat template | the checkpoint's `chat_template.jinja` |
+```bash
+vllm serve /path/to/nemotron_3_5_super_vl_w4a4_hf \
+    --tensor-parallel-size 4 --enable-expert-parallel \
+    --trust-remote-code \
+    --max-model-len 262144 \
+    --kv-cache-dtype fp8 \
+    --mamba-ssm-cache-dtype float16 \
+    --mamba-cache-dtype float32 \
+    --enable-mamba-cache-stochastic-rounding \
+    --mamba-cache-philox-rounds 5 \
+    --reasoning-parser nemotron_v3 \
+    --enable-auto-tool-choice --tool-call-parser qwen3_coder \
+    --chat-template /path/to/nemotron_3_5_super_vl_w4a4_hf/chat_template.jinja \
+    --enable-prefix-caching --enable-chunked-prefill \
+    --gpu-memory-utilization 0.8
+```
+
+MMLU-ProX was served with `--max-model-len 32768`. The BF16 baseline uses `--kv-cache-dtype auto`.

examples/megatron_bridge/tutorials/README.md

@@ -8,5 +8,5 @@ Each one walks through a complete workflow using the scripts in [examples/megatr
 | Tutorial | What it covers |
 | --- | --- |
 | [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/README.md) | End-to-end optimization of the Nemotron-3-Nano-30B-A3B-BF16 (MoE + Mamba-Transformer hybrid) model: Minitron structured pruning (31.6B/A3.6B → 22B/A3.0B) → two-phase knowledge distillation (100B tokens, 8K then 32K seq length) → quantization → vLLM deployment. Includes data-blend preparation, evaluation setup, and detailed pruning / data-blend / long-context ablations. |
-| [NemotronH-Omni-W4A4-QAD](NemotronH-Omni-W4A4-QAD/README.md) | **W4A4** quantization of the language model of a NemotronH omni vision-language checkpoint (`NemotronH_Omni_Reasoning_V3`, Mamba-Transformer MoE): mixed NVFP4/FP8 PTQ → KD-only QAD on chat data → Hugging Face export → vLLM serving. |
+| [NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16](NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16/README.md) | **W4A4** quantization of the language model of NVIDIA Nemotron 3.5 Super VL (`NemotronH_Omni_Reasoning_V3`, Mamba-Transformer MoE): mixed NVFP4/FP8 PTQ → KD-only QAD on chat data → Hugging Face export → vLLM serving. |
 | [Qwen3.6-35B-A3B](Qwen3.6-35B-A3B/README.md) | End-to-end **W4A4 NVFP4** optimization of the Qwen3.6-35B-A3B (MoE + hybrid linear/full attention VLM) model: NVFP4 W4A4 post-training quantization → quantization-aware distillation (QAD) to recover the accuracy W4A4 costs → evaluation → vLLM deployment. Includes data-format (pre-tokenized vs chat SFT) and SciCode non-termination ablations. |

modelopt/torch/utils/plugins/prepare_megatron_data_blend.py

@@ -193,7 +193,7 @@ def _to_chat(
     """
     if content_field == "messages":
         messages = row.get("messages")
-        if not messages:
+        if not messages or not isinstance(messages, list):
             return None
         messages = prepare_messages_for_chat_template(messages, reasoning_content=reasoning_content)
         chat: dict[str, Any] = {"messages": messages}
@@ -203,8 +203,8 @@ def _to_chat(
             if row.get(key):
                 chat[key] = row[key]
     else:
-        text = row.get("text") or ""
-        if not text:
+        text = row.get("text")
+        if not text or not isinstance(text, str):
             return None
         if "<extra_id_" not in text:
             return {"messages": [{"role": "assistant", "content": text}]}
@@ -279,14 +279,27 @@ def _take_chat_rows(
     from megatron.bridge.data.conversation_processing import is_text_only_chat_example
 
     name = f"{source['hf_dataset']} {source.get('split') or source.get('files')}"
+    content_field = source["content_field"]  # a config error, not a bad row: fail here
     rows: list[dict[str, Any]] = []
-    rejected = 0
+    rejected = unconvertible = 0
     max_rejected = 10 * want + 100
     try:
         for row in _iter_rows(source, output_dir, shuffle, seed):
-            chat = _to_chat(row, source["content_field"], reasoning_content, loss_mode)
-            # Judge the source row's top-level media keys too, which the conversion does not keep.
-            if chat is None or not is_text_only_chat_example({**row, **chat}):
+            try:
+                chat = _to_chat(row, content_field, reasoning_content, loss_mode)
+                # Judge the source row's top-level media keys too, which the conversion drops.
+                if chat is not None and not is_text_only_chat_example({**row, **chat}):
+                    chat = None
+            except (AttributeError, KeyError, TypeError, ValueError) as e:
+                # As in the tokenizing reader: one malformed record must not lose the source.
+                # Warn on the first few only; the summary line carries the count.
+                if unconvertible < 5:
+                    print(
+                        f"[WARN] {name}: skipping a row that cannot be converted: {e!r}", flush=True
+                    )
+                unconvertible += 1
+                chat = None
+            if chat is None:
                 rejected += 1
                 if rejected > max_rejected:
                     break
@@ -299,8 +312,11 @@ def _take_chat_rows(
     if len(rows) < want:
         gave_up = rejected > max_rejected
         reason = f"gave up after {rejected} rejected rows" if gave_up else "source exhausted"
+        if unconvertible:
+            reason += f"; {unconvertible} rows could not be converted"
         raise RuntimeError(f"{name}: only {len(rows)}/{want} usable rows ({reason})")
-    print(f"{name}: {len(rows)} rows ({rejected} rejected)", flush=True)
+    detail = f", {unconvertible} unconvertible" if unconvertible else ""
+    print(f"{name}: {len(rows)} rows ({rejected} rejected{detail})", flush=True)
     return rows
 
 

tests/gpu_megatron/torch/utils/plugins/test_prepare_megatron_data_blend.py

@@ -176,6 +176,7 @@ def test_to_chat_handles_all_row_shapes():
     msgs = [{"role": "user", "content": "hi"}, {"role": "assistant", "content": "hello"}]
     assert blend_module._to_chat({"messages": msgs}, "messages") == {"messages": msgs}
     assert blend_module._to_chat({"messages": []}, "messages") is None
+    assert blend_module._to_chat({"messages": "hello"}, "messages") is None
     # Nothing would be trained under assistant-only loss; loss_mode full trains the user turn.
     assert blend_module._to_chat({"messages": msgs[:1]}, "messages") is None
     assert blend_module._to_chat({"messages": msgs[:1]}, "messages", loss_mode="full") == {
@@ -206,6 +207,7 @@ def test_to_chat_handles_all_row_shapes():
         "messages": [{"role": "assistant", "content": "def f(): pass"}]
     }
     assert blend_module._to_chat({"text": ""}, "text") is None
+    assert blend_module._to_chat({"text": ["x"]}, "text") is None
     # Markers that parse to no turns: dropped, never trained as assistant text.
     assert blend_module._to_chat({"text": "<extra_id_0>\nraw body\n"}, "text") is None
     # A role without a mapping would cut its turn out of the middle: the whole row is dropped.
@@ -226,7 +228,9 @@ def test_to_chat_handles_all_row_shapes():
         assert blend_module._to_chat({"text": text}, "text") is None, text
 
 
-def test_take_chat_rows_skips_non_text_rows_and_fails_without_enough(tmp_path: Path, monkeypatch):
+def test_take_chat_rows_skips_non_text_rows_and_fails_without_enough(
+    tmp_path: Path, monkeypatch, capsys
+):
     source = _source("a")
     # Media is rejected inside message content and as a top-level key the conversion drops.
     rows = [_chat(0), _image_chat(1), {**_chat(9), "images": ["a.png"]}, _chat(2), _chat(3)]
@@ -239,6 +243,22 @@ def test_take_chat_rows_skips_non_text_rows_and_fails_without_enough(tmp_path: P
     with pytest.raises(RuntimeError, match=r"only 0/2 usable rows \(gave up after 121 rejected"):
         blend_module._take_chat_rows(source, tmp_path, 2, False, 0, "native", "assistant")
 
+    # Unconvertible rows are skipped, but only the first few are warned about.
+    bad = [{"messages": ["hello"]} for _ in range(20)]
+    monkeypatch.setattr(blend_module, "_iter_rows", lambda *a: iter([*bad, _chat(0)]))
+    capsys.readouterr()
+    assert (
+        len(blend_module._take_chat_rows(source, tmp_path, 1, False, 0, "native", "assistant")) == 1
+    )
+    out = capsys.readouterr().out
+    assert out.count("cannot be converted") == 5
+    assert "1 rows (20 rejected, 20 unconvertible)" in out
+    # A config error is not a bad row: it fails before any row is skipped.
+    monkeypatch.setattr(blend_module, "_iter_rows", lambda *a: iter(bad))
+    no_field = {k: v for k, v in source.items() if k != "content_field"}
+    with pytest.raises(KeyError, match="content_field"):
+        blend_module._take_chat_rows(no_field, tmp_path, 1, False, 0, "native", "assistant")
+
 
 def test_chat_blend_splits_weighted_rows(tmp_path: Path, chat_blend, monkeypatch):
     write_config, calls = chat_blend
@@ -365,8 +385,10 @@ def test_write_jsonl_is_atomic(tmp_path: Path):
 def test_chat_blend_reads_files_sources(tmp_path: Path, monkeypatch):
     plain, packed = tmp_path / "a.jsonl", tmp_path / "b.jsonl.gz"
     lines = [json.dumps(_chat(i)) for i in range(20)]
-    # A malformed record is skipped with a warning, as in the tokenizing reader.
-    plain.write_text("\n".join([*lines[:10], '{"messages": [', *lines[10:]]) + "\n", "utf-8")
+    # Malformed records, unparseable or unconvertible, are skipped with a warning as in the
+    # tokenizing reader.
+    bad = ['{"messages": [', '{"messages": ["hello"]}']
+    plain.write_text("\n".join([*lines[:10], *bad, *lines[10:]]) + "\n", "utf-8")
     with gzip.open(packed, "wt", encoding="utf-8") as f:
         f.writelines(json.dumps(_chat(i)) + "\n" for i in range(20, 40))
     paths = {"data/a.jsonl": str(plain), "data/b.jsonl.gz": str(packed)}