Repository navigation
Commit af6d7d9
Leave MTP layers unquantized at any depth in the default PTQ exclusions (#2911)
### What does this PR do?
Type of change: Bug fix
The default PTQ exclusions
(`configs/ptq/units/default_disabled_quantizers`, imported by the
`general/ptq/*` recipes and `mtq.*_CFG` presets) disable
multi-token-prediction layers with `mtp.*`, which only matches an MTP
block at the model root (HF checkpoints, Megatron LLMs). Megatron
vision-language models nest it under the language model
(`language_model.mtp.layers.0...`), so their MTP layers were quantized.
For Qwen3.5/3.6 VLMs quantized with
`examples/megatron_bridge/quantize.py` and a general recipe,
`export_quantized_megatron_to_hf.py --vllm_fakequant_export` then
refuses the checkpoint ("MTP quantization is not supported by vLLM
fakequant export/reload"). The pattern is now `*mtp.*`, matching the
model-specific recipes that already use `*mtp*`.
Models whose MTP block was nested and previously quantized by a general
recipe now keep it in BF16, which is what the exclusion intends. Recipes
that quantize MTP on purpose (e.g. the Nemotron 3.5 Super ones) do not
import this unit after their MTP entries, so they are unaffected.
### Usage
No API change.
### Testing
- New
`tests/unit/recipe/test_presets.py::test_default_exclusions_disable_mtp_at_any_depth`:
with `general/ptq/fp8_default-kv_fp8`, MTP quantizers at the root and
under `language_model.` are disabled and decoder quantizers stay
enabled; fails without the change.
- `tests/unit/recipe`: 492 passed.
- Found while validating vLLM fake-quant export of Qwen3.6-35B-A3B
(#2889): the default `nvfp4_default-kv_fp8` checkpoint was refused on
`language_model.mtp.layers.0.eh_proj.input_quantizer`.
### Before your PR is "*Ready for review*"
- Is this change backward compatible?: ✅ (nested MTP layers that a
general recipe used to quantize now stay in BF16, as the exclusion
intends)
- If you copied code from any other sources or added a new PIP
dependency, did you follow guidance in `CONTRIBUTING.md`: N/A
- Did you write any new necessary tests?: ✅
- Did you update
[Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?:
✅
- Did you get Claude approval on this PR?: ❌
### Additional Information
Related to #2889 (vLLM fake-quant export through Megatron-Bridge).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Corrected default post-training quantization recipes for Megatron
vision-language models so MTP layers remain unquantized consistently,
including nested language-model layers.
* Preserved quantization behavior for unrelated decoder projection
layers.
* Added regression coverage to verify both root-level and nested MTP
exclusions in the default FP8 recipe.
* **Documentation**
* Added a changelog entry for the 0.49.0 bug fix.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>1 parent 754d3bc commit af6d7d9
3 files changed
Lines changed: 22 additions & 1 deletion
File tree
- modelopt_recipes/configs/ptq/units
- tests/unit/recipe
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| 40 | + | |
40 | 41 | | |
41 | 42 | | |
42 | 43 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| 27 | + | |
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
| |||
96 | 97 | | |
97 | 98 | | |
98 | 99 | | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
99 | 119 | | |
100 | 120 | | |
101 | 121 | | |
| |||
0 commit comments