Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ Model Optimizer is also integrated with [NVIDIA Megatron-Bridge](https://github.
## Latest News

- [2026/10/15] [NVIDIA Nemotron 3.5 Super VL 120B-A12B NVFP4 checkpoint](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4): Quantized with Model Optimizer using post-training quantization (PTQ) and quantization-aware distillation (QAD). For details, see the Quantization section (Section 4) of the [technical report](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Super-and-Lightning-Technical-Report.pdf).
- [2026/10/08] [BLOG: AutoQuant KV: Automatic Mixed-Precision KV Cache Quantization](https://nvidia.github.io/Model-Optimizer/announcements/autoquant-kv.html): Per-layer KV format search, GEMM composition, and a Qwen3-8B accuracy case study.
- [2026/09/16] [BLOG: Scale Learning: Training NVFP4 Block Scales to Recover Accuracy](https://nvidia.github.io/Model-Optimizer/announcements/scale-learning.html)
- [2026/10/08] [BLOG: AutoQuant KV: Automatic Mixed-Precision KV Cache Quantization](https://nvidia.github.io/Model-Optimizer/announcements/20261008-autoquant-kv.html): Per-layer KV format search, GEMM composition, and a Qwen3-8B accuracy case study.
- [2026/09/16] [BLOG: Scale Learning: Training NVFP4 Block Scales to Recover Accuracy](https://nvidia.github.io/Model-Optimizer/announcements/20260916-scale-learning.html)
- [2026/09/16] [**End-to-end W4A4 NVFP4 + QAD tutorial for Qwen3.6-35B-A3B**](./examples/megatron_bridge/tutorials/Qwen3.6-35B-A3B): NVFP4 W4A4 PTQ plus quantization-aware distillation, reaching up to 1.30x vLLM throughput over BF16 and 3.1x smaller checkpoints while recovering the accuracy W4A4 costs.
- [2026/09/09] [BLOG: Improving NVFP4 Accuracy with Local-Hessian Weight Scales](https://nvidia.github.io/Model-Optimizer/announcements/local-hessian.html)
- [2026/08/24] [BLOG: AutoQuantize: A Fast Automatic Mixed-Precision Assignment](https://nvidia.github.io/Model-Optimizer/announcements/autoquantize.html)
- [2026/09/09] [BLOG: Improving NVFP4 Accuracy with Local-Hessian Weight Scales](https://nvidia.github.io/Model-Optimizer/announcements/20260909-local-hessian.html)
- [2026/08/24] [BLOG: AutoQuantize: A Fast Automatic Mixed-Precision Assignment](https://nvidia.github.io/Model-Optimizer/announcements/20260824-autoquantize.html)
- [2026/08/17] [BLOG: Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer](https://developer.nvidia.com/blog/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer/): Learn how quantization-aware distillation recovers accuracy from aggressive NVFP4 quantization while reducing model size and increasing throughput.
- [2026/06/26] [BLOG: Creating the NVIDIA Nemotron 3 Ultra NVFP4 Checkpoint with NVIDIA Model Optimizer](https://developer.nvidia.com/blog/creating-the-nvidia-nemotron-3-ultra-nvfp4-checkpoint-with-nvidia-model-optimizer/): How we quantized Nemotron 3 Ultra (550B) to NVFP4 with Model Optimizer — up to 5.9× higher decode-heavy inference throughput than GLM-5.1 754B FP4 while matching BF16 accuracy. [NVFP4 Checkpoint](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4) on Hugging Face.
- [2026/05/27] [**End-to-end Optimization tutorial for Nemotron-3-Nano-30B-A3B**](./examples/megatron_bridge/tutorials/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16): Pruning + two-phase distillation + FP8 quantization achieving 2.6× vLLM throughput and 2.6× memory reduction.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ What Changes
Authoring Flow
**************

Add a Sphinx page under ``docs/source/announcements/``, add its ``.announcement-card`` metadata and link to ``docs/source/index.rst``, and link it from the announcements toctree when applicable. The GitHub Pages workflow rebuilds the static site from committed source, so every announcement follows the same review path as code and docs.
Add a Sphinx page under ``docs/source/announcements/``, named ``<yyyymmdd>-<slug>.rst`` after its publication date so the pages sort by date, add its ``.announcement-card`` metadata and link to ``docs/source/index.rst``, and link it from the announcements toctree when applicable. The GitHub Pages workflow rebuilds the static site from committed source, so every announcement follows the same review path as code and docs.
18 changes: 18 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@
"sphinx_copybutton", # line numbers getting copied so cannot use `:linenos:`
"sphinx_inline_tabs",
"sphinx_togglebutton",
"sphinxext.rediraffe", # redirect pages for moved docs, see rediraffe_redirects
"sphinxcontrib.autodoc_pydantic",
"modelopt_autodoc_pydantic",
]
Expand Down Expand Up @@ -217,6 +218,23 @@ def visit_reference(self, node: Element) -> None:
self.body.append(("%s" + self.secnumber_suffix) % ".".join(map(str, node["secnumber"])))


# Pages that moved, as {old docname: new docname}; sphinxext-rediraffe writes a redirect page at
# each old path. Announcements gained a <yyyymmdd>- prefix so the source tree sorts by date.
rediraffe_redirects = {
f"announcements/{slug}": f"announcements/{date}-{slug}"
for date, slug in [
("20260713", "dspark-vs-domino"),
("20260813", "github-pages-announcements"),
("20260824", "autoquantize"),
("20260909", "local-hessian"),
("20260916", "qwen36-w4a4-qad"),
("20260916", "scale-learning"),
("20260928", "single-gpu-ptq"),
("20261008", "autoquant-kv"),
]
}


def setup(app: sphinx.application.Sphinx) -> None:
"""Setup according to the Sphinx extension API."""
app.set_translator("html", PatchedHTMLTranslator)
16 changes: 8 additions & 8 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,49 +33,49 @@ Release notes, technical updates, examples, and deployment stories from the Mode
<div class="announcement-grid" id="announcement-grid">
<article class="announcement-card" data-date="2026-10-08" data-title="AutoQuant KV: Automatic Mixed-Precision KV Cache Quantization" data-summary="Learn how AutoQuant selects per-layer KV formats under a storage budget and combines with GEMM quantization, with Qwen3-8B as a case study." data-tags="autoquantize kv-cache quantization mixed-precision modelopt">
<div class="announcement-card-meta">October 8, 2026 &middot; Model Optimizer Team</div>
<h2><a href="announcements/autoquant-kv.html">AutoQuant KV: Automatic Mixed-Precision KV Cache Quantization</a></h2>
<h2><a href="announcements/20261008-autoquant-kv.html">AutoQuant KV: Automatic Mixed-Precision KV Cache Quantization</a></h2>
Comment thread
kevalmorabia97 marked this conversation as resolved.
<p>Learn how AutoQuant selects per-layer KV formats under a storage budget and combines with GEMM quantization, with Qwen3-8B as a case study.</p>
<div class="announcement-card-tags"><span>autoquantize</span><span>kv-cache</span><span>quantization</span><span>mixed-precision</span><span>modelopt</span></div>
</article>
<article class="announcement-card" data-date="2026-09-28" data-title="Quantizing a 4.9 TB Qwen3.8 Model on a Single GPU" data-summary="Layerwise calibration and per-layer shard export drop the memory floor for PTQ from one model to one layer: a 4.9 TB Qwen3.8 checkpoint quantized on a single GB300." data-tags="quantization nvfp4 layerwise moe single-gpu modelopt">
<div class="announcement-card-meta">September 28, 2026 &middot; Model Optimizer Team</div>
<h2><a href="announcements/single-gpu-ptq.html">Quantizing a 4.9 TB Qwen3.8 Model on a Single GPU</a></h2>
<h2><a href="announcements/20260928-single-gpu-ptq.html">Quantizing a 4.9 TB Qwen3.8 Model on a Single GPU</a></h2>
<p>Layerwise calibration and per-layer shard export drop the memory floor for PTQ from one model to one layer: a 4.9 TB Qwen3.8 checkpoint quantized on a single GB300.</p>
<div class="announcement-card-tags"><span>quantization</span><span>nvfp4</span><span>layerwise</span><span>moe</span><span>single-gpu</span><span>modelopt</span></div>
</article>
<article class="announcement-card" data-date="2026-09-16" data-title="Scale Learning: Training NVFP4 Block Scales to Recover Accuracy" data-summary="How ModelOpt learns NVFP4 per-block weight scales with gradients instead of setting them with a calibration rule, and what that recovers." data-tags="scale-learning quantization nvfp4 qad modelopt">
<div class="announcement-card-meta">September 16, 2026 &middot; Model Optimizer Team</div>
<h2><a href="announcements/scale-learning.html">Scale Learning: Training NVFP4 Block Scales to Recover Accuracy</a></h2>
<h2><a href="announcements/20260916-scale-learning.html">Scale Learning: Training NVFP4 Block Scales to Recover Accuracy</a></h2>
<p>How ModelOpt learns NVFP4 per-block weight scales with gradients instead of setting them with a calibration rule, and what that recovers.</p>
<div class="announcement-card-tags"><span>scale-learning</span><span>quantization</span><span>nvfp4</span><span>qad</span><span>modelopt</span></div>
</article>
<article class="announcement-card" data-date="2026-09-16" data-title="Recovering W4A4 NVFP4 Accuracy with Quantization-Aware Distillation" data-summary="Weight-only NVFP4 is slower than BF16 on Blackwell; W4A4 unlocks the FP4 kernels, and QAD recovers the accuracy it costs." data-tags="quantization nvfp4 w4a4 qad distillation megatron-bridge">
<div class="announcement-card-meta">September 16, 2026 &middot; Model Optimizer Team</div>
<h2><a href="announcements/qwen36-w4a4-qad.html">Recovering W4A4 NVFP4 Accuracy with Quantization-Aware Distillation</a></h2>
<h2><a href="announcements/20260916-qwen36-w4a4-qad.html">Recovering W4A4 NVFP4 Accuracy with Quantization-Aware Distillation</a></h2>
<p>Weight-only NVFP4 is <em>slower</em> than BF16 on Blackwell. W4A4 beats it in 9 of 12 shapes, and QAD recovers the accuracy W4A4 costs on Qwen3.6-35B-A3B.</p>
<div class="announcement-card-tags"><span>quantization</span><span>nvfp4</span><span>w4a4</span><span>qad</span><span>distillation</span><span>megatron-bridge</span></div>
</article>
<article class="announcement-card" data-date="2026-09-09" data-title="Improving NVFP4 Accuracy with Local-Hessian Weight Scales" data-summary="How Local-Hessian selects NVFP4 block scales to minimize layer output error, and how it compares with max, MSE, Four-over-six, and GPTQ." data-tags="local-hessian quantization nvfp4 calibration modelopt">
<div class="announcement-card-meta">September 9, 2026 &middot; Model Optimizer Team</div>
<h2><a href="announcements/local-hessian.html">Improving NVFP4 Accuracy with Local-Hessian Weight Scales</a></h2>
<h2><a href="announcements/20260909-local-hessian.html">Improving NVFP4 Accuracy with Local-Hessian Weight Scales</a></h2>
<p>How Local-Hessian selects NVFP4 block scales to minimize layer output error, and how it compares with max, MSE, Four-over-six, and GPTQ.</p>
<div class="announcement-card-tags"><span>local-hessian</span><span>quantization</span><span>nvfp4</span><span>calibration</span><span>modelopt</span></div>
</article>
<article class="announcement-card" data-date="2026-08-24" data-title="AutoQuantize: A Fast Automatic Mixed-Precision Assignment" data-summary="AutoQuantize finds low-sensitivity mixed-precision assignments with gradient-based scoring under a modeled effective-bits budget." data-tags="autoquantize quantization mixed-precision modelopt">
<div class="announcement-card-meta">August 24, 2026 &middot; Model Optimizer Team</div>
<h2><a href="announcements/autoquantize.html">AutoQuantize: A Fast Automatic Mixed-Precision Assignment</a></h2>
<h2><a href="announcements/20260824-autoquantize.html">AutoQuantize: A Fast Automatic Mixed-Precision Assignment</a></h2>
<p>AutoQuantize finds low-sensitivity mixed-precision assignments with gradient-based scoring under a modeled effective-bits budget.</p>
<div class="announcement-card-tags"><span>autoquantize</span><span>quantization</span><span>mixed-precision</span><span>modelopt</span></div>
</article>
<article class="announcement-card" data-date="2026-08-13" data-title="Model Optimizer announcements are moving to GitHub Pages" data-summary="The public Model Optimizer site is gaining a PR-updated announcements landing page within the existing Sphinx documentation." data-tags="release docs github-pages">
<div class="announcement-card-meta">August 13, 2026 &middot; Model Optimizer Team</div>
<h2><a href="announcements/github-pages-announcements.html">Model Optimizer announcements are moving to GitHub Pages</a></h2>
<h2><a href="announcements/20260813-github-pages-announcements.html">Model Optimizer announcements are moving to GitHub Pages</a></h2>
<p>The GitHub Pages site now starts with announcements while the existing API documentation remains available in the docs navigation.</p>
<div class="announcement-card-tags"><span>release</span><span>docs</span><span>github-pages</span></div>
</article>
<article class="announcement-card" data-date="2026-07-13" data-title="DSpark vs Domino: Same DFlash Backbone, Different Correction Heads" data-summary="DSpark and Domino both build on block-parallel DFlash draft generation but diverge in their token-level correction heads." data-tags="speculative-decoding dflash dspark domino architecture">
<div class="announcement-card-meta">July 13, 2026 &middot; Model Optimizer Team</div>
<h2><a href="announcements/dspark-vs-domino.html">DSpark vs Domino: Same DFlash Backbone, Different Correction Heads</a></h2>
<h2><a href="announcements/20260713-dspark-vs-domino.html">DSpark vs Domino: Same DFlash Backbone, Different Correction Heads</a></h2>
<p>DSpark and Domino share a DFlash backbone but make different correction-head tradeoffs: ModelOpt's <code>vanilla</code> Markov head versus a GRU.</p>
<div class="announcement-card-tags"><span>speculative-decoding</span><span>dflash</span><span>dspark</span><span>domino</span><span>architecture</span></div>
</article>
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ dev-docs = [
"sphinx-inline-tabs==2025.12.21.14; python_version >= '3.12'",
"shibuya~=2026.7.0; python_version >= '3.12'",
"sphinx-togglebutton~=0.4.0; python_version >= '3.12'",
"sphinxext-rediraffe~=0.3.0; python_version >= '3.12'",
]
dev-test = [
"coverage[toml]~=7.14.0", # a1_coverage.pth for subprocess tracking requires this
Expand Down
Loading