[zhenshanx-nv]

Background

families/pixart builds PixArt-Sigma XL (PixArt-alpha/PixArt-Sigma-XL-2-1024-MS) as T5 encoder, DiT and VAE decoder engines (fp16 at 512 x 512 and 1024 x 1024, fp32 with tensor parallel size 4) and has no quantized build. No FP8 or NVFP4 checkpoint or official quantization config exists for PixArt-Sigma, so FP8 self-quantization of the DiT linears (per-tensor FP8 weights at build time, static activation scales calibrated on the PyTorch model, fp16 Dequantize) was built and measured on TensorRT 11.4 and 11.1, B200 and B300. FP8 does not pay for this family: the variants that give a speedup (feed-forward and self-attention q/k/v) give 1.05x to 1.15x on the DiT call at batch 1, but their denoiser error is 3.5x to 6x the fp16 floor and LPIPS against the fp32 reference grows from 0.016 to 0.04 to 0.07 to 0.20. The only variant at the fp16 floor (cross-attention q/k/v) gives 1.00x to 1.01x. This change adds a README for the family that records the study, like the yolov10 (#2093), bart (#2004), timesfm (#2005) and marian (#2035) studies, so that it does not have to be repeated. The FP8 code is not part of this change. NVFP4 was not tried.

Exit Criteria

• The README records the recipe, the method, DiT speed and plan size on B200 and B300 (TensorRT 11.4, 11.1, batch 1 and 2), quality per projection group against fp32 PyTorch with the fp16 floor, the time split that bounds the gain, variants that did not help, NVFP4 and when to revisit.
• No code, test, manifest or bundle change; builds are unchanged.
• Non-goals: shipping FP8 or NVFP4 builds, an FP8 manifest or CI case.

Implementation

• families/pixart/README.md (new): a short family description and the section "FP8 quantization study (not shipped)": summary and decision, recipe, method, results (speed, batch 2, TensorRT 11.1, quality per group, time split), why no gain, NVFP4, what was not tried, when to revisit.

Recipe that was measured (code kept out of the repository):

• FP8 for groups of the DiT linears in all 28 blocks: feed-forward ff.net.0.proj and ff.net.2, self-attention attn1.to_q/k/v, cross-attention attn2.to_q/k/v. Embeddings, adaptive layer norm, attention matmuls, attn1.to_out.0, norm_out, proj_out, T5 and the VAE stay unquantized.
• Weights: per-tensor scale amax / 448 at build time. Activations: static per-tensor max scales from the PyTorch model over the 20 denoising steps, at the target resolution. fp16 Dequantize (fp32 Dequantize gives no gain: 1.00x at 512 on TensorRT 11.1).

Why no gain: the error is FP8 rounding noise amplified by the 28 blocks and guidance (clipping, per-block exclusion of blocks 0 to 3 did not bring it near the floor); the insensitive cross-attention linears are a small share of the time; half of the DiT call is not GEMM time.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

python3 -m tools.model_ci validate
python3 -m pytest tools/tests/test_architecture.py tools/tests/test_public_source_hygiene.py -q
python3 tools/pr_metadata.py validate --event <event.json>   # Pull-request metadata is complete.
git diff --check

Documentation only.

Check	Result
python3 -m tools.model_ci validate	pass
pytest tools/tests/test_architecture.py tools/tests/test_public_source_hygiene.py (CPU)	62 passed
Markdown check of the README (table column counts, code fences)	pass
git diff --check	pass
python3 tools/pr_metadata.py validate on this description	pass

The numbers in the README come from the local study and are not re-run by this change (DiT call at batch 1, x is fp16 time over FP8 time, FF = feed-forward):

GPU	TensorRT	Size	fp16 ms	FF x	FF + self q/k/v x	FF + self and cross q/k/v x	cross q/k/v x	plan GiB fp16 to widest
B200	11.4	512	4.095	1.05	1.07	1.08	1.01	1.114 to 0.630
B200	11.4	1024	11.487	1.10	1.14	1.15	1.00	1.114 to 0.630
B300	11.4	512	3.871	1.05	1.07	1.07	1.00	1.114 to 0.630
B300	11.4	1024	10.181	1.06	1.09	1.09	1.00	1.114 to 0.630
B200	11.1	512 / 1024	4.143 / 11.150	1.05 / 1.07	1.07 / 1.10	1.08 / 1.11	0.99 / 0.98 (with out)	same
B300	11.1	512 / 1024	3.920 / 10.178	1.05 / 1.07	1.08 / 1.08	1.09 / 1.09	0.98 / 1.00 (with out)	same

Quality (TensorRT 11.1 study, fp32 PyTorch reference, 20 steps, guidance 4.5): step error 0.0089 (fp16) vs 0.051 (FF) and 0.070 (all linears) at 512; LPIPS 0.016 (fp16) vs 0.081 (FF) at 512 and 0.041 vs 0.177 at 1024. No non-finite output in any FP8 engine.

Hardware, Environment, and Revisions

• Repository: this commit on top of upstream/main at 5801406c.
• Checkpoints: PixArt-alpha/PixArt-Sigma-XL-2-1024-MS at e102b3591cc82e97071b8b4cb90d834d0c487207 (manifest revision); PixArt-alpha/PixArt-Sigma-XL-2-512-MS for the 512 quality runs.
• Study: B200 and B300, one GPU per job, TensorRT 11.4.0.106 (target) and 11.1.0.106; standalone DiT engines with the family's graph, all variants built and timed in one job, median of 7 interleaved rounds. Reference: diffusers PixArt-Sigma in fp32.

Not Run / Remaining Gaps

• No code was run for this change; the study cannot be rerun from this repository because the FP8 code was not merged.
• Quality was measured on TensorRT 11.1 (B200) and with PyTorch fake quantization, which matched the engines; the 11.4 runs measured speed, plan size and non-finite outputs.
• Not measured: per-timestep scales, SmoothQuant, quantization-aware checkpoints, T5 and VAE quantization, H100, the native runtime with an FP8 bundle.
• No problem on main was found during the study.

Contributor Self-Review

• I have completed a self-review of this change.

Notes For Future Readers

• Suggested review order: "Summary and decision", the quality per projection group table, "When to revisit".
• Revisit FP8 when a recipe keeps the feed-forward and self-attention error near the fp16 floor. Larger levers without FP8: one batch 2 DiT call for the guidance pair (1.35x to 1.38x per image at 512) and a fused masked cross-attention.

Risk level

• Low
• Medium
• High

Risk rationale: documentation only; one README added, no code, build, bundle or test change.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.

Next included review available in 24 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 12 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: NVIDIA/TensorRT-Model-Connect
• Branch: main
• File: .coderabbit.yaml
• Commit: 5801406
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 3a6412cd-c1ea-4b84-9763-882f8de31db5

Commits

Reviewing files that changed from the base of the PR and between 5801406 and c1ba524.

Files selected for processing: 1

• families/pixart/README.md