[zhenshanx-nv]

Background

families/timm_mobilevit builds timm/mobilevit_xxs, mobilevit_xs and mobilevit_s (cvnets_in1k) as one classifier engine (256 x 256, batch 1, fp16 in the manifests) and has no quantized build. No FP8 or NVFP4 checkpoint of these models exists on the Hub, so FP8 self-quantization of the transformer linears (attn.qkv, mlp.fc1, mlp.fc2; build-time FP8 weights, ModelOpt-calibrated static activation scales, the ModelOpt ViT recipe) was built and measured for all three checkpoints on TensorRT 11.4 and 11.1. FP8 does not pay for this family: it is slower than fp16 on B200 and B300 (0.89x to 0.96x at the family's builder optimization level 1, 0.94x to 0.97x at level 3), top-1 agreement with the fp16 engine drops to 95.7% to 98.7%, and the plan is only 4% to 19% smaller. This change adds a README for the family that records the study, like the yolov10 (#2093) and timm_efficientnet (#2102) studies, so that it does not have to be repeated. The FP8 code is not part of this change. NVFP4 was not tried.

Exit Criteria

• The README records the recipe, the method, speed and plan size on B200 and B300 (TensorRT 11.4, 11.1) at builder optimization levels 1 and 3, quality against PyTorch fp32 and the fp16 engine, the non-finite scan, why there is no gain, NVFP4 and when to revisit.
• No code, test, manifest or bundle change; builds are unchanged.
• Non-goals: shipping FP8 or NVFP4 builds, an FP8 manifest or CI case, changing the builder optimization level (proposed separately in perf(timm): build 21 timm families with TensorRT optimization level 3 #2123; this change touches only the README, so the two do not conflict).

Implementation

• families/timm_mobilevit/README.md (new): a short family description and the section "FP8 quantization study (not shipped)": summary and decision, recipe, method, results (speed and size at levels 1 and 3, layers, quality, non-finite scan), NVFP4, what was not tried, when to revisit.

Recipe that was measured (code kept out of the repository):

• FP8 for attn.qkv, mlp.fc1 and mlp.fc2 of the 9 transformer layers of each model; convolutions, attention matmuls, layer norms, classifier and attn.proj stay fp16 (a Quantize before attn.proj makes the TensorRT 11.4 attention fusion fail to build).
• Weights: per-tensor scale amax / 448 at build time. Activations: static per-tensor scales from ModelOpt FP8_DEFAULT_CFG (max) on the timm PyTorch model, 500 ImageNet validation images. fp16 Dequantize.

Why no gain: the transformer GEMMs are tiny (token width 64 to 240) and the batch-1 engine is dominated by convolutions and per-layer launch time; FP8 adds 9 to 12 Quantize/Dequantize layers that cost more than the FP8 GEMMs save.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

python3 -m tools.model_ci validate
python3 -m pytest tools/tests/test_architecture.py tools/tests/test_public_source_hygiene.py -q
python3 -m pytest families/timm_mobilevit/tests -q      # GPU, TensorRT 11.4 and 11.1
python3 tools/pr_metadata.py validate --event <event.json>   # Pull-request metadata is complete.
git diff --check

Check	Result
python3 -m tools.model_ci validate	valid
pytest tools/tests/test_architecture.py tools/tests/test_public_source_hygiene.py (CPU)	62 passed
pytest families/timm_mobilevit/tests (B200, TensorRT 11.4 / 11.1)	20 passed, 2 skipped / 20 passed, 2 skipped (E2E needs explicit selection; no code change)
Markdown check of the README (table column counts, code fences)	pass
git diff --check	pass
python3 tools/pr_metadata.py validate on this description	pass

The numbers in the README come from the local study (TensorRT 11.4 re-measured on the current main build code; the FP8 build is the held code) and are not re-run by this change. Batch 1, ms per engine call, x is fp16 time over FP8 time, L1 and L3 are builder optimization levels:

Model	GPU	TensorRT	fp16 L1	FP8 L1	x L1	fp16 L3	FP8 L3	x L3	plan MiB fp16 to FP8 (L1)
s	B200	11.4	0.431	0.462	0.93	0.398	0.414	0.96	12.6 to 10.2
s	B300	11.4	0.420	0.442	0.95	0.371	0.388	0.96	12.7 to 10.3
xs	B200	11.4	0.408	0.449	0.91	0.385	0.401	0.96	6.3 to 5.6
xs	B300	11.4	0.406	0.425	0.96	0.355	0.367	0.97	6.4 to 5.6
xxs	B200	11.4	0.357	0.403	0.89	0.338	0.358	0.94	4.3 to 4.0
xxs	B300	11.4	0.363	0.385	0.94	0.313	0.331	0.95	4.3 to 4.0
s	B200	11.1	0.447	0.462	0.97	0.412	0.410	1.01	12.6 to 10.3

Quality (1500 held-out ImageNet images, TensorRT 11.4, level 1, B200): FP8 top-1 agreement with the fp16 engine 98.5% (s), 97.7% (xs), 95.8% (xxs); top-1 against labels 78.07% / 74.20% / 68.40% (fp16 78.00% / 74.47% / 68.73%, PyTorch fp32 78.00% / 74.33% / 68.87%). No non-finite logits on the images. One of 8 synthetic stress inputs gives non-finite logits on mobilevit_xxs in fp16 and FP8 alike (PyTorch fp32 is finite); this is fp16 behavior of the existing build.

Hardware, Environment, and Revisions

• Repository: this commit on top of upstream/main at 012df437.
• Checkpoints: timm/mobilevit_s.cvnets_in1k at b531fe564ef88515b1aa1743ad4c40452b9f80ee, timm/mobilevit_xxs.cvnets_in1k at 2c62d71504c6fbba9bb322f728586c4a235e5fa4 (manifest revisions), timm/mobilevit_xs.cvnets_in1k at 7a989ba56f36ed9f230044306e1e31563132c35e.
• Data: clip-benchmark/wds_imagenet1k validation images, every 25th of the class-sorted set (2000): 500 for calibration, 1500 held out.
• Study: B200 and B300, one GPU per job, all variants of one model built and timed in the same job, TensorRT 11.4.0.106 (target) and 11.1.0.106, ModelOpt 0.46.1, timm 1.0.28 (reference). Reference: the timm PyTorch model in fp32.

Not Run / Remaining Gaps

• The study cannot be rerun from this repository because the FP8 code was not merged.
• Not measured: FP8 convolutions, FP8 attention matmuls, percentile or entropy calibration, batch above 1 (the family builds batch 1 only), H100, the native runtime with an FP8 bundle, NVFP4.
• attn.proj in FP8 was only tried on TensorRT 11.4 (build fails).
• No new problem on main was found. The fp16 non-finite logits on one synthetic stress input of mobilevit_xxs (a synthetic input, not a natural image) are existing behavior, also seen in perf(timm): build 21 timm families with TensorRT optimization level 3 #2123.

Contributor Self-Review

• I have completed a self-review of this change.

Notes For Future Readers

• Suggested review order: "Summary and decision", the speed table, "When to revisit".
• Revisit FP8 when TensorRT fuses the Quantize into the producing layer for small GEMMs, or when the family gets a batched or larger-resolution variant.
• Can merge before or after perf(timm): build 21 timm families with TensorRT optimization level 3 #2123 (builder optimization level 1 to 3); the README already reports both levels.

Risk level

• Low
• Medium
• High

Risk rationale: documentation only; one README added, no code, build, bundle or test change.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.

Next included review available in 37 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 12 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: NVIDIA/TensorRT-Model-Connect
• Branch: main
• File: .coderabbit.yaml
• Commit: 012df43
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 9cede5d1-0ace-4db6-b1f6-4d270253157f

Commits

Reviewing files that changed from the base of the PR and between 012df43 and 4b31509.

Files selected for processing: 1

• families/timm_mobilevit/README.md