[zhenshanx-nv]

Background

families/timm_xcit builds timm XCiT classifiers (manifests: xcit-nano-12-p16-224-fb-in1k and xcit-tiny-12-p16-224-fb-in1k, fp16, batch 1) and has no quantized build. No FP8 or NVFP4 checkpoint of any XCiT classifier exists on the Hub, so FP8 self-quantization of the attention and MLP linears (build-time FP8 weights, ModelOpt-calibrated static activation scales, the timm_vit flow) was built and measured on TensorRT 11.4 and 11.1, on B200 and B300. FP8 does not pay for this family: at batch 1 the FP8 engine is 0.94x to 0.95x of the fp16 engine at the family's builder optimization level 1, and 1.01x to 1.03x at level 3 (the pending level change in #2123). The engine is launch bound at batch 1; the only clear gain is a 19% to 31% smaller plan (2 to 5 MiB for the manifest checkpoints). This change adds the study to the family README, like the yolov10 (#2093) and timm_efficientnet (#2102) studies, so that it does not have to be repeated. The FP8 code is not part of this change. NVFP4 was not tried.

Exit Criteria

• The README records the recipe, the method, speed and plan size on B200 and B300 (TensorRT 11.4, level 1 and level 3; 11.1 on B200), quality against PyTorch fp32 with the fp16 engine as the floor, the non-finite scan, the larger configs, NVFP4 and when to revisit.
• No code, test, manifest or bundle change; builds are unchanged.
• Non-goals: shipping FP8 or NVFP4 builds, an FP8 manifest or CI case, the builder optimization level change (perf(timm): build 21 timm families with TensorRT optimization level 3 #2123).

Implementation

• families/timm_xcit/README.md: new section "FP8 quantization study (not shipped)": summary and decision, recipe, method, results (two studies), NVFP4, what was not tried, when to revisit. Only this section is added; the existing text is unchanged, so this does not conflict with perf(timm): build 21 timm families with TensorRT optimization level 3 #2123 (which changes model.py only).

Recipe that was measured (code kept out of the repository):

• FP8 for attn.qkv, attn.proj, mlp.fc1, mlp.fc2 of every block and the class-attention attn.kv; stem, depthwise convolutions, layer norms, channel-attention matmuls, one-token class-attention GEMVs and the head stay fp16.
• Weights: per-tensor scale amax / 448 at build time. Activations: static per-tensor scales from ModelOpt FP8_DEFAULT_CFG (max) on the timm PyTorch model, 500 ImageNet validation images. fp16 Dequantize.

Why no gain: at batch 1 each GEMM takes about 10 microseconds at either precision (196 tokens, 128 to 768 wide). At level 1 the 12 extra Quantize layers are not fused and make FP8 slower; at level 3 they are fused and FP8 gains 1% to 3%.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

python3 -m tools.model_ci validate
python3 -m pytest tools/tests/test_architecture.py tools/tests/test_public_source_hygiene.py -q
python3 tools/pr_metadata.py validate --event <event.json>   # Pull-request metadata is complete.
git diff --check

Documentation only.

Check	Result
python3 -m tools.model_ci validate	pass
pytest tools/tests/test_architecture.py tools/tests/test_public_source_hygiene.py (CPU)	62 passed
Markdown check of the README (table column counts, code fences)	pass
git diff --check	pass
python3 tools/pr_metadata.py validate on this description	pass

The numbers in the README come from the local study and are not re-run by this change (manifest checkpoints, batch 1, x is fp16 time over FP8 time at the same builder optimization level, all variants of a model built and timed in one job, median of 21 interleaved rounds):

Model	GPU	TensorRT	x level 1	x level 3	plan MiB fp16 to FP8 (level 1)	FP8 top-1 agreement with PyTorch (level 1)
nano	B200	11.4	0.95	1.01	7.4 to 5.6	96.4%
nano	B300	11.4	0.94	1.01	6.9 to 5.2	96.2%
tiny	B200	11.4	0.95	1.03	14.5 to 9.9	98.4%
tiny	B300	11.4	0.94	1.03	14.1 to 9.6	98.2%
nano	B200	11.1	0.92	1.00	7.4 to 5.6	96.0%
tiny	B200	11.1	0.92	1.01	14.5 to 9.9	98.8%

The fp16 engine agrees with PyTorch on 99.6% to 99.8% of the images. Five larger configs (small_12_p8 to large_24_p8, earlier study, TensorRT 11.4, level 1) reach 0.97x to 1.07x at batch 1 and up to 1.15x at batch 32 (scratch build). No NaN or Inf on 500 to 1500 held-out images and 17 to 26 stress inputs, any variant.

Hardware, Environment, and Revisions

• Repository: this commit on top of upstream/main at f6550736.
• Checkpoints: timm/xcit_nano_12_p16_224.fb_in1k at b181b4623913d78fd0927601c36e40b9d8bc74ce, timm/xcit_tiny_12_p16_224.fb_in1k at 060449891838eaf301bdb3673e3daaf58e0370de (manifest revisions); larger configs timm/xcit_<config>_224.fb_in1k.
• Data: ImageNet validation images from clip-benchmark/wds_imagenet1k (500 for calibration; 500 and 1500 held out).
• Study: B200 and B300, one GPU per job, TensorRT 11.4.0.106 (target) and 11.1.0.106, torch 2.14, timm 1.0.28, ModelOpt 0.46.1. Reference: the timm PyTorch model in fp32.

Not Run / Remaining Gaps

• No code was run for this change; the study cannot be rerun from this repository because the FP8 code was not merged.
• Not measured: FP8 depthwise convolutions, one-token GEMVs or channel-attention matmuls, layer-norm output Q/DQ, percentile calibration, 384 px configs, H100, native runtime with an FP8 bundle, TensorRT 11.1 on B300.
• No problem on main was found during the study.

Contributor Self-Review

• I have completed a self-review of this change.

Notes For Future Readers

• Suggested review order: "Summary and decision", the study 2 results table, "When to revisit".
• Revisit FP8 when the family builds batch sizes above 1 (1.08x to 1.15x at batch 32 for the larger configs), or when larger configs become manifest checkpoints.
• Can merge before or after perf(timm): build 21 timm families with TensorRT optimization level 3 #2123; the README section already reports both builder optimization levels.

Risk level

• Low
• Medium
• High

Risk rationale: documentation only; one README section added, no code, build, bundle or test change.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.

Next included review available in 57 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 12 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: NVIDIA/TensorRT-Model-Connect
• Branch: main
• File: .coderabbit.yaml
• Commit: f655073
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: c98ccbde-2201-4e04-833e-12a965c5052d

Commits

Reviewing files that changed from the base of the PR and between f655073 and 835402b.

Files selected for processing: 1

• families/timm_xcit/README.md