[zhenshanx-nv]

Background

families/yolo11 builds yolo11n (Ultralytics/YOLO11) as one detector engine (640 x 640, batch 1, fp16 in the manifest; class-aware NMS in the runtime) and has no quantized build. No FP8 or NVFP4 checkpoint of any YOLO detector exists on the Hub, so FP8 self-quantization of the convolutions (the recipe of the yolov10 study, #2093) was built and measured on TensorRT 11.4 and 11.1. FP8 does not pay for this family: it is slower than fp16 (0.90x to 0.93x at batch 1, 0.88x to 0.89x at batch 8), the plan saves only about 2 MiB (6.56 to 4.65 MiB on B200), and COCO AP drops from 0.4158 to 0.4034. This change adds a README for the family that records the study, like the yolov10 (#2093) and yolov8 (#2101) studies, so that it does not have to be repeated. The FP8 code is not part of this change. NVFP4 was not tried: TensorRT does not support NVFP4 on convolutions.

Exit Criteria

• The README records the recipe, the method, speed and plan size on B200 and B300 (TensorRT 11.4, 11.1), AP against PyTorch fp32, why there is no speedup, the head-in-fp16 variant, NVFP4 and when to revisit.
• No code, test, manifest or bundle change; builds are unchanged.
• Non-goals: shipping FP8 or NVFP4 builds, an FP8 manifest or CI case.

Implementation

• families/yolo11/README.md (new): a short family description and the section "FP8 quantization study (not shipped)": summary and decision, recipe, method, results (speed, size, quality, variant), NVFP4, what was not tried, when to revisit.

Recipe that was measured (code kept out of the repository):

• FP8 for the 77 of 87 convolutions of yolo11n whose input and output channels are multiples of 16. The stem (3 input channels), the 2 convolutions with 8 channels in model.2, the 7 depthwise convolutions, the DFL decode and the top-1 class stay fp16.
• Weights: per-output-channel scale amax / 448 of the batch-norm-folded kernel at build time. Activations: static per-tensor scales from max calibration (the ModelOpt FP8_DEFAULT_CFG statistic) on the family's PyTorch reference, 256 COCO 2017 validation images. fp16 Dequantize.

Why no gain: the convolutions run on FP8 kernels, but bias, SiLU and the next Quantize are not fused into the FP8 convolution, so the generated pointwise kernels grow from 45 to 64 (plan layers 137 to 159) and the small, launch-bound model ends up slower. FP8 depthwise convolutions have no TensorRT 11.4 kernel ("no tactics"), so the 7 depthwise convolutions of yolo11n stay fp16.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

python3 -m tools.model_ci validate
python3 -m pytest tools/tests/test_architecture.py tools/tests/test_public_source_hygiene.py -q
python3 tools/pr_metadata.py validate --event <event.json>   # Pull-request metadata is complete.
git diff --check

Documentation only.

Check	Result
python3 -m tools.model_ci validate	pass
pytest tools/tests/test_architecture.py tools/tests/test_public_source_hygiene.py (CPU)	62 passed
Markdown check of the README (table column counts, code fences)	pass
git diff --check	pass
python3 tools/pr_metadata.py validate on this description	pass

The numbers in the README come from the local study and are not re-run by this change (yolo11n, x is fp16 time over FP8 time):

GPU	TensorRT	fp16 ms	FP8 ms	x batch 1	x batch 8	plan MiB fp16 to FP8	AP fp16 / FP8
B200	11.4	0.433	0.467	0.93	0.89	6.56 to 4.65	0.4158 / 0.4034
B300	11.4	0.400	0.443	0.90	0.88	6.68 to 4.66	0.4158 / 0.4034
B200	11.1	0.434	0.469	0.92	0.88	6.61 to 4.62	0.4157 / 0.4042

No NaN or Inf on 500 held-out images and 3 stress inputs.

Hardware, Environment, and Revisions

• Repository: this commit on top of upstream/main at f8800ce3. The study ran on upstream/main at f8800ce3; families/yolo11 has not changed since.
• Checkpoint: Ultralytics/YOLO11 (yolo11n.pt) at 8b8ac7d1fae7468f85dbf89670dd66f41f485aab (manifest revision).
• Data: detection-datasets/coco validation images 0 to 255 (calibration) and 300 to 799 (held out), with their COCO 2017 annotations.
• Study: B200 and B300, one GPU per job, TensorRT 11.4.0.106 (target) and 11.1.0.106 (B200), pycocotools. Reference: the family's PyTorch yolo11n in fp32.

Not Run / Remaining Gaps

• No code was run for this change; the study cannot be rerun from this repository because the FP8 code was not merged.
• TensorRT 11.1 was measured on B200 only.
• Not measured: percentile or entropy calibration, per-tensor weight scales, a SiLU form that TensorRT might fuse into the FP8 convolution epilogue, larger widths, the full 5000-image COCO validation set, H100, the native runtime with an FP8 bundle.
• No problem on main was found during the study.

Contributor Self-Review

• I have completed a self-review of this change.

Notes For Future Readers

• Suggested review order: "Summary and decision", the speed table, "When to revisit".
• Revisit FP8 when a TensorRT release fuses bias, SiLU and the next Quantize into the FP8 convolution epilogue or adds FP8 depthwise kernels, or when the family adds a larger width.

Risk level

• Low
• Medium
• High

Risk rationale: documentation only; one README added, no code, build, bundle or test change.

🤖 Generated with Claude Code

[coderabbitai]

No actionable comments were generated in the recent review.

Summary

Summary

Adds families/yolo11/README.md to document an FP8 self-quantization study of yolo11n on TensorRT 11.4 and 11.1. The README reports that FP8 ran slower than FP16, reduced plan size by about 2 MiB, and lowered COCO AP by 1.2 points. It records the recipe, measurement method, an FP16-head variant, NVFP4 limitations, untested options, and conditions for revisiting FP8.

The README states that the FP8 implementation was not merged. This change adds no model code or quantized build.

Architecture impact

• Family-owned files: Adds one file, families/yolo11/README.md.
• Shared surfaces: No shared code or public API changes are present in the diff.
• Dependency direction: No new code dependencies are introduced.
• Affected consumers: No runtime or build consumers are changed. The README references the yolov8 and yolov10 studies.
• Unresolved blast radius: The reported benchmark and quality results were not independently verified from this documentation change.
• Review outcome: HUMAN REVIEW REQUIRED for the study’s measurement evidence. This is not a finding of an architecture violation.

Validation

CI, build, and test results on the current head: not available.

The new YOLO11 README describes the yolo11n TensorRT engine and an FP8 quantization study that is not shipped. It records the measured performance, plan size, and COCO AP results, along with study limitations and conditions for revisiting FP8.

Changes: YOLO11 documentation

Layer / File(s)	Summary
Engine and FP8 study documentation families/yolo11/README.md	The README documents the batch-1 fp16 engine, the FP8 recipe and measurements, study limitations, and revisit criteria. It reports that FP8 was slower than fp16, produced a smaller plan, and reduced COCO AP.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Other

Possibly related PRs

• NVIDIA/TensorRT-Model-Connect#2101: Documents an FP8 study of a YOLO detection engine and reports a related TensorRT fusion limitation.
• NVIDIA/TensorRT-Model-Connect#2093: Documents an FP8 study of a YOLO detection engine and reports a related TensorRT fusion limitation.

Merge Risk: Minimal · up to 7a8e1

The README documents an unshipped study and existing engine limits. No merge-blocking risk remains.

Architecture risk: Low · up to 7a8e1

The change affects 1 system.

Changed systems: families

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

• observed — families (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

• observed — Modified behavior in families/yolo11/README.md: Adds README documentation for the YOLO11 yolo11n TensorRT engine and an unshipped FP8 study, including its quantization recipe, benchmark and quality results, limitations, untried configurations, and revisit criteria. The document states that the family builds batch 1 only, has no quantized build, and that FP8 measured slower than fp16 with lower COCO AP despite a smaller plan.

Pre-merge checks: 8 · 1

Failed checks: 1 inconclusive

Check name	Status	Explanation	Resolution
Benchmark Validation Integrity		The new report compares fp16 and FP8 latency using CUDA graph replay in interleaved rounds (families/yolo11/README.md:73-84). The study code is not in the repository, so the timed regions and synchr…	Provide the study harness or its timing contract for both variants, including synchronization, transfers, and any output processing.

Passed checks: 8 passed

Check name	Status	Explanation
Title check		The title clearly summarizes the main change: documenting why FP8 is not shipped for YOLO11.
Description check		The description covers the required sections, identifies the documentation-only change, records validation results and study limitations, and includes a self-review and risk rationale.
Docstring Coverage		No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Family Ownership Boundary		The PR adds only families/yolo11/README.md. Lines 41–42 and 46–48 refer to sibling studies and their recipe, but do not introduce a dependency on another family's implementation or artifacts. No reg…
Shared Semantic Neutrality		The PR adds only families/yolo11/README.md. It does not change shared code.
Shared Change Blast Radius		The PR adds only families/yolo11/README.md. The diff does not change a shared surface, so the check does not apply.

Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

Full details: Family Ownership Boundary

Explanation

The PR adds only families/yolo11/README.md. Lines 41–42 and 46–48 refer to sibling studies and their recipe, but do not introduce a dependency on another family's implementation or artifacts. No registry or other family changes were introduced.

Full details: Benchmark Validation Integrity

Explanation

The new report compares fp16 and FP8 latency using CUDA graph replay in interleaved rounds (families/yolo11/README.md:73-84). The study code is not in the repository, so the timed regions and synchronization for both variants cannot be verified.

Resolution

Provide the study harness or its timing contract for both variants, including synchronization, transfers, and any output processing.

Recent review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/TensorRT-Model-Connect
• Branch: main
• File: .coderabbit.yaml
• Commit: f8800ce
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: f6df093b-ebfc-4fc5-86f5-067c59b67aed

Commits

Reviewing files that changed from the base of the PR and between f8800ce and 7a8e1d9.

Files selected for processing: 1

• families/yolo11/README.md

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.