[zhenshanx-nv]

Background

families/yolov10 builds jameslahm/yolov10n, yolov10s and yolov10x as one end-to-end detector engine (640 x 640, batch 1, fp16 in the manifests) and has no quantized build. No FP8 or NVFP4 checkpoint of any YOLO detector exists on the Hub, so FP8 self-quantization of the convolutions (build-time per-channel FP8 weights, ModelOpt-calibrated static activation scales, the convolution recipe of the stable_diffusion FP8 build) was built and measured for yolov10x on TensorRT 11.4 and 11.1. FP8 does not pay for this family: the plan is 43% smaller (61.9 to 35.5 MiB), but latency stays at the fp16 level (0.98x to 1.02x at batch 1) and COCO AP drops from 0.5730 to 0.5697; yolov10n and yolov10s are slower than fp16 in FP8. This change adds a README for the family that records the study, like the bart (#2004), timesfm (#2005) and marian (#2035) studies, so that it does not have to be repeated. The FP8 code is not part of this change. NVFP4 was not tried: TensorRT does not support NVFP4 on convolutions.

Exit Criteria

• The README records the recipe, the method, speed and plan size on B200 and B300 (TensorRT 11.4, 11.1), AP against PyTorch fp32, the kernel breakdown that explains the missing speedup, the recipe variants, yolov10n and yolov10s, NVFP4 and when to revisit.
• No code, test, manifest or bundle change; builds are unchanged.
• Non-goals: shipping FP8 or NVFP4 builds, an FP8 manifest or CI case.

Implementation

• families/yolov10/README.md (new): a short family description and the section "FP8 quantization study (not shipped)": summary and decision, recipe, method, results (speed, quality, kernel time per group, variants), NVFP4, what was not tried, when to revisit.

Recipe that was measured (code kept out of the repository):

• FP8 for the 108 dense convolutions of yolov10x whose input and output channels are multiples of 16; the stem, all 64 depthwise convolutions, the attention matmuls and the decode stay fp16.
• Weights: per-output-channel scale amax / 448 of the batch-norm-folded kernel at build time. Activations: static per-tensor scales from ModelOpt FP8_DEFAULT_CFG (max) on the family's PyTorch reference, 256 COCO 2017 validation images. fp16 Dequantize.

Why no gain: the FP8 convolutions are about 11% faster, but bias, SiLU and the next Quantize are not fused into the FP8 convolution, so the pointwise kernels grow from 62 to 104 and take the time back. FP8 depthwise convolutions have no TensorRT kernel.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

python3 -m tools.model_ci validate
python3 -m pytest tools/tests/test_architecture.py tools/tests/test_public_source_hygiene.py -q
python3 tools/pr_metadata.py validate --event <event.json>   # Pull-request metadata is complete.
git diff --check

Documentation only.

Check	Result
python3 -m tools.model_ci validate	pass
pytest tools/tests/test_architecture.py tools/tests/test_public_source_hygiene.py (CPU)	62 passed
Markdown check of the README (table column counts, code fences)	pass
git diff --check	pass
python3 tools/pr_metadata.py validate on this description	pass

The numbers in the README come from the local study and are not re-run by this change (yolov10x, batch 1, x is fp16 time over FP8 time):

GPU	TensorRT	fp16 ms	FP8 ms	x	plan MiB fp16 to FP8	AP fp16 / FP8
B200	11.4	1.406	1.377	1.02	61.9 to 35.5	0.5730 / 0.5697
B300	11.4	1.323	1.319	1.00	62.0 to 35.5	0.5734 / 0.5701
B200	11.1	1.412	1.385	1.02	61.6 to 35.1	same as 11.4
B300	11.1	1.305	1.329	0.98	61.7 to 35.2	same as 11.4

yolov10n and yolov10s with the same recipe: 0.93x and 0.97x at batch 1 (B200), 0.35 to 0.9 AP point lower. No NaN or Inf on 500 held-out images and 20 stress inputs.

Hardware, Environment, and Revisions

• Repository: this commit on top of upstream/main at 7d410eda.
• Checkpoints: jameslahm/yolov10x at 6d7c9b6871ffa50dfd2764fa896bc0e33294233c, jameslahm/yolov10n at 3b91f578c5b492080f191a26aa35711bb44f608c, jameslahm/yolov10s at 9096a248f4ab5dd21b3d9e9fa0295534db1b87ff (manifest revisions).
• Data: detection-datasets/coco validation images 0 to 255 (calibration) and 300 to 799 (held out), with their COCO 2017 annotations.
• Study: B200 and B300, one GPU per job, TensorRT 11.4.0.106 (target) and 11.1.0.106, torch 2.14.0 (CUDA 13), ModelOpt 0.46.1, pycocotools 2.0.11. Reference: the family's PyTorch yolov10x in fp32.

Not Run / Remaining Gaps

• No code was run for this change; the study cannot be rerun from this repository because the FP8 code was not merged.
• Not measured: FP8 attention matmuls, percentile or entropy calibration, a SiLU form that TensorRT might fuse into the FP8 convolution epilogue, the full 5000-image COCO validation set, H100, the native runtime with an FP8 bundle.
• No problem on main was found during the study.

Contributor Self-Review

• I have completed a self-review of this change.

Notes For Future Readers

• Suggested review order: "Summary and decision", the kernel time table, "When to revisit".
• Revisit FP8 when a TensorRT release fuses bias, SiLU and the next Quantize into the FP8 convolution epilogue, or adds an FP8 depthwise convolution kernel.

Risk level

• Low
• Medium
• High

Risk rationale: documentation only; one README added, no code, build, bundle or test change.

🤖 Generated with Claude Code

[coderabbitai]

Summary

Summary

Adds families/yolov10/README.md with the YOLOv10 engine configuration and an FP8 quantization study that is not shipped. The documented recipe quantizes 108 of yolov10x’s 173 convolutions and keeps depthwise convolutions, attention matmuls, and decode in FP16.

The README reports that yolov10x FP8 plans are about 43% smaller than FP16, with batch-1 latency near parity and a 0.3-point COCO AP decrease. It attributes the limited speedup to separate pointwise kernels for bias, SiLU, and the next Quantize operation. It also reports slower FP8 results for yolov10n and yolov10s. The README records the study method, variants, limitations, and conditions for revisiting FP8. No FP8 build is added.

Architecture impact

• Family-owned files: One new file, families/yolov10/README.md.
• Shared surfaces: None changed.
• Dependency directions: None added.
• Affected consumers: YOLOv10 documentation readers. No runtime or build consumers change.
• Unresolved blast-radius questions: The README says the study was not rerun for this change. It also lists full COCO validation, H100, and native runtime with an FP8 bundle as unmeasured.
• Review outcome: HUMAN REVIEW REQUIRED. The change is confined to family documentation, but the historical study measurements were not rerun for this change. Reviewers cannot verify those measurements from completed checks on the current head.
• CI, build, and test results: not available.

The YOLOv10 README now documents the TensorRT engine configuration and an FP8 quantization study. It records the study’s recipe, evaluation methods, reported measurements, tested and untried variants, and conditions for revisiting FP8. It states that the family has no quantized build.

Changes: YOLOv10 FP8 Study Documentation

Layer / File(s)	Summary
Engine configuration and FP8 decision families/yolov10/README.md	The README describes the engine configuration, reports FP8 comparison results, and states that FP8 is not shipped.
FP8 recipe and evaluation method families/yolov10/README.md	The README records the quantization recipe, evaluation setup, quality checks, and timing procedure.
Measured results and variants families/yolov10/README.md	The README reports performance, plan-size, quality, kernel timing, and model-variant measurements.
Untried options and revisit conditions families/yolov10/README.md	The README lists untried approaches and evaluation configurations, and states conditions for revisiting FP8.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Merge Risk: Minimal · up to 100a6

The change documents existing behavior. The measured batch-size results remain near parity, and no merge-blocking risk is established.

Architecture risk: Low · up to 100a6

The change affects 1 system.

Changed systems: families

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

• observed — families (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

• observed — Modified behavior in families/yolov10/README.md: The README describes the YOLOv10 engine configuration and documents the FP8 study as not shipped, including its reported plan-size, latency, and quality results.
• observed — Modified behavior in families/yolov10/README.md: The README records the FP8 comparison metrics and the stated decision not to ship FP8, with reported reasons including latency, depthwise convolution support, and smaller-model results.
• observed — Modified behavior in families/yolov10/README.md: The README documents the measured FP8 recipe, including which convolution groups were quantized or kept in fp16, scale choices, calibration data, and the reported outlier and non-finite checks.
• observed — Modified behavior in families/yolov10/README.md: The README describes the study’s hardware, TensorRT versions, quality and agreement measurements, non-finite scan, and timing procedure; it notes that batch 8 was a scratch build.

Pre-merge checks: 8 · 1

Failed checks: 1 inconclusive

Check name	Status	Explanation	Resolution
Benchmark Validation Integrity		The new report compares FP16 and FP8 latency, but the measurement boundaries cannot be verified. families/yolov10/README.md:70-71 states that the study used CUDA graph replay, but does not describe …	Provide the FP16 and FP8 study measurement code or a detailed timing contract for both paths, including synchronization, transfers, and timed regions.

Passed checks: 8 passed

Check name	Status	Explanation
Title check		The title clearly and concisely describes the README’s documentation of why FP8 is not shipped for YOLOv10.
Description check		The description covers the required sections, including background, exit criteria, implementation, change category, validation, environment, remaining gaps, self-review, and risk. It explains the docu…
Docstring Coverage		No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Family Ownership Boundary		The PR only adds families/yolov10/README.md. Lines 44–47 describe guidance used for stable_diffusion, but do not introduce a code or artifact dependency. Lines 15–17 state that the study code was …
Shared Semantic Neutrality		The PR adds only families/yolov10/README.md. It does not change shared code.
Shared Change Blast Radius		The PR adds only families/yolov10/README.md. The diff does not change a shared surface, so the shared change check does not apply.

Full details: Description check

Explanation

The description covers the required sections, including background, exit criteria, implementation, change category, validation, environment, remaining gaps, self-review, and risk. It explains the documentation-only scope and reports the study’s limitations. It does not link an originating issue or discussion, but the rest of the description is complete.

Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

Full details: Family Ownership Boundary

Explanation

The PR only adds families/yolov10/README.md. Lines 44–47 describe guidance used for stable_diffusion, but do not introduce a code or artifact dependency. Lines 15–17 state that the study code was not merged.

Full details: Benchmark Validation Integrity

Explanation

The new report compares FP16 and FP8 latency, but the measurement boundaries cannot be verified. families/yolov10/README.md:70-71 states that the study used CUDA graph replay, but does not describe synchronization or transfers for each engine. The study code was not merged. The existing PyTorch reference benchmark uses a different timing path and cannot establish the study's timing contract.

Resolution

Provide the FP16 and FP8 study measurement code or a detailed timing contract for both paths, including synchronization, transfers, and timed regions.

[coderabbitai]

Additional comments: 1

families/yolov10/README.md-142-143: 1

142-143: Minor · Functional Correctness · Quick win

Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Limit the batch-size conclusion to the measured cases.

On B300 with TensorRT 11.1, the yolov10x ratio changes from 0.98x at batch 1 to 1.02x at batch 8. No batch above 8 was measured. The results therefore do not establish that larger batches cannot change the outcome. (raw.githubusercontent.com)

Review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/TensorRT-Model-Connect
• Branch: main
• File: .coderabbit.yaml
• Commit: 7d410ed
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 196ce923-542b-44ce-8871-e70f9d3b4c1f

Commits

Reviewing files that changed from the base of the PR and between 7d410ed and 100a67c.

Files selected for processing: 1

• families/yolov10/README.md

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 6 remain after this review.