From 113be24e730ff8f7cdf792695504d93a7771f226 Mon Sep 17 00:00:00 2001 From: Zhenshan Xie Date: Mon, 9 Nov 2026 17:01:02 -0800 Subject: [PATCH] docs(timm_regnet): record why FP8 is not shipped Add a README to the timm_regnet family with the FP8 self-quantization study of regnety_040 on TensorRT 11.4 (B200, B300) and 11.1: recipe (FP8 weights and static per-tensor activation scales on the 48 dense 1x1 convolutions; stem, grouped 3x3 and squeeze-excite convolutions kept fp16), measurement method, speed, plan size, ImageNet top-1 against PyTorch fp32, kernel time per group and the variants. FP8 is slower than fp16 at batch 1 (0.95x to 0.96x) and at batch 32 (0.94x to 0.95x): the small 1x1 convolutions save less time than the stand-alone Quantize kernels after residual adds and squeeze-excite products cost. In FP8, TensorRT splits each grouped convolution into one convolution per group (165 to 335 kernels, 0.53x to 0.72x), so grouped convolutions stay fp16. With builder optimization level 3 FP8 is still slower than fp16 (0.94x to 0.97x). FP8 is not shipped. NVFP4 was not tried (no NVFP4 convolution in TensorRT). Documentation only; no code change. Signed-off-by: Zhenshan Xie --- families/timm_regnet/README.md | 169 +++++++++++++++++++++++++++++++++ 1 file changed, 169 insertions(+) create mode 100644 families/timm_regnet/README.md diff --git a/families/timm_regnet/README.md b/families/timm_regnet/README.md new file mode 100644 index 0000000000..218194379c --- /dev/null +++ b/families/timm_regnet/README.md @@ -0,0 +1,169 @@ +# timm RegNet + +This family builds `timm/regnety_040.ra3_in1k` (RegNetY-4.0GF, 224 x 224) for +`image_to_class_scores` as one TensorRT engine at batch 1. The manifest builds +fp16 (fp32 is also supported). The family has no quantized build. + +## FP8 quantization study (not shipped) + +FP8 self-quantization of the convolutions was built and measured on TensorRT +11.4 (and 11.1) for `timm/regnety_040.ra3_in1k`. It is not shipped: FP8 is +slower than fp16 at batch 1 (0.94x to 0.96x) and at batch 32 (0.94x to 0.96x) +on B200 and B300, and FP8 grouped convolutions are split into one convolution +per group (0.53x to 0.72x). This section records the recipe, the method and +the numbers, so that the study does not have to be repeated. The FP8 code was +not merged; this family has no FP8 build. + +### Summary and decision + +| Item | fp16 (manifest) | FP8 (fp16 engine) | FP8 vs fp16 | +|---|---|---|---| +| Latency, batch 1, B200 / B300 (TensorRT 11.4) | 0.520 / 0.510 ms | 0.541 / 0.538 ms | 0.96x / 0.95x | +| Latency, batch 32, B200 / B300 (TensorRT 11.4) | 1.258 / 1.188 ms | 1.320 / 1.264 ms | 0.95x / 0.94x | +| Plan size, batch 1, B200 / B300 | 40.5 / 40.0 MiB | 30.0 / 29.5 MiB | -26% | +| ImageNet top-1, B200 (PyTorch fp32 82.67%) | 82.73% | 82.40% | -0.3 points | +| Top-1 agreement with PyTorch, B200 | 99.93% | 96.87% | -3.1 points | +| Non-finite outputs | 0 | 0 | none | + +Decision: do not ship FP8 for timm RegNet. + +- Speed: the 48 FP8 1x1 convolutions are small at batch 1 (7 x 7 to 112 x 112 + maps) and save less kernel time than the stand-alone Quantize kernels cost. + Each block input feeds both the shortcut add and the first 1x1 convolution, + and the squeeze-excite gate multiplies the 3x3 convolution output, so + TensorRT cannot write the FP8 activation from the producing kernel: it adds + separate multiply-and-cast kernels (165 to 187 kernels per call). +- Grouped convolutions: in FP8, TensorRT 11.4 splits each of the 22 grouped + 3x3 convolutions (group width 64) into one convolution per group (152 FP8 + convolution kernels instead of 22; plan 167 to 337 layers), and the engine + is about 2x slower at batch 1. So the grouped convolutions stay fp16. +- Plan size: the 26% smaller plan (38% with the grouped convolutions in FP8) + comes with no speed gain, so it is not a reason to ship. + +### Recipe that was measured + +No FP8 or NVFP4 checkpoint of this model exists on the Hub, and no official +quantization config exists for it. Each quantized convolution is built as +`input -> Quantize -> Dequantize(fp16) -> Convolution(+bias)`, with the kernel +`FP8 constant -> Dequantize(fp16)`. + +| Choice | Decision | +|---|---| +| Quantized (48 of 115 convolutions) | every dense convolution on a spatial map larger than 1x1 (the 1x1 convolutions of each block and the downsample shortcuts) | +| fp16 | the 3x3 stem (3 input channels), the 22 grouped 3x3 convolutions, the 44 squeeze-excite convolutions (on a 1x1 map), the classifier | +| Weight scale | per tensor (amax / 448) of the batch-norm-folded kernel at build time; per output channel measured as a variant | +| Activation scale | static, per tensor, max \|input\| / 448 over 500 calibration images (the statistic of ModelOpt max calibration), read from an fp32 engine of the family graph | +| Outlier rule | a scale above 20x the median of its kind stays fp16; measured: no outlier (largest ratio 2.1x) | +| Dequantize type | fp16; no non-finite value, fp32 Dequantize was not needed | + +### Method + +- GPUs: B200 and B300, one GPU per job, all variants built and timed in the + same job; TensorRT 11.4.0.106 (target) and 11.1.0.106. +- Data: 2000 ImageNet validation images from `clip-benchmark/wds_imagenet1k`: + 500 for calibration, the other 1500 held out. Family preprocessing. +- Quality: top-1 against the labels, top-1 agreement and KL divergence against + the timm PyTorch fp32 model on the 1500 held-out images. +- Non-finite scan: the 2000 images and 29 stress inputs (noise with standard + deviation 1, 3 and 10, constants), all variants, B200 and B300. +- Speed: CUDA graph replay, median of 9 interleaved rounds of 100 calls, + spread at most 2.7%. Batch 32 is a scratch build (the family builds batch 1 + only). The engines use the family graph and builder settings (builder + optimization level 1); the family code did not change between the study + and this note. + +### Results + +Speed and size, TensorRT 11.4 (ms per engine call; x is fp16 time over FP8 +time): + +| GPU | Batch | fp32 ms | fp16 ms | FP8 ms | x | FP8 per-channel ms (x) | FP8 + grouped ms (x) | +|---|---|---|---|---|---|---|---| +| B200 | 1 | 0.814 | 0.520 | 0.541 | 0.96 | 0.543 (0.96) | 0.964 (0.54) | +| B200 | 32 | 2.176 | 1.258 | 1.320 | 0.95 | 1.316 (0.96) | 1.759 (0.72) | +| B300 | 1 | 0.779 | 0.510 | 0.538 | 0.95 | 0.542 (0.94) | 0.970 (0.53) | +| B300 | 32 | 2.088 | 1.188 | 1.264 | 0.94 | 1.263 (0.94) | 1.693 (0.70) | + +Plan size and layers (batch 1, B300): fp16 40.0 MiB and 167 layers, FP8 +29.5 MiB and 189 layers, FP8 + grouped 24.8 MiB and 337 layers. + +TensorRT 11.1 sanity check (B200, same method, latest main): + +| Batch | fp16 ms | FP8 ms | x | FP8 + grouped ms (x) | Top-1 fp16 / FP8 | +|---|---|---|---|---|---| +| 1 | 0.498 | 0.539 | 0.92 | 0.935 (0.53) | 82.67% / 82.33% | +| 32 | 1.254 | 1.310 | 0.96 | 1.750 (0.72) | 82.73% / 82.73% | + +TensorRT 11.1 gives the same kernel counts as 11.4 (165 fp16, 187 FP8, 335 +FP8 + grouped), so the grouped convolutions are also split per group on 11.1. + +Quality (1500 held-out images; TensorRT 11.4): + +| Engine | Top-1 B200 | Top-1 B300 | Agreement with PyTorch top-1 (B200) | KL mean (B200) | KL p99 (B200) | +|---|---|---|---|---|---| +| PyTorch fp32 | 82.67% | - | - | - | - | +| fp16 engine | 82.73% | 82.67% | 99.93% | < 0.0001 | 0.0001 | +| FP8 | 82.40% | 82.87% | 96.87% | 0.0107 | 0.162 | +| FP8 per-channel weights | 83.00% | 82.60% | 96.87% | 0.0091 | 0.108 | +| FP8 + grouped | 82.60% | 82.73% | 96.20% | 0.0139 | 0.170 | + +No variant gives a NaN or Inf on the 2000 images and 29 stress inputs (B200, +B300, batch 1 and 32). The FP8 KL is about 3x the KL of the same recipe on +timm ConvNeXt (0.0107 against 0.0032, B200, batch 1). + +Kernel time per group, TensorRT 11.4, batch 1 (CUDA profiler, sum of kernel +time per call, ms): + +| GPU | Kernel group | fp16 | FP8 | FP8 + grouped | +|---|---|---|---|---| +| B200 | convolutions and GEMMs | 0.508 | 0.498 | 1.133 | +| B200 | other (pointwise, Quantize, pooling) | 0.127 | 0.188 | 0.241 | +| B200 | kernels per call | 165 | 187 | 335 | +| B300 | convolutions and GEMMs | 0.598 | 0.558 | 1.166 | +| B300 | other (pointwise, Quantize, pooling) | 0.053 | 0.107 | 0.171 | +| B300 | kernels per call | 165 | 187 | 335 | + +The FP8 convolutions save 0.01 to 0.04 ms; the added Quantize kernels cost +0.05 to 0.06 ms. + +### Builder optimization level 3 + +The numbers above use builder optimization level 1, the family setting when +the study was made. Level 3 makes the fp16 engine faster (batch 1, TensorRT +11.4, family benchmark: 0.501 to 0.448 ms on B200 and 0.506 to 0.453 ms on +B300, 1.12x). The same FP8 recipe built at level 3 is still slower than fp16 +at level 3 (TensorRT 11.4, same method as above, one job per GPU): + +| GPU | Batch | fp16 ms | FP8 ms | x | FP8 + grouped ms (x) | +|---|---|---|---|---|---| +| B200 | 1 | 0.453 | 0.482 | 0.94 | 0.888 (0.51) | +| B200 | 32 | 1.175 | 1.209 | 0.97 | 1.669 (0.70) | +| B300 | 1 | 0.472 | 0.490 | 0.96 | 0.874 (0.54) | +| B300 | 32 | 1.116 | 1.159 | 0.96 | 1.583 (0.71) | + +At level 3 FP8 top-1 is 82.60% (B200) and 82.67% (B300) against 82.67% for +fp16, with no NaN or Inf, and the FP8 + grouped engine still runs 331 kernels +per call against 164 for fp16 (the grouped convolutions are still split per +group). + +### NVFP4 (not tried) + +TensorRT does not support NVFP4 on convolutions (block-scaled FP4 needs a 2-D +GEMM), FP8 already loses to fp16, and the engine is not limited by weight +reads (plan 40 MiB). + +### What was not tried + +| Item | Note | +|---|---| +| Percentile or entropy calibration, quantization-aware training | not measured; could lower the KL, not the latency | +| Other RegNet sizes, H100, native runtime with an FP8 bundle | not measured | + +### When to revisit + +- A TensorRT release that fuses the Quantize into the producing layer after + residual adds and squeeze-excite products. +- A TensorRT FP8 kernel for grouped convolutions that keeps one layer per + convolution. +- A larger RegNet in this family, or larger batches, where the 1x1 + convolutions are limited by math rather than by activation traffic.