Repository navigation
[Engine] HIP gfx103x: the start-up pcie_frac (bandwidth only) is too high when the CPU pool is fast; the PcieAuto refinement (opt-in there) finds 0.20 — enable it by default on gfx103x? #4007
Description
What I see
On gfx103x the native pack's PCIe share comes from the start-up rule only, 0.55 x min(1, probed GB/s / 20): the runtime refinement (PcieAuto) is the default on gfx12 (decode_defaults_arch_ok) and opt-in elsewhere (STRATA_PCIE_FRAC_DEFAULT=auto). The start-up rule looks at the link and not at the CPU pool, and with a fast pool the share it picks makes the GPU wait for its PCIe experts.
RX 6900 XT, Qwen3.8-Flash-Next IQ3_S pack, main 7dd5cf1, single card, benchmark.py-style 4K / 32K prompts, two runs per arm, interleaved cold starts:
| machine | link (probe) | start-up share | refined share (log) | decode 4K / 32K, start-up | decode 4K / 32K, refined |
|---|---|---|---|---|---|
| Windows 11, i9-10900K, PCIe 3.0 x16 | 12.6 GB/s | 0.35 | 0.20 (pool 70 us per missed expert, 29.4 GB/s) | 35.4, 35.5 / 37.1, 34.8 | 42.6, 42.5 / 43.8, 44.0 |
| Linux, Ryzen 5 5600X, PCIe 4.0 x8 | 14.1 GB/s | 0.39 | 0.25 then 0.20 (pool 84 then 77 us, 26.7 GB/s) | 48.6, 48.7 / 50.5, 50.6 | 49.6, 49.6 / 51.0, 51.0 |
Prompt reads do not change (Windows 796-802 / 1,154-1,163 tok/s in every arm, Linux 951-959 / 1,134-1,137).
A fixed sweep on the Windows box agrees with the refined value: decode 4K / 32K at --pcie-frac 0 45.4 / 45.9, 0.1 44.9 / 43.6, 0.2 46.5 / 47.6, 0.3 39.1 / 40.8, 0.35 (auto) 36.4 / 36.9 (two runs each, interleaved). On Windows the share costs more than on Linux because every DMA batch is a WDDM submission (~120 µs more per batch end to end than on Linux in a HIP probe), but the refinement does not need to know that: the pool is simply faster than the link, and the formula link / (link + 1.85 x pool rate) lands on the measured best.
The 4K decode on Windows stays below the fixed 0.2 (42.6 vs 46.5) because the first update comes after 48 windows, so a short answer runs mostly at the start-up share.
Question
Would you take a change that makes PcieAuto the default on gfx103x (pcie_auto_fork_arch_ok returning true there unless STRATA_PCIE_FRAC_DEFAULT=old)? I can open it with 10 interleaved pairs on both machines. I have no gfx11 to say anything about the 7900-series reports in the code comment (one "no regression", one "never engages").
Found while measuring the prompt path on Windows for #4005.
Developed with an AI coding assistant; every number above was measured on the two machines named in the table.
Activity
Niko1221 commentedon Oct 29, 2026
Thank you. Your numbers agree with #3984: on RDNA2 with a fast CPU pool the start-up PCIe share is too high, and
PcieAutoor--pcie-frac 0.2gave +20% decode on your Windows RX 6900 XT and +13.7% on an RX 6700 XT in #3984. The start-up share was not changed in 0.2.7. Yes, please open the change that makesPcieAutothe default on gfx103x with the ten interleaved pairs on both machines; we have no gfx103x card, so those pairs are what it would be judged on.kulight commentedon Oct 29, 2026
A correction to the RX 6700 XT figure quoted above from #3984: the +13.7% there was measured against a hand-set
--pcie-frac 0.75, not the start-up share. On that machine the start-up rule picks 0.55, and against that--pcie-frac 0.2is about +12% in decode (45.3 → 50.6 tok/s, medians of 3, same session, all other flags fixed; full sweep with the peak at 0.2 in #3984 (comment)). Same direction, slightly smaller. That run used an abliterated IQ2_XS derivative, andPcieAutoitself was not measured there.