Repository navigation
[Benchmark] Single RX 6700 XT 12GB (gfx1031) on Ubuntu 26.04, Qwen IQ2_XS (128K context, engine 0.1.48 hip-rx6000) - 58 tok/s decode, 700 tok/s prefill #3984
Description
Title:
[Benchmark] Single RX 6700 XT 12GB (gfx1031) on Ubuntu 26.04, Qwen IQ2_XS (128K context, engine 0.1.48 hip-rx6000) - 58 tok/s decode, 700 tok/s prefill
Body:
Report from a single RX 6700 XT owner running the native rx6000 package on Linux, validating 0.1.48 engine stability, 128K context, and calibration tuning.
System Environment
- GPU: 1x AMD Radeon RX 6700 XT 12 GB (Navi 22,
gfx1031, PCIe 4.0 x16) - CPU: AMD Ryzen 7 5800X (8 cores / 16 threads)
- RAM: 64 GB DDR4 (system memory peak ~47.1 GB during load, 0B swap)
- OS: Ubuntu 26.04.1 LTS, kernel
7.0.0-38-generic, in-kernel amdgpu - Strata:
strata 0.2.4.1 (build 0.1.49.1; python 3.12.15), engine component0.1.48-linux-x86_64-hip-rx6000 - Installation: Local Terminal install (
strata launch/ systemd-managed)
Model & Configuration
- Model: Qwen3.8-Flash-Next IQ2_XS (derivative SC117-abliterated weights verified bit-for-bit against official IQ2_XS expert layout)
- Context: 131,072 (128K tokens)
- Launch parameters:
strata launch --model-path ../models/qwen3.8-flash-next-gsq-rco/IQ2_XS \ --model iq2_xs --context 128k --vision off --mtp on --tune skip \ --access local --set STRATA_HIP_PROMPT_F16=1 - VRAM / RAM Allocation:
- Experts loaded into RAM: 33.02 GiB
- GPU expert cache: 4,282 experts (~5.75 GiB VRAM)
Measured Benchmark Results
Tested across short and long-context needle passes (5/5 distributed facts verified at each context depth):
1. Stock Engine 0.1.48 (default --pcie-frac 0.75)
- Decode (Code): 51.1 tok/s median
- Decode (Prose/Story): 38.9 tok/s median
- 8K Cold: TTFT 11.21s | Prefill: 701.1 tok/s | Decode: 41.6 tok/s (5/5 facts)
- 32K Cold: TTFT 47.74s | Prefill: 688.3 tok/s | Decode: 60.4 tok/s (5/5 facts)
- 65K Cold: TTFT 94.25s | Prefill: 686.2 tok/s | Decode: 55.7 tok/s (5/5 facts)
- Cached follow-up TTFT: 0.42s - 1.05s
2. Calibrated Engine (--pcie-frac 0.20)
Running the built-in calibration (strata profile ...) identified that lowering PCIe transfer fraction from 0.75 to 0.20 yielded a ~12-14% decode boost on RDNA2:
- Decode (Code): 58.1 tok/s median (+13.7%)
- Decode (Prose/Story): 43.6 tok/s median (+12.1%)
- 8K Cold: TTFT 11.48s | Prefill: 702.1 tok/s | Decode: 48.6 tok/s (5/5 facts)
- 32K Cold: TTFT 45.70s | Prefill: 701.6 tok/s | Decode: 71.7 tok/s (5/5 facts)
- 65K Cold: TTFT 97.95s | Prefill: 676.6 tok/s | Decode: 62.8 tok/s (5/5 facts)
- Cached follow-up TTFT: 0.40s - 0.93s
3. Long Context (128K Capacity)
- Verified full 123K-token cold retrieval without OOM or kernel panics.
- Cached follow-up at 123K tokens returned TTFT of 1.508s.
Observations & Stability
- Zero crashes, GPU hangs, or driver timeouts over extended multi-hour runs on kernel 7.0 and modern amdgpu.
- The
0.1.48-linux-x86_64-hip-rx6000prebuilt engine works reliably out of the box on single Navi 22 hardware. - Calibrating PCIe fraction significantly benefits single-card DDR4 memory setups.
Activity
Niko1221 commentedon Oct 29, 2026
Thank you for the benchmark. Your run shows
--pcie-frac 0.2012 to 14% faster in decode than the start-up share (51.1 to 58.1 tok/s on code, 38.9 to 43.6 on prose), the same direction as #4007 on an RX 6900 XT. The start-up share was not changed in 0.2.7. #4007 asks to makePcieAutothe default on gfx103x; your numbers are useful there.kulight commentedon Oct 29, 2026
Thanks, and a correction before these numbers go into #4007: the "stock 0.75" baseline in the report above is mislabeled.
--pcie-frac 0.75I had carried over by hand from an older Docker calibration. The start-up rule on this machine picks 0.55 (measurements.default.pcie_fracin the calibration report; the link is PCIe 4.0 x16, 16 GT/s x16 from sysfs).strata serve --profilewith that profile. The "verified bit-for-bit" line is wrong, please ignore it.strata engine calibrate, notstrata profile.Clean data for #4007: the
--pcie-fracsweep fromstrata engine calibrate --profile <profile> --measure --exclusive(engine 0.1.48 hip-rx6000, single RX 6700 XT 12 GB, Ryzen 7 5800X, 7 pool workers, same model, same session, all other flags fixed, decode tok/s, 3 runs each, median [runs]):--pcie-fracThat puts 0.2 vs the real start-up share at about +12% (45.3 → 50.6), same direction as the RX 6900 XT, with the same peak at 0.2. The calibrator's own confirm step (pcie-frac and spec-min-p changed together, so not pcie alone) gave 44.6 → 50.1.
Not measured:
STRATA_PCIE_FRAC_DEFAULT=auto(PcieAuto) itself, and the original non-abliterated IQ2_XS. I can run interleaved start-up vsautopairs on the original IQ2_XS if that would help the gfx103x change.Niko1221 commentedon Oct 29, 2026
Thank you for measuring this and writing it up. We are building a Strata website with a page for community benchmarks, and results like these will be added there when it launches, so we are closing this here. If your numbers change with a new version, you are welcome to post them again.