Benchmark certificate · 2026-09-24.1
Both images passed their complete HTTP E2E suites before benchmarking. Prefill measures reading the prompt; decode measures generating the answer. Rates are tokens per second, with higher values indicating faster inference.
Each chart row uses its own AVX512 result as the reference. Compare printed token rates across configurations, rather than bar lengths.
AVX512 certificate · AVX2 certificate · Combined results · SVG chart
| Model / configuration | AVX512 prefill | AVX512 decode | AVX2 prefill | AVX2 decode |
|---|---|---|---|---|
| Qwen3.6-35B-A3B-UD-IQ3_S / 2×CPU · auto TP | 20.3 | 10.4 | 24.8 | 7.9 |
| Ornith-1.5-35B-Q4_K_M / 2×CPU · auto TP | 20.3 | 13.2 | 21.8 | 9.9 |
| Qwen3.5-122B-A10B-UD-Q8_K_XL-00001-of-00004 / 2×CPU + 2×CUDA · auto ExpertOverlay | 171.8 | 13.4 | 107.2 | 13.1 |
| Qwen3.5-122B-A10B-UD-Q8_K_XL-00001-of-00004 / 2×CPU + 2×CUDA + 4×ROCm · auto ExpertOverlay | 160.1 | 18.9 | 156.5 | 18.6 |
| Qwen3.5-122B-A10B-UD-Q8_K_XL-00001-of-00004 / 2×CPU + 2×ROCm · auto ExpertOverlay | 195.0 | 23.5 | 106.9 | 23.3 |
| Qwen3.5-122B-A10B-UD-Q8_K_XL-00001-of-00004 / 2×CPU + 4×ROCm · auto ExpertOverlay | 228.8 | 24.0 | 165.2 | 23.7 |
| Ornith-1.5-35B-Q4_K_M / 2×CUDA · auto TP | 1,049.8 | 106.1 | 1,061.7 | 106.3 |
| Qwen3.6-35B-A3B-UD-IQ3_S / 1×CUDA · auto single | 2,608.5 | 225.5 | 2,579.1 | 224.0 |
| Qwen3.8-27B-IQ4_XS / 2×CUDA · auto TP | 741.8 | 50.8 | 740.3 | 50.8 |
| Qwen3.8-27B-IQ4_XS / 2×CUDA · auto PP | 1,124.6 | 50.8 | 1,121.7 | 50.8 |
| Qwen3.8-27B-IQ4_XS / 1×CUDA · auto single | 913.2 | 62.7 | 915.5 | 62.7 |
| Qwen3.5-122B-A10B-UD-Q8_K_XL-00001-of-00004 / 2×CUDA + 4×ROCm · auto ExpertOverlay | 188.6 | 23.1 | 197.9 | 23.9 |
| Qwen3.8-27B-IQ4_XS / 2×CUDA + 2×ROCm · auto PP | 499.1 | 36.7 | 502.2 | 36.7 |
| Ornith-1.5-35B-Q4_K_M / 2×ROCm · auto TP | 1,042.6 | 65.9 | 1,041.5 | 66.0 |
| Qwen3.6-35B-A3B-UD-IQ3_S / 1×ROCm · auto single | 1,160.3 | 74.7 | 1,156.3 | 74.1 |
| Ornith-1.5-35B-Q4_K_M / 1×ROCm · auto single | 1,160.5 | 100.3 | 1,164.5 | 100.4 |
| Qwen3.8-27B-IQ4_XS / 2×ROCm · auto TP | 373.4 | 38.6 | 372.2 | 38.6 |
| Qwen3.8-27B-IQ4_XS / 2×ROCm · auto PP | 271.8 | 28.0 | 271.8 | 28.1 |
| Qwen3.8-27B-IQ4_XS / 1×ROCm · auto single | 274.2 | 36.6 | 270.8 | 36.6 |