In case someone is curious, here are the results of the llama.cpp benchmark on the Orion O6 board running Ubuntu 25.04, but with a discrete GPU, namely AMD Radeon RX 7900 XTX, accessed via the Vulkan API:
$ ./llama-bench -m Qwen_Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon RX 7900 XTX (RADV NAVI31) (radv) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | Vulkan | 99 | pp512 | 1698.22 ± 28.68 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | Vulkan | 99 | tg128 | 57.64 ± 0.19 |
build: c55d53ace (6854)
llama.cpp has been built simply with:
cmake -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON
P.S. I forgot the CPU-only results, though they are in line with what has already been presented here:
$ ./llama-bench -m Qwen_Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf -t 8
| model | size | params | backend | threads | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | CPU | 8 | pp512 | 27.44 ± 0.17 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | CPU | 8 | tg128 | 16.84 ± 0.05 |
build: c55d53ace (6854)
llama.cpp has been built in the same way, except that -DGGML_VULKAN=ON has been omitted.