Does Vulkan work?

[jimhamiru]

I did manage to get KleidAI to build, but it actually caused worse performance for me:

1. I did manage a successful build with KleidiAI support - but prompt ingestion performance did not appear to improve and token generation was actually significantly slower (tg on Qwen3 A3B:30B:Q4_K_M down to around 12 t/s).

Llama.cpp - Benchmarks

Hi all, I bought the Orion O6 primarily with the intent of running LLM’s on it (I needed something with relatively low power consumption - I run exclusively on solar). I figure some benchmarks for Llama.cpp might interest others. Do not expect great speeds - it is constrained by the RAM bandwidth for token generation - and prompt ingestion is currently CPU-only (though would love to see how it would perform using the NPU). A preliminary note, I built Llama.cpp using the following: # NOTE: Th…