Hello, Since AI/LLMs/NPUs are being pushed into every part of our lives these days i feel like it’s appropriate to ask about this.
As far as i know there’s little to no (or none at all) support for NPUs in upstream linux or llama.cpp.
Rockchip boards (mainly rk3588) recently had a rocket driver upstreamed which allows for basic tflite usage but not full LLM acceleration.
I’m wondering if we’ll ever get the NPU driver on the orion upstreamed or at least usable with llama.cpp on the vendor kernel.
Highly doubt it given the track record of rockchip or other companies..
Bit annoying when boards are advertised as “being able to do x” but only if you use non standard/outdated software/solutions.
As far as I’m concerned if something doesn’t work with mainline linux or whatever most people use then it does not work at all.
A hack is a hack not a solution.