Llama ia meta utile?

#1

quel qu’un à deja tester “llama” sur rock 3 ? le npu peut servir …?

#2

I’ll reply based on translated output from Google. I tried Llama2 using llama.cpp but it only runs on the CPU, not even the GPU. I’ve finally managed to set up OpenCL 2.1 via libmali but have yet to test whether llama.cpp successfully uses the GPU(via OpenCL) to accelerate Llama.

Unfortunately, according to the developer of llama.cpp, the NPU of rock 3/RK3568 doesn’t support matrix multiplication, so it can’t be used to accelerate LLM like Llama2. Only the GPU can be used.

#3

Merci, donc Llama 3 … :joy::skull: