Rock 5C lite rkllm

ROCK 5 Series 5C
1

I want to run qwen2.5 on the Rock 5C lite as explained in the documentation. When running it, i only get a cpu mismatch error, since i have to compile for the rk3588 and not the rk3582.

./llm_demo /home/radxa/Qwen2.5-1.5B-Instruct_W8A8_RK3588.rkllm 2048 4096
rkllm init start
I rkllm: rkllm-runtime version: 1.2.3, rknpu driver version: 0.9.8, platform: RK3588
I rkllm: loading rkllm model from /home/radxa/Qwen2.5-1.5B-Instruct_W8A8_RK3588.rkllm
I rkllm: rkllm-toolkit version: 1.2.3, max_context_limit: 4096, npu_core_num: 2, target_platform: RK3588, model_dtype: W8A8
I rkllm: Enabled cpus: [4, 5, 7]
I rkllm: Enabled cpus num: 4
E rkllm: Mismatch between enabled CPUs mask and expected count. Please check the configuration.
rkllm init failed

Im using the newest radxa os and reduced the npu_core_num to 2 before cross compiling.
Is there any know issue or solution?

1 Like
2

I got same issues too.

3

RK3582 has 2 fewer CPU cores, you need to edit the code to incorporate this.

4

Hi, @Melonmain

Please try compile Qwen2.5 to rkllm model with 3 NPU CORES and change int8_t enabled_cpus_num to 3

best,
Morgan