I tried compiling llama.cpp with opencl support to use the Adreno GPU, but it doesn’t work. clinfo shows zero platforms. Do I need to download qualcomm drivers separately? Or is this currently not possible? I’m running the default radxa image radxa-airbox-q900-ubuntu-noble-gnome-t4.
Let me try to reproduce this and hopefully give you a solution soon.
clinfo indeed gets zero platform by default:
radxa@airbox:~$ clinfo
Number of platforms 0
ICD loader properties
ICD loader Name OpenCL ICD Loader
ICD loader Vendor OCL Icd free software
ICD loader Version 2.3.2
ICD loader Profile OpenCL 3.0
after running sudo apt install qcom-adreno-cl-dev
and then clinfo shows adreno gpu device
radxa@airbox:~$ clinfo
clinfo: /lib/aarch64-linux-gnu/libOpenCL.so.1: no version information available (required by clinfo)
Number of platforms 1
Platform Name QUALCOMM Snapdragon(TM)
Platform Vendor QUALCOMM
Platform Version OpenCL 3.0 QUALCOMM build: 0808.11
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_icd
Platform Extensions with Version cl_khr_icd 0xc00000 (3.0.0)
Platform Numeric Version 0xc00000 (3.0.0)
Platform Extensions function suffix QCOM
Platform Host timer resolution 0ns
Platform Name QUALCOMM Snapdragon(TM)
Number of devices 1
Device Name QUALCOMM Adreno(TM) 663
Device Vendor QUALCOMM
Device Vendor ID 0x5143
Device Version OpenCL 3.0 Adreno(TM) 663
Device Numeric Version 0xc00000 (3.0.0)
Driver Version OpenCL 3.0 QUALCOMM build: 0808.11 Compiler E031.49.02.00
Device OpenCL C Version OpenCL C 3.0 Adreno(TM) 663
Device OpenCL C all versions OpenCL C 0xc00000 (3.0.0)
OpenCL C 0x800000 (2.0.0)
OpenCL C 0x402000 (1.2.0)
OpenCL C 0x401000 (1.1.0)
OpenCL C 0x400000 (1.0.0)
Device OpenCL C features __opencl_c_3d_image_writes 0xc00000 (3.0.0)
__opencl_c_atomic_order_acq_rel 0xc00000 (3.0.0)
__opencl_c_atomic_order_seq_cst 0xc00000 (3.0.0)
__opencl_c_atomic_scope_device 0xc00000 (3.0.0)
__opencl_c_atomic_scope_all_devices 0xc00000 (3.0.0)
__opencl_c_device_enqueue 0xc00000 (3.0.0)
__opencl_c_generic_address_space 0xc00000 (3.0.0)
__opencl_c_images 0xc00000 (3.0.0)
__opencl_c_int64 0xc00000 (3.0.0)
__opencl_c_pipes 0xc00000 (3.0.0)
__opencl_c_program_scope_global_variables 0xc00000 (3.0.0)
__opencl_c_read_write_images 0xc00000 (3.0.0)
__opencl_c_subgroups 0xc00000 (3.0.0)
__opencl_c_work_group_collective_functions 0xc00000 (3.0.0)
Latest conformance test passed v2024-05-29-01
Device Type GPU
Device Profile FULL_PROFILE
Device Available Yes
Compiler Available Yes
Linker Available Yes
Max compute units 3
Max clock frequency 1MHz
Device Partition (core)
Max number of sub-devices 1
Supported partition types None
Supported affinity domains (n/a)
Max work item dimensions 3
Max work item sizes 1024x1024x1024
Max work group size 1024
Preferred work group size multiple (device) 64
Preferred work group size multiple (kernel) 128
Max sub-groups per work group 16
3 Likes
Looks good after installing qcom-adreno-cl-dev with llama.cpp backend ops test
radxa@airbox:~/llama.cpp/build$ ./bin/test-backend-ops
ggml_opencl: selected platform: 'QUALCOMM Snapdragon(TM)'
ggml_opencl: device: 'QUALCOMM Adreno(TM) 663 (OpenCL 3.0 Adreno(TM) 663)'
ggml_opencl: OpenCL driver: OpenCL 3.0 QUALCOMM build: 0808.11 Compiler E031.49.02.00
ggml_opencl: vector subgroup broadcast support: true
ggml_opencl: device FP16 support: true
ggml_opencl: mem base addr align: 128
ggml_opencl: max mem alloc size: 256 MB
ggml_opencl: device max workgroup size: 1024
ggml_opencl: SVM coarse grain buffer support: false
ggml_opencl: SVM fine grain buffer support: false
ggml_opencl: SVM fine grain system support: false
ggml_opencl: SVM atomics support: false
ggml_opencl: flattening quantized weights representation as struct of arrays (GGML_OPENCL_SOA_Q)
ggml_opencl: using kernels optimized for Adreno (GGML_OPENCL_USE_ADRENO_KERNELS)
ggml_opencl: loading OpenCL kernels.............................................................................
ggml_opencl: A_q_d buffer size reduced from 311164928 to 268435456 due to device limitations.
ggml_opencl: default device: 'QUALCOMM Adreno(TM) 663 (OpenCL 3.0 Adreno(TM) 663)'
Testing 2 devices
Backend 1/2: GPUOpenCL
Device description: QUALCOMM Adreno(TM) 663
Device memory: 0 MB (0 MB free)
ABS(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
ABS(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
SGN(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
SGN(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
NEG(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
NEG(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
STEP(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
STEP(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
TANH(type=f16,ne_a=[128,2,2,2],v=0): OK
TANH(type=f16,ne_a=[5,7,11,13],v=0): OK
ELU(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
ELU(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
RELU(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
RELU(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
SIGMOID(type=f16,ne_a=[128,2,2,2],v=0): OK
SIGMOID(type=f16,ne_a=[5,7,11,13],v=0): OK
GELU(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
GELU(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
GELU_QUICK(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
GELU_QUICK(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
SILU(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
SILU(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
HARDSWISH(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
HARDSWISH(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
HARDSIGMOID(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
HARDSIGMOID(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
EXP(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
EXP(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
EXPM1(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
EXPM1(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
SOFTPLUS(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
SOFTPLUS(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
GELU_ERF(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
GELU_ERF(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
FLOOR(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
FLOOR(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
CEIL(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
CEIL(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
ROUND(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
ROUND(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
TRUNC(type=f16,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
TRUNC(type=f16,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
ABS(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
ABS(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
SGN(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
SGN(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
NEG(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
NEG(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
STEP(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
STEP(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
TANH(type=f16,ne_a=[128,2,2,2],v=1): OK
TANH(type=f16,ne_a=[5,7,11,13],v=1): OK
ELU(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
ELU(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
RELU(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
RELU(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
SIGMOID(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
SIGMOID(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
GELU(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
GELU(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
GELU_QUICK(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
GELU_QUICK(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
SILU(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
SILU(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
HARDSWISH(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
HARDSWISH(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
HARDSIGMOID(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
HARDSIGMOID(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
EXP(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
EXP(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
EXPM1(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
EXPM1(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
SOFTPLUS(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
SOFTPLUS(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
GELU_ERF(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
GELU_ERF(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
FLOOR(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
FLOOR(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
CEIL(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
CEIL(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
ROUND(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
ROUND(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
TRUNC(type=f16,ne_a=[128,2,2,2],v=1): not supported [OpenCL]
TRUNC(type=f16,ne_a=[5,7,11,13],v=1): not supported [OpenCL]
ABS(type=f32,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
ABS(type=f32,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
SGN(type=f32,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
SGN(type=f32,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
NEG(type=f32,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
NEG(type=f32,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
STEP(type=f32,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
STEP(type=f32,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
TANH(type=f32,ne_a=[128,2,2,2],v=0): OK
TANH(type=f32,ne_a=[5,7,11,13],v=0): OK
ELU(type=f32,ne_a=[128,2,2,2],v=0): not supported [OpenCL]
ELU(type=f32,ne_a=[5,7,11,13],v=0): not supported [OpenCL]
RELU(type=f32,ne_a=[128,2,2,2],v=0): OK
RELU(type=f32,ne_a=[5,7,11,13],v=0): OK
SIGMOID(type=f32,ne_a=[128,2,2,2],v=0): OK
SIGMOID(type=f32,ne_a=[5,7,11,13],v=0): OK
GELU(type=f32,ne_a=[128,2,2,2],v=0): OK
GELU(type=f32,ne_a=[5,7,11,13],v=0): OK
GELU_QUICK(type=f32,ne_a=[128,2,2,2],v=0): OK
GELU_QUICK(type=f32,ne_a=[5,7,11,13],v=0): OK
SILU(type=f32,ne_a=[128,2,2,2],v=0): OK
SILU(type=f32,ne_a=[5,7,11,13],v=0): OK
...
1 Like
This is working now, thanks! Turns out to have been a permission issue: the problem went away when I used sudo.
1 Like