I’d like to understand the feasibility of porting a GGML model to the HiFi 5 DSP in the SoC, particularly with respect to the memory accessible by the DSP. Based on the core configuration shown here ( Radxa Orion O6 | Tensilica Tools ), the System RAM region starting at 0x20400000 appears to be only 1 MB in size. Is this System RAM mapped to the LPDDR5 memory, or is it a separate on-chip memory region? Additionally, where can I find detailed documentation on the SoC memory map, including the HiFi 5 memory architecture and its access to system memory? Thanks.
The HiFi5 DSP actually has 384k of local RAM (in three banks of 128KB) plus 1MB of on-chip external SRAM (from 0x20400000) as well as a mapped 2MB window for host communications (not all of which is allocated by default) and 16MB LPDDR-backed DSP working memory plus a second 16MB LPDDR-backed XAF client data pool. However the first 16MB is scratch-memory and state, so only the second 16MB would be available for weights, etc.
Your challenges would therefore be that you’ll struggle to fit a full model into 16MB, and offloading individual small operations would probably not be worthwhile due to the communications overhead.
You’d also not be able to productively use generic GGML C code, but instead would have to re-write the model as Xtensa/HiFi SIMD matrix and quantisation kernels, probably as custom XAF components.
The CIX P1 TRM omits detailed HiFi5-local and LPDDR-window architecture detail, so it’s difficult to know how amenable the DSP would be to enlarging the client-pool beyond 16MB in size – which would also require custom DSP- as well as system- firmware.