Linux ‘average load’ is different to any other kernel (OS) known: Linux Load Averages: Solving the Mystery
Check which processes are in uninterruptible state, this is a common issue with ‘boards’ in ‘bring-up’ stage, at least I’ve seen this with plenty of BSP kernels here and there (though usually loadavg not dropping below 2 or 1 and not 4 as you report – haven’t bootet ‘my’ O6 for months since interested in ‘energy efficient computing’ and anything Cix P1 currently is a joke)