On ARM there is only one cpufreq policy per cluster so cpu0 adjusts all four A55 (which isn’t a bad thing). Anyway, it’s not cpufreq scaling but still a single A55 being the bottleneck since %sys and %irq are both handled here.
Can you please make a diff between /proc/interrupts contents before/after another iperf3 run? The lines with significant differences are the interesting ones (to send this specific interrupt later to another CPU).