The custom kernel looks for firmware in a different place than the radxa supplied kernel. I had to make some symbolic links to have wlan working. Check the syslog output for clues. I remember that I had to create /system/etc directory and create a link from /lib/firmware to /system/etc/firmware. I also think that I had to copy some files from a subdirectory of /lib/firmware to the /lib/firmware itself, but I forgot which. Looking which file are mentioned in the log, it is probably one of those:
fw_bcm43456c5_ag.bin
nvram_ap6256.txt
clm_bcmdhd.blob