Worked on it a bit over the weekend and got around the issue.
Hopefully the binaries will get updated soon and we’ll be able to flash them as shown in the instructions.
Basically, I flashed an SD Card with DietPi and loaded u-boot.bin using RZ_USB_Boot_Helper_V1.0.0.exe. This allowed me to boot from the SD card temporarily.
After that from within DietPi I flashed u-boot.bin to the eMMC (/dev/mmcblk2 in my case) with the command:
sudo dd if=u-boot.bin of=/dev/mmcblk2 bs=512 seek=1
Lastly, I flashed the eMMC with DietPi (since it was the OS I needed) with the command:
sudo dd if=DietPi_RadxaZero-ARMv8-Bullseye.img of=/dev/mmcblk2 bs=1M
Hopefully this will help others.