Can you try overriding and increasing the pull-up level (drive-strength) of all the data lines here: kernel/rk3588s-pinctrl.dtsi at stable-5.10-rock5 · radxa/kernel · GitHub ?
PCB traces may be a bit too long (or other issue) for the default settings, hence why it works fine in lower HS mode or inherently slower cards (like the 16 GB one).