Ok, after doing some further reading in the thread about working pd power supplies I came onto a git page from some guy doing power and performance benchmarks who also had underpowering issues and linked to this forum post:
This shows the exact problem I get when attaching it on usb3 with a usb3 cable, and this also hints at underpower. The op solved this by adding external power to the drive which unfortunately is not a workable solution for me. However as my testing suggests, throttling the device cuts power usage(and I think the particular issue here might actually be spikes in power usage) enough for the device to become a reliable USB endpoint. As soon as I find my USB power meter I’ll check for that