Kernel doesn't recognize M.2 slot with PCIe backplane

RockPi4b, this happens with both the stock Debian distribution and the stock Ubuntu distribution.

I have an M.2 to PCIe 4X single-slot adapter. In that slot, I want to run a multi-port USB card, but I can’t get it to enumerate any PCIe 4X card I put in there.

I can use an nVME card in the M.2 slot, so I know it’s good. I also have an external nVME->USB 3.0 enclosure that I can use with this adapter to see the USB card on my laptop, so I know the adapter is good.

I think the PCI device scan is disabled in the kernel. I tried rebuilding the kernel with turning on virtually every option that says “PCI” but it didn’t help.

What do I do to make it see this card?