M.2 e key PCIe topology

#1

The documentation for the orion o6 board indicates that it has two gen4 lanes via the m.2 e key slot. I have a gen 4 NIC (Mellanox Technologies MT2894 Family [ConnectX-6 Lx]) attached there via an e key to PCIe x4 (openended) adapter. It works but I notice that the device is only getting a single lane of gen4. While this is fast enough for a single port of 10g connectivity, if I wanted to light up both interfaces or drive them at 25g it would not and so I was hoping for x2.

I did some digging and it seems that while m.2 e key can expose 2 lanes of PCIe, it’s unclear to me whether it’s an x2 link or as a pair of x1 that can be used by 2 distinct devices bifurcation-style. If so, one lane is all that I should expect and I should live with this. Or perhaps the adapter I bought only supports x1 (the docs for it don’t specify how many lanes it carries now that I read the fine print, though the physical interface is x4).

Is anybody who is more well-versed with the ins-and-outs of m.2 and PCIe able to add anything to what I’m observing? Is it possible to get a single x2 connection from the m.2 e key port? I’m trying to take advantage of all of the IO on this board since in theory it should have plenty, but I don’t want to use up the PCIe slot or the m.2 m key for a faster NIC.

#2

I think two lanes are routed there, because my understanding is that the SoC has 16 total lanes:

  • 8 for pcie connector
  • 4 for M2 M SSD
  • 2 for the 2 onboard NICs
  • 2 left for M2 E

Howwever, the PCIe spec only mandates support for x1 and max width for devices. This means that an x4 or x8 device which doesn’t find all of its lanes is not forced to support x2 or x4 in degraded mode. Since x2 was very rare before M2, it’s very possible that Mellanox never made the effort to support x2 on their board. IMHO given that x1 or x2 bandwidth is already huge for an SSD, you should use adapters to move your storage to the M2E and use another x4 adapter for the M2M to your NIC. That would allow you to reach line rate with limited effort. I’ve used an M2 E->M adapter in my Rock5 ITX for this, and placed the storage on a 2242 SSD attached there in order to use the M2M for the 10G NIC. It looks like what you’re trying to do with the Orion :wink: