@Semi
Thank you very much for sharing your design with us, it looks perfect and I am happy to try out a solution similar to yours, using the Rock 5 as a NAS with Sata and a NVME.
Radxa M.2 E key to SATA breakout card
I tested two 5-Port jmb585 controllers in m.2 format. I consistently get problems after a couple of hours, especially if I move around a lot of data or read a lot of data (test case is usually first moving a couple hundred gb from one drive to another and then scan a plex library). Timeouts, drives spinning down and spinning up, finally leading to readonly failing filesystems. I ruled out: cabling, drives, cooling of the jmb chip (still not sure, that’s a ridiculous small heatsink), and power supply.
I’m using armbian, booting from sd, rootfs on drive 1 (SSD) and the 4 others are spinning rust.
Anyone else having problems? I’ll attach a log once i’m back at my main workstation.
This adapter looks interesting for me, have you tested two nvme ssd on this?
@Semi Have you thought of using an ASM2812 (or PEX8712 / PEX8714) PCIe Gen3 Packet Switch to get either 2x4 or 4x2?
I also wonder whether having 1x4 and 4x1 with such a switch is possible so that we can have a full x4 slot for either ssd or gpu and four x1 slot for wifi card, sata card, or even multiple 2.5GbE (or 5GbE) LAN card.
@dominik do you think it is possible to do a 3 slot splitter that can work either in 2x 2x or 2x 1x 1x?
For example first two slot will be 2x slot, if both slot has 2 lanes device plugged in, the third slot will not work, but if the second slot got a 1-lane device plugged in, we can plug a 1-lane device on the third slot and config the rock 5b in 2x 1x 1x
Have You tried that with @semi extension or just on m.2 port?
JMB585 is same as the one from quad sata hat by radxa and also wired with pcie on rock4 or rock3a boards. Take a look at this thread for some other issue, both probably caused by same thing?
should be possible as far as I know, but powering up that may be another issue here…
Radxa used available clocks for different peripheral.
Rk3588 supports 4, 2-2, 2-1-1, 1-1-1-1, but Rock5b only 4, 2-2.
(i may be wrong)
Is it okay to have different length between of RX_N and RX_P traces? (the differential pair within one lane)
What is the reason that the Rock5B can’t do 4x1 despite the general capabilities of the RK3588? Can someone please confirm this?
The datasheet states even 1 more mode: 1x4, 2x2, 4x1, 1x2+2x1.
Thanks for your great work semi! Really appreciate it and i actually had get this board made too. However, i was stucked in utilizing this board as i cannot get the kernel actually recognize both of the pcie slot. Did you actually get it working? Could you share more information on how to get the kernel to use both of the board? Really appreciate your kind help!
I planed to use rock 5b with 10gbps nic and a ssd as NAS.
The problem shall be the perstn signal. The kernel require a perstn signal for each of the pcie slot, but in the m2 slot we got just have 2 possible signals for that. This is so far i know… Not sure i am right or not…
FYI, today i noticed that 6.7-RC8 contains native support for bifurcation on rock 5b:
best
Thanks for the information! This sound promising!
But when looking into the code, i cannot find the bifurcation related parts… Would you please give me some clues on it? Thanks!
Hi There,
I’m no dts guru either but I can see that there are two configurations for rk3588: In the line I linked:
“pcie3x4” so this should be for PCIe 3 1x4 and below you can see “pcie3x2” this should be the part for PCIe 3 2x2. What all these interrupts do etc. I have not really a clue
Update: Output ChatGPT
pcie3x4: pcie@fe150000 {
This line defines a PCIe controller node with the name "pcie3x4" at the physical address "fe150000".
compatible = "rockchip,rk3588-pcie", "rockchip,rk3568-pcie";
Indicates compatible PCIe controller models: "rk3588-pcie" and "rk3568-pcie".
#address-cells = <3>;
Specifies that the controller uses three cells to represent addresses.
#size-cells = <2>;
Specifies that the controller uses two cells to represent sizes.
bus-range = <0x00 0x0f>;
Defines the PCI bus range supported by this controller from 0x00 to 0x0f.
clocks = <&cru ACLK_PCIE_4L_MSTR>, <&cru ACLK_PCIE_4L_SLV>,...;
Lists the clock sources required by the PCIe controller.
clock-names = "aclk_mst", "aclk_slv",...;
Associates names with the clock sources listed in the previous line.
device_type = "pci";
Specifies that the device type is PCI.
interrupts = <GIC_SPI 263 IRQ_TYPE_LEVEL_HIGH 0>,...;
Lists interrupt sources and their properties.
interrupt-names = "sys", "pmc", "msg", "legacy", "err";
Associates names with the interrupts listed in the previous line.
#interrupt-cells = <1>;
Indicates that each interrupt specifier has one cell.
interrupt-map-mask = <0 0 0 7>;
Specifies the mask for the interrupt mapping.
interrupt-map = <0 0 0 1 &pcie3x4_intc 0>,...;
Maps PCIe interrupts to corresponding interrupt controllers.
linux,pci-domain = <0>;
Specifies the Linux PCI domain.
max-link-speed = <3>;
Indicates the maximum link speed (Gen3).
msi-map = <0x0000 &its1 0x0000 0x1000>;
Maps Message Signaled Interrupts (MSI) to the Interrupt Translation Service (ITS).
num-lanes = <4>;
Specifies the number of PCIe lanes.
phys = <&pcie30phy>;
Associates the PCIe controller with a PHY (physical layer) named "pcie30phy".
phy-names = "pcie-phy";
Associates a name with the PHY listed in the previous line.
power-domains = <&power RK3588_PD_PCIE>;
Specifies the power domain for the PCIe controller.
ranges = <0x01000000 0x0 0xf0100000 0x0 0xf0100000 0x0 0x00100000>,...;
Describes the address ranges that the PCIe controller can access.
reg = <0xa 0x40000000 0x0 0x00400000>,...;
Specifies register ranges for the PCIe controller.
reg-names = "dbi", "apb", "config";
Associates names with the register ranges listed in the previous line.
resets = <&cru SRST_PCIE0_POWER_UP>, <&cru SRST_P_PCIE0>;
Specifies the resets required for the PCIe controller.
reset-names = "pwr", "pipe";
Associates names with the resets listed in the previous line.
status = "disabled";
Indicates that the PCIe controller is currently disabled.
pcie3x4_intc: legacy-interrupt-controller {...};
Defines a legacy interrupt controller named "pcie3x4_intc" for handling interrupts associated with this PCIe controller.
};
Closes the definition for the "pcie3x4" PCIe controller node.
pcie3x2: pcie@fe160000 {...};
Similar to the "pcie3x4" node, this defines another PCIe controller named "pcie3x2" at the physical address "fe160000".
...
The definition for "pcie3x2" follows a similar structure as "pcie3x4", with differences in parameters such as bus range, number of lanes, and interrupts.
pcie3x2_intc: legacy-interrupt-controller {...};
Defines a legacy interrupt controller named "pcie3x2_intc" for handling interrupts associated with the "pcie3x2" PCIe controller.
};
Closes the definition for the "pcie3x2" PCIe controller node.
So for the dts i used Rockchip_Developer_Guide_PCIe_CN. Section 2.3.3 describes how to configure pcie to 2x2 lanes (don’t forget to set the correct mode, see 2.4.3).
As for the perstn signal, pcie has three control signals: perst, wake and clkreq. Since the clock can’t be disabled on the rock 5b we don’t need the clkreq signal. As i did not need the wake function we can also get rid of this signal too (I think the wake is not even configured in the original dts). So we have two unused pins for control signal, one of which (I think it’s GPIO4_B4) is used as a reset signal for the second m.2 slot.
Thanks for the information! I’ve actually revealed the same findings after inspecting the board and schematic carefully. Anyways, really appreciate for your great work! I can now access to both of the nvme now. Next I would try to get access to 10Gbps NIC
Hi There, I have no experience in ordering custom boards. Is there any buyable option? Such a board would be nice.
you may need to ask Radxa for that i think Stay tuned, I heard that they are coming to launch some new boards on rk3588 soon.