Oh must’ve confused them then bc for a long time (maybe still?) the BSP device tree for NanoPi-R5S falsely had set 2.0 speed on that port
News about the ROCK 5B Plus! ;)
IIRC both Gen3 lanes on R5S are attached to RTL8125BG Ethernet controllers which are only Gen2 capable. So setting the speed to Gen2 in the first place saves 0.5 seconds booting time for each controller since with max-link-speed = <3>; each and every boot the PCIe controller would’ve tried Gen3 first just to do a fallback to Gen2 speeds shortly after.
Anyway: confusion is resolved and RK3588’s PCIe capabilities should be clear now 
Reading into the PCIe spec (ignoring overhead) RTL8126 should work just fine attached to the 2.1 Port. Only being limited to 5Gbit in one direction by bandwidth instead of both at full usage which shouldn’t really matter too much in most usecases.
It’s 5 Gbps (Ethernet data rate) vs. 4 Gbps PCIe data rate (5 GT/s at 8b10b coding). So you can’t saturate 5GbE even in a single direction. Taking other overhead into account you’re maxing out at ~75% of the Ethernet connection’s theoretical peak bandwidth.
3 Likes
Taking other overhead into account you’re maxing out at ~75% of the Ethernet connection’s theoretical peak bandwidth.
Yep that’s usually it, but it also depends on the NIC’s descriptors size etc, which vary from NIC to NIC. For example on an intel XL710 with 2x25G connected in Gen3 x8, one port reaches 24.6G, two ports reach 23.66G each, or 47.32G total, for a theoretical bus speed limit of 63G. That’s 75.1% here as well but for en3. You can imagine the same type of NIC on Gen2, it would probably be around 61%.
But actually on SBCs with few PCIe lanes, it’s better to saturate PCIe lanes and have slightly less bandwidth than the opposite. We’re always complaining that there are not enough lanes, it’s not for starting to waste their bandwidth. It’s always a difficult match between Ethernet and PCIe, as you often have to choose which one to waste. And the 2x25G is the first great match I’ve seen to date where you can saturate PCIe by losing only a few percent of Ethernet’s bandwidth.
I’ve seen one or two only because of simple thing - 2.5G can be used with older ethernet cables.
5G as far as I remember needs upgrade, so there is much more work to do and probably You will consider some kind of fiber which uses less energy (and heat!).
I think that 5G is just death end. Sure with 10G switch it will probably work on 5G mode, but it also means that You are already on 10G side
1 Like
Nope, ‘real’ 10GbE switches (IEEE 802.3an-2006) don’t do Nbase-T (IEEE 802.3bz) and as such work only in 10GbE or 1GbE mode but nothing in between. And while there are currently +100 switches with at least one 5GbE port available I second your ‘death end’ statement wrt 5GbE.
2.5GbE works with old and boring Cat 5e cables, power requirements aren’t that dramatic and it allows for PoE (unlike ‘standard’ 10Gbase-T). With 5GbE usually the host interface is the bottleneck (be it PCIe or USB) as such real-world performance is only slightly better than 2.5GbE or you are going to ‘waste’ precious PCIe lanes, they’re more expensive and you need at least Cat 6 cables.
If you need more than 2.5GbE then 10GbE/SFP+ seems to be the more reasonable choice
And BTW: SMB Multichannel exists (unfortunately still most consumers think bonding/trunking would do the same).
2 Likes
I completely agree, 5G requires special cabling as well as selected choice of network equipment. For sure asus released some compatible routers, STH reviewed cheap network switches with 5G fallback mode, but in fact this requires extra attention to complete right stuff and for sure it’s just problematic. Most of use will skip that in favor of 10G which is probably much more popular at same price.
Consumers don’t care that much about ethernet today, because wifi is getting better and better. Those are also advertised as capable of 4Gbit in some theoretical conditions.
1 Like
I 've bought 3 switches at home, one of them support 1Gbps/10Gbps SFP+, 2 of them support 1Gbps/2Gbps/10Gbps SFP+, but none of them support 5Gbps. 
I think the cost difference for building 10Gbps network is not much higher than 5Gbps nowadays.
I would not discourage people from trying 5Gbps network, but I would go for 10GbE myself
Anyways, I concur with you and tkaiser that 5GbE would be ‘death end’
Yes, the SMB Multichannel would be extremely useful especially when you are using 10GbE for NAS.
I would also like to test the SMB direct RoCE / InfiniteBand. Had you tested that on the Rock5B ITX? Could that reduce the CPU usage?
Just like @tkaiser said - none pro network equipment consider 5G network speed,
but for sure there are some home consumers devices like those from asus and this requires such hardware.
2.5G jump is relatively cheap and easy. I completely agree that 5G is bad idea compared to 10G at same price and much more devices available. I don’t thing this will change anywhere in future.
1 Like
Looks like the RPI-5 handles the 5GbE nic at ~3.3Gbit @ PCIe 2.0. Doesn’t seem worth it from 2.5GbE except if some PCIe 3 lanes are left over.
https://fixupx.com/will_whang/status/1797053374199959813
2 Likes
Thanks for that comparison, I still think that this particular speed will be skipped by most of us, it’s just way too expensive compared to 10G and just problematic in hardware (transceivers, switches, routers). 2.5G was cheap bonus for gigabit and easy upgrade
Any news on 5B+?
It’s been a while.
given that there are still strange/bad performance measurements on the DDR5 with the latest DDR SPL code on Rock5 ITX (see DRAM speed on ROCK 5 ITX), I think it’s prudent to let engineers try to sort everything out before deciding what to do/act on. If in the end they have to choose a different DRAM chip or slightly adjust routing to use the optimal timings, you’ll probably be very glad to have waited a little bit more for an optimal product.
1 Like
from the source I know, there was plan to release Rock 5B Plus in Jul, so stay tune! 
Completely agree,
We all hope that DDR5 can increase RAM speed compared to DDR4. For now all other boards released with this pair - RK3588 and DDR5 have none speed improvement, if final result is really based on RAM chip then simple software update will not help those.
With most recent v1.16 DRAM initialization BLOBs it’s even worse compared to LPDDR4X, just check Willy’s aforementioned tests with different BLOBs.
1 Like
Confirmed, I even ran a quick synthetic test that was twice as slow to execute on the ITX than the rock5b. I need to see if I’m still having the code, in order to share it.
Edit: here it is:
rock@rock-5b:~$ time taskset -c 4 ./a.out 10
malloc...
fill...
scan...
real 6m2.604s
user 6m1.861s
sys 0m0.648s
willy@rock5-itx:~$ time taskset -c 4 ./a.out 10
malloc...
fill...
scan...
real 11m15.793s
user 11m14.447s
sys 0m1.101s
Both run at the same CPU frequency, and rock5-itx is twice slower due to v1.16’s DDR5 timings.
I’m going to put the tool on my github for easier testing, will update the URL shortly.
1 Like
Sure. Sure. I don’t want anyone to rush the board to market, I was just curious because everything has gone pretty much silent for a longer while and I’ve been waiting for any news at all.