ROCK 5B Debug Party Invitation

tkaiser · July 26, 2022, 11:42am

@jack for your wiki article about successfully tested chargers I added those I personally confirmed to work with Radxa’s Ubuntu 20.04 image to my review.

Important note: the two Apple USB-C chargers (96W and 140W) do not work with Armbian. Here the board stalls in a reboot loop. Initially (see many posts above) I thought it’s related to adjusting the PDO_FIXED device-tree nodes but it’s related to Armbian. At least the image I tested with (Armbian_20220707f-rpardini_Rock-5b_jammy_legacy_5.10.66.img.xz) shows this behaviour with 2 out of 5 chargers.

Now that I learned that I’m participating in the most stupid ‘debug party’ ever (not just some babbling in closed/hidden Discord channels but also in ‘radxa’s official QQ group’ whatever that is) it’s really time to stop.

Edit: debugfs info queried that way: cat /sys/kernel/debug/usb/fusb302-4-0022 /sys/kernel/debug/usb/tcpm-4-0022 | sort | curl -s -F 'f:1=<-' ix.io

Stat_headcrabed · July 26, 2022, 12:05pm

@tkaiser That QQ group is just for Chinese users since most people in China don’t use discord. And actually amazingfate is the only one who gets board and lives in China.

Stat_headcrabed · July 26, 2022, 12:20pm

Maybe you can get a PD protocol analyzer (such as power-z’s) to investigate what actually happens during reboot.

tkaiser · July 26, 2022, 12:31pm

Me? Seriously?

Maybe @amazingfate can try out another image with those problematic USB-C chargers and report back here? If it works with Radxa’s image then someone willing to help could start to bisect?

BTW: I put the Ubuntu image I tested with online since it disappeared from releases page.

Stat_headcrabed · July 26, 2022, 12:44pm

He often reports in radxa’s public discord group, have you checked yet?

tkaiser · July 26, 2022, 12:52pm

Nope, I can’t check since I’m not using this Discord crap for obvious reasons. If participating in such a ‘debug party’ requires me handing my mobile number over to some shitty US tech company that is specialised in user tracking then I guess I shouldn’t be part of this ‘party’.

Asides that I have no idea why Radxa printed this when they actively force fragmentation of user discussions:

Stat_headcrabed · July 26, 2022, 1:10pm

Well seems that most board owners are discussing there. If you don’t want to expose your phone number to discord, you can try google voice, which creates a virtual phone number.

stuartiannaylor · July 26, 2022, 2:16pm

Discord is there and has a more informal chat basis and generally from what I have seen often a more formal summary is posted here, as many use both.
If you don’t want to sign up to discord then don’t, but its not as if you are missing all that much.
Generally there are enough on both and likely would post anything missing that needs to be.

Tonymac32 · July 26, 2022, 3:43pm

See if Radxa will make FPC adapter cables. The RPi “Standard” is based on the limitations of an arm11 SoC from forever ago. I don’t disagree that compatible ports are nice, but trying to merge 2 connectors to one device is way harder than just selecting the desired signals with a flex cable.

Stat_headcrabed · July 26, 2022, 5:13pm

Actually I mean rpi cm4io’s CSI/DSI ports, which has full 4-lane bus. Beaglebone AI-64 is also using this port. I think convert cable is also a good choice, but if it’s done by a dedicated pcb, there would be a mess to use those ports. @Tonymac32

nyanmisaka · July 26, 2022, 6:14pm

The latest board layout from Allnet China.

Tonymac32 · July 26, 2022, 6:32pm

So a less common connector then than even the normal RPi. 22-pin, 2 or 4 lane (1 of each on cm4io). An adapter ribbon cable should be simple I would think.

stuartiannaylor · July 26, 2022, 7:46pm

Just a tangential question on PCB art but the crazy track paths you often see on memory is that to make the track length all equal as is it that sensitive in timing?

Tonymac32 · July 26, 2022, 8:22pm

Yep. Ideally you match impedance, too.

willy · July 26, 2022, 9:05pm

Great job! Looks like the heatsink will hold firmly in place now, that was worth it!

willy · July 26, 2022, 9:43pm

I could finally run some network tests with a 4x10G NIC on my M.2->PCIe adapter. The NIC is the common dual-82599 behind a PCIe bridge. The PCIe bridge is PCIe 3.0 x8 converting to two PCIe 2.0 x8 for the network controllers. That’s perfect because it can convert the 8GT x4 to 5GT x8 with limited losses. We have 32Gbps theoretical PCIe bandwidth (27G effective with the standard 128-byte payload).

[edit: I forgot the mandatory setup photo]

Well, to make a story short, that board is a serious bit mover! Please see the graphs below. I’ve exchanged HTTP traffic between the board and 3 other devices (I needed that to make it give up). One was my mcbin (10G) and the two other ones were HoneyComb LX2 (one 10G port each). There was not much point going further, due to the bus width.

I started a server (httpterm) on each board. First tests consisted in using the Rock5B as a client only (download test). It reached 26.1 Gbps stable over the 3 NICs (measured at the ethernet layer, as reported by the interface), and there was still 24% idle CPU, indicating I was saturating the communication channel. It’s very close to the limit anyway:

30g-single

Then I tested the traffic in bidirectional mode. Rock5B had 3 clients (one per interface) and each other machine also had one client attacking Rock5B. Result: 23.0 Gbps in each direction and 0% idle! So the CPU can almost saturate the PCIe in both directions at once while processing network traffic (probably that with larger objects it could, they were only 1MB, that’s almost 3000 requests/s when you think about it). There were 2.1 million packets/s in each direction. There were It’s visible on the graph that the Rx traffic dropped a little bit when the Tx traffic arrived:

30g-bidir

Finally I wanted to see what happens when the network definitely is the limit. I started the 20G bidirectional test (only two ports). It saturated both links in both directions, leaving 12% idle:

20g-bidir

A few other points, I used all 8 CPUs during these tests. They seem to complement well, as the 4 little cores are able to download 14.5 Gbps together, which is pretty good!

A final test (not graphed) consisted in attacking the board with multiple HTTP clients sending small requests. It saturated at 540000 requests/s, which is also excellent for such a device.

Finally there was nothing in dmesg along all the tests, so that’s pretty reassuring, and indicates that the instabilities I faced with the older Myricom NIC was related to the card itself.

Thus I’m pretty sure we’ll start to see network gear and NAS devices made around this chip for its PCIe capabilities and its memory bandwidth, which play in the yard of entry-level Xeons. That’s pretty cool.

NekoMay · July 27, 2022, 7:39am

There is no such thing as a “standard” CSI or DSI port; they’re deliberately left undefined by MIPI. It’s a major problem with the standard, IMO.

jack · July 27, 2022, 8:57am

Neither QQ group or Discord is a good place for technical discussion since they are not public search engine indexing.

I think why there is official QQ group or discord is social purpose.

kangdei · July 27, 2022, 3:09pm

Blockquote我十分赞同,QQ群,微信群,TG群都不是什么好的地方,讨论出来的问题和解决方法哪怕是群成员都不能及时看到,并且消息过多,更多的可能是直接清理掉了,在论坛的话,不仅仅是可以被搜索到,有同样问题并解决了的人可以风险出自己解决问题的办法,而不是不厌其烦的问群里面的技术比较好的人,技术比较好的人面对同样的问题也是会厌烦的,同时社区越活跃对RADXA有积极的正面影响,可以让RADXA成为下一个树莓派的可能,社区越活跃资源也将越丰富,同时我也提一个建议,就是能不能让社区的服务器运行的更流畅一些呀,国内的用户访问卡卡卡卡的,

willy · July 28, 2022, 6:02pm

I just received my PCIe USB3.1 adapter this evening. The model is “StarTech.com PEXUSB312A3” sold as “StarTech USB3.1 adapter with UASP support”, which shows up like this:

rock@rock-5b:~$ lspci -nn
0000:00:00.0 PCI bridge [0604]: Fuzhou Rockchip Electronics Co., Ltd Device [1d87:3588] (rev 01)
0000:01:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM2142 USB 3.1 Host Controller [1b21:2142]
0004:40:00.0 PCI bridge [0604]: Fuzhou Rockchip Electronics Co., Ltd Device [1d87:3588] (rev 01)
0004:41:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 04)

Thus I could run the test again with the Crucial X8 SSD. TL;DR: it works perfectly now :

# dd if=/dev/sda of=/dev/null bs=1M status=progress
12495880192 bytes (12 GB, 12 GiB) copied, 15 s, 832 MB/s^C

Look at the “bi” column during the test, I’ve seen it reach 1 GB/s once:

rock@rock-5b:~$ vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 3505692  23376 132156    0    0 23475    18  463  450  1  4 94  1  0
 1  0      0 3505692  23376 132156    0    0     0     0 1226 1561  0  1 99  0  0
 2  0      0 3248208 281420 131996    0    0 192584    36 2904 2637  0  3 96  1  0
 2  0      0 2328076 1198868 134476    0    0 982980     0 9351 6751  0  8 87  4  0
 0  1      0 1408228 2116316 136324    0    0 917448     0 9566 6934  0  9 86  5  0
 1  1      0 488048 3033764 138620    0    0 917448     0 8981 6404  0  9 86  4  0
 2  0      0 246848 3276656 139924    0    0 775352     0 9026 6328  0 14 83  4  0
 2  0      0 250976 3271576 141448    0    0 765112     0 8572 6469  0 14 82  4  0
 2  0      0 243664 3277176 142964    0    0 742984    12 9057 6661  0 15 81  4  0
 0  1      0 260476 3257040 144916    0    0 832076     0 9725 7007  0 15 81  4  0
 2  0      0 245108 3272296 146544    0    0 774548     0 9136 7019  0 14 82  3  0
 1  0      0 251768 3264428 148080    0    0 816308     0 9735 7497  0 15 81  4  0
 0  1      0 276760 3236952 149696    0    0 733800     0 8483 6261  0 14 82  4  0
 1  0      0 263156 3249680 151388    0    0 896260     0 10610 8162  0 13 83  4  0
 2  0      0 247468 3263040 153240    0    0 769476     0 9244 6621  0 15 81  3  0
 0  1      0 261800 3247116 154872    0    0 811076     0 9356 6831  0 15 81  3  0
 3  0      0 242204 3263760 156572    0    0 787256     0 9312 6775  0 14 82  4  0
^C

So that tells us that:

the USB stack and xhci driver in the Rockchip BSP kernel are OK
the SSD indeed supports UAS well, just like the kernel
the issues when using the local USB ports (either USB2 or USB3) are definitely related to the rockchip USB controller or to the connection between it and the USB3 connectors. I tend to think that it’s very likely an issue within the driver itself given that it doesn’t particularly work better in USB2 (at least we should hope for this).

For those interested, here’s what the kernel says about the USB card:

[    2.729508] xhci_hcd 0000:01:00.0: xHCI Host Controller
[    2.729533] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 10
[    2.729555] xhci_hcd 0000:01:00.0: Host supports USB 3.1 Enhanced SuperSpeed
[    2.729652] usb usb10: We don't know the algorithms for LPM for this host, disabling LPM.
[    2.729785] usb usb10: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.10
[    2.729800] usb usb10: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    2.729813] usb usb10: Product: xHCI Host Controller
[    2.729825] usb usb10: Manufacturer: Linux 5.10.66-267861-g55f540ce97a3 xhci-hcd
[    2.729837] usb usb10: SerialNumber: 0000:01:00.0
[    2.730589] hub 10-0:1.0: USB hub found

what appears in kernel messages when I connect the SSD:

[   32.805961] usb 10-1: new SuperSpeedPlus Gen 2 USB device number 2 using xhci_hcd
[   32.833698] usb 10-1: New USB device found, idVendor=0634, idProduct=5600, bcdDevice= 1.00
[   32.833718] usb 10-1: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[   32.833726] usb 10-1: Product: Crucial X8 SSD
[   32.833732] usb 10-1: Manufacturer: Micron Technology Inc
[   32.833738] usb 10-1: SerialNumber: 2124E3297853
[   32.865379] scsi host0: uas
[   33.807846] scsi 0:0:0:0: Direct-Access     Micron   Crucial X8 SSD   0    PQ: 0 ANSI: 6
[   33.809603] sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB)
[   33.809723] sd 0:0:0:0: [sda] Write Protect is off
[   33.809732] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
[   33.809905] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   33.810593] sd 0:0:0:0: [sda] Optimal transfer size 33553920 bytes
[   33.837781]  sda: sda1
[   33.840421] sd 0:0:0:0: [sda] Attached SCSI disk
[   33.852544] vcc3v3_pcie2x1l0: disabling