SPI Image with current U-boot?

[JingleFl]

Hi everyone,

I’m seeking for help as I’m trying to boot my Rock 3B via SPI from SSD. Unfortunately my SBC gets recognized as Rock 3A, hence I assume that’s the reason why the 2nd Ethernet-Port is not available

U-Boot SPL board init
U-Boot SPL 2017.09-gef9f6b1caad-211223 #stephen (Sep 24 2022 - 17:13:02)
Trying to boot from MMC2
MMC error: The cmd index is 1, ret is -110
Card did not respond to voltage select!
mmc_init: -95, time 10
spl: mmc init failed with error: -95
Trying to boot from MMC1
MMC error: The cmd index is 0, ret is -110
mmc_init: -110, time 5
spl: mmc init failed with error: -110
Trying to boot from MTD2
No misc partition
Jumping to U-Boot(0x00a00000) via ARM Trusted Firmware(0x00040000)
Total: 539.182 ms
INFO: Preloader serial: 2
NOTICE: BL31: v2.3():v2.3-181-gc9a647cae:cl
NOTICE: BL31: Built : 10:55:41, Oct 18 2021
INFO: GICv3 without legacy support detected.
INFO: ARM GICv3 driver initialized in EL3
INFO: pmu v1 is valid
INFO: dfs DDR fsp_param[0].freq_mhz= 1056MHz
INFO: dfs DDR fsp_param[1].freq_mhz= 324MHz
INFO: dfs DDR fsp_param[2].freq_mhz= 528MHz
INFO: dfs DDR fsp_param[3].freq_mhz= 780MHz
INFO: Using opteed sec cpu_context!
INFO: boot cpu mask: 0
INFO: BL31: Initializing runtime services
WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UN
K
ERROR: Error initializing runtime service opteed_fast
INFO: BL31: Preparing for EL3 exit to normal world
INFO: Entry point address = 0xa00000
INFO: SPSR = 0x3c9

U-Boot 2017.09-gef9f6b1caad-211223 #stephen (Sep 24 2022 - 17:13:02 +0800)
Model: Radxa ROCK3 Model A
PreSerial: 2, raw, 0xfe660000
DRAM: 7.7 GiB
Sysmem: init
Relocation Offset: ed349000
Relocation fdt: eb9f75a0 - eb9fecd0
CR: M/C/I
Using default environment

whereas, when I booting from SD-Card, my SBC gets correctly identified as Rock 3B and with the 2nd Ethernet available:

U-Boot SPL board init
U-Boot SPL 2017.09-gef9f6b1caad-211223 #stephen (Sep 24 2022 - 17:13:02)
Trying to boot from MMC2
Jumping to U-Boot(0x00a00000) via ARM Trusted Firmware(0x00040000)
Total: 817.932 ms
INFO: Preloader serial: 2
NOTICE: BL31: v2.3():v2.3-181-gc9a647cae-dirty:xsf
NOTICE: BL31: Built : 11:30:09, Oct 18 2022
INFO: GICv3 without legacy support detected.
INFO: ARM GICv3 driver initialized in EL3
INFO: pmu v1 is valid
INFO: dfs DDR fsp_param[0].freq_mhz= 1056MHz
INFO: dfs DDR fsp_param[1].freq_mhz= 324MHz
INFO: dfs DDR fsp_param[2].freq_mhz= 528MHz
INFO: dfs DDR fsp_param[3].freq_mhz= 780MHz
INFO: Using opteed sec cpu_context!
INFO: boot cpu mask: 0
INFO: BL31: Initializing runtime services
WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UN
K
ERROR: Error initializing runtime service opteed_fast
INFO: BL31: Preparing for EL3 exit to normal world
INFO: Entry point address = 0xa00000
INFO: SPSR = 0x3c9

U-Boot latest-2023.07.02-6-4257d241-g4257d241 (Oct 12 2023 - 07:58:46 +0000)
Model: Radxa ROCK 3 Model B
DRAM: 8 GiB (effective 7.7 GiB)
PMIC: RK8090 (on=0x40, off=0x00)
Core: 340 devices, 31 uclasses, devicetree: separate
MMC: mmc@fe2b0000: 1, mmc@fe310000: 0
Loading Environment from nowhere… OK
In: serial@fe660000
Out: serial@fe660000
Err: serial@fe660000
Model: Radxa ROCK 3 Model B
Net: No ethernet found.

I took the SPI Image from https://dl.radxa.com/rock3/images/loader/ per https://wiki.radxa.com/Rock3/install/spi#Get_RK356X_loader_and_U-Boot_images

Is there any SPI-Image for the Rock 3B or can please anyone tell me how to build SPI-Image with latest u-boot?

[dominik]

I’m seeking for help as I’m trying to boot my Rock 3B via SPI from SSD. Unfortunately my SBC gets recognized as Rock 3A, hence I assume that’s the reason why the 2nd Ethernet-Port is not available

Are You trying to boot from ethernet or was this port available later in system?
If You are using image for 3A probably second port is just not configured, but this has nothing to do about u-boot and ssd booting. Probably You have two different system images?
I was able to boot on Rock 3A from nvme, haven’t tested that with 3B yet but I expect that it should be the same. You need good pair of spi image for booting and system image. Not every combination works, usually different u-boot was needed to spin up armbian.

[JingleFl]

Hi dominik,

thx a lot for your reply. I kept on playing around a little bit, so here is what I did / what I noticed:

• I flashed the same jammy CLI-image (/github.com/radxa-build/rock-3b/releases/tag/b18) on SD-Card and on NVME

Booting from SD-Card, I have both ethernets available and everything seems to work perfectly fine and within the boot-log (see previous post, 2nd log), U-Boot is using a Rock3b configuration.

• Now, trying to make the Rock3b boot from NVME:

1. I flashed SPI-Image per /wiki.radxa.com/Rock3/install/spi as I assumed some compatibility based on the same rk3568 chip
2. jammy-CLI is successfully booting from NVME, but I’m missing the 2nd Ethernet and within the boot-log (see previous post, 1st log), U-Boot is using a Rock3a configuration.

• After putting back the SD-Card, everything is working fine with both ethernets and per log U-Boot is using a Rock3b configuration again - but OS is running from SD-Card.

So my conclusion is (correct me, if I’m wrong), I need a SPI-Image with U-boot properly configured for Rock 3b, so that the OS is picking up the right configuration-tree on its own?

So I found this repository https://github.com/radxa/rockchip-bsp which “stable-4.19-rock3”-branch supports rock-3b. I built the SPI-Image per README, but it completely failed booting from NVME (I’m still using the same image as mentioned before):

U-Boot SPL board init
U-Boot SPL 2017.09-g5d7d46a482-211223 #test-env (Feb 24 2024 - 10:10:39)
Trying to boot from MMC2
MMC error: The cmd index is 1, ret is -110
Card did not respond to voltage select!
mmc_init: -95, time 10
spl: mmc init failed with error: -95
Trying to boot from MMC1
MMC error: The cmd index is 0, ret is -110
mmc_init: -110, time 5
spl: mmc init failed with error: -110
Trying to boot from MTD2
No misc partition

I have the impression, this is somehow related with https://github.com/edk2-porting/edk2-rk3588/issues/93 but unfortunately I do not understand what kind of magic is needed from here on.

Any help or poke to the right direction is of course highly appreciated, but anticipate me being an enthusiastic hobbyist who is willing to learn

[t4_4t]

It is recommended that the versions of “SPL” and “u-boot” match.
Looking at log you provided, there are some version inconsistencies.

I can’t decide whether that’s what you intended or not.
However, if you combine versions that are too far apart (2017.09 vs 2023.07.02),
I don’t think there’s anything strange about it not working properly.

“SPL” which is originally paired with “latest-2023.07.02-6-4257d241-g4257d241” (SBC gets correctly identified as Rock 3B and with the 2nd Ethernet available) which you are reporting relatively good working results is as follows.

U-Boot SPL latest-2023.07.02-6-4257d241-g4257d241 (Oct 12 2023 - 07:58:46 +0000)

And the image provided by radxa “rock-3b_ubuntu_jammy_cli_b18.img.xz”, of course the two are configured with the same version combination. (See below)

# hexdump -C rock-3b_ubuntu_jammy_cli_b18.img | less
000320a0  0a 55 2d 42 6f 6f 74 20  53 50 4c 20 6c 61 74 65  |.U-Boot SPL late|
000320b0  73 74 2d 32 30 32 33 2e  30 37 2e 30 32 2d 36 2d  |st-2023.07.02-6-|
000320c0  34 32 35 37 64 32 34 31  2d 67 34 32 35 37 64 32  |4257d241-g4257d2|
000320d0  34 31 20 28 4f 63 74 20  31 32 20 32 30 32 33 20  |41 (Oct 12 2023 |
000320e0  2d 20 30 37 3a 35 38 3a  34 36 20 2b 30 30 30 30  |- 07:58:46 +0000|
000320f0  29 0a 00 6c 69 74 74 6c  65 2d 65 6e 64 69 61 6e  |)..little-endian|

"U-Boot SPL latest-2023.07.02-6-4257d241-g4257d241 (Oct 12 2023 - 07:58:46 +0000)"
---

008c1590  6e 75 78 20 20 20 00 55  2d 42 6f 6f 74 20 6c 61  |nux   .U-Boot la|
008c15a0  74 65 73 74 2d 32 30 32  33 2e 30 37 2e 30 32 2d  |test-2023.07.02-|
008c15b0  36 2d 34 32 35 37 64 32  34 31 2d 67 34 32 35 37  |6-4257d241-g4257|
008c15c0  64 32 34 31 20 28 4f 63  74 20 31 32 20 32 30 32  |d241 (Oct 12 202|
008c15d0  33 20 2d 20 30 37 3a 35  38 3a 34 36 20 2b 30 30  |3 - 07:58:46 +00|
008c15e0  30 30 29 00 25 73 3a 20  65 78 74 65 6e 64 65 64  |00).%s: extended|

"U-Boot latest-2023.07.02-6-4257d241-g4257d241 (Oct 12 2023 - 07:58:46 +0000)"

Related files are located under “/usr/lib/u-boot/rock-3b/”.

# ls -l /usr/lib/u-boot/rock-3b/
-rw-r--r--  1 root root  194560 10月 12 16:59 idbloader.img
-rw-r--r--  1 root root  457152 10月 12 16:59 rkboot.bin
-rw-r--r--  1 root root  270784 10月 12 16:59 rkboot_SPI_NAND.bin
-rwxr-xr-x  1 root root    4914 10月 12 16:59 setup.sh
-rw-r--r--  1 root root 1210368 10月 12 16:59 u-boot.itb

I do not own “rock-3B” myself, so I have not confirmed whether the above information actually works correctly.

[JingleFl]

Thx a lot for your hint where to find a compatible SPL! I flashed the new spi-image and corresponding OS where they belong to, but sadly I’m banging my head on the next problem as it looks like the kernel is crashing (jammy / bullseye, same result).

First part looks very promising, as you can see:

U-Boot SPL latest-2023.07.02-6-4257d241-g4257d241 (Oct 12 2023 - 07:58:46 +0000)
Trying to boot from SPI
[…]
INFO: Preloader serial: 2
NOTICE: BL31: v2.3():v2.3-181-gc9a647cae-dirty:xsf
NOTICE: BL31: Built : 11:30:09, Oct 18 2022
[…]
INFO: BL31: Initializing runtime services
WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UN
K
ERROR: Error initializing runtime service opteed_fast
INFO: BL31: Preparing for EL3 exit to normal world
INFO: Entry point address = 0xa00000
INFO: SPSR = 0x3c9

U-Boot latest-2023.07.02-6-4257d241-g4257d241 (Oct 12 2023 - 07:58:46 +0000)

Model: Radxa ROCK 3 Model B
DRAM: 8 GiB (effective 7.7 GiB)
PMIC: RK8090 (on=0x40, off=0x00)
Core: 340 devices, 31 uclasses, devicetree: separate
MMC: mmc@fe2b0000: 1, mmc@fe310000: 0
Loading Environment from nowhere… OK
In: serial@fe660000
Out: serial@fe660000
Err: serial@fe660000
Model: Radxa ROCK 3 Model B
Net: No ethernet found.

starting USB…
[…]
pcie_dw_rockchip pcie@fe260000: PCIe-0 Link Fail
** Booting bootflow ‘nvme#0.blk#1.bootdev.part_3’ with extlinux
U-Boot menu
1: Debian GNU/Linux 11 (bullseye) 5.10.160-18-rk356x
2: Debian GNU/Linux 11 (bullseye) 5.10.160-18-rk356x (rescue target)
Enter choice: 1: Debian GNU/Linux 11 (bullseye) 5.10.160-18-rk356x
Retrieving file: /boot/vmlinuz-5.10.160-18-rk356x
Retrieving file: /boot/initrd.img-5.10.160-18-rk356x
append: root=UUID=d210e617-6a4c-4771-b955-ddd835a32d2b quiet splash loglevel=4 rw earlycon consoleblank=0 console=tty0 console=ttyFIQ0,
1500000n8 console=ttyAML0,115200n8 console=ttyS2,1500000n8 console=ttyS0,1500000n8 coherent_pool=2M irqchip.gicv3_pseudo_nmi=0 cgroup_e
nable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1
Retrieving file: /usr/lib/linux-image-5.10.160-18-rk356x/rockchip/rk3568-rock-3b.dtb
Moving Image from 0x2080000 to 0x2200000, end=3d90000
'## Flattened Device Tree blob at 0a100000
Booting using the fdt blob at 0xa100000
Working FDT set to a100000
Host not halted after 16000 microseconds.
Loading Ramdisk to ec6bd000, end eceb0c4c … OK
ERROR: reserving fdt memory region failed (addr=0 size=0 flags=0)
ERROR: reserving fdt memory region failed (addr=0 size=0 flags=0)
Loading Device Tree to 00000000ec690000, end 00000000ec6bc0f4 … OK
Working FDT set to ec690000

Starting kernel …

But then, on NVME, it hangs on this, whereas on SD-Card its running smoothly till login prompt:

[ 11.880893] fiq_debugger fiq_debugger.0: IRQ fiq not found
[ 11.880945] fiq_debugger fiq_debugger.0: IRQ wakeup not found
[ 11.880976] fiq_debugger_probe: could not install nmi irq handler
[ 13.366432] rk-pcie 3c0000000.pcie: IRQ msi not found
[ 13.366520] rk-pcie 3c0000000.pcie: Missing config reg space
[ 13.366820] rk-pcie 3c0000000.pcie: Missing config reg space
[ 13.366905] rk-pcie 3c0000000.pcie: invalid resource
[ 13.376203] mpp-iep2 fdef0000.iep: allocate roi buffer failed
[ 13.377018] rk-pcie 3c0800000.pcie: IRQ msi not found
[ 13.377082] rk-pcie 3c0800000.pcie: Missing config reg space
[ 13.377393] rk-pcie 3c0800000.pcie: Missing config reg space
[ 13.377475] rk-pcie 3c0800000.pcie: invalid resource
[ 13.398870] arm-scmi firmware:scmi: Failed. SCMI protocol 22 not active.
[ 13.423598] dwhdmi-rockchip fe0a0000.hdmi: IRQ index 1 not found
[ 13.427275] no ATF memory for init
[ 13.448630] no ATF memory for init
[ 13.463759] rk_gmac-dwmac fe010000.ethernet: no regulator found
[ 13.698974] rk_gmac-dwmac fe2a0000.ethernet: no regulator found
[ 14.058211] … rk pwm remotectl v2.0 init
[ 14.129933] rk817-battery rk817-battery: Failed to find matching dt id
[ 14.130355] rk817-charger rk817-charger: Failed to find matching dt id
[ 14.180353] arm-scmi firmware:scmi: Failed. SCMI protocol 17 not active.
[ 14.186372] rksfc_base v1.1 2016-01-08
[ 14.221158] mpp_rkvenc fdf40000.rkvenc: Failed to get leakage
[ 14.221547] mpp_rkvenc fdf40000.rkvenc: failed to find power_model node
[ 14.221566] mpp_rkvenc fdf40000.rkvenc: failed to initialize power model
[ 14.221578] mpp_rkvenc fdf40000.rkvenc: failed to get dynamic-coefficient
[ 14.223006] mpp_rkvdec2 fdf80200.rkvdec: shared_niu_a is not found!
[ 14.223030] rkvdec2_init:1022: No niu aclk reset resource define
[ 14.223044] mpp_rkvdec2 fdf80200.rkvdec: shared_niu_h is not found!
[ 14.223056] rkvdec2_init:1025: No niu hclk reset resource define
[ 14.223724] mpp_rkvdec2 fdf80200.rkvdec: failed to find power_model node
[ 14.223743] mpp_rkvdec2 fdf80200.rkvdec: failed to initialize power model
[ 14.223756] mpp_rkvdec2 fdf80200.rkvdec: failed to get dynamic-coefficient
[ 14.226780] rockchip-dmc dmc: failed to get vop pn to msch rl
[ 14.227786] rockchip-dmc dmc: could not find power_model node
[ 16.257372] rk-pcie 3c0000000.pcie: PCIe Link Fail
[ 16.257509] rk-pcie 3c0000000.pcie: failed to initialize host
[ 45.484938] blk_update_request: I/O error, dev nvme0n1, sector 976772992 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[ 45.560878] Buffer I/O error on dev nvme0n1, logical block 122096624, async page read
[ 45.570609] Buffer I/O error on dev nvme0n1p1, logical block 4080, async page read
[ 45.570619] Buffer I/O error on dev nvme0n1p3, logical block 4115200, async page read
[ 45.570639] Buffer I/O error on dev nvme0n1p3, logical block 4115201, async page read
[ 45.570783] Buffer I/O error on dev nvme0n1p3, logical block 4115202, async page read
[ 45.570815] Buffer I/O error on dev nvme0n1p3, logical block 4115203, async page read
[ 45.570848] Buffer I/O error on dev nvme0n1p3, logical block 4115204, async page read
[ 45.570878] Buffer I/O error on dev nvme0n1p3, logical block 4115205, async page read
[ 45.570907] Buffer I/O error on dev nvme0n1p3, logical block 4115206, async page read
[ 45.570936] Buffer I/O error on dev nvme0n1p3, logical block 4115207, async page read

I would have blamed my 500GB samsung NVME, but after completely rolling back to the rock-3a configuration, jammy was doing fine again (apart from the missing 2nd ethernet adapter).

Is that a bug? Any idea how to solve this issue or shall I simply order an EMMC module and try again?

[t4_4t]

Judging from the logs you provided, I believe that NVME booting from “U-Boot @ SPI-Flash” is working fine.

starting USB…
[…]
pcie_dw_rockchip pcie@fe260000: PCIe-0 Link Fail
** Booting bootflow ‘nvme#0.blk#1.bootdev.part_3’ with extlinux
U-Boot menu
1: Debian GNU/Linux 11 (bullseye) 5.10.160-18-rk356x
2: Debian GNU/Linux 11 (bullseye) 5.10.160-18-rk356x (rescue target)
...
     Starting kernel...

Please pay attention to the following part of the log you presented.

...
[ 45.484938] blk_update_request: I/O error, dev nvme0n1, sector 976772992 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
...

The above log indicates that there is a problem with the PCIEx communication route, indicating that reliable communication may not be possible.

The problem seems to be in the kernel or device (500GB samsung NVME).

---

The article below is for “rock-3a”, but please note that such symptoms have been reported as a reference.

M2.ssd will randomly fail to write on rock3a

hey there, i have been running a rock3a for a while with a poe hat and nvme attached to it and it randomly starts to fail to write to the disk and then the kernel remounts it as readonly. you can see the dmesg log here gist:dfb76e9401ebb932bbfdb90734a4ebc8 (github.com) and the kernel commit i am running is the following arm64: dts: radxa cm3 io: add HD101BOE9365 Display support · radxa/kernel@12d0b2b (github.com) with this only change on the config diff --git a/arch/arm64/configs/rockchip_li…

I personally do not think there is a problem with the device (NVME), but considering that there are many similar reports,
I think it is reasonable to assume that the hardware (board ?) margin is small.

===
Try adding constraints to the PCIEx communication speed and see if there is any improvement.
The parts where constraints are set are the “max-link-speed” and “num-lanes” parts below.
First, try lowering it step by step as “max-link-speed” = <3>, -> <2>, -> <1>.
And see if the situation improves.

If you don’t see any improvement, try restricting it further with “num-lanes” = <1>;.

     pcie@fe280000 {
...
        max-link-speed = <0x03>;    // PCIEx Speed = Gen3
        num-lanes = <0x02>;         // Lane-Num = 2
...
        pinctrl-0 = <0xb6>;         // <&pcie30x2m1-pins>;
        vpcie3v3-supply = <0xb3>;   // <&gpio-regulator>;
...
        status = "okay";
     };

The change method is as follows.
Please back up the relevant files as your necessary.

dtc -O dts -I dtb -o rk3568-rock-3b.dts /usr/lib/linux-image-5.10.160-18-rk356x/rockchip/rk3568-rock-3b.dtb
[" Modified by text-editor" ]
dtc -O dtb -I dts -o rk3568-rock-3b.dtb rk3568-rock-3b.dts

cp -av rk3568-rock-3b.dtb /usr/lib/linux-image-5.10.160-18-rk356x/rockchip/

If there is still no change, there are only passive measures such as trying another “NVME device” or increasing the power supply capacity.
Or you’ll have no choice but to give up.

[devon]

Not sure exactly what software stack you are after, but FYI, rk3568 is well supported by mainline uboot and the linux kernel. For u-boot, I built an SPI image using the rock-3a config. Just assure you enable CONFIG_ROCKCHIP_SPI_IMAGE and the compiler will spit out: u-boot-rockchip-spi.bin. You can write that directly to SPI.

As for enabling board features, I’m just using the upstream Rock3a dts, for now. That means a few things are not working including ethernet #2, but that will be fixed once everything is enabled in the dts. I’ll share once I do it, if you are interested in running upstream software.

If you need any details, just shout.

edit: oh yeah, i had many issues with my rock 5 and samsung nvme drives. now, i stick to SK Hynix Gold P31 which have been flawless with SBCs from a few Chinese companies.

[onolide]

For those who might need this, here’s the spi image rock-3b-spi_image.zip generated using u-boot.itb and idbloader.img provided with radxa’s official b18 release at /usr/lib/u-boot/rock-3b/. Have flashed it to my Rock 3B’s SPI flash and boots from SSD just fine.

Just unzip it and flash the rock-3b-spi_image.img image using RKDevTool as explained at the Rock 3A spi page (exact same steps, including the same spl loader bin, just with this spi image instead of the Rock 3A one)

Update:
I’ve also compiled u-boot with Armbian patches to generate this spi image rock-3b-spi_image.zip Boots just fine from SSD after flashing it on my Rock 3B’s SPI flash as well