Idea about plug AMD GPU via NVME

Hi, I used Phytoum D2000 which has Linux mainline kernel support,
I used it rum archlinuxarm and equiped a Radeon RX5500XT, it runs well after some magic tune🤣

2 Likes

very surprised, and never heard of that arm chip before!
also you were lucky archlinuxarm was available…

Looking a bit to specification of the Phytium D2000, looks like a very uncommon machine:

  • SoC – Phytium D2000 ARMv8 octa-core custom Armv8 (FTC663) desktop processor @ 2.3 – 2.6 GHz with 8MB L2 Cache (2MB per dual-core clusters) and 4MB L3 Cache; 14nm process; 25W TDP; 1144-pin FCBGA package (35×35 mm)
  • GPU – AMD Radeon RX 550 MXM 3.0a graphics card with 2GB VRAM

They didn’t go “all-in” as Apple with M1 which did both CPU and GPU on same soc,
but chose to mix Arm with classic external graphic card
Also they use 14nm which is much more than 8nm process used in RK3588 as it seems

@geerlingguy tried that on pi with some progress on hist yt channel,
I don’t expect that to be fully working anytime soon, but interestingly this may work some day.

I found someone did this thing, but can’t find who did this :slight_smile:

1 Like

How did you managed to do that?
I ended with this

[  124.474708] [drm] amdgpu kernel modesetting enabled.
[  124.476599] amdgpu 0002:23:00.0: enabling device (0000 -> 0003)
[  124.476635] [drm] initializing kernel modesetting (NAVI14 0x1002:0x7340 0x1DA2:0xE421 0xC5).
[  124.476650] amdgpu 0002:23:00.0: amdgpu: Trusted Memory Zone (TMZ) feature disabled as experimental (default)
[  124.476695] [drm] register mmio base: 0xF2200000
[  124.476701] [drm] register mmio size: 524288
[  124.476731] [drm] PCIE atomic ops is not supported
[  124.478522] [drm] add ip block number 0 <nv_common>
[  124.478530] [drm] add ip block number 1 <gmc_v10_0>
[  124.478538] [drm] add ip block number 2 <navi10_ih>
[  124.478557] [drm] add ip block number 3 <psp>
[  124.478573] [drm] add ip block number 4 <smu>
[  124.478588] [drm] add ip block number 5 <gfx_v10_0>
[  124.478603] [drm] add ip block number 6 <sdma_v5_0>
[  124.478616] [drm] add ip block number 7 <vcn_v2_0>
[  124.478622] [drm] add ip block number 8 <jpeg_v2_0>
[  124.512534] amdgpu 0002:23:00.0: amdgpu: Fetched VBIOS from ROM BAR
[  124.512554] amdgpu: ATOM BIOS: 113-E4210SM-U03
[  124.512602] [drm] VCN decode is enabled in VM mode
[  124.512607] [drm] VCN encode is enabled in VM mode
[  124.512612] [drm] JPEG decode is enabled in VM mode
[  124.512639] [drm] GPU posting now...
[  124.512815] [drm] vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit
[  124.512860] amdgpu 0002:23:00.0: BAR 2: releasing [mem 0x990000000-0x9901fffff 64bit pref]
[  124.512872] amdgpu 0002:23:00.0: BAR 0: releasing [mem 0x980000000-0x98fffffff 64bit pref]
[  124.512981] pcieport 0002:22:00.0: BAR 9: releasing [mem 0x980000000-0x997ffffff 64bit pref]
[  124.512994] pcieport 0002:21:00.0: BAR 9: releasing [mem 0x980000000-0x997ffffff 64bit pref]
[  124.513003] pcieport 0002:20:00.0: BAR 9: releasing [mem 0x980000000-0x997ffffff 64bit pref]
[  124.513044] pcieport 0002:20:00.0: BAR 9: no space for [mem size 0x300000000 64bit pref]
[  124.513054] pcieport 0002:20:00.0: BAR 9: failed to assign [mem size 0x300000000 64bit pref]
[  124.513069] pcieport 0002:21:00.0: BAR 9: no space for [mem size 0x300000000 64bit pref]
[  124.513078] pcieport 0002:21:00.0: BAR 9: failed to assign [mem size 0x300000000 64bit pref]
[  124.513093] pcieport 0002:22:00.0: BAR 9: no space for [mem size 0x300000000 64bit pref]
[  124.513102] pcieport 0002:22:00.0: BAR 9: failed to assign [mem size 0x300000000 64bit pref]
[  124.513117] amdgpu 0002:23:00.0: BAR 0: no space for [mem size 0x200000000 64bit pref]
[  124.513127] amdgpu 0002:23:00.0: BAR 0: failed to assign [mem size 0x200000000 64bit pref]
[  124.513143] amdgpu 0002:23:00.0: BAR 2: no space for [mem size 0x00200000 64bit pref]
[  124.513153] amdgpu 0002:23:00.0: BAR 2: failed to assign [mem size 0x00200000 64bit pref]
[  124.513166] pcieport 0002:20:00.0: PCI bridge to [bus 21-23]
[  124.513178] pcieport 0002:20:00.0:   bridge window [io  0x1000-0x1fff]
[  124.513191] pcieport 0002:20:00.0:   bridge window [mem 0xf2200000-0xf23fffff]
[  124.513208] pcieport 0002:20:00.0: PCI bridge to [bus 21-23]
[  124.513217] pcieport 0002:20:00.0:   bridge window [io  0x1000-0x1fff]
[  124.513226] pcieport 0002:20:00.0:   bridge window [mem 0xf2200000-0xf23fffff]
[  124.513235] pcieport 0002:20:00.0:   bridge window [mem 0x980000000-0x997ffffff 64bit pref]
[  124.513245] pcieport 0002:21:00.0: PCI bridge to [bus 22-23]
[  124.513259] pcieport 0002:21:00.0:   bridge window [io  0x1000-0x1fff]
[  124.513281] pcieport 0002:21:00.0:   bridge window [mem 0xf2200000-0xf22fffff]
[  124.513297] pcieport 0002:21:00.0:   bridge window [mem 0x980000000-0x997ffffff 64bit pref]
[  124.513324] pcieport 0002:22:00.0: PCI bridge to [bus 23]
[  124.513336] pcieport 0002:22:00.0:   bridge window [io  0x1000-0x1fff]
[  124.513357] pcieport 0002:22:00.0:   bridge window [mem 0xf2200000-0xf22fffff]
[  124.513374] pcieport 0002:22:00.0:   bridge window [mem 0x980000000-0x997ffffff 64bit pref]
[  124.513433] [drm] Not enough PCI address space for a large BAR.
[  124.513447] amdgpu 0002:23:00.0: BAR 0: assigned [mem 0x980000000-0x98fffffff 64bit pref]
[  124.513492] amdgpu 0002:23:00.0: BAR 2: assigned [mem 0x990000000-0x9901fffff 64bit pref]
[  124.513552] amdgpu 0002:23:00.0: amdgpu: VRAM: 8176M 0x0000008000000000 - 0x00000081FEFFFFFF (8176M used)
[  124.513564] amdgpu 0002:23:00.0: amdgpu: GART: 512M 0x0000000000000000 - 0x000000001FFFFFFF
[  124.513575] [drm] Detected VRAM RAM=8176M, BAR=256M
[  124.513582] [drm] RAM width 128bits GDDR6
[  124.514182] [TTM] Zone  kernel: Available graphics memory: 1913096 KiB
[  124.514188] [TTM] Initializing pool allocator
[  124.514223] [TTM] Initializing DMA pool allocator
[  124.514358] [drm] amdgpu: 8176M of VRAM memory ready
[  124.514375] [drm] amdgpu: 2802M of GTT memory ready.
[  124.514391] [drm] GART: num cpu pages 131072, num gpu pages 131072
[  124.514595] [drm] PCIE GART of 512M enabled (table at 0x0000008000000000).
[  124.575879] [drm] Found VCN firmware Version ENC: 1.16 DEC: 5 VEP: 0 Revision: 4
[  124.575934] amdgpu 0002:23:00.0: amdgpu: Will use PSP to load VCN firmware
[  127.521038] [drm:psp_hw_start [amdgpu]] *ERROR* PSP load kdb failed!
[  127.521880] [drm:psp_hw_init [amdgpu]] *ERROR* PSP firmware loading failed
[  127.522743] [drm:amdgpu_device_fw_loading [amdgpu]] *ERROR* hw_init of IP block <psp> failed -22
[  127.522791] amdgpu 0002:23:00.0: amdgpu: amdgpu_device_ip_init failed
[  127.522812] amdgpu 0002:23:00.0: amdgpu: Fatal error during GPU init
[  127.523047] amdgpu 0002:23:00.0: amdgpu: amdgpu: finishing device.

Probably related to the card model. GCN cards are known to be working but RDNA cards not so much. When I was playing with RISC-V development boards that does not have integrated GPUs, GCN cards boots just fine but I cannot use any RDNA cards on that yet.

But this looks like bars problem.
It looks like pcie ranges are no properly set in dts

Did you done anything special RADXA?
[https://twitter.com/theradxa/status/1592374915231289344/photo/1]

2 Likes

Poor Jeff Geerling xD He has tried getting an external GPU to work on the Pi4 and failed miserably. Some unknown guy does it on the Rock5 in a few weeks

2 Likes

@geerlingguy work maybe opened doors to this, I don’t think that all that effort was useless and sure was fun to watch.

i think @geerlingguy had to struggle with the pi because of its wierd construction of its soc but the rock 5b is more standard

Of course he did. I also found the idea very interesting and fun to watch. I also didn’t mean that the effort was useless.

I’m trying to replicate this on my own Rock 5 model B … it would be nice to know how @radxa was able to do this. I’m still searching around for any way I can recompile the kernel with support for radeon or amdgpu drivers. Might try installing them from package too, to see if that just works. We’ll see.

4 Likes

@geerlingguy i think with this beast of an sbc it will be not so much work like with the raspberry pi 4
(maybe its even faster if you compile the linux kernel on the rock 5b instead of your mac studio)

You can build image with armbian and you can configure linux kernel build in armbian :slight_smile:

I found instructions for recompiling the kernel here: https://wiki.radxa.com/Rock5/guide/build-kernel-on-5b

When I did that (enabling the ATI radeon graphics module), the kernel compiled okay, but when I tried switching to it, the Rock 5 would no longer boot (the blue LED would flash in a pattern with two blinks, and no display output would happen).

@geerlingguy I know for your videos/reviews you’d probably prefer to work with Radxa’s image, but if you want to get it to work I’d suggest working it with the Armbian kernel and build platform.

2 Likes

If it is flashing with two blinks the kernel is probably loaded but the GPU failed to initialize. You can use an uart serial to see what is happening.

thats never a bad idea hooking up a serial cable