I have a 2 node 16GB Rock5b ProxMox cluster running now! I have also deployed a debian 12.1 VM.
I will add additional information once I have my notes straight & a completely solid recipe.
Rock 5B Virtualisation
2 Likes
https://wiki.radxa.com/Rock5/downloads
This image https://dietpi.com/downloads/images/DietPi_ROCK5B-ARMv8-Bullseye.7z
Install OpenSSH Server ( My Preference over dropbear )
dietpi-config
Network Options: Adapters
Turn off IPV6
IPV4 Turn off DHCP & copy current settings to static
nano /etc/hostname ##set hostname
curl https://raw.githubusercontent.com/pimox/pimox7/master/KEY.gpg | apt-key add -
apt update
curl https://raw.githubusercontent.com/pimox/pimox7/master/RPiOS64-IA-Install.sh > RPiOS64-IA-Install.sh
chmod +x RPiOS64-IA-Install.sh
from hard wired console ./RPiOS64-IA-Install.sh
from hard wired console after reboot —> apt upgrade -y
reboot
===========================================
References:
Guide https://pycvala.de/blog/raspberry-pi/raspberry-pi-installing-proxmox-ve-7-on-the-pi-4/
Container
https://uk.lxd.images.canonical.com/images/ubuntu/jammy/arm64/default/20230830_07:57/
VM Images
https://cdimage.debian.org/debian-cd/current/arm64/iso-cd/
4 Likes
longshot, but have you tried to pass through the gpu?
@shelter ARM64 SoC:s do support virtualisation.
@flyingRich Sorry the instructions you shared are a bit unclear.
Radxa’s Debian distribution for Rock5B appears to bundle a KVM module but it’s not loaded by default.
Has anyone had success with running KVM VM:s on Rock5B?
QEMU and LibVirt work equally well?
I have tried using KVM and it seems to work but I couldn’t get windows ARM to boot: Actually booting Windows 11 using a KVM virtual machine
Libvirt and KVM themselves work pretty fine, there just seems to be a problem with UEFI. I managed to boot Windows XP, Ubuntu x64 and Windows 7 using qemu. Haven’t tried native KVM with a Linux distro but I think it will work, you just might need to turn the a55 cores offline (there is a way, search for it).
Proxmox has basically the same functionality as Virt-manager but as a webUI
Why would you need to disable the A55 cores to use KVM?
Could you share the QEMU command line you used to launch installer and launch the machine
I think the spaghetti qemu commands are a bit useless and prefer to use Virt-manager, but if you really want I can find out and share it (but I couldn’t get Windows to work anyway, so I don’t know of what use it would be). As said above, the systems that worked used QEMU emulation, not KVM, but I expect the problem with Windows was not with KVM itself but with UEFI being badly implemented in libvirt.
As for why the A55 should be disabled - I’m not sure, it was in one of the tutorials I found. Might be related to this: https://forum.pine64.org/showthread.php?tid=18202 (A53 and A72 here). Or this: https://patchwork.kernel.org/project/linux-arm-kernel/patch/20191216115631.17804-4-steven.price@arm.com/
What is your question?
If I Can post a video, I will.
Hi,
I also stumbled across this thread while playing around with kubevirt on the Rock 5B. And it seems that QEMU in KVM mode only works with older EFI versions. The last EFI version that worked for me was EFI ‘2022.02’ from https://packages.ubuntu.com/jammy/qemu-efi-aarch64 - all later versions didn’t boot up. I tried later Ubuntu versions, alpine and the ‘stock’ version from kubevirt. But all other Qemu versions work with the 2022.02 EFI from Ubuntu in KVM mode.
If anyone else wants to use kubevirt on Rock5: Here are some examples.
I only tried Linux as a guest OS, so I have no idea if it works for Windows.
Just wanted to share my findings in the hope that it might be useful 
Bye,
Chris
2 Likes
Wow, this might be the key to solve this: Actually booting Windows 11 using a KVM virtual machine - I never got it to work with so many qemu and libvirt versions.
if this works on chromeOS it must work on any linux either and apparently can virtualize windows 11 too.
I havent tested it yet… as it didnt worked on openfyde sokething on my .yaml config was wrong…
In this example they aren’t using native KVM for virtualization of windows, but emulation.
What is everyone’s results with KVM hardware virtualisation on the Rock 5B now?
1 Like
It works when you disable all A55 cores or all A76 cores (mixing cores makes it crash). Not the fault of Rock 5B but of qemu.
1 Like
How do you disable the slow cores in Linux, from boot time?
And then, what is your command line to launch QEMU with one Linux installer image e.g. as virtual USB memory stick to boot the device from in one image file, and then a virtual SSD in another image file?
Maybe cpu pinning?
(I havent tried it yet, i would like to know how well it performs too)
This issue deals with it a bit: https://github.com/Joshua-Riek/ubuntu-rockchip/issues/731
Briefly, to disable A55 cores, do this:
sudo chcpu --disable 0,1,2,3
Pinning works in theory but did not work for me.
1 Like
Will the chcpu gain effect right away?
What arguments do you pass to QEMU to get Linux installer on a USB memory stick from an image file, and have a virtual SSD on another image file?
Yes it’s immediate.
Not sure about arguments, I use the virt-manager GUI.
1 Like
Windows on ARM just doesn’t run well with vCPUs having differenct characteristics for some reason, even when it boots it will eventually cause some of the vCPUs to hang with 100% utilization. Linux on QEMU/KVM runs just fine.
You don’t need to disable cores, pinning each vCPU to a physical core would make Windows happy as long as there is no mixing of A76 and A55 cores.
To do this with libvirt, add the following to the domain configuration under <domain> tag:
<cputune>
<vcpupin vcpu="0" cpuset="4"/>
<vcpupin vcpu="1" cpuset="5"/>
<vcpupin vcpu="2" cpuset="6"/>
<vcpupin vcpu="3" cpuset="7"/>
</cputune>
This would pin all 4 cores of the VM to 4 A76 cores.
I tried this before with mainline kernel and QEMU 8.0 on openSUSE and Windows runs fine, though the performance isn’t impressive.