Anyone got Docker working on 5B with Debian/Ubuntu?

guywhosteals · November 18, 2022, 2:48am

Hi, on both distros I get this:

$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
7050e35b49f5: Pull complete
Digest: sha256:faa03e786c97f07ef34423fccceeec2398ec8a5759259f94d99078f264e9d7af
Status: Downloaded newer image for hello-world:latest
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: bpf_prog_query(BPF_CGROUP_DEVICE) failed: function not implemented: unknown.
ERRO[0008] error waiting for container: context canceled

This seems to be a missing CGROUP on the kernels on both images. Any way around this?

EDIT: Of course recompiling the kernel (https://wiki.radxa.com/Rock5/guide/build-kernel-on-5b) should work, but IMO Docker should work out of the box because it is not a weird edge-case use. I assume most people who run Rock 5B use it to run Docker containers.

gnattu · November 18, 2022, 3:01am

It IS a weird edge-case use for Rockchip. The default BSP kernel is nothing more than an Android Kernel and it is very feature lacking if you want to use it on regular GNU Linux. The Radxa repo has merged kernel configs to enable features required for containers, for example https://github.com/radxa/kernel/pull/36 but that’s for Kubernetes, I think you need to enable additional features to support docker.

igorp · November 18, 2022, 8:01am

It works, yes:

guywhosteals · November 18, 2022, 3:24pm

I did these steps already. The error that comes from before doing these steps is about the Docker daemon not starting.

In this case, the daemon has started, but fails on docker run command.

kangdei · November 18, 2022, 5:05pm

我同样遇到了这样的问题,但是官方目前没有解决

lin_yang · November 19, 2022, 8:11am

如图中，需要把 "systemd.unified_cgroup_hierarchy=0"加到/boot/config.txt，这是给内核的参数
比如：cmdline: systemd.unified_cgroup_hierarchy=0
最后执行sudo update_extlinux.sh并重启系统即可

Google translate：You need to add “systemd.unified_cgroup_hierarchy=0” to /boot/config.txt, which is a parameter for the kernel
For example: “cmdline: systemd.unified_cgroup_hierarchy=0”
Finally execute sudo update_extlinux.sh and restart the system

guywhosteals · November 19, 2022, 8:11am

This fixed it, thanks a lot!

How do you know this solution, if you don’t mind me asking 谢谢您 ‍♂

stuartiannaylor · November 19, 2022, 2:57pm

Google and some hair pulling usually. but some are docker regulars and pretty much know it inside out
So great as docker was wanting with the Debian image

Satoshi_Owada · November 21, 2022, 7:31am

Today I did apt upgrade. Then I got kernel 5.10.66-28.
It looks this version is configured with CGROUP enabled.

mcooper59 · November 21, 2022, 11:56am

Hello Guys,

These are the ones I use (cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 swapaccount=1) in the cmdline.txt. I also turn off wifi and bluetooth in the config.txt.

Depending on whether you are using RPiOS or Ubuntu of course
RPiOS - /boot/cmdline.txt and /boot/configtxt
Ubuntu - /boot/firmware/cmdline.txt and /boot/firmware/config.txt

Let me know if you need my exact commands for each file.

I am going to assume you mean Rock Pi 5B as opposed to Rock Pi 4b v1.5?

Good Luck,
Michael

stuartiannaylor · November 21, 2022, 12:32pm

append /boot/extlinux/extlinux.conf with systemd.unified_cgroup_hierarchy=0 is all that seems to be needed as we are not using a Pi.

Yep Rock-5b

Append systemd.unified_cgroup_hierarchy=0
to the /boot/extlinux/extlinux.conf 

Install iptables
update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

/etc/sysctl.conf
net.ipv4.ip_forward = 1

reboot
install docker

Thanks to milnepe

kangdei · November 21, 2022, 2:37pm

感谢,通过这个方法,我的Docker正常工作了

williamvm · November 21, 2022, 3:01pm

i run most of my software as containers so not ‘weird edge-case’ for me.

murpf · November 22, 2022, 8:37pm

I packaged the latest podman version for debian/ubuntu in aarch64, if you want to give it a try. Didn’t really get to taking it for a spin, since the little one got a tough cold. But I don’t see anything problematic. I stopped at the setup of firewalld to redirect to a non-privileged port. But it seems, that netfilter and nftables is problematic in this setup (or, the much more plausible option, the problem is on layer 8… happens a lot with me)

repo for podman:

stuartiannaylor · November 22, 2022, 9:37pm

Prob needed as does docker
https://wiki.debian.org/iptables
After installing iptables

update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

murpf · November 24, 2022, 12:09pm

Just checked… apparently the kernel is so old, that its still on iptables/xtables ?!

stuartiannaylor · November 24, 2022, 12:31pm

Its actually the oppisite as we have to swap from nftables to iptables-legacy as that is the problem with Docker as it doesn’t work with the newer nf_tables

That is why you need to

update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

So setting back to legacy iptables from nftables

murpf · November 24, 2022, 10:30pm

weird. i shouldnt need iptables (podman instead of docker) i’m using armbian. just installed nftables

andi@rock-5b:~$ sudo nft add rule ip filter INPUT ip saddr 192.0.2.0/24 counter accept
netlink: Error: cache initialization failed: Invalid argument

this doesn’t look right. checking and I can only see iptables/xtables being loaded.

andi@rock-5b:~$ lsmod 
Module                  Size  Used by
nfnetlink              16384  0
overlay               106496  0
zstd                   16384  8
fuse                  106496  1
ip_tables              24576  0
x_tables               32768  1 ip_tables
ipv6                  413696  62
panfrost               53248  0
gpu_sched              28672  1 panfrost
pgdrv                  16384  0
r8125                 139264  0

shouldn’t there be at least some nf* module loaded?
as an example from my rockpro64 running armbian
nf_tables 212992 1567 nft_compat,nft_counter,nft_chain_nat

stuartiannaylor · November 24, 2022, 11:14pm

x_tables is netfilter so is nfnetlink iptables is just the userspace program using them.

Apart from that haven’t a clue as rarely use and was searching for where iptables had gone.
Guess you will have to do some reading of https://wiki.debian.org/nftables as just dunno

Etienne · November 30, 2022, 4:13pm

I had 2 kernels version listed in /boot/extlinux/extlinux.conf, I updated most recent one by adding

systemd.unified_cgroup_hierarchy=0

to the “append” section.

Not sure why but after rebooting, my network interface disappeared.
Once changes reverted, it was back.

Any idea? Did I correctly updated conf?

Btw, seems like working as I could run sudo docker run hello-world successfully