Image size after above compilation is around 4Gbyte size, but SDnand is only 1Gbyte.
I need to find a way to build smaller image.
Do you have any suggestions ?
Investigating smallest possible Debian Rootfs for RockPi-S
Stock minimal Armbian image will fit perfectly on 1Gb.
Whatever you need extra:
https://docs.armbian.com/Developer-Guide_User-Configurations/#user-provided-image-customization-script
You can build a custom OS using yocto. The base minimal gpt image will be less than 300mb and console image of 600mb. You could follow the instructions given here to build your own image: https://github.com/nishantpoorswani/meta-radxa
Or
https://wiki.radxa.com/Yocto-layer-for-radxa-boards
You can customize and add whatever you need
If you need that and if time input in learning and dealing with Yocto is justified. Yocto is nice but mastering it takes time - Yocto and buildroot compared with Armbian build system.
We’ve kept the readme simple and it is easy to build now currently. I honestly like Armbian btw 
1 Like
@nish Thank you very much. I considered Yocto for a long time, and unfortunately I need a debian based rootfs with APT. Can I have APT with these instructions ?
Update/edit: Installed official yocto image:
- /dev/root 125.1M 97.7M 18.1M 84% /
No rootfs resize?- No APT ?
- No NetworkManager ?
- No SSH ?
Therefore I am considering ISAR ( https://github.com/ilbers/isar ) for many SBC’s in the market, which uses bitbake + debian based rootfs.
Unfortunately, as Igor suggested, initial porting + learning curve to have a Isar based build system is too expensive for hobby projects.
These days, I also considered DEBOS ( https://github.com/go-debos/debos ) which also lacks documentation for porting to a new device.
@igorp Nice to see you here Igor. And thanks for your suggestion for armbian.
I also considered learning armbian, and I have some criticisms for it:
- No Incremental build: I couldn’t find a clear documentation about how to achieve incremental ( delta ) builds with armbian. For example, if I change only one line in u-boot or kernel or userpatches, how can I rebuild whole image in less than 2 mins ?
- Not enough documentation for porting armbian to a new board: I know you don’t want to maintain lots of custom boards which maintainers can not access. This is fair. But this avoids for someone like me who works on multiple Allwinner & RockChip & IMX8 based boards, and to have a generic build system for all of it.
- Not enough documentation for customisation considerations: How to create a extremely minimal rootfs, just to have Ethernet + Debian APT with Armbian ?
Finally, an answer to both of you. I have 3 projects which I would like to achieve with RockPi-S. One of them is to run docker-pihole on it. And this requires around 500Mbyte of storage space.
Even minimal image that I downloaded from radxa website is around 900+Mbyte when installed to SDnand. If I am not mistaken, it is almost not possible to install docker with apt ?
Am I missing anything ?
Need your suggestions please…
That very important feature is supported since early days, but not enabled by default - check documentation CLEAN parameter in the default config. One can easily recompile kernel & u-boot image in less then 1 minute (on a modern desktop CPU), but for the whole image you need several minutes. Everything that is possible is cached but there is a small room for improvements - if you remove one apt update that is done each time, you can squeeze some more but its not recommended. And if you develop multi-core packages deployment, a bit more.
It depends. For the purpose you are describing more then enough.
Armbin provides pretty generic system - just do some more research and check support term in case you want more help / shortcuts https://github.com/armbian/build#support Adding a new board? Relatively simple: https://github.com/armbian/build/commit/23604e8a0dcdf81ec6c28ccd4b2a64b90816d8e7#diff-7ec8d78f3fb79b669285f5a5c42069e3 … killing BSP complexity is ofc your cost. The same as with Yocto or buildroot.
RockpiS is fully supported for several months. You don’t need to waste your own time for that, so you just (read the docs) and make a custom image.
Ofc its possible.
@igorp thank you.
I will try to find out how to create an extremely minimal image ( only Network + APT + Docker ) for RockPi-S via Armbian. I will give a next try with CLEAN parameter, and share my build duration results, and try to document it.
@nish I am trying to follow your readme ( building via yocto ), and on my ancient build machine ( FX6300 + 12GB DRAM + multiple 250GByte Samsung Evo SSDs ), it is still building after 2 hours.
I really can’t understand why modern build systems are so heavyweight 
But I will keep trying, lets see what will happen.
Many thanks.
You can build the console image. That has way more features like SSH, networkmanager, htop, dialog etc. You can get get APT but you will need to compile other packages using yocto and host them. Better to just add packages you want using IMAGE_INSTALL_append in the local.conf. We will be adding more instructions in the readme which will help new users to add packages and configure kernel according to their need in the coming days.
That’s because everything is fetched and compiled from scratch(including your toolchain) so hence the pain. The first build takes time but then subsequent builds are just incremental and don’t take much time.
I agree it is time consuming but I really like it from a deployment perspective.
Edit: apt is not there, but apt-get is there. Unfortunately, I can’t install any package because perl version in the yocto layer is higher than the one used in apt, and trying to find a way to fix this
Ok, first of all, for anyone interested, I will provide some amateur information about the system (HW/OS) below.
May be it helps somebody else who is experimenting on this nice board.
SDnand on the RockPi-S has following approximate sequential write speed:
root@rockpi-s-rk3308:~# dd if=/dev/zero of=/dev/mmcblk1 bs=16M status=progress
1006632960 bytes (1.0 GB, 960 MiB) copied, 92 s, 10.9 MB/s
dd: error writing '/dev/mmcblk1': No space left on device
61+0 records in
60+0 records out
1010827264 bytes (1.0 GB, 964 MiB) copied, 99.1463 s, 10.2 MB/s
root@rockpi-s-rk3308:~#
@nish thank you, I finally completed building console image:
stulluk /F250/ROCKPI-S/yocto/poky/build (zeus) $ `bitbake -k radxa-console-image`
WARNING: Host distribution "linuxmint-19.3" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Loading cache: 100% |##################################################################################################| Time: 0:00:00
Loaded 3580 entries from dependency cache.
Parsing recipes: 100% |################################################################################################| Time: 0:00:00
Parsing of 2453 .bb files complete (2452 cached, 1 parsed). 3581 targets, 115 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION = "1.44.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal"
TARGET_SYS = "aarch64-poky-linux"
MACHINE = "rockpi-s-rk3308"
DISTRO = "poky"
DISTRO_VERSION = "3.0.4"
TUNE_FEATURES = "aarch64 cortexa35 crc crypto"
TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp = "zeus:d88d62c20d7d8da85f02edb170dae0280624ad7e"
meta-oe
meta-networking
meta-python
meta-multimedia
meta-gnome
meta-xfce = "zeus:2b5dd1eb81cd08bc065bc76125f2856e9383e98b"
meta-radxa = "zeus:d489b2909b45112434d843fba1aa18f8229949f2"
Initialising tasks: 100% |#############################################################################################| Time: 0:00:03
Sstate summary: Wanted 410 Found 0 Missed 410 Current 966 (0% match, 70% complete)
NOTE: Executing Tasks
NOTE: Setscene tasks completed
NOTE: Tasks Summary: Attempted 3884 tasks of which 3117 didn't need to be rerun and all succeeded.
Summary: There was 1 WARNING message shown.
stulluk /F250/ROCKPI-S/yocto/poky/build (zeus) $
The image created is ( radxa-console-image-rockpi-s-rk3308-gpt.img ) around 400Mbyte:
stulluk /F250/ROCKPI-S/yocto/poky/build (zeus) $ ls -lah tmp/deploy/images/rockpi-s-rk3308/
total 776M
drwxr-xr-x 4 stulluk stulluk 4,0K Sep 20 15:40 .
drwxrwxr-x 3 stulluk stulluk 4,0K Sep 20 14:22 ..
-rw-r--r-- 2 stulluk stulluk 2,2K Sep 20 15:36 boot.cmd
-rw-r--r-- 2 stulluk stulluk 2,3K Sep 20 15:36 boot.scr
-rw-r--r-- 1 stulluk stulluk 127K Sep 20 15:40 idbloader.img
lrwxrwxrwx 2 stulluk stulluk 52 Sep 20 14:18 Image -> Image--4.4.143-r1-rockpi-s-rk3308-20200920092156.bin
-rw-r--r-- 2 stulluk stulluk 12M Sep 20 14:18 Image--4.4.143-r1-rockpi-s-rk3308-20200920092156.bin
lrwxrwxrwx 2 stulluk stulluk 52 Sep 20 14:18 Image-rockpi-s-rk3308.bin -> Image--4.4.143-r1-rockpi-s-rk3308-20200920092156.bin
-rwxr-xr-x 1 stulluk stulluk 190K Sep 20 15:35 mkimage
-rw-rw-r-- 2 stulluk stulluk 178M Sep 20 14:21 modules--4.4.143-r1-rockpi-s-rk3308-20200920092156.tgz
lrwxrwxrwx 2 stulluk stulluk 54 Sep 20 14:21 modules-rockpi-s-rk3308.tgz -> modules--4.4.143-r1-rockpi-s-rk3308-20200920092156.tgz
drwxr-xr-x 2 stulluk stulluk 4,0K Sep 20 14:21 overlays
drwxr-xr-x 2 stulluk stulluk 4,0K Sep 20 15:36 radxa-binary
-rw-r--r-- 2 stulluk stulluk 267M Sep 20 15:40 radxa-console-image-rockpi-s-rk3308-20200920123221.rootfs.ext4
-rw-r--r-- 2 stulluk stulluk 59K Sep 20 15:39 radxa-console-image-rockpi-s-rk3308-20200920123221.rootfs.manifest
-rw-r--r-- 2 stulluk stulluk 266K Sep 20 15:39 radxa-console-image-rockpi-s-rk3308-20200920123221.testdata.json
-rw-r--r-- 1 stulluk stulluk 112M Sep 20 15:40 radxa-console-image-rockpi-s-rk3308-boot.img
lrwxrwxrwx 2 stulluk stulluk 62 Sep 20 15:40 radxa-console-image-rockpi-s-rk3308.ext4 -> radxa-console-image-rockpi-s-rk3308-20200920123221.rootfs.ext4
-rw-r--r-- 1 stulluk stulluk 396M Sep 20 15:40 radxa-console-image-rockpi-s-rk3308-gpt.img
lrwxrwxrwx 2 stulluk stulluk 66 Sep 20 15:39 radxa-console-image-rockpi-s-rk3308.manifest -> radxa-console-image-rockpi-s-rk3308-20200920123221.rootfs.manifest
lrwxrwxrwx 2 stulluk stulluk 64 Sep 20 15:39 radxa-console-image-rockpi-s-rk3308.testdata.json -> radxa-console-image-rockpi-s-rk3308-20200920123221.testdata.json
-rw-r--r-- 2 stulluk stulluk 63K Sep 20 14:21 rk3308-rock-pi-s--4.4.143-r1-rockpi-s-rk3308-20200920092156.dtb
lrwxrwxrwx 2 stulluk stulluk 63 Sep 20 14:21 rk3308-rock-pi-s.dtb -> rk3308-rock-pi-s--4.4.143-r1-rockpi-s-rk3308-20200920092156.dtb
lrwxrwxrwx 2 stulluk stulluk 63 Sep 20 14:21 rk3308-rock-pi-s-rockpi-s-rk3308.dtb -> rk3308-rock-pi-s--4.4.143-r1-rockpi-s-rk3308-20200920092156.dtb
-rw-r--r-- 1 stulluk stulluk 1,0M Sep 20 15:40 trust.img
-rw-r--r-- 1 stulluk stulluk 173 Sep 20 15:40 trust.ini
lrwxrwxrwx 2 stulluk stulluk 33 Sep 20 15:36 u-boot.bin -> u-boot-rockpi-s-rk3308-1.0-r0.bin
-rw-r--r-- 1 stulluk stulluk 1,0M Sep 20 15:40 u-boot.img
-rw-r--r-- 2 stulluk stulluk 618K Sep 20 15:36 u-boot-rockpi-s-rk3308-1.0-r0.bin
lrwxrwxrwx 2 stulluk stulluk 33 Sep 20 15:36 u-boot-rockpi-s-rk3308.bin -> u-boot-rockpi-s-rk3308-1.0-r0.bin
-rw-r--r-- 2 stulluk stulluk 149 Sep 20 15:36 uEnv.txt
stulluk /F250/ROCKPI-S/yocto/poky/build (zeus) $
I don’t know which of these files correspond to bootloader, like the file:
rk3308_loader_uart0_m0_emmc_port_support_sd_20190717.bin
Building took around 2+ hours. Since Yocto is quite incremental, I can modify something, and trigger :
bitbake -k radxa-console-image
I burned this file to a fast SDcard via etcher, and the RockPi-S boots (drop to shell prompt) in around 10 seconds.
https://pastebin.pl/view/ce63d072
Quite impressive.
And it resizes rootfs during the 1st bootup.
In order to login, username is “root” and password is “rock”.
After bootup, storage consumption is:
root@rockpi-s-rk3308:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 14G 185M 14G 2% /
devtmpfs 212M 0 212M 0% /dev
tmpfs 213M 0 213M 0% /dev/shm
tmpfs 213M 8.7M 204M 5% /run
tmpfs 213M 0 213M 0% /sys/fs/cgroup
tmpfs 213M 0 213M 0% /tmp
tmpfs 213M 16K 213M 1% /var/volatile
tmpfs 43M 0 43M 0% /run/user/0
root@rockpi-s-rk3308:~#
And ssh is OK:
root@rockpi-s-rk3308:~# systemctl | grep ssh
sshd@1-192.168.1.111:22-192.168.1.247:35218.service loaded active running OpenSSH Per-Conne
sshdgenkeys.service loaded active exited OpenSSH Key Gener
system-sshd.slice loaded active active system-sshd.slice
sshd.socket loaded active listening sshd.socket
root@rockpi-s-rk3308:~#
And NetworkManager is also OK:
root@rockpi-s-rk3308:~# systemctl | grep NetworkManager
NetworkManager.service loaded active running Network Manager
root@rockpi-s-rk3308:~#
Unfortunately, there is no APT (yet):
root@rockpi-s-rk3308:~# apt
-sh: apt: command not found
root@rockpi-s-rk3308:~#
Then, I decided to test to burn this image to SDnand:
stulluk /F250/ROCKPI-S/yocto/poky/build (zeus) $ sudo rkdeveloptool db ../../../IMAGES/rk3308_loader_uart0_m0_emmc_port_support_sd_20190717.bin
Downloading bootloader succeeded.
stulluk /F250/ROCKPI-S/yocto/poky/build (zeus) $ sudo rkdeveloptool wl 0 tmp/deploy/images/rockpi-s-rk3308/radxa-console-image-rockpi-s-rk3308-gpt.img
Write LBA from file (100%)
stulluk /F250/ROCKPI-S/yocto/poky/build (zeus) $
After bootup, storage consumption is same as SDcard:
root@rockpi-s-rk3308:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 802M 185M 578M 25% /
devtmpfs 212M 0 212M 0% /dev
tmpfs 213M 0 213M 0% /dev/shm
tmpfs 213M 8.7M 204M 5% /run
tmpfs 213M 0 213M 0% /sys/fs/cgroup
tmpfs 213M 0 213M 0% /tmp
tmpfs 213M 16K 213M 1% /var/volatile
tmpfs 43M 0 43M 0% /run/user/0
root@rockpi-s-rk3308:~#
@nish : Thank you.
You can get get APT but you will need to compile other packages using yocto and host them. Better to just add packages you want using IMAGE_INSTALL_append in the local.conf.
How to do that? To be more precise, I want to run docker-pihole ( https://www.smarthomebeginner.com/run-pihole-in-docker-on-ubuntu-with-reverse-proxy/ ) on RockPi-S.
I did this on my DS167 board easily ( Allwinner A20 + 1GBRAM + 8Gbyte eMMC).
If I had APT + DOCKER in this yocto image, I thought that I could do it on RockPi-S too.
What is your suggestion to achieve this (docker-pihole on RockPi-S) via IMAGE_INSTALL_append ?
It is the loader.bin file in build/tmp/deploy/images/rockpi-s-rk3308.conf/radxa-binary folder.
You can search for list of packages that you build here: https://layers.openembedded.org/layerindex/branch/zeus/recipes/
As you will see the docker-ce recipe is given in meta-virtualization layer (https://layers.openembedded.org/layerindex/recipe/112720/)
These would be the steps to add docker to the image:
- Fetch the layer using git in the poky folder where other layers like meta,meta-openembedded are present:
~/yocto/poky $ git clone git://git.yoctoproject.org/meta-virtualization -b zeus
- Add meta-virtualization layer to the build/conf/bblayers.conf along with the dependencies (https://layers.openembedded.org/layerindex/branch/zeus/layer/meta-virtualization/)
Your build/conf/bblayers.conf should contain these:
${TOPDIR}/../meta \
${TOPDIR}/../meta-poky \
${TOPDIR}/../meta-yocto-bsp \
${TOPDIR}/../meta-openembedded/meta-oe \
${TOPDIR}/../meta-openembedded/meta-networking \
${TOPDIR}/../meta-openembedded/meta-python \
${TOPDIR}/../meta-openembedded/meta-multimedia \
${TOPDIR}/../meta-openembedded/meta-gnome \
${TOPDIR}/../meta-openembedded/meta-xfce \
${TOPDIR}/../meta-radxa \
${TOPDIR}/../meta-openembedded/meta-filesystems \
${TOPDIR}/../meta-virtualization \
Note: meta-virtualization and meta-filesystem are added here. meta-filesystems was a dependency which wasn’t present before in the bblayers.conf
- Now to instruct the build system to add the docker-ce package in our image we need to add it using the IMAGE_INSTALL_append variable in the
build/conf/local.conffile:
IMAGE_INSTALL_append = " docker-ce"
You can add it to the end of the file as well.
- Setup the environment using
~/yocto/poky $ source oe-init-build-env
- Run the command to build the console image:
~/yocto/poky/build $ bitbake -k radxa-console-image
At the end of the build you should have docker-ce in your console-image. I hope you find this useful.
1 Like
@nish amazing explanation, thank you very much.
stulluk /F250/ROCKPI-S/yocto/poky (zeus) $ cat build/conf/bblayers.conf
# POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
POKY_BBLAYERS_CONF_VERSION = "2"
BBPATH = "${TOPDIR}"
BBFILES ?= ""
BBLAYERS ?= " \
${TOPDIR}/../meta \
${TOPDIR}/../meta-poky \
${TOPDIR}/../meta-yocto-bsp \
${TOPDIR}/../meta-openembedded/meta-oe \
${TOPDIR}/../meta-openembedded/meta-networking \
${TOPDIR}/../meta-openembedded/meta-python \
${TOPDIR}/../meta-openembedded/meta-multimedia \
${TOPDIR}/../meta-openembedded/meta-gnome \
${TOPDIR}/../meta-openembedded/meta-xfce \
${TOPDIR}/../meta-radxa \
${TOPDIR}/../meta-openembedded/meta-filesystems \
${TOPDIR}/../meta-virtualization \
"
stulluk /F250/ROCKPI-S/yocto/poky (zeus) $
And:
stulluk /F250/ROCKPI-S/yocto/poky (zeus) $ cat build/conf/local.conf
#
# This file is your local configuration file and is where all local user settings
..........
DISTRO_FEATURES_append = " pam systemd x11"
VIRTUAL-RUNTIME_init_manager = "systemd"
PACKAGECONFIG_append_pn-systemd = " resolved networkd"
DISTRO_FEATURES_remove = "wayland"
IMAGE_INSTALL_append = "docker-ce"
stulluk /F250/ROCKPI-S/yocto/poky (zeus) $
Unfortunately, here is the status:
stulluk /F250/ROCKPI-S/yocto/poky (zeus) $ vi build/conf/bblayers.conf
stulluk /F250/ROCKPI-S/yocto/poky (zeus) $ vi build/conf/local.conf
stulluk /F250/ROCKPI-S/yocto/poky (zeus) $ source oe-init-build-env
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-sato
meta-toolchain
meta-ide-support
You can also run generated qemu images with a command like 'runqemu qemux86'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks
stulluk /F250/ROCKPI-S/yocto/poky/build (zeus) $ bitbake -k radxa-console-image
WARNING: You have included the meta-virtualization layer, but 'virtualization' has not been enabled in your DISTRO_FEATURES. Some bbappend files may not take effect. See the meta-virtualization README for details on enabling virtualization support.
WARNING: Host distribution "linuxmint-19.3" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
WARNING: /F250/ROCKPI-S/yocto/poky/build/../meta-radxa/recipes-connectivity/bluetooth/bluetooth-rockpi-e.bb: Unable to get checksum for bluetooth-rockpi-e SRC_URI entry realtek-btfw-load.service: file could not be found | ETA: 0:0WARNING: /F250/ROCKPI-S/yocto/poky/build/../meta-radxa/recipes-connectivity/bluetooth/bluetooth-rockpi-e.bb: Unable to get checksum for bluetooth-rockpi-e SRC_URI entry realtek-btfw-load.sh: file could not be found
Parsing recipes: 100% |################################################################################################| Time: 0:01:15
Parsing of 2594 .bb files complete (0 cached, 2594 parsed). 3735 targets, 128 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION = "1.44.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal"
TARGET_SYS = "aarch64-poky-linux"
MACHINE = "rockpi-s-rk3308"
DISTRO = "poky"
DISTRO_VERSION = "3.0.4"
TUNE_FEATURES = "aarch64 cortexa35 crc crypto"
TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp = "zeus:d88d62c20d7d8da85f02edb170dae0280624ad7e"
meta-oe
meta-networking
meta-python
meta-multimedia
meta-gnome
meta-xfce = "zeus:2b5dd1eb81cd08bc065bc76125f2856e9383e98b"
meta-radxa = "zeus:d489b2909b45112434d843fba1aa18f8229949f2"
meta-filesystems = "zeus:2b5dd1eb81cd08bc065bc76125f2856e9383e98b"
meta-virtualization = "zeus:48ec1b49871fef7d331ac1d0680a48d362c7482b"
Initialising tasks: 100% |#############################################################################################| Time: 0:00:03
Sstate summary: Wanted 163 Found 0 Missed 163 Current 1369 (0% match, 89% complete)
NOTE: Executing Tasks
NOTE: Setscene tasks completed
WARNING: radxa-console-image-1.0-r0 do_rootfs: Postinstall for package docker-ce failed with 1:
update-rc.d: /F250/ROCKPI-S/yocto/poky/build/tmp/work/rockpi_s_rk3308-poky-linux/radxa-console-image/1.0-r0/rootfs/etc/init.d/docker.init: file does not exist
ERROR: radxa-console-image-1.0-r0 do_rootfs: Postinstall scriptlets of ['docker-ce'] have failed. If the intention is to defer them to first boot,
then please place them into pkg_postinst_ontarget_${PN} ().
Deferring to first boot via 'exit 1' is no longer supported.
Details of the failure are in /F250/ROCKPI-S/yocto/poky/build/tmp/work/rockpi_s_rk3308-poky-linux/radxa-console-image/1.0-r0/temp/log.do_rootfs.
ERROR: Logfile of failure stored in: /F250/ROCKPI-S/yocto/poky/build/tmp/work/rockpi_s_rk3308-poky-linux/radxa-console-image/1.0-r0/temp/log.do_rootfs.12748
ERROR: Task (/F250/ROCKPI-S/yocto/poky/build/../meta-radxa/recipes-core/images/radxa-console-image.bb:do_rootfs) failed with exit code '1'
NOTE: Tasks Summary: Attempted 4258 tasks of which 3869 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/F250/ROCKPI-S/yocto/poky/build/../meta-radxa/recipes-core/images/radxa-console-image.bb:do_rootfs
Summary: There were 5 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
stulluk /F250/ROCKPI-S/yocto/poky/build (zeus) $
Here is the content of the error log file:
https://pastebin.pl/view/0d2cf8e8
See the end of it:
NOTE: Executing preinstall for package: docker-ce ...
NOTE: Running groupadd commands...
NOTE: docker-ce: Performing groupadd with [--root /F250/ROCKPI-S/yocto/poky/build/tmp/work/rockpi_s_rk3308-poky-linux/radxa-console-image/1.0-r0/rootfs -r docker]
NOTE: docker-ce: group docker already exists, not re-creating it
NOTE: Executing postinstall for package: docker-ce ...
WARNING: Postinstall for package docker-ce failed with 1:
update-rc.d: /F250/ROCKPI-S/yocto/poky/build/tmp/work/rockpi_s_rk3308-poky-linux/radxa-console-image/1.0-r0/rootfs/etc/init.d/docker.init: file does not exist
ERROR: Postinstall scriptlets of ['docker-ce'] have failed. If the intention is to defer them to first boot,
then please place them into pkg_postinst_ontarget_${PN} ().
Deferring to first boot via 'exit 1' is no longer supported.
Details of the failure are in /F250/ROCKPI-S/yocto/poky/build/tmp/work/rockpi_s_rk3308-poky-linux/radxa-console-image/1.0-r0/temp/log.do_rootfs.
DEBUG: Python function do_rootfs finished
stulluk /F250/ROCKPI-S/yocto/poky/build (zeus) $
Is it a bug, or did I miss any step ?
@igorp I made a small investigation on Armbian minimal Ubuntu image for RockPi-S.
Below log is just after burning this image to SDnand via rkdeveloptool, without installing anything:
stulluk /F250/ROCKPI-S/IMAGES $ ssh rockpi
Warning: Permanently added '192.168.1.111' (ECDSA) to the list of known hosts.
root@192.168.1.111's password:
____ _ _ ____
| _ \ ___ ___| | ___ __ (_) / ___|
| |_) / _ \ / __| |/ / '_ \| | \___ \
| _ < (_) | (__| <| |_) | | ___) |
|_| \_\___/ \___|_|\_\ .__/|_| |____/
|_|
Welcome to Armbian 20.08.1 Focal with Linux 4.4.228-rockpis
System load: 0.65 0.28 0.10 Up time: 1 min
Memory usage: 11 % of 487MB IP: 192.168.1.111
CPU temp: 43°C
Usage of /: 68% of 870M
Last login: Sun Sep 20 22:58:56 2020
root@rockpi-s:~# systemctl | grep -i network
networking.service loaded active exited Raise network interfaces
NetworkManager.service loaded active running Network Manager
systemd-resolved.service loaded active running Network Name Resolution
network-online.target loaded active active Network is Online
network-pre.target loaded active active Network (Pre)
network.target loaded active active Network
nss-lookup.target loaded active active Host and Network Name Lookups
root@rockpi-s:~# systemctl | grep -i ssh
ssh.service loaded active running OpenBSD Secure Shell server
root@rockpi-s:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 240M 0 240M 0% /dev
tmpfs 49M 1.9M 47M 4% /run
/dev/mmcblk1p1 870M 553M 265M 68% /
tmpfs 244M 0 244M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 244M 0 244M 0% /sys/fs/cgroup
tmpfs 244M 4.0K 244M 1% /tmp
/dev/zram0 49M 748K 45M 2% /var/log
tmpfs 49M 0 49M 0% /run/user/0
root@rockpi-s:~# apt update
Hit:1 http://ports.ubuntu.com focal InRelease
Get:2 http://ports.ubuntu.com focal-security InRelease [107 kB]
Get:4 http://ports.ubuntu.com focal-updates InRelease [111 kB]
Get:5 http://ports.ubuntu.com focal-backports InRelease [98.3 kB]
Get:6 http://ports.ubuntu.com focal-security/main armhf Packages [172 kB]
Get:7 http://ports.ubuntu.com focal-security/universe arm64 Packages [67.0 kB]
Get:8 http://ports.ubuntu.com focal-security/main arm64 Packages [181 kB]
Get:9 http://ports.ubuntu.com focal-security/universe armhf Packages [66.4 kB]
Get:10 http://ports.ubuntu.com focal-updates/main arm64 Packages [407 kB]
Get:11 http://ports.ubuntu.com focal-updates/universe arm64 Packages [204 kB]
Get:12 http://ports.ubuntu.com focal-updates/main armhf Packages [397 kB]
Get:13 http://ports.ubuntu.com focal-updates/universe armhf Packages [201 kB]
Get:3 https://us.mirrors.fossho.st/armbian/apt focal InRelease [18.2 kB]
Get:14 http://ports.ubuntu.com focal-backports/universe armhf Packages [3205 B]
Get:15 http://ports.ubuntu.com focal-backports/universe arm64 Packages [3196 B]
Get:16 https://us.mirrors.fossho.st/armbian/apt focal/focal-utils arm64 Packages [3994 B]
Get:17 https://us.mirrors.fossho.st/armbian/apt focal/main armhf Packages [161 kB]
Get:18 https://us.mirrors.fossho.st/armbian/apt focal/main arm64 Packages [179 kB]
Get:19 https://us.mirrors.fossho.st/armbian/apt focal/focal-utils armhf Packages [3998 B]
Fetched 2385 kB in 8s (306 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
25 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@rockpi-s:~# apt install docker.io
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
containerd iptables libip6tc2 libnetfilter-conntrack3 libnfnetlink0 libnftnl11 runc
Suggested packages:
aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils firewalld nftables
Recommended packages:
cgroupfs-mount | cgroup-lite git pigz ubuntu-fan apparmor
The following NEW packages will be installed:
containerd docker.io iptables libip6tc2 libnetfilter-conntrack3 libnfnetlink0 libnftnl11 runc
0 upgraded, 8 newly installed, 0 to remove and 25 not upgraded.
Need to get 48.4 MB of archives.
After this operation, 272 MB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.
root@rockpi-s:~#
So it is not possible to install docker, (Available 265Mbyte, only docker engine needs 272Mbyte) am I wrong?
If I am not mistaken, even if I build armbian for Rockpi-S by myself, it will not be possible to install docker, because it will not fit into total 870Mbyte of space.
Am I wrong ? Is there a way to reduce armbian rootfs size even more by customising build options? How much space could be gained ?
Many thanks for your answers.
You need to install Docker, find a way.
Correct. Perhaps remove things to get there?
There seems to a be a conflict in systemd and sysvinit. Could you try this out.
In the local.conf could you add sysvinit to DISTRO_FEATURES_remove. So your variable should look like:
DISTRO_FEATURES_remove = “wayland sysvinit”
Do this. Source and fire the build again.
If this doesn’t work, revert back to original and try this out: https://github.com/Xilinx/meta-virtualization/issues/4
You can create a file named docker-ce_%.bbappend in meta-radxa/recipes-connectivity with the following content:
do_install_append() {
install -d ${D}${sysconfdir}/init.d
install -m 0755 ${WORKDIR}/docker.init ${D}${sysconfdir}/init.d/docker.init
}
Ideally this should be addressed by the maintainer of the meta-virtualization layer
Confirmed that this has worked, and I have docker in my image. Thank you up to this point.
Unfortunately, I still couldn’t be able to :
docker pull pihole/pihole:latest
Original yocto image was setting up boot partition size around 128MByte, and remaining as rootfs. See file meta-radxa/classes/rockchip-radxa-gpt-img.bbclass :
# More info at http://rockchip.wikidot.com/partitions
LOADER1_SIZE = "8000"
RESERVED1_SIZE = "128"
RESERVED2_SIZE = "8192"
LOADER2_SIZE = "8192"
ATF_SIZE = "8192"
#BOOT_SIZE = "229376"
BOOT_SIZE = "32768"
As seen above, I reduced boot partition size to 16Mbyte, and it didn’t affect bootup, and I got following new size:
root@rockpi-s-rk3308:/# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 863M 402M 412M 50% /
devtmpfs 212M 0 212M 0% /dev
tmpfs 213M 0 213M 0% /dev/shm
tmpfs 213M 8.7M 204M 5% /run
tmpfs 213M 0 213M 0% /sys/fs/cgroup
tmpfs 213M 0 213M 0% /tmp
tmpfs 213M 24K 213M 1% /var/volatile
tmpfs 43M 0 43M 0% /run/user/0
root@rockpi-s-rk3308:/#
At this point, I am a little confused (see http://rockchip.wikidot.com/partitions) :
- 16MByte reserved
- 16Mbyte boot partition as I set above
- 864Mbyte in Rootfs
Then where is the remaining: 960MByte total ( see above message of my dd speed test) - 864 -16 -16 = 60Mbyte is missing somewhere.
I checked reserved block percentage :
stulluk /F250/ROCKPI-S/yocto/poky (zeus) $ tune2fs -l /F250/ROCKPI-S/yocto/poky/build/tmp/deploy/images/rockpi-s-rk3308/radxa-console-image-rockpi-s-rk3308.ext4 | grep "Reserved block count"
Reserved block count: 6836
stulluk /F250/ROCKPI-S/yocto/poky (zeus) $
So, 6836*4096/1024/1024 = 26Mbytes ( am I calculating correctly ? )…
Original pihole docker image on my host seems less than 300Mbyte:
stulluk /F250/ROCKPI-S/yocto/poky (zeus) $ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
stulluk/rockpis-builder latest 3467f6915865 2 days ago 502MB
stulluk/rockpis-builder <none> 5cf255b1a0f2 2 days ago 484MB
pihole/pihole latest 4642d275ab73 6 weeks ago 296MB
stulluk /F250/ROCKPI-S/yocto/poky (zeus) $
But unfortunately, again, docker-pihole doesn’t fit to remaining space on rockpi-S:
root@rockpi-s-rk3308:~# docker pull pihole/pihole:latest
latest: Pulling from pihole/pihole
c408adf5aab3: Pull complete
e1760851e187: Pull complete
0f00de0e80bf: Pull complete
a42b41aa464a: Extracting [==================================================>] 115.3MB/115.3MB
018c74957032: Download complete
0e261c69a668: Download complete
fdc2ebbe8553: Download complete
11921923a10e: Download complete
failed to register layer: Error processing tar file(exit status 1): write /var/www/html/admin/.git/objects/pack/pack-69011510806132c1126562871583280a974cf494.pack: no space left on device
root@rockpi-s-rk3308:~#
Here is latest directory sizes on yocto image on rockpi-S:
root@rockpi-s-rk3308:/# du -sh * | sort -nr 2>/dev/null
du: cannot access 'proc/761/task/761/fd/4': No such file or directory
du: cannot access 'proc/761/task/761/fdinfo/4': No such file or directory
du: cannot access 'proc/761/fd/4': No such file or directory
du: cannot access 'proc/761/fdinfo/4': No such file or directory
291M usr
62M lib
28K home
20M var
16K opt
12M boot
11M etc
8.7M run
5.7M bin
4.0K mnt
4.0K media
2.3M sbin
0 tmp
0 sys
0 proc
0 dev
root@rockpi-s-rk3308:/#
Most of the space is consumed by docker & kernel modules…etc
I also tried to remove some apps:
# MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "\
# kernel-modules \
# rtl-tools \
# rkwifibt-firmware-rtl8723ds-bt \
# bluetooth-rockpi-s \
# resize-helper \
# "
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "\
kernel-modules \
resize-helper \
"
It seems I have to give-up my dream 
1Gbyte SDnand will not be enough for my use-case.
Actually, I like the general idea of SDnand, since lifetime of such storage device could be much more durable than commodity eMMC’s, and it could be easier to replace.
May be in future, there would be 2Gbyte or 4Gbyte versions of RockPi-S, and they would definitely be much more useful for my use-cases.
Again, many thanks to @nish and @igorp for your valuable recommendations.
I hope this investigation could be more useful for somebody else.
Still thinking: Could there be a way to strip down docker-pihole image?
Stop looking for shortcuts and you will perhaps get somewhere.
We wasted weeks to come out with Armbian minimal - similar goal, but not that extreme since we still need to have regular functionality, certain tools, etc., Docker must be supported kernel wise but engine not present. You want to get more and better in much less time … also with less experiences.
- getting minimal possible kernel config that run docker (few days, perhaps shortcut exits but I don’t know it)
- optimising boot process to get some space - removing ramdisk (1h - few days)
- remove python and other non-essential packages (few days)
Then you will perhaps have the needed space.
There are few suggestions that I can give. First of a good attempt. Maybe suggestion 5 is the best one as it uses openwrt which is essentially made for router-os and for low contrained env.
Suggestion 1:
Use the minimal image and add only the necessary packages using IMAGE_INSTALL_append = “”. Also remove x11 from distro features which will reduce some pacakage sizes. Your local.conf would look something like this:
DISTRO_FEATURES_append = " pam systemd" VIRTUAL-RUNTIME_init_manager = “systemd” PACKAGECONFIG_append_pn-systemd = " resolved networkd" DISTRO_FEATURES_remove = “wayland x11 sysvinit” IMAGE_INSTALL_append = "nmtui docker-ce"
You can also remove rtl-tools, bluetooth-rockpi-s, rkwifibt-firmware which are required for bluetooth. Also maybe you can remove the packages used by resize-helper after first boot.
Suggestion 2:
The packages for the console image come from: (https://github.com/nishantpoorswani/meta-radxa/blob/zeus/recipes-core/packagegroups/packagegroup-radxa-console.bb) and (https://github.com/nishantpoorswani/meta-radxa/blob/zeus/recipes-core/images/radxa-console-image.bb)
You can remove the things you don’t require from here and build your image. Ideally method 1 is better.
Suggestion 3:
As the current system uses systemd, python3 becomes a dependency. Hence, it is difficult to further reduce. What you can try is that replace systemd by busybox. This will remove python3 as it won’t be a runtime dependency. You can see that the distro understands to use systemd from 2 places:
1st is given here: https://github.com/nishantpoorswani/meta-radxa/blob/zeus/recipes-core/images/radxa-minimal-image.bb. You will need to remove systemd from here.
2nd in the local.conf in the DISTRO_FEATURE_append. So remove it from there and place it in the DISTO_FEATURE_REMOVE. Also you would need to change the init manager. Your local.conf would look like this:
DISTRO_FEATURES_append = " pam"
DISTRO_FEATURES_remove = "wayland systemd sysvinit x11"
VIRTUAL-RUNTIME_login_manager = "busybox"
VIRTUAL-RUNTIME_init_manager = "busybox"
IMAGE_INSTALL_append = “docker-ce networkmanager”
Suggestion 4:
By default in the local.conf you can see that the distro selected is poky which is a default distro you yocto as give in the local.conf using the variable: DISTRO ?= “poky”. poky uses glibc and systemd and other things by default. Yocto also provides another distro if you want to reduce the size called poky-tiny. This will serverly reduce the size as it uses musl and busybox. python3 and unnecessary stuff is reduced. Your local.conf would look like this:
MACHINE ?= “rockpi-s-rk3308”
DISTRO ?= “poky-tiny”We default to ipk:
PACKAGE_CLASSES ?= “package_ipk”
EXTRA_IMAGE_FEATURES ?= “debug-tweaks”
USER_CLASSES ?= “buildstats image-mklibs image-prelink”
PATCHRESOLVE = “noop”BB_DISKMON_DIRS ??= "
STOPTASKS,${TMPDIR},1G,100K
STOPTASKS,${DL_DIR},1G,100K
STOPTASKS,${SSTATE_DIR},1G,100K
STOPTASKS,/tmp,100M,100K
ABORT,${TMPDIR},100M,1K
ABORT,${DL_DIR},100M,1K
ABORT,${SSTATE_DIR},100M,1K
ABORT,/tmp,10M,1K"CONF_VERSION = “1”
IMAGE_INSTALL_append = “docker-ce networkmanager”
I’m not sure that you can compiler docker or no using this but I found links which suggest you can: https://linuxfr.org/users/xberger/journaux/yocto-docker-continuons-l-experience
Also remove all unnecessary packages like the ones for bluetooth and all systemd one’s like resizehelper. You will have to maybe write your own script for resize.
Also you will need to remove systemd as stated above from the layer in the minimal-image. Also maybe you use the minimal-image given by yocto instead of radxa-minimal-image. You can use the command: bitbake -k core-image-minimal inplace of radxa-minimal-image
Ref link: https://wiki.yoctoproject.org/wiki/Poky-Tiny
Suggestion 5:
Well now that all options inside yocto are exhausted you can try openwrt. Here is the link to the post: OpenWrt for ROCK Pi S [testing]
Helpful link: 1) https://www.reddit.com/r/pihole/comments/apquar/dockerpihole_openwrt/
This might be easier than the yocto way.
I hope all this info helps
1 Like
Considered checking out OpenWrt with docker?
I notice that buildroot has docker packages. Have you considered trying buildroot with the docker packages ?
Buildroot provides these packages for installation :
docker