Armbian reboot hangs and is unresponsive

Rockpi5b:
Armbian 23.02.2 Jammy with Linux 5.10.110-rockchip-rk3588
The system boots from its “nvme” drive.

I have cron that executes every night. I have tried both: “reboot” and “shutdown -r now”. Some nights it re boots fine. Sometimes it makes the system unresponsive.

Anyone know how to debug this or know a fix?

Turn on verbose output and try to catch error on UART. Try same thing with fresh image on card/different drive.
If it’s unresponsive there may be some kernel panic left on console.
for hard reboot - push power button little longer

Turned on logging. I set cron to reboot every 3 min. After about 10 reboots it hung. Weirdly power button reset, no matter how long I hold it wont reboot. Pulled the power and it started again. The following is what looks odd in the logs.
Looks like its an erorr trying to read data from the ES8316 audio codec chip using the ASoC (Audio System on Chip) driver. Do you think its related?

What do you think?

/var/log/dmesg
[ 11.109125] kernel: es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
[ 11.112614] kernel: es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
[ 11.115903] kernel: es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
[ 11.119001] kernel: es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
[ 11.121915] kernel: es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
[ 11.124901] kernel: es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
[ 11.127670] kernel: es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
[ 11.130755] kernel: es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
[ 11.133423] kernel: es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
[ 11.136123] kernel: es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6

/var/kern.log
Mar 27 12:13:05 pg-592 kernel: [ 9.147656] zram2: detected capacity change from 0 to 4031315968
Mar 27 12:13:05 pg-592 kernel: [ 9.236142] EXT4-fs (zram2): mounted filesystem without journal. Opts: discard
Mar 27 12:13:05 pg-592 kernel: [ 9.298587] EXT4-fs (zram1): mounted filesystem without journal. Opts: discard
Mar 27 12:13:05 pg-592 kernel: [ 11.109125] es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
Mar 27 12:13:05 pg-592 kernel: [ 11.112614] es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
Mar 27 12:13:05 pg-592 kernel: [ 11.115903] es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
Mar 27 12:13:05 pg-592 kernel: [ 11.119001] es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
Mar 27 12:13:05 pg-592 kernel: [ 11.121915] es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
Mar 27 12:13:05 pg-592 kernel: [ 11.124901] es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
Mar 27 12:13:05 pg-592 kernel: [ 11.127670] es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
Mar 27 12:13:05 pg-592 kernel: [ 11.130755] es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6
Mar 27 12:13:05 pg-592 kernel: [ 11.133423] es8316 7-0011: ASoC: error at soc_component_read_no_lock on es8316.7-0011: -6

Its indeed an issue… I get these a lot everytime I opt by doing shutdown other than reboot and most of times it requires me do a power cut… nobody seems to talk about this much but I doubt I am the only one with this issue… a fix seems to be non-existent so far.

1 Like

I just attempted to disable it, as I don’t need audio. Rebooting a bunch of times to see if this fixes it.

#!/bin/bash

Navigate to the /boot directory

cd /boot

Check if the armbianEnv.txt file exists

if [ ! -f armbianEnv.txt ]; then
echo “Error: armbianEnv.txt file not found.”
exit 1
fi

Check if the disable-wm8960 parameter already exists

if grep -q “disable-wm8960” armbianEnv.txt; then
echo “ES8316 audio codec chip is already disabled.”
exit 0
fi

Add the disable-wm8960 parameter to the overlays line

sed -i ‘s/^overlays=.*/& disable-wm8960/’ armbianEnv.txt

echo “ES8316 audio codec chip disabled. Reboot the system for the changes to take effect.”

It happens on other images too not just on armbian.
It wasnt happening on default armbian 22.0 but on 23.0 is… I started to have the same issue on openfyde and its hard to figure why

Well that didn’t work. But I think this did. I recompiled the kernel editing es8316 in rk3588-rock-5b.dts. Give it a try if you don’t need the audio. place in /boot/dtb/rockchip. backup before you try it.

rk3588-rock-5b.dtb.zip (55.3 KB)

I am going to reboot a bunch of times over the night. error msg went away at least.

2 Likes

Thanks Nate but I am not even using armbian atm and off course I need audio but I will keep this in mind.

Thanks for sharing that.
You needed to recompile kernel or just replaced dtbo for device?
Should overlay do the same?

I recompiled the .dts to a .dtb . You just need to drop the.dtb in /boot/dtb/rockchip

I ran it overnight. rebooting every 5 min via cron. crashed again. So it is not the audio. I don’t really have any other logging that looks helpful so I’m going to try dietpi next or ubunto.

1 Like

Let me know how it goes. By the way you are just using it via nvme?

Both ubuntu and dietpi both hang on reboot too. I have also tried multiple nmve cards and a 5b 4GB and a 5B8GB board. All the same behavior. So at this point I’m kind a stumped. Maybe its a firmware issue?

Reboot/hang is pretty random. It often takes 1 time to fail and hang. Sometimes 10 + . Any idea would be appreciated. I was planning on using these on an iot edge project but it’s now a no go if I can’t get a reliable reboot.

Set up a cron (every 2 min or whatever) : */2 * * * * /usr/sbin/reboot
See if you get the same problem.

Yes just using a nvme only.

1 Like

Maybe panfrost is the cause? I saw panfrost exceptions when played with the hibernation mode.

These are all the error I could find in /var/log files:

armbian-hardware-monitor.log:

[ 5.778531] rockchip-dmc dmc: normal_rate = 1560000000
[ 5.784831] W : [File] : drivers/gpu/arm/bifrost/platform/rk/mali_kbase_config_rk.c; [Line] : 136; [Func] : kbase_platform_rk_init(); power-off-delay-ms not available.
[ 5.875279] mali fb000000.gpu: Error -22, no DT entry: mali-simple-power-model.dynamic-coefficient = 1*[0]
[ 5.882418] RKNPU fdab0000.npu: Adding to iommu group 0
[ 5.883047] mali fb000000.gpu: Error -22, no DT entry: mali-simple-power-model.ts = 4*[0]
[ 5.884881] RKNPU fdab0000.npu: RKNPU: rknpu iommu is enabled, using iommu mode
[ 5.886560] mali fb000000.gpu: Error -22, no DT entry: mali-simple-power-model.thermal-zone = ‘’
[ 6.043374] cfg80211: Loaded X.509 cert ‘sforshee: 00b28ddf47aef9cea7’
[ 6.048745] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[ 6.053124] cfg80211: failed to load regulatory.db
[ 6.057273] I : [File] : drivers/gpu/arm/mali400/mali/linux/mali_kernel_linux.c; [Line] : 409; [Func] : mali_module_init(); svn_rev_string_from_arm of this mali_ko is ‘’, rk_ko_ver is ‘5’, built at ‘23:42:35’, on ‘Feb 17 2023’.
[ 6.063858] Mali:
[ 5.548624] SMCCC: SOC_ID: ARCH_FEATURES(ARCH_SOC_ID) returned error: fffffffffffffffd
5.576275] optee: probe of firmware:optee failed with error -22
5.806086] mali fb000000.gpu: Error -22, no DT entry: mali-simple-power-model.static-coefficient = 1*[0]
[ 5.806441] mali fb000000.gpu: Error -22, no DT entry: mali-simple-power-model.dynamic-coefficient = 1*[0]
[ 5.819468] mali fb000000.gpu: Error -22, no DT entry: mali-simple-power-model.ts = 4*[0]
[ 5.821301] mali fb000000.gpu: Error -22, no DT entry: mali-simple-power-model.thermal-zone = ‘’

dmesg:

5.407494] kernel: SMCCC: SOC_ID: ARCH_FEATURES(ARCH_SOC_ID) returned error: fffffffffffffffd
[ 5.433900] kernel: optee: probe of firmware:optee failed with error -22
[ 5.834056] kernel: mali fb000000.gpu: Error -22, no DT entry: mali-simple-power-model.static-coefficient = 1*[0]
[ 5.838581] kernel: rockchip-dmc dmc: performance_rate = 2112000000
[ 5.840881] kernel: mali fb000000.gpu: Error -22, no DT entry: mali-simple-power-model.dynamic-coefficient = 1*[0]
[ 5.845215] kernel: rockchip-dmc dmc: hdmirx_rate = 2112000000
[ 5.847537] kernel: mali fb000000.gpu: Error -22, no DT entry: mali-simple-power-model.ts = 4*[0]
[ 5.852035] kernel: rockchip-dmc dmc: failed to get vop bandwidth to dmc rate
[ 5.854570] kernel: mali fb000000.gpu: Error -22, no DT entry: mali-simple-power-model.thermal-zone = ‘’

Do you have mali-g610-firmware installed?

I’m not sure. How can I verify its installed?

Use apt to try to install it

Also, your error seems to be related to SMCCC: https://developer.arm.com/Architectures/Secure%20Monitor%20Calling%20Convention
Are you trying to run some hypervisors on Rock5?