On a brand new (had it for a few weeks) Zero 3W, get the following error " BZ2_bzread: /var/lib/apt/lists/partial/deb.debian.org_debian_dists_bullseye_main_i18n_Translation-en.bz2 Read error (-4: DATA_ERROR)" This I would suspect is a microSD card error at its root cause. But running fsck forced, repair mode… it passed. This happens during a common ‘apt update’ scenario. We get what appears to be a panic crash as well, randomly (in terms of time after reboot). Or various segmentation faults. I plan to replace the microSD card, but thought I would ask here if there should be other things we should look for, any hardware faults we could check for?
What is read error (-4: DATA_ERROR)?
Well, it appears my Zero 3W has failed. Not good, it was only 4 weeks old. I re-imaged a new SD card with the B6 debian XFCE image, same as I did weeks ago. And nothing is successful, it boots, fsck passes, but when I rsetup, segmentation faults. Even ‘sudo apt update’ segmentation faults. I hope the 3E I have does not end up in the same state.
I have also tried a couple of different power supplies, even though the power supplies via meter, show 5.1v and 3a.
Here is the kernel panic traces information…
jibun@radxa-zero3:~$ sudo apt update
[sudo] password for jibun:
Get:1 http://security.debian.org bookworm-security InRelease [48.0 kB]
Get:2 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:3 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
Get:4 http://deb.debian.org/debian bookworm-backports InRelease [56.6 kB]
Get:5 http://armbian.chi.auroradev.org/apt bookworm InRelease [53.3 kB]
Get:6 http://security.debian.org bookworm-security/main arm64 Packages [176 kB]
Get:7 http://deb.debian.org/debian bookworm/main arm64 Packages [11.9 MB]
Get:8 http://deb.debian.org/debian bookworm-backports/contrib arm64 Packages.diff/Index [37.5 kB]
Get:9 http://deb.debian.org/debian bookworm-backports/main arm64 Packages.diff/Index [63.3 kB]
Get:10 http://armbian.chi.auroradev.org/apt bookworm/bookworm-utils arm64 Packages [16.6 kB]
Get:11 http://deb.debian.org/debian bookworm-backports/contrib arm64 Packages T-2024-08-27-1426.43-F-2024-08-27-1426.43.pdiff [198 B]
Get:11 http://deb.debian.org/debian bookworm-backports/contrib arm64 Packages T-2024-08-27-1426.43-F-2024-08-27-1426.43.pdiff [198 B]
95% [7 Packages store 0 B] [Waiting for headers] [Waiting for headers][ 375.378822] Internal error: Oops: 0000000096000004 [#1] SMP
[ 375.379373] Modules linked in: r8153_ecm r8152 zstd zram zsmalloc binfmt_misc aic8800_fdrv aic8800_bsp nvmem_rockchip_otp rk805_pwrkey panfrost drm_shmem_helper gpu_sched fuse dm_mod ip_tables ipv6
[ 375.380986] CPU: 2 PID: 1320 Comm: store Tainted: G W 6.1.75-vendor-rk35xx #1
[ 375.381742] Hardware name: Radxa ZERO 3 (DT)
[ 375.382131] pstate: a0400009 (NzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 375.382750] pc : kmem_cache_alloc+0x158/0x1b8
[ 375.383171] lr : kmem_cache_alloc+0x80/0x1b8
[ 375.383558] sp : ffff80000c9bb8c0
[ 375.383863] x29: ffff80000c9bb8c0 x28: fffffc00006187c0 x27: 000000000000739b
[ 375.384509] x26: ffffffffffffffff x25: 0000000000000068 x24: ffff8000082c9afc
[ 375.385152] x23: ffff8000082c9afc x22: ffff000000c6c200 x21: ffff800009a06fe0
[ 375.385795] x20: 0000000000408d40 x19: ffff000000c6c200 x18: 0000000000000000
[ 375.386438] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[ 375.387081] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
[ 375.387723] x11: 0000000000000000 x10: 0000000000000000 x9 : ffff80000826ecd8
[ 375.388365] x8 : ffff00000d38b478 x7 : ffff80000c9bb9a0 x6 : 0000000000000402
[ 375.389007] x5 : ffff0000ef7727b0 x4 : ffff0000ef77dfe0 x3 : 00000000000029a0
[ 375.389649] x2 : ffff0000ef773180 x1 : 0000000000000030 x0 : ffdd000018074138
[ 375.390293] Call trace:
[ 375.390521] kmem_cache_alloc+0x158/0x1b8
[ 375.390889] alloc_buffer_head+0x28/0x74
[ 375.391250] alloc_page_buffers+0x84/0xf0
[ 375.391617] create_empty_buffers+0x2c/0x120
[ 375.392007] create_page_buffers+0x60/0x8c
[ 375.392384] __block_write_begin_int+0x8c/0x504
[ 375.392796] __block_write_begin+0x40/0x54
[ 375.393173] ext4_da_write_begin+0x144/0x1fc
[ 375.393569] generic_perform_write+0xb4/0x1bc
[ 375.393972] ext4_buffered_write_iter+0x88/0x10c
[ 375.394398] ext4_file_write_iter+0x118/0x4b0
[ 375.394799] vfs_write+0x154/0x1b8
[ 375.395112] ksys_write+0x78/0xe4
[ 375.395420] __arm64_sys_write+0x20/0x2c
[ 375.395775] invoke_syscall+0x8c/0x128
[ 375.396124] el0_svc_common.constprop.0+0xd8/0x128
[ 375.396558] do_el0_svc+0xac/0xbc
[ 375.396866] el0_svc+0x2c/0x54
[ 375.397159] el0t_64_sync_handler+0xac/0x13c
[ 375.397547] el0t_64_sync+0x19c/0x1a0
[Redacted register and memory dump]
[ 375.787728] ---[ end trace 0000000000000000 ]---
And…
[ 408.231025] Unable to handle kernel paging request at virtual address ffdd000018074168
[ 408.231889] Mem abort info:
[ 408.232185] ESR = 0x0000000096000004
[ 408.232570] EC = 0x25: DABT (current EL), IL = 32 bits
[ 408.233098] SET = 0, FnV = 0
[ 408.233684] EA = 0, S1PTW = 0
[ 408.234030] FSC = 0x04: level 0 translation fault
[ 408.234521] Data abort info:
[ 408.234825] ISV = 0, ISS = 0x00000004
[ 408.235209] CM = 0, WnR = 0
[ 408.235514] [ffdd000018074168] address between user and kernel address ranges
[ 408.236209] Internal error: Oops: 0000000096000004 [#2] SMP
[ 408.236770] Modules linked in: r8153_ecm r8152 zstd zram zsmalloc binfmt_misc aic8800_fdrv aic8800_bsp nvmem_rockchip_otp rk805_pwrkey panfrost drm_shmem_helper gpu_sched fuse dm_mod ip_tables ipv6
[ 408.238606] CPU: 2 PID: 48 Comm: kworker/u8:3 Tainted: G D W 6.1.75-vendor-rk35xx #1
[ 408.239476] Hardware name: Radxa ZERO 3 (DT)
[ 408.239914] Workqueue: writeback wb_workfn (flush-179:0)
[ 408.240505] pstate: a0400009 (NzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 408.241184] pc : kmem_cache_alloc+0x158/0x1b8
[ 408.241644] lr : kmem_cache_alloc+0x80/0x1b8
[ 408.242085] sp : ffff80000a9f35c0
[ 408.242427] x29: ffff80000a9f35c0 x28: ffff0000075a8d80 x27: 000000000000739b
[ 408.243146] x26: ffff80000a3e0000 x25: 0000000000000068 x24: ffff800008346f48
[ 408.243863] x23: ffff800008346f48 x22: ffff000000c6c200 x21: ffff800009a06fe0
[ 408.244579] x20: 0000000000000d40 x19: ffff000000c6c200 x18: 0000000000100010
[ 408.245294] x17: 000000040044ffff x16: 005000f2b5503510 x15: 0000000000000000
[ 408.246010] x14: ffff000000d78000 x13: ffffffffffffffff x12: 0000000000000040
[ 408.246724] x11: 0000000000000228 x10: ffff000004afcb68 x9 : ffff80000826ec90
[ 408.247441] x8 : ffff000003f1b088 x7 : 0000000000000000 x6 : 000000000000003f
[ 408.248155] x5 : 0000000000001000 x4 : ffff0000ef77dfe0 x3 : 0000000000000001
[ 408.248870] x2 : ffff80000a9f3630 x1 : 0000000000000030 x0 : ffdd000018074138
[ 408.249586] Call trace:
[ 408.249856] kmem_cache_alloc+0x158/0x1b8
[ 408.250280] ext4_mb_new_blocks+0x440/0xbd0
[ 408.250720] ext4_ext_map_blocks+0x10e8/0x11d0
[ 408.251184] ext4_map_blocks+0x2b0/0x4cc
[ 408.251595] ext4_writepages+0x4fc/0xafc
[ 408.252006] do_writepages+0xc0/0x130
[ 408.252402] __writeback_single_inode+0x90/0x2c8
[ 408.252879] writeback_sb_inodes+0x23c/0x420
[ 408.253324] __writeback_inodes_wb+0xa0/0xec
[ 408.253767] wb_writeback+0x240/0x264
[ 408.254149] wb_workfn+0x2d8/0x3bc
[ 408.254506] process_one_work+0x1c0/0x274
[ 408.254934] worker_thread+0x1dc/0x274
[ 408.255330] kthread+0xd4/0xe4
[ 408.255662] ret_from_fork+0x10/0x20
[Redacted register and memory dump]
[ 408.684797] ---[ end trace 0000000000000000 ]---
[ 408.715653] ------------[ cut here ]------------
[ 408.716130] WARNING: CPU: 2 PID: 48 at kernel/exit.c:814 do_exit+0x94/0x88c
[ 408.716801] Modules linked in: r8153_ecm r8152 zstd zram zsmalloc binfmt_misc aic8800_fdrv aic8800_bsp nvmem_rockchip_ot
I have tried 3 different OS images, and several different microSD cards, the result is always the same, I try to update the OS, via apt update… and kernel panic results.
What is frustrating, beyond this, is that this happened 16 days after I received the SBC, so the vendor I ordered the board from is not being helpful. They told me to talk to RADXA directly, so i opened a support request, but likely it is going to cost more to RMA the board back to RADXA then the purchase price.
Not fun. Really wanted these boards to be solid, like their feature set. I still have a 3E, now I wonder how long it is going to work.
Tried again to use B6 (XFCE Debian Image) download per RADXA suggestion. Most the time the board will not load the OS, bootloader works but actual OS failed. When the OS load happens to work… maybe 1 out 6 or 7 tries… I cannot run any command successfully. I get segmentation fault, or kernel panic, or WIFI fails to load. And NOW… I cannot even issue a reboot…
radxa@radxa-zero3:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
radxa@radxa-zero3:~$ sudo reboot
[ 1013.970720] systemd-journald[258]: Failed to send WATCHDOG=1 notification message: Transport endpoint is not connected
Failed to open initctl fifo: No such device or address
Failed to talk to init daemon.
The more I try to use this board, THE WORSE IT GETS. What will it take to get someone at RADXA to realize that this board has failed, after just hours of total use. This is NOT just a case of a bad image or the image does not work. The board is systematically getting worse, now WIFI refuses to work, now I cannot even issue a reboot from the console.
Vendor requested proof of crash… This is with a new microSD card, validated, as the Debian 11 XFCE B6 image… This SAME image works fine on my existing, still working, 3E. But the 3W, completely useless at this point, every boot, the same failure, sometimes it takes a few seconds, sometimes a few minutes. I just let it sit there at the console login prompt, and it will fail.
[ OK ] Listening on Load/Save RF …itch Status /dev/rfkill Watch.
Mounting /boot/efi...
Starting Save/Restore Sound Card State...
Starting CUPS Scheduler...
Starting dnsmasq - A light…DHCP and caching DNS server...
Starting Authorization Manager...
[ OK ] Started Reset AIC8800 Bluetooth on ttyS1.
Starting hciattach on ttyS1...
Starting Hostname Service...
Starting Permit User Sessions...
[ OK ] Mounted /boot/efi.
[ OK ] Started CUPS Scheduler.
[ OK ] Finished Save/Restore Sound Card State.
[ OK ] Finished Permit User Sessions.
[ OK ] Started Authorization Manager.
[ OK ] Started Accounts Service.
[ OK ] Reached target Sound Card.
Mounting /config...
Starting Light Display Manager...
Starting Hold until boot process finishes up...
Starting Load/Save RF Kill Switch Status...
[ OK ] Started hciattach on ttyS1.
[ OK ] Mounted /config.
[ OK ] Started Load/Save RF Kill Switch Status.
Starting Bluetooth service...
Debian GNU/Linux 11 radxa-zero3 ttyFIQ0
radxa-zero3 login: [ 24.057718] Bluetooth: hci0: command 0x1003 tx timeout
[ 26.117511] Unable to handle kernel paging request at virtual address ffdd000006111c58
[ 26.118281] Mem abort info:
[ 26.118551] ESR = 0x96000004
[ 26.118836] EC = 0x25: DABT (current EL), IL = 32 bits
[ 26.119313] SET = 0, FnV = 0
[ 26.119595] EA = 0, S1PTW = 0
[ 26.119875] Data abort info:
[ 26.120133] ISV = 0, ISS = 0x00000004
[ 26.120472] CM = 0, WnR = 0
[ 26.120742] [ffdd000006111c58] address between user and kernel address ranges
[ 26.121463] Internal error: Oops: 96000004 [#1] SMP
[ 26.121913] Modules linked in: bnep hci_uart btqca btrtl btbcm btintel bluetooth ecdh_generic ecc vfat fat snd_soc_simple_card snd_soc_simple_card_utils aic8800_bsp(O) snd_soc_hdmi_codec snd_soc_rockchip_i2s_tdm snd_soc_core snd_pcm_dmaengine snd_pcm snd_timer zstd zstd_compress snd zram soundcore bifrost_kbase ledtrig_timer ledtrig_pattern ledtrig_heartbeat ledtrig_default_on fuse ip_tables x_tables ipv6 sdhci_of_dwcmshc dw_hdmi_i2s_audio dw_hdmi_cec rtc_rk808
[ 26.125584] CPU: 1 PID: 693 Comm: amixer Tainted: G O 5.10.160-38-rk356x #60b0d0cca
[ 26.126376] Hardware name: Radxa ZERO 3 (DT)
[ 26.126766] pstate: 80400009 (Nzcv daif +PAN -UAO -TCO BTYPE=--)
[ 26.127318] pc : unmap_page_range+0x60/0x4bc
[ 26.127711] lr : unmap_single_vma+0xe4/0xe8
[ 26.128086] sp : ffff80000c533ac0
[ 26.128392] x29: ffff80000c533ac0 x28: ffff00000c9a3d00
[ 26.128875] x27: 0000000000000000 x26: 0000000000000000
[ 26.129358] x25: 0000000000000000 x24: ffff000018074900
[ 26.129839] x23: 0000000000000000 x22: ffff80000c533c98
[ 26.130319] x21: 0000ffff8e114000 x20: ffff80000c533c98
[ 26.130799] x19: 0000ffff8e113000 x18: 0000000000000000
[ 26.131279] x17: 0000000000000000 x16: 0000000000000000
[ 26.131760] x15: 0000000000000000 x14: 0000000000000000
[ 26.132241] x13: 0000000000000000 x12: 0000000000000000
[ 26.132722] x11: 0000000000000000 x10: 0000000000000000