Kernel Panic with External WiFi Dongle

#1

I am using an Edimax AC-1200 WiFi USB dongle to get a second WiFi connection on my system. I’m using kernel 4.4.154-110-rockchip-gcef30e88a9f5 and have built/installed the 88x2bu driver for this chip and all works just fine.

Then I made a copy of my MMC card and that copy also runs just fine.

Then I plugged that MMC card into a second Rock Pi 4 board with the same WiFi dongle and I get kernel panics and hangs on that system (see below). If I boot my system without the dongle, all is fine. If I boot with it, I see a panic like this 35 seconds or so after boot and any use of networking seems to hang the CPU. If I type “ifconfig eth0” for example; hangs the CPU. If I try to connect through the onboard WiFi, things will hang.

Is there some flash/firmware on the boards that I need to update that could be causing this? That’s the only thing I can think of that’s different. I am using the same MMC and the same physical dongle on both boards. Only the board where I built/configured the system works.

Thanks!

Here’s by dump if helpful:

Sep 7 18:18:51 localhost kernel: [ 2884.506394] usb 2-1: new high-speed USB device number 2 using ehci-platform
Sep 7 18:18:51 localhost kernel: [ 2884.640539] usb 2-1: New USB device found, idVendor=7392, idProduct=b822
Sep 7 18:18:51 localhost kernel: [ 2884.640564] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Sep 7 18:18:51 localhost kernel: [ 2884.640579] usb 2-1: Product: AC1200 MU-MIMO USB2.0 Adapter
Sep 7 18:18:51 localhost kernel: [ 2884.640592] usb 2-1: Manufacturer: Realtek
Sep 7 18:18:51 localhost kernel: [ 2884.640606] usb 2-1: SerialNumber: 123456
…
Sep 7 18:19:12 localhost kernel: [ 2905.299179] BUG: spinlock bad magic on CPU#5, ip/13972
Sep 7 18:19:12 localhost kernel: [ 2905.299195] lock: 0xffffffc0e4aa7568, .magic: 00000000, .owner: /-1, .owner_cpu: 0
Sep 7 18:19:12 localhost kernel: [ 2905.299205] CPU: 5 PID: 13972 Comm: ip Tainted: G O 4.4.154-110-rockchip-gcef30e88a9f5 #1
Sep 7 18:19:12 localhost kernel: [ 2905.299211] Hardware name: ROCK PI 4B (DT)
Sep 7 18:19:12 localhost kernel: [ 2905.299216] Call trace:
Sep 7 18:19:12 localhost kernel: [ 2905.299235] [] dump_backtrace+0x0/0x220
Sep 7 18:19:12 localhost kernel: [ 2905.299243] [] show_stack+0x24/0x30
Sep 7 18:19:12 localhost kernel: [ 2905.299252] [] dump_stack+0x98/0xc0
Sep 7 18:19:12 localhost kernel: [ 2905.299260] [] spin_dump+0x84/0xa4
Sep 7 18:19:12 localhost kernel: [ 2905.299267] [] spin_bug+0x30/0x3c
Sep 7 18:19:12 localhost kernel: [ 2905.299273] [] do_raw_spin_lock+0x40/0x164
Sep 7 18:19:12 localhost kernel: [ 2905.299283] [] _raw_spin_lock_irqsave+0x28/0x38
Sep 7 18:19:12 localhost kernel: [ 2905.299612] [] wl_cfg80211_netdev_notifier_call+0x204/0x2a8 [bcmdhd]
Sep 7 18:19:12 localhost kernel: [ 2905.299622] [] notifier_call_chain+0x70/0x90
Sep 7 18:19:12 localhost kernel: [ 2905.299629] [] raw_notifier_call_chain+0x3c/0x4c
Sep 7 18:19:12 localhost kernel: [ 2905.299638] [] call_netdevice_notifiers_info+0x64/0x74
Sep 7 18:19:12 localhost kernel: [ 2905.299644] [] call_netdevice_notifiers+0x2c/0x38
Sep 7 18:19:12 localhost kernel: [ 2905.299650] [] __dev_close_many+0x9c/0xe0
Sep 7 18:19:12 localhost kernel: [ 2905.299657] [] __dev_close+0x2c/0x44
Sep 7 18:19:12 localhost kernel: [ 2905.299663] [] __dev_change_flags+0xa8/0x13c
Sep 7 18:19:12 localhost kernel: [ 2905.299668] [] dev_change_flags+0x34/0x70
Sep 7 18:19:12 localhost kernel: [ 2905.299676] [] do_setlink+0x400/0x844
Sep 7 18:19:12 localhost kernel: [ 2905.299683] [] rtnl_newlink+0x34c/0x648
Sep 7 18:19:12 localhost kernel: [ 2905.299690] [] rtnetlink_rcv_msg+0x1a0/0x1a8
Sep 7 18:19:12 localhost kernel: [ 2905.299699] [] netlink_rcv_skb+0x78/0xcc
Sep 7 18:19:12 localhost kernel: [ 2905.299706] [] rtnetlink_rcv+0x2c/0x3c
Sep 7 18:19:12 localhost kernel: [ 2905.299713] [] netlink_unicast+0x160/0x200
Sep 7 18:19:12 localhost kernel: [ 2905.299720] [] netlink_sendmsg+0x2ec/0x30c
Sep 7 18:19:12 localhost kernel: [ 2905.299728] [] sock_sendmsg+0x4c/0x68
Sep 7 18:19:12 localhost kernel: [ 2905.299735] [] ___sys_sendmsg+0x1dc/0x228
Sep 7 18:19:12 localhost kernel: [ 2905.299741] [] __sys_sendmsg+0x50/0x78
Sep 7 18:19:12 localhost kernel: [ 2905.299746] [] SyS_sendmsg+0x38/0x50
Sep 7 18:19:12 localhost kernel: [ 2905.299753] [] el0_svc_naked+0x24/0x28
Sep 7 18:19:12 localhost kernel: [ 2905.299765] Unable to handle kernel NULL pointer dereference at virtual address 00000000
Sep 7 18:19:12 localhost kernel: [ 2905.299770] pgd = ffffffc0df68e000
Sep 7 18:19:12 localhost kernel: [ 2905.299774] [00000000] *pgd=0000000000000000, *pud=0000000000000000
Sep 7 18:19:12 localhost kernel: [ 2905.299786] Internal error: Oops: 96000005 [#1] SMP
Sep 7 18:19:12 localhost kernel: [ 2905.299792] Modules linked in: 88x2bu(O) w1_therm w1_gpio wire bcmdhd autofs4
Sep 7 18:19:12 localhost kernel: [ 2905.299815] CPU: 5 PID: 13972 Comm: ip Tainted: G O 4.4.154-110-rockchip-gcef30e88a9f5 #1
Sep 7 18:19:12 localhost kernel: [ 2905.299820] Hardware name: ROCK PI 4B (DT)
Sep 7 18:19:12 localhost kernel: [ 2905.299824] task: ffffffc0ddc3c600 task.stack: ffffffc0debb0000
Sep 7 18:19:12 localhost kernel: [ 2905.300140] PC is at wl_cfg80211_netdev_notifier_call+0x218/0x2a8 [bcmdhd]
Sep 7 18:19:12 localhost kernel: [ 2905.300457] LR is at wl_cfg80211_netdev_notifier_call+0x204/0x2a8 [bcmdhd]
Sep 7 18:19:12 localhost kernel: [ 2905.300463] pc : [] lr : [] pstate: 800001c5
Sep 7 18:19:12 localhost kernel: [ 2905.300467] sp : ffffffc0debb3560
Sep 7 18:19:12 localhost kernel: [ 2905.300471] x29: ffffffc0debb3560 x28: 0000000000000000
Sep 7 18:19:12 localhost kernel: [ 2905.300480] x27: ffffffc0e5ed7b00 x26: ffffffc0debb3918
Sep 7 18:19:12 localhost kernel: [ 2905.300488] x25: ffffffc0debd5210 x24: ffffff800119f200
Sep 7 18:19:12 localhost kernel: [ 2905.300495] x23: ffffff80011f8978 x22: ffffffc0e4aa7568
Sep 7 18:19:12 localhost kernel: [ 2905.300502] x21: 0000000000000009 x20: ffffffc0df58c000
Sep 7 18:19:12 localhost kernel: [ 2905.300509] x19: ffffffc0e4aa6fc0 x18: 0000000000000005
Sep 7 18:19:12 localhost kernel: [ 2905.300516] x17: 0000007f7f9e05e8 x16: ffffff8008a3a338
Sep 7 18:19:12 localhost kernel: [ 2905.300523] x15: 0000000000000000 x14: 000000000000000a
Sep 7 18:19:12 localhost kernel: [ 2905.300530] x13: ffffff80893f48cb x12: 0000000000000030
Sep 7 18:19:12 localhost kernel: [ 2905.300537] x11: 0000000000003ff0 x10: 0000000000000005
Sep 7 18:19:12 localhost kernel: [ 2905.300544] x9 : ffffffc0debb3400 x8 : 00000040eed2d000
Sep 7 18:19:12 localhost kernel: [ 2905.300551] x7 : 0000000000003fff x6 : 00000000000011b0
Sep 7 18:19:12 localhost kernel: [ 2905.300558] x5 : 0000000000000000 x4 : ffffffffffffee50
Sep 7 18:19:12 localhost kernel: [ 2905.300565] x3 : 00000040eed2d000 x2 : ffffffffffffee50
Sep 7 18:19:12 localhost kernel: [ 2905.300572] x1 : ffffffc0e4aa7558 x0 : 0000000000000140
…
[ details of memory dump deleted ]
…
Sep 7 18:19:12 localhost kernel: [ 2905.304333] [] wl_cfg80211_netdev_notifier_call+0x218/0x2a8 [bcmdhd]
Sep 7 18:19:12 localhost kernel: [ 2905.304341] [] notifier_call_chain+0x70/0x90
Sep 7 18:19:12 localhost kernel: [ 2905.304348] [] raw_notifier_call_chain+0x3c/0x4c
Sep 7 18:19:12 localhost kernel: [ 2905.304356] [] call_netdevice_notifiers_info+0x64/0x74
Sep 7 18:19:12 localhost kernel: [ 2905.304362] [] call_netdevice_notifiers+0x2c/0x38
Sep 7 18:19:12 localhost kernel: [ 2905.304368] [] __dev_close_many+0x9c/0xe0
Sep 7 18:19:12 localhost kernel: [ 2905.304375] [] __dev_close+0x2c/0x44
Sep 7 18:19:12 localhost kernel: [ 2905.304380] [] __dev_change_flags+0xa8/0x13c
Sep 7 18:19:12 localhost kernel: [ 2905.304386] [] dev_change_flags+0x34/0x70
Sep 7 18:19:12 localhost kernel: [ 2905.304393] [] do_setlink+0x400/0x844
Sep 7 18:19:12 localhost kernel: [ 2905.304400] [] rtnl_newlink+0x34c/0x648
Sep 7 18:19:12 localhost kernel: [ 2905.304407] [] rtnetlink_rcv_msg+0x1a0/0x1a8
Sep 7 18:19:12 localhost kernel: [ 2905.304414] [] netlink_rcv_skb+0x78/0xcc
Sep 7 18:19:12 localhost kernel: [ 2905.304421] [] rtnetlink_rcv+0x2c/0x3c
Sep 7 18:19:12 localhost kernel: [ 2905.304427] [] netlink_unicast+0x160/0x200
Sep 7 18:19:12 localhost kernel: [ 2905.304434] [] netlink_sendmsg+0x2ec/0x30c
Sep 7 18:19:12 localhost kernel: [ 2905.304442] [] sock_sendmsg+0x4c/0x68
Sep 7 18:19:12 localhost kernel: [ 2905.304449] [] ___sys_sendmsg+0x1dc/0x228
Sep 7 18:19:12 localhost kernel: [ 2905.304454] [] __sys_sendmsg+0x50/0x78
Sep 7 18:19:12 localhost kernel: [ 2905.304460] [] SyS_sendmsg+0x38/0x50
Sep 7 18:19:12 localhost kernel: [ 2905.304466] [] el0_svc_naked+0x24/0x28
Sep 7 18:19:12 localhost kernel: [ 2905.304473] Code: 928235e4 91166261 d2823606 8b040042 (f948d843)
Sep 7 18:19:12 localhost kernel: [ 2905.309522] —[ end trace 998417cddfae61dd ]—

Bad Memory modules on RockPi 4?
#2

What if you plug your first sd card, not the copied one, to the 2nd ROCK Pi 4, will this happen?

#3

I’m using eMMC cards, but, yes. The problem stays with the boards. My original board works no matter which card I use. None of my three other boards work with any of them.

Also tried moving the WiFi dongles and only the original board works with them. None work on the other boards.

This is why I was thinking maybe there’s some firmware flashed in the boards that could be different?

#4

Currently there is SPI flash on the board if you remove the eMMC. You can verify by erase the SPI flash on all boards.

What if you re-build the wifi driver on other boards. The crash seems from the wifi driver itself.

#5

OK. I rebuilt the driver on the “bad” board and the problem is unchanged. Same kernel dump and then trying to do anything related to networking hangs the processor.

I’m not sure I want to simply erase the SPI. Isn’t the bootloader in there? I have multiple boards and none work but the one I used to build my initial image.system. This is so odd. Same hardware (unchanged from the factory except to set up my MMC image). Same MMC chip/wifi adapters that work on the one board fail to work on any others.

This is showstopper. I have four boards and only one works with the same adapter and MMC card. If there is no explanation for this, I’ll need to find other hardware fr my product. I can’t waste much more time on this unless there is some explanation so I can try to fix it.

I can send my log output in its entirety if that would help.

#6

In case you can’t waste your time, hire someone.

BTW. In Armbian (K5.8.y) this wireless adapter works fine OOB - Plug and play. Before you argue for more, check support terms: https://github.com/armbian/build#support

#7

I have a system that works.

But moving the same hardware, including the MMC card, to another copy of the Rock Pi 4 kernel panics and locks up. So there seems to be something different on those boards.

If there’s a software explanation for this, I’d love to know so I can fix it. But it seems that there are hardware differences between boards which is not something I can deal with when building a solution I need multiple copies of and needs to be reliable.

#8

Obviously not. You are using so called board support package based Linux kernel. And the only people that will deal with this are a few Radxa engineers.

My advise also contains cheapest and fastest way to verify your hunch.

#9

It just comes to me, what power adapter are you using?

#10

I’m using a 45W PD USB power supply which pulls 0.5A @ 12V. Same power supply I use on the board that works.

I also power off my HAT board I built. Also works on the one board but not the others. But most testing is with the USB power supply, now, as I don’t need my HAT for this test and it fails on the bare board with nothing else attached to the USB ports, even.