Unexpected rebooting of Ubuntu Server (ROCK Pi 4B + NVMe booting + POE Hat)

My ROCK Pi’s periodicaly unexpectedly halts and reboots. CPU temperature is normal (40-50 °C), cores are not overloaded.

i found these logs before reboot:

sudo journalctl _TRANSPORT=kernel

Apr 28 09:13:38 rockpi4b-node4 kernel: dwmmc_rockchip fe320000.dwmmc: GPIO lookup for consumer cd
Apr 28 09:13:38 rockpi4b-node4 kernel: dwmmc_rockchip fe320000.dwmmc: using device tree for GPIO lookup
Apr 28 09:13:38 rockpi4b-node4 kernel: of_get_named_gpiod_flags: can't parse 'cd-gpios' property of node '/dwmmc@fe320000[0]'
Apr 28 09:13:38 rockpi4b-node4 kernel: of_get_named_gpiod_flags: can't parse 'cd-gpio' property of node '/dwmmc@fe320000[0]'
Apr 28 09:13:38 rockpi4b-node4 kernel: dwmmc_rockchip fe320000.dwmmc: using lookup tables for GPIO lookup
Apr 28 09:13:38 rockpi4b-node4 kernel: dwmmc_rockchip fe320000.dwmmc: lookup for GPIO cd failed
Apr 28 09:13:38 rockpi4b-node4 kernel: dwmmc_rockchip fe320000.dwmmc: GPIO lookup for consumer wp
Apr 28 09:13:38 rockpi4b-node4 kernel: dwmmc_rockchip fe320000.dwmmc: using device tree for GPIO lookup
Apr 28 09:13:38 rockpi4b-node4 kernel: of_get_named_gpiod_flags: can't parse 'wp-gpios' property of node '/dwmmc@fe320000[0]'
Apr 28 09:13:38 rockpi4b-node4 kernel: of_get_named_gpiod_flags: can't parse 'wp-gpio' property of node '/dwmmc@fe320000[0]'
Apr 28 09:13:38 rockpi4b-node4 kernel: dwmmc_rockchip fe320000.dwmmc: using lookup tables for GPIO lookup
Apr 28 09:13:38 rockpi4b-node4 kernel: dwmmc_rockchip fe320000.dwmmc: lookup for GPIO wp failed
Apr 28 09:13:38 rockpi4b-node4 kernel: vcc_sd: unsupportable voltage range: 3300000-3000000uV
Apr 28 09:13:38 rockpi4b-node4 kernel: rockchip-iodomain ff770000.syscon:io-domains: Setting to 3000000 done
Apr 28 09:13:38 rockpi4b-node4 kernel: rockchip-iodomain ff770000.syscon:io-domains: Setting to 3000000 done
Apr 28 09:13:38 rockpi4b-node4 kernel: mmc_host mmc0: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0)
Apr 28 09:13:38 rockpi4b-node4 kernel: dwmmc_rockchip fe320000.dwmmc: 1 slots initialized
Apr 28 09:13:38 rockpi4b-node4 kernel: sdhci-pltfm: SDHCI platform and OF driver helper
Apr 28 09:13:38 rockpi4b-node4 kernel: sdhci-arasan fe330000.sdhci: GPIO lookup for consumer wp
Apr 28 09:13:38 rockpi4b-node4 kernel: sdhci-arasan fe330000.sdhci: using device tree for GPIO lookup
Apr 28 09:13:38 rockpi4b-node4 kernel: of_get_named_gpiod_flags: can't parse 'wp-gpios' property of node '/sdhci@fe330000[0]'
Apr 28 09:13:38 rockpi4b-node4 kernel: of_get_named_gpiod_flags: can't parse 'wp-gpio' property of node '/sdhci@fe330000[0]'
Apr 28 09:13:38 rockpi4b-node4 kernel: sdhci-arasan fe330000.sdhci: using lookup tables for GPIO lookup
Apr 28 09:13:38 rockpi4b-node4 kernel: sdhci-arasan fe330000.sdhci: lookup for GPIO wp failed
Apr 28 09:13:38 rockpi4b-node4 kernel: sdhci-arasan fe330000.sdhci: Looking up vmmc-supply from device tree
Apr 28 09:13:38 rockpi4b-node4 kernel: sdhci-arasan fe330000.sdhci: Looking up vmmc-supply property in node /sdhci@fe330000 failed
Apr 28 09:13:38 rockpi4b-node4 kernel: sdhci-arasan fe330000.sdhci: Looking up vqmmc-supply from device tree
Apr 28 09:13:38 rockpi4b-node4 kernel: sdhci-arasan fe330000.sdhci: Looking up vqmmc-supply property in node /sdhci@fe330000 failed
Apr 28 09:13:38 rockpi4b-node4 kernel: sdhci-arasan fe330000.sdhci: No vmmc regulator found
Apr 28 09:13:38 rockpi4b-node4 kernel: sdhci-arasan fe330000.sdhci: No vqmmc regulator found
Apr 28 09:13:38 rockpi4b-node4 kernel: mmc1: SDHCI controller on fe330000.sdhci [fe330000.sdhci] using ADMA
Apr 28 09:13:38 rockpi4b-node4 kernel: of_get_named_gpiod_flags: parsed 'gpios' property of node '/gpio-leds/user-led1[0]' - status (0)
Apr 28 09:13:38 rockpi4b-node4 kernel: of_get_named_gpiod_flags: parsed 'gpios' property of node '/gpio-leds/user-led2[0]' - status (0)
Apr 28 09:13:38 rockpi4b-node4 kernel: ledtrig-cpu: registered to indicate activity on CPUs
Apr 28 09:13:38 rockpi4b-node4 kernel: hidraw: raw HID events driver (C) Jiri Kosina
Apr 28 09:13:38 rockpi4b-node4 kernel: usbcore: registered new interface driver usbhid
Apr 28 09:13:38 rockpi4b-node4 kernel: usbhid: USB HID core driver
Apr 28 09:13:38 rockpi4b-node4 kernel: rockchip-saradc ff100000.saradc: Looking up vref-supply from device tree
Apr 28 09:13:38 rockpi4b-node4 kernel: rockchip-saradc ff100000.saradc: Looking up vref-supply property in node /saradc@ff100000 failed
Apr 28 09:13:38 rockpi4b-node4 kernel: ff100000.saradc supply vref not found, using dummy regulator
...skipping...
Apr 30 15:11:16 rockpi4b-node4 kernel: dw_wdt: unexpected close, system will reboot soon
Apr 30 15:11:16 rockpi4b-node4 systemd-shutdown[1]: Syncing filesystems and block devices.
Apr 30 15:11:16 rockpi4b-node4 systemd-shutdown[1]: Sending SIGTERM to remaining processes...
-- Reboot --

Another information:

cat /etc/os-release

NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

hostnamectl

   Static hostname: rockpi4b-node4
         Icon name: computer
        Machine ID: 4fad13488e9d4a808b226d351b60eb49
           Boot ID: c0049999dde8468990d5bcb40f4c1394
  Operating System: Ubuntu 20.04.4 LTS
            Kernel: Linux 4.4.154-116-rockchip-g86a614bc15b3
      Architecture: arm64

sudo strings /dev/mtd0 | grep U-Boot

** Invalid partition type "%.32s" (expect "U-Boot")
Enter U-Boot charging mode
scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${script}; then echo Found U-Boot script ${prefix}${script}; run boot_a_script; echo SCRIPT FAILED: continuing...; fi; done
U-Boot.armv8
U-Boot 2017.09-2700-g70b4cfe057

What PoE HAT are you using and what is your PoE switch?

D-Link DGS-1005P/A1A 5G 1 PoE 15.4W + ROCK Pi 23W PoE HAT