AFAIR this JMicron does not reliably work in UAS mode with RPi4. It should be well-known, I even think USB quirks are applied by default in the RPi kernel, but not sure about that.
At least smartctl needs UAS. When you put the thing in MSD mode (so explicitly add USB quirks yourself to kernel cmdline), you should not see any USB3 resets I think. And of course smartctl and fstrim won’t work then.
Also be aware that this setup makes little sense; Usually RAID is applied to enhance uptime of a storage solution, but this just gets the whole solution in trouble (all 4 devices), not just 1 SSD. Also max total I/O speed is only 500GBps (due to 8/10 channel coding PCIe2 for BCM2711(RPi4), meaning a modern SSD can already max out I/O bandwidth. So I would just use 1 SSD with the RPi4 and use the other 3 for other purposes, like attaching to an SBC/SoC that has native SATA, like RK35xx.
Hi, radrocks-san,
Thanks a lot for your comment, really appreciated.
To summarize our case very simply:
We can reproduce RAID10 degrade when smartctl -x -j is run in burst across 4 disks during md resync.
Right after that we see reset SuperSpeed / USB disconnect / I/O error / Disk failure.
So for now, we changed operation to:
no burst SMART on all disks,
staggered one-disk collection,
skip SMART while resync/recovery/check is running.
Also, one small feedback:
I understand docs contain some limitations, but current wording can still be confusing for users.
It is advertised as a “complete NAS solution”, while practical RAID constraints over USB bridge are quite important.
Maybe it would help to put these caveats in a more visible place (early warning / best-practice section).
This is known bug of JMS controller on board, and the only solution available was to alter smartctl binary (or something that is using it like omv) to not make certain calls. There were some patches for that to skip right part, but it’s not best way to solve such issues.
Disabling UAS helps to skip the problem, but it’s also not the best way, as far as I remember still RAID will brake from time to time. The best approach is to use JMS hardware RAID option, this one will work with SMART and UAS, and it’s the only stable option.
I agree that just the 4-SATA HAT is not a complete NAS solution. It is only a component, 1 of the several you need to build a NAS. You need SSD’s, a powersupply, cables and a computer board. The latter 1 is already the problem: RPi4 is called a “Single Board Computer” but it has no storage like a smartphone or laptop. It is at least not a “PC motherboard” where you need to plug a CPU and RAM and other PCIe cards maybe.
So it is marketing. And because it is all cheap or it seems cheap, people buy things and might not have thought about all system design issues.
If you want NAS (and RAID) your storage devices are most important I think. So in this case SSD’s with SATA interface. It is then typically strange that the host platform (RPI4) is used as it does not have SATA connectors. It has an USB3 chip (VIA) and that you connect to another USB3 chip (JMicron) then connect to the SSD(’s). So 2 chips in the chain that you won’t have on for example Intel platform with 2 or 4 or 6 SATA ports.
It all can work via USB3, but as you experience, it has issues.
Hi, dominik-san,
Thank you for the detailed explanation.
I tested the JMS hardware RAID option too. It looked better in short tests, but in my longer test I still saw repeated USB reset/disconnect events, and the array became unstable again.
So I decided to return the board. Maybe it is my unit/cable/power combination, but I could not make it stable enough for NAS use.
Thanks again, your advice helped me understand the issue much better.
Hi, radrocks-san,
Thank you, I agree with your point.
After testing, I also feel the weak point is the whole system design, not only one command or one RAID mode. RPi4 + USB3 bridge + JMS bridge + SATA SSDs is a long chain, and in my case the USB reset/disconnect happened too often.
So I decided this board is not suitable for my NAS use case, and I will return it. I will probably choose a platform with native SATA or PCIe storage path next time.
I have used 6-port SATA traditional PC for 4x HDD (in raid10 like mode) in the past, but stopped using multi-disk setups. I feel more comfortable with just 1 large enough HDD (or SSD) and multiple SBC boards. So I have ROCK3A and ROCK5B with that simple M.2 E-key SATA break-out board in the slot where you usually put a WiFi card and use the M.2 M-key slot for NVME SSD. The NVME contains OS and various other partitions including 1 to cache the HDD (based on LVM). It is not as easy to build as plugging USB3 things, but at least RK3568 and RK3588 have SATA on-chip so no multipe USB3 chips in the chain and also all smartctl is no problem, also spindown HDD (power-save) works fine.
If I would need multiple SATA, I would now maybe buy a ROCK5B+ as it has 2 M.2 M-key slots, so 1 for NVME and the other for a 6-port ASM1166 M.2 M-key based module. It needs own 12V and 5V power cables, but that is not a problem for me. This has enough bandwidth to do Linux/software based RAID, including Btrfs or ZFS. ROCK5B runs standard OS/kernel (Debian Sid or Opensuse Tumbleweed).
If you want ready-made small ARM based solution, you have to look around, other vendors might have better solutions.
Hi, radrocks-san,
Thank you for the detailed suggestion.
I agree with your point. Native SATA or PCIe storage path is much better than using multiple USB3 bridge chips in the chain. In my case, I was not trying to build the best NAS system from scratch. I already had four 1TB SATA SSDs, so I wanted to reuse them if possible. That was the main reason why I tried this Quad SATA HAT.
But now PC parts and NAS related parts are quite expensive, so for me it is not a good timing to buy many new parts immediately. Your suggestion is reasonable, but I will take more time. I am now thinking about another option, maybe a used small PC / SFF PC with a SATA HBA or RAID card. It may be a better way to reuse my SATA SSDs and avoid the USB bridge chain problem.
For now, I returned this board and will slowly look for the next platform.
Thank you again. Your advice was very helpful.