Indeed, it is possible. Rock 5B allows direct plug of a M2-M 2280 SATA controller. Easy to find modules exist based on ASM1166 and JMB585 chips. No specific drivers required, all these modules work out-of-the-box with all Armbian or Debian/Ubuntu CLI releases I have tested.
However, use of the Radxa Quad/Penta SATA hats is not obvious with this board, so I would stick to a 2280 module with regular SATA data cables and a cage for Molex/SATA power. For the HDDs, an alternate power supply is required. On my end, I found that 4pin Molex from a 150~250 ATX or Flex power supply is fine and it can feed the Rock 5B as well ditrectly from the 20/24 pin ATX power cable (5V+ground to USB-C or maybe to GPIO)
Please check my modest contribution here Rock boards as NAS solution 
This is working quite fine with software RAID and I am about to finalize a NAS including surveillance station based on Rock 5B board in a Fractal Design Node 304 NAS case. In the end, this is not really cheaper compared to a low power x86 ITX board, but it is perfectly possible!