How much memory does your 3e have?
RGA crashes on Zero 3 with 8 GB RAM, not with less
@nyanmisaka, Jellyfin succeeded!
Since RGA succeeds with jellyfin-ffmpeg, this should be a hardware limitation of the RGA2E on RK356x/3588 not supporting 4GB+ memory. You need to use the DMA32 flag to allocate DMA memory and have the RGA input and output use it.
I’m sorry to inform you that Jellyfin doesn’t use the RGA, when playing:
./ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -i INPUT.mp4 \ -vf scale_rkrga=format=nv12,hwmap,format=nv12 -f null -
Simply, because to mpp output is already in NV12, there is no need for a RGA accelerated nv12->BGR conversion. Looking at the ffmpeg info output on the console, you see no RGA being initialized.
My Zero3E has 8 GB of RAM
free -m:
Mem: 7688
Swap: 3844
ffmpeg will follow the filters you give it, even if it’s copying from NV12 to NV12 using RGA. There’s no passthrough.
And the RGA initialization log somehow seems to have disappeared on newer librga. You can add -v debug to determine if it is working.
I will recompile the kernel with the DMA32 flag and let you know.
Thanks for pointing me in the right direction. 
Another issue is that the DMA-BUF allocator in the BSP 6.1 kernel does not support the DMA32 flag, only the DRM allocator does.
How on earth did Jellyfin get the RGA working on the 8 GB Zero3?
At last, I got it working, but it was a huge ordeal!
First, I had to recompile the Linux kernel to get an /dev/dma_heap/system-uncached-dma32.
Next, I had to manually allocate DMA memory for the RGA, using the example given by Rockchip. (https://github.com/airockchip/librga/blob/main/samples/allocator_demo/src/rga_allocator_dma32_demo.cpp)
Still, I face problems with allocating continuous memory for all my threads running the RGA.
In the end, the most simple and secure way was limiting the memory to 4 GB, in the /boot/extlinux/extlinux.conf (adding mem=4096M cma=256M).