I have been messing around with the simple task of setting up an RTSP server from a mipi camera and receiving this stream with minimal delays for about half a year now.
In my tests, I first receive the Happytime Rtsp Client X64 stream on Windows, and only then try to receive it on RAdxa 3W or Orangepi CM4.
RTSP server based on RADXA 3W
And I encountered the fact that there is a fake number of frames and resolution in the case of the Gstreamer. It gave me 4K and 20 FPS as real, but the receiver displayed the resolution that was set, or if you do not set the resolution, it gave 4K 120 FPS, although the camera physically cannot give this.
In a neighboring topic, I encountered the output of the camera to HDMI with the fact that the camera does not switch modes for some reason and as I assume, I may be wrong, we have an interpolated resolution from the base 4K 20 fps. Although the camera supports other modes, for testing I took Raspb Camera V2 with the same sensor, it behaves the same, but for some reason the modes do not switch. Therefore, it is not yet clear what we actually get.
The server based on ffmpeg gives worse delays. I refused it.
In my opinion, the compiled gst-rtsp-server-1.18.4 gives better quality and minimal delays. I am currently receiving the stream on a Windows PC. Stream parameters (number of frames and FPS there seem to be correctly shown)
You can compile version 1.20.3 using this instruction
I have not had time to test it yet.
I started testing the FPS quantity and quality with a wave of my hand and finger 
And the second problem is to receive this stream with minimal delays as on a PC. It is obvious that it can be received using a gstreamer
At the moment, I am trying to figure out how to receive a stream on an Orange pi CM4 board, which has a similar processor. I chose Armbian as the OS, but in the minimal IOT version I could not get Gstreamer and MPP to work together
Now I have installed Armbian with a desktop and deleted it, there is a built-in gstreamer and MPP, but so far I have encountered an access rights error, while I am trying to figure this out, although I have deleted gnome and X. The command gst-launch-1.0 videotestsrc ! kmssink works correctly.
I will switch to Radxa 3W as a receiver after CM4.
There is a problem with Armbian for Radxa 3W with a desktop, which I described on the Armbian forum.
There used to be an Armbian Server image, but for some reason they removed it. Theoretically, you can build your own image with the necessary components using the Armbian constructor.
At the moment, I don’t have a good solution yet.