Hdmi capture / mpph26xenc latency

[Max_Skydrone]

Hi,

I’m working with official ubuntu 20.04.6 LTS from https://wiki.radxa.com/Rock5/downloads

I’m doing HDMI capture, h264/h265 hardware encoding, and rtsp-streaming with gstreamer.

I get help from these posts :

Ubuntu 20.04.6 LTS - gstreamer HW encoding

Hi, I’m having issues trying to h264/h265 HW encode with gtreamer (rtsp-server). Ubuntu 20.04.6 LTS on ROCK 5B Software encoding works fine with “videotestsrc ! x264enc ! rtph264pay name=pay0 pt=96” I should use this for hardware encoding : “videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! videoconvert ! mpph264enc ! rtph264pay name=pay0 pt=96” But mpph264enc is missing I’ve installed mpp (https://wiki.radxa.com/Rock5/guide/rockchip-mpp) and gstreamer1.0-roc…

IMX415 + NPU demo on ROCK 5B

There is a little gem here: There you find all pre-built deb packages if you don’t want to spend time building/compiling and fixing all stuff. Start with a minimal image that does not have any broken dependencies and then install the required packages. I think rkr1 was meant for kernel 5.10.66 ( ~ 72 - patches) and rkr4 for 5.10.110. But rkr1 will work on 5.10.110 with some minor features missing. Care must be taken by the order you install the packages. And don’t make the mistake to ins…

ROCK 5B Camera streaming with RTSP / RTMP

I will find a 4k tv monitor to try out. Looks like 1080@60 is BGR as you noted. Converting from BGR to NV12 does not look like a good solution, i think it will be by SW conversion. rock@rock5b:~$ v4l2-ctl -d /dev/video20 --all Driver Info: Driver name : rk_hdmirx Card type : rk_hdmirx Bus info : fdee0000.hdmirx-controller Driver version : 5.10.66 Capabilities : 0x84201000 Video Capture Multiplanar Streaming Extended Pix Format Device Capabilities Devic…

It’s now working, but it needs some improvments

This command works :
gst-rtsp-server/examples/test-launch “( v4l2src device=/dev/video0 io-mode=dmabuf ! video/x-raw,format=BGR,width=1920,height=1080,framerate=60/1 ! videoconvert ! video/x-raw,width=1920,height=1080,format=NV12,framerate=60/1 ! videorate ! mpph265enc bps=3000000 bps-max=3172000 rc-mode=vbr ! rtph265pay name=pay0 pt=96 )”

If I remove “videorate” it doesn’t work without huge buffer on video player (like 1s) and there is a lot of lost frames (around 15fps only are displayed).
If I add this “videorate” in the pipe, it run without buffering on video player (vlc) and aroung 60fps are displayed

I can let this “videorate”, but I wonder why this could be needed … Any idea ?

My main question is about Latency. I’m getting around 1sec of latency for the sequence hdmi_in_capture => encoding => rtsp/wifi => player

From previous work on this subject, I assume rtsp/wifi and player is about 100 ms.
Do you know a way to calculate hdmi_in_capture and mpph26x encoding latency, and a way to reduce them to their minimum ?

Additionnal question : I could try all this on ubuntu 22.04, where do you get it ? I only see 20.04.6 LTS from https://wiki.radxa.com/Rock5/downloads

Thanks for your help !

Max

[avaf]

I could try all this on ubuntu 22.04, where do you get it

I use my own custom image (hand-made, not automated, and X11) and i build everything on the board.
I think you are looking for this: (Introduction to rockchip-multimedia ppa for ubuntu jammy)

It’s Wayland but i think you can have X11 as well.
Looks like Wayland is 3x faster for video accel. but not stable as X11.

About the latency, i think there is a parameter/property to suppress latency.

Maybe this can help:
(https://stackoverflow.com/questions/49284547/gstreamer-rtsp-creating-a-pipeline-to-reduce-latency-due-to-rtpjitterbuffer)