HDMI Input on Rock 5B

#1

Can anyone help me to get the HDMI Input working on the Rock 5B running Ubuntu 22.04?
Thanks

HDMI-in audio recording
#2

For audio:

current_pulse_sink=$(pactl get-default-sink)
gst-launch-1.0 -e pulsesrc device=alsa_input.platform-hdmiin-sound.stereo-fallback ! audioconvert ! audioresample ! queue ! pulsesink device=${current_pulse_sink}

For video:

First you have to check the pixel format of the hdmi source:

v4l2-ctl --get-fmt-video|grep Pixel|awk '{print $4}'

If you get BGR3:

GST_VIDEO_CONVERT_USE_RGA=1 gst-launch-1.0 -e v4l2src device=/dev/video0 ! videoconvert ! waylandsink

If you get NV12:

GST_GL_API=gles2 GST_GL_PLATFORM=egl gst-launch-1.0 -e v4l2src device=/dev/video0 ! glimagesink
2 Likes
#3

Thanks so much for your reply. It was nice to finally see something working.
I am however only getting a ate of about 1 frame per second along with the message;

WARNING: from element /GstPipeline:pipeline0/GstWaylandSink:waylandsink0: A lot of buffers are being dropped.
Additional debug info:
ā€¦/libs/gst/base/gstbasesink.c(3143): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstWaylandSink:waylandsink0:
There may be a timestamping problem, or this computer is too slow.

Is the a way Video0 can be used capture device by say VLC or Javascriptā€™s getUserMedia?

#4

You need a gst base plugin patched with this patch to make use of rga accel in video convert.
I have packaged it in my ppa: https://launchpad.net/~liujianfeng1994/+archive/ubuntu/rockchip-multimedia

2 Likes
#5

Sorry for the delay in replying to you. Thanks the above worked really well! I struggled a bit with the audio.

It still does not however work the way I had hope. Iā€™m in a bit of trouble with a job now where we used to use CM4 module and a HDMI to USB where none of this was an issue.

I guess Iā€™ll have to get the 11 boxes 3D printed again to allow for the HDMI to USB. I really thought the HDMI input on the Rock 5b was just that.

Unless Iā€™m still missing something, I donā€™t think this can work. I need /dev/video0 to be either an input capture device OR somehow stream the input with both audio and video that can then be displayed in a browser. I did manage to get really bad streaming working but still not with audio.

Anyway thanks again for the input and help.

#6

There is no hardware issues now. You have to deal with gstreamer. I canā€™t help you much because I am not a gstreamer expert.

#7

Thanks you already helped a lot

#8

this great helpful!! thank you!

and some other questions,

1, do you know can i record the hdmi using gst while open /dev/video0 with opencv?

2, or can i use hardware encoder function in opencv?

3, can i mmap hdmi-input to hdmi2 output by hardware?

Thank thee very much!

#9

Sorry I know less about opencv.

#10

Thank you all the same!!

so do you know how to stream hdmi-in to network with low-latency

#11

Use gstreamer to stream rtmp:

GST_VIDEO_CONVERT_USE_RGA=1 gst-launch-1.0 -e v4l2src device=/dev/video0 ! videoconvert ! video/x-raw,format=NV12,width=3840,height=2160 ! mpph264enc bps=2000 rc-mode=cbr ! h264parse ! flvmux ! rtmpsink sync=true async=true location="rtmp://your.rtmp.server"
2 Likes
#12

thatā€™s WOW

the only problem is the latency is about 2~3sec, my network is 2.5G, is there any way to low the latency < 0.5s or less ?

thank you!

#13

Latency is introduced by the client.

As an example, a remote client: ffplay playing the camera stream (3840x2160) from Rock 5B (server and publisher)

5 secs latency:
ffplay -i http://192.168.254.10:8080/live/stream.flv

~300 ms latency:
ffplay -fflags nobuffer -i http://192.168.254.10:8080/live/stream.flv

Maybe you get a few ms less with:
location=ā€œrtmp://your.rtmp.server live=1ā€ and sync=false

playback rtmp:
ffplay -fflags nobuffer -i rtmp://192.168.254.10:1935/live/stream

gstreamer playback equivalent:
gst-launch-1.0 rtmpsrc location=rtmp://192.168.254.10:1935/live/stream ! flvdemux name=demux demux.video ! queue ! h264parse ! avdec_h264 ! autovideosink

#14

Thank you so much!!

i use mpv with --profile low-latency for similar effect.

maybe i need recompile my opencv to support gstreamer next stepā€¦

Thank you !!!

#15

Try this: --profile=low-latency --no-cache --untimed

1 Like
#16

very strangeā€¦

now when i run the command, it show:

 WARNING: erroneous pipeline: no property "bps" in element "mpph264enc0"

something changed?

#17

my mistakeā€¦ use radax apt ā€¦

Armbian - the best ARM framework šŸ§
#18

Hi, I try with librga but without success. Maybe anyone have some experience and thought. HDMI RX format is BGR3

image
image.png1698Ɨ753 142 KB

Best regards

#19

@Sidhion gst-mpp doesnā€™t support rgb input yet, try rockchip-ffmpeg or change the pixel format of the input source