Hello all,
I am trying to live stream a USB webcam using Radxa CM5 with RPi4 CM4 io-board.
OS: Linux radxa-cm5-rpi-cm4-io 6.1.84-6-rk2410 #6 SMP
Camera: Logitech C270 connected over USB2.0
When I try to stream the webcam using the following gstreamer string using software H264, video feed has a huge delay as expected but is smooth without any interruptions / hitches:
./test-launch “( v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! x264enc ! rtph264pay name=pay0 pt=97 )”
But when I try to use the hardware encoder, the video feed is dropping frames, and the frames have some sort of visual artifacts. The string I used here is:
./test-launch “( v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! mpph264enc ! rtph264pay name=pay0 pt=97 )”
The log is as follows:
stream ready at rtsp://127.0.0.1:8554/test 0:00:00.579223243 3288 0xaaaafff3bcc0 WARN mpp gstmpp.c:121:gst_mpp_use_rga: RGA enabled 0:00:00.612141367 3288 0xffffa0010c60 WARN v4l2 gstv4l2object.c:4410:gst_v4l2_object_set_crop:<v4l2src0:src> VIDIOC_S_CROP failed 0:00:00.648434894 3288 0xffffa0010c60 WARN v4l2 gstv4l2object.c:3228:gst_v4l2_object_reset_compose_region:<v4l2src0:src> Failed to get default compose rectangle with VIDIOC_G_SELECTION: Invalid argument 0:00:03.623937968 3288 0xffffa0010f00 WARN rtpsession gstrtpsession.c:2435:gst_rtp_session_chain_send_rtp_common:<rtpsession0> Can't determine running time for this packet without knowing configured latency 0:00:03.624451293 3288 0xaaaafff3bcc0 FIXME rtspmedia rtsp-media.c:4622:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing 0:00:03.707274219 3288 0xffffa0010f00 WARN v4l2 gstv4l2object.c:4410:gst_v4l2_object_set_crop:<v4l2src1:src> VIDIOC_S_CROP failed 0:00:03.743811284 3288 0xffffa0010f00 WARN v4l2 gstv4l2object.c:3228:gst_v4l2_object_reset_compose_region:<v4l2src1:src> Failed to get default compose rectangle with VIDIOC_G_SELECTION: Invalid argument 0:00:04.005183475 3288 0xffffa0010c60 WARN rtpsession gstrtpsession.c:2435:gst_rtp_session_chain_send_rtp_common:<rtpsession1> Can't determine running time for this packet without knowing configured latency 0:00:04.005480387 3288 0xaaaafff3bcc0 FIXME rtspmedia rtsp-media.c:4622:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing 0:00:04.007455521 3288 0xaaaafff3bcc0 FIXME rtspmedia rtsp-media.c:4622:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing 0:00:04.007484687 3288 0xaaaafff3bcc0 WARN rtspmedia rtsp-media.c:4661:gst_rtsp_media_suspend: media 0xffffa80553b0 was not prepared 0:00:04.014181827 3288 0xaaaafff3bcc0 FIXME rtspmedia rtsp-media.c:2900:gst_rtsp_media_seek_trickmode:<GstRTSPMedia@0xffffa80553b0> Handle going back to 0 for none live not seekable streams. 0:00:04.014852649 3288 0xffffa0010c60 WARN rtpsession gstrtpsession.c:2435:gst_rtp_session_chain_send_rtp_common:<rtpsession1> Can't determine running time for this packet without knowing configured latency 0:00:06.869702504 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 4 - ts: 0:00:02.815922449 0:00:06.873295778 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 9 - ts: 0:00:03.151793449 0:00:13.518124947 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 1 - ts: 0:00:07.403717448 0:00:13.520568490 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 4 - ts: 0:00:07.567672448 0:00:13.523042366 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 21 - ts: 0:00:08.307669448 0:00:13.525574865 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 1 - ts: 0:00:08.375714449 0:00:13.528188739 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 26 - ts: 0:00:09.275919448 0:00:13.533403945 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 14 - ts: 0:00:09.807632740 0:00:15.128077115 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 26 - ts: 0:00:11.407607741 0:00:16.269517090 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 6 - ts: 0:00:12.543635740 0:00:18.199555607 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 32 - ts: 0:00:14.475567740 0:00:19.036063109 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 1 - ts: 0:00:15.307542740 0:00:24.685638476 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 4 - ts: 0:00:17.175573032 0:00:24.689312540 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 6 - ts: 0:00:17.407516032 0:00:24.691960538 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 35 - ts: 0:00:18.607486740 0:00:24.696727168 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 27 - ts: 0:00:19.747638740 0:00:24.708223895 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 7 - ts: 0:00:20.299537032 0:00:24.795726245 3288 0xffffa0010f00 WARN v4l2src gstv4l2src.c:1351:gst_v4l2src_create:<v4l2src1> lost frames detected: count = 9 - ts: 0:00:20.903417740
It is the same when using mpp265enc as well.
To debug further I tried the ffmpeg + mediamtx for RTSP from the Radxa wiki, this seems to be working, the commands are:
nohup ./mediamtx &
ffmpeg-rockchip/ffmpeg -f v4l2 -i /dev/video0 -vf “fps=30” -s 640x480 -c:v h264_rkmpp -rc_mode AVBR -b:v 20M -minrate 10M -maxrate 40M -profile:v main -level 5.1 -f rtsp rtsp://0.0.0.0:8554/stream
So the the hardware seems to be fine, looks like it is an issue with gstreamer implementations of the encoders perhaps.
Any guidance on how to resolve the issue with gstreamer would be greatly appreciated as my whole application is built upon gstreamer.
Thank You