Need to run libcamera-vid in ROCK3A/3C board

#1

Hi,
*I have Rock 3A/3C board. We used rock-3a_debian_bullseye_xfce_b25.img in the 3A board. Now I want to run libcamera tools to test live streaming (i.e libcamera-vid) on Rock 3A board. With the help of this tool, we want to pass the captured video to live555 server to check the live stream with rtsp://ip_address in other machine with VLC media player.
I checked on Debian sites, It seems libcamera-tools is supported only in Debian bookworm and not in Debian bullseye. Here are the Debian links for your reference.
https://packages.debian.org/bullseye/libs/
https://packages.debian.org/bookworm/libs/
Is there any place where I can get ROCK 3A/Rock 3C image with Debian bookworm, which support libcamera tool?
Is there any other tool in existing image of Rock 3A where I can check live stream by capturing video from camera and send it to live555 server?
FYI, I tried cheese to check camera working condition.
*

#2

If you want to test live streaming you can follow these instructions here:

It works for Rock 3A and possibly 3C.

pre-requisites:

  • rkmpp and gstreamer rkmpp plugin
    rock@rock3a:~$ gst-inspect-1.0|grep -i rock
    rkisp: rkisp: Gstreamer Plugin For Rockchip ISP Source
    rockchipmpp: mpph264enc: Rockchip Mpp H264 Encoder
    rockchipmpp: mpph265enc: Rockchip Mpp H265 Encoder
    rockchipmpp: mppjpegdec: Rockchip’s MPP JPEG image decoder
    rockchipmpp: mppjpegenc: Rockchip Mpp JPEG Encoder
    rockchipmpp: mppvideodec: Rockchip’s MPP video decoder
    rockchipmpp: mppvpxalphadecodebin: VP8/VP9 Alpha Decoder

  • gstreamer 1.20 or higher (with rockchip patches)
    GStreamer 1.20.3
    https://launchpad.net/distros/ubuntu/+source/gstreamer1.0

  • working camera
    H264 USB camera
    ov5647 (NV12)
    imx219 (NV12)

There are some other RTSP servers out there, but i don’t think they are ready to fire you always end up with some needed changes.

  • Alternatives to RTSP
    RTMP server - you push the encoded frames using gstreamer to the RTMP server and ffplay, vlc or another client can play the video.
    nginx-rtmp (RTMP)
    live555 server (RTSP)
    RSTP server
1 Like
#3

We are trying to use live555 and not GStreamer.
But for testing to check live streaming, I followed the steps to build gstreamer rtsp server as you mentioned. I tried both gst-rtsp-server-1.18.4 and gst-rtsp-server-1.18.5.
When I give following command, It gives me error
sudo meson --prefix=/usr --wrap-mode=nofallback -D buildty
pe=release -D package-origin=https://gstreamer.freedesktop.org/src/gstreamer/ -D package-name=“GStreamer 1.18.5” …

rock@rock-3a:/rockchip/gstreamer/rtsp-server/gst-rtsp-server-1.18.5/build$ sudo meson --prefix=/usr --wrap-mode=nofallback -D buildty
pe=release -D package-origin=https://gstreamer.freedesktop.org/src/gstreamer/ -D package-name=“GStreamer 1.18.5” …
The Meson build system
Version: 1.3.0
Source dir: /rockchip/gstreamer/rtsp-server/gst-rtsp-server-1.18.5
Build dir: /rockchip/gstreamer/rtsp-server/gst-rtsp-server-1.18.5/build
Build type: native build
Project name: gst-rtsp-server
Project version: 1.18.5
C compiler for the host machine: cc (gcc 10.2.1 “cc (Debian 10.2.1-6) 10.2.1 20210110”)
C linker for the host machine: cc ld.bfd 2.35.2
Host machine cpu family: aarch64
Host machine cpu: aarch64
Compiler for C supports link arguments -Wl,-Bsymbolic-functions: YES
Compiler for C supports arguments -fvisibility=hidden: YES
Compiler for C supports arguments -fno-strict-aliasing: YES
Message: Disabling GLib cast checks
Compiler for C supports arguments -Wmissing-declarations: YES
Compiler for C supports arguments -Wmissing-prototypes: YES
Compiler for C supports arguments -Wredundant-decls: YES
Compiler for C supports arguments -Wundef: YES
Compiler for C supports arguments -Wwrite-strings: YES
Compiler for C supports arguments -Wformat: YES
Compiler for C supports arguments -Wformat-nonliteral: YES
Compiler for C supports arguments -Wformat-security: YES
Compiler for C supports arguments -Wold-style-definition: YES
Compiler for C supports arguments -Waggregate-return: YES
Compiler for C supports arguments -Winit-self: YES
Compiler for C supports arguments -Wmissing-include-dirs: YES
Compiler for C supports arguments -Waddress: YES
Compiler for C supports arguments -Wno-multichar: YES
Compiler for C supports arguments -Wdeclaration-after-statement: YES
Compiler for C supports arguments -Wvla: YES
Compiler for C supports arguments -Wpointer-arith: YES
Found pkg-config: YES (/usr/bin/pkg-config) 0.29.2
Run-time dependency glib-2.0 found: YES 2.66.8
Did not find CMake ‘cmake’
Found CMake: NO
Run-time dependency gstreamer-1.0 found: NO (tried pkgconfig and cmake)
Not looking for a fallback subproject for the dependency gstreamer-1.0 because:
Use of fallback dependencies is disabled.

…/meson.build:138:10: ERROR: Dependency ‘gstreamer-1.0’ is required but not found.

when I checked the meson-log.txt , it is looking for gstreamer path into /usr/lib/gcc/aarch64-linux-gnu/10 and it is not there.

When I check for gstreamer-1.0, it is under /usr/lib/aarch64-linux-gnu.

Could you please guide how to process further?

#4

You need gtreamer and all gstreamer plugins. Install it before building gst-server.
I will provide detailed instructions in another post.

 gir1.2-gst-plugins-bad-1.0
 gir1.2-gst-plugins-base-1.0
 gir1.2-gstreamer-1.0
 gstreamer1.0-alsa
 gstreamer1.0-gl
 gstreamer1.0-gtk3
 gstreamer1.0-opencv
 gstreamer1.0-plugins-bad
 gstreamer1.0-plugins-bad-apps
 gstreamer1.0-plugins-base
 gstreamer1.0-plugins-base-apps
 gstreamer1.0-plugins-good
 gstreamer1.0-pulseaudio
 gstreamer1.0-qt5
 gstreamer1.0-rockchip1 - The Gstreamer plugins for Rockchip platforms.
 gstreamer1.0-tools
 gstreamer1.0-wpe
 gstreamer1.0-x
 libgstreamer-gl1.0-0
 libgstreamer-opencv1.0-0
 libgstreamer-plugins-bad1.0-0
 libgstreamer-plugins-bad1.0-dev
 libgstreamer-plugins-base1.0-0
 libgstreamer-plugins-base1.0-dev
 libgstreamer-plugins-good1.0-0
 libgstreamer-plugins-good1.0
 libgstreamer1.0-0
 libgstreamer1.0-dev

You will have to build your live555 server then.
live555 vs Gstremaer

#5

Thanks for the information, I will check all the gstreamer plugin are installed or not and if something is missing, will install it.

For live555, I have already build live555 server in Rock 3A board. The only thing is how should I pass the capture video frames to live555 server, that is why I want libcamera tool to be installed on 3A board.

I have done POC on Raspberry Pi for live 555 with Debian 12 (bookworm). I followed below link for that and able to run the LIVE STREAMING successfully.

https://forums.raspberrypi.com/viewtopic.php?t=52071

raspvid is for older version and libcamera for new versions. So with below command I was able to do live streaming in raspberry pi with live555 RTSP server.

pi@Raspberry ~/live/raspi $ libcamera-vid -b 300000 -t 99999999 -o - | ./testRaspi

please let me know, If I can do same in Rock 3A board, because our preference is to use live555.

#6

There is no support for libcamera on RK35xx AFAIK (yet).

I think you can solve this (using the live555 server) by:

  • Build your own live555 server (link provided) that pulls frames and encodes the frames for your camera
    or
  • Use gstreamer to pull the frames, encode it, and pipe them to your live555 server, as you do with raspi

I think you better find a ready-made OS Image (distro) that has encoder/decoder/gstreamer working or you end up compiling every component yourself (which is time-consuming but some people really like that :slight_smile: )

Here is how you can solve your problem:

  • create named pipe file

    mkfifo /tmp/video

  • Pull frames and encode it with gstreamer and write it to the named pipe

    gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=NV12,width=640,height=480' ! mpph264enc ! filesink location=/tmp/video

  • make live555 read the named pipe /tmp/video

You you want to test it with distro gstreamer, use x264enc instead of mpph264enc

#7

I am able to successfully build gst-server and did live streaming with following command on ROCK 3A board.

sudo ./test-launch “( v4l2src device=/dev/video0 ! video/x-raw,format=NV12,width=1920,height=1080 ! mpph264enc ! rtph264pay name=pay0 pt=96 )”

Now, I want to stream audio as well and for that I tried following command and It gives me error as following. Is there anything else I need to install or change any setting in commnad?

rock@rock-3a:~/rockchip/gstreamer/rtsp-server/gst-rtsp-server-1.18.4/build/examples$ sudo ./test-launch --gst-debug=3 “( videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! mpph264enc ! rtph264pay name=pay0 pt=96 ! audiotestsrc ! audio/x-raw-int,rate=8000 ! alawenc ! rtppcmapay name=pay1 pt=97)”
[sudo] password for rock:
stream ready at rtsp://127.0.0.1:8554/test

(test-launch:3924): GStreamer-WARNING **: 17:41:57.843: 0.10-style raw audio caps are being created. Should be audio/x-raw,format=(string)… now.
0:00:17.096596592 3924 0xaaaaed776860 ERROR GST_PIPELINE gst/parse/grammar.y:459:gst_parse_element_set: could not set property “pt” in element “pay1” to “97)”
0:00:17.096816218 3924 0xaaaaed776860 WARN default gst/parse/grammar.y:1181:priv_gst_parse_yyerror: Error during parsing: syntax error
0:00:17.096850634 3924 0xaaaaed776860 ERROR GST_PIPELINE gst/parse/grammar.y:1105:priv_gst_parse_yyparse: syntax error
0:00:17.096898468 3924 0xaaaaed776860 ERROR GST_PIPELINE gst/parse/grammar.y:1221:priv_gst_parse_launch: Unrecoverable syntax error while parsing pipeline ( videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! mpph264enc ! rtph264pay name=pay0 pt=96 ! audiotestsrc ! audio/x-raw-int,rate=8000 ! alawenc ! rtppcmapay name=pay1 pt=97)

** (test-launch:3924): CRITICAL **: 17:41:57.858: could not parse launch syntax (( videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! mpph264enc ! rtph264pay name=pay0 pt=96 ! audiotestsrc ! audio/x-raw-int,rate=8000 ! alawenc ! rtppcmapay name=pay1 pt=97)): could not set property “pt” in element “pay1” to “97)”

** (test-launch:3924): CRITICAL **: 17:41:57.859: could not create element
0:00:17.102777593 3924 0xaaaaed776860 ERROR rtspclient rtsp-client.c:1069:find_media: client 0xaaaaed7731c0: can’t create media
0:00:17.103485177 3924 0xaaaaed776860 ERROR rtspclient rtsp-client.c:3346:handle_describe_request: client 0xaaaaed7731c0: no media

(test-launch:3924): GStreamer-WARNING **: 17:42:02.008: 0.10-style raw audio caps are being created. Should be audio/x-raw,format=(string)… now.
0:00:21.251334552 3924 0xaaaaed776860 ERROR GST_PIPELINE gst/parse/grammar.y:459:gst_parse_element_set: could not set property “pt” in element “pay1” to “97)”
0:00:21.251451510 3924 0xaaaaed776860 WARN default gst/parse/grammar.y:1181:priv_gst_parse_yyerror: Error during parsing: syntax error
0:00:21.251477468 3924 0xaaaaed776860 ERROR GST_PIPELINE gst/parse/grammar.y:1105:priv_gst_parse_yyparse: syntax error
0:00:21.251519468 3924 0xaaaaed776860 ERROR GST_PIPELINE gst/parse/grammar.y:1221:priv_gst_parse_launch: Unrecoverable syntax error while parsing pipeline ( videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! mpph264enc ! rtph264pay name=pay0 pt=96 ! audiotestsrc ! audio/x-raw-int,rate=8000 ! alawenc ! rtppcmapay name=pay1 pt=97)

** (test-launch:3924): CRITICAL **: 17:42:02.009: could not parse launch syntax (( videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! mpph264enc ! rtph264pay name=pay0 pt=96 ! audiotestsrc ! audio/x-raw-int,rate=8000 ! alawenc ! rtppcmapay name=pay1 pt=97)): could not set property “pt” in element “pay1” to “97)”

** (test-launch:3924): CRITICAL **: 17:42:02.009: could not create element
0:00:21.251918468 3924 0xaaaaed776860 ERROR rtspclient rtsp-client.c:1069:find_media: client 0xaaaaed773300: can’t create media
0:00:21.252124385 3924 0xaaaaed776860 ERROR rtspclient rtsp-client.c:3064:handle_setup_request: client 0xaaaaed773300: media ‘/test’ not found

#8

If there’s something I never get right in linux, it’s the sound.
Anyway, i think there’s an error in your pipeline, audio/x-raw-int should be audio/x-raw. I am not an expert on gstreamer.

Thsi works for me here:

./test-launch "( v4l2src device=/dev/video0 io-mode=dmabuf ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! mpph264enc ! rtph264pay name=pay0 pt=96 ! audiotestsrc ! alawenc ! rtppcmapay name=pay1 pt=97 )"

and

./test-launch "( v4l2src device=/dev/video0 io-mode=dmabuf ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! mpph264enc ! rtph264pay name=pay0 pt=96 ! audiotestsrc ! audio/x-raw,rate=8000 ! alawenc ! rtppcmapay name=pay1 pt=97 )"

#9

I tried audio test pipeline, It gives continuous single tone. What to do if we want real time audio from rock 3A board? Some mic need to connect on board?

One more exercise I am doing is about multiple stream with different resolution from single source stream. I tried following command by using tee of gstreamer to generate multiple stream/sub stream.
gst-launch-1.0 -vvv --gst-debug-level=3 v4l2src device=/dev/video0 ! tee name=t \t. ! queue ! videoconvert! videoscale ! ‘video/x-raw,format=NV12,width=1920,height=1080’ ! autovideosink \t. ! queue ! videoconvert! videoscale ! ‘video/x-raw,format=NV12,width=1280,height=720’ ! autovideosink \t. ! queue ! videoconvert! videoscale ! ‘video/x-raw,format=NV12,width=640,height=480’ ! autovideosink

This command works and shows different resolution windows of live stream on local machine, but its all raw data and high cpu usage, so I need to encode it, I tried different encoder format, but then it ends up in errors.

My other query is , is it possible to run multiple streams from test-launch? I tried to change the code to run multiple stream but I think its taking same stream only.
Below is the code I added in test-launch.c file.

  factory = gst_rtsp_media_factory_new ();
  gst_rtsp_media_factory_set_launch (factory, argv[1]);
  gst_rtsp_media_factory_set_shared (factory, TRUE);
  gst_rtsp_mount_points_add_factory (mounts, "/stream1", factory);

  gst_rtsp_media_factory_set_launch (factory, argv[2]);
  gst_rtsp_media_factory_set_shared (factory, TRUE);
  gst_rtsp_mount_points_add_factory (mounts, "/stream2", factory);

I passed to different streams in command line as below
sudo ./test-launch “( v4l2src device=/dev/video0 ! video/x-raw,format=NV12,bitrate=1000,width=1920,height=1080 ! mpph264enc ! rtph264pay name=pay1 pt=96 )” “( v4l2src device=/dev/video0 ! video/x-raw,format=NV12,bitrate=500,width=640,height=480 ! mpph264enc ! rtph264pay name=pay0 pt=96 )”

When I tried two different streams in VLC media player (stream1 and stream2), it is showing same stream and not changing resolution and yes at a time only one stream is working. Any other way to make all the pipeline working simultaneously?

Please let me know if you have any suggestion for the above queries.