ROCK 5B Camera streaming with RTSP / RTMP

avaf · April 21, 2023, 1:41pm

Here is the instructions for Ubuntu 22.04 , X11, xfce4, latest kernel.
Using the @bytecode64 suggestion, i passed the parms to the encoder that reduces the bandwidth , but if you happen (or anyone else) to know better parms to reduce even more bandwidth and keep quality, please suggest.

Build the server:

mkdir -p rtsp-server
cd rtsp-server/
wget https://gstreamer.freedesktop.org/src/gst-rtsp-server/gst-rtsp-server-1.20.3.tar.xz
tar -xf gst-rtsp-server-1.20.3.tar.xz
cd gst-rtsp-server-1.20.3/
mkdir build && cd build
meson --prefix=/usr --wrap-mode=nofallback -D buildtype=release -D package-origin=https://gstreamer.freedesktop.org/src/gstreamer/ -D package-name="GStreamer 1.20.3" ..
ninja -j8
sudo ninja install
sudo ldconfig
cd examples/

Run the server with 4k streaming:

./test-launch "( v4l2src device=/dev/video11 io-mode=dmabuf ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! mpph265enc bps=2000000 bps-max=3172000 rc-mode=vbr ! rtph265pay name=pay0 pt=96 )"

CPU Usage average: 408 ~1200 Mhz ( 8 hours testing the best streaming 4k 30 fps)

To be tested (audio):

./test-launch "( v4l2src device=/dev/video11 io-mode=dmabuf ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! mpph265enc bps=1000000 bps-max=3172000 rc-mode=vbr ! rtph265pay name=pay0 pt=96 ! audiotestsrc ! alawenc ! rtppcmapay name=pay1 pt=97 )"

AC3 test

./test-launch "( v4l2src device=/dev/video11 io-mode=dmabuf ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 do-timestamp=true ! mpph265enc bps=1000000 bps-max=3172000 rc-mode=vbr ! rtph265pay name=pay0 pt=96 ! audiotestsrc ! avenc_ac3 ! rtpac3pay name=pay1 pt=97 )"

and:

./test-launch "( v4l2src device=/dev/video11 io-mode=dmabuf ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! mpph265enc bps=1000000 bps-max=3172000 rc-mode=vbr ! rtph265pay name=pay0 pt=96 ! alsasrc device=sysdefault:CARD=rockchipes8316 ! alawenc ! rtppcmapay name=pay1 pt=97 )"

Taken from:

record -L
null
    Discard all samples (playback) or generate zero samples (capture)
samplerate
    Rate Converter Plugin Using Samplerate Library
speexrate
    Rate Converter Plugin Using Speex Resampler
jack
    JACK Audio Connection Kit
oss
    Open Sound System
pulse
    PulseAudio Sound Server
upmix
    Plugin for channel upmix (4,6,8)
vdownmix
    Plugin for channel downmix (stereo) with a simple spacialization
default
    Playback/recording through the PulseAudio sound server
usbstream:CARD=rockchiphdmi0
    rockchip-hdmi0
    USB Stream Output
usbstream:CARD=rockchiphdmi1
    rockchip-hdmi1
    USB Stream Output
usbstream:CARD=rockchipdp0
    rockchip,dp0
    USB Stream Output
hw:CARD=rockchipes8316,DEV=0
    rockchip-es8316, fe470000.i2s-ES8316 HiFi ES8316 HiFi-0
    Direct hardware device without any conversions
plughw:CARD=rockchipes8316,DEV=0
    rockchip-es8316, fe470000.i2s-ES8316 HiFi ES8316 HiFi-0
    Hardware device with all software conversions
sysdefault:CARD=rockchipes8316
    rockchip-es8316, fe470000.i2s-ES8316 HiFi ES8316 HiFi-0
    Default Audio Device
dsnoop:CARD=rockchipes8316,DEV=0
    rockchip-es8316, fe470000.i2s-ES8316 HiFi ES8316 HiFi-0
    Direct sample snooping device
usbstream:CARD=rockchipes8316
    rockchip-es8316
    USB Stream Output
hw:CARD=rockchiphdmiin,DEV=0
    rockchip,hdmiin, fddf8000.i2s-dummy_codec hdmiin-dc-0
    Direct hardware device without any conversions
plughw:CARD=rockchiphdmiin,DEV=0
    rockchip,hdmiin, fddf8000.i2s-dummy_codec hdmiin-dc-0
    Hardware device with all software conversions
sysdefault:CARD=rockchiphdmiin
    rockchip,hdmiin, fddf8000.i2s-dummy_codec hdmiin-dc-0
    Default Audio Device
dsnoop:CARD=rockchiphdmiin,DEV=0
    rockchip,hdmiin, fddf8000.i2s-dummy_codec hdmiin-dc-0
    Direct sample snooping device
usbstream:CARD=rockchiphdmiin
    rockchip,hdmiin
    USB Stream Output

And finally, if you want to play the rtsp stream in your ROCK 5B:

GST_GL_API=gles2 GST_GL_PLATFORM=egl gst-launch-1.0 uridecodebin uri=rtsp://192.168.254.10:8554/test ! glimagesink
Setting pipeline to PAUSED ...
arm_release_ver of this libmali is 'g6p0-01eac0', rk_so_ver is '5'.
Pipeline is live and does not need PREROLL ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayGBM\)\ gldisplaygbm0";
Progress: (open) Opening Stream
Pipeline is PREROLLED ...
Prerolled, waiting for progress to finish...
Progress: (connect) Connecting to rtsp://192.168.254.10:8554/test
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Redistribute latency...
Redistribute latency...
Redistribute latency...
Progress: (request) Sending PLAY request
Redistribute latency...
Progress: (request) Sent PLAY request
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:25.353149438
Setting pipeline to NULL ...
Freeing pipeline ...

That’s all folks!