RTSP Streaming with Zero 3E (Issues with Zero3E and Camera)

Hey folks!

I’ve hit a wall with my testing using the new Zero 3E with a IMX219 camera. I’ve got the camera working (the cheese app displays the image). I can get the camera to output locally with gstreamer but the gstreamer rtsp client sink does not work. It throws a “generic error” message.

I poked at using ffmpeg to output the stream to rtsp as well but that ffmpeg doesn’t seem to like reading in /dev/video0.

Any suggestions?

Maybe try https://github.com/AlexxIT/go2rtc

PS what permissions are on /dev//dev/video0 as can you access in your use space?
Presume there is a group video and check if you are in it?
https://trac.ffmpeg.org/wiki/Capture/Webcam has some good start points

Thanks for the suggestions!

Love the go2rtc app! I gave that a go but it looks like it takes in ffmpeg and that’s what I can’t seem to get working with the camera.

When I run ffmpeg -f v4l2 -list_formats all -i /dev/video0 it comes back saying that it’s not a valid capture device. I feel like I’m missing something around how video output works with gstreamer but not with ffmpeg.

v4l2-ctl --list-devices is no different?

This is what I’m seeing with v4l2-ctl:

rkisp-statistics (platform: rkisp):
	/dev/video7
	/dev/video8

rkisp_mainpath (platform:rkisp-vir0):
	/dev/video0
	/dev/video1
	/dev/video2
	/dev/video3
	/dev/video4
	/dev/video5
	/dev/video6
	/dev/media0

And this is what I see with ffmpeg -f v4l2 -list_formats all -i /dev/video0:

ffmpeg -f v4l2 -list_formats all -i /dev/video0
ffmpeg version 4.3.6-0+deb11u1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr --extra-version=0+deb11u1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
[video4linux2,v4l2 @ 0xaaaabb8ec220] Not a video capture device.
/dev/video0: No such device

Dunno been meaning to hook up a cam to the 03W just haven’t got round to it.


Confused as gstreamer works so you have the right overlay and guess your running debian desktop b6 being newest.

I guess you could also try the Ubuntu from https://github.com/Joshua-Riek/ubuntu-rockchip/releases/tag/v1.33 as even though the W think the ethernet is the insternal onboard that is still enabled ?

Any idea how to get the camera enabled under ubuntu?

In dtbo in /boot think the cam dtbo should be in there.

/boot/ubuntuEnv.txt add the overlays

On my Zero3W the only cam I have to hand is a Arducam IMX219 which when connected causes the board to heat up and boot to fail… ?

Well that’s concerning… I might hold off on ubuntu then.

Is there a way to get someone from Radxa to comment in this thread with ideas/suggestions?

@RadxaYuntian Might be able to help


Or ask on the radxa discord

The Debian image does the same must be my cam its IMX219 rooted out of a box of old Pi stuff so thought it would work.

You need a modified ffmpeg and gstreamer with support for MIPI cameras, see my comment in the Rock 5B section and how to build the RTSP gstreamer.

This one avaf? ROCK 5B Camera streaming with RTSP / RTMP

Yeap.

I might provide some tips on this as soon as i connect my wifi the old way. Wifi is up but i still receive tons of LOGINFO / LOGTRACE even with debugging disabled.

Wifi is up and connected, so here is the tip, the instructions still valid but make sure you have:

gst-inspect-1.0|grep -i rock
rkisp: rkisp: Gstreamer Plugin For Rockchip ISP Source
rockchipmpp: mpph264enc: Rockchip Mpp H264 Encoder
rockchipmpp: mpph265enc: Rockchip Mpp H265 Encoder
rockchipmpp: mppjpegdec: Rockchip’s MPP JPEG image decoder
rockchipmpp: mppjpegenc: Rockchip Mpp JPEG Encoder
rockchipmpp: mppvideodec: Rockchip’s MPP video decoder
rockchipmpp: mppvpxalphadecodebin: VP8/VP9 Alpha Decoder

1 Like

Think mine is just wrong cam as guessing the arducam imx279 is not same pin out as Raspberry v2 cam…

@avaf I noticed you have done some fast C code with rknpu which is prob great for running on a rk3588 identify model.
Some code here and ignore the python but that method is that the simplest and movement tracker method https://pastebin.com/FYQ3Ltez is it worthwhile dropping the numpy/cv2 calls and doing in C?
I think that is about the fastest ignoring IR or wind, as they need a cleverer prob ML model.

I sort of like what frigate does and don’t as situation, target type and model can vary massively, which makes the embedded models and manner constraining.
I would prefer motion detector and identity model to be different instances and a choice of motion detectors and identity models.
Like Frigate is great as quick easy up and running in HA but it doesn’t pan the motion detection area and letterbox in Yolo input sizes based on tracked item size and many seem to be doing ‘Its a dog’ at 100fps…

If you are ever doing a frigate like app as saw some of your examples, as it would be great if you did.

@setq Please restore the RTSP documentation.

Radxa Docs: Zero3 / RTSP Streaming

1 Like

I did attempt to follow this for ffmpeg and I ran into the following error:

ffmpeg version 65f90322bf Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --enable-rkmpp --enable-version3 --enable-libdrm --enable-libx264 --enable-gpl
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[video4linux2,v4l2 @ 0xaaaabfe444a0] ioctl(VIDIOC_G_INPUT): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0xaaaabfe444a0] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0xaaaabfe444a0] Time per frame unknown
[video4linux2,v4l2 @ 0xaaaabfe444a0] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 73917.489180, bitrate: N/A
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080, 1000k tbr, 1000k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> hevc (hevc_rkmpp_encoder))
Press [q] to stop, [?] for help
[vost#0:0/hevc_rkmpp_encoder @ 0xaaaabfe4a490] Frame rate very high for a muxer not efficiently supporting it.
Please consider specifying a lower framerate, a different muxer or setting vsync/fps_mode to vfr
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Picture format is yuv420p.
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Rate Control mode is set to VBR
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Bitrate Target/Min/Max is set to 6000000/375000/6375000
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Profile is set to MAIN
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Level is set to 10
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Quality Min/Max is set to 50%(Quant=30) / 100%(Quant=10)
[tcp @ 0xaaaabfe5ef50] Connection to tcp://192.168.42.226:8554?timeout=0 failed: Connection refused
[out#0/rtsp @ 0xaaaabfe47ac0] Could not write header (incorrect codec parameters ?): Connection refused
[vost#0:0/hevc_rkmpp_encoder @ 0xaaaabfe4a490] Error initializing output stream: 
Conversion failed!

I also tried the steps linked for mediamtx and that ended up with ffmpeg showing this:

The last lines just repeat basically.

radxa@radxa-zero3:~/FFmpeg/mediamtx$ sudo ffmpeg -re -f v4l2 -i /dev/video0 -c:v hevc -rc_mode 0 -level 30 -f rtsp rtsp://0.0.0.0:8554/stream
    ffmpeg version 65f90322bf Copyright (c) 2000-2023 the FFmpeg developers
      built with gcc 10 (Debian 10.2.1-6)
      configuration: --enable-rkmpp --enable-version3 --enable-libdrm --enable-libx264 --enable-gpl
      libavutil      58.  2.100 / 58.  2.100
      libavcodec     60.  3.100 / 60.  3.100
      libavformat    60.  3.100 / 60.  3.100
      libavdevice    60.  1.100 / 60.  1.100
      libavfilter     9.  3.100 /  9.  3.100
      libswscale      7.  1.100 /  7.  1.100
      libswresample   4. 10.100 /  4. 10.100
      libpostproc    57.  1.100 / 57.  1.100
    [video4linux2,v4l2 @ 0xaaab033a34a0] ioctl(VIDIOC_G_INPUT): Inappropriate ioctl for device
    [video4linux2,v4l2 @ 0xaaab033a34a0] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
    [video4linux2,v4l2 @ 0xaaab033a34a0] Time per frame unknown
    2024/02/06 14:53:12 INF [RTSP] [conn 127.0.0.1:50380] opened
    2024/02/06 14:53:12 INF [RTSP] [conn 127.0.0.1:50380] closed: no one is publishing to path 'stream'
    2024/02/06 14:53:12 ERR [path cam] [RTSP source] bad status code: 404 (Not Found)
    [video4linux2,v4l2 @ 0xaaab033a34a0] Stream #0: not enough frames to estimate rate; consider increasing probesize
    Input #0, video4linux2,v4l2, from '/dev/video0':
      Duration: N/A, start: 139250.093516, bitrate: N/A
      Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080, 1000k tbr, 1000k tbn
    Stream mapping:
      Stream #0:0 -> #0:0 (rawvideo (native) -> hevc (hevc_rkmpp_encoder))
    Press [q] to stop, [?] for help
    [vost#0:0/hevc_rkmpp_encoder @ 0xaaab033a9490] Frame rate very high for a muxer not efficiently supporting it.
    Please consider specifying a lower framerate, a different muxer or setting vsync/fps_mode to vfr
    [hevc_rkmpp_encoder @ 0xaaab033a97c0] Picture format is yuv420p.
    [hevc_rkmpp_encoder @ 0xaaab033a97c0] Rate Control mode is set to VBR
    [hevc_rkmpp_encoder @ 0xaaab033a97c0] Bitrate Target/Min/Max is set to 6000000/375000/6375000
    [hevc_rkmpp_encoder @ 0xaaab033a97c0] Profile is set to MAIN
    [hevc_rkmpp_encoder @ 0xaaab033a97c0] Level is set to 10
    [hevc_rkmpp_encoder @ 0xaaab033a97c0] Quality Min/Max is set to 50%(Quant=30) / 100%(Quant=10)
    2024/02/06 14:53:12 INF [RTSP] [conn 127.0.0.1:50384] opened
    2024/02/06 14:53:12 INF [RTSP] [session 95ccc229] created by 127.0.0.1:50384
    2024/02/06 14:53:12 INF [RTSP] [session 95ccc229] is publishing to path 'stream', 1 track (H265)
    Output #0, rtsp, to 'rtsp://0.0.0.0:8554/stream':
      Metadata:
        encoder         : Lavf60.3.100
      Stream #0:0: Video: hevc (Main), yuv420p(progressive), 1920x1080, q=2-31, 6000 kb/s, 1000k fps, 90k tbn
        Metadata:
          encoder         : Lavc60.3.100 hevc_rkmpp_encoder
    [hevc_rkmpp_encoder @ 0xaaab033a97c0] Reconfigured with w=1920, h=1080, format=yuv420p.
    [vost#0:0/hevc_rkmpp_encoder @ 0xaaab033a9490] More than 1000 frames duplicated  0x    
    [rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 0, current: 0; changing to 1. This may result in incorrect timestamps in the output file.
    [rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 1, current: 0; changing to 2. This may result in incorrect timestamps in the output file.
    [rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 2, current: 0; changing to 3. This may result in incorrect timestamps in the output file.
    [rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 3, current: 0; changing to 4. This may result in incorrect timestamps in the output file.
    [rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 4, current: 0; changing to 5. This may result in incorrect timestamps in the output file.
    [rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 5, current: 1; changing to 6. This may result in incorrect timestamps in the output file.
    [rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 6, current: 1; changing to 7. This may result in incorrect timestamps in the output file.
    [rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 7, current: 1; changing to 8. This may result in incorrect timestamps in the output file.
    [rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 8, current: 1; changing to 9. This may result in incorrect timestamps in the output file.
    [rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 9, current: 1; changing to 10. This may result in incorrect timestamps in the output file.
    [rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 10, current: 1; changing to 11. This may result in incorrect timestamps in the output file.

I still feel like I’m missing something. This seems like it shouldn’t be too difficult.

FWIW, I’m using an IMX219 camera. It’s from waveshare I think.

I tried your suggestions for the gst rtsp setup. It does run but errors out when attempting to access the stream:

radxa@radxa-zero3:~/rockchip/gstreamer/rtsp-server/gst-rtsp-server-1.18.4/build/examples$ sudo ./test-launch "( v4l2src device=/dev/video0 io-mode=dmabuf ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! mpph264enc ! rtph264pay name=pay0 pt=96 )"
stream ready at rtsp://127.0.0.1:8554/test
rga_api version 1.3.1_[11] (RGA is compiling with meson base: $PRODUCT_BASE)
 RgaBlit(1356) RGA_BLIT fail: Invalid argument RgaBlit(1357) RGA_BLIT fail: Invalid argument
fd-vir-phy-hnd-format[0, 0xffff81c880a0, (nil), (nil), 0]
rect[0, 0, 1920, 1080, 1920, 1088, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
fd-vir-phy-hnd-format[39, (nil), (nil), (nil), 0]
rect[0, 0, 1920, 1080, 1920, 1088, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
This output the user patamaters when rga call blit fail
 RgaBlit(1356) RGA_BLIT fail: Invalid argument RgaBlit(1357) RGA_BLIT fail: Invalid argument
fd-vir-phy-hnd-format[0, 0xffff8148f0a0, (nil), (nil), 0]
rect[0, 0, 1920, 1080, 1920, 1088, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
fd-vir-phy-hnd-format[43, (nil), (nil), (nil), 0]
rect[0, 0, 1920, 1080, 1920, 1088, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
This output the user patamaters when rga call blit fail