Hey folks!
I’ve hit a wall with my testing using the new Zero 3E with a IMX219 camera. I’ve got the camera working (the cheese app displays the image). I can get the camera to output locally with gstreamer but the gstreamer rtsp client sink does not work. It throws a “generic error” message.
I poked at using ffmpeg to output the stream to rtsp as well but that ffmpeg doesn’t seem to like reading in /dev/video0.
Any suggestions?
Maybe try https://github.com/AlexxIT/go2rtc
PS what permissions are on /dev//dev/video0 as can you access in your use space?
Presume there is a group video and check if you are in it?
https://trac.ffmpeg.org/wiki/Capture/Webcam has some good start points
Thanks for the suggestions!
Love the go2rtc app! I gave that a go but it looks like it takes in ffmpeg and that’s what I can’t seem to get working with the camera.
When I run ffmpeg -f v4l2 -list_formats all -i /dev/video0 it comes back saying that it’s not a valid capture device. I feel like I’m missing something around how video output works with gstreamer but not with ffmpeg.
v4l2-ctl --list-devices is no different?
This is what I’m seeing with v4l2-ctl:
rkisp-statistics (platform: rkisp):
/dev/video7
/dev/video8
rkisp_mainpath (platform:rkisp-vir0):
/dev/video0
/dev/video1
/dev/video2
/dev/video3
/dev/video4
/dev/video5
/dev/video6
/dev/media0
And this is what I see with ffmpeg -f v4l2 -list_formats all -i /dev/video0:
ffmpeg -f v4l2 -list_formats all -i /dev/video0
ffmpeg version 4.3.6-0+deb11u1 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 10 (Debian 10.2.1-6)
configuration: --prefix=/usr --extra-version=0+deb11u1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
[video4linux2,v4l2 @ 0xaaaabb8ec220] Not a video capture device.
/dev/video0: No such deviceDunno been meaning to hook up a cam to the 03W just haven’t got round to it.
Confused as gstreamer works so you have the right overlay and guess your running debian desktop b6 being newest.
I guess you could also try the Ubuntu from https://github.com/Joshua-Riek/ubuntu-rockchip/releases/tag/v1.33 as even though the W think the ethernet is the insternal onboard that is still enabled ?
Any idea how to get the camera enabled under ubuntu?
In dtbo in /boot think the cam dtbo should be in there.
/boot/ubuntuEnv.txt add the overlays
On my Zero3W the only cam I have to hand is a Arducam IMX219 which when connected causes the board to heat up and boot to fail… ?
Well that’s concerning… I might hold off on ubuntu then.
Is there a way to get someone from Radxa to comment in this thread with ideas/suggestions?
@RadxaYuntian Might be able to help
Or ask on the radxa discord
The Debian image does the same must be my cam its IMX219 rooted out of a box of old Pi stuff so thought it would work.
You need a modified ffmpeg and gstreamer with support for MIPI cameras, see my comment in the Rock 5B section and how to build the RTSP gstreamer.
Yeap.
I might provide some tips on this as soon as i connect my wifi the old way. Wifi is up but i still receive tons of LOGINFO / LOGTRACE even with debugging disabled.
Wifi is up and connected, so here is the tip, the instructions still valid but make sure you have:
gst-inspect-1.0|grep -i rock
rkisp: rkisp: Gstreamer Plugin For Rockchip ISP Source
rockchipmpp: mpph264enc: Rockchip Mpp H264 Encoder
rockchipmpp: mpph265enc: Rockchip Mpp H265 Encoder
rockchipmpp: mppjpegdec: Rockchip’s MPP JPEG image decoder
rockchipmpp: mppjpegenc: Rockchip Mpp JPEG Encoder
rockchipmpp: mppvideodec: Rockchip’s MPP video decoder
rockchipmpp: mppvpxalphadecodebin: VP8/VP9 Alpha Decoder
Think mine is just wrong cam as guessing the arducam imx279 is not same pin out as Raspberry v2 cam…
@avaf I noticed you have done some fast C code with rknpu which is prob great for running on a rk3588 identify model.
Some code here and ignore the python but that method is that the simplest and movement tracker method https://pastebin.com/FYQ3Ltez is it worthwhile dropping the numpy/cv2 calls and doing in C?
I think that is about the fastest ignoring IR or wind, as they need a cleverer prob ML model.
I sort of like what frigate does and don’t as situation, target type and model can vary massively, which makes the embedded models and manner constraining.
I would prefer motion detector and identity model to be different instances and a choice of motion detectors and identity models.
Like Frigate is great as quick easy up and running in HA but it doesn’t pan the motion detection area and letterbox in Yolo input sizes based on tracked item size and many seem to be doing ‘Its a dog’ at 100fps…
If you are ever doing a frigate like app as saw some of your examples, as it would be great if you did.
I did attempt to follow this for ffmpeg and I ran into the following error:
ffmpeg version 65f90322bf Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 10 (Debian 10.2.1-6)
configuration: --enable-rkmpp --enable-version3 --enable-libdrm --enable-libx264 --enable-gpl
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
[video4linux2,v4l2 @ 0xaaaabfe444a0] ioctl(VIDIOC_G_INPUT): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0xaaaabfe444a0] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0xaaaabfe444a0] Time per frame unknown
[video4linux2,v4l2 @ 0xaaaabfe444a0] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 73917.489180, bitrate: N/A
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080, 1000k tbr, 1000k tbn
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> hevc (hevc_rkmpp_encoder))
Press [q] to stop, [?] for help
[vost#0:0/hevc_rkmpp_encoder @ 0xaaaabfe4a490] Frame rate very high for a muxer not efficiently supporting it.
Please consider specifying a lower framerate, a different muxer or setting vsync/fps_mode to vfr
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Picture format is yuv420p.
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Rate Control mode is set to VBR
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Bitrate Target/Min/Max is set to 6000000/375000/6375000
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Profile is set to MAIN
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Level is set to 10
[hevc_rkmpp_encoder @ 0xaaaabfe4a7c0] Quality Min/Max is set to 50%(Quant=30) / 100%(Quant=10)
[tcp @ 0xaaaabfe5ef50] Connection to tcp://192.168.42.226:8554?timeout=0 failed: Connection refused
[out#0/rtsp @ 0xaaaabfe47ac0] Could not write header (incorrect codec parameters ?): Connection refused
[vost#0:0/hevc_rkmpp_encoder @ 0xaaaabfe4a490] Error initializing output stream:
Conversion failed!I also tried the steps linked for mediamtx and that ended up with ffmpeg showing this:
The last lines just repeat basically.
radxa@radxa-zero3:~/FFmpeg/mediamtx$ sudo ffmpeg -re -f v4l2 -i /dev/video0 -c:v hevc -rc_mode 0 -level 30 -f rtsp rtsp://0.0.0.0:8554/stream
ffmpeg version 65f90322bf Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 10 (Debian 10.2.1-6)
configuration: --enable-rkmpp --enable-version3 --enable-libdrm --enable-libx264 --enable-gpl
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
[video4linux2,v4l2 @ 0xaaab033a34a0] ioctl(VIDIOC_G_INPUT): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0xaaab033a34a0] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0xaaab033a34a0] Time per frame unknown
2024/02/06 14:53:12 INF [RTSP] [conn 127.0.0.1:50380] opened
2024/02/06 14:53:12 INF [RTSP] [conn 127.0.0.1:50380] closed: no one is publishing to path 'stream'
2024/02/06 14:53:12 ERR [path cam] [RTSP source] bad status code: 404 (Not Found)
[video4linux2,v4l2 @ 0xaaab033a34a0] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 139250.093516, bitrate: N/A
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080, 1000k tbr, 1000k tbn
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> hevc (hevc_rkmpp_encoder))
Press [q] to stop, [?] for help
[vost#0:0/hevc_rkmpp_encoder @ 0xaaab033a9490] Frame rate very high for a muxer not efficiently supporting it.
Please consider specifying a lower framerate, a different muxer or setting vsync/fps_mode to vfr
[hevc_rkmpp_encoder @ 0xaaab033a97c0] Picture format is yuv420p.
[hevc_rkmpp_encoder @ 0xaaab033a97c0] Rate Control mode is set to VBR
[hevc_rkmpp_encoder @ 0xaaab033a97c0] Bitrate Target/Min/Max is set to 6000000/375000/6375000
[hevc_rkmpp_encoder @ 0xaaab033a97c0] Profile is set to MAIN
[hevc_rkmpp_encoder @ 0xaaab033a97c0] Level is set to 10
[hevc_rkmpp_encoder @ 0xaaab033a97c0] Quality Min/Max is set to 50%(Quant=30) / 100%(Quant=10)
2024/02/06 14:53:12 INF [RTSP] [conn 127.0.0.1:50384] opened
2024/02/06 14:53:12 INF [RTSP] [session 95ccc229] created by 127.0.0.1:50384
2024/02/06 14:53:12 INF [RTSP] [session 95ccc229] is publishing to path 'stream', 1 track (H265)
Output #0, rtsp, to 'rtsp://0.0.0.0:8554/stream':
Metadata:
encoder : Lavf60.3.100
Stream #0:0: Video: hevc (Main), yuv420p(progressive), 1920x1080, q=2-31, 6000 kb/s, 1000k fps, 90k tbn
Metadata:
encoder : Lavc60.3.100 hevc_rkmpp_encoder
[hevc_rkmpp_encoder @ 0xaaab033a97c0] Reconfigured with w=1920, h=1080, format=yuv420p.
[vost#0:0/hevc_rkmpp_encoder @ 0xaaab033a9490] More than 1000 frames duplicated 0x
[rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 0, current: 0; changing to 1. This may result in incorrect timestamps in the output file.
[rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 1, current: 0; changing to 2. This may result in incorrect timestamps in the output file.
[rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 2, current: 0; changing to 3. This may result in incorrect timestamps in the output file.
[rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 3, current: 0; changing to 4. This may result in incorrect timestamps in the output file.
[rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 4, current: 0; changing to 5. This may result in incorrect timestamps in the output file.
[rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 5, current: 1; changing to 6. This may result in incorrect timestamps in the output file.
[rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 6, current: 1; changing to 7. This may result in incorrect timestamps in the output file.
[rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 7, current: 1; changing to 8. This may result in incorrect timestamps in the output file.
[rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 8, current: 1; changing to 9. This may result in incorrect timestamps in the output file.
[rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 9, current: 1; changing to 10. This may result in incorrect timestamps in the output file.
[rtsp @ 0xaaab033a6b90] Non-monotonous DTS in output stream 0:0; previous: 10, current: 1; changing to 11. This may result in incorrect timestamps in the output file.
I still feel like I’m missing something. This seems like it shouldn’t be too difficult.
FWIW, I’m using an IMX219 camera. It’s from waveshare I think.
I tried your suggestions for the gst rtsp setup. It does run but errors out when attempting to access the stream:
radxa@radxa-zero3:~/rockchip/gstreamer/rtsp-server/gst-rtsp-server-1.18.4/build/examples$ sudo ./test-launch "( v4l2src device=/dev/video0 io-mode=dmabuf ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! mpph264enc ! rtph264pay name=pay0 pt=96 )"
stream ready at rtsp://127.0.0.1:8554/test
rga_api version 1.3.1_[11] (RGA is compiling with meson base: $PRODUCT_BASE)
RgaBlit(1356) RGA_BLIT fail: Invalid argument RgaBlit(1357) RGA_BLIT fail: Invalid argument
fd-vir-phy-hnd-format[0, 0xffff81c880a0, (nil), (nil), 0]
rect[0, 0, 1920, 1080, 1920, 1088, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
fd-vir-phy-hnd-format[39, (nil), (nil), (nil), 0]
rect[0, 0, 1920, 1080, 1920, 1088, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
This output the user patamaters when rga call blit fail
RgaBlit(1356) RGA_BLIT fail: Invalid argument RgaBlit(1357) RGA_BLIT fail: Invalid argument
fd-vir-phy-hnd-format[0, 0xffff8148f0a0, (nil), (nil), 0]
rect[0, 0, 1920, 1080, 1920, 1088, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
fd-vir-phy-hnd-format[43, (nil), (nil), (nil), 0]
rect[0, 0, 1920, 1080, 1920, 1088, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
This output the user patamaters when rga call blit fail