SDL_ROCKCHIP is used only with KMSDRM, if you are with X11 or Wayland there will be no problem.
Object detection with NPU - h264 streams - h264 camera - rtsp
1 Like
Hello avaf! I had left this on the side for some time but I am picking it up again
I get this error when trying to run testegles2 on X11 (and all platforms really). I used the SDL3 commit you gave me
arm_release_ver of this libmali is 'g2p0-01eac0', rk_so_ver is '4'.
INFO: SDF is enabled
INFO: Load image file: /tests/testgles2_sdf_img_sdf.bmp
INFO: Screen bpp: 32
INFO:
INFO: Vendor : (null)
INFO: Renderer : (null)
INFO: Version : (null)
INFO: Extensions : (null)
INFO:
INFO: SDL_GL_RED_SIZE: requested 5, got 0
INFO: SDL_GL_GREEN_SIZE: requested 5, got 0
INFO: SDL_GL_BLUE_SIZE: requested 5, got 0
INFO: SDL_GL_DEPTH_SIZE: requested 16, got 0
INFO: Shader compilation failed:
Weird right? I can run glmark2-es2 perfectly fine. I even played Chocolate Doom on old SDL2
I should probably mention this as it may be important as well:
testgles2 first complained about libGLESv2.so.2 and libEGL.so.1 being missing. So I just made symbolic links to libGLESv2.so and libEGL.so which are present on my board. This may have been wrong but what else could I do?
Also I just tried the last commit of SDL3 and still same issue
Please, note i am using libmali. libmali has a feature that wraps libEGLESv2 and libEGL automagically.
The drawback of this is if libmali version changes, it breaks everything and you need to re-build all deb packages again with the new libmali version.
I think you donāt need to worry about this if you use panfrost. My guess is you are using panfrost and you need to link explicitly. That means latest SDL2 will compile and work, forget about the SDL_ROCKCHIP hack.
See how it is linkedoin RK3588 for example (SDL2), same as RK356x:
ldd /usr/lib/aarch64-linux-gnu/installed-tests/SDL2/testgles2
linux-vdso.so.1 (0x0000007fbef48000)
libSDL2-2.0.so.0 => /lib/aarch64-linux-gnu/libSDL2-2.0.so.0 (0x0000007fbed30000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007fbeb80000)
/lib/ld-linux-aarch64.so.1 (0x0000007fbef0f000)
libasound.so.2 => /lib/aarch64-linux-gnu/libasound.so.2 (0x0000007fbea60000)
libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007fbe9c0000)
libpulse.so.0 => /lib/aarch64-linux-gnu/libpulse.so.0 (0x0000007fbe960000)
libX11.so.6 => /lib/aarch64-linux-gnu/libX11.so.6 (0x0000007fbe810000)
libXext.so.6 => /lib/aarch64-linux-gnu/libXext.so.6 (0x0000007fbe7e0000)
libXcursor.so.1 => /lib/aarch64-linux-gnu/libXcursor.so.1 (0x0000007fbe7c0000)
libXinerama.so.1 => /lib/aarch64-linux-gnu/libXinerama.so.1 (0x0000007fbe7a0000)
libXi.so.6 => /lib/aarch64-linux-gnu/libXi.so.6 (0x0000007fbe770000)
libXfixes.so.3 => /lib/aarch64-linux-gnu/libXfixes.so.3 (0x0000007fbe750000)
libXrandr.so.2 => /lib/aarch64-linux-gnu/libXrandr.so.2 (0x0000007fbe730000)
libXss.so.1 => /lib/aarch64-linux-gnu/libXss.so.1 (0x0000007fbe710000)
libXxf86vm.so.1 => /lib/aarch64-linux-gnu/libXxf86vm.so.1 (0x0000007fbe6f0000)
libdrm.so.2 => /lib/aarch64-linux-gnu/libdrm.so.2 (0x0000007fbe6c0000)
libgbm.so.1 => /usr/lib/aarch64-linux-gnu/mali/libgbm.so.1 (0x0000007fbe6a0000)
libwayland-egl.so.1 => /lib/aarch64-linux-gnu/libwayland-egl.so.1 (0x0000007fbe680000)
libwayland-client.so.0 => /lib/aarch64-linux-gnu/libwayland-client.so.0 (0x0000007fbe660000)
libwayland-cursor.so.0 => /lib/aarch64-linux-gnu/libwayland-cursor.so.0 (0x0000007fbe640000)
libxkbcommon.so.0 => /lib/aarch64-linux-gnu/libxkbcommon.so.0 (0x0000007fbe5e0000)
libdecor-0.so.0 => /lib/aarch64-linux-gnu/libdecor-0.so.0 (0x0000007fbe5c0000)
libpulsecommon-15.99.so => /usr/lib/aarch64-linux-gnu/pulseaudio/libpulsecommon-15.99.so (0x0000007fbe530000)
libdbus-1.so.3 => /lib/aarch64-linux-gnu/libdbus-1.so.3 (0x0000007fbe4d0000)
libxcb.so.1 => /lib/aarch64-linux-gnu/libxcb.so.1 (0x0000007fbe490000)
libXrender.so.1 => /lib/aarch64-linux-gnu/libXrender.so.1 (0x0000007fbe470000)
libmali-hook.so.1 => /lib/aarch64-linux-gnu/libmali-hook.so.1 (0x0000007fbe450000)
libmali.so.1 => /lib/aarch64-linux-gnu/libmali.so.1 (0x0000007fb7ac0000)
libffi.so.8 => /lib/aarch64-linux-gnu/libffi.so.8 (0x0000007fb7aa0000)
libsndfile.so.1 => /lib/aarch64-linux-gnu/libsndfile.so.1 (0x0000007fb7a10000)
libX11-xcb.so.1 => /lib/aarch64-linux-gnu/libX11-xcb.so.1 (0x0000007fb79f0000)
libsystemd.so.0 => /lib/aarch64-linux-gnu/libsystemd.so.0 (0x0000007fb7910000)
libasyncns.so.0 => /lib/aarch64-linux-gnu/libasyncns.so.0 (0x0000007fb78f0000)
libapparmor.so.1 => /lib/aarch64-linux-gnu/libapparmor.so.1 (0x0000007fb78c0000)
libXau.so.6 => /lib/aarch64-linux-gnu/libXau.so.6 (0x0000007fb78a0000)
libXdmcp.so.6 => /lib/aarch64-linux-gnu/libXdmcp.so.6 (0x0000007fb7880000)
libxcb-dri2.so.0 => /lib/aarch64-linux-gnu/libxcb-dri2.so.0 (0x0000007fb7860000)
libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007fb7840000)
libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007fb7820000)
libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007fb75f0000)
libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007fb75c0000)
libFLAC.so.8 => /lib/aarch64-linux-gnu/libFLAC.so.8 (0x0000007fb7570000)
libvorbis.so.0 => /lib/aarch64-linux-gnu/libvorbis.so.0 (0x0000007fb7530000)
libvorbisenc.so.2 => /lib/aarch64-linux-gnu/libvorbisenc.so.2 (0x0000007fb7470000)
libopus.so.0 => /lib/aarch64-linux-gnu/libopus.so.0 (0x0000007fb7400000)
libogg.so.0 => /lib/aarch64-linux-gnu/libogg.so.0 (0x0000007fb73e0000)
liblzma.so.5 => /lib/aarch64-linux-gnu/liblzma.so.5 (0x0000007fb73a0000)
libzstd.so.1 => /lib/aarch64-linux-gnu/libzstd.so.1 (0x0000007fb72d0000)
liblz4.so.1 => /lib/aarch64-linux-gnu/liblz4.so.1 (0x0000007fb72a0000)
libcap.so.2 => /lib/aarch64-linux-gnu/libcap.so.2 (0x0000007fb7280000)
libgcrypt.so.20 => /lib/aarch64-linux-gnu/libgcrypt.so.20 (0x0000007fb7190000)
libbsd.so.0 => /lib/aarch64-linux-gnu/libbsd.so.0 (0x0000007fb7160000)
libgpg-error.so.0 => /lib/aarch64-linux-gnu/libgpg-error.so.0 (0x0000007fb7120000)
libmd.so.0 => /lib/aarch64-linux-gnu/libmd.so.0 (0x0000007fb7100000)
See my other post on ff-rknn which uses SDL2 instead of SDL3
Hey Avaf I saw your other thread. Iāve done a lot of work speeding up the NPU on my Rk3566. I am sure we could collaborate and achieve greatness 
I have around 25fps~ currently without SDL2/SDL3. Just OpenCV and Rkximagesink (Gstreamer) for display. I am pretty sure I am being bottlenecked by OpenCV as my inference times were around 0.02ms
I will share some videos later and some code if thatās helpful to you
If you invoke gst from the main program and not indirectly by OpenCV, be my guest.
Looks like Drawing Rects and Text is to blame.
Youāre probably giving me too much credit, I am invoking gst from OpenCVās imwrite function. But it still worked out great. Iāll be uploading the source code to Github today
Also some other things I noticed was the input resolution to OpenCV having a big impact on the framerate. For example if I specify 1024x600 on the gst input pipeline ( āvideo/x-raw, width=1024, height=600ā ) I get near perfect framerate, this resolution is actually almost perfect for my use case because my RK3566 device is kind of like a tablet ( https://imgur.com/a/86vrNDY ) and thatās the max screen resolution so there is no noticeable more quality if I increase it anymore than that. But if I try higher, like 1920x1080, then framerate bombs. And I still would like to have 1920x1080 for RTSP streaming and file storage. I am using librga for resizing, so that should not be at fault, I think OpenCV is doing a CPU-intensive task when I try a high res input. Actually I confirmed this by looking at the CPU usage, at 1920x1080 CPU usage is at maximum.
Hey avaf, to elaborate on that, it seems I had some very weird mix between libmali and mesa3d libraries. You kept bringing up Mesa3D/Panfrost so I thought maybe you noticed something I didnāt
I am using Buildroot so I probably enabled mesa3d along the way and forgot to delete those files
I deleted all of my libGLES and libEGL files and then proceeded to replace them with symbolic links to my Bifrost Libmali driver and it now works!
So all is good
Hello Alexander, I am having some trouble running your code. I get this error everytime:
[h264 @ 0x557b40fa90] decode_slice_header error
[h264 @ 0x557b40fa90] no frame!
Error sending a packet for decoding
If I run ffmpeg manually via cli it works perfectly fine. I also did try many different .h264 files and even a rtsp stream. Same error
I am using ffmpeg 7.0 from https://github.com/nyanmisaka/ffmpeg-rockchip , along with jellyfin-mpp and jellyfin-rga as recommended
Do you have any idea or suggestion? Did you use a different version of any of the aforementioned packages?
I used the master branch.
Just used master branch and still same issue. Will be looking into it
Just tried out the examples from FFMPEG and the decode_video.c example seems to work fine. Thatās mpeg1 though
If I am not able to fix it Iāll probably try to bypass h264 altogether and just try out the raw stream from camera, or try another codec like mpeg1. Whatās really weird to me is that I donāt get any error by doing h264 decoding via cli
I also do not have libsndio which was a build requirement for ff-rknn I disabled during compilation. I donāt think thatās related as, according to the internet, libsndio is a library for MIDI audio and hardware? But at this point I may consider installing that first and rebuilding the app just in case