i figured out how to use gstreamer with hw decoding on Ubuntu. The only issue i had is with hevc 10bit, surprisingly ffplay has no issues playing it, but the way it displays the video on screen is a bit annoying, it tries to scale the 1920x1080 video to something like 1920x9??.
Playing Big_Buck_Bunny_1080_10s_10MB.mp4 (H264)
gst-launch-1.0 filesrc location=Big_Buck_Bunny_1080_10s_10MB.mp4 ! qtdemux ! h264parse ! mppvideodec ! glimagesink
Setting pipeline to PAUSED ...
arm_release_ver of this libmali is 'g6p0-01eac0', rk_so_ver is '5'.
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayGBM\)\ gldisplaygbm0";
Pipeline is PREROLLED ...0 %)
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:10.000724467
Setting pipeline to NULL ...
Freeing pipeline ...
CPU usage:
Playing jellyfish-30-mbps-hd-hevc.mkv (HEVC)
gst-launch-1.0 filesrc location=jellyfish-30-mbps-hd-hevc.mkv ! matroskademux ! h265parse ! mppvideodec ! glimagesink
Setting pipeline to PAUSED ...
arm_release_ver of this libmali is 'g6p0-01eac0', rk_so_ver is '5'.
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayGBM\)\ gldisplaygbm0";
Pipeline is PREROLLED ...0 %)
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:30.031330520
Setting pipeline to NULL ...
Freeing pipeline ...
CPU usage:
Playing tos-4096x1720-tiles.mkv (4K) on 1920x1080
gst-launch-1.0 filesrc location=tos-4096x1720-tiles.mkv ! matroskademux ! h265parse ! mppvideodec ! glimagesink
Setting pipeline to PAUSED ...
arm_release_ver of this libmali is 'g6p0-01eac0', rk_so_ver is '5'.
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayGBM\)\ gldisplaygbm0";
Pipeline is PREROLLED ...0 %)
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:12:14.167728212
Setting pipeline to NULL ...
Freeing pipeline ...
CPU usage:
Now i need to find out how to replicate it to Debianā¦
Reminder: If you have a board with dram > 4 GB you need to wait for the librga fix.
UPDATE
Cross-compiled the kernel for Debian 11.5 Image and i can confirm that building the kernel natively breaks rkdvec2 (or mpp, whatever) , FFmpeg works fine but i still have a lot of dropped frames with gstreamer. Need to find where i got the correct plugin that works on Ubuntu.
Playing jellyfish-20-mbps-hd-hevc-10bit.mkv
CPU usage
UPDATE 2
Finally, gstreamer is working with Debian 11.5 X11. The missing bits are:
- HEVC 10bit does not work
- Works only for boards with 4 GB RAM (we need to wait for the new SDK or if Radxa can handle the source code for the librga which runs on boards with 8 GB or more)
Thatās all Folks!
Playing jellyfish-5-mbps-hd-hevc.mkv (HEVC - H265)
sudo GST_GL_API=gles2 GST_GL_PLATFORM=egl gst-launch-1.0 filesrc location=jellyfish-5-mbps-hd-hevc.mkv ! matroskademux ! h265parse ! mppvideodec ! glimagesink
CPU Usage:
UPDATE 3
Play youtube videos, sometimes it gets stuck or breaks the pipe, experimental.
-
Install youtube-dl
sudo curl -L https://yt-dl.org/downloads/latest/youtube-dl -o /usr/local/bin/youtube-dl
sudo chmod a+rx /usr/local/bin/youtube-dl
-
Get a real youtube link, for example: https://youtu.be/HSE_X3XUoiw?list=RDMMHSE_X3XUoiw
-
play youtube with hw accel
sudo /usr/local/bin/youtube-dl https://www.youtube.com/watch?v=HSE_X3XUoiw -o - | GST_GL_API=gles2 GST_GL_PLATFORM=egl gst-launch-1.0 filesrc location=/dev/stdin ! qtdemux ! h264parse ! mppvideodec ! glimagesink
You can also try with ffplay, i get better results plus audio until i find a way to play video + audio with gstreamer.
sudo /usr/local/bin/youtube-dl https://www.youtube.com/watch?v=HSE_X3XUoiw -o - | ffplay - -autoexit -loglevel quiet
Important: run sudo ls before firing the youtube-dl command or it get stuck waiting for the password
UPDATE 4
If you want to play with more than one camera on Rock 5B, you can add an extra H264 USB Camera (30 fps).
Tested with gstreamer and ffmpeg.
CSI camera is on /dev/video11 and H264 USB Camera is on /dev/video23 for this test. Attached to USB 3.0
sudo GST_GL_API=gles2 GST_GL_PLATFORM=egl gst-launch-1.0 v4l2src device=/dev/video23 ! video/x-h264,width=1920,height=1080,framerate=30/1 ! h264parse ! mppvideodec ! glimagesink
CPU usage:
the LAST one, testing encode and decode:
gst-launch-1.0 filesrc location=Big_Buck_Bunny_1080_10s_10MB.mp4 ! qtdemux ! h264parse ! mppvideodec ! video/x-raw,formate=NV12 ! mpph264enc ! avimux ! filesink location=Big_Buck_Bunny_1080_10s_10MB.avi
Thatās it.
