Will have board of Roc 2A or 2F support of hardware accelerated video decoding on linux?
By Gstreamer or FFmpeg.
ROCK 2A/F Video accelerate
You can check this post. Rock 2A/2F uses RK3528, which is also supported by MPP and RGA.
Thanx, works but only for yuv pixel format.
I need video player in java application.
I try gstreamer, works but after add caps acceleration is disabled.
video/x-raw,format=BGRx
or
video/x-raw,format=xRGB
By ffmpeg is failing because of “-pix_fmt rgb24”
ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -i ~/input.mp4 \
-c:v h264_rkmpp -pix_fmt rgb24 -f image2pipe -
ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -i ~/input.mp4 \
-c:v hvec_rkmpp -pix_fmt rgb24 -f image2pipe -
Question:
How can i use hardware acceleration + read frames as RGB or BGR
It sounds like you want to convert the decoded YUV image to RGB24 and output it in rawvideo muxer. If so, then you don’t need an encoder -c:v
, but instead use the RGA filter.
ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -i ~/input.mp4 -an -sn \
-vf scale_rkrga=format=rgb24,hwmap,format=rgb24 -f rawvideo -
FHD video (1920x1080) of 30 FPS
-> decoding at 55FPS
That is great
But
UHD video (3840x2160) of 30 FPS
-> decoding at 14FPS
That can not be used.
Any sugesstions?
ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -i ~/sample.mp4 -sn -an \
-vf scale_rkrga=format=rgb24,hwmap,format=rgb24 -f rawvideo - | cat > /dev/null
ffmpeg version b81c3bf Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 10 (Debian 10.2.1-6)
configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-libdrm --enable-rkmpp --enable-rkrga --enable-shared --enable-nonfree --enable-zlib
libavutil 58. 29.100 / 58. 29.100
libavcodec 60. 31.102 / 60. 31.102
libavformat 60. 16.100 / 60. 16.100
libavdevice 60. 3.100 / 60. 3.100
libavfilter 9. 12.100 / 9. 12.100
libswscale 7. 5.100 / 7. 5.100
libswresample 4. 12.100 / 4. 12.100
libpostproc 57. 3.100 / 57. 3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/rock/sample.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2mp41
encoder : Lavf58.76.100
Duration: 00:00:08.04, start: 0.000000, bitrate: 2689 kb/s
Stream #0:0[0x1](eng): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv, bt709, progressive), 3840x2160, 2683 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
Metadata:
handler_name : ?Mainconcept Video Media Handler
vendor_id : [0][0][0][0]
Stream mapping:
Stream #0:0 -> #0:0 (hevc (hevc_rkmpp) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, rawvideo, to 'pipe:':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2mp41
encoder : Lavf60.16.100
Stream #0:0(eng): Video: rawvideo (RGB[24] / 0x18424752), rgb24(pc, bt709/unknown/unknown, progressive), 3840x2160, q=2-31, 5966001 kb/s, 29.97 fps, 29.97 tbn (default)
Metadata:
handler_name : ?Mainconcept Video Media Handler
vendor_id : [0][0][0][0]
encoder : Lavc60.31.102 rawvideo
[out#0/rawvideo @ 0xaaaae5bbc110] video:5856300kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
frame= 241 fps= 14 q=-0.0 Lsize= 5856300kB time=00:00:08.00 bitrate=5990860.3kbits/s speed=0.472x
Because using ffmpeg this way is just a demo target for debugging.
Muxer -f rawvideo -
means 3840x2160x3x8 bytes memory copy per second, you should access these functions through libav interface to avoid copying.
How can i do that?
I mean, any preffered binding for java / kotlin ?
I am kotlin programmer + i can some things put together in shell , but C / C++ only basics.
Connecting the hardware decoder and video renderer and achieving 0-copy is a pretty low-level task. I’m not sure how to do it in java either. Since you are familiar with kotlin/java, why not install the Android image and use MediaCodec there, Android should take care of everything.
Option which i know
- FFmpeg (not fast enought because pipeline)
- Gstreamer try multiple variants but RGB by RGA seems to be not working
- full speed YUV
gst-launch-1.0 filesrc location=$src ! parsebin ! mppvideodec ! appsink
- 0.5 FPS RGB
gst-launch-1.0 filesrc location=$src ! qtdemux ! mppvideodec ! videoconvert ! 'video/x-raw,format=RGB' ! appsink
- full speed YUV
- OpenCV compilation everitime fail on avcodec_get_context_defaults3
- Rock 5B working, no HW acceleration but cpu is fast enought
Android can not be used because infinite problems with display setup on LED Board, TV …
There is indeed a ffmpeg4j wrapper project may be useful for you.
But you have to handle avcodec, avfilter etc manually.