Updated FFMpeg with mpp

#42

Checked dmesg, rock5b has 2 video decoder cores or am I missing something?

[80946.788050] mpp_rkvdec2 fdc48100.rkvdec-core: resetting…
[80946.788129] mpp_rkvdec2 fdc48100.rkvdec-core: reset done
[80946.788134] mpp_rkvdec2 fdc38100.rkvdec-core: resetting…
[80946.788187] mpp_rkvdec2 fdc38100.rkvdec-core: reset done
[80946.979655] mpp_rkvdec2 fdc48100.rkvdec-core: resetting…
[80946.979747] mpp_rkvdec2 fdc48100.rkvdec-core: reset done
[80946.979752] mpp_rkvdec2 fdc38100.rkvdec-core: resetting…
[80946.979823] mpp_rkvdec2 fdc38100.rkvdec-core: reset done
[80947.033336] mpp_rkvdec2 fdc48100.rkvdec-core: resetting…
[80947.033397] mpp_rkvdec2 fdc48100.rkvdec-core: reset done
[80947.033400] mpp_rkvdec2 fdc38100.rkvdec-core: resetting…
[80947.033450] mpp_rkvdec2 fdc38100.rkvdec-core: reset done

#43 
top - 22:42:48 up 22:48,  3 users,  load average: 9.68, 8.98, 8.54
Tasks: 289 total,   5 running, 284 sleeping,   0 stopped,   0 zombie
%Cpu(s): 35.1 us, 11.3 sy,  0.0 ni, 52.5 id,  0.0 wa,  0.0 hi,  1.1 si,  0.0 st
MiB Mem :   7691.9 total,   1443.5 free,   1546.9 used,   4701.4 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.   6070.0 avail Mem 

PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                     
 624407 root      20   0 1343264 125860  21420 S 114.3   1.6  21:51.15 ffmpeg                                                      
 624523 root      20   0 1197236 126204  19584 S  89.4   1.6  19:40.99 ffmpeg                                                      
 617288 root      20   0 1568444 155012  17004 R  20.9   2.0  13:00.66 frigate.process                                             
 617295 root      20   0 1334180 151528   8780 S  18.9   1.9   8:41.81 frigate.capture                                             
 617299 root      20   0 1364556 140908   8688 S  18.9   1.8   9:14.33 frigate.capture                                             
 617292 root      20   0 1558868 148332  17004 S  16.6   1.9   4:55.23 frigate.process                                             
 617289 root      20   0 1602600 188932  17004 R  14.3   2.4   9:01.81 frigate.process                                             
 617255 root      20   0 1692884 122324  12476 S  13.0   1.6   5:14.51 frigate.output                                              
 617313 root      20   0 1144564  41308  19472 S  10.0   0.5   3:02.44 ffmpeg                                                      
 617314 root      20   0 1145544  43688  19472 S  10.0   0.6   3:03.62 ffmpeg                                                      
 616890 root      20   0 2300004 314188  54900 S   9.6   4.0   4:01.43 python3                                                     
 617253 root      20   0 1238228 134368  22248 S   7.3   1.7   5:01.70 frigate.detecto                                             
 617290 root      20   0 1555228 144704  17004 R   7.0   1.8   3:20.56 frigate.process                                             
 624414 root      20   0  262444  24816  18480 S   3.3   0.3   0:40.90 ffmpeg                                                      
 617244 root      20   0 1048480 181148  20384 S   2.7   2.3   0:40.68 frigate.recordi                                             
 617303 root      20   0 1334516 135508   8652 S   2.7   1.7   0:51.38 frigate.capture                                             
 617307 root      20   0 1334856 135376   8584 S   2.3   1.7   0:50.97 frigate.capture

and this is the camera with the highest cpu usage… I replace pipe with /tmp/tmp_frigate_rawvideo

 ffmpeg -threads 2 -hwaccel drm -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -analyzeduration 1000M -probesize 1000M -fflags +genpts+discardcorrupt -use_wallclock_as_timestamps 1 -i "rtsp://uername:pasword@hostname:port/the?1080p&stream" -r 25 -vf fps=25,scale=1920:1080 -movflags frag_keyframe+empty_moov -f rawvideo -pix_fmt yuv420p /tmp/tmp_frigate_rawvideo version 656cd334fa Copyright (c) 2000-2023 the FFmpeg developers
      built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
      configuration: --enable-rkmpp --extra-cflags='-I/usr/local/include -march=armv8-a+crc' --extra-ldflags=-L/usr/local/lib --extra-libs='-lpthread -lm -latomic' --arch=arm64 --enable-gmp --enable-gpl --enable-libaom --enable-libass --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libssh --enable-libvorbis --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-nonfree --enable-version3 --target-os=linux --enable-pthreads --enable-openssl
      libavutil      58.  2.100 / 58.  2.100
      libavcodec     60.  3.100 / 60.  3.100
      libavformat    60.  3.100 / 60.  3.100
      libavdevice    60.  1.100 / 60.  1.100
      libavfilter     9.  3.100 /  9.  3.100
      libswscale      7.  1.100 /  7.  1.100
      libswresample   4. 10.100 /  4. 10.100
      libpostproc    57.  1.100 / 57.  1.100
    Input #0, rtsp, from 'rtsp://uername:pasword@hostname:port/the?1080p&stream':
      Metadata:
        title           : Media Server
      Duration: N/A, start: 1699830079.675756, bitrate: N/A
      Stream #0:0: Video: hevc (Main), yuvj420p(pc, bt470bg/bt470bg/smpte170m), 1920x1080, 25 fps, 25 tbr, 90k tbn
    File '/tmp/tmp_frigate_rawvideo' already exists. Overwrite? [y/N] y
    Stream mapping:
      Stream #0:0 -> #0:0 (hevc (hevc_rkmpp_decoder) -> rawvideo (native))
    Press [q] to stop, [?] for help
    [hevc_rkmpp_decoder @ 0x55ce2ba450] Pixfmt (yuyv422), Conversion (nv12->yuyv422)
    rga_api version 1.9.3_[0]
    Output #0, rawvideo, to '/tmp/tmp_frigate_rawvideo':
      Metadata:
        title           : Media Server
        encoder         : Lavf60.3.100
      Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(tv, bt470bg/bt470bg/smpte170m, progressive), 1920x1080, q=2-31, 622080 kb/s, 25 fps, 25 tbn
        Metadata:
          encoder         : Lavc60.3.100 rawvideo
    [rtsp @ 0x55ce27c2c0] max delay reached. need to consume packet bitrate=  -0.0kbits/s speed=N/A    
    [rtsp @ 0x55ce27c2c0] RTP: missed 377 packets
    frame=  260 fps= 24 q=-0.0 Lsize=  789750kB time=00:00:14.44 bitrate=448035.5kbits/s speed=1.35x    
    video:789750kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
    Exiting normally, received signal 2.

anything I can do to make it use less cpu?

#44

Why threads 2?, why do you alloc drm device? Why do you vf_scale?i cant say i got what this ffmpeg chain is doing

#45

this ffmpeg is generated by frigate based on this config.yml from kcembrey

#46

essentially this should be enough.

FFMPEG_RKMPP_PIXFMT=yuv420p ffmpeg -i "rtsp://uername:pasword@hostname:port/the?1080p&stream" -f rawvideo -pix_fmt yuv420p /tmp/tmp_frigate_rawvideo

if you need further tuning you can add those extra arguments to process timestamps and snyhronisation.

FFMPEG_RKMPP_PIXFMT=yuv420p might not even be necessary

but do not use -vf fps=25,scale=1920:1080 this will cause software scaling and use lots of cpu

1 Like
#47

any hardware accelerated scaler on rock5b that ffmpeg can use?

#48

@nyanmisaka has a pre alpha in below link

Usage is documented here

He has a hardware scaler, but i am not sure he supports this branch yet, you can experiment it.

1 Like
#49

Here’s another branch for AFBC + RGA async. It’s not finalized yet due to the unstable RGA driver.

2 Likes
#50

experimenting… so far I’ve come to the conclusion I can’t do

-c:v copy and -vf scale_rkrga=w=1920:h=1080:format=yuv420p:core=rga3_core0+rga3_core1+rga2_core0

#51

lol, this is how i feel for the last 1 year.
1 remark, dont assign to rga3 cores when using planar formats (the ones ending with p, ie: yuv420p)
better not to force any core, it is easier this way, rga is a b**ch

#52

My filters have been designed to automatically set the RGA core internally. There is no need to set RGA core manually unless you know what you are doing.

1 Like
#53

Hi @boogiepop,

What is the current support pixel formats and resolutions for hevc & mjpeg encoders ?
is yuv444p pixel format supported for 4k ?

also -
is there slicing support ? and how do control the number of encoding threads ?

thanks ahead,

#54

It accepts as input, it encodes in 420 format. Because simply encoder hardware does not support it.

no the hardware is frame based (to my knowledhe), however there is something called split mode which reduces latency, it could be may be what you are looking, but i have not implemented it yet

you do not need to, because encoder hardware is single core, even if there were multiple cores of hardware encoders, the underlying mpp api has its own internal threads to handle this parallelism, so ffmpeg threads are not necessary, at least from todays POV.

#55

thanks for your prompt answer.
and regarding mjpeg encoder, can it encode in 444 ? datasheet doesn’t mention specifically the capability (just the throughput).

#56

if you mean, encode to, then no, encode from yes, and that is to 420 format and done thorugh rga+libyuv conversion (image is subsampled in a lossy way 444->420).

#57

TRM says no.

image
image.png917×713 107 KB

2 Likes
#58

i wonder how does internal encoder handles only 420 if the input were rgb888 or some funny format. Is it using RGA in a way, may be it is possible to exploit this part of hardware for other tasks.

#59

Hello,
Since some time ffmpeg-mpp, ffmpeg4.4-mpp expired from AUR.
iI this due to a new ffmpeg version ?
Can I get ffmpeg-mpp in AUR back ?
I feeel that kodi is not anylonger running that smooth.
Kind regards
Uli

1 Like
#60

It is provided here: https://github.com/7Ji/archrepo

I removed the aur packages due to some useless drama. This version of ffmpeg is the most compatible version. Ffmpeg-rockchip will evetually replace it but it is too soon for that.

1 Like
#61

Thanks for your help.
It works again.
regards
Uli

1 Like