Debian RockPi4 SE - GPU with CPU via GStreamer for improved performance

Hi,

I have a question regarding the possibility of utilizing the GPU in parallel with the CPU through GStreamer. Our goal is to enhance performance. Currently, we are using GStreamer command-line tools for streaming (udpsink) over the network, and while it is functional, we are experiencing a slow frame rate of approximately 3fps on the target host with a resolution of 1344x968. Here is gstreamer command line we are using:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, width=1344, height=968 ! videoconvert ! x264enc tune=zerolatency bitrate=10000 speed-preset=superfast ! queue ! rtph264pay ! udpsink host=192.168.1.10 port=5000

We are using an OnSemi-Cam Sensor through the MIPI interface. The issue we are facing is that GStreamer is utilizing 100% of the CPUs rather than the GPU, resulting in the slow frame rate. Our target is to use the hardware video acceleration via gstreamer elements. So our idea would be something like this:

gst-launch-1.0 v4l2src device=/dev/video0 ! glimagesink name=gl ! videoconvert ! x264enc tune=zerolatency bitrate=10000 speed-preset=superfast ! queue ! rtph264pay ! udpsink host=192.168.2.255 port=5000

I would greatly appreciate it if you could advise us on how to install the suitable GPU drivers and to configure GStreamer to leverage the GPU’s capabilities and improve the overall performance.

Thanks,
Rami

Did you ever find a way? I have the same question. Did you look in to the rock MPP plugins for gstreamer?