How to use Radxa Camera 4K

ROCK 5 Series
2

You asked for, here is the shortest and most intense course for beginners.

  • Camera
  • Environment
  • Working with the camera, examples
  • References

1. Camera

The camera has a Sony IMX-415 sensor, 4K (8MP), Low-light sensitivity, and fixed focus with MIPI interface. The current driver is able to get a picture (frame) of 3840x2160 pixels (high resolution) in NV12 format.

2. Environment

Debian 11 is used here, with the camera_engine_rkaiq installed and running, there should be a file in your setups like this one or a similar file:
/etc/iqfiles/imx415_CMK-OT2022-PX1_IR0147-50IRC-8M-F20.json
Note: You might have a newer file from radxa:
/etc/iqfiles/imx415_RADXA-CAMERA-4K_DEFAULT.json

Basically, when you take a picture, the camera engine does some pre-processing using that appropriate file, helping the camera adjust colors, light, and exposure. It runs in the background in auto mode. Without the camera engine, you get a dark and green image.

Gstreamer is the software that can work with the camera, gstreamer works with plugins and you need a special plugin called rockchipmpp that can encode the frames into H264/H265 with hardware acceleration. If you want to save a video in an MP4 container you will use the plugin. Other software that usually works with USB cameras, like FFmpeg or OpenCV, will not work and it will need some modifications in order to work properly.

Before we work with the camera, we check if we have the correct setup.

Camera nodes (11 will be our device):
When Linux detects a camera it creates a device node for that camera. Our camera will be in /dev/video11

rock@rock5b:~/rockchip/camera/python$ ls /dev/video*
/dev/video0 /dev/video13 /dev/video18 /dev/video22 /dev/video5
/dev/video1 /dev/video14 /dev/video19 /dev/video23 /dev/video6
/dev/video10 /dev/video15 /dev/video2 /dev/video24 /dev/video7
/dev/video11 /dev/video16 /dev/video20 /dev/video3 /dev/video8
/dev/video12 /dev/video17 /dev/video21 /dev/video4 /dev/video9

Gstreamer plugin:

Rock@rock5b:~/rockchip/camera/python$ gst-inspect-1.0 |grep mpp
rockchipmpp: mppjpegdec: Rockchip’s MPP JPEG image decoder
rockchipmpp: mppvideodec: Rockchip’s MPP video decoder
rockchipmpp: mppjpegenc: Rockchip Mpp JPEG Encoder
rockchipmpp: mpph265enc: Rockchip Mpp H265 Encoder
rockchipmpp: mpph264enc: Rockchip Mpp H264 Encoder

Camera engine:

rock@rock5b:~/rockchip/camera/python$ systemctl status rkaiq_3A
● rkaiq_3A.service - rkisp 3A engine
Loaded: loaded (/lib/systemd/system/rkaiq_3A.service; enabled; vendor pres>
Active: active (running) since Sat 2022-11-26 11:52:57 UTC; 8h ago
Process: 359 ExecStart=/etc/init.d/rkaiq_3A.sh start (code=exited, status=0>
Tasks: 7 (limit: 18511)
Memory: 22.7M
CPU: 1min 6.898s
CGroup: /system.slice/rkaiq_3A.service
├─376 /usr/bin/rkaiq_3A_server
└─377 logger -t rkaiq

Python:

rock@rock5b:~/rockchip/camera/python$ python3 --version
Python 3.9.2

If you don’t have the correct environment, this guide will not work.

3. Working with the camera, examples

Finding video formats for our device node (/dev/video11) examples:

rock@rock5b:~/rockchip/camera/python$ v4l2-ctl --device /dev/video11 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture Multiplanar

	[0]: 'UYVY' (UYVY 4:2:2)
		Size: Stepwise 32x16 - 3840x2160 with step 8/8
	[1]: 'NV16' (Y/CbCr 4:2:2)
		Size: Stepwise 32x16 - 3840x2160 with step 8/8
	[2]: 'NV61' (Y/CrCb 4:2:2)
		Size: Stepwise 32x16 - 3840x2160 with step 8/8
	[3]: 'NV21' (Y/CrCb 4:2:0)
		Size: Stepwise 32x16 - 3840x2160 with step 8/8
	[4]: 'NV12' (Y/CbCr 4:2:0)
		Size: Stepwise 32x16 - 3840x2160 with step 8/8
	[5]: 'NM21' (Y/CrCb 4:2:0 (N-C))
		Size: Stepwise 32x16 - 3840x2160 with step 8/8
	[6]: 'NM12' (Y/CbCr 4:2:0 (N-C))
		Size: Stepwise 32x16 - 3840x2160 with step 8/8

For this example, we use python3, but i think python2 will also work.

rock@rock5b:~/rockchip/camera/python$ python3 --version
Python 3.9.2

Our python example calls gstreamer to grab the frames or a frame from the camera and save video in MP4 or JPG.

Take a picture python command line example (640x480 in size):

  • command line argument (pass a file name)

take_photo.py [file name.jpg]

  • running

rock@rock5b:~/rockchip/camera/python$ python3 take_photo.py photo.jpg
rga_api version 1.8.1_[4]
-rw-r–r-- 1 rock rock 234576 Nov 26 19:51 photo.jpg

Python File:

import gi
import sys
import time

gi.require_version('Gst', '1.0')
from gi.repository import Gst
# initialize GStreamer
Gst.init(sys.argv)

pipeline = Gst.parse_launch ("v4l2src device=/dev/video11 io-mode=dmabuf num-buffers=1 ! video/x-raw,format=NV12,width=640,height=480 ! mppjpegenc ! filesink location=" + sys.argv[1])
pipeline.set_state(Gst.State.PLAYING)
time.sleep(1)
pipeline.set_state(Gst.State.NULL)

Take a video in MP4 (h264 640x480)

  • command line example (640x480 in size):

take_video.py [name.mp4] [time in seconds approximate]

  • running

rock@rock5b:~/rockchip/camera/python$ python3 take_video.py video_30s_30fps_640x480.mp4 30
rga_api version 1.8.1_[4]
-rw-r–r-- 1 rock rock 4311511 Nov 26 19:49 video_30s_30fps_640x480.mp4

Python file:

import gi
import sys
import time

gi.require_version('Gst', '1.0')
from gi.repository import Gst
# initialize GStreamer
Gst.init(sys.argv)

pipeline = Gst.parse_launch ("v4l2src device=/dev/video11 io-mode=dmabuf ! video/x-raw,format=NV12,width=640,height=480,framerate=30/1 ! mpph264enc ! filesink location=" + sys.argv[1])
pipeline.set_state(Gst.State.PLAYING)
time.sleep(int(sys.argv[2]))
pipeline.set_state(Gst.State.NULL)

4. References

https://www.electronicdesign.com/technologies/communications/article/21799475/understanding-mipi-alliance-interface-specifications

Now that you can get pictures from the camera, please, move the thread to ROCK 5 series and not ROCK 4 series.

5 Likes