IMX219 + Rock 3A + rkaiq_3A_server how it made working? Where get correct calibration files?

Hello! If short: where i can get 3A calibration files for IMX219 for correct version for this board?

If detailed… I have Rock 3A board and IMX219 Raspberry Pi 2.1 sensor. Use Debian image. Obviously, without the 3A service, I have green and dark photos, and the camera cannot be used. There is no build-in automatic 3A in this camera, unlike the ov5647 camera. My kernel version 4.19.193-56-rockchip-gf62b47f70096, In this kernel used ISP 2.1 version. And at all in kernels for RK356* CPU used ISP 2.1.
I can found 2 package in Raxda Debian repository.

rkisp-engine - here has 3A service for version 1.x, and it is not fit for 2.1, and here has calibration for IMX219 sensor, but in old format, and i do not know where i can get sources of this package (i think for RK3399 Rock Pi4 this package)?
camera-engine-rkaiq - here has also 3A service, for version 2.1, but it service not working, becouse compiled with differnt version kernel “rk-camera-module.h” header file, and seems with other differnces (i think it is for Rock Pi5), i also do not know where can i get sources of this package…

Then I started looking for something from other manufacturers, in particular FireFly.


There is a correct version of the service here, however for a newer kernel than the one used in Rock PI3 at the moment, the difference in rk-camera-module.h. Then, i replace this file in original kernel sources. Now it normal process, IOCTL RKMODULE_GET_MODULE_INFO and now it can find correct file of JSON, in correct path…
Next problem it is IOCTL call in VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL, to fix it i change call, in imx219.c:

static int imx219_enum_frame_interval(struct v4l2_subdev *sd,
                                       struct v4l2_subdev_pad_config *cfg,
                                       struct v4l2_subdev_frame_interval_enum *fie)
{
        struct i2c_client *client = v4l2_get_subdevdata(sd);
        struct imx219 *priv = to_imx219(client);

        if (fie->index >= priv->cfg_num)
                return -EINVAL;

//      if (fie->code != MEDIA_BUS_FMT_SRGGB10_1X10)
//              return -EINVAL;

        fie->code = MEDIA_BUS_FMT_SRGGB10_1X10;
        fie->width = supported_modes[fie->index].width;
        fie->height = supported_modes[fie->index].height;
        fie->interval = supported_modes[fie->index].max_fps;
        fie->reserved[0] = NO_HDR;
        return 0;
}

and now i have fixed both errors: Get sensor module info failed, and this: Enum sensor frame interval failed, errors, not happened anymore, victory!
I try this rkaiq_3A_server from FireFly rposytory/firmware (i also was able to build it from sources), with calibration gc2093_default_JX8006.json, and it working, images is not green! (rkaiq_3A_server from camera-engine-rkaiq - from radxa repo still not working).

But calibration is wrong, with low lights images is dark. And camera can working only in 1920x1080 resolution, because other is not supported in this calibration file.
I attach here patch to commit e1d21c3a8bea69ff0cf38f9e01b6a3e5f75fcb66 of kernel https://drive.google.com/file/d/1iqQS0OQphueTrutn5n3YntUgledsRhrK/view?usp=sharing
I also found in FireFlay repository converter from RKISP 1.0 to 2.0 and 2.1 format, tools\iqConverTer and try to converter, but get a lot errors, in parsing imx219_rpi-camera-v2_default.xml file and it is not success ( https://drive.google.com/file/d/1iqQS0OQphueTrutn5n3YntUgledsRhrK/view?usp=sharing )… Also here has RKISP2x_Tuner_v2.0.3_Alpha_Release and calibration server, which would allow to calibrate the camera from scratch.

It seems to me that such camera support is a disaster. Most users will not be able to handle this. Or do I not understand something?.. Maybe someone can help me… In general, where can I get the correct imx219_rpi-camera-v2_default.json?

I don’t think there are public calibration files for 4.19 from Rockchip.

Anyway, I think I’ve got a little progress with the imx219 sensor without a camera engine.
I can get the frames without the calibration, similar to ov5647 in RAW mode. Here is a photo (resized) to fit in here.
The image is not as bad as the photo below, but it still needs a denoise filter and a white balance filter.

The driver used is a mix of mainline and 4.x.
Streaming 1920x1080 is 25FPS. I don’t think i can get any better than this without spending a lot of time and effort.

The image was obtained with:
V4L2_CID_ANALOGUE_GAIN = 2800
V4L2_CID_EXPOSURE = 1853

You can try the following:

  1. Stop the camera engine
  2. Set Analogue gain and exposure during the initialization of your application.

Please, note the following doesn’t help, i had to set the V4L2_CID inside the application while streaming video or before capturing the frame. I think there is something about AUTO EXPOSURE, but i get an error trying to disable it.
v4l2-ctl --device /dev/video0 --set-ctrl analogue_gain=2800
v4l2-ctl --device /dev/video0 --set-ctrl exposure=1850

Another thing you can try is libcamera. I think you have to modify or create some dummy V4L2_CID one to ignore some errors in libcamera.

I was able to achieve a little more than before. I also had to make corrections to the camera driver. Among other things, the problem is incorrect handling of exposure and gain… The driver forgets these settings when restarting the thread each time, because it is just has not code to set this values in registers, when camera start… thus, if we restart the camera and do not set values different from the previous ones (we set the same), the values are not set, and remain the default. Maybe it is same problem about which you wrote above, with setting values. That is why you first need to start capturing the camera and then change the values. Look at the kernel patch I have attached to the imx219.c file

(I can also add that I am very confused now by the calculation in the driver of the gain value that is written to the register, and this is mixing with the analog gain / digital gain parameter. I want to remove this, because knowing the analog gain according to the sensor specification varies in the range from 1 - 10.66, using a value in the range from 1 to 232 … Here it is set by some strange interval, in the upper part of which there is already a digital gain. But it is not so critical…)

i attach new kernel patch: https://drive.google.com/file/d/1hdViVd62H4PByqHQotY9C1h6AGJN9j0S/view?usp=sharing

I also some mix two calibration files, this XML and one other (gc2093_default_JX8006.json), to get some JSON:
https://drive.google.com/file/d/14PPB_m3xiB023yO92NOe_-_TPjG4Mav2/view?usp=sharing It’s not what is need… and there’s a lot of misinformation out there, but it’s better than nothing.

also i made patch to tool server, but it while not fully good working…

https://drive.google.com/file/d/1n6SoCIUdSJkKwhJYBKj3xLg8frjBP2Vw/view?usp=sharing
tool server i get from FireFly repository, and it made access to wrong device…

Unfortunately, I don’t have a light box to fully calibrate the camera… If you have a light box to simulate standard light sources, or are there those on the forum who have it? It would be possible to calibrate the camera and make it all good.

Regarding libcamera, this is not a bad idea, but as far as I understand, there is no version for rk3568 (there is no support for RKISP2.1 either?).

Images is not so bad now, it was used 3A server from FireFly sources:

About full resolution, you can use this commands to get it:
media-ctl -d /dev/media0 --set-v4l2 ‘“m01_f_imx219 5-0010”:0 [fmt:SRGGB10_1X10/3280x2464]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0 [fmt:SRGGB10_1X10/3280x2464]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0 [crop:(0,0)/3280x2464]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:2 [crop:(0,0)/3280x2464]’
v4l2-ctl -d /dev/video0 --set-selection=target=crop,top=0,left=0,width=3280,height=2464

media-ctl -p -d /dev/media0 

Exposure and gain now working:

Calibration software i also get from FireFly repository… https://wiki.t-firefly.com/en/iCore-3568JQ/prepare_compile_linux.html in this wiki has guide how to download full sources of this firefly board. And here it has.

XML file RK3399 version
https://drive.google.com/file/d/10qnbsNG52buhoVYqtKDhCtdK3qHRWsuf/view?usp=sharing
In sources of firefly also has converter, but it convert from differnt format and not fit…

As in ov5647, the imx219 driver on my setup has the pads broken. Changing mode must be done in another way, in my case.

media-ctl -d /dev/media0 --set-v4l2 ‘“m00_b_imx219 5-0010”:0 [fmt:SRGGB10_1X10/3280x2464]’
Unable to setup formats: Invalid argument (22)

I can see you get much better results with the 3A.

It would be interesting if you could push it to github, i mean with all the patches already applied.

PS. i don’t have a light box

there is generally more of a problem with writing quotes (" and ') in the forum, it replaces them. Also use command:

media-ctl -p -d /dev/media0 

to get available tree.

ov5647 at all have automatic 3A and it can be enabled (V4L2_CID_EXPOSURE_AUTO, V4L2_CID_AUTO_WHITE_BALANCE, V4L2_CID_AUTOGAIN), it is build in sensor, without 3A engine.

Although the result of the work is not quite good…
3A in general is much more than these three parameters. There are many different processing. (attached instructions obtained from the FireFly repository)
https://drive.google.com/file/d/1LCw0m96MkMYoTLzIcOxeLXAjFp4vLBqK/view?usp=sharing

Copy + Paste gone wrong.

media-ctl -d /dev/media0 --set-v4l2 "'m00_b_imx219 5-0010':0 [fmt:SRGGB10_1X10/3280x2464]"
media-ctl -d /dev/media0 --set-v4l2 "'rkisp-isp-subdev':0 [fmt:SRGGB10_1X10/3280x2464]"
media-ctl -d /dev/media0 --set-v4l2 "'rkisp-isp-subdev':0 [crop:(0,0)/3280x2464]"
media-ctl -d /dev/media0 --set-v4l2 "'rkisp-isp-subdev':2 [crop:(0,0)/3280x2464]"
v4l2-ctl -d /dev/video0 --set-selection=target=crop,top=0,left=0,width=3280,height=2464

3280x2464@21fps

v4l2-ctl --device /dev/video0 --stream-mmap=4 --stream-count=1 --stream-skip=150 --set-fmt-video=width=3280,height=2464,pixelformat=NV12
<<<<<<<<<<<<<<<<<<<<<<< 21.16 fps
<<<<<<<<<<<<<<<<<<<<< 21.16 fps
<<<<<<<<<<<<<<<<<<<<< 21.16 fps
<<<<<<<<<<<<<<<<<<<<< 21.16 fps
<<<<<<<<<<<<<<<<<<<<< 21.16 fps
<<<<<<<<<<<<<<<<<<<<< 21.16 fps
<<<<<<<<<<<<<<<<<<<<<< 21.16 fps

21FPS it is maximum for this camera with 2 lane (raspberry pi camera board have only 2 lane on connector) configuration :slight_smile:


And this selected in driver.