I’m trying to enable IMX290 IR-CUT / IMX519 / IMX708 / IMX477 on Radxa Zero 3W.
| Sensor | Status | IQfile | FPS |
|---|---|---|---|
| IMX290 | i2c and MIPI errors / driver | no | |
| IMX519 | driver working / manual focus | done | 10 FPS / 60 FPS / 120 FPS |
| IMX708 | driver working with auto-focus | done | 14 FPS / 60 FPS / 120 FPS |
| IMX477 | driver working | done | 10 FPS / 40 FPS / 50 FPS |
IMX708 is missing the auto-focus, don’t have any rockchip example to be based on. Current limitation is max window size of 4080x2568.
IMX708 1920x1080 60 FPS
IMX708 1536x864 120 FPS
IMX477 working on 1920x1080 40 fps, current limitation is the max window size 4048x3040
IMX519 1920x1080 (out of focus)
IMX519 4080x3024 (no focus)
I can’t find a working AK7375, still hopes. Focus works issuing a v4l2 command.
IMX519 1280x720 120 FPS
References:
imx519: linux/drivers/media/i2c/imx519.c at rpi-5.15.y · raspberrypi/linux · GitHub
imx708: add support for imx708 raspberry pi v3 camera by ackPeng · Pull Request #457 · armbian/linux-rockchip · GitHub
imx477: IMX477 / IMX577 / IMX766 on Rock 5 - #10 by avaf
1 Like
Regarding IMX290 , the lack of hardware knowledge prevents me from getting this working, i2c detect the hardware but failed to send / receive commands, also i get MIPI errors so i think i am missing something.
Reference: linux/drivers/media/i2c/imx290.c at rpi-5.15.y · raspberrypi/linux · GitHub
After replacing the cable and testing all the sensors, the connector lock became loose, so care must be taken.
I will order a new rock 3w and try again if someone spot something wrong with dtsi or if someone at radxa can post here that it’s not possible to use this sensor on 3W, then that’s it.
Anyway, if someone wants to give it a try and fix what i missed, here is the DTSI i used:
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/pinctrl/rockchip.h>
/ {
imx290_vana_2v8: 2p8v {
compatible = "regulator-fixed";
regulator-name = "camera_vana";
regulator-min-microvolt = <2800000>;
regulator-max-microvolt = <2800000>;
regulator-always-on;
};
imx290_vdig_1v2: 1p2v {
compatible = "regulator-fixed";
regulator-name = "camera_vdig";
regulator-min-microvolt = <1200000>;
regulator-max-microvolt = <1200000>;
regulator-always-on;
};
imx290_vddl_1v8: 1p8v {
compatible = "regulator-fixed";
regulator-name = "camera_vddl";
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <1800000>;
regulator-always-on;
};
clk_cam_37m: external-camera-clock-37m {
status = "okay";
compatible = "fixed-clock";
clock-frequency = <37125000>;
clock-output-names = "clk_cam_37m";
#clock-cells = <0>;
};
camera_pwdn_gpio: camera-pwdn-gpio {
status = "okay";
compatible = "regulator-fixed";
regulator-name = "camera_pwdn_gpio";
regulator-always-on;
regulator-boot-on;
enable-active-high;
gpio = <&gpio3 RK_PC6 GPIO_ACTIVE_HIGH>;
};
};
&i2c2 {
status = "okay";
pinctrl-names = "default";
pinctrl-0 = <&i2c2m1_xfer>;
#address-cells = <1>;
#size-cells = <0>;
camera_imx290: camera-imx290@1a {
status = "okay";
compatible = "sony,imx290";
reg = <0x1a>;
clocks = <&clk_cam_37m>;
assigned-clocks = <&clk_cam_37m>;
assigned-clock-rates = <37125000>;
clock-frequency = <37125000>;
clock-names = "xclk";
/* pwdn-gpios = <&gpio3 RK_PC6 GPIO_ACTIVE_HIGH>; */
rockchip,camera-vflip = <1>;
rockchip,camera-hflip = <1>;
rockchip,camera-mode = <0>;
rockchip,camera-module-index = <0>;
rockchip,camera-module-facing = "back";
rockchip,camera-module-name = "imx290";
rockchip,camera-module-lens-name = "default";
rotation = <0>;
orientation = <2>;
vdda-supply = <&imx290_vana_2v8>; /* 2.8v */
vddd-supply = <&imx290_vdig_1v2>; /* 1.2v */
vdddo-supply = <&imx290_vddl_1v8>; /* 1.8v */
port {
ucam_out0: endpoint {
remote-endpoint = <&mipi_in_ucam0>;
data-lanes = <1 2>;
link-frequencies = /bits/ 64 <445500000 297000000>;
};
};
};
};
&csi2_dphy_hw {
status = "okay";
};
&csi2_dphy0 {
status = "okay";
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;
mipi_in_ucam0: endpoint@1 {
reg = <1>;
remote-endpoint = <&ucam_out0>;
data-lanes = <1 2>;
};
};
port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;
dphy0_out: endpoint@1 {
reg = <1>;
remote-endpoint = <&isp0_in>;
};
};
};
};
&rkisp_vir0 {
status = "okay";
port {
#address-cells = <1>;
#size-cells = <0>;
isp0_in: endpoint@0 {
reg = <0>;
remote-endpoint = <&dphy0_out>;
};
};
};
&rkisp {
status = "okay";
};
&rkisp_mmu {
status = "okay";
};
&rkcif_mmu {
status = "okay";
};
&rkcif {
status = "okay";
};
Nice work, getting cameras going is a bit tricky when information is hard to come by (sensor datasheets, PCB connector pin outs, broken I2C drivers etc). My work on this front has been put on hold whilst I have other projects to get through and that work has only been on RK3576 so far, where the 3W would be slightly different again with its 22 pin connector.
My problems with the IMX290 came down to the I2C driver that ships with the Rockchip 6.1 kernel being totally broken which would explain your MIPI errors. I ended up back porting a working RPI Linux 6.1 driver to be compatible with Rockchips way of doing things with the RKAIQ server.
What are you doing for the IQfile tuning, do you have a proper Light box to use along with RKISP?
For obtaining the “confidential” Sony sensor datasheets I found these are available on Chinese forums, usually for sale for a couple of dollars.
I compiled and ran your driver for the imx290 on my Rock 5C. Unfortunately, I wasn’t successful on the first attempt; I received some error/warning messages, although, at least, there were no I2C errors. I need to review the dtsi file.
Made a mistake, its working. Can get some image now.
First picture, 60 fps only? ops, forgot its two lane.
v4l2-ctl --device /dev/video11 --stream-mmap=4 --stream-count=1 --stream-skip=150 --set-fmt-video=width=1920,height=1080,pixelformat=NV12
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps, dropped buffers: 12
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps, dropped buffers: 10
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
No proper Light box here.
Its nice you have access to some register to improve the driver/drivers.
By the way, looks like Radxa is launching IMX577 soon.
Question for you: I see you use isp39 on your iqfile, is it new camera engine? And it’s working for you, or is just for reference?
ISP39 is for RK3576 as it uses the newer ISP version. For other Rockchip SoC models you have to use the ISP version made specifically for it. RK3588 uses ISP30.
@3djelly Thanks for your work. Here my iqfile for Rock 5C / Rock 5C lite / Rock 5B.
Tested with IR to see the result , but still needs some adjustments (pink). going day to night with IR works , switching off IR sometimes need to switch a few times back and forth to recover.
IR off.
IR on
imx290_imx290_default.json.zip (96.9 KB)
In the past couple of days I have been working on the OV9282 sensor and came up against a FPS limit also, however 2 MIPI lanes does have the bandwidth to achieve 120 FPS.
The first of the issues was the ov9282 I2C driver being non functional, which required fixing to match the working ov9281 driver, but I also had to add in support for 30/60/120 FPS.
Secondly I was initially using OpenCV to use the camera which required a JPEG encode step that was being offloaded to the CPU and only achieved a 60 FPS rate. I switched to using Gstreamer with the Rockchip mppjpegenc hardware encoder and that then allowed the full 120 FPS rate.
Nice. My OV9281 camera is on the way.
Received the camera today. Initial test on ov9281 with legacy driver. Need a way to HFlip and Tuning seems to be hard…
Yes tuning is hard, I am still getting the resources to do it properly.
One problem I ran up against is actually a bug in the RKISP tuner software where it is not drawing a particular UI option properly so silently makes a setting in the IQ file which results in the image output being pure black. If you come up against that, the fix is to;
Change the IQ JSON file field from;
"hw_btnrCfg_pixDomain_mode": "btnr_pixLog2Domain_mode"
to;
"hw_btnrCfg_pixDomain_mode": "btnr_pixLinearDomain_mode"
For HFlip you can change the registers in the sensor by editing the driver source. See datasheet section 4.1 
With your driver, and first try with iqfile.
Much better now:
ov9281 30 fps, daylight ambient, indoor with led lamp, dusty keyboard:
ov9282 120 fps, daylight ambient, indoor with led lamp, dusty keyboard:
Waiting for a new 3W to arrive and then try to fix the imx290.
As your working with the 3W, did you manage to achieve the 120FPS rate? As you have the FPS and frame stats overlaid on the image its possible that could slow things down a bit from achieving the full throughput.
Grabbing the frame at 120FPS, but 3W can’t keep-up with this rate if you display it on screen, avarage 80FPS using hw acceleration.
v4l2-ctl --device /dev/video0 --stream-mmap=4 --stream-count=1 --stream-skip=350 --set-fmt-video=width=1280,height=800,pixelformat=NV12
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 120.65 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 120.64 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
PS: Maybe rkaiq_3A is doing a lot of things it should not.
All these tests were conducted in an indoor environment, i believe the results would be better outdoors (daylight).
IMX290 update:
So, the I2C error has been fixed, but there are still MIPI errors. The cable is 30 cm long, hopefully, the cable is the problem,otherwise, it’s the end… Yep, there is a typo, but too late to fix it with this cable.
[ 9.781709] rkisp_hw fdff0000.rkisp: Adding to iommu group 8
[ 9.781984] rkisp_hw fdff0000.rkisp: is_thunderboot: 0
[ 9.781996] rkisp_hw fdff0000.rkisp: max input:0x0@0fps
[ 9.782030] rkisp_hw fdff0000.rkisp: can’t request region for resource [mem 0xfdff0000-0xfdffffff]
[ 9.782599] rkisp_hw fdff0000.rkisp: no find phandle sram
[ 9.783253] rkisp rkisp-vir0: rkisp driver version: v02.02.01
[ 9.783455] rkisp rkisp-vir0: No memory-region-thunderboot specified
[ 9.783726] rkisp rkisp-vir0: Entity type for entity rkisp-isp-subdev was not initialized!
[ 9.783748] rkisp rkisp-vir0: Entity type for entity rkisp-csi-subdev was not initialized!
[ 9.786838] rkisp-vir0: Async subdev notifier completed
[ 10.455988] rkisp rkisp-vir0: clear unready subdev num: 0
[ 64.796253] rkisp_hw fdff0000.rkisp: set isp clk = 297000000Hz
[ 66.057252] rkisp-vir0: MIPI error: packet: 0x00100000
[ 66.557240] rkisp-vir0: MIPI error: packet: 0x00100000
[ 68.691470] rkisp-vir0: MIPI error: packet: 0x00010010
media-ctl -d /dev/media0 -p
Media controller API version 5.10.160
Media device information
driver rkisp-vir0
model rkisp0
serial
bus info
hw revision 0x0
driver version 5.10.160
Device topology
entity 1: rkisp-isp-subdev (4 pads, 7 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none
crop.bounds:(0,0)/1920x1080
crop:(0,0)/1920x1080]
← “rkisp-csi-subdev”:1 [ENABLED]
← “rkisp_rawrd0_m”:0
← “rkisp_rawrd2_s”:0
pad1: Sink
← “rkisp-input-params”:0 [ENABLED]
pad2: Source
[fmt:YUYV8_2X8/1920x1080 field:none colorspace:smpte170m quantization:full-range
crop.bounds:(0,0)/1920x1080
crop:(0,0)/1920x1080]
→ “rkisp_mainpath”:0 [ENABLED]
→ “rkisp_selfpath”:0 [ENABLED]
pad3: Source
→ “rkisp-statistics”:0 [ENABLED]
entity 6: rkisp-csi-subdev (6 pads, 5 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none]
← “rockchip-csi2-dphy0”:1 [ENABLED]
pad1: Source
[fmt:SRGGB10_1X10/1920x1080 field:none]
→ “rkisp-isp-subdev”:0 [ENABLED]
pad2: Source
→ “rkisp_rawwr0”:0 [ENABLED]
pad3: Source
pad4: Source
→ “rkisp_rawwr2”:0 [ENABLED]
pad5: Source
→ “rkisp_rawwr3”:0 [ENABLED]
entity 13: rkisp_mainpath (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
← “rkisp-isp-subdev”:2 [ENABLED]
entity 19: rkisp_selfpath (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video1
pad0: Sink
← “rkisp-isp-subdev”:2 [ENABLED]
entity 25: rkisp_rawwr0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video2
pad0: Sink
← “rkisp-csi-subdev”:2 [ENABLED]
entity 31: rkisp_rawwr2 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video3
pad0: Sink
← “rkisp-csi-subdev”:4 [ENABLED]
entity 37: rkisp_rawwr3 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video4
pad0: Sink
← “rkisp-csi-subdev”:5 [ENABLED]
entity 43: rkisp_rawrd0_m (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video5
pad0: Sourc
e
→ “rkisp-isp-subdev”:0
entity 49: rkisp_rawrd2_s (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video6
pad0: Source
→ “rkisp-isp-subdev”:0
entity 55: rkisp-statistics (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video7
pad0: Sink
← “rkisp-isp-subdev”:3 [ENABLED]
entity 61: rkisp-input-params (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video8
pad0: Source
→ “rkisp-isp-subdev”:1 [ENABLED]
entity 67: rockchip-csi2-dphy0 (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none]
← “m00_b_imx209 2-001a”:0 [ENABLED]
pad1: Source
→ “rkisp-csi-subdev”:0 [ENABLED]
entity 70: m00_b_imx209 2-001a (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev3
pad0: Source
[fmt:SRGGB10_1X10/1920x1080 field:none]
→ “rockchip-csi2-dphy0”:0 [ENABLED]
In your output its Linux 5.10 right? My driver if that is the one your using is made for 6.1 due to V4L2 API differences between the two versions.
Also in your output it thinks you have an IMX209 sensor.
entity 70: m00_b_imx209 2-001a (1 pad, 1 link)
Downgrading your driver was tricky for me. It looks like you have the Zero 3E, is there a chance you could work on legacy kernel?
Fixed. Here is the printscreen without iqfile + 500 mm fpc, a bit greenish, but it’s a start. Tuning is another thing.
@3djelly Did you succeed to tune the imx290 in low light condition?