Rkisp-vir0 Error on Rock 3a

[rice]

Hi:
I transplanted the ov5640 driver on Rock and successfully generated the device node /dev/video*…
But using cheese to open the camera fails!
dmesg log:

root@linaro-alip:~# ls /dev/video* -la
crw-rw----+ 1 root video 81, 0 Feb 14  2019 /dev/video0
crw-rw----+ 1 root video 81, 1 Feb 14  2019 /dev/video1
crw-rw----+ 1 root video 81, 2 Feb 14  2019 /dev/video2
crw-rw----+ 1 root video 81, 3 Feb 14  2019 /dev/video3
crw-rw----+ 1 root video 81, 4 Feb 14  2019 /dev/video4
crw-rw----+ 1 root video 81, 5 Feb 14  2019 /dev/video5
crw-rw----+ 1 root video 81, 6 Feb 14  2019 /dev/video6
crw-rw----+ 1 root video 81, 7 Feb 14  2019 /dev/video7
crw-rw----+ 1 root video 81, 8 Feb 14  2019 /dev/video8
root@linaro-alip:~# dmesg | grep rkisp-vir0
[    0.753990] rkisp rkisp-vir0: rkisp driver version: v01.06.01
[    0.754138] rkisp rkisp-vir0: No memory-region-thunderboot specified
[    0.754436] rkisp rkisp-vir0: Entity type for entity rkisp-isp-subdev was not initialized!
[    0.754480] rkisp rkisp-vir0: Entity type for entity rkisp-csi-subdev was not initialized!
[    0.757514] rkisp-vir0: update sensor failed
[    0.757924] rkisp-vir0: Failed to register subdev notifier(-22)
root@linaro-alip:~# cheese


(cheese:2451): dbind-WARNING **: 03:54:28.729: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files


[ 2239.655004] rkisp-vir0: rkisp_enum_frameintervals Not active sensor
[ 2239.655126] rkisp-vir0: rkisp_enum_frameintervals Not active sensor
[ 2239.655156] rkisp-vir0: rkisp_enum_frameintervals Not active sensor
[ 2239.655194] rkisp-vir0: rkisp_enum_frameintervals Not active sensor
[ 2239.655226] rkisp-vir0: rkisp_enum_frameintervals Not active sensor
[ 2239.655256] rkisp-vir0: rkisp_enum_frameintervals Not active sensor
[ 2239.655284] rkisp-vir0: rkisp_enum_frameintervals Not active sensor
[ 2239.655317] rkisp-vir0: rkisp_enum_frameintervals Not active sensor
[ 2239.655348] rkisp-vir0: rkisp_enum_frameintervals Not active sensor
[ 2239.655381] rkisp-vir0: rkisp_enum_frameintervals Not active sensor
[ 2239.655405] rkisp-vir0:

Configure the dts of the camera as follows:

&rkisp {
	status = "okay";
};

// add by fl start
&rkisp_vir0 {
   status = "okay";
   port {
      #address-cells = <1>;
      #size-cells = <0>;

      isp0_in: endpoint@0 {
         reg = <0>;
         remote-endpoint = <&csidphy_out>;
      };
   };
};

&csi2_dphy0 {
        status = "okay";
        /*
        * dphy0 only used for full mode,
        * full mode and split mode are mutually exclusive
        */
        ports {
                #address-cells = <1>;
                #size-cells = <0>;
                port@0 {
                        reg = <0>;
                        #address-cells = <1>;
                        #size-cells = <0>;
                        mipi_in_ucam0: endpoint@1 {
                                reg = <1>;
                                remote-endpoint = <&ov5640_out>;
                                data-lanes = <1 2>;
                        };
                };
                port@1 {
                        reg = <1>;
                        #address-cells = <1>;
                        #size-cells = <0>;

                        csidphy_out: endpoint@0 {
                                reg = <0>;
                                remote-endpoint = <&isp0_in>;
                        };
                };
        };
};
&csi2_dphy_hw {
   status = "okay";
};


&csi2_dphy1 {
        status = "disabled";
};

&csi2_dphy2 {
        status = "okay";
};

// end

how to fix this isp driver error？？

[rice]

I am not sure whether the following log information affects the work of csi sensor, can someone provide some help information? thanks…

root@linaro-alip:~# dmesg | grep rockchip-csi2
[    0.285755] rockchip-csi2-dphy csi2-dphy0: csi2 dphy0 probe successfully!
[    0.285975] rockchip-csi2-dphy csi2-dphy2: Err:csi2 dphy hw has been set as full mode by phy0, target mode is:split
[    0.286007] rockchip-csi2-dphy csi2-dphy2: csi2 dphy hw can't be attached, register dphy2 failed!
[    0.286180] rockchip-csi2-dphy-hw fe870000.csi2-dphy-hw: csi2 dphy hw probe successfully!
[    0.759336] rockchip-csi2-dphy csi2-dphy0: dphy0 matches ov5640 5-003c:bus type 4

[francesco.agostini]

Hi, I’m trying with OV5647 camera module. At the moment I’m not using the dphy0 (4 lane), I’m trying with dphy1 (splitted mode that use lane 1 and 2). But I have some trouble with the ISP configuration.

[    0.807003] iommu: Adding device fdff0000.rkisp to group 7
[    0.807063] rkisp_hw fdff0000.rkisp: Linked as a consumer to fdff1a00.iommu
[    0.807327] rkisp_hw fdff0000.rkisp: is_thunderboot: 0
[    0.807349] rkisp_hw fdff0000.rkisp: max input:0x0@0fps
[    0.807400] rkisp_hw fdff0000.rkisp: can't request region for resource [mem 0xfdff0000-0xfdffffff]
[    0.808742] rkisp rkisp-vir0: rkisp driver version: v01.06.01
[    0.808888] rkisp rkisp-vir0: No memory-region-thunderboot specified
[    0.809167] rkisp rkisp-vir0: Entity type for entity rkisp-isp-subdev was not initialized!
[    0.809208] rkisp rkisp-vir0: Entity type for entity rkisp-csi-subdev was not initialized!
[    1.045365] rkisp rkisp-vir0: clear unready subdev num: 1
[    1.045783] rkisp-vir0: update sensor failed
[   63.603294] rkisp-vir0: update sensor info failed -19

This is my DTS code:

/* Enable CSI PHY */
&csi2_dphy_hw {
status = "okay";
};

/* Disable full mode mipi csi 4 lane - mutual exclusion in split mode */

&csi2_dphy0 {
    status = "disabled";
};

/* Enable mipi csi single mode lane 1 - 2 */

&csi2_dphy1 {
    status = "okay";
    power-supply = <&vcc_camera>;
    ports {
        #address-cells = <1>;
        #size-cells = <0>;
        port@0 {
                reg = <0>;
                #address-cells = <1>;
                #size-cells = <0>;

                dphy1_in: endpoint@1 {
                        reg = <1>;
                        remote-endpoint = <&ov5647_out>;
                        data-lanes = <1 2>;
                };
        };
        port@1 {
                reg = <1>;
                #address-cells = <1>;
                #size-cells = <0>;

                dphy1_out: endpoint@0 {
                        reg = <0>;
                        remote-endpoint = <&isp0_in>;
                    };
        	};
    };
};


&csi2_dphy2 {
    status = "disabled";
};


&rkisp {
    status = "okay";
};

&rkisp_mmu {
    status = "okay";
};

&rkisp_vir0 {
    status = "okay";
    port {
            #address-cells = <1>;
            #size-cells = <0>;

            isp0_in: endpoint@0 {
                    reg = <0>;
                    remote-endpoint = <&dphy1_out>;
            };
    };
};

[abel]

you need to open the camera power, and ov5647 driver also need to do some change

[francesco.agostini]

Hi! Thank you for your suggestions, camera power seems to be ok, I added to the ov5647 driver the GPIO power control. And the camera driver do correctly the camera detection via I2C. This is what I did on the camera driver in the probe function, read the pwdn GPIO from the device tree and open the camera power. Maybe a power init sequence issue?

/* Request the power down GPIO asserted. */
sensor->pwdn = devm_gpiod_get_optional(dev, "pwdn", GPIOD_OUT_HIGH);
if (IS_ERR(sensor->pwdn)) {
	dev_err(dev, "Failed to get 'pwdn' gpio\n");
	return -EINVAL;
}

/* Turning on pwdn gpio */
gpiod_set_value_cansleep(sensor->pwdn, 1);
msleep(PWDN_ACTIVE_DELAY_MS);

[abel]

1、First you will change you dts code:

/ {
	ext_cam_clk: external-camera-clock {
        compatible = "fixed-clock";
        clock-frequency = <25000000>;
        clock-output-names = "CLK_CAMERA_25MHZ";
        #clock-cells = <0>;
    };
};

&i2c5 {
	status = "okay";	
	pinctrl-0 = <&i2c5m0_xfer>;
	
	ov5647: ov5647@36 {
		compatible = "ovti,ov5647";
		reg = <0x36>;
		status = "okay";
		clocks = <&ext_cam_clk>;
		pwdn-gpios = <&gpio4 RK_PD2 GPIO_ACTIVE_LOW>;

		port {
			camera_out: endpoint {
				remote-endpoint = <&dphy_in>;
				data-lanes = <1 2>;
			};
		};
	};
};

&gpio1 {
	cam_power {
		gpio-hog;
		gpios = <RK_PB1 GPIO_ACTIVE_HIGH>;
		output-high;
	};
};

&rkisp {
	status = "okay";
};

&rkisp_mmu {
	status = "okay";
};

&csi2_dphy0 {
	status = "okay";
	ports {
		#address-cells = <1>;
		#size-cells = <0>;
		port@0 {
			reg = <0>;
			#address-cells = <1>;
			#size-cells = <0>;

			dphy_in: endpoint@0 {
				reg = <0>;
				remote-endpoint = <&camera_out>;
				data-lanes = <1 2>;
			};
		};
		port@1 {
			reg = <1>;
			#address-cells = <1>;
			#size-cells = <0>;

			dphy_out: endpoint@0 {
				reg = <0>;
				remote-endpoint = <&isp0_in>;
			};
		};
	};
};

&csi2_dphy_hw {
      status = "okay";
};

&rkisp_vir0 {
	status = "okay";

	port {
		#address-cells = <1>;
		#size-cells = <0>;

		isp0_in: endpoint@0 {
			reg = <0>;
			remote-endpoint = <&dphy_out>;
		};
	};

};

2、Second replace your ov5647 driver file at /driver/media/i2c/
ov5647.zip (8.8 KB)

3、I only used on rock3a board, and my kernel version is 4.19. 219, other boards have not been verified

[avaf]

@abel, nice work. Thank you.

[abel]

You’re welcome.

[gacktex]

Great job! THX

[jack]

Nice work. Thanks for the sharing. We will integrate this into the new image release.

[francesco.agostini]

It works! Thanks for your support @abel

[abel]

thant’s all right

[yoshis]

Hi,
Please help me on building new kernel and device overlay file. for ov5647.
I’m stuck with an error and I can not resolve this since I don’t have enough knowledge about device tree overlays.

I did like below.
1, I pasted the dts code and save it as arch/arm64/boot/dts/rockchip/overlay/ov5647.dts.
2, I replaced the ov5647 driver file at /driver/media/i2c/
3, I added a line of ‘CONFIG_VIDEO_OV5647=y’ to ‘rockchip_linux_defconfig’ file.
4. I also added a line of ‘ov5647.dtbo’ to ‘…arch/arm64/boot/dts/rockchip/overlay/Makefile’.

Then, I got the following syntax error ,

-------- error log --------

./build/mk-kernel.sh rk3568-rock-3-a

Building kernel for rk3568-rock-3-a board!
4.19.193
CALL scripts/checksyscalls.sh
CHK include/generated/compile.h
DTCO arch/arm64/boot/dts/rockchip/overlay/ov5647.dtbo
Error: arch/arm64/boot/dts/rockchip/overlay/ov5647.dts:22.24-25 syntax error
FATAL ERROR: Unable to parse input tree
make[3]: *** [scripts/Makefile.lib:324: arch/arm64/boot/dts/rockchip/overlay/ov5647.dtbo] Error 1
make[2]: *** [scripts/Makefile.build:637: arch/arm64/boot/dts/rockchip/overlay] Error 2
make[1]: *** [scripts/Makefile.build:637: arch/arm64/boot/dts/rockchip] Error 2
make: *** [arch/arm64/Makefile:150: dtbs] Error 2
make: *** Waiting for unfinished jobs…
MAKE KERNEL IMAGE FAILED.
-------- end of log --------

Would anyone let me know the correct procedure?

Thanks in advance.

[avaf]

That is not an overlay.

Try this:

1. rename ov5647.dts to rock-3a-ov5647.dtsi
2. move the file to …/dts/rockchip
3. restore your overlay Makefile
4. edit the file rk3568-rock-3-a.dts
5. rebuild

Something like:

/dts-v1/;
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/pwm/pwm.h>
#include <dt-bindings/pinctrl/rockchip.h>
#include <dt-bindings/input/rk-input.h>
#include <dt-bindings/display/drm_mipi_dsi.h>
#include <dt-bindings/sensor-dev.h>
#include “rk3568.dtsi”

#include "rock-3a-ov5647.dtsi"

[yoshis]

Hi, avat,
Thank you so much. I could complete to build a new kernel without errors.
However, after setting up the OV5647 camera and update and upgrade,
‘fswebcam’ generated an error like,

root@rock-3a:/home/rock# fswebcam -d /dev/video0 ./test.png
— Opening /dev/video0…
Trying source module v4l2…
/dev/video0 opened.
No input was specified, using the first.
Error selecting input 0
VIDIOC_S_INPUT: Inappropriate ioctl for device

root@rock-3a:/home/rock# dmesg|grep OV5647
[ 2.396140] ov5647 5-0036: OV5647 power on
[ 2.916626] ov5647 5-0036: OV5647 power off
[ 2.926588] ov5647 5-0036: OmniVision OV5647 camera driver probed

I wonder if my camera is broken, or if I’ve made some mistakes.
Do you know what this error means?
The kernel version is based on
linux-image-4.19.193-36-rockchip-gd05f98887579.

Thanks.

[avaf]

Camera is not broken, you just need a modified fswebcam to bypass that error. i have done that long time ago, should be in an sd card somewhere here.

To test your camera you can do this:

v4l2-ctl -d /dev/video0 --list-ctrls
v4l2-ctl -d /dev/video0 -vwidth=1920,height=1080,pixelformat=NV12 --stream-mmap --stream-skip=3

[yoshis]

Thanks a lot. The v4l2-ctl works fine. The result was 15.63 fps. Please share me the modified version of the fswebcam if you would find it.
Anyway, It seems that I need to study the v4l2-utils.

[avaf]

I think there is updated fswebcam on the net, but you can search Armbian forum, there was plenty of discussion about the camera and optimized fswebcam. I posted my findings/mods there long ago, it might work with this camera.

Meanwhile, you can try this to grab a png image:

v4l2-ctl --device /dev/video0 --stream-mmap=3 --stream-count=1 --stream-skip=50 --stream-to=1920x1080.nv12 --set-fmt-video=width=1920,height=1080,pixelformat=NV12
ffmpeg -y -f rawvideo -s 1920x1080 -pix_fmt nv12 -i 1920x1080.nv12  -pix_fmt rgb24 1920x1080.nv12.png

Not sure –stream-mmap=3 is really necessary.

[yoshis]

I will search Armbian forum.
Thank you for your kind help.

[abel]

to change mode (width and hight) and other param, could look here:

Rock 3A Camera support

use cmd “v4l2-ctl -d /dev/video0 -l”, and you will see: root@rock-3a:~# v4l2-ctl -d /dev/video0 -l User Controls white_balance_automatic 0x0098090c (bool) : default=0 value=0 exposure 0x00980911 (int) : min=4 max=500 step=1 default=500 value=500 gain_automatic 0x00980912 (bool) : default=0 value=0 horizontal_flip 0x00980914 (bool) : default=0 value=0 flags=modify-layout vertical_flip 0x00980915 (bool) : default=0 value=…

Rock 3A Camera support

I know how to change mode. kernel driver is no bug, and you need’t to recompliling kernel or decompiling dtb. you can use cmd like this to change mode: media-ctl -d /dev/media0 --set-v4l2 ‘“ov5647 5-0036”:0[fmt:SGBRG10_1X10/2592x1944]’ media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[fmt:SGBRG10_1X10/2592x1944]’ media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[crop:(0,0)/2592x1944]’ media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:2[crop:(0,0)/2592x1944]’ v4l2-c…