ADV7280M with Rock Pi 4B - problems

Hi,
I have to connect ADV7280M (contained in the EVAL-ADV7280MEBZ board) to Rock Pi 4B.
I have recompiled the kernel with added adv7180.c driver. The procedure for recompilation of the kernel I have taken from: https://wiki.radxa.com/Rockpi4/dev/kernel-4.4
After that I had to add ADV7280M to the device tree.
First I have found, that the driver does not control the powerdown pin (GPIO1 B5), therefore I have added an overlay that sets this pin to “1”:
/dts-v1/;
/plugin/;

/ {
	model = "ROCK PI 4B";
	compatible = "rockchip,rockpi","rockchip,rk3399";

	fragment@0 {
		target-path = "/";
		__overlay__ {
			wzgpios {
				compatible = "gpio-leds";
				wzcam_on {
					label = "WZcam On";
					gpios = <&gpio1 13 0>;
					default-state = "on";
				};
			};
		};
	};
};

After that I have connected the ADV7280M itself:
/dts-v1/;
/plugin/;

/ {
        model = "ROCK PI 4B";
        compatible = "rockchip,rockpi","rockchip,rk3399";

	fragment@0 {
		target = <&i2c4>;
		__overlay__ {
			#address-cells = <1>;
			#size-cells = <0>;
			status = "okay";
			adv7280@21 {
				compatible = "adi,adv7280-m";
				reg = <0x21>;
				status = "okay";
				pinctrl-names = "rockchip,camera_default";
				port {
					adv7280_out: endpoint {
						remote-endpoint = <&mipi_in_ucam0>;
						data-lanes = <1>;
					};
				};
			};
		};
	};

	fragment@1 {
		target = <&mipi_dphy_rx0>;
		__overlay__ {
			status = "okay";
		};
	};

	fragment@2 {
		target = <&mipi_in_ucam0>;
		__overlay__ {
			remote-endpoint = <&adv7280_out>;
			data-lanes = <1>; 
		};
	};
};

With that DT, i have obtained correct output of “media-ctl -p” command, but when I try to start the streaming of data with:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=NV12,width=1920,height=1080, framerate=30/1 ! videoconvert !  x264enc tune=zerolatency bitrate=500 speed-preset=superfast ! rtph264pay ! udpsink host=172.19.1.1 port=5000

I get the following error and system crashes:

[ 246.739221] adv7180 4-0021: No active sensor
[ 246.739613] rkisp1: open cif pipeline failed -32

I have found, that the problem is caused by the fact that adv7180.c driver does not set the sd->entity.type to MEDIA_ENT_T_V4L2_SUBDEV_SENSOR, and the
drivers/media/platform/rockchip/isp1/dev.c file checks it in lines 153-164:

	/* find the subdev of active sensor */
	sd = p->subdevs[0];
	for (i = 0; i < p->num_subdevs; i++) {
		sd = p->subdevs[i];
		if (sd->entity.type == MEDIA_ENT_T_V4L2_SUBDEV_SENSOR)
			break;
	}

	if (i == p->num_subdevs) {
		v4l2_warn(sd, "No active sensor\n");
		return -EPIPE;
	}

The question is, if the adv7180.c should set this type. If not, then I should define another pipeline element, corresponding to the analog video input. However, I couldn’t find any info how to do that.
Therefore, I have modified the adv7180.c to set the required type.
Unfortunately, it does not help, as my device fails the next check (lines 165-169 in dev.c):

	ctrl = v4l2_ctrl_find(sd->ctrl_handler, V4L2_CID_PIXEL_RATE);
	if (!ctrl) {
		v4l2_warn(sd, "No pixel rate control in subdev\n");
		return -EPIPE;
	}

After an attempt to start streaming, I get

adv7180 4-0021: No pixel rate control in subdev

and the system crashes again. (BTW, why video_rkisp1 crashes the system when starting the pipelione fails?)

Of course, I can add a fake PIXEL_RATE control to adv7180.c, but it does not seem to be the right solution. Probably indeed, it is necessary to define a
“composite video input” element in the pipeline, that would have the entity type set to MEDIA_ENT_T_V4L2_SUBDEV_SENSOR and would provide the V4L2_CID_PIXEL_RATE control.
I’ll appreciate any suggestions how to do it in the right way.

TIA & Regards,
Wojtek

I have found the following device tree binding:


However, the associated driver https://github.com/radxa/kernel/blob/release-4.4-rockpi4/drivers/video/fbdev/omap2/displays-new/connector-analog-tv.c does not seem to be related to V4L…

OK. It appears that to have ADV7280A-M working with rkisp1 in standard 4.4.154 kernel, I need to add in 1dv7180.c not only the V4L2_CID_PIXEL_RATE control, but also V4L2_CID_LINK_FREQ control.
I alsoo needed to add
sd->entity.type = MEDIA_ENT_T_V4L2_SUBDEV_SENSOR;
After that, an attempt to start streaming does not hang the system, but data are still not flowing…

1 Like

I have verified with the oscilloscope that the data are delivered to the MIPI port of Rock Pi 4B. It seems, that for certain reason rkisp1 does not detect frames in the incoming data.
When I connect the old OV5647 based Raspberry Pi camera to the same board, data are accepted and I can stream the video data, so Rock Pi 4B is OK.
Unfortunately, there is no detailed information about the rkisp1 hardware available. Is there any way to inspect its status to check why it does not accept the data?

I have added the PIXEL_RATE and LINK_FREQ controls. Now the pipeline gets initialized, and capturing data stops, when ISP1 waits for data.
When I have switched on debugging, I get the following output:

[ 1102.156898] RSZ_CTRL 0x0000000f/0x0000000f
[ 1102.156898] RSZ_SCALE_HY 58975/58975
[ 1102.156898] RSZ_SCALE_HCB 58966/58966
[ 1102.156898] RSZ_SCALE_HCR 58966/58966
[ 1102.156898] RSZ_SCALE_VY 62911/62911
[ 1102.156898] RSZ_SCALE_VC 31401/31401
[ 1102.156898] RSZ_PHASE_HY 0/0
[ 1102.156898] RSZ_PHASE_HC 0/0
[ 1102.156898] RSZ_PHASE_VY 0/0
[ 1102.156898] RSZ_PHASE_VC 0/0
[ 1102.159816] rkisp1: stream 0 crop: 720x576 -> 800x600
[ 1102.160747] rkisp1: CIF_ID 0x10853017
[ 1102.161098]   MIPI_CTRL 0x00060f00
[ 1102.161098]   MIPI_IMG_DATA_SEL 0x0000001e
[ 1102.161098]   MIPI_STATUS 0x00002000
[ 1102.161098]   MIPI_IMSC 0x03f33333
[ 1102.162544]   MI_CTRL 0x007a2001
[ 1102.164204]   ISP_CTRL 0x00006195 MIPI_CTRL 0x00060f01
[ 1102.165924] vb2: vb2_core_streamon: successful

Unfortunately, after that nothing happens, until I press CTRL+C.
When I do the testing with OV5647, I get:

[   84.096199] RSZ_CTRL 0x0000007f/0x0000007f
[   84.096199] RSZ_SCALE_HY 27286/27286
[   84.096199] RSZ_SCALE_HCB 27266/27266
[   84.096199] RSZ_SCALE_HCR 27266/27266
[   84.096199] RSZ_SCALE_VY 36381/36381
[   84.096199] RSZ_SCALE_VC 58972/58972
[   84.096199] RSZ_PHASE_HY 0/0
[   84.096199] RSZ_PHASE_HC 0/0
[   84.096199] RSZ_PHASE_VY 0/0
[   84.096199] RSZ_PHASE_VC 0/0
[   84.099407] rkisp1: stream 0 crop: 1296x972 -> 800x600
[   84.102404] rkisp1: CIF_ID 0x10853017
[   84.102957]   MIPI_CTRL 0x00061f00
[   84.102957]   MIPI_IMG_DATA_SEL 0x0000002a
[   84.102957]   MIPI_STATUS 0x00002000
[   84.102957]   MIPI_IMSC 0x03f33333
[   84.104440]   MI_CTRL 0x007a2001
[   84.106151]   ISP_CTRL 0x00006197 MIPI_CTRL 0x00061f01
[   84.108238] rkisp1: MIPI mis error: 0x00800000
[   84.110177] vb2: vb2_core_streamon: successful
[   84.155821] vb2: vb2_buffer_done: done processing on buffer 0, state: 6
[   84.156432] vb2: vb2_buffer_done: call_memop(ffffffc0ed0f89c0, 0, finish)
[   84.157034] vb2: vb2_buffer_done: call_memop(ffffffc0ed0f89c0, 0, finish)

The last three lines inform that reception of data has started.
The differences between the MIPI and ISP registers seem to be justified by different formats.
I have verified with the scope that ADV7280-M sends the data via MIPI to the processor. Why those data are ignored by the ISP1?

I have also performed tests with the newest kernel 5.5 branch rockchip/isp/v13 from Collabora: https://gitlab.collabora.com/koike/linux/tree/rockchip/isp/v13
Results are practically the same as described above…