Rock 3A Camera support

avaf · April 1, 2022, 2:23pm

I tried but only got a black frame, although i was at 60 fps.

Available pixel formats for /dev/video0:
  UYVY (59565955), UYVY 4:2:2, flags = 0
  422P (50323234), Planar YUV 4:2:2, flags = 0
  NV16 (3631564e), Y/CbCr 4:2:2, flags = 0
  NV61 (3136564e), Y/CrCb 4:2:2, flags = 0
  YM16 (36314d59), Planar YUV 4:2:2 (N-C), flags = 0
  NV21 (3132564e), Y/CrCb 4:2:0, flags = 0
  NV12 (3231564e), Y/CbCr 4:2:0, flags = 0
  NM21 (31324d4e), Y/CrCb 4:2:0 (N-C), flags = 0
  NM12 (32314d4e), Y/CbCr 4:2:0 (N-C), flags = 0
  YU12 (32315559), Planar YUV 4:2:0, flags = 0
  YM24 (34324d59), Planar YUV 4:4:4 (N-C), flags = 0
  RGGB (42474752), 8-bit Bayer RGRG/GBGB, flags = 0
  GRBG (47425247), 8-bit Bayer GRGR/BGBG, flags = 0
  GBRG (47524247), 8-bit Bayer GBGB/RGRG, flags = 0
  BA81 (31384142), 8-bit Bayer BGBG/GRGR, flags = 0
  RG10 (30314752), 10-bit Bayer RGRG/GBGB, flags = 0
  BA10 (30314142), 10-bit Bayer GRGR/BGBG, flags = 0
  GB10 (30314247), 10-bit Bayer GBGB/RGRG, flags = 0
  BG10 (30314742), 10-bit Bayer BGBG/GRGR, flags = 0
  RG12 (32314752), 12-bit Bayer RGRG/GBGB, flags = 0
  BA12 (32314142), 12-bit Bayer GRGR/BGBG, flags = 0
  GB12 (32314247), 12-bit Bayer GBGB/RGRG, flags = 0
  BG12 (32314742), 12-bit Bayer BGBG/GRGR, flags = 0

Control id: 9963788 - White Balance, Automatic
Control id: 9963793 - Exposure
Control id: 9963794 - Gain, Automatic
Control id: 9963796 - Horizontal Flip
Control id: 9963797 - Vertical Flip
Control id: 10094849 - Auto Exposure
Control id: 10356993 - Vertical Blanking
Control id: 10356994 - Horizontal Blanking
Control id: 10356995 - Analogue Gain
Control id: 10422529 - Link Frequency
Control id: 10422530 - Pixel Rate
Setting pixel_format: NV12 (640x480)
Image size: 614400 (640x480)
frame count: 1 - frames to skip: 602 - max. time: 0
fps: 62 - frame: 603 - tot. msecs: 9684939.

avaf · April 1, 2022, 9:17pm

1920x1080@30fps works
640x480@60fps works but was really, really dark, that was the problem.

abel · April 2, 2022, 3:20pm

use cmd “v4l2-ctl -d /dev/video0 -l”, and you will see:

root@rock-3a:~# v4l2-ctl -d /dev/video0 -l

User Controls

    white_balance_automatic 0x0098090c (bool)   : default=0 value=0
                   exposure 0x00980911 (int)    : min=4 max=500 step=1 default=500 value=500
             gain_automatic 0x00980912 (bool)   : default=0 value=0
            horizontal_flip 0x00980914 (bool)   : default=0 value=0 flags=modify-layout
              vertical_flip 0x00980915 (bool)   : default=0 value=0 flags=modify-layout

     Camera Controls

              auto_exposure 0x009a0901 (menu)   : min=0 max=1 default=1 value=1

    Image Source Controls

          vertical_blanking 0x009e0901 (int)    : min=24 max=32287 step=1 default=24 value=24
        horizontal_blanking 0x009e0902 (int)    : min=1212 max=1212 step=1 default=1212 value=1212 flags=read-only
              analogue_gain 0x009e0903 (int)    : min=16 max=1023 step=1 default=32 value=128

     Image Processing Controls

             link_frequency 0x009f0901 (intmenu): min=0 max=3 default=0 value=0
                 pixel_rate 0x009f0902 (int64)  : min=55000000 max=55000000 step=1 default=55000000 value=55000000 flags=read-only

then you can use cmd “v4l2-ctl --set-ctrl=analogue_gain=xxx” to change some param for camera, and you can change any of list ctrl(analogue_gain exposure white_balance_automatic … and so on).

for example: “v4l2-ctl --set-ctrl=analogue_gain=512”, you will see like this:

avaf · April 2, 2022, 6:06pm

No problem, i have done that. I was streaming video at night…

avaf · April 2, 2022, 9:08pm

Just for fun, I wasn’t sure if it was possible to change the mode with v4l2-ctl, so i added a property to switch mode with overlay (or decompiling dtb and changing mode and then recompiling) without recompiling the kernel.

Maybe someone finds this useful.

Property is camera-mode:

&i2c5 {
	status = "okay";
	pinctrl-0 = <&i2c5m0_xfer>;

	ov5647: ov5647@36 {
		compatible = "ovti,ov5647";
		reg = <0x36>;
		status = "okay";
		clocks = <&ext_cam_clk>;
		pwdn-gpios = <&gpio4 RK_PD2 GPIO_ACTIVE_LOW>;
		camera-mode = <0>; /* 0 - full, 1 - 192x01080@30, 2, 3 - 640x480@60 */

		port {
			camera_out: endpoint {
				remote-endpoint = <&dphy_in>;
				data-lanes = <1 2>;
			};
		};
	};
};

add_ov5647_prop_mode.patch.zip (596 Bytes)

abel · April 4, 2022, 2:09pm

I know how to change mode. kernel driver is no bug, and you need’t to recompliling kernel or decompiling dtb. you can use cmd like this to change mode:

media-ctl -d /dev/media0 --set-v4l2 ‘“ov5647 5-0036”:0[fmt:SGBRG10_1X10/2592x1944]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[fmt:SGBRG10_1X10/2592x1944]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[crop:(0,0)/2592x1944]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:2[crop:(0,0)/2592x1944]’
v4l2-ctl -d /dev/video0 --set-selection=target=crop,top=0,left=0,width=2592,height=1944

media-ctl -d /dev/media0 --set-v4l2 ‘“ov5647 5-0036”:0[fmt:SGBRG10_1X10/1920x1080]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[fmt:SGBRG10_1X10/1920x1080]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[crop:(0,0)/1920x1080]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:2[crop:(0,0)/1920x1080]’
v4l2-ctl -d /dev/video0 --set-selection=target=crop,top=0,left=0,width=1920,height=1080

media-ctl -d /dev/media0 --set-v4l2 ‘“ov5647 5-0036”:0[fmt:SGBRG10_1X10/640x480]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[fmt:SGBRG10_1X10/640x480]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[crop:(0,0)/640x480]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:2[crop:(0,0)/640x480]’
v4l2-ctl -d /dev/video0 --set-selection=target=crop,top=0,left=0,width=640,height=480

avaf · April 4, 2022, 3:24pm

Nice.

I would like to comment about the hflip and vflip, they do flip but the color of the image changes while streaming. I don’t see anything wrong with the code.

Can you test it and see what you get?

vflip=1

radbis · May 25, 2022, 11:56am

Greetings. Nice job. I’m very happy you solved it already.

radbis · May 25, 2022, 12:02pm

Guys, I’m positively surprised, really.
Can you use h.265 encoding from camera stream with 1080p@100fps or at least 60fps?
I would use it together with this very sensitive camera:

Camera itself has 1080p@60fps and Sony claims their IMX327 sensor has 1080p@100fps.
That would be great to build drone camera module with OpenHD - fork of EZ-Wifibroadcast

avaf · May 25, 2022, 1:49pm

Interesting, there is support for imx327 in the kernel and rockchip camera engine, at least on rk3399. The driver setup is to 4 lanes, i don’t know if you just change it to 2 lanes in the code, and will work. If you have the camera you could try to enable the driver and add the node for the camera, but with the correct pins, something like:

&mipi_dphy_rx0 {
	status = "okay";

	ports {
		#address-cells = <1>;
		#size-cells = <0>;

		port@0 {
			reg = <0>;
			#address-cells = <1>;
			#size-cells = <0>;

			mipi_in_ucam0: endpoint@1 {
				reg = <1>;
				remote-endpoint = <&ucam_out0>;
				data-lanes = <1 2>;
			};
		};
     };
};

&i2c5 {
	status = "okay";
	pinctrl-0 = <&i2c5m0_xfer>;

	imx327: imx327@1a {
		compatible = "sony,imx327";
		status = "okay";
		reg = <0x1a>;
		clocks = <&cru SCLK_CIF_OUT>;
		clock-names = "xvclk";
		// find the correct pins for reset and pwdn, if needed
		// ?????????? reset-gpios = <&gpio4 26 GPIO_ACTIVE_HIGH>;
		pwdn-gpios = <&gpio4 RK_PD2 GPIO_ACTIVE_HIGH>;
		pinctrl-names = "default";
		pinctrl-0 = <&cif_clkout>;
		rockchip,camera-module-index = <0>;
		rockchip,camera-module-facing = "front";
		rockchip,camera-module-name = "TongJu";
		rockchip,camera-module-lens-name = "CHT842-MD";
		port {
			ucam_out0: endpoint {
				remote-endpoint = <&mipi_in_ucam0>;
				data-lanes = <1 2>;
			};
		};
	};
};

Let’s see what @abel has to say.

Regarding h265 encoding (hw), i tried 1080@30 and got reasonable to poor results, but without trying to understand how to change parameters or find ways to get better results. I doubt it will work with 60 fps, but it is just an opinion…

radbis · May 25, 2022, 8:04pm

Wow! Wow!!! Okay I’m excited now. I have unapacked new Rock Pi 4B 1 GB on RK3399 and new boxed IMX327. Jeez!
What to do to start testing both? I installed Debian image from march this year but there was a lot of problems en wasn’t even able to run connected OV5647 - there wasn’t v4l-utils, there was nothing at all. Still can’t find test_camera-rkisp1.sh. I even not imagine about conntecting 327 to this board in this situation
If you can say me first steps and how and where to do this. I’m not totally rookie, but on this platform I’m totally green.
I’m okay with h.264 on this board. For h.265 I know now that RV1126 is the best and thinking about invest into this board later on.

avaf · May 25, 2022, 8:46pm

The first step is to know which imx327 chip you have in the camera. i googled a bit and there are some variants. IMX327LQR-C, IMX327S. They have different chip id. Seems to be 0xb2 or 0x6, possibly.
Cross your finger the chip id is 0xb2.

◆Readout rate
Maximum frame rate in Full HD 1080p mode: 60 frame / s

In the second step, imx327 must be enabled if not already, so a kernel rebuild is likely. Get used to the build process, i build everything manually so i don’t know much about the radxa build system, can’t help on this.

CONFIG_VIDEO_IMX327=y (or m if you need to rebuild the driver)

Rock3A is with kernel 4.19.y , the RockPi 4 is with kernel 4.4.y (so i think, i don’t have one).
It is possible to run kernel 4.19.y on rk3399 , i have a nanopi m4 running with kernel 4.19.193. You need to check out the latest imx327.c in kernel 4.4.y , or build 4.19.193 for your board.

Before you build, change IMX327_4LANES to IMX327_2LANES in imx327.c

Third step, is to create the new overlay for the camera imx327 (rockpi 4B dtso) with the code i posted above or edit the rockpi 4B dts and add the code there.

We need some advice about the pins for your board, we need the guys who know about the hardware, poke them:
reset-gpios = <&gpio4 26 GPIO_ACTIVE_HIGH>;
pwdn-gpios = <&gpio4 25 GPIO_ACTIVE_HIGH>;

The last step, make sure you have the iqfile imx327_TongJu_CHT842-MD.xml and the camera engine.

radbis · May 25, 2022, 9:29pm

Thank you. I think I’ll be able to follow your instruction, but will be hard I feel It’s definitely LQR regarding above innomaker info. Not sure what id it has - didn’t conect it yet.

avaf · May 28, 2022, 12:16am

Here is my take on the matter, kernel 4.4 (RockPi 4B):

&mipi_dphy_rx0 {
	status = "okay";

	ports {
		#address-cells = <1>;
		#size-cells = <0>;

		port@0 {
			reg = <0>;
			#address-cells = <1>;
			#size-cells = <0>;

			mipi_in_ucam0: endpoint@1 {
				reg = <1>;
				remote-endpoint = <&ucam_out0>;
				data-lanes = <1 2>;
			};
		};
     };
};

&i2c4 {
	status = "okay";

	imx327: imx327@1a {
		compatible = "sony,imx327";
		status = "okay";
		reg = <0x1a>;
		clocks = <&cru SCLK_CIF_OUT>;
		clock-names = "xvclk";
		pwdn-gpios = <&gpio1 RK_PB5 GPIO_ACTIVE_HIGH>;
		pinctrl-names = "rockchip,camera_default", "rockchip,camera_sleep";
		pinctrl-0 = <&cam0_default_pins &cif_clkout_a>;
		pinctrl-1 = <&cam0_default_pins>;
		rockchip,camera-module-index = <0>;
		rockchip,camera-module-facing = "front";
		rockchip,camera-module-name = "TongJu";
		rockchip,camera-module-lens-name = "CHT842-MD";
		port {
			ucam_out0: endpoint {
				remote-endpoint = <&mipi_in_ucam0>;
				data-lanes = <1 2>;
			};
		};
	};
};

Or a simpler way is to edit this file, and change it to imx327:

github.com

radxa/kernel/blob/release-4.4-rockpi4/arch/arm64/boot/dts/rockchip/rockpi-4-linux.dtsi#L632


		reg = <0x14>;
		max-x = <800>;
		max-y = <480>;
		tp-size = <911>;
	};
};


&i2c4 {
	camera1: camera-module@10 {
		status = "disabled";
		compatible = "sony,imx219";
		reg = <0x10>;


		clocks = <&cru SCLK_CIF_OUT>;
		clock-names = "clk_cif_out";
		pinctrl-names = "rockchip,camera_default";
		pwdn-gpios = <&gpio1 13 GPIO_ACTIVE_HIGH>;


		port {
			ucam_out0: endpoint {
				remote-endpoint = <&mipi_in_ucam0>;

Use this driver here and comment on the line:

github.com

rockchip-linux/kernel/blob/develop-4.4/drivers/media/i2c/imx327.c#L1250


	ret = of_property_read_u32(node, RKMODULE_CAMERA_MODULE_INDEX,
				   &imx327->module_index);
	ret |= of_property_read_string(node, RKMODULE_CAMERA_MODULE_FACING,
				       &imx327->module_facing);
	ret |= of_property_read_string(node, RKMODULE_CAMERA_MODULE_NAME,
				       &imx327->module_name);
	ret |= of_property_read_string(node, RKMODULE_CAMERA_LENS_NAME,
				       &imx327->len_name);
	if (ret) {
		dev_err(dev, "could not get module information!\n");
		return -EINVAL;
	}


	imx327->client = client;
	imx327->cur_mode = &imx327->support_modes[0];


	imx327->xvclk = devm_clk_get(dev, "xvclk");
	if (IS_ERR(imx327->xvclk)) {
		dev_err(dev, "Failed to get xvclk\n");
		return -EINVAL;
	}

Edit the Makefile and add the new driver:

github.com

rockchip-linux/kernel/blob/develop-4.4/drivers/media/i2c/Makefile#L116


obj-$(CONFIG_VIDEO_OV5647)	+= ov5647.o
obj-$(CONFIG_VIDEO_JX_H65)	+= jx_h65.o
obj-$(CONFIG_VIDEO_IMX214)	+= imx214.o
obj-$(CONFIG_VIDEO_IMX219)	+= imx219.o
obj-$(CONFIG_VIDEO_IMX258)	+= imx258.o
obj-$(CONFIG_VIDEO_IMX258_EEPROM)	+= imx258_eeprom.o
obj-$(CONFIG_VIDEO_IMX291)	+= imx291.o
obj-$(CONFIG_VIDEO_IMX307)	+= imx307.o
obj-$(CONFIG_VIDEO_IMX317)	+= imx317.o
obj-$(CONFIG_VIDEO_IMX323)	+= imx323.o
obj-$(CONFIG_VIDEO_IMX327)	+= imx327.o
obj-$(CONFIG_VIDEO_GC0403)	+= gc0403.o
obj-$(CONFIG_VIDEO_GC2155)	+= gc2155.o
obj-$(CONFIG_VIDEO_GC0312)	+= gc0312.o
obj-$(CONFIG_VIDEO_GC2145)	+= gc2145.o
obj-$(CONFIG_VIDEO_GC0329)	+= gc0329.o
obj-$(CONFIG_VIDEO_GC2035)	+= gc2035.o
obj-$(CONFIG_VIDEO_GC2053)	+= gc2053.o
obj-$(CONFIG_VIDEO_GC2355)	+= gc2355.o
obj-$(CONFIG_VIDEO_GC2385)	+= gc2385.o
obj-$(CONFIG_VIDEO_GC5025)	+= gc5025.o

Edit Kconfig and add imx327 config:

github.com

rockchip-linux/kernel/blob/develop-4.4/drivers/media/i2c/Kconfig#L560


	  This driver supports IMX317 cameras from Sony


config VIDEO_IMX323
	tristate "Sony imx323 sensor support"
	depends on VIDEO_V4L2 && I2C
	depends on MEDIA_CAMERA_SUPPORT
	select V4L2_FWNODE
	help
	  This driver supports IMX323 cameras from Sony


config VIDEO_IMX327
	tristate "Sony imx327 sensor support"
	depends on VIDEO_V4L2 && I2C
	depends on MEDIA_CAMERA_SUPPORT
	select V4L2_FWNODE
	help
	  This driver supports IMX327 cameras from Sony


config VIDEO_VIRT_CAMERA
	tristate "Rockchip virtual sensor support"
	depends on VIDEO_V4L2 && I2C

Create the new overlay with the new name overlays-rockpi4/imx327-overlay.dts, use this as a base:

github.com

radxa/kernel/blob/release-4.4-rockpi4/arch/arm64/boot/dts/rockchip/overlays-rockpi4/imx219-overlay.dts

/dts-v1/;
/plugin/;

/ {
	model = "ROCK PI 4B";
	compatible = "rockchip,rockpi","rockchip,rk3399";

	fragment@0 {
		target = <&i2c4>;

		__overlay__ {
			status = "okay";
		};
	};

	fragment@1 {
		target = <&camera1>;

		__overlay__ {
			status = "okay";

This file has been truncated. show original

Add the new overlay just created:

imx219.dtbo \
imx327.dtbo \

github.com

radxa/kernel/blob/release-4.4-rockpi4/arch/arm64/boot/dts/rockchip/overlays-rockpi4/Makefile#L20


	pcie-gen2.dtbo \
	spi1-waveshare35c.dtbo \
	spi1-waveshare35b-v2.dtbo \
	spi1-flash.dtbo \
	spi1-mcp2515-can0.dtbo \
	hifiberry-dac.dtbo \
	hifiberry-dacplus.dtbo \
	raspberrypi-7-inch-lcd.dtbo \
	gt9xx.dtbo \
	cpufreq.dtbo \
	imx219.dtbo \
	ov5647.dtbo \
	w1-gpio4-30.dtbo


targets += dtbs dtbs_install
targets += $(dtbo-y)


always		:= $(dtbo-y)
clean-files	:= *.dtbo

You tell me if this works and I will be forced to buy this sensor for my Rock3A.

Please, post your progress.

francesco.agostini · June 7, 2022, 7:39am

Hi @abel thank you for your support in the kernel driver and in the community. Using your driver and your media-ctl example finally I’m able to set camera format and stream video. I have just some issues with the ov5647 sensor, in any possible format, the max FPS that I can reach is 30 with 640x480 resolution. And there is some documentation about how to use the ISP scaler in the selfpath? This is my media-ctl setup:

media-ctl -d /dev/media0 --set-v4l2 '"ov5647 5-0036":0[fmt:Y10_1X10/640x480]'
media-ctl -d /dev/media0 --set-v4l2 '"rkisp-isp-subdev":0[fmt:Y10_1X10/640x480]'
media-ctl -d /dev/media0 --set-v4l2 '"rkisp-isp-subdev":0[crop:(0,0)/640x480]'
media-ctl -d /dev/media0 --set-v4l2 '"rkisp-isp-subdev":2[crop:(0,0)/640x480]'
v4l2-ctl -d /dev/video0 --set- selection=target=crop,top=0,left=0,width=640,height=480

for the ISP scale operation, I tried to follow this guide but the reported entity in this guide is not present with our ISP kernel driver.
https://www.kernel.org/doc/Documentation/admin-guide/media/rkisp1.rst

jmcguire525 · July 30, 2022, 8:00pm

@avaf Did you end up purchasing a IMX327 camera or do you know of a well supported camera that can stream 1080p/60 or 720p/120 with the Rock 3a?

avaf · August 2, 2022, 7:14pm

No, I didn’t buy IMX327. I’m still using 4.19.193 which is a bit outdated but works fine with OV5647. Maybe if you try to get it to work you should go with 4.19.219.

Sorry, I don’t know of any other camera to support 1080p/60, maybe the veye-327e with its driver can.

Update: I have ordered an imx219 camera (US$ 10) to see if i can get it to work, if it work i will be tempted to get IMX327. It takes at least ~45 days to get here.

imz · August 26, 2022, 12:11pm

Dot (need reminder if you comment for imx219)

avaf · November 15, 2023, 2:41pm

I found the SD card today and reviewed the result of ov5647 640x480@60. imx327 is still expensive to try out. imx219 has better quality, but the calibration file given to me is a bit off for my kernel. I will try the radxa calibration file and compare the results.

Avinadad_Mendez · February 18, 2024, 5:10am

Hello everyone, I’ve been reading all your posts

I was able to get rkaiq_3A_server running with my IMX586

Colors look good. But I don’t have auto-exposure, it is not working. Exposure is not adjusting itself

I am using Buildroot and compiled rkaiq_3A_server from Firefly repos

Any suggestion as to what to test or what to look for?