Rock 3A Camera support

I tried but only got a black frame, although i was at 60 fps.

Available pixel formats for /dev/video0:
  UYVY (59565955), UYVY 4:2:2, flags = 0
  422P (50323234), Planar YUV 4:2:2, flags = 0
  NV16 (3631564e), Y/CbCr 4:2:2, flags = 0
  NV61 (3136564e), Y/CrCb 4:2:2, flags = 0
  YM16 (36314d59), Planar YUV 4:2:2 (N-C), flags = 0
  NV21 (3132564e), Y/CrCb 4:2:0, flags = 0
  NV12 (3231564e), Y/CbCr 4:2:0, flags = 0
  NM21 (31324d4e), Y/CrCb 4:2:0 (N-C), flags = 0
  NM12 (32314d4e), Y/CbCr 4:2:0 (N-C), flags = 0
  YU12 (32315559), Planar YUV 4:2:0, flags = 0
  YM24 (34324d59), Planar YUV 4:4:4 (N-C), flags = 0
  RGGB (42474752), 8-bit Bayer RGRG/GBGB, flags = 0
  GRBG (47425247), 8-bit Bayer GRGR/BGBG, flags = 0
  GBRG (47524247), 8-bit Bayer GBGB/RGRG, flags = 0
  BA81 (31384142), 8-bit Bayer BGBG/GRGR, flags = 0
  RG10 (30314752), 10-bit Bayer RGRG/GBGB, flags = 0
  BA10 (30314142), 10-bit Bayer GRGR/BGBG, flags = 0
  GB10 (30314247), 10-bit Bayer GBGB/RGRG, flags = 0
  BG10 (30314742), 10-bit Bayer BGBG/GRGR, flags = 0
  RG12 (32314752), 12-bit Bayer RGRG/GBGB, flags = 0
  BA12 (32314142), 12-bit Bayer GRGR/BGBG, flags = 0
  GB12 (32314247), 12-bit Bayer GBGB/RGRG, flags = 0
  BG12 (32314742), 12-bit Bayer BGBG/GRGR, flags = 0

Control id: 9963788 - White Balance, Automatic
Control id: 9963793 - Exposure
Control id: 9963794 - Gain, Automatic
Control id: 9963796 - Horizontal Flip
Control id: 9963797 - Vertical Flip
Control id: 10094849 - Auto Exposure
Control id: 10356993 - Vertical Blanking
Control id: 10356994 - Horizontal Blanking
Control id: 10356995 - Analogue Gain
Control id: 10422529 - Link Frequency
Control id: 10422530 - Pixel Rate
Setting pixel_format: NV12 (640x480)
Image size: 614400 (640x480)
frame count: 1 - frames to skip: 602 - max. time: 0
fps: 62 - frame: 603 - tot. msecs: 9684939.

1920x1080@30fps works
640x480@60fps works but was really, really dark, that was the problem.

1 Like

use cmd “v4l2-ctl -d /dev/video0 -l”, and you will see:

root@rock-3a:~# v4l2-ctl -d /dev/video0 -l

User Controls

    white_balance_automatic 0x0098090c (bool)   : default=0 value=0
                   exposure 0x00980911 (int)    : min=4 max=500 step=1 default=500 value=500
             gain_automatic 0x00980912 (bool)   : default=0 value=0
            horizontal_flip 0x00980914 (bool)   : default=0 value=0 flags=modify-layout
              vertical_flip 0x00980915 (bool)   : default=0 value=0 flags=modify-layout

     Camera Controls

              auto_exposure 0x009a0901 (menu)   : min=0 max=1 default=1 value=1

    Image Source Controls

          vertical_blanking 0x009e0901 (int)    : min=24 max=32287 step=1 default=24 value=24
        horizontal_blanking 0x009e0902 (int)    : min=1212 max=1212 step=1 default=1212 value=1212 flags=read-only
              analogue_gain 0x009e0903 (int)    : min=16 max=1023 step=1 default=32 value=128

     Image Processing Controls

             link_frequency 0x009f0901 (intmenu): min=0 max=3 default=0 value=0
                 pixel_rate 0x009f0902 (int64)  : min=55000000 max=55000000 step=1 default=55000000 value=55000000 flags=read-only

then you can use cmd “v4l2-ctl --set-ctrl=analogue_gain=xxx” to change some param for camera, and you can change any of list ctrl(analogue_gain exposure white_balance_automatic … and so on).

for example: “v4l2-ctl --set-ctrl=analogue_gain=512”, you will see like this:

1

No problem, i have done that. I was streaming video at night… :slight_smile:

Just for fun, I wasn’t sure if it was possible to change the mode with v4l2-ctl, so i added a property to switch mode with overlay (or decompiling dtb and changing mode and then recompiling) without recompiling the kernel.

Maybe someone finds this useful.

Property is camera-mode:

&i2c5 {
	status = "okay";
	pinctrl-0 = <&i2c5m0_xfer>;

	ov5647: ov5647@36 {
		compatible = "ovti,ov5647";
		reg = <0x36>;
		status = "okay";
		clocks = <&ext_cam_clk>;
		pwdn-gpios = <&gpio4 RK_PD2 GPIO_ACTIVE_LOW>;
		camera-mode = <0>; /* 0 - full, 1 - 192x01080@30, 2, 3 - 640x480@60 */

		port {
			camera_out: endpoint {
				remote-endpoint = <&dphy_in>;
				data-lanes = <1 2>;
			};
		};
	};
};

add_ov5647_prop_mode.patch.zip (596 Bytes)

I know how to change mode. kernel driver is no bug, and you need’t to recompliling kernel or decompiling dtb. you can use cmd like this to change mode:

media-ctl -d /dev/media0 --set-v4l2 ‘“ov5647 5-0036”:0[fmt:SGBRG10_1X10/2592x1944]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[fmt:SGBRG10_1X10/2592x1944]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[crop:(0,0)/2592x1944]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:2[crop:(0,0)/2592x1944]’
v4l2-ctl -d /dev/video0 --set-selection=target=crop,top=0,left=0,width=2592,height=1944

media-ctl -d /dev/media0 --set-v4l2 ‘“ov5647 5-0036”:0[fmt:SGBRG10_1X10/1920x1080]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[fmt:SGBRG10_1X10/1920x1080]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[crop:(0,0)/1920x1080]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:2[crop:(0,0)/1920x1080]’
v4l2-ctl -d /dev/video0 --set-selection=target=crop,top=0,left=0,width=1920,height=1080

media-ctl -d /dev/media0 --set-v4l2 ‘“ov5647 5-0036”:0[fmt:SGBRG10_1X10/640x480]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[fmt:SGBRG10_1X10/640x480]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:0[crop:(0,0)/640x480]’
media-ctl -d /dev/media0 --set-v4l2 ‘“rkisp-isp-subdev”:2[crop:(0,0)/640x480]’
v4l2-ctl -d /dev/video0 --set-selection=target=crop,top=0,left=0,width=640,height=480

1 Like

Nice.

I would like to comment about the hflip and vflip, they do flip but the color of the image changes while streaming. I don’t see anything wrong with the code.

Can you test it and see what you get?

vflip=1

Greetings. Nice job. I’m very happy you solved it already.

Guys, I’m positively surprised, really. :heart:
Can you use h.265 encoding from camera stream with 1080p@100fps or at least 60fps?
I would use it together with this very sensitive camera:


Camera itself has 1080p@60fps and Sony claims their IMX327 sensor has 1080p@100fps.
That would be great to build drone camera module with OpenHD - fork of EZ-Wifibroadcast :slight_smile:

Interesting, there is support for imx327 in the kernel and rockchip camera engine, at least on rk3399. The driver setup is to 4 lanes, i don’t know if you just change it to 2 lanes in the code, and will work. If you have the camera you could try to enable the driver and add the node for the camera, but with the correct pins, something like:

&mipi_dphy_rx0 {
	status = "okay";

	ports {
		#address-cells = <1>;
		#size-cells = <0>;

		port@0 {
			reg = <0>;
			#address-cells = <1>;
			#size-cells = <0>;

			mipi_in_ucam0: endpoint@1 {
				reg = <1>;
				remote-endpoint = <&ucam_out0>;
				data-lanes = <1 2>;
			};
		};
     };
};

&i2c5 {
	status = "okay";
	pinctrl-0 = <&i2c5m0_xfer>;

	imx327: imx327@1a {
		compatible = "sony,imx327";
		status = "okay";
		reg = <0x1a>;
		clocks = <&cru SCLK_CIF_OUT>;
		clock-names = "xvclk";
		// find the correct pins for reset and pwdn, if needed
		// ?????????? reset-gpios = <&gpio4 26 GPIO_ACTIVE_HIGH>;
		pwdn-gpios = <&gpio4 RK_PD2 GPIO_ACTIVE_HIGH>;
		pinctrl-names = "default";
		pinctrl-0 = <&cif_clkout>;
		rockchip,camera-module-index = <0>;
		rockchip,camera-module-facing = "front";
		rockchip,camera-module-name = "TongJu";
		rockchip,camera-module-lens-name = "CHT842-MD";
		port {
			ucam_out0: endpoint {
				remote-endpoint = <&mipi_in_ucam0>;
				data-lanes = <1 2>;
			};
		};
	};
};

Let’s see what @abel has to say.

Regarding h265 encoding (hw), i tried 1080@30 and got reasonable to poor results, but without trying to understand how to change parameters or find ways to get better results. I doubt it will work with 60 fps, but it is just an opinion…

Wow! Wow!!! Okay I’m excited now. I have unapacked new Rock Pi 4B 1 GB on RK3399 and new boxed IMX327. Jeez!
What to do to start testing both? I installed Debian image from march this year but there was a lot of problems en wasn’t even able to run connected OV5647 - there wasn’t v4l-utils, there was nothing at all. Still can’t find test_camera-rkisp1.sh. I even not imagine about conntecting 327 to this board in this situation :confused:
If you can say me first steps and how and where to do this. I’m not totally rookie, but on this platform I’m totally green.
I’m okay with h.264 on this board. For h.265 I know now that RV1126 is the best and thinking about invest into this board later on.

The first step is to know which imx327 chip you have in the camera. i googled a bit and there are some variants. IMX327LQR-C, IMX327S. They have different chip id. Seems to be 0xb2 or 0x6, possibly.
Cross your finger the chip id is 0xb2.

◆Readout rate
Maximum frame rate in Full HD 1080p mode: 60 frame / s

In the second step, imx327 must be enabled if not already, so a kernel rebuild is likely. Get used to the build process, i build everything manually so i don’t know much about the radxa build system, can’t help on this.

CONFIG_VIDEO_IMX327=y (or m if you need to rebuild the driver)

Rock3A is with kernel 4.19.y , the RockPi 4 is with kernel 4.4.y (so i think, i don’t have one).
It is possible to run kernel 4.19.y on rk3399 , i have a nanopi m4 running with kernel 4.19.193. You need to check out the latest imx327.c in kernel 4.4.y , or build 4.19.193 for your board.

Before you build, change IMX327_4LANES to IMX327_2LANES in imx327.c

Third step, is to create the new overlay for the camera imx327 (rockpi 4B dtso) with the code i posted above or edit the rockpi 4B dts and add the code there.

We need some advice about the pins for your board, we need the guys who know about the hardware, poke them:
reset-gpios = <&gpio4 26 GPIO_ACTIVE_HIGH>;
pwdn-gpios = <&gpio4 25 GPIO_ACTIVE_HIGH>;

The last step, make sure you have the iqfile imx327_TongJu_CHT842-MD.xml and the camera engine.

Thank you. I think I’ll be able to follow your instruction, but will be hard I feel :slight_smile: It’s definitely LQR regarding above innomaker info. Not sure what id it has - didn’t conect it yet.

Here is my take on the matter, kernel 4.4 (RockPi 4B):

&mipi_dphy_rx0 {
	status = "okay";

	ports {
		#address-cells = <1>;
		#size-cells = <0>;

		port@0 {
			reg = <0>;
			#address-cells = <1>;
			#size-cells = <0>;

			mipi_in_ucam0: endpoint@1 {
				reg = <1>;
				remote-endpoint = <&ucam_out0>;
				data-lanes = <1 2>;
			};
		};
     };
};

&i2c4 {
	status = "okay";

	imx327: imx327@1a {
		compatible = "sony,imx327";
		status = "okay";
		reg = <0x1a>;
		clocks = <&cru SCLK_CIF_OUT>;
		clock-names = "xvclk";
		pwdn-gpios = <&gpio1 RK_PB5 GPIO_ACTIVE_HIGH>;
		pinctrl-names = "rockchip,camera_default", "rockchip,camera_sleep";
		pinctrl-0 = <&cam0_default_pins &cif_clkout_a>;
		pinctrl-1 = <&cam0_default_pins>;
		rockchip,camera-module-index = <0>;
		rockchip,camera-module-facing = "front";
		rockchip,camera-module-name = "TongJu";
		rockchip,camera-module-lens-name = "CHT842-MD";
		port {
			ucam_out0: endpoint {
				remote-endpoint = <&mipi_in_ucam0>;
				data-lanes = <1 2>;
			};
		};
	};
};

Or a simpler way is to edit this file, and change it to imx327:

Use this driver here and comment on the line:

Edit the Makefile and add the new driver:

Edit Kconfig and add imx327 config:

Create the new overlay with the new name overlays-rockpi4/imx327-overlay.dts, use this as a base:

Add the new overlay just created:

imx219.dtbo \
imx327.dtbo \

You tell me if this works and I will be forced to buy this sensor for my Rock3A.

Please, post your progress.

Hi @abel thank you for your support in the kernel driver and in the community. Using your driver and your media-ctl example finally I’m able to set camera format and stream video. I have just some issues with the ov5647 sensor, in any possible format, the max FPS that I can reach is 30 with 640x480 resolution. And there is some documentation about how to use the ISP scaler in the selfpath? This is my media-ctl setup:

media-ctl -d /dev/media0 --set-v4l2 '"ov5647 5-0036":0[fmt:Y10_1X10/640x480]'
media-ctl -d /dev/media0 --set-v4l2 '"rkisp-isp-subdev":0[fmt:Y10_1X10/640x480]'
media-ctl -d /dev/media0 --set-v4l2 '"rkisp-isp-subdev":0[crop:(0,0)/640x480]'
media-ctl -d /dev/media0 --set-v4l2 '"rkisp-isp-subdev":2[crop:(0,0)/640x480]'
v4l2-ctl -d /dev/video0 --set- selection=target=crop,top=0,left=0,width=640,height=480

for the ISP scale operation, I tried to follow this guide but the reported entity in this guide is not present with our ISP kernel driver.
https://www.kernel.org/doc/Documentation/admin-guide/media/rkisp1.rst

@avaf Did you end up purchasing a IMX327 camera or do you know of a well supported camera that can stream 1080p/60 or 720p/120 with the Rock 3a?

No, I didn’t buy IMX327. I’m still using 4.19.193 which is a bit outdated but works fine with OV5647. Maybe if you try to get it to work you should go with 4.19.219.

Sorry, I don’t know of any other camera to support 1080p/60, maybe the veye-327e with its driver can.

Update: I have ordered an imx219 camera (US$ 10) to see if i can get it to work, if it work i will be tempted to get IMX327. It takes at least ~45 days to get here.

Dot (need reminder if you comment for imx219)

I found the SD card today and reviewed the result of ov5647 640x480@60. imx327 is still expensive to try out. imx219 has better quality, but the calibration file given to me is a bit off for my kernel. I will try the radxa calibration file and compare the results.

Hello everyone, I’ve been reading all your posts

I was able to get rkaiq_3A_server running with my IMX586

Colors look good. But I don’t have auto-exposure, it is not working. Exposure is not adjusting itself

I am using Buildroot and compiled rkaiq_3A_server from Firefly repos

Any suggestion as to what to test or what to look for?