How to use Radxa Camera 4K

avaf · August 3, 2023, 6:39pm

compiled on board?

sergei_gagarin · August 3, 2023, 10:59pm

Did you ask me?
Yes, compiled on board. With small obvious changes (mostly concerning compiler and link flags).

avaf · August 3, 2023, 11:34pm

Mind sharing a tar file of your complete build tree and command?

@sergei_gagarin
I clicked <-+ Reply button to reply, but this did not work.
If the tar file can’t be shared here, please use mega.nz or some other link to download, thanks.

sergei_gagarin · August 4, 2023, 2:00am

Here are the files I edited and a small instruction how to compile server and library on your board.
build_rkaiq.zip (2.8 KB)

May be some standard packages are required (sure, gcc, cmake and so on are obligated).

Please note that I compiled it some months ago (that’s why “git reset” in instruction). Recently Firefly added some changes (new version 5 of rkaiq). I did not work with it. May be additional editing will be required in that case.

avaf · August 4, 2023, 3:25pm

With that repo and your changes, i was able to compile. i did not have to follow your libdrm and librga changes:

rock@rock5b:/apps/rockchip/camera_engine_rkaiq/build/linux$ ldd output/aarch64/all_lib/RelWithDebInfo/librkaiq.so
	linux-vdso.so.1 (0x0000007f85983000)
	libdrm.so.2 => /lib/aarch64-linux-gnu/libdrm.so.2 (0x0000007f85300000)
	libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f850d0000)
	libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f85030000)
	libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f85000000)
	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f84e50000)
	/lib/ld-linux-aarch64.so.1 (0x0000007f8594a000)
rock@rock5b:/apps/rockchip/camera_engine_rkaiq/build/linux$ ldd output/aarch64/exe/RelWithDebInfo/rkaiq_3A_server
	linux-vdso.so.1 (0x0000007f91652000)
	librkaiq.so => /lib/librkaiq.so (0x0000007f90fb0000)
	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f90e00000)
	/lib/ld-linux-aarch64.so.1 (0x0000007f91619000)
	libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f90de0000)
    libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f90bb0000)
	libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f90b10000)
	libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f90ae0000)

The next step is to check if it works , thanks.

Just for the record.
I am on Ubuntu:

lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.2 LTS
Release:	22.04
Codename:	jammy

the new version:

sudo /usr/bin/rkaiq_3A_server
Cound not find rkisp dev names, skipped /dev/media0
ERR: Bad media topology for: /dev/media0
DBG: get rkisp-isp-subdev devname: /dev/v4l-subdev3
DBG: get rkisp-input-params devname: /dev/video19
DBG: get rkisp-statistics devname: /dev/video18
DBG: get rkisp_mainpath devname: /dev/video11
rkaiq log level ff0

************************** VERSION INFOS **************************
version release date: 2022-09-20
         AIQ:       AIQ v3.0x9.1
   IQ PARSER:       Calib v1.4.8,magicCode:1170944
************************ VERSION INFOS END ************************
ERR: Bad media topology for: /dev/media2
ERR: Bad media topology for: /dev/media3
ERR: Bad media topology for: /dev/media4
ERR: Bad media topology for: /dev/media5
ERR: Bad media topology for: /dev/media6
ERR: Bad media topology for: /dev/media7
ERR: Bad media topology for: /dev/media8
ERR: Bad media topology for: /dev/media9
ERR: Bad media topology for: /dev/media10
ERR: Bad media topology for: /dev/media11
ERR: Bad media topology for: /dev/media12
ERR: Bad media topology for: /dev/media13
ERR: Bad media topology for: /dev/media14
ERR: Bad media topology for: /dev/media15
Segmentation fault

the old unknown version:

sudo /usr/bin/rkaiq_3A_server
media get entity by name: stream_cif_dvp_id0 is null
media get entity by name: stream_cif_dvp_id1 is null
media get entity by name: stream_cif_dvp_id2 is null
media get entity by name: stream_cif_dvp_id3 is null
media get entity by name: rkcif-mipi-luma is null
media get entity by name: rkcif-lvds-subdev is null
media get entity by name: rkcif-lite-lvds-subdev is null
media get entity by name: rockchip-mipi-dphy-rx is null
media get entity by name: stream_cif is null
media get entity by name: rkcif-dvp-sof is null
media get entity by name: rkisp-csi-subdev is null
media get entity by name: rkisp-mpfbc-subdev is null
media get entity by name: rkisp_rawwr0 is null
media get entity by name: rkisp_rawwr1 is null
media get entity by name: rkisp_rawwr2 is null
media get entity by name: rkisp_rawwr3 is null
media get entity by name: rkisp_dmapath is null
media get entity by name: rkisp-mipi-luma is null
media get entity by name: rockchip-mipi-dphy-rx is null
media get entity by name: rockchip-csi2-dphy0 is null
media get entity by name: rkcif_dvp is null
media get entity by name: rkcif_dvp is null
media get entity by name: rkcif_lite_mipi_lvds is null
media get entity by name: rkcif_mipi_lvds is null
media get entity by name: rkcif_mipi_lvds1 is null
media get entity by name: rkcif_mipi_lvds2 is null
media get entity by name: rkcif_mipi_lvds3 is null
media get entity by name: rkcif_mipi_lvds4 is null
media get entity by name: rkcif_mipi_lvds5 is null
media get entity by name: rkcif-mipi-lvds is null
media get entity by name: rkcif-mipi-lvds1 is null
media get entity by name: rkcif-mipi-lvds3 is null
media get entity by name: rkcif-mipi-lvds4 is null
media get entity by name: rkcif-mipi-lvds5 is null
media get entity by name: rkisp-isp-subdev is null
media get entity by name: rkisp-input-params is null
media get entity by name: rkisp-statistics is null
media get entity by name: rkisp_mainpath is null
Cound not find rkisp dev names, skipped /dev/media0
ERR: Bad media topology for: /dev/media0
DBG: get rkisp-isp-subdev devname: /dev/v4l-subdev3
DBG: get rkisp-input-params devname: /dev/video19
DBG: get rkisp-statistics devname: /dev/video18
DBG: get rkisp_mainpath devname: /dev/video11
ERR: Bad media topology for: /dev/media2
ERR: Bad media topology for: /dev/media3
ERR: Bad media topology for: /dev/media4
ERR: Bad media topology for: /dev/media5
ERR: Bad media topology for: /dev/media6
ERR: Bad media topology for: /dev/media7
ERR: Bad media topology for: /dev/media8
ERR: Bad media topology for: /dev/media9
ERR: Bad media topology for: /dev/media10
ERR: Bad media topology for: /dev/media11
ERR: Bad media topology for: /dev/media12
ERR: Bad media topology for: /dev/media13
ERR: Bad media topology for: /dev/media14
ERR: Bad media topology for: /dev/media15
[11:59:50.553631][XCORE]:XCAM ERROR rk_aiq_user_api_sysctl.cpp:392: Should use json instead of xml
[11:59:50.631867][XCORE]:XCAM ERROR RkAiqCalibDbV2.cpp:253: write /tmp/iq_dump.json failed!
[11:59:50.631915][XCORE]:XCAM ERROR rk_aiq_user_api_sysctl.cpp:408: Failed to find params of : scene in json, using default scene
[11:59:50.635584][XCORE]:XCAM ERROR socket_server.cpp:481: Error bind Address already in use

[11:59:50.635913][CAMHW]:XCAM ERROR SensorHw.cpp:1321: failed to set hdr mode 0
[11:59:50.635942][CAMHW]:XCAM ERROR SensorHw.cpp:1291: failed to set sync mode 0
[11:59:50.636741][ADEHAZE]:XCAM ERROR rk_aiq_adehaze_algo.cpp:2177: AdehazeGetCurrData:PreResBuf is NULL!

DBG: subscribe events from /dev/video19 success !
DBG: /dev/media1: wait stream start event...

Looks like it can’t find the config file.

avaf · August 4, 2023, 11:02pm

I am upgrading to rkr4.1 and see what i get…

zoldaten · August 9, 2023, 1:45am

i moved from Debian to /etc/iqfiles Ubuntu 2 files:

imx415_CMK-OT2022-PX1_IR0147-50IRC-8M-F20.json
imx415_RADXA-CAMERA-4K_DEFAULT.json
restarted and checked.
As there was no result i renamed imx415_CMK-OT2022-PX1_IR0147-50IRC-8M-F20.json to imx415_RADXA-CAMERA-4K_DEFAULT.json
and restarted to check again - no result.

rkaiq_3A.service not crashing but gives some errors as i think:

rkaiq_3A.service - rkisp 3A engine
Loaded: loaded (/lib/systemd/system/rkaiq_3A.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2023-08-04 22:03:00 MSK; 21h ago
Process: 886 ExecStart=/etc/init.d/rkaiq_3A.sh start (code=exited, status=0/SUCCESS)
Tasks: 7 (limit: 4370)
Memory: 19.6M
CGroup: /system.slice/rkaiq_3A.service
├─889 /usr/bin/rkaiq_3A_server
└─890 logger -t rkaiq

авг 04 22:15:30 rock rkaiq[890]: DBG: device manager start
авг 04 22:15:30 rock rkaiq[890]: DBG: rkisp_init engine succeed
авг 04 22:15:30 rock rkaiq[890]: DBG: /dev/media1: wait stream stop event…
авг 04 22:15:30 rock rkaiq[890]: [22:15:30.564231][CAMHW]:XCAM ERROR Isp20Params.cpp:4036: get blc params fa>
авг 04 22:15:30 rock rkaiq[890]:
авг 04 22:16:25 rock rkaiq[890]: DBG: /dev/media1: wait stream stop event success …
авг 04 22:16:25 rock rkaiq[890]: [22:16:25.063288][CAMHW]:XCAM ERROR SensorHw.cpp:1291: failed to set sync m>
авг 04 22:16:25 rock rkaiq[890]: DBG: subscribe events from /dev/video19 success !
авг 04 22:16:25 rock rkaiq[890]: DBG: subscribe events from /dev/video19 success !
авг 04 22:16:25 rock rkaiq[890]: DBG: /dev/media1: wait stream start event…

i also tried to go on with this issue - Has anyone actually succeeded in getting hardware video encoding working?
but only i got - working OBS studio. not bad as Kasam not working there recording just black screen.

zoldaten · August 5, 2023, 6:00pm

i tried to compile but at the last step:
./make-Makefiles-aarch64.bash
i see:

./make-Makefiles-aarch64.bash
~/Downloads/test/camera_engine_rkaiq/build/linux/output/aarch64 ~/Downloads/test/camera_engine_rkaiq/build/linux
CMake Deprecation Warning at CMakeLists.txt:11 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.

Update the VERSION argument value or use a … suffix to tell
CMake that the project does not need compatibility with older versions.

– The C compiler identification is unknown
– The CXX compiler identification is unknown
CMake Error at CMakeLists.txt:13 (PROJECT):
The CMAKE_C_COMPILER:
/home/camera/camera/rk356x_linux/buildroot/output/rockchip_rk3566/host/bin/aarch64-buildroot-linux-gnu-gcc
is not a full path to an existing compiler tool.

Tell CMake where to find the compiler by setting either the environment
variable “CC” or the CMake cache entry CMAKE_C_COMPILER to the full path to
the compiler, or to the compiler name if it is in the PATH.

CMake Error at CMakeLists.txt:13 (PROJECT):
The CMAKE_CXX_COMPILER:
/home/camera/camera/rk356x_linux/buildroot/output/rockchip_rk3566/host/bin/aarch64-buildroot-linux-gnu-g++
is not a full path to an existing compiler tool.

Tell CMake where to find the compiler by setting either the environment
variable “CXX” or the CMake cache entry CMAKE_CXX_COMPILER to the full path
to the compiler, or to the compiler name if it is in the PATH.

– Configuring incomplete, errors occurred!
See also “/home/rock/Downloads/test/camera_engine_rkaiq/build/linux/output/aarch64/CMakeFiles/CMakeOutput.log”.
See also “/home/rock/Downloads/test/camera_engine_rkaiq/build/linux/output/aarch64/CMakeFiles/CMakeError.log”.
~/Downloads/test/camera_engine_rkaiq/build/linux

incognito · August 5, 2023, 9:27pm

Looks like they are hardcoding paths to their executables. Sloppy

avaf · August 6, 2023, 12:24am

FYI, this version worked for my kernel:

************************** VERSION INFOS **************************
version release date: 2023-03-02
         AIQ:       AIQ v5.0x1.2-rc2
   IQ PARSER:       Calib v1.4.8,magicCode:1170944
************************ VERSION INFOS END ************************

sergei_gagarin · August 7, 2023, 4:15pm

Looks like you did not un-tar archive changed.tgz (or un-tared it in wrong directory).

sergei_gagarin · August 7, 2023, 4:18pm

Try to use the library and server from here: https://disk.yandex.ru/d/tPnXrLB5xSy0vg
(ignore JSON-file in this archive).

zoldaten · August 7, 2023, 5:09pm

overwrited and reboot - server is not starting at all:

./rkaiq_3A_server
Cound not find rkisp dev names, skipped /dev/media0
ERR: Bad media topology for: /dev/media0
DBG: get rkisp-isp-subdev devname: /dev/v4l-subdev3
DBG: get rkisp-input-params devname: /dev/video19
DBG: get rkisp-statistics devname: /dev/video18
DBG: get rkisp_mainpath devname: /dev/video11
rkaiq log level ff0

************************** VERSION INFOS **************************
version release date: 2022-09-20
AIQ: AIQ v3.0x9.1
IQ PARSER: Calib v1.4.8,magicCode:1170944
************************ VERSION INFOS END ************************
ERR: Bad media topology for: /dev/media2
ERR: Bad media topology for: /dev/media3
ERR: Bad media topology for: /dev/media4
ERR: Bad media topology for: /dev/media5
ERR: Bad media topology for: /dev/media6
ERR: Bad media topology for: /dev/media7
ERR: Bad media topology for: /dev/media8
ERR: Bad media topology for: /dev/media9
ERR: Bad media topology for: /dev/media10
ERR: Bad media topology for: /dev/media11
ERR: Bad media topology for: /dev/media12
ERR: Bad media topology for: /dev/media13
ERR: Bad media topology for: /dev/media14
ERR: Bad media topology for: /dev/media15
Segmentation fault (core dumped)

zoldaten · August 7, 2023, 6:19pm

ok. i built it with no errors (i think).
see 2 dirs in output: aarch64 and arm.

what should next ?

sergei_gagarin · August 7, 2023, 11:05pm

Either JSON files in /etc/iqfiles are not compatible with this version of rkaiq, or names of your camera are different.

First of all, find your camera’s DTS settings.
How did you added support of your camera ? Did you manually add some overlay to DTS or the camera is already supported by your system ?

If you use overlay then look for “imx415” node in some “i2c-???” section in overlay file and find values of parameters “rockchip,camera-module-name” and “rockchip,camera-module-lens-name”.
Note. If your overlay is a dtbo-file, then you can convert it to text format with command:
dtc -I dtb -O dts -o my.dtso YOUR_DTBO_FILE
and look at my.dtso file.

If you do not use overlay (or even if you use it ) then go to directory /sys/bus/i2c/drivers/imx415. You will find there some directory (symbolic link) named like “3-0036” (digits will differ), go to this directory and execute commands:
cat of_node/rockchip,camera-module-name && echo
cat of_node/rockchip,camera-module-lens-name && echo

Now you know your camera names (module name and lens name). JSON file for your camera must have name “imx415_MODULENAME_LENSNAME.json”.

If server still crashes then this JSON file is not compatible with server’s version. You should search them (server/library and/or JSON files) in other places. I have no such camera, so I cannot help here.
You could try to ask “avaf” for version 5 (AIQ v5), see his post above.

sergei_gagarin · August 7, 2023, 10:59pm

Server executable must be in subdirectory in directory aarch64/exe.
Library (librkaiq.so) must be in subdirectory in directory aarch64/all_lib.

But Indeed you don’t need these files already, as server from my archive is runnable.

zoldaten · August 8, 2023, 6:34am

ive replaced files but server doesnt start after reboot:

./rkaiq_3A_server
Cound not find rkisp dev names, skipped /dev/media0
ERR: Bad media topology for: /dev/media0
DBG: get rkisp-isp-subdev devname: /dev/v4l-subdev3
DBG: get rkisp-input-params devname: /dev/video19
DBG: get rkisp-statistics devname: /dev/video18
DBG: get rkisp_mainpath devname: /dev/video11
rkaiq log level ff0

************************** VERSION INFOS **************************
version release date: 2022-09-20
AIQ: AIQ v3.0x9.1
IQ PARSER: Calib v1.4.8,magicCode:1170944
************************ VERSION INFOS END ************************
ERR: Bad media topology for: /dev/media2
ERR: Bad media topology for: /dev/media3
ERR: Bad media topology for: /dev/media4
ERR: Bad media topology for: /dev/media5
ERR: Bad media topology for: /dev/media6
ERR: Bad media topology for: /dev/media7
ERR: Bad media topology for: /dev/media8
ERR: Bad media topology for: /dev/media9
ERR: Bad media topology for: /dev/media10
ERR: Bad media topology for: /dev/media11
ERR: Bad media topology for: /dev/media12
ERR: Bad media topology for: /dev/media13
ERR: Bad media topology for: /dev/media14
ERR: Bad media topology for: /dev/media15
Segmentation fault (core dumped)

zoldaten · August 8, 2023, 7:34am

How did you added support of your camera ? Did you manually add some overlay to DTS or the camera is already supported by your system ?

as noted in image wiki github com/Joshua-Riek/ubuntu-rockchip/wiki/Device-Tree-Overlays i added overlay:
sudo nano /boot/firmware/ubuntuEnv.txt
overlays=rock-5b-radxa-camera-4k

If your overlay is a dtbo-file, then you can convert it to text format with command:
dtc -I dtb -O dts -o my.dtso YOUR_DTBO_FILE
and look at my.dtso file.

in /boot/firmware/dtbs/overlays of the distro i see rock-5b-radxa-camera-4k.dtbo

dtc -I dtb -O dts -o my.dtso rock-5b-radxa-camera-4k.dtbo

while conversion i see:
my.dtso: Warning (unit_address_vs_reg): /local_fixups/fragment@0/overlay/imx415@1a: node has a unit name, but no reg or ranges property
my.dtso: Warning (unit_address_vs_reg): /local_fixups/fragment@2/overlay/ports/port@0: node has a unit name, but no reg or ranges property
my.dtso: Warning (unit_address_vs_reg): /local_fixups/fragment@2/overlay/ports/port@0/endpoint@1: node has a unit name, but no reg or ranges property
my.dtso: Warning (unit_address_vs_reg): /local_fixups/fragment@2/overlay/ports/port@1: node has a unit name, but no reg or ranges property
my.dtso: Warning (unit_address_vs_reg): /local_fixups/fragment@2/overlay/ports/port@1/endpoint@0: node has a unit name, but no reg or ranges property
my.dtso: Warning (unit_address_vs_reg): /local_fixups/fragment@3/overlay/ports/port@0: node has a unit name, but no reg or ranges property
my.dtso: Warning (unit_address_vs_reg): /local_fixups/fragment@3/overlay/ports/port@0/endpoint@1: node has a unit name, but no reg or ranges property
my.dtso: Warning (unit_address_vs_reg): /local_fixups/fragment@3/overlay/ports/port@1: node has a unit name, but no reg or ranges property
my.dtso: Warning (unit_address_vs_reg): /local_fixups/fragment@3/overlay/ports/port@1/endpoint@0: node has a unit name, but no reg or ranges property
my.dtso: Warning (unit_address_vs_reg): /local_fixups/fragment@10/overlay/port/endpoint@0: node has a unit name, but no reg or ranges property
my.dtso: Warning (clocks_property): /fragment@0/overlay/imx415@1a:clocks: Could not get phandle node for (cell 1)
my.dtso: Warning (power_domains_property): /fragment@0/overlay/imx415@1a:power-domains: Could not get phandle node for (cell 1)
my.dtso: Warning (gpios_property): /fragment@0/overlay/imx415@1a: Missing property ‘#gpio-cells’ in node /fragment@6/overlay/port/endpoint or bad phandle (referred from pwdn-gpios[1])
my.dtso: Warning (gpios_property): /fragment@0/overlay/imx415@1a: Missing property ‘#gpio-cells’ in node /fragment@2/overlay/ports/port@0/endpoint@1 or bad phandle (referred from reset-gpios[2])
my.dtso: Warning (graph_child_address): /fragment@2/overlay/ports/port@1: graph node has single child node ‘endpoint@0’, #address-cells/#size-cells are not necessary
my.dtso: Warning (graph_child_address): /fragment@3/overlay/ports/port@1: graph node has single child node ‘endpoint@0’, #address-cells/#size-cells are not necessary
my.dtso: Warning (graph_child_address): /fragment@10/overlay/port: graph node has single child node ‘endpoint@0’, #address-cells/#size-cells are not necessary

If you do not use overlay (or even if you use it ) then go to directory /sys/bus/i2c/drivers/imx415. You will find there some directory (symbolic link) named like “3-0036” (digits will differ), go to this directory and exec commands:
cat of_node/rockchip,camera-module-name && echo
cat of_node/rockchip,camera-module-lens-name && echo

i see:
RADXA-CAMERA-4K
and
DEFAULT

zoldaten · August 8, 2023, 9:40am

hi.
what image ubuntu 22.04 do u use ?

avaf · August 8, 2023, 12:54pm

It’s my custom image for development. It’s always behind the known distros

I think the camera engine must match the json file and i suspect modern gcc optimization break the code somewhere (the parser ?) if not cross-compiled.

You can try to build this one: (https://github.com/radxa/meta-rockchip/blob/kirkstone-radxa/recipes-multimedia/rockchip-rkaiq/rockchip-rkaiq.bb)

Here is the json file i use:

rw-rw-rw- 1 root root 732267 Aug 7 2022 imx415_CMK-OT2022-PX1_IR0147-50IRC-8M-F20.json

so
imx415_CMK-OT2022-PX1_IR0147-50IRC-8M-F20.json
imx415_RADXA-CAMERA-4K_DEFAULT.json
should be the same.

In theory, copying all relevant files from Debian to Ubuntu should work:

rock@rock5b:~$ cat /usr/lib/systemd/system/rkaiq_3A.service
[Unit]
Description=Enable Rockchip camera engine rkaiq
DefaultDependencies=no
Before=rockchip.service

[Service]
Type=forking
ExecStart=/etc/init.d/rkaiq_3A.sh start
ExecStop=/etc/init.d/rkaiq_3A.sh stop

[Install]
WantedBy=sysinit.target
rock@rock5b:~$ cat /etc/init.d/rkaiq_3A.sh
#!/bin/bash
### BEGIN INIT INFO
# Provides:          rkaiq_3a
# Required-Start:
# Required-Stop:
# Default-Start:
# Default-Stop:
# Short-Description:
# Description:       Setup rkaiq 3a daemon
### END INIT INFO

start_3A()
{
    /usr/bin/rkaiq_3A_server 2>&1 | logger -t rkaiq &
}

stop_3A()
{
    killall rkaiq_3A_server || true
}


case "$1" in
  start)
    start_3A
    ;;
  stop)
    stop_3A
    ;;
  reload)
    stop_3A
    start_3A
    ;;
  *)
    echo "Usage: $0 {start|stop}"
    exit 1
    ;;
esac
exit 0

Plus the files you listed, but the kernel version must be the same