Crap, I was under the impression that I had the datasheet for the IMX477, but only now did I notice that the page numbering conveniently skips from 24 to 40… But the RPi driver for the IMX477 contains registers I think.
Connect Arducam HQ Camera to ROCK5B
For my purposes, I’m fine with just getting 8 or 10 bit raw sensor data. No concerns about Bayer color filters or AWB, for example. If I can skip the ISP and any associated latency, all the better. The VICAP module appears to be capable of DMAing frames without a trip through the ISP, based on the diagram in the TRM. However, maybe disabling all of the ISP processing stages will be just about as good. It’s disappointing that the 5,000 page TRM lacks a chapter for the ISP — I assume that’s intentional.
If I2C is doing it’s thing, I’m wondering why the doubt about the camera being detected. Seems like I should also be able to get raw pixel values streaming via CSI. The (valid) concern raised seems to be about getting the necessary calibration data for the ISP to do it’s thing and get nice looking color images.
In the BSP kernel, it looks like the RK8588S ISP is designated isp30 in drivers/media/platform/rockchip/isp and you can set up the ISP through a V4L2 parameter subdevice. Not all that different from what is being done in libcamera for rkisp1.
@Zipdox This document gives you an idea of the process involved in fine tuning for RPi ISP. I think the process is much the same for Rockchip. Maybe it’s possible to convert a substantial part of the RPi JSON files over? The ISPs would be doing pretty much the same tasks, but likely have different requirements for number and format for parameters.
Hi,
I’m also working on similar project that adapts the PI HQ camera for another RK3588S board (cool pi 4b).
The board shares the same csi-2 interface with the raspberry pi 4b.
So, my work mostly focusing on the driver of the camera and ISP tuning.
I’m able to get the driver side working and sampling some dark-green NV12 images.
However, tuning the RK3588s ISP is not a single-man or DIY job. (For its ancestor chips, a OEM tuning binary is required but I could not find the one for RK3588s.)
Therefore, I’m thinking about getting a software ISP.
@eshelton would you mind providing any advises on how to turn off the hardware ISP and how to get the raw sensor data?
I really appreciate that!
BTW, the driver provided from the RPI would also need some modifications to fit into the rock-chip BSP.
You got the kernel driver and device overlay working? I’m fairly sure there’s a /dev/video device that bypasses the ISP.
Yes, I got the kernel driver and device tree working.
Cool pi 4b + Ubuntu don’t use the overlay, therefore I modified the device tree and got it working.
Yes, there supposes to be a node (/dev/video0 or /dev/video1) that bypasses the ISP.
But the image samples are not correct, something like the attached image.
Yeah that looks like an incorrect sample format. Could you share some details about your software setup? Also it is worthy of note that the cool PI 4B only has 2 CSI lanes since it uses the same connector. That means you can’t use the full potential of the RK3588 and camera.
Yes, I used the v4l2-ctl command to sample a 1920x1080 stream ()
sudo v4l2-ctl --verbose -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat='NV12' --stream-mmap=4 --set-selection=target=crop,flags=0,top=0,left=0,width=1920,height=1080 --stream-to=out1.yuv
And then used the ffplay to visualize it.
ffplay out1.yuv -pixel_format nv12 -video_size 1920x1080
(There is some buffer bug inside the rockchip mipi driver that I couldn’t stream 4K resolution in RAW format, but I could stream the 4K resolution after ISP.)
Cannot complain, but have to say the rockchip documents are not well organized.
Yes, it only comes with the 2 data lanes.
I planned to use the HQ camera for star rail photography of which the bottleneck is the post-processing (Raspberry PI 4B is so so slow on that).
But I came across the same problem as @Raphael discussed. (All my inquires to Rockchip were ignored, even though my startup has made a fair amount of purchases from them)
Therefore, I decided to work on the software ISP.
exactly what I thought would happen 
I had a meeting with Arducam yesterday, they’re considering supporting the Rock5 and are working on providing cameras for this platform.
It may take some time, though, but the interest is there and they already started on that.
I also told them about the very bad tuned radxa 4k camera, maybe they’re able to make it a little better.
When I know more I’ll post a few infos here and I’ll ask Arducam to join this forum 
2 Likes
Tell them we’re looking forward to it. Would buy it no questions asked.
This is the most inspiring news I had this week:)
Thanks @Raphael so much for forwarding our issue to Arducam:)
If there is anything I could help (like drivers, testing, etc), please don’t hesitate to let me know 
Bring it on.
I am willing to help to test it and give some feedback, or help somehow. Arducam, if you are following this, take note, please.
1 Like
Interesting, I think years ago we contacted Arducam for collaboration on the camera modules but we never got a reply from them.
@Raphael any news from the Arducam team? Are they still working on it, or the project is cancelled and we we proceed with our own hand-made adapters and kernel drivers? 
It isn’t cancelled, but I also don’t know any publishable facts yet.
What I can tell is that arducam and radxa had/have a meeting today
Sounds like a really long-term project to me
I did a quick research and found that RPI hq camera indeed has voltage step-down bundled inside the camera board, where RPI’s 3.3 volts become 1.8 which sensor requires, so there is no way that adapter layout proposed before would work (I checked, even i2c is unstable on 1.8v). The MIPI pinout and voltage seem to be part of some standart, so there should not be any issues there.
I managed to create a small board which converts 1.8v signals received from OPI 5 to 3.3v expected by the arducam/rpi hq camera, In couple of weeks I’ll see if it works.
Also I found this article where it is explained how to make IMX415 sensor work with similar processor and same kernel version, so it seems there should not be a big issue to make IMX477 work too.
Of course, by “not a big issue” I mean that I’m expecting something like proof-of-concept and not hi-end quality camera without any bugs and issues
1 Like
/sorry for posting another message in the row, but it appears that newbies can add only 2 links in a single message/
Btw, here are some existing adapter boards which do the same stuff that I did:
https://wiki.veye.cc/index.php/ADP-Tfirefly
https://wiki.veye.cc/index.php/ADP-MV2_Adapter_Board_Data_Sheet/zh
First one should be easily modified if mine would not work properly.
UPD 26.05
My custom 1.8 -> 3.3 volts converter board worked and now I’m able to see my imx477 on i2c and even get it’s chip id propery.
Even with probe() method of the driver working properly, I don’t see any new devices in media-ctl -p output and also none of /dev/video* are linked to my camera. That is next challange which I think would take a lot of time, because the issue might be in dts config, which I’m not familiar with at all.
@Raphael I understand that making isp work properly might take a lot of time, but my goal for now is at least get raw image, like the green one that you referenced to.
BTW, radxa, “newbies are limited by 3 posts in single topic”, wtf?
The issue here is a little more in depth than you would think.
Let*s start with the imx415, this camera has already been tuned by rockchip and is officially supported.
Without this tuning you’ll see the green image which I posted a few comments back.
Veye is another thing, they’re using a isp directly on their camera-board, and also have drivers for some rockchip boards, so in theory it shouldn’t be a big hassle to get them working.
Here is a link of somebody who played around some time ago with the 415, in the middle you see a green picture, that’s without the isp
1 Like
Will this adapter work on Rock 5B with with OV5647 Video Camera Module For Raspberry Pi 2/4/3B ?