Oh shit I totally forgot about that. I just assumed it would 3.3V, guess I was wrong.
On the flip side, the Rock5B camera connector has 3.3V and 5V power pins, so it would be possible to put regulators and level shifters on the adapter board.
The IMX477 doesn’t use 3.3V anywhere, so the HQ camera probably has level shifters on it. That would mean that we would be level shifting twice.
I’m quite curious why no RADXA member posted here, but you do realise that the camera won’t output anything readable, do you ?
To get a camera supported, the easiest step is doing the adapter.
After that you need a kernel overlay, a kernel driver and tuning files.
All of that is really expensive to create, thats the main reason why no rockship/allwinner/… does support multiple cameras.
The simple way to get cameras working is to choose one with an integrated ISP, but those are quite rare (like the veye cameras), and then port the drivers and overlays, because the cameras don’t need to be tuned to work with the radxa isp.
Without the tuning files you’ll get either garbage or nothing.
@Arducam cameras… the IMX477 may have a kernel driver in newer kernels, but no overlay and also no tuning files
@ ArduCam 16MP AutoFocus … also called imx519 will most likely not have any kernel drivers
Creating those Overlays and tuning files usually needs a sensor-datasheet and somebody with good knowledge in that area. The datasheets are under NDA and the vendors who do that kind of tuning start at about 10Grand.
Yes I am aware that it’s a very involved process, and I don’t expect to get a pretty picture without professional tuning. I tried to reach out both to Radxa and Arducam but I didn’t get any responses.
The Kernel driver is probably the easiest software part (relatively speaking). The Rockchip kernel already has an IMX577 driver in it, which is very similar to the IMX477, and by peeking at the RPi IMX477 driver, I’m sure something can be written.
As for the DT overlay, I’m not too familiar with that. Tuning the ISP will be a pain. Are tuning files specific to ISPs? If so, how specific? Maybe something can be lifted from the RPi.
Most bigger companies will just ignore requests or even emails from private persons, or even smaller companies.
I know how that feels, I’m working on an open source project and in the beginning nobody listened to us.
Right now we’re official partners with Arducam and Veye-Imaging.
In general you normally need to hire people to do the work you’re talking about. The Overlays are device specific, can’t be easily ported, it may be doable with a lot experience but not without.
The tuning files are ISP specific, sometimes even ISP and device specific. Some manufacturers also wrap those tuning files into blobs, so that they’re not readable, because generating them is really expensive.
To be honest I would love to see more cameras on that platform, but the way you’re trying to achieve it is wrong. Most companies will just laugh about the work you’ve done.
To get more cameras supported needs knocking on doors and trying to get attention of big camera-producers, just trying it without datasheets is nearly impossible and will result in frustrations.
For the time being I would just go with the radxa 4k (imx415) since it is a starvis sensor it ain’t half bad. Even better than the imx477
What I can do is ask Linda from Arducam if they looked at the newer rockship boards, but afaik they’re more focussed on other “vendors”
Yes, please ask Linda. I think they should be interested in the RK3588. That would allow recording 4K video or high refresh rate video. I’m honestly surprised they haven’t done anything with it yet.
Is it really such a tremendous effort on the software side to get a CSI-2 camera working to a basic degree? For example, for plenty of applications niceties like AWB, auto exposure, vignetting correction and distortion correction aren’t required. I’ve got a monochrome OV9281 where I would be satisfied with simply being able to adjust resolution, frame rate, and exposure and stream in raw pixel values via V4L2.
A fairly thorough roadmap for putting the overlay together is out there at https://wiki.t-firefly.com/en/ROC-RK3588S-PC/usage_camera.html
For the kernel driver, the goal is to create an adapter that allows the existing ecosystem of RPi camera modules to be used. With an overlay that identifies the correct I2C and the regulator enable GPIO, is there any reason the existing V4L2 kernel drivers won’t work?
If the camera is used via V4L2, does a tuning file even come into play? I thought that was just used by libcamera, but I admit not knowing about the various image processing pipelines.
If libcamera is of interest, although there are different ISPs in RPi and RK, it appears rkisp1 support (including tuning files for a few cameras) has been added to the repo in recent months.
Just saying it seems like there is plenty of low hanging fruit to get some of the existing RPi MIPI camera modules up and running on RK8588/RK8588S. I’ve got some PCBs in hand for OPi5 and just waiting on a few bits of hardware, so soon I will learn firsthand just how tough it is to get the software stack going.
Now for those looking to offer camera modules commercially: yeah, more polish along the lines of what @Raphael laid out is likely going to be expected from your customers.
Like I said, you’ll be able to connect and power it, but nothing more. If you’re lucky it’ll get detected.
Since most Raspberry-Pi cameras there is no ISP integrated, which means you need to tune it to work on any ISP, regardless on what driver you use.
That’s also the reason why in the FPV hobby all vendors put the ISP on the Camera itself, so it doesn’t need said tuning.
With rockship (at least the boards I know of) there is a tuning untillity, but this is kept behind a commercial NDA.
@Libcamera … there is no support for the Rk3588, yet and it doesn’t use the fairly old rkisp1.
@Overlay, this looks kinda nice, but remember that is a different CPU even if it is quite similiar.
Here is a little sample how pictures look without isp tuning :
and a diagram how an isp works
I think what @eshelton is saying is that he doesn’t require good ISP tuning. He only needs ballpark processing. As for the picture you posted, which one did you say is without ISP tuning? The green picture on the left is the unprocessed image AFAIK, so is the one on the right processed but without proper tuning? Because in that case it would be acceptable.
without tuning it is the green one, since the isp doesn’t have any settings to improve the picture, it’ll just convert it to be readable, no dead pixel correction, no lens shading, no color corection, nothing the right one is with proper tuning
Well of course if you provide no settings at all then it’s gonna output a green image. I know you need professional tools to get a high quality image, but are you saying you can’t eyeball a half decent tuning to get a usable image? 1 minute in GIMP using the channel mixer and brightness curve I can get it to look like this.
that are a few parameters, but tuning involves how it looks with different exposure times, isos and genera lightning situations. A tuning involves a lot of labor.
But even if you could do something by hand … how do you do it without the registers you need to set for the settings and the tuning untillity, which is all behind a NDA.
Crap, I was under the impression that I had the datasheet for the IMX477, but only now did I notice that the page numbering conveniently skips from 24 to 40… But the RPi driver for the IMX477 contains registers I think.
For my purposes, I’m fine with just getting 8 or 10 bit raw sensor data. No concerns about Bayer color filters or AWB, for example. If I can skip the ISP and any associated latency, all the better. The VICAP module appears to be capable of DMAing frames without a trip through the ISP, based on the diagram in the TRM. However, maybe disabling all of the ISP processing stages will be just about as good. It’s disappointing that the 5,000 page TRM lacks a chapter for the ISP — I assume that’s intentional.
If I2C is doing it’s thing, I’m wondering why the doubt about the camera being detected. Seems like I should also be able to get raw pixel values streaming via CSI. The (valid) concern raised seems to be about getting the necessary calibration data for the ISP to do it’s thing and get nice looking color images.
In the BSP kernel, it looks like the RK8588S ISP is designated isp30 in drivers/media/platform/rockchip/isp and you can set up the ISP through a V4L2 parameter subdevice. Not all that different from what is being done in libcamera for rkisp1.
@Zipdox This document gives you an idea of the process involved in fine tuning for RPi ISP. I think the process is much the same for Rockchip. Maybe it’s possible to convert a substantial part of the RPi JSON files over? The ISPs would be doing pretty much the same tasks, but likely have different requirements for number and format for parameters.
Hi,
I’m also working on similar project that adapts the PI HQ camera for another RK3588S board (cool pi 4b).
The board shares the same csi-2 interface with the raspberry pi 4b.
So, my work mostly focusing on the driver of the camera and ISP tuning.
I’m able to get the driver side working and sampling some dark-green NV12 images.
However, tuning the RK3588s ISP is not a single-man or DIY job. (For its ancestor chips, a OEM tuning binary is required but I could not find the one for RK3588s.)
Therefore, I’m thinking about getting a software ISP.
@eshelton would you mind providing any advises on how to turn off the hardware ISP and how to get the raw sensor data?
I really appreciate that!
BTW, the driver provided from the RPI would also need some modifications to fit into the rock-chip BSP.
You got the kernel driver and device overlay working? I’m fairly sure there’s a /dev/video device that bypasses the ISP.
Yes, I got the kernel driver and device tree working.
Cool pi 4b + Ubuntu don’t use the overlay, therefore I modified the device tree and got it working.
Yes, there supposes to be a node (/dev/video0 or /dev/video1) that bypasses the ISP.
But the image samples are not correct, something like the attached image.
Yeah that looks like an incorrect sample format. Could you share some details about your software setup? Also it is worthy of note that the cool PI 4B only has 2 CSI lanes since it uses the same connector. That means you can’t use the full potential of the RK3588 and camera.
Yes, I used the v4l2-ctl command to sample a 1920x1080 stream ()
sudo v4l2-ctl --verbose -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat='NV12' --stream-mmap=4 --set-selection=target=crop,flags=0,top=0,left=0,width=1920,height=1080 --stream-to=out1.yuv
And then used the ffplay to visualize it.
ffplay out1.yuv -pixel_format nv12 -video_size 1920x1080
(There is some buffer bug inside the rockchip mipi driver that I couldn’t stream 4K resolution in RAW format, but I could stream the 4K resolution after ISP.)
Cannot complain, but have to say the rockchip documents are not well organized.
Yes, it only comes with the 2 data lanes.
I planned to use the HQ camera for star rail photography of which the bottleneck is the post-processing (Raspberry PI 4B is so so slow on that).
But I came across the same problem as @Raphael discussed. (All my inquires to Rockchip were ignored, even though my startup has made a fair amount of purchases from them)
Therefore, I decided to work on the software ISP.
exactly what I thought would happen 
I had a meeting with Arducam yesterday, they’re considering supporting the Rock5 and are working on providing cameras for this platform.
It may take some time, though, but the interest is there and they already started on that.
I also told them about the very bad tuned radxa 4k camera, maybe they’re able to make it a little better.
When I know more I’ll post a few infos here and I’ll ask Arducam to join this forum 