50MP / 8K camera for RK3588 boards, interest to make this work on the Rock 5 boards?

#1

I’ve just launched my new hardware project: A much better camera for the common embedded systems used in hardware projects such as robotics (my regular bread & butter) and drones. Besides being my own customer I hope these will enable communities like these to build better products.

It would be great if we can make these work on the Rock 5 boards as well. I saw the work done on a conversion cable, which would be a good starting point. Would there be interest from Raxda to work on this?

About the camera:

The sensor is the Samsung S5KGN2 which you can find in (amongst others) the Xiaomi Mi 11 Ultra and upcoming Google Pixel 8 Pro phones. The module is approx the same size as the RPi HQ modules without lens. This module (dubbed ‘C50M’) has auto-focus and a 90-degree FoV F1.83 lens. The module has a 22-pin 0.5mm pitch FFC connector with the ‘standard’ layout for use with Nvidia Jetson and Raspberry Pi boards (exposing 4 CSI-2 lanes).

As there’s a minimum order size to get access to the manufacturing and support (further documentation / driver code) I’ve started a campaign to collect pre-orders:

Please let me know what you think.

2 Likes
#2

Looks interesting, although from looking at the Sony specs it is 4 lanes CSI to get max resolutions/framerate. The standard rpi camera connector only supports 2 lanes (same with Jeston Nano/AGX). I known because was I porting the v1 rpi camera to the Jetson . To get the most out of the camera it will need 4 lanes boards which would mean CM4 although from what I understand the ISP (software) may not be able to cope with the throughput at higher resolutions/framerates.

For RK3588 max lane speed is 2.5Gbps while S5KGN2 goes up to 6.5Gbps for higher resolutions/video rates. So would be interesting to see where it maxes out for resolutions/framerates. Can help with driver development.

#3

Hi! Thanks for your insight.

The standard rpi camera connector only supports 2 lanes (same with Jeston Nano/AGX).

Yes, I chose to implement a 4-lane version that can be converted to the Pi/Jetson connector with just a cable. Easy testing and more chance I get this thing into mass production. It’s also why I chose not to go with C-PHY for now, even though that would be better for performance.

You’re right that each of these platforms has limitations, but even with lower frame rates these cameras open up new possibilities (light weight surveying UAVs for example).

#4

Hi Stefan,
I’m working on UAV’s for a few years now, and helping to design Cameras together with arducam.
Let’s give it to you streight away, that camera is not usable on drones :wink:

The type of Autofocus mobile phones use is very sensitive to vibrations, and drones vibrate as hell.
Also 50MP is a great marketing phrase, but how good is the binning, how much does the sensor crop when using the industry standard 720p/1080p.
Additional the lens is quite small and most likely plastic, this will result in image degradation and less light that reaches the sensor.

Another question would be, what isp are you using, since when you want inter compatibility with different boards the only cheap way without tuning all isps is to use a dedicated one. Those are often expensive and if not … very slow which results in huge latency.

#5

Both Arducam and Raspberry Pi advertise the same VCM type auto-focus systems as their current featured cameras. Neither states / warns their cameras can’t be used in drones. Seems to conflict your claim?

Show me a 1080p cropped ‘industry standard’ camera with better image quality than any current high-end smartphone. Check the Xiaomi Mi 11 Ultra for a phone with the same sensor: https://www.dxomark.com/xiaomi-mi-11-ultra-camera-review-large-sensor-power/

This module doesn’t use a separate ISP for the same reasons, it would be a shame to use one. Yes, that makes driver development more expensive, but the results are far better too. It’s great not getting stuck at 1080p30. That said, Samsung has more hardware image processing features built-in the sensor than for example Sony and can output 50MP bayer, 50MP QBR or even 100MP QBR (as the sensor has 2 green sub-pixels for each node it can generate 100MP raw data if needed), along with the usual set of binning features.

Small additional note: If not having auto-focus is that important for you, there will be versions with a CS-mount (likely) later on, as soon as these ship out well and I can offer a decent value set of lenses that will actually work okay for this resolution (not that easy with CS mount).

#6

Marketing :wink:
They don’t say that they can be used, but they have a whole page about usable cameras for drones :slight_smile:
https://www.arducam.com/openhd/
No camera of those supports autofocus, somewhere they also say that autofocus is not good for vibrations, but can’t find the exact phrase now.
I don’t say that the image quality is bad on your module, but drone pilots hate cropping, the fov is highly impaled by cropping instead of binning.
On the phone nobody cares about latency or if the sensor is highly cropped. They don’t even use lower resolution modes on such modules, since a phone can record 4k or even more …
A radio link can’t transmit that, it needs to be lower resolution to be realtime decoded/encoded and even send over a less ideal link.

@Driver, I actually like this step, it’s the cleaner way of creating a Camera-Module. But then you need to keep in mind that the Raspberry can only do 1080/30|720p/60 and this will most likely be highly cropped.

Also camera modules for drones are currently very small, so no CS-mount please … we’re using M12, and the new Arducam modules are far bigger than everything that is currently used in commercial links like HD-Zero and DJI. Those cameras are 18*18mm, but it is still very small compared to a module with cs mount.

It may be a great module for research, but with the info I have it’ll be not usable on drones

1 Like
#7

Pretty interesting and promising project. That’s exactly what I always wish to see to be available on the RK3588, as one of the biggest reason I joined this community and got my Rock 5 is the promising spec of RK3588’s VPU,ISP, and the high speed 4-lane MIPI CSI that creates the possibility of building a high quality camera drone. Pretty sad to see that this campaign is unfortunately closed.

But honestly, not even the existing ArduCam modules have a driver working for the Rock 5B, I bought the 16MP AF Camera from ArduCam last year hoping to do some test (as that was the best camera module for the Pi at that moment when the Pi HQ Camera is too big for drones) and was hoping to get that camera module working on the Rock 5B.

Also, thanks to this post, we got to learnt why we don’t see drones on the market support Autofocus.

Pretty interested in getting one of these module if one day it becomes possible to get this into market.