IMX415 + NPU demo on ROCK 5B

This is a demo of Rock 5B with an IMX415 camera and NPU to detect objects in real-time.
FPS: 13 ~19
Image size: 1920x1080

rknn

Video:
https://drive.google.com/file/d/1t8MuSeTZjkzzqXmhvVqH7FbDSeDb6-BN/view?usp=sharing

Based on this:

3 Likes

Thank you for sharing!

Are you downscaling the image from 1920x1080 to 640x640 before feeding to OpenCV?

If not, what FPS do you get on a 640x640 downscaled image?

Yes.

On the upper left corner of the screenshot is the FPS.

  • grab a frame from the camera (1980x1080) - 30FPS
  • downscale to 640x640
  • run NPU
  • render it on screen (1920x1080) with sdl - (13 ~19 FPS)

Unfortunately, I left a 10ms delay on each frame while debugging the code. I think that slowed things down a bit. I will re-run the code without the delay. Maybe i can get 20 FPS or more.

Little gain. I think it depends on the number of objects detected. There were some spikes of 20 FPS. (14 ~ 20 FPS)

That’s it for now!

1 Like

Thanks for trying that out, I would have expected higher FPS as Google Coral’s USB at 4 TOPS gives FPS in the mid teens.
When my Rock Pi is delivered I will try it out and profile each step of the process in case there is some other bottle neck resulting in the lower than anticipated FPS.