[GUIDE] How to get the Rock Pi N10 up and running with NPU inference

I recently bought the Rock Pi N10, with the primary intention of exploring the NPU capabilities, with hopes to boost the performance of my image analytics projects.

Getting to the point where I was able to use the RKNN SDK and try things out, was however more complicated than expected and more difficult than it had to be. The information on the wiki and this forum was not always consistent and a number of things threw me off in the wrong direction.

To help others out, and my self if I need to reinstall, I’ve written down the steps I took. There might be better ways of doing this, but here are the steps I took along with some key explanations.

Disclaimer: Your mileage may vary in getting this to work and I don’t claim to understand how everything works and I take no responsibility of anything, etc etc.

Where to find things
Going through the forum and wiki, there are a number of different links to resources needed/useful for the installation and not all links are obvious to find.

http://repo.rock-chips.com/pypi/simple/
Pre-built python wheels for some of the dependencies and the RKNN toolkit

https://github.com/lhelontra/tensorflow-on-arm/releases
Pre-built python wheels for tensorflow

https://rockchips-my.sharepoint.com/:f:/g/personal/addy_ke_rockchips_onmicrosoft_com/ErFTw5A6uYtBvtg-1ksWIQEByi5eoqA4hIWHh7xPD8SRpg?e=q3kbhe
More pre-built wheels for the RKNN toolkit and some dependencies, this one is linked from the wiki, but after finding the first link above to the RK repo, that one is a better choice as it’s more complete.

https://github.com/rockchip-linux/rknn-toolkit
https://github.com/rockchip-linux/rknn-toolkit/releases
Another location for the RKNN toolkit and some of the dependencies

https://wiki.radxa.com/RockpiN10/downloads
Official Rock Pi N10 images

First mistakes
The first thing that threw me off, was that following the wiki and the different links to python wheels etc, is a mix of different python versions with not all dependencies possible to fulfil with all versions of python and OS version. Some of the links above covers some versions, other links cover others etc. That led me on a wild goose chase for some time, great learning experience, but mostly a waste of time and source of frustration. :crazy_face:

Start off right
With that out of the way, what we want to do is to use the latest version of the RKNN toolkit, which at time of writing is 1.4.0. It’s available in the following flavours:

rknn-1.4.0-cp35-cp35m-linux_aarch64.whl
rknn-1.4.0-cp37-cp37m-linux_aarch64.whl

That tells us, that we can either choose python 3.5 or 3.7. (And the wheels to look for have to be aarch64, as that is what the RK3399 CPU runs.) It also, implies that dependencies might be built towards an OS version that has 3.5 or 3.7 python by default.

Ignoring Fedora, which seems to not be updated, we have the following official images for the Rock Pi N10 to choose from,

Ubuntu Bionic Server (2020-07-10)
Debian 10 Buster Desktop (2020-07-10)

You might be tempted to pick the Ubuntu image if you have no intention of using the desktop, but that would be a mistake. Ubuntu Bionic contains python 3.6 and is older than Debian 10 Buster. Python 3.6 will not match any of the available 1.4.0 toolkits and even if you upgrade python to 3.7, the older OS means that you will have problems to resolve some of the dependencies later in the installation. The right choice for using 1.4.0 toolkit is then Debian 10 Buster, which comes with python 3.7 by default and seems to be the same OS version as the toolkit is built on.

(Please remember to pick the right build of the image, based on which Rock Pi model A/B&C you have.)

Burn the Debian image to an SD card and boot it up (https://wiki.radxa.com/RockpiN10/install/microSD).

Prepare the OS
Once the OS is booted up, access a terminal and run the following commands to prepare the OS.

#Add Radxa repository and update image
export DISTRO=buster-stable
echo “deb http://apt.radxa.com/$DISTRO/ ${DISTRO%-*} main” | sudo tee -a /etc/apt/sources.list.d/apt-radxa-com.list
sudo apt-get update
sudo apt-get install wget
wget -O - apt.radxa.com/$DISTRO/public.key | sudo apt-key add -
sudo apt install -y rockchip-overlay
sudo apt-get install -y linux-4.4-rockpin10-latest

#NPU
sudo apt-get install -y rockchip-npu

#Initialize and start the NPU – You have to do this after each reboot
sudo /etc/init.d/npuservice start

#Check if NPU is running or not
lsusb
#Bus 008 Device 003: ID 2207:18?? – Before init/start
#Bus 008 Device 003: ID 2207:1808 – After

#Make sure essential build tools are installed
sudo apt-get install build-essential checkinstall
sudo apt-get install cmake
sudo apt-get install gcc gfortran python-dev python3-dev libopenblas-dev liblapack-dev cython cython3
sudo apt-get install protobuf-compiler libprotoc-dev
sudo apt-get install pybind11-dev
sudo apt-get install autoconf automake libtool curl make g++ unzip git

One optional step before moving on, is to move /tmp to RAM, currently it’s mapped to the SD card we booted from. Moving it to RAM will reduce wear and tear of the SD card and provide some performance improvement, at least if you will be using /tmp a lot while processing data.

#Add line to /etc/fstab
tmpfs /tmp tmpfs defaults,nosuid 0 0

At this point the OS is in a good state and we can focus on getting the RKNN dependencies in place.

Start by installing the following packages, as described in the wiki.

sudo apt-get install python3-numpy python3-scipy
sudo apt-get install python3-opencv
sudo apt-get install python3-h5py
sudo apt-get install python3-grpcio
sudo apt-get install python3-pip

Even if installing these, some will not be of the right version and will be re-built as part of installing other dependencies.

You will need scipy >= 1.1.0, either see if you can find it (cp37 & aarch64), if you do, try to install it using the wheel file. Otherwise build it. (With the various build tools we installed earlier, this should work without any issues, but take some time.)

#Build scipy
python3 -m pip install Pillow==5.3.0
pip3 install --upgrade scipy

Next we will need h5py == 2.8.0, either see if you find it (cp37 & aarch64) or build it as below.

#Build h5py
sudo apt-get install libhdf5-dev
export HDF5_DIR=/usr/lib/aarch64-linux-gnu/hdf5/serial/
python3 -m pip install h5py==2.8.0

If you run into build problems with above, related to Cython and “await”, then you need to upgrade Cython using below command.

python3 -m pip install --upgrade Cython

Besides scipy and h5py, the other big packages needed are onnx == 1.4.1 and grpcio (version?), if you can find the right wheels, that will save you some time, otherwise they will be built when the remaining packages are installed.

Next we need to install Tensorflow, download it from https://github.com/lhelontra/tensorflow-on-arm/releases/download/v1.14.0-buster/tensorflow-1.14.0-cp37-none-linux_aarch64.whl or from rock chip repo.

pip3 install tensorflow-1.14.0-cp37-none-linux_aarch64.whl

Once tensor flow is installed, continue with the following for preparing the RKNN install.

pip3 install gluoncv
sudo apt-get install libjpeg-dev
python3 -m pip install --upgrade Pillow

We can now finally install RKNN toolkit.

pip3 install rknn_toolkit-1.4.0-cp37-cp37m-linux_aarch64.whl

Once installed, proceed with running the sample script from the examples folder in RKNN toolkit, as described in section “6.3 Running the sample attached in the installation package” of the quick start guide.

Starting off right, avoiding the first mistakes and installing all the needed build tools from the beginning, the installation process is not that painful.

I hope this is useful for someone, it would have saved me a lot of time. :smile: :sweat:
If there are errors or better ways of doing it, please reply with corrections and other tips.

4 Likes

Hello. First of all thank you for the guide.
Unfortunately I am having problems with executing the tflite example on the rk3399pro.
I always get
–> Init runtime environment
E Only support ntb mode on ARM64 platform. But can not find device with ntb mode.
E Catch exception when init runtime!
E Traceback (most recent call last):
E File “rknn/api/rknn_base.py”, line 815, in rknn.api.rknn_base.RKNNBase.init_runtime
E File “rknn/api/rknn_runtime.py”, line 170, in rknn.api.rknn_runtime.RKNNRuntime.init
E File “rknn/api/rknn_platform_utils.py”, line 307, in rknn.api.rknn_platform_utils.start_ntp_or_adb
E Exception: Init runtime environment failed!
E Current device id is: 0123456789ABCDEF
E Devices connected:
E [‘0123456789ABCDEF’]
Init runtime environment failed

Maybe you have encountered this before.