Use YoloV8 in RK3588 NPU

avaf · April 7, 2023, 9:45pm

ock@rock5b:~/rockchip/RKNN$ git clone https://github.com/ultralytics/ultralytics --depth=1
Cloning into 'ultralytics'...
remote: Enumerating objects: 248, done.
remote: Counting objects: 100% (248/248), done.
remote: Compressing objects: 100% (223/223), done.
remote: Total 248 (delta 36), reused 102 (delta 20), pack-reused 0
Receiving objects: 100% (248/248), 620.63 KiB | 4.89 MiB/s, done.
Resolving deltas: 100% (36/36), done.
rock@rock5b:~/rockchip/RKNN$ cd ultralytics/
rock@rock5b:~/rockchip/RKNN/ultralytics$ pip install -r requirements.txt
Defaulting to user installation because normal site-packages is not writeable
Collecting matplotlib>=3.2.2
  Downloading matplotlib-3.7.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.3/11.3 MB 2.9 MB/s eta 0:00:00
Collecting numpy>=1.21.6
  Downloading numpy-1.24.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 26.3 MB/s eta 0:00:00
Collecting opencv-python>=4.6.0
  Downloading opencv_python-4.7.0.72-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (40.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.4/40.4 MB 21.0 MB/s eta 0:00:00
Requirement already satisfied: Pillow>=7.1.2 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 8)) (9.0.1)
Requirement already satisfied: PyYAML>=5.3.1 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 9)) (5.4.1)
Requirement already satisfied: requests>=2.23.0 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 10)) (2.25.1)
Collecting scipy>=1.4.1
  Downloading scipy-1.10.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (30.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.9/30.9 MB 2.7 MB/s eta 0:00:00
Collecting torch>=1.7.0
  Downloading torch-2.0.0-1-cp310-cp310-manylinux2014_aarch64.whl (74.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 74.3/74.3 MB 13.2 MB/s eta 0:00:00
Collecting torchvision>=0.8.1
  Downloading torchvision-0.15.1-cp310-cp310-manylinux2014_aarch64.whl (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 22.3 MB/s eta 0:00:00
Collecting tqdm>=4.64.0
  Downloading tqdm-4.65.0-py3-none-any.whl (77 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.1/77.1 KB 12.1 MB/s eta 0:00:00
Collecting pandas>=1.1.4
  Downloading pandas-2.0.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.6/11.6 MB 28.4 MB/s eta 0:00:00
Collecting seaborn>=0.11.0
  Downloading seaborn-0.12.2-py3-none-any.whl (293 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 293.3/293.3 KB 31.6 MB/s eta 0:00:00
Collecting psutil
  Downloading psutil-5.9.4.tar.gz (485 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 485.8/485.8 KB 32.7 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting thop>=0.1.1
  Downloading thop-0.1.1.post2209072238-py3-none-any.whl (15 kB)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/lib/python3/dist-packages (from matplotlib>=3.2.2->-r requirements.txt (line 5)) (2.4.7)
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.4.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 31.6 MB/s eta 0:00:00
Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3/dist-packages (from matplotlib>=3.2.2->-r requirements.txt (line 5)) (2.8.1)
Collecting cycler>=0.10
  Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting fonttools>=4.22.0
  Downloading fonttools-4.39.3-py3-none-any.whl (1.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 32.2 MB/s eta 0:00:00
Collecting packaging>=20.0
  Downloading packaging-23.0-py3-none-any.whl (42 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.7/42.7 KB 7.1 MB/s eta 0:00:00
Collecting contourpy>=1.0.1
  Downloading contourpy-1.0.7-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (283 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 283.7/283.7 KB 33.3 MB/s eta 0:00:00
Collecting filelock
  Downloading filelock-3.11.0-py3-none-any.whl (10.0 kB)
Collecting networkx
  Downloading networkx-3.1-py3-none-any.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 31.6 MB/s eta 0:00:00
Collecting typing-extensions
  Downloading typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Collecting jinja2
  Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 KB 18.5 MB/s eta 0:00:00
Collecting sympy
  Downloading sympy-1.11.1-py3-none-any.whl (6.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 24.1 MB/s eta 0:00:00
Collecting python-dateutil>=2.7
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 KB 28.9 MB/s eta 0:00:00
Collecting tzdata>=2022.1
  Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 341.8/341.8 KB 34.7 MB/s eta 0:00:00
Collecting pytz>=2020.1
  Downloading pytz-2023.3-py2.py3-none-any.whl (502 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 502.3/502.3 KB 35.3 MB/s eta 0:00:00
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.7->matplotlib>=3.2.2->-r requirements.txt (line 5)) (1.16.0)
Collecting MarkupSafe>=2.0
  Downloading MarkupSafe-2.1.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (26 kB)
Collecting mpmath>=0.19
  Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 KB 34.8 MB/s eta 0:00:00
Building wheels for collected packages: psutil
  Building wheel for psutil (pyproject.toml) ... done
  Created wheel for psutil: filename=psutil-5.9.4-cp310-abi3-linux_aarch64.whl size=278808 sha256=4214dc365f2af49265df4fd669c41695d14c31c529fd3b696082b9242e726d5a
  Stored in directory: /home/rock/.cache/pip/wheels/8c/f0/2c/0d7d6e64319efa308609a6844a955ef94aa797388843265de0
Successfully built psutil
Installing collected packages: pytz, mpmath, tzdata, typing-extensions, tqdm, sympy, python-dateutil, psutil, packaging, numpy, networkx, MarkupSafe, kiwisolver, fonttools, filelock, cycler, scipy, pandas, opencv-python, jinja2, contourpy, torch, matplotlib, torchvision, thop, seaborn
Successfully installed MarkupSafe-2.1.2 contourpy-1.0.7 cycler-0.11.0 filelock-3.11.0 fonttools-4.39.3 jinja2-3.1.2 kiwisolver-1.4.4 matplotlib-3.7.1 mpmath-1.3.0 networkx-3.1 numpy-1.24.2 opencv-python-4.7.0.72 packaging-23.0 pandas-2.0.0 psutil-5.9.4 python-dateutil-2.8.2 pytz-2023.3 scipy-1.10.1 seaborn-0.12.2 sympy-1.11.1 thop-0.1.1.post2209072238 torch-2.0.0 torchvision-0.15.1 tqdm-4.65.0 typing-extensions-4.5.0 tzdata-2023.3
rock@rock5b:~/rockchip/RKNN/ultralytics$ pip install -e '.[dev]'
Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///home/rock/rockchip/RKNN/ultralytics
  Preparing metadata (setup.py) ... done
Requirement already satisfied: Pillow>=7.1.2 in /usr/lib/python3/dist-packages (from ultralytics==8.0.69) (9.0.1)
Requirement already satisfied: PyYAML>=5.3.1 in /usr/lib/python3/dist-packages (from ultralytics==8.0.69) (5.4.1)
Requirement already satisfied: matplotlib>=3.2.2 in /home/rock/.local/lib/python3.10/site-packages (from ultralytics==8.0.69) (3.7.1)
Requirement already satisfied: numpy>=1.21.6 in /home/rock/.local/lib/python3.10/site-packages (from ultralytics==8.0.69) (1.24.2)
Requirement already satisfied: opencv-python>=4.6.0 in /home/rock/.local/lib/python3.10/site-packages (from ultralytics==8.0.69) (4.7.0.72)
Requirement already satisfied: pandas>=1.1.4 in /home/rock/.local/lib/python3.10/site-packages (from ultralytics==8.0.69) (2.0.0)
Requirement already satisfied: psutil in /home/rock/.local/lib/python3.10/site-packages (from ultralytics==8.0.69) (5.9.4)
Requirement already satisfied: requests>=2.23.0 in /usr/lib/python3/dist-packages (from ultralytics==8.0.69) (2.25.1)
Requirement already satisfied: scipy>=1.4.1 in /home/rock/.local/lib/python3.10/site-packages (from ultralytics==8.0.69) (1.10.1)
Requirement already satisfied: seaborn>=0.11.0 in /home/rock/.local/lib/python3.10/site-packages (from ultralytics==8.0.69) (0.12.2)
Collecting sentry_sdk
  Downloading sentry_sdk-1.19.1-py2.py3-none-any.whl (199 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.2/199.2 KB 3.9 MB/s eta 0:00:00
Requirement already satisfied: thop>=0.1.1 in /home/rock/.local/lib/python3.10/site-packages (from ultralytics==8.0.69) (0.1.1.post2209072238)
Requirement already satisfied: torch>=1.7.0 in /home/rock/.local/lib/python3.10/site-packages (from ultralytics==8.0.69) (2.0.0)
Requirement already satisfied: torchvision>=0.8.1 in /home/rock/.local/lib/python3.10/site-packages (from ultralytics==8.0.69) (0.15.1)
Requirement already satisfied: tqdm>=4.64.0 in /home/rock/.local/lib/python3.10/site-packages (from ultralytics==8.0.69) (4.65.0)
Collecting check-manifest
  Downloading check_manifest-0.49-py3-none-any.whl (20 kB)
Collecting coverage
  Downloading coverage-7.2.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (228 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 228.8/228.8 KB 24.1 MB/s eta 0:00:00
Collecting mkdocs-material
  Downloading mkdocs_material-9.1.6-py3-none-any.whl (7.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 29.7 MB/s eta 0:00:00
Collecting mkdocstrings[python]
  Downloading mkdocstrings-0.21.2-py3-none-any.whl (26 kB)
Collecting pytest
  Downloading pytest-7.2.2-py3-none-any.whl (317 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.2/317.2 KB 36.4 MB/s eta 0:00:00
Collecting pytest-cov
  Downloading pytest_cov-4.0.0-py3-none-any.whl (21 kB)
Requirement already satisfied: packaging>=20.0 in /home/rock/.local/lib/python3.10/site-packages (from matplotlib>=3.2.2->ultralytics==8.0.69) (23.0)
Requirement already satisfied: contourpy>=1.0.1 in /home/rock/.local/lib/python3.10/site-packages (from matplotlib>=3.2.2->ultralytics==8.0.69) (1.0.7)
Requirement already satisfied: python-dateutil>=2.7 in /home/rock/.local/lib/python3.10/site-packages (from matplotlib>=3.2.2->ultralytics==8.0.69) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/rock/.local/lib/python3.10/site-packages (from matplotlib>=3.2.2->ultralytics==8.0.69) (1.4.4)
Requirement already satisfied: cycler>=0.10 in /home/rock/.local/lib/python3.10/site-packages (from matplotlib>=3.2.2->ultralytics==8.0.69) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /home/rock/.local/lib/python3.10/site-packages (from matplotlib>=3.2.2->ultralytics==8.0.69) (4.39.3)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/lib/python3/dist-packages (from matplotlib>=3.2.2->ultralytics==8.0.69) (2.4.7)
Requirement already satisfied: pytz>=2020.1 in /home/rock/.local/lib/python3.10/site-packages (from pandas>=1.1.4->ultralytics==8.0.69) (2023.3)
Requirement already satisfied: tzdata>=2022.1 in /home/rock/.local/lib/python3.10/site-packages (from pandas>=1.1.4->ultralytics==8.0.69) (2023.3)
Requirement already satisfied: sympy in /home/rock/.local/lib/python3.10/site-packages (from torch>=1.7.0->ultralytics==8.0.69) (1.11.1)
Requirement already satisfied: jinja2 in /home/rock/.local/lib/python3.10/site-packages (from torch>=1.7.0->ultralytics==8.0.69) (3.1.2)
Requirement already satisfied: filelock in /home/rock/.local/lib/python3.10/site-packages (from torch>=1.7.0->ultralytics==8.0.69) (3.11.0)
Requirement already satisfied: typing-extensions in /home/rock/.local/lib/python3.10/site-packages (from torch>=1.7.0->ultralytics==8.0.69) (4.5.0)
Requirement already satisfied: networkx in /home/rock/.local/lib/python3.10/site-packages (from torch>=1.7.0->ultralytics==8.0.69) (3.1)
Collecting tomli
  Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting build>=0.1
  Downloading build-0.10.0-py3-none-any.whl (17 kB)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from check-manifest->ultralytics==8.0.69) (59.6.0)
Collecting mkdocs>=1.4.2
  Downloading mkdocs-1.4.2-py3-none-any.whl (3.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.7/3.7 MB 26.5 MB/s eta 0:00:00
Collecting mkdocs-material-extensions>=1.1
  Downloading mkdocs_material_extensions-1.1.1-py3-none-any.whl (7.9 kB)
Collecting pygments>=2.14
  Downloading Pygments-2.14.0-py3-none-any.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 34.6 MB/s eta 0:00:00
Collecting pymdown-extensions>=9.9.1
  Downloading pymdown_extensions-9.11-py3-none-any.whl (239 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 239.9/239.9 KB 24.4 MB/s eta 0:00:00
Requirement already satisfied: colorama>=0.4 in /usr/lib/python3/dist-packages (from mkdocs-material->ultralytics==8.0.69) (0.4.4)
Collecting requests>=2.23.0
  Downloading requests-2.28.2-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 KB 10.4 MB/s eta 0:00:00
Collecting regex>=2022.4.24
  Downloading regex-2023.3.23-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (768 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 769.0/769.0 KB 31.7 MB/s eta 0:00:00
Collecting markdown>=3.2
  Downloading Markdown-3.4.3-py3-none-any.whl (93 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 93.9/93.9 KB 13.7 MB/s eta 0:00:00
Collecting charset-normalizer<4,>=2
  Downloading charset_normalizer-3.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (195 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 195.9/195.9 KB 28.5 MB/s eta 0:00:00
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests>=2.23.0->ultralytics==8.0.69) (2020.6.20)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3/dist-packages (from requests>=2.23.0->ultralytics==8.0.69) (1.26.5)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests>=2.23.0->ultralytics==8.0.69) (3.3)
Requirement already satisfied: MarkupSafe>=1.1 in /home/rock/.local/lib/python3.10/site-packages (from mkdocstrings[python]->ultralytics==8.0.69) (2.1.2)
Collecting mkdocs-autorefs>=0.3.1
  Downloading mkdocs_autorefs-0.4.1-py3-none-any.whl (9.8 kB)
Collecting mkdocstrings-python>=0.5.2
  Downloading mkdocstrings_python-0.9.0-py3-none-any.whl (38 kB)
Collecting attrs>=19.2.0
  Downloading attrs-22.2.0-py3-none-any.whl (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.0/60.0 KB 8.1 MB/s eta 0:00:00
Collecting exceptiongroup>=1.0.0rc8
  Downloading exceptiongroup-1.1.1-py3-none-any.whl (14 kB)
Collecting pluggy<2.0,>=0.12
  Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting iniconfig
  Downloading iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.15-py2.py3-none-any.whl (140 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.9/140.9 KB 20.2 MB/s eta 0:00:00
Collecting pyproject_hooks
  Downloading pyproject_hooks-1.0.0-py3-none-any.whl (9.3 kB)
Collecting markdown>=3.2
  Downloading Markdown-3.3.7-py3-none-any.whl (97 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.8/97.8 KB 8.3 MB/s eta 0:00:00
Collecting mergedeep>=1.3.4
  Downloading mergedeep-1.3.4-py3-none-any.whl (6.4 kB)
Collecting watchdog>=2.0
  Downloading watchdog-3.0.0-py3-none-manylinux2014_aarch64.whl (82 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 82.0/82.0 KB 6.8 MB/s eta 0:00:00
Requirement already satisfied: click>=7.0 in /usr/lib/python3/dist-packages (from mkdocs>=1.4.2->mkdocs-material->ultralytics==8.0.69) (8.0.3)
Collecting ghp-import>=1.0
  Downloading ghp_import-2.1.0-py3-none-any.whl (11 kB)
Collecting pyyaml-env-tag>=0.1
  Downloading pyyaml_env_tag-0.1-py3-none-any.whl (3.9 kB)
Collecting griffe>=0.24
  Downloading griffe-0.26.0-py3-none-any.whl (88 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.3/88.3 KB 12.2 MB/s eta 0:00:00
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.7->matplotlib>=3.2.2->ultralytics==8.0.69) (1.16.0)
Requirement already satisfied: mpmath>=0.19 in /home/rock/.local/lib/python3.10/site-packages (from sympy->torch>=1.7.0->ultralytics==8.0.69) (1.3.0)
Installing collected packages: watchdog, urllib3, tomli, regex, pyyaml-env-tag, pygments, pluggy, mkdocs-material-extensions, mergedeep, markdown, iniconfig, griffe, exceptiongroup, coverage, charset-normalizer, attrs, sentry_sdk, requests, pytest, pyproject_hooks, pymdown-extensions, ghp-import, pytest-cov, mkdocs, build, ultralytics, mkdocs-material, mkdocs-autorefs, check-manifest, mkdocstrings, mkdocstrings-python
  Running setup.py develop for ultralytics
Successfully installed attrs-22.2.0 build-0.10.0 charset-normalizer-3.1.0 check-manifest-0.49 coverage-7.2.3 exceptiongroup-1.1.1 ghp-import-2.1.0 griffe-0.26.0 iniconfig-2.0.0 markdown-3.3.7 mergedeep-1.3.4 mkdocs-1.4.2 mkdocs-autorefs-0.4.1 mkdocs-material-9.1.6 mkdocs-material-extensions-1.1.1 mkdocstrings-0.21.2 mkdocstrings-python-0.9.0 pluggy-1.0.0 pygments-2.14.0 pymdown-extensions-9.11 pyproject_hooks-1.0.0 pytest-7.2.2 pytest-cov-4.0.0 pyyaml-env-tag-0.1 regex-2023.3.23 requests-2.28.2 sentry_sdk-1.19.1 tomli-2.0.1 ultralytics urllib3-1.26.15 watchdog-3.0.0
rock@rock5b:~/rockchip/RKNN/ultralytics$ python3 -m pip list | grep numpy
numpy                      1.24.2
rock@rock5b:~/rockchip/RKNN/ultralytics$ python3 -m pip list | grep onnx
rock@rock5b:~/rockchip/RKNN/ultralytics$ pip install onnx
Defaulting to user installation because normal site-packages is not writeable
Collecting onnx
  Downloading onnx-1.13.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (13.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.8/13.8 MB 2.2 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.16.6 in /home/rock/.local/lib/python3.10/site-packages (from onnx) (1.24.2)
Collecting protobuf<4,>=3.20.2
  Downloading protobuf-3.20.3-cp310-cp310-manylinux2014_aarch64.whl (918 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 918.4/918.4 KB 30.8 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions>=3.6.2.1 in /home/rock/.local/lib/python3.10/site-packages (from onnx) (4.5.0)
Installing collected packages: protobuf, onnx
Successfully installed onnx-1.13.1 protobuf-3.20.3
rock@rock5b:~/rockchip/RKNN/ultralytics$ pip install onnxruntime
Defaulting to user installation because normal site-packages is not writeable
Collecting onnxruntime
  Downloading onnxruntime-1.14.1-cp310-cp310-manylinux_2_27_aarch64.whl (4.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/4.7 MB 3.3 MB/s eta 0:00:00
Collecting coloredlogs
  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.0/46.0 KB 5.9 MB/s eta 0:00:00
Requirement already satisfied: sympy in /home/rock/.local/lib/python3.10/site-packages (from onnxruntime) (1.11.1)
Requirement already satisfied: numpy>=1.21.6 in /home/rock/.local/lib/python3.10/site-packages (from onnxruntime) (1.24.2)
Requirement already satisfied: protobuf in /home/rock/.local/lib/python3.10/site-packages (from onnxruntime) (3.20.3)
Collecting flatbuffers
  Downloading flatbuffers-23.3.3-py2.py3-none-any.whl (26 kB)
Requirement already satisfied: packaging in /home/rock/.local/lib/python3.10/site-packages (from onnxruntime) (23.0)
Collecting humanfriendly>=9.1
  Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.8/86.8 KB 12.9 MB/s eta 0:00:00
Requirement already satisfied: mpmath>=0.19 in /home/rock/.local/lib/python3.10/site-packages (from sympy->onnxruntime) (1.3.0)
Installing collected packages: flatbuffers, humanfriendly, coloredlogs, onnxruntime
Successfully installed coloredlogs-15.0.1 flatbuffers-23.3.3 humanfriendly-10.0 onnxruntime-1.14.1
rock@rock5b:~/rockchip/RKNN/ultralytics$ python3 -m pip list | grep torch
torch                      2.0.0
torchvision                0.15.1
rock@rock5b:~/rockchip/RKNN/ultralytics$ yolo export model=yolov8s.pt imgsz=640,640 format=onnx opset=12
Downloading https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s.pt to yolov8s.pt...
100%|██████████████████████████████████████| 21.5M/21.5M [00:00<00:00, 33.8MB/s]
Ultralytics YOLOv8.0.69 🚀 Python-3.10.6 torch-2.0.0 CPU
YOLOv8s summary (fused): 168 layers, 11156544 parameters, 0 gradients, 28.6 GFLOPs

PyTorch: starting from yolov8s.pt with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (21.5 MB)

ONNX: starting export with onnx 1.13.1 opset 12...
================ Diagnostic Run torch.onnx.export version 2.0.0 ================
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

ONNX: export success ✅ 2.1s, saved as yolov8s.onnx (42.8 MB)

Export complete (5.2s)
Results saved to /home/rock/rockchip/RKNN/ultralytics
Predict:         yolo predict task=detect model=yolov8s.onnx imgsz=640 
Validate:        yolo val task=detect model=yolov8s.onnx imgsz=640 data=coco.yaml 
Visualize:       https://netron.app
rock@rock5b:~/rockchip/RKNN/ultralytics$ find . -name *.onnx
./yolov8s.onnx
rock@rock5b:~/rockchip/RKNN/ultralytics$

Unfortunately, i can’t continue from here, if anyone wants to convert and share the final rknn in the proper platform, please do so, here is the file (mega.nz):
https://mega.nz/file/kbB3QCpC#7YB2Gnz5WsF4F-2oXBztK03SSiELrVRFmI4SadKgNPM

Thanks for sharing the instructions and thanks to everyone for interacting, please don’t take things personally, we’re here to have fun.

Let’s have a beer together one day, cheers.

nickliu973 · April 7, 2023, 10:56pm

@jack, @avaf I may consider to post the code, but this forum needs to exclude this raciest, LGBT-hate and bully @JonGroff first.

rippanda12 · April 9, 2023, 8:22pm

Y’all are funny where’s my popcorn

DarkevilPT · April 10, 2023, 7:23am

Dropbox? file.io uploadnow.io maybe?

phiber · April 10, 2023, 5:22pm

great. I’m looking forward to it.
i want to connect this to my camera rtsp stream and output an rtsp stream with boxes around objects that the model recognizes.

milas · April 16, 2023, 8:05pm

Thanks so much for sharing @nickliu973!

I’ve been working on reproducing this following your instructions and am close but still running into some trouble.

Since others are interested in the models, I’ll share what I’ve got here in the hopes that it helps someone else.

I’m doing everything from Docker:

github.com

milas/rock5-toolchain/blob/rknn/rknn/Dockerfile

# syntax=docker/dockerfile:1-labs

FROM scratch AS git-rknntoolkit2

ADD https://github.com/rockchip-linux/rknn-toolkit2.git /

# --------------------------------------------------------------------------- #

FROM scratch AS git-rknpu2

ADD https://github.com/rockchip-linux/rknpu2.git /

# --------------------------------------------------------------------------- #

FROM --platform=linux/amd64 ubuntu:20.04 AS rknn-toolkit2

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
    build-essential \
    python3-pip \

This file has been truncated. show original

If you want to export the YOLOv8 (yolov8s) model in ONNX & RKNN format:

docker buildx bake \
  --set='*.context=https://github.com/milas/rock5-toolchain.git#rknn:rknn' \
  'https://github.com/milas/rock5-toolchain.git#rknn' \
  yolov8

That will result in the following in your current directory:

./out/rknn
├── yolov8s.onnx
└── yolov8s.rknn

NOTE: This will ONLY work from an AMD64 host, as the “full” rknn-toolkit2 only has a x86_64 Python wheel. The rknn-toolkit-lite2 is for ARM64/aarch64, but I haven’t tried using it currently.

I also built the rknn_benchmark and have an image with the yolov8s model and benchmark:

docker run --rm -it --ipc=host --privileged \
  docker.io/milas/rknn-benchmark:yolov8s \
  rknn_benchmark /sdk/models/yolov8s.rknn 5 7

NOTE: This will ONLY work on a Rock 5 (RK3588) device.

This is currently where I’m at, but it fails for me:

rknn_api/rknnrt version: 1.4.0 (a10f100eb@2022-09-09T09:07:14), driver version: 0.8.2
total weight size: 11893952, total internal size: 6150080
total dma used size: 26640384
model input num: 1, output num: 1
input tensors:
index=0, name=images, n_dims=4, dims=[1, 480, 640, 3], n_elems=921600, size=921600, w_stride = 640, size_with_stride=921600, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
output tensors:
index=0, name=output0, n_dims=4, dims=[1, 84, 6300, 1], n_elems=529200, size=529200, w_stride = 0, size_with_stride=529200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=2.493242
custom string:
Warmup ...
0: Elapse Time = 69.47ms, FPS = 14.39
1: Elapse Time = 57.34ms, FPS = 17.44
E RKNN: [17:33:58.876] failed to submit!, op id: 12, op name: Add:/model.2/m.0/Add, flags: 0x5, task start: 3410, task number: 3, run task counter: 0, int status: 0
rknn run error -1

EDIT: This is what’s showing up in the kernel logs when it happens:

err: [2023-04-16T20:00:25.726217426Z]: RKNPU: failed to wait job, task counter: 0, flags: 0x5, ret = 0, elapsed time: 6135901us
err: [2023-04-16T20:00:25.832819426Z]: RKNPU: job timeout, flags: 0x0, irq status: 0x0, raw status: 0xc0000000, require mask: 0x300, task counter: 0x0, elapsed time: 6242507us
info: [2023-04-16T20:00:25.939628426Z]: RKNPU: soft reset

I saw different results on different runs, e.g. it sometimes makes it further than that but sometimes fails even earlier. I started trying different core masks and actually locked up the kernel at one point and had to power cycle

I’m still iterating on the Docker build setup; once I get things working and that improved, I’ll make a new thread with the details. If you’re feeling adventurous, take a look at the rknn/yolo targets in docker-bake.hcl on the rknn branch.

avaf · April 16, 2023, 9:39pm

nickliu973 · April 17, 2023, 12:28am

Hi, @milas.

Thanks for the message.
Sorry, it will take me a bit while to get back to this thread.
I did not try the rknn docker before.
I’m wondering if the docker is running in simulator, debug or physical model of the RK3588 NPU?

The message shows the output dimension is [84,6300] instead of the [84, 8400] which I was used.
Did you make any change to the model?

The error message implies the scheduling issue of the rknn runtime.
It indicates the add node in the model 2 of the neural network caused the problem which is weird as this data path should not cause the data sync problem.

The issue I came across in physical mode of the RK3588 NPU is the add model in the model.22 (post-processing stage of the bounding-box generation) and the issue is consistent in different runs.
And the solution is to remove the node which causes the issue and re-program it input the post-processing runtime.

Can you try to run the model in the physical mode and let me know the issue?

milas · April 17, 2023, 11:14pm

Running on a physical Rock 5B (retail version from Allnet).

I did not - using the stock model and trying to follow your instructions. I might have messed something up though, thanks for the tip - I will look at this again!

phiber · April 28, 2023, 8:13pm

hi everyone. anybody made progress past the last error @nickliu973 mentioned?

arzen · April 28, 2023, 9:31pm

@nickliu973 could you please share the v8 test.py file? thanks

ubombi · May 2, 2023, 10:14pm

What is this nice UI from a screenshot?

nickliu973 · May 3, 2023, 1:57am

Hi, please check this web: https://netron.app/

nickliu973 · May 3, 2023, 2:04am

You can post your issue and I will help you.

Dbenton · May 3, 2023, 8:56pm

I’m trying to follow the guide here and immediately i’m met with this:

I was able to install ultralytics using PIP but attempting to run anything related to Yolov8 in python is restarting the board. I’m using the ameridroid branded power adapter.

nickliu973 · May 4, 2023, 2:57am

I didn’t try directly run the yolov8 in rock5b.
You need to run the yolov8 in a PC and convert the model (some modifications) using RKNN toolkit2.
Then, run the converted model in rock5b.
You can use this command to clone the yolov8
https://github.com/ultralytics/ultralytics.git

You may want to try the yolov5 examples in the rknn-toolkit2 before working on YoloV8.

Dbenton · May 4, 2023, 1:20pm

I managed to get it just after posting, but yes still have reboot issues. I will try v5.

Dbenton · May 4, 2023, 1:30pm

Seems I have found a solution. Running the board using the Ameridroid power adapter, I had previously been using a USB-A to USB-C cable. Currently using a USB-C to USB-C and it trained all 3 epochs without a reboot.

nickliu973 · May 4, 2023, 6:26pm

Nice! If there is any problems, don’t hesitate to let me know.

arzen · May 5, 2023, 5:01am

convert yolov8 onnx to rknn, show error when run it on rk3588, error message:
failed to submit!, op id: 173, op name: Add:Add_243

convert script setting:

please help，thanks!