Thanks so much for sharing @nickliu973!
I’ve been working on reproducing this following your instructions and am close but still running into some trouble.
Since others are interested in the models, I’ll share what I’ve got here in the hopes that it helps someone else.
I’m doing everything from Docker:
If you want to export the YOLOv8 (yolov8s) model in ONNX & RKNN format:
docker buildx bake \
--set='*.context=https://github.com/milas/rock5-toolchain.git#rknn:rknn' \
'https://github.com/milas/rock5-toolchain.git#rknn' \
yolov8
That will result in the following in your current directory:
./out/rknn
├── yolov8s.onnx
└── yolov8s.rknn
NOTE: This will ONLY work from an AMD64 host, as the “full” rknn-toolkit2 only has a x86_64 Python wheel. The rknn-toolkit-lite2 is for ARM64/aarch64, but I haven’t tried using it currently.
I also built the rknn_benchmark and have an image with the yolov8s model and benchmark:
docker run --rm -it --ipc=host --privileged \
docker.io/milas/rknn-benchmark:yolov8s \
rknn_benchmark /sdk/models/yolov8s.rknn 5 7
NOTE: This will ONLY work on a Rock 5 (RK3588) device.
This is currently where I’m at, but it fails for me:
rknn_api/rknnrt version: 1.4.0 (a10f100eb@2022-09-09T09:07:14), driver version: 0.8.2
total weight size: 11893952, total internal size: 6150080
total dma used size: 26640384
model input num: 1, output num: 1
input tensors:
index=0, name=images, n_dims=4, dims=[1, 480, 640, 3], n_elems=921600, size=921600, w_stride = 640, size_with_stride=921600, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
output tensors:
index=0, name=output0, n_dims=4, dims=[1, 84, 6300, 1], n_elems=529200, size=529200, w_stride = 0, size_with_stride=529200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=2.493242
custom string:
Warmup ...
0: Elapse Time = 69.47ms, FPS = 14.39
1: Elapse Time = 57.34ms, FPS = 17.44
E RKNN: [17:33:58.876] failed to submit!, op id: 12, op name: Add:/model.2/m.0/Add, flags: 0x5, task start: 3410, task number: 3, run task counter: 0, int status: 0
rknn run error -1
EDIT: This is what’s showing up in the kernel logs when it happens:
err: [2023-04-16T20:00:25.726217426Z]: RKNPU: failed to wait job, task counter: 0, flags: 0x5, ret = 0, elapsed time: 6135901us
err: [2023-04-16T20:00:25.832819426Z]: RKNPU: job timeout, flags: 0x0, irq status: 0x0, raw status: 0xc0000000, require mask: 0x300, task counter: 0x0, elapsed time: 6242507us
info: [2023-04-16T20:00:25.939628426Z]: RKNPU: soft reset
I saw different results on different runs, e.g. it sometimes makes it further than that but sometimes fails even earlier. I started trying different core masks and actually locked up the kernel at one point and had to power cycle 
I’m still iterating on the Docker build setup; once I get things working and that improved, I’ll make a new thread with the details. If you’re feeling adventurous, take a look at the rknn/yolo targets in docker-bake.hcl on the rknn branch.
