Running custom frigate with full npu support Dragon Q6a

Dragon Series Q6A
1

Guide: Running Frigate NVR 0.17 on Radxa Dragon Q6A (Qualcomm QCS6490) using Hexagon NPU (QNN/QAIRT 2.46)

Hi everyone!

I want to share a complete breakthrough in running hardware-accelerated object detection on the Radxa Dragon Q6A (Snapdragon QCS6490 architecture) inside Frigate NVR 0.17.

By bypassing the standard cloud/CPU bottlenecks and utilizing Qualcomm’s Hexagon NPU via custom ZeroMQ coupling, we achieved mind-blowing performance:

  • Inference Time: 10ms (YOLOv8n)

  • Detector CPU Usage: 0.2%

  • Detector Memory Usage: 1.2%

Here is a step-by-step summary of the challenges we solved and how to replicate this ultra-efficient local Edge AI setup.

The Architecture Setup

Instead of trying to force Frigate’s built-in ONNX detector plugin to load Qualcomm’s massive native compilation stacks directly, we decoupled the execution. We used a dual-process paradigm inside the container:

  1. A standalone Python ZeroMQ server (qnn_detector.py) managing the Qualcomm QAIRT SDK session.

  2. Frigate’s native zmq detector type connecting locally over TCP.

Step 1: Kernel & System Dependencies

To allow the container to talk to the hardware, the host system requires specific low-level nodes. Ensure your kernel exposes FastRPC and ION memory allocation:

  • /dev/fastrpc-cdsp (Hexagon Compute DSP access)

  • /dev/ion (Shared memory management)

  • Copied firmware binaries mapping to /lib/firmware on the host.

Step 2: Crafting the custom Dockerfile

We built a local image (frigate-local-qnn) extending Frigate’s stable standard arm64 layer. It embeds the Qualcomm QAIRT SDK binaries, installs necessary compilation toolchains, and sets up communication protocols.

Dockerfile

FROM ghcr.io/blakeblackshear/frigate:stable-standard-arm64

RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    python3-dev \
    cmake \
    && rm -rf /var/lib/apt/lists/*

COPY ./frigate/qnn_sdk_build /opt/qnn_sdk
ENV QNN_SDK_ROOT=/opt/qnn_sdk
ENV PIP_BREAK_SYSTEM_PACKAGES=1

RUN pip install --no-cache-dir --upgrade pip setuptools wheel --break-system-packages
RUN pip install --no-cache-dir onnxruntime-qnn pyzmq --break-system-packages

RUN mkdir -p /custom_detector
COPY ./qnn_detector.py /custom_detector/qnn_detector.py
RUN chmod +x /custom_detector/qnn_detector.py

# --- CRITICAL FIX: Overcoming Frigate 0.17 NoneType Bug ---
COPY ./frigate/patch_zmq.py /tmp/patch_zmq.py
RUN python3 /tmp/patch_zmq.py && rm /tmp/patch_zmq.py

Step 3: Patching Frigate 0.17 Core (patch_zmq.py)

In Frigate 0.17, there is a known schema strictness conflict. The Pydantic validator for type: zmq disallows passing model.path or model.name configuration arguments (throwing Extra inputs are not permitted). However, internally, frigate/detectors/plugins/zmq_ipc.py strictly attempts to read model_path via os.path.basename, causing a critical bootloop crashing with TypeError: expected str, bytes or os.PathLike object, not NoneType.

We resolved this dynamically during the Docker build stage using a python patch script (patch_zmq.py):

Python

import os

target_file = None
for root, dirs, files in os.walk('/opt/frigate/frigate'):
    if 'zmq_ipc.py' in files:
        target_file = os.path.join(root, 'zmq_ipc.py')
        break

if not target_file:
    print("ERROR: zmq_ipc.py not found!")
    exit(1)

with open(target_file, 'r') as f:
    content = f.read()

# Dynamically fallback to a default string if configuration object is None
content = content.replace(
    "return os.path.basename(model_path)",
    "return os.path.basename(model_path) if model_path else 'yolov8n.onnx'"
)

with open(target_file, 'w') as f:
    f.write(content)

Step 4: Docker Compose Orchestration

The environment variables require meticulous path definitions to hook into the FastRPC framework and load Qualcomm’s backend library (libQnnHtp.so) with High Performance INT8 modes activated.

YAML

version: '3.9'
services:
  frigate:
    image: frigate-local-qnn:latest
    container_name: frigate
    privileged: true
    shm_size: "256mb"
    devices:
      - /dev/ion:/dev/ion
      - /dev/fastrpc-cdsp:/dev/fastrpc-cdsp
    environment:
      - LD_LIBRARY_PATH=/opt/qnn_libs:/usr/lib/aarch64-linux-gnu:/usr/lib/dsp:/opt/qnn_sdk/qairt/2.46.0.260424/lib/aarch64-ubuntu-c++-gcc11
      - ADSP_LIBRARY_PATH=/opt/qnn_libs;/usr/lib/dsp
      - ONNXRUNTIME_QNN_BACKEND_PATH=/opt/qnn_sdk/qairt/2.46.0.260424/lib/aarch64-ubuntu-c++-gcc11/libQnnHtp.so
      - ONNXRUNTIME_QNN_HTP_PERFORMANCE_MODE=high_performance
      - ONNXRUNTIME_QNN_HTP_PRECISION=int8
      - ONNXRUNTIME_QNN_CONTEXT_ENABLE=1
      - ONNXRUNTIME_QNN_CONTEXT_CACHE_PATH=/config/model_cache
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /usr/lib/dsp:/usr/lib/dsp:ro
      - /home/radxa/qnn_libs:/opt/qnn_libs:ro
      - ./config:/config
      - ./storage:/media/frigate

Step 5: Clean config.yml

By shifting to type: zmq and keeping the model block empty of unaccepted keys, the validation passes smoothly:

YAML

detectors:
  qualcomm_npu:
    type: zmq
    endpoint: tcp://127.0.0.1:5555
    model:
      width: 640
      height: 640

Conclusion

By mapping host DSP libs, containerizing the QAIRT pipeline, spinning up a local ZeroMQ server wrapper for the model inference, and patching Frigate’s minor core discrepancy, the Radxa Dragon Q6A works flawlessly.

CPU utilization dropped to 0.2% for detection, turning the board into an industrial-grade, icy-cold, hyper-fast NVR system! Special thanks to the community for pointing out earlier hardware iterations.

1 Like