Running LLMs on Dragon Q6A NPU - Setup Guide & Questions

Rahul007 · July 7, 2026, 2:09pm

I went down exactly the same rabbit hole. Spent weeks trying QAIRT, QNN, AI Hub, ExecuTorch, compiled binaries, and even rented a RunPod instance just to verify I wasn’t losing my mind. Nothing worked. I also found it very difficult to follow the Qualcomm documentation as several examples and workflows either didn’t match reality or simply didn’t work as expected.

Finally got Llama 3.2 1B running on the Hexagon NPU using TrailCurrent’s script, and I can also confirm it works on Armbian 26.2.4 (Ubuntu 24.04 LTS), not just the official Radxa OS.

Qualcomm’s AI ecosystem definitely isn’t for the faint-hearted. It feels like solving an escape room before you can run your first prompt.

My only remaining disappointment is the apparent 1024-token context window and ~250-token output limit. Is that a limitation of the compiled QAIRT model, Genie, or the backend itself?

The script I used downloads the precompiled binaries from ModelScope:

github.com/trailcurrentoss/YouTubeCodeSamples

install-genie-npu.sh

main

#!/usr/bin/env bash
#
# Bring up a Qualcomm Genie NPU LLM server on a Radxa Dragon Q6A.
#
# Starts from a stock Radxa OS R2 (or newer) image, downloads the
# NPU-optimized Llama 3.2 1B bundle, installs the TrailCurrent Peregrine
# genie_server.py + systemd unit, and starts the service. Result: an
# Ollama-compatible HTTP endpoint on http://localhost:11434 serving the
# model from the Hexagon NPU at ~12 tokens/s.
#
# Run as the `radxa` user (not root). Requires sudo for the systemd
# install step and internet access for the model + repo files.
#
# Usage:
#     bash install-genie-npu.sh

set -eu

CTX_LENGTH="${CTX_LENGTH:-1024}"
MODEL_REPO="radxa/Llama3.2-1B-${CTX_LENGTH}-qairt-v68"

This file has been truncated. show original

At least it finally got the NPU working. Hopefully larger context windows become possible in the future.