[BUG] Gemma 4 E2B infinite generation loop on AX-M1 via AXCL PCIe (x86 host)

Hardware and peripherals
1

Hi,

I’m running the Radxa AICore AX-M1 on an x86 host and experiencing an infinite generation loop with the Gemma 4 E2B model.

Hardware:

  • Host: Intel i7-10700K, x86_64, Debian 13 (Trixie)
  • AI accelerator: Radxa AICore AX-M1 (AX8850, PCIe M.2)
  • Second card: M5Stack LLM-8850 (AX8850, PCIe M.2)
  • AXCL Driver: V3.6.5_P1_20251018122826

Software:

  • axllm binary: e75d3022642d, built Jun 8 2026, backend: AXCL x86
  • Model: AXERA-TECH/gemma-4-E2B-it-GPTQ-INT4 (downloaded from HuggingFace)
  • Running inside Proxmox LXC with AXCL device passthrough

Symptom:
Model loads correctly, remain_cmm ~5005 MB, TTFT ~373ms, but instead of valid text generates an infinite loop of repeated symbols (I!I!I! or I I I I). Decode speed ~9.83 tok/s — tokens are being generated but content is garbage.

What works:

  • DeepSeek-R1-Distill-Qwen-7B-GPTQ-Int4 works correctly on the same hardware at ~4.5 tok/s
  • axcl-smi correctly detects both AX8850 cards

Related:

  • AXERA-TECH/ax-llm issue #39 (gemma4 infinite generation loop)
  • AXERA-TECH/ax-llm issue #21 (similar issue with Qwen3-1.7B on AX8850 AXCL)
  • AXERA confirmed in issue #39 that Gemma4 currently does not support inference on AXCL compute cards

Is there any update on AXCL PCIe support for Gemma4 on AX-M1? Is this something Radxa is working on with AXERA, or should this be tracked purely upstream?

Thanks