ARM SPE on Nio 12L

Hi,
I am actually trying to get the Statistical Profiling Extension (SPE) to run on the NIO 12L with Ubuntu. The Cortex-A78 cores should have this feature (the corresponding bits in the ID_AA64DFR0_EL1 register are also set).

However, apart from passing kpti=off to the kernel and loading the corresponding kernel module (arm_spe_pmu), SPE must be also enabled in the Firmware, so I tried to add a SPE_PMU entry to the device tree, since arm_spe_0 is not available in /sys/bus/event_source/devices/. I simply used the examples given here: https://www.kernel.org/doc/Documentation/devicetree/bindings/arm/spe-pmu.txt

So the devicetree entry looks like:

spe-pmu {
compatible = “arm,statistical-profiling-extension-v1”;
interrupts = <0x01 0x05 0x04>;
interrupt-parent = <0x01>;
};

The board boots correctly, however dmesg now shows arm_spe_pmu spe-pmu: IRQ index 0 not found during boot, which suggests, that the settings I used were somehow wrong or incomplete. Do I have do modify anything else in the devicetree, to get SPE enabled?

Edit: Since the genio1200-radxa-nio-12l.dtb devicetree uses PPI partitions and the interrupt field needs a fourth field (https://www.kernel.org/doc/Documentation/devicetree/bindings/interrupt-controller/arm%2Cgic-v3.txt), that links to the partition of the big cores, I added the phandle of the PPI partition of the big cores (0x16) to the Interrupt field, unfortunately the board then does not start anymore. I can see the grub entry via UART but then the screen is blank.

spe-pmu {
compatible = “arm,statistical-profiling-extension-v1”;
interrupts = <0x01 0x05 0x04 0x16>;
interrupt-parent = <0x01>;
};

OK, it seems, that it is not simply done by adding a SPE node in the device tree, the firmware has to be adapted as well.
When I add a SPE-node to the device tree and assign it to the interrupt partition of the cortex a78, the device does not boot. I assume that the problem is similar to the problem stated here (https://patchwork.kernel.org/project/linux-mediatek/patch/[email protected]/).

As far as I understand now, the feature has to be activated in the firmware. Any help on this issue would be really great.

I cam’t really help, but I’m seeing the same basic issue in the 5B+. I can add the arm_spe_pmu module okay, and it loads, but the interrupt is missing, and therefore, the event sources aren’t set up. We really need someone who knows the hardware to guide us here.

I’ll try these on the 5B+ and see if they yield insights.

Hi,

I’ve been diving deep into getting the ARM Statistical Profiling Extension (SPE) working on both the Radxa NIO 12L (MediaTek MT8395 / Genio 1200) and the Radxa Dragon Q6A (Qualcomm QCS6490 / Snapdragon).

Since ARM SPE requires modifications across EL3 (TF-A) and Device Tree, here is a detailed breakdown of what I discovered on both platforms, including the technical root causes for both the NIO 12L and Dragon Q6A.

Summary: I failed to get SPE fully working on both SBCs. (Radxa NIO 12L works without the interruption)


1. Radxa NIO 12L (MediaTek MT8395)

Status Overview

  • What works: SPE instruction sampling & recording (~2MB of profiling data is successfully written to DRAM, PMBPTR_EL1 advances, and PMBSR_EL1.S = 1 sets the buffer-full flag).
  • What does NOT work: The hardware buffer-full interrupt (PPI 5) is not delivered properly when PMBSR_EL1.S = 1.

Custom Firmware (TF-A / BL31) Modifications

To allow Non-Secure (Linux) access to SPE, the following modifications in TF-A BL31 are required:

SPE Access & EL3 Traps (MDCR_EL3)

I compiled TF-A with ENABLE_SPE_FOR_LOWER_ELS=1 (ENABLE_SPE_FOR_NS=1 for the current version of TF-A?).

  • Clear MDCR_EL3.SDD (Bit 16 = 0): Disables Secure Debug trap, allowing Non-Secure SPE access.
  • Set MDCR_EL3.NSPB (Bits [13:12]) to 0b11 (0x3000): Enables SPE for Non-Secure EL1/EL2.
Interrupt & Debug Subsystem Unlocking

(Attempted to configure CoreSight CTI for interrupt routing; however, CTI appears to be unrelated to SPE interrupt delivery on this SoC)

  • DevAPC Firewall (MediaTek Security Firewall):
    • Set DEVAPC_DEBUGSYS_INDEX (Index 17: 0x0D000000 CoreSight / DEBUGSYS region) to NO_PROTECTION across all 16 security domains in devapc.c.
  • CoreSight CTI (Cross Trigger Interface):
    • Unlocked all CoreSight CTI instances using the standard ARM software lock key 0xC5ACCE55 and set CTICONTROL = 1.

Kernel Device Tree (DTS)

spe-pmu {
	compatible = "arm,statistical-profiling-extension-v1";
	interrupt-parent = <&gic>;
	interrupts = <GIC_PPI 5 IRQ_TYPE_LEVEL_HIGH &ppi_cluster1>;
	interrupt-affinity = <&cpu4>, <&cpu5>, <&cpu6>, <&cpu7>;
};

Adding the spe-pmu node as shown above enables the arm_spe_pmu kernel module to probe successfully.
However, the buffer-full interrupt does not fire when expected.
GIC_PPI 5 seems to be constantly asserted.

Conclusion for NIO 12L

SPE hardware profiling itself is fully functional, but the hardware interrupt delivery does not work as expected.

  • SPE Hardware is Functional The Cortex-A78 cores successfully sample instructions and write ~2MB of profiling data into DRAM (PMBPTR_EL1 advances and PMBSR_EL1.S = 1 sets the buffer-full flag).
  • The Interrupt Issue (PPI 5 IRQ):
    • Despite setting GIC GICR_IGROUPR0 to Non-Secure Group 1, unlocking DevAPC, configuring MDCR_EL3, enabling the relevant debug power domains, and enabling CTI 8x8 routing, the hardware interrupt line (PPI 5 / IntID 21) is asserted to the GIC before PMBSR_EL1.S = 1 rather than when the buffer is full.

Possible root causes are:

  1. The SPE interrupt output pin is physically unwired in MediaTek’s MT8395 silicon design.
  2. The interrupt is routed to a different interrupt number (e.g., an SPI instead of PPI 5).
  3. A necessary debug/PMU clock or power domain remains powered down.
  4. Additional MediaTek-specific CoreSight/SPM configuration is required.

2. Radxa Dragon Q6A (Qualcomm QCS6490 / Snapdragon)

I also evaluated enabling SPE on the Radxa Dragon Q6A to see if a similar approach works. The QCS6490’s Cortex-A78 cores hardware-support SPE, but Non-Secure access is blocked by stock EL3 firmware.

Diagnostic Findings

With the following DTS node:

spe-pmu {
        compatible = "arm,statistical-profiling-extension-v1";
        interrupt-parent = <&gic>;
        interrupts = <GIC_PPI 5 IRQ_TYPE_LEVEL_HIGH>;
};

Upon loading the arm_spe_pmu driver in Linux, the kernel logs the following error:

arm_spe_pmu spe-pmu: profiling buffer owned by higher exception level

This error occurs because the CPU’s PMBIDR_EL1 register indicates that SPE buffer programming is locked by a higher Exception Level. Specifically, Qualcomm’s proprietary EL3 firmware (tz.mbn / QTEE) leaves MDCR_EL3.NSPBE (Bit 18, Non-secure Statistical Profiling Buffer Enable) set to 0 (Disabled).

Custom Firmware Attempt & SBL1 Rejection

I attempted to build and flash open-source trusted firmware, but the board’s bootloader rejects it:

  1. Building Custom Open-Source TF-A:
    • I built Qualcomm’s official open-source TF-A (qualcomm-linux/trusted-firmware-a, PLAT=rb3gen2) and OP-TEE (qualcomm-linux/optee_os, PLATFORM_FLAVOR=kodiak) with ENABLE_SPE_FOR_NS=2 (ENABLE_SPE_FOR_LOWER_ELS=1).
    • This successfully compiled bl2.mbn (TF-A BL2) and fip.elf (TF-A BL31 + OP-TEE + UEFI).
  2. SBL1 Rejection:
    • Replacing Qualcomm’s stock tz.mbn with the custom bl2.mbn results in a boot failure at the Primary Boot Loader / XBL (SBL1) stage:
      B - 941352 - QSEE - Image Load, Start
      D - 854 - Auth Metadata
      B - 945957 - Error code 65 at sbl1_mc.c Line 2089
      
    • I think this error suggests that SBL1 rejects non-signed (or test-signed) TF-A / BL2 images (though I haven’t completely ruled out other potential header format or signing issues).

Conclusion for Dragon Q6A

Unless Qualcomm or Radxa provides an officially signed tz.mbn (or signed TF-A release) with MDCR_EL3.NSPBE = 1 enabled, it seems end-users cannot simply run a custom, non-signed TF-A on the Dragon Q6A.


Summary & Takeaways

  • Radxa NIO 12L (MT8395): SPE hardware works and records data to DRAM (~2MB written). Custom TF-A can be flashed without any issue (signing is not required). Interrupt delivery does not function (possibly unwired in silicon), but can be worked around via kernel polling (hrtimer).
  • Radxa Dragon Q6A (QCS6490): SPE is disabled by stock tz.mbn (MDCR_EL3.NSPBE = 0). Replacing tz.mbn with custom TF-A triggers Error code 65 in XBL, which (may) indicates that SBL1 rejects non-signed TF-A images.

I hope a more detailed document for the MT8395 becomes available, or that Radxa releases updated firmware enabling SPE for Radxa Dragon Q6A(and Q8B).

If anyone would like my working code or repository, please let me know!