Thin NeuralSPOT-X adapter for the stock ExecuTorch Cortex-M runtime.
The package keeps ExecuTorch unmodified and supplies:
- an NSX CMake target,
nsx::executorch; - caller-owned, heap-free method and tensor arenas;
- total
Method::execute()cycle measurement; - optional per-operator callbacks backed by ExecuTorch
EventTracer; - deterministic, offline source and host-tool pins.
Set NSX_EXECUTORCH_CMSIS_NN_PROVIDER before adding the module:
set(NSX_EXECUTORCH_CMSIS_NN_PROVIDER arm CACHE STRING "" FORCE)arm(default) resolves thearm-cmsis-nnNSX module (repoAmbiqAI/arm-cmsis-nn, exported targetnsx::arm_cmsis_nn), which itself vendors pinned, stock upstream CMSIS-NN and CMSIS 6 sources.nsresolvesnsx-cmsis-nnv7.29.2 (project repoAmbiqAI/ns-cmsis-nn). Its NSX target is bridged to the stockcmsis-nntarget expected by ExecuTorch.
Neither CMSIS-NN nor CMSIS 6 is vendored directly in this repository anymore;
both are consumed as NSX modules. NSX does not resolve optional dependencies
automatically, so an app must list the selected provider immediately before
nsx-executorch in nsx.yml:
modules:
- name: arm-cmsis-nn # or nsx-cmsis-nn
- name: nsx-executorchSet NSX_EXECUTORCH_CMSIS_NN_PROVIDER before nsx_bootstrap_app(). Keeping
the provider first lets NSX configure it exactly once despite stock
ExecuTorch's non-idempotent CMSIS-NN add_subdirectory().
For standalone use (development, CI, helia-profiler materialization, or direct
add_subdirectory()), set the provider root in the CMake cache:
# provider=arm
set(NSX_EXECUTORCH_ARM_CMSIS_NN_ROOT "/path/to/arm-cmsis-nn" CACHE PATH "" FORCE)
# provider=ns
set(NSX_EXECUTORCH_NS_CMSIS_NN_ROOT "/path/to/ns-cmsis-nn" CACHE PATH "" FORCE)The ns override accepts either the repository root or its nsx/ module
directory. NSX_EXECUTORCH_SOURCE_ROOT similarly overrides the pinned
ExecuTorch checkout. CMake never fetches dependencies: all provider and
ExecuTorch sources must be materialized before configure. This is a
build-tree NSX module; stock ExecuTorch package-install/export rules are
suppressed because they cannot encode an app-selected NSX provider.
Both providers consume the same stock Cortex-M operators and the same PTE. Provider selection never changes operator schemas or ExecuTorch lowering. NS-CMSIS-NN v7.29.2 adds a weight-sum context to convolution, depthwise convolution, and transpose-convolution wrappers. A private adapter extends ExecuTorch's temporary allocations, precomputes those sums, and passes the extra contexts without modifying the pinned ExecuTorch source.
Enable layer callbacks with:
set(NSX_EXECUTORCH_ENABLE_PROFILING ON CACHE BOOL "" FORCE)Each callback receives an OperatorEvent: the ExecuTorch chain and
instruction identifiers plus what the instruction runs — the serialized
operator name and overload (aten::add / out,
cortex_m::quantized_conv2d / out) for kernel calls, the backend id for
delegate calls. A caller such as Helia Profiler labels its per-layer rows from
the name and samples PMU counters around every operator.
The ns provider ships kernels beyond stock CMSIS-NN. This repo can expose a
first tier of them — sub, hardswish, mean, standalone
relu/relu6/hardtanh/clamp, and leaky_relu — to ExecuTorch models as
quantized int8 operators in a dedicated cortex_m_ns:: namespace. Everything
composes stock ExecuTorch through public extension points (ops-ns/ operator
YAML + codegen, aot/nsx_cortex_m/ Python package); the pinned submodule is
never modified.
Namespace contract:
- PTEs containing only stock
cortex_m::ops run on both providers. - PTEs containing
cortex_m_ns::ops require an ns build with NS ops enabled; on any other build they fail fast atMethod::load()with "operator missing" instead of silently miscomputing.
Runtime gating (default OFF; provider=arm builds are byte-identical and gain no new targets):
set(NSX_EXECUTORCH_CMSIS_NN_PROVIDER ns CACHE STRING "" FORCE)
set(NSX_EXECUTORCH_ENABLE_NS_OPS ON CACHE BOOL "" FORCE)| provider | NSX_EXECUTORCH_ENABLE_NS_OPS |
result |
|---|---|---|
| arm | OFF (default) | stock build, no new targets |
| arm | ON | configure error (fail fast) |
| ns | OFF (default) | stock ns build, no new targets |
| ns | ON | adds cortex_m_ns_kernels + cortex_m_ns_ops_lib, raises MAX_KERNEL_NUM to 96 |
Enabling NS ops additionally requires PyYAML on the host Python (ExecuTorch
codegen); the vendored tools/python tree covers torchgen only.
AOT export (requires a host ExecuTorch install, see aot/):
from nsx_cortex_m import export
result = export(model, example_inputs, kernel_provider="ns") # or "arm"
result.write_pte("model.pte") # also writes the model.pte.json sidecar
print(result.edge_ops) # lowered operator histogram
print(result.portable_select_ops_list) # NSX_EXECUTORCH_PORTABLE_SELECT_OPS_LIST valueaot/ builds as the helia-torch pip package (see aot/pyproject.toml;
its torch/executorch pins must match this repository's pinned submodule).
The CLI is file-in/file-out: the user runs torch.export.save() in their
own training environment and hands the .pt2 artifact to the tool — no
model code executes here.
helia-torch compile model.pt2 --provider ns --calibrate calib.npz
helia-torch report model.pte # kernel lowering report from the sidecar
helia-torch inspect model.pte # sidecar manifest as JSONcompile writes model.pte plus a model.pte.json sidecar (schema
nsx-executorch.pte-manifest/1), SHA-256-bound to that exact PTE. The
sidecar records everything a target build needs to register the right
kernels and size its memory: kernel provider, requires_ns_ops, the
portable select-ops list, the planned arena size, and per-tensor I/O sizes.
Consumers verify the hash and self-configure — helia-profiler reads it
automatically, so a profiling config reduces to the model path, this
checkout's path, and a board. The compile step also prints a lowering
report that flags any [PORTABLE] fallback loudly: a portable op runs on
the standard float kernels and typically dominates per-op cost on target.
Memory placement is intentionally not in the sidecar: sizes are export
facts, but which RAM region each caller-owned arena lives in is a
board-pressure decision made by the consumer (helia-profiler's
engine.config.*_location keys, or the application's linker attributes).
Splitting activations across multiple planned arenas at export time
(per-tensor placement) is tracked in
issue #3.
kernel_provider="arm" reproduces the stock flow exactly (zero
cortex_m_ns:: ops). With "ns", ops that fail a lowering qualifier stay
portable ATen ops (correct but slow) and are reported in
portable_fallback_ops so the matching
NSX_EXECUTORCH_PORTABLE_SELECT_OPS_LIST value can be applied.
Tier 1 restrictions (documented qualifiers; anything else falls back):
- int8 per-tensor quantization only (int16 is out of scope for this tier).
sub: channel broadcast only, mirroring stockquantized_mul, and only for channels_last tensors (stockCortexMAddMulCheckpolicy).mean: rank ≤ 4, contiguous (non-channels_last) tensors, compile-time reduce dims (maps ontoarm_mean_s8's NHWC axis mask; the element count is folded into the output requantization AOT).relu/relu6/hardtanh/clamp: compile-time scalar bounds; bounds are quantized into the output domain and requantization is supported.leaky_relu: compile-time positive slope (two AOT requant pairs).hardswish: usesarm_hard_swish_precise_s8withprescale=0(matches the pinned ns-cmsis-nn unit tests; guarded byrelu_q6 ≤ 2^22).
Follow-up tiers (later PRs): sigmoid/tanh int16, data-movement ops, further reductions, LSTM/SVDF.
Clone with the pinned ExecuTorch submodule:
git clone --recurse-submodules https://github.com/AmbiqAI/nsx-executorch.gitFor a minimal Cortex-M checkout, initialize external/executorch and the
ExecuTorch submodules used by its stock CMake configuration:
git submodule update --init external/executorch
git -C external/executorch submodule update --init \
backends/xnnpack/third-party/FXdiv \
third-party/flatbuffers third-party/flatcc third-party/gflags third-party/jsonThen verify the exact revision and configure the profiling-enabled smoke
projects for both providers (using the checked-in test fixtures that stand in
for real arm-cmsis-nn / ns-cmsis-nn checkouts):
python3 tests/verify_source_pins.py
cmake -S tests/smoke -B build/configure-smoke-arm -G Ninja
cmake -S tests/smoke-ns -B build/configure-smoke-ns -G Ninja
cmake -S tests/smoke-standalone -B build/configure-standalone-arm -G Ninja \
-DNSX_EXECUTORCH_TEST_PROVIDER=arm
cmake -S tests/smoke-standalone -B build/configure-standalone-ns -G Ninja \
-DNSX_EXECUTORCH_TEST_PROVIDER=nsCI additionally builds the stock Cortex-M kernels with the exact Arm and NS
provider revisions listed in PROVENANCE.md.
docs/build-options.md is the audit record for every ExecuTorch option this
module forces (and the ones it deliberately leaves off), how compiler flags
reach the subtree, and hardware before/after numbers for the choices that
were changed. The short version: the module owns the subtree's optimization
level via NSX_EXECUTORCH_OPTIMIZATION (speed = -O3 default, size =
-Os), strips the app policy flags (-O3 -ffast-math) that the board
interface would otherwise impose, and defaults
NSX_EXECUTORCH_ENABLE_PROGRAM_VERIFICATION=OFF (the sidecar SHA covers PTE
integrity). Hold NSX_EXECUTORCH_OPTIMIZATION constant across arm-vs-ns
comparisons.
The module builds under arm-none-eabi-gcc and ATfE (Arm Toolchain for
Embedded, LLVM-based). armclang is not yet supported. Cross-compiling test
fixtures carry one toolchain file each: tests/real-provider/arm-none-eabi-gcc.cmake
and tests/real-provider/atfe.cmake (requires the ATFE_ROOT env var pointing
at an ATfE install with the newlib overlay; skip the ATfE legs when unset).
All per-toolchain accommodations in the module live in the single
CMAKE_CXX_COMPILER_ID STREQUAL "Clang" block at the end of CMakeLists.txt.
Validated: ATfE 22.1.0 (newlib overlay) on Cortex-M55 / Apollo510 EVB, both
providers, NSX_EXECUTORCH_ENABLE_NS_OPS on and off, plus portable ops via
NSX_EXECUTORCH_PORTABLE_SELECT_OPS_LIST=aten::clamp.out,aten::leaky_relu.out,aten::sub.out
(the tier-1 arm PTE's fallback list). Hardware numbers for the tier-1 arm PTE (96 MHz LP,
median of 25, helia-profiler hpx compare): total 3.51M cycles (GCC 15.2)
vs 2.77M cycles (ATfE 22.1.0), −21%; firmware .text −5.5%. Most of the
delta is clang auto-vectorizing float portable kernels with MVE tail
predication (aten::clamp.out 63x); aten::leaky_relu.out regressed ~54%
under ATfE — see the issue #5 flag audit. Numerical parity between the
toolchains is not yet exercised: hpx validate still gates ExecuTorch to
GCC (AmbiqAI/helia-profiler#141).
Known toolchain differences (issue #6):
- The fixture board flags pass no
-mfpufor Cortex-M55 (matching NSX'snsx_apply_toolchain_flags): an explicit-mfpu=fpv5-sp-d16downgrades clang to integer-only MVE (__ARM_FEATURE_MVE=1), silently breaking ExecuTorch's Helium float kernels; GCC ignores-mfpufor MVE derivation. - Upstream CMSIS-NN defaults
CMSIS_OPTIMIZATION_LEVELto-Ofast, which clang 22 deprecates (warning only).