Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,8 +180,8 @@ mlops:

`cbuild setup --active SSE-320-U85` resolves it into
`cmsis-executorch.cbuild-mlops.yml`, which contains the processor, NPU
and Vela options. `create_ai_layer.py` reads those options and passes them to
ExecuTorch's `EthosUCompileSpec`, so the target configuration is never
and Vela options. `create_ai_layer.py` reads those options and passes all of
them to ExecuTorch's `EthosUCompileSpec`, so the target configuration is never
duplicated in Python. The script then writes:

- `ai_layer/ai_layer.clayer.yml`: the CMSIS components required by the model.
Expand Down Expand Up @@ -213,7 +213,8 @@ and build.

To target another Ethos-U configuration, update the target and `mlops:`
settings in the CMSIS solution and re-run all three steps. The generated Vela
options then follow that configuration automatically. Moving to a different
options then follow that configuration automatically; `vela: misc:` takes any
further Vela option, for example `--optimise Size`. Moving to a different
board or reference platform also requires the corresponding device pack, board
support, memory layout, and FVP configuration.

Expand Down
50 changes: 42 additions & 8 deletions create_ai_layer.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@

import os
import re
import shlex
import subprocess
import sys
from pathlib import Path
Expand Down Expand Up @@ -101,6 +102,42 @@ def executorch_pack(version: str) -> Path:
return pack_root() / vendor / pack / version


# Vela options that are arguments of EthosUCompileSpec, and the ones the export
# sets itself: the configuration file comes from vela.ini of the cbuild-mlops.yml.
SPEC_OPTIONS = ("accelerator-config", "system-config", "memory-mode")
EXPORT_OPTIONS = ("config", "output-format", "output-dir")


def vela_options(options: str) -> tuple[dict[str, str], list[str]]:
"""Split vela.options of the cbuild-mlops.yml: the EthosUCompileSpec arguments, and every other option.

The other options are the `misc:` of the csolution's mlops: node (for
example `--optimise Size`); they reach Vela as extra flags, written as
`--name=value`.
"""
tokens = shlex.split(options)
spec, extra = {}, []
while tokens:
token = tokens.pop(0)
if not token.startswith("--"):
sys.exit(f"vela options: unexpected '{token}' in '{options}'")
name, has_value, value = token[2:].partition("=")
if not has_value and tokens and not tokens[0].startswith("-"):
value = tokens.pop(0)
if name in EXPORT_OPTIONS:
sys.exit(
f"vela options: --{name} is set by the export; "
"the configuration file is vela: ini: of the mlops: node"
)
if name in SPEC_OPTIONS:
if name in spec:
sys.exit(f"vela options: --{name} is given twice in '{options}'")
spec[name] = value
else:
extra.append(f"--{name}={value}" if value else f"--{name}")
return spec, extra


def compile_spec(mlops: dict, mlops_dir: Path):
"""EthosUCompileSpec from the npu: and vela: nodes of the cbuild-mlops.yml."""
from executorch.backends.arm.ethosu import EthosUCompileSpec
Expand All @@ -109,17 +146,14 @@ def compile_spec(mlops: dict, mlops_dir: Path):
if not npu:
sys.exit("the solution's mlops: node names no NPU; this example needs an Ethos-U")
vela = mlops.get("vela", {})
options = vela.get("options", "")

def option(name: str) -> str | None:
found = re.search(rf"--{name}[= ](\S+)", options)
return found.group(1) if found else None
spec, extra = vela_options(vela.get("options", ""))

target = option("accelerator-config") or f"{npu['type'].lower()}-{npu.get('macs', 256)}"
target = spec.get("accelerator-config") or f"{npu['type'].lower()}-{npu.get('macs', 256)}"
kwargs = {
"target": target,
"system_config": option("system-config"),
"memory_mode": option("memory-mode"),
"system_config": spec.get("system-config"),
"memory_mode": spec.get("memory-mode"),
"extra_flags": extra,
}
if vela.get("ini"):
# ExecuTorch stores the path in the compile spec, and the spec ends up
Expand Down
4 changes: 3 additions & 1 deletion documentation/mlops-flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,9 @@ for an MLOps system. It reads the file and:

1. builds ExecuTorch's `EthosUCompileSpec` from `npu:` and `vela:` -- the
accelerator (`ethos-u85-256`), system config and memory mode come from
there, so the Python code contains no NPU configuration;
there, and every other option in `vela.options` (the `misc:` of the
csolution's `vela:` node) reaches Vela as an extra flag, so the Python code
contains no NPU configuration;
2. exports `model/model.py`: quantizes it, delegates the whole graph to the
Ethos-U and compiles it with Vela;
3. reads the operators the resulting program still calls on the CPU and looks
Expand Down
Loading