Phase 3: tiny-tpu backend (Tile IR, tiling, .ttpu.json, real-RTL executor)

CLAUDE.md

@@ -18,6 +18,7 @@ python3 -m venv .venv && .venv/bin/pip install -e ".[dev]" && .venv/bin/pip inst
 source scripts/env.sh
 tinyc examples/mm_relu.tiny --locs          # optimized Graph IR (--emit ast, -O0, --target gpu|tpu,
                                             #   --print-after-all, --verify-each, --time-passes, --disable PASS)
+tinyc examples/mm_relu.tiny --target tpu --run          # compile + run on the real RTL (also --emit tile|ttpu, --stats)
 pytest -q                                   # full suite
 pytest --update-golden                      # rewrite tests/golden/* after reviewing the diff
 cd web && pnpm dev | build | lint | typecheck | test

CONTEXT.md

@@ -13,9 +13,12 @@ faked; host-side work is always labelled as host work.
 - Phase 0 (scaffold + drive both targets from pytest): DONE. 17 tests green (runners + target contract), ruff/black clean.
 - Phase 1 (frontend + Graph IR + oracle): DONE. 90 tests green: lexer/parser/diagnostics (20 golden error messages), Graph IR
   printer/parser round trip, verifier, numpy oracle + independent scalar interpreter, 6 examples.
-- Phase 2 (pass manager + graph passes): DONE. 183 tests green. Passes: shape-infer, canonicalize, const-fold, cse, dce,
+- Phase 2 (pass manager + graph passes): DONE. Passes: shape-infer, FIXPOINT{canonicalize, const-fold, cse, dce},
   fuse (matmul{epilogue=bias,relu}), range-check, legalize-tpu/gpu; `--verify-each`, `--print-after-all`, `--time-passes`.
-- Next: Phase 3 (TPU backend: Tile IR, tiling, `.ttpu.json`, cost model, `ttpu-exec` full command list).
+- Phase 3 (TPU backend): DONE. 287 tests green. Graph IR -> Tile IR (`tinyc/tpu/`) -> `.ttpu.json` (schema `docs/ttpu.schema.json`,
+  spec `docs/TTPU_FORMAT.md`); `runtime/ttpu-exec.mjs` runs whole programs on the real RTL; cost model == measurement (pinned
+  in `tests/perf_baseline.json`). `tinyc FILE --target tpu --run` works.
+- Next: Phase 4 (GPU backend -O0: Kernel IR, memory planner, regalloc, `.tgpu`, gpu_runner).
 
 ## Fixed decisions (do not re-litigate; change only via docs/ARCHITECTURE.md)
 | Decision | Choice |
@@ -28,7 +31,9 @@ faked; host-side work is always labelled as host work.
 | GPU mapping | One thread per output element: block = row i, thread = column j |
 | GPU regalloc | Linear scan, budget 16 by default, no spilling (compile error) |
 | TPU mapping | 4×4 int8 passes, zero padding, K-accumulation + relu/bias as `host.*` ops |
-| TPU executor | Exactly one: `runtime/ttpu-exec.mjs` (Node in tests, browser in UI) |
+| TPU executor | Exactly one: `runtime/ttpu-exec.mjs` (Node in tests, browser in UI). `tinyc/tpu/interp.py` is a second, independent *reference* reader of the same JSON spec, never a second executor |
+| TPU host work | `acc`/`epilogue`/`host_add`/`host_relu`/`alloc` are host commands, counted separately from chip passes everywhere |
+| TPU cost | `passes * 15` RTL cycles; utilization = useful MACs (`M*K*N`) / (64 * passes) |
 | Overflow | `range-check` proves every value fits i32 by intervals; otherwise reject (tiny-gpu raises on overflow). i32 *inputs* are assumed within ±2^20 (`PassContext.i32_input_range`); runners must respect it |
 | Fusion | `matmul{epilogue=bias,relu}`; the fused op takes the outermost absorbed op's loc. Backends must lower the epilogue |
 | Verification | Oracle (numpy) + per-pass translation validation + real targets + Hypothesis differential + golden + perf pins |

README.md

@@ -4,10 +4,11 @@ One tiny tensor language, two real chips: `C = relu(A @ B)` compiles to
 [tiny-gpu](https://github.com/deaneeth/tiny-gpu) assembly or to a
 [tiny-tpu](https://github.com/deaneeth/tiny-tpu) tile schedule, and runs on the real simulator / RTL.
 
-**Status: Phase 2.** The frontend, Graph IR and optimizing pass pipeline work
-(`tinyc examples/mm_bias_relu.tiny --target gpu --print-after-all`); the chip backends come next.
+**Status: Phase 3.** The frontend, optimizing pass pipeline and the **tiny-tpu backend** work:
+`tinyc examples/mm_relu.tiny --target tpu --run` compiles to tile passes and runs them on the real
+RTL, bit-exact against numpy. The tiny-gpu backend comes next.
 Both targets can be driven from pytest. See [docs/PLAN.md](docs/PLAN.md) for the roadmap,
-[docs/LANGUAGE.md](docs/LANGUAGE.md), [docs/IR.md](docs/IR.md) and [docs/PASSES.md](docs/PASSES.md).
+[docs/LANGUAGE.md](docs/LANGUAGE.md), [docs/IR.md](docs/IR.md) and [docs/PASSES.md](docs/PASSES.md) and [docs/TTPU_FORMAT.md](docs/TTPU_FORMAT.md).
 
 ```bash
 git clone --recurse-submodules <repo> && cd tiny-compiler

docs/PASSES.md

@@ -3,10 +3,17 @@
 `tinyc/driver.py` builds the pipeline; `tinyc/passes/manager.py` runs it.
 
 ```
--O1:  shape-infer, canonicalize, const-fold, canonicalize, cse, dce, fuse, range-check, [legalize-<target>]
+-O1:  shape-infer, FIXPOINT{canonicalize, const-fold, cse, dce}, fuse, range-check, [legalize-<target>]
 -O0:  shape-infer, range-check, [legalize-<target>]
 ```
 
+The four simplifications run as a **fixpoint group** (`passes.manager.Fixpoint`): they enable each
+other, so the whole group repeats until a full round changes nothing. A single sweep is not enough:
+CSE can merge an op into an earlier one and leave a commutative `add` with its operands in
+non-canonical order (Hypothesis found this; `tests/passes/test_manager.py` keeps the shrunk
+program). Every sub-pass is still verified, recorded and dumped on every round, so
+`--print-after-all` shows each round.
+
 | Pass | What it does |
 |---|---|
 | `shape-infer` | Recomputes every result type from its operands (repairs parsed or hand-edited IR). |
@@ -48,3 +55,11 @@ tinyc examples/mm_relu.tiny --disable fuse      # switch a pass off
 ```
 Dumps and timings go to stderr; the final IR goes to stdout. Exit codes: 0 ok, 1 compile or
 legalization error, 2 usage error, 3 internal compiler error (a pass broke the program).
+
+## Tile passes (tiny-tpu backend)
+After lowering to Tile IR (`tinyc/tpu/lower.py`) the `-O1` pipeline runs `skip-zero-passes`:
+a pass whose A or B window is a constant all-zero tile is removed together with its accumulation
+(its product is zero). If that accumulation carried the output tile's epilogue, the epilogue moves
+to the previous kept accumulation, becomes a standalone `epilogue` (bias present), or disappears
+(relu only: relu of the zero-initialized tile is a no-op). Pass ids are renumbered. `--verify-each`
+validates the schedule after lowering and after each tile pass, by interpreting the emitted JSON.

docs/PLAN.md

@@ -153,7 +153,7 @@ tpu.schedule @main tile=4x4 {
   schedule order matches tiny-tpu's `tilingSchedule()` (output tile row-major, K innermost),
   so the existing tiny-tpu L3 view can display it.
 - **Tile passes:** skip passes whose A or B tile is all zero (constants only), sink `relu`/bias
-  into the last `host.acc` of each output tile, and report utilization (useful MACs ÷ 16 per pass).
+  into the last `host.acc` of each output tile, and report utilization (useful MACs ÷ 64 per pass: a pass is 4×4×4 MACs).
 - **Emit:** `.ttpu.json` = a buffer table, int8 tile payloads (or slice refs), and a command list
   (`pass`, `acc`, `relu`, `add_bias`, `store`). The format is versioned and has a JSON Schema
   in `docs/TTPU_FORMAT.md`.

docs/TTPU_FORMAT.md

@@ -0,0 +1,58 @@
+# `.ttpu.json` (version 1)
+
+The program a tiny-tpu run executes. Machine-checked by `docs/ttpu.schema.json`
+(`tests/tpu/test_ttpu_format.py` validates every emitted program against it).
+Consumers: `runtime/ttpu-exec.mjs` (the executor; Node and browser) and `tinyc/tpu/interp.py`
+(an independent reference interpreter written from this spec).
+
+```json
+{ "format": "ttpu", "version": 1, "tile": 4, "name": "main",
+  "buffers": { "A": {"dtype":"i8","shape":[4,8],"kind":"input"},
+               "1": {"dtype":"i32","shape":[4,4],"kind":"temp"} },
+  "inputs": ["A", "B"], "outputs": ["1"],
+  "commands": [ ... ], "estimate": { ... } }
+```
+
+## Buffers
+`dtype` is `i8` or `i32`; `shape` has rank 1 or 2 (row-major). `kind`:
+`input` (supplied at run time), `const` (carries `data` as nested lists) or `temp` (zero-filled
+at start). Buffer names are the Graph IR value names without `%`.
+
+## Commands, in execution order
+Only `pass` runs on the chip. **Everything else is host work**, counted separately and never
+attributed to the chip.
+
+A *window* `{"buf", "r", "c"}` is the 4x4 region of a rank-2 buffer whose top-left element is
+`(r, c)`. Reading outside the buffer yields 0 (zero padding); writing outside it is dropped.
+
+| `op` | Fields | Meaning |
+|---|---|---|
+| `alloc` | `buf` | **host.** Zero the buffer. |
+| `pass` | `id`, `a`, `b` (windows) | **chip.** `tmp[id] = a_window @ b_window` (int8 x int8 -> int32, 4x4). Each pass starts from psum = 0 and reloads its weights. |
+| `acc` | `pass`, `dst` (window), optional `bias` (buffer name), optional `relu: true` | **host.** `dst += tmp[pass]` (in-bounds part), then the epilogue: `dst[r][c] += bias[c]`, then `max(dst, 0)`. |
+| `epilogue` | `dst` (window), optional `bias`, `relu` | **host.** The epilogue alone (an output tile that no pass feeds). |
+| `host_add` | `dst`, `a`, `b` | **host.** Whole-buffer add; same shape, or a rank-1 operand broadcast over the rows of a rank-2 one. |
+| `host_relu` | `dst`, `src` | **host.** `dst = max(src, 0)`. |
+
+Every pass is consumed by exactly one `acc`, after it ran. Every value must fit in i32 or the
+executor fails (`RangeError`); `range-check` proves this at compile time for any legal input.
+
+## Tiling order
+For `C = A @ B`, `A: [M,K]`, `B: [K,N]`: output tiles row-major, K innermost, matching tiny-tpu's own
+`tilingSchedule()`. `ceil(M/4) * ceil(N/4) * ceil(K/4)` passes before optimization. The last `acc`
+of an output tile carries that tile's epilogue.
+
+## `estimate`
+Static cost model; the executor must measure exactly this (`tests/tpu/test_tpu_rtl.py`).
+
+| Key | Meaning |
+|---|---|
+| `passes`, `passes_skipped` | chip passes left, and passes removed by `skip-zero-passes` |
+| `cycles_per_pass` | 15 (14 from start to done, +1 for the last element to latch; `getCycleCount()`) |
+| `rtl_cycles` | `passes * 15` |
+| `host_ops` | counts per host command kind |
+| `useful_macs` / `capacity_macs` / `utilization` | non-padding multiply-adds (`M*K*N` for the passes kept) over `64 * passes` (a pass is 4x4x4 MACs) |
+
+## Executor protocol (`runtime/ttpu-cli.mjs`)
+Request `{ "program": <this document>, "inputs": { name: nested lists } }` ->
+`{ "outputs": { name: {shape, data} }, "passes": [{id, cycles}], "totalCycles", "hostOps" }`.

docs/ttpu.schema.json

@@ -0,0 +1,83 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://github.com/deaneeth/tiny-compiler/docs/ttpu.schema.json",
+  "title": "ttpu program (version 1)",
+  "type": "object",
+  "required": ["format", "version", "tile", "name", "buffers", "inputs", "outputs", "commands", "estimate"],
+  "additionalProperties": false,
+  "properties": {
+    "format": { "const": "ttpu" },
+    "version": { "const": 1 },
+    "tile": { "const": 4 },
+    "name": { "type": "string" },
+    "buffers": {
+      "type": "object",
+      "additionalProperties": { "$ref": "#/$defs/buffer" }
+    },
+    "inputs": { "type": "array", "items": { "type": "string" } },
+    "outputs": { "type": "array", "items": { "type": "string" }, "minItems": 1 },
+    "commands": { "type": "array", "items": { "$ref": "#/$defs/command" } },
+    "estimate": {
+      "type": "object",
+      "required": ["passes", "passes_skipped", "cycles_per_pass", "rtl_cycles", "host_ops", "useful_macs", "capacity_macs", "utilization"],
+      "additionalProperties": false,
+      "properties": {
+        "passes": { "type": "integer", "minimum": 0 },
+        "passes_skipped": { "type": "integer", "minimum": 0 },
+        "cycles_per_pass": { "type": "integer", "minimum": 1 },
+        "rtl_cycles": { "type": "integer", "minimum": 0 },
+        "host_ops": {
+          "type": "object",
+          "required": ["alloc", "acc", "epilogue", "host_add", "host_relu"],
+          "additionalProperties": { "type": "integer", "minimum": 0 }
+        },
+        "useful_macs": { "type": "integer", "minimum": 0 },
+        "capacity_macs": { "type": "integer", "minimum": 0 },
+        "utilization": { "type": ["number", "null"], "minimum": 0, "maximum": 1 }
+      }
+    }
+  },
+  "$defs": {
+    "buffer": {
+      "type": "object",
+      "required": ["dtype", "shape", "kind"],
+      "additionalProperties": false,
+      "properties": {
+        "dtype": { "enum": ["i8", "i32"] },
+        "shape": { "type": "array", "items": { "type": "integer", "minimum": 1 }, "minItems": 1, "maxItems": 2 },
+        "kind": { "enum": ["input", "const", "temp"] },
+        "data": { "type": "array", "description": "const buffers only: nested lists matching shape" }
+      }
+    },
+    "ref": {
+      "type": "object",
+      "required": ["buf", "r", "c"],
+      "additionalProperties": false,
+      "properties": {
+        "buf": { "type": "string" },
+        "r": { "type": "integer", "minimum": 0 },
+        "c": { "type": "integer", "minimum": 0 }
+      }
+    },
+    "command": {
+      "oneOf": [
+        { "type": "object", "required": ["op", "buf"], "additionalProperties": false,
+          "properties": { "op": { "const": "alloc" }, "buf": { "type": "string" } } },
+        { "type": "object", "required": ["op", "id", "a", "b"], "additionalProperties": false,
+          "properties": { "op": { "const": "pass" }, "id": { "type": "integer", "minimum": 0 },
+                          "a": { "$ref": "#/$defs/ref" }, "b": { "$ref": "#/$defs/ref" } } },
+        { "type": "object", "required": ["op", "pass", "dst"], "additionalProperties": false,
+          "properties": { "op": { "const": "acc" }, "pass": { "type": "integer", "minimum": 0 },
+                          "dst": { "$ref": "#/$defs/ref" }, "bias": { "type": "string" }, "relu": { "const": true } } },
+        { "type": "object", "required": ["op", "dst"], "additionalProperties": false,
+          "properties": { "op": { "const": "epilogue" }, "dst": { "$ref": "#/$defs/ref" },
+                          "bias": { "type": "string" }, "relu": { "const": true } } },
+        { "type": "object", "required": ["op", "dst", "a", "b"], "additionalProperties": false,
+          "properties": { "op": { "const": "host_add" }, "dst": { "type": "string" },
+                          "a": { "type": "string" }, "b": { "type": "string" } } },
+        { "type": "object", "required": ["op", "dst", "src"], "additionalProperties": false,
+          "properties": { "op": { "const": "host_relu" }, "dst": { "type": "string" }, "src": { "type": "string" } } }
+      ]
+    }
+  }
+}

pyproject.toml

@@ -10,7 +10,7 @@ requires-python = ">=3.12"
 dependencies = ["numpy>=2.0"]
 
 [project.optional-dependencies]
-dev = ["pytest>=8", "hypothesis>=6.100", "black>=24.10", "ruff>=0.6"]
+dev = ["pytest>=8", "hypothesis>=6.100", "jsonschema>=4", "black>=24.10", "ruff>=0.6"]
 
 [tool.setuptools.packages.find]
 include = ["tinyc*", "runtime*"]

runtime/tpu_runner.py

@@ -10,7 +10,7 @@
 import subprocess
 import tempfile
 from collections.abc import Sequence
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from pathlib import Path
 
 import numpy as np
@@ -42,20 +42,49 @@ def run_tpu_passes(passes: Sequence[tuple[np.ndarray, np.ndarray]]) -> TpuResult
             for a, b in passes
         ]
     }
+    out = _call(payload)
+    return TpuResult(
+        passes=[
+            PassResult(np.asarray(p["c"], dtype=np.int32).reshape(4, 4), p["cycles"])
+            for p in out["passes"]
+        ],
+        total_cycles=out["totalCycles"],
+    )
+
+
+@dataclass
+class ProgramResult:
+    outputs: dict[str, np.ndarray]  # int64, original shapes
+    pass_cycles: list[int]  # measured per pass (RTL clock cycles)
+    total_cycles: int
+    host_ops: dict[str, int]  # counted by the executor, never attributed to the chip
+    passes: list[dict] = field(default_factory=list)
+
+
+def _call(request: dict) -> dict:
     with tempfile.TemporaryDirectory() as tmp:
-        path = os.path.join(tmp, "passes.json")
+        path = os.path.join(tmp, "request.json")
         with open(path, "w") as f:
-            json.dump(payload, f)
+            json.dump(request, f)
         proc = subprocess.run(
-            ["node", str(CLI), path], capture_output=True, text=True, check=False, timeout=120
+            ["node", str(CLI), path], capture_output=True, text=True, check=False, timeout=300
         )
     if proc.returncode != 0:
         raise RuntimeError(f"ttpu-cli failed ({proc.returncode}): {proc.stderr.strip()}")
-    out = json.loads(proc.stdout)
-    return TpuResult(
-        passes=[
-            PassResult(np.asarray(p["c"], dtype=np.int32).reshape(4, 4), p["cycles"])
-            for p in out["passes"]
-        ],
+    return json.loads(proc.stdout)
+
+
+def run_ttpu(program: dict, inputs: dict) -> ProgramResult:
+    """Execute a `.ttpu.json` program on the real tiny-tpu RTL through the one JS executor."""
+    payload = {k: np.asarray(v, dtype=np.int64).tolist() for k, v in inputs.items()}
+    out = _call({"program": program, "inputs": payload})
+    return ProgramResult(
+        outputs={
+            name: np.asarray(o["data"], dtype=np.int64).reshape(o["shape"])
+            for name, o in out["outputs"].items()
+        },
+        pass_cycles=[p["cycles"] for p in out["passes"]],
         total_cycles=out["totalCycles"],
+        host_ops=out["hostOps"],
+        passes=out["passes"],
     )

runtime/ttpu-cli.mjs

@@ -1,10 +1,14 @@
 #!/usr/bin/env node
-// ttpu-cli.mjs - run tiny-tpu passes from a JSON file and print JSON results.
+// ttpu-cli.mjs - run tiny-tpu work from a JSON file and print JSON results.
 //
-//   node runtime/ttpu-cli.mjs <passes.json>
+//   node runtime/ttpu-cli.mjs <request.json>
 //
-//   in : { "passes": [ { "a": [16 ints], "b": [16 ints] }, ... ] }
-//   out: { "passes": [ { "c": [16 ints], "cycles": n }, ... ], "totalCycles": n }
+//   request A (raw passes):  { "passes": [ { "a": [16 ints], "b": [16 ints] }, ... ] }
+//     -> { "passes": [ { "c": [16 ints], "cycles": n }, ... ], "totalCycles": n }
+//
+//   request B (a program):   { "program": <.ttpu.json>, "inputs": { name: nested lists } }
+//     -> { "outputs": { name: {shape, data} }, "passes": [ {id, cycles} ], "totalCycles": n,
+//          "hostOps": { alloc, acc, epilogue, host_add, host_relu } }
 //
 // The tiny-tpu build is taken from $TINY_TPU_DIST (a dir holding tiny_tpu.mjs + tiny_tpu.wasm),
 // defaulting to the pinned submodule's web/public.
@@ -19,17 +23,27 @@ const dist = process.env.TINY_TPU_DIST ?? join(here, "..", "targets", "tiny-tpu"
 
 const [file] = process.argv.slice(2);
 if (!file) {
-  console.error("usage: ttpu-cli.mjs <passes.json>");
+  console.error("usage: ttpu-cli.mjs <request.json>");
   process.exit(2);
 }
 
 const { default: factory } = await import(pathToFileURL(resolve(dist, "tiny_tpu.mjs")).href);
 const tpu = await createTpu(factory);
-const { passes } = JSON.parse(readFileSync(file, "utf8"));
-const results = passes.map(({ a, b }) => {
-  const { c, cycles } = tpu.runPass(a, b);
-  return { c, cycles };
-});
-console.log(
-  JSON.stringify({ passes: results, totalCycles: results.reduce((n, r) => n + r.cycles, 0) }),
-);
+const request = JSON.parse(readFileSync(file, "utf8"));
+
+try {
+  if (request.program) {
+    console.log(JSON.stringify(tpu.executeProgram(request.program, request.inputs ?? {})));
+  } else {
+    const results = request.passes.map(({ a, b }) => {
+      const { c, cycles } = tpu.runPass(a, b);
+      return { c, cycles };
+    });
+    console.log(
+      JSON.stringify({ passes: results, totalCycles: results.reduce((n, r) => n + r.cycles, 0) }),
+    );
+  }
+} catch (err) {
+  console.error(`${err.name}: ${err.message}`);
+  process.exit(1);
+}