refactor(runner): replace warpforth-runner with a Python device worker

.claude/skills/verify-warpforth/SKILL.md

@@ -1,14 +1,14 @@
 ---
 name: verify-warpforth
-description: Drive the real WarpForth toolchain to prove a change works. Compiles Forth with build/bin/warpforthc (and the warpforth-translate | warpforth-opt stages) locally, and executes kernels on a rented Vast.ai GPU via warpforth-runner. Use after changing the parser, dialect, conversions, pipeline, or CLI flags, when lit tests alone don't show the kernel actually computes the right numbers.
+description: Drive the real WarpForth toolchain to prove a change works. Compiles Forth with build/bin/warpforthc (and the warpforth-translate | warpforth-opt stages) locally, and executes kernels on a rented Vast.ai GPU through the warpforth_gpu device worker. Use after changing the parser, dialect, conversions, pipeline, or CLI flags, when lit tests alone don't show the kernel actually computes the right numbers.
 ---
 
 # Verify WarpForth
 
-WarpForth is a CLI compiler: the user writes a `.forth` kernel, runs `warpforthc` to get PTX, and runs that PTX on an NVIDIA GPU with `warpforth-runner`. There is no server or UI. This machine (macOS) has no CUDA GPU, no `nvcc`, no `ptxas`, so:
+WarpForth is a CLI compiler: the user writes a `.forth` kernel, runs `warpforthc` to get PTX, and runs that PTX on an NVIDIA GPU through `warpforth_gpu.client.Device`, which talks to a stdlib-only Python worker (`warpforth_gpu/worker.py`) over newline-delimited JSON. There is no server or UI. This machine (macOS) has no CUDA GPU and no `ptxas`, so:
 
 - **Local surface** (free, always available): `warpforthc`, `warpforth-translate`, `warpforth-opt`. Proves parsing, diagnostics, lowering, and that PTX is emitted.
-- **GPU surface** (costs money, needs `VASTAI_API_KEY`): rents one Vast.ai GPU (cap $0.50/hr, `nvidia/cuda:12.4.0-devel-ubuntu22.04`, host driver must support CUDA 12.4), builds `warpforth-runner` there with `nvcc`, executes PTX, reads back a buffer. This is the only proof that a kernel computes the right values. **Ask the user before each GPU run**: it spends their money.
+- **GPU surface** (costs money, needs `VASTAI_API_KEY`): rents one Vast.ai GPU (cap $0.50/hr, `nvidia/cuda:12.4.0-devel-ubuntu22.04`, host driver must support CUDA 12.4), starts the device worker there over SSH (its source is passed with `python3 -c`, nothing is copied), executes PTX, reads back a buffer. This is the only proof that a kernel computes the right values. **Ask the user before each GPU run**: it spends their money.
 
 Other surfaces, not mapped yet: the GPT-2 demo (`demo/`, `docs/demo.md`) needs a PyTorch Vast.ai image and a manual scp.
 
@@ -24,13 +24,13 @@ cmake --build build
 
 Ready when it exits 0 and `$S/doctor.sh` prints no `FAIL`. Each compile is its own short-lived process, so any number of agents can compile side by side. Teardown: nothing.
 
-GPU launch happens inside `$S/gpu_run.py` (below): it searches offers, creates an instance labeled `warpforth-test-<timestamp>-<uuid>`, waits for sshd, compiles the runner with `nvcc`. Usually 2-7 minutes (the first proof run took 6: 5 to boot, 1 for nvcc and the jobs). Concurrent GPU runs are safe: each owns its own label and destroys only that.
+GPU launch happens inside `$S/gpu_run.py` (below): it searches offers, creates an instance labeled `warpforth-test-<timestamp>-<uuid>`, waits for sshd, installs `python3` if the image lacks it. Usually 2-7 minutes, almost all of it instance boot. Concurrent GPU runs are safe: each owns its own label and destroys only that.
 
 ## Doctor
 
 ```bash
 $S/doctor.sh            # local: binaries exist, `ninja -n` shows no pending build steps, smoke compile emits `.visible .entry main`
-$S/doctor.sh --remote   # plus: uv/ssh/scp on PATH, Vast.ai key accepted, lists any running warpforth-test-* instances
+$S/doctor.sh --remote   # plus: uv/ssh on PATH, Vast.ai key accepted, lists any running warpforth-test-* instances
 ```
 
 Read-only (the smoke compile writes only to a mktemp dir). Exit 0 means worth driving. A stale build is the most common false result: a verification run against old binaries proves nothing, so rebuild when doctor says stale. `--remote` warnings about existing instances are informational; they may be another agent's live run.
@@ -51,7 +51,7 @@ Runs exactly `build/bin/warpforthc <file> --arch <arch> -o kernel.ptx` (the user
 uv run python $S/gpu_run.py <spec.json>
 ```
 
-One spec = one rented GPU for all its jobs. Spec format is in the script's docstring: per job a `.forth` source, `params` by header name, `grid`, `block`, `output_param` (index into the `\! param` list, must be an array), `output_count`, and optional `expect` (a prefix of the output; floats compared with `isclose`). It reuses `gpu_test/conftest.py`'s `VastSession` and `KernelRunner`, the same path as `uv run pytest -m gpu`. Exit 0 only if every `expect` matched and instance destruction was verified. A ready-made spec covering int arithmetic, multi-thread matmul (the README quickstart), and f64 math is at `.claude/skills/verify-warpforth/fixtures/smoke/spec.json`. For a change, write a new spec next to the kernels you want to prove, in scratch space, not inside the skill.
+One spec = one rented GPU for all its jobs. Spec format is in the script's docstring: per job a `.forth` source, `params` by header name, `grid`, `block`, `output_param` (index into the `\! param` list, must be an array), `output_count`, and optional `expect` (a prefix of the output; floats compared with `isclose`). It reuses `warpforth_gpu.vast.VastSession` and `gpu_test/conftest.py`'s `KernelRunner`, the same path as `uv run pytest -m gpu`. The worker protocol itself is covered locally, without a GPU, by `uv run pytest gpu_test/test_worker.py`. Exit 0 only if every `expect` matched and instance destruction was verified. A ready-made spec covering int arithmetic, multi-thread matmul (the README quickstart), and f64 math is at `.claude/skills/verify-warpforth/fixtures/smoke/spec.json`. For a change, write a new spec next to the kernels you want to prove, in scratch space, not inside the skill.
 
 The full pytest GPU suite (`uv run pytest -v -m gpu`, ~100+ kernels) is the regression net; `gpu_run.py` is for proving the specific kernel your change affects with captured evidence.
 
@@ -64,11 +64,11 @@ Everything lands in `build/verify-evidence/<run-id>/` (`build/` is gitignored; n
 
 Proof standards:
 
-- Drive `warpforthc` / `warpforth-runner` as a user would. Do not call internal C++ APIs or hand-edit MLIR to make a case pass.
+- Drive `warpforthc` and the `warpforth_gpu` client as a user would. Do not call internal C++ APIs or hand-edit MLIR to make a case pass.
 - A compile proof shows the source, the command's exit code, and the PTX (or the diagnostic for an error case, with its `file:line:col`). "It compiled" is not proof of correctness. Correct values need a GPU run.
 - A GPU proof shows the kernel source, the PTX that ran, the inputs (the spec), and the read-back buffer matching an expectation you computed independently, not one copied from the output.
 - Quote the `evidence:` path in your report.
-- `--output-param` reads back one buffer. To prove a side effect in a second buffer, add a second job that outputs it.
+- `output_param` reads back one buffer. To prove a side effect in a second buffer, add a second job that outputs it.
 
 ## Cleanup
 

.claude/skills/verify-warpforth/features/gpu-execution.md

@@ -1,6 +1,6 @@
 # GPU execution
 
-A user runs compiled PTX on an NVIDIA GPU with `warpforth-runner`, passing array and scalar parameters, a grid and block shape, and reads back one array parameter as comma-separated values. This is the only journey that proves a kernel computes the right numbers.
+A user runs compiled PTX on an NVIDIA GPU through `warpforth_gpu.client.Device`, uploading array buffers, passing scalar values, choosing a grid and block shape, and reading buffers back as numpy arrays. This is the only journey that proves a kernel computes the right numbers.
 
 ## Sub-features
 
@@ -12,7 +12,7 @@ A user runs compiled PTX on an NVIDIA GPU with `warpforth-runner`, passing array
 
 ## How to get to it (user POV)
 
-- Compile locally with `warpforthc`, then on a GPU host: `warpforth-runner kernel.ptx --kernel main --param 'i64[]:1,2,3' --param i64:42 --grid 6,1,1 --block 1,1,1 --output-param 0 --output-count 3`.
+- Compile locally with `warpforthc`, then on a GPU host, from the repo root: `Device.spawn(worker_command())`, `device.load(ptx).kernel("main").launch(grid=..., block=..., args=[device.upload(array), 42])`, `buffer.read(np.int64)`. For a remote host, prefix the ssh argv: `Device.spawn([*ssh_argv, shlex.join(worker_command())])`.
 - `uv run pytest -v -m gpu` (regression suite, same harness).
 
 ## Driving it with gpu_run.py
@@ -33,6 +33,7 @@ Preconditions:
 
 - Rental can fail with `No suitable GPU offers found` under the $0.50/hr cap; retry later or report it, don't raise the cap without the user.
 - `output_param` must index an array param; scalars cannot be read back.
+- A CUDA error during a launch (for example an illegal address) is fatal to the worker; `KernelRunner` starts a fresh one for the next job, and that job's `result.json` shows the `DeviceError`.
 - `params` values are padded with zeros to the declared array size; `expect` is a prefix match, so trailing zeros need not be listed.
 - Higher `arch` in the spec (e.g. `sm_90`) narrows offers to newer GPUs and raises cost and wait.
-- Instance boot plus `nvcc` takes minutes; run in the background and don't kill it mid-run. If you do, run `cleanup.py` on its evidence dir.
+- Instance boot takes minutes; run in the background and don't kill it mid-run. If you do, run `cleanup.py` on its evidence dir.

.claude/skills/verify-warpforth/scripts/cleanup.py

@@ -26,7 +26,7 @@ def owned(sdk: VastAI, label: str) -> list[int]:
     instances = sdk.show_instances()
     if not isinstance(instances, list):
         msg = "Vast.ai returned an invalid instance listing"
-        raise RuntimeError(msg)
+        raise TypeError(msg)
     return [int(i["id"]) for i in instances if isinstance(i, dict) and i.get("label") == label]
 
 

.claude/skills/verify-warpforth/scripts/doctor.sh

@@ -29,7 +29,7 @@ rm -rf "$smoke"
 
 if [ "${1:-}" = "--remote" ]; then
   command -v uv >/dev/null && ok "uv on PATH" || bad "uv missing"
-  command -v ssh >/dev/null && command -v scp >/dev/null && ok "ssh/scp on PATH" || bad "ssh/scp missing"
+  command -v ssh >/dev/null && ok "ssh on PATH" || bad "ssh missing"
   uv run --quiet python - <<'PY' || fail=1
 import sys
 sys.path.insert(0, ".claude/skills/verify-warpforth/scripts")

.claude/skills/verify-warpforth/scripts/gpu_run.py

@@ -33,13 +33,15 @@
 import os
 import sys
 import traceback
+from contextlib import closing
 from datetime import UTC, datetime
 from pathlib import Path
 
 REPO = Path(__file__).resolve().parents[4]
 sys.path.insert(0, str(REPO))
 
-from gpu_test.conftest import Compiler, KernelRunner, VastSession  # noqa: E402
+from gpu_test.conftest import Compiler, KernelRunner, open_vast_device  # noqa: E402
+from warpforth_gpu.vast import VastSession  # noqa: E402
 
 
 def load_api_key() -> str | None:
@@ -99,9 +101,9 @@ def main() -> int:
     (out / "owned_label.txt").write_text(session.instance_label + "\n")
     print(f"evidence: {out}", flush=True)
     try:
-        with session:
+        runner = KernelRunner(lambda: open_vast_device(session), compiler)
+        with session, closing(runner):
             summary["instance_id"] = session.instance_id
-            runner = KernelRunner(session, compiler)
             for job in spec["jobs"]:
                 job_dir = out / job["name"]
                 job_dir.mkdir()

.github/workflows/ci.yml

@@ -86,7 +86,7 @@ jobs:
         run: cmake --build build --target check-format
 
       - name: Ruff check
-        run: uv run ruff check gpu_test/ && uv run ruff format --check gpu_test/
+        run: uv run ruff check gpu_test/ warpforth_gpu/ && uv run ruff format --check gpu_test/ warpforth_gpu/
 
       - name: Test GPU infrastructure
         run: uv run pytest -q -m "not gpu"

AGENTS.md

@@ -38,7 +38,11 @@ CMake.
 **Tools:**
 - `tools/warpforth-translate/warpforth-translate.cpp` - Translation tool entry point
 - `tools/warpforth-opt/warpforth-opt.cpp` - Optimization tool entry point
-- `tools/warpforth-runner/warpforth-runner.cpp` - PTX execution tool for GPU kernels
+
+**GPU execution:**
+- `warpforth_gpu/worker.py` - Stdlib-only device worker; drives the CUDA driver via ctypes and speaks newline-delimited JSON on stdin/stdout
+- `warpforth_gpu/client.py` - `Device`, `Module`, `Kernel`, `Buffer` client for the worker
+- `warpforth_gpu/vast.py` - `VastSession`, rents a Vast.ai GPU and exposes `ssh_argv()`
 
 ## Tools Usage
 
@@ -54,9 +58,21 @@ CMake.
 ./build/bin/warpforth-translate --forth-to-mlir test/example.forth | \
   ./build/bin/warpforth-opt --warpforth-pipeline | \
   ./build/bin/warpforth-translate --mlir-to-ptx > kernel.ptx
+```
+
+```python
+# Execute PTX on a GPU (locally; prepend ssh argv and shlex.join the command for a remote host)
+from pathlib import Path
+
+import numpy as np
+
+from warpforth_gpu.client import Device, worker_command
 
-# Execute PTX on GPU
-./warpforth-runner kernel.ptx --param i64[]:1,2,3 --param i64:42 --output-param 0 --output-count 3
+with Device.spawn(worker_command()) as device:
+    kernel = device.load(Path("kernel.ptx").read_text()).kernel("main")
+    data = device.upload(np.array([1, 2, 3], dtype=np.int64))
+    kernel.launch(grid=(1, 1, 1), block=(3, 1, 1), args=[data, 42])
+    print(data.read(np.int64))
 ```
 
 ## Adding New Operations
@@ -79,15 +95,15 @@ Add corresponding conversion pattern in `lib/Conversion/ForthToMemRef/ForthToMem
 
 ## GPU Tests
 
-End-to-end GPU execution tests live in `gpu_test/`. They compile Forth kernels locally, rent a GPU on Vast.ai, and verify output.
+End-to-end GPU execution tests live in `gpu_test/`. They compile Forth kernels locally, rent a GPU on Vast.ai, run them through a device worker started over SSH, and verify output. `gpu_test/test_worker.py` covers the worker protocol locally with a fake driver.
 
 ```bash
 # Run GPU tests
 VASTAI_API_KEY=xxx uv run pytest -v -m gpu
 
 # Lint and format Python code
-uv run ruff check gpu_test/
-uv run ruff format gpu_test/
+uv run ruff check gpu_test/ warpforth_gpu/
+uv run ruff format gpu_test/ warpforth_gpu/
 ```
 
 ## Architecture Notes

README.md

@@ -9,7 +9,7 @@ See the [documentation](https://tetsuo-cpp.github.io/warpforth/).
 - LLVM/MLIR
 - CMake
 - C++17 compiler
-- CUDA toolkit (for GPU execution)
+- An NVIDIA driver and Python 3 on the GPU host (for GPU execution)
 - [uv](https://github.com/astral-sh/uv) (for Python test tooling)
 
 ## Building
@@ -56,25 +56,38 @@ Compile to PTX:
 ./build/bin/warpforthc matmul.forth -o matmul.ptx --arch sm_80
 ```
 
-Test on a GPU (A is 2x4 row-major, B is 4x3 row-major, C is 2x3 output):
+Run it on a machine with an NVIDIA GPU, with the repository root on the Python
+path (for example `PYTHONPATH=. uv run python run_matmul.py`). A is 2x4
+row-major, B is 4x3 row-major, C is 2x3 output:
 
-```bash
-./build/bin/warpforth-runner matmul.ptx \
-  --param 'i64[]:1,2,3,4,5,6,7,8' \
-  --param 'i64[]:1,2,3,4,5,6,7,8,9,10,11,12' \
-  --param 'i64[]:0,0,0,0,0,0' \
-  --grid 6,1,1 --block 1,1,1 \
-  --output-param 2 --output-count 6
+```python
+from pathlib import Path
+
+import numpy as np
+
+from warpforth_gpu.client import Device, worker_command
+
+with Device.spawn(worker_command()) as device:
+    kernel = device.load(Path("matmul.ptx").read_text()).kernel("main")
+    a = device.upload(np.arange(1, 9, dtype=np.int64))
+    b = device.upload(np.arange(1, 13, dtype=np.int64))
+    c = device.alloc(6 * 8)
+    kernel.launch(grid=(6, 1, 1), block=(1, 1, 1), args=[a, b, c])
+    print(c.read(np.int64))  # [ 70  80  90 158 184 210]
 ```
 
+`worker_command()` runs the device worker from its inlined source, so the
+same call works over SSH on a remote GPU host:
+`Device.spawn(["ssh", host, shlex.join(worker_command())])`.
+
 ## Toolchain
 
 | Tool | Description |
 |------|-------------|
 | `warpforthc` | Compiles Forth source to PTX |
 | `warpforth-translate` | Translates from Forth source to MLIR and MLIR to PTX assembly |
 | `warpforth-opt` | Runs individual MLIR passes or entire pipeline |
-| `warpforth-runner` | Executes PTX kernels on a GPU for testing |
+| `warpforth_gpu` | Python package that loads PTX and runs kernels on an NVIDIA GPU, locally or over SSH |
 
 These tools can be composed for debugging or inspecting intermediate stages: