Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 27 additions & 4 deletions docs/adapters/grok.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
- [Guard behavior](#guard-behavior)
- [Verify](#verify)
- [Update and uninstall](#update-and-uninstall)
- [Spec-loop runner](#spec-loop-runner)
- [Known limitations](#known-limitations)
- [Developer checks](#developer-checks)
- [Upstream references](#upstream-references)
Expand All @@ -26,8 +27,9 @@

**Capability:** platform:harness

Grok Build is supported for the deterministic Magpie action guard.
This does not add Grok spec-loop, headless-runner, or dedicated sandbox-profile support.
Grok Build is supported for the deterministic Magpie action guard and the
spec-loop headless runner.
This does not add a dedicated Grok sandbox profile.

## Harness contract

Expand Down Expand Up @@ -166,10 +168,31 @@ A hand-edited copy must be shown for review before replacement or removal.
Repository uninstall removes only the Magpie-owned project hook.
It does not uninstall a user's marketplace plugin and does not change Grok trust.

## Spec-loop runner

Grok Build 1.0.50 was live-verified as a headless spec-loop runtime.

The runner uses `--prompt-file` for the assembled prompt, `--cwd` for the
repository root, and `--always-approve` for unattended local edits inside the
external Magpie sandbox.
`SPEC_LOOP_OUTPUT_FORMAT=text` maps to Grok's `plain` output and
`stream-json` maps to `streaming-json`.
`SPEC_LOOP_MODEL` maps to `--model`, while `SPEC_LOOP_EFFORT` maps to
`--reasoning-effort low|medium|high`.

The runner also supplies explicit `--deny` rules for `git push` and `gh`.
On Grok Build 1.0.50 these exact rule shapes were live-verified to deny the
matching shell calls before execution even with `--always-approve`.

A disposable repository probe confirmed that the same profile can create a
local file headlessly while leaving unrelated tracked files unchanged.

## Known limitations

This integration is action-guard only.
It does not add a Grok-specific spec-loop runner or headless execution profile.
Grok spec-loop support relies on the same external OS sandbox required by the
other unattended harness profiles.
The per-invocation deny rules are defense in depth rather than a replacement
for that sandbox.

Malformed or unrelated input handled inside the adapter is deliberately
fail-open. Hook-process exit 2 is fail-closed on the Grok 1.0.50 runtime tested:
Expand Down
2 changes: 1 addition & 1 deletion docs/adapters/registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ extension point = a documented, labelled slot with a tracking issue.
| [`tools/vcs`](../../tools/vcs/) | Git, Mercurial, Fossil | Subversion [\#602](https://github.com/apache/magpie/issues/602), Jujutsu [\#603](https://github.com/apache/magpie/issues/603), Perforce [\#605](https://github.com/apache/magpie/issues/605) |
| Forge / tracker | [`github`](../../tools/github/), [`forgejo`](../../tools/forgejo/) `partial` foundation, [`jira`](../../tools/jira/), [`bitbucket`](../../tools/bitbucket/) `partial-read-only` foundation, [`sourcehut`](../../tools/sourcehut/), [`fossil`](../../tools/fossil/), [`gitlab`](../../tools/gitlab/) `partial-read-only` foundation | Forgejo/Gitea [\#310](https://github.com/apache/magpie/issues/310), Pagure [\#312](https://github.com/apache/magpie/issues/312), deeper Bitbucket/Jira coverage [\#606](https://github.com/apache/magpie/issues/606), GitLab [\#305](https://github.com/apache/magpie/issues/305), Bugzilla [\#302](https://github.com/apache/magpie/issues/302) |
| [`tools/chat`](../../tools/chat/) | [`chat-slack`](../../tools/chat-slack/), [`chat-discord`](../../tools/chat-discord/) | Matrix [\#309](https://github.com/apache/magpie/issues/309), Zulip [\#308](https://github.com/apache/magpie/issues/308) |
| Agent harness | Claude Code, [Codex](codex.md) `experimental` ([#313](https://github.com/apache/magpie/issues/313)), [Gemini CLI](gemini.md) `experimental` ([#314](https://github.com/apache/magpie/issues/314)), [Local LLM (Ollama / llama.cpp / vLLM)](local-llm.md) ([#315](https://github.com/apache/magpie/issues/315)), [Cursor](cursor.md) ([#316](https://github.com/apache/magpie/issues/316)), [Goose](goose.md) `guide only` ([#319](https://github.com/apache/magpie/issues/319)), [Aider](aider.md) `guide only` ([#317](https://github.com/apache/magpie/issues/317)), [GitHub Copilot](copilot.md) `guide only` ([#318](https://github.com/apache/magpie/issues/318)), [Grok](grok.md) `reviewer backend + action guard` ([#1416](https://github.com/apache/magpie/issues/1416)) | Amazon Q [#320](https://github.com/apache/magpie/issues/320)–OpenHands [#322](https://github.com/apache/magpie/issues/322) |
| Agent harness | Claude Code, [Codex](codex.md) `experimental` ([#313](https://github.com/apache/magpie/issues/313)), [Gemini CLI](gemini.md) `experimental` ([#314](https://github.com/apache/magpie/issues/314)), [Local LLM (Ollama / llama.cpp / vLLM)](local-llm.md) ([#315](https://github.com/apache/magpie/issues/315)), [Cursor](cursor.md) ([#316](https://github.com/apache/magpie/issues/316)), [Goose](goose.md) `guide only` ([#319](https://github.com/apache/magpie/issues/319)), [Aider](aider.md) `guide only` ([#317](https://github.com/apache/magpie/issues/317)), [GitHub Copilot](copilot.md) `guide only` ([#318](https://github.com/apache/magpie/issues/318)), [Grok](grok.md) `reviewer backend + action guard + spec-loop` ([#1416](https://github.com/apache/magpie/issues/1416)) | Amazon Q [#320](https://github.com/apache/magpie/issues/320)–OpenHands [#322](https://github.com/apache/magpie/issues/322) |
| Security cross-ref | [`tools/osv`](../../tools/osv/) | — |
| [`tools/typed-decision`](../../tools/typed-decision/) | TypeSafe Jev API, in-tree local provider (Ollama / llama.cpp / vLLM) ([#1431](https://github.com/apache/magpie/issues/1431)) | — |
| [`tools/people`](../../tools/people/) | [`github`](../../tools/github/), [`jira`](../../tools/jira/) (profile lookup) | GitLab, Forgejo, Bitbucket |
Expand Down
4 changes: 2 additions & 2 deletions docs/vendor-neutrality.md
Original file line number Diff line number Diff line change
Expand Up @@ -648,7 +648,7 @@ Organization scope (declared, orthogonal to vendor): ASF = 16, agnostic = 63.
| `skill-reconciler-diff` | framework-dev | any | ✅ agnostic |
| `skill-token-count` | framework-dev, analytics | any | ✅ agnostic |
| `spec-inventory` | framework-dev, analytics | any | ✅ agnostic |
| `spec-loop` | framework-dev | Claude Code, Codex, Cursor, Gemini CLI, Kiro, OpenCode | ✅ portable |
| `spec-loop` | framework-dev | Claude Code, Codex, Cursor, Gemini CLI, Grok, Kiro, OpenCode | ✅ portable |
| `spec-status-index` | framework-dev, analytics | any | ✅ agnostic |
| `spec-validator` | framework-dev | any | ✅ agnostic |
| `symlink-lint` | framework-dev | any | ✅ agnostic |
Expand All @@ -662,7 +662,7 @@ Harness → substrate tools it supports:
- **Copilot** (1): `agent-guard`
- **Cursor** (2): `sandbox-lint`, `spec-loop`
- **Gemini CLI** (3): `agent-guard`, `sandbox-lint`, `spec-loop`
- **Grok** (1): `agent-guard`
- **Grok** (2): `agent-guard`, `spec-loop`
- **Kiro** (3): `agent-guard`, `sandbox-lint`, `spec-loop`
- **OpenCode** (3): `agent-guard`, `sandbox-lint`, `spec-loop`
- **any harness** (29): `adversarial-review`, `agent-isolation`, `container-gateway`, `contributor-metrics`, `dashboard-generator`, `dev`, `egress-gateway`, `maven-artifact-verify`, `permission-audit`, `pilot-report-validator`, `pr-management`, `preflight-audit`, `privacy-llm`, `probe-templates`, `release-config`, `release-verify`, `reproducible-archive`, `security-tracker-stats-dashboard`, `setup-preflight`, `skill-and-tool-validator`, `skill-evals`, `skill-reconciler-diff`, `skill-token-count`, `spec-inventory`, `spec-status-index`, `spec-validator`, `symlink-lint`, `vendor-neutrality-score`, `vetted-ops`
Expand Down
29 changes: 18 additions & 11 deletions tools/spec-loop/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

**Capability:** substrate:framework-dev

**Harness:** Claude Code, Codex, Cursor, Gemini CLI, OpenCode, Kiro
**Harness:** Claude Code, Codex, Cursor, Gemini CLI, OpenCode, Kiro, Grok

A spec-driven build loop for this framework, in the general
[Ralph](https://ghuntley.com/ralph/) style (run a fresh agent context
Expand All @@ -27,11 +27,13 @@ picks its run convention:
- `gemini` — `gemini --approval-mode default --prompt "<prompt>"`.
- `opencode` — `opencode run --auto --model … "<prompt>"`, prompt as a
positional argument.
- `grok` — `grok --always-approve --cwd … --prompt-file …`, with explicit
deny rules for remote mutation.

`SPEC_LOOP_HARNESS` defaults from the agent basename, so
`SPEC_LOOP_AGENT=codex`, `SPEC_LOOP_AGENT=cursor-agent`,
`SPEC_LOOP_AGENT=gemini`, or `SPEC_LOOP_AGENT=opencode` is usually all that
is needed:
`SPEC_LOOP_AGENT=gemini`, `SPEC_LOOP_AGENT=opencode`, or
`SPEC_LOOP_AGENT=grok` is usually all that is needed:

```bash
SPEC_LOOP_AGENT=codex tools/spec-loop/loop.sh build 5
Expand All @@ -46,15 +48,20 @@ SPEC_LOOP_AGENT=opencode SPEC_LOOP_MODEL=anthropic/claude-sonnet-4-5 \
tools/spec-loop/loop.sh build 5
```

```bash
SPEC_LOOP_AGENT=grok tools/spec-loop/loop.sh build 5
```

**Gemini CLI:** the launcher uses `--approval-mode default` instead of `--yolo`.
With the shipped profile, calls requiring confirmation are refused headlessly, so unattended build iterations that need edits cannot complete.
See the [Gemini runner notes](../../docs/adapters/gemini.md#spec-loop-runner) for supported use and upgrade impact.

All conventions run the agent non-interactively.
The profiles other than Gemini request automatic approval; their safety rails (never push, never open a PR) come from
the OS sandbox, missing push/write credentials, repo hooks, and the loop's own
guards. Claude also gets per-invocation `--disallowedTools` hard-deny flags;
the other harnesses rely on their own policy/config plus the external sandbox.
guards. Claude gets per-invocation `--disallowedTools` hard-deny flags.
Grok gets equivalent per-invocation `--deny` rules for `git push` and `gh`.
The remaining harnesses rely on their own policy/config plus the external sandbox.
See the SECURITY notes in [`loop.sh`](loop.sh).

## Prerequisites
Expand All @@ -64,7 +71,7 @@ See the SECURITY notes in [`loop.sh`](loop.sh).
- **CLIs:** `git` (required — must run inside a git checkout), `gh`
(for open-PR duplicate-work checks), and one supported headless agent
CLI (`SPEC_LOOP_AGENT`, default `claude`; also supports `codex`,
`cursor` / `cursor-agent`, `gemini`, and `opencode`).
`cursor` / `cursor-agent`, `gemini`, `opencode`, `kiro-cli`, and `grok`).
- **Credentials / auth:** `gh` must be authenticated for the PR checks;
the loop is designed to run with **no** push/write credentials in the
environment (it hard-denies `git push` and `gh` writes).
Expand Down Expand Up @@ -131,13 +138,13 @@ for Claude, as defence in depth, the loop also hard-denies `git push` and
- `SPEC_LOOP_BASE` — branch to fork work items from. Defaults to `main`;
set it explicitly to build on top of a different branch.
- `SPEC_LOOP_AGENT` — supported headless agent CLI or wrapper to run
(default `claude`; `codex`, `cursor`, `gemini`, and `opencode` are
first-class harnesses).
(default `claude`; `codex`, `cursor`, `gemini`, `opencode`, `kiro-cli`,
and `grok` are first-class harnesses).
- `SPEC_LOOP_HARNESS` — override the invocation convention when the CLI
name does not imply it (`claude`, `codex`, `cursor`, `gemini`, or
`opencode`).
name does not imply it (`claude`, `codex`, `cursor`, `gemini`,
`opencode`, `kiro`, or `grok`).
- `SPEC_LOOP_MODEL` — model passed to the agent CLI. Defaults to `sonnet`
for Claude; Codex/Cursor/Gemini/OpenCode use their configured default
for Claude; Codex/Cursor/Gemini/OpenCode/Grok use their configured default
unless this is set.
- `SPEC_LOOP_EFFORT` — reasoning-effort band: `low`, `medium`, or `high`.
Unset means no effort flag is passed. Mapped per harness (Claude/Kiro
Expand Down
22 changes: 22 additions & 0 deletions tools/spec-loop/lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,28 @@ spec_loop_launch_agent() {
"$agent" chat --no-interactive \
${kiro_effort_args[@]+"${kiro_effort_args[@]}"} \
"$(cat "$prompt_file")" &
elif [ "$harness" = "grok" ]; then
# Grok Build headless mode accepts the assembled prompt directly from
# a file. `--always-approve` permits unattended local work while the
# explicit deny rules keep remote mutation blocked in depth alongside
# the external sandbox and agent guard.
local grok_output_format=plain
local grok_effort_args=()
[ "$output_format" = "stream-json" ] && grok_output_format=streaming-json
case "$effort" in
low) grok_effort_args=(--reasoning-effort low) ;;
medium) grok_effort_args=(--reasoning-effort medium) ;;
high) grok_effort_args=(--reasoning-effort high) ;;
esac
"$agent" \
--always-approve \
--cwd "$root" \
--deny "Bash(git push:*)" \
--deny "Bash(gh:*)" \
--output-format "$grok_output_format" \
${model_args[@]+"${model_args[@]}"} \
${grok_effort_args[@]+"${grok_effort_args[@]}"} \
--prompt-file "$prompt_file" &
else
local verbose_args=()
local claude_effort_args=()
Expand Down
13 changes: 8 additions & 5 deletions tools/spec-loop/loop.sh
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,12 @@
# chosen by SPEC_LOOP_HARNESS below.
# SPEC_LOOP_HARNESS which harness's headless-run convention to use when
# invoking SPEC_LOOP_AGENT: `claude` (default), `codex`,
# `cursor`, `gemini`, or `opencode`. Defaults from the
# agent basename, so setting SPEC_LOOP_AGENT=codex,
# SPEC_LOOP_AGENT=cursor-agent, SPEC_LOOP_AGENT=gemini, or
# SPEC_LOOP_AGENT=opencode is usually enough.
# `cursor`, `gemini`, `opencode`, `kiro`, or `grok`.
# Defaults from the agent basename, so setting
# SPEC_LOOP_AGENT=codex, SPEC_LOOP_AGENT=cursor-agent,
# SPEC_LOOP_AGENT=gemini, SPEC_LOOP_AGENT=opencode,
# SPEC_LOOP_AGENT=kiro-cli, or SPEC_LOOP_AGENT=grok is
# usually enough.
# SPEC_LOOP_MODEL model passed to the agent CLI. Defaults to `sonnet` for
# Claude; all other harnesses use their configured default
# unless this is set.
Expand Down Expand Up @@ -112,6 +114,7 @@ case "${SPEC_LOOP_HARNESS:-$(basename "$AGENT")}" in
*gemini*) HARNESS=gemini ;;
*opencode*) HARNESS=opencode ;;
*kiro*) HARNESS=kiro ;;
*grok*) HARNESS=grok ;;
*) HARNESS=claude ;;
esac
if [ -n "${SPEC_LOOP_MODEL:-}" ]; then
Expand Down Expand Up @@ -192,7 +195,7 @@ fi

if ! command -v "$AGENT" >/dev/null 2>&1; then
echo "Error: agent CLI '$AGENT' not found on PATH." >&2
echo "Set SPEC_LOOP_AGENT to a headless agent CLI (claude / codex / cursor / gemini / opencode / a wrapper)" >&2
echo "Set SPEC_LOOP_AGENT to a headless agent CLI (claude / codex / cursor / gemini / opencode / kiro / grok / a wrapper)" >&2
echo "and SPEC_LOOP_HARNESS to its run convention if the name doesn't imply it." >&2
exit 1
fi
Expand Down
14 changes: 8 additions & 6 deletions tools/spec-loop/specs/spec-loop-runner.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ acceptance:
- Open PRs and local work-item branches are passed into plan/build prompts
as in-flight work so the loop does not duplicate already-built items.
- The headless-agent harness contract is explicit for Claude Code, Codex,
Cursor, Gemini CLI, OpenCode, and Kiro.
Cursor, Gemini CLI, OpenCode, Kiro, and Grok Build.
- The update marker `tools/spec-loop/.last-sync` is owned by the runner,
not by the prompt.
---
Expand Down Expand Up @@ -119,6 +119,7 @@ sandbox, and stop without pushing or opening a PR.
| Gemini CLI | `--prompt "<prompt>"` | launched from repo root | `--approval-mode default` | `--model` | `--output-format` | tool-sandboxing and Gemini policy; headless actions needing confirmation are refused |
| OpenCode | positional prompt to `opencode run` | launched from repo root | `--auto` | `--model` | `--format json` for stream JSON | external sandbox and OpenCode policy |
| Kiro | positional prompt to `kiro-cli chat` | launched from repo root | `--no-interactive` | agent config (`.kiro/agents`) | plain text only | external sandbox and the agent-guard `--kiro` hook |
| Grok Build | `--prompt-file <assembled-prompt>` | `--cwd "$ROOT"` | `--always-approve` | `--model` | `plain`; `streaming-json` for stream JSON | `--deny` blocks `git push` and `gh`; external sandbox and agent guard remain active |

`SPEC_LOOP_EFFORT` is the harness-neutral reasoning-effort band
(`low` / `medium` / `high`). Unset means no effort flag is added. Invalid
Expand All @@ -140,6 +141,7 @@ harness CLI, re-check `--help` and update this table and
| Kiro | `--effort` | [Kiro CLI effort docs](https://kiro.dev/docs/cli/chat/effort/) (`low\|medium\|high\|xhigh\|max`) | `low` | `medium` | `max` |
| Cursor | (none — omitted silently) | `cursor agent --help` (no effort/thinking-level flag) | — | — | — |
| Gemini CLI | (none — omitted silently) | `gemini --help` (no effort/thinking-level flag) | — | — | — |
| Grok Build | `--reasoning-effort` | `grok 1.0.50 --help` (`--effort` alias also accepted) | `low` | `medium` | `high` |

`SPEC_LOOP_AGENT` chooses the CLI. `SPEC_LOOP_HARNESS` chooses the
invocation convention and defaults from the agent basename. Adding a new
Expand Down Expand Up @@ -168,8 +170,8 @@ in the same change.
branch context.
4. Update prompts receive incremental-scope guidance from `.last-sync`
when present, but the runner remains the only writer of `.last-sync`.
5. Claude Code, Codex, Cursor, Gemini CLI, OpenCode, and Kiro are documented in
the headless harness matrix and implemented in `loop.sh`.
5. Claude Code, Codex, Cursor, Gemini CLI, OpenCode, Kiro, and Grok Build are
documented in the headless harness matrix and implemented by the runner.
6. Operator docs and prompts agree on branch naming: build uses bare
`<slug>` branches; update uses `sync-specs-<timestamp>` branches.
7. Security docs describe unattended agent flags as harness-specific
Expand All @@ -186,9 +188,9 @@ uv run --project tools/spec-validator --group dev spec-validate

## Known gaps

- The non-Claude harnesses rely on external policy/config plus the OS
sandbox for push/PR denial; only Claude has a per-invocation hard-deny
flag in the current runner.
- Cursor, Gemini CLI, OpenCode, and Kiro rely on external policy/config plus
the OS sandbox for push/PR denial. Claude Code and Grok Build additionally
receive per-invocation hard-deny rules from the runner.
- The deterministic scope helper this bullet used to call missing has
shipped: `loop.sh` pipes the incremental diff through
`tools/spec-inventory`'s `spec-scope`.
Expand Down
24 changes: 24 additions & 0 deletions tools/spec-loop/tests/test_runner_fixtures.sh
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,30 @@ test_harness_command_construction() {
assert_contains "$TMPDIR_TEST/gemini.log" "<--output-format> <stream-json>"
assert_contains "$TMPDIR_TEST/gemini.log" "<--model> <gemini-test>"
assert_contains "$TMPDIR_TEST/gemini.log" "<--prompt> <PROMPT BODY>"

# Grok Build 1.0.50: prompt-file headless mode, explicit cwd,
# unattended approval, remote-write denies, output mapping, model,
# and framework effort mapping.
make_fake_agent "$TMPDIR_TEST/grok" "$TMPDIR_TEST/grok.log"
spec_loop_launch_agent grok "$TMPDIR_TEST/grok" /repo "$TMPDIR_TEST/prompt.md" grok-test stream-json high
wait "$SPEC_LOOP_AGENT_PID"
assert_contains "$TMPDIR_TEST/grok.log" "<--always-approve>"
assert_contains "$TMPDIR_TEST/grok.log" "<--cwd> </repo>"
assert_contains "$TMPDIR_TEST/grok.log" "<--deny> <Bash(git push:*)>"
assert_contains "$TMPDIR_TEST/grok.log" "<--deny> <Bash(gh:*)>"
assert_contains "$TMPDIR_TEST/grok.log" "<--output-format> <streaming-json>"
assert_contains "$TMPDIR_TEST/grok.log" "<--model> <grok-test>"
assert_contains "$TMPDIR_TEST/grok.log" "<--reasoning-effort> <high>"
assert_contains "$TMPDIR_TEST/grok.log" "<--prompt-file> <$TMPDIR_TEST/prompt.md>"
assert_not_contains "$TMPDIR_TEST/grok.log" "stdin:PROMPT BODY"

# Default text output maps to Grok's plain headless format and unset
# effort must not add a reasoning-effort flag.
spec_loop_launch_agent grok "$TMPDIR_TEST/grok" /repo "$TMPDIR_TEST/prompt.md" "" text
wait "$SPEC_LOOP_AGENT_PID"
assert_contains "$TMPDIR_TEST/grok.log" "<--output-format> <plain>"
assert_not_contains "$TMPDIR_TEST/grok.log" "<--reasoning-effort>"
assert_not_contains "$TMPDIR_TEST/grok.log" "<--model>"
}

# loop.sh validates SPEC_LOOP_EFFORT at startup (reject anything other than
Expand Down