Repository navigation
feat(spec-loop): add Grok headless harness profile - #1593
Conversation
|
@potiuk You can post this as the maintainer-facing PR comment: |
potiuk
left a comment
There was a problem hiding this comment.
LGTM — Grok slots into the existing harness switch with a --prompt-file / --cwd / --always-approve profile, and its deny rules are exactly Claude's (Bash(git push:*), Bash(gh:*)), live-verified on 1.0.50 to block under --always-approve. The fixture test pins every flag plus the text-default path without effort or model (the fake agent truncates its log per run, so the "absent" assertions are real), and the README, spec matrix, known gaps, registry and vendor-neutrality index all agree. Good call keeping the docs explicit that the denies are defence in depth on top of the OS sandbox. CI is green.
This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.More on how Apache Magpie handles maintainer review:
Contributing guide.
Summary
tools/spec-loop.--prompt-file,--cwd,--always-approve, output/model/effort flags, and explicit deny rules for remote mutation.Type of change
.claude/skills/<name>/) — eval fixtures updated belowtools/<system>/*.md)tools/*/withpyproject.toml)docs/,README.md,CONTRIBUTING.md)projects/_template/)prek, workflows, validators)Test plan
prek run --all-filespassesuv run pytest/ruff check/mypypasses(
PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)(a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)
bash tools/spec-loop/tests/test_runner_fixtures.shbash -n tools/spec-loop/loop.shbash -n tools/spec-loop/lib.shbash -n tools/spec-loop/tests/test_runner_fixtures.shBash(git push:*)andBash(gh:*)deny rules block matching commands even with--always-approveRFC-AI-0004 compliance
git pushandghfrom the Grok invocation; the spec-loop remains local-only and does not autonomously open or push PRsLinked issues
Refs #1416
Notes for reviewers (optional)
The Grok profile was validated against Grok Build 1.0.50 rather than inferred from another harness.
The runner maps:
--prompt-file--cwd--always-approveSPEC_LOOP_OUTPUT_FORMAT=text->plainSPEC_LOOP_OUTPUT_FORMAT=stream-json->streaming-jsonSPEC_LOOP_MODEL->--modelSPEC_LOOP_EFFORT=low|medium|high->--reasoning-effort low|medium|highThe exact deny rules used in the runner were also exercised against the installed CLI:
Bash(git push:*)Bash(gh:*)Both were denied before execution under
--always-approve.A disposable-repository probe was used instead of running a destructive full spec-loop build against the Magpie checkout. The profile successfully made the requested local edit and did not modify unrelated tracked files.