diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9fa0df28..b8fc6ae5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -112,6 +112,8 @@ jobs: run: sh scripts/test-zb.sh - name: scripts/check-kickoff run: python3 scripts/test-check-kickoff.py + - name: scripts/agent-prepare, agent-finish + run: bash scripts/test-agent-work.sh test: name: Tests (${{ matrix.name }}) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f847b0bd..d553aac6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -8,6 +8,11 @@ once, and these rules are what keep their work from colliding. would need adds it to this file in a PR. Agent memory is for one person's context: their preferences, findings still in flight. +Agent adapters use the single worker contract in [`docs/AGENT_WORK.md`](docs/AGENT_WORK.md). +`scripts/agent-prepare ` creates the isolated checkout; `scripts/agent-finish ` +verifies and publishes a draft with evidence. Keep worker rules there rather than copying them +into kickoff prompts, wrappers or runners. + ## One plan step, one branch, one PR - **A plan is an issue** with a checklist of steps, or a plan doc merged on its own and linked diff --git a/docs/AGENT_WORK.md b/docs/AGENT_WORK.md index d12e49b5..493c9f92 100644 --- a/docs/AGENT_WORK.md +++ b/docs/AGENT_WORK.md @@ -34,14 +34,14 @@ Builds on: Branch: / Files/area: Gates: scripts/check-fmt.sh; scripts/zb build, scripts/zb build test, scripts/zb build test-integration, scripts/zb build check-web, scripts/zb build test-sdk-version (read "N/N steps succeeded") -Rules: follow AGENTS.md, CONTRIBUTING.md, docs/STYLE.md and the AGENTS.md of each directory touched. One PR, <=800 lines outside tests/docs. PRs say "Part of fizzyedit/work#N". If the SDK fingerprint moves, record it and leave sdk_version alone. +Rules: docs/AGENT_WORK.md (the shared worker contract). PRs say "Part of fizzyedit/work#N". Done: draft PR on first push; gates pass; PR body says what was run and how verified (test, tape, measurement). Notes: ``` Before assigning a worker, run `scripts/check-kickoff ` (issue numbers are read from `fizzyedit/work` through `gh`). It reports every missing or empty template -field, bodies over 1,600 characters, invalid branch names, `scripts/zb zig build` in Gates, +field, bodies over 1,600 characters, invalid branch names, a `zig` command wrapped in `scripts/zb` in Gates, and Rules without a `fizzyedit/work#N` reference. It accepts the template above and GitHub issue-form `### Field` headings. The template is the source of the required field list. Run `python3 scripts/test-check-kickoff.py` for the validator tests. @@ -57,7 +57,45 @@ Run `python3 scripts/test-check-kickoff.py` for the validator tests. - The workflow reads labels and events only. It never checks out code, uses no secrets, calls no model, and never puts issue text in a shell. -## Worker rules +## Shared worker contract + +Every adapter (cloud kickoff, Codex/agy/Jules wrapper, workflow or runner) points the worker +at this file. Adapters carry task data and invoke the helpers; they do not copy these rules. +An explicit run restriction, such as an edit-only sandbox, still applies: its trusted harness +runs the publishing helper after collecting the edits. + +1. Run `scripts/agent-prepare /` from the source checkout. It creates a sibling + jj workspace at `main@origin`, or a git worktree at `origin/main` when jj is unavailable, + and prints the absolute rules path. Read that file and work in its checkout. Existing + destinations and invalid branch names fail rather than reusing another worker's work. + It also refuses a temp-like destination (`/tmp`, `/private/tmp`, `/var/tmp`, a scratchpad), + which the OS clears, and fetches `origin` first so the checkout starts at current `main`; + `AGENT_PREPARE_ALLOW_TMP=1` lifts the temp refusal for tests only. + On Linux without Zig it runs `scripts/cloud-setup.sh`; later shells may need + `$HOME/.local/bin` added to `PATH` as setup reports. +2. Read `AGENTS.md`, `CONTRIBUTING.md`, `docs/STYLE.md` and each touched directory's + `AGENTS.md`. Follow the issue's scope. Keep one PR and at most 800 changed lines outside + tests and docs. Record a moved SDK fingerprint; leave `sdk_version` alone. +3. Verify with `scripts/check-fmt.sh`, `scripts/zb build`, `scripts/zb build test`, + `scripts/zb build test-integration`, `scripts/zb build check-web` and + `scripts/zb build test-sdk-version`, plus the issue's gates. Let the build lock wait. + Read each Build Summary: only `N/N steps succeeded` is a pass. Never launch a window + unless the run explicitly authorizes it. +4. After the first push, run `scripts/agent-finish --claim ` (a number or + `fizzyedit/work#N`): it pushes the branch and opens the draft PR saying + `Part of fizzyedit/work#N` with "gates not run yet", so the claim is visible early. +5. When the work is done run `scripts/agent-finish ` in the worker checkout. It checks + formatting and the build gates, records the current change (keeping your description; it + only fills in an empty one), pushes, and updates the PR body, creating the draft if the + claim step was skipped. The body always includes `Part of fizzyedit/work#N`, the diff stat + and the last 50 lines of each gate's output, even if the worker supplied no final message. + Additional gate evidence and explanations belong in the PR before review. + +Both helpers exit non-zero and print a one-line reason on failure. A failed prepare can leave +its workspace for inspection; a failed finish can leave a published branch if GitHub refused +PR creation. Fix the reported problem and retry finish. Neither helper merges or marks ready. + +## Review - Draft PR on the first push; take it to ready only when the gates pass and CI is green. - A different worker than the author reviews the PR. diff --git a/scripts/agent-finish b/scripts/agent-finish new file mode 100755 index 00000000..fa47012f --- /dev/null +++ b/scripts/agent-finish @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +# Publish a worker change with evidence that does not depend on a model's final message. +# `agent-finish --claim ` opens the draft PR early (no gates run); a plain run updates its body. +set -euo pipefail +reason='unexpected failure' +trap 'echo "agent-finish: $reason" >&2' ERR +fail() { echo "agent-finish: $*" >&2; exit 1; } +claim=false +if [[ ${1:-} == --claim ]]; then claim=true; shift; fi +[[ $# == 1 ]] || fail 'usage: agent-finish [--claim] ' +issue=${1#fizzyedit/work#} +[[ $issue =~ ^[1-9][0-9]*$ ]] || fail 'issue must be a positive number or fizzyedit/work#N' +cd "$(dirname "$0")/.." +command -v gh >/dev/null 2>&1 || fail 'gh is required' +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT +branch_re='^[a-z][a-z0-9_-]*/[a-z0-9][a-z0-9_-]*$' +use_jj=false +if command -v jj >/dev/null 2>&1 && jj root >/dev/null 2>&1; then + use_jj=true + reason='could not identify the working bookmark' + # The bookmark sits on @, or on @- when the working copy is a fresh empty change above it. + rev=@ + branch=$(jj log -r @ --no-graph -T 'local_bookmarks.map(|b| b.name()).join("\n")' | grep -E "$branch_re" || true) + if [[ -z $branch ]]; then + rev=@- + branch=$(jj log -r @- --no-graph -T 'local_bookmarks.map(|b| b.name()).join("\n")' | grep -E "$branch_re" || true) + fi +else + branch=$(git symbolic-ref --quiet --short HEAD) +fi +[[ $branch =~ $branch_re ]] || fail 'exactly one area/slug branch must point at the working change' +title="${branch%%/*}: ${branch#*/}" +if $use_jj; then + # Keep the worker's own description; only fill in an empty one. + reason='could not read the change description' + desc=$(jj log -r "$rev" --no-graph -T 'description.first_line()') + if [[ -n $desc ]]; then + title=$desc + else + jj describe -r "$rev" -m "$title" + fi +fi +reason='could not compare with origin/main' +if $use_jj; then + jj diff --from main@origin --to "$rev" --stat >"$tmp/stat" +else + # Inside this clone only; .gitignore keeps zig-pkg, .zig-cache, zig-out and .DS_Store out. + git add -A + git diff origin/main --stat >"$tmp/stat" +fi +$claim || [[ -s $tmp/stat ]] || fail 'no changes relative to main' +{ + printf 'Part of fizzyedit/work#%s\n\n## Changes\n\n```text\n' "$issue" + head -n 100 "$tmp/stat" + printf '```\n\n## Gates\n\n' +} >"$tmp/body" +if $claim; then + printf 'Gates not run yet; `scripts/agent-finish %s` replaces this with their output.\n' "$issue" >>"$tmp/body" +else + run_gate() { + local status=0 + "$@" >"$tmp/gate" 2>&1 || status=$? + cat "$tmp/gate" + { + printf '### `%s` (exit %s)\n\n```text\n' "$*" "$status" + # GitHub caps a PR body near 65k characters; the tail holds the summary. + if [[ $(wc -l <"$tmp/gate") -gt 50 ]]; then printf '... (earlier output omitted)\n'; fi + tail -n 50 "$tmp/gate" + printf '\n```\n\n' + } >>"$tmp/body" + [[ $status == 0 ]] || fail "gate failed: $*" + if [[ $1 == scripts/zb ]]; then + # A successful test count can accompany a failed build step; zig appends + # "; M/M tests passed" to the steps line when tests ran. + awk ' + /Build Summary:/ { + split($3, steps, "/") + if (steps[1] > 0 && steps[1] == steps[2] && + $4 == "steps" && $5 ~ /^succeeded/) passed = 1 + } + END { exit !passed } + ' "$tmp/gate" || fail "missing successful Build Summary: $*" + fi + } + run_gate scripts/check-fmt.sh + run_gate scripts/zb build --summary all + for gate in test test-integration check-web test-sdk-version; do + run_gate scripts/zb build "$gate" --summary all + done +fi +reason='could not publish branch' +if $use_jj; then + jj git push --bookmark "$branch" +else + if ! git diff --cached --quiet; then + git commit -m "$title" + elif [[ $(git rev-list --count origin/main..HEAD) == 0 ]]; then + # A claim before any work: gh pr create needs a commit ahead of main. + git commit --allow-empty -m "$title: claim" + fi + git push --set-upstream origin "$branch" +fi +reason='could not look up pull request' +pr=$(gh pr list --head "$branch" --base main --json number --jq '.[0].number // empty') +reason='could not write draft pull request' +if [[ -n $pr ]]; then + gh pr edit "$pr" --body-file "$tmp/body" +else + gh pr create --draft --base main --head "$branch" --title "$title" --body-file "$tmp/body" +fi diff --git a/scripts/agent-prepare b/scripts/agent-prepare new file mode 100755 index 00000000..87ad2f4d --- /dev/null +++ b/scripts/agent-prepare @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +# Create an isolated worker checkout and print its shared rules path. +set -euo pipefail +reason='unexpected failure' +trap 'echo "agent-prepare: $reason" >&2' ERR +fail() { echo "agent-prepare: $*" >&2; exit 1; } +[[ $# == 1 ]] || fail 'usage: agent-prepare ' +branch=$1 +[[ $branch =~ ^[a-z][a-z0-9_-]*/[a-z0-9][a-z0-9_-]*$ ]] || fail 'branch must be area/slug' +cd "$(dirname "$0")/.." +root=$PWD +workspace="$(dirname "$root")/$(basename "$root")-${branch//\//-}" +case $workspace in + /tmp/* | /private/tmp/* | /var/tmp/* | *claude-*/scratchpad/*) + [[ ${AGENT_PREPARE_ALLOW_TMP:-} == 1 ]] || fail "workspace would sit in a temp directory the OS clears: $workspace" ;; +esac +[[ ! -e $workspace ]] || fail "workspace already exists: $workspace" +reason='could not create workspace' +if command -v jj >/dev/null 2>&1 && jj root >/dev/null 2>&1; then + jj git fetch >&2 || echo 'agent-prepare: warning: jj git fetch failed; main@origin may be stale' >&2 + jj workspace add --name "${branch//\//-}" -r main@origin "$workspace" >&2 + cd "$workspace" + jj describe -m "${branch%%/*}: ${branch#*/}" >&2 + jj bookmark create "$branch" -r @ >&2 +else + git fetch origin main >&2 || echo 'agent-prepare: warning: git fetch failed; origin/main may be stale' >&2 + git worktree add -b "$branch" "$workspace" origin/main >&2 + cd "$workspace" +fi +if [[ $(uname -s) == Linux ]] && ! command -v zig >/dev/null 2>&1; then + reason='cloud setup failed' + scripts/cloud-setup.sh >&2 +fi +printf '%s/docs/AGENT_WORK.md\n' "$workspace" diff --git a/scripts/test-agent-work.sh b/scripts/test-agent-work.sh new file mode 100755 index 00000000..ecfc6d01 --- /dev/null +++ b/scripts/test-agent-work.sh @@ -0,0 +1,149 @@ +#!/usr/bin/env bash +# Exercise worker mechanics with fake tools; no network, builds or repository writes. +set -euo pipefail +cd "$(dirname "$0")/.." +source_root=$PWD +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT +fail() { echo "FAIL: $*" >&2; exit 1; } +mkdir -p "$tmp/repo/scripts" "$tmp/bin" +cp scripts/agent-{prepare,finish} "$tmp/repo/scripts/" +export AGENT_TEST_DIR=$tmp +export AGENT_PREPARE_ALLOW_TMP=1 # the test checkout may itself sit under /tmp +export PATH="$tmp/bin:$PATH" +cat >"$tmp/bin/jj" <<'FAKE' +#!/usr/bin/env bash +set -e +printf '%s\n' "$*" >>"$AGENT_TEST_DIR/calls" +if [[ ${AGENT_TEST_GIT:-} == 1 && $1 == root ]]; then exit 1; fi +case "$1 $2" in + 'workspace add') + destination=${*: -1} + mkdir -p "$destination/docs" "$destination/scripts" + if [[ -e $AGENT_TEST_DIR/cloud-setup.sh ]]; then + cp "$AGENT_TEST_DIR/cloud-setup.sh" "$destination/scripts/" + fi + ;; + 'log -r') + if [[ $* == *description* ]]; then + [[ -e $AGENT_TEST_DIR/no-desc ]] || echo 'ci: worker title' + else echo ci/example; fi ;; + 'diff --from') echo ' scripts/example | 1 +' ;; +esac +FAKE +cat >"$tmp/bin/gh" <<'FAKE' +#!/usr/bin/env bash +set -e +printf '%s\n' "$*" >>"$AGENT_TEST_DIR/calls" +if [[ $1 == pr && $2 == list && ${AGENT_TEST_EXISTING:-} == 1 ]]; then echo 12; fi +if [[ $1 == pr && ( $2 == create || $2 == edit ) ]]; then + while [[ $# -gt 0 ]]; do + if [[ $1 == --body-file ]]; then cp "$2" "$AGENT_TEST_DIR/body"; break; fi + shift + done +fi +FAKE +cat >"$tmp/repo/scripts/check-fmt.sh" <<'FAKE' +#!/usr/bin/env bash +if [[ -e $AGENT_TEST_DIR/fail ]]; then exit 1; fi +echo formatted +FAKE +cat >"$tmp/repo/scripts/zb" <<'FAKE' +#!/usr/bin/env bash +if [[ -e $AGENT_TEST_DIR/no-summary ]]; then exit 0; fi +echo "Build Summary: 1/1 steps succeeded$(cat "$AGENT_TEST_DIR/suffix" 2>/dev/null)" +if [[ -e $AGENT_TEST_DIR/long ]]; then seq 1 200; echo 'Build Summary: 1/1 steps succeeded'; fi +FAKE +chmod +x "$tmp/bin/"* "$tmp/repo/scripts/"* +prepare=$tmp/repo/scripts/agent-prepare +finish=$tmp/repo/scripts/agent-finish +"$prepare" ci/example >"$tmp/output" +grep -qx "$tmp/repo-ci-example/docs/AGENT_WORK.md" "$tmp/output" || fail 'rules path' +if "$prepare" ci/example >"$tmp/output" 2>&1; then fail 'existing workspace accepted'; fi +if "$prepare" ../bad >"$tmp/output" 2>&1; then fail 'unsafe branch accepted'; fi +if "$finish" bad >"$tmp/output" 2>&1; then fail 'invalid issue accepted'; fi +"$finish" fizzyedit/work#42 >"$tmp/output" +grep -q 'Part of fizzyedit/work#42' "$tmp/body" || fail 'issue reference' +grep -q 'scripts/example' "$tmp/body" || fail 'diff evidence' +grep -q '1/1 steps succeeded' "$tmp/body" || fail 'gate evidence' +grep -q 'pr create --draft' "$tmp/calls" || fail 'draft PR' +grep -q -- '--title ci: worker title' "$tmp/calls" || fail 'PR title from description' +! grep -q 'describe -r' "$tmp/calls" || fail 'worker description overwritten on first finish' +printf ' (3/3 tests passed)' >"$tmp/suffix" +"$finish" 42 >"$tmp/output" +grep -q 'tests passed' "$tmp/body" || fail 'tests suffix gate' +rm "$tmp/suffix" +touch "$tmp/long" +"$finish" 42 >"$tmp/output" +! grep -qx 100 "$tmp/body" || fail 'gate output not tailed' +grep -qx 200 "$tmp/body" || fail 'gate output tail missing' +! grep -qx 150 "$tmp/body" || fail 'gate output tail longer than 50 lines' +grep -q 'earlier output omitted' "$tmp/body" || fail 'tail marker' +rm "$tmp/long" +: >"$tmp/calls" +touch "$tmp/no-desc" +"$finish" 42 >"$tmp/output" +grep -q 'describe -r @ -m ci: example' "$tmp/calls" || fail 'empty description filled' +rm "$tmp/no-desc" +: >"$tmp/calls" +"$finish" 42 >"$tmp/output" +! grep -q 'describe' "$tmp/calls" || fail 'worker description overwritten' +touch "$tmp/fail" +: >"$tmp/calls" +"$finish" --claim 42 >"$tmp/output" +grep -q 'pr create --draft' "$tmp/calls" || fail 'claim draft PR' +grep -q 'Gates not run yet' "$tmp/body" || fail 'claim body' +rm "$tmp/fail" +rm "$tmp/body" +touch "$tmp/fail" +if "$finish" 42 >"$tmp/output" 2>&1; then fail 'failed gate accepted'; fi +[[ ! -e $tmp/body ]] || fail 'published after failed gate' +grep -q '^agent-finish: gate failed:' "$tmp/output" || fail 'failure reason' +! grep -q 'scripts/zb zig build' "$source_root/docs/AGENT_WORK.md" || fail 'invalid zb invocation' +rm "$tmp/fail" +touch "$tmp/no-summary" +if "$finish" 42 >"$tmp/output" 2>&1; then fail 'missing summary accepted'; fi +[[ ! -e $tmp/body ]] || fail 'published without Build Summary' +rm "$tmp/no-summary" +bad=$(mktemp -d /tmp/agent-test.XXXXXX) +mkdir "$bad/scripts" +cp "$tmp/repo/scripts/agent-prepare" "$bad/scripts/" +if AGENT_PREPARE_ALLOW_TMP= "$bad/scripts/agent-prepare" ci/example >"$tmp/output" 2>&1; then rm -rf "$bad"; fail 'tmp workspace accepted'; fi +rm -rf "$bad" +cat >"$tmp/bin/git" <<'FAKE' +#!/usr/bin/env bash +set -e +printf 'git %s\n' "$*" >>"$AGENT_TEST_DIR/calls" +case "$1 $2" in + 'worktree add') mkdir -p "${*: -2:1}" ;; + 'symbolic-ref --quiet') echo ci/git ;; + 'rev-list --count') echo "${AGENT_TEST_AHEAD:-1}" ;; + 'diff origin/main') echo ' scripts/git-example | 1 +' ;; +esac +FAKE +chmod +x "$tmp/bin/git" +AGENT_TEST_GIT=1 "$prepare" ci/git >"$tmp/output" +grep -q 'git worktree add -b ci/git' "$tmp/calls" || fail 'git worktree fallback' +AGENT_TEST_GIT=1 "$finish" 42 >"$tmp/output" +grep -q 'git push --set-upstream origin ci/git' "$tmp/calls" || fail 'git push fallback' +: >"$tmp/calls" +AGENT_TEST_GIT=1 AGENT_TEST_AHEAD=0 "$finish" --claim 42 >"$tmp/output" +grep -q 'git commit --allow-empty -m ci: git: claim' "$tmp/calls" || fail 'git claim empty commit' +: >"$tmp/calls" +AGENT_TEST_GIT=1 "$finish" --claim 42 >"$tmp/output" +! grep -q 'allow-empty' "$tmp/calls" || fail 'empty commit despite commits ahead' +AGENT_TEST_EXISTING=1 "$finish" 42 >"$tmp/output" +grep -q 'pr edit 12 --body-file' "$tmp/calls" || fail 'existing PR refresh' +cat >"$tmp/bin/uname" <<'FAKE' +#!/usr/bin/env bash +echo Linux +FAKE +cat >"$tmp/cloud-setup.sh" <<'FAKE' +#!/usr/bin/env bash +touch "$AGENT_TEST_DIR/setup-ran" +FAKE +chmod +x "$tmp/bin/uname" "$tmp/cloud-setup.sh" +# System bins provide the shell and basic tools, without the developer's Zig launcher. +PATH="$tmp/bin:/usr/bin:/bin" "$prepare" ci/cloud >"$tmp/output" +[[ -e $tmp/setup-ran ]] || fail 'Linux without Zig did not run cloud setup' +echo ok