From d84735d09982dfb3829e1da2e2fbb236eaf86d26 Mon Sep 17 00:00:00 2001 From: dmccoystephenson Date: Sun, 16 Aug 2026 02:12:11 +0000 Subject: [PATCH 1/3] Harden Phase 8's rebase fence and state the merge-authorization precedence rule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The post-rebase verification ran the test command alone, so a required check that also runs a linter or static analysis survived the force-push and was rediscovered from CI minutes later. It now runs the same verification set Phase 3 runs. The bare --force-with-lease has three unrelated causes that share one '(stale info)' rejection — a restricted fetch refspec, an abbreviated SHA, and a genuine concurrent push. Only the third justifies backing off, so the fence now reads a full 40-character lease value from the remote at push time and the disambiguation rule is written down. Phase 8 also gains an explicit precedence rule: a run-level merge pre-authorization conveys permission, not validation, so it does not satisfy Phase 4's anchor gate on an UNVERIFIED PR. Closes #115 Closes #117 Closes #118 Co-Authored-By: Claude Opus 5 (1M context) --- create-dev-loop.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/create-dev-loop.md b/create-dev-loop.md index 4124396..207275b 100644 --- a/create-dev-loop.md +++ b/create-dev-loop.md @@ -471,6 +471,8 @@ git checkout {{DEFAULT_BRANCH}} && git pull **Review-ready hand-off is a valid terminal state — not a failed cycle.** On a repo with no autonomous merge path (branch-protected `{{DEFAULT_BRANCH}}` requiring an approving review, and/or repository auto-merge disabled), a clean, anchor-green PR's true terminal state is *open + self-review posted + awaiting human approval*. Expect `gh pr merge` → `the base branch policy prohibits the merge` and `--auto` → `Auto merge is not allowed for this repository`. **Never use `--admin`** to bypass a deliberately-configured human gate, and a later self-audit must not score "could not auto-merge" as a failure. A do-not-auto-merge path match blocks **autonomous** merge only: if the human codeowner explicitly authorizes the merge after being shown which protected path matched, that authorization satisfies the hold — proceed (still run the Phase 7 docs sweep + the regression gate above first), and state in the merge report which protected path was overridden and by whose authorization. +**Standing merge authorization does not satisfy the anchor gate.** A run-level merge pre-authorization granted before this PR existed (e.g. a headless dispatch launched with a blanket "may merge" flag) conveys *permission*, not *validation* — where Phase 4 has marked the PR UNVERIFIED because the anchor could not run on anchor-relevant files, the hand-off stands regardless of merge authority. The codeowner exception above is retrospective by construction: it applies only to authorization given *after* the specific protected path has been surfaced to the codeowner, so a flag passed before the PR was written does not satisfy it. The terminal state in that case is open + self-review posted + awaiting a human, which the paragraph above already declares valid rather than failed. + **Worktree note:** under a `git worktree` workflow (main checkout kept on `{{DEFAULT_BRANCH}}`), `gh pr merge --delete-branch`'s *local* branch deletion can fail with `fatal: '{{DEFAULT_BRANCH}}' is already checked out` — the merge and remote-branch deletion still succeed. Verify the real state with `gh pr view --json state` and delete the remote branch separately rather than treating the local error as a failed merge. Verify issues auto-closed. Close any that did not: @@ -547,13 +549,20 @@ Return to Phase 1. **A `gh` command fails with a transient network/transport error (`http2: client conn could not be established`, `unexpected EOF`):** retry once or twice before treating it as blocked — do not treat the first failure as terminal and silently drop a self-review, commit, or comment. **`gh pr create` fails with "you must first push the current branch to a remote" despite a successful `git push -u`:** the clone's fetch refspec may be restricted to the default branch only (common in gardener-managed or otherwise restricted dedicated checkouts) — confirm with `git config --get-all remote.origin.fetch`. A restricted refspec means `git fetch` never populates a remote-tracking ref for feature branches, so `gh pr create`'s and `git branch --set-upstream-to`'s auto-detection both fail even though the push and upstream config succeeded. Pass `--head ` explicitly to `gh pr create` to bypass the remote-tracking-ref lookup rather than retrying the push. {{#if REVIEWER}}**Reviewer addition fails:** proceed to the self-review step in Phase 4. Do not skip to Phase 5 without posting self-review comments — Phase 6 addresses those comments like any other review.{{/if}} -**Branch is behind main or has a merge conflict at Phase 8:** rebase onto main, re-run tests, and force-push before retrying the merge: +**Branch is behind main or has a merge conflict at Phase 8:** rebase onto main, re-run the **same verification set Phase 3 runs** — not just the test command — and force-push before retrying the merge: \`\`\`bash git fetch origin git rebase origin/{{DEFAULT_BRANCH}} +{{COMPILE_CMD}} {{TEST_CMD}} -git push --force-with-lease +{{#if LINT_CMD}}{{LINT_CMD}}{{/if}} +git ls-remote origin refs/heads/ # full 40-char SHA for the lease below +git push origin : --force-with-lease=: \`\`\` +**Verify with everything the required checks run, not a subset.** A force-push commits to the result, so a repository whose required checks also run a formatter, linter or static analysis will otherwise go red on a task the test command never covered, minutes later, from CI. Where the repository's required checks exceed the Phase 3 verification set, run those too rather than trusting the subset. + +**`(stale info)` has three unrelated causes; only one justifies backing off.** A bare `--force-with-lease` compares against a maintained remote-tracking ref, which a restricted clone does not have (fetch refspec covering `{{DEFAULT_BRANCH}}` only — see the `gh pr create` entry above); the push is rejected as `! [rejected] -> (stale info)`. The explicit `--force-with-lease=:` form fixes that, but the expected value is compared against the remote ref literally and abbreviations are never expanded — so a short SHA copied from `git log --oneline` is rejected with the *same* message. A restricted fetch refspec, an abbreviated SHA, and a genuine concurrent push therefore all read identically. Do not treat `(stale info)` as "another session pushed" until the expected SHA has been confirmed **full 40-character** and re-read from the remote at push time (`git ls-remote`; `git rev-parse origin/` also yields the full form but only reflects the last fetch, so it is the weaker source where the lease is meant to be meaningful). Never fall back to a plain `--force`, which clobbers unseen work in exactly the one case that warranted stopping. + If the rebase produces conflicts that cannot be resolved automatically, close the PR and delete the branch, then return to Phase 1: \`\`\`bash gh pr close --comment "Closing: unresolvable merge conflict after rebase." @@ -588,9 +597,9 @@ Use findings from Step 2 to substitute each `{{placeholder}}`. The table below c | `SKILL_REPO_OWNER` | `gh api user -q .login` — the authenticated GitHub account that ran `/create-dev-loop`, i.e. the owner of the skill-tracker repo created in Step 6. Not necessarily the same as `GITHUB_OWNER` when the target project lives under an org. | | `DEFAULT_BRANCH` | `gh repo view --json defaultBranchRef -q .defaultBranchRef.name` | | `BRANCH_PREFIX` | From CONTRIBUTING.md, or `feature` if not specified | -| `COMPILE_CMD` | Fastest command that catches syntax/import errors without running tests. For Maven: `MVN=$([ -f ./mvnw ] && echo ./mvnw \|\| echo mvn) && $MVN compile`. **If the repo has no build system** (docs-only, config-only, single-template repos), substitute the cheapest mechanical consistency check that would actually catch breakage — e.g. a `grep` for unresolved placeholders, `jq -e . `, a YAML/schema lint. If no such check exists, substitute a shell comment naming what gates correctness instead (e.g. `# no build system — see the validation steps below`) so the fence stays valid bash. | +| `COMPILE_CMD` | Fastest command that catches syntax/import errors without running tests. For Maven: `MVN=$([ -f ./mvnw ] && echo ./mvnw \|\| echo mvn) && $MVN compile`. **If the repo has no build system** (docs-only, config-only, single-template repos), substitute the cheapest mechanical consistency check that would actually catch breakage — e.g. a `grep` for unresolved placeholders, `jq -e . `, a YAML/schema lint. If no such check exists, substitute a shell comment naming what gates correctness instead (e.g. `# no build system — see the validation steps below`) so the fence stays valid bash. Substitute the same text in both places it appears (Phase 3 verify block, Phase 8 rebase fence). | | `TEST_CMD` | Full test suite command from CI workflow or README. For Maven: `$MVN test` (reuse the `MVN` variable set above). **If the repo has no automated test suite** — confirm it (`CLAUDE.md` says so, no test directory, no CI test job); don't assume — substitute a shell comment pointing at the project's manual validation steps, e.g. `# re-run the Phase 3 manual validation checklist`, naming the same check `EXTERNAL_SIGNAL_CMD` renders. Substitute the same text everywhere `{{TEST_CMD}}` appears (Phase 3 verify block, Phase 8 rebase fence). | -| `LINT_CMD` | Linter/formatter command if present in CI; omit section if absent | +| `LINT_CMD` | Linter/formatter command if present in CI; omit section if absent. Appears in both the Phase 3 verify block and the Phase 8 rebase fence — include or omit it in both, so the post-rebase verification covers the same tasks the required checks do | | `VALIDATION_NOTE` | Only when `COMPILE_CMD` or `TEST_CMD` is a shell comment rather than a real command: a one-line parenthetical placed directly under the Phase 3 verify fence, stating what actually gates correctness — e.g. `(no automated test command or linter exists for this repo — the manual validation checklist in Phase 4 is the real gate)`. Omit the conditional block entirely when both are real commands. | | `REVALIDATE_INSTRUCTION` | Imperative clause opening the Phase 6 re-verification sentence (capitalized, no trailing period; it is followed by "after every fix"). With a test suite: `Run \`$MVN test\``. Without one: `Re-run the Phase 3 manual validation checklist`. Must name the same check as `TEST_CMD` and `EXTERNAL_SIGNAL_CMD`. | | `EXTERNAL_SIGNAL_LABEL` | Short label for the anchor used in Phase 4 self-review. `CI` when the repo has CI workflows; `manual validation` when the project uses a fixture or test-command checklist (e.g. a doc-only repo); otherwise a project-specific phrase like `snapshot regression suite`. | @@ -606,7 +615,7 @@ Use findings from Step 2 to substitute each `{{placeholder}}`. The table below c | `TEMPLATE_VERSION` | Short commit SHA of the `create-dev-loop` repo at generation time. Capture with `git -C rev-parse --short HEAD`. Becomes part of an HTML comment at the top of the generated skill so future cycles can detect template drift. | | `GENERATED_AT` | ISO-8601 UTC timestamp at generation time. Capture with `date -u +%Y-%m-%dT%H:%M:%SZ`. Pairs with `TEMPLATE_VERSION` in the HTML-comment header. | -**When the repo has no build system or no automated test suite**, `COMPILE_CMD`, `TEST_CMD`, `REVALIDATE_INSTRUCTION`, `VALIDATION_NOTE`, and `EXTERNAL_SIGNAL_CMD` must all name the *same* check — whatever the project actually gates correctness on. Also reword the Edge cases entries that name tests — "Tests fail during implementation", "Tests fail after addressing a comment", "A test fails intermittently", and the "re-run tests" step of "Branch is behind main or has a merge conflict at Phase 8" — so they refer to that check rather than a test suite; keep the entry order, wording of the surrounding rules, and phase numbers unchanged. +**When the repo has no build system or no automated test suite**, `COMPILE_CMD`, `TEST_CMD`, `REVALIDATE_INSTRUCTION`, `VALIDATION_NOTE`, and `EXTERNAL_SIGNAL_CMD` must all name the *same* check — whatever the project actually gates correctness on. Also reword the Edge cases entries that name tests — "Tests fail during implementation", "Tests fail after addressing a comment", "A test fails intermittently", and the post-rebase verification step of "Branch is behind main or has a merge conflict at Phase 8" — so they refer to that check rather than a test suite; keep the entry order, wording of the surrounding rules, and phase numbers unchanged. For `SCAN_CHECKLIST`, always include these universal items plus any repo-specific ones: - Missing tests on new public methods From 9d7af166a6d4777395f4a7d032a06f6016dd3eed Mon Sep 17 00:00:00 2001 From: dmccoystephenson Date: Sun, 16 Aug 2026 02:13:25 +0000 Subject: [PATCH 2/3] Record PR #119 under RESEARCH.md finding 1 Co-Authored-By: Claude Opus 5 (1M context) --- RESEARCH.md | 1 + 1 file changed, 1 insertion(+) diff --git a/RESEARCH.md b/RESEARCH.md index 2541e58..932c84c 100644 --- a/RESEARCH.md +++ b/RESEARCH.md @@ -48,6 +48,7 @@ Last reviewed: 2026-08-04. - PR #54 (Apply consolidated template-rule backlog, #44/#46/#47): shipped 2026-06-14. Observed effect: pending — needs N cycles of data. Sharpens the external-anchor definition (#44: UNVERIFIED handling when the anchor can't run; #46: green CI is not verification when CI's scope excludes the changed files) and grounds the Tests-fix rubric item empirically (#47: stash-and-run FAIL→PASS, not scored from reasoning alone) — directly targeting the self-preference failure mode this finding's evidence documents for judgment-based rubric scoring. - PR #57 (Fix documentation drift across the three sources of truth): shipped 2026-07-25. Observed effect: pending — needs N cycles of data. Restores `{{EXTERNAL_SIGNAL_LABEL}}` in the Phase 1 carried-over-PR bullet, where PR #54 had reintroduced a hardcoded "CI must be green first" after PR #28 generalized the anchor. An anchor named `CI` in a repo that has no CI is an anchor the loop cannot check, which degrades the self-review to exactly the unanchored critique this finding shows is neutral-to-harmful. - PR #67 (Add Tests-fix CI-revert fallback ladder): shipped 2026-07-26. Observed effect: pending — needs N cycles of data. Closes a gap left by PR #54: the Tests-fix rubric item required a local `git stash` experiment with no substitute when the local anchor can't run, even though the same PR's UNVERIFIED handling fully anticipates that case. Without a defined fallback, Tests-fix silently degraded back to unanchored judgment on exactly the repos this finding is about — confirmed twice, in two separate PRs opened by a generated child skill running against a repo whose local test anchor could not be run. +- PR #119 (Harden Phase 8's rebase fence and state the merge-authorization precedence rule): shipped 2026-08-16. Observed effect: pending — needs N cycles of data. Two more ways the anchor silently weakens at the merge phase: the post-rebase verification ran the test command alone, so a required check covering a linter or static analysis was never part of the signal a force-push committed to (#115); and nothing stated whether a run-level merge pre-authorization could stand in for a Phase 4 UNVERIFIED result (#118). Both are the same failure this finding's PR #54 entry records for CI scope — an anchor that does not cover the changed surface is not an anchor — applied to permission rather than coverage in the second case. --- From 6bffb115c7462a9b625c6c2dda83b2ec9cc7fb2a Mon Sep 17 00:00:00 2001 From: Daniel McCoy Stephenson Date: Sat, 22 Aug 2026 01:02:18 -0600 Subject: [PATCH 3/3] Correct the ship date on RESEARCH.md's PR #119 implementations entry The entry was written on the day the PR was opened and dated 2026-08-16, but the PR did not merge that day. Re-date it to the day the branch was finalized so the finding does not carry a ship date that never happened. --- RESEARCH.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RESEARCH.md b/RESEARCH.md index 5f0acee..e02960e 100644 --- a/RESEARCH.md +++ b/RESEARCH.md @@ -48,7 +48,7 @@ Last reviewed: 2026-08-04. - PR #54 (Apply consolidated template-rule backlog, #44/#46/#47): shipped 2026-06-14. Observed effect: pending — needs N cycles of data. Sharpens the external-anchor definition (#44: UNVERIFIED handling when the anchor can't run; #46: green CI is not verification when CI's scope excludes the changed files) and grounds the Tests-fix rubric item empirically (#47: stash-and-run FAIL→PASS, not scored from reasoning alone) — directly targeting the self-preference failure mode this finding's evidence documents for judgment-based rubric scoring. - PR #57 (Fix documentation drift across the three sources of truth): shipped 2026-07-25. Observed effect: pending — needs N cycles of data. Restores `{{EXTERNAL_SIGNAL_LABEL}}` in the Phase 1 carried-over-PR bullet, where PR #54 had reintroduced a hardcoded "CI must be green first" after PR #28 generalized the anchor. An anchor named `CI` in a repo that has no CI is an anchor the loop cannot check, which degrades the self-review to exactly the unanchored critique this finding shows is neutral-to-harmful. - PR #67 (Add Tests-fix CI-revert fallback ladder): shipped 2026-07-26. Observed effect: pending — needs N cycles of data. Closes a gap left by PR #54: the Tests-fix rubric item required a local `git stash` experiment with no substitute when the local anchor can't run, even though the same PR's UNVERIFIED handling fully anticipates that case. Without a defined fallback, Tests-fix silently degraded back to unanchored judgment on exactly the repos this finding is about — confirmed twice, in two separate PRs opened by a generated child skill running against a repo whose local test anchor could not be run. -- PR #119 (Harden Phase 8's rebase fence and state the merge-authorization precedence rule): shipped 2026-08-16. Observed effect: pending — needs N cycles of data. Two more ways the anchor silently weakens at the merge phase: the post-rebase verification ran the test command alone, so a required check covering a linter or static analysis was never part of the signal a force-push committed to (#115); and nothing stated whether a run-level merge pre-authorization could stand in for a Phase 4 UNVERIFIED result (#118). Both are the same failure this finding's PR #54 entry records for CI scope — an anchor that does not cover the changed surface is not an anchor — applied to permission rather than coverage in the second case. +- PR #119 (Harden Phase 8's rebase fence and state the merge-authorization precedence rule): shipped 2026-08-22. Observed effect: pending — needs N cycles of data. Two more ways the anchor silently weakens at the merge phase: the post-rebase verification ran the test command alone, so a required check covering a linter or static analysis was never part of the signal a force-push committed to (#115); and nothing stated whether a run-level merge pre-authorization could stand in for a Phase 4 UNVERIFIED result (#118). Both are the same failure this finding's PR #54 entry records for CI scope — an anchor that does not cover the changed surface is not an anchor — applied to permission rather than coverage in the second case. - PR #121 (Define the structurally-red-anchor path and require the anchor to have executed tests): shipped 2026-08-16. Observed effect: pending — needs N cycles of data. Extends the same argument as PR #54's "green CI is not verification when CI's scope excludes the changed files" to two further ways an anchor can carry no information: a required job that has *never* succeeded on the base branch (observed on a repo whose CI jobs have failed on empty credentials since February 2026, so every cycle terminated in a hand-off regardless of quality), and a build tool that reports success having executed zero tests (`:test NO-SOURCE` still prints `BUILD SUCCESSFUL`). Both were previously read as ordinary signals — the first as a blocking red, the second as a green — which is unanchored critique wearing an anchor's clothes. ---