Skip to content

Review rules as a fork feature: finish the judge (C3, C4) and the lens tools (T3-T5) #888

Open

Description

@scuttletoby

What: review rules as a fork feature. A project's config says which review lenses a change needs (by changed path and tier), the merge gate judges it, and T3 tools start lenses, record their reports and arm the PR, so the rules are enforced by code and not by the orchestrator's memory.

Why the owner wants it: reviews decide whether unattended work is safe to merge; today much of it is prose rules and a project-specific gate.

Already built: C1 config #301, C2 path judge #340 (+ #359), T1 contracts #291, T6 Settings panel #318 (+ #372), shared logic on shared vectors #781, release guide #778, general review kit #790. Also merged: T3a lens_start logic #825, general review skill #845. Open: follow-ups #797 #799 #841.

Left (checklist):

  • C3 judge: rounds, approach lens, blocker: / major: and state: lines
  • C4 release tag, gate caller example, docs
  • T3 lens_start (first part T3a Review rules T3a: lens_start plan, prompt, refusals and request ids (pure logic) #825 merged)
  • T4 lens_report: server-written header, real commit
  • T5 lens_arm: refusals, ready, pinned auto-merge, check wait
  • Each project pins a release, runs the review block in shadow mode, then enforce, and deletes its old gate code
  • Outside reviewers as a config key

Owner decisions (all decided 2026-11-14): #81 (one shared gate App), #82 (one extra agent review for workflow and gate settings), #99 (block on blocker: / major:), #100 (cap of 4 rounds, approach from round 3).

Related: #848 (one merge path), #862 (batched merge queue), #865 (look-alike lens comments), #870 (orchestrator logic as software).

Done when: every project's merge gate reads one review config from the fork, lenses are started and recorded by T3 tools, and a PR without its required lenses cannot merge.

🤖 Generated with Claude Code

Activity

  1. scuttletoby commented on Nov 22, 2026

    @scuttletoby
    OwnerAuthor

    Two details carried over from a consumer project, for T5 (lens_arm) and C3:

    • Arm must refuse a quick check that says nit-only: no. The consumer's rule: a quick check writes nit-only: yes|no, and a merge may be armed only on yes. Its ready command takes the reason as free text, so a quick check that said no could still be used to arm. Wanted in lens_arm: when the reason names a quick-check comment, read it and refuse unless it says nit-only: yes at the PR head and has no blocker: or major: line. Test: a quick check with nit-only: no is refused. Extra cases for the review-gate switch: setting "off" in the environment, a linked worktree, a PR branch that commits "off", and a main with no gate key.
    • One parser for arm and for the head check. The consumer's ready command used its own copy of the lens rules and read each comment one way, while pr-queue/lens.py reads it two ways. An odd lens header (brackets around the sha, a broken HTML tag) passed arm but turned the head check red, and the PR sat stuck. It can only block, never let a PR through (no fail-open in 300,000 random comments). Wanted: lens_arm runs the pinned lens.py, and the "each project deletes its old gate code" step removes such copies.

    🤖 Generated with Claude Code

  2. scuttletoby commented on Nov 22, 2026

    @scuttletoby
    OwnerAuthor

    Two more details carried over from a consumer project, for T5 (lens_arm, "ready") and the head check:

    • After an override, re-run every red head-check run on the head, from every workflow. In the consumer project every PR head gets two required-name check runs (one from the main CI workflow, one from the workflow for edited PRs). After a valid override, its watcher re-ran only the newest red one; the other stayed red and GitHub kept four PRs blocked. Wanted: after posting an override, arm/ready re-runs every red run of the required check on that head, whatever workflow made it. Also decide whether two workflows should report a check of the same name at all. Test: two red runs on one head (one per workflow) are both re-run after an override.
    • Race: the head check starts before the override is posted. A push starts CI, the head check reads the lens comments at once, and the ready command posts the override for the new head a little later. The check has already failed with " is not the change read at " and the run stays red; a manual re-run passes (seen twice on one PR). Wanted: once the override is posted, ready re-runs a head check that failed only on that reason (not a fixed wait). Test: push, then ready with an override, gives a green head check without a manual re-run.

    State question: the re-run is asked twice, the override is posted twice, or a newer push lands between the post and the re-run (never re-run a check for a head that is no longer the PR head).

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions