feat(roadmap): every step repairs before it gives up, in bounded rounds #3053

What happened

A step that fails stops the item: one more turn after a ruff or mypy error, then "all your work is lost". Examples: #2175 stopped on its own scratch file; XML tool-call parse errors (#2953) end a step; a lost test ends a step. The agent has no way to try another approach.

What should happen

• A step runs in attempts. On a failure the agent gets the failure class, the evidence (command output, trace) and a diagnosis hint, and tries again. After round 1 the strategy changes: more context from the repo map, a smaller step (split the group), a different model tier (D4).
• The number of rounds, tokens and time per step and per item are limits in config. Each attempt is in the trace. Only an exhausted budget, or a failure class that no attempt can fix (a false premise), stops the item.

Acceptance criteria

• A replay with a failing check passes on the second attempt and the trace shows both.
• The rounds, tokens and time limits are settings with defaults, and each stop names the limit that ended it.
• A failure of class plan defect or premise false stops at once with the evidence (F1).
• No attempt repeats a previous failed action unchanged (a test with a stub model).

Related: tracker for this set (filed with it), #2908 (issue workflow), #2867 (draft PR evaluator), #3005 (task kind), #3010 to #3015 (plan gates), #2919 (stuck-item metrics), #2953 (Ollama parser).

Delivery

This issue is delivered in parts. Each part has its own change folder. The pull request of the last part to merge closes this issue.

• Delivered by feat(roadmap): a step stops at its budgets and records attempts #3568 (merged): the budgets, the named stops, the single attempt of a fatal class, and the attempt events of the trace. This covers the criteria "The rounds, tokens and time limits are settings with defaults, and each stop names the limit that ended it" and "A failure of class plan defect or premise false stops at once".
• Delivered by the pull request of this issue: the class parser. The resend carries one message that asks for a smaller edit, and a second parser failure of the step stops it with class parser. Issue feat(roadmap): a parser failure gets one smaller-edit message #3571 holds the same work and closes as delivered.
• Moved to feat(roadmap): each repair round changes the strategy of a step #3569: the strategy of each round, and the name of roadmap.implement.ladder in the stop of a one-tier ladder.
• Moved to feat(roadmap): a repair attempt cannot repeat a failed tool call #3570: "No attempt repeats a previous failed action unchanged".
• Moved to fix(roadmap): a step whose agent moved HEAD is reset, not stopped #3575: the reset of a moved HEAD.
• Moved to feat(roadmap): a group that changes no file returns for a smaller one #3576: a group that changes no file returns for a smaller replacement.
• Moved to feat(roadmap): a failed verification command gets a repair round #3577: the repair round of a failed verification command, which holds the criterion "A replay with a failing check passes on the second attempt and the trace shows both".

Alternatives check of 2026-11-04: partly.
Tool: pydantic/pydantic-ai (https://github.com/pydantic/pydantic-ai), MIT, last activity 2026-11-04, maintained.

Proposed design

Problem statement

A step that fails stops the whole item after one more turn. The agent cannot try another approach. The item loses the work of the step. This change gives each step bounded repair rounds. Each round carries the failure class, the evidence and a diagnosis hint. The rounds, token and time limits are settings, and each stop names the limit that ended it.

Acceptance criteria

Requirement: Repair on the second attempt

A step SHALL retry a failed check within its round budget and pass on the second attempt.

Scenario: Failing check passes on the second attempt

• WHEN A replay runs a step whose first attempt fails a check and whose second attempt passes.
• THEN The step succeeds and the trace holds one entry for each attempt with its class and evidence.

Requirement: Limits are settings with defaults

The round, token and time limits of a step and of an item SHALL be settings with defaults.

Scenario: Token budget ends a step

• WHEN A step uses more input tokens than roadmap.implement.step_token_budget.
• THEN The step stops and the stop comment names that setting and its value.

Requirement: Fatal classes stop at once

A failure of class plan defect or premise false SHALL stop the item at once with its evidence.

Scenario: A false premise stops before a retry

• WHEN The first attempt of a step fails with class premise false.
• THEN The item stops with the evidence and the trace holds no second attempt.

Requirement: No repeated failed action

A repair attempt MUST NOT repeat a previous failed action unchanged.

Scenario: A stub model repeats its tool call

• WHEN A stub model sends the same tool call with the same arguments as a failed earlier attempt.
• THEN The executor refuses the call and returns the earlier failure and its diagnosis hint to the model.

Requirement: Strategy change between rounds

Each attempt after a failure SHALL change the strategy of the step, as the round and the failure class decide.

Scenario: The second attempt widens the context

• WHEN The first attempt of a step fails with class test.
• THEN The second attempt carries more repo-map context and the trace records the strategy of each round.

Requirement: Parser failure gets a smaller edit

A parser failure SHALL add one message that asks for a smaller edit, and a second parser failure MUST stop the step.

Scenario: Two parser errors stop the step

• WHEN Two attempts of one step fail with class parser.
• THEN The step stops with class parser and the work of the group stays.

Requirement: Moved HEAD is reset

A step whose failure says the agent moved HEAD SHALL reset the branch to the start commit with the changes kept.

Scenario: The agent commits during a step

• WHEN A step fails with class ci_infra and the text The agent moved HEAD.
• THEN The job runs git reset --soft and the next attempt starts from the start commit with the changes in the worktree.

Requirement: No-op group is a plan defect

A task group that changes no file SHALL be recorded as class plan defect and returned for a smaller replacement.

Scenario: One no-op group and one real group

• WHEN A step runs one group that changes no file and one group that changes a file.
• THEN The trace records class plan defect for the no-op group only and the next round asks for a smaller replacement of that group.

Requirement: One-tier ladder names its setting

A change_tier action on a ladder with one tier SHALL name roadmap.implement.ladder in the stop comment.

Scenario: Tier change with one tier configured

• WHEN A step reaches the tier round and roadmap.implement.ladder holds one tier.
• THEN The stop comment names roadmap.implement.ladder and its value.

Existing Tool Considered

pydantic/pydantic-ai (MIT, last activity 2026-11-04, maintained) covers bounded attempts, failure feedback with ModelRetry, usage budgets with a named limit, and a message history that keeps each attempt. It lacks a failure-class taxonomy, a strategy change between rounds, per-step and per-item round and time budgets with a named stop, and a guard against a repeated failed action. The project builds these missing parts: about 260000 tokens to build, about 120000 tokens to keep and fix in the first three months, and about 4 working days.

Key questions

• Which failure classes does a step distinguish, and which of them stop the item at once?: A step records one class for each failure: plan defect, premise false, parser, test, typecheck, lint, commit and ci_infra. Class plan defect and class premise false stop the item at once with the evidence. The other classes get repair rounds. (best practice) [sources: src/devops_cli/roadmap/failure.py, tests/roadmap/test_roadmap_failure.py]
  • Chosen: Class from the failing check and the failure text. The check name gives test, typecheck and lint; the failure text gives plan defect, premise false, parser and ci_infra.
  • Rejected: Two classes only, fatal and repairable. The strategy change and the stop message need the finer class.
  • Rejected: Class from a model call. A model call adds cost and gives a different answer on equal input.
• What are the default round, token and time limits for a step and for an item?: roadmap.implement.repair_rounds defaults to 3 attempts for each step. roadmap.implement.step_token_budget defaults to 300000 input tokens. roadmap.implement.item_token_budget defaults to 2000000 input tokens. roadmap.implement.step_time_limit defaults to 30 minutes. roadmap.implement.item_time_limit defaults to 4 hours. A person can change each setting in .github/roadmap.toml. (best practice) [sources: src/devops_cli/config/defaults.py, src/devops_cli/config/settings.py, src/devops_cli/config/constants.py]
  • Chosen: 3 rounds, 300000 tokens and 30 minutes for each step. The 39 stops in the evaluation cost about 45000 input tokens each; 300000 allows repairs and still stops a runaway step.
  • Rejected: One repair turn, as today. One turn cannot change the strategy and fails the second-attempt criterion.
  • Rejected: Rounds only, with no token or time limit. A step that repeats a large prompt can exhaust the item budget with no named stop.
• How does the strategy change after the first failed attempt?: Attempt 1 runs the step as planned. A failed attempt gives the next attempt the class, the evidence and a diagnosis hint. Attempt 2 adds context from the repo map and, when the failure names a group or the inputs exceed the step budget, asks for a smaller group. Attempt 3 changes the model tier when roadmap.implement.ladder holds more than one tier; otherwise the stop names that setting. (best practice) [sources: src/devops_cli/roadmap/implement_step.py, tests/roadmap/test_roadmap_implement_step_repair.py]
  • Chosen: A fixed ladder: context, then smaller group, then tier. It is predictable, it changes the input in each round, and a stop names the round that ended.
  • Rejected: Class-driven strategy only. Two failures of the same class would repeat the same strategy.
  • Rejected: Tier change first. A tier change costs more and does not fix a group that is too large.
• How does the loop stop an attempt from repeating a failed action unchanged?: The step keeps a digest of each failed tool call: the tool name and its normalized arguments. The executor refuses a call whose digest is in the set. It returns the earlier failure and its diagnosis hint to the model as a new turn. (best practice) [sources: src/devops_cli/roadmap/runner_tools.py, tests/roadmap/test_roadmap_runner_tools.py]
  • Chosen: A digest guard in the tool executor. The executor sees every call and can refuse one before it runs.
  • Rejected: A prompt instruction only. A model can ignore the instruction, and the stub-model test needs a hard refusal.
  • Rejected: ModelRetry alone. ModelRetry returns a message, but it does not record the failed actions of earlier rounds.
• What does a parser failure class do in the repair loop?: The failure is deterministic for one conversation. The next attempt carries one message that asks for a smaller edit. A second parser failure stops the step with class parser, and the work of the group stays. (best practice) [sources: src/devops_cli/roadmap/implement_step.py, tests/roadmap/test_roadmap_implement_step.py]
  • Chosen: One smaller-edit message, then stop. A resend of the same conversation cannot change a deterministic parse error.
  • Rejected: Resend the same conversation. _turn does this today, and the upstream report says the error repeats.
  • Rejected: Change the model tier on the first parser error. A tier change is the third round, and the parse error needs a smaller edit first.
• What repair does a step get when the failure says the agent moved HEAD?: The job runs git reset --soft to move the branch back to the start commit and keeps the changes in the worktree. The next attempt runs the checks on that result. (best practice) [sources: src/devops_cli/roadmap/implement_step.py, src/devops_cli/roadmap/runner_tools.py, tests/roadmap/test_roadmap_implement_shell_policy.py]
  • Chosen: git reset --soft to the start commit. It keeps the work and puts the branch where the checks expect it.
  • Rejected: Refuse git commands in the step shell. Refused commands are the largest stop class, and feat(roadmap): run the implement agent shell in a real sandbox #2695 records the sandbox decision.
  • Rejected: Discard the work of the step. The work is valid; only the branch position is wrong.
• What happens when a task group changes no file?: The step records class plan defect for the group and returns it for a smaller replacement within the round budget. The stop names the group. The owner decision of 2026-11-02 applies: a plan defect stops at once unless the bounded amendment covers it. (best practice) [sources: src/devops_cli/roadmap/implement_step.py, tests/roadmap/test_roadmap_implement_step.py]
  • Chosen: Plan defect with a smaller replacement inside the budget. It follows the owner decision and keeps the work of the group.
  • Rejected: Stop the item at once. A group that is too large is not a false premise, and the bounded amendment covers it.
  • Rejected: Mark the group ticked. The plan did not do its work, and a tick would hide it.
• What does each attempt record, and what does a stop name?: The trace holds one entry for each attempt with its class, its evidence and the strategy of the round. A stop names the limit that ended it and its value, for example roadmap.implement.repair_rounds=3. (best practice) [sources: tests/roadmap/test_roadmap_implement_trace.py, src/devops_cli/roadmap/failure.py]
  • Chosen: One trace entry for each attempt; the stop names the limit and its value. The replay test can check both attempts, and a person sees the setting to change.
  • Rejected: One trace entry for the step. The second-attempt criterion needs both attempts in the trace.
  • Rejected: The stop names the class only. The criterion requires the limit that ended it.

Fit

Fits in one pull request.

Suspected block

None.

Dependencies

None.