Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/roadmap/alternatives.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,14 +74,16 @@ The check is off for a private repository and when no search provider is configu

An item is skipped, reported and recorded in its job record when its model call fails, when its run stops, or when the job refuses its reply. The job writes the other items. The run takes a skipped item again after `roadmap.alternatives.retry_days`.

The job writes the check of an item before it checks the next item. A kill of the worker loses at most the check in progress. A rate limit on a write ends the round, and the checks that are written stay.

The command exits 1 only when it tried items and checked none.

## Modes

| Flag | What the run does |
| :--- | :--- |
| `--plan` (the default) | Runs the checks and prints them. It writes nothing. |
| `--confirm` | Writes each check to GitHub: its comment, its body block and its job record. |
| `--confirm` | Writes each check to GitHub as soon as the item is checked: its comment, its body block and its job record. |
| `--dry-run` | Makes no request. It prints the requests that a run makes, with placeholders. |

## Where the rules are
Expand Down
4 changes: 3 additions & 1 deletion docs/roadmap/stage.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ For each item that a label filter names, the job does these things in order:
4. It opens a draft pull request. The title is `docs(openspec): plan #<issue> <summary>`. The body starts with `Plans #<issue>` and holds no closing keyword. `build_plan_body` of `pull_form` builds it, and a body that breaks the plan form goes back to the model once, as a refused plan does. It adds the `type/*` and `scope/*` labels of the title to the draft.
5. It writes one comment on the issue. The comment names the branch, the pull request, and the task groups with their measures. A label that GitHub refuses becomes a note in the comment.

The job writes each item when the item is planned, and it plans the next item after that. A stop of the Service loses at most the item in progress. Before the first plan, it removes the staged branches of closed items and writes the rejected plans of the items that the round does not select, because these need no model. It writes the rejection of a selected item after the plan of that item and before the push. It reads the paths that moved on the default branch for one item, just before its push.

The Status of the item stays Ready. The job does not mark the pull request ready. The Service does it, in its row `stage_ready`, when all checks of the draft have passed.

The overlap read is one GraphQL request for each 100 open pull requests. The context lists each pull request whose files meet a path that the item body cites, with its number, title, state, plan mark and shared paths, and each other file that holds a cited metric name. The prompt writes each entry into `## Dependencies`, and the gate refuses a plan with an empty `dependencies` field when the context lists an overlap. The comment of the staged item states the number of GraphQL requests.
Expand Down Expand Up @@ -93,7 +95,7 @@ Refine reads the comment as `stage_return`, because it is a comment of the stage
| Flag | What the run does |
| :--- | :--- |
| `--plan` (the default) | Reads GitHub and calls the model. It prints the changes that `--confirm` writes. It writes nothing. |
| `--confirm` | Plans as `--plan` does, then makes the writes in git and on GitHub. |
| `--confirm` | Plans as `--plan` does, and makes the writes in git and on GitHub for each item as soon as the item is planned. |
| `--dry-run` | Makes no request. It prints the requests and the git commands that a run makes, with placeholders. |

## Where the rules are
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-11-07
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Design

## Decisions

- **Two adapter classes, one in `stage.py` and one in `alternatives.py`.**: Each row writes a different set of artifacts, so a shared type would hold the union of both. `RefineWriter` is tied to `RefinedItem`, `RefineFailure` and `_apply_result`.
- **`plan_stage` and `plan_alternatives` take an optional writer.**: The loop holds the result of each item the moment it is done. A callback adds no thread and no second pass. With no writer the plan stays read-only, so `--plan` makes no write.
- **Stage order: unselected writes, then plan and write each selected item.**: The removals and the rejections of unselected items need no model, so they go before the first plan call. A stop then loses only the unfinished items.
- **A rejection of a selected item goes after its plan and before its publish.**: The flag `staged` of `_reject_plan` is then true only when the plan is ready, and the branch of the rejected plan is deleted before the new push.
- **The moved paths are read for one item, just before its publish.**: The read is git only. It sends no GitHub request, and it sees a commit that landed during the plans of the earlier items.
- **A stop of the Service and rejected credentials end a stage round.**: Both make every later request fail. The round records the failures of the finished items, then raises. Every other error is the failure of one item.
- **Alternatives: write check N before check N+1, and end on a rate limit.**: A kill loses only the check in progress. A rate limit blocks every later write, so the round must not spend model tokens on the other items.

## Rejected alternatives

- **One shared Protocol for both rows.**: `RefineWriter` is tied to its own types, so the shared type would carry the union of what stage and alternatives write.
- **Extend `RefineWriter` to all three rows.**: Stage writes publish, return, rejection and moved paths, and alternatives writes a body block, a record and a comment. `RefineWriter` holds none of these.
- **Keep the five-phase `apply_stage` after all plans.**: A stop loses every finished plan of the round. This is the defect.
- **Plan and write every item, including the unselected ones.**: Removals and rejections need no model, so a plan call for them wastes tokens and time.
- **End the round on every error.**: One error of one item would drop the finished items of the round.
- **End the round on no error.**: A stop or a refused identity would waste the rest of the request window.
- **Retry a rate-limited write until it succeeds.**: A rate limit blocks every later write, so the round cannot finish in the request window.
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Proposal

## Why

The stage row and the alternatives row of the Service plan every selected item first and write the results after. A stop of the Service, or a kill of the worker when the drain time ends, loses every finished plan of the round. A stage plan can use up to 1.5 million input tokens, so the next round pays for the same work again. #3631 removed this fault for the refine row only (#4034).

Two more faults come from the same order:

- `_plan_each` in `src/devops_cli/roadmap/stage/stage.py` does not catch `ServiceStoppingError`. The error leaves the loop and every finished plan is lost.
- `apply_alternatives` in `src/devops_cli/roadmap/alternatives.py` has no handler for `GitHubRateLimitError`. The round goes on to plan every item although every later write fails.

## What Changes

- The stage row plans one item, writes its result at once, and then plans the next item. The removals and the rejections of the items that the round does not select are written before the first plan call, because they need no model. A rejection of a selected item is written after its plan and before its publish. The moved paths are read for one item, just before its publish.
- A stop of the Service or rejected GitHub credentials end the round. The failures of the finished items are recorded first. Any other error is the failure of one item, and the round goes on.
- The alternatives row writes the body block, the record and the comment of check N before check N+1 starts. A write that fails with a rate limit ends the round and keeps the checks that are written.
- `plan_stage` and `plan_alternatives` take an optional writer, as `plan_refine` takes a `RefineWriter`. With no writer they write nothing, so `--plan` makes no GitHub write. `devops roadmap stage --confirm`, `devops roadmap alternatives --confirm` and the two rows use the same writers.
- The stage writer and the alternatives writer are two adapter classes, one in each module. The stage and alternatives specs gain requirements because the Service behavior changes.
- The change fits one pull request.

## Capabilities

### New Capabilities

None.

### Modified Capabilities

- `roadmap-stage`: A round writes each item when the item is done
- `roadmap-alternatives`: A round writes each check when the check is done

## Impact

- Code: `src/devops_cli/roadmap/stage/stage.py`, `src/devops_cli/roadmap/alternatives.py`, `src/devops_cli/roadmap/run.py`, `src/devops_cli/commands/roadmap.py`
- Tests: `tests/roadmap/stage/test_roadmap_stage_item_writes.py`, `tests/roadmap/test_roadmap_alternatives_item_writes.py`

## Feasibility

- I read the code at origin/main. `plan_stage` calls `_plan_each` over all selected targets. `_plan_each` catches `AICredentialsError` (and re-raises it), `AIClientError`, `RoadmapStageError` and `ToolExecutionError` only, so `ServiceStoppingError` leaves the loop. `apply_stage` runs five phases after all plans and calls `_moved_by_item` once for the plan. `_run_stage_adapter` in `src/devops_cli/roadmap/run.py` is `plan_stage` and then `apply_stage`.
- `plan_alternatives` loops `_check_one` over all items, and `apply_alternatives` writes after the loop. It has no handler for `GitHubRateLimitError`. `_run_alternatives_adapter` is `plan_alternatives` and then `apply_alternatives`.
- The pattern exists and is merged: `RefineWriter` in `src/devops_cli/roadmap/refine/refine.py` with `begin`, `settle`, `end` and `raise_for_failures`, and `plan_refine(writer=None)` (#3631). `RefineWriter` is tied to `RefinedItem`, `RefineFailure` and `_apply_result`, so it does not fit stage or alternatives. Each row needs its own adapter.
- The new tests ran against origin/main. The kill test, the stop test and the order test of the stage row fail there, because item 1 has no draft pull request until `apply_stage` runs, and because the first plan call comes before the first removal. The kill test and the rate-limit test of the alternatives row fail there, because check 1 is not written when check 3 is killed, and because the round asks the model about all five items after the write failed. `devops ci fail-to-pass --base origin/main` runs them.
- Three tests are guards that pass on origin/main: the plan-only test of each row, and the test of the rejection of a selected item. They keep the old behavior (a plan makes no write, and the rejection comes between the plan and the publish) while the order changes.
- No merged change and no open pull request does this work, and no ADR or newer issue replaces it.

## Existing Tool Considered

Existing project code. No open library covers the write order of two roadmap rows. The nearest code is `RefineWriter` (#3631). The change follows its pattern with two adapter classes, and it deletes the five-phase `apply_stage` as a batch over a whole plan and the batch loop of `apply_alternatives`. The written cost is about 80k tokens and 3.5 hours to build with the tests, about 3k tokens a quarter to keep, and about 20k tokens and 1 hour for each fault.

## Dependencies

None.
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
## ADDED Requirements

### Requirement: An alternatives round writes each check when the check is done
An alternatives round SHALL write the body block, the record and the comment of check N before check N+1 starts. A kill of the worker SHALL lose at most the check in progress.

#### Scenario: Kill in item 3 of 5
- **WHEN** a round selects 5 items and the model call of the third item raises a `BaseException` that no handler catches
- **THEN** checks 1 and 2 have their body block, record and comment, and the other items have none

### Requirement: A rate limit ends an alternatives round and keeps the written checks
An alternatives round SHALL end when a write fails with a GitHub rate limit. It SHALL keep the checks that it wrote and SHALL NOT check the items after the failed one.

#### Scenario: Rate limit in check 2
- **WHEN** the write of the body of item 2 of 5 fails with a rate-limit error
- **THEN** the round raises the error, item 1 keeps its written check, and the model gets no request for items 3, 4 and 5

### Requirement: An alternatives plan writes nothing
`devops roadmap alternatives --plan` SHALL make no write to the roadmap store.

#### Scenario: Plan only
- **WHEN** the command runs with `--plan` on 3 items that have no check
- **THEN** the store receives no write
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
## ADDED Requirements

### Requirement: A stage round writes each item when the item is done
A stage round SHALL plan one selected item, write its result, and only then plan the next item. The result is the publish of a ready change, the return of a blocked item, or the stop comment of an item over the budget. A kill of the worker SHALL lose at most the item in progress.

#### Scenario: Kill between item 1 and item 2
- **WHEN** a round selects 3 items and the model call of item 2 raises a `BaseException` that no handler catches
- **THEN** item 1 has its pushed branch, its draft pull request, its record and its comment, and items 2 and 3 have no branch, no record and no comment

### Requirement: A stop ends a stage round after it records the finished items
A stop of the Service or rejected GitHub credentials SHALL end a stage round. The round SHALL record the failure of each item that failed before the stop, and SHALL keep the written result of each finished item. Any other error SHALL be the failure of one item, and the round SHALL go on.

#### Scenario: Stop of the Service in item 3
- **WHEN** item 1 is staged, the plan of item 2 fails, and the Service stops in the plan of item 3 of 4 items
- **THEN** item 1 stays staged, item 2 has a failure record, items 3 and 4 have no record and no comment, and the round ends with the stop

### Requirement: A stage round writes the unselected items first
A stage round SHALL write the removals of closed items and the rejections of items that it does not select before its first plan call. A rejection of a selected item SHALL be written after the plan of that item and before its publish. The round SHALL read the moved paths of an item just before it publishes that item.

#### Scenario: Removal and rejection of unselected items
- **WHEN** a round has a closed item with a staged branch, an item with a rejected plan that the round leaves out, and one item to plan
- **THEN** the removal and the rejection are written before the first plan call

#### Scenario: Rejection of a selected item
- **WHEN** a person names an item whose plan pull request was closed with no merge
- **THEN** the rejection is written after the plan of the item and before its publish, the staged branch of the rejected plan is deleted before the new push, and the Status of the item stays Ready

#### Scenario: A commit lands during the plan of item 2
- **WHEN** a commit that changes a path named by both plans lands on the default branch during the plan of item 2
- **THEN** the comment of item 1 lists no moved path and the comment of item 2 lists the path

### Requirement: A stage plan writes nothing
`devops roadmap stage --plan` SHALL make no write to GitHub or to the remote repository, also when the round holds a removal or a rejection.

#### Scenario: Plan only
- **WHEN** the command runs with `--plan` on a roadmap that holds a removal, a rejection and a ready item
- **THEN** the store receives no write, no branch is pushed or deleted, and no GitHub write request is sent
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Tasks

## 1. The stage row writes each item when that item is done

Kind: behavior.

Reads: `src/devops_cli/roadmap/stage/stage.py`, `src/devops_cli/roadmap/run.py`, `src/devops_cli/commands/roadmap.py`, `src/devops_cli/roadmap/refine/refine.py`. Changes: `src/devops_cli/roadmap/stage/stage.py`, `src/devops_cli/roadmap/run.py`, `src/devops_cli/commands/roadmap.py`. Creates: `tests/roadmap/stage/test_roadmap_stage_item_writes.py`.

- [x] 1.1 In `tests/roadmap/stage/test_roadmap_stage_item_writes.py`, write the cases: a kill between item 1 and item 2, a stop of the Service in item 3 with the failure of item 2 recorded, the order of the removals, the rejections, the plan, the publish and the moved paths, and a plan that writes nothing
- Verification: `$ uv run devops ci test tests/roadmap/stage/test_roadmap_stage_item_writes.py` fails on the base
- [x] 1.2 In `src/devops_cli/roadmap/stage/stage.py`, add the stage writer with the phase order of the design and the optional `writer` of `plan_stage`. Delete the five-phase batch of `apply_stage` and the single call of `_moved_by_item` for a whole plan. In `src/devops_cli/roadmap/run.py` and `src/devops_cli/commands/roadmap.py`, run `--confirm` and the row with the writer, and `--plan` with none
- Verification: `$ uv run devops ci test tests/roadmap/stage/test_roadmap_stage_item_writes.py` passes
- [x] 1.3 Keep the other stage tests green with the new order
- Verification: `$ uv run devops ci test tests/roadmap/stage/test_roadmap_stage_apply.py`

## 2. The alternatives row writes each check when that check is done

Kind: behavior.

Reads: `src/devops_cli/roadmap/alternatives.py`, `src/devops_cli/roadmap/run.py`, `src/devops_cli/commands/roadmap.py`. Changes: `src/devops_cli/roadmap/alternatives.py`, `src/devops_cli/roadmap/run.py`, `src/devops_cli/commands/roadmap.py`. Creates: `tests/roadmap/test_roadmap_alternatives_item_writes.py`.

- [x] 2.1 In `tests/roadmap/test_roadmap_alternatives_item_writes.py`, write the cases: a kill in item 3 of 5, a rate limit in the write of check 2, and a plan that writes nothing
- Verification: `$ uv run devops ci test tests/roadmap/test_roadmap_alternatives_item_writes.py` fails on the base
- [x] 2.2 In `src/devops_cli/roadmap/alternatives.py`, add the alternatives writer and the optional `writer` of `plan_alternatives`, end the round on a rate limit, and delete the batch loop of `apply_alternatives`. In `src/devops_cli/roadmap/run.py` and `src/devops_cli/commands/roadmap.py`, run `--confirm` and the row with the writer, and `--plan` with none
- Verification: `$ uv run devops ci test tests/roadmap/test_roadmap_alternatives_item_writes.py` passes
- [x] 2.3 Keep the other alternatives tests green with the new order
- Verification: `$ uv run devops ci test tests/roadmap/test_roadmap_alternatives.py`

## 3. The documents match the new order

Kind: documents.

Changes: the documents that state "plan all, then write" for the stage and alternatives rows, if any.

- [x] 3.1 Fix each document that says the stage row or the alternatives row writes after it plans all items
- Verification: `$ uv run devops ci docs`

## Criteria

- Group 1: A stage round SHALL write the result of each item before it plans the next item.
- Group 1: A stop of the Service SHALL end a stage round after it records the failures of the finished items.
- Group 1: A stage round SHALL write the removals and the rejections of unselected items before its first plan call.
- Group 1: A stage plan SHALL write nothing.
- Group 2: An alternatives round SHALL write the check of an item before it checks the next item.
- Group 2: A rate limit SHALL end an alternatives round and keep the written checks.
- Group 2: An alternatives plan SHALL write nothing.

## Workflow follow-up

- Pending a person: review the diff in tuicr (tuicr -r <base>..HEAD).
Loading
Loading