feat(roadmap): keep the gated stage plan as a file #4064

Problem

A failed run drops the gated plan. The item then waits for the retry time. The next run explores from the first tool call again. One plan used up to 1.5 million input tokens (#3880, closed).

devops roadmap stage --plan prints text and writes nothing. --confirm plans again. So a person who reads a plan gets another plan.

The item record cannot hold the gated plan. It holds marks only (stage.sha, stage.groups, stage.text_hash). A plan is too large for a mark.

Measure, taken on the live Service on 2026-11-07 with uv run devops prometheus query 'sum by (result) (increase(devops_cli_roadmap_stage_total[7d]))':

• failed: 388 items in 7 days.
• staged: 116, returned: 34.

The 388 is an upper bound. The result failed also counts a failed write, and a plan staged with an open review finding (stage_results). Both keep a plan. The metric cannot tell them apart from a failed planner run. The AI token metrics have no task label, so the mean input tokens of a planner run are not measurable today.

So the hit rate of a stored plan is unmeasured. This issue makes it measurable in the same pull request, with a counter of hits and misses. If the counter shows no hit after 14 days, a person decides to remove the file.

Expected

Add a file record StagePlanRecord in a new module roadmap/stage/stage_plan_record.py. It holds the gated StagedChange, the findings and the StageRequest of the issue #4046.

• Write the file atomically with write_json_file(..., atomic=True), as IntakeRecord is written.
• Raise an error that names the file when it cannot be read.
• The key is the sha256 of five values:
  • the item number;
  • the hash of the title and body (body_hashes);
  • the model name of the task stage;
  • the toolset name;
  • the prompt version.
• The prompt version is the sha256 of the text of the prompts that the planner reads. These are roadmap_stage_change.md, roadmap_stage_split.md and the tools prompt of the toolset (load_task_prompt). The code has no prompt version today, so this issue defines it as that hash.
• The staged sha is not part of the key, because main moves several times a day. The sha stays inside the record.
• A hit skips the planner model call. The gates and the review run again on the current sha, from the stored StageRequest.
• The directory is roadmap/stage/<owner>/<repo>/ under the data directory (resolve_data_path(load_settings().data.dir), the path that service_state_dir uses).
• Delete the file when the item is staged or when its key changes.

--plan then --confirm:

• --plan writes the file and makes no GitHub write.
• --confirm reads the file by the same key and sends the planner no request.
• Today the help of --plan says that it writes nothing. Change that text in lang/en/help.py (stage_plan) and in docs/roadmap/stage.md (line 95). Say that --plan writes one local file and no GitHub write.

Counter. Add devops_cli_roadmap_stage_plan_record_total with the label result (hit, miss, written). Document it in docs/TELEMETRY.md. It has 3 series.

Not in this issue: a file of the refine proposal. File that only if the measured rate of stops between investigation and apply is above 1 refine item in 20 in one week. #3631 (closed) writes each refine item when it is done, so a stop loses at most one item.

Acceptance criteria

• uv run pytest tests/roadmap/stage/test_roadmap_stage_plan_record.py -k review_fails - a failed review run and a rerun make zero planner requests on the rerun.
• uv run pytest tests/roadmap/stage/test_roadmap_stage_plan_record.py -k unreadable - a corrupt file raises an error that names the file.
• uv run pytest tests/roadmap/stage/test_roadmap_stage_plan_record.py -k key - a changed item body gives a new key. A changed prompt text gives a new key. A moved staged sha keeps the key and runs the gates again.
• uv run pytest tests/roadmap/stage/test_roadmap_stage_plan_record.py -k plan_then_confirm - --plan then --confirm on a stub model send the planner one request in total. --plan makes no GitHub write.
• uv run pytest tests/roadmap/stage/test_roadmap_stage_plan_record.py -k delete - a staged item removes its file.
• uv run pytest tests/roadmap/stage/test_roadmap_stage_plan_record.py -k counter - a hit, a miss and a write each add 1 to the counter with their result.
• uv run pytest tests/roadmap/test_roadmap_command_help.py -k stage_plan_help - the help of --plan says that it writes one local file and no GitHub write.

Every command above fails on the unchanged code. The pull request cites the fail-to-pass run (devops ci fail-to-pass --base <ref>).

Feasibility

I read these files at origin/main: stage.py (_Planner._request, _settle, _ask, stage_results), run.py (service_state_dir, read_intake_record, _write_intake_record), the prompt files under ai/tasks, help.py (stage_plan) and docs/roadmap/stage.md.

I ran the measure above. The numbers are in the Problem section. I did not run the Service or any test.

What is not proved: the hit rate. The upper bound is large (388 against 116 staged), but it is not the rate of failed planner runs. The counter in this issue gives the real rate.

The change makes no GitHub request. The earlier idea of a key by the staged sha is dropped. It rarely hits.

Existing tool considered

The atomic writer of IntakeRecord (write_json_file(..., atomic=True)) and the hash helpers of body_hash.py. Argo artifacts map to these files one to one later (ADR 0008). No library is needed.

Cost

Build: about 70k tokens and 3.5 h (file record, key, --plan contract, help and docs, counter, 8 tests). Keep: about 4k tokens per quarter. Fix: about 20k tokens and 1.5 h.

Related

Part of #3027. Depends on the issue #4046. #3105 and #3677 (implement only), #3880 (closed), #3529. Order: after the issues that change stage.py before it.