Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 0 additions & 59 deletions openspec/changes/3308-the-plan-check-compares-a-number/tasks.md

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@
- **Carry the constants in `CaseFacts` with an empty default, and pass them through `PlanUnderTest` to `check_proposal`.**: The recorded cases keep no text of a file, so an empty mapping leaves them as they are and a later case can record the values it needs.
- **Name the constant, the plan's number and the staged value in one problem line.**: The model corrects the number in one turn, as `SUMMARY_MAX_CHARS is 60 in the staged tree, but the plan says 40` shows.
- **Add the citation rule to the plan stage prompt beside the rule that asks for the quoted probe lines.**: The model has the read tools, and it writes the plan before the check runs, so the rule makes the first reply pass.
- **Keep the check in a new module `src/devops_cli/roadmap/stage_constants.py` and call it from `check_proposal`.**: `check_proposal` is the one seam that the stage job and the replay both call, so the rule runs in both paths.
- **Keep the check in a new module `src/devops_cli/roadmap/stage/stage_constants.py` and call it from `check_proposal`.**: `check_proposal` is the one seam that the stage job and the replay both call, so the rule runs in both paths.

## Rejected alternatives

- **Import the staged modules to read the constant values.**: `src/devops_cli/roadmap/stage_probe.py` reads the commit with `git cat-file`, `git grep` and `git log` only, so the check runs no staged code.
- **Import the staged modules to read the constant values.**: `src/devops_cli/roadmap/stage/stage_probe.py` reads the commit with `git cat-file`, `git grep` and `git log` only, so the check runs no staged code.
- **Find the pairs with a regular expression over the source text.**: `src/devops_cli/roadmap/pull_form.py` holds the docstring `Its pattern `^.{1,60}$` runs in JavaScript` and the assignment `BODY_MAX_CHARS: Final[int] = CONST_ROADMAP_IMPLEMENT_BODY_MAX_CHARS`, so a regex over text reads 60 from a docstring and finds no literal for an aliased constant.
- **Read every Python file of the staged tree, with no `git grep` first.**: The tree holds hundreds of files, and a plan cites a few names. `git grep` gives the files to parse and keeps the read inside the command budget.
- **Scan every file of the change folder, `tasks.md` included.**: `src/devops_cli/roadmap/stage_change.py` renders `tasks.md` from `group_text`, where a verify command holds a test path and a number that is no claim about a constant.
- **Scan every file of the change folder, `tasks.md` included.**: `src/devops_cli/roadmap/stage/stage_change.py` renders `tasks.md` from `group_text`, where a verify command holds a test path and a number that is no claim about a constant.
- **Pair every number in a sentence with every constant in it.**: A sentence that says `SUMMARY_MAX_CHARS` is 60, while the tree says 47, and the issue says 40 is a counterexample and not a claim. The strict rule skips it.
- **Refuse a plan that names a constant which the staged tree does not define.**: The plan folder `openspec/changes/3200-refuse-a-plan-with-placeholder-or/design.md` names `PLAN_SUMMARY_MAX_CHARS = 47`, which `src/` does not define, so a refusal would stop a valid plan.
- **Read the constants through `StagedTree`.**: The class holds file names only, and every test that builds a tree would need the source and the commit. The explicit mapping leaves the tree and those tests alone.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@ A sentence that holds more than one number is ambiguous: it can show an example,

## What Changes

- Add `src/devops_cli/roadmap/stage_constants.py`:
- Add `src/devops_cli/roadmap/stage/stage_constants.py`:
- `candidate_names(texts)` reads the constant names that the plan text cites.
- `staged_constants(source, sha, names)` finds the definition of each name with `git grep` at the staging commit, reads only those files with `git show`, parses them with `ast`, and resolves an alias across modules. It imports and runs no staged code.
- `constant_problems(texts, constants)` returns one problem for each sentence that names one known constant and holds one candidate number that differs.
- Add `plan_texts(proposal)` to `src/devops_cli/roadmap/stage_change.py`: the plan fields that render to `proposal.md`, `design.md` and the spec deltas. It leaves out every task group, so `tasks.md` is never scanned.
- Add a `constants` keyword to `check_proposal` in `stage_change.py`, and pass the mapping that `Stage._settle` reads with `staged_constants` from `self._source` and `self._sha`.
- Add `plan_texts(proposal)` to `src/devops_cli/roadmap/stage/stage_change.py`: the plan fields that render to `proposal.md`, `design.md` and the spec deltas. It leaves out every task group, so `tasks.md` is never scanned.
- Add a `constants` keyword to `check_proposal` in `src/devops_cli/roadmap/stage/stage_change.py`, and pass the mapping that `Stage._settle` reads with `staged_constants` from `self._source` and `self._sha`.
- Carry recorded constants through the stage eval replay: `CaseFacts` gets an optional `constants` field, and `PlanUnderTest` passes it to `check_proposal`. A case with no field keeps an empty mapping, so the recorded cases do not change.
- Add the message that names the constant, the plan's number and the staged value to `src/devops_cli/lang/en/messages.py`.
- Add the citation rule to `src/devops_cli/ai/tasks/roadmap_stage_change.md`: cite the staged value of each constant that the plan relies on, and do not give a new name to the bound of a constant that the staged tree defines.
Expand All @@ -32,21 +32,21 @@ None.

## Impact

- Code: `src/devops_cli/roadmap/stage_constants.py`, `src/devops_cli/roadmap/stage_change.py`, `src/devops_cli/roadmap/stage.py`, `src/devops_cli/roadmap/stage_eval_cases.py`, `src/devops_cli/roadmap/stage_eval_gates.py`, `src/devops_cli/lang/en/messages.py`, `openspec/changes/3308-the-plan-check-compares-a-number/.openspec.yaml`
- Documents: `src/devops_cli/ai/tasks/roadmap_stage_change.md`, `openspec/changes/3308-the-plan-check-compares-a-number/proposal.md`, `openspec/changes/3308-the-plan-check-compares-a-number/design.md`, `openspec/changes/3308-the-plan-check-compares-a-number/tasks.md`, `openspec/changes/3308-the-plan-check-compares-a-number/specs/roadmap-stage/spec.md`
- Tests: `tests/roadmap/test_roadmap_stage_constants.py`, `tests/roadmap/test_roadmap_stage_plan.py`, `tests/roadmap/test_roadmap_stage_prompt_rules.py`, `tests/roadmap/test_roadmap_stage_eval_gates.py`
- Code: `src/devops_cli/roadmap/stage/stage_constants.py`, `src/devops_cli/roadmap/stage/stage_change.py`, `src/devops_cli/roadmap/stage/stage.py`, `src/devops_cli/roadmap/stage/stage_eval_cases.py`, `src/devops_cli/roadmap/stage/stage_eval_gates.py`, `src/devops_cli/lang/en/messages.py`
- Documents: `src/devops_cli/ai/tasks/roadmap_stage_change.md`
- Tests: `tests/roadmap/stage/test_roadmap_stage_constants.py`, `tests/roadmap/stage/test_roadmap_stage_plan.py`, `tests/roadmap/stage/test_roadmap_stage_prompt_rules.py`, `tests/roadmap/stage/test_roadmap_stage_eval_gates.py`

## Feasibility

- `read_file` on `src/devops_cli/roadmap/pull_form.py` showed `SUMMARY_MAX_CHARS: Final[int] = 60`, and `src/devops_cli/roadmap/stage_change.py` showed `STAGE_SUMMARY_MAX_CHARS: Final[int] = SUMMARY_MAX_CHARS - summary_length(`, so the premise holds: the plan of the pull request for #3200 claimed 40.
- The probes give `The path `src/devops_cli/roadmap/stage_change.py` is a file of the staging commit.`, and that file holds `def check_proposal(` with `problems.extend(path_problems(proposal.groups, tree))`, so the plan check has one place for the new rule.
- The probes give `The path `src/devops_cli/roadmap/stage_paths.py` is a file of the staging commit.`, and `class StagedTree` holds `self._files = frozenset(files)` and no text of a file. The content therefore comes from `Stage._settle`, which holds `self._source` and `self._sha` and calls `check_proposal(`, and not from the tree.
- The probes give `The path `src/devops_cli/roadmap/stage_probe.py` is a file of the staging commit.`, and that file holds `probe_premise(source: Path, sha: str, body: str)`, the read-only probe that the new reader follows with `git grep` and `git show`.
- `src/devops_cli/roadmap/stage_change.py` holds `def render_change(` and `def _spec_file(`, so the plan fields that render to the change files are known and the check can read them without the change folder name.
- `src/devops_cli/roadmap/stage_eval_cases.py` holds `class CaseFacts` with `tracked` and `capabilities`, so the replay holds no file text. An optional `constants` field keeps every recorded case valid and lets a later case record the values.
- `src/devops_cli/roadmap/stage/stage_change.py` holds `def render_change(` and `def _spec_file(`, so the plan fields that render to the change files are known and the check can read them without the change folder name.
- `src/devops_cli/roadmap/stage/stage_eval_cases.py` holds `class CaseFacts` with `tracked` and `capabilities`, so the replay holds no file text. An optional `constants` field keeps every recorded case valid and lets a later case record the values.
- The plan of the pull request for #3260 said `StageProposal.summary` takes the bound `PLAN_SUMMARY_MAX_CHARS = 47`, and that the issue text says 40 and the schema says 60. The staged tree defines no `PLAN_SUMMARY_MAX_CHARS`, so the check skips that name, as the design decided to avoid a false refusal. The check refuses the explicit claim that the issue records, `SUMMARY_MAX_CHARS` beside 40, and the prompt rule makes the model state the staged 60.
- The probes give `The path `src/devops_cli/ai/tasks/roadmap_stage_change.md` is a file of the staging commit.`, so the prompt rule goes into a file of the commit.
- First checks against the real system: `uv run devops ci test tests/roadmap/test_roadmap_stage_constants.py`, then `uv run devops openspec check`, then `uv run devops roadmap eval stage --dry-run` to see the gate on the golden set.
- First checks against the real system: `uv run devops ci test tests/roadmap/stage/test_roadmap_stage_constants.py`, then `uv run devops openspec check`, then `uv run devops roadmap eval stage --dry-run` to see the gate on the golden set.

## Existing Tool Considered

Expand Down
Loading
Loading