Skip to content

fix(roadmap): Refuse criteria entries not in the issue body #3391

New issue
New issue
Open
Open
fix(roadmap): Refuse criteria entries not in the issue body#3391
Labels
effort/mediumEffort Mediumpriority/p2-mediumStandard priority feature work and architectural improvementsscope/roadmapThe roadmap jobs: intake, refine, stage, implement and the Service that runs themsource/agentFiled by an agentstatus/blockedExecution blocked by external dependency or upstream reviewtype/bugDefect, regression, or incorrect runtime behaviorvalue/mediumValue Medium

Description

@dan-petty
dan-petty
opened on Nov 3, 2026

Problem or Motivation

The stage job accepts a plan whose ## Criteria section holds an acceptance criterion that the issue does not hold. The plan for #3320 in PR #3333 shows it. Issue #3320 has no Acceptance criteria heading in the part that a person wrote. Its only list of criteria sits in the devops-roadmap-refine block. The prompt src/devops_cli/ai/tasks/roadmap_stage_change.md says to give one entry for each item of the person-written list and none for the list in the refine block, and an issue with no such list gets an empty list.

The plan copied the five requirements of the refine block into ## Criteria under groups 1, 1, 2, 2 and 3. criterion_problems in src/devops_cli/roadmap/stage_criteria.py refuses only a missing criterion and a group number over the group count. It accepts an entry that no person wrote. The implement job reads this section, so it verifies steps against criteria that the model invented, and it hides that the person gave no acceptance criteria.

Proposed Solution & Architectural Design

  1. In criterion_problems, refuse an entry whose text is not one of acceptance_criteria(issue_body). The function already cuts off the refine block, so the check reads only the person-written part.
  2. Add a test in tests/roadmap/test_roadmap_stage_criteria.py for a body with no person-written heading and a reply whose criteria holds an entry: the check names the entry.
  3. Add a test for a body whose only list is in the refine block: the check accepts an empty criteria list and refuses a copied entry.
  4. State in src/devops_cli/ai/tasks/roadmap_stage_change.md that a plan whose issue has no person-written list gives an empty criteria list, so the returned plan tells the model the same rule.

Target Subsystem

src/devops_cli/roadmap/stage_criteria.py (criterion_problems), src/devops_cli/ai/tasks/roadmap_stage_change.md, tests/roadmap/test_roadmap_stage_criteria.py.

Existing Tool Considered

No open, maintained tool covers the need. The rule compares a reply with the text of one issue of this repository, so it lives in criterion_problems. The change deletes no internal code: acceptance_criteria and criterion_problems stay and gain one comparison and one problem text.

Source: #3333

Proposed design

Problem statement

The stage criteria check accepts an entry in a plan's ## Criteria section even when the issue has no person-written criterion with that text. The plan for #3320 copied criteria from the refine block. The implement job then verifies steps against criteria that the model invented. The prompt already states the empty-list rule, and the check does not enforce it. The change makes the check refuse any entry that is not one of the person-written criteria.

Acceptance criteria

Requirement: Refuse an entry that no person wrote

criterion_problems SHALL return a problem for each criteria entry that is not a person-written criterion of the issue.

Scenario: An entry copies a refine-block criterion

  • WHEN the issue body has no person-written Acceptance criteria heading and the reply holds one criterion from the refine block
  • THEN criterion_problems returns one problem that quotes the copied criterion.

Requirement: Accept an empty list when no person wrote a criterion

criterion_problems MUST return no problem for an empty criteria list when the issue has no person-written acceptance criteria.

Scenario: An issue has no person-written list

  • WHEN the issue body has no Acceptance criteria heading outside the refine block and the reply has an empty criteria list
  • THEN criterion_problems returns an empty problem list.

Requirement: Refuse a copied refine-block entry

criterion_problems SHALL refuse a criteria entry copied from the refine block when that block is the only acceptance criteria list.

Scenario: The refine block is the only list

  • WHEN the issue body has an Acceptance criteria heading only inside the refine block and the reply copies one entry
  • THEN criterion_problems returns a problem that names the copied entry.

Requirement: Test the two issue bodies

The stage criteria test module MUST cover a body with no person-written heading and a body with only a refine-block list.

Scenario: The two issue bodies have tests

  • WHEN the stage criteria test module runs
  • THEN the tests for the no-heading body and the refine-only body pass.

Requirement: State the empty-list rule in the stage prompt

The stage change prompt SHALL state that an issue with no person-written acceptance criteria list gets an empty criteria list.

Scenario: The prompt states the empty-list rule

  • WHEN a test reads src/devops_cli/ai/tasks/roadmap_stage_change.md
  • THEN the file contains the rule that an issue with no person-written list gets an empty criteria list.

Existing Tool Considered

No alternatives check has run. No open, maintained tool covers the need. The rule compares a reply with the text of one issue in this repository, and it belongs in criterion_problems. The change adds one comparison and one problem text to existing code. The written cost is about 3k tokens to build, keep and fix the code, and under 1 hour.

Key questions

  • Does acceptance_criteria read only the person-written part of the issue body?: Yes. acceptance_criteria calls _without_refine_block and reads only the text before and after the refine block. A body with no Acceptance criteria heading outside that block returns an empty list. [sources: src/devops_cli/roadmap/stage_criteria.py]
  • Should criterion_problems refuse an entry that no person wrote?: Yes. The check must enforce the prompt rule. An entry that is not in acceptance_criteria(issue_body) gets a problem. (best practice) [sources: src/devops_cli/roadmap/stage_criteria.py, src/devops_cli/ai/tasks/roadmap_stage_change.md]
    • Chosen: Reject unknown entries. The stage prompt says to copy only person-written criteria. The check must enforce that rule.
    • Rejected: Accept unknown entries. The implement job then verifies invented criteria and hides that the person gave no criteria.
    • Rejected: Reject only refine-block text. The check cannot tell every invented text from refine-block text. It must use the person-written list as the only source.
  • How should the check compare an entry with a person-written criterion?: Use _one_line on the entry and compare it with the set from acceptance_criteria. The comparison is exact after whitespace normalization. (best practice) [sources: src/devops_cli/roadmap/stage_criteria.py]
    • Chosen: Normalized exact match. The existing check normalizes spacing with _one_line, and the prompt requires copied text.
    • Rejected: Raw text match. Whitespace changes would refuse a valid copy.
    • Rejected: Substring match. A partial text could match a different criterion and weaken the check.
  • Where should the new problem text live?: Add a new message under MESSAGES.roadmap and use it in criterion_problems. The message names the entry and says that it is not a person-written criterion. (best practice) [sources: src/devops_cli/roadmap/stage_criteria.py]
    • Chosen: New localized message. The module already reads problem texts from MESSAGES.roadmap, and tests assert the exact text.
    • Rejected: Inline string. It breaks the existing message pattern and makes the text harder to keep.
    • Rejected: Reuse the group message. That message names a group count, not an unknown criterion.
  • What should the stage prompt state for an issue with no person-written list?: The prompt already says that an issue with no such list gets an empty criteria list. Keep that sentence and add a test that reads it. No prompt edit is needed. (best practice) [sources: src/devops_cli/ai/tasks/roadmap_stage_change.md]
    • Chosen: Keep the existing empty-list rule. The prompt already has the rule, and a test can check it.
    • Rejected: Add a second statement. It would duplicate the rule and add no behavior.
    • Rejected: Remove the rule. The model then may copy refine-block entries.
  • Which tests should cover the new rule?: Add two tests in tests/roadmap/test_roadmap_stage_criteria.py. One test uses a body with no person-written heading and an entry. The other uses a body whose only list is in the refine block, accepts an empty list and refuses a copied entry. (best practice) [sources: tests/roadmap/test_roadmap_stage_criteria.py, src/devops_cli/roadmap/stage_criteria.py]
    • Chosen: Two focused tests. They match the two failure cases in the issue and assert the problem text.
    • Rejected: One broad test. It would hide which case failed.
    • Rejected: No new test. The check would have no regression guard.

Fit

Fits in one pull request.

Suspected block

None.

Dependencies

None.

Alternatives check of 2026-11-06: none.
Tool: none found.

Activity

  1. dan-petty
    added
    type/bugDefect, regression, or incorrect runtime behavior
    priority/p2-mediumStandard priority feature work and architectural improvements
    source/agentFiled by an agent
    scope/roadmapThe roadmap jobs: intake, refine, stage, implement and the Service that runs them
    status/newStatus New: the item is not ready yet
    value/mediumValue Medium
    effort/mediumEffort Medium
    on Nov 3, 2026
  2. devops-cli-service
    added
    status/readyStatus Ready: problem, acceptance criteria and an answer to each key question
    and removed
    status/newStatus New: the item is not ready yet
    on Nov 4, 2026

  3. devops-cli-service commented on Nov 4, 2026

    @devops-cli-service
    devops-cli-serviceBot
    on Nov 4, 2026
    Contributor

    Status set to Ready at 46fc42e (see proposed design in issue body).

  4. dan-petty
    changed the title [-]fix(roadmap): the stage criteria check refuses an entry the issue does not hold[/-] [+]fix(roadmap): Refuse criteria entries not in the issue body[/+] on Nov 6, 2026

  5. dan-petty commented on Nov 6, 2026

    @dan-petty
    dan-petty
    on Nov 6, 2026
    OwnerAuthor

    The job shortened the title of this item. The summary had 65 characters, and the limit is 60. The type and the scope stay.

    • Before: fix(roadmap): the stage criteria check refuses an entry the issue does not hold
    • After: fix(roadmap): Refuse criteria entries not in the issue body
    • Source of the summary: the background model

  6. devops-cli-service commented on Nov 6, 2026

    @devops-cli-service
    devops-cli-serviceBot
    on Nov 6, 2026
    Contributor

    Alternatives check: none

    Tool: none found

    Lacks:

    • No open, maintained tool checks a model's criteria reply against the person-written acceptance criteria of one issue in this repository; the rule belongs in criterion_problems next to acceptance_criteria.
    • The check must cut off the devops-roadmap-refine block, so criteria copied from that block are refused and an issue with no person-written list accepts an empty criteria list — a repo-specific reading of one issue body, not a general-purpose function.
    • The work is a small in-repo change: one comparison against acceptance_criteria(issue_body), one new problem text under MESSAGES.roadmap, and two focused tests in tests/roadmap/test_roadmap_stage_criteria.py.
    • The stage prompt rule (an issue with no person-written list gets an empty criteria list) is already stated in src/devops_cli/ai/tasks/roadmap_stage_change.md; only a test that reads it is added.
    • The repository ADRs name no tool for this need: they decide workflow, label, transport and CI tooling, and none concerns validating a reply against an issue body.

    Searches run:

    • library validate acceptance criteria list against issue body text
    • markdown section parser extract acceptance criteria heading python tool
    • python library parse markdown issue body acceptance criteria section
    • markdown-it-py extract section by heading
    • markdown section extraction python
    • existing tool validate roadmap acceptance criteria

    Checked on 2026-11-06. A person decides what this means for the item. The check changes no label and closes no issue.

  7. devops-cli-service commented on Nov 7, 2026

    @devops-cli-service
    devops-cli-serviceBot
    on Nov 7, 2026
    Contributor

    Staged: branch fix/3391-refuse-criteria-entries-not-in-the, draft pull request #4069. The pull request plans the item and closes no issue.

    The task groups, with the tokens of the inputs of each step, for a budget of 64000 tokens:

    1. Give the replay's paths gate the issue body: 26137 tokens
    2. Refuse a criteria entry that no person wrote: 30571 tokens
    3. State the refusal in the stage prompt: 25065 tokens

    A model review of the plan found these points. They stop no draft, and a person judges them:

    • false_premise at feasibility: The plan says 3411 is the only golden case with a criteria list and that the new rule refuses no recorded plan, but 3174's plan.json also holds a criteria list (empty), and the replay's _path_gate calls check_proposal without issue_body, so the new rule would refuse 3411's entry there. Evidence: ""criteria": []"
    • task_order at group 1: Group 1 adds the unknown-entry rule to criterion_problems, which the replay's paths gate also calls with no issue body; the existing replay test that asserts a mapped plan is not refused fails until group 2 passes plan.issue_body. Group 1 therefore needs group 2's result, and the groups should be reordered or merged. Evidence: "assert not replay_case(_with_criteria(good, both)).refused"
    • false_premise at feasibility: The plan says the known defect of case 3174 is a placeholder group at the content gate, but the case file pins that defect to no gate; the plan's reasoning about the replay's pinned counts rests on a wrong fact. Evidence: ""gate": "none""

    The runs of the planner and the review made 237 model request(s), and used 16299016 input tokens and 227273 output tokens. The tool calls, in order:

    1. read_file({"path":"src/devops_cli/roadmap/stage/stage_criteria.py"})
    2. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_criteria.py"})
    3. search_code({"text":"stage_problem_criterion"})
    4. search_code({"text":"an issue with no such list gets an empty list"})
    5. read_file({"path":"src/devops_cli/lang/en/messages.py","start_line":3595,"end_line":3640})
    6. read_file({"path":"src/devops_cli/ai/tasks/roadmap_stage_change.md","start_line":38,"end_line":60})
    7. search_code({"text":"criterion_problems("})
    8. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py"})
    9. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":975,"end_line":1010})
    10. read_file({"path":"src/devops_cli/roadmap/stage/stage.py","start_line":1140,"end_line":1180})
    11. read_spec({"capability":"roadmap-stage","requirement":"An unmapped criterion is refused"})
    12. read_spec({"capability":"roadmap-stage","requirement":"Criteria mapping in the stage reply"})
    13. search_code({"text":"roadmap_stage_change.md"})
    14. search_code({"text":"test_roadmap_stage_prompt_rules"})
    15. list_files({"directory":"tests/roadmap/stage"})
    16. search_code({"text":"STAGE_EVAL_TRIGGER_PATHS"})
    17. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_prompt_rules.py"})
    18. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":160,"end_line":185})
    19. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":860,"end_line":915})
    20. read_file({"end_line":200,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":180})
    21. read_file({"end_line":1000,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":911})
    22. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_text_lines.py","start_line":120,"end_line":150})
    23. search_code({"text":"stage_problem_criterion_unknown"})
    24. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_criterion_dropped.py"})
    25. list_files({"directory":"tests/fixtures/roadmap_eval/stage"})
    26. search_code({"text":"\"criteria\""})
    27. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/plan.json","start_line":60,"end_line":90})
    28. search_code({"text":"Acceptance criteria"})
    29. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/issue.txt"})
    30. read_file({"end_line":300,"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py","start_line":255})
    31. search_code({"text":"def _overlap_problems"})
    32. read_issue({"number":3989})
    33. read_file({"end_line":1175,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":1120})
    34. search_code({"text":"3347"})
    35. search_code({"text":"_TOKEN_COST"})
    36. read_file({"end_line":120,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":100})
    37. search_code({"text":"leaves one item out"})
    38. search_code({"text":"one item out"})
    39. search_code({"text":"devops ci test tests/"})
    40. read_file({"path":"src/devops_cli/roadmap/stage/stage_criteria.py"})
    41. list_files({"directory":"tests/roadmap/stage"})
    42. search_code({"text":"stage_problem_criterion"})
    43. search_code({"text":"An issue with no such list gets an empty list"})
    44. read_file({"path":"src/devops_cli/lang/en/messages.py","start_line":3605,"end_line":3630})
    45. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_prompt_rules.py","start_line":1,"end_line":80})
    46. search_code({"text":"criteria"})
    47. search_code({"text":"--replay"})
    48. search_code({"text":"criterion_problems"})
    49. search_code({"text":"criteria gate"})
    50. read_spec({"capability":"roadmap-stage","requirement":"An unmapped criterion is refused"})
    51. search_code({"text":"Criteria mapping in the stage reply"})
    52. list_files({"directory":"tests/fixtures/roadmap_eval/stage"})
    53. read_spec({"capability":"roadmap-stage","requirement":"Criteria mapping in the stage reply"})
    54. search_code({"text":"\"criteria\""})
    55. search_code({"text":"## Acceptance criteria"})
    56. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/plan.json","start_line":60,"end_line":90})
    57. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/issue.txt","start_line":1,"end_line":40})
    58. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_split.py","start_line":30,"end_line":110})
    59. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_criteria.py"})
    60. search_code({"text":"criterion_problems("})
    61. search_code({"text":"First criterion."})
    62. read_file({"end_line":150,"path":"tests/roadmap/stage/test_roadmap_stage_text_lines.py","start_line":115})
    63. read_file({"end_line":60,"path":"tests/roadmap/implement/test_roadmap_implement_resplit.py","start_line":30})
    64. search_code({"text":"STAGE_EVAL_TRIGGER_PATHS"})
    65. search_code({"text":"criterion_unknown"})
    66. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":160,"end_line":200})
    67. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":865,"end_line":915})
    68. search_code({"text":"def _quotes_probe"})
    69. read_file({"end_line":1010,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":915})
    70. search_code({"text":"criteria"})
    71. search_code({"text":"def _plan_citation_problems"})
    72. read_file({"end_line":866,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":700})
    73. search_code({"text":"_LINE_CITATION"})
    74. read_file({"end_line":160,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":130})
    75. search_code({"text":"GroupKind.BEHAVIOR"})
    76. read_file({"end_line":1300,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":1240})
    77. read_file({"end_line":470,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":370})
    78. read_spec({"capability":"roadmap-stage"})
    79. list_files({"directory":"src/devops_cli/roadmap/stage"})
    80. read_spec({"capability":"roadmap-stage","requirement":"An unmapped criterion is refused"})
    81. read_spec({"capability":"roadmap-stage","requirement":"Criteria mapping in the stage reply"})
    82. search_code({"text":"An issue with no such list gets an empty list"})
    83. search_code({"text":"stage_problem_criterion"})
    84. search_code({"text":"ci test"})
    85. list_files({"directory":"tests/fixtures/roadmap_eval/stage"})
    86. search_code({"text":"\"criteria\""})
    87. search_code({"text":"criteria:"})
    88. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/plan.json","start_line":60,"end_line":100})
    89. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/issue.txt"})
    90. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":960,"end_line":1030})
    91. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py","start_line":70,"end_line":120})
    92. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py","start_line":240,"end_line":340})
    93. search_code({"text":"ACCEPTANCE_BODY"})
    94. search_code({"text":"criterion_problems"})
    95. search_code({"text":"criteria=["})
    96. read_file({"end_line":180,"path":"tests/roadmap/implement/test_roadmap_implement_resplit.py","start_line":130})
    97. read_file({"end_line":320,"path":"tests/roadmap/implement/test_roadmap_implement_resplit.py","start_line":270})
    98. search_code({"text":"criterion_problems("})
    99. read_file({"path":"src/devops_cli/roadmap/implement/implement_run.py","start_line":840,"end_line":950})
    100. search_code({"text":"First criterion."})
    101. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_text_lines.py","start_line":100,"end_line":160})
    102. read_file({"end_line":90,"path":"tests/roadmap/implement/test_roadmap_implement_resplit.py","start_line":25})
    103. read_file({"end_line":2320,"path":"tests/roadmap/stage/test_roadmap_stage_plan.py","start_line":2180})
    104. search_code({"text":"roadmap eval stage"})
    105. read_file({"path":"src/devops_cli/lang/en/messages.py","start_line":3600,"end_line":3640})
    106. search_code({"text":"STAGE_EVAL_TRIGGER_PATHS"})
    107. search_code({"text":"def eval_replay_problems"})
    108. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":160,"end_line":195})
    109. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":865,"end_line":900})
    110. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":900,"end_line":930})
    111. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":760,"end_line":870})
    112. search_code({"text":"def _self_acceptance_problems"})
    113. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":660,"end_line":762})
    114. search_code({"text":"_PLAN_CHECK_FILE"})
    115. list_files({"directory":"src/devops_cli/lang"})
    116. list_files({"directory":"tests/roadmap/stage"})
    117. search_code({"text":"criteria"})
    118. search_code({"text":"An issue with no such list"})
    119. search_code({"text":"criteria"})
    120. read_file({"end_line":110,"path":"tests/roadmap/stage/test_roadmap_stage_split.py","start_line":30})
    121. read_spec({"capability":"roadmap-stage","requirement":"A plan that adds a text rule carries its self-acceptance test"})
    122. read_spec({"capability":"roadmap-stage","requirement":"A plan that changes no plan check needs no self-acceptance test"})
    123. search_code({"text":"stage_problem_criterion_group"})
    124. search_code({"text":"stage_problem_criterion_unknown"})
    125. read_spec({"capability":"roadmap-stage","requirement":"The replay judges the criteria gate on plans with a criteria list"})
    126. read_spec({"capability":"roadmap-stage","requirement":"Criteria section in tasks.md"})
    127. read_file({"path":"tests/fixtures/roadmap_eval/stage/3174-a-plan-with-a-placeholder-group/issue.txt"})
    128. read_file({"path":"tests/fixtures/roadmap_eval/stage/3174-a-plan-with-a-placeholder-group/case.json"})
    129. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py","start_line":120,"end_line":230})
    130. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/case.json"})
    131. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py","start_line":230,"end_line":340})
    132. search_code({"text":"3411"})
    133. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py","start_line":1,"end_line":120})
    134. search_code({"text":"replay_case("})
    135. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_criterion_dropped.py"})
    136. search_code({"text":"what-changes"})
    137. search_code({"text":"The stage check refuses a What Changes item"})
    138. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_command.py","start_line":70,"end_line":120})
    139. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_run.py","start_line":150,"end_line":200})
    140. search_code({"text":"class RoadmapMessages"})
    141. list_files({"directory":"tests/lang"})
    142. read_file({"path":"tests/lang/test_language_catalogs.py"})
    143. find_issues({"terms":"3989"})
    144. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_render.py"})
    145. read_file({"path":"docs/roadmap/plan-check.md","start_line":1,"end_line":140})
    146. read_spec({"capability":"roadmap-stage","requirement":"The stage context lists an overlapping pull request"})
    147. read_spec({"capability":"roadmap-stage","requirement":"The gate refuses an empty Dependencies part on an overlap"})
    148. read_issue({"number":3989})
    149. read_spec({"capability":"roadmap-stage","requirement":"A closed plan is no dependency"})
    150. search_code({"text":"_ISSUE_DEPENDENCY"})
    151. read_spec({"capability":"roadmap-plan-check"})
    152. read_spec({"capability":"roadmap-plan-check","requirement":"Plan-check reports an open issue dependency"})
    153. read_spec({"capability":"roadmap-plan-check","requirement":"The plan reader reads the dependencies of a plan"})
    154. read_spec({"capability":"roadmap-stage","requirement":"The stage prompt states the Dependencies rule"})
    155. search_code({"text":"dependencies' field"})
    156. search_code({"text":"'dependencies':"})
    157. read_spec({"capability":"roadmap-stage","requirement":"A task group renders the tasks of its kind"})
    158. read_spec({"capability":"roadmap-stage","requirement":"A task group has a kind"})
    159. git_log({"path":"src/devops_cli/roadmap/stage/stage_criteria.py"})
    160. git_log({"path":"src/devops_cli/roadmap/stage_criteria.py"})
    161. search_code({"text":"StageCriterion("})
    162. read_file({"path":"src/devops_cli/roadmap/stage/stage_criteria.py"})
    163. search_code({"text":"_path_gate"})
    164. list_files({"directory":"tests/fixtures/roadmap_eval/stage"})
    165. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py","start_line":100,"end_line":220})
    166. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/plan.json"})
    167. read_file({"path":"tests/fixtures/roadmap_eval/stage/3174-a-plan-with-a-placeholder-group/plan.json"})
    168. search_code({"text":"def check_proposal"})
    169. search_code({"text":"criterion_problems("})
    170. search_code({"text":"\"criteria\": ["})
    171. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":955,"end_line":1060})
    172. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py","start_line":1,"end_line":100})
    173. read_file({"end_line":400,"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py","start_line":216})
    174. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/issue.txt"})
    175. read_file({"path":"tests/fixtures/roadmap_eval/stage/3174-a-plan-with-a-placeholder-group/issue.txt"})
    176. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/case.json"})
    177. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_criteria.py"})
    178. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py","start_line":1,"end_line":120})
    179. read_spec({"capability":"roadmap-stage","requirement":"An unmapped criterion is refused"})
    180. read_spec({"capability":"roadmap-stage","requirement":"The replay judges the criteria gate on plans with a criteria list"})
    181. search_code({"text":"stage_problem_criterion"})
    182. read_file({"path":"src/devops_cli/lang/en/messages.py","start_line":3600,"end_line":3630})
    183. search_code({"text":"An issue with no such list"})
    184. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_plan.py","start_line":280,"end_line":340})
    185. search_code({"text":"eval stage"})
    186. callers({"symbol":"check_gates"})
    187. search_code({"text":"check_proposal("})
    188. read_file({"path":"src/devops_cli/roadmap/plan_recheck.py","start_line":260,"end_line":330})
    189. read_file({"path":"src/devops_cli/roadmap/plan_check.py","start_line":340,"end_line":400})
    190. read_file({"path":"src/devops_cli/roadmap/implement/implement.py","start_line":385,"end_line":425})
    191. search_code({"text":"test_roadmap_stage_prompt_rules"})
    192. list_files({"directory":"tests/roadmap/stage"})
    193. read_file({"end_line":150,"path":"tests/roadmap/stage/test_roadmap_stage_text_lines.py","start_line":115})
    194. read_file({"end_line":80,"path":"tests/roadmap/stage/test_roadmap_stage_criterion_dropped.py","start_line":1})
    195. search_code({"text":"def make_stage_proposal"})
    196. read_file({"end_line":120,"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py","start_line":120})
    197. read_file({"end_line":200,"path":"tests/support/builders.py","start_line":140})
    198. read_file({"path":"src/devops_cli/roadmap/stage/stage_criteria.py"})
    199. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py"})
    200. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py","start_line":150,"end_line":330})
    201. read_spec({"capability":"roadmap-stage"})
    202. read_spec({"capability":"roadmap-stage","requirement":"An unmapped criterion is refused"})
    203. read_spec({"capability":"roadmap-stage","requirement":"The replay judges the criteria gate on plans with a criteria list"})
    204. search_code({"text":"def check_proposal"})
    205. search_code({"text":"criterion_problems("})
    206. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":930,"end_line":1050})
    207. search_code({"text":"def make_stage_proposal"})
    208. search_code({"text":"stage_problem_criterion"})
    209. read_file({"path":"src/devops_cli/lang/en/messages.py","start_line":3600,"end_line":3640})
    210. read_file({"path":"tests/support/builders.py","start_line":130,"end_line":220})
    211. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py","start_line":100,"end_line":250})
    212. list_files({"directory":"tests/fixtures/roadmap_eval/stage"})
    213. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/plan.json"})
    214. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/case.json"})
    215. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/issue.txt"})
    216. search_code({"text":"\"criteria\""})
    217. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py","start_line":246,"end_line":340})
    218. read_file({"path":"src/devops_cli/ai/tasks/roadmap_stage_change.md","start_line":30,"end_line":70})
    219. git_log({"path":"src/devops_cli/roadmap/stage/stage_criteria.py"})
    220. read_file({"path":"tests/invariants/test_roadmap_workflow_doc.py"})
    221. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_text_lines.py","start_line":120,"end_line":160})
    222. read_file({"path":"tests/fixtures/roadmap_eval/stage/3174-a-plan-with-a-placeholder-group/case.json"})
    223. read_file({"path":"tests/fixtures/roadmap_eval/stage/3174-a-plan-with-a-placeholder-group/issue.txt"})
    224. search_code({"text":"acceptance_criteria("})
    225. search_code({"text":"stage_problem_criterion_unknown"})
    226. read_file({"path":"src/devops_cli/roadmap/stage/stage.py","start_line":1120,"end_line":1185})
    227. list_files({"directory":"src/devops_cli/roadmap"})
    228. find_issues({"terms":"roadmap stage change prompt criteria"})
    229. read_issue({"number":3989})
    230. read_issue({"number":3830})
    231. search_code({"text":"score before and after"})
    232. search_code({"text":"eval stage --replay"})
    233. search_code({"text":"STAGE_EVAL_TRIGGER_PATHS"})
    234. search_code({"text":"def eval_replay_problems"})
    235. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":160,"end_line":195})
    236. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":860,"end_line":905})
    237. search_code({"text":"\"criterion\":"})
    238. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_plan.py","start_line":2170,"end_line":2330})
    239. read_file({"path":"tests/roadmap/test_roadmap_plan_check.py","start_line":300,"end_line":380})
    240. read_file({"path":"tests/roadmap/test_roadmap_plan_recheck.py","start_line":180,"end_line":250})
    241. search_code({"text":"check_proposal("})
    242. read_file({"path":"src/devops_cli/roadmap/plan_check.py","start_line":350,"end_line":410})
    243. read_file({"path":"src/devops_cli/roadmap/implement/implement.py","start_line":380,"end_line":430})
    244. read_file({"path":"tests/roadmap/test_roadmap_plan_check.py","start_line":1,"end_line":140})
    245. search_code({"text":"_gate_problems("})
    246. read_file({"path":"src/devops_cli/roadmap/plan_check.py","start_line":1,"end_line":140})
    247. read_file({"path":"src/devops_cli/roadmap/plan_check.py","start_line":470,"end_line":550})
    248. read_spec({"capability":"roadmap-stage","requirement":"Criteria mapping in the stage reply"})
    249. read_spec({"capability":"roadmap-stage","requirement":"The prompt asks feasibility to quote the probes"})
    250. read_file({"path":"tests/fixtures/roadmap_eval/stage/3174-a-plan-with-a-placeholder-group/plan.json","start_line":55,"end_line":75})
    251. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_task_verification.py","start_line":200,"end_line":245})
    252. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_text_lines.py","start_line":190,"end_line":225})
    253. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py","start_line":340,"end_line":460})
    254. search_code({"text":"person wrote"})
    255. search_code({"text":"not a person-written"})
    256. list_files({"directory":"openspec/changes/3320-a-plan-that-deletes-a-guard"})
    257. read_file({"path":"openspec/changes/3320-a-plan-that-deletes-a-guard/tasks.md"})
    258. search_code({"text":"criteria"})
    259. search_code({"text":"Criteria"})
    260. search_code({"text":"Acceptance criteria"})
    261. search_code({"text":"criteria"})
    262. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_prompt_rules.py","start_line":100,"end_line":200})
    263. search_code({"text":"def registry_table"})
    264. search_code({"text":"def registry_problems"})
    265. read_file({"path":"src/devops_cli/roadmap/stage/stage_registry.py","start_line":80,"end_line":120})
    266. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":1057,"end_line":1095})
    267. search_code({"text":"def recheck_plans"})
    268. read_file({"path":"src/devops_cli/roadmap/stage/stage_criteria.py"})
    269. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py"})
    270. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py","start_line":155,"end_line":280})
    271. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py"})
    272. search_code({"text":"_with_criteria"})
    273. search_code({"text":"\"gate\": \"none\""})
    274. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py","start_line":270,"end_line":340})
    275. read_file({"path":"tests/fixtures/roadmap_eval/stage/3174-a-plan-with-a-placeholder-group/case.json"})
    276. read_file({"end_line":340,"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py","start_line":278})
    277. search_code({"text":"\"criteria\":"})
    278. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/case.json"})
    279. search_code({"text":"check_proposal("})
    280. read_file({"path":"tests/fixtures/roadmap_eval/stage/3174-a-plan-with-a-placeholder-group/issue.txt"})
    281. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/plan.json","start_line":60,"end_line":85})
    282. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/issue.txt"})
    283. read_file({"end_line":2240,"path":"tests/roadmap/stage/test_roadmap_stage_plan.py","start_line":2170})
    284. search_code({"text":"criterion_problems"})
    285. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_criteria.py"})
    286. search_code({"text":"stage_problem_criterion"})
    287. search_code({"text":"An issue with no such list gets an empty list"})
    288. read_file({"path":"src/devops_cli/lang/en/messages.py","start_line":3605,"end_line":3630})
    289. search_code({"text":"roadmap_stage_change.md"})
    290. search_code({"text":"criteria"})
    291. read_file({"path":"src/devops_cli/roadmap/implement/implement.py","start_line":392,"end_line":420})
    292. read_file({"path":"src/devops_cli/roadmap/plan_check.py","start_line":368,"end_line":400})
    293. list_files({"directory":"tests/roadmap/stage"})
    294. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_prompt_rules.py","end_line":80})
    295. search_code({"text":"criteria"})
    296. read_spec({"capability":"roadmap-stage","requirement":"An unmapped criterion is refused"})
    297. read_spec({"capability":"roadmap-stage","requirement":"The replay judges the criteria gate on plans with a criteria list"})
    298. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py","start_line":196,"end_line":270})
    299. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py","start_line":200,"end_line":400})
    300. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":1,"end_line":80})
    301. search_code({"text":"_path_gate"})
    302. search_code({"text":"def check_proposal"})
    303. read_file({"path":"src/devops_cli/roadmap/stage/stage_eval_gates.py","start_line":120,"end_line":200})
    304. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":1000,"end_line":1070})
    305. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_gates.py","start_line":140,"end_line":420})
    306. read_spec({"capability":"roadmap-stage"})
    307. read_spec({"capability":"roadmap-stage","requirement":"An unmapped criterion is refused"})
    308. read_spec({"capability":"roadmap-stage","requirement":"The replay judges the criteria gate on plans with a criteria list"})
    309. search_code({"text":"Acceptance criteria"})
    310. search_code({"text":"criteria"})
    311. search_code({"text":"An issue with no such list gets an empty list"})
    312. search_code({"text":"stage_problem_criterion"})
    313. search_code({"text":"ci test"})
    314. search_code({"text":"criterion_problems("})
    315. search_code({"text":"issue_body="})
    316. read_file({"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":940,"end_line":1005})
    317. read_file({"end_line":400,"path":"src/devops_cli/roadmap/plan_check.py","start_line":330})
    318. read_file({"end_line":320,"path":"src/devops_cli/roadmap/plan_recheck.py","start_line":270})
    319. search_code({"text":"\"criteria\""})
    320. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/plan.json","start_line":55,"end_line":90})
    321. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/issue.txt"})
    322. read_file({"path":"tests/fixtures/roadmap_eval/stage/3174-a-plan-with-a-placeholder-group/case.json"})
    323. read_file({"path":"tests/fixtures/roadmap_eval/stage/3174-a-plan-with-a-placeholder-group/issue.txt"})
    324. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_eval_command.py","start_line":60,"end_line":140})
    325. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_text_lines.py","start_line":120,"end_line":160})
    326. read_file({"path":"docs/ROADMAP_WORKFLOW.md","start_line":300,"end_line":330})
    327. search_code({"text":"3174"})
    328. search_code({"text":"3411"})
    329. read_file({"path":"tests/fixtures/roadmap_eval/stage/3411-a-what-changes-item-that-names/case.json"})
    330. git_log({"path":"src/devops_cli/ai/tasks/roadmap_stage_change.md"})
    331. read_issue({"number":3989})
    332. search_code({"text":"Criteria mapping in the stage reply"})
    333. read_spec({"capability":"roadmap-stage","requirement":"Criteria mapping in the stage reply"})
    334. search_code({"text":"stage prompt states"})
    335. search_code({"text":"def make_stage_proposal"})
    336. read_file({"path":"src/devops_cli/roadmap/plan_recheck.py","start_line":1,"end_line":80})
    337. search_code({"text":"open_items"})
    338. search_code({"text":"_gate_problems("})
    339. read_file({"end_line":545,"path":"src/devops_cli/roadmap/plan_check.py","start_line":470})
    340. read_file({"end_line":240,"path":"tests/support/builders.py","start_line":120})
    341. read_file({"end_line":2240,"path":"tests/roadmap/stage/test_roadmap_stage_plan.py","start_line":2180})
    342. read_file({"end_line":580,"path":"tests/roadmap/test_roadmap_plan_check.py","start_line":520})
    343. search_code({"text":"class GroupKind"})
    344. read_file({"end_line":370,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":335})
    345. search_code({"text":"criteria"})
    346. file_symbols({"path":"tests/roadmap/stage/test_roadmap_stage_prompt_rules.py"})
    347. read_file({"end_line":540,"path":"tests/roadmap/stage/test_roadmap_stage_prompt_rules.py","start_line":425})
    348. list_files({"directory":"tests/lang"})
    349. list_files({"directory":"tests/roadmap/stage"})
    350. read_file({"path":"tests/roadmap/stage/test_roadmap_stage_criterion_dropped.py"})
    351. list_files({"directory":"tests/fixtures/roadmap_eval/stage"})
    352. read_file({"path":"tests/fixtures/roadmap_eval/stage/3410-the-eval-field-group-is-dropped/plan.json"})
    353. search_code({"text":"\"criteria\""})
    354. read_file({"end_line":110,"path":"tests/roadmap/stage/test_roadmap_stage_split.py","start_line":1})
    355. read_file({"end_line":1180,"path":"src/devops_cli/roadmap/stage/stage.py","start_line":1100})
    356. search_code({"text":"It works"})
    357. search_code({"text":"dependencies"})
    358. read_file({"end_line":470,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":370})
    359. read_file({"end_line":560,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":470})
    360. search_code({"text":"def _overlap_problems"})
    361. read_file({"end_line":1160,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":1120})
    362. search_code({"text":"def content_problems"})
    363. read_file({"end_line":940,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":860})
    364. search_code({"text":"def _self_acceptance_problems"})
    365. read_file({"end_line":790,"path":"src/devops_cli/roadmap/stage/stage_change.py","start_line":660})

    Tool calls: callers=1 (empty=0), file_symbols=1 (empty=0), find_issues=2 (empty=0), git_log=4 (empty=0), list_files=18 (empty=0), read_file=171 (empty=0), read_issue=5 (empty=0), read_spec=33 (empty=0), search_code=130 (empty=6)

    The stage job read the open pull requests and their files in 1 GraphQL request(s).

  8. devops-cli-service
    added
    owner/serviceOwner Service: the roadmap Service works on the item in a Job
    status/in-progressActively being designed or implemented by developer or agent
    and removed
    status/readyStatus Ready: problem, acceptance criteria and an answer to each key question
    on Nov 7, 2026
  9. dan-petty
    added 3 commits that reference this issue on Nov 7, 2026

    docs(openspec): plan #3391 Refuse criteria entries not in the issue body

    9065c9e

    docs(openspec): refine plan #3391 tasks and dependencies

    c26d7da

    docs(openspec): plan #3391 Refuse criteria entries not in the issue b…

    60063b1

  10. devops-cli-service commented on Nov 7, 2026

    @devops-cli-service
    devops-cli-serviceBot
    on Nov 7, 2026
    Contributor

    Stopped at archive.

    Failed check: uv run devops openspec archive 3391-refuse-criteria-entries-not-in-the.

    Failure class: test.
    Action taken: none.
    Action left: repair: give the failure to a fix step.

    Step Status Attempts Tokens in/out Last failing command
    group 1 done 1 4269320/16910 pytest on base: tests/roadmap/stage/test_roadmap_stage_eval_gates.py
    group 2 done 1 6928186/18149 pytest on base: tests/roadmap/stage/test_roadmap_stage_criteria.py
    group 3 done 1 1122140/6650 -
    archive stopped 1 0/0 uv run devops openspec archive 3391-refuse-criteria-entries-not-in-the

    Reason:

    uv run devops openspec archive 3391-refuse-criteria-entries-not-in-the exited with status 1.
✓ spec/threat-intel-lookup-outcome
✓ spec/traefik-release-pin
✓ spec/valkey-cache-tier
✓ spec/valkey-endpoint
✓ spec/vault-lease-lifecycle
✓ spec/vault-status
✓ spec/web-search
✓ spec/workstation-dashboard
Totals: 235 passed, 1 failed (236 items)
Details: openspec validate roadmap-stage --type spec
ℹ [INFO] file: skip_specs is set in .openspec.yaml: change declares no spec-level behavior changes, zero deltas accepted
  ℹ [INFO] file: skip_specs is set in .openspec.yaml: change declares no spec-level behavior changes, zero deltas accepted
  ℹ [INFO] file: skip_specs is set in .openspec.yaml: change declares no spec-level behavior changes, zero deltas accepted
  ℹ [INFO] file: skip_specs is set in .openspec.yaml: change declares no spec-level behavior changes, zero deltas accepted
  ℹ [INFO] file: skip_specs is set in .openspec.yaml: change declares no spec-level behavior changes, zero deltas accepted
  ℹ [INFO] file: skip_specs is set in .openspec.yaml: change declares no spec-level behavior changes, zero deltas accepted
✗ spec/roadmap-stage
  ⚠ [WARNING] requirements[60]: Requirement text is very long (>500 characters). Move examples and edge cases into scenarios, or split it into separate requirements that each state one behavior.
✗ openspec validate --all --strict failed with exit code 1. The archive is applied. Fix the specs that the tool names above. Then run `uv run devops openspec check`. Do not run the archive again.
Elapsed: 3.87s

    Last message of the agent (a claim that no check verified): none.

    The item is Blocked, and the job will not run it. Fix the cause. Then comment implement: retry. The Service clears the stop on its own when a new runner image rolls out, but only if a retry can fix the class of the stop and the limits of roadmap.implement.digest_clears_per_class and roadmap.implement.digest_clears_per_item allow it.
    The push of the commits that passed failed too.
    git push --force-with-lease=fix/3391-refuse-criteria-entries-not-in-the:57efbfb9ea5afbcf9daa29f01000da369e8ca3dd origin fix/3391-refuse-criteria-entries-not-in-the failed: To https://github.com/dan-petty/devops-cli.git ! [rejected] fix/3391-refuse-criteria-entries-not-in-the -> fix/3391-refuse-criteria-entries-not-in-the (stale info) error: failed to push some refs to 'https://github.com/dan-petty/devops-cli.git'

  11. devops-cli-service
    added
    status/blockedExecution blocked by external dependency or upstream review
    and removed
    status/in-progressActively being designed or implemented by developer or agent
    on Nov 7, 2026

  12. devops-cli-service commented on Nov 7, 2026

    @devops-cli-service
    devops-cli-serviceBot
    on Nov 7, 2026
    Contributor

    The runner Job devops-cli-runner-20261107-201633-cba7 ended: succeeded.
    Image digest: sha256:9a1af5a95b44197c2568d011d53c1b9a84748cf35f9721e302fe56d860d9ae06

    The last lines of its log:

    Tool call: read_file, ok.
Tool call: read_file, ok.
Tool call: read_file, ok.
Tool call: grep, ok.
Tool call: grep, ok.
Tool call: grep, ok.
Tool call: grep, ok.
Tool call: grep, ok.
Tool call: shell, started: uv run devops roadmap eval stage ...
Tool call: shell, program uv, ok.
Tool call: edit_file, ok.
Tool call: shell, started: uv run devops ci test tests/roadmap/stage/test_roadmap_stage_prompt_rules.py
Tool call: shell, program uv, ok.
Tool call: edit_file, ok.
Tool call: shell, started: uv run devops ci test tests/roadmap/stage/test_roadmap_stage_prompt_rules.py
Tool call: shell, program uv, ok.
Tool call: shell, started: uv run devops ci docs
Tool call: shell, program uv, ok.
Tool call: shell, started: uv run devops roadmap eval stage ...
Tool call: shell, program uv, ok.
Tool call: shell, started: uv run ruff check tests/roadmap/stage/test_roadmap_stage_prompt_rules.py
Tool call: grep, ok.
Tool call: shell, program uv, ok.
Tool call: read_file, ok.
Tool call: edit_file, ok.
Tool call: shell, started: git diff --stat
Tool call: shell, program git, ok.
Tool call: shell, started: git diff
Tool call: shell, program git, ok.
Tool call: read_file, ok.
Tool call: shell, started: git status --short
Tool call: shell, program git, ok.
Tool call: read_file, ok.
Tool call: shell, started: git log --oneline -3
Tool call: shell, program git, ok.
Tool call: shell, started: git show --stat HEAD
Tool call: shell, program git, ok.
Failure class test. Action left: repair.
✓ Took 0 item(s) to a ready pull request, stopped 1.
Elapsed: 3h 6m 52s
  13. devops-cli-service
    removed
    owner/serviceOwner Service: the roadmap Service works on the item in a Job
    on Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    effort/mediumEffort Mediumpriority/p2-mediumStandard priority feature work and architectural improvementsscope/roadmapThe roadmap jobs: intake, refine, stage, implement and the Service that runs themsource/agentFiled by an agentstatus/blockedExecution blocked by external dependency or upstream reviewtype/bugDefect, regression, or incorrect runtime behaviorvalue/mediumValue Medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions