feat(roadmap): the implement agent can use the project's development tools #3041

What happened

The implement shell allows only git, uv, ruff and mypy. Calls to find, cat, python and others are refused and count against a limit, and "if you use the limit up, the step stops and all your work is lost". The agent cannot use devops ci, the AST graph, repomap, RAG, devops review or the mutation check. It guesses where the project's own tools would tell.

What should happen

• A declared tool catalog for the agent, with a policy per tool (read-only, scoped write, network or none) and a budget: devops ci test <path>, devops ci scoped to the changed paths, devops ci fail-to-pass, devops ai ast-graph, devops ai repomap, devops ai rag-query, devops review on the diff, devops docs generate, devops openspec check, kubectl and kube-linter read-only for manifests.
• The runner image carries every tool in the catalog (see F4). A refused call names the tool to use instead, and does not count against the stop limit when the agent then uses it.

Acceptance criteria

• The catalog lives in one table that the prompt, the shell guard and the docs read.
• Each tool has a test that the agent can call it in the runner shell and that its policy holds (a write tool cannot leave the checkout).
• The refusal limit counts only calls that the catalog has no entry for.
• The prompt lists the tools from the table, and a test fails if the table and the prompt disagree.

Related: tracker for this set (filed with it), #2908 (issue workflow), #2867 (draft PR evaluator), #3005 (task kind), #3010 to #3015 (plan gates), #2919 (stuck-item metrics), #2953 (Ollama parser).

Delivery

The pull request of this issue lists the local devops commands (ci test, ai repomap, ai ast, docs generate, openspec check) in the shell catalog and the prompt, from one table (LOCAL_COMMANDS). Each part of the item is here, in an earlier pull request, or in another issue:

• Catalog in one table that the prompt, the shell guard and the docs read: done for programs by feat(runner): the runner image carries every development tool, and the shell policy is one table #3072 (PR feat(roadmap): add gate tools to the runner, one shell policy #3111). For the local devops commands, this pull request adds LOCAL_COMMANDS, which the prompt and the docs read. The shell guard does not read it: a call of uv run devops ... passes because uv is a program of SHELL_PROGRAMS. A guard that reads the access class of a local command is part of feat(roadmap): enforce scoped writes and refusal budgets for catalog tools #3565.
• The prompt lists the tools from the table, and a test fails when they differ: delivered by feat(runner): the runner image carries every development tool, and the shell policy is one table #3072 and by this pull request.
• A test that the agent can call each local command in the runner shell: delivered by this pull request. The test that a write tool cannot leave the checkout: feat(roadmap): enforce scoped writes and refusal budgets for catalog tools #3565.
• The refusal limit counts only calls that the catalog has no entry for: feat(roadmap): enforce scoped writes and refusal budgets for catalog tools #3565.
• kubectl and kube-linter in the catalog and the runner image: feat(roadmap): Add kubectl and kube-linter to runner tool catalog #3562.
• devops review and devops ai rag-query from the shell: feat(roadmap): allow scoped review and RAG calls from the implement shell #3563.

Alternatives check of 2026-11-04: partly.
Tool: open-policy-agent/opa (https://github.com/open-policy-agent/opa), Apache-2.0, last activity 2026-11-04, maintained.

Proposed design

Problem statement

The implement agent does not know which local devops commands it may run. The shell already allows uv, so these commands run, but the prompt lists only uv run devops ci test. The agent guesses or uses general commands. This change adds the local read, check and docs commands to the existing shell catalog and prompt table.

Acceptance criteria

Requirement: Local devops commands in catalog

The implement shell catalog SHALL list devops ai ast, devops ai repomap, devops docs generate, and devops openspec check as local commands with a read-only or checkout-write policy.

Scenario: Catalog entries exist

• WHEN The shell guard loads the catalog
• THEN Each of the four commands has an entry with its policy and no network policy.

Requirement: Prompt lists local commands

The implement prompt SHALL list each local devops command from the catalog table.

Scenario: Prompt matches catalog

• WHEN The prompt is generated for an implement step
• THEN Every local devops command in the catalog appears in the prompt and no other local devops command appears.

Requirement: Catalog and prompt consistency

A test SHALL fail when the catalog table and the implement prompt disagree on a local devops command.

Scenario: Missing prompt entry

• WHEN A local devops command is added to the catalog but not to the prompt
• THEN The consistency test fails and names the missing command.

Requirement: Cataloged command does not count

A call to a cataloged local devops command SHALL not increment the refusal counter.

Scenario: Cataloged local call

• WHEN The agent calls uv run devops ai repomap
• THEN The call is allowed and the refusal counter is unchanged.

Existing Tool Considered

open-policy-agent/opa (Apache-2.0, maintained) covers per-tool allow and deny decisions and parameter-level rules. It lacks a single catalog table that the prompt and docs read, per-tool per-step budgets and a refusal counter, the project's own tools and runner image, scoped-write confinement, and a refusal message that names the tool to use instead. The project keeps its typed Python catalog module. The written cost is about 10,000 tokens to build and test the table, guard, prompt and docs, and about 2,000 tokens per month to keep them in sync, over 2 to 3 days.

Key questions

• Where does the catalog table live in the codebase?: The catalog stays in src/devops_cli/roadmap/implement_shell_policy.py as a typed Python table. The guard, the prompt generator and the docs read it. A TOML, YAML or JSON file would add a parser and a second source. (best practice) [sources: src/devops_cli/roadmap/implement_shell_policy.py, docs/ROADMAP_WORKFLOW.md]
  • Chosen: Typed Python module. The guard is Python code. Importing the table is the shortest path. Type checking catches errors at build time.
  • Rejected: TOML or YAML file. Adds a parser and a separate file to keep in sync. The guard needs a parser dependency.
  • Rejected: JSON file. Adds a parser and no schema validation without another dependency.
• What is the budget per tool?: The budget is a maximum call count per tool per step. The catalog entry holds the number, and the guard resets the count at each step. This change does not change the budget. The enforcement of refused arguments stays deferred. (best practice) [sources: src/devops_cli/roadmap/implement_shell_policy.py, docs/ROADMAP_WORKFLOW.md]
  • Chosen: Per-step call count. The step is the unit of work. A reset each step keeps a long session from using up the budget.
  • Rejected: Per-session call count. A session spans steps. This is too restrictive for multi-step items.
  • Rejected: No budget. The issue requires a budget.
• What are the policy values for each tool?: devops ai ast and devops ai repomap are read-only. devops openspec check is read-only. devops docs generate is checkout-write and may write only inside the checkout. kubectl, kube-linter, devops review and devops ai rag-query are out of this change. (best practice) [sources: src/devops_cli/roadmap/implement_shell_policy.py, src/devops_cli/roadmap/implement_run.py, docs/ROADMAP_WORKFLOW.md]
  • Chosen: Read-only and checkout-write. The four local commands read code or write only generated docs in the checkout. They need no network.
  • Rejected: Network policy for review and RAG. The shell has no key. The job process holds the credentials. This part waits on a separate security design.
  • Rejected: Read-only for every local command. devops docs generate writes files. A read-only label would be false.
• How does the refusal message work?: The guard reads the refusal table and names the cataloged tool for the refused program. An uncataloged program increments the refusal counter. The change adds the local commands to that table and keeps the existing hint path. [sources: src/devops_cli/roadmap/implement_shell_policy.py, docs/ROADMAP_WORKFLOW.md]
• Which tools are in the initial catalog?: The initial catalog keeps the current shell programs and adds uv run devops ci test, devops ai ast, devops ai repomap, devops docs generate and devops openspec check. It leaves out kubectl, kube-linter, devops review and devops ai rag-query. (best practice) [sources: src/devops_cli/roadmap/implement_shell_policy.py, src/devops_cli/roadmap/runner_tools.py, docs/ROADMAP_WORKFLOW.md]
  • Chosen: Current programs plus local devops commands. The shell already allows uv, so these commands run. The prompt only needs to name them.
  • Rejected: Add kubectl and kube-linter. NOT_IN_RUNNER leaves them out. A Job has no cluster token, and a scanner on PATH fails tests that pass in the gate.
  • Rejected: Add review and RAG. The shell has a scratch HOME and no key. The job process holds the credentials.
• Should the project adopt open-policy-agent/opa for the catalog?: No. OPA covers per-tool allow and deny decisions, but it has no single catalog table that the prompt and docs read, no per-tool per-step budget or refusal counter, and no scoped-write confinement. The project keeps its typed Python table. (best practice) [sources: src/devops_cli/roadmap/implement_shell_policy.py]
  • Chosen: Typed Python catalog. It is the single source that the guard, prompt and docs already read. It needs no new service.
  • Rejected: OPA policy engine. It covers policy evaluation but lacks the catalog, budget, prompt consistency and scoped-write enforcement.
  • Rejected: OPA plus a Python catalog. Two sources must stay in sync. The current table already covers the need.
• Is scoped-write enforcement part of this change?: No. The shell check reads the first word only and is no security boundary. The pod is the sandbox. A test that a write cannot leave the checkout needs the real sandbox of feat(roadmap): run the implement agent shell in a real sandbox #2695. This part is deferred. (best practice) [sources: src/devops_cli/roadmap/implement_run.py, CONTEXT.md]
  • Chosen: Defer to feat(roadmap): run the implement agent shell in a real sandbox #2695. The current shell cannot enforce path confinement. The sandbox item is open and is a person item.
  • Rejected: Enforce in the shell guard. uv run python can write anywhere the user can. A first-word check cannot stop it.
  • Rejected: Drop the requirement. The requirement protects the checkout. It stays as a deferred issue.

Fit

Fits in one pull request.

Suspected block

None.

Dependencies

None.