@@ -24,26 +24,31 @@
-cometcli is a local-first operations terminal for validators running
-[Cosmos-EVM](https://github.com/cosmos/evm) chains. It answers
-*"is my node healthy?"* — and lets you act on the answer — from one place.
-
-- **Observe** — CometBFT RPC, Cosmos gRPC, and EVM JSON-RPC in one view:
- signing health, peers, drift, gov, upgrade plans
-- **Operate** — transactions (simulate → decode → approve → broadcast),
- service control, upgrades, state-sync, security audits
-- **Watch** — live TUI dashboard and an alert engine that pages you on
- Slack, Discord, or Telegram
-- **Automate** — runbooks for jail recovery, halts, migrations; the same
- tools drive an optional AI agent
-
-No dashboards to wire up. No context-switching between `evmd`, `systemctl`,
-`curl`, and explorer tabs.
-
-Every capability is a deterministic subcommand **and** a tool the optional AI
-agent can call — one registry, two front-ends. The deterministic layer is
-boring and correct; the agent composes it. See [PLAN.md](PLAN.md) for the
-architecture and design rationale.
+cometcli is a terminal you talk to, the way you use Claude Code, built for running
+[Cosmos SDK](https://github.com/cosmos/cosmos-sdk) and
+[Cosmos-EVM](https://github.com/cosmos/evm) validators. Ask *"is anyone jailed?"* or
+*"why is my validator missing blocks?"*. It inspects the node, chain and host, works
+out the cause, proposes the fix, and carries it out once you approve.
+
+- **Ask** — plain-language questions about your machine, nodes and chain; it checks
+ instead of guessing
+- **Diagnose** — one sweep of ~50 signals, matched against a knowledge base of 77
+ known failure cases, root cause first
+- **Fix** — incidents worked end to end: fix the cause, wait for sync, unjail, check
+ it signs again; every change and transaction waits for your approval
+- **Operate** — transactions, governance proposals and votes, upgrades, state sync,
+ service control, security audits
+- **Watch** — live dashboard and alerts to Slack, Discord or Telegram
+- **Extend** — settings, memory, custom commands, hooks, MCP servers and subagents,
+ as in Claude Code
+
+
+ 
+
+
+Every capability is a deterministic subcommand **and** a tool the agent can call —
+one registry, two front-ends. The deterministic layer is boring and correct; the
+agent composes it. See [PLAN.md](PLAN.md) for the architecture.
## Install
@@ -60,177 +65,273 @@ Details — custom install dirs, checksum/cosign verification, SBOMs — in
## Quick start
+**1. Pick a model** (once). Keys are read hidden and saved to
+`~/.cometcli/credentials` (mode 0600); an environment variable of the same name
+always wins.
+
+```bash
+# low-cost Claude
+cometcli config set agent.provider anthropic
+cometcli config set agent.model claude-haiku-4-5-20251001
+cometcli config set-key ANTHROPIC_API_KEY
+
+# or free models via OpenRouter
+cometcli config set agent.provider openrouter # model defaults to openrouter/free
+cometcli config set-key OPENROUTER_API_KEY
+```
+
+**2. Talk to it.**
+
+```bash
+cometcli # chat on this machine
+cometcli "why is the disk filling up?" # start with a question
+```
+
+**3. Add your node** and ask about it — the agent moves onto the node by itself when
+a question needs it.
+
```bash
-# 1. Guided setup — probes your node and fills in chain-id, bech32 prefix,
-# bond denom, and evm-chain-id automatically
-cometcli init
+cometcli init # guided: probes the node, fills chain-id, prefix, denoms
+cometcli one myval # or pin a session to the node
+cometcli one myval "is my validator healthy?"
+cometcli doctor # one-shot health check, no model needed
+```
+
+Or add the profile declaratively (`profile add` merges — re-run to change a field):
-# (or declaratively — `profile add` merges, re-run to update one field)
+```bash
cometcli profile add myval \
- --chain-id mychain-1 --evm-chain-id 9000 --bech32-prefix cosmos \
+ --chain-id mychain-1 --bech32-prefix cosmos \
--role validator --home /var/lib/evmd --binary evmd \
--comet tcp://127.0.0.1:26657 --grpc 127.0.0.1:9090 \
--evm http://127.0.0.1:8545 \
--service systemd --unit evmd.service \
--signer ops --signer-backend file --fee-denom uatom
+```
+
+`--evm` is optional: leave it out for a plain Cosmos SDK chain.
+
+## The terminal
-# 2. Add an ops key (transaction signing only — consensus keys are never touched)
-cometcli keys add --name ops # generate
-cometcli keys add --name ops --recover # import BIP-39 mnemonic
-cometcli keys add --name ops --privkey-hex
# import raw secp256k1 hex
+An inline chat like Claude Code's: answers land in your normal scrollback, and only
+the bottom of the screen redraws.
-# 3. Check everything
-cometcli doctor
```
+> is anyone jailed?
+⏺ Use node(val01)
+ ⎿ node mode: val01
+
+⏺ chain.validators(status=jailed)
+ ⎿ no validator is jailed
+
+⏺ No — none of primium-1's validators is jailed.
```
-✓ rpc reachable height 4464
-✓ not catching up catching_up=false
-✓ has peers 3 peers
-✓ port exposure clean
-✗ key file perms 1 problems ← config dir is 0755, want 0700
-✓ signing health missed 5/10000
-✓ jail status not jailed
-✓ upgrade pending none
-✓ host disk
-✓ ntp/clock
-✓ evm parity drift 0
+
+| Key / prefix | Does |
+|---|---|
+| `/` | command menu — `/incident`, `/status`, `/one`, `/model`, `/cost`, `/compact`, … |
+| `@path` · `!cmd` · `#text` | attach a file · run a command in your own terminal · save to memory |
+| `esc` | interrupt; anything you typed meanwhile is sent at once |
+| `ctrl+r` · `ctrl+o` | expand the last tool output · show the model's thinking |
+| `shift+tab` | cycle permission mode: ops → accept-edits → read-only |
+
+Type while it works and your message reaches the model at its next step, so you can
+redirect or stop it mid-task. Approvals offer **Yes**, **Yes, and don't ask again**
+for that kind of command, and **No**. Full list in
+[docs/COMMANDS.md](docs/COMMANDS.md#the-interactive-terminal).
+
+**General mode** (`cometcli`) gives the agent `bash`, file read/write/edit,
+`glob`/`grep` and `web_fetch` on your machine. **Node mode** (`cometcli one
+`, `/one `, or the agent's own `use_node`) adds the node, chain,
+validator and incident tools, runs the shell on the node's host (local or SSH), applies
+the validator safety rules and keeps a live snapshot of the node in context.
+
+Headless, for scripts and CI:
+
+```bash
+cometcli -p "summarize docker container health"
+journalctl -u evmd -n 300 | cometcli one myval -p "why did it halt?"
+cometcli -p --output-format json "…" # result + usage as JSON
+cometcli -p --output-format stream-json "…" # one JSON event per line
+cometcli -c # continue the last session here
+cometcli -r # resume one (cometcli sessions)
```
-## Command surface
+## Incidents
-Tools are grouped by domain. Required args also bind positionally —
-`cometcli tx get ` works like `evmd q tx `.
+Validators fail in a few hundred known ways. cometcli starts every incident from
+evidence: `node.triage` collects ~50 signals at once — sync, peers, signing, jail,
+consensus key, upgrade plan, governance, disk, memory, clock, process restarts and
+OOM kills, log error categories, config invariants, EVM — and matches them against a
+knowledge base of known cases for CometBFT, the Cosmos SDK and Cosmos-EVM.
-| Domain | Highlights |
-|---|---|
-| `node` | `status`, `peers`, `health`, `consensus`, `logs`, `config --action lint`, `service status\|restart`, `version-check` |
-| `val` | `status`, `signing`, `rewards`, `votes` · on-chain: `unjail`, `withdraw`, `vote`, `edit`, `create` |
-| `chain` | `validators`, `params`, `gov`, `upgrade-plan`, `pool`, `balance` |
-| `evm` | `chainid` (profile-vs-RPC sanity), `parity` (comet↔JSON-RPC drift), `gasprice`, `txpool` |
-| `tx` | `send`, `delegate`, `undelegate`, `redelegate`, `get` — every tx tool takes `--gas-price`, `--gas-limit`, `--fee-denom`, `--seq` |
-| `keys` | `add` (`--recover`/`--privkey-hex`), `list`, `show`, `rm`, `convert` (bech32↔0x) — `eth_secp256k1` by default |
-| `sec` | `exposure` (listening-port audit), `perms` (key-file permissions), `doublesign` (priv_validator_state HRS check) |
-| `mon` | `snapshot`, `watch` (live TUI), `alerts` (rules → stdout/Slack/Discord/Telegram; `--once`, `--mute`, `--repeat-minutes`) |
-| `ui` | chat-first TUI: talk to the agent, `/run` tools, approvals in-app; overview/fleet/logs/send panes behind it |
-| `upgrade` | `check` (plan + binary + upstream release), `prepare` (cosmovisor staging), `watch` |
-| `snap` | `list`, `prune`, `statesync` (fetch trust height, write `[statesync]`) |
-| `runbook` | `list`, `show`, `run` — builtins + your own YAML in `~/.cometcli/runbooks/` |
-| `net` | `add-peer`, `rm-peer` (persistent peers in config.toml) |
-| `fleet` | `status` (health matrix), `exec` (fan out a read-only tool), `shell` (command on every host, one approval) |
-| `agent`/`ask` | interactive AI SRE over the same registry |
+```
+$ cometcli node triage
+matched cases (root causes, most severe first):
+ 1. cfg-parse-error [critical] Config file fails to parse — config.parse_error == true
+symptoms of the above (clear once the root is fixed): node-crash-loop, node-rpc-down,
+ val-jailed-downtime
+→ kb.show for confirm steps, fix and verify; confirm before acting.
+```
+
+Each case lists how to confirm it, its causes, fix steps tagged `[read]` /
+`[change]` / `[tx]`, how to verify, and what never to do. In a session,
+**`/incident [what you see]`** works the top case to resolution: confirm → fix the
+root cause → wait (`wait.until`, without spending model calls) → verify → report.
+When the agent finds a cause no case covered, it offers to record it with `kb.add`.
+Add your own cases in `~/.cometcli/kb/` — format in
+[docs/EXTENDING.md](docs/EXTENDING.md#knowledge-base-cases).
-Global flags: `--profile` (override active), `--json` (structured output),
-`-y/--yes` (auto-approve observe/diagnose prompts — never on-chain).
+Jail recovery is careful about fees: `val.unjail` refuses while the jail period runs,
+the node isn't synced, the signer's saved state is ahead of the chain, the node signs
+with a different consensus key than the one registered, self-delegation is below the
+minimum, or the account can't pay the fee. `/recover-jail` runs the whole procedure.
-## Transactions
+## Transactions and governance
-Every on-chain op goes through **build → simulate → decode → approve →
-broadcast → confirm**:
+Every on-chain op goes through **build → simulate → decode → approve → broadcast →
+confirm**, and always waits for you:
```
⚠ [on-chain] broadcast transaction
{
+ "signer": "container val-node (key validator, test keyring)",
"chain_id": "mychain-1",
- "account": "cosmos10pmprk9…",
- "sequence": 1,
- "messages": ["/cosmos.bank.v1beta1.MsgSend {…}"],
- "fee": "155401875000000uatom",
- "gas_limit": 138135
+ "messages": ["/cosmos.slashing.v1beta1.MsgUnjail {…}"],
+ "fee": "130408000000000uatom",
+ "gas_limit": 130408
}
Proceed? [y/N]
```
-Nothing hits the wire without an explicit `y`. Sync-acceptance is not
-success — cometcli polls for the committed result and reports the real
-`code`, height, and gas. Sequence drift auto-heals once.
+Two signers: cometcli's own keyring, or **the node container's keyring** — the key
+never leaves the node; cometcli has `evmd tx sign` run inside the container. When both
+can sign, you pick one first. A local key is only offered if it is the validator's
+operator account. Sync-acceptance is not success — cometcli polls for the committed
+result and reports the real code, height and gas.
-## The agent
+Governance: `gov propose` (text, software upgrade, any messages), `gov deposit`,
+`val vote`, and `wait until --condition proposal-status`. Upgrade heights the vote
+can't reach in time are refused before you sign. See
+[docs/COMMANDS.md](docs/COMMANDS.md#governance).
-`cometcli agent` (or `cometcli ask "why is my validator missing blocks?"`)
-runs an LLM in a loop over the same tool registry:
-
-```bash
-cometcli profile add myval --agent-provider anthropic --agent-model claude-sonnet-4-5
-export ANTHROPIC_API_KEY=… # or OPENAI_API_KEY, OLLAMA_API_KEY;
- # COMETCLI_LLM_API_KEY is a universal fallback
-cometcli ask "summarize signing health and flag any exposure risks"
-```
+## Command surface
-Providers: `anthropic`, `openai`, `groq` (`GROQ_API_KEY`, default model
-`llama-3.3-70b-versatile`), `gemini` (`GEMINI_API_KEY`, default model
-`gemini-3.8-flash` — Interactions API, `store=false` so nothing is kept
-server-side), `openai-compat` (Ollama, vLLM, LM Studio — any OpenAI-shaped
-endpoint via `--agent-base-url`), or `off`.
+Tools are grouped by domain. Required args also bind positionally —
+`cometcli tx get ` works like `evmd q tx `.
-The model sees tool calls as function calls; every on-chain or local-change
-call still goes through the same approval gate — **the model can propose,
-only you approve.**
+| Domain | Highlights |
+|---|---|
+| `node` | `triage`, `status`, `peers`, `health`, `consensus`, `logs`, `config --action lint`, `service status\|restart`, `version-check` |
+| `val` | `status`, `signing`, `jail-check`, `consensus`, `rewards`, `votes` · on-chain: `unjail`, `withdraw`, `vote`, `edit`, `create` |
+| `chain` | `validators` (`--status jailed`), `params`, `gov`, `upgrade-plan`, `pool`, `balance` |
+| `gov` | `propose` (text / upgrade / messages), `deposit` |
+| `kb` | `search`, `show`, `add` — the knowledge base of failure cases |
+| `wait` | `until` — synced, height, unjailable, signing, in-consensus, tx-committed, proposal-status, or any triage signal |
+| `evm` | `chainid`, `parity` (comet↔JSON-RPC drift), `gasprice`, `txpool` |
+| `tx` | `send`, `delegate`, `undelegate`, `redelegate`, `get` — every tx tool takes `--gas-price`, `--gas-limit`, `--fee-denom`, `--seq` |
+| `keys` | `add` (`--recover`/`--privkey-hex`), `list`, `show`, `rm`, `convert` (bech32↔0x) |
+| `sec` | `exposure` (listening ports), `perms` (key-file permissions), `doublesign` (priv_validator_state check) |
+| `mon` | `snapshot`, `watch` (live TUI), `alerts` (rules → stdout/Slack/Discord/Telegram) |
+| `upgrade` | `check`, `prepare` (download, verify, stage for cosmovisor), `watch` |
+| `snap` | `list`, `prune`, `statesync` |
+| `runbook` | `list`, `show`, `run` — builtins + your own YAML |
+| `net` | `add-peer`, `rm-peer` |
+| `fleet` | `status`, `exec` (a read-only tool on every profile), `shell` (one command on every host, one approval) |
+
+Global flags: `--profile`, `--json`, `-y/--yes` (auto-approve observe/diagnose
+prompts — never on-chain).
+
+## Models
+
+| Provider | Key | Default model |
+|---|---|---|
+| `anthropic` | `ANTHROPIC_API_KEY` | `claude-opus-5-5` (`claude-haiku-4-5-20251001` for low cost) |
+| `openrouter` | `OPENROUTER_API_KEY` | `openrouter/free` — a free model that supports tools |
+| `openai` | `OPENAI_API_KEY` | `gpt-4.1` |
+| `groq` | `GROQ_API_KEY` | `llama-3.3-70b-versatile` |
+| `gemini` | `GEMINI_API_KEY` | `gemini-3.8-flash` (`store=false`: nothing kept server-side) |
+| `openai-compat` | `--agent-base-url` | Ollama, vLLM, LM Studio — any OpenAI-shaped endpoint |
+
+Set per machine (`cometcli config set agent.provider …`) or per profile; switch for
+one session with `--model` or `/model`. `/cost` shows token use; prompt caching is on
+for Anthropic. Thinking streams live where the model offers it, and effort
+(`--effort` or `/effort`) is sent only to models that accept it.
+
+The model proposes; **only you approve.** Every change and every transaction goes
+through the same approval gate whichever model you use.
Bounded runs for automation:
```bash
-cometcli agent --task "check fleet, report anomalies" --budget 8 --max-iter 6
cometcli ask "is anything wrong?" --safe # read-only: mutating tools refused
cometcli mon alerts --triage # every alert ships with an AI diagnosis
```
-`--triage` runs a bounded, safe-mode agent per firing alert and appends the
-diagnosis to whatever the webhook delivers — you get *"missed 9931/10000 —
-likely jailed; run `cometcli val unjail` after jailed_until"* instead of a
-bare alarm.
-
### MCP server
-`cometcli mcp` exposes the whole registry as a Model Context Protocol
-server on stdio — point any MCP client (Claude Desktop, Cursor, a custom
-orchestrator) at it:
+`cometcli mcp` exposes the whole registry as a Model Context Protocol server on
+stdio — point Claude Desktop, Cursor or your own orchestrator at it:
```json
{ "mcpServers": { "cometcli": { "command": "cometcli", "args": ["mcp"] } } }
```
-Observe/diagnose tools run freely and carry `readOnlyHint`; mutating tiers
-are marked `destructiveHint` and refused — stdio has no human to approve,
-run those via the CLI.
+Observe/diagnose tools run freely and carry `readOnlyHint`; mutating tiers are
+marked `destructiveHint` and refused — stdio has no human to approve, run those via
+the CLI. cometcli is also an MCP *client*: `cometcli mcp add` connects servers to the
+agent.
## Safety model
-- **Tiers** — `observe → diagnose → local-change → on-chain`. Read-only by
- default; every mutation prompts.
-- **Consensus keys are radioactive** — `priv_validator_key.json` is never
- read into memory or sent to a model. Only `priv_validator_state.json`
- HRS is inspected for double-sign guards.
-- **Redaction** — keys, mnemonics, JWTs, bearer tokens, and URL credentials
- are scrubbed from all text entering and leaving the agent.
-- **Audit** — every tool call, shell command, prompt, and transaction lands
- in `~/.cometcli/audit/YYYY-MM-DD.jsonl` (`cometcli audit` to tail it).
-- **Bounded execution** — every tool runs under a deadline; a dead endpoint
- fails fast instead of hanging.
+- **Tiers** — `observe → diagnose → local-change → on-chain`. Reads run; changes ask;
+ transactions always ask.
+- **Permission rules** — allow / ask / deny in Claude Code syntax
+ (`bash(docker logs:*)`, `read(./config/**)`, `node.*`); deny beats ask beats allow.
+ Every shell command is classified first, scripts and `docker exec` included.
+- **Consensus keys are radioactive** — `priv_validator_key.json` is never read into
+ memory or sent to a model; state resets are refused outright. Key management is
+ operator-only.
+- **Redaction** — keys, mnemonics, JWTs, bearer tokens and URL credentials are
+ scrubbed from everything entering and leaving the agent (tx hashes stay visible).
+- **Audit** — every tool call, shell command, prompt and transaction lands in
+ `~/.cometcli/audit/YYYY-MM-DD.jsonl` (`cometcli audit` to tail it).
+- **Bounded execution** — every tool runs under a deadline; long requests pause
+ after 200 model steps (`agent.max_turns`) and continue on "continue".
## Profiles & config
State lives in `~/.cometcli/` (`COMETCLI_HOME` to override):
```
-config.yaml # profiles + active pointer
-keys/ # ops keyring (file backend)
-runbooks/ # your custom YAML playbooks
-audit/ # JSONL audit trail
+config.yaml # profiles, active pointer, agent settings
+credentials # API keys saved by `config set-key` (0600)
+settings.json # permission rules, hooks, MCP servers (also per project: .cometcli/)
+COMET.md # memory the agent always reads (also per project)
+kb/ # your own knowledge-base cases
+keys/ # ops keyring (file backend)
+sessions/ # saved conversations (cometcli -c / -r)
+runbooks/ # your custom YAML playbooks
+audit/ # JSONL audit trail
```
Key env vars: `COMETCLI_PROFILE`, `COMETCLI_KEYRING_PASSWORD`,
-`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `COMETCLI_LLM_API_KEY`.
+`COMETCLI_CONTAINER_KEYRING_PASSWORD`, the provider keys above, and
+`COMETCLI_LLM_API_KEY` as a universal fallback.
## Documentation
+- [docs/COMMANDS.md](docs/COMMANDS.md) — full command reference: chat, incidents,
+ signing, governance, every tool
+- [docs/EXTENDING.md](docs/EXTENDING.md) — settings, memory, custom commands, hooks,
+ MCP, subagents, knowledge-base cases
- [docs/INSTALL.md](docs/INSTALL.md) — install options, checksum & cosign verification
-- [docs/COMMANDS.md](docs/COMMANDS.md) — full command reference with examples
-- [TESTING.md](TESTING.md) — test locally against docker validators, with a free LLM key (Groq/Ollama)
+- [TESTING.md](TESTING.md) — test locally against docker validators
- [PLAN.md](PLAN.md) — architecture and design rationale
- [docs/RUNBOOKS.md](docs/RUNBOOKS.md) — builtin playbooks + authoring your own
-- [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) — endpoints, keyrings,
- tx errors, SSH, state sync, agent config
+- [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) — endpoints, keyrings, tx errors,
+ SSH, state sync, agent config
- [docs/RELEASE.md](docs/RELEASE.md) — release checklist and verification
- [CONTRIBUTING.md](CONTRIBUTING.md) · [SECURITY.md](SECURITY.md)
@@ -240,12 +341,16 @@ Key env vars: `COMETCLI_PROFILE`, `COMETCLI_KEYRING_PASSWORD`,
go build ./... # build all packages
go vet ./... # vet
golangci-lint run # lint (config: .golangci.yml)
-go test ./... # unit tests (keys KAT, redaction, lint, agent loop…)
+go test -race ./... # unit + scenario tests, incl. a full jail recovery on a
+ # simulated chain (fake gRPC node, CometBFT RPC and docker)
# e2e (needs a live node):
COMETCLI_E2E=1 COMETCLI_E2E_GRPC=127.0.0.1:9090 \
COMETCLI_E2E_COMET=tcp://127.0.0.1:26657 go test ./test/e2e/...
+# container signing against a real evmd image:
+go test -tags realdocker ./internal/tools/common/
+
# SSH transport e2e (needs a reachable sshd):
COMETCLI_SSH_TEST_HOST=127.0.0.1 COMETCLI_SSH_TEST_PORT=2222 \
COMETCLI_SSH_TEST_USER=ops COMETCLI_SSH_TEST_KEY=~/.ssh/id_ed25519 \
@@ -254,16 +359,14 @@ go test ./internal/client/host -run SSHLive -v
## Ecosystem
-Built for the Cosmos stack — chains running
-[CometBFT](https://github.com/cometbft/cometbft) consensus with the
-[Cosmos EVM](https://github.com/cosmos/evm) module (`evmd` and derivatives).
-If your chain exposes a CometBFT RPC, Cosmos gRPC, and Ethereum JSON-RPC,
-cometcli speaks to all three.
+Built for chains running [CometBFT](https://github.com/cometbft/cometbft) consensus
+with the [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) — plain SDK chains and
+[Cosmos-EVM](https://github.com/cosmos/evm) chains (`evmd` and derivatives) alike. It
+speaks CometBFT RPC, Cosmos gRPC and, where present, Ethereum JSON-RPC.
-**Nothing is chain-specific.** Chain-id, bech32 prefix, fee denom, EVM
-chain-id, ports, binary name — all live in the profile, and `cometcli init`
-auto-detects most of them by probing the node. Point it at any evmd-based
-chain and it just works.
+**Nothing is chain-specific.** Chain-id, bech32 prefix, fee denom, EVM chain-id, ports
+and binary name live in the profile, and `cometcli init` detects most of them by
+probing the node. Nodes can run under systemd or docker, locally or over SSH.
## Contributing
diff --git a/docs/COMMANDS.md b/docs/COMMANDS.md
index 06d0cdf..1a108ed 100644
--- a/docs/COMMANDS.md
+++ b/docs/COMMANDS.md
@@ -9,6 +9,160 @@ Global flags: `--profile ` · `--json` · `-y/--yes` (auto-approve
observe/diagnose only — never on-chain) · `COMETCLI_PROFILE`,
`COMETCLI_KEYRING_PASSWORD` env vars.
+## Chat
+
+```bash
+cometcli # general mode: SRE agent on this machine (TUI)
+cometcli "prompt" # same, starting with a prompt
+cometcli one [profile] ["prompt"] # node mode for a profile (active profile if omitted)
+cometcli -p "prompt" # headless: print the answer and exit
+cmd | cometcli -p "prompt" # piped input is attached as context
+cometcli -c / -r # continue the latest session here / resume one
+cometcli config show # global agent settings (general mode; profiles override)
+cometcli config set agent.provider anthropic # or openrouter, openai, groq, gemini, openai-compat
+cometcli config set agent.model claude-haiku-4-5-20251001
+cometcli config set-key ANTHROPIC_API_KEY # saves a key (read hidden from stdin) to ~/.cometcli/credentials, 0600
+cometcli config set agent.max_turns 300 # model steps per request before it pauses (default 200)
+cometcli trust # let this project's hooks and MCP servers run
+cometcli mcp add|list|remove … # MCP servers the agent connects to
+```
+
+Headless flags: `--output-format text|json|stream-json`, `--include-partial-messages`
+(token deltas in stream-json), `--verbose` (tool calls on stderr in text mode),
+`--append-system-prompt "…"`, `--max-turns N`. Headless runs never read approvals from
+stdin: anything that needs approval is denied unless `--allowedTools` or
+`--permission-mode` allows it. An explicit `--profile` on bare `cometcli` also means
+node mode.
+
+`json` prints one object: `{"type":"result","subtype":"success|error","is_error",
+"result","session_id","num_turns","duration_ms","provider","model","mode","usage"}`.
+`stream-json` prints a `system/init` line, then `assistant`, `tool_use`, `tool_result`,
+`notice` and `todos` events, then the same `result` object.
+
+### The interactive terminal
+
+`cometcli` and `cometcli one` open an inline chat: answers land in your normal
+scrollback (scroll, copy, search as usual); only the bottom area — streaming text,
+input, approval dialog, status line — redraws.
+
+| Key / prefix | Does |
+|---|---|
+| `enter` · `\`+`enter` / `ctrl+j` | send · new line |
+| `↑` `↓` | prompt history (or move in a menu) |
+| `/` | command menu as you type; `tab` completes, `enter` runs |
+| `@path` | file completion; the file's content is attached to your message |
+| `!cmd` | run `cmd` **in your terminal** — the UI steps aside, so prompts and passwords work (e.g. `!./upgrade/vote-upgrade.sh`). Output is never sent to the model; it's told the command and exit code. In node mode over SSH it runs on the node (`ssh -t`) |
+| `#text` | add a line to memory (`/remember`) |
+| `shift+tab` | cycle mode: ops → accept-edits → read-only (→ bypass, if started with it) |
+| `esc` | interrupt the running turn (or close a menu); a message you typed meanwhile is sent right away |
+| `ctrl+r` | expand the last collapsed tool output |
+| `ctrl+o` | show the model's last thinking |
+| `ctrl+c` | interrupt · clear input · twice to exit |
+| `ctrl+d` | exit (empty input) |
+| `?` | shortcut help |
+
+A message typed while a turn runs reaches the model at its next step (after the current
+tool call), so you can redirect or stop it mid-task; `esc` interrupts and sends it at
+once. Tool output is collapsed to a few lines (`ctrl+r` expands it) and thinking to a
+"Thought for Ns" line (`ctrl+o` shows it). If a request runs out of steps it pauses —
+say "continue". In general mode the agent moves onto a node profile by itself when a
+question needs one; `/one ` pins one. Approvals offer
+**Yes**, **Yes, and don't ask again for ``** (saved to the project's
+`.cometcli/settings.local.json`, or your user settings outside a project), and **No**
+(declines and interrupts so you can redirect). Transactions offer only Yes/No.
+
+`cometcli ui` still opens the full-screen dashboard (overview, fleet, logs, send).
+
+## Incidents
+
+### Triage and the knowledge base
+
+Validators fail in a few hundred known ways. cometcli carries them as a **knowledge base
+of cases** (Cosmos SDK, CometBFT and Cosmos-EVM), and starts every incident with one
+deterministic sweep instead of the model guessing which tool to call:
+
+```bash
+cometcli node triage # ~50 signals in parallel + the known cases they match
+cometcli node triage --since 2h # wider log window
+cometcli node triage --signals val. # just one family
+cometcli kb search --query "app hash mismatch"
+cometcli kb show --id val-jailed-downtime
+```
+
+`node.triage` collects from every source at once — CometBFT RPC, chain gRPC (staking,
+slashing, upgrade plan, gov), host (disk, inodes, memory, load, NTP), the process (state,
+restarts, OOM kill, exit code), logs (counted by category: panic, OOM, disk, app hash, DB
+lock/corruption, privval, height regression, double sign, upgrade halt, state sync, port
+in use, config error, chain mismatch, EVM JSON-RPC…), config.toml/app.toml invariants,
+priv_validator_state, and on Cosmos-EVM chains the JSON-RPC endpoint. A source that's
+down is reported, never fatal. Signals are flat names (`val.jailed`,
+`host.disk_used_pct`, `logs.apphash`, `node.key_mismatch`…).
+
+Each **case** says which signals point to it, how to confirm it, causes, fix steps tagged
+`[read]` / `[change]` / `[tx]`, how to verify, and what never to do. In node mode,
+**`/incident [what you see]`** runs the method: triage → confirm the top case → fix the
+root cause first (approvals as usual) → wait with `wait.until` → verify → re-triage →
+report. When the agent finds a cause no case covered, it offers to record one with
+`kb.add` (you approve the write), so the next triage recognizes it.
+
+Chain family comes from the profile (`metadata.chain_type: cosmos-sdk|cosmos-evm`, else any
+EVM endpoint/chain id/`*evm*` binary means cosmos-evm); EVM-only cases are skipped on plain
+SDK chains. Add or override cases in `~/.cometcli/kb/*.yaml` or `.cometcli/kb/*.yaml` —
+format in [EXTENDING.md](EXTENDING.md#knowledge-base-cases).
+
+`wait.until --condition signal` waits on any triage expression:
+
+```bash
+cometcli wait until --condition signal --value "host.disk_used_pct < 85 && proc.running == true"
+```
+
+### Incident tools
+
+Tools built for running an incident to resolution — usable directly or by the agent:
+
+```bash
+cometcli val jail-check # why jailed, root-cause evidence from the node's logs,
+ # restarts/OOM/disk, and every unjail blocker with its fix
+cometcli val consensus --window 50 # bonded, in the validator set, signing recent blocks?
+cometcli wait until --condition synced # block until caught up (live progress, ETA)
+cometcli wait until --condition unjailable # until the jail period is over
+cometcli wait until --condition in-consensus --window 50 --min_signed_pct 95
+cometcli wait until --condition tx-committed --value
+cometcli wait until --condition proposal-status --value 12:PASSED
+```
+
+`wait.until` polls in-process, so an agent waiting on it spends no tokens; it returns
+`done=false` with the last state on timeout. `val.unjail` refuses on hard blockers
+(jail period not over, node not synced, self-delegation below minimum, no fee balance,
+tombstoned) instead of paying for a doomed tx, and every rejected transaction comes back
+with a cause → next step hint.
+
+In a node-mode session, **`/recover-jail [notes]`** runs the whole procedure: diagnose →
+stop if tombstoned → fix the root cause → wait for sync → wait out the jail → unjail
+(you approve) → on failure fix the cause and retry → verify it signs → report.
+
+### Signing transactions
+
+Two signers; when both can sign, you pick one first, then approve the tx (the approval shows which signer will sign):
+
+- **cometcli's keyring** — `signer.key` (import with `cometcli keys add --recover`).
+- **the node container's own keyring** — the key never leaves the node: cometcli builds
+ and simulates the tx, you approve, then `evmd tx sign` runs inside the container (a file
+ keyring's password is asked for then, never stored or sent to the model).
+
+```yaml
+signer:
+ key: ops # cometcli keyring (optional)
+ mode: "" # "" = ask when both work | local | container
+ container: primium-validator0 # default: service.unit for docker services
+ container_key: val0 # key name in the container's keyring
+ container_keyring: "" # test | file — detected when empty
+ container_home: /data/node0/evmd
+```
+
+Headless runs (`-p`) must set `signer.mode`; a container file-keyring password comes from
+`COMETCLI_CONTAINER_KEYRING_PASSWORD` there.
+
## Setup
```bash
@@ -63,7 +217,10 @@ cometcli val signing # missed-block counter + uptime %
cometcli val rewards # outstanding rewards + commission
cometcli val votes # active proposals vs recorded votes
-cometcli val unjail # MsgUnjail (fails at simulation if still jailed)
+cometcli val jail-check # why jailed, evidence from the logs, every unjail blocker
+cometcli val unjail # MsgUnjail — refused while the jail period runs, the node isn't
+ # synced, the signer's state is ahead, the node runs another
+ # consensus key, self-delegation is short or fees are missing
cometcli val vote --proposal 7 --option yes
cometcli val edit --commission-rate 0.05 --moniker new-name
cometcli val withdraw # rewards + commission to signer
@@ -72,6 +229,26 @@ cometcli val create --amount 1000000uatom \
--moniker myval --commission-rate 0.05
```
+## Governance
+
+```bash
+cometcli chain gov # proposals in voting (--status deposit|passed|rejected|all)
+cometcli val votes # open proposals and how this validator voted
+cometcli gov propose --kind text --title "…" --summary "…"
+cometcli gov propose --kind upgrade --title "v2" --summary "…" --name v2 --in_blocks 20000
+cometcli gov propose --kind messages --title "…" --summary "…" \
+ --messages '[{"@type":"/cosmos.distribution.v1beta1.MsgCommunityPoolSpend", …}]'
+cometcli gov deposit --proposal 7 # tops up exactly what's missing to the minimum
+cometcli val vote --proposal 7 --option yes
+cometcli wait until --condition proposal-status --value 7:PASSED
+```
+
+The deposit defaults to the chain's minimum and is checked against its
+`min_deposit_ratio` before you approve. An upgrade height the vote can't reach in time
+(voting period at the measured block time) is refused, with the earliest height that
+works. Messages are signed with whichever signer you pick (see
+[Signing transactions](#signing-transactions)).
+
## Transactions
```bash
@@ -88,7 +265,7 @@ All tx commands accept: `--gas-price` · `--fee-denom` · `--gas-limit` ·
## Chain (gRPC queries)
```bash
-cometcli chain validators # bonded set
+cometcli chain validators # bonded set (--status jailed|all|unbonding|unbonded)
cometcli chain params # slashing window, unbonding, inflation
cometcli chain pool # bonded vs unbonded stake
cometcli chain gov # proposals in voting period
@@ -201,37 +378,113 @@ cometcli audit # today's audit log (prompts, LLM turns,
cometcli audit sessions [--all] # agent sessions recorded in the log
cometcli audit replay # reproduce a session offline: recorded LLM turns +
# tool output re-driven through the agent loop
+cometcli sessions [--all] # saved agent conversations, newest first
cometcli completion zsh # shell completion
cometcli version
```
Agent flags (`agent`, `ask`, `serve`): `--mode ops|readonly`, `--safe` (=readonly),
-`--autopilot local-change`, `--budget N` tool calls per turn, `--max-iter N`, `--no-stream`.
+`--autopilot local-change`, `--budget N` tool calls per turn, `--max-iter N` (default 50), `--no-stream`.
+
+Session flags (bare `cometcli`, `ui`, `agent`, `ask`): `-c/--continue` resumes the latest
+session started in this directory, `-r/--resume ` a specific one (an id prefix works),
+`--model`, `--effort low|medium|high|xhigh|max`, `--max-tokens N`. Every conversation is
+saved after each turn to `~/.cometcli/sessions/` (0600, already redacted).
### Agent session commands (TUI, REPL, web)
| Command | Effect |
|---|---|
-| `/mode [ops\|readonly]` | show or set the approval posture; readonly hides mutating tools from the model |
+| `/mode [ops\|accept-edits\|readonly\|bypass]` | show or set the approval posture; readonly hides mutating tools from the model |
| `/approve local-change on\|off` | autopilot local-change tools; on-chain can never be autopiloted |
| `/model [name \| provider name]` | show or switch the LLM mid-conversation |
| `/runbook [name]` | list runbooks, or have the agent run one through the normal approval gate |
| `/tools` | every tool with its tier and how this session treats it (auto/confirm/deny) |
| `/profile`, `/audit`, `/reset` | profile + policy, audit path + session id, new session |
+| `/compact [focus]` | summarize the conversation to free context (also automatic near the limit) |
+| `/cost` | session token usage, cache hits, and current context size |
+| `/effort [level\|default]` | show or set reasoning depth for the rest of the session |
+| `/sessions`, `/resume ` | list saved sessions, or load one into the current session |
+| `/permissions`, `/allow\|/ask\|/deny ` | show rules; add one for this session |
+| `/memory`, `/remember [--user\|--node] ` | memory files loaded; add a line to COMET.md |
+| `/mcp`, `/agents` | connected MCP servers; subagents for the task tool |
+| `/` | commands from `.cometcli/commands/*.md` — see [EXTENDING.md](EXTENDING.md) |
+| `/incident [what you see]` | node mode: triage → known case → fix root cause → wait → verify → report |
+| `/recover-jail [notes]` | node mode: run the full jail-recovery procedure |
+| `/one [profile\|off]` | switch the session to node mode for a profile, or back to general (conversation kept) |
+
+### General tools and permissions
+
+Besides the node tools, the agent has general SRE tools that run on the profile's host
+(local, or over SSH — same tools either way):
+
+| Tool | What it does |
+|---|---|
+| `bash` | any shell command; the working directory persists, stdin is closed (prompts get EOF), default timeout 2 min (max 10) |
+| `read`, `write`, `edit` | files with line numbers; edits need a fresh read and show a diff for approval |
+| `glob`, `grep` | find files by pattern, search contents (ripgrep when installed) |
+| `web_fetch` | release notes, docs, a node's REST endpoint; each new domain is approved, localhost isn't |
+| `todo_write` | a visible checklist for multi-step work |
+
+Every shell command is classified before it runs: reads (`ls`, `journalctl`, `docker
+logs`, `systemctl status`, `evmd q …`, JSON-RPC queries via curl, …) run at once;
+anything that changes the host asks; a transaction — `evmd tx …` directly, inside
+`docker exec`/`docker run`, or inside a script the command runs — **always** asks, in
+every mode. Consensus keys, mnemonics, key ceremonies (`keys add/export`, scripts that
+run them) and state resets (`unsafe-reset-all`, deleting `priv_validator_state.json`)
+are refused outright: the agent hands those steps to you.
+
+Modes (`--permission-mode`, `/mode`): `ops` (default — changes ask), `accept-edits`
+(file edits inside the working directory don't ask), `readonly` (nothing that changes
+anything can run; the shell is limited to reads), `bypass` (local changes don't ask;
+transactions still do).
+
+Rules refine that per session (`/allow`, `/ask`, `/deny`, `--allowedTools`,
+`--disallowedTools`) or per profile (`agent.permissions`). Deny beats ask beats allow,
+and an allow must cover every part of a compound command:
+
+```yaml
+agent:
+ permissions:
+ allow: ["bash(docker logs:*)", "bash(./info.sh)", "web_fetch(domain:github.com)", "node.*"]
+ ask: ["bash(docker compose:*)"]
+ deny: ["bash(rm:*)", "edit(/etc/**)", "val.unjail"]
+```
### Agent config (per profile)
```yaml
agent:
provider: anthropic # anthropic | openai | groq | gemini | openai-compat/ollama | off
- model: claude-sonnet-4-5
+ model: claude-opus-5-5 # default per provider when omitted
mode: ops # ops (default) | readonly
autopilot: [local-change] # optional; on-chain is rejected
redact_hosts: [val.internal] # masked before any text reaches the LLM
redact_endpoints: true # also mask this profile's endpoint/SSH hosts
no_stream: false # set true for endpoints that mishandle streaming
+ effort: medium # reasoning depth; mapped per provider (see below)
+ max_tokens: 0 # output cap per round; 0 = provider default
+ max_turns: 50 # model rounds per prompt
+ context_window: 0 # override the model's context size (tokens)
+ compact_at: 0 # prompt size that triggers auto-compaction; 0 = 80% of window, ≤200k
+ tools: "" # "all" sends every tool schema each request; default loads on demand
```
+Each request carries a core tool set (the general tools plus node.status/health/logs and
+val.status/signing) and a catalog of the rest by name; the model loads others with
+`tool_search` as it needs them. That keeps a request near 2.5k tokens instead of ~7.5k —
+small enough for free-tier per-minute limits — and cheaper on every provider.
+
+`effort` maps to each provider's own control: Anthropic `output_config.effort`, OpenAI
+and Groq `reasoning_effort` (`xhigh`/`max` → `high`; reasoning models only), Gemini
+`generation_config.thinking_level`. Leave it unset for models without a reasoning knob.
+
+Long sessions stay healthy on every provider: transient API failures (429, 5xx,
+overloaded) are retried with backoff honoring `Retry-After`; a reply cut off at the
+output limit is continued automatically; the system prompt is frozen per session (later
+node snapshots ride on your messages) so provider prompt caches stay warm; and when
+the prompt nears `compact_at`, the conversation is summarized at the next turn boundary.
+
`COMETCLI_OFFLINE=1` disables the agent entirely (ask/agent/serve chat refuse); every
tool subcommand and `cometcli mcp` keep working. Mnemonics (checked against the BIP-39
wordlist), hex/base64 key material, tokens and URL credentials are scrubbed from user
diff --git a/docs/EXTENDING.md b/docs/EXTENDING.md
new file mode 100644
index 0000000..90bd38a
--- /dev/null
+++ b/docs/EXTENDING.md
@@ -0,0 +1,227 @@
+# Extending cometcli
+
+Settings, memory, custom commands, hooks, MCP servers and subagents all live in plain
+files. The formats follow Claude Code's, so configs written for it mostly carry over.
+
+A **project** is the nearest directory above where you start cometcli that holds a
+`.cometcli/` folder, a `COMET.md` or a `.git`.
+
+## Settings — `settings.json`
+
+| File | Scope |
+|---|---|
+| `~/.cometcli/settings.json` | you, everywhere |
+| `/.cometcli/settings.json` | the project — commit it |
+| `/.cometcli/settings.local.json` | you, in this project — gitignore it |
+
+Later files override single values (`model`, `effort`, `permissions.defaultMode`);
+lists (`permissions.allow/ask/deny`), `env`, `hooks` and `mcpServers` accumulate.
+`config.yaml`'s `agent:` section still picks the provider; settings refine it.
+
+```json
+{
+ "model": "openai/gpt-oss-120b",
+ "effort": "medium",
+ "permissions": {
+ "allow": ["bash(docker logs:*)", "bash(./info.sh)", "web_fetch(domain:github.com)"],
+ "ask": ["bash(docker compose:*)"],
+ "deny": ["bash(rm:*)", "edit(/etc/**)", "val.unjail"],
+ "defaultMode": "acceptEdits"
+ },
+ "env": { "GROQ_API_KEY": "…" },
+ "hooks": { … },
+ "mcpServers": { … }
+}
+```
+
+`env` is exported before the provider starts, without overriding variables already set.
+`defaultMode` takes `default`/`ops`, `acceptEdits`, `plan`/`readonly`, `bypassPermissions`.
+
+## Trust
+
+Project hooks and project MCP servers run commands on your machine — on a validator
+host, that matters. They are **ignored until you trust the project**:
+
+```bash
+cometcli trust # trust the project you're in (covers subfolders)
+cometcli trust --off # revoke
+cometcli trust --list
+```
+
+Untrusted items are listed when a session starts. Your user-level settings are always
+trusted. A project command's `allowed-tools` also needs trust.
+
+## Memory — `COMET.md`
+
+Standing instructions loaded into every session's system prompt:
+
+1. `~/.cometcli/COMET.md` — yours, everywhere
+2. `COMET.md` in each directory from the project root down to where you started
+ (`AGENTS.md` is read where a directory has no `COMET.md`), plus `COMET.local.md`
+3. In node mode, `~/.cometcli/memory/.md` — notes about that node
+
+A line `@path/to/file.md` imports another file (key material is never imported).
+
+```markdown
+# primium node-setup
+- Validators run in docker; containers are primium-validator0..3.
+- Never restart validator0 without asking — it's the genesis proposer.
+- Upgrades follow upgrade/README.md.
+@docs/ports.md
+```
+
+In a session: `/memory` lists what's loaded; `/remember ` appends to the
+project's `COMET.md` (`--user`, `--node` for the others) and tells the model right away.
+
+## Custom slash commands
+
+Markdown files in `/.cometcli/commands/` or `~/.cometcli/commands/` become
+commands. `ops/vote.md` is `/ops:vote`. A project command shadows a user one.
+
+```markdown
+---
+description: Vote on an upgrade proposal
+argument-hint:
+allowed-tools: bash(./upgrade/vote-upgrade.sh:*), read
+---
+Vote $2 on proposal $1.
+
+Current proposals: !`docker exec primium-validator0 evmd q gov proposals -o json | jq -r '.proposals[] | "\(.id) \(.status)"'`
+Network config: @run-genesis/network-config.env
+
+Read upgrade/vote-upgrade.sh first, confirm the proposal is in its voting period,
+then run it. Report the tx hash.
+```
+
+- `$ARGUMENTS` is everything after the command; `$1`…`$9` are the words (quotes group).
+- `` !`cmd` `` runs while expanding and inlines the output — **read-only commands only**
+ (the same classifier as the bash tool); anything else is left for the agent to run
+ with approval.
+- `@path` inlines a file (never key material).
+- `allowed-tools` adds allow rules for that one turn. Transactions still always ask.
+
+Commands work headlessly too: `cometcli -p "/ops:vote 7 yes"`.
+
+## Hooks
+
+Commands run at points in the loop. Events: `PreToolUse`, `PostToolUse`,
+`UserPromptSubmit`, `Stop`, `SessionStart`.
+
+```json
+{
+ "hooks": {
+ "PreToolUse": [
+ { "matcher": "bash", "hooks": [{ "type": "command", "command": ".cometcli/hooks/guard.sh", "timeout": 10 }] }
+ ],
+ "PostToolUse": [
+ { "matcher": "edit|write", "hooks": [{ "type": "command", "command": "evmd validate-genesis 2>&1 | tail -3" }] }
+ ],
+ "UserPromptSubmit": [
+ { "hooks": [{ "type": "command", "command": "echo \"on-call: $(cat ~/.oncall)\"" }] }
+ ]
+ }
+}
+```
+
+`matcher` is a case-insensitive regex over tool names (`bash`, `edit`, `node.status`,
+`mcp__grafana__.*`; Claude Code's `Bash` works); empty or `*` matches all. Hooks run
+from the project root with `COMETCLI_PROJECT_DIR` (and `CLAUDE_PROJECT_DIR`) set, and
+get the event as JSON on stdin:
+
+```json
+{"session_id":"…","transcript_path":"…","cwd":"…","hook_event_name":"PreToolUse",
+ "permission_mode":"ops","tool_name":"bash","tool_input":{"command":"…"}}
+```
+
+PostToolUse adds `tool_response`, UserPromptSubmit `prompt`, Stop `stop_hook_active`,
+SessionStart `source` (`startup`|`resume`).
+
+| Exit | Effect |
+|---|---|
+| 0 | pass. UserPromptSubmit/SessionStart stdout is added as context for the model |
+| 2 | **block** — PreToolUse: the call doesn't run and stderr goes to the model; PostToolUse: stderr goes to the model; UserPromptSubmit: the prompt is rejected; Stop: the agent keeps working on stderr (at most 3 times) |
+| other | non-blocking error, shown to you |
+
+Exit 0 with JSON on stdout can decide too: `{"hookSpecificOutput":{"permissionDecision":
+"allow|deny|ask","permissionDecisionReason":"…","additionalContext":"…"}}`,
+`{"decision":"block","reason":"…"}`, `{"systemMessage":"…"}`, `{"continue":false}`.
+A hook `allow` skips the prompt for a host change — **never for a transaction**, and
+it can't unlock key material or state resets.
+
+## MCP servers
+
+```bash
+cometcli mcp add grafana -- npx -y @grafana/mcp-grafana
+cometcli mcp add k8s -e KUBECONFIG=~/.kube/config -- kubectl-mcp-server
+cometcli mcp add pd --transport http https://mcp.example.com/mcp -H "Authorization: Bearer ${PD_TOKEN}"
+cometcli mcp add tools --scope project -- ./scripts/mcp.sh # writes .mcp.json, shared
+cometcli mcp list --check # connect and count tools
+cometcli mcp remove grafana
+```
+
+Scopes: `user` (default, `~/.cometcli/settings.json`), `project` (`.mcp.json` — the
+same file Claude Code reads), `local` (`.cometcli/settings.local.json`). `${VAR}` and
+`${VAR:-default}` expand in commands, args, env, URLs and headers.
+
+Servers start with each session; their tools are named `mcp____` and,
+like node tools, load on demand — the system prompt lists them, the model calls
+`tool_search` to load the ones it needs. Tools marked `readOnlyHint` run without a
+prompt; others ask. Rules: `mcp__grafana` covers a whole server. `/mcp` shows what's
+connected. (`cometcli mcp` with no subcommand still *serves* cometcli's own tools.)
+
+## Subagents
+
+The agent can hand a self-contained investigation to a subagent with the `task` tool:
+it gets a fresh context, the same tools and permissions, works through the task and
+returns only its report — so long log trawls don't fill the main conversation (or a
+small per-minute token budget). Subagents can't spawn subagents.
+
+Define specialized ones in `/.cometcli/agents/.md` or
+`~/.cometcli/agents/`:
+
+```markdown
+---
+name: log-digger
+description: Trawls container and journal logs for errors around a time window
+tools: bash, grep, read
+---
+You read logs. Start with docker logs --since for each validator container,
+then journalctl. Report each distinct error with first/last timestamp and count.
+```
+
+`tools` restricts what it can use. `/agents` lists them.
+
+## Knowledge-base cases
+
+`node.triage` matches its signals against cases. Built-in cases ship in the binary
+(`internal/kb/cases/`); yours go in `~/.cometcli/kb/*.yaml` (all nodes) or
+`/.cometcli/kb/*.yaml`. Same `id` overrides a built-in. A file holds one case or a
+list of them:
+
+```yaml
+- id: sentry-lost # lowercase, digits, dashes
+ title: Validator lost both sentries
+ chains: [cosmos-sdk] # or cosmos-evm; omit for all (cosmos-sdk includes evm chains)
+ severity: high # critical | high | medium | low
+ symptoms: peers drop to 0 after sentry maintenance
+ match:
+ all: ["node.peers == 0", "config.pex == false"] # every one must hold
+ any: ["logs.peer_net > 0"] # at least one, when given
+ confirm: ["node.peers; can the validator reach sentry-1:26656?"]
+ causes: ["sentries restarted with new node ids"]
+ fix:
+ - "[read] get the sentries' node ids"
+ - "[change] net.add-peer the new ids; restart"
+ verify: ["node.peers >= 2"]
+ warnings: ["never open the validator's p2p port publicly as a shortcut"]
+ test:
+ signals: {node.peers: 0, config.pex: false, logs.peer_net: 3} # must match
+```
+
+Conditions are ` ` with `== != > >= < <= contains exists missing`;
+values are numbers, `true`/`false`, or strings. A signal triage couldn't collect never
+satisfies a condition (except `missing`). `cometcli node triage` prints every signal name
+and value — write conditions against those. Fix steps must start with `[read]`, `[change]`
+or `[tx]`; the agent still goes through the normal approvals for each. Ranking: more matched
+conditions first, then severity. The built-in suite checks every case's `test` fixture
+ranks it in the top 3 and that a healthy node matches nothing.
diff --git a/docs/images/cometcli-startup.png b/docs/images/cometcli-startup.png
new file mode 100644
index 0000000..bdbbc30
Binary files /dev/null and b/docs/images/cometcli-startup.png differ
diff --git a/internal/agent/anthropic.go b/internal/agent/anthropic.go
index 1c1a2bc..c727393 100644
--- a/internal/agent/anthropic.go
+++ b/internal/agent/anthropic.go
@@ -1,11 +1,11 @@
package agent
import (
- "bytes"
"context"
"encoding/json"
"fmt"
"net/http"
+ "regexp"
"sort"
"strings"
"time"
@@ -15,126 +15,244 @@ import (
type anthropic struct {
key, model, base string
hc *http.Client
+ noEffort bool // the model rejected output_config.effort
}
func (a *anthropic) Name() string { return "anthropic" }
func (a *anthropic) client() *http.Client {
if a.hc == nil {
- a.hc = &http.Client{Timeout: 300 * time.Second}
+ a.hc = &http.Client{Timeout: 10 * time.Minute}
}
return a.hc
}
type anthBlock map[string]any
-func (a *anthropic) body(r *Request) map[string]any {
+// Models that take adaptive thinking (4.6+ Opus/Sonnet, the 5 family,
+// Fable/Mythos). Older ones (Haiku 4.5, Sonnet 4.5, …) reject it.
+var anthAdaptiveRe = regexp.MustCompile(`^claude-(fable|mythos)-|^claude-(opus|sonnet)-(4-[6-9]|5)`)
+
+// Models that take output_config.effort (Opus 4.5+, Sonnet 4.6+, the 5
+// family, Fable/Mythos). Haiku 4.5 and Sonnet 4.5 reject it with a 400.
+var anthEffortRe = regexp.MustCompile(`^claude-(fable|mythos)-|^claude-opus-(4-[5-9]|5)|^claude-sonnet-(4-[6-9]|5)|^claude-haiku-[5-9]`)
+
+// Models that accept the server-side refusal fallback ("default" routing).
+var anthFallbackRe = regexp.MustCompile(`^claude-(fable-5-1|opus-5|sonnet-5-5)`)
+
+const anthFallbackBeta = "server-side-fallback-2026-07-01"
+
+func (a *anthropic) firstParty() bool {
+ return strings.Contains(a.base, "api.anthropic.com")
+}
+
+func (a *anthropic) body(r *Request, stream bool) map[string]any {
+ maxTok := r.MaxTok
+ if maxTok == 0 {
+ // streaming has no HTTP-timeout concern, so give room for thinking
+ maxTok = 16000
+ if stream {
+ maxTok = 64000
+ }
+ }
body := map[string]any{
"model": r.Model,
- "max_tokens": r.MaxTok,
- "system": r.System,
+ "max_tokens": maxTok,
"messages": a.convMessages(r.Messages),
+ // auto-places a cache breakpoint on the last cacheable block, so
+ // the frozen system prompt + tools + history prefix is reused
+ "cache_control": map[string]any{"type": "ephemeral"},
+ }
+ if r.System != "" {
+ body["system"] = []anthBlock{{"type": "text", "text": r.System}}
}
if len(r.Tools) > 0 {
var tools []map[string]any
for _, t := range r.Tools {
- tools = append(tools, map[string]any{
+ td := map[string]any{
"name": t.Name,
"description": t.Desc,
"input_schema": t.Schema,
- })
+ }
+ if stream {
+ td["eager_input_streaming"] = true
+ }
+ tools = append(tools, td)
}
body["tools"] = tools
}
+ if anthAdaptiveRe.MatchString(r.Model) {
+ body["thinking"] = map[string]any{"type": "adaptive", "display": "summarized"}
+ }
+ if r.Effort != "" && anthEffortRe.MatchString(r.Model) && !a.noEffort {
+ body["output_config"] = map[string]any{"effort": r.Effort}
+ }
+ if a.firstParty() && anthFallbackRe.MatchString(r.Model) {
+ body["fallbacks"] = "default"
+ }
return body
}
func (a *anthropic) post(ctx context.Context, body map[string]any) (*http.Response, error) {
- raw, _ := json.Marshal(body)
- req, err := http.NewRequestWithContext(ctx, "POST", strings.TrimSuffix(a.base, "/")+"/v1/messages", bytes.NewReader(raw))
+ raw, err := json.Marshal(body)
if err != nil {
return nil, err
}
- req.Header.Set("x-api-key", a.key)
- req.Header.Set("anthropic-version", "2023-06-01")
- req.Header.Set("content-type", "application/json")
- return a.client().Do(req)
+ hdr := map[string]string{
+ "x-api-key": a.key,
+ "anthropic-version": "2023-06-01",
+ }
+ if _, ok := body["fallbacks"]; ok {
+ hdr["anthropic-beta"] = anthFallbackBeta
+ }
+ resp, err := doHTTP(ctx, a.client(), strings.TrimSuffix(a.base, "/")+"/v1/messages", raw, hdr)
+ if err != nil {
+ return nil, err
+ }
+ if resp.StatusCode/100 != 2 {
+ defer resp.Body.Close()
+ err := apiError("anthropic", resp)
+ // a model we didn't know lacks effort: drop it for the session
+ if _, has := body["output_config"]; has && resp.StatusCode == 400 && strings.Contains(err.Error(), "effort") {
+ a.noEffort = true
+ delete(body, "output_config")
+ return a.post(ctx, body)
+ }
+ return nil, err
+ }
+ return resp, nil
+}
+
+// anthUsage is the Messages API usage object; input_tokens excludes the
+// cached parts, so total prompt size is the sum of all three.
+type anthUsage struct {
+ Input int `json:"input_tokens"`
+ CacheWrite int `json:"cache_creation_input_tokens"`
+ CacheRead int `json:"cache_read_input_tokens"`
+ Output int `json:"output_tokens"`
+}
+
+func (u anthUsage) usage() Usage {
+ return Usage{Input: u.Input + u.CacheWrite + u.CacheRead, Output: u.Output, CacheRead: u.CacheRead}
+}
+
+type anthStopDetails struct {
+ Category string `json:"category"`
+ Explanation string `json:"explanation"`
+}
+
+func anthStop(reason string, d *anthStopDetails) (StopReason, string) {
+ switch reason {
+ case "end_turn", "stop_sequence":
+ return StopEnd, ""
+ case "tool_use":
+ return StopToolUse, ""
+ case "max_tokens", "model_context_window_exceeded":
+ return StopMaxTokens, ""
+ case "refusal":
+ detail := ""
+ if d != nil {
+ detail = strings.TrimSpace(d.Category + " " + d.Explanation)
+ }
+ return StopRefusal, detail
+ }
+ return "", ""
+}
+
+// anthResponse builds a Response from the final content blocks. The raw
+// blocks (thinking signatures included) are kept for verbatim replay.
+func anthResponse(blocks []json.RawMessage, stop string, sd *anthStopDetails, u anthUsage) *Response {
+ res := &Response{Usage: u.usage(), RawSteps: mustJSON(blocks)}
+ res.Stop, res.StopDetail = anthStop(stop, sd)
+ var text, thinking strings.Builder
+ for _, raw := range blocks {
+ var b struct {
+ Type string `json:"type"`
+ Text string `json:"text"`
+ Thinking string `json:"thinking"`
+ ID string `json:"id"`
+ Name string `json:"name"`
+ Input json.RawMessage `json:"input"`
+ }
+ if json.Unmarshal(raw, &b) != nil {
+ continue
+ }
+ switch b.Type {
+ case "text":
+ text.WriteString(b.Text)
+ case "thinking":
+ thinking.WriteString(b.Thinking)
+ case "tool_use":
+ args := b.Input
+ if len(args) == 0 {
+ args = json.RawMessage(`{}`)
+ }
+ res.Calls = append(res.Calls, Call{ID: b.ID, Name: b.Name, Args: args})
+ }
+ }
+ res.Text, res.Thinking = text.String(), thinking.String()
+ res.Done = len(res.Calls) == 0
+ return res
}
func (a *anthropic) Chat(ctx context.Context, r *Request) (*Response, error) {
- resp, err := a.post(ctx, a.body(r))
+ resp, err := a.post(ctx, a.body(r, false))
if err != nil {
return nil, err
}
defer resp.Body.Close()
var out struct {
- Content []struct {
- Type string `json:"type"`
- Text string `json:"text"`
- ID string `json:"id"`
- Name string `json:"name"`
- Input json.RawMessage `json:"input"`
- } `json:"content"`
- StopReason string `json:"stop_reason"`
- Error *struct {
- Message string `json:"message"`
- } `json:"error"`
+ Content []json.RawMessage `json:"content"`
+ StopReason string `json:"stop_reason"`
+ StopDetails *anthStopDetails `json:"stop_details"`
+ Usage anthUsage `json:"usage"`
}
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
return nil, err
}
- if out.Error != nil {
- return nil, fmt.Errorf("anthropic: %s", out.Error.Message)
- }
- res := &Response{Done: out.StopReason == "end_turn"}
- for _, b := range out.Content {
- switch b.Type {
- case "text":
- res.Text += b.Text
- case "tool_use":
- res.Calls = append(res.Calls, Call{ID: b.ID, Name: b.Name, Args: b.Input})
- }
- }
- return res, nil
+ return anthResponse(out.Content, out.StopReason, out.StopDetails, out.Usage), nil
}
-// Stream consumes the Messages API SSE stream: text_delta chunks are
-// forwarded to onText as they arrive; tool_use input arrives as
-// input_json_delta fragments and is assembled per content-block index.
+// Stream consumes the Messages API SSE stream. Each content block is
+// rebuilt from its start event plus deltas — text, thinking (with its
+// signature), redacted thinking, and tool_use input (partial JSON) — so
+// the assistant turn can be replayed exactly as the model produced it.
func (a *anthropic) Stream(ctx context.Context, r *Request, onText func(string)) (*Response, error) {
- body := a.body(r)
+ body := a.body(r, true)
body["stream"] = true
resp, err := a.post(ctx, body)
if err != nil {
return nil, err
}
defer resp.Body.Close()
- if resp.StatusCode/100 != 2 {
- return nil, apiError("anthropic", resp)
- }
type block struct {
- typ, id, name string
- input strings.Builder
+ obj map[string]any
+ text strings.Builder // text or thinking
+ sig strings.Builder
+ input strings.Builder
}
blocks := map[int]*block{}
- var text strings.Builder
- stop := ""
+ var usage anthUsage
+ var stop string
+ var sd *anthStopDetails
err = readSSE(resp.Body, func(data string) error {
var ev struct {
- Type string `json:"type"`
- Index int `json:"index"`
- ContentBlock struct {
- Type string `json:"type"`
- ID string `json:"id"`
- Name string `json:"name"`
- Text string `json:"text"`
- } `json:"content_block"`
+ Type string `json:"type"`
+ Index int `json:"index"`
+ ContentBlock map[string]any `json:"content_block"`
+ Message struct {
+ Usage anthUsage `json:"usage"`
+ } `json:"message"`
Delta struct {
- Type string `json:"type"`
- Text string `json:"text"`
- PartialJSON string `json:"partial_json"`
- StopReason string `json:"stop_reason"`
+ Type string `json:"type"`
+ Text string `json:"text"`
+ Thinking string `json:"thinking"`
+ Signature string `json:"signature"`
+ PartialJSON string `json:"partial_json"`
+ StopReason string `json:"stop_reason"`
+ StopDetails *anthStopDetails `json:"stop_details"`
} `json:"delta"`
+ Usage *anthUsage `json:"usage"`
Error *struct {
Message string `json:"message"`
} `json:"error"`
@@ -143,26 +261,44 @@ func (a *anthropic) Stream(ctx context.Context, r *Request, onText func(string))
return nil // tolerate keepalives / unknown payloads
}
switch ev.Type {
+ case "message_start":
+ usage = ev.Message.Usage
case "content_block_start":
- blocks[ev.Index] = &block{typ: ev.ContentBlock.Type, id: ev.ContentBlock.ID, name: ev.ContentBlock.Name}
- if ev.ContentBlock.Text != "" {
- text.WriteString(ev.ContentBlock.Text)
- onText(ev.ContentBlock.Text)
+ b := &block{obj: ev.ContentBlock}
+ if t, _ := b.obj["text"].(string); t != "" {
+ b.text.WriteString(t)
+ onText(t)
}
+ blocks[ev.Index] = b
case "content_block_delta":
+ b := blocks[ev.Index]
+ if b == nil {
+ return nil
+ }
switch ev.Delta.Type {
case "text_delta":
- text.WriteString(ev.Delta.Text)
+ b.text.WriteString(ev.Delta.Text)
onText(ev.Delta.Text)
- case "input_json_delta":
- if b := blocks[ev.Index]; b != nil {
- b.input.WriteString(ev.Delta.PartialJSON)
+ case "thinking_delta":
+ b.text.WriteString(ev.Delta.Thinking)
+ if r.OnThinking != nil {
+ r.OnThinking(ev.Delta.Thinking)
}
+ case "signature_delta":
+ b.sig.WriteString(ev.Delta.Signature)
+ case "input_json_delta":
+ b.input.WriteString(ev.Delta.PartialJSON)
}
case "message_delta":
if ev.Delta.StopReason != "" {
stop = ev.Delta.StopReason
}
+ if ev.Delta.StopDetails != nil {
+ sd = ev.Delta.StopDetails
+ }
+ if ev.Usage != nil {
+ usage.Output = ev.Usage.Output
+ }
case "error":
if ev.Error != nil {
return fmt.Errorf("anthropic: %s", ev.Error.Message)
@@ -174,33 +310,79 @@ func (a *anthropic) Stream(ctx context.Context, r *Request, onText func(string))
if err != nil {
return nil, err
}
- res := &Response{Text: text.String(), Done: stop == "end_turn"}
idx := make([]int, 0, len(blocks))
for i := range blocks {
idx = append(idx, i)
}
sort.Ints(idx)
+ var out []json.RawMessage
+ bad := map[string]string{} // tool_use id → unparseable input
for _, i := range idx {
b := blocks[i]
- if b.typ != "tool_use" {
- continue
+ switch b.obj["type"] {
+ case "text":
+ b.obj["text"] = b.text.String()
+ case "thinking":
+ b.obj["thinking"] = b.text.String()
+ if b.sig.Len() > 0 {
+ b.obj["signature"] = b.sig.String()
+ }
+ case "tool_use":
+ // eager input streaming means the server no longer validates
+ // the JSON (and max_tokens can cut it off): replay an empty
+ // object, but hand the raw text to the loop so the call fails
+ // as "bad args" and the model re-issues it
+ in := strings.TrimSpace(b.input.String())
+ if in == "" {
+ in = "{}"
+ }
+ if json.Valid([]byte(in)) {
+ b.obj["input"] = json.RawMessage(in)
+ } else {
+ b.obj["input"] = map[string]any{}
+ id, _ := b.obj["id"].(string)
+ bad[id] = in
+ }
}
- args := b.input.String()
- if strings.TrimSpace(args) == "" {
- args = "{}"
+ out = append(out, mustJSON(b.obj))
+ }
+ res := anthResponse(out, stop, sd, usage)
+ for j, c := range res.Calls {
+ if in, ok := bad[c.ID]; ok {
+ res.Calls[j].Args = json.RawMessage(in)
}
- res.Calls = append(res.Calls, Call{ID: b.id, Name: b.name, Args: json.RawMessage(args)})
}
return res, nil
}
+// convMessages maps neutral history to Messages API turns. Assistant
+// turns produced by this provider replay their raw blocks verbatim
+// (thinking blocks must come back unchanged); consecutive tool results
+// are merged into one user message, as parallel tool use expects.
func (a *anthropic) convMessages(in []Msg) []map[string]any {
var out []map[string]any
+ var results []anthBlock
+ flush := func() {
+ if len(results) > 0 {
+ out = append(out, map[string]any{"role": "user", "content": results})
+ results = nil
+ }
+ }
for _, m := range in {
+ if m.Role != "tool" {
+ flush()
+ }
switch m.Role {
case "user":
out = append(out, map[string]any{"role": "user", "content": []anthBlock{{"type": "text", "text": m.Text}}})
case "assistant":
+ if m.RawProvider == "anthropic" && len(m.RawSteps) > 0 {
+ var blocks []json.RawMessage
+ if json.Unmarshal(m.RawSteps, &blocks) == nil && len(blocks) > 0 {
+ out = append(out, map[string]any{"role": "assistant", "content": blocks})
+ continue
+ }
+ }
var blocks []anthBlock
if m.Text != "" {
blocks = append(blocks, anthBlock{"type": "text", "text": m.Text})
@@ -212,13 +394,17 @@ func (a *anthropic) convMessages(in []Msg) []map[string]any {
}
blocks = append(blocks, anthBlock{"type": "tool_use", "id": cl.ID, "name": cl.Name, "input": input})
}
+ if len(blocks) == 0 {
+ blocks = append(blocks, anthBlock{"type": "text", "text": "(no output)"})
+ }
out = append(out, map[string]any{"role": "assistant", "content": blocks})
case "tool":
- out = append(out, map[string]any{"role": "user", "content": []anthBlock{{
+ results = append(results, anthBlock{
"type": "tool_result", "tool_use_id": m.CallID,
"content": m.Text, "is_error": m.IsError,
- }}})
+ })
}
}
+ flush()
return out
}
diff --git a/internal/agent/anthropic_effort_test.go b/internal/agent/anthropic_effort_test.go
new file mode 100644
index 0000000..394ba24
--- /dev/null
+++ b/internal/agent/anthropic_effort_test.go
@@ -0,0 +1,56 @@
+package agent
+
+import (
+ "context"
+ "encoding/json"
+ "io"
+ "net/http"
+ "net/http/httptest"
+ "strings"
+ "sync/atomic"
+ "testing"
+)
+
+func TestAnthropicEffortOnlyWhereSupported(t *testing.T) {
+ a := &anthropic{base: "https://api.anthropic.com"}
+ for model, want := range map[string]bool{
+ "claude-haiku-4-5-20251001": false, "claude-sonnet-4-5": false,
+ "claude-opus-4-5": true, "claude-sonnet-4-6": true, "claude-opus-5-5": true, "claude-sonnet-5-5": true, "claude-fable-5-1": true,
+ } {
+ _, has := a.body(&Request{Model: model, Effort: "low"}, false)["output_config"]
+ if has != want {
+ t.Errorf("%s: effort sent = %v, want %v", model, has, want)
+ }
+ }
+}
+
+func TestAnthropicRetriesWithoutEffortOn400(t *testing.T) {
+ var calls atomic.Int32
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ calls.Add(1)
+ raw, _ := io.ReadAll(r.Body)
+ var body map[string]any
+ _ = json.Unmarshal(raw, &body)
+ if _, has := body["output_config"]; has {
+ w.WriteHeader(400)
+ _, _ = w.Write([]byte(`{"type":"error","error":{"type":"invalid_request_error","message":"This model does not support the effort parameter."}}`))
+ return
+ }
+ _, _ = w.Write([]byte(`{"content":[{"type":"text","text":"4 validators"}],"stop_reason":"end_turn","usage":{"input_tokens":10,"output_tokens":3}}`))
+ }))
+ defer srv.Close()
+ a := &anthropic{base: srv.URL}
+ // a model the pattern thinks supports effort, but the API says no
+ res, err := a.Chat(context.Background(), &Request{Model: "claude-opus-5-5", Effort: "low", Messages: []Msg{{Role: "user", Text: "hi"}}})
+ if err != nil || !strings.Contains(res.Text, "4 validators") {
+ t.Fatalf("%v %v", res, err)
+ }
+ if calls.Load() != 2 || !a.noEffort {
+ t.Fatalf("calls=%d noEffort=%v", calls.Load(), a.noEffort)
+ }
+ // later requests skip it straight away
+ _, _ = a.Chat(context.Background(), &Request{Model: "claude-opus-5-5", Effort: "low", Messages: []Msg{{Role: "user", Text: "hi"}}})
+ if calls.Load() != 3 {
+ t.Fatalf("retried again: calls=%d", calls.Load())
+ }
+}
diff --git a/internal/agent/builtin_cmds.go b/internal/agent/builtin_cmds.go
new file mode 100644
index 0000000..db8ad2b
--- /dev/null
+++ b/internal/agent/builtin_cmds.go
@@ -0,0 +1,43 @@
+package agent
+
+import "github.com/abhijitkrm/cometcli/internal/settings"
+
+// builtinCommands ship with cometcli. A user or project command with the
+// same name overrides one.
+var builtinCommands = map[string]*settings.Command{
+ "incident": {
+ Name: "incident", Scope: "builtin", ArgHint: "[what you see]",
+ Description: "Work an incident to resolution: triage, match a known case, fix the root cause, wait, verify, report",
+ Body: `Work this incident on the node to resolution. Keep the operator's checklist current with todo_write; don't stop between steps unless something needs their decision.
+
+1. node.triage (since=2h if the problem started earlier). Note which sources were unavailable.
+2. Take the top matched case: kb.show it, run its confirm steps. If confirmation fails, move to the next case or kb.search the symptoms/log lines. No case at all → investigate from the signals and logs (node.logs, bash read-only).
+3. If several cases match, fix the one that CAUSES the others first (disk full → crash → behind → jailed: disk first).
+4. Apply the fix steps by CALLING the tools. [change] and [tx] steps are approved at the tool's own prompt: state in one line what you're doing and why, then make the call in the same step — never stop to ask for approval in text. Never do anything a case lists under NEVER.
+5. Wait for the effect with wait.until (synced, signing, in-consensus, or condition=signal value=""). If progress stalls, re-triage instead of waiting longer.
+6. Verify with the case's verify steps, then node.triage again: the case must no longer match.
+7. Report: root cause with evidence, actions taken (tx hashes), final state. If you found a cause no case described, propose a kb.add case (with a test fixture) for the operator to approve.
+
+What the operator sees: $ARGUMENTS`,
+ },
+ "recover-jail": {
+ Name: "recover-jail", Scope: "builtin", ArgHint: "[notes]",
+ Description: "Recover a jailed validator end to end: diagnose, fix the cause, sync, unjail, verify it signs",
+ Body: `The validator on this node is jailed (or may be). Run the recovery to completion — keep the operator's checklist current with todo_write and don't stop between steps unless something needs their decision.
+
+1. Diagnose with val.jail-check. If it is TOMBSTONED, stop: explain, and do not unjail or restart the signer.
+2. Root cause: from its evidence (jail line, log errors, restarts, OOM kills, disk, peers) name the most likely cause. If it is still present — no peers, disk full, signer errors, crash loop — fix it first (propose the exact change; the operator approves). Otherwise the validator will be jailed again.
+3. Sync: if the node isn't synced, wait.until condition=synced timeout=3600. If the progress says it is NOT advancing, investigate (node.peers, node.logs) instead of waiting longer.
+4. If the jail period hasn't ended, wait.until condition=unjailable.
+5. val.unjail. If it fails, read the reason and the → hint, fix that cause (raise gas-price, self-delegate to the minimum, fund fees, wait longer…) and retry — at most 3 attempts.
+6. Confirm: wait.until condition=in-consensus window=50 min_signed_pct=95 timeout=1800, then val.consensus.
+7. Report: why it was jailed and the root cause (with the evidence), what you did (tx hashes), and the final consensus numbers.
+
+(This is the knowledge-base case val-jailed-downtime as a ready procedure; for anything else use /incident.)
+
+Operator notes: $ARGUMENTS`,
+ },
+}
+
+// builtinNeedsNode lists built-ins that only make sense in node mode.
+var builtinNeedsNode = map[string]bool{"recover-jail": true, "incident": true}
diff --git a/internal/agent/commands.go b/internal/agent/commands.go
index c524d2e..9e2ac7f 100644
--- a/internal/agent/commands.go
+++ b/internal/agent/commands.go
@@ -1,11 +1,16 @@
package agent
import (
+ "context"
"errors"
"fmt"
+ "path/filepath"
+ "sort"
"strings"
+ "github.com/abhijitkrm/cometcli/internal/config"
"github.com/abhijitkrm/cometcli/internal/runbook"
+ "github.com/abhijitkrm/cometcli/internal/settings"
"github.com/abhijitkrm/cometcli/internal/toolkit"
)
@@ -21,17 +26,29 @@ type CmdResult struct {
// /runbook so the run goes through the normal approval gate and
// streams like any other request.
Prompt string
+ // Job, when set, is slow work (a model call) the front-end runs off
+ // its UI loop; its returned text is shown like Text.
+ Job func(context.Context) (string, error)
}
// CommandHelp lists the shared slash commands; front-ends append their own.
-const CommandHelp = ` /mode [ops|readonly] show or set the approval posture
+const CommandHelp = ` /mode [ops|accept-edits|readonly|bypass] show or set the approval posture
/approve [local-change on|off] show or toggle autopilot (on-chain can never be autopiloted)
/model [name | provider name] show or switch the LLM
/runbook [name] list runbooks, or have the agent run one
/tools list tools with how this session treats each
+ /one [profile|off] switch to node mode for a profile, or back to general
/profile active profile
/audit audit log path + session id
- /reset clear conversation, start a new audit session`
+ /reset, /clear clear conversation, start a new audit session
+ /status current setup: model, mode, scope, memory, MCP (+ live node snapshot in node mode)
+ /compact [focus] summarize the conversation to free context
+ /cost token usage for this session
+ /effort [low|medium|high|xhigh|max|default] show or set reasoning depth
+ /permissions show permission rules
+ /allow|/ask|/deny add a rule for this session, e.g. /allow bash(docker logs:*)
+ /sessions list saved sessions
+ /resume load a saved session into this one`
// RunCommand executes a shared slash command. a may be nil when no LLM
// provider is configured — commands that need the agent say so.
@@ -45,11 +62,57 @@ func RunCommand(a *Agent, c *toolkit.Context, reg *toolkit.Registry, line string
if Offline() {
return ErrOffline
}
- return fmt.Errorf("no agent configured — set agent.provider in the profile")
+ return fmt.Errorf("no agent provider — run `cometcli config set agent.provider `")
}
return nil
}
switch f[0] {
+ case "/help":
+ return CmdResult{Text: HelpText(a)}, nil
+
+ case "/clear":
+ return RunCommand(a, c, reg, "/reset")
+
+ case "/status":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ // the current setup, like Claude Code's /status; in node mode also
+ // a live snapshot of the node (as a background job: never block)
+ a.loadMemory()
+ var b strings.Builder
+ scope := "general (this machine)"
+ if a.Node() {
+ p := a.Ctx.Profile
+ scope = fmt.Sprintf("node %s — %s, %s via %s", p.Name, p.ChainID, p.Role, orNone(p.Transport.Type))
+ }
+ fmt.Fprintf(&b, "scope: %s\n", scope)
+ fmt.Fprintf(&b, "model: %s/%s", a.Provider.Name(), a.Model)
+ if a.Effort != "" {
+ fmt.Fprintf(&b, " · effort %s", a.Effort)
+ }
+ fmt.Fprintf(&b, "\nmode: %s\n", strings.TrimPrefix(a.Policy.String(), "mode "))
+ fmt.Fprintf(&b, "cwd: %s\n", a.WorkRoot)
+ fmt.Fprintf(&b, "session: %s", a.ID())
+ if pct := a.ContextPercent(); pct > 0 {
+ fmt.Fprintf(&b, " · context %d%% used", pct)
+ }
+ var mem []string
+ if a.ext != nil {
+ for _, m := range a.ext.memory {
+ mem = append(mem, m.Path)
+ }
+ }
+ fmt.Fprintf(&b, "\nmemory: %s\n", orNone(strings.Join(mem, ", ")))
+ fmt.Fprintf(&b, "mcp: %s", orNone(strings.Join(a.MCPServers(), ", ")))
+ res := CmdResult{Text: b.String()}
+ if a.Node() {
+ res.Job = func(ctx context.Context) (string, error) {
+ return strings.TrimSpace(a.Redact.Text(SnapshotText(a.Ctx))), nil
+ }
+ }
+ return res, nil
+
case "/profile":
if c == nil || c.Profile == nil {
return CmdResult{Text: "no active profile"}, nil
@@ -164,6 +227,209 @@ func RunCommand(a *Agent, c *toolkit.Context, reg *toolkit.Registry, line string
a.Provider, a.Model, a.conf = p, modelOf(p), conf
return CmdResult{Text: fmt.Sprintf("switched to %s/%s (conversation kept)", p.Name(), a.Model)}, nil
+ case "/one", "/node":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ if len(f) == 1 {
+ txt := "general mode — shell and files on this machine; /one for node work"
+ if a.Node() {
+ txt = "node mode: " + a.Ctx.Profile.Name + " — /one off for general mode"
+ }
+ if cfg := a.Ctx.Cfg; cfg != nil && len(cfg.Profiles) > 0 {
+ var names []string
+ for n := range cfg.Profiles {
+ names = append(names, n)
+ }
+ sort.Strings(names)
+ txt += "\nprofiles: " + strings.Join(names, ", ")
+ }
+ return CmdResult{Text: txt}, nil
+ }
+ if f[1] == "off" || f[1] == "general" {
+ a.SwitchScope(nil)
+ return CmdResult{Text: "general mode — node tools off; the shell runs on this machine"}, nil
+ }
+ if a.Ctx.Cfg == nil {
+ return CmdResult{}, fmt.Errorf("no config loaded")
+ }
+ p, ok := a.Ctx.Cfg.Profiles[f[1]]
+ if !ok {
+ return CmdResult{}, fmt.Errorf("no profile %q — /one lists them", f[1])
+ }
+ p.Name = f[1]
+ a.SwitchScope(p)
+ return CmdResult{Text: fmt.Sprintf("node mode: %s (%s, %s via %s) — conversation kept", p.Name, p.Role, p.ChainID, orNone(p.Transport.Type))}, nil
+
+ case "/compact":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ focus := strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), "/compact"))
+ return CmdResult{Text: "compacting…", Job: func(ctx context.Context) (string, error) {
+ if err := a.Compact(ctx, focus); err != nil {
+ return "", err
+ }
+ return "compacted — the summary is attached to your next message", nil
+ }}, nil
+
+ case "/cost", "/usage":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ u, last := a.Usage()
+ txt := fmt.Sprintf("input %s (cached %s) · output %s", humanTok(u.Input), humanTok(u.CacheRead), humanTok(u.Output))
+ if last > 0 {
+ txt += fmt.Sprintf("\ncontext %s of %s — auto-compacts at %s", humanTok(last), humanTok(a.contextWindow()), humanTok(a.compactThreshold()))
+ }
+ return CmdResult{Text: txt}, nil
+
+ case "/effort":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ if len(f) > 1 {
+ e := strings.ToLower(f[1])
+ if e == "default" || e == "auto" {
+ e = ""
+ }
+ if err := ValidEffort(e); err != nil {
+ return CmdResult{}, err
+ }
+ a.Effort = e
+ }
+ if a.Effort == "" {
+ return CmdResult{Text: "effort: model default"}, nil
+ }
+ return CmdResult{Text: "effort: " + a.Effort}, nil
+
+ case "/permissions", "/rules":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ allow, ask, deny := a.Rules.Lists()
+ var b strings.Builder
+ fmt.Fprintf(&b, "%s\n", a.Policy)
+ for _, l := range []struct {
+ name string
+ rules []string
+ }{{"deny", deny}, {"ask", ask}, {"allow", allow}} {
+ if len(l.rules) > 0 {
+ fmt.Fprintf(&b, "%-5s %s\n", l.name, strings.Join(l.rules, ", "))
+ }
+ }
+ b.WriteString("always: transactions ask; key material and state resets are refused")
+ return CmdResult{Text: b.String()}, nil
+
+ case "/allow", "/ask", "/deny":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ rule := strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), f[0]))
+ if rule == "" {
+ return CmdResult{}, fmt.Errorf("usage: %s — e.g. bash(systemctl status:*), edit(./config/**), val.unjail", f[0])
+ }
+ if a.Rules == nil {
+ a.Rules, _ = toolkit.NewRules(nil, nil, nil)
+ }
+ if err := a.Rules.Add(f[0][1:], rule); err != nil {
+ return CmdResult{}, err
+ }
+ return CmdResult{Text: fmt.Sprintf("%s %s (this session; add it to agent.permissions in the profile to keep it)", f[0][1:], rule)}, nil
+
+ case "/sessions":
+ all, err := ListSessions()
+ if err != nil {
+ return CmdResult{}, err
+ }
+ if len(all) == 0 {
+ return CmdResult{Text: "no saved sessions"}, nil
+ }
+ var b strings.Builder
+ for i, s := range all {
+ if i == 20 {
+ fmt.Fprintf(&b, "… %d more\n", len(all)-20)
+ break
+ }
+ fmt.Fprintf(&b, "%s %s %-10s %s\n", s.ID, s.Updated.Local().Format("Jan 02 15:04"), s.Profile, s.Title)
+ }
+ b.WriteString("resume with /resume (a prefix is enough)")
+ return CmdResult{Text: b.String()}, nil
+
+ case "/resume":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ if len(f) < 2 {
+ return RunCommand(a, c, reg, "/sessions")
+ }
+ sf, err := LoadSession(f[1])
+ if err != nil {
+ return CmdResult{}, err
+ }
+ a.Restore(sf)
+ return CmdResult{Text: fmt.Sprintf("resumed %s — %s (%d messages)", sf.ID, sf.Title, len(sf.History))}, nil
+
+ case "/memory":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ a.loadMemory()
+ var b strings.Builder
+ if a.ext == nil || len(a.ext.memory) == 0 {
+ b.WriteString("no memory files loaded\n")
+ }
+ if a.ext != nil {
+ for _, m := range a.ext.memory {
+ fmt.Fprintf(&b, "%-8s %s (%d lines)\n", m.Scope, m.Path, strings.Count(m.Content, "\n")+1)
+ }
+ }
+ b.WriteString("add with /remember (project COMET.md), /remember --user|--node ")
+ return CmdResult{Text: b.String()}, nil
+
+ case "/remember", "/#":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ rest := strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), f[0]))
+ path, err := a.memoryTarget(&rest)
+ if err != nil {
+ return CmdResult{}, err
+ }
+ if rest == "" {
+ return CmdResult{}, fmt.Errorf("usage: /remember [--user|--node] ")
+ }
+ if err := settings.AppendMemory(path, rest); err != nil {
+ return CmdResult{}, err
+ }
+ // the system prompt is frozen: tell the model now, the file
+ // carries it into future sessions
+ a.carry = strings.TrimSpace(a.carry + "\n\n[Operator added to memory (" + path + "): " + a.Redact.Text(rest) + "]")
+ return CmdResult{Text: "remembered in " + path}, nil
+
+ case "/mcp":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ srv := a.MCPServers()
+ if len(srv) == 0 {
+ return CmdResult{Text: "no MCP servers connected — add one with `cometcli mcp add -- [args…]`"}, nil
+ }
+ return CmdResult{Text: "connected: " + strings.Join(srv, ", ") + "\ntheir tools load on demand (tool_search)"}, nil
+
+ case "/agents":
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ if a.ext == nil || len(a.ext.Agents) == 0 {
+ return CmdResult{Text: "only the general subagent — define more in .cometcli/agents/.md (frontmatter: name, description, tools; body: its instructions)"}, nil
+ }
+ var b strings.Builder
+ for _, d := range a.ext.Agents {
+ fmt.Fprintf(&b, "%-16s %s (%s)\n", d.Name, d.Description, d.Path)
+ }
+ return CmdResult{Text: strings.TrimRight(b.String(), "\n")}, nil
+
case "/runbook", "/runbooks":
if len(f) == 1 {
var b strings.Builder
@@ -191,5 +457,54 @@ func RunCommand(a *Agent, c *toolkit.Context, reg *toolkit.Registry, line string
Prompt: fmt.Sprintf("Run runbook %q with runbook.run, then summarize each step's outcome and anything that needs my attention.", rb.Name),
}, nil
}
+ name := strings.TrimPrefix(f[0], "/")
+ args := strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), f[0]))
+ if a != nil && a.ext != nil {
+ if cmd, ok := a.ext.Commands[name]; ok {
+ return a.runCustomCommand(c, cmd, args)
+ }
+ }
+ if cmd, ok := builtinCommands[name]; ok {
+ if err := needAgent(); err != nil {
+ return CmdResult{}, err
+ }
+ if builtinNeedsNode[name] && !a.Node() {
+ return CmdResult{}, fmt.Errorf("/%s needs node mode — /one first", name)
+ }
+ return CmdResult{Text: "/" + name, Prompt: cmd.Expand(args, nil, nil)}, nil
+ }
return CmdResult{}, ErrUnknownCommand
}
+
+// memoryTarget picks the file /remember writes: --user, --node, or the
+// project's COMET.md (the user file without a project).
+func (a *Agent) memoryTarget(rest *string) (string, error) {
+ switch {
+ case strings.HasPrefix(*rest, "--user"):
+ *rest = strings.TrimSpace(strings.TrimPrefix(*rest, "--user"))
+ d, err := config.Dir()
+ return filepath.Join(d, "COMET.md"), err
+ case strings.HasPrefix(*rest, "--node"):
+ *rest = strings.TrimSpace(strings.TrimPrefix(*rest, "--node"))
+ if !a.Node() {
+ return "", fmt.Errorf("--node needs node mode (/one )")
+ }
+ return settings.NodeMemoryPath(a.Ctx.Profile.Name)
+ }
+ if a.ext != nil && a.ext.Settings.Root != "" {
+ return filepath.Join(a.ext.Settings.Root, "COMET.md"), nil
+ }
+ d, err := config.Dir()
+ return filepath.Join(d, "COMET.md"), err
+}
+
+// humanTok renders a token count compactly: 950, 12.3k, 1.2M.
+func humanTok(n int) string {
+ switch {
+ case n >= 1_000_000:
+ return fmt.Sprintf("%.1fM", float64(n)/1e6)
+ case n >= 1000:
+ return fmt.Sprintf("%.1fk", float64(n)/1e3)
+ }
+ return fmt.Sprint(n)
+}
diff --git a/internal/agent/compact.go b/internal/agent/compact.go
new file mode 100644
index 0000000..69a5cb8
--- /dev/null
+++ b/internal/agent/compact.go
@@ -0,0 +1,148 @@
+package agent
+
+import (
+ "context"
+ "fmt"
+ "strings"
+)
+
+// compactPrompt asks the model to summarize the session so far. The
+// summary replaces the whole history ("simple compaction"): nothing from
+// before it is replayed, so no provider-bound state (thinking blocks,
+// thought signatures) survives to be invalidated.
+const compactPrompt = `Summarize this conversation so it can continue from the summary alone — earlier messages and tool output will be discarded.
+
+Do not call any tools. Reply with the summary only, using these sections:
+1. Goal — what the operator asked for, in their words where it matters.
+2. Findings — facts established, with exact values (heights, versions, hosts, paths, error strings, tx hashes).
+3. Actions taken — commands and tools run, changes made, and their results.
+4. Open items — what is still pending, blocked, or awaiting approval.
+5. Next step — what you were about to do.`
+
+// contextWindow returns the model's context size in tokens.
+func (a *Agent) contextWindow() int {
+ if a.conf.ContextWindow > 0 {
+ return a.conf.ContextWindow
+ }
+ m := strings.ToLower(a.Model)
+ switch a.Provider.Name() {
+ case "anthropic":
+ if strings.Contains(m, "haiku") || strings.Contains(m, "-3-") || strings.Contains(m, "-4-5") || strings.Contains(m, "-4-1") || strings.Contains(m, "-4-0") {
+ return 200_000
+ }
+ return 1_000_000
+ case "gemini":
+ return 1_000_000
+ case "openai":
+ if strings.HasPrefix(m, "gpt-4.1") {
+ return 1_000_000
+ }
+ return 128_000
+ case "groq":
+ return 128_000
+ case "openrouter":
+ // the free router picks a model per request; the smallest free
+ // contexts are ~32k, so stay safely under that unless configured
+ return 32_000
+ }
+ return 32_000 // local models: conservative
+}
+
+// compactThreshold is the prompt size that triggers auto-compaction.
+func (a *Agent) compactThreshold() int {
+ if a.CompactAt > 0 {
+ return a.CompactAt
+ }
+ t := a.contextWindow() * 8 / 10
+ if t > 200_000 {
+ t = 200_000 // bound per-request cost on 1M-token models
+ }
+ return t
+}
+
+// promptTokens is the latest prompt size: reported by the provider when
+// available, otherwise estimated at ~4 bytes per token.
+func (a *Agent) promptTokens() int {
+ if a.lastIn > 0 {
+ return a.lastIn
+ }
+ n := len(a.sys)
+ for _, m := range a.history {
+ n += len(m.Text) + len(m.RawSteps)
+ for _, c := range m.Calls {
+ n += len(c.Args) + len(c.Name)
+ }
+ }
+ return n / 4
+}
+
+func (a *Agent) needsCompact() bool {
+ return len(a.history) >= 2 && a.promptTokens() >= a.compactThreshold()
+}
+
+// Compact summarizes the conversation and replaces the history with the
+// summary, which is attached to the next user turn. instructions, when
+// set, tell the summarizer what to focus on (/compact ).
+func (a *Agent) Compact(ctx context.Context, instructions string) error {
+ return a.compactNow(ctx, instructions, false)
+}
+
+// compactNow does the work. midTurn means a tool round is in progress:
+// the summary becomes a user message asking the model to carry on.
+func (a *Agent) compactNow(ctx context.Context, instructions string, midTurn bool) error {
+ if len(a.history) == 0 {
+ return fmt.Errorf("nothing to compact")
+ }
+ prompt := compactPrompt
+ if instructions != "" {
+ prompt += "\n\nOperator focus for this summary: " + a.Redact.Text(instructions)
+ }
+ hist := append([]Msg(nil), a.history...)
+ // a summarizer turn must follow a complete exchange: a trailing user
+ // message would make two user turns in a row on some providers, so
+ // fold the prompt into it
+ if n := len(hist); n > 0 && hist[n-1].Role == "user" {
+ hist[n-1].Text += "\n\n" + prompt
+ } else {
+ hist = append(hist, Msg{Role: "user", Text: prompt})
+ }
+ // same system + tools as the session, so the summarizer reads the
+ // cached prefix; the prompt forbids tool calls and any are ignored
+ req := &Request{
+ Model: a.Model, System: a.system(), Messages: hist,
+ Tools: a.toolDefs(), MaxTok: 8192, Effort: a.Effort,
+ }
+ resp, err := a.Provider.Chat(ctx, req)
+ if err != nil {
+ return err
+ }
+ a.total.Add(resp.Usage)
+ summary := strings.TrimSpace(a.Redact.Text(resp.Text))
+ if summary == "" {
+ return fmt.Errorf("model returned an empty summary")
+ }
+ block := "\n" + summary + "\n"
+ a.history, a.lastIn = nil, 0
+ if midTurn {
+ a.history = []Msg{{Role: "user", Text: block + "\n\nContinue the task in progress from this summary."}}
+ } else {
+ a.carry = block
+ }
+ if lg := a.Audit(); lg != nil {
+ _ = lg.Log("compact", a.profileName(), map[string]any{"summary": summary, "mid_turn": midTurn})
+ }
+ a.emit(Event{Kind: EvNotice, Text: fmt.Sprintf("conversation compacted (%d-char summary)", len(summary))})
+ return nil
+}
+
+// isContextOverflow recognizes "prompt too long" errors across providers.
+func isContextOverflow(err error) bool {
+ s := strings.ToLower(err.Error())
+ return strings.Contains(s, "prompt is too long") ||
+ strings.Contains(s, "context_length_exceeded") ||
+ strings.Contains(s, "context length") ||
+ strings.Contains(s, "context window") ||
+ strings.Contains(s, "maximum context") ||
+ strings.Contains(s, "request too large") || // Groq 413: prompt exceeds the per-minute token budget
+ (strings.Contains(s, "too many tokens") && strings.Contains(s, "input"))
+}
diff --git a/internal/agent/events.go b/internal/agent/events.go
index e42eb17..f470e70 100644
--- a/internal/agent/events.go
+++ b/internal/agent/events.go
@@ -21,6 +21,17 @@ const (
EvToolStart EventKind = "tool_start"
// EvToolResult fires after a tool runs (Err set on failure).
EvToolResult EventKind = "tool_result"
+ // EvThinking is streamed model reasoning (a summary, where the
+ // provider exposes one). Front-ends may show it dimmed or hide it.
+ EvThinking EventKind = "thinking"
+ // EvNotice is a status line from the loop itself: continuation after
+ // an output cutoff, compaction, a refusal, a failed session save.
+ EvNotice EventKind = "notice"
+ // EvProgress is a live status line from a long-running tool (Tool,
+ // Text); front-ends show the latest one instead of printing each.
+ EvProgress EventKind = "progress"
+ // EvTodos carries the agent's updated task checklist (Todos).
+ EvTodos EventKind = "todos"
// EvApproval asks the human to approve a gated action; answer it via
// Approval.Answer. Only emitted when the context uses EventApprover.
EvApproval EventKind = "approval"
@@ -36,6 +47,9 @@ type Event struct {
Args map[string]any `json:"args,omitempty"`
Err string `json:"error,omitempty"`
Approval *Approval `json:"approval,omitempty"`
+ Todos []Todo `json:"todos,omitempty"`
+ // Output is a multi-line preview of a tool's result (EvToolResult).
+ Output string `json:"output,omitempty"`
}
// Approval is a pending human decision. Answer it exactly once.
diff --git a/internal/agent/ext.go b/internal/agent/ext.go
new file mode 100644
index 0000000..7868c56
--- /dev/null
+++ b/internal/agent/ext.go
@@ -0,0 +1,364 @@
+package agent
+
+import (
+ "context"
+ "fmt"
+ "os"
+ "path/filepath"
+ "sort"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/abhijitkrm/cometcli/internal/client/host"
+ "github.com/abhijitkrm/cometcli/internal/hooks"
+ "github.com/abhijitkrm/cometcli/internal/mcpclient"
+ "github.com/abhijitkrm/cometcli/internal/settings"
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+ "github.com/abhijitkrm/cometcli/internal/tools/shell"
+)
+
+// ext holds what settings files contribute to a session.
+type ext struct {
+ Settings *settings.Settings
+ Hooks *hooks.Runner
+ Commands map[string]*settings.Command
+ Agents map[string]*settings.AgentDef
+ mcp []*mcpclient.Client
+ notes []string // startup notices (MCP failures, untrusted items)
+ memory []settings.Memory
+ resumed bool // SessionStart source
+ turnRule []string // allow rules granted for the current turn only
+}
+
+// loadExtensions reads settings, memory and commands for cwd, connects
+// MCP servers (registering their tools) and builds the hook runner.
+// Settings env vars are exported before this returns, so the provider
+// sees API keys set there.
+func loadExtensions(cwd string, reg *toolkit.Registry) (*ext, error) {
+ st, err := settings.Load(cwd)
+ if err != nil {
+ return nil, err
+ }
+ st.ApplyEnv()
+ dir := st.Root
+ if dir == "" {
+ dir = cwd
+ }
+ e := &ext{
+ Settings: st,
+ Hooks: &hooks.Runner{Matchers: st.Hooks, Dir: dir},
+ Commands: settings.LoadCommands(st.Root),
+ Agents: settings.LoadAgents(st.Root),
+ }
+ if len(st.Ignored) > 0 {
+ e.notes = append(e.notes, fmt.Sprintf("project settings in %s are not trusted, so these were skipped: %s — run `cometcli trust` there to enable them",
+ st.Root, strings.Join(st.Ignored, ", ")))
+ }
+ e.connectMCP(reg)
+ return e, nil
+}
+
+func (e *ext) connectMCP(reg *toolkit.Registry) {
+ names := make([]string, 0, len(e.Settings.MCPServers))
+ for n := range e.Settings.MCPServers {
+ names = append(names, n)
+ }
+ sort.Strings(names)
+ var mu sync.Mutex
+ var wg sync.WaitGroup
+ for _, n := range names {
+ wg.Add(1)
+ go func(n string, cfg settings.MCPServer) {
+ defer wg.Done()
+ ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
+ defer cancel()
+ c, err := mcpclient.Connect(ctx, n, cfg)
+ mu.Lock()
+ defer mu.Unlock()
+ if err != nil {
+ e.notes = append(e.notes, err.Error())
+ return
+ }
+ e.mcp = append(e.mcp, c)
+ }(n, e.Settings.MCPServers[n])
+ }
+ wg.Wait()
+ for _, c := range e.mcp {
+ for _, t := range c.Tools {
+ reg.Upsert(mcpclient.NewAgentTool(c, t))
+ }
+ }
+}
+
+// close stops MCP servers.
+func (e *ext) close(reg *toolkit.Registry) {
+ if e == nil {
+ return
+ }
+ for _, c := range e.mcp {
+ for _, t := range c.Tools {
+ reg.Remove(mcpclient.ToolName(c.Name, t.Name))
+ }
+ c.Close()
+ }
+ e.mcp = nil
+}
+
+// isMCP reports whether t came from an MCP server.
+func isMCP(t toolkit.Tool) bool { return strings.HasPrefix(t.Name(), "mcp__") }
+
+// MCPServers lists connected servers with their tool counts.
+func (a *Agent) MCPServers() []string {
+ if a.ext == nil {
+ return nil
+ }
+ var out []string
+ for _, c := range a.ext.mcp {
+ out = append(out, fmt.Sprintf("%s (%d tools)", c.Name, len(c.Tools)))
+ }
+ return out
+}
+
+// memoryText loads COMET.md memory for the current scope.
+func (a *Agent) memoryText() string {
+ if a.ext == nil {
+ return ""
+ }
+ cwd := a.WorkRoot
+ profile := ""
+ if a.Node() {
+ profile = a.Ctx.Profile.Name
+ }
+ a.ext.memory = settings.LoadMemory(a.ext.Settings.Root, cwd, profile)
+ return a.Redact.Text(settings.RenderMemory(a.ext.memory))
+}
+
+// loadMemory reads the memory files if the system prompt hasn't yet
+// (cheap: no node probe, unlike building the whole prompt).
+func (a *Agent) loadMemory() {
+ if a.ext != nil && a.ext.memory == nil {
+ a.memoryText()
+ }
+}
+
+// --- hooks -----------------------------------------------------------------
+
+func (a *Agent) hookInput(event string) hooks.Input {
+ in := hooks.Input{SessionID: a.ID(), Cwd: a.Tools.Cwd(a.WorkRoot), Event: event, PermissionMode: string(a.Policy.Mode)}
+ if p, err := sessionPath(a.ID()); err == nil {
+ in.TranscriptPath = p
+ }
+ return in
+}
+
+// runHook runs an event's hooks and surfaces their operator messages.
+func (a *Agent) runHook(ctx context.Context, in hooks.Input) hooks.Outcome {
+ if a.ext == nil || !a.ext.Hooks.Has(in.Event) {
+ return hooks.Outcome{}
+ }
+ out := a.ext.Hooks.Run(ctx, in)
+ for _, m := range out.Messages {
+ a.emit(Event{Kind: EvNotice, Text: a.Redact.Text(m)})
+ }
+ if lg := a.Audit(); lg != nil && (out.Block || out.Decision != "") {
+ _ = lg.Log("hook", a.profileName(), map[string]any{"event": in.Event, "tool": in.ToolName, "block": out.Block, "decision": out.Decision, "reason": a.Redact.Text(out.Reason)})
+ }
+ return out
+}
+
+// --- custom slash commands ---------------------------------------------------
+
+// runCustomCommand expands a markdown command into a prompt.
+func (a *Agent) runCustomCommand(c *toolkit.Context, cmd *settings.Command, args string) (CmdResult, error) {
+ h, herr := a.Ctx.Host()
+ cwd := a.Tools.Cwd(a.WorkRoot)
+ run := func(command string) string {
+ if herr != nil {
+ return "[" + herr.Error() + "]"
+ }
+ binary := ""
+ if a.Node() {
+ binary = a.Ctx.Profile.Binary
+ }
+ v := shell.Classify(command, shell.Opts{Binary: binary})
+ if v.Forbidden != "" || v.Tier > toolkit.TierDiagnose {
+ return fmt.Sprintf("[!`%s` not run: only read-only commands expand inline — the agent can run it with approval]", command)
+ }
+ ctx, cancel := context.WithTimeout(c, 30*time.Second)
+ defer cancel()
+ script := command
+ if cwd != "" {
+ script = "cd '" + strings.ReplaceAll(cwd, "'", `'\''`) + "' && " + command
+ }
+ res, err := host.Exec(ctx, h, script, 16_000)
+ if err != nil {
+ return "[" + err.Error() + "]"
+ }
+ return strings.TrimRight(res.Output, "\n")
+ }
+ read := func(p string) (string, bool) {
+ if strings.HasPrefix(p, "~/") {
+ home, _ := os.UserHomeDir()
+ p = filepath.Join(home, p[2:])
+ } else if !filepath.IsAbs(p) {
+ p = filepath.Join(cwd, p)
+ }
+ if _, local := h.(*host.Local); !local || shell.Classify("cat "+p, shell.Opts{}).Forbidden != "" {
+ return "", false
+ }
+ b, err := os.ReadFile(p)
+ if err != nil || len(b) > 100_000 {
+ return "", false
+ }
+ return string(b), true
+ }
+ prompt := a.Redact.Text(cmd.Expand(args, run, read))
+ if prompt == "" {
+ return CmdResult{}, fmt.Errorf("/%s expanded to nothing", cmd.Name)
+ }
+ text := fmt.Sprintf("/%s (%s)", cmd.Name, cmd.Scope)
+ if len(cmd.AllowedTools) > 0 {
+ if cmd.Scope == "project" && !a.ext.Settings.Trusted {
+ text += " — allowed-tools ignored: project not trusted (`cometcli trust`)"
+ } else {
+ a.ext.turnRule = append(a.ext.turnRule, cmd.AllowedTools...)
+ text += " — allowing for this turn: " + strings.Join(cmd.AllowedTools, ", ")
+ }
+ }
+ return CmdResult{Text: text, Prompt: prompt}, nil
+}
+
+// grantTurnRules applies a command's allowed-tools for one turn and
+// returns the cleanup.
+func (a *Agent) grantTurnRules() func() {
+ if a.ext == nil || len(a.ext.turnRule) == 0 {
+ return func() {}
+ }
+ rules := a.ext.turnRule
+ a.ext.turnRule = nil
+ if a.Rules == nil {
+ a.Rules, _ = toolkit.NewRules(nil, nil, nil)
+ }
+ var added []string
+ for _, r := range rules {
+ if a.Rules.Add("allow", r) == nil {
+ added = append(added, r)
+ }
+ }
+ return func() {
+ for _, r := range added {
+ a.Rules.Remove("allow", r)
+ }
+ }
+}
+
+// HelpText is the slash-command help, including custom commands.
+func HelpText(a *Agent) string {
+ s := CommandHelp + "\n /memory, /remember show memory files; add a line to COMET.md\n /mcp connected MCP servers\n /agents subagents available to the task tool"
+ cmds := map[string]*settings.Command{}
+ for n, c := range builtinCommands {
+ cmds[n] = c
+ }
+ if a != nil && a.ext != nil {
+ for n, c := range a.ext.Commands {
+ cmds[n] = c
+ }
+ }
+ var b strings.Builder
+ b.WriteString(s + "\nprocedures and custom commands:")
+ for _, c := range settings.SortedCommands(cmds) {
+ name := "/" + c.Name
+ if c.ArgHint != "" {
+ name += " " + c.ArgHint
+ }
+ fmt.Fprintf(&b, "\n %-29s %s (%s)", name, c.Description, c.Scope)
+ }
+ return b.String()
+}
+
+// --- subagents ---------------------------------------------------------------
+
+const taskToolName = "task"
+
+func (a *Agent) taskToolDef() ToolDef {
+ desc := "Delegate a self-contained investigation to a subagent with its own fresh context — log trawls, multi-file reads, comparing hosts. It has the same tools and permissions, works through the task, and returns only its final report, keeping your context small. Give it everything it needs in prompt."
+ props := map[string]any{
+ "description": toolkit.Str("3-5 word label shown to the operator"),
+ "prompt": toolkit.Str("the complete task: goal, where to look, what to report"),
+ }
+ if a.ext != nil && len(a.ext.Agents) > 0 {
+ var names []string
+ var lines []string
+ for _, d := range a.ext.Agents {
+ names = append(names, d.Name)
+ lines = append(lines, d.Name+": "+d.Description)
+ }
+ sort.Strings(names)
+ sort.Strings(lines)
+ props["agent"] = toolkit.Enum("a specialized subagent (optional): "+strings.Join(lines, "; "), names...)
+ }
+ return ToolDef{Name: taskToolName, Desc: desc, Schema: toolkit.ObjSchema(props, "description", "prompt")}
+}
+
+// runTask runs a subagent to completion and returns its report.
+func (a *Agent) runTask(ctx context.Context, call Call, args map[string]any) Msg {
+ prompt, _ := args["prompt"].(string)
+ label, _ := args["description"].(string)
+ if strings.TrimSpace(prompt) == "" {
+ return a.toolErr(call, taskToolName, "prompt is required")
+ }
+ var def *settings.AgentDef
+ if n, _ := args["agent"].(string); n != "" && a.ext != nil {
+ if def = a.ext.Agents[n]; def == nil {
+ return a.toolErr(call, taskToolName, "no subagent "+n)
+ }
+ }
+ child := &Agent{
+ Provider: a.Provider, Model: a.Model, Reg: a.Reg, Ctx: a.Ctx,
+ MaxIter: 25, Policy: a.Policy, Redact: a.Redact, Stream: a.Stream,
+ SnapshotFn: a.SnapshotFn, MaxTokens: a.MaxTokens, Effort: a.Effort,
+ Rules: a.Rules, Tools: toolkit.NewSession(a.Tools.Cwd("")), WorkRoot: a.WorkRoot,
+ conf: a.conf, ext: a.ext, parent: a, def: def, id: a.ID(), audit: a.audit,
+ }
+ child.OnEvent = func(e Event) {
+ switch e.Kind {
+ case EvToolStart, EvToolResult, EvNotice:
+ e.Tool = "↳ " + e.Tool
+ a.emit(e)
+ }
+ }
+ a.emit(Event{Kind: EvToolStart, Tool: taskToolName, Args: map[string]any{"description": label}})
+ out, err := child.Run(ctx, prompt)
+ a.total.Add(child.total)
+ if err != nil {
+ return a.toolErr(call, taskToolName, "subagent failed: "+err.Error())
+ }
+ a.emit(Event{Kind: EvToolResult, Tool: taskToolName, Text: firstLine(out)})
+ return Msg{Role: "tool", CallID: call.ID, ToolName: taskToolName, Text: clip(out, 30_000)}
+}
+
+// subagentPrompt is appended to a subagent's system prompt.
+func (a *Agent) subagentPrompt() string {
+ s := "\nYOU ARE A SUBAGENT: another agent delegated one task to you. Work through it with your tools, then reply with a concise, complete report — findings with exact evidence, what you changed, what remains. Only your final message is returned.\n"
+ if a.def != nil && a.def.Prompt != "" {
+ s += "\n" + a.def.Prompt + "\n"
+ }
+ return s
+}
+
+// allowedByDef restricts a custom subagent to its declared tools.
+func (a *Agent) allowedByDef(name string) bool {
+ if a.def == nil || len(a.def.Tools) == 0 {
+ return true
+ }
+ for _, t := range a.def.Tools {
+ if r, err := toolkit.ParseRule(t); err == nil && strings.EqualFold(r.Tool, name) {
+ return true
+ }
+ if strings.HasSuffix(t, "*") && strings.HasPrefix(name, strings.TrimSuffix(t, "*")) {
+ return true
+ }
+ }
+ return false
+}
diff --git a/internal/agent/gemini.go b/internal/agent/gemini.go
index 530fc8c..a2aa17c 100644
--- a/internal/agent/gemini.go
+++ b/internal/agent/gemini.go
@@ -1,7 +1,6 @@
package agent
import (
- "bytes"
"context"
"encoding/json"
"fmt"
@@ -58,7 +57,7 @@ func gemInput(msgs []Msg) []json.RawMessage {
{"type": "text", "text": m.Text},
}})
case "assistant":
- if len(m.RawSteps) > 0 {
+ if len(m.RawSteps) > 0 && (m.RawProvider == "" || m.RawProvider == "gemini") {
var steps []json.RawMessage
if json.Unmarshal(m.RawSteps, &steps) == nil {
out = append(out, steps...)
@@ -108,8 +107,16 @@ func (g *gemini) body(r *Request, stream bool) map[string]any {
if len(tools) > 0 {
b["tools"] = tools
}
+ gc := map[string]any{}
if r.MaxTok > 0 {
- b["generation_config"] = map[string]any{"max_output_tokens": r.MaxTok}
+ gc["max_output_tokens"] = r.MaxTok
+ }
+ if r.Effort != "" {
+ gc["thinking_level"] = gemEffort(r.Effort)
+ gc["thinking_summaries"] = "auto"
+ }
+ if len(gc) > 0 {
+ b["generation_config"] = gc
}
if stream {
b["stream"] = true
@@ -117,18 +124,58 @@ func (g *gemini) body(r *Request, stream bool) map[string]any {
return b
}
+// gemEffort maps the neutral effort scale onto thinking_level
+// (minimal | low | medium | high).
+func gemEffort(e string) string {
+ switch e {
+ case "xhigh", "max":
+ return "high"
+ }
+ return e
+}
+
+// gemUsage is the Interaction usage object.
+type gemUsage struct {
+ Input int `json:"total_input_tokens"`
+ Cached int `json:"total_cached_tokens"`
+ Output int `json:"total_output_tokens"`
+ Thought int `json:"total_thought_tokens"`
+}
+
+func (u gemUsage) usage() Usage {
+ return Usage{Input: u.Input, Output: u.Output + u.Thought, CacheRead: u.Cached}
+}
+
+func gemStop(status string, calls int) StopReason {
+ switch status {
+ case "incomplete":
+ return StopMaxTokens
+ case "requires_action":
+ return StopToolUse
+ case "completed":
+ if calls > 0 {
+ return StopToolUse
+ }
+ return StopEnd
+ }
+ return ""
+}
+
func (g *gemini) post(ctx context.Context, body map[string]any) (*http.Response, error) {
- raw, _ := json.Marshal(body)
- url := strings.TrimSuffix(g.base, "/") + "/v1beta/interactions"
- req, err := http.NewRequestWithContext(ctx, "POST", url, bytes.NewReader(raw))
+ raw, err := json.Marshal(body)
if err != nil {
return nil, err
}
- req.Header.Set("content-type", "application/json")
- if g.key != "" {
- req.Header.Set("x-goog-api-key", g.key)
+ resp, err := doHTTP(ctx, g.client(), strings.TrimSuffix(g.base, "/")+"/v1beta/interactions", raw,
+ map[string]string{"x-goog-api-key": g.key})
+ if err != nil {
+ return nil, err
+ }
+ if resp.StatusCode/100 != 2 {
+ defer resp.Body.Close()
+ return nil, apiError(g.Name(), resp)
}
- return g.client().Do(req)
+ return resp, nil
}
// gemStepText extracts assistant text from a parsed step.
@@ -160,6 +207,8 @@ func gemResponse(steps []json.RawMessage, outputText string) *Response {
continue
}
switch st.Type {
+ case "thought":
+ res.Thinking += gemThoughtText(s)
case "function_call":
args := st.Arguments
if len(args) == 0 {
@@ -183,18 +232,42 @@ func gemResponse(steps []json.RawMessage, outputText string) *Response {
return res
}
+// gemThoughtText pulls readable text out of a thought step's summary.
+func gemThoughtText(raw json.RawMessage) string {
+ var t struct {
+ Summary json.RawMessage `json:"summary"`
+ }
+ if json.Unmarshal(raw, &t) != nil || len(t.Summary) == 0 {
+ return ""
+ }
+ var parts []struct {
+ Text string `json:"text"`
+ }
+ if json.Unmarshal(t.Summary, &parts) == nil {
+ var b strings.Builder
+ for _, p := range parts {
+ b.WriteString(p.Text)
+ }
+ return b.String()
+ }
+ var str string
+ if json.Unmarshal(t.Summary, &str) == nil {
+ return str
+ }
+ return ""
+}
+
func (g *gemini) Chat(ctx context.Context, r *Request) (*Response, error) {
resp, err := g.post(ctx, g.body(r, false))
if err != nil {
return nil, err
}
defer resp.Body.Close()
- if resp.StatusCode/100 != 2 {
- return nil, apiError(g.Name(), resp)
- }
var out struct {
OutputText string `json:"output_text"`
Steps []json.RawMessage `json:"steps"`
+ Status string `json:"status"`
+ Usage gemUsage `json:"usage"`
}
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
return nil, err
@@ -202,7 +275,9 @@ func (g *gemini) Chat(ctx context.Context, r *Request) (*Response, error) {
if len(out.Steps) == 0 && out.OutputText == "" {
return nil, fmt.Errorf("%s: empty interaction", g.Name())
}
- return gemResponse(out.Steps, out.OutputText), nil
+ res := gemResponse(out.Steps, out.OutputText)
+ res.Stop, res.Usage = gemStop(out.Status, len(res.Calls)), out.Usage.usage()
+ return res, nil
}
// gemEvent is one SSE data payload from the Interactions stream.
@@ -218,6 +293,8 @@ type gemEvent struct {
Interaction struct {
Steps []json.RawMessage `json:"steps"`
OutputText string `json:"output_text"`
+ Status string `json:"status"`
+ Usage gemUsage `json:"usage"`
} `json:"interaction"`
}
@@ -226,6 +303,8 @@ type gemStreamState struct {
steps map[int]*json.RawMessage // index → assembled step
order []int
argBuf map[int]*strings.Builder
+ status string
+ usage gemUsage
}
func newGemStream() *gemStreamState {
@@ -266,6 +345,7 @@ func (s *gemStreamState) handle(ev *gemEvent, onText func(string)) (completed *R
}
}
case "interaction.completed", "interaction.complete":
+ s.status, s.usage = ev.Interaction.Status, ev.Interaction.Usage
if len(ev.Interaction.Steps) > 0 {
return &Response{RawSteps: mustJSON(ev.Interaction.Steps), Text: ev.Interaction.OutputText}, nil
}
@@ -310,9 +390,6 @@ func (g *gemini) Stream(ctx context.Context, r *Request, onText func(string)) (*
return nil, err
}
defer resp.Body.Close()
- if resp.StatusCode/100 != 2 {
- return nil, apiError(g.Name(), resp)
- }
st := newGemStream()
var text strings.Builder
var completed *Response
@@ -339,12 +416,13 @@ func (g *gemini) Stream(ctx context.Context, r *Request, onText func(string)) (*
if err != nil {
return nil, err
}
+ var res *Response
if completed != nil && len(completed.RawSteps) > 0 {
- res := gemResponse(mustUnmarshalSteps(completed.RawSteps), firstNonEmpty(completed.Text, text.String()))
- return res, nil
+ res = gemResponse(mustUnmarshalSteps(completed.RawSteps), firstNonEmpty(completed.Text, text.String()))
+ } else {
+ res = gemResponse(st.assemble(), text.String())
}
- steps := st.assemble()
- res := gemResponse(steps, text.String())
+ res.Stop, res.Usage = gemStop(st.status, len(res.Calls)), st.usage.usage()
return res, nil
}
diff --git a/internal/agent/loop.go b/internal/agent/loop.go
index aa592e5..cd91f86 100644
--- a/internal/agent/loop.go
+++ b/internal/agent/loop.go
@@ -6,13 +6,18 @@ import (
"encoding/hex"
"encoding/json"
"fmt"
+ "os"
"regexp"
"strings"
+ "sync"
"time"
+ "unicode/utf8"
"github.com/abhijitkrm/cometcli/internal/audit"
"github.com/abhijitkrm/cometcli/internal/config"
+ "github.com/abhijitkrm/cometcli/internal/hooks"
"github.com/abhijitkrm/cometcli/internal/redact"
+ "github.com/abhijitkrm/cometcli/internal/settings"
"github.com/abhijitkrm/cometcli/internal/toolkit"
)
@@ -20,6 +25,10 @@ import (
// session: it owns the conversation, the approval policy, the redactor, and
// a session id stamped on every audit event.
type Agent struct {
+ // steer holds operator messages typed mid-turn (see Steer).
+ steerMu sync.Mutex
+ steer []string
+
Provider Provider
Model string
Reg *toolkit.Registry
@@ -42,22 +51,83 @@ type Agent struct {
// defaults to SnapshotText. Refreshed at most every SnapshotTTL.
SnapshotFn func(*toolkit.Context) string
SnapshotTTL time.Duration
+ // MaxTokens caps each model round's output (0 = provider default).
+ MaxTokens int
+ // Effort is the reasoning depth passed to the provider ("" = default).
+ Effort string
+ // CompactAt is the prompt size, in tokens, at which history is
+ // summarized at the next turn boundary (0 = derived from the model's
+ // context window).
+ CompactAt int
+ // Persist saves the session after every turn so it can be resumed.
+ Persist bool
+ // Rules are the session's permission rules (allow/ask/deny).
+ Rules *toolkit.Rules
+ // Tools is shell/file state shared by general tools (working
+ // directory, files read).
+ Tools *toolkit.Session
+ // WorkRoot anchors relative permission patterns and accept-edits.
+ WorkRoot string
+ // AppendSystem is operator text added to the system prompt
+ // (--append-system-prompt).
+ AppendSystem string
conf config.AgentConf
id string
+ created time.Time
audit *audit.Logger
history []Msg
+ sys string // system prompt, frozen per session (cache prefix)
+ toolSig string // tool names last advertised
+ carry string // compaction summary awaiting the next user turn
snap string
snapAt time.Time
calls int
toolTrunc int
+ lastIn int // prompt tokens of the latest round (0 = unknown)
+ total Usage // session token totals
+ todos []Todo
+ loaded map[string]bool // tools loaded with tool_search
+ rounds int // model rounds in the latest Run
+ ownCtx bool // a.Ctx was built by SwitchScope
+ ext *ext // settings, hooks, commands, MCP servers
+ parent *Agent // set on subagents
+ def *settings.AgentDef
}
-// New builds an agent for a context from the profile's agent config.
+// New builds an agent for a context. With a profile it runs in node mode
+// (node tools, validator rules, live snapshot); without one, in general
+// mode on the operator's machine. Agent settings come from the global
+// agent section overridden by the profile's.
func New(c *toolkit.Context, reg *toolkit.Registry) (*Agent, error) {
- ac := c.Profile.Agent
+ var ac config.AgentConf
+ switch {
+ case c.Cfg != nil:
+ ac = c.Cfg.AgentFor(c.Profile)
+ case c.Profile != nil:
+ ac = c.Profile.Agent
+ }
+ cwd, _ := os.Getwd()
+ e, err := loadExtensions(cwd, reg) // exports settings env before the provider reads keys
+ if err != nil {
+ return nil, err
+ }
+ st := e.Settings
+ if st.Model != "" {
+ ac.Model = st.Model
+ }
+ if st.Effort != "" {
+ ac.Effort = st.Effort
+ }
+ if st.Permissions.DefaultMode != "" {
+ ac.Mode = st.Permissions.DefaultMode
+ }
+ ac.Permissions.Allow = append(ac.Permissions.Allow, st.Permissions.Allow...)
+ ac.Permissions.Ask = append(ac.Permissions.Ask, st.Permissions.Ask...)
+ ac.Permissions.Deny = append(ac.Permissions.Deny, st.Permissions.Deny...)
prov, err := NewProvider(ac)
if err != nil {
+ e.close(reg)
return nil, err
}
pol, err := PolicyFrom(ac)
@@ -65,16 +135,28 @@ func New(c *toolkit.Context, reg *toolkit.Registry) (*Agent, error) {
return nil, err
}
a := &Agent{
- Provider: prov,
- Model: def(ac.Model, ""),
- Reg: reg,
- Ctx: c,
- MaxIter: 16,
- Policy: pol,
- Redact: RedactorFor(c.Profile),
- Stream: !ac.NoStream,
- conf: ac,
+ Provider: prov,
+ Model: def(ac.Model, ""),
+ Reg: reg,
+ Ctx: c,
+ MaxIter: def0(ac.MaxTurns, 200),
+ Policy: pol,
+ Redact: RedactorFor(c.Profile),
+ Stream: !ac.NoStream,
+ MaxTokens: ac.MaxTokens,
+ Effort: ac.Effort,
+ CompactAt: ac.CompactAt,
+ conf: ac,
+ }
+ if err := ValidEffort(a.Effort); err != nil {
+ return nil, err
+ }
+ if a.Rules, err = toolkit.NewRules(ac.Permissions.Allow, ac.Permissions.Ask, ac.Permissions.Deny); err != nil {
+ return nil, err
}
+ a.WorkRoot = cwd
+ a.Tools = toolkit.NewSession("")
+ a.ext = e
if a.Model == "" {
a.Model = modelOf(prov)
}
@@ -103,10 +185,28 @@ func modelOf(p Provider) string {
return t.model
case *openai:
return t.model
+ case *gemini:
+ return t.model
}
return ""
}
+func def0(v, d int) int {
+ if v == 0 {
+ return d
+ }
+ return v
+}
+
+// ValidEffort accepts "" or one of the neutral effort levels.
+func ValidEffort(e string) error {
+ switch e {
+ case "", "minimal", "low", "medium", "high", "xhigh", "max":
+ return nil
+ }
+ return fmt.Errorf("unknown effort %q — want low | medium | high | xhigh | max", e)
+}
+
// ID returns the session id (assigned lazily on first use).
func (a *Agent) ID() string {
if a.id == "" {
@@ -129,9 +229,17 @@ func (a *Agent) Audit() *audit.Logger {
func (a *Agent) Reset() {
a.history = nil
a.id, a.audit = "", nil
- a.snap = ""
+ a.snap, a.sys, a.toolSig, a.carry = "", "", "", ""
+ a.lastIn, a.total = 0, Usage{}
+ a.created = time.Time{}
+ a.todos, a.loaded = nil, nil
+ a.Tools = toolkit.NewSession("")
}
+// Usage returns the session's accumulated token usage and the size of
+// the latest prompt (0 when the provider doesn't report it).
+func (a *Agent) Usage() (total Usage, lastPrompt int) { return a.total, a.lastIn }
+
func (a *Agent) emit(e Event) {
if a.OnEvent != nil {
a.OnEvent(e)
@@ -139,7 +247,7 @@ func (a *Agent) emit(e Event) {
}
func (a *Agent) profileName() string {
- if a.Ctx != nil && a.Ctx.Profile != nil {
+ if a.Node() {
return a.Ctx.Profile.Name
}
return ""
@@ -150,10 +258,13 @@ func (a *Agent) profileName() string {
func (a *Agent) toolDefs() []ToolDef {
var out []ToolDef
for _, t := range a.Reg.All() {
- if toolkit.IsLongRunning(t) {
- continue // watchers never return inside a turn
+ if toolkit.IsLongRunning(t) || toolkit.IsOperatorOnly(t) {
+ continue // watchers never return inside a turn; key management is the operator's
}
- if !a.Policy.Allows(t.Tier()) {
+ if !a.Policy.Allows(t.Tier()) && !toolkit.IsDynamic(t) {
+ continue
+ }
+ if !a.advertised(t) || !a.allowedByDef(t.Name()) {
continue
}
out = append(out, ToolDef{
@@ -162,54 +273,227 @@ func (a *Agent) toolDefs() []ToolDef {
Schema: t.Schema(),
})
}
+ out = append(out, todoToolDef)
+ if a.deferred() {
+ out = append(out, toolSearchDef)
+ }
+ if a.parent == nil {
+ out = append(out, a.taskToolDef())
+ }
+ if d, ok := a.useNodeDef(); ok {
+ out = append(out, d)
+ }
return out
}
func toolFnName(n string) string { return strings.ReplaceAll(n, ".", "__") }
+// maxContinuations bounds automatic "continue" rounds after a reply is
+// cut off by the output-token limit.
+const maxContinuations = 3
+
// Run processes one user turn, executing tools until the model finishes.
func (a *Agent) Run(ctx context.Context, input string) (string, error) {
+ out, err := a.run(ctx, input)
+ if a.Persist {
+ if serr := a.Save(); serr != nil && err == nil {
+ a.emit(Event{Kind: EvNotice, Text: "session not saved: " + serr.Error()})
+ }
+ }
+ return out, err
+}
+
+func (a *Agent) run(ctx context.Context, input string) (string, error) {
a.calls = 0
+ if a.created.IsZero() {
+ a.created = time.Now()
+ }
+ // compact only at a turn boundary — never inside a tool round
+ if a.needsCompact() {
+ if err := a.Compact(ctx, ""); err != nil {
+ a.emit(Event{Kind: EvNotice, Text: "auto-compact failed: " + err.Error()})
+ }
+ }
+ if a.ext != nil && a.parent == nil {
+ for _, n := range a.ext.notes {
+ a.emit(Event{Kind: EvNotice, Text: n})
+ }
+ a.ext.notes = nil
+ if a.sys == "" || a.ext.resumed {
+ in := a.hookInput(hooks.SessionStart)
+ in.Source = "startup"
+ if a.ext.resumed {
+ in.Source = "resume"
+ }
+ a.ext.resumed = false
+ if out := a.runHook(ctx, in); out.Context != "" {
+ a.carry = strings.TrimSpace(a.carry + "\n\n\n" + a.Redact.Text(out.Context) + "\n")
+ }
+ }
+ in := a.hookInput(hooks.UserPromptSubmit)
+ in.Prompt = input
+ out := a.runHook(ctx, in)
+ if out.Block {
+ return "", fmt.Errorf("prompt blocked by hook: %s", out.Reason)
+ }
+ if out.Context != "" {
+ input += "\n\n\n" + out.Context + "\n"
+ }
+ }
+ defer a.grantTurnRules()()
input = a.Redact.Text(input)
- a.history = append(a.history, Msg{Role: "user", Text: input})
if lg := a.Audit(); lg != nil {
_ = lg.Log(audit.KindPrompt, a.profileName(), map[string]any{"text": input})
}
+ a.history = append(a.history, Msg{Role: "user", Text: a.turnPrefix() + input})
+ var texts []string
+ conts, compacted, stops := 0, false, 0
+ a.rounds = 0
for i := 0; i < a.MaxIter; i++ {
- req := &Request{
- Model: a.Model,
- System: a.system(),
- Messages: a.history,
- Tools: a.toolDefs(),
- MaxTok: 4096,
- }
+ a.rounds++
+ req := a.request()
resp, err := a.chat(ctx, req)
+ if err != nil && !compacted && isContextOverflow(err) {
+ // the prompt outgrew the window mid-turn: summarize everything
+ // so far and carry on from the summary
+ compacted = true
+ a.emit(Event{Kind: EvNotice, Text: "context window full — compacting and continuing"})
+ if cerr := a.compactNow(ctx, "", true); cerr != nil {
+ return "", fmt.Errorf("%w (compaction failed: %v)", err, cerr)
+ }
+ i--
+ continue
+ }
if err != nil {
return "", err
}
- a.history = append(a.history, Msg{Role: "assistant", Text: resp.Text, Calls: resp.Calls, RawSteps: resp.RawSteps})
+ a.total.Add(resp.Usage)
+ if resp.Usage.Input > 0 {
+ a.lastIn = resp.Usage.Input + resp.Usage.Output
+ }
+ a.history = append(a.history, Msg{
+ Role: "assistant", Text: resp.Text, Calls: resp.Calls,
+ RawSteps: resp.RawSteps, RawProvider: rawProvider(a.Provider, resp),
+ })
a.logLLM(i, resp)
shown := a.Redact.Text(resp.Text)
if shown != "" {
a.emit(Event{Kind: EvText, Text: shown})
+ texts = append(texts, shown)
+ }
+ switch {
+ case resp.Stop == StopRefusal:
+ msg := "the model declined this request"
+ if resp.StopDetail != "" {
+ msg += " (" + resp.StopDetail + ")"
+ }
+ a.emit(Event{Kind: EvNotice, Text: msg})
+ return strings.Join(texts, "\n"), fmt.Errorf("%s", msg)
+ case resp.Stop == StopMaxTokens && len(resp.Calls) == 0:
+ if conts >= maxContinuations {
+ a.emit(Event{Kind: EvNotice, Text: "reply truncated at the output-token limit"})
+ return strings.Join(texts, ""), nil
+ }
+ conts++
+ a.emit(Event{Kind: EvNotice, Text: "reply hit the output-token limit — continuing"})
+ a.history = append(a.history, Msg{Role: "user", Text: "Your reply was cut off by the output limit. Continue exactly where you stopped, without repeating anything."})
+ continue
+ }
+ if (resp.Done || len(resp.Calls) == 0) && a.parent == nil && stops < 3 && a.ext != nil && a.ext.Hooks.Has(hooks.Stop) {
+ in := a.hookInput(hooks.Stop)
+ in.StopHookActive = stops > 0
+ if out := a.runHook(ctx, in); out.Block {
+ stops++
+ a.emit(Event{Kind: EvNotice, Text: "Stop hook: " + a.Redact.Text(out.Reason)})
+ a.history = append(a.history, Msg{Role: "user", Text: "[Stop hook] " + a.Redact.Text(out.Reason) + "\nAddress this, then finish."})
+ continue
+ }
}
if resp.Done || len(resp.Calls) == 0 {
+ if conts > 0 {
+ return strings.Join(texts[len(texts)-conts-1:], ""), nil
+ }
return shown, nil
}
for _, call := range resp.Calls {
if err := ctx.Err(); err != nil {
- return "", err
+ // answer every outstanding call so the history stays valid
+ a.history = append(a.history, Msg{Role: "tool", CallID: call.ID, ToolName: toolkit.ResolveName(call.Name), Text: "cancelled by operator", IsError: true})
+ continue
}
a.history = append(a.history, a.execCall(ctx, call))
}
+ if err := ctx.Err(); err != nil {
+ return "", err
+ }
+ if a.parent == nil {
+ a.deliverSteer() // what the operator typed meanwhile reaches the next step
+ }
+ }
+ // out of steps: pause, don't fail — the history is valid (every call
+ // answered), so "continue" picks up exactly here
+ a.emit(Event{Kind: EvNotice, Text: fmt.Sprintf("paused after %d steps — say \"continue\" to keep going (raise the limit with agent.max_turns)", a.MaxIter)})
+ return strings.Join(texts, "\n"), nil
+}
+
+// rawProvider tags raw output with the provider that produced it, so it
+// is only ever replayed to that provider.
+func rawProvider(p Provider, resp *Response) string {
+ if len(resp.RawSteps) == 0 {
+ return ""
+ }
+ return p.Name()
+}
+
+// request builds one model round. The system prompt is frozen for the
+// session and the tool set is stable, so providers can reuse the cached
+// prefix; when the tool set does change (e.g. /mode readonly), replayed
+// thinking bound to the old prefix is dropped first.
+func (a *Agent) request() *Request {
+ tools := a.toolDefs()
+ names := make([]string, len(tools))
+ for i, t := range tools {
+ names[i] = t.Name
+ }
+ sig := strings.Join(names, ",")
+ if a.toolSig != "" && sig != a.toolSig {
+ a.dropBoundRaw()
+ }
+ a.toolSig = sig
+ return &Request{
+ Model: a.Model,
+ System: a.system(),
+ Messages: a.history,
+ Tools: tools,
+ MaxTok: a.MaxTokens,
+ Effort: a.Effort,
+ OnThinking: func(t string) {
+ if a.OnEvent != nil && t != "" {
+ a.emit(Event{Kind: EvThinking, Text: a.Redact.Text(t)})
+ }
+ },
+ }
+}
+
+// dropBoundRaw discards raw assistant output whose validity is tied to
+// the exact request prefix (Anthropic thinking signatures). Those turns
+// fall back to their plain text + tool calls.
+func (a *Agent) dropBoundRaw() {
+ for i := range a.history {
+ if a.history[i].RawProvider == "anthropic" {
+ a.history[i].RawSteps, a.history[i].RawProvider = nil, ""
+ }
}
- return "", fmt.Errorf("agent exceeded %d iterations", a.MaxIter)
}
// chat performs one model round, streaming text deltas when possible.
+// Retries surface as notices so a backoff never looks like a hang.
func (a *Agent) chat(ctx context.Context, req *Request) (*Response, error) {
- if s, ok := a.Provider.(Streamer); ok && a.Stream && a.OnEvent != nil {
+ ctx = WithRetryNotice(ctx, func(attempt int, wait time.Duration, reason string) {
+ a.emit(Event{Kind: EvNotice, Text: fmt.Sprintf("%s: %s — retrying in %s (attempt %d)", a.Provider.Name(), reason, wait.Round(100*time.Millisecond), attempt+1)})
+ })
+ if s, ok := a.Provider.(Streamer); ok && a.Stream {
return s.Stream(ctx, req, func(chunk string) {
a.emit(Event{Kind: EvDelta, Text: chunk})
})
@@ -245,16 +529,33 @@ func (a *Agent) execCall(ctx context.Context, call Call) Msg {
if err := json.Unmarshal(call.Args, &args); err != nil {
return a.toolErr(call, name, "bad args: "+err.Error())
}
+ switch name {
+ case todoToolName:
+ return a.todoWrite(call, args)
+ case toolSearchName:
+ return a.toolSearch(call, args)
+ case useNodeName:
+ if a.parent == nil {
+ return a.useNode(call, args)
+ }
+ case taskToolName:
+ if a.parent == nil {
+ return a.runTask(ctx, call, args)
+ }
+ }
t, ok := a.Reg.Get(name)
if !ok {
return a.toolErr(call, name, "no such tool: "+name)
}
shownArgs := a.Redact.Args(args)
a.emit(Event{Kind: EvToolStart, Tool: name, Tier: t.Tier().String(), Args: shownArgs})
+ if toolkit.IsOperatorOnly(t) {
+ return a.toolErr(call, name, "refused: "+name+" handles key material and is operator-only — give the operator the command: cometcli "+strings.ReplaceAll(name, ".", " "))
+ }
if toolkit.IsLongRunning(t) {
return a.toolErr(call, name, "refused: "+name+" is a long-running watcher and cannot finish inside an agent turn — suggest `cometcli "+strings.ReplaceAll(name, ".", " ")+"` to the operator")
}
- if !a.Policy.Allows(t.Tier()) {
+ if !a.Policy.Allows(t.Tier()) && !toolkit.IsDynamic(t) {
msg := fmt.Sprintf("blocked: %s is a %s tool and the session is in readonly mode — recommend it to the operator instead", name, t.Tier())
a.Audit().ToolSeen(a.profileName(), name, t.Tier().String(), shownArgs, nil, fmt.Errorf("%s", msg), "")
return a.toolErr(call, name, msg)
@@ -264,10 +565,39 @@ func (a *Agent) execCall(ctx context.Context, call Call) Msg {
return a.toolErr(call, name, fmt.Sprintf("tool-call budget exhausted (%d) — stop calling tools and summarize findings so far", a.MaxCalls))
}
- runCtx, cancel := a.toolCtx(ctx, t)
- res, err := t.Run(runCtx, args)
+ runCtx, cancel := a.toolCtx(ctx, t, args)
+ var res *toolkit.Result
+ in := a.hookInput(hooks.PreToolUse)
+ in.ToolName, in.ToolInput = name, args
+ pre := a.runHook(ctx, in)
+ var err error
+ switch {
+ case pre.Block || pre.Decision == "deny":
+ err = fmt.Errorf("blocked by PreToolUse hook: %s", pre.Reason)
+ default:
+ runCtx.HookDecision = pre.Decision
+ err = a.ruleGate(runCtx, t, shownArgs)
+ }
+ if err == nil {
+ res, err = t.Run(runCtx, args)
+ }
runCtx.Close()
cancel()
+ var postNote string
+ if a.ext != nil && a.ext.Hooks.Has(hooks.PostToolUse) && !pre.Block && pre.Decision != "deny" {
+ in := a.hookInput(hooks.PostToolUse)
+ in.ToolName, in.ToolInput = name, args
+ resp := map[string]any{"success": err == nil}
+ if err != nil {
+ resp["error"] = err.Error()
+ } else if res != nil {
+ resp["output"] = res.Text
+ }
+ in.ToolResponse = resp
+ if post := a.runHook(ctx, in); post.Block || post.Context != "" {
+ postNote = strings.TrimSpace(post.Reason + "\n" + post.Context)
+ }
+ }
var text string
if err == nil {
@@ -276,13 +606,14 @@ func (a *Agent) execCall(ctx context.Context, call Call) Msg {
text = res.JSON()
}
limit := a.toolTrunc
+ if ol, ok := t.(toolkit.OutputLimiter); ok && limit == 0 {
+ limit = ol.OutputLimit()
+ }
if limit == 0 {
- limit = 8192
+ limit = 16_000
}
text = ansiRe.ReplaceAllString(text, "") // colored logs waste tokens and confuse models
- if len(text) > limit { // bound context growth
- text = text[:limit] + "\n…[truncated]"
- }
+ text = clip(text, limit) // bound context growth
text = a.Redact.Text(text)
}
if lg := a.Audit(); lg != nil {
@@ -292,10 +623,14 @@ func (a *Agent) execCall(ctx context.Context, call Call) Msg {
}
lg.ToolSeen(a.profileName(), name, t.Tier().String(), shownArgs, data, err, text)
}
+ if postNote != "" {
+ postNote = "\n[PostToolUse hook] " + a.Redact.Text(postNote)
+ }
if err != nil {
- return a.toolErr(call, name, "error: "+err.Error())
+ return a.toolErr(call, name, "error: "+err.Error()+postNote)
}
- a.emit(Event{Kind: EvToolResult, Tool: name, Tier: t.Tier().String(), Text: firstLine(text)})
+ text += postNote
+ a.emit(Event{Kind: EvToolResult, Tool: name, Tier: t.Tier().String(), Text: firstLine(text), Output: preview(text)})
return Msg{Role: "tool", CallID: call.ID, ToolName: name, Text: text}
}
@@ -303,40 +638,122 @@ func (a *Agent) execCall(ctx context.Context, call Call) Msg {
// 90s deadline for short tools, the session audit logger, and the policy's
// approval threshold. Read-only sessions hard-deny any approval request
// that slips past filtering.
-func (a *Agent) toolCtx(ctx context.Context, t toolkit.Tool) (*toolkit.Context, context.CancelFunc) {
+func (a *Agent) toolCtx(ctx context.Context, t toolkit.Tool, args toolkit.Args) (*toolkit.Context, context.CancelFunc) {
parent := &toolkit.Context{
Context: ctx, Profile: a.Ctx.Profile, Cfg: a.Ctx.Cfg, Out: a.Ctx.Out,
- Audit: a.Audit(), Approver: a.Ctx.Approver,
+ Audit: a.Audit(), Approver: a.Ctx.Approver, Chooser: a.Ctx.Chooser, Secret: a.Ctx.Secret,
+ Session: a.Tools, Rules: a.Rules, WorkRoot: a.WorkRoot,
+ ReadOnly: a.Policy.ReadOnly(), AcceptEdits: a.Policy.AcceptEdits(),
}
var sub *toolkit.Context
var cancel context.CancelFunc
- if toolkit.IsLongRunning(t) {
+ switch {
+ case toolkit.IsLongRunning(t):
sub, cancel = toolkit.WithCancel(parent)
- } else {
- sub, cancel = toolkit.WithDeadline(parent, 90*time.Second)
+ default:
+ sub, cancel = toolkit.WithDeadline(parent, toolkit.CallTimeout(t, args))
}
sub.AutoApproveBelow = a.Policy.AutoApproveBelow()
+ sub.ToolName = t.Name()
+ name := t.Name()
+ sub.Progress = func(s string) { a.emit(Event{Kind: EvProgress, Tool: name, Text: a.Redact.Text(s)}) }
if a.Policy.ReadOnly() {
sub.Approver = toolkit.DenyApprover
}
return sub, cancel
}
-// system returns the system prompt plus a cached, redacted live snapshot.
+// ruleGate applies name-level permission rules to registry tools that
+// don't gate themselves (general tools match rules on their own specs):
+// deny refuses, ask confirms first, allow skips the tool's local-change
+// prompt. Transactions keep their own approval regardless.
+func (a *Agent) ruleGate(c *toolkit.Context, t toolkit.Tool, shownArgs map[string]any) error {
+ if toolkit.IsAgentOnly(t) {
+ return nil
+ }
+ d, rule := a.Rules.Decide(toolkit.Request{Tool: t.Name()})
+ switch {
+ case d != toolkit.DecideDeny && c.HookDecision == "ask":
+ d, rule = toolkit.DecideAsk, "PreToolUse hook"
+ case d == toolkit.DecideDefault && c.HookDecision == "allow":
+ d = toolkit.DecideAllow
+ }
+ switch d {
+ case toolkit.DecideDeny:
+ return fmt.Errorf("denied by permission rule %s", rule)
+ case toolkit.DecideAsk:
+ return toolkit.RequireApproval(c, fmt.Sprintf("run %s %s", t.Name(), CompactArgs(shownArgs)), t.Tier(), map[string]any{"rule": rule})
+ case toolkit.DecideAllow:
+ c.AutoApproveBelow = toolkit.TierOnChain
+ }
+ return nil
+}
+
+// system returns the session's system prompt: instructions plus the live
+// snapshot taken when the session started. It is frozen until Reset or
+// Compact — rebuilding it every request would defeat prompt caching and
+// invalidate replayed thinking. Later snapshots ride on user turns.
func (a *Agent) system() string {
+ if a.sys == "" {
+ if a.Node() {
+ a.sys = a.Redact.Text(SystemPrompt(a.Ctx)) + a.memoryText() + a.toolCatalog() + a.snapshot()
+ } else {
+ a.sys = a.Redact.Text(GeneralPrompt(a.Ctx, a.Tools.Cwd(a.WorkRoot))) + a.memoryText() + a.toolCatalog()
+ }
+ if a.parent != nil {
+ a.sys += a.subagentPrompt()
+ }
+ if a.AppendSystem != "" {
+ a.sys += "\n" + a.Redact.Text(a.AppendSystem) + "\n"
+ }
+ }
+ return a.sys
+}
+
+// Node reports whether the session is scoped to a node profile.
+func (a *Agent) Node() bool { return a.Ctx != nil && a.Ctx.Profile != nil }
+
+// Rounds is the number of model rounds the latest Run took.
+func (a *Agent) Rounds() int { return a.rounds }
+
+// snapshot probes the live node digest and records when.
+func (a *Agent) snapshot() string {
+ fn := a.SnapshotFn
+ if fn == nil {
+ fn = SnapshotText
+ }
+ a.snap = a.Redact.Text(fn(a.Ctx))
+ a.snapAt = time.Now()
+ return a.snap
+}
+
+// turnPrefix is prepended to a user turn: a pending compaction summary,
+// and a refreshed live snapshot once the previous one is older than
+// SnapshotTTL. Appending keeps the request prefix byte-stable.
+func (a *Agent) turnPrefix() string {
+ var b strings.Builder
+ if a.carry != "" {
+ b.WriteString(a.carry)
+ b.WriteString("\n\n")
+ a.carry = ""
+ }
+ if a.sys == "" {
+ a.system() // first turn: the snapshot goes in the system prompt
+ return b.String()
+ }
+ if !a.Node() {
+ return b.String()
+ }
ttl := a.SnapshotTTL
if ttl == 0 {
ttl = 60 * time.Second
}
- if a.snap == "" || time.Since(a.snapAt) > ttl {
- fn := a.SnapshotFn
- if fn == nil {
- fn = SnapshotText
- }
- a.snap = a.Redact.Text(fn(a.Ctx))
- a.snapAt = time.Now()
+ if time.Since(a.snapAt) > ttl {
+ b.WriteString("\n")
+ b.WriteString(strings.TrimSpace(a.snapshot()))
+ b.WriteString("\n\n\n")
}
- return a.Redact.Text(SystemPrompt(a.Ctx)) + a.snap
+ return b.String()
}
// History returns a copy of the conversation (for tests and replay).
@@ -345,12 +762,53 @@ func (a *Agent) History() []Msg { return append([]Msg(nil), a.history...) }
// ansiRe matches terminal escape sequences (colors, cursor moves).
var ansiRe = regexp.MustCompile(`\x1b\[[0-9;?]*[ -/]*[@-~]`)
+// preview is the first lines of tool output, for front-ends.
+func preview(s string) string {
+ lines := strings.Split(strings.TrimRight(s, "\n"), "\n")
+ more := 0
+ if len(lines) > 8 {
+ more, lines = len(lines)-8, lines[:8]
+ }
+ out := safeCut(strings.Join(lines, "\n"), 1500)
+ if more > 0 {
+ out += fmt.Sprintf("\n… +%d lines", more)
+ }
+ return out
+}
+
+// clip bounds s to about limit bytes, keeping the head and the tail (log
+// tails usually hold the error) and cutting only at UTF-8 boundaries.
+func clip(s string, limit int) string {
+ if len(s) <= limit {
+ return s
+ }
+ head, tail := limit*2/3, limit/3
+ h := safeCut(s, head)
+ t := s[len(s)-tail:]
+ for len(t) > 0 && !utf8.RuneStart(t[0]) {
+ t = t[1:]
+ }
+ return fmt.Sprintf("%s\n…[%d bytes truncated]…\n%s", h, len(s)-len(h)-len(t), t)
+}
+
+// safeCut returns the longest prefix of s within n bytes that ends on a
+// rune boundary.
+func safeCut(s string, n int) string {
+ if len(s) <= n {
+ return s
+ }
+ for n > 0 && !utf8.RuneStart(s[n]) {
+ n--
+ }
+ return s[:n]
+}
+
func firstLine(s string) string {
if i := strings.IndexByte(s, '\n'); i >= 0 {
s = s[:i]
}
if len(s) > 120 {
- return s[:120] + "…"
+ return safeCut(s, 120) + "…"
}
return s
}
diff --git a/internal/agent/loop_test.go b/internal/agent/loop_test.go
index d28ed75..7bae35f 100644
--- a/internal/agent/loop_test.go
+++ b/internal/agent/loop_test.go
@@ -163,8 +163,11 @@ func TestLoop_IterationLimit(t *testing.T) {
return &toolkit.Result{Text: "ok"}, nil
}})
_, err := a.Run(context.Background(), "loop forever")
- if err == nil || !strings.Contains(err.Error(), "iterations") {
- t.Fatalf("expected iteration limit error, got %v", err)
+ if err != nil {
+ t.Fatalf("the step limit pauses the turn, it doesn't fail it: %v", err)
+ }
+ if prov.calls != a.MaxIter {
+ t.Fatalf("ran %d rounds, limit %d", prov.calls, a.MaxIter)
}
}
diff --git a/internal/agent/openai.go b/internal/agent/openai.go
index c147309..8b12c05 100644
--- a/internal/agent/openai.go
+++ b/internal/agent/openai.go
@@ -1,7 +1,6 @@
package agent
import (
- "bytes"
"context"
"encoding/json"
"fmt"
@@ -87,21 +86,97 @@ func (o *openai) body(r *Request) map[string]any {
if len(tools) > 0 {
body["tools"] = tools
}
+ if r.MaxTok > 0 {
+ if o.hosted() {
+ body["max_completion_tokens"] = r.MaxTok
+ } else {
+ body["max_tokens"] = r.MaxTok // llama.cpp / older vLLM / Ollama
+ }
+ }
+ if r.Effort != "" && o.hosted() {
+ // local OpenAI-compatible servers may reject unknown fields
+ body["reasoning_effort"] = oaiEffort(r.Effort)
+ }
+ if o.Name() == "openrouter" {
+ // OpenRouter's normalized knob: always ask for the reasoning
+ // stream (models without reasoning ignore it)
+ rs := map[string]any{"enabled": true}
+ if r.Effort != "" {
+ rs["effort"] = oaiEffort(r.Effort)
+ }
+ body["reasoning"] = rs
+ }
return body
}
+// hosted reports whether this is OpenAI or Groq proper (vs a local
+// OpenAI-compatible server that may lack newer request fields).
+func (o *openai) hosted() bool {
+ n := o.Name()
+ return n == "openai" || n == "groq"
+}
+
+// oaiEffort maps the neutral effort scale onto reasoning_effort, which
+// tops out at "high" on OpenAI and Groq reasoning models.
+func oaiEffort(e string) string {
+ switch e {
+ case "xhigh", "max":
+ return "high"
+ }
+ return e
+}
+
+func oaiStop(finish string) StopReason {
+ switch finish {
+ case "stop":
+ return StopEnd
+ case "tool_calls", "function_call":
+ return StopToolUse
+ case "length":
+ return StopMaxTokens
+ case "content_filter":
+ return StopRefusal
+ }
+ return ""
+}
+
+// oaiUsage is the chat-completions usage object.
+type oaiUsage struct {
+ Prompt int `json:"prompt_tokens"`
+ Completion int `json:"completion_tokens"`
+ Details struct {
+ Cached int `json:"cached_tokens"`
+ } `json:"prompt_tokens_details"`
+}
+
+func (u *oaiUsage) usage() Usage {
+ if u == nil {
+ return Usage{}
+ }
+ return Usage{Input: u.Prompt, Output: u.Completion, CacheRead: u.Details.Cached}
+}
+
func (o *openai) post(ctx context.Context, body map[string]any) (*http.Response, error) {
- raw, _ := json.Marshal(body)
- url := strings.TrimSuffix(o.base, "/") + "/v1/chat/completions"
- req, err := http.NewRequestWithContext(ctx, "POST", url, bytes.NewReader(raw))
+ raw, err := json.Marshal(body)
if err != nil {
return nil, err
}
- req.Header.Set("content-type", "application/json")
+ hdr := map[string]string{}
if o.key != "" {
- req.Header.Set("authorization", "Bearer "+o.key)
+ hdr["authorization"] = "Bearer " + o.key
}
- return o.client().Do(req)
+ if o.Name() == "openrouter" {
+ hdr["x-title"] = "cometcli" // app attribution on openrouter.ai
+ }
+ resp, err := doHTTP(ctx, o.client(), strings.TrimSuffix(o.base, "/")+"/v1/chat/completions", raw, hdr)
+ if err != nil {
+ return nil, err
+ }
+ if resp.StatusCode/100 != 2 {
+ defer resp.Body.Close()
+ return nil, apiError(o.Name(), resp)
+ }
+ return resp, nil
}
func (o *openai) Chat(ctx context.Context, r *Request) (*Response, error) {
@@ -113,11 +188,15 @@ func (o *openai) Chat(ctx context.Context, r *Request) (*Response, error) {
var out struct {
Choices []struct {
Message struct {
- Content string `json:"content"`
- ToolCalls []oaiToolCall `json:"tool_calls"`
+ Content string `json:"content"`
+ Reasoning string `json:"reasoning"`
+ // DeepSeek / vLLM / some OpenRouter upstreams
+ ReasoningContent string `json:"reasoning_content"`
+ ToolCalls []oaiToolCall `json:"tool_calls"`
} `json:"message"`
FinishReason string `json:"finish_reason"`
} `json:"choices"`
+ Usage *oaiUsage `json:"usage"`
Error *struct {
Message string `json:"message"`
} `json:"error"`
@@ -132,7 +211,13 @@ func (o *openai) Chat(ctx context.Context, r *Request) (*Response, error) {
return nil, fmt.Errorf("%s: no choices", o.Name())
}
ch := out.Choices[0].Message
- res := &Response{Text: ch.Content, Done: len(ch.ToolCalls) == 0}
+ if ch.Reasoning == "" {
+ ch.Reasoning = ch.ReasoningContent
+ }
+ res := &Response{
+ Text: ch.Content, Thinking: ch.Reasoning, Done: len(ch.ToolCalls) == 0,
+ Stop: oaiStop(out.Choices[0].FinishReason), Usage: out.Usage.usage(),
+ }
for _, tc := range ch.ToolCalls {
res.Calls = append(res.Calls, Call{
ID: tc.ID, Name: tc.Function.Name, Args: json.RawMessage(tc.Function.Arguments),
@@ -148,29 +233,34 @@ func (o *openai) Chat(ctx context.Context, r *Request) (*Response, error) {
func (o *openai) Stream(ctx context.Context, r *Request, onText func(string)) (*Response, error) {
body := o.body(r)
body["stream"] = true
+ if o.Name() == "openai" {
+ body["stream_options"] = map[string]any{"include_usage": true}
+ }
resp, err := o.post(ctx, body)
if err != nil {
return nil, err
}
defer resp.Body.Close()
- if resp.StatusCode/100 != 2 {
- return nil, apiError(o.Name(), resp)
- }
type partial struct {
id, name string
args strings.Builder
}
calls := map[int]*partial{}
- var text strings.Builder
+ var text, reasoning strings.Builder
+ var finish string
+ var usage *oaiUsage
err = readSSE(resp.Body, func(data string) error {
if data == "[DONE]" {
return nil
}
var ch struct {
Choices []struct {
- Delta struct {
- Content string `json:"content"`
- ToolCalls []struct {
+ FinishReason string `json:"finish_reason"`
+ Delta struct {
+ Content string `json:"content"`
+ Reasoning string `json:"reasoning"`
+ ReasoningContent string `json:"reasoning_content"`
+ ToolCalls []struct {
Index int `json:"index"`
ID string `json:"id"`
Function struct {
@@ -180,6 +270,10 @@ func (o *openai) Stream(ctx context.Context, r *Request, onText func(string)) (*
} `json:"tool_calls"`
} `json:"delta"`
} `json:"choices"`
+ Usage *oaiUsage `json:"usage"`
+ XGroq *struct {
+ Usage *oaiUsage `json:"usage"`
+ } `json:"x_groq"` // Groq reports stream usage here
Error *struct {
Message string `json:"message"`
} `json:"error"`
@@ -190,10 +284,27 @@ func (o *openai) Stream(ctx context.Context, r *Request, onText func(string)) (*
if ch.Error != nil {
return fmt.Errorf("%s: %s", o.Name(), ch.Error.Message)
}
+ if ch.Usage != nil {
+ usage = ch.Usage
+ } else if ch.XGroq != nil && ch.XGroq.Usage != nil {
+ usage = ch.XGroq.Usage
+ }
if len(ch.Choices) == 0 {
return nil
}
+ if f := ch.Choices[0].FinishReason; f != "" {
+ finish = f
+ }
d := ch.Choices[0].Delta
+ if d.Reasoning == "" {
+ d.Reasoning = d.ReasoningContent
+ }
+ if d.Reasoning != "" {
+ reasoning.WriteString(d.Reasoning)
+ if r.OnThinking != nil {
+ r.OnThinking(d.Reasoning)
+ }
+ }
if d.Content != "" {
text.WriteString(d.Content)
onText(d.Content)
@@ -217,7 +328,7 @@ func (o *openai) Stream(ctx context.Context, r *Request, onText func(string)) (*
if err != nil {
return nil, err
}
- res := &Response{Text: text.String()}
+ res := &Response{Text: text.String(), Thinking: reasoning.String(), Stop: oaiStop(finish), Usage: usage.usage()}
idx := make([]int, 0, len(calls))
for i := range calls {
idx = append(idx, i)
diff --git a/internal/agent/phase1_test.go b/internal/agent/phase1_test.go
new file mode 100644
index 0000000..80129e5
--- /dev/null
+++ b/internal/agent/phase1_test.go
@@ -0,0 +1,584 @@
+package agent
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "net/http"
+ "net/http/httptest"
+ "strings"
+ "sync/atomic"
+ "testing"
+ "time"
+ "unicode/utf8"
+
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+)
+
+// noSleep makes retry backoff instant for the duration of a test.
+func noSleep(t *testing.T) *[]time.Duration {
+ t.Helper()
+ var waits []time.Duration
+ old := sleep
+ sleep = func(ctx context.Context, d time.Duration) error {
+ waits = append(waits, d)
+ return ctx.Err()
+ }
+ t.Cleanup(func() { sleep = old })
+ return &waits
+}
+
+func TestRetryHonorsRetryAfterThenSucceeds(t *testing.T) {
+ waits := noSleep(t)
+ var n atomic.Int32
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ switch n.Add(1) {
+ case 1:
+ w.Header().Set("Retry-After", "7")
+ w.WriteHeader(429)
+ case 2:
+ w.WriteHeader(529) // Anthropic overloaded
+ default:
+ io.WriteString(w, "ok")
+ }
+ }))
+ defer srv.Close()
+ resp, err := doHTTP(context.Background(), srv.Client(), srv.URL, []byte(`{}`), nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ resp.Body.Close()
+ if resp.StatusCode != 200 || n.Load() != 3 {
+ t.Fatalf("status %d after %d attempts", resp.StatusCode, n.Load())
+ }
+ if len(*waits) != 2 || (*waits)[0] != 7*time.Second {
+ t.Fatalf("waits = %v, want Retry-After 7s first", *waits)
+ }
+}
+
+func TestRetrySkipsClientErrors(t *testing.T) {
+ noSleep(t)
+ var n atomic.Int32
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ n.Add(1)
+ w.WriteHeader(400)
+ }))
+ defer srv.Close()
+ resp, err := doHTTP(context.Background(), srv.Client(), srv.URL, []byte(`{}`), nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ resp.Body.Close()
+ if resp.StatusCode != 400 || n.Load() != 1 {
+ t.Fatalf("400 retried: %d attempts", n.Load())
+ }
+}
+
+func TestRetryGivesUpAfterMax(t *testing.T) {
+ noSleep(t)
+ var n atomic.Int32
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ n.Add(1)
+ w.WriteHeader(503)
+ }))
+ defer srv.Close()
+ resp, err := doHTTPWith(context.Background(), srv.Client(), srv.URL, []byte(`{}`), nil, retryPolicy{Max: 2, Base: time.Millisecond, Cap: time.Second})
+ if err != nil {
+ t.Fatal(err)
+ }
+ resp.Body.Close()
+ if resp.StatusCode != 503 || n.Load() != 3 {
+ t.Fatalf("want final 503 after 3 attempts, got %d after %d", resp.StatusCode, n.Load())
+ }
+}
+
+// sseServer serves a fixed SSE body and records each request body.
+func sseServer(t *testing.T, events string, bodies *[]map[string]any, hdrs *[]http.Header) *httptest.Server {
+ t.Helper()
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ var b map[string]any
+ _ = json.NewDecoder(r.Body).Decode(&b)
+ *bodies = append(*bodies, b)
+ if hdrs != nil {
+ *hdrs = append(*hdrs, r.Header.Clone())
+ }
+ w.Header().Set("content-type", "text/event-stream")
+ io.WriteString(w, events)
+ }))
+ t.Cleanup(srv.Close)
+ return srv
+}
+
+func sseBody(evs ...string) string {
+ var b strings.Builder
+ for _, e := range evs {
+ b.WriteString("data: " + e + "\n\n")
+ }
+ return b.String()
+}
+
+func TestAnthropicStreamThinkingToolUseAndReplay(t *testing.T) {
+ var bodies []map[string]any
+ srv := sseServer(t, sseBody(
+ `{"type":"message_start","message":{"usage":{"input_tokens":10,"cache_read_input_tokens":90,"cache_creation_input_tokens":0}}}`,
+ `{"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":""}}`,
+ `{"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"check the "}}`,
+ `{"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"node"}}`,
+ `{"type":"content_block_delta","index":0,"delta":{"type":"signature_delta","signature":"SIG123"}}`,
+ `{"type":"content_block_start","index":1,"content_block":{"type":"text","text":""}}`,
+ `{"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"Looking."}}`,
+ `{"type":"content_block_start","index":2,"content_block":{"type":"tool_use","id":"tu1","name":"node__status","input":{}}}`,
+ `{"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"{\"lines\":"}}`,
+ `{"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"5}"}}`,
+ `{"type":"message_delta","delta":{"stop_reason":"tool_use"},"usage":{"output_tokens":42}}`,
+ ), &bodies, nil)
+ p := &anthropic{key: "k", model: "claude-opus-5-5", base: srv.URL, hc: srv.Client()}
+ var thought, text string
+ req := &Request{Model: "claude-opus-5-5", System: "sys", Effort: "high",
+ Messages: []Msg{{Role: "user", Text: "hi"}},
+ Tools: []ToolDef{{Name: "node__status", Schema: map[string]any{"type": "object"}}},
+ OnThinking: func(s string) { thought += s }}
+ res, err := p.Stream(context.Background(), req, func(s string) { text += s })
+ if err != nil {
+ t.Fatal(err)
+ }
+ if thought != "check the node" || text != "Looking." || res.Thinking != "check the node" {
+ t.Fatalf("thinking=%q text=%q", thought, text)
+ }
+ if res.Stop != StopToolUse || len(res.Calls) != 1 || string(res.Calls[0].Args) != `{"lines":5}` {
+ t.Fatalf("stop=%s calls=%+v", res.Stop, res.Calls)
+ }
+ if res.Usage != (Usage{Input: 100, Output: 42, CacheRead: 90}) {
+ t.Fatalf("usage = %+v", res.Usage)
+ }
+ if !strings.Contains(string(res.RawSteps), `"signature":"SIG123"`) {
+ t.Fatalf("signature lost from raw blocks: %s", res.RawSteps)
+ }
+ b := bodies[0]
+ if b["cache_control"] == nil {
+ t.Error("no top-level cache_control")
+ }
+ if th, _ := b["thinking"].(map[string]any); th["type"] != "adaptive" {
+ t.Errorf("thinking = %v", b["thinking"])
+ }
+ if oc, _ := b["output_config"].(map[string]any); oc["effort"] != "high" {
+ t.Errorf("output_config = %v", b["output_config"])
+ }
+ if tl, _ := b["tools"].([]any); len(tl) != 1 || tl[0].(map[string]any)["eager_input_streaming"] != true {
+ t.Errorf("tools = %v", b["tools"])
+ }
+ if _, ok := b["fallbacks"]; ok {
+ t.Error("fallbacks sent to a non-first-party base URL")
+ }
+
+ // the next request replays the assistant turn byte-for-byte, with both
+ // tool results in one user message
+ hist := []Msg{
+ {Role: "user", Text: "hi"},
+ {Role: "assistant", Text: res.Text, Calls: res.Calls, RawSteps: res.RawSteps, RawProvider: "anthropic"},
+ {Role: "tool", CallID: "tu1", Text: "height 5"},
+ {Role: "tool", CallID: "tu2", Text: "peers 3"},
+ }
+ msgs := p.convMessages(hist)
+ if len(msgs) != 3 {
+ t.Fatalf("want user/assistant/user, got %d messages", len(msgs))
+ }
+ got, _ := json.Marshal(msgs[1]["content"])
+ if !jsonEqual(t, got, res.RawSteps) {
+ t.Fatalf("assistant not replayed verbatim:\n got %s\nwant %s", got, res.RawSteps)
+ }
+ if c, _ := msgs[2]["content"].([]anthBlock); len(c) != 2 {
+ t.Fatalf("tool results not merged: %v", msgs[2]["content"])
+ }
+ // raw from another provider is never replayed to Anthropic
+ hist[1].RawProvider = "gemini"
+ got, _ = json.Marshal(p.convMessages(hist)[1]["content"])
+ if strings.Contains(string(got), "SIG123") {
+ t.Fatal("foreign raw steps replayed")
+ }
+}
+
+func jsonEqual(t *testing.T, a, b []byte) bool {
+ t.Helper()
+ var x, y any
+ if json.Unmarshal(a, &x) != nil || json.Unmarshal(b, &y) != nil {
+ return false
+ }
+ ja, _ := json.Marshal(x)
+ jb, _ := json.Marshal(y)
+ return string(ja) == string(jb)
+}
+
+func TestAnthropicFallbackOnFirstParty(t *testing.T) {
+ p := &anthropic{model: "claude-opus-5-5", base: "https://api.anthropic.com"}
+ b := p.body(&Request{Model: "claude-opus-5-5"}, false)
+ if b["fallbacks"] != "default" || b["max_tokens"] != 16000 {
+ t.Fatalf("body = %v", b)
+ }
+ b = p.body(&Request{Model: "claude-haiku-4-5"}, true)
+ if _, ok := b["fallbacks"]; ok {
+ t.Fatal("fallbacks sent for a model without it")
+ }
+ if _, ok := b["thinking"]; ok {
+ t.Fatal("adaptive thinking sent to haiku")
+ }
+ if b["max_tokens"] != 64000 {
+ t.Fatalf("stream max_tokens = %v", b["max_tokens"])
+ }
+}
+
+func TestAnthropicRefusalAndMaxTokens(t *testing.T) {
+ r := anthResponse(nil, "refusal", &anthStopDetails{Category: "cyber"}, anthUsage{})
+ if r.Stop != StopRefusal || r.StopDetail != "cyber" {
+ t.Fatalf("refusal = %s %q", r.Stop, r.StopDetail)
+ }
+ if r := anthResponse(nil, "max_tokens", nil, anthUsage{}); r.Stop != StopMaxTokens {
+ t.Fatalf("stop = %s", r.Stop)
+ }
+}
+
+func TestAnthropicInvalidToolInputReportedNotReplayed(t *testing.T) {
+ var bodies []map[string]any
+ srv := sseServer(t, sseBody(
+ `{"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"tu1","name":"x","input":{}}}`,
+ `{"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"a\":"}}`,
+ `{"type":"message_delta","delta":{"stop_reason":"max_tokens"}}`,
+ ), &bodies, nil)
+ p := &anthropic{key: "k", base: srv.URL, hc: srv.Client()}
+ res, err := p.Stream(context.Background(), &Request{Model: "m"}, func(string) {})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if string(res.Calls[0].Args) != `{"a":` {
+ t.Fatalf("truncated input should reach the loop as-is, got %s", res.Calls[0].Args)
+ }
+ if !json.Valid(res.RawSteps) || !strings.Contains(string(res.RawSteps), `"input":{}`) {
+ t.Fatalf("raw blocks must stay valid JSON: %s", res.RawSteps)
+ }
+}
+
+func TestOpenAIGroqStreamUsageReasoningAndLength(t *testing.T) {
+ var bodies []map[string]any
+ srv := sseServer(t, sseBody(
+ `{"choices":[{"delta":{"reasoning":"hmm "}}]}`,
+ `{"choices":[{"delta":{"content":"partial"}}]}`,
+ `{"choices":[{"delta":{},"finish_reason":"length"}],"x_groq":{"usage":{"prompt_tokens":120,"completion_tokens":30,"prompt_tokens_details":{"cached_tokens":100}}}}`,
+ `[DONE]`,
+ ), &bodies, nil)
+ o := &openai{name: "groq", model: "openai/gpt-oss-120b", base: srv.URL, hc: srv.Client()}
+ var thought string
+ res, err := o.Stream(context.Background(), &Request{Model: o.model, MaxTok: 2048, Effort: "max",
+ Messages: []Msg{{Role: "user", Text: "x"}}, OnThinking: func(s string) { thought += s }}, func(string) {})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if res.Stop != StopMaxTokens || res.Text != "partial" || thought != "hmm " {
+ t.Fatalf("stop=%s text=%q thought=%q", res.Stop, res.Text, thought)
+ }
+ if res.Usage != (Usage{Input: 120, Output: 30, CacheRead: 100}) {
+ t.Fatalf("usage = %+v", res.Usage)
+ }
+ b := bodies[0]
+ if b["reasoning_effort"] != "high" || b["max_completion_tokens"] != float64(2048) {
+ t.Fatalf("body = %v", b)
+ }
+ if _, ok := b["stream_options"]; ok {
+ t.Fatal("stream_options sent to groq")
+ }
+}
+
+func TestOpenAICompatUsesMaxTokens(t *testing.T) {
+ o := &openai{model: "qwen3:32b", base: "http://localhost:11434", name: "openai-compat"}
+ b := o.body(&Request{Model: "qwen3:32b", MaxTok: 100})
+ if b["max_tokens"] != 100 || b["max_completion_tokens"] != nil || b["reasoning_effort"] != nil {
+ t.Fatalf("body = %v", b)
+ }
+}
+
+func TestOpenAIErrorStatusSurfaced(t *testing.T) {
+ noSleep(t)
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ w.WriteHeader(401)
+ io.WriteString(w, `{"error":{"message":"bad key"}}`)
+ }))
+ defer srv.Close()
+ o := &openai{name: "groq", base: srv.URL, hc: srv.Client()}
+ _, err := o.Chat(context.Background(), &Request{Model: "m"})
+ if err == nil || !strings.Contains(err.Error(), "bad key") || !strings.Contains(err.Error(), "401") {
+ t.Fatalf("err = %v", err)
+ }
+}
+
+func TestGeminiEffortUsageAndStatus(t *testing.T) {
+ var bodies []map[string]any
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ var b map[string]any
+ _ = json.NewDecoder(r.Body).Decode(&b)
+ bodies = append(bodies, b)
+ io.WriteString(w, `{"status":"incomplete","output_text":"cut","steps":[{"type":"thought","signature":"s","summary":[{"type":"text","text":"plan"}]},{"type":"model_output","content":[{"type":"text","text":"cut"}]}],
+ "usage":{"total_input_tokens":50,"total_cached_tokens":20,"total_output_tokens":5,"total_thought_tokens":7}}`)
+ }))
+ defer srv.Close()
+ g := &gemini{model: "gemini-x", base: srv.URL, hc: srv.Client()}
+ res, err := g.Chat(context.Background(), &Request{Model: "gemini-x", Effort: "xhigh", MaxTok: 900})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if res.Stop != StopMaxTokens || res.Thinking != "plan" || res.Usage != (Usage{Input: 50, Output: 12, CacheRead: 20}) {
+ t.Fatalf("res = %+v", res)
+ }
+ gc, _ := bodies[0]["generation_config"].(map[string]any)
+ if gc["thinking_level"] != "high" || gc["max_output_tokens"] != float64(900) {
+ t.Fatalf("generation_config = %v", gc)
+ }
+}
+
+func TestSystemFrozenSnapshotRefreshAppended(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{{Text: "a", Done: true}, {Text: "b", Done: true}}}
+ a := newTestAgent(t, prov)
+ n := 0
+ a.SnapshotFn = func(*toolkit.Context) string { n++; return fmt.Sprintf("LIVE: height=%d", n) }
+ a.SnapshotTTL = time.Nanosecond
+ a.Run(context.Background(), "one")
+ time.Sleep(time.Millisecond)
+ a.Run(context.Background(), "two")
+ if prov.reqs[0].System != prov.reqs[1].System {
+ t.Fatal("system prompt changed between turns — breaks the cached prefix")
+ }
+ if !strings.Contains(prov.reqs[0].System, "height=1") {
+ t.Fatalf("first snapshot missing from system: %q", prov.reqs[0].System)
+ }
+ last := prov.reqs[1].Messages[len(prov.reqs[1].Messages)-1].Text
+ if !strings.Contains(last, "") || !strings.Contains(last, "height=2") || !strings.HasSuffix(last, "two") {
+ t.Fatalf("refresh not appended to the user turn: %q", last)
+ }
+ // earlier messages are untouched (append-only history)
+ if prov.reqs[1].Messages[0].Text != prov.reqs[0].Messages[0].Text {
+ t.Fatal("earlier turn edited")
+ }
+}
+
+func TestMaxTokensAutoContinues(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{
+ {Text: "first half ", Stop: StopMaxTokens},
+ {Text: "second half", Stop: StopEnd, Done: true},
+ }}
+ a := newTestAgent(t, prov)
+ var notices []string
+ a.OnEvent = func(e Event) {
+ if e.Kind == EvNotice {
+ notices = append(notices, e.Text)
+ }
+ }
+ out, err := a.Run(context.Background(), "write it")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if out != "first half second half" || prov.calls != 2 || len(notices) != 1 {
+ t.Fatalf("out=%q calls=%d notices=%v", out, prov.calls, notices)
+ }
+}
+
+func TestRefusalSurfacesAsError(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{{Stop: StopRefusal, StopDetail: "cyber"}}}
+ a := newTestAgent(t, prov)
+ _, err := a.Run(context.Background(), "x")
+ if err == nil || !strings.Contains(err.Error(), "cyber") {
+ t.Fatalf("err = %v", err)
+ }
+}
+
+func TestUsageAccumulates(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{{ID: "c1", Name: "node__status", Args: json.RawMessage(`{}`)}}, Usage: Usage{Input: 100, Output: 10}},
+ {Text: "ok", Done: true, Usage: Usage{Input: 150, Output: 5, CacheRead: 100}},
+ }}
+ tool := stubTool{name: "node.status", tier: toolkit.TierObserve, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) {
+ return &toolkit.Result{Text: "h"}, nil
+ }}
+ a := newTestAgent(t, prov, tool)
+ a.Run(context.Background(), "x")
+ tot, last := a.Usage()
+ if tot != (Usage{Input: 250, Output: 15, CacheRead: 100}) || last != 155 {
+ t.Fatalf("total=%+v last=%d", tot, last)
+ }
+}
+
+// compactProvider answers summarizer requests with a summary and records
+// what each normal request carried.
+type compactProvider struct {
+ mockProvider
+ overflowOnce bool
+}
+
+func (c *compactProvider) Chat(ctx context.Context, r *Request) (*Response, error) {
+ last := r.Messages[len(r.Messages)-1].Text
+ if strings.Contains(last, "Summarize this conversation") {
+ return &Response{Text: "SUMMARY: disk was 95%", Done: true}, nil
+ }
+ if c.overflowOnce {
+ c.overflowOnce = false
+ return nil, errors.New("anthropic: prompt is too long: 1000001 tokens > 1000000 maximum (HTTP 400)")
+ }
+ return c.mockProvider.Chat(ctx, r)
+}
+
+func TestAutoCompactAtTurnBoundary(t *testing.T) {
+ prov := &compactProvider{mockProvider: mockProvider{responses: []*Response{
+ {Text: "one", Done: true, Usage: Usage{Input: 900, Output: 200}},
+ {Text: "two", Done: true},
+ }}}
+ a := newTestAgent(t, prov)
+ a.CompactAt = 1000
+ a.Run(context.Background(), "check disk")
+ a.Run(context.Background(), "and now?")
+ req := prov.lastReq
+ if len(req.Messages) != 1 {
+ t.Fatalf("history not replaced by summary: %d messages", len(req.Messages))
+ }
+ m := req.Messages[0].Text
+ if !strings.Contains(m, "SUMMARY: disk was 95%") || !strings.HasSuffix(m, "and now?") {
+ t.Fatalf("summary not attached to next turn: %q", m)
+ }
+}
+
+func TestContextOverflowCompactsAndRetries(t *testing.T) {
+ prov := &compactProvider{overflowOnce: true, mockProvider: mockProvider{responses: []*Response{{Text: "recovered", Done: true}}}}
+ a := newTestAgent(t, prov)
+ a.history = []Msg{{Role: "user", Text: "old"}, {Role: "assistant", Text: "old reply"}}
+ out, err := a.Run(context.Background(), "go")
+ if err != nil || out != "recovered" {
+ t.Fatalf("out=%q err=%v", out, err)
+ }
+ if !strings.Contains(prov.lastReq.Messages[0].Text, "Continue the task in progress") {
+ t.Fatalf("mid-turn summary missing: %q", prov.lastReq.Messages[0].Text)
+ }
+}
+
+func TestToolSetChangeDropsBoundThinking(t *testing.T) {
+ prov := &mockProvider{}
+ ro := stubTool{name: "node.status", tier: toolkit.TierObserve, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) { return &toolkit.Result{}, nil }}
+ rw := stubTool{name: "node.restart", tier: toolkit.TierLocalChange, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) { return &toolkit.Result{}, nil }}
+ a := newTestAgent(t, prov, ro, rw)
+ a.conf.Tools = "all"
+ a.Run(context.Background(), "one")
+ a.history[1].RawSteps, a.history[1].RawProvider = json.RawMessage(`[{"type":"thinking","signature":"s"}]`), "anthropic"
+ a.Policy.Mode = ModeReadOnly // hides node.restart → tool set changes
+ a.Run(context.Background(), "two")
+ if a.history[1].RawSteps != nil {
+ t.Fatal("thinking bound to the old tool set was replayed")
+ }
+}
+
+func TestSessionSaveRestoreRoundTrip(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{{Text: "hello", Done: true, Usage: Usage{Input: 10, Output: 2}}}}
+ a := newTestAgent(t, prov)
+ a.Persist = true
+ if _, err := a.Run(context.Background(), "first question"); err != nil {
+ t.Fatal(err)
+ }
+ list, err := ListSessions()
+ if err != nil || len(list) != 1 || list[0].Title != "first question" || list[0].History != nil {
+ t.Fatalf("list = %+v err=%v", list, err)
+ }
+ sf, err := LoadSession(a.ID()[:12])
+ if err != nil {
+ t.Fatal(err)
+ }
+ b := newTestAgent(t, &mockProvider{})
+ b.Restore(sf)
+ if b.ID() != a.ID() || len(b.History()) != 2 || b.system() != a.system() {
+ t.Fatalf("restore lost state: id=%s hist=%d", b.ID(), len(b.History()))
+ }
+ if tot, _ := b.Usage(); tot.Input != 10 {
+ t.Fatalf("usage not restored: %+v", tot)
+ }
+ if _, err := LoadSession("../etc/passwd"); err == nil {
+ t.Fatal("path traversal accepted")
+ }
+}
+
+func TestClipKeepsHeadTailAndUTF8(t *testing.T) {
+ s := strings.Repeat("é", 5000) + "TAIL-ERROR"
+ out := clip(s, 1000)
+ if !utf8.ValidString(out) || !strings.HasSuffix(out, "TAIL-ERROR") || len(out) > 1100 {
+ t.Fatalf("clip: valid=%v len=%d tail=%q", utf8.ValidString(out), len(out), out[len(out)-12:])
+ }
+ if clip("short", 1000) != "short" {
+ t.Fatal("short input altered")
+ }
+}
+
+func TestValidEffort(t *testing.T) {
+ for _, e := range []string{"", "low", "medium", "high", "xhigh", "max"} {
+ if ValidEffort(e) != nil {
+ t.Fatalf("%q rejected", e)
+ }
+ }
+ if ValidEffort("ultra") == nil {
+ t.Fatal("bad effort accepted")
+ }
+}
+
+func TestRetryIsAnnouncedAndTimeoutsAreNot(t *testing.T) {
+ noSleep(t)
+ var n atomic.Int32
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ if n.Add(1) == 1 {
+ w.WriteHeader(503)
+ return
+ }
+ io.WriteString(w, "ok")
+ }))
+ defer srv.Close()
+ var notes []string
+ ctx := WithRetryNotice(context.Background(), func(attempt int, wait time.Duration, reason string) {
+ notes = append(notes, fmt.Sprintf("%d %s", attempt, reason))
+ })
+ resp, err := doHTTP(ctx, srv.Client(), srv.URL, []byte(`{}`), nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ resp.Body.Close()
+ if len(notes) != 1 || notes[0] != "1 HTTP 503" {
+ t.Fatalf("notices = %v", notes)
+ }
+
+ slow := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ n.Add(1)
+ time.Sleep(200 * time.Millisecond)
+ }))
+ defer slow.Close()
+ n.Store(0)
+ hc := &http.Client{Timeout: 20 * time.Millisecond}
+ if _, err := doHTTP(context.Background(), hc, slow.URL, []byte(`{}`), nil); err == nil {
+ t.Fatal("want timeout error")
+ }
+ if n.Load() > 1 { // 0 when the timeout fires before the handler runs
+ t.Fatalf("timed-out request retried: %d attempts", n.Load())
+ }
+}
+
+func TestRetrySkipsDailyQuota(t *testing.T) {
+ noSleep(t)
+ var n atomic.Int32
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ n.Add(1)
+ w.WriteHeader(429)
+ _, _ = w.Write([]byte(`{"error":{"message":"Rate limit exceeded: free-models-per-day. Add 10 credits to unlock 1000 free model requests per day"}}`))
+ }))
+ defer srv.Close()
+ resp, err := doHTTP(context.Background(), srv.Client(), srv.URL, []byte(`{}`), nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer resp.Body.Close()
+ if n.Load() != 1 {
+ t.Fatalf("daily quota retried: %d attempts", n.Load())
+ }
+ if err := apiError("openrouter", resp); err == nil || !strings.Contains(err.Error(), "free-models-per-day") {
+ t.Fatalf("body lost: %v", err)
+ }
+}
diff --git a/internal/agent/phase2_test.go b/internal/agent/phase2_test.go
new file mode 100644
index 0000000..62ece14
--- /dev/null
+++ b/internal/agent/phase2_test.go
@@ -0,0 +1,253 @@
+package agent
+
+import (
+ "context"
+ "encoding/json"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "github.com/abhijitkrm/cometcli/internal/client/host"
+ "github.com/abhijitkrm/cometcli/internal/config"
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+ "github.com/abhijitkrm/cometcli/internal/tools/shell"
+)
+
+func bashCall(id, cmd string) Call {
+ b, _ := json.Marshal(map[string]any{"command": cmd})
+ return Call{ID: id, Name: "bash", Args: b}
+}
+
+func newShellAgent(t *testing.T, prov Provider, tools ...toolkit.Tool) (*Agent, string) {
+ a := newTestAgent(t, prov, append([]toolkit.Tool{shell.Bash{}}, tools...)...)
+ dir := t.TempDir()
+ a.Tools = toolkit.NewSession(dir)
+ a.WorkRoot = dir
+ a.Ctx.SetHost(&host.Local{})
+ return a, dir
+}
+
+func lastToolText(a *Agent) string {
+ h := a.History()
+ for i := len(h) - 1; i >= 0; i-- {
+ if h[i].Role == "tool" {
+ return h[i].Text
+ }
+ }
+ return ""
+}
+
+func TestReadOnlyModeShellReadsButNeverWrites(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{bashCall("c1", "echo hello")}},
+ {Calls: []Call{bashCall("c2", "touch made.txt")}},
+ {Text: "done", Done: true},
+ }}
+ a, dir := newShellAgent(t, prov)
+ a.Policy.Mode = ModeReadOnly
+ if _, err := a.Run(context.Background(), "go"); err != nil {
+ t.Fatal(err)
+ }
+ var names []string
+ for _, td := range prov.reqs[0].Tools {
+ names = append(names, td.Name)
+ }
+ if !strings.Contains(strings.Join(names, ","), "bash") {
+ t.Fatalf("bash hidden in readonly: %v", names)
+ }
+ h := a.History()
+ if !strings.Contains(h[2].Text, "hello") {
+ t.Fatalf("read command blocked: %q", h[2].Text)
+ }
+ if !strings.Contains(h[4].Text, "read-only") {
+ t.Fatalf("write not blocked: %q", h[4].Text)
+ }
+ if _, err := os.Stat(filepath.Join(dir, "made.txt")); err == nil {
+ t.Fatal("file created in readonly mode")
+ }
+}
+
+func TestDenyRuleBlocksRegistryTool(t *testing.T) {
+ ran := false
+ unjail := stubTool{name: "val.unjail", tier: toolkit.TierOnChain, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) {
+ ran = true
+ return &toolkit.Result{}, nil
+ }}
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{{ID: "c1", Name: "val__unjail", Args: json.RawMessage(`{}`)}}},
+ {Text: "ok", Done: true},
+ }}
+ a, _ := newShellAgent(t, prov, unjail)
+ a.Rules, _ = toolkit.NewRules(nil, nil, []string{"val.unjail"})
+ a.Run(context.Background(), "unjail")
+ if ran || !strings.Contains(lastToolText(a), "denied by permission rule val.unjail") {
+ t.Fatalf("ran=%v result=%q", ran, lastToolText(a))
+ }
+}
+
+func TestAllowRuleSkipsLocalChangePrompt(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{bashCall("c1", "touch ok.txt")}},
+ {Text: "ok", Done: true},
+ }}
+ a, dir := newShellAgent(t, prov)
+ asked := 0
+ a.Ctx.Approver = func(*toolkit.Context, string, toolkit.Tier, map[string]any) (bool, error) { asked++; return false, nil }
+ a.Ctx.AutoApproveBelow = toolkit.TierLocalChange
+ a.Rules, _ = toolkit.NewRules([]string{"bash(touch:*)"}, nil, nil)
+ a.Run(context.Background(), "x")
+ if asked != 0 {
+ t.Fatal("allow rule still asked")
+ }
+ if _, err := os.Stat(filepath.Join(dir, "ok.txt")); err != nil {
+ t.Fatalf("allowed command didn't run: %s", lastToolText(a))
+ }
+}
+
+func TestTodoWriteEmitsChecklist(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{{ID: "c1", Name: "todo_write", Args: json.RawMessage(`{"todos":[{"content":"stop node","status":"completed"},{"content":"swap binary","status":"in_progress"},{"content":"start node","status":"pending"}]}`)}}},
+ {Text: "ok", Done: true},
+ }}
+ a, _ := newShellAgent(t, prov)
+ var got []Todo
+ a.OnEvent = func(e Event) {
+ if e.Kind == EvTodos {
+ got = e.Todos
+ }
+ }
+ a.Run(context.Background(), "upgrade")
+ if len(got) != 3 || !strings.Contains(lastToolText(a), "1/3 done") {
+ t.Fatalf("todos=%v result=%q", got, lastToolText(a))
+ }
+ if r := RenderTodos(got); !strings.Contains(r, "☒ stop node") || !strings.Contains(r, "▸ swap binary") {
+ t.Fatalf("render = %q", r)
+ }
+}
+
+func TestBashOutputUsesToolLimit(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{bashCall("c1", "head -c 20000 /dev/zero | tr '\\0' 'a'")}},
+ {Text: "ok", Done: true},
+ }}
+ a, _ := newShellAgent(t, prov)
+ a.Run(context.Background(), "x")
+ if n := len(lastToolText(a)); n < 19000 || n > 30500 {
+ t.Fatalf("bash output length %d — should keep up to 30k", n)
+ }
+}
+
+func TestDeferredToolsLoadOnDemand(t *testing.T) {
+ gov := stubTool{name: "chain.gov", tier: toolkit.TierObserve, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) {
+ return &toolkit.Result{Text: "proposal 3 voting"}, nil
+ }}
+ core := stubTool{name: "node.status", tier: toolkit.TierObserve, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) {
+ return &toolkit.Result{}, nil
+ }}
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{{ID: "c1", Name: "tool_search", Args: json.RawMessage(`{"query":"select:chain.gov"}`)}}},
+ {Calls: []Call{{ID: "c2", Name: "chain__gov", Args: json.RawMessage(`{}`)}}},
+ {Text: "ok", Done: true},
+ }}
+ a, _ := newShellAgent(t, prov, gov, core)
+ a.Run(context.Background(), "any proposals?")
+ names := func(r *Request) string {
+ var n []string
+ for _, td := range r.Tools {
+ n = append(n, td.Name)
+ }
+ return strings.Join(n, ",")
+ }
+ if first := names(prov.reqs[0]); strings.Contains(first, "chain__gov") || !strings.Contains(first, "node__status") || !strings.Contains(first, "tool_search") {
+ t.Fatalf("first request tools = %s", first)
+ }
+ if !strings.Contains(prov.reqs[0].System, "- chain: gov") {
+ t.Fatalf("catalog missing from system prompt: %q", prov.reqs[0].System)
+ }
+ if second := names(prov.reqs[1]); !strings.Contains(second, "chain__gov") {
+ t.Fatalf("loaded tool not advertised: %s", second)
+ }
+ if !strings.Contains(lastToolText(a), "proposal 3") {
+ t.Fatalf("loaded tool didn't run: %q", lastToolText(a))
+ }
+ // keyword search ranks by name/description
+ if got := a.searchTools("governance gov proposals", 3); len(got) == 0 || got[0].Name() != "chain.gov" {
+ t.Fatalf("search = %v", got)
+ }
+ if lt := a.LoadedTools(); len(lt) != 1 || lt[0] != "chain.gov" {
+ t.Fatalf("loaded = %v", lt)
+ }
+}
+
+func TestGeneralModeAndScopeSwitch(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{{Text: "a", Done: true}, {Text: "b", Done: true}, {Text: "c", Done: true}}}
+ nodeTool := stubTool{name: "node.status", tier: toolkit.TierObserve, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) { return &toolkit.Result{}, nil }}
+ a, _ := newShellAgent(t, prov, nodeTool)
+ prof := a.Ctx.Profile
+ prof.Role, prof.ChainID = "validator", "primium-1"
+ a.Ctx.Cfg = &config.Config{Profiles: map[string]*config.Profile{"testp": prof}}
+ a.SwitchScope(nil) // start in general mode
+ a.carry = ""
+ a.Run(context.Background(), "one")
+ r := prov.reqs[0]
+ if !strings.Contains(r.System, "working in the operator's terminal") || !strings.Contains(r.System, "testp (validator, primium-1)") {
+ t.Fatalf("general prompt = %q", r.System)
+ }
+ for _, td := range r.Tools {
+ if td.Name == "node__status" || td.Name == toolSearchName {
+ t.Fatalf("node tool %s advertised in general mode", td.Name)
+ }
+ }
+ if strings.Contains(r.System, "LIVE SNAPSHOT") {
+ t.Fatal("snapshot in general mode")
+ }
+
+ a.SwitchScope(prof)
+ a.Run(context.Background(), "two")
+ r = prov.reqs[1]
+ if !strings.Contains(r.System, "Cosmos-EVM validators") || !strings.Contains(r.System, "LIVE: height=42") {
+ t.Fatalf("node prompt = %q", r.System)
+ }
+ var names []string
+ for _, td := range r.Tools {
+ names = append(names, td.Name)
+ }
+ if !strings.Contains(strings.Join(names, ","), "node__status") {
+ t.Fatalf("node tools missing: %v", names)
+ }
+ last := r.Messages[len(r.Messages)-1].Text
+ if !strings.Contains(last, "switched this session to node mode for profile testp") || !strings.HasSuffix(last, "two") {
+ t.Fatalf("switch note missing: %q", last)
+ }
+ if len(r.Messages) != 3 {
+ t.Fatalf("conversation not kept: %d messages", len(r.Messages))
+ }
+}
+
+type opOnly struct{ stubTool }
+
+func (opOnly) OperatorOnly() bool { return true }
+
+func TestOperatorOnlyToolsNeverReachTheAgent(t *testing.T) {
+ ran := false
+ keyAdd := opOnly{stubTool{name: "keys.add", tier: toolkit.TierLocalChange, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) {
+ ran = true
+ return &toolkit.Result{Text: "mnemonic: …"}, nil
+ }}}
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{{ID: "c1", Name: "keys__add", Args: json.RawMessage(`{"name":"x"}`)}}},
+ {Text: "ok", Done: true},
+ }}
+ a, _ := newShellAgent(t, prov, keyAdd)
+ a.conf.Tools = "all"
+ a.Run(context.Background(), "make me a key")
+ for _, td := range prov.reqs[0].Tools {
+ if td.Name == "keys__add" {
+ t.Fatal("operator-only tool advertised")
+ }
+ }
+ if ran || !strings.Contains(lastToolText(a), "operator-only") {
+ t.Fatalf("ran=%v result=%q", ran, lastToolText(a))
+ }
+}
diff --git a/internal/agent/phase4_test.go b/internal/agent/phase4_test.go
new file mode 100644
index 0000000..a09d17f
--- /dev/null
+++ b/internal/agent/phase4_test.go
@@ -0,0 +1,239 @@
+package agent
+
+import (
+ "context"
+ "encoding/json"
+ "os"
+ "os/exec"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "github.com/abhijitkrm/cometcli/internal/hooks"
+ "github.com/abhijitkrm/cometcli/internal/settings"
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+)
+
+func hookScript(t *testing.T, dir, name, body string) string {
+ p := filepath.Join(dir, name)
+ os.WriteFile(p, []byte("#!/bin/sh\n"+body+"\n"), 0o755)
+ return p
+}
+
+func withExt(a *Agent, dir string, hm map[string][]settings.HookMatcher) {
+ a.ext = &ext{Settings: &settings.Settings{Root: dir}, Hooks: &hooks.Runner{Matchers: hm, Dir: dir},
+ Commands: map[string]*settings.Command{}, Agents: map[string]*settings.AgentDef{}}
+}
+
+func TestPreAndPostToolUseHooks(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{bashCall("c1", "echo should-not-run > x.txt")}},
+ {Calls: []Call{bashCall("c2", "echo fine")}},
+ {Text: "done", Done: true},
+ }}
+ a, dir := newShellAgent(t, prov)
+ pre := hookScript(t, dir, "pre.sh", `grep -q 'x.txt' && { echo "writing x.txt is not allowed" >&2; exit 2; }; exit 0`)
+ post := hookScript(t, dir, "post.sh", `echo '{"decision":"block","reason":"remember to check peers"}'`)
+ withExt(a, dir, map[string][]settings.HookMatcher{
+ hooks.PreToolUse: {{Matcher: "bash", Hooks: []settings.HookCommand{{Command: pre}}}},
+ hooks.PostToolUse: {{Matcher: "Bash", Hooks: []settings.HookCommand{{Command: post}}}},
+ })
+ a.Run(context.Background(), "go")
+ h := a.History()
+ if !strings.Contains(h[2].Text, "blocked by PreToolUse hook: writing x.txt is not allowed") {
+ t.Fatalf("pre hook: %q", h[2].Text)
+ }
+ if _, err := os.Stat(filepath.Join(dir, "x.txt")); err == nil {
+ t.Fatal("blocked command ran")
+ }
+ if !strings.Contains(h[4].Text, "fine") || !strings.Contains(h[4].Text, "[PostToolUse hook] remember to check peers") {
+ t.Fatalf("post hook: %q", h[4].Text)
+ }
+}
+
+func TestPreToolUseAllowSkipsPromptButNotForTx(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{bashCall("c1", "touch ok.txt")}},
+ {Calls: []Call{bashCall("c2", "evmd tx gov vote 1 yes --from v")}},
+ {Text: "done", Done: true},
+ }}
+ a, dir := newShellAgent(t, prov)
+ asked := []string{}
+ a.Ctx.Approver = func(_ *toolkit.Context, p string, _ toolkit.Tier, _ map[string]any) (bool, error) {
+ asked = append(asked, p)
+ return false, nil
+ }
+ a.Ctx.AutoApproveBelow = toolkit.TierLocalChange
+ allow := hookScript(t, dir, "allow.sh", `echo '{"hookSpecificOutput":{"permissionDecision":"allow"}}'`)
+ withExt(a, dir, map[string][]settings.HookMatcher{hooks.PreToolUse: {{Hooks: []settings.HookCommand{{Command: allow}}}}})
+ a.Run(context.Background(), "go")
+ if _, err := os.Stat(filepath.Join(dir, "ok.txt")); err != nil {
+ t.Fatal("hook allow didn't skip the prompt")
+ }
+ if len(asked) != 1 || !strings.Contains(asked[0], "evmd tx") {
+ t.Fatalf("a tx must still ask despite a hook allow: %v", asked)
+ }
+}
+
+func TestUserPromptSubmitAndStopHooks(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{{Text: "first", Done: true}, {Text: "second", Done: true}}}
+ a, dir := newShellAgent(t, prov)
+ ups := hookScript(t, dir, "ups.sh", `in=$(cat); case "$in" in *secret-project*) echo "not here" >&2; exit 2;; esac; echo "on-call: alice"`)
+ stop := hookScript(t, dir, "stop.sh", `in=$(cat); case "$in" in *'"stop_hook_active":true'*) exit 0;; esac; echo '{"decision":"block","reason":"verify with node status"}'`)
+ withExt(a, dir, map[string][]settings.HookMatcher{
+ hooks.UserPromptSubmit: {{Hooks: []settings.HookCommand{{Command: ups}}}},
+ hooks.Stop: {{Hooks: []settings.HookCommand{{Command: stop}}}},
+ })
+ if _, err := a.Run(context.Background(), "tell me about secret-project"); err == nil || !strings.Contains(err.Error(), "not here") {
+ t.Fatalf("prompt not blocked: %v", err)
+ }
+ if prov.calls != 0 {
+ t.Fatal("blocked prompt reached the model")
+ }
+ out, err := a.Run(context.Background(), "status?")
+ if err != nil || out != "second" || prov.calls != 2 {
+ t.Fatalf("stop hook should force one more round: out=%q calls=%d err=%v", out, prov.calls, err)
+ }
+ if !strings.Contains(prov.reqs[0].Messages[0].Text, "on-call: alice") {
+ t.Fatalf("hook context missing: %q", prov.reqs[0].Messages[0].Text)
+ }
+ last := prov.reqs[1].Messages[len(prov.reqs[1].Messages)-1].Text
+ if !strings.Contains(last, "[Stop hook] verify with node status") {
+ t.Fatalf("stop reason not fed back: %q", last)
+ }
+}
+
+func TestCustomCommandExpandsAndGrantsTurnRules(t *testing.T) {
+ prov := &mockProvider{}
+ a, dir := newShellAgent(t, prov)
+ withExt(a, dir, nil)
+ a.ext.Settings.Trusted = true
+ os.WriteFile(filepath.Join(dir, "network-config.env"), []byte("CHAIN_ID=primium-1\n"), 0o644)
+ a.ext.Commands["vote"] = &settings.Command{Name: "vote", Scope: "project", AllowedTools: []string{"bash(./upgrade/vote-upgrade.sh:*)"},
+ Body: "Vote $1 on proposal $2.\nHost: !`uname -s`\nRefused: !`touch x`\nConfig: @network-config.env"}
+ res, err := RunCommand(a, a.Ctx, a.Reg, "/vote yes 7")
+ if err != nil {
+ t.Fatal(err)
+ }
+ for _, want := range []string{"Vote yes on proposal 7.", "Host: " + uname(t), "not run: only read-only", "CHAIN_ID=primium-1"} {
+ if !strings.Contains(res.Prompt, want) {
+ t.Errorf("prompt missing %q:\n%s", want, res.Prompt)
+ }
+ }
+ if !strings.Contains(res.Text, "allowing for this turn") {
+ t.Fatalf("text = %q", res.Text)
+ }
+ done := a.grantTurnRules()
+ if d, _ := a.Rules.Decide(toolkit.Request{Tool: "bash", Kind: "command", Specs: []string{"./upgrade/vote-upgrade.sh 7 yes"}}); d != toolkit.DecideAllow {
+ t.Fatal("turn rule not granted")
+ }
+ done()
+ if d, _ := a.Rules.Decide(toolkit.Request{Tool: "bash", Kind: "command", Specs: []string{"./upgrade/vote-upgrade.sh 7 yes"}}); d != toolkit.DecideDefault {
+ t.Fatal("turn rule outlived the turn")
+ }
+ // untrusted project: allowed-tools ignored
+ a.ext.Settings.Trusted = false
+ res, _ = RunCommand(a, a.Ctx, a.Reg, "/vote yes 7")
+ if !strings.Contains(res.Text, "ignored") || len(a.ext.turnRule) != 0 {
+ t.Fatalf("untrusted grant: %q %v", res.Text, a.ext.turnRule)
+ }
+}
+
+func TestMemoryInSystemPrompt(t *testing.T) {
+ prov := &mockProvider{}
+ a, dir := newShellAgent(t, prov)
+ withExt(a, dir, nil)
+ os.WriteFile(filepath.Join(dir, "COMET.md"), []byte("Validators run in docker; never restart val0 without asking."), 0o644)
+ a.WorkRoot = dir
+ a.Run(context.Background(), "hi")
+ if !strings.Contains(prov.reqs[0].System, "never restart val0 without asking") {
+ t.Fatalf("memory missing from system prompt")
+ }
+ res, err := RunCommand(a, a.Ctx, a.Reg, "/remember peers live in config.toml")
+ if err != nil || !strings.Contains(res.Text, filepath.Join(dir, "COMET.md")) {
+ t.Fatalf("/remember: %v %v", res, err)
+ }
+ a.Run(context.Background(), "next")
+ last := prov.reqs[1].Messages[len(prov.reqs[1].Messages)-1].Text
+ if !strings.Contains(last, "Operator added to memory") || prov.reqs[1].System != prov.reqs[0].System {
+ t.Fatalf("memory update should ride on the turn, system frozen: %q", last)
+ }
+}
+
+func TestSubagentTask(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{{ID: "t1", Name: "task", Args: json.RawMessage(`{"description":"scan logs","prompt":"find panics in the logs"}`)}}},
+ {Calls: []Call{bashCall("s1", "echo 'panic: oom'")}}, // subagent round 1
+ {Text: "REPORT: one OOM panic", Done: true}, // subagent final
+ {Text: "The node OOM-panicked once.", Done: true}, // parent final
+ }}
+ a, _ := newShellAgent(t, prov)
+ var notices []string
+ a.OnEvent = func(e Event) {
+ if e.Kind == EvToolStart {
+ notices = append(notices, e.Tool)
+ }
+ }
+ out, err := a.Run(context.Background(), "check logs")
+ if err != nil || out != "The node OOM-panicked once." {
+ t.Fatalf("out=%q err=%v", out, err)
+ }
+ if got := a.History()[2].Text; got != "REPORT: one OOM panic" {
+ t.Fatalf("task result = %q", got)
+ }
+ child := prov.reqs[1]
+ if !strings.Contains(child.System, "YOU ARE A SUBAGENT") || len(child.Messages) != 1 {
+ t.Fatalf("subagent request: %d msgs", len(child.Messages))
+ }
+ for _, td := range child.Tools {
+ if td.Name == taskToolName {
+ t.Fatal("subagent can spawn subagents")
+ }
+ }
+ if strings.Join(notices, ",") != "task,↳ bash" {
+ t.Fatalf("events = %v", notices)
+ }
+ if len(a.History()) != 4 {
+ t.Fatalf("subagent turns leaked into the parent history: %d", len(a.History()))
+ }
+}
+
+func TestMCPToolsDeferredInGeneralMode(t *testing.T) {
+ mcpTool := stubTool{name: "mcp__grafana__query", tier: toolkit.TierDiagnose, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) {
+ return &toolkit.Result{Text: "p99 120ms"}, nil
+ }}
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{{ID: "c1", Name: "tool_search", Args: json.RawMessage(`{"query":"select:mcp__grafana__query"}`)}}},
+ {Calls: []Call{{ID: "c2", Name: "mcp__grafana__query", Args: json.RawMessage(`{}`)}}},
+ {Text: "ok", Done: true},
+ }}
+ a, _ := newShellAgent(t, prov, mcpTool)
+ a.Ctx.Profile = nil // general mode
+ a.Run(context.Background(), "latency?")
+ first := strings.Join(toolNames(prov.reqs[0]), ",")
+ if strings.Contains(first, "mcp__grafana__query") || !strings.Contains(first, "tool_search") {
+ t.Fatalf("first tools = %s", first)
+ }
+ if !strings.Contains(prov.reqs[0].System, "mcp__grafana: query") {
+ t.Fatalf("catalog missing MCP tools")
+ }
+ if !strings.Contains(strings.Join(toolNames(prov.reqs[1]), ","), "mcp__grafana__query") || !strings.Contains(lastToolText(a), "p99 120ms") {
+ t.Fatalf("MCP tool not loaded/called: %q", lastToolText(a))
+ }
+}
+
+func uname(t *testing.T) string {
+ out, err := exec.Command("uname", "-s").Output()
+ if err != nil {
+ t.Skip("no uname")
+ }
+ return strings.TrimSpace(string(out))
+}
+
+func toolNames(r *Request) []string {
+ var n []string
+ for _, td := range r.Tools {
+ n = append(n, td.Name)
+ }
+ return n
+}
diff --git a/internal/agent/policy.go b/internal/agent/policy.go
index c146662..264af2c 100644
--- a/internal/agent/policy.go
+++ b/internal/agent/policy.go
@@ -16,19 +16,34 @@ const (
// autopilots if enabled), on-chain always confirms.
ModeOps Mode = "ops"
// ModeReadOnly: only observe/diagnose tools exist for the model;
- // anything mutating is neither advertised nor runnable.
+ // anything mutating is neither advertised nor runnable (the shell
+ // stays, limited to read-only commands).
ModeReadOnly Mode = "readonly"
+ // ModeAcceptEdits: like ops, but file writes and edits inside the
+ // working root run without a prompt.
+ ModeAcceptEdits Mode = "accept-edits"
+ // ModeBypass: every local-change runs without a prompt. Transactions
+ // still always ask; key material and state resets stay refused.
+ ModeBypass Mode = "bypass"
)
-// ParseMode accepts ops | readonly (plus "safe"/"ro" aliases).
+// Modes lists the approval postures in shift+tab cycling order.
+var Modes = []Mode{ModeOps, ModeAcceptEdits, ModeReadOnly, ModeBypass}
+
+// ParseMode accepts ops | readonly | accept-edits | bypass, plus Claude
+// Code's names (default, plan, acceptEdits, bypassPermissions).
func ParseMode(s string) (Mode, error) {
switch strings.ToLower(strings.TrimSpace(s)) {
- case "", "ops", "rw":
+ case "", "ops", "rw", "default":
return ModeOps, nil
- case "readonly", "read-only", "ro", "safe":
+ case "readonly", "read-only", "ro", "safe", "plan":
return ModeReadOnly, nil
+ case "accept-edits", "acceptedits", "edits":
+ return ModeAcceptEdits, nil
+ case "bypass", "bypasspermissions", "yolo":
+ return ModeBypass, nil
}
- return "", fmt.Errorf("unknown mode %q — want ops or readonly", s)
+ return "", fmt.Errorf("unknown mode %q — want ops, readonly, accept-edits or bypass", s)
}
// Policy maps tool tiers to auto | confirm | deny for one session.
@@ -66,12 +81,15 @@ func (p Policy) Allows(t toolkit.Tier) bool {
// AutoApproveBelow is the toolkit threshold this policy implies. on-chain
// is never below it, so a tx always reaches the human.
func (p Policy) AutoApproveBelow() toolkit.Tier {
- if p.AutoLocal && !p.ReadOnly() {
+ if (p.AutoLocal || p.Mode == ModeBypass) && !p.ReadOnly() {
return toolkit.TierOnChain
}
return toolkit.TierLocalChange
}
+// AcceptEdits reports whether in-root file edits skip the prompt.
+func (p Policy) AcceptEdits() bool { return p.Mode == ModeAcceptEdits }
+
// Decision describes how a tier is handled: auto | confirm | deny.
func (p Policy) Decision(t toolkit.Tier) string {
switch {
@@ -79,6 +97,8 @@ func (p Policy) Decision(t toolkit.Tier) string {
return "deny"
case t < p.AutoApproveBelow():
return "auto"
+ case t == toolkit.TierLocalChange && p.AcceptEdits():
+ return "confirm (edits in root: auto)"
default:
return "confirm"
}
diff --git a/internal/agent/prompt.go b/internal/agent/prompt.go
index dc26e40..d5959cd 100644
--- a/internal/agent/prompt.go
+++ b/internal/agent/prompt.go
@@ -14,20 +14,40 @@ import (
func SystemPrompt(c *toolkit.Context) string {
p := c.Profile
var b strings.Builder
- b.WriteString(`You are cometcli — an expert SRE agent for Cosmos-EVM validators
-(CometBFT consensus + Ethereum-compatible execution via the cosmos/evm stack).
+ b.WriteString(`You are cometcli — an expert SRE agent for Cosmos SDK and Cosmos-EVM validators
+(CometBFT consensus; on Cosmos-EVM chains also Ethereum-compatible execution via cosmos/evm).
You operate ONE node via tools. Rules you must follow:
+- Questions are about THIS node: inspect it with your tools and answer
+ from what you find — never with a generic explanation unless the
+ operator asks how something works in general.
- Prefer read-only tools first; diagnose before proposing changes.
- NEVER ask for or echo mnemonics, private keys, or priv_validator contents.
- For on-chain actions: explain the tx you're about to build, then call the
tool — the human still has to approve at the tx gate.
-- For jail recovery: check val.signing + chain params BEFORE suggesting
- unjail; a tombstoned validator must never be restarted to sign.
+- A tombstoned validator must never be unjailed or restarted to sign.
- Reference exact numbers (heights, missed counts, drift) in answers.
- Keep answers terse, technical, and actionable. Use markdown sparingly.
+Incidents and "is something wrong?" questions — work the method:
+1. node.triage first: one sweep of every signal, ranked known cases.
+2. kb.show the top case; run its confirm steps. Trust evidence over the
+ match — if confirmation fails, try the next case or kb.search.
+3. Fix the ROOT CAUSE before the symptom (disk before restart, cause
+ before unjail) by calling the tools: [change]/[tx] steps are approved
+ at the tool's own prompt, so say what and why in one line and make
+ the call — don't stop to ask in text. Never do what a case lists
+ under NEVER.
+4. Wait for progress with wait.until (synced, signing, in-consensus,
+ height, or signal="") instead of guessing; if progress
+ stalls, re-triage.
+5. Verify with the case's verify steps, then re-run node.triage.
+6. Report cause + evidence, actions (tx hashes), final state. If no case
+ fit and you found the cause, offer to record it with kb.add.
+
`)
+ b.WriteString(strings.Replace(generalToolsText, "General tools:", "General tools (they run on the node's host — local or over SSH):", 1))
+ b.WriteString("\n")
fmt.Fprintf(&b, "NODE PROFILE: %s (role=%s, chain-id=%s, evm-chain-id=%d, binary=%s, home=%s)\n",
p.Name, p.Role, p.ChainID, p.EVMChainID, p.Binary, p.Home)
fmt.Fprintf(&b, "ENDPOINTS: comet=%s grpc=%s evm=%s transport=%s service=%s:%s\n\n",
diff --git a/internal/agent/provider.go b/internal/agent/provider.go
index f060176..6cb9e39 100644
--- a/internal/agent/provider.go
+++ b/internal/agent/provider.go
@@ -26,9 +26,11 @@ type Msg struct {
ToolName string // for role=tool
IsError bool // tool result flagged error
JSONArgs json.RawMessage // for role=assistant with calls
- // RawSteps carries a provider's verbatim turn steps (Gemini Interactions
- // echoes them back in stateless mode, preserving thought signatures).
- RawSteps json.RawMessage
+ // RawSteps carries a provider's verbatim turn output (Gemini steps with
+ // thought signatures, Anthropic content blocks with thinking). It is
+ // replayed unchanged, but only to the provider named in RawProvider.
+ RawSteps json.RawMessage
+ RawProvider string `json:",omitempty"`
}
// Call is a tool invocation requested by the model.
@@ -51,7 +53,38 @@ type Request struct {
System string
Messages []Msg
Tools []ToolDef
- MaxTok int
+ // MaxTok caps output tokens; 0 lets the provider pick its default.
+ MaxTok int
+ // Effort is the reasoning depth: low | medium | high | xhigh | max, or
+ // "" for the model default. Each provider maps it to its own knob.
+ Effort string
+ // OnThinking, when set, receives streamed reasoning text (providers
+ // that expose none never call it).
+ OnThinking func(string) `json:"-"`
+}
+
+// StopReason is why a model round ended, normalized across providers.
+type StopReason string
+
+const (
+ StopEnd StopReason = "end" // finished normally
+ StopToolUse StopReason = "tool_use" // wants tool results
+ StopMaxTokens StopReason = "max_tokens" // output cut off at MaxTok
+ StopRefusal StopReason = "refusal" // declined by a safety filter
+)
+
+// Usage is the token accounting of one model round.
+type Usage struct {
+ Input int `json:"input"` // prompt tokens, including cached
+ Output int `json:"output"` // generated tokens, including thinking
+ CacheRead int `json:"cache_read"` // prompt tokens served from cache
+}
+
+// Add accumulates u2 into u.
+func (u *Usage) Add(u2 Usage) {
+ u.Input += u2.Input
+ u.Output += u2.Output
+ u.CacheRead += u2.CacheRead
}
// Response is the model's reply.
@@ -59,7 +92,16 @@ type Response struct {
Text string
Calls []Call
Done bool // true when the model finished (no tool calls pending)
- // RawSteps is the provider's raw step list for this turn (Gemini only).
+ // Stop is the normalized stop reason ("" when the provider didn't say).
+ Stop StopReason
+ // StopDetail explains a refusal when the provider gives a category.
+ StopDetail string
+ // Usage is this round's token accounting (zero when not reported).
+ Usage Usage
+ // Thinking is visible reasoning text, when the provider returns any.
+ Thinking string
+ // RawSteps is the provider's raw output for this turn, replayed
+ // verbatim in later requests (see Msg.RawSteps).
RawSteps json.RawMessage
}
@@ -96,18 +138,24 @@ func NewProvider(ac config.AgentConf) (Provider, error) {
}
key := func(env string) string {
if ac.APIKeyEnv != "" {
- return os.Getenv(ac.APIKeyEnv)
+ if v := os.Getenv(ac.APIKeyEnv); v != "" {
+ return v
+ }
+ return config.Credential(ac.APIKeyEnv)
}
if v := os.Getenv("COMETCLI_LLM_API_KEY"); v != "" {
return v
}
- return os.Getenv(env)
+ if v := os.Getenv(env); v != "" {
+ return v
+ }
+ return config.Credential(env) // saved by `cometcli config set-key`
}
switch strings.ToLower(ac.Provider) {
case "anthropic", "claude":
return &anthropic{
key: key("ANTHROPIC_API_KEY"),
- model: def(ac.Model, "claude-sonnet-4-5"),
+ model: def(ac.Model, "claude-opus-5-5"),
base: def(ac.BaseURL, "https://api.anthropic.com"),
}, nil
case "openai":
@@ -124,6 +172,15 @@ func NewProvider(ac config.AgentConf) (Provider, error) {
model: def(ac.Model, "llama-3.3-70b-versatile"),
base: def(ac.BaseURL, "https://api.groq.com/openai"),
}, nil
+ case "openrouter":
+ // OpenRouter — https://openrouter.ai; openrouter/free routes each
+ // request to a free model that supports what it needs (tools…)
+ return &openai{
+ name: "openrouter",
+ key: key("OPENROUTER_API_KEY"),
+ model: def(ac.Model, "openrouter/free"),
+ base: def(ac.BaseURL, "https://openrouter.ai/api"),
+ }, nil
case "gemini", "google":
// Gemini Interactions API — https://aistudio.google.com/apikey
return &gemini{
@@ -137,12 +194,13 @@ func NewProvider(ac config.AgentConf) (Provider, error) {
base = "http://localhost:11434" // ollama default
}
return &openai{
+ name: "openai-compat",
key: key("OLLAMA_API_KEY"),
model: def(ac.Model, "qwen3:32b"),
base: base,
}, nil
case "off", "none", "":
- return nil, fmt.Errorf("agent disabled — set agent.provider in the profile")
+ return nil, fmt.Errorf("no agent provider — run `cometcli config set agent.provider openrouter` (or groq, gemini, anthropic, openai, openai-compat), or set one in a profile")
default:
return nil, fmt.Errorf("unknown agent.provider %q", ac.Provider)
}
diff --git a/internal/agent/repl.go b/internal/agent/repl.go
index 7cfb5ac..6c88486 100644
--- a/internal/agent/repl.go
+++ b/internal/agent/repl.go
@@ -44,11 +44,36 @@ type Printer struct {
Out io.Writer
MD *glamour.TermRenderer
streamed bool
+ thinking bool
}
// Handle renders one event.
func (p *Printer) Handle(e Event) {
+ if p.thinking && e.Kind != EvThinking {
+ fmt.Fprintln(p.Out)
+ p.thinking = false
+ }
switch e.Kind {
+ case EvThinking:
+ if !p.thinking {
+ fmt.Fprint(p.Out, toolSt.Render("∴ "))
+ p.thinking = true
+ }
+ fmt.Fprint(p.Out, toolSt.Render(e.Text))
+ case EvTodos:
+ if p.streamed {
+ fmt.Fprintln(p.Out)
+ p.streamed = false
+ }
+ fmt.Fprintln(p.Out, toolSt.Render(RenderTodos(e.Todos)))
+ case EvProgress:
+ fmt.Fprintf(p.Out, "%s %s\n", toolSt.Render("…"), toolSt.Render(e.Text))
+ case EvNotice:
+ if p.streamed {
+ fmt.Fprintln(p.Out)
+ p.streamed = false
+ }
+ fmt.Fprintf(p.Out, "%s %s\n", toolSt.Render("·"), toolSt.Render(e.Text))
case EvDelta:
p.streamed = true
fmt.Fprint(p.Out, e.Text)
@@ -82,9 +107,13 @@ func (r *REPL) Run() error {
pr := &Printer{Out: r.Out, MD: r.md}
a.OnEvent = pr.Handle
+ scope := "general"
+ if a.Node() {
+ scope = a.Ctx.Profile.Name
+ }
fmt.Fprintf(r.Out, "%s — %s/%s on %s · %s\n%s\n\n",
promptSt.Render("cometcli agent"), a.Provider.Name(), a.Model,
- a.Ctx.Profile.Name, a.Policy, toolSt.Render("type /help for commands, ctrl+c cancels a running turn, /exit quits"))
+ scope, a.Policy, toolSt.Render("type /help for commands, ctrl+c cancels a running turn, /exit quits"))
sc := bufio.NewScanner(r.In)
sc.Buffer(make([]byte, 1<<20), 1<<20)
@@ -142,7 +171,7 @@ func (r *REPL) slash(cmd string) (prompt string, quit bool) {
case "/exit", "/quit", "/q":
return "", true
case "/help":
- fmt.Fprintln(r.Out, "Commands:\n"+CommandHelp+"\n /exit quit")
+ fmt.Fprintln(r.Out, "Commands:\n"+HelpText(r.Agent)+"\n /exit quit")
return "", false
}
res, err := RunCommand(r.Agent, r.Agent.Ctx, r.Agent.Reg, cmd)
@@ -155,6 +184,13 @@ func (r *REPL) slash(cmd string) (prompt string, quit bool) {
if res.Text != "" {
fmt.Fprintln(r.Out, toolSt.Render(res.Text))
}
+ if res.Job != nil {
+ if txt, err := res.Job(r.Agent.Ctx); err != nil {
+ fmt.Fprintln(r.Out, errSt.Render(err.Error()))
+ } else if txt != "" {
+ fmt.Fprintln(r.Out, toolSt.Render(txt))
+ }
+ }
}
return res.Prompt, false
}
@@ -175,7 +211,7 @@ func CompactArgs(args map[string]any) string {
}
s := strings.Join(parts, " ")
if len(s) > 100 {
- s = s[:100] + "…"
+ s = safeCut(s, 100) + "…"
}
return s
}
diff --git a/internal/agent/retry.go b/internal/agent/retry.go
new file mode 100644
index 0000000..97ea74c
--- /dev/null
+++ b/internal/agent/retry.go
@@ -0,0 +1,188 @@
+package agent
+
+import (
+ "bytes"
+ "context"
+ "errors"
+ "fmt"
+ "io"
+ "math/rand/v2"
+ "net"
+ "net/http"
+ "os"
+ "regexp"
+ "strconv"
+ "strings"
+ "time"
+)
+
+// RetryNotice is told about each retry so front-ends can show it — a
+// silent backoff looks exactly like a hang.
+type RetryNotice func(attempt int, wait time.Duration, reason string)
+
+type retryNoticeKey struct{}
+
+// WithRetryNotice attaches a retry observer to ctx.
+func WithRetryNotice(ctx context.Context, fn RetryNotice) context.Context {
+ return context.WithValue(ctx, retryNoticeKey{}, fn)
+}
+
+// debugOut receives one line per HTTP attempt when COMETCLI_DEBUG is set.
+var debugOut io.Writer = os.Stderr
+
+func debugf(format string, args ...any) {
+ if v := os.Getenv("COMETCLI_DEBUG"); v != "" && v != "0" {
+ fmt.Fprintf(debugOut, "[debug] "+format+"\n", args...)
+ }
+}
+
+// retryPolicy bounds how hard doHTTP tries before surfacing an error.
+type retryPolicy struct {
+ Max int // retries after the first attempt
+ Base time.Duration // first backoff
+ Cap time.Duration // longest single wait
+}
+
+var defaultRetry = retryPolicy{Max: 4, Base: time.Second, Cap: 30 * time.Second}
+
+// sleep is swapped out in tests.
+var sleep = func(ctx context.Context, d time.Duration) error {
+ t := time.NewTimer(d)
+ defer t.Stop()
+ select {
+ case <-ctx.Done():
+ return ctx.Err()
+ case <-t.C:
+ return nil
+ }
+}
+
+// retryable reports whether a status is worth another attempt: timeouts,
+// conflicts, rate limits, and server-side failures (incl. Anthropic 529).
+func retryable(code int) bool {
+ return code == http.StatusRequestTimeout || code == http.StatusConflict ||
+ code == http.StatusTooManyRequests || code >= 500
+}
+
+// doHTTP sends a JSON POST, retrying connection errors and retryable
+// statuses with jittered exponential backoff, honoring Retry-After. It
+// only retries before a body is read, so a stream is never replayed
+// half-consumed. The returned response is the caller's to close.
+func doHTTP(ctx context.Context, hc *http.Client, url string, body []byte, hdr map[string]string) (*http.Response, error) {
+ return doHTTPWith(ctx, hc, url, body, hdr, defaultRetry)
+}
+
+func doHTTPWith(ctx context.Context, hc *http.Client, url string, body []byte, hdr map[string]string, rp retryPolicy) (*http.Response, error) {
+ for attempt := 0; ; attempt++ {
+ req, err := http.NewRequestWithContext(ctx, "POST", url, bytes.NewReader(body))
+ if err != nil {
+ return nil, err
+ }
+ req.Header.Set("content-type", "application/json")
+ for k, v := range hdr {
+ if v != "" {
+ req.Header.Set(k, v)
+ }
+ }
+ start := time.Now()
+ resp, err := hc.Do(req)
+ var wait time.Duration
+ var reason string
+ switch {
+ case err != nil:
+ debugf("POST %s attempt %d: %v (%s)", redactURL(url), attempt+1, err, time.Since(start).Round(time.Millisecond))
+ if ctx.Err() != nil || !transient(err) || attempt >= rp.Max {
+ return nil, err
+ }
+ reason = "connection error"
+ case resp.StatusCode == 429 && quotaExhausted(resp):
+ // a daily/credit quota won't reset in seconds: fail now
+ debugf("POST %s attempt %d: HTTP 429 quota exhausted — not retrying", redactURL(url), attempt+1)
+ return resp, nil
+ case retryable(resp.StatusCode) && attempt < rp.Max:
+ debugf("POST %s attempt %d: HTTP %d (%s)", redactURL(url), attempt+1, resp.StatusCode, time.Since(start).Round(time.Millisecond))
+ wait = retryAfter(resp.Header)
+ reason = fmt.Sprintf("HTTP %d", resp.StatusCode)
+ resp.Body.Close()
+ default:
+ debugf("POST %s attempt %d: HTTP %d, headers in %s", redactURL(url), attempt+1, resp.StatusCode, time.Since(start).Round(time.Millisecond))
+ return resp, nil
+ }
+ if wait == 0 {
+ wait = backoff(rp, attempt)
+ }
+ if wait > rp.Cap {
+ wait = rp.Cap
+ }
+ if fn, ok := ctx.Value(retryNoticeKey{}).(RetryNotice); ok && fn != nil {
+ fn(attempt+1, wait, reason)
+ }
+ if err := sleep(ctx, wait); err != nil {
+ return nil, err
+ }
+ }
+}
+
+// transient reports whether a transport error may succeed on retry:
+// refused/reset connections and DNS hiccups. Timeouts are not retried —
+// the client timeout is already minutes long, and repeating it would
+// turn one slow request into a silent multi-minute hang.
+func transient(err error) bool {
+ var ne net.Error
+ if errors.As(err, &ne) && ne.Timeout() {
+ return false
+ }
+ var oe *net.OpError
+ var de *net.DNSError
+ return errors.As(err, &oe) || errors.As(err, &de) || errors.Is(err, net.ErrClosed) || errors.Is(err, io.ErrUnexpectedEOF)
+}
+
+// redactURL drops the query string (some APIs take keys there).
+func redactURL(u string) string {
+ if i := strings.IndexByte(u, '?'); i >= 0 {
+ return u[:i]
+ }
+ return u
+}
+
+func backoff(rp retryPolicy, attempt int) time.Duration {
+ d := rp.Base << attempt
+ if d <= 0 || d > rp.Cap {
+ d = rp.Cap
+ }
+ // full jitter in [d/2, d)
+ return d/2 + time.Duration(rand.Int64N(int64(d/2)+1))
+}
+
+var quotaRe = regexp.MustCompile(`(?i)per[- ]day|daily|quota exceeded|insufficient (quota|credits)|add \d+ credits|billing`)
+
+// quotaExhausted peeks a 429 body for a daily/credit quota (as opposed to
+// a per-minute rate limit), restoring the body for the caller.
+func quotaExhausted(resp *http.Response) bool {
+ raw, _ := io.ReadAll(io.LimitReader(resp.Body, 64<<10))
+ resp.Body.Close()
+ resp.Body = io.NopCloser(bytes.NewReader(raw))
+ return quotaRe.Match(raw)
+}
+
+// retryAfter reads retry-after-ms, then Retry-After (seconds or HTTP date).
+func retryAfter(h http.Header) time.Duration {
+ if v := h.Get("retry-after-ms"); v != "" {
+ if ms, err := strconv.ParseFloat(v, 64); err == nil && ms > 0 {
+ return time.Duration(ms * float64(time.Millisecond))
+ }
+ }
+ v := h.Get("Retry-After")
+ if v == "" {
+ return 0
+ }
+ if s, err := strconv.ParseFloat(v, 64); err == nil && s > 0 {
+ return time.Duration(s * float64(time.Second))
+ }
+ if t, err := http.ParseTime(v); err == nil {
+ if d := time.Until(t); d > 0 {
+ return d
+ }
+ }
+ return 0
+}
diff --git a/internal/agent/scope.go b/internal/agent/scope.go
new file mode 100644
index 0000000..26ce244
--- /dev/null
+++ b/internal/agent/scope.go
@@ -0,0 +1,121 @@
+package agent
+
+import (
+ "fmt"
+ "os"
+ "runtime"
+ "sort"
+ "strings"
+ "time"
+
+ "github.com/abhijitkrm/cometcli/internal/config"
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+)
+
+// GeneralPrompt is the system prompt for general mode: an SRE agent in
+// the operator's terminal, with no node attached.
+func GeneralPrompt(c *toolkit.Context, cwd string) string {
+ host, _ := os.Hostname()
+ var b strings.Builder
+ fmt.Fprintf(&b, `You are cometcli — an expert SRE agent working in the operator's terminal.
+You run shell commands and edit files on their machine through tools; they
+watch and approve anything that changes the system.
+
+Rules you must follow:
+- Questions are about THIS machine unless the operator says otherwise.
+ "Summarize container health", "why is disk full", "check nginx" mean:
+ inspect it now with your tools (docker ps / docker inspect, df -h,
+ systemctl status, logs …) and answer from what you found. Never answer
+ such a question with a generic explanation; give one only when asked
+ how something works in general.
+- Investigate before changing anything; reference exact evidence (log
+ lines, versions, exit codes) in answers.
+- Keep answers terse, technical, and actionable. Use markdown sparingly.
+- NEVER ask for or echo secrets: mnemonics, private keys, tokens.
+- For destructive or irreversible steps, say what will happen first.
+
+%s
+ENVIRONMENT: %s/%s, host %s, working directory %s, date %s
+`, generalToolsText, runtime.GOOS, runtime.GOARCH, host, cwd, time.Now().Format("2006-01-02"))
+ if c != nil && c.Cfg != nil && len(c.Cfg.Profiles) > 0 {
+ var names []string
+ for n, p := range c.Cfg.Profiles {
+ names = append(names, fmt.Sprintf("%s (%s, %s)", n, orNone(p.Role), orNone(p.ChainID)))
+ }
+ sort.Strings(names)
+ active := ""
+ if ap, err := c.Cfg.ActiveProfile(""); err == nil && ap != nil {
+ active = " Active profile: " + ap.Name + "."
+ }
+ fmt.Fprintf(&b, "NODE PROFILES: %s.%s For any question about a node, validator or chain (jailing, sync, peers, upgrades, governance, txs), call use_node with the right profile yourself and continue — never ask the operator to switch. Any profile on a chain answers chain-wide questions; pick the active one when nothing points elsewhere.\n", strings.Join(names, "; "), active)
+ }
+ return b.String()
+}
+
+// generalToolsText describes the general tools; shared by both modes.
+const generalToolsText = `General tools:
+- bash: any shell command; the working directory persists between calls.
+ Read-only commands run at once; changes wait for the operator's
+ approval; transactions always do. stdin is closed: a command or script
+ that prompts (read -p, sudo password, y/N) gets EOF — ask the operator
+ to run those in their terminal instead.
+- read / grep / glob for files — prefer them over cat/grep/find in bash.
+ edit and write need a read of the file first; keep edits minimal.
+- Before running a script, read it and say what it will do.
+- Refusals are final: consensus keys, key ceremonies (keys add/export),
+ mnemonics and state resets never pass through you. Don't retry
+ variations of a refused or denied command; explain and hand the step
+ to the operator with the exact command.
+- web_fetch for release notes and docs. todo_write to track work of 3+
+ steps (an upgrade, a migration) so the operator sees progress.
+`
+
+// SwitchScope moves the session to a node profile (nil = general mode),
+// keeping the conversation. The system prompt and tool set are rebuilt,
+// so provider-bound replay state is dropped and the cache restarts; the
+// model is told about the switch on the next turn.
+func (a *Agent) SwitchScope(p *config.Profile) {
+ note := "[The operator switched this session to general mode: node tools are gone; the shell runs on their machine.]"
+ if p != nil {
+ note = fmt.Sprintf("[The operator switched this session to node mode for profile %s (role %s, chain %s, transport %s); node tools, rules and a live snapshot are now in your instructions.]",
+ p.Name, orNone(p.Role), orNone(p.ChainID), orNone(p.Transport.Type))
+ }
+ a.switchScope(p, note)
+}
+
+// switchScope moves the session to p (nil = general); note, if any, is
+// carried into the next user turn (the model's own use_node needs none).
+func (a *Agent) switchScope(p *config.Profile, note string) {
+ old := a.Ctx
+ c := &toolkit.Context{
+ Context: old.Context, Profile: p, Cfg: old.Cfg, Out: old.Out,
+ Audit: old.Audit, Approver: old.Approver, AutoApproveBelow: old.AutoApproveBelow,
+ Chooser: old.Chooser, Secret: old.Secret,
+ }
+ if a.ownCtx {
+ old.Close()
+ }
+ a.Ctx, a.ownCtx = c, true
+ a.Redact = RedactorFor(p)
+ a.sys, a.snap, a.toolSig = "", "", ""
+ a.loaded = nil
+ a.Tools = toolkit.NewSession("")
+ a.dropBoundRaw()
+ if note == "" {
+ return
+ }
+ if a.carry != "" {
+ a.carry += "\n\n"
+ }
+ a.carry += note
+}
+
+// Close releases a context created by SwitchScope and stops MCP servers.
+func (a *Agent) Close() {
+ if a.ownCtx {
+ a.Ctx.Close()
+ }
+ if a.parent == nil {
+ a.ext.close(a.Reg)
+ }
+}
diff --git a/internal/agent/session.go b/internal/agent/session.go
new file mode 100644
index 0000000..005205c
--- /dev/null
+++ b/internal/agent/session.go
@@ -0,0 +1,221 @@
+package agent
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "os"
+ "path/filepath"
+ "sort"
+ "strings"
+ "time"
+
+ "github.com/abhijitkrm/cometcli/internal/config"
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+)
+
+// SessionFile is a saved conversation, written after every turn to
+// ~/.cometcli/sessions/.json (0600). Everything in it already passed
+// the redactor on its way to the model.
+type SessionFile struct {
+ Version int `json:"version"`
+ ID string `json:"id"`
+ Created time.Time `json:"created"`
+ Updated time.Time `json:"updated"`
+ Title string `json:"title"`
+ Cwd string `json:"cwd"`
+ Profile string `json:"profile,omitempty"`
+ Provider string `json:"provider"`
+ Model string `json:"model"`
+ Usage Usage `json:"usage"`
+ // System is the frozen system prompt, restored verbatim so a resumed
+ // session keeps its cached prefix (and replayable thinking) intact.
+ System string `json:"system"`
+ ToolSig string `json:"tool_sig,omitempty"`
+ Carry string `json:"carry,omitempty"`
+ History []Msg `json:"history"`
+ // ShellCwd is the bash tool's working directory; Todos the checklist.
+ ShellCwd string `json:"shell_cwd,omitempty"`
+ Todos []Todo `json:"todos,omitempty"`
+ Loaded []string `json:"loaded_tools,omitempty"`
+}
+
+const sessionVersion = 1
+
+// ErrNoSession is returned when there is nothing to resume.
+var ErrNoSession = errors.New("no saved session to resume")
+
+func sessionDir() (string, error) {
+ d, err := config.Path("sessions")
+ if err != nil {
+ return "", err
+ }
+ return d, os.MkdirAll(d, 0o700)
+}
+
+// sessionPath maps an id to its file, rejecting anything path-like.
+func sessionPath(id string) (string, error) {
+ if id == "" || strings.ContainsAny(id, `/\`) || strings.Contains(id, "..") {
+ return "", fmt.Errorf("invalid session id %q", id)
+ }
+ d, err := sessionDir()
+ if err != nil {
+ return "", err
+ }
+ return filepath.Join(d, id+".json"), nil
+}
+
+// Save writes the session atomically. A session with no turns is skipped.
+func (a *Agent) Save() error {
+ if len(a.history) == 0 && a.carry == "" {
+ return nil
+ }
+ p, err := sessionPath(a.ID())
+ if err != nil {
+ return err
+ }
+ cwd, _ := os.Getwd()
+ sf := SessionFile{
+ Version: sessionVersion, ID: a.ID(), Created: a.created, Updated: time.Now(),
+ Title: a.title(), Cwd: cwd, Profile: a.profileName(),
+ Provider: a.Provider.Name(), Model: a.Model, Usage: a.total,
+ System: a.sys, ToolSig: a.toolSig, Carry: a.carry, History: a.history,
+ ShellCwd: a.Tools.Cwd(""), Todos: a.todos, Loaded: a.LoadedTools(),
+ }
+ raw, err := json.Marshal(sf)
+ if err != nil {
+ return err
+ }
+ tmp := p + ".tmp"
+ if err := os.WriteFile(tmp, raw, 0o600); err != nil {
+ return err
+ }
+ return os.Rename(tmp, p)
+}
+
+// title is the first user prompt, minus injected prefixes, on one line.
+func (a *Agent) title() string {
+ for _, m := range a.history {
+ if m.Role != "user" {
+ continue
+ }
+ t := m.Text
+ for _, tag := range []string{"", ""} {
+ if i := strings.LastIndex(t, tag); i >= 0 {
+ t = t[i+len(tag):]
+ }
+ }
+ t = strings.Join(strings.Fields(t), " ")
+ if len(t) > 80 {
+ t = safeCut(t, 80) + "…"
+ }
+ if t != "" {
+ return t
+ }
+ }
+ return "(compacted session)"
+}
+
+// Restore loads a saved session into the agent. The conversation is kept
+// even if the provider or model changed since; raw output from another
+// provider is simply not replayed.
+func (a *Agent) Restore(sf *SessionFile) {
+ a.Reset()
+ a.id, a.created = sf.ID, sf.Created
+ a.history, a.sys, a.toolSig, a.carry = sf.History, sf.System, sf.ToolSig, sf.Carry
+ a.total = sf.Usage
+ a.todos = sf.Todos
+ a.loaded = map[string]bool{}
+ for _, n := range sf.Loaded {
+ a.loaded[n] = true
+ }
+ a.Tools = toolkit.NewSession(sf.ShellCwd)
+ if a.ext != nil {
+ a.ext.resumed = true
+ }
+ // the frozen system prompt carries the old snapshot: force a refresh
+ // onto the next turn
+ a.snapAt = time.Time{}
+}
+
+// LoadSession reads one saved session by id (a unique prefix works).
+func LoadSession(id string) (*SessionFile, error) {
+ p, err := sessionPath(id)
+ if err != nil {
+ return nil, err
+ }
+ raw, err := os.ReadFile(p)
+ if errors.Is(err, os.ErrNotExist) {
+ all, lerr := ListSessions()
+ if lerr != nil {
+ return nil, lerr
+ }
+ var hit *SessionFile
+ for _, s := range all {
+ if strings.HasPrefix(s.ID, id) {
+ if hit != nil {
+ return nil, fmt.Errorf("session id %q is ambiguous", id)
+ }
+ hit = s
+ }
+ }
+ if hit == nil {
+ return nil, fmt.Errorf("no session %q", id)
+ }
+ return LoadSession(hit.ID)
+ }
+ if err != nil {
+ return nil, err
+ }
+ var sf SessionFile
+ if err := json.Unmarshal(raw, &sf); err != nil {
+ return nil, fmt.Errorf("session %s: %w", id, err)
+ }
+ return &sf, nil
+}
+
+// ListSessions returns saved sessions, most recently updated first, with
+// History omitted.
+func ListSessions() ([]*SessionFile, error) {
+ d, err := sessionDir()
+ if err != nil {
+ return nil, err
+ }
+ ents, err := os.ReadDir(d)
+ if err != nil {
+ return nil, err
+ }
+ var out []*SessionFile
+ for _, e := range ents {
+ if e.IsDir() || !strings.HasSuffix(e.Name(), ".json") {
+ continue
+ }
+ raw, err := os.ReadFile(filepath.Join(d, e.Name()))
+ if err != nil {
+ continue
+ }
+ var sf SessionFile
+ if json.Unmarshal(raw, &sf) != nil || sf.ID == "" {
+ continue
+ }
+ sf.History, sf.System = nil, ""
+ out = append(out, &sf)
+ }
+ sort.Slice(out, func(i, j int) bool { return out[i].Updated.After(out[j].Updated) })
+ return out, nil
+}
+
+// LatestSession returns the most recent session started in cwd (and on
+// profile, when set) — what `--continue` resumes.
+func LatestSession(cwd, profile string) (*SessionFile, error) {
+ all, err := ListSessions()
+ if err != nil {
+ return nil, err
+ }
+ for _, s := range all {
+ if s.Cwd == cwd && (profile == "" || s.Profile == profile) {
+ return LoadSession(s.ID)
+ }
+ }
+ return nil, ErrNoSession
+}
diff --git a/internal/agent/session_test.go b/internal/agent/session_test.go
index a85b7fc..dea917e 100644
--- a/internal/agent/session_test.go
+++ b/internal/agent/session_test.go
@@ -159,8 +159,10 @@ func TestReadOnlyHidesAndBlocks(t *testing.T) {
if ran {
t.Fatal("local-change tool ran in readonly mode")
}
- if len(prov.reqs[0].Tools) != 0 {
- t.Fatalf("readonly advertised mutating tools: %+v", prov.reqs[0].Tools)
+ for _, td := range prov.reqs[0].Tools {
+ if td.Name != todoToolName && td.Name != toolSearchName && td.Name != taskToolName {
+ t.Fatalf("readonly advertised mutating tool %s", td.Name)
+ }
}
}
diff --git a/internal/agent/steer.go b/internal/agent/steer.go
new file mode 100644
index 0000000..a98607f
--- /dev/null
+++ b/internal/agent/steer.go
@@ -0,0 +1,44 @@
+package agent
+
+import "strings"
+
+// Steer queues a message the operator typed while a turn is running. It
+// reaches the model at its next step — appended to the latest tool
+// result, which keeps the message structure valid for every provider —
+// instead of waiting for the whole turn to end. Safe to call from any
+// goroutine.
+func (a *Agent) Steer(text string) {
+ if text = strings.TrimSpace(text); text == "" {
+ return
+ }
+ a.steerMu.Lock()
+ a.steer = append(a.steer, text)
+ a.steerMu.Unlock()
+}
+
+// TakeSteer returns (and clears) messages not yet delivered — the front
+// end sends them as the next turn when the current one ends first.
+func (a *Agent) TakeSteer() []string {
+ a.steerMu.Lock()
+ defer a.steerMu.Unlock()
+ s := a.steer
+ a.steer = nil
+ return s
+}
+
+// deliverSteer folds queued operator messages into the last tool result.
+func (a *Agent) deliverSteer() {
+ msgs := a.TakeSteer()
+ if len(msgs) == 0 {
+ return
+ }
+ i := len(a.history) - 1
+ if i < 0 || a.history[i].Role != "tool" {
+ a.Steer(strings.Join(msgs, "\n\n")) // nothing to attach to: keep for the next turn
+ return
+ }
+ note := "\n\n[The operator, while you were working: " + a.Redact.Text(strings.Join(msgs, "\n\n")) +
+ "]\nTake this into account now — it may change or stop what you were doing."
+ a.history[i].Text += note
+ a.emit(Event{Kind: EvNotice, Text: "your message reached the agent"})
+}
diff --git a/internal/agent/steer_test.go b/internal/agent/steer_test.go
new file mode 100644
index 0000000..a9c31fe
--- /dev/null
+++ b/internal/agent/steer_test.go
@@ -0,0 +1,76 @@
+package agent
+
+import (
+ "context"
+ "encoding/json"
+ "fmt"
+ "strings"
+ "testing"
+
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+)
+
+func TestSteerReachesTheNextStep(t *testing.T) {
+ var a *Agent
+ slow := stubTool{name: "node.status", tier: toolkit.TierObserve, run: func(c *toolkit.Context, _ toolkit.Args) (*toolkit.Result, error) {
+ a.Steer("stop — don't restart anything") // typed while the tool runs
+ return &toolkit.Result{Text: "height 42"}, nil
+ }}
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{{ID: "c1", Name: "node__status", Args: json.RawMessage(`{}`)}}},
+ {Text: "ok, stopping", Done: true},
+ }}
+ a = newTestAgent(t, prov, slow)
+ if _, err := a.Run(context.Background(), "check and restart the node"); err != nil {
+ t.Fatal(err)
+ }
+ last := prov.reqs[1].Messages
+ got := last[len(last)-1]
+ if got.Role != "tool" || !strings.Contains(got.Text, "stop — don't restart anything") || !strings.Contains(got.Text, "height 42") {
+ t.Fatalf("steer not delivered with the tool result: %+v", got)
+ }
+ if left := a.TakeSteer(); len(left) != 0 {
+ t.Fatalf("delivered message still queued: %v", left)
+ }
+}
+
+func TestSteerTypedDuringFinalAnswerIsKept(t *testing.T) {
+ prov := &mockProvider{responses: []*Response{{Text: "done", Done: true}}}
+ a := newTestAgent(t, prov)
+ a.Steer("also check peers")
+ if _, err := a.Run(context.Background(), "hi"); err != nil {
+ t.Fatal(err)
+ }
+ // no tool step happened: the front end gets it back to send next
+ if left := a.TakeSteer(); len(left) != 1 || left[0] != "also check peers" {
+ t.Fatalf("left = %v", left)
+ }
+}
+
+func TestStepLimitPausesInsteadOfFailing(t *testing.T) {
+ loop := stubTool{name: "node.status", tier: toolkit.TierObserve, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) {
+ return &toolkit.Result{Text: "still syncing"}, nil
+ }}
+ var resps []*Response
+ for i := 0; i < 10; i++ {
+ resps = append(resps, &Response{Calls: []Call{{ID: fmt.Sprint("c", i), Name: "node__status", Args: json.RawMessage(`{}`)}}})
+ }
+ a := newTestAgent(t, &mockProvider{responses: resps}, loop)
+ a.MaxIter = 3
+ var notices []string
+ a.OnEvent = func(e Event) {
+ if e.Kind == EvNotice {
+ notices = append(notices, e.Text)
+ }
+ }
+ if _, err := a.Run(context.Background(), "wait for sync"); err != nil {
+ t.Fatalf("step limit should pause, got error: %v", err)
+ }
+ if len(notices) == 0 || !strings.Contains(notices[len(notices)-1], `say "continue"`) {
+ t.Fatalf("notices: %v", notices)
+ }
+ h := a.History()
+ if h[len(h)-1].Role != "tool" {
+ t.Fatal("history must end with answered calls so 'continue' works")
+ }
+}
diff --git a/internal/agent/todo.go b/internal/agent/todo.go
new file mode 100644
index 0000000..c6a9426
--- /dev/null
+++ b/internal/agent/todo.go
@@ -0,0 +1,85 @@
+package agent
+
+import (
+ "fmt"
+ "strings"
+)
+
+// Todo is one item of the agent's task list (todo_write).
+type Todo struct {
+ Content string `json:"content"`
+ Status string `json:"status"` // pending | in_progress | completed
+}
+
+const todoToolName = "todo_write"
+
+// todoToolDef is a built-in tool: the agent keeps a visible checklist for
+// multi-step work (an upgrade, a migration) and updates it as it goes.
+var todoToolDef = ToolDef{
+ Name: todoToolName,
+ Desc: "Replace the session's task checklist, shown to the operator. Use it for work with 3+ steps: list the steps, keep exactly one in_progress, mark each completed as soon as it is done. Send the full list every time.",
+ Schema: map[string]any{
+ "type": "object",
+ "properties": map[string]any{
+ "todos": map[string]any{
+ "type": "array",
+ "items": map[string]any{
+ "type": "object",
+ "properties": map[string]any{
+ "content": map[string]any{"type": "string", "description": "the step, imperative"},
+ "status": map[string]any{"type": "string", "enum": []string{"pending", "in_progress", "completed"}},
+ },
+ "required": []string{"content", "status"},
+ },
+ },
+ },
+ "required": []string{"todos"},
+ },
+}
+
+// Todos returns the current checklist.
+func (a *Agent) Todos() []Todo { return append([]Todo(nil), a.todos...) }
+
+func (a *Agent) todoWrite(call Call, args map[string]any) Msg {
+ raw, _ := args["todos"].([]any)
+ var todos []Todo
+ for _, r := range raw {
+ m, _ := r.(map[string]any)
+ c, _ := m["content"].(string)
+ s, _ := m["status"].(string)
+ if strings.TrimSpace(c) == "" {
+ continue
+ }
+ switch s {
+ case "pending", "in_progress", "completed":
+ default:
+ return a.toolErr(call, todoToolName, fmt.Sprintf("bad status %q — want pending, in_progress or completed", s))
+ }
+ todos = append(todos, Todo{Content: a.Redact.Text(c), Status: s})
+ }
+ a.todos = todos
+ a.emit(Event{Kind: EvTodos, Todos: a.Todos()})
+ done := 0
+ for _, t := range todos {
+ if t.Status == "completed" {
+ done++
+ }
+ }
+ return Msg{Role: "tool", CallID: call.ID, ToolName: todoToolName, Text: fmt.Sprintf("checklist updated: %d/%d done", done, len(todos))}
+}
+
+// RenderTodos draws the checklist as plain lines.
+func RenderTodos(todos []Todo) string {
+ var b strings.Builder
+ for _, t := range todos {
+ mark := "☐"
+ switch t.Status {
+ case "in_progress":
+ mark = "▸"
+ case "completed":
+ mark = "☒"
+ }
+ fmt.Fprintf(&b, "%s %s\n", mark, t.Content)
+ }
+ return strings.TrimRight(b.String(), "\n")
+}
diff --git a/internal/agent/toolsearch.go b/internal/agent/toolsearch.go
new file mode 100644
index 0000000..d0a112e
--- /dev/null
+++ b/internal/agent/toolsearch.go
@@ -0,0 +1,224 @@
+package agent
+
+import (
+ "fmt"
+ "sort"
+ "strings"
+
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+)
+
+// Deferred tool loading. Sending all ~65 tool schemas costs ~6.5k tokens
+// per request — more than some providers' whole per-minute budget — so
+// by default each request carries a core set, the system prompt lists the
+// rest by name, and tool_search loads full definitions on demand. Loaded
+// tools stay for the session (and are saved with it).
+
+const toolSearchName = "tool_search"
+
+// coreTools are always advertised (with the agent-only general tools).
+var coreTools = map[string]bool{
+ "node.status": true, "node.health": true, "node.logs": true,
+ "val.status": true, "val.signing": true,
+ "val.jail-check": true, "val.consensus": true, "wait.until": true, "chain.validators": true,
+ "node.triage": true, "kb.show": true, "kb.search": true,
+}
+
+var toolSearchDef = ToolDef{
+ Name: toolSearchName,
+ Desc: "Load more tools so you can call them. Pass names from the tool catalog in the system prompt (\"select:val.unjail,chain.gov\") or keywords (\"governance proposals vote\"). Loaded tools are callable from your next step on.",
+ Schema: toolkit.ObjSchema(map[string]any{
+ "query": toolkit.Str(`"select:name1,name2" for exact tools, or keywords`),
+ }, "query"),
+}
+
+// deferred reports whether this session loads tools on demand: node
+// tools in node mode, MCP server tools in either mode.
+func (a *Agent) deferred() bool {
+ if a.conf.Tools == "all" {
+ return false
+ }
+ if a.Node() {
+ return true
+ }
+ for _, t := range a.Reg.All() {
+ if isMCP(t) {
+ return true
+ }
+ }
+ return false
+}
+
+// deferrable reports whether t is listed in the catalog rather than sent.
+func (a *Agent) deferrable(t toolkit.Tool) bool {
+ if isMCP(t) {
+ return true
+ }
+ return a.Node() && !toolkit.IsAgentOnly(t) && !coreTools[t.Name()] && !toolkit.IsLongRunning(t) && !toolkit.IsOperatorOnly(t) && strings.Contains(t.Name(), ".")
+}
+
+// advertised reports whether t goes into this request's tool list.
+func (a *Agent) advertised(t toolkit.Tool) bool {
+ if isMCP(t) {
+ return a.conf.Tools == "all" || a.loaded[t.Name()]
+ }
+ if toolkit.IsAgentOnly(t) {
+ return true
+ }
+ if !a.Node() {
+ return false // node tools need a node: /one
+ }
+ if !a.deferred() || coreTools[t.Name()] {
+ return true
+ }
+ return a.loaded[t.Name()]
+}
+
+// toolCatalog renders the not-yet-advertised tools by domain, names only,
+// for the frozen system prompt.
+func (a *Agent) toolCatalog() string {
+ if !a.deferred() {
+ return ""
+ }
+ groups := map[string][]string{}
+ var order []string
+ for _, t := range a.Reg.All() {
+ if !a.deferrable(t) {
+ continue
+ }
+ dom, verb, ok := strings.Cut(t.Name(), ".")
+ if isMCP(t) {
+ rest := strings.TrimPrefix(t.Name(), "mcp__")
+ srv, tool, _ := strings.Cut(rest, "__")
+ dom, verb, ok = "mcp__"+srv, tool, true
+ }
+ if !ok {
+ continue
+ }
+ if _, seen := groups[dom]; !seen {
+ order = append(order, dom)
+ }
+ groups[dom] = append(groups[dom], verb)
+ }
+ if len(order) == 0 {
+ return ""
+ }
+ var b strings.Builder
+ b.WriteString("TOOL CATALOG — load with tool_search before calling (e.g. select:val.unjail, select:mcp__server__tool):\n")
+ for _, d := range order {
+ fmt.Fprintf(&b, "- %s: %s\n", d, strings.Join(groups[d], ", "))
+ }
+ return b.String() + "\n"
+}
+
+func (a *Agent) toolSearch(call Call, args map[string]any) Msg {
+ q := strings.TrimSpace(fmt.Sprint(args["query"]))
+ if q == "" || q == "" {
+ return a.toolErr(call, toolSearchName, "query is required")
+ }
+ var picks []toolkit.Tool
+ var missing []string
+ if names, ok := strings.CutPrefix(q, "select:"); ok {
+ for _, n := range strings.Split(names, ",") {
+ n = toolkit.ResolveName(strings.TrimSpace(n))
+ if t, ok := a.Reg.Get(n); ok {
+ picks = append(picks, t)
+ } else if n != "" {
+ missing = append(missing, n)
+ }
+ }
+ } else {
+ picks = a.searchTools(q, 5)
+ }
+ if a.loaded == nil {
+ a.loaded = map[string]bool{}
+ }
+ var b strings.Builder
+ var names []string
+ for _, t := range picks {
+ if toolkit.IsLongRunning(t) {
+ continue
+ }
+ if !a.deferrable(t) && !coreTools[t.Name()] {
+ missing = append(missing, t.Name()+" (node tool: call use_node with a profile first)")
+ continue
+ }
+ a.loaded[t.Name()] = true
+ names = append(names, toolFnName(t.Name()))
+ note := ""
+ if !a.Policy.Allows(t.Tier()) && !toolkit.IsDynamic(t) {
+ note = " (not callable in readonly mode)"
+ }
+ fmt.Fprintf(&b, "- %s [%s] %s%s\n", toolFnName(t.Name()), t.Tier(), t.Desc(), note)
+ }
+ if len(names) == 0 {
+ msg := "no tools matched " + q + " — use names from the catalog"
+ if len(missing) > 0 {
+ msg = "unknown tools: " + strings.Join(missing, ", ")
+ }
+ return a.toolErr(call, toolSearchName, msg)
+ }
+ text := "loaded: " + strings.Join(names, ", ") + " — callable from your next step\n" + b.String()
+ if len(missing) > 0 {
+ text += "unknown: " + strings.Join(missing, ", ") + "\n"
+ }
+ a.emit(Event{Kind: EvToolResult, Tool: toolSearchName, Text: "loaded " + strings.Join(names, ", ")})
+ return Msg{Role: "tool", CallID: call.ID, ToolName: toolSearchName, Text: strings.TrimRight(text, "\n")}
+}
+
+// searchTools ranks tools by keyword hits in name (weighted) and
+// description.
+func (a *Agent) searchTools(q string, n int) []toolkit.Tool {
+ words := strings.Fields(strings.ToLower(q))
+ type hit struct {
+ t toolkit.Tool
+ score int
+ }
+ var hits []hit
+ for _, t := range a.Reg.All() {
+ if !a.deferrable(t) && !coreTools[t.Name()] {
+ continue
+ }
+ name, desc := strings.ToLower(t.Name()), strings.ToLower(t.Desc())
+ s := 0
+ for _, w := range words {
+ if len(w) < 3 && name != w {
+ continue
+ }
+ if strings.Contains(name, w) {
+ s += 3
+ }
+ if strings.Contains(desc, w) {
+ s++
+ }
+ }
+ if s > 0 {
+ hits = append(hits, hit{t, s})
+ }
+ }
+ sort.SliceStable(hits, func(i, j int) bool { return hits[i].score > hits[j].score })
+ var out []toolkit.Tool
+ for i := 0; i < len(hits) && i < n; i++ {
+ out = append(out, hits[i].t)
+ }
+ return out
+}
+
+// LoadedTools returns the names loaded with tool_search, sorted.
+func (a *Agent) LoadedTools() []string {
+ var out []string
+ for n := range a.loaded {
+ out = append(out, n)
+ }
+ sort.Strings(out)
+ return out
+}
+
+// ToolNames lists the tools the next request would advertise.
+func (a *Agent) ToolNames() []string {
+ var out []string
+ for _, d := range a.toolDefs() {
+ out = append(out, d.Name)
+ }
+ return out
+}
diff --git a/internal/agent/uisupport.go b/internal/agent/uisupport.go
new file mode 100644
index 0000000..bf1be1c
--- /dev/null
+++ b/internal/agent/uisupport.go
@@ -0,0 +1,181 @@
+package agent
+
+import (
+ "fmt"
+ "os"
+ "path/filepath"
+ "regexp"
+ "sort"
+ "strings"
+
+ "github.com/abhijitkrm/cometcli/internal/settings"
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+ "github.com/abhijitkrm/cometcli/internal/tools/shell"
+)
+
+// CmdInfo describes a slash command for completion menus.
+type CmdInfo struct {
+ Name string // with the leading slash
+ Args string
+ Desc string
+}
+
+var helpLineRe = regexp.MustCompile(`^\s+(/\S+)(.*?)\s{2,}(\S.*)$`)
+
+// Commands lists the slash commands available in this session: the
+// shared ones, front-end ones, and custom commands.
+func Commands(a *Agent) []CmdInfo {
+ var out []CmdInfo
+ seen := map[string]bool{}
+ add := func(c CmdInfo) {
+ for _, n := range strings.Split(c.Name, ",") {
+ n = strings.TrimSpace(n)
+ if n == "" || seen[n] {
+ continue
+ }
+ seen[n] = true
+ out = append(out, CmdInfo{Name: n, Args: c.Args, Desc: c.Desc})
+ }
+ }
+ for _, l := range strings.Split(HelpText(nil)+"\n /exit quit", "\n") {
+ l = strings.ReplaceAll(l, ", /", "|/") // "/reset, /clear" names two commands
+ m := helpLineRe.FindStringSubmatch(l)
+ if m == nil {
+ continue
+ }
+ names := strings.Split(m[1], "|")
+ args := strings.TrimSpace(m[2])
+ for _, n := range names {
+ if !strings.HasPrefix(n, "/") {
+ n = "/" + n
+ }
+ add(CmdInfo{Name: strings.TrimSuffix(n, ","), Args: args, Desc: m[3]})
+ }
+ }
+ for _, c := range settings.SortedCommands(builtinCommands) {
+ if a != nil && a.ext != nil && a.ext.Commands[c.Name] != nil {
+ continue // overridden
+ }
+ add(CmdInfo{Name: "/" + c.Name, Args: c.ArgHint, Desc: c.Description})
+ }
+ if a != nil && a.ext != nil {
+ for _, c := range settings.SortedCommands(a.ext.Commands) {
+ add(CmdInfo{Name: "/" + c.Name, Args: c.ArgHint, Desc: c.Description + " (" + c.Scope + ")"})
+ }
+ }
+ sort.SliceStable(out, func(i, j int) bool { return out[i].Name < out[j].Name })
+ return out
+}
+
+var mentionRe = regexp.MustCompile(`(^|\s)@([\w./~-][\w./~:-]*)`)
+
+// ExpandMentions attaches the content of @path mentions (files on this
+// machine, relative to the shell's working directory) to a prompt. Key
+// material and large or unreadable files are left as plain text.
+func (a *Agent) ExpandMentions(input string) string {
+ cwd := a.Tools.Cwd(a.WorkRoot)
+ var files []string
+ for _, m := range mentionRe.FindAllStringSubmatch(input, 10) {
+ p := m[2]
+ full := p
+ if strings.HasPrefix(p, "~/") {
+ home, _ := os.UserHomeDir()
+ full = filepath.Join(home, p[2:])
+ } else if !filepath.IsAbs(p) {
+ full = filepath.Join(cwd, p)
+ }
+ if shell.Classify("cat "+full, shell.Opts{}).Forbidden != "" {
+ continue
+ }
+ fi, err := os.Stat(full)
+ if err != nil || fi.IsDir() || fi.Size() > 100_000 {
+ continue
+ }
+ b, err := os.ReadFile(full)
+ if err != nil || strings.IndexByte(string(b[:min(len(b), 8192)]), 0) >= 0 {
+ continue
+ }
+ if a.Tools != nil {
+ h, _ := a.Ctx.Host()
+ key := "local:" + full
+ if h != nil {
+ key = h.String() + ":" + full
+ }
+ a.Tools.MarkRead(key, b) // the model has seen it: edits may follow
+ }
+ files = append(files, fmt.Sprintf("\n%s\n", full, strings.TrimRight(string(b), "\n")))
+ }
+ if len(files) == 0 {
+ return input
+ }
+ return input + "\n\n" + strings.Join(files, "\n")
+}
+
+// ContextPercent is how full the context window is (0 when unknown).
+func (a *Agent) ContextPercent() int {
+ _, last := a.Usage()
+ if last == 0 {
+ return 0
+ }
+ return min(100, last*100/a.contextWindow())
+}
+
+// AddNote tells the model something on its next turn (e.g. the operator
+// ran a command in their own terminal).
+func (a *Agent) AddNote(note string) {
+ a.carry = strings.TrimSpace(a.carry + "\n\n" + a.Redact.Text(note))
+}
+
+// AllowAlways adds an allow rule to the session and saves it to the
+// project's settings.local.json (or the user settings without a
+// project). Returns where it was saved.
+func (a *Agent) AllowAlways(rule string) (string, error) {
+ if a.Rules == nil {
+ a.Rules, _ = toolkit.NewRules(nil, nil, nil)
+ }
+ if err := a.Rules.Add("allow", rule); err != nil {
+ return "", err
+ }
+ path, err := settings.UserPath()
+ if a.ext != nil && a.ext.Settings.Root != "" {
+ path, err = filepath.Join(a.ext.Settings.Root, ".cometcli", "settings.local.json"), nil
+ }
+ if err != nil {
+ return "", err
+ }
+ return path, settings.EditFile(path, func(f *settings.File) error {
+ for _, r := range f.Permissions.Allow {
+ if r == rule {
+ return nil
+ }
+ }
+ f.Permissions.Allow = append(f.Permissions.Allow, rule)
+ return nil
+ })
+}
+
+// ProjectFiles lists files under the working directory for @ completion
+// (skipping VCS, dependency and build directories), up to limit.
+func (a *Agent) ProjectFiles(limit int) []string {
+ root := a.Tools.Cwd(a.WorkRoot)
+ var out []string
+ skip := map[string]bool{".git": true, "node_modules": true, "vendor": true, ".build": true, "dist": true, "evm": true, ".mnemonics": true}
+ _ = filepath.WalkDir(root, func(p string, d os.DirEntry, err error) error {
+ if err != nil {
+ return nil
+ }
+ if d.IsDir() {
+ if p != root && (skip[d.Name()] || strings.HasPrefix(d.Name(), ".") && d.Name() != ".cometcli") {
+ return filepath.SkipDir
+ }
+ return nil
+ }
+ rel, _ := filepath.Rel(root, p)
+ out = append(out, rel)
+ if len(out) >= limit {
+ return filepath.SkipAll
+ }
+ return nil
+ })
+ return out
+}
diff --git a/internal/agent/usenode.go b/internal/agent/usenode.go
new file mode 100644
index 0000000..aa5b62b
--- /dev/null
+++ b/internal/agent/usenode.go
@@ -0,0 +1,52 @@
+package agent
+
+import (
+ "fmt"
+ "sort"
+ "strings"
+)
+
+const useNodeName = "use_node"
+
+// useNodeDef lets the model move the session onto a node profile itself
+// (or back to general), instead of asking the operator to type /one.
+func (a *Agent) useNodeDef() (ToolDef, bool) {
+ if a.parent != nil || a.Ctx == nil || a.Ctx.Cfg == nil || len(a.Ctx.Cfg.Profiles) == 0 {
+ return ToolDef{}, false
+ }
+ names := []string{"off"}
+ for n := range a.Ctx.Cfg.Profiles {
+ names = append(names, n)
+ }
+ sort.Strings(names[1:])
+ return ToolDef{
+ Name: useNodeName,
+ Desc: "[observe] Work on a node: switch this session to a node profile (adds its node, chain, validator and incident tools, " +
+ "safety rules and a live snapshot), or 'off' for general mode. Call it yourself whenever a question is about a node, " +
+ "validator or chain — never ask the operator to switch. Any profile on a chain answers chain-wide questions.",
+ Schema: map[string]any{"type": "object", "properties": map[string]any{
+ "profile": map[string]any{"type": "string", "enum": names, "description": "profile name, or off"},
+ }, "required": []string{"profile"}},
+ }, true
+}
+
+func (a *Agent) useNode(call Call, args map[string]any) Msg {
+ name := strings.TrimSpace(fmt.Sprint(args["profile"]))
+ a.emit(Event{Kind: EvToolStart, Tool: useNodeName, Args: map[string]any{"profile": name}})
+ if name == "off" || name == "" {
+ a.switchScope(nil, "")
+ a.emit(Event{Kind: EvToolResult, Tool: useNodeName, Text: "general mode"})
+ return Msg{Role: "tool", CallID: call.ID, ToolName: useNodeName, Text: "now in general mode: shell, files and web on the operator's machine"}
+ }
+ p, ok := a.Ctx.Cfg.Profiles[name]
+ if !ok {
+ return a.toolErr(call, useNodeName, "no profile "+name)
+ }
+ p.Name = name
+ a.switchScope(p, "")
+ text := fmt.Sprintf("now working on %s (role %s, chain %s): node tools are callable from your next step — "+
+ "node.triage for health/incidents, chain.validators (status=jailed lists jailed validators), val.* for this validator; "+
+ "load others with tool_search", name, orNone(p.Role), orNone(p.ChainID))
+ a.emit(Event{Kind: EvToolResult, Tool: useNodeName, Text: "node mode: " + name})
+ return Msg{Role: "tool", CallID: call.ID, ToolName: useNodeName, Text: text}
+}
diff --git a/internal/agent/usenode_test.go b/internal/agent/usenode_test.go
new file mode 100644
index 0000000..a8c1458
--- /dev/null
+++ b/internal/agent/usenode_test.go
@@ -0,0 +1,68 @@
+package agent
+
+import (
+ "context"
+ "encoding/json"
+ "strings"
+ "testing"
+
+ "github.com/abhijitkrm/cometcli/internal/config"
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+)
+
+// "is anyone jailed?" in general mode: the model switches to a node
+// profile itself and answers — the operator never types /one.
+func TestModelSwitchesToNodeItself(t *testing.T) {
+ var sawProfile string
+ validators := stubTool{name: "chain.validators", tier: toolkit.TierObserve, run: func(c *toolkit.Context, a toolkit.Args) (*toolkit.Result, error) {
+ if c.Profile != nil {
+ sawProfile = c.Profile.Name
+ }
+ return &toolkit.Result{Text: "val-3 UNBONDING JAILED"}, nil
+ }}
+ prov := &mockProvider{responses: []*Response{
+ {Calls: []Call{{ID: "c1", Name: "use_node", Args: json.RawMessage(`{"profile":"val01"}`)}}},
+ {Calls: []Call{{ID: "c2", Name: "chain__validators", Args: json.RawMessage(`{"status":"jailed"}`)}}},
+ {Text: "val-3 is jailed", Done: true},
+ }}
+ a := newTestAgent(t, prov, validators)
+ cfg := &config.Config{Profiles: map[string]*config.Profile{
+ "val01": {ChainID: "primium-1", Role: "validator"},
+ "val02": {ChainID: "primium-1", Role: "validator"},
+ }}
+ a.Ctx.Profile, a.Ctx.Cfg = nil, cfg // general mode
+ if a.Node() {
+ t.Fatal("should start in general mode")
+ }
+ out, err := a.Run(context.Background(), "is anyone jailed?")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if !a.Node() || a.Ctx.Profile.Name != "val01" || sawProfile != "val01" {
+ t.Fatalf("node=%v profile=%v tool saw %q", a.Node(), a.Ctx.Profile, sawProfile)
+ }
+ if !strings.Contains(out, "val-3 is jailed") {
+ t.Fatalf("answer %q", out)
+ }
+ // the first request (general mode) offered use_node and told the model to use it
+ first := prov.reqs[0]
+ offered := false
+ for _, d := range first.Tools {
+ offered = offered || d.Name == "use_node"
+ }
+ if !offered || !strings.Contains(first.System, "call use_node") || strings.Contains(first.System, "operator switches with /one") {
+ t.Fatalf("general prompt/tools don't steer to use_node")
+ }
+ // after the switch the node prompt and node tools were in play
+ if !strings.Contains(prov.reqs[1].System, "You operate ONE node") {
+ t.Fatal("system prompt not switched to node mode mid-turn")
+ }
+ // and the model can go back
+ a.execCall(context.Background(), Call{ID: "c3", Name: "use_node", Args: json.RawMessage(`{"profile":"off"}`)})
+ if a.Node() {
+ t.Fatal("off didn't return to general mode")
+ }
+ if m := a.execCall(context.Background(), Call{ID: "c4", Name: "use_node", Args: json.RawMessage(`{"profile":"nope"}`)}); !m.IsError {
+ t.Fatal("unknown profile accepted")
+ }
+}
diff --git a/internal/audit/audit_test.go b/internal/audit/audit_test.go
new file mode 100644
index 0000000..3a3f703
--- /dev/null
+++ b/internal/audit/audit_test.go
@@ -0,0 +1,103 @@
+package audit
+
+import (
+ "bufio"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "os"
+ "path/filepath"
+ "sync"
+ "testing"
+)
+
+func TestConcurrentSessionsKeepJSONLIntact(t *testing.T) {
+ t.Setenv("COMETCLI_HOME", t.TempDir())
+ lg, err := Open("val01")
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer lg.Close()
+ var wg sync.WaitGroup
+ for s := 0; s < 8; s++ {
+ wg.Add(1)
+ go func(s int) {
+ defer wg.Done()
+ l := lg.WithSession(fmt.Sprintf("sess-%d", s))
+ _ = l.Log(KindPrompt, "val01", map[string]any{"text": fmt.Sprintf("prompt %d", s)})
+ for i := 0; i < 50; i++ {
+ l.ToolSeen("val01", "node.status", "observe", map[string]any{"i": i}, map[string]any{"ok": true}, nil, "height 42")
+ }
+ l.Approval("val01", "broadcast tx", s%2 == 0)
+ l.Tool("val01", "val.unjail", "on-chain", nil, nil, errors.New("denied"))
+ }(s)
+ }
+ wg.Wait()
+
+ f, _ := os.Open(lg.Path())
+ defer f.Close()
+ sc := bufio.NewScanner(f)
+ n := 0
+ for sc.Scan() {
+ var e Event
+ if err := json.Unmarshal(sc.Bytes(), &e); err != nil {
+ t.Fatalf("line %d is not valid JSON (interleaved write?): %q", n+1, sc.Text())
+ }
+ n++
+ }
+ if n != 8*(1+50+2) {
+ t.Fatalf("got %d events, want %d", n, 8*53)
+ }
+ fi, _ := os.Stat(lg.Path())
+ di, _ := os.Stat(filepath.Dir(lg.Path()))
+ if fi.Mode().Perm() != 0o600 || di.Mode().Perm() != 0o700 {
+ t.Fatalf("permissions: file %v dir %v", fi.Mode().Perm(), di.Mode().Perm())
+ }
+
+ evs, err := ReadSession("sess-3")
+ if err != nil || len(evs) != 53 || evs[0].Kind != KindPrompt || evs[52].Detail["error"] != "denied" {
+ t.Fatalf("ReadSession: %d events err=%v", len(evs), err)
+ }
+ if evs[1].Detail["digest"] == nil || evs[1].Detail["seen"] != "height 42" {
+ t.Fatalf("tool event = %v", evs[1].Detail)
+ }
+ ss, err := Sessions(lg.Path())
+ if err != nil || len(ss) != 8 {
+ t.Fatalf("Sessions = %v %v", ss, err)
+ }
+ for _, s := range ss {
+ if s[1] == "" {
+ t.Fatalf("session %s has no first prompt", s[0])
+ }
+ }
+}
+
+func TestReadSessionAcrossDays(t *testing.T) {
+ home := t.TempDir()
+ t.Setenv("COMETCLI_HOME", home)
+ dir := filepath.Join(home, "audit")
+ os.MkdirAll(dir, 0o700)
+ line := func(kind, text string) string {
+ b, _ := json.Marshal(Event{Kind: Kind(kind), Session: "s1", Detail: map[string]any{"text": text}})
+ return string(b) + "\n"
+ }
+ os.WriteFile(filepath.Join(dir, "2026-10-05.jsonl"), []byte(line("prompt", "before midnight")+"not json\n"), 0o600)
+ os.WriteFile(filepath.Join(dir, "2026-10-06.jsonl"), []byte(line("llm", "after midnight")), 0o600)
+ evs, err := ReadSession("s1")
+ if err != nil || len(evs) != 2 || evs[0].Detail["text"] != "before midnight" || evs[1].Detail["text"] != "after midnight" {
+ t.Fatalf("events = %+v err=%v", evs, err)
+ }
+ files, _ := List()
+ if len(files) != 2 || filepath.Base(files[0]) != "2026-10-06.jsonl" {
+ t.Fatalf("List = %v", files)
+ }
+}
+
+func TestNilLoggerIsNoop(t *testing.T) {
+ var l *Logger
+ if l.Log(KindTool, "", nil) != nil || l.Path() != "" || l.Session() != "" || l.Close() != nil || l.WithSession("x") != nil {
+ t.Fatal("nil logger must be a no-op")
+ }
+ l.Shell("p", "ls", 0)
+ l.Tx("p", "simulate", nil)
+}
diff --git a/internal/cli/agent.go b/internal/cli/agent.go
index 61a83e7..4a09cda 100644
--- a/internal/cli/agent.go
+++ b/internal/cli/agent.go
@@ -13,7 +13,21 @@ func agentFlags(cmd *cobra.Command) {
cmd.Flags().StringSlice("autopilot", nil, "tiers that skip the confirm prompt (only local-change; on-chain never)")
cmd.Flags().Bool("no-stream", false, "disable token streaming")
cmd.Flags().Int("budget", 0, "max tool calls per turn (0 = unlimited)")
- cmd.Flags().Int("max-iter", 0, "max model iterations per turn (default 16)")
+ cmd.Flags().Int("max-iter", 0, "max model iterations per turn (default 50)")
+ sessionFlags(cmd)
+}
+
+// sessionFlags wires model and session options shared by every agent
+// front-end, bare `cometcli` included.
+func sessionFlags(cmd *cobra.Command) {
+ cmd.Flags().BoolP("continue", "c", false, "resume the most recent session started in this directory")
+ cmd.Flags().StringP("resume", "r", "", "resume a saved `id` (list them with: cometcli sessions)")
+ cmd.Flags().String("model", "", "model for this session (overrides the profile)")
+ cmd.Flags().String("effort", "", "reasoning depth: low | medium | high | xhigh | max")
+ cmd.Flags().Int("max-tokens", 0, "max output tokens per model round")
+ cmd.Flags().String("permission-mode", "", "approval posture: ops | accept-edits | readonly | bypass")
+ cmd.Flags().StringSlice("allowedTools", nil, `permission rules to allow, e.g. "bash(git status:*)","read"`)
+ cmd.Flags().StringSlice("disallowedTools", nil, `permission rules to deny, e.g. "bash(rm:*)","web_fetch"`)
}
// AgentCmd opens the agentic REPL. Implemented in internal/agent.
diff --git a/internal/cli/agent_impl.go b/internal/cli/agent_impl.go
index c4fb091..b5ff7fb 100644
--- a/internal/cli/agent_impl.go
+++ b/internal/cli/agent_impl.go
@@ -1,6 +1,9 @@
package cli
import (
+ "fmt"
+ "os"
+
"github.com/spf13/cobra"
"github.com/abhijitkrm/cometcli/internal/agent"
@@ -20,9 +23,14 @@ func runAgentImpl(cmd *cobra.Command, reg *toolkit.Registry, oneshot string) err
if err != nil {
return err
}
- if err := applyAgentFlags(cmd, a); err != nil {
+ defer a.Close()
+ note, err := applyAgentFlags(cmd, a)
+ if err != nil {
return err
}
+ if note != "" {
+ fmt.Fprintln(cmd.ErrOrStderr(), note)
+ }
if oneshot != "" {
pr := &agent.Printer{Out: c.Out}
a.OnEvent = pr.Handle
@@ -33,13 +41,19 @@ func runAgentImpl(cmd *cobra.Command, reg *toolkit.Registry, oneshot string) err
return repl.Run()
}
-// applyAgentFlags layers --mode/--safe/--autopilot/--budget/--max-iter/
-// --no-stream over the profile's agent defaults.
-func applyAgentFlags(cmd *cobra.Command, a *agent.Agent) error {
- if m, _ := cmd.Flags().GetString("mode"); m != "" {
+// applyAgentFlags layers the agent and session flags over the profile's
+// agent defaults, turns on session saving, and resumes a session when
+// asked (--continue / --resume). Flags a command doesn't define read as
+// unset. The returned note describes a resumed session.
+func applyAgentFlags(cmd *cobra.Command, a *agent.Agent) (string, error) {
+ m, _ := cmd.Flags().GetString("mode")
+ if pm, _ := cmd.Flags().GetString("permission-mode"); pm != "" {
+ m = pm
+ }
+ if m != "" {
mode, err := agent.ParseMode(m)
if err != nil {
- return err
+ return "", err
}
a.Policy.Mode = mode
}
@@ -49,7 +63,7 @@ func applyAgentFlags(cmd *cobra.Command, a *agent.Agent) error {
if tiers, _ := cmd.Flags().GetStringSlice("autopilot"); len(tiers) > 0 {
for _, t := range tiers {
if err := a.Policy.SetAutopilot(t, true); err != nil {
- return err
+ return "", err
}
}
}
@@ -66,5 +80,53 @@ func applyAgentFlags(cmd *cobra.Command, a *agent.Agent) error {
// hard-refuse anything that slips past filtering
a.Ctx.Approver = toolkit.DenyApprover
}
- return nil
+ if m, _ := cmd.Flags().GetString("model"); m != "" {
+ a.Model = m
+ }
+ for list, flag := range map[string]string{"allow": "allowedTools", "deny": "disallowedTools"} {
+ rules, _ := cmd.Flags().GetStringSlice(flag)
+ for _, r := range rules {
+ if err := a.Rules.Add(list, r); err != nil {
+ return "", fmt.Errorf("--%s: %w", flag, err)
+ }
+ }
+ }
+ if e, _ := cmd.Flags().GetString("effort"); e != "" {
+ if err := agent.ValidEffort(e); err != nil {
+ return "", err
+ }
+ a.Effort = e
+ }
+ if n, _ := cmd.Flags().GetInt("max-tokens"); n > 0 {
+ a.MaxTokens = n
+ }
+ a.Persist = true
+ return resumeSession(cmd, a)
+}
+
+// resumeSession loads the session named by --resume, or the latest one in
+// this directory for --continue.
+func resumeSession(cmd *cobra.Command, a *agent.Agent) (string, error) {
+ id, _ := cmd.Flags().GetString("resume")
+ cont, _ := cmd.Flags().GetBool("continue")
+ var sf *agent.SessionFile
+ var err error
+ switch {
+ case id != "":
+ sf, err = agent.LoadSession(id)
+ case cont:
+ cwd, _ := os.Getwd()
+ profile := ""
+ if a.Ctx.Profile != nil {
+ profile = a.Ctx.Profile.Name
+ }
+ sf, err = agent.LatestSession(cwd, profile)
+ default:
+ return "", nil
+ }
+ if err != nil {
+ return "", err
+ }
+ a.Restore(sf)
+ return fmt.Sprintf("resumed session %s — %s (%d messages)", sf.ID, sf.Title, len(sf.History)), nil
}
diff --git a/internal/cli/chat.go b/internal/cli/chat.go
new file mode 100644
index 0000000..91500a9
--- /dev/null
+++ b/internal/cli/chat.go
@@ -0,0 +1,353 @@
+package cli
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "os"
+ "strings"
+ "time"
+
+ "github.com/spf13/cobra"
+
+ "github.com/abhijitkrm/cometcli/internal/agent"
+ "github.com/abhijitkrm/cometcli/internal/audit"
+ "github.com/abhijitkrm/cometcli/internal/config"
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+ "github.com/abhijitkrm/cometcli/internal/tui/chatui"
+)
+
+// maxStdin bounds piped input; larger input keeps its head and tail.
+const maxStdin = 200_000
+
+// printFlags wires headless-mode flags (bare cometcli and `one`).
+func printFlags(cmd *cobra.Command) {
+ cmd.Flags().BoolP("print", "p", false, "answer once and exit — no TUI; for scripts, pipes and CI")
+ cmd.Flags().String("output-format", "text", "with -p: text | json | stream-json")
+ cmd.Flags().Bool("include-partial-messages", false, "with stream-json: also emit token deltas")
+ cmd.Flags().Bool("verbose", false, "with -p text: show tool calls and notices on stderr")
+ cmd.Flags().String("append-system-prompt", "", "extra instructions appended to the system prompt")
+ cmd.Flags().Int("max-turns", 0, "max model rounds per prompt (same as --max-iter)")
+}
+
+// oneCmd is node mode: `cometcli one [profile] [prompt]`.
+func oneCmd(reg *toolkit.Registry) *cobra.Command {
+ cmd := &cobra.Command{
+ Use: "one [profile] [prompt]",
+ Short: "Node mode — chat scoped to one node profile (validator tools, safety rules, live snapshot)",
+ Long: `Node mode scopes the session to one profile: the node tools (val, chain,
+tx, upgrade, …) join the general ones, the shell runs on the node's host
+(local or SSH), the validator safety rules apply, and a live snapshot
+of the node rides along. Without a profile name the active profile is
+used. Inside any session, /one and /one off switch modes.`,
+ Args: cobra.ArbitraryArgs,
+ ValidArgsFunction: func(cmd *cobra.Command, args []string, _ string) ([]string, cobra.ShellCompDirective) {
+ if len(args) > 0 {
+ return nil, cobra.ShellCompDirectiveNoFileComp
+ }
+ cfg, err := config.Load()
+ if err != nil {
+ return nil, cobra.ShellCompDirectiveNoFileComp
+ }
+ var names []string
+ for n := range cfg.Profiles {
+ names = append(names, n)
+ }
+ return names, cobra.ShellCompDirectiveNoFileComp
+ },
+ RunE: func(cmd *cobra.Command, args []string) error {
+ cfg, err := config.Load()
+ if err != nil {
+ return err
+ }
+ name := flagProfile
+ if len(args) > 0 {
+ if _, ok := cfg.Profiles[args[0]]; ok {
+ name, args = args[0], args[1:]
+ }
+ }
+ p, err := cfg.ActiveProfile(name)
+ if err != nil {
+ return fmt.Errorf("%w — `cometcli one `, or `cometcli init` to add one", err)
+ }
+ return runChat(cmd, reg, p, args)
+ },
+ }
+ agentFlags(cmd)
+ printFlags(cmd)
+ return cmd
+}
+
+// runChat is the shared entry for bare `cometcli` and `cometcli one`:
+// interactive TUI on a terminal, headless (-p) otherwise. p == nil is
+// general mode.
+func runChat(cmd *cobra.Command, reg *toolkit.Registry, p *config.Profile, args []string) error {
+ // `cometcli doctr` is a typo, not a prompt — checked only for a single
+ // word typed interactively, long enough that a near miss is meaningful
+ if p, _ := cmd.Flags().GetBool("print"); !p && len(args) == 1 && len(args[0]) >= 4 && !strings.ContainsAny(args[0], " \t?") {
+ cmd.Root().SuggestionsMinimumDistance = 1
+ if len(args[0]) >= 6 {
+ cmd.Root().SuggestionsMinimumDistance = 2
+ }
+ if sug := cmd.Root().SuggestionsFor(args[0]); len(sug) > 0 {
+ return fmt.Errorf("unknown command %q — did you mean %s? (quote a one-word prompt with a ?)", args[0], strings.Join(sug, ", "))
+ }
+ }
+ prompt := strings.TrimSpace(strings.Join(args, " "))
+ piped, err := readPipedStdin(cmd.InOrStdin())
+ if err != nil {
+ return err
+ }
+ if piped != "" {
+ if prompt == "" {
+ prompt = piped
+ } else {
+ prompt += "\n\n\n" + piped + "\n"
+ }
+ }
+ printMode, _ := cmd.Flags().GetBool("print")
+ headless := printMode || piped != "" || (!isTerminal(cmd.OutOrStdout()) && prompt != "")
+ if !headless && !isTerminal(cmd.InOrStdin()) {
+ if prompt == "" {
+ return cmd.Help()
+ }
+ headless = true
+ }
+
+ c, err := chatCtx(cmd, p, headless)
+ if err != nil {
+ return err
+ }
+ defer c.Close()
+ if c.Audit != nil {
+ defer c.Audit.Close()
+ }
+ if headless {
+ if prompt == "" {
+ return fmt.Errorf("no prompt — pass one as an argument or pipe input in")
+ }
+ return runPrint(cmd, c, reg, prompt)
+ }
+ a, err := agent.New(c, reg)
+ if err != nil {
+ return err
+ }
+ defer a.Close()
+ note, err := configureChatAgent(cmd, a)
+ if err != nil {
+ return err
+ }
+ var notes []string
+ if note != "" {
+ notes = append(notes, note)
+ }
+ return chatui.Run(chatui.Options{Agent: a, Prompt: prompt, Notes: notes})
+}
+
+// chatCtx builds the tool context for a chat session. Headless sessions
+// never read approvals from stdin (it may be the piped input): anything
+// needing approval is denied unless rules or the mode allow it.
+func chatCtx(cmd *cobra.Command, p *config.Profile, headless bool) (*toolkit.Context, error) {
+ cfg, err := config.Load()
+ if err != nil {
+ return nil, err
+ }
+ name := "general"
+ if p != nil {
+ name = p.Name
+ }
+ lg, _ := audit.Open(name)
+ c := &toolkit.Context{
+ Context: cmd.Context(), Profile: p, Cfg: cfg, Out: cmd.OutOrStdout(), Audit: lg,
+ }
+ c.Approver, c.Chooser = StdinPrompts(cmd.InOrStdin())
+ c.Secret = TTYSecret
+ if headless {
+ // never read prompts from stdin (it may be the piped input); a
+ // secret still comes from the terminal or the env var
+ c.Out = cmd.ErrOrStderr()
+ c.Approver, c.Chooser = headlessApprover, nil
+ }
+ if flagYes {
+ c.AutoApproveBelow = toolkit.TierLocalChange
+ }
+ return c, nil
+}
+
+func headlessApprover(_ *toolkit.Context, prompt string, tier toolkit.Tier, _ map[string]any) (bool, error) {
+ return false, fmt.Errorf("not approved: %s needs approval and -p has no one to ask — allow it with --allowedTools or --permission-mode, or run interactively", tier)
+}
+
+// configureChatAgent applies the agent flags plus the chat-only ones.
+func configureChatAgent(cmd *cobra.Command, a *agent.Agent) (string, error) {
+ if s, _ := cmd.Flags().GetString("append-system-prompt"); s != "" {
+ a.AppendSystem = s
+ }
+ if n, _ := cmd.Flags().GetInt("max-turns"); n > 0 {
+ a.MaxIter = n
+ }
+ return applyAgentFlags(cmd, a)
+}
+
+// readPipedStdin returns piped or redirected input. Terminals, /dev/null
+// and other devices are skipped, so an idle stdin never blocks.
+func readPipedStdin(r io.Reader) (string, error) {
+ f, ok := r.(*os.File)
+ if !ok {
+ return "", nil
+ }
+ fi, err := f.Stat()
+ if err != nil || fi.Mode()&(os.ModeNamedPipe) == 0 && !fi.Mode().IsRegular() {
+ return "", nil
+ }
+ b, err := io.ReadAll(io.LimitReader(f, 8*maxStdin))
+ if err != nil {
+ return "", err
+ }
+ s := strings.TrimSpace(string(b))
+ if len(s) > maxStdin {
+ head, tail := s[:maxStdin*2/3], s[len(s)-maxStdin/3:]
+ s = head + fmt.Sprintf("\n…[%d bytes of input omitted]…\n", len(s)-len(head)-len(tail)) + tail
+ }
+ return s, nil
+}
+
+// runPrint answers one prompt headlessly in the requested format.
+func runPrint(cmd *cobra.Command, c *toolkit.Context, reg *toolkit.Registry, prompt string) error {
+ format, _ := cmd.Flags().GetString("output-format")
+ switch format {
+ case "text", "json", "stream-json":
+ default:
+ return fmt.Errorf("--output-format %q: want text, json or stream-json", format)
+ }
+ a, err := agent.New(c, reg)
+ if err != nil {
+ return err
+ }
+ defer a.Close()
+ note, err := configureChatAgent(cmd, a)
+ if err != nil {
+ return err
+ }
+ if note != "" {
+ fmt.Fprintln(cmd.ErrOrStderr(), note)
+ }
+ out := cmd.OutOrStdout()
+ enc := json.NewEncoder(out)
+ partial, _ := cmd.Flags().GetBool("include-partial-messages")
+ verbose, _ := cmd.Flags().GetBool("verbose")
+
+ switch format {
+ case "stream-json":
+ _ = enc.Encode(map[string]any{
+ "type": "system", "subtype": "init", "session_id": a.ID(), "cwd": a.WorkRoot,
+ "provider": a.Provider.Name(), "model": a.Model, "mode": scopeName(a),
+ "permission_mode": a.Policy.Mode, "tools": a.ToolNames(),
+ })
+ a.OnEvent = func(e agent.Event) {
+ if line := streamEvent(e, partial); line != nil {
+ _ = enc.Encode(line)
+ }
+ }
+ case "text":
+ if verbose {
+ pr := &agent.Printer{Out: cmd.ErrOrStderr()}
+ a.OnEvent = func(e agent.Event) {
+ if e.Kind != agent.EvDelta && e.Kind != agent.EvText && e.Kind != agent.EvThinking {
+ pr.Handle(e)
+ }
+ }
+ }
+ }
+ // slash commands work headlessly too: `cometcli -p "/vote yes 7"`
+ if strings.HasPrefix(prompt, "/") {
+ cr, err := agent.RunCommand(a, c, reg, prompt)
+ if err != nil {
+ if errors.Is(err, agent.ErrUnknownCommand) {
+ return fmt.Errorf("unknown command %s", strings.Fields(prompt)[0])
+ }
+ return err
+ }
+ if cr.Job != nil {
+ txt, err := cr.Job(cmd.Context())
+ if err != nil {
+ return err
+ }
+ cr.Text = txt
+ }
+ if cr.Prompt == "" {
+ if cr.Text != "" {
+ fmt.Fprintln(out, cr.Text)
+ }
+ return nil
+ }
+ if verbose && cr.Text != "" {
+ fmt.Fprintln(cmd.ErrOrStderr(), cr.Text)
+ }
+ prompt = cr.Prompt
+ }
+
+ start := time.Now()
+ res, runErr := a.Run(cmd.Context(), prompt)
+ if format == "text" {
+ if res != "" {
+ fmt.Fprintln(out, res)
+ }
+ return runErr
+ }
+ usage, _ := a.Usage()
+ result := map[string]any{
+ "type": "result", "subtype": "success", "is_error": runErr != nil, "result": res,
+ "session_id": a.ID(), "num_turns": a.Rounds(), "duration_ms": time.Since(start).Milliseconds(),
+ "provider": a.Provider.Name(), "model": a.Model, "mode": scopeName(a),
+ "usage": map[string]int{"input_tokens": usage.Input, "output_tokens": usage.Output, "cache_read_input_tokens": usage.CacheRead},
+ }
+ if runErr != nil {
+ result["subtype"] = "error"
+ result["error"] = runErr.Error()
+ if errors.Is(runErr, cmd.Context().Err()) && cmd.Context().Err() != nil {
+ result["subtype"] = "cancelled"
+ }
+ }
+ _ = enc.Encode(result)
+ return runErr
+}
+
+func scopeName(a *agent.Agent) string {
+ if a.Node() {
+ return "node:" + a.Ctx.Profile.Name
+ }
+ return "general"
+}
+
+// streamEvent maps an agent event to a stream-json line (nil = skip).
+func streamEvent(e agent.Event, partial bool) map[string]any {
+ switch e.Kind {
+ case agent.EvDelta:
+ if partial {
+ return map[string]any{"type": "stream_event", "delta": e.Text}
+ }
+ case agent.EvText:
+ return map[string]any{"type": "assistant", "text": e.Text}
+ case agent.EvThinking:
+ if partial {
+ return map[string]any{"type": "thinking", "delta": e.Text}
+ }
+ case agent.EvToolStart:
+ return map[string]any{"type": "tool_use", "tool": e.Tool, "tier": e.Tier, "input": e.Args}
+ case agent.EvToolResult:
+ m := map[string]any{"type": "tool_result", "tool": e.Tool, "is_error": e.Err != ""}
+ if e.Err != "" {
+ m["output"] = e.Err
+ } else {
+ m["output"] = e.Text
+ }
+ return m
+ case agent.EvNotice:
+ return map[string]any{"type": "notice", "text": e.Text}
+ case agent.EvTodos:
+ return map[string]any{"type": "todos", "todos": e.Todos}
+ }
+ return nil
+}
diff --git a/internal/cli/chat_test.go b/internal/cli/chat_test.go
new file mode 100644
index 0000000..cfa9c48
--- /dev/null
+++ b/internal/cli/chat_test.go
@@ -0,0 +1,49 @@
+package cli
+
+import (
+ "os"
+ "strings"
+ "testing"
+
+ "github.com/abhijitkrm/cometcli/internal/agent"
+)
+
+func TestReadPipedStdin(t *testing.T) {
+ r, w, err := os.Pipe()
+ if err != nil {
+ t.Fatal(err)
+ }
+ go func() {
+ w.WriteString(" panic: boom\n")
+ w.Close()
+ }()
+ got, err := readPipedStdin(r)
+ if err != nil || got != "panic: boom" {
+ t.Fatalf("pipe = %q %v", got, err)
+ }
+ null, _ := os.Open(os.DevNull)
+ defer null.Close()
+ if got, _ := readPipedStdin(null); got != "" {
+ t.Fatalf("/dev/null read %q", got)
+ }
+ big, _ := os.CreateTemp(t.TempDir(), "in")
+ big.WriteString(strings.Repeat("x", maxStdin) + "TAIL")
+ big.Seek(0, 0)
+ got, _ = readPipedStdin(big)
+ if len(got) > maxStdin+100 || !strings.HasSuffix(got, "TAIL") || !strings.Contains(got, "omitted") {
+ t.Fatalf("large input: len=%d", len(got))
+ }
+}
+
+func TestStreamEvent(t *testing.T) {
+ if streamEvent(agent.Event{Kind: agent.EvDelta, Text: "x"}, false) != nil {
+ t.Fatal("delta emitted without --include-partial-messages")
+ }
+ m := streamEvent(agent.Event{Kind: agent.EvToolResult, Tool: "bash", Err: "denied"}, false)
+ if m["type"] != "tool_result" || m["is_error"] != true || m["output"] != "denied" {
+ t.Fatalf("tool_result = %v", m)
+ }
+ if m := streamEvent(agent.Event{Kind: agent.EvText, Text: "hi"}, false); m["type"] != "assistant" {
+ t.Fatalf("assistant = %v", m)
+ }
+}
diff --git a/internal/cli/configcmd.go b/internal/cli/configcmd.go
new file mode 100644
index 0000000..4ac3df7
--- /dev/null
+++ b/internal/cli/configcmd.go
@@ -0,0 +1,181 @@
+package cli
+
+import (
+ "bufio"
+ "fmt"
+ "os"
+ "sort"
+ "strconv"
+ "strings"
+
+ "github.com/spf13/cobra"
+ "golang.org/x/term"
+ "gopkg.in/yaml.v3"
+
+ "github.com/abhijitkrm/cometcli/internal/agent"
+ "github.com/abhijitkrm/cometcli/internal/config"
+)
+
+// agentKeys are the settable global agent keys and their setters.
+var agentKeys = map[string]func(*config.AgentConf, string) error{
+ "provider": func(a *config.AgentConf, v string) error { a.Provider = v; return nil },
+ "model": func(a *config.AgentConf, v string) error { a.Model = v; return nil },
+ "base_url": func(a *config.AgentConf, v string) error { a.BaseURL = v; return nil },
+ "api_key_env": func(a *config.AgentConf, v string) error { a.APIKeyEnv = v; return nil },
+ "effort": func(a *config.AgentConf, v string) error {
+ if err := agent.ValidEffort(v); err != nil {
+ return err
+ }
+ a.Effort = v
+ return nil
+ },
+ "mode": func(a *config.AgentConf, v string) error {
+ if _, err := agent.ParseMode(v); err != nil {
+ return err
+ }
+ a.Mode = v
+ return nil
+ },
+ "tools": func(a *config.AgentConf, v string) error {
+ if v != "" && v != "all" && v != "auto" {
+ return fmt.Errorf("tools: want all or auto")
+ }
+ a.Tools = v
+ return nil
+ },
+ "max_tokens": intKey(func(a *config.AgentConf) *int { return &a.MaxTokens }),
+ "max_turns": intKey(func(a *config.AgentConf) *int { return &a.MaxTurns }),
+ "context_window": intKey(func(a *config.AgentConf) *int { return &a.ContextWindow }),
+ "compact_at": intKey(func(a *config.AgentConf) *int { return &a.CompactAt }),
+ "no_stream": func(a *config.AgentConf, v string) error {
+ b, err := strconv.ParseBool(v)
+ a.NoStream = b
+ return err
+ },
+}
+
+func intKey(f func(*config.AgentConf) *int) func(*config.AgentConf, string) error {
+ return func(a *config.AgentConf, v string) error {
+ n, err := strconv.Atoi(v)
+ if err != nil {
+ return fmt.Errorf("want a number, got %q", v)
+ }
+ *f(a) = n
+ return nil
+ }
+}
+
+// configCmd reads and writes the global agent settings in config.yaml —
+// the defaults general mode uses and profiles override.
+func configCmd() *cobra.Command {
+ cmd := &cobra.Command{Use: "config", Short: "Show or change global agent settings (agent.provider, agent.model, …)"}
+ keys := make([]string, 0, len(agentKeys))
+ for k := range agentKeys {
+ keys = append(keys, "agent."+k)
+ }
+ sort.Strings(keys)
+ cmd.AddCommand(&cobra.Command{
+ Use: "set-key ",
+ Short: "Save an API key (read from stdin) to ~/.cometcli/credentials (0600), e.g. set-key OPENROUTER_API_KEY",
+ Long: "Reads the key from stdin (hidden when it's a terminal) and saves it to ~/.cometcli/credentials, mode 0600. " +
+ "Providers use it when the environment variable isn't set. An empty input removes the key.",
+ Args: cobra.ExactArgs(1),
+ RunE: func(cmd *cobra.Command, args []string) error {
+ v, err := readSecretStdin(cmd, args[0])
+ if err != nil {
+ return err
+ }
+ if err := config.SetCredential(args[0], strings.TrimSpace(v)); err != nil {
+ return err
+ }
+ p, _ := config.Path("credentials")
+ if strings.TrimSpace(v) == "" {
+ fmt.Fprintf(cmd.OutOrStdout(), "removed %s from %s\n", args[0], p)
+ } else {
+ fmt.Fprintf(cmd.OutOrStdout(), "saved %s to %s (0600)\n", args[0], p)
+ }
+ return nil
+ },
+ })
+ cmd.AddCommand(&cobra.Command{
+ Use: "show",
+ Short: "Print the global agent section",
+ RunE: func(cmd *cobra.Command, _ []string) error {
+ cfg, err := config.Load()
+ if err != nil {
+ return err
+ }
+ b, _ := yaml.Marshal(map[string]any{"agent": cfg.Agent})
+ fmt.Fprint(cmd.OutOrStdout(), string(b))
+ if cfg.Agent.Provider == "" {
+ fmt.Fprintln(cmd.OutOrStdout(), "# no global provider — general mode uses the active profile's agent settings")
+ }
+ return nil
+ },
+ }, &cobra.Command{
+ Use: "set ",
+ Short: "Set a key: " + strings.Join(keys, ", "),
+ Args: cobra.ExactArgs(2),
+ ValidArgs: keys,
+ RunE: func(cmd *cobra.Command, args []string) error {
+ k := strings.TrimPrefix(args[0], "agent.")
+ set, ok := agentKeys[k]
+ if !ok {
+ return fmt.Errorf("unknown key %q — one of %s", args[0], strings.Join(keys, ", "))
+ }
+ cfg, err := config.Load()
+ if err != nil {
+ return err
+ }
+ if err := set(&cfg.Agent, args[1]); err != nil {
+ return err
+ }
+ if err := cfg.Save(); err != nil {
+ return err
+ }
+ fmt.Fprintf(cmd.OutOrStdout(), "agent.%s = %s\n", k, args[1])
+ return nil
+ },
+ }, &cobra.Command{
+ Use: "unset ",
+ Short: "Clear a key",
+ Args: cobra.ExactArgs(1),
+ RunE: func(cmd *cobra.Command, args []string) error {
+ k := strings.TrimPrefix(args[0], "agent.")
+ set, ok := agentKeys[k]
+ if !ok {
+ return fmt.Errorf("unknown key %q", args[0])
+ }
+ cfg, err := config.Load()
+ if err != nil {
+ return err
+ }
+ zero := ""
+ if strings.HasPrefix(k, "max_") || k == "context_window" || k == "compact_at" {
+ zero = "0"
+ } else if k == "no_stream" {
+ zero = "false"
+ }
+ if err := set(&cfg.Agent, zero); err != nil {
+ return err
+ }
+ return cfg.Save()
+ },
+ })
+ return cmd
+}
+
+// readSecretStdin reads one line from stdin, without echo on a terminal.
+func readSecretStdin(cmd *cobra.Command, name string) (string, error) {
+ if fd := int(os.Stdin.Fd()); term.IsTerminal(fd) {
+ fmt.Fprintf(cmd.ErrOrStderr(), "%s: ", name)
+ b, err := term.ReadPassword(fd)
+ fmt.Fprintln(cmd.ErrOrStderr())
+ return string(b), err
+ }
+ line, err := bufio.NewReader(os.Stdin).ReadString('\n')
+ if err != nil && line == "" {
+ return "", fmt.Errorf("no key on stdin")
+ }
+ return line, nil
+}
diff --git a/internal/cli/e2e_test.go b/internal/cli/e2e_test.go
new file mode 100644
index 0000000..b034fc7
--- /dev/null
+++ b/internal/cli/e2e_test.go
@@ -0,0 +1,275 @@
+package cli
+
+import (
+ "bytes"
+ "encoding/json"
+ "fmt"
+ "net/http"
+ "net/http/httptest"
+ "os"
+ "path/filepath"
+ "strings"
+ "sync"
+ "testing"
+
+ "github.com/spf13/cobra"
+
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+ "github.com/abhijitkrm/cometcli/internal/tools"
+)
+
+// fakeLLM is a scripted OpenAI-compatible chat-completions server. It
+// answers by looking at the last message: user prompts are matched to a
+// script entry (which may call a tool first), tool results get a final
+// answer quoting the tool output.
+type fakeLLM struct {
+ mu sync.Mutex
+ requests []map[string]any
+ script map[string]string // prompt substring → shell command to run first
+}
+
+func (f *fakeLLM) handler(w http.ResponseWriter, r *http.Request) {
+ var req map[string]any
+ _ = json.NewDecoder(r.Body).Decode(&req)
+ f.mu.Lock()
+ f.requests = append(f.requests, req)
+ f.mu.Unlock()
+ msgs, _ := req["messages"].([]any)
+ last, _ := msgs[len(msgs)-1].(map[string]any)
+ var text string
+ var call map[string]any
+ switch last["role"] {
+ case "tool":
+ text = "Result: " + strings.SplitN(fmt.Sprint(last["content"]), "\n", 2)[0]
+ default:
+ prompt := fmt.Sprint(last["content"])
+ text = "answer to: " + strings.SplitN(prompt, "\n", 2)[0]
+ for k, cmd := range f.script {
+ if strings.Contains(prompt, k) {
+ args, _ := json.Marshal(map[string]any{"command": cmd})
+ call = map[string]any{"index": 0, "id": "call_1", "type": "function",
+ "function": map[string]any{"name": "bash", "arguments": string(args)}}
+ }
+ }
+ }
+ w.Header().Set("content-type", "text/event-stream")
+ send := func(v any) { b, _ := json.Marshal(v); fmt.Fprintf(w, "data: %s\n\n", b) }
+ if call != nil {
+ send(map[string]any{"choices": []any{map[string]any{"delta": map[string]any{"tool_calls": []any{call}}}}})
+ send(map[string]any{"choices": []any{map[string]any{"delta": map[string]any{}, "finish_reason": "tool_calls"}},
+ "usage": map[string]any{"prompt_tokens": 500, "completion_tokens": 20}})
+ } else {
+ send(map[string]any{"choices": []any{map[string]any{"delta": map[string]any{"content": text}}}})
+ send(map[string]any{"choices": []any{map[string]any{"delta": map[string]any{}, "finish_reason": "stop"}},
+ "usage": map[string]any{"prompt_tokens": 600, "completion_tokens": 10}})
+ }
+ fmt.Fprint(w, "data: [DONE]\n\n")
+}
+
+func (f *fakeLLM) count() int {
+ f.mu.Lock()
+ defer f.mu.Unlock()
+ return len(f.requests)
+}
+
+func (f *fakeLLM) last() map[string]any {
+ f.mu.Lock()
+ defer f.mu.Unlock()
+ return f.requests[len(f.requests)-1]
+}
+
+type env struct {
+ t *testing.T
+ llm *fakeLLM
+ home string
+ work string
+ reg *toolkit.Registry
+}
+
+func newEnv(t *testing.T) *env {
+ t.Helper()
+ f := &fakeLLM{script: map[string]string{}}
+ srv := httptest.NewServer(http.HandlerFunc(f.handler))
+ t.Cleanup(srv.Close)
+ home, work := t.TempDir(), t.TempDir()
+ t.Setenv("HOME", home)
+ t.Setenv("COMETCLI_HOME", filepath.Join(home, ".cometcli"))
+ t.Setenv("OLLAMA_API_KEY", "x")
+ os.MkdirAll(filepath.Join(home, ".cometcli"), 0o700)
+ cfg := fmt.Sprintf("agent:\n provider: openai-compat\n base_url: %s\n model: fake-1\nprofiles: {}\n", srv.URL)
+ os.WriteFile(filepath.Join(home, ".cometcli", "config.yaml"), []byte(cfg), 0o600)
+ wd, _ := os.Getwd()
+ t.Cleanup(func() { os.Chdir(wd) })
+ os.Chdir(work)
+ reg := toolkit.NewRegistry()
+ tools.RegisterAll(reg)
+ return &env{t: t, llm: f, home: home, work: work, reg: reg}
+}
+
+// run executes the CLI in-process and returns stdout, stderr and error.
+func (e *env) run(args ...string) (string, string, error) {
+ e.t.Helper()
+ root := NewRoot(e.reg, []*cobra.Command{DoctorCmd(e.reg), AskCmd(e.reg), AgentCmd(e.reg), UICmd(e.reg), MCPCmd(e.reg), ServeCmd(e.reg)})
+ var out, errb bytes.Buffer
+ root.SetOut(&out)
+ root.SetErr(&errb)
+ root.SetIn(strings.NewReader(""))
+ root.SetArgs(args)
+ err := root.Execute()
+ return out.String(), errb.String(), err
+}
+
+func TestE2EPrintTextAndJSON(t *testing.T) {
+ e := newEnv(t)
+ e.llm.script["disk"] = "echo disk-ok"
+ out, _, err := e.run("-p", "how is the disk?")
+ if err != nil || strings.TrimSpace(out) != "Result: disk-ok" {
+ t.Fatalf("text: out=%q err=%v", out, err)
+ }
+ // general mode: only general tools, general prompt
+ req := e.llm.last()
+ names := toolNamesOf(req)
+ if !strings.Contains(names, "bash") || strings.Contains(names, "node__status") {
+ t.Fatalf("tools = %s", names)
+ }
+ sys := fmt.Sprint(req["messages"].([]any)[0].(map[string]any)["content"])
+ if !strings.Contains(sys, "working in the operator's terminal") {
+ t.Fatalf("system prompt: %.200s", sys)
+ }
+
+ out, _, err = e.run("-p", "--output-format", "json", "hello")
+ var res map[string]any
+ if err != nil || json.Unmarshal([]byte(out), &res) != nil {
+ t.Fatalf("json: %q %v", out, err)
+ }
+ if res["type"] != "result" || res["subtype"] != "success" || res["result"] != "answer to: hello" || res["mode"] != "general" {
+ t.Fatalf("json result = %v", res)
+ }
+ if u := res["usage"].(map[string]any); u["input_tokens"].(float64) != 600 {
+ t.Fatalf("usage = %v", u)
+ }
+}
+
+func TestE2EStreamJSONAndHeadlessDenial(t *testing.T) {
+ e := newEnv(t)
+ target := filepath.Join(e.work, "made.txt")
+ e.llm.script["create"] = "touch " + target
+ out, _, err := e.run("-p", "--output-format", "stream-json", "create the file")
+ if err != nil {
+ t.Fatal(err)
+ }
+ var types []string
+ var denied bool
+ for _, l := range strings.Split(strings.TrimSpace(out), "\n") {
+ var ev map[string]any
+ if json.Unmarshal([]byte(l), &ev) != nil {
+ t.Fatalf("not JSON: %q", l)
+ }
+ types = append(types, fmt.Sprint(ev["type"]))
+ if ev["type"] == "tool_result" && ev["is_error"] == true && strings.Contains(fmt.Sprint(ev["output"]), "-p has no one to ask") {
+ denied = true
+ }
+ }
+ if got := strings.Join(types, ","); got != "system,tool_use,tool_result,assistant,result" {
+ t.Fatalf("event types = %s", got)
+ }
+ if !denied {
+ t.Fatalf("headless write wasn't denied:\n%s", out)
+ }
+ if _, err := os.Stat(target); err == nil {
+ t.Fatal("headless run created a file without approval")
+ }
+ // allowed explicitly → runs
+ if _, _, err := e.run("-p", "--allowedTools", "bash(touch:*)", "create the file"); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Stat(target); err != nil {
+ t.Fatal("--allowedTools didn't allow the command")
+ }
+}
+
+func TestE2ESessionsContinueAndResume(t *testing.T) {
+ e := newEnv(t)
+ if _, _, err := e.run("-p", "first question"); err != nil {
+ t.Fatal(err)
+ }
+ out, _, err := e.run("sessions")
+ if err != nil || !strings.Contains(out, "first question") {
+ t.Fatalf("sessions: %q %v", out, err)
+ }
+ id := strings.Fields(strings.Split(out, "\n")[1])[0]
+ if _, _, err := e.run("-p", "-c", "second question"); err != nil {
+ t.Fatal(err)
+ }
+ msgs := e.llm.last()["messages"].([]any)
+ if len(msgs) != 4 { // system, first user, first answer, second user
+ t.Fatalf("continue didn't carry the conversation: %d messages", len(msgs))
+ }
+ if _, stderr, err := e.run("-p", "-r", id[:10], "third"); err != nil || !strings.Contains(stderr, "resumed session") {
+ t.Fatalf("resume: %q %v", stderr, err)
+ }
+ if _, _, err := e.run("-p", "-r", "no-such-id", "x"); err == nil {
+ t.Fatal("bad resume id accepted")
+ }
+}
+
+func TestE2EPipedStdinSlashCommandsConfigAndTypos(t *testing.T) {
+ e := newEnv(t)
+ // piped input: a real file as stdin
+ in := filepath.Join(e.work, "log.txt")
+ os.WriteFile(in, []byte("panic: out of memory\n"), 0o644)
+ f, _ := os.Open(in)
+ defer f.Close()
+ root := NewRoot(e.reg, nil)
+ var out bytes.Buffer
+ root.SetOut(&out)
+ root.SetErr(&bytes.Buffer{})
+ root.SetIn(f)
+ root.SetArgs([]string{"-p", "why did it crash?"})
+ if err := root.Execute(); err != nil {
+ t.Fatal(err)
+ }
+ user := fmt.Sprint(e.llm.last()["messages"].([]any)[1].(map[string]any)["content"])
+ if !strings.Contains(user, "\npanic: out of memory\n") {
+ t.Fatalf("stdin not attached: %q", user)
+ }
+
+ // a slash command headlessly, no model call
+ before := e.llm.count()
+ if out, _, err := e.run("-p", "/permissions"); err != nil || !strings.Contains(out, "transactions ask") || e.llm.count() != before {
+ t.Fatalf("/permissions: %q %v", out, err)
+ }
+ // a custom command expands into a prompt
+ os.MkdirAll(filepath.Join(e.work, ".cometcli", "commands"), 0o755)
+ os.WriteFile(filepath.Join(e.work, ".cometcli", "commands", "greet.md"), []byte("Say hi to $1"), 0o644)
+ if out, _, err := e.run("-p", "/greet operator"); err != nil || strings.TrimSpace(out) != "answer to: Say hi to operator" {
+ t.Fatalf("/greet: %q %v", out, err)
+ }
+
+ if out, _, err := e.run("config", "set", "agent.effort", "low"); err != nil || !strings.Contains(out, "agent.effort = low") {
+ t.Fatalf("config set: %q %v", out, err)
+ }
+ if _, _, err := e.run("-p", "hi"); err != nil {
+ t.Fatal(err)
+ }
+ if e.llm.last()["reasoning_effort"] != nil {
+ // openai-compat (local servers) never gets reasoning_effort
+ t.Fatalf("reasoning_effort sent to an openai-compat server")
+ }
+ if _, _, err := e.run("doctr"); err == nil || !strings.Contains(err.Error(), "did you mean doctor") {
+ t.Fatalf("typo: %v", err)
+ }
+ if _, _, err := e.run("one"); err == nil || !strings.Contains(err.Error(), "init") {
+ t.Fatalf("one without a profile: %v", err)
+ }
+}
+
+func toolNamesOf(req map[string]any) string {
+ var n []string
+ tl, _ := req["tools"].([]any)
+ for _, t := range tl {
+ fn := t.(map[string]any)["function"].(map[string]any)
+ n = append(n, fmt.Sprint(fn["name"]))
+ }
+ return strings.Join(n, ",")
+}
diff --git a/internal/cli/mcp.go b/internal/cli/mcp.go
index 09c3207..104e18b 100644
--- a/internal/cli/mcp.go
+++ b/internal/cli/mcp.go
@@ -15,9 +15,13 @@ import (
// Observe/diagnose tools run freely; anything that would prompt (host
// mutations, on-chain txs) is denied — stdio has no human to approve.
func MCPCmd(reg *toolkit.Registry) *cobra.Command {
- return &cobra.Command{
+ cmd := &cobra.Command{
Use: "mcp",
- Short: "Serve cometcli tools over MCP (stdio) for external agents",
+ Short: "Serve cometcli tools over MCP (stdio); add/list/remove MCP servers the agent uses",
+ Long: `With no subcommand, serves cometcli's node tools over MCP on stdio for
+external agents. The subcommands manage MCP servers that cometcli's own
+agent connects to (their tools appear as mcp____).`,
+ Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
c, err := NewCtx(cmd, true)
if err != nil {
@@ -32,4 +36,6 @@ func MCPCmd(reg *toolkit.Registry) *cobra.Command {
}).Serve(cmd.Context(), cmd.InOrStdin(), cmd.OutOrStdout())
},
}
+ cmd.AddCommand(mcpAddCmd(), mcpListCmd(), mcpRemoveCmd())
+ return cmd
}
diff --git a/internal/cli/mcpservers.go b/internal/cli/mcpservers.go
new file mode 100644
index 0000000..a87d017
--- /dev/null
+++ b/internal/cli/mcpservers.go
@@ -0,0 +1,256 @@
+package cli
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "os"
+ "path/filepath"
+ "sort"
+ "strings"
+ "text/tabwriter"
+ "time"
+
+ "github.com/spf13/cobra"
+
+ "github.com/abhijitkrm/cometcli/internal/mcpclient"
+ "github.com/abhijitkrm/cometcli/internal/settings"
+)
+
+// mcpScopePath returns the file a scope's servers live in: user settings,
+// the project's .mcp.json (shared with Claude Code), or local settings.
+func mcpScopePath(scope string) (string, error) {
+ switch scope {
+ case "user", "":
+ return settings.UserPath()
+ case "project", "local":
+ cwd, _ := os.Getwd()
+ root := settings.FindRoot(cwd)
+ if root == "" {
+ root = cwd
+ }
+ if scope == "project" {
+ return filepath.Join(root, ".mcp.json"), nil
+ }
+ return filepath.Join(root, ".cometcli", "settings.local.json"), nil
+ }
+ return "", fmt.Errorf("--scope: want user, project or local")
+}
+
+// editServers mutates the mcpServers map of a scope's file.
+func editServers(scope string, fn func(map[string]settings.MCPServer) error) (string, error) {
+ p, err := mcpScopePath(scope)
+ if err != nil {
+ return "", err
+ }
+ if scope != "project" {
+ return p, settings.EditFile(p, func(f *settings.File) error {
+ if f.MCPServers == nil {
+ f.MCPServers = map[string]settings.MCPServer{}
+ }
+ return fn(f.MCPServers)
+ })
+ }
+ // .mcp.json: {"mcpServers": {...}}, other keys preserved
+ doc := map[string]any{}
+ if raw, err := os.ReadFile(p); err == nil {
+ if err := json.Unmarshal(raw, &doc); err != nil {
+ return p, fmt.Errorf("%s: %w", p, err)
+ }
+ } else if !errors.Is(err, os.ErrNotExist) {
+ return p, err
+ }
+ servers := map[string]settings.MCPServer{}
+ if raw, ok := doc["mcpServers"]; ok {
+ b, _ := json.Marshal(raw)
+ _ = json.Unmarshal(b, &servers)
+ }
+ if err := fn(servers); err != nil {
+ return p, err
+ }
+ doc["mcpServers"] = servers
+ out, _ := json.MarshalIndent(doc, "", " ")
+ return p, os.WriteFile(p, append(out, '\n'), 0o644)
+}
+
+func mcpAddCmd() *cobra.Command {
+ var scope, transport string
+ var env, headers []string
+ cmd := &cobra.Command{
+ Use: "add [--transport http ] | [-- [args…]]",
+ Short: "Add an MCP server for the agent",
+ Example: ` cometcli mcp add grafana -- npx -y @grafana/mcp-grafana
+ cometcli mcp add k8s -e KUBECONFIG=~/.kube/config -- kubectl-mcp-server
+ cometcli mcp add pagerduty --transport http https://mcp.example.com/mcp --header "Authorization: Bearer ${PD_TOKEN}"
+ cometcli mcp add tools --scope project -- ./scripts/mcp.sh # shared via .mcp.json`,
+ Args: cobra.MinimumNArgs(2),
+ RunE: func(cmd *cobra.Command, args []string) error {
+ name := args[0]
+ srv := settings.MCPServer{Type: transport}
+ switch transport {
+ case "http":
+ srv.URL = args[1]
+ case "stdio", "":
+ srv.Type = ""
+ srv.Command, srv.Args = args[1], args[2:]
+ default:
+ return fmt.Errorf("--transport: want stdio or http")
+ }
+ for _, kv := range env {
+ k, v, ok := strings.Cut(kv, "=")
+ if !ok {
+ return fmt.Errorf("-e %q: want KEY=value", kv)
+ }
+ if srv.Env == nil {
+ srv.Env = map[string]string{}
+ }
+ srv.Env[k] = v
+ }
+ for _, h := range headers {
+ k, v, ok := strings.Cut(h, ":")
+ if !ok {
+ return fmt.Errorf("--header %q: want 'Name: value'", h)
+ }
+ if srv.Headers == nil {
+ srv.Headers = map[string]string{}
+ }
+ srv.Headers[strings.TrimSpace(k)] = strings.TrimSpace(v)
+ }
+ p, err := editServers(scope, func(m map[string]settings.MCPServer) error {
+ m[name] = srv
+ return nil
+ })
+ if err != nil {
+ return err
+ }
+ fmt.Fprintf(cmd.OutOrStdout(), "added MCP server %s to %s\n", name, p)
+ if scope == "project" || scope == "local" {
+ fmt.Fprintln(cmd.OutOrStdout(), "project servers run once the folder is trusted: cometcli trust")
+ }
+ return nil
+ },
+ }
+ cmd.Flags().StringVarP(&scope, "scope", "s", "user", "user | project (.mcp.json, shared) | local")
+ cmd.Flags().StringVarP(&transport, "transport", "t", "stdio", "stdio | http")
+ cmd.Flags().StringArrayVarP(&env, "env", "e", nil, "KEY=value for a stdio server (${VAR} expands)")
+ cmd.Flags().StringArrayVarP(&headers, "header", "H", nil, "'Name: value' for an http server (${VAR} expands)")
+ return cmd
+}
+
+func mcpRemoveCmd() *cobra.Command {
+ var scope string
+ cmd := &cobra.Command{
+ Use: "remove ",
+ Short: "Remove an MCP server",
+ Args: cobra.ExactArgs(1),
+ RunE: func(cmd *cobra.Command, args []string) error {
+ p, err := editServers(scope, func(m map[string]settings.MCPServer) error {
+ if _, ok := m[args[0]]; !ok {
+ return fmt.Errorf("no MCP server %q in %s scope", args[0], scope)
+ }
+ delete(m, args[0])
+ return nil
+ })
+ if err != nil {
+ return err
+ }
+ fmt.Fprintf(cmd.OutOrStdout(), "removed %s from %s\n", args[0], p)
+ return nil
+ },
+ }
+ cmd.Flags().StringVarP(&scope, "scope", "s", "user", "user | project | local")
+ return cmd
+}
+
+func mcpListCmd() *cobra.Command {
+ var check bool
+ cmd := &cobra.Command{
+ Use: "list",
+ Short: "List MCP servers the agent will use here (--check connects to each)",
+ RunE: func(cmd *cobra.Command, _ []string) error {
+ cwd, _ := os.Getwd()
+ st, err := settings.Load(cwd)
+ if err != nil {
+ return err
+ }
+ names := make([]string, 0, len(st.MCPServers))
+ for n := range st.MCPServers {
+ names = append(names, n)
+ }
+ sort.Strings(names)
+ tw := tabwriter.NewWriter(cmd.OutOrStdout(), 0, 2, 2, ' ', 0)
+ fmt.Fprintln(tw, "NAME\tSCOPE\tTARGET\tSTATUS")
+ for _, n := range names {
+ srv := st.MCPServers[n]
+ target := srv.URL
+ if target == "" {
+ target = strings.TrimSpace(srv.Command + " " + strings.Join(srv.Args, " "))
+ }
+ status := "-"
+ if check {
+ ctx, cancel := context.WithTimeout(cmd.Context(), 20*time.Second)
+ c, err := mcpclient.Connect(ctx, n, srv)
+ cancel()
+ if err != nil {
+ status = "✗ " + err.Error()
+ } else {
+ status = fmt.Sprintf("✓ %d tools", len(c.Tools))
+ c.Close()
+ }
+ }
+ fmt.Fprintf(tw, "%s\t%s\t%s\t%s\n", n, srv.Scope, target, status)
+ }
+ if len(names) == 0 {
+ fmt.Fprintln(tw, "(none)\t\t\t")
+ }
+ for _, ig := range st.Ignored {
+ fmt.Fprintf(tw, "skipped\t\t%s\tproject not trusted — cometcli trust\n", ig)
+ }
+ return tw.Flush()
+ },
+ }
+ cmd.Flags().BoolVar(&check, "check", false, "connect to each server and count its tools")
+ return cmd
+}
+
+// trustCmd marks a project directory as trusted so its hooks and MCP
+// servers may run.
+func trustCmd() *cobra.Command {
+ var off, list bool
+ cmd := &cobra.Command{
+ Use: "trust [dir]",
+ Short: "Trust a project folder: allow its hooks and MCP servers to run",
+ Long: `Project settings (.cometcli/settings.json, settings.local.json, .mcp.json)
+can define hooks and MCP servers — commands that run on this machine. They
+are ignored until you trust the folder. Trust covers subdirectories.`,
+ Args: cobra.MaximumNArgs(1),
+ RunE: func(cmd *cobra.Command, args []string) error {
+ if list {
+ for _, d := range settings.Trusted() {
+ fmt.Fprintln(cmd.OutOrStdout(), d)
+ }
+ return nil
+ }
+ dir, _ := os.Getwd()
+ if len(args) == 1 {
+ dir = args[0]
+ } else if root := settings.FindRoot(dir); root != "" {
+ dir = root
+ }
+ if err := settings.Trust(dir, !off); err != nil {
+ return err
+ }
+ abs, _ := filepath.Abs(dir)
+ verb := "trusted"
+ if off {
+ verb = "no longer trusted"
+ }
+ fmt.Fprintf(cmd.OutOrStdout(), "%s: %s\n", verb, abs)
+ return nil
+ },
+ }
+ cmd.Flags().BoolVar(&off, "off", false, "revoke trust")
+ cmd.Flags().BoolVar(&list, "list", false, "list trusted folders")
+ return cmd
+}
diff --git a/internal/cli/root.go b/internal/cli/root.go
index 365fb3d..4457758 100644
--- a/internal/cli/root.go
+++ b/internal/cli/root.go
@@ -6,10 +6,12 @@ package cli
import (
"bufio"
"fmt"
+ "golang.org/x/term"
"io"
+ "os"
"sort"
+ "strconv"
"strings"
- "time"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
@@ -28,38 +30,53 @@ var (
// NewRoot builds the root command and registers all subcommands.
func NewRoot(reg *toolkit.Registry, extra []*cobra.Command) *cobra.Command {
root := &cobra.Command{
- Use: "cometcli",
- Short: "Agentic SRE terminal for Cosmos-EVM validators",
- Long: `cometcli — an agentic SRE terminal for Cosmos-EVM validators.
+ Use: "cometcli [prompt]",
+ Short: "Agentic SRE terminal — general ops, plus Cosmos-EVM validators",
+ Long: `cometcli — an agentic SRE terminal.
-Every capability is a deterministic subcommand (cometcli val status,
-cometcli doctor, cometcli tx unjail) AND a tool the agent can call.
-Run 'cometcli' with no arguments to open the chat terminal, 'cometcli
-ask "..."' for one-shot questions, or 'cometcli serve' for a local web chat.`,
+ cometcli chat on this machine (shell, files, web)
+ cometcli "why is disk full?" start the chat with a prompt
+ cometcli -p "…" answer once and exit (scripts, pipes, CI)
+ cmd | cometcli -p "…" piped input becomes context
+ cometcli one [profile] […] node mode: validator tools, rules, live snapshot
+
+Every node capability is also a deterministic subcommand (cometcli val
+status, cometcli doctor, cometcli tx unjail) and a tool the agent calls.`,
SilenceUsage: true,
SilenceErrors: true,
- Args: cobra.NoArgs,
- RunE: func(cmd *cobra.Command, _ []string) error {
- // Bare `cometcli` opens the chat TUI when there's a terminal and
- // a profile; otherwise it's the classic help screen.
- if !isTerminal(cmd.InOrStdin()) || !hasProfile() {
- if isTerminal(cmd.InOrStdin()) {
- fmt.Fprint(cmd.OutOrStdout(), "no profile yet — run `cometcli init` to connect a node, then `cometcli` opens the chat.\n\n")
+ Args: cobra.ArbitraryArgs,
+ RunE: func(cmd *cobra.Command, args []string) error {
+ // an explicit --profile means node mode, like `cometcli one`
+ var p *config.Profile
+ if cmd.Flags().Changed("profile") {
+ cfg, err := config.Load()
+ if err != nil {
+ return err
+ }
+ if p, err = cfg.ActiveProfile(flagProfile); err != nil {
+ return err
}
- return cmd.Help()
}
- return runUI(cmd, reg, 5*time.Second)
+ return runChat(cmd, reg, p, args)
},
}
pf := root.PersistentFlags()
pf.StringVar(&flagProfile, "profile", "", "profile to use (env COMETCLI_PROFILE)")
pf.BoolVar(&flagJSON, "json", false, "emit structured JSON")
pf.BoolVarP(&flagYes, "yes", "y", false, "auto-approve observe/diagnose prompts (never on-chain)")
+ agentFlags(root)
+ printFlags(root)
+ pf.Bool("debug", false, "log every LLM API attempt (status, timing, retries) to stderr")
+ root.PersistentPreRun = func(cmd *cobra.Command, _ []string) {
+ if d, _ := cmd.Flags().GetBool("debug"); d {
+ _ = os.Setenv("COMETCLI_DEBUG", "1")
+ }
+ }
for _, c := range toolGroupCommands(reg) {
root.AddCommand(c)
}
- root.AddCommand(profileCmd(), auditCmd(), versionCmd(), initCmd())
+ root.AddCommand(profileCmd(), auditCmd(), versionCmd(), initCmd(), sessionsCmd(), oneCmd(reg), configCmd(), trustCmd())
root.AddCommand(extra...)
return root
}
@@ -69,8 +86,8 @@ func toolGroupCommands(reg *toolkit.Registry) []*cobra.Command {
groups := map[string]*cobra.Command{}
for _, t := range reg.All() {
domain, _, ok := strings.Cut(t.Name(), ".")
- if !ok {
- continue
+ if !ok || toolkit.IsAgentOnly(t) {
+ continue // general agent tools (bash, read, …) have no subcommand
}
parent, ok := groups[domain]
if !ok {
@@ -79,7 +96,7 @@ func toolGroupCommands(reg *toolkit.Registry) []*cobra.Command {
}
parent.AddCommand(toolCmd(t))
}
- preferred := []string{"node", "val", "chain", "evm", "keys", "tx", "upgrade", "snap", "mon", "sec", "net", "runbook", "fleet"}
+ preferred := []string{"node", "val", "chain", "gov", "evm", "keys", "tx", "upgrade", "snap", "mon", "sec", "net", "runbook", "fleet"}
var out []*cobra.Command
for _, d := range preferred {
if p, ok := groups[d]; ok {
@@ -165,7 +182,7 @@ func RunTool(cmd *cobra.Command, t toolkit.Tool, args toolkit.Args) error {
}
defer c.Close()
if !toolkit.IsLongRunning(t) {
- sub, cancel := toolkit.WithDeadline(c, 90*time.Second)
+ sub, cancel := toolkit.WithDeadline(c, toolkit.CallTimeout(t, args))
defer cancel()
defer sub.Close()
c = sub
@@ -218,28 +235,44 @@ func NewCtx(cmd *cobra.Command, requireProfile bool) (*toolkit.Context, error) {
if flagYes {
c.AutoApproveBelow = toolkit.TierLocalChange
}
- c.Approver = StdinApprover(cmd.InOrStdin())
+ c.Approver, c.Chooser = StdinPrompts(cmd.InOrStdin())
+ c.Secret = TTYSecret
return c, nil
}
-// hasProfile reports whether an active profile is configured.
-func hasProfile() bool {
- cfg, err := config.Load()
- if err != nil {
- return false
- }
- p, err := cfg.ActiveProfile(flagProfile)
- return err == nil && p != nil
-}
-
// StdinApprover prompts y/N on the terminal.
func StdinApprover(in io.Reader) toolkit.Approver {
+ a, _ := StdinPrompts(in)
+ return a
+}
+
+// StdinPrompts returns a y/N approver and a numbered chooser sharing one
+// reader (two readers on one stdin would steal each other's input).
+func StdinPrompts(in io.Reader) (toolkit.Approver, toolkit.Chooser) {
reader := bufio.NewReader(in)
+ choose := func(c *toolkit.Context, prompt string, options []string) (int, error) {
+ fmt.Fprintf(c.Out, "\n%s\n", prompt)
+ for i, o := range options {
+ fmt.Fprintf(c.Out, " %d. %s\n", i+1, o)
+ }
+ fmt.Fprintf(c.Out, "Choose [1-%d]: ", len(options))
+ line, err := reader.ReadString('\n')
+ if err != nil && strings.TrimSpace(line) == "" {
+ return 0, fmt.Errorf("no choice made: stdin closed")
+ }
+ n, err := strconv.Atoi(strings.TrimSpace(line))
+ if err != nil || n < 1 || n > len(options) {
+ return 0, fmt.Errorf("invalid choice %q", strings.TrimSpace(line))
+ }
+ return n - 1, nil
+ }
return func(c *toolkit.Context, prompt string, tier toolkit.Tier, detail map[string]any) (bool, error) {
fmt.Fprintf(c.Out, "\n⚠ [%s] %s\n", tier, prompt)
keys := make([]string, 0, len(detail))
for k := range detail {
- keys = append(keys, k)
+ if !strings.HasPrefix(k, "_") { // front-end hints, not for display
+ keys = append(keys, k)
+ }
}
sort.Strings(keys)
for _, k := range keys {
@@ -255,9 +288,29 @@ func StdinApprover(in io.Reader) toolkit.Approver {
}
fmt.Fprint(c.Out, "Proceed? [y/N] ")
line, err := reader.ReadString('\n')
- if err != nil {
+ if err == io.EOF && strings.TrimSpace(line) == "" {
+ fmt.Fprintln(c.Out)
+ return false, fmt.Errorf("not approved: no terminal to answer on (stdin closed) — run interactively, or allow it with --allowedTools / agent.permissions")
+ }
+ if err != nil && err != io.EOF {
return false, err
}
return strings.EqualFold(strings.TrimSpace(line), "y"), nil
+ }, choose
+}
+
+// TTYSecret reads a secret from the controlling terminal with echo off.
+func TTYSecret(c *toolkit.Context, prompt string) (string, error) {
+ tty, err := os.OpenFile("/dev/tty", os.O_RDWR, 0)
+ if err != nil {
+ return "", fmt.Errorf("%s: no terminal available", prompt)
+ }
+ defer tty.Close()
+ fmt.Fprintf(tty, "%s: ", prompt)
+ b, err := term.ReadPassword(int(tty.Fd()))
+ fmt.Fprintln(tty)
+ if err != nil {
+ return "", err
}
+ return string(b), nil
}
diff --git a/internal/cli/serve.go b/internal/cli/serve.go
index abed279..a3368cf 100644
--- a/internal/cli/serve.go
+++ b/internal/cli/serve.go
@@ -36,7 +36,7 @@ page. To reach it from another machine, tunnel it: ssh -L 8765:127.0.0.1:8765 ho
defer c.Audit.Close()
a, agentErr := agent.New(c, reg)
if agentErr == nil {
- if err := applyAgentFlags(cmd, a); err != nil {
+ if _, err := applyAgentFlags(cmd, a); err != nil {
return err
}
}
diff --git a/internal/cli/sessions.go b/internal/cli/sessions.go
new file mode 100644
index 0000000..5d2fb89
--- /dev/null
+++ b/internal/cli/sessions.go
@@ -0,0 +1,48 @@
+package cli
+
+import (
+ "encoding/json"
+ "fmt"
+ "text/tabwriter"
+
+ "github.com/spf13/cobra"
+
+ "github.com/abhijitkrm/cometcli/internal/agent"
+)
+
+// sessionsCmd lists saved agent sessions for --resume.
+func sessionsCmd() *cobra.Command {
+ var all bool
+ cmd := &cobra.Command{
+ Use: "sessions",
+ Short: "List saved agent sessions (resume with -r or -c)",
+ Args: cobra.NoArgs,
+ RunE: func(cmd *cobra.Command, _ []string) error {
+ list, err := agent.ListSessions()
+ if err != nil {
+ return err
+ }
+ if flagJSON {
+ enc := json.NewEncoder(cmd.OutOrStdout())
+ enc.SetIndent("", " ")
+ return enc.Encode(list)
+ }
+ if len(list) == 0 {
+ fmt.Fprintln(cmd.OutOrStdout(), "no saved sessions")
+ return nil
+ }
+ tw := tabwriter.NewWriter(cmd.OutOrStdout(), 0, 2, 2, ' ', 0)
+ fmt.Fprintln(tw, "ID\tUPDATED\tPROFILE\tMODEL\tTITLE")
+ for i, s := range list {
+ if i == 20 && !all {
+ fmt.Fprintf(tw, "… %d more (--all)\t\t\t\t\n", len(list)-20)
+ break
+ }
+ fmt.Fprintf(tw, "%s\t%s\t%s\t%s\t%s\n", s.ID, s.Updated.Local().Format("Jan 02 15:04"), s.Profile, s.Model, s.Title)
+ }
+ return tw.Flush()
+ },
+ }
+ cmd.Flags().BoolVar(&all, "all", false, "list every session")
+ return cmd
+}
diff --git a/internal/cli/ui.go b/internal/cli/ui.go
index ce8362a..a7a1530 100644
--- a/internal/cli/ui.go
+++ b/internal/cli/ui.go
@@ -2,7 +2,6 @@ package cli
import (
"fmt"
- "io"
"os"
"time"
@@ -10,6 +9,7 @@ import (
"github.com/spf13/cobra"
+ "github.com/abhijitkrm/cometcli/internal/agent"
"github.com/abhijitkrm/cometcli/internal/toolkit"
"github.com/abhijitkrm/cometcli/internal/tui"
)
@@ -27,11 +27,13 @@ func UICmd(reg *toolkit.Registry) *cobra.Command {
},
}
cmd.Flags().IntVar(&interval, "interval", 5, "refresh interval seconds")
+ sessionFlags(cmd)
return cmd
}
-func isTerminal(r io.Reader) bool {
- f, ok := r.(*os.File)
+// isTerminal reports whether a stream (stdin or stdout) is a terminal.
+func isTerminal(s any) bool {
+ f, ok := s.(*os.File)
return ok && term.IsTerminal(int(f.Fd()))
}
@@ -46,5 +48,7 @@ func runUI(cmd *cobra.Command, reg *toolkit.Registry, interval time.Duration) er
}
defer c.Close()
defer c.Audit.Close()
- return tui.RunApp(c, reg, interval)
+ return tui.RunApp(c, reg, interval, func(a *agent.Agent) (string, error) {
+ return applyAgentFlags(cmd, a)
+ })
}
diff --git a/internal/client/evm/evm.go b/internal/client/evm/evm.go
index 6c9dc82..354b7c4 100644
--- a/internal/client/evm/evm.go
+++ b/internal/client/evm/evm.go
@@ -7,8 +7,10 @@ import (
"context"
"encoding/json"
"fmt"
+ "io"
"net/http"
"strconv"
+ "strings"
"sync/atomic"
"time"
)
@@ -56,9 +58,13 @@ func (c *Client) Call(ctx context.Context, method string, params []any, out any)
return err
}
defer resp.Body.Close()
+ if resp.StatusCode/100 != 2 {
+ b, _ := io.ReadAll(io.LimitReader(resp.Body, 300))
+ return fmt.Errorf("%s: HTTP %d: %s", method, resp.StatusCode, strings.TrimSpace(string(b)))
+ }
var r rpcResp
if err := json.NewDecoder(resp.Body).Decode(&r); err != nil {
- return err
+ return fmt.Errorf("%s: bad JSON-RPC response: %w", method, err)
}
if r.Error != nil {
return fmt.Errorf("%s: rpc error %d: %s", method, r.Error.Code, r.Error.Message)
@@ -69,18 +75,36 @@ func (c *Client) Call(ctx context.Context, method string, params []any, out any)
return json.Unmarshal(r.Result, out)
}
-func hexToUint64(s string) uint64 {
- n, _ := strconv.ParseUint(s, 0, 64)
- return n
+// hexToUint64 parses a JSON-RPC quantity ("0x1a"). A malformed value is
+// an error, never a silent 0 (which would read as block 0 and fake a
+// huge parity drift).
+func hexToUint64(s string) (uint64, error) {
+ if !strings.HasPrefix(s, "0x") && !strings.HasPrefix(s, "0X") {
+ return 0, fmt.Errorf("bad quantity %q: want 0x-prefixed hex", s)
+ }
+ n, err := strconv.ParseUint(s[2:], 16, 64)
+ if err != nil {
+ return 0, fmt.Errorf("bad quantity %q: %w", s, err)
+ }
+ return n, nil
}
-// BlockNumber returns the latest EVM block number.
-func (c *Client) BlockNumber(ctx context.Context) (uint64, error) {
+// quantity calls a method returning one hex quantity.
+func (c *Client) quantity(ctx context.Context, method string) (uint64, error) {
var hex string
- if err := c.Call(ctx, "eth_blockNumber", nil, &hex); err != nil {
+ if err := c.Call(ctx, method, nil, &hex); err != nil {
return 0, err
}
- return hexToUint64(hex), nil
+ n, err := hexToUint64(hex)
+ if err != nil {
+ return 0, fmt.Errorf("%s: %w", method, err)
+ }
+ return n, nil
+}
+
+// BlockNumber returns the latest EVM block number.
+func (c *Client) BlockNumber(ctx context.Context) (uint64, error) {
+ return c.quantity(ctx, "eth_blockNumber")
}
// Syncing returns nil if synced, or the sync progress object.
@@ -103,20 +127,12 @@ func (c *Client) Syncing(ctx context.Context) (map[string]any, error) {
// GasPrice returns the current gas price in wei.
func (c *Client) GasPrice(ctx context.Context) (uint64, error) {
- var hex string
- if err := c.Call(ctx, "eth_gasPrice", nil, &hex); err != nil {
- return 0, err
- }
- return hexToUint64(hex), nil
+ return c.quantity(ctx, "eth_gasPrice")
}
// PeerCount returns net_peerCount.
func (c *Client) PeerCount(ctx context.Context) (uint64, error) {
- var hex string
- if err := c.Call(ctx, "net_peerCount", nil, &hex); err != nil {
- return 0, err
- }
- return hexToUint64(hex), nil
+ return c.quantity(ctx, "net_peerCount")
}
// ClientVersion returns web3_clientVersion.
@@ -128,11 +144,7 @@ func (c *Client) ClientVersion(ctx context.Context) (string, error) {
// ChainID returns eth_chainId (the EIP-155 id).
func (c *Client) ChainID(ctx context.Context) (uint64, error) {
- var hex string
- if err := c.Call(ctx, "eth_chainId", nil, &hex); err != nil {
- return 0, err
- }
- return hexToUint64(hex), nil
+ return c.quantity(ctx, "eth_chainId")
}
// TxPoolStatus returns txpool_status {pending, queued} counts.
@@ -143,7 +155,11 @@ func (c *Client) TxPoolStatus(ctx context.Context) (map[string]uint64, error) {
}
out := map[string]uint64{}
for k, v := range m {
- out[k] = hexToUint64(v)
+ n, err := hexToUint64(v)
+ if err != nil {
+ return nil, fmt.Errorf("txpool_status %s: %w", k, err)
+ }
+ out[k] = n
}
return out, nil
}
diff --git a/internal/client/evm/evm_test.go b/internal/client/evm/evm_test.go
new file mode 100644
index 0000000..6c5b907
--- /dev/null
+++ b/internal/client/evm/evm_test.go
@@ -0,0 +1,77 @@
+package evm
+
+import (
+ "context"
+ "encoding/json"
+ "net/http"
+ "net/http/httptest"
+ "strings"
+ "testing"
+)
+
+func server(t *testing.T, results map[string]string, status int) *Client {
+ t.Helper()
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ if status != 200 {
+ w.WriteHeader(status)
+ w.Write([]byte("bad gateway"))
+ return
+ }
+ var req rpcReq
+ json.NewDecoder(r.Body).Decode(&req)
+ res, ok := results[req.Method]
+ if !ok {
+ w.Write([]byte(`{"jsonrpc":"2.0","id":1,"error":{"code":-32601,"message":"method not found"}}`))
+ return
+ }
+ w.Write([]byte(`{"jsonrpc":"2.0","id":1,"result":` + res + `}`))
+ }))
+ t.Cleanup(srv.Close)
+ return New(srv.URL)
+}
+
+func TestQuantitiesAndSyncing(t *testing.T) {
+ c := server(t, map[string]string{
+ "eth_blockNumber": `"0x279c8"`, "eth_gasPrice": `"0x3b9aca00"`, "net_peerCount": `"0x3"`,
+ "eth_chainId": `"0x1e241"`, "eth_syncing": `false`, "web3_clientVersion": `"evmd/v0.7.2"`,
+ "txpool_status": `{"pending":"0x2","queued":"0x0"}`,
+ }, 200)
+ ctx := context.Background()
+ if n, err := c.BlockNumber(ctx); err != nil || n != 162248 {
+ t.Fatalf("block = %d %v", n, err)
+ }
+ if n, _ := c.GasPrice(ctx); n != 1_000_000_000 {
+ t.Fatalf("gas = %d", n)
+ }
+ if n, _ := c.ChainID(ctx); n != 123457 {
+ t.Fatalf("chain id = %d", n)
+ }
+ if s, err := c.Syncing(ctx); err != nil || s != nil {
+ t.Fatalf("syncing false → nil: %v %v", s, err)
+ }
+ if v, _ := c.ClientVersion(ctx); v != "evmd/v0.7.2" {
+ t.Fatalf("version = %q", v)
+ }
+ if p, err := c.TxPoolStatus(ctx); err != nil || p["pending"] != 2 {
+ t.Fatalf("txpool = %v %v", p, err)
+ }
+ c2 := server(t, map[string]string{"eth_syncing": `{"currentBlock":"0x10","highestBlock":"0x20"}`}, 200)
+ if s, err := c2.Syncing(ctx); err != nil || s["highestBlock"] != "0x20" {
+ t.Fatalf("syncing object: %v %v", s, err)
+ }
+}
+
+func TestMalformedAndErrorsAreErrors(t *testing.T) {
+ ctx := context.Background()
+ c := server(t, map[string]string{"eth_blockNumber": `"162248"`}, 200)
+ if _, err := c.BlockNumber(ctx); err == nil || !strings.Contains(err.Error(), "bad quantity") {
+ t.Fatalf("decimal string must not parse as a block number: %v", err)
+ }
+ if _, err := c.GasPrice(ctx); err == nil || !strings.Contains(err.Error(), "method not found") {
+ t.Fatalf("rpc error: %v", err)
+ }
+ c = server(t, nil, 502)
+ if _, err := c.BlockNumber(ctx); err == nil || !strings.Contains(err.Error(), "HTTP 502") {
+ t.Fatalf("http error: %v", err)
+ }
+}
diff --git a/internal/client/grpc/account.go b/internal/client/grpc/account.go
index 0639edd..382fb40 100644
--- a/internal/client/grpc/account.go
+++ b/internal/client/grpc/account.go
@@ -6,6 +6,7 @@ import (
authv1beta1 "cosmossdk.io/api/cosmos/auth/v1beta1"
"google.golang.org/protobuf/encoding/protowire"
"google.golang.org/protobuf/proto"
+ "google.golang.org/protobuf/types/known/anypb"
)
// DecodeAccount extracts (account_number, sequence) from an account Any.
@@ -27,6 +28,23 @@ func DecodeAccount(typeURL string, value []byte) (num, seq uint64, err error) {
return acc.AccountNumber, acc.Sequence, nil
}
+// DecodeAccountPubKey extracts the account's public key Any (nil when the
+// account has never signed a transaction).
+func DecodeAccountPubKey(typeURL string, value []byte) (*anypb.Any, error) {
+ raw := value
+ if typeURL == "/ethermint.types.v1.EthAccount" || typeURL == "/cosmos.evm.types.v1.EthAccount" {
+ raw = fieldBytes(value, 1)
+ }
+ var acc authv1beta1.BaseAccount
+ if err := proto.Unmarshal(raw, &acc); err != nil {
+ return nil, fmt.Errorf("decoding %s: %w", typeURL, err)
+ }
+ if acc.PubKey == nil || acc.PubKey.TypeUrl == "" {
+ return nil, nil
+ }
+ return &anypb.Any{TypeUrl: acc.PubKey.TypeUrl, Value: acc.PubKey.Value}, nil
+}
+
// fieldBytes returns the bytes of a length-delimited field, or nil.
func fieldBytes(b []byte, num protowire.Number) []byte {
for len(b) > 0 {
diff --git a/internal/client/grpc/grpc.go b/internal/client/grpc/grpc.go
index f339671..8897b86 100644
--- a/internal/client/grpc/grpc.go
+++ b/internal/client/grpc/grpc.go
@@ -12,6 +12,7 @@ import (
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/credentials/insecure"
+ "google.golang.org/protobuf/types/known/anypb"
authv1beta1 "cosmossdk.io/api/cosmos/auth/v1beta1"
bankv1beta1 "cosmossdk.io/api/cosmos/bank/v1beta1"
@@ -84,6 +85,19 @@ func (c *Conn) RawConn() *grpc.ClientConn { return c.cc }
func (c *Conn) Close() error { return c.cc.Close() }
+// AccountPubKey returns the public key the chain has on record for an
+// address (nil if it never signed).
+func (c *Conn) AccountPubKey(ctx context.Context, bech32Addr string) (*anypb.Any, error) {
+ res, err := c.Auth.Account(ctx, &authv1beta1.QueryAccountRequest{Address: bech32Addr})
+ if err != nil {
+ return nil, err
+ }
+ if res.Account == nil {
+ return nil, nil
+ }
+ return DecodeAccountPubKey(res.Account.TypeUrl, res.Account.Value)
+}
+
// Account fetches account_number/sequence for an address. Cosmos-EVM chains
// may return either BaseAccount or EthAccount; both are handled.
func (c *Conn) Account(ctx context.Context, bech32Addr string) (num, seq uint64, err error) {
diff --git a/internal/client/host/exec.go b/internal/client/host/exec.go
new file mode 100644
index 0000000..94c43b9
--- /dev/null
+++ b/internal/client/host/exec.go
@@ -0,0 +1,209 @@
+package host
+
+import (
+ "bytes"
+ "context"
+ "errors"
+ "fmt"
+ "os"
+ "os/exec"
+ "sync"
+ "time"
+
+ "golang.org/x/crypto/ssh"
+)
+
+// ExecResult is the outcome of a shell script run by Exec.
+type ExecResult struct {
+ Output string // stdout and stderr, interleaved as written
+ Code int // exit status (-1 when killed)
+ TimedOut bool
+ Truncated bool // Output kept only its head and tail
+}
+
+// Execer runs a script with combined output, an output cap, and
+// process-tree cleanup on cancel. Local and SSH implement it; other Host
+// implementations fall back to Run.
+type Execer interface {
+ Exec(ctx context.Context, script string, maxOut int) (ExecResult, error)
+}
+
+// StdinExecer runs a script with input on stdin (secrets, documents).
+type StdinExecer interface {
+ ExecIn(ctx context.Context, script string, stdin []byte, maxOut int) (ExecResult, error)
+}
+
+// ExecIn runs script on h with stdin. Hosts without stdin support error.
+func ExecIn(ctx context.Context, h Host, script string, stdin []byte, maxOut int) (ExecResult, error) {
+ e, ok := h.(StdinExecer)
+ if !ok {
+ return ExecResult{}, fmt.Errorf("host %s can't pass stdin to commands", h)
+ }
+ return e.ExecIn(ctx, script, stdin, maxOut)
+}
+
+// Exec runs script on h via Execer when available.
+func Exec(ctx context.Context, h Host, script string, maxOut int) (ExecResult, error) {
+ if e, ok := h.(Execer); ok {
+ return e.Exec(ctx, script, maxOut)
+ }
+ out, code, err := h.Run(ctx, script)
+ if err != nil && code != 0 {
+ out += "\n" + err.Error()
+ err = nil
+ }
+ return ExecResult{Output: out, Code: code, TimedOut: errors.Is(ctx.Err(), context.DeadlineExceeded)}, err
+}
+
+// nonInteractiveEnv keeps tools from waiting on a terminal that isn't
+// there (stdin is /dev/null): pagers, git credential prompts, apt.
+var nonInteractiveEnv = []string{
+ "PAGER=cat", "GIT_PAGER=cat", "SYSTEMD_PAGER=", "GIT_TERMINAL_PROMPT=0",
+ "DEBIAN_FRONTEND=noninteractive", "TERM=dumb", "NO_COLOR=1",
+}
+
+// capWriter keeps the first and last half of max bytes.
+type capWriter struct {
+ mu sync.Mutex
+ max int
+ head bytes.Buffer
+ tail []byte
+ truncated bool
+}
+
+func (w *capWriter) Write(p []byte) (int, error) {
+ w.mu.Lock()
+ defer w.mu.Unlock()
+ n := len(p)
+ if room := w.max/2 - w.head.Len(); room > 0 {
+ k := min(room, len(p))
+ w.head.Write(p[:k])
+ p = p[k:]
+ }
+ if len(p) > 0 {
+ w.tail = append(w.tail, p...)
+ if over := len(w.tail) - w.max/2; over > 0 {
+ w.tail = w.tail[over:]
+ w.truncated = true
+ }
+ }
+ return n, nil
+}
+
+func (w *capWriter) String() string {
+ w.mu.Lock()
+ defer w.mu.Unlock()
+ if !w.truncated {
+ return w.head.String() + string(w.tail)
+ }
+ return fmt.Sprintf("%s\n…[output truncated]…\n%s", w.head.String(), w.tail)
+}
+
+func shellBin() string {
+ if p, err := exec.LookPath("bash"); err == nil {
+ return p
+ }
+ return "/bin/sh"
+}
+
+// Exec runs script under bash (or sh) with stdin from /dev/null. On
+// cancel or timeout the whole process group gets SIGTERM, then SIGKILL.
+func (l *Local) Exec(ctx context.Context, script string, maxOut int) (ExecResult, error) {
+ return l.ExecIn(ctx, script, nil, maxOut)
+}
+
+// ExecIn is Exec with stdin (nil = /dev/null).
+func (l *Local) ExecIn(ctx context.Context, script string, stdin []byte, maxOut int) (ExecResult, error) {
+ c := exec.Command(shellBin(), "-c", script)
+ c.Env = append(os.Environ(), nonInteractiveEnv...)
+ if stdin != nil {
+ c.Stdin = bytes.NewReader(stdin)
+ }
+ w := &capWriter{max: maxOut}
+ c.Stdout, c.Stderr = w, w
+ setProcessGroup(c)
+ if err := c.Start(); err != nil {
+ return ExecResult{}, err
+ }
+ done := make(chan error, 1)
+ go func() { done <- c.Wait() }()
+ var err error
+ timedOut := false
+ select {
+ case err = <-done:
+ case <-ctx.Done():
+ timedOut = errors.Is(ctx.Err(), context.DeadlineExceeded)
+ killGroup(c, false)
+ select {
+ case err = <-done:
+ case <-time.After(2 * time.Second):
+ killGroup(c, true)
+ err = <-done
+ }
+ }
+ res := ExecResult{Output: w.String(), Truncated: w.truncated, TimedOut: timedOut}
+ var ee *exec.ExitError
+ switch {
+ case err == nil:
+ case errors.As(err, &ee):
+ res.Code = ee.ExitCode()
+ default:
+ return res, err
+ }
+ if ctx.Err() != nil && res.Code == 0 {
+ res.Code = -1
+ }
+ return res, nil
+}
+
+// Exec runs script on the remote host. Cancel kills the session.
+func (s *SSH) Exec(ctx context.Context, script string, maxOut int) (ExecResult, error) {
+ return s.ExecIn(ctx, script, nil, maxOut)
+}
+
+// ExecIn is Exec with stdin.
+func (s *SSH) ExecIn(ctx context.Context, script string, stdin []byte, maxOut int) (ExecResult, error) {
+ sess, err := s.client.NewSession()
+ if err != nil {
+ return ExecResult{}, err
+ }
+ defer sess.Close()
+ w := &capWriter{max: maxOut}
+ sess.Stdout, sess.Stderr = w, w
+ if stdin != nil {
+ sess.Stdin = bytes.NewReader(stdin)
+ }
+ env := ""
+ for _, e := range nonInteractiveEnv {
+ env += e + " "
+ }
+ q := shellQuote(script)
+ cmd := fmt.Sprintf("export %s; if command -v bash >/dev/null 2>&1; then exec bash -c %s; else exec sh -c %s; fi", env, q, q)
+ done := make(chan error, 1)
+ go func() { done <- sess.Run(cmd) }()
+ res := ExecResult{}
+ select {
+ case err = <-done:
+ case <-ctx.Done():
+ res.TimedOut = errors.Is(ctx.Err(), context.DeadlineExceeded)
+ _ = sess.Signal(ssh.SIGKILL)
+ sess.Close()
+ err = <-done
+ res.Code = -1
+ }
+ res.Output, res.Truncated = w.String(), w.truncated
+ var ee *ssh.ExitError
+ switch {
+ case err == nil:
+ case errors.As(err, &ee):
+ res.Code = ee.ExitStatus()
+ case res.Code == -1:
+ default:
+ var missing *ssh.ExitMissingError
+ if !errors.As(err, &missing) {
+ return res, err
+ }
+ res.Code = -1
+ }
+ return res, nil
+}
diff --git a/internal/client/host/exec_other.go b/internal/client/host/exec_other.go
new file mode 100644
index 0000000..b399fbb
--- /dev/null
+++ b/internal/client/host/exec_other.go
@@ -0,0 +1,13 @@
+//go:build !unix
+
+package host
+
+import "os/exec"
+
+func setProcessGroup(*exec.Cmd) {}
+
+func killGroup(c *exec.Cmd, _ bool) {
+ if c.Process != nil {
+ _ = c.Process.Kill()
+ }
+}
diff --git a/internal/client/host/exec_test.go b/internal/client/host/exec_test.go
new file mode 100644
index 0000000..2b2fdea
--- /dev/null
+++ b/internal/client/host/exec_test.go
@@ -0,0 +1,57 @@
+package host
+
+import (
+ "context"
+ "strings"
+ "testing"
+ "time"
+)
+
+func TestLocalExecCombinedOutputAndCodes(t *testing.T) {
+ h := &Local{}
+ res, err := h.Exec(context.Background(), "echo out; echo err >&2; exit 3", 1000)
+ if err != nil || res.Code != 3 || !strings.Contains(res.Output, "out") || !strings.Contains(res.Output, "err") {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ // stdin is closed: a prompt gets EOF instead of hanging
+ res, _ = h.Exec(context.Background(), "read x; echo got=[$x] rc=$?", 1000)
+ if !strings.Contains(res.Output, "got=[]") {
+ t.Fatalf("read from stdin: %q", res.Output)
+ }
+ // non-interactive env
+ res, _ = h.Exec(context.Background(), "echo $GIT_TERMINAL_PROMPT $PAGER", 1000)
+ if strings.TrimSpace(res.Output) != "0 cat" {
+ t.Fatalf("env: %q", res.Output)
+ }
+}
+
+func TestLocalExecCapsOutputKeepingHeadAndTail(t *testing.T) {
+ res, err := (&Local{}).Exec(context.Background(), "echo HEAD; head -c 100000 /dev/zero | tr '\\0' x; echo; echo TAIL", 2000)
+ if err != nil || !res.Truncated || len(res.Output) > 2200 || !strings.HasPrefix(res.Output, "HEAD") || !strings.Contains(res.Output, "TAIL") {
+ t.Fatalf("truncated=%v len=%d", res.Truncated, len(res.Output))
+ }
+}
+
+func TestLocalExecTimeoutKillsChildren(t *testing.T) {
+ ctx, cancel := context.WithTimeout(context.Background(), 300*time.Millisecond)
+ defer cancel()
+ start := time.Now()
+ res, err := (&Local{}).Exec(ctx, "sleep 30 & sleep 30; wait", 1000)
+ if err != nil || !res.TimedOut || time.Since(start) > 5*time.Second {
+ t.Fatalf("res=%+v err=%v took %s", res, err, time.Since(start))
+ }
+}
+
+func TestLocalFiles(t *testing.T) {
+ dir := t.TempDir()
+ h := &Local{}
+ p := dir + "/f.toml"
+ if err := h.WriteFile(context.Background(), p, []byte("a=1"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ b, err := h.ReadFile(context.Background(), p)
+ fi, _ := h.Stat(context.Background(), p)
+ if err != nil || string(b) != "a=1" || fi.Mode().Perm() != 0o600 {
+ t.Fatalf("read=%q err=%v mode=%v", b, err, fi.Mode().Perm())
+ }
+}
diff --git a/internal/client/host/exec_unix.go b/internal/client/host/exec_unix.go
new file mode 100644
index 0000000..c81dda1
--- /dev/null
+++ b/internal/client/host/exec_unix.go
@@ -0,0 +1,25 @@
+//go:build unix
+
+package host
+
+import (
+ "os/exec"
+ "syscall"
+)
+
+func setProcessGroup(c *exec.Cmd) {
+ c.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
+}
+
+// killGroup signals the command's whole process group, so children
+// (docker compose, build steps) die with the shell.
+func killGroup(c *exec.Cmd, hard bool) {
+ if c.Process == nil {
+ return
+ }
+ sig := syscall.SIGTERM
+ if hard {
+ sig = syscall.SIGKILL
+ }
+ _ = syscall.Kill(-c.Process.Pid, sig)
+}
diff --git a/internal/config/config.go b/internal/config/config.go
index c502c88..f2ce061 100644
--- a/internal/config/config.go
+++ b/internal/config/config.go
@@ -39,10 +39,78 @@ const configFile = "config.yaml"
// Config is the root of ~/.cometcli/config.yaml.
type Config struct {
- Active string `yaml:"active"`
+ Active string `yaml:"active"`
+ // Agent is the default LLM setup, used on its own in general mode
+ // (no node) and as the base a profile's agent section overrides.
+ Agent AgentConf `yaml:"agent,omitempty"`
Profiles map[string]*Profile `yaml:"profiles"`
}
+// AgentFor resolves the agent settings for a profile (nil = general
+// mode): the global agent section, overridden field by field by the
+// profile's. In general mode without a global provider, the active
+// profile's settings are used, so existing setups keep working.
+func (c *Config) AgentFor(p *Profile) AgentConf {
+ if c == nil {
+ if p != nil {
+ return p.Agent
+ }
+ return AgentConf{}
+ }
+ if p == nil {
+ if c.Agent.Provider != "" {
+ return c.Agent
+ }
+ if ap, err := c.ActiveProfile(""); err == nil && ap != nil {
+ return MergeAgent(c.Agent, ap.Agent)
+ }
+ return c.Agent
+ }
+ return MergeAgent(c.Agent, p.Agent)
+}
+
+// MergeAgent overlays o's set fields onto base. A different provider in o
+// drops base's model, endpoint and key (they belong to the old provider);
+// permission rules accumulate.
+func MergeAgent(base, o AgentConf) AgentConf {
+ r := base
+ if o.Provider != "" && o.Provider != base.Provider {
+ r.Provider, r.Model, r.BaseURL, r.APIKeyEnv = o.Provider, "", "", ""
+ }
+ str := func(dst *string, v string) {
+ if v != "" {
+ *dst = v
+ }
+ }
+ num := func(dst *int, v int) {
+ if v != 0 {
+ *dst = v
+ }
+ }
+ str(&r.Model, o.Model)
+ str(&r.BaseURL, o.BaseURL)
+ str(&r.APIKeyEnv, o.APIKeyEnv)
+ str(&r.Mode, o.Mode)
+ str(&r.Effort, o.Effort)
+ str(&r.Tools, o.Tools)
+ num(&r.MaxTokens, o.MaxTokens)
+ num(&r.MaxTurns, o.MaxTurns)
+ num(&r.ContextWindow, o.ContextWindow)
+ num(&r.CompactAt, o.CompactAt)
+ if len(o.Autopilot) > 0 {
+ r.Autopilot = o.Autopilot
+ }
+ r.RedactHosts = append(append([]string{}, base.RedactHosts...), o.RedactHosts...)
+ r.RedactEndpoints = base.RedactEndpoints || o.RedactEndpoints
+ r.NoStream = base.NoStream || o.NoStream
+ r.Permissions = Permissions{
+ Allow: append(append([]string{}, base.Permissions.Allow...), o.Permissions.Allow...),
+ Ask: append(append([]string{}, base.Permissions.Ask...), o.Permissions.Ask...),
+ Deny: append(append([]string{}, base.Permissions.Deny...), o.Permissions.Deny...),
+ }
+ return r
+}
+
// Profile describes one managed node.
type Profile struct {
Name string `yaml:"-"`
@@ -70,6 +138,9 @@ type Endpoints struct {
GRPC string `yaml:"grpc,omitempty"` // host:port
LCD string `yaml:"lcd,omitempty"` // http(s)://host:1317
EVM string `yaml:"evm,omitempty"` // http(s)://host:8545 (RPC/sentry nodes only)
+ // FallbackGRPC is another node of the same chain (sentry, RPC node,
+ // public endpoint) used for chain queries while this node is down.
+ FallbackGRPC string `yaml:"fallback_grpc,omitempty"`
}
// Transport selects how host-plane operations reach the machine.
@@ -95,6 +166,16 @@ type Signer struct {
CoinType uint32 `yaml:"coin_type,omitempty"`
Account uint32 `yaml:"account,omitempty"`
Index uint32 `yaml:"index,omitempty"`
+ // Mode picks the signer when both are available: "" asks at signing
+ // time, "local" uses cometcli's keyring, "container" the node's.
+ Mode string `yaml:"mode,omitempty"`
+ // Container signing: the key lives in the node container's own
+ // keyring and evmd signs there (`docker exec … evmd tx sign`).
+ Container string `yaml:"container,omitempty"` // default: service.unit for docker services
+ ContainerKey string `yaml:"container_key,omitempty"` // default: key
+ ContainerKeyring string `yaml:"container_keyring,omitempty"` // test | file (default: detect)
+ ContainerHome string `yaml:"container_home,omitempty"` // evmd --home inside the container
+ ContainerBinary string `yaml:"container_binary,omitempty"` // default: the profile binary
}
// AgentConf configures the LLM backend for agent mode.
@@ -114,6 +195,31 @@ type AgentConf struct {
RedactEndpoints bool `yaml:"redact_endpoints,omitempty"`
// NoStream disables token streaming for endpoints that mishandle it.
NoStream bool `yaml:"no_stream,omitempty"`
+ // Effort is the reasoning depth: low | medium | high | xhigh | max
+ // (empty = model default). Mapped to each provider's own control.
+ Effort string `yaml:"effort,omitempty"`
+ // MaxTokens caps output tokens per model round (0 = provider default).
+ MaxTokens int `yaml:"max_tokens,omitempty"`
+ // MaxTurns bounds model rounds per prompt (0 = 200).
+ MaxTurns int `yaml:"max_turns,omitempty"`
+ // ContextWindow overrides the model's context size in tokens.
+ ContextWindow int `yaml:"context_window,omitempty"`
+ // CompactAt is the prompt size in tokens that triggers summarizing
+ // the conversation (0 = 80% of the window, at most 200k).
+ CompactAt int `yaml:"compact_at,omitempty"`
+ // Tools is "all" to send every tool schema on every request; the
+ // default loads non-core tools on demand via tool_search.
+ Tools string `yaml:"tools,omitempty"`
+ // Permissions are allow/ask/deny rules, e.g. "bash(systemctl status:*)",
+ // "edit(./config/**)", "web_fetch(domain:github.com)", "val.unjail".
+ Permissions Permissions `yaml:"permissions,omitempty"`
+}
+
+// Permissions holds permission rule lists. Deny beats ask beats allow.
+type Permissions struct {
+ Allow []string `yaml:"allow,omitempty"`
+ Ask []string `yaml:"ask,omitempty"`
+ Deny []string `yaml:"deny,omitempty"`
}
// Alerts configures notification sinks for the monitor.
diff --git a/internal/config/config_test.go b/internal/config/config_test.go
index 6907800..e5e42a6 100644
--- a/internal/config/config_test.go
+++ b/internal/config/config_test.go
@@ -64,3 +64,34 @@ func TestOverride(t *testing.T) {
t.Fatalf("override failed: %v", p.ChainID)
}
}
+
+func TestAgentForMergesAndFallsBack(t *testing.T) {
+ c := &Config{
+ Active: "v1",
+ Agent: AgentConf{Provider: "groq", Model: "openai/gpt-oss-120b", Effort: "low", Permissions: Permissions{Deny: []string{"bash(rm:*)"}}},
+ Profiles: map[string]*Profile{
+ "v1": {Agent: AgentConf{Model: "llama-3.3-70b-versatile", Permissions: Permissions{Allow: []string{"node.*"}}}},
+ "v2": {Agent: AgentConf{Provider: "gemini"}},
+ },
+ }
+ g := c.AgentFor(nil)
+ if g.Provider != "groq" || g.Model != "openai/gpt-oss-120b" {
+ t.Fatalf("general = %+v", g)
+ }
+ n := c.AgentFor(c.Profiles["v1"])
+ if n.Provider != "groq" || n.Model != "llama-3.3-70b-versatile" || n.Effort != "low" {
+ t.Fatalf("node override = %+v", n)
+ }
+ if len(n.Permissions.Deny) != 1 || len(n.Permissions.Allow) != 1 {
+ t.Fatalf("rules not accumulated: %+v", n.Permissions)
+ }
+ if m := c.AgentFor(c.Profiles["v2"]); m.Provider != "gemini" || m.Model != "" {
+ t.Fatalf("provider switch kept the old model: %+v", m)
+ }
+ // no global provider: general mode borrows the active profile's
+ c.Agent = AgentConf{}
+ c.Profiles["v1"].Agent.Provider = "gemini"
+ if g := c.AgentFor(nil); g.Provider != "gemini" {
+ t.Fatalf("fallback = %+v", g)
+ }
+}
diff --git a/internal/config/credentials.go b/internal/config/credentials.go
new file mode 100644
index 0000000..416a05c
--- /dev/null
+++ b/internal/config/credentials.go
@@ -0,0 +1,72 @@
+package config
+
+import (
+ "bufio"
+ "fmt"
+ "os"
+ "sort"
+ "strings"
+)
+
+const credentialsFile = "credentials"
+
+// Credential returns an API key saved by `cometcli config set-key`
+// (~/.cometcli/credentials, mode 0600, NAME=value lines), or "".
+// Environment variables always take precedence over this file.
+func Credential(name string) string {
+ p, err := Path(credentialsFile)
+ if err != nil {
+ return ""
+ }
+ f, err := os.Open(p)
+ if err != nil {
+ return ""
+ }
+ defer f.Close()
+ sc := bufio.NewScanner(f)
+ for sc.Scan() {
+ k, v, ok := strings.Cut(strings.TrimSpace(sc.Text()), "=")
+ if ok && strings.TrimSpace(k) == name {
+ return strings.TrimSpace(v)
+ }
+ }
+ return ""
+}
+
+// SetCredential saves (or, with an empty value, removes) a key.
+func SetCredential(name, value string) error {
+ if name == "" || strings.ContainsAny(name, "=\n ") || strings.ContainsAny(value, "\n\r") {
+ return fmt.Errorf("invalid credential name or value")
+ }
+ p, err := Path(credentialsFile)
+ if err != nil {
+ return err
+ }
+ kv := map[string]string{}
+ if raw, err := os.ReadFile(p); err == nil {
+ for _, l := range strings.Split(string(raw), "\n") {
+ if k, v, ok := strings.Cut(strings.TrimSpace(l), "="); ok {
+ kv[strings.TrimSpace(k)] = strings.TrimSpace(v)
+ }
+ }
+ }
+ if value == "" {
+ delete(kv, name)
+ } else {
+ kv[name] = value
+ }
+ names := make([]string, 0, len(kv))
+ for k := range kv {
+ names = append(names, k)
+ }
+ sort.Strings(names)
+ var b strings.Builder
+ for _, k := range names {
+ fmt.Fprintf(&b, "%s=%s\n", k, kv[k])
+ }
+ tmp := p + ".tmp"
+ if err := os.WriteFile(tmp, []byte(b.String()), 0o600); err != nil {
+ return err
+ }
+ return os.Rename(tmp, p)
+}
diff --git a/internal/config/credentials_test.go b/internal/config/credentials_test.go
new file mode 100644
index 0000000..177a10b
--- /dev/null
+++ b/internal/config/credentials_test.go
@@ -0,0 +1,33 @@
+package config
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+)
+
+func TestCredentials(t *testing.T) {
+ dir := t.TempDir()
+ t.Setenv("COMETCLI_HOME", dir)
+ if Credential("X_KEY") != "" {
+ t.Fatal("unexpected key")
+ }
+ if err := SetCredential("X_KEY", "abc=123"); err != nil {
+ t.Fatal(err)
+ }
+ _ = SetCredential("Y_KEY", "y")
+ if got := Credential("X_KEY"); got != "abc=123" {
+ t.Fatalf("got %q", got)
+ }
+ fi, _ := os.Stat(filepath.Join(dir, "credentials"))
+ if fi.Mode().Perm() != 0o600 {
+ t.Fatalf("mode %v", fi.Mode().Perm())
+ }
+ _ = SetCredential("X_KEY", "")
+ if Credential("X_KEY") != "" || Credential("Y_KEY") != "y" {
+ t.Fatal("remove")
+ }
+ if SetCredential("BAD NAME", "v") == nil || SetCredential("K", "a\nb") == nil {
+ t.Fatal("invalid accepted")
+ }
+}
diff --git a/internal/hooks/hooks.go b/internal/hooks/hooks.go
new file mode 100644
index 0000000..8022dbd
--- /dev/null
+++ b/internal/hooks/hooks.go
@@ -0,0 +1,249 @@
+// Package hooks runs operator-defined commands at points in the agent
+// loop, with Claude Code's contract: the event arrives as JSON on stdin;
+// exit 0 passes (stdout may carry JSON decisions or extra context); exit
+// 2 blocks, with stderr as the reason; any other exit is a non-blocking
+// error shown to the operator.
+package hooks
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "os"
+ "os/exec"
+ "regexp"
+ "strings"
+ "time"
+
+ "github.com/abhijitkrm/cometcli/internal/settings"
+)
+
+// Hook events.
+const (
+ PreToolUse = "PreToolUse"
+ PostToolUse = "PostToolUse"
+ UserPromptSubmit = "UserPromptSubmit"
+ Stop = "Stop"
+ SessionStart = "SessionStart"
+)
+
+// Events lists the supported events.
+var Events = []string{PreToolUse, PostToolUse, UserPromptSubmit, Stop, SessionStart}
+
+// Runner executes the hooks configured for a session.
+type Runner struct {
+ Matchers map[string][]settings.HookMatcher
+ // Dir is where hooks run and what COMETCLI_PROJECT_DIR points at.
+ Dir string
+}
+
+// Input is the JSON document a hook reads on stdin.
+type Input struct {
+ SessionID string `json:"session_id"`
+ TranscriptPath string `json:"transcript_path,omitempty"`
+ Cwd string `json:"cwd"`
+ Event string `json:"hook_event_name"`
+ PermissionMode string `json:"permission_mode,omitempty"`
+ ToolName string `json:"tool_name,omitempty"`
+ ToolInput map[string]any `json:"tool_input,omitempty"`
+ ToolResponse map[string]any `json:"tool_response,omitempty"`
+ Prompt string `json:"prompt,omitempty"`
+ StopHookActive bool `json:"stop_hook_active,omitempty"`
+ Source string `json:"source,omitempty"`
+}
+
+// Outcome is what the hooks for one event decided, combined.
+type Outcome struct {
+ // Block stops the action (tool call, prompt) or, for Stop, makes the
+ // agent continue; Reason says why (fed to the model for tool events).
+ Block bool
+ Reason string
+ // Decision is a PreToolUse permission decision: allow | deny | ask.
+ Decision string
+ // Context is extra text for the model (UserPromptSubmit, SessionStart
+ // stdout; additionalContext).
+ Context string
+ // Messages are shown to the operator (non-blocking errors, systemMessage).
+ Messages []string
+}
+
+// Has reports whether any hook is configured for the event.
+func (r *Runner) Has(event string) bool {
+ return r != nil && len(r.Matchers[event]) > 0
+}
+
+// Run executes every hook for event whose matcher fits in.ToolName.
+// Decisions combine conservatively: deny > ask > allow, any block blocks.
+func (r *Runner) Run(ctx context.Context, in Input) Outcome {
+ var out Outcome
+ if !r.Has(in.Event) {
+ return out
+ }
+ in.Cwd = def(in.Cwd, r.Dir)
+ payload, _ := json.Marshal(in)
+ var reasons, contexts []string
+ for _, m := range r.Matchers[in.Event] {
+ if !matches(m.Matcher, in.ToolName) {
+ continue
+ }
+ for _, h := range m.Hooks {
+ if h.Type != "" && h.Type != "command" || strings.TrimSpace(h.Command) == "" {
+ continue
+ }
+ res := r.exec(ctx, h, payload)
+ switch {
+ case res.err != nil && res.code < 0:
+ out.Messages = append(out.Messages, fmt.Sprintf("%s hook %q failed: %v", in.Event, h.Command, res.err))
+ case res.code == 2:
+ out.Block = true
+ reasons = append(reasons, firstNonEmpty(strings.TrimSpace(res.stderr), "blocked by "+in.Event+" hook"))
+ case res.code != 0:
+ msg := strings.TrimSpace(res.stderr)
+ out.Messages = append(out.Messages, fmt.Sprintf("%s hook %q exited %d: %s", in.Event, h.Command, res.code, msg))
+ default:
+ r.parseStdout(in.Event, res.stdout, &out, &reasons, &contexts)
+ }
+ }
+ }
+ out.Reason = strings.Join(reasons, "\n")
+ out.Context = strings.Join(contexts, "\n")
+ return out
+}
+
+func (r *Runner) parseStdout(event, stdout string, out *Outcome, reasons, contexts *[]string) {
+ s := strings.TrimSpace(stdout)
+ if s == "" {
+ return
+ }
+ if !strings.HasPrefix(s, "{") {
+ if event == UserPromptSubmit || event == SessionStart {
+ *contexts = append(*contexts, s)
+ }
+ return
+ }
+ var j struct {
+ Continue *bool `json:"continue"`
+ StopReason string `json:"stopReason"`
+ Decision string `json:"decision"`
+ Reason string `json:"reason"`
+ SystemMessage string `json:"systemMessage"`
+ Specific struct {
+ PermissionDecision string `json:"permissionDecision"`
+ PermissionDecisionReason string `json:"permissionDecisionReason"`
+ AdditionalContext string `json:"additionalContext"`
+ } `json:"hookSpecificOutput"`
+ }
+ if json.Unmarshal([]byte(s), &j) != nil {
+ if event == UserPromptSubmit || event == SessionStart {
+ *contexts = append(*contexts, s)
+ }
+ return
+ }
+ if j.SystemMessage != "" {
+ out.Messages = append(out.Messages, j.SystemMessage)
+ }
+ if j.Continue != nil && !*j.Continue {
+ out.Block = true
+ *reasons = append(*reasons, firstNonEmpty(j.StopReason, j.Reason, "stopped by hook"))
+ }
+ switch j.Decision {
+ case "block":
+ out.Block = true
+ *reasons = append(*reasons, firstNonEmpty(j.Reason, "blocked by hook"))
+ case "approve":
+ out.Decision = stricter(out.Decision, "allow")
+ }
+ if d := j.Specific.PermissionDecision; d == "allow" || d == "deny" || d == "ask" {
+ out.Decision = stricter(out.Decision, d)
+ if d == "deny" {
+ *reasons = append(*reasons, firstNonEmpty(j.Specific.PermissionDecisionReason, "denied by PreToolUse hook"))
+ }
+ }
+ if j.Specific.AdditionalContext != "" {
+ *contexts = append(*contexts, j.Specific.AdditionalContext)
+ }
+}
+
+// stricter keeps the more restrictive permission decision.
+func stricter(a, b string) string {
+ rank := map[string]int{"": 0, "allow": 1, "ask": 2, "deny": 3}
+ if rank[b] > rank[a] {
+ return b
+ }
+ return a
+}
+
+type execResult struct {
+ code int
+ stdout, stderr string
+ err error
+}
+
+const maxHookOutput = 64 << 10
+
+func (r *Runner) exec(ctx context.Context, h settings.HookCommand, payload []byte) execResult {
+ to := time.Duration(h.Timeout) * time.Second
+ if to <= 0 {
+ to = 60 * time.Second
+ }
+ ctx, cancel := context.WithTimeout(ctx, to)
+ defer cancel()
+ c := exec.CommandContext(ctx, "sh", "-c", h.Command)
+ c.Dir = r.Dir
+ c.Env = append(os.Environ(), "COMETCLI_PROJECT_DIR="+r.Dir, "CLAUDE_PROJECT_DIR="+r.Dir)
+ c.Stdin = bytes.NewReader(payload)
+ isolate(c)
+ c.WaitDelay = time.Second // don't wait on pipes a killed hook left open
+ var so, se bytes.Buffer
+ c.Stdout, c.Stderr = &so, &se
+ err := c.Run()
+ res := execResult{stdout: trim(so.String()), stderr: trim(se.String())}
+ var ee *exec.ExitError
+ switch {
+ case err == nil:
+ case ctx.Err() != nil:
+ res.code, res.err = -1, fmt.Errorf("timed out after %s", to)
+ case errors.As(err, &ee):
+ res.code = ee.ExitCode()
+ default:
+ res.code, res.err = -1, err
+ }
+ return res
+}
+
+func matches(matcher, tool string) bool {
+ m := strings.TrimSpace(matcher)
+ if m == "" || m == "*" || tool == "" {
+ return true
+ }
+ re, err := regexp.Compile("(?i)^(?:" + m + ")$")
+ if err != nil {
+ return strings.EqualFold(m, tool)
+ }
+ return re.MatchString(tool)
+}
+
+func trim(s string) string {
+ if len(s) > maxHookOutput {
+ return s[:maxHookOutput] + "…[truncated]"
+ }
+ return s
+}
+
+func def(v, d string) string {
+ if v == "" {
+ return d
+ }
+ return v
+}
+
+func firstNonEmpty(vs ...string) string {
+ for _, v := range vs {
+ if v != "" {
+ return v
+ }
+ }
+ return ""
+}
diff --git a/internal/hooks/hooks_test.go b/internal/hooks/hooks_test.go
new file mode 100644
index 0000000..c5c6592
--- /dev/null
+++ b/internal/hooks/hooks_test.go
@@ -0,0 +1,76 @@
+package hooks
+
+import (
+ "context"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "github.com/abhijitkrm/cometcli/internal/settings"
+)
+
+func runner(t *testing.T, event, matcher, script string) *Runner {
+ t.Helper()
+ dir := t.TempDir()
+ p := filepath.Join(dir, "hook.sh")
+ os.WriteFile(p, []byte("#!/bin/sh\n"+script+"\n"), 0o755)
+ return &Runner{Dir: dir, Matchers: map[string][]settings.HookMatcher{
+ event: {{Matcher: matcher, Hooks: []settings.HookCommand{{Type: "command", Command: p, Timeout: 5}}}},
+ }}
+}
+
+func TestExitCodes(t *testing.T) {
+ ctx := context.Background()
+ out := runner(t, PreToolUse, "bash", `echo "no rm here" >&2; exit 2`).Run(ctx, Input{Event: PreToolUse, ToolName: "bash"})
+ if !out.Block || out.Reason != "no rm here" {
+ t.Fatalf("exit 2: %+v", out)
+ }
+ out = runner(t, PreToolUse, "bash", `exit 1`).Run(ctx, Input{Event: PreToolUse, ToolName: "bash"})
+ if out.Block || len(out.Messages) != 1 {
+ t.Fatalf("exit 1 should be non-blocking: %+v", out)
+ }
+ // matcher doesn't fit → not run
+ out = runner(t, PreToolUse, "edit|write", `exit 2`).Run(ctx, Input{Event: PreToolUse, ToolName: "bash"})
+ if out.Block {
+ t.Fatal("hook ran for a non-matching tool")
+ }
+ out = runner(t, PreToolUse, "Bash", `exit 2`).Run(ctx, Input{Event: PreToolUse, ToolName: "bash"})
+ if !out.Block {
+ t.Fatal("matcher should be case-insensitive (Claude Code's Bash)")
+ }
+}
+
+func TestStdinPayloadAndJSONDecisions(t *testing.T) {
+ ctx := context.Background()
+ r := runner(t, PreToolUse, "*", `in=$(cat); case "$in" in *'"command":"rm -rf x"'*) echo '{"hookSpecificOutput":{"permissionDecision":"deny","permissionDecisionReason":"rm blocked"}}';; *) echo '{"hookSpecificOutput":{"permissionDecision":"allow"}}';; esac`)
+ out := r.Run(ctx, Input{Event: PreToolUse, ToolName: "bash", ToolInput: map[string]any{"command": "rm -rf x"}})
+ if out.Decision != "deny" || !strings.Contains(out.Reason, "rm blocked") {
+ t.Fatalf("deny: %+v", out)
+ }
+ out = r.Run(ctx, Input{Event: PreToolUse, ToolName: "bash", ToolInput: map[string]any{"command": "ls"}})
+ if out.Decision != "allow" || out.Block {
+ t.Fatalf("allow: %+v", out)
+ }
+ out = runner(t, UserPromptSubmit, "", `echo "on-call: alice"`).Run(ctx, Input{Event: UserPromptSubmit, Prompt: "hi"})
+ if out.Context != "on-call: alice" {
+ t.Fatalf("context: %+v", out)
+ }
+ out = runner(t, Stop, "", `echo '{"decision":"block","reason":"run the tests first"}'`).Run(ctx, Input{Event: Stop})
+ if !out.Block || out.Reason != "run the tests first" {
+ t.Fatalf("stop block: %+v", out)
+ }
+ out = runner(t, PostToolUse, "", `echo $COMETCLI_PROJECT_DIR > /dev/null; echo '{"systemMessage":"lint ok"}'`).Run(ctx, Input{Event: PostToolUse, ToolName: "edit"})
+ if len(out.Messages) != 1 || out.Messages[0] != "lint ok" {
+ t.Fatalf("systemMessage: %+v", out)
+ }
+}
+
+func TestTimeout(t *testing.T) {
+ r := runner(t, PreToolUse, "", `sleep 10`)
+ r.Matchers[PreToolUse][0].Hooks[0].Timeout = 1
+ out := r.Run(context.Background(), Input{Event: PreToolUse, ToolName: "bash"})
+ if out.Block || len(out.Messages) != 1 || !strings.Contains(out.Messages[0], "timed out") {
+ t.Fatalf("timeout: %+v", out)
+ }
+}
diff --git a/internal/hooks/proc_other.go b/internal/hooks/proc_other.go
new file mode 100644
index 0000000..154e44a
--- /dev/null
+++ b/internal/hooks/proc_other.go
@@ -0,0 +1,7 @@
+//go:build !unix
+
+package hooks
+
+import "os/exec"
+
+func isolate(*exec.Cmd) {}
diff --git a/internal/hooks/proc_unix.go b/internal/hooks/proc_unix.go
new file mode 100644
index 0000000..cdec21c
--- /dev/null
+++ b/internal/hooks/proc_unix.go
@@ -0,0 +1,20 @@
+//go:build unix
+
+package hooks
+
+import (
+ "os/exec"
+ "syscall"
+)
+
+// isolate runs the hook in its own process group and makes cancel kill
+// the whole group, so a hook's children can't outlive its timeout.
+func isolate(c *exec.Cmd) {
+ c.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
+ c.Cancel = func() error {
+ if c.Process == nil {
+ return nil
+ }
+ return syscall.Kill(-c.Process.Pid, syscall.SIGKILL)
+ }
+}
diff --git a/internal/kb/cases/config.yaml b/internal/kb/cases/config.yaml
new file mode 100644
index 0000000..4430eb2
--- /dev/null
+++ b/internal/kb/cases/config.yaml
@@ -0,0 +1,110 @@
+- id: cfg-parse-error
+ title: Config file fails to parse / unknown field
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded, node-exit-config-error]
+ severity: critical
+ match:
+ any: ["config.parse_error == true", "app.parse_error == true", "logs.config_parse > 0"]
+ confirm:
+ - "node.config show config.toml / app.toml; node.logs for the exact error"
+ fix:
+ - "[change] fix the TOML (edit) — compare with a freshly generated config of the same version"
+ test:
+ signals: {logs.config_parse: 2, proc.running: false}
+
+- id: cfg-min-gas-prices-empty
+ title: minimum-gas-prices empty in app.toml
+ kind: advisory
+ severity: medium
+ match:
+ all: ["app.min_gas_prices_empty == true"]
+ causes:
+ - fresh config; the node accepts zero-fee spam (some versions refuse to start)
+ fix:
+ - "[change] set minimum-gas-prices to the chain's recommended value (e.g. 0.025); restart"
+ test:
+ signals: {app.min_gas_prices_empty: true}
+
+- id: cfg-db-backend-mismatch
+ title: config.toml db_backend differs from app.toml app-db-backend
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: high
+ match:
+ all: ["config.db_backend_mismatch == true"]
+ confirm:
+ - "node.config both files; which backend does the existing data use (ls data/*.db — pebble vs goleveldb files)?"
+ fix:
+ - "[change] set both to the backend the data was created with; restart"
+ warnings:
+ - switching backend on existing data does not convert it — the node fails to open or starts empty
+ test:
+ signals: {config.db_backend_mismatch: true, config.db_backend: goleveldb, app.app_db_backend: pebbledb}
+
+- id: cfg-mempool-mismatch
+ title: CometBFT app mempool selected but the app mempool is disabled
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: high
+ match:
+ all: ["config.mempool_mismatch == true"]
+ confirm:
+ - "node.config: config.toml [mempool] type vs app.toml [mempool] max-txs"
+ fix:
+ - "[change] set app.toml mempool.max-txs >= 0 (e.g. 5000), or use the flood mempool in config.toml; restart"
+ test:
+ signals: {config.mempool_mismatch: true, config.mempool_type: app, app.mempool_max_txs: -1}
+
+- id: cfg-validator-api-exposed
+ title: Validator exposes API/gRPC (attack surface, memory load)
+ kind: advisory
+ severity: medium
+ match:
+ all: ["profile.role == validator", "app.api_enable == true"]
+ confirm:
+ - "sec.exposure — bound to 0.0.0.0?"
+ fix:
+ - "[change] disable [api] on the validator or bind to 127.0.0.1; serve queries from sentries/RPC nodes"
+ test:
+ signals: {profile.role: validator, app.api_enable: true}
+
+- id: cfg-validator-pruning-nothing
+ title: Validator keeps all state (pruning = nothing)
+ kind: advisory
+ severity: low
+ match:
+ all: ["profile.role == validator", "app.pruning == nothing"]
+ fix:
+ - "[change] pruning=custom keep-recent=100 interval=10 on validators (archive belongs on dedicated nodes)"
+ test:
+ signals: {profile.role: validator, app.pruning: nothing}
+
+- id: cfg-no-persistent-peers
+ title: No persistent peers or seeds configured
+ kind: advisory
+ severity: medium
+ match:
+ all: ["config.persistent_peers == 0", "config.seeds == 0"]
+ fix:
+ - "[change] net.add-peer a few reliable peers (sentries for validators)"
+ test:
+ signals: {config.persistent_peers: 0, config.seeds: 0}
+
+- id: cfg-double-sign-check-off
+ title: double_sign_check_height disabled on a validator
+ kind: advisory
+ severity: low
+ match:
+ all: ["profile.role == validator", "config.double_sign_check_height == 0"]
+ fix:
+ - "[change] set consensus.double_sign_check_height = 10: the node refuses to start signing if its key signed recently elsewhere"
+ test:
+ signals: {profile.role: validator, config.double_sign_check_height: 0}
+
+- id: cfg-mempool-full
+ title: Mempool full — txs rejected
+ severity: medium
+ match:
+ all: ["logs.mempool_full > 0"]
+ fix:
+ - "[read] evm.txpool / unconfirmed txs; is the chain congested or is this node stuck?"
+ - "[change] raise mempool size (config.toml [mempool] size or app max-txs) if resources allow; restart"
+ test:
+ signals: {logs.mempool_full: 50}
diff --git a/internal/kb/cases/evm.yaml b/internal/kb/cases/evm.yaml
new file mode 100644
index 0000000..df6933f
--- /dev/null
+++ b/internal/kb/cases/evm.yaml
@@ -0,0 +1,80 @@
+- id: evm-rpc-down
+ title: EVM JSON-RPC not responding
+ severity: high
+ chains: [cosmos-evm]
+ match:
+ all: ["evm.reachable == false", "node.reachable == true"]
+ confirm:
+ - "node.config app.toml [json-rpc] enable/address; sec.exposure for 8545; logs.evm_rpc samples"
+ causes:
+ - json-rpc disabled or bound to 127.0.0.1 while queried remotely; port conflict; indexer error
+ fix:
+ - "[change] enable [json-rpc] and set address (0.0.0.0:8545 only behind a proxy/firewall); restart"
+ verify:
+ - "evm.syncing / evm.chainid respond"
+ test:
+ signals: {evm.reachable: false, node.reachable: true}
+
+- id: evm-json-rpc-disabled
+ title: JSON-RPC disabled in app.toml on an RPC node
+ kind: advisory
+ severity: medium
+ chains: [cosmos-evm]
+ match:
+ all: ["app.json_rpc_enable == false", "profile.role != validator"]
+ fix:
+ - "[change] set [json-rpc] enable = true; restart"
+ test:
+ signals: {app.json_rpc_enable: false, profile.role: rpc}
+
+- id: evm-height-drift
+ title: EVM RPC height lags the CometBFT height
+ severity: medium
+ chains: [cosmos-evm]
+ match:
+ any: ["evm.drift > 5", "evm.drift < -5"]
+ confirm:
+ - "evm.parity — compare eth_blockNumber with comet height twice"
+ causes:
+ - EVM indexer behind or stuck, JSON-RPC served from another node behind a load balancer
+ fix:
+ - "[change] restart the node; if persistent, check [json-rpc] enable-indexer and indexer logs"
+ test:
+ signals: {evm.drift: 120}
+
+- id: evm-chain-id-mismatch
+ title: EVM chain id differs from app.toml evm-chain-id
+ severity: high
+ chains: [cosmos-evm]
+ match:
+ all: ["evm.chain_id_mismatch == true"]
+ confirm:
+ - "evm.chainid vs app.toml [evm] evm-chain-id vs the chain's documented EVM chain id"
+ fix:
+ - "[change] set the documented evm-chain-id in app.toml; restart (wallets sign with the chain id — a mismatch breaks txs)"
+ test:
+ signals: {evm.chain_id_mismatch: true}
+
+- id: evm-rpc-errors
+ title: EVM JSON-RPC / indexer errors in logs
+ severity: medium
+ chains: [cosmos-evm]
+ match:
+ all: ["logs.evm_rpc > 10"]
+ fix:
+ - "[read] node.logs grep=json-rpc; kb.search the error text"
+ test:
+ signals: {logs.evm_rpc: 40}
+
+- id: evm-validator-json-rpc-public
+ title: Validator serves EVM JSON-RPC
+ kind: advisory
+ severity: medium
+ chains: [cosmos-evm]
+ match:
+ all: ["profile.role == validator", "app.json_rpc_enable == true"]
+ any: ["app.json_rpc_address contains 0.0.0.0"]
+ fix:
+ - "[change] disable JSON-RPC on validators or bind it to 127.0.0.1; serve RPC from dedicated nodes"
+ test:
+ signals: {profile.role: validator, app.json_rpc_enable: true, app.json_rpc_address: "0.0.0.0:8545"}
diff --git a/internal/kb/cases/host.yaml b/internal/kb/cases/host.yaml
new file mode 100644
index 0000000..67a98a5
--- /dev/null
+++ b/internal/kb/cases/host.yaml
@@ -0,0 +1,153 @@
+- id: host-disk-full
+ title: Disk full or nearly full on the node data volume
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded, state-db-corrupt, host-disk-high, host-disk-free-low]
+ severity: critical
+ match:
+ any: ["host.disk_used_pct >= 95", "logs.disk_io > 0"]
+ confirm:
+ - "bash: df -h ; du -sh /data/* (largest dirs)"
+ causes:
+ - unpruned state (app.pruning=nothing/default on a validator), old snapshots, huge logs, cosmovisor backups
+ fix:
+ - "[read] snap.prune advice; snap.list"
+ - "[change] remove old snapshots / rotated logs / cosmovisor backup dirs (operator confirms each delete)"
+ - "[change] set app.toml pruning=custom (keep-recent 100, interval 10) for validators; restart"
+ - "[change] longer-term: grow the volume"
+ verify:
+ - "host.disk_used_pct < 85"
+ warnings:
+ - never delete data/priv_validator_state.json or anything under config/
+ - never delete application.db/blockstore.db on a validator to free space
+ test:
+ signals: {host.disk_used_pct: 98}
+
+- id: host-disk-high
+ title: Disk usage high (80-95%)
+ severity: medium
+ match:
+ all: ["host.disk_used_pct >= 80", "host.disk_used_pct < 95"]
+ fix:
+ - "[read] snap.prune; du of data dirs"
+ - "[change] plan pruning / cleanup / resize before it fills"
+ test:
+ signals: {host.disk_used_pct: 88}
+
+- id: host-inodes-exhausted
+ title: Inodes exhausted
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: critical
+ match:
+ all: ["host.inode_used_pct >= 95"]
+ confirm:
+ - "bash: df -i; find -xdev -type f | cut -d/ -f1-5 | sort | uniq -c | sort -n | tail"
+ fix:
+ - "[change] remove the directory with millions of small files (often old snapshot chunks or wasm cache)"
+ test:
+ signals: {host.inode_used_pct: 100}
+
+- id: host-oom-killed
+ title: Node killed for out-of-memory
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded, host-memory-pressure]
+ severity: critical
+ match:
+ any: ["proc.oom_killed == true", "logs.oom > 0", "proc.oom_kills > 0"]
+ confirm:
+ - "bash: dmesg -T | grep -i oom (or journalctl -k); host.mem_used_pct"
+ causes:
+ - too little RAM, iavl cache too large, large query load on a validator (API/gRPC open), memory leak in a version
+ fix:
+ - "[change] lower app.toml iavl-cache-size, disable public API/gRPC on validators, add swap as a stopgap, add RAM"
+ - "[change] node.service restart"
+ verify:
+ - "proc.running and memory stable for an hour"
+ test:
+ signals: {proc.oom_killed: true}
+
+- id: host-memory-pressure
+ title: Memory pressure (>90% used)
+ severity: medium
+ match:
+ all: ["host.mem_used_pct >= 90"]
+ fix:
+ - "[read] bash: ps aux --sort=-rss | head"
+ - "[change] reduce caches / other processes; add RAM"
+ test:
+ signals: {host.mem_used_pct: 94}
+
+- id: host-overloaded
+ title: Host CPU overloaded
+ explains: [node-consensus-timeouts, val-signing-degraded, node-blocksync-slow]
+ severity: medium
+ match:
+ all: ["host.load_per_cpu >= 2"]
+ fix:
+ - "[read] bash: top -bn1 | head -20"
+ - "[change] move other workloads off the validator host"
+ test:
+ signals: {host.load_per_cpu: 3.5}
+
+- id: host-clock-unsynced
+ title: System clock not NTP-synchronized
+ explains: [node-consensus-timeouts, val-signing-degraded, val-jail-imminent]
+ severity: high
+ match:
+ all: ["clock.ntp_synced == false"]
+ confirm:
+ - "bash: timedatectl; chronyc tracking"
+ fix:
+ - "[change] enable chrony/systemd-timesyncd: timedatectl set-ntp true"
+ verify:
+ - "clock.ntp_synced true"
+ test:
+ signals: {clock.ntp_synced: false}
+
+- id: host-too-many-files
+ title: Too many open files
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded, node-low-peers, node-no-peers]
+ severity: high
+ match:
+ all: ["logs.too_many_files > 0"]
+ confirm:
+ - "bash: cat /proc/$(pgrep -f )/limits | grep 'open files'"
+ fix:
+ - "[change] systemd: LimitNOFILE=65535 in the unit; docker: --ulimit nofile=65535:65535; restart"
+ test:
+ signals: {logs.too_many_files: 10}
+
+- id: host-port-in-use
+ title: Node can't bind a port (address already in use)
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: high
+ match:
+ all: ["logs.port_in_use > 0"]
+ confirm:
+ - "bash: ss -ltnp | grep -E '26656|26657|9090|1317|8545'"
+ causes:
+ - a second instance of the node running, another service on the port
+ fix:
+ - "[change] stop the duplicate (CHECK it is not a second validator instance with the same key) or change the port"
+ warnings:
+ - two instances of a validator = double-sign risk; stop one immediately
+ test:
+ signals: {logs.port_in_use: 2}
+
+- id: container-memory-limit
+ title: Container memory limit too low for the node (OOM kills inside the container)
+ explains: [node-crash-loop, node-process-down, node-rpc-down, node-restarted-recently, node-killed-sigkill, host-oom-killed, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, node-catching-up, evm-rpc-down]
+ severity: critical
+ symptoms: restarts with no error in the logs; docker OOMKilled may be false because the node runs under a wrapper shell
+ match:
+ all: ["proc.mem_limit_mb < 4096"]
+ any: ["proc.oom_kills > 0", "proc.mem_used_pct_of_limit >= 90", "proc.restarts >= 3"]
+ confirm:
+ - "docker inspect -f '{{.HostConfig.Memory}}' (bytes; 0 = unlimited) and docker stats --no-stream"
+ - "a node needs several GB under load — a limit of a few hundred MB will always be killed"
+ causes:
+ - memory limit set in compose (mem_limit / deploy.resources.limits.memory) or with docker update
+ fix:
+ - "[change] raise or remove the limit: docker update --memory g --memory-swap g (and fix compose so it survives recreation)"
+ - "[change] restart the container if it's not recovering on its own"
+ verify:
+ - "proc.restarts stops increasing; proc.mem_used_pct_of_limit well below 90; wait.until synced"
+ test:
+ signals: {proc.mem_limit_mb: 120, proc.oom_kills: 14, proc.restarts: 7}
diff --git a/internal/kb/cases/more.yaml b/internal/kb/cases/more.yaml
new file mode 100644
index 0000000..2bd4945
--- /dev/null
+++ b/internal/kb/cases/more.yaml
@@ -0,0 +1,158 @@
+- id: val-signing-degraded
+ title: Validator intermittently missing blocks
+ severity: medium
+ match:
+ all: ["val.in_set == true", "val.signed_recent_pct >= 50", "val.signed_recent_pct < 90"]
+ confirm:
+ - "val.consensus window=100 — pattern of misses; host.load_per_cpu, logs.clock, node.peers"
+ causes:
+ - overloaded host or slow disk, flaky network to other validators, clock drift, remote signer latency
+ fix:
+ - "[read] node.consensus round info; bash: iostat/top on the host"
+ - "[change] remove load / improve peers (sentries) / fix NTP"
+ verify:
+ - "val.signed_recent_pct >= 98 over a few hundred blocks"
+ test:
+ signals: {val.in_set: true, val.signed_recent_pct: 75}
+
+- id: val-in-set-while-syncing
+ title: Validator in the active set while its node is catching up
+ severity: critical
+ symptoms: every block missed until synced; jail risk grows each block
+ match:
+ all: ["val.in_set == true", "node.catching_up == true"]
+ fix:
+ - "[read] wait.until synced and watch val.downtime_budget_used_pct"
+ - "[change] if it won't make it in time, the fastest sync path (snap.statesync on a NEW data dir) — never a second signer"
+ warnings:
+ - do not bring up a backup node with the same key while this one runs
+ test:
+ signals: {val.in_set: true, node.catching_up: true}
+
+- id: node-restarted-recently
+ title: Node restarted in the last few minutes
+ severity: low
+ match:
+ all: ["proc.uptime_s < 300", "proc.restarts > 0"]
+ confirm:
+ - "node.logs from before the restart (since=15m) — crash or manual?"
+ fix:
+ - "[read] find who/what restarted it; watch for a crash loop (re-triage in 10 min)"
+ test:
+ signals: {proc.uptime_s: 120, proc.restarts: 1}
+
+- id: node-killed-sigkill
+ title: Node exited with 137 (SIGKILL — OOM killer or forced stop)
+ explains: [node-process-down, node-rpc-down, evm-rpc-down, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: high
+ match:
+ all: ["proc.running == false", "proc.exit_code == 137"]
+ confirm:
+ - "bash: dmesg -T | grep -i -E 'killed process|oom'; docker events / journalctl for a stop timeout"
+ causes:
+ - kernel OOM killer; a stop that exceeded the stop timeout (container stop -t / TimeoutStopSec)
+ fix:
+ - "[change] OOM: follow host-oom-killed; stop timeout: raise it so the node flushes state on shutdown"
+ test:
+ signals: {proc.running: false, proc.exit_code: 137}
+
+- id: node-exit-config-error
+ title: Node exits at startup with a config / flag error
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: critical
+ match:
+ all: ["proc.running == false", "logs.config_parse > 0"]
+ fix:
+ - "[read] node.logs — the exact field/flag"
+ - "[change] fix config (edit) or the service's command flags; start"
+ test:
+ signals: {proc.running: false, logs.config_parse: 1, proc.exit_code: 1}
+
+- id: host-disk-free-low
+ title: Less than 20 GB free on the data volume
+ severity: high
+ match:
+ all: ["host.disk_free_gb < 20"]
+ fix:
+ - "[read] snap.prune; du of data dirs"
+ - "[change] prune / clean / grow the volume before the next state growth or upgrade migration"
+ warnings:
+ - upgrades often run store migrations that need extra free space
+ test:
+ signals: {host.disk_free_gb: 8}
+
+- id: cfg-grpc-disabled
+ title: gRPC disabled in app.toml (queries and cometcli chain tools fail)
+ severity: medium
+ match:
+ all: ["app.grpc_enable == false"]
+ fix:
+ - "[change] [grpc] enable = true, address 127.0.0.1:9090 on validators; restart"
+ test:
+ signals: {app.grpc_enable: false}
+
+- id: cfg-prometheus-off
+ title: Prometheus metrics disabled
+ kind: advisory
+ severity: low
+ match:
+ all: ["config.prometheus == false", "profile.role == validator"]
+ fix:
+ - "[change] [instrumentation] prometheus = true (listen on localhost or a private interface) and scrape it"
+ test:
+ signals: {config.prometheus: false, profile.role: validator}
+
+- id: val-local-privval-errors
+ title: Local priv_validator errors (file key)
+ explains: [val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: high
+ match:
+ all: ["config.remote_signer == false", "logs.privval > 0"]
+ confirm:
+ - "sec.perms on config/priv_validator_key.json and data/priv_validator_state.json (owner, mode, read-only fs?)"
+ causes:
+ - key or state file unreadable / not writable by the node user (ownership, mode, read-only filesystem)
+ - "'no SignBytes found': priv_validator_state.json was hand-edited (or restored) to a height/round/step without its sign bytes — the signer refuses that exact HRS and recovers on the next height"
+ fix:
+ - "[change] fix ownership/permissions so the node user can read the key and write the state file; restart"
+ warnings:
+ - never edit or reset priv_validator_state.json to silence errors
+ test:
+ signals: {config.remote_signer: false, logs.privval: 6}
+
+- id: evm-indexer-behind
+ title: EVM endpoint reports syncing while the node is synced
+ severity: medium
+ chains: [cosmos-evm]
+ match:
+ all: ["evm.syncing == true", "node.catching_up == false"]
+ fix:
+ - "[read] evm.syncing details; logs.evm_rpc samples"
+ - "[change] restart the node; if persistent, rebuild the EVM indexer per the chain docs"
+ test:
+ signals: {evm.syncing: true, node.catching_up: false}
+
+- id: upgrade-scheduled
+ title: Upgrade scheduled (more than ~1 day away)
+ kind: advisory
+ severity: low
+ match:
+ all: ["upgrade.pending == true", "upgrade.blocks_to_halt >= 20000"]
+ fix:
+ - "[read] upgrade.check — release notes, required version, config changes"
+ - "[change] upgrade.prepare well before the height"
+ test:
+ signals: {upgrade.pending: true, upgrade.blocks_to_halt: 80000}
+
+- id: node-behind-but-not-reporting
+ title: Node far behind while reporting catching_up=false and peers present
+ severity: high
+ match:
+ all: ["node.stalled == true", "node.peers > 0"]
+ any: ["logs.apphash > 0", "logs.panic > 0", "logs.db_corrupt > 0", "logs.disk_io > 0"]
+ confirm:
+ - "node.logs around the stall height — the block it is stuck on"
+ fix:
+ - "[read] kb.show the matching state/host case (apphash, db corruption, disk)"
+ test:
+ signals: {node.stalled: true, node.peers: 6, logs.apphash: 2}
diff --git a/internal/kb/cases/state.yaml b/internal/kb/cases/state.yaml
new file mode 100644
index 0000000..185a61b
--- /dev/null
+++ b/internal/kb/cases/state.yaml
@@ -0,0 +1,85 @@
+- id: state-apphash-mismatch
+ title: App hash mismatch (state diverged from the network)
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded, upgrade-version-behind]
+ severity: critical
+ symptoms: wrong Block.Header.AppHash / CONSENSUS FAILURE; node halts on a block
+ match:
+ any: ["logs.apphash > 0"]
+ confirm:
+ - "node.logs grep -i apphash: at which height? node.version-check — same version as the network?"
+ causes:
+ - wrong binary version (missed or early upgrade, non-deterministic build), different config affecting state (e.g. iavl/fastnode), corrupt state
+ fix:
+ - "[read] verify the exact binary version the network runs at that height"
+ - "[change] install the correct binary; if state is corrupt, restore from a trusted snapshot or snap.statesync"
+ warnings:
+ - rollback commands change state — operator only; keep priv_validator_state.json
+ test:
+ signals: {logs.apphash: 5, logs.panic: 1}
+
+- id: state-db-locked
+ title: Database locked by another process
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: high
+ match:
+ all: ["logs.db_lock > 0"]
+ confirm:
+ - "bash: ps aux | grep — two instances? lsof /data/*/LOCK"
+ fix:
+ - "[change] stop the duplicate process (verify it is not a second validator instance first)"
+ test:
+ signals: {logs.db_lock: 3}
+
+- id: state-db-corrupt
+ title: Database corruption
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: critical
+ match:
+ all: ["logs.db_corrupt > 0"]
+ any: ["logs.panic > 0", "proc.running == false", "proc.restarts > 0"]
+ confirm:
+ - "node.logs — which db (application, blockstore, state)?"
+ causes:
+ - unclean shutdown, disk errors (logs.disk_io), full disk
+ fix:
+ - "[change] restore data from a trusted snapshot or snap.statesync (operator confirms)"
+ warnings:
+ - preserve data/priv_validator_state.json across any restore
+ test:
+ signals: {logs.db_corrupt: 4, logs.panic: 1}
+
+- id: state-statesync-failing
+ title: State sync failing
+ severity: high
+ match:
+ all: ["logs.statesync_fail > 0"]
+ confirm:
+ - "node.config statesync section; are rpc_servers reachable and serving snapshots? trust height recent?"
+ causes:
+ - trust height/hash stale or wrong, rpc servers without snapshots, snapshot format mismatch, too few peers with snapshots
+ fix:
+ - "[change] snap.statesync to refresh trust height/hash from a trusted RPC; add peers that serve snapshots; restart"
+ test:
+ signals: {logs.statesync_fail: 8, config.statesync_enable: true}
+
+- id: state-statesync-left-enabled
+ title: statesync.enable left on after the node synced
+ kind: advisory
+ severity: low
+ match:
+ all: ["config.statesync_enable == true", "node.catching_up == false"]
+ fix:
+ - "[change] set [statesync] enable=false so a future restart with empty data doesn't surprise you"
+ test:
+ signals: {config.statesync_enable: true, node.catching_up: false}
+
+- id: state-no-snapshots-served
+ title: Node does not produce state-sync snapshots
+ kind: advisory
+ severity: low
+ match:
+ all: ["app.snapshot_interval == 0", "profile.role != validator"]
+ fix:
+ - "[change] RPC/sentry nodes: set state-sync snapshot-interval (e.g. 1000) and keep-recent to help peers"
+ test:
+ signals: {app.snapshot_interval: 0, profile.role: rpc}
diff --git a/internal/kb/cases/sync.yaml b/internal/kb/cases/sync.yaml
new file mode 100644
index 0000000..d35524c
--- /dev/null
+++ b/internal/kb/cases/sync.yaml
@@ -0,0 +1,231 @@
+- id: node-rpc-down
+ title: Node RPC unreachable (process down or RPC disabled)
+ explains: [evm-rpc-down, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: critical
+ match:
+ all: ["node.reachable == false"]
+ confirm:
+ - "node.health; node.service status"
+ - "proc.running and proc.exit_code; node.logs tail"
+ causes:
+ - process crashed/exited (see logs.panic, logs.config_parse, logs.port_in_use)
+ - RPC laddr changed or bound to another interface
+ - host unreachable
+ fix:
+ - "[read] node.logs last 200 lines for the exit reason"
+ - "[change] fix the reason, then node.service restart"
+ verify:
+ - "node.health ok; wait.until synced"
+ test:
+ signals: {node.reachable: false}
+
+- id: node-process-down
+ title: Node process/container not running
+ explains: [node-rpc-down, evm-rpc-down, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: critical
+ match:
+ all: ["proc.running == false"]
+ confirm:
+ - "node.service status; node.logs (last lines before exit); proc.exit_code"
+ causes:
+ - crash (panic), OOM kill, bad config/flag, port in use, disk full, upgrade halt (exits by design)
+ fix:
+ - "[read] node.logs to find why it exited — do not restart blindly"
+ - "[change] fix the reason; node.service start"
+ verify:
+ - "proc.running true and stays up >5 min; wait.until synced"
+ warnings:
+ - an upgrade halt exits on purpose — restarting the old binary re-halts; follow upgrade-halt-binary-missing
+ test:
+ signals: {proc.running: false, proc.state: exited}
+
+- id: node-crash-loop
+ title: Node crash-looping (restarting repeatedly)
+ explains: [node-process-down, node-rpc-down, evm-rpc-down, node-restarted-recently, node-killed-sigkill, node-catching-up, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: critical
+ match:
+ all: ["proc.restarts >= 3", "proc.uptime_s < 600"]
+ confirm:
+ - "node.logs for the panic/exit message repeated each start"
+ causes:
+ - panic on a block (app hash / bug → needs a fixed binary), bad config, corrupt DB, upgrade halt with old binary
+ fix:
+ - "[read] node.logs; kb.search the panic message"
+ - "[change] apply the matching fix; stop the restart loop if it makes things worse (node.service stop)"
+ test:
+ signals: {proc.restarts: 12, proc.uptime_s: 40, logs.panic: 5}
+
+- id: node-catching-up
+ title: Node catching up (block sync in progress)
+ explains: [val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: medium
+ match:
+ all: ["node.catching_up == true"]
+ confirm:
+ - "node.status twice ~30s apart: is the height increasing? how fast?"
+ causes:
+ - restarted after downtime, restored from snapshot, slow disk/peers
+ fix:
+ - "[read] wait.until synced (progress shows blocks/s and ETA)"
+ - "[change] if far behind (days), consider snap.statesync instead of replaying"
+ verify:
+ - "node.catching_up false, node.block_age_s < 30"
+ test:
+ signals: {node.catching_up: true, node.block_age_s: 7200}
+
+- id: node-stalled
+ title: Node stalled — not catching up but no new blocks
+ explains: [val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: critical
+ symptoms: catching_up=false but latest block is minutes old
+ match:
+ all: ["node.stalled == true"]
+ confirm:
+ - "node.status twice; node.peers; is the whole chain halted? compare a public RPC (web_fetch /status)"
+ causes:
+ - whole chain halted (upgrade height, consensus failure) — check upgrade.blocks_to_halt, logs.upgrade_needed
+ - this node lost peers or is stuck on a bad block / app hash
+ fix:
+ - "[read] decide: chain-wide halt vs local stall (public RPC height)"
+ - "[change] local: fix peers (net.add-peer), restart; chain-wide: follow the chain's coordination channel"
+ verify:
+ - "node.block_age_s < 30"
+ test:
+ signals: {node.stalled: true, node.block_age_s: 900, node.catching_up: false}
+
+- id: node-no-peers
+ title: Node has no peers
+ explains: [node-stalled, node-catching-up, node-blocksync-slow, node-peer-churn, node-p2p-port-exposure, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: critical
+ match:
+ all: ["node.peers == 0"]
+ confirm:
+ - "node.peers; logs.peer_net / logs.p2p_auth samples; sec.exposure (26656 reachable?)"
+ causes:
+ - persistent_peers/seeds empty or stale, firewall blocks 26656, wrong chain-id/genesis (peers reject), external_address wrong
+ fix:
+ - "[read] get current peers from the chain's registry or a public RPC /net_info"
+ - "[change] net.add-peer (2-5 healthy peers); node.service restart"
+ verify:
+ - "node.peers >= 3 and node.block_age_s < 30"
+ test:
+ signals: {node.peers: 0}
+
+- id: node-low-peers
+ title: Node has very few peers
+ severity: medium
+ match:
+ all: ["node.peers > 0", "node.peers < 3"]
+ fix:
+ - "[change] net.add-peer more persistent peers; check config.pex and max_inbound_peers"
+ test:
+ signals: {node.peers: 1}
+
+- id: node-peer-churn
+ title: Heavy peer connection errors
+ severity: low
+ match:
+ all: ["logs.peer_net > 200"]
+ confirm:
+ - "node.peers count stable? if peers OK this is usually harmless noise"
+ fix:
+ - "[change] remove dead persistent peers (net.rm-peer) and add healthy ones"
+ test:
+ signals: {logs.peer_net: 900, node.peers: 8}
+
+- id: node-wrong-network
+ title: Peers reject us — different network / genesis
+ explains: [node-no-peers, node-low-peers, node-stalled, node-catching-up, node-p2p-port-exposure, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: critical
+ match:
+ any: ["logs.chain_mismatch > 0"]
+ confirm:
+ - "node.status network vs profile chain_id; sha256 of config/genesis.json vs the official one"
+ causes:
+ - wrong genesis.json, wrong chain-id in config/client, joined testnet peers on mainnet
+ fix:
+ - "[read] compare genesis checksum with the chain's official genesis (web_fetch)"
+ - "[change] replace genesis with the official one (operator confirms), reset is NOT done by the agent"
+ warnings:
+ - never run unsafe-reset-all on a validator without the operator — it is refused by cometcli
+ test:
+ signals: {logs.chain_mismatch: 30, node.peers: 0}
+
+- id: node-p2p-port-exposure
+ title: Peers cannot dial us (p2p port not reachable)
+ severity: medium
+ match:
+ all: ["node.peers < 3", "config.pex == true"]
+ any: ["logs.p2p_auth > 0", "logs.peer_net > 0"]
+ confirm:
+ - "sec.exposure — is 26656 listening on a public interface? check external_address in config.toml"
+ fix:
+ - "[change] set p2p.external_address, open firewall for 26656, restart"
+ test:
+ signals: {node.peers: 1, config.pex: true, logs.p2p_auth: 20}
+
+- id: node-consensus-timeouts
+ title: Consensus round timeouts / slow votes
+ severity: medium
+ match:
+ all: ["logs.clock > 20", "node.catching_up == false"]
+ confirm:
+ - "node.consensus — round > 0 often? clock.ntp_synced? host.load_per_cpu?"
+ causes:
+ - clock skew, overloaded host, slow disk, poor connectivity to other validators
+ fix:
+ - "[change] fix NTP / load / peers; consider a sentry topology"
+ test:
+ signals: {logs.clock: 50, node.catching_up: false}
+
+- id: node-blocksync-slow
+ title: Catching up too slowly to ever reach the tip
+ explains: [node-catching-up, val-not-signing, val-jail-imminent]
+ severity: high
+ match:
+ all: ["node.catching_up == true", "node.block_age_s > 86400"]
+ confirm:
+ - "two node.status calls 60s apart: blocks/s vs chain block rate"
+ fix:
+ - "[change] snap.statesync (or restore a trusted snapshot) instead of replaying"
+ - "[change] faster disk (NVMe), more peers"
+ test:
+ signals: {node.catching_up: true, node.block_age_s: 400000}
+
+- id: node-network-detached
+ title: Node container has no network attached
+ explains: [node-rpc-down, node-rpc-path-broken, node-no-peers, node-low-peers, node-stalled, node-catching-up, evm-rpc-down, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent]
+ severity: critical
+ symptoms: process running, but no peers and its published ports are gone
+ match:
+ all: ["proc.networks == 0", "proc.running == true"]
+ confirm:
+ - "docker inspect -f '{{json .NetworkSettings.Networks}}' → {}"
+ - "which network should it be on? (compose file / the other nodes' docker inspect)"
+ causes:
+ - docker network disconnect (manual or tooling), network removed/recreated under a running container
+ fix:
+ - "[change] docker network connect [--ip ] — keep the IP if peers dial it by address"
+ - "[change] or recreate it with docker compose up -d "
+ verify:
+ - "node.reachable true, node.peers >= 2, wait.until synced"
+ test:
+ signals: {proc.networks: 0, proc.running: true, node.reachable: false}
+
+- id: node-rpc-path-broken
+ title: Node healthy inside its container but unreachable from the host
+ explains: [node-rpc-down, evm-rpc-down]
+ severity: high
+ symptoms: host RPC calls fail while the node answers on localhost inside the container
+ match:
+ all: ["node.unreachable_from_host == true"]
+ confirm:
+ - "docker port ; rpc laddr in config.toml (must bind 0.0.0.0 inside a container); host firewall"
+ causes:
+ - port mapping lost or changed, laddr bound to 127.0.0.1 inside the container, host firewall, network detached
+ fix:
+ - "[change] fix the port mapping / laddr (tcp://0.0.0.0:26657) and recreate or restart the container"
+ verify:
+ - "node.reachable true from the host"
+ test:
+ signals: {node.unreachable_from_host: true, node.reachable: false}
diff --git a/internal/kb/cases/upgrade.yaml b/internal/kb/cases/upgrade.yaml
new file mode 100644
index 0000000..deb89a3
--- /dev/null
+++ b/internal/kb/cases/upgrade.yaml
@@ -0,0 +1,97 @@
+- id: upgrade-halt-binary-missing
+ title: Chain halted at an upgrade height — new binary not running
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded, state-apphash-mismatch]
+ severity: critical
+ symptoms: logs show UPGRADE "" NEEDED at height; node stopped or crash-looping
+ match:
+ any: ["logs.upgrade_needed > 0", "upgrade.blocks_to_halt <= 0"]
+ confirm:
+ - "logs.upgrade_name; chain.upgrade-plan; upgrade.check (local version vs required)"
+ - "is the whole chain halted? (other validators' heights / a public RPC) — an upgrade halt is chain-wide"
+ fix:
+ - "[read] upgrade.check — find the release for the plan name"
+ - "[change] upgrade.prepare (download + checksum verify + stage for cosmovisor, or swap the binary/image)"
+ - "[change] node.service restart with the new binary"
+ - "[change] ONLY if the chain's coordinators announce skipping this upgrade: restart with --unsafe-skip-upgrades (needs ≥2/3 of voting power doing the same); remove the flag afterwards"
+ verify:
+ - "node.block_age_s < 30 after the network reaches 2/3; val.consensus"
+ warnings:
+ - never skip the upgrade by restarting the old binary with --unsafe-skip-upgrades unless the chain coordinators say so
+ test:
+ signals: {logs.upgrade_needed: 3, logs.upgrade_name: v2.0.0, proc.restarts: 4}
+
+- id: upgrade-imminent
+ title: Scheduled upgrade soon — binary must be staged
+ severity: high
+ match:
+ all: ["upgrade.pending == true", "upgrade.blocks_to_halt > 0", "upgrade.blocks_to_halt < 20000"]
+ confirm:
+ - "upgrade.check — is the new binary staged (cosmovisor upgrades//bin)?"
+ fix:
+ - "[change] upgrade.prepare now"
+ - "[read] upgrade.watch near the height"
+ verify:
+ - "upgrade.check shows staged and checksum-verified"
+ test:
+ signals: {upgrade.pending: true, upgrade.blocks_to_halt: 5000, upgrade.name: v2.0.0}
+
+- id: upgrade-binary-too-early
+ title: New binary started before the upgrade height
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded, state-apphash-mismatch]
+ severity: critical
+ match:
+ any: ["logs.binary_early > 0"]
+ confirm:
+ - "node.logs grep 'BINARY UPDATED BEFORE TRIGGER'; upgrade.check"
+ fix:
+ - "[change] switch back to the previous binary until the upgrade height, then switch"
+ test:
+ signals: {logs.binary_early: 1}
+
+- id: upgrade-halt-height-set
+ title: halt-height configured in app.toml
+ explains: [node-process-down, node-rpc-down, node-crash-loop, node-stalled, node-catching-up, node-killed-sigkill, node-restarted-recently, node-blocksync-slow, evm-rpc-down, val-in-set-while-syncing, node-behind-but-not-reporting, val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: high
+ match:
+ all: ["app.halt_height > 0"]
+ confirm:
+ - "node.config app.toml halt-height"
+ causes:
+ - left over from a manual coordinated upgrade
+ fix:
+ - "[change] set halt-height = 0 after the coordinated halt is done; restart"
+ test:
+ signals: {app.blocks_to_halt_height: 200, app.halt_height: 1200}
+
+- id: upgrade-version-behind
+ title: Upgrade passed but node still on an older version
+ severity: high
+ match:
+ all: ["upgrade.pending == false", "logs.apphash > 0", "node.catching_up == false"]
+ confirm:
+ - "node.version-check; compare with the version other validators run"
+ fix:
+ - "[change] upgrade.prepare the current release; restart"
+ test:
+ signals: {upgrade.pending: false, logs.apphash: 2, node.catching_up: false}
+
+- id: upgrade-replay-past-upgrade
+ title: Node syncing from an old height halts at an upgrade the chain already passed
+ explains: [upgrade-halt-binary-missing, node-crash-loop, node-process-down, node-rpc-down, node-restarted-recently, node-catching-up, node-stalled, evm-rpc-down, val-jailed-downtime, val-jailed-waiting-period, val-not-signing]
+ severity: critical
+ symptoms: a catching-up node logs UPGRADE "x" NEEDED, but the chain has no pending plan — it is replaying history
+ match:
+ all: ["logs.upgrade_needed > 0", "upgrade.pending == false"]
+ confirm:
+ - "logs.upgrade_name and the halt height vs the chain's current height (far above it)"
+ - "how did the network pass that height: an upgrade binary, or a coordinated --unsafe-skip-upgrades?"
+ causes:
+ - block-syncing from genesis or an old snapshot with only the latest binary
+ fix:
+ - "[change] the network applied the upgrade: run each historical binary up to its upgrade height (cosmovisor with every upgrades//bin staged), or"
+ - "[change] the network skipped it: start with --unsafe-skip-upgrades like the validators did"
+ - "[change] or avoid replay entirely: snap.statesync / a recent trusted snapshot"
+ verify:
+ - "node.height passes the upgrade height; wait.until synced"
+ test:
+ signals: {logs.upgrade_needed: 2, upgrade.pending: false, node.catching_up: true}
diff --git a/internal/kb/cases/validator.yaml b/internal/kb/cases/validator.yaml
new file mode 100644
index 0000000..e2fc879
--- /dev/null
+++ b/internal/kb/cases/validator.yaml
@@ -0,0 +1,280 @@
+- id: val-jailed-downtime
+ title: Validator jailed for downtime (missed too many blocks)
+ severity: critical
+ symptoms: validator jailed, not tombstoned; zero voting power; signing stopped earlier
+ match:
+ all: ["val.jailed == true", "val.tombstoned == false"]
+ confirm:
+ - "val.jail-check — jailed_until, unjail blockers, and log forensics around the jailing time"
+ - "the cause must be found and fixed before unjailing, or it will be jailed again"
+ causes:
+ - node was down, crashed, or OOM-killed (check proc.restarts, proc.oom_killed, logs.panic, logs.oom)
+ - node fell behind / stalled (peers lost, disk full, slow IO)
+ - signer broken (remote signer down, wrong priv_validator_key after migration)
+ - halted at an upgrade height without the new binary
+ fix:
+ - "[read] val.jail-check to get the jail cause from logs and the remaining jail time"
+ - "[change] fix the root cause found (restart the service, free disk, fix peers, install upgrade binary)"
+ - "[read] wait.until synced — never unjail a node that is not caught up"
+ - "[read] wait.until unjailable (jailed_until passed)"
+ - "[tx] val.unjail (asks which signer: cometcli keyring or the container keyring)"
+ - "[read] if the tx fails, read the error (TxHint explains known codes), fix, retry"
+ verify:
+ - "wait.until in-consensus (bonded, in the validator set, signing recent blocks)"
+ - "val.consensus shows signed ≥90% of the last 20 blocks"
+ warnings:
+ - never unjail while catching up — it misses blocks again immediately
+ - never copy priv_validator_key to a second running node to "get back fast" (double sign → tombstone)
+ test:
+ signals: {val.jailed: true, val.tombstoned: false}
+
+- id: val-tombstoned
+ title: Validator tombstoned (double sign) — permanent
+ severity: critical
+ symptoms: validator jailed and tombstoned; cannot be unjailed ever
+ match:
+ all: ["val.tombstoned == true"]
+ confirm:
+ - "val.status / val.jail-check shows tombstoned=true"
+ - "sec.doublesign and logs for duplicate vote evidence around the slash height"
+ causes:
+ - the same consensus key signed on two nodes (failover without stopping the old node, copied key, restored backup while live)
+ - priv_validator_state.json reset or restored to an older height while the key was active elsewhere
+ fix:
+ - "[read] identify every machine holding this priv_validator_key and stop all but none — the key is burned"
+ - "[change] stop the node signing with the tombstoned key to avoid confusion"
+ - "[tx] a NEW validator is required: new consensus key (operator generates it, never the agent), val.create, ask delegators to redelegate"
+ verify:
+ - new validator bonded and signing (val.consensus with the new operator)
+ warnings:
+ - unjail is impossible for a tombstoned validator — do not attempt val.unjail
+ - never reuse the tombstoned consensus key
+ test:
+ signals: {val.tombstoned: true, val.jailed: true}
+
+- id: val-jail-imminent
+ title: Downtime budget nearly exhausted — jail imminent
+ severity: critical
+ symptoms: missed blocks counter close to the slashing window limit
+ match:
+ all: ["val.jailed == false", "val.downtime_budget_used_pct >= 70"]
+ confirm:
+ - "val.signing — missed counter vs window and min_signed_per_window"
+ - "val.consensus — is it signing NOW? if yes, the counter drains as the window slides"
+ causes:
+ - recent outage; node may still be unhealthy
+ fix:
+ - "[read] node.triage signals for the reason it is missing blocks (sync, peers, signer, process)"
+ - "[change] fix that cause immediately — every missed block counts"
+ verify:
+ - "val.signed_recent_pct back to ~100 and val.missed stops increasing"
+ warnings:
+ - do not restart repeatedly — each restart costs blocks while it replays and reconnects
+ test:
+ signals: {val.jailed: false, val.downtime_budget_used_pct: 85}
+
+- id: val-not-signing
+ title: Validator in the active set but not signing blocks
+ severity: critical
+ symptoms: bonded, not jailed, voting power > 0, but absent from recent commits
+ match:
+ all: ["val.jailed == false", "val.in_set == true", "val.signed_recent_pct < 50"]
+ confirm:
+ - "val.consensus window=50"
+ - "node.status — is the node synced? node.key_mismatch? logs.privval / logs.height_regression?"
+ causes:
+ - node not synced or stalled
+ - node runs a different consensus key than the chain has registered (node.key_mismatch)
+ - remote signer / KMS disconnected
+ - priv_validator_state height ahead of the chain (signing refused)
+ - clock skew causing late votes
+ fix:
+ - "[read] node.triage → follow the more specific case it matches"
+ - "[change] restore signing: fix signer connection / key / sync as found"
+ verify:
+ - "wait.until signing"
+ warnings:
+ - never start a second node with the same key to "fix" signing
+ test:
+ signals: {val.jailed: false, val.in_set: true, val.signed_recent_pct: 0}
+
+- id: val-key-mismatch
+ title: Node runs a different consensus key than the validator registered on chain
+ explains: [val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded, val-in-set-while-syncing]
+ severity: critical
+ symptoms: node status shows a consensus address different from the validator's; node is not signing
+ match:
+ all: ["node.key_mismatch == true", "profile.role == validator"]
+ confirm:
+ - "node.status validator address vs val.status consensus address"
+ - "sec.perms on config/priv_validator_key.json (file replaced? new init?)"
+ causes:
+ - node re-initialized (init generated a fresh priv_validator_key)
+ - migrated to a new machine without moving the key
+ - the real key is on ANOTHER machine that may still be running
+ fix:
+ - "[read] find where the registered key lives; confirm the old machine is stopped (double-sign risk)"
+ - "[change] the operator places the correct priv_validator_key.json (agent never handles key material) and restarts the node"
+ verify:
+ - "node.key_mismatch == false after restart; wait.until signing"
+ warnings:
+ - make sure only ONE machine ever runs the registered key
+ - cometcli never reads or copies consensus keys
+ test:
+ signals: {node.key_mismatch: true, profile.role: validator}
+
+- id: val-not-in-active-set
+ title: Validator bonded status lost — unbonding/unbonded, not jailed
+ severity: high
+ symptoms: validator not jailed but not in the active set
+ match:
+ all: ["val.jailed == false", "val.bonded == false"]
+ confirm:
+ - "chain.validators — compare our tokens with the last active validator (max_validators cutoff)"
+ - "val.status — tokens and status"
+ causes:
+ - stake fell below the active-set cutoff (delegators left, others gained)
+ - self-delegation below min_self_delegation (validator auto-unbonds)
+ fix:
+ - "[read] chain.params staking max_validators, chain.validators"
+ - "[tx] tx.delegate more stake (operator decision)"
+ verify:
+ - "val.status BONDED; val.in_set true on the next epoch/block"
+ test:
+ signals: {val.jailed: false, val.bonded: false, val.status: UNBONDING}
+
+- id: val-self-delegation-below-min
+ title: Self-delegation below min_self_delegation
+ explains: [val-not-in-active-set]
+ severity: high
+ match:
+ all: ["val.self_below_min == true"]
+ confirm:
+ - "val.status self_delegation vs min_self_delegation"
+ causes:
+ - operator undelegated or was slashed below the minimum
+ fix:
+ - "[tx] tx.delegate from the operator account to bring self-delegation above the minimum"
+ - "[tx] then val.unjail if jailed"
+ verify:
+ - "val.self_below_min == false"
+ test:
+ signals: {val.self_below_min: true}
+
+- id: val-no-fee-balance
+ title: Operator account has no balance for fees (unjail/vote will fail)
+ severity: medium
+ match:
+ all: ["val.fee_balance_zero == true"]
+ confirm:
+ - "chain.balance of the operator account in the fee denom"
+ fix:
+ - "[tx] fund the operator account (tx.send from another account the operator controls) or val.withdraw rewards first"
+ verify:
+ - "chain.balance > estimated fee"
+ test:
+ signals: {val.fee_balance_zero: true}
+
+- id: val-jailed-waiting-period
+ title: Jailed and still inside the downtime jail period
+ severity: high
+ match:
+ all: ["val.jailed == true", "val.jail_remaining_s > 0", "val.tombstoned == false"]
+ confirm:
+ - "val.jail-check — jailed_until"
+ fix:
+ - "[read] kb.show val-jailed-downtime — the full recovery procedure; this case only adds the wait"
+ - "[change] use the wait to fix the cause and get the node synced"
+ - "[read] wait.until unjailable"
+ - "[tx] val.unjail"
+ verify:
+ - "wait.until in-consensus"
+ warnings:
+ - an unjail tx before jailed_until fails with 'validator still jailed' and costs fees
+ test:
+ signals: {val.jailed: true, val.jail_remaining_s: 300, val.tombstoned: false}
+
+- id: val-gov-vote-due
+ title: Open governance proposals not voted on
+ severity: medium
+ match:
+ all: ["gov.unvoted > 0"]
+ confirm:
+ - "val.votes — proposals in voting period and our vote status"
+ fix:
+ - "[read] chain.gov show each proposal; summarize for the operator"
+ - "[tx] val.vote with the operator's choice (never choose for them)"
+ verify:
+ - "val.votes shows our vote"
+ test:
+ signals: {gov.unvoted: 2, gov.unvoted_ends_in_h: 30}
+
+- id: val-gov-vote-urgent
+ title: Governance vote ending within 24h without our vote
+ severity: high
+ match:
+ all: ["gov.unvoted > 0", "gov.unvoted_ends_in_h < 24"]
+ fix:
+ - "[read] chain.gov the proposal; upgrade proposals also need a binary ready"
+ - "[tx] val.vote with the operator's decision"
+ test:
+ signals: {gov.unvoted: 1, gov.unvoted_ends_in_h: 5}
+
+- id: val-remote-signer-down
+ title: Remote signer (tmkms/horcrux) connection failing
+ explains: [val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded]
+ severity: critical
+ match:
+ all: ["config.remote_signer == true"]
+ any: ["logs.privval > 0", "val.signed_recent_pct < 50"]
+ confirm:
+ - "node.logs grep=privval|signer"
+ - "check the signer host/process and the priv_validator_laddr port"
+ causes:
+ - signer process down, network/firewall between node and signer, signer state height mismatch
+ fix:
+ - "[read] inspect the signer service on its host (bash, read-only first)"
+ - "[change] restart the signer, then the node if it does not reconnect"
+ verify:
+ - "wait.until signing"
+ warnings:
+ - never point two nodes at the same signer as an active-active setup without a threshold signer (horcrux)
+ test:
+ signals: {config.remote_signer: true, logs.privval: 40}
+
+- id: val-pvs-ahead
+ title: priv_validator_state height ahead of the chain — signing refused
+ explains: [val-jailed-downtime, val-jailed-waiting-period, val-not-signing, val-jail-imminent, val-signing-degraded, val-local-privval-errors]
+ severity: high
+ symptoms: node synced but won't sign; logs show height regression / conflicting data
+ match:
+ all: ["pvs.ahead_by > 1"]
+ any: ["node.catching_up == false", "logs.height_regression > 0"]
+ confirm:
+ - "sec.doublesign shows priv_validator_state height vs node height"
+ causes:
+ - node data restored from an older snapshot while priv_validator_state kept the newer height (this is the protection working)
+ fix:
+ - "[read] wait — the node will sign again once the chain passes pvs.height; wait.until height "
+ - "[change] only if the chain was rolled back/forked by coordinated governance, the operator may adjust state per the chain's instructions"
+ warnings:
+ - never reset priv_validator_state.json to 0 on a live validator — that is how double signs happen
+ test:
+ signals: {pvs.ahead_by: 5000, node.catching_up: false, logs.height_regression: 12}
+
+- id: val-double-sign-evidence
+ title: Double-sign evidence in logs
+ explains: [val-tombstoned]
+ severity: critical
+ match:
+ any: ["logs.double_sign > 0"]
+ confirm:
+ - "sec.doublesign; val.status tombstoned?"
+ - "find every machine running this consensus key"
+ fix:
+ - "[change] STOP the duplicate signer immediately (operator confirms which machine)"
+ - "[read] then follow val-tombstoned if tombstoned"
+ warnings:
+ - every additional block signed on two machines can produce more evidence
+ test:
+ signals: {logs.double_sign: 3}
diff --git a/internal/kb/expr.go b/internal/kb/expr.go
new file mode 100644
index 0000000..2d94d88
--- /dev/null
+++ b/internal/kb/expr.go
@@ -0,0 +1,138 @@
+package kb
+
+import (
+ "fmt"
+ "strconv"
+ "strings"
+)
+
+// Signals are the facts triage collected, by name: "node.peers",
+// "val.jailed", "host.disk_used_pct"… Values are bool, float64 or string.
+type Signals map[string]any
+
+// Cond is one comparison: .
+type Cond struct {
+ Raw string
+ Signal string
+ Op string // == != > >= < <= contains exists missing
+ Value any // bool, float64 or string
+}
+
+var ops = []string{">=", "<=", "==", "!=", ">", "<", " contains ", " exists", " missing"}
+
+// ParseCond parses "val.jailed == true", "host.disk_used_pct >= 90",
+// `logs.panic > 0`, "evm.endpoint exists", `config.mempool_type != "app"`.
+func ParseCond(s string) (Cond, error) {
+ raw := strings.TrimSpace(s)
+ for _, op := range ops {
+ i := strings.Index(raw, op)
+ if i <= 0 {
+ continue
+ }
+ name := strings.TrimSpace(raw[:i])
+ rest := strings.TrimSpace(raw[i+len(op):])
+ op = strings.TrimSpace(op)
+ if strings.ContainsAny(name, " \t") {
+ return Cond{}, fmt.Errorf("condition %q: bad signal name %q", raw, name)
+ }
+ c := Cond{Raw: raw, Signal: name, Op: op}
+ if op == "exists" || op == "missing" {
+ if rest != "" {
+ return Cond{}, fmt.Errorf("condition %q: %s takes no value", raw, op)
+ }
+ return c, nil
+ }
+ if rest == "" {
+ return Cond{}, fmt.Errorf("condition %q: missing value", raw)
+ }
+ c.Value = literal(rest)
+ return c, nil
+ }
+ return Cond{}, fmt.Errorf("condition %q: want with op one of == != > >= < <= contains exists missing", raw)
+}
+
+func literal(s string) any {
+ switch s {
+ case "true":
+ return true
+ case "false":
+ return false
+ }
+ if f, err := strconv.ParseFloat(s, 64); err == nil {
+ return f
+ }
+ return strings.Trim(s, `"'`)
+}
+
+// Eval reports whether the condition holds. A signal that wasn't
+// collected never holds (except for "missing").
+func (c Cond) Eval(s Signals) bool {
+ v, ok := s[c.Signal]
+ switch c.Op {
+ case "exists":
+ return ok
+ case "missing":
+ return !ok
+ }
+ if !ok || v == nil {
+ return false
+ }
+ switch c.Op {
+ case "contains":
+ return strings.Contains(strings.ToLower(fmt.Sprint(v)), strings.ToLower(fmt.Sprint(c.Value)))
+ case "==", "!=":
+ eq := equal(v, c.Value)
+ if c.Op == "==" {
+ return eq
+ }
+ return !eq
+ }
+ a, aok := num(v)
+ b, bok := num(c.Value)
+ if !aok || !bok {
+ return false
+ }
+ switch c.Op {
+ case ">":
+ return a > b
+ case ">=":
+ return a >= b
+ case "<":
+ return a < b
+ case "<=":
+ return a <= b
+ }
+ return false
+}
+
+func equal(a, b any) bool {
+ if x, ok := num(a); ok {
+ if y, ok := num(b); ok {
+ return x == y
+ }
+ }
+ if x, ok := a.(bool); ok {
+ y, ok := b.(bool)
+ return ok && x == y
+ }
+ return strings.EqualFold(fmt.Sprint(a), fmt.Sprint(b))
+}
+
+func num(v any) (float64, bool) {
+ switch x := v.(type) {
+ case float64:
+ return x, true
+ case float32:
+ return float64(x), true
+ case int:
+ return float64(x), true
+ case int64:
+ return float64(x), true
+ case uint64:
+ return float64(x), true
+ case string:
+ f, err := strconv.ParseFloat(x, 64)
+ return f, err == nil
+ }
+ return 0, false
+}
diff --git a/internal/kb/kb.go b/internal/kb/kb.go
new file mode 100644
index 0000000..772b268
--- /dev/null
+++ b/internal/kb/kb.go
@@ -0,0 +1,353 @@
+// Package kb is cometcli's knowledge base of known validator failure
+// modes. Each case says which triage signals point to it, how to confirm
+// it, its causes, the fix (each step tagged read / change / tx), how to
+// verify the fix, and what never to do. Cases are data (YAML): built-in
+// ones ship in the binary; ~/.cometcli/kb/ and /.cometcli/kb/
+// add or override.
+package kb
+
+import (
+ "embed"
+ "fmt"
+ "io/fs"
+ "os"
+ "path/filepath"
+ "regexp"
+ "sort"
+ "strings"
+
+ "gopkg.in/yaml.v3"
+)
+
+//go:embed cases/*.yaml
+var builtin embed.FS
+
+// Case is one known failure mode.
+type Case struct {
+ ID string `yaml:"id"`
+ Title string `yaml:"title"`
+ Chains []string `yaml:"chains,omitempty"` // cosmos-sdk, cosmos-evm (empty = all)
+ Severity string `yaml:"severity"` // critical | high | medium | low
+ // Kind is "incident" (default: something is wrong now) or "advisory"
+ // (hardening/hygiene — listed separately, never outranks an incident).
+ Kind string `yaml:"kind,omitempty"`
+ Symptoms string `yaml:"symptoms"`
+ Match struct {
+ All []string `yaml:"all,omitempty"`
+ Any []string `yaml:"any,omitempty"`
+ } `yaml:"match"`
+ // Explains lists cases this one causes (a crash loop explains the
+ // RPC being down and the downtime jail): when both match, this one
+ // is the root and ranks first.
+ Explains []string `yaml:"explains,omitempty"`
+ Confirm []string `yaml:"confirm,omitempty"`
+ Causes []string `yaml:"causes,omitempty"`
+ Fix []string `yaml:"fix,omitempty"` // "[read] …", "[change] …", "[tx] …"
+ Verify []string `yaml:"verify,omitempty"`
+ Warnings []string `yaml:"warnings,omitempty"`
+ Refs []string `yaml:"refs,omitempty"`
+ Test struct {
+ Signals Signals `yaml:"signals"`
+ } `yaml:"test,omitempty"`
+
+ Source string `yaml:"-"` // builtin | user | project
+ all []Cond
+ any []Cond
+}
+
+var severityRank = map[string]int{"critical": 4, "high": 3, "medium": 2, "low": 1}
+
+// compile parses and checks a case.
+func (c *Case) compile() error {
+ if c.ID == "" || c.Title == "" {
+ return fmt.Errorf("case needs id and title")
+ }
+ if _, ok := severityRank[c.Severity]; !ok {
+ return fmt.Errorf("case %s: severity %q: want critical, high, medium or low", c.ID, c.Severity)
+ }
+ switch c.Kind {
+ case "":
+ c.Kind = "incident"
+ case "incident", "advisory":
+ default:
+ return fmt.Errorf("case %s: kind %q: want incident or advisory", c.ID, c.Kind)
+ }
+ if len(c.Match.All)+len(c.Match.Any) == 0 {
+ return fmt.Errorf("case %s: match needs all or any conditions", c.ID)
+ }
+ c.all, c.any = nil, nil
+ for _, s := range c.Match.All {
+ cond, err := ParseCond(s)
+ if err != nil {
+ return fmt.Errorf("case %s: %w", c.ID, err)
+ }
+ c.all = append(c.all, cond)
+ }
+ for _, s := range c.Match.Any {
+ cond, err := ParseCond(s)
+ if err != nil {
+ return fmt.Errorf("case %s: %w", c.ID, err)
+ }
+ c.any = append(c.any, cond)
+ }
+ for _, x := range c.Explains {
+ if x == c.ID {
+ return fmt.Errorf("case %s explains itself", c.ID)
+ }
+ }
+ for _, f := range c.Fix {
+ if !strings.HasPrefix(f, "[read]") && !strings.HasPrefix(f, "[change]") && !strings.HasPrefix(f, "[tx]") {
+ return fmt.Errorf("case %s: fix step %q must start with [read], [change] or [tx]", c.ID, f)
+ }
+ }
+ return nil
+}
+
+// AppliesTo reports whether the case is for this chain type.
+func (c *Case) AppliesTo(chain string) bool {
+ if len(c.Chains) == 0 || chain == "" {
+ return true
+ }
+ for _, x := range c.Chains {
+ if x == chain || (x == "cosmos-sdk" && chain == "cosmos-evm") {
+ return true // cosmos-evm chains are cosmos-sdk chains too
+ }
+ }
+ return false
+}
+
+// Hit is a case whose conditions matched.
+type Hit struct {
+ Case *Case
+ Score int
+ Matched []string // the conditions that held
+ // SymptomOf is the matched case that explains this one ("" = root).
+ SymptomOf string
+}
+
+// Matches reports whether the case's conditions hold for the signals.
+func (c *Case) Matches(s Signals) (bool, []string) {
+ var matched []string
+ for _, cond := range c.all {
+ if !cond.Eval(s) {
+ return false, nil
+ }
+ matched = append(matched, cond.Raw)
+ }
+ if len(c.any) > 0 {
+ n := 0
+ for _, cond := range c.any {
+ if cond.Eval(s) {
+ matched = append(matched, cond.Raw)
+ n++
+ }
+ }
+ if n == 0 {
+ return false, nil
+ }
+ }
+ return true, matched
+}
+
+// Base is a loaded knowledge base.
+type Base struct {
+ Cases map[string]*Case
+ Errs []string // cases that failed to load (reported, not fatal)
+}
+
+// Load reads the built-in cases, then ~/.cometcli/kb and the project's
+// .cometcli/kb (later ones override by id).
+func Load(userDir, projectRoot string) *Base {
+ b := &Base{Cases: map[string]*Case{}}
+ _ = fs.WalkDir(builtin, "cases", func(p string, d fs.DirEntry, err error) error {
+ if err == nil && !d.IsDir() && strings.HasSuffix(p, ".yaml") {
+ raw, _ := builtin.ReadFile(p)
+ b.add(raw, "builtin", p)
+ }
+ return nil
+ })
+ for _, src := range []struct{ dir, name string }{{userDir, "user"}, {filepath.Join(projectRoot, ".cometcli", "kb"), "project"}} {
+ if src.dir == "" || (src.name == "project" && projectRoot == "") {
+ continue
+ }
+ files, _ := filepath.Glob(filepath.Join(src.dir, "*.yaml"))
+ for _, f := range files {
+ if raw, err := os.ReadFile(f); err == nil {
+ b.add(raw, src.name, f)
+ }
+ }
+ }
+ return b
+}
+
+// add parses a file holding one case or a list of cases.
+func (b *Base) add(raw []byte, source, path string) {
+ var many []*Case
+ if err := yaml.Unmarshal(raw, &many); err != nil {
+ var one Case
+ if err := yaml.Unmarshal(raw, &one); err != nil {
+ b.Errs = append(b.Errs, fmt.Sprintf("%s: %v", path, err))
+ return
+ }
+ many = []*Case{&one}
+ }
+ for _, c := range many {
+ if err := c.compile(); err != nil {
+ b.Errs = append(b.Errs, fmt.Sprintf("%s: %v", path, err))
+ continue
+ }
+ c.Source = source
+ b.Cases[c.ID] = c
+ }
+}
+
+// Match returns the cases the signals point to, best first.
+func (b *Base) Match(s Signals, chain string) []Hit {
+ var hits []Hit
+ for _, c := range b.Cases {
+ if !c.AppliesTo(chain) {
+ continue
+ }
+ if ok, matched := c.Matches(s); ok {
+ // severity leads (a dead node beats a busy log), specificity
+ // breaks ties; advisories always sort after incidents
+ score := severityRank[c.Severity]*10 + len(matched)*7
+ if c.Kind == "advisory" {
+ score -= 1000
+ }
+ hits = append(hits, Hit{Case: c, Score: score, Matched: matched})
+ }
+ }
+ byScore := func(i, j int) bool {
+ if hits[i].Score != hits[j].Score {
+ return hits[i].Score > hits[j].Score
+ }
+ return hits[i].Case.ID < hits[j].Case.ID
+ }
+ sort.Slice(hits, byScore)
+ // a hit explained by another matched hit (that it doesn't explain
+ // back) is a symptom: roots first, each in score order
+ matched := map[string]*Case{}
+ for _, h := range hits {
+ matched[h.Case.ID] = h.Case
+ }
+ explains := func(a *Case, id string) bool {
+ for _, x := range a.Explains {
+ if x == id {
+ return true
+ }
+ }
+ return false
+ }
+ for i := range hits {
+ for _, h := range hits { // best-scored explainer wins
+ if h.Case.ID != hits[i].Case.ID && explains(h.Case, hits[i].Case.ID) && !explains(hits[i].Case, h.Case.ID) {
+ hits[i].SymptomOf = h.Case.ID
+ break
+ }
+ }
+ }
+ sort.SliceStable(hits, func(i, j int) bool {
+ ai, aj := hits[i].Case.Kind == "advisory", hits[j].Case.Kind == "advisory"
+ if ai != aj {
+ return aj
+ }
+ ri, rj := hits[i].SymptomOf == "", hits[j].SymptomOf == ""
+ if ri != rj {
+ return ri
+ }
+ return byScore(i, j)
+ })
+ return hits
+}
+
+// Search ranks cases by keyword hits in id, title, symptoms and causes.
+func (b *Base) Search(q, chain string, n int) []*Case {
+ words := strings.Fields(strings.ToLower(q))
+ type scored struct {
+ c *Case
+ s int
+ }
+ var out []scored
+ for _, c := range b.Cases {
+ if !c.AppliesTo(chain) {
+ continue
+ }
+ hay := strings.ToLower(c.ID + " " + c.Title + " " + c.Symptoms + " " + strings.Join(c.Causes, " ") + " " + strings.Join(c.Match.All, " ") + " " + strings.Join(c.Match.Any, " "))
+ s := 0
+ for _, w := range words {
+ if len(w) < 3 {
+ continue
+ }
+ if strings.Contains(strings.ToLower(c.Title+" "+c.ID), w) {
+ s += 3
+ }
+ if strings.Contains(hay, w) {
+ s++
+ }
+ }
+ if s > 0 {
+ out = append(out, scored{c, s})
+ }
+ }
+ sort.Slice(out, func(i, j int) bool {
+ if out[i].s != out[j].s {
+ return out[i].s > out[j].s
+ }
+ return out[i].c.ID < out[j].c.ID
+ })
+ var res []*Case
+ for i := 0; i < len(out) && i < n && out[i].s*2 >= out[0].s; i++ {
+ res = append(res, out[i].c)
+ }
+ return res
+}
+
+// Render formats a case as the card the agent reads.
+func (c *Case) Render() string {
+ var b strings.Builder
+ fmt.Fprintf(&b, "CASE %s — %s [%s]\n", c.ID, c.Title, c.Severity)
+ if c.Symptoms != "" {
+ fmt.Fprintf(&b, "symptoms: %s\n", c.Symptoms)
+ }
+ section := func(name string, items []string) {
+ if len(items) == 0 {
+ return
+ }
+ fmt.Fprintf(&b, "%s:\n", name)
+ for i, it := range items {
+ fmt.Fprintf(&b, " %d. %s\n", i+1, it)
+ }
+ }
+ section("confirm", c.Confirm)
+ section("causes", c.Causes)
+ section("fix", c.Fix)
+ section("verify", c.Verify)
+ section("NEVER", c.Warnings)
+ if len(c.Refs) > 0 {
+ fmt.Fprintf(&b, "refs: %s\n", strings.Join(c.Refs, " "))
+ }
+ return strings.TrimRight(b.String(), "\n")
+}
+
+// Validate loads a single case file's bytes (for kb.add).
+func Validate(raw []byte) (*Case, error) {
+ var c Case
+ if err := yaml.Unmarshal(raw, &c); err != nil {
+ return nil, err
+ }
+ if err := c.compile(); err != nil {
+ return nil, err
+ }
+ if len(c.Test.Signals) > 0 {
+ if ok, _ := c.Matches(c.Test.Signals); !ok {
+ return nil, fmt.Errorf("case %s doesn't match its own test signals", c.ID)
+ }
+ }
+ return &c, nil
+}
+
+var idRe = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{1,63}$`)
+
+// ValidID reports whether id is safe as a file name.
+func ValidID(id string) bool { return idRe.MatchString(id) }
diff --git a/internal/kb/kb_test.go b/internal/kb/kb_test.go
new file mode 100644
index 0000000..574b338
--- /dev/null
+++ b/internal/kb/kb_test.go
@@ -0,0 +1,173 @@
+package kb
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+)
+
+func TestBuiltinCasesLoadAndSelfMatch(t *testing.T) {
+ b := Load("", "")
+ if len(b.Errs) > 0 {
+ t.Fatalf("load errors: %v", b.Errs)
+ }
+ if len(b.Cases) < 50 {
+ t.Fatalf("only %d cases", len(b.Cases))
+ }
+ for id, c := range b.Cases {
+ if len(c.Test.Signals) == 0 {
+ t.Errorf("%s: no test fixture", id)
+ continue
+ }
+ if len(c.Fix) == 0 {
+ t.Errorf("%s: no fix steps", id)
+ }
+ chain := "cosmos-sdk"
+ if len(c.Chains) == 1 {
+ chain = c.Chains[0]
+ }
+ hits := b.Match(c.Test.Signals, chain)
+ rank := -1
+ for i, h := range hits {
+ if h.Case.ID == id {
+ rank = i
+ }
+ }
+ if rank < 0 || rank > 2 {
+ var top []string
+ for i := 0; i < len(hits) && i < 4; i++ {
+ top = append(top, hits[i].Case.ID)
+ }
+ t.Errorf("%s: fixture ranks %d (top: %v)", id, rank, top)
+ }
+ }
+}
+
+func TestHealthyNodeMatchesNothing(t *testing.T) {
+ healthy := Signals{
+ "chain.type": "cosmos-evm", "profile.role": "validator",
+ "node.reachable": true, "node.height": 1000.0, "node.catching_up": false, "node.block_age_s": 2.0,
+ "node.peers": 12.0, "node.stalled": false, "node.key_mismatch": false,
+ "val.jailed": false, "val.tombstoned": false, "val.bonded": true, "val.status": "BONDED", "val.in_set": true,
+ "val.signed_recent_pct": 100.0, "val.missed_pct": 0.1, "val.downtime_budget_used_pct": 0.2, "val.self_below_min": false,
+ "val.fee_balance_zero": false, "upgrade.pending": false, "gov.unvoted": 0.0,
+ "host.disk_used_pct": 40.0, "host.inode_used_pct": 5.0, "host.mem_used_pct": 50.0, "host.load_per_cpu": 0.4,
+ "clock.ntp_synced": true, "proc.running": true, "proc.restarts": 0.0, "proc.oom_killed": false, "proc.uptime_s": 86400.0,
+ "logs.panic": 0.0, "logs.apphash": 0.0, "logs.peer_net": 12.0,
+ "config.db_backend_mismatch": false, "config.mempool_mismatch": false, "config.remote_signer": false,
+ "config.persistent_peers": 3.0, "config.seeds": 1.0, "config.statesync_enable": false, "config.pex": true,
+ "config.double_sign_check_height": 10.0,
+ "app.min_gas_prices_empty": false, "app.pruning": "custom", "app.api_enable": false, "app.json_rpc_enable": false,
+ "evm.reachable": true, "evm.drift": 0.0, "evm.chain_id_mismatch": false, "pvs.ahead_by": 0.0,
+ }
+ if hits := Load("", "").Match(healthy, "cosmos-evm"); len(hits) > 0 {
+ var ids []string
+ for _, h := range hits {
+ ids = append(ids, h.Case.ID)
+ }
+ t.Fatalf("healthy node matched %v", ids)
+ }
+}
+
+func TestCond(t *testing.T) {
+ s := Signals{"a": 5.0, "b": true, "c": "Pebbledb", "d": "0.0.0.0:8545"}
+ for expr, want := range map[string]bool{
+ "a > 4": true, "a >= 5": true, "a < 5": false, "a == 5": true, "a != 5": false,
+ "b == true": true, "b == false": false, "c == pebbledb": true, `c != "goleveldb"`: true,
+ "d contains 0.0.0.0": true, "x exists": false, "x missing": true, "a exists": true,
+ "x > 0": false, "x == false": false, // uncollected never holds
+ } {
+ c, err := ParseCond(expr)
+ if err != nil {
+ t.Fatalf("%s: %v", expr, err)
+ }
+ if got := c.Eval(s); got != want {
+ t.Errorf("%s = %v, want %v", expr, got, want)
+ }
+ }
+ for _, bad := range []string{"a", "a >", "a b > 1", "exists"} {
+ if _, err := ParseCond(bad); err == nil {
+ t.Errorf("%q parsed", bad)
+ }
+ }
+}
+
+func TestChainFiltering(t *testing.T) {
+ b := Load("", "")
+ s := Signals{"evm.reachable": false, "node.reachable": true}
+ for _, h := range b.Match(s, "cosmos-sdk") {
+ if h.Case.ID == "evm-rpc-down" {
+ t.Fatal("evm case matched a plain cosmos-sdk chain")
+ }
+ }
+ found := false
+ for _, h := range b.Match(s, "cosmos-evm") {
+ found = found || h.Case.ID == "evm-rpc-down"
+ }
+ if !found {
+ t.Fatal("evm case missed on cosmos-evm")
+ }
+}
+
+func TestUserOverrideAndValidate(t *testing.T) {
+ dir := t.TempDir()
+ raw := []byte(`id: node-no-peers
+title: Custom no peers
+severity: low
+match: {all: ["node.peers == 0"]}
+fix: ["[change] call the NOC"]
+test: {signals: {node.peers: 0}}
+`)
+ if _, err := Validate(raw); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(filepath.Join(dir, "x.yaml"), raw, 0o600); err != nil {
+ t.Fatal(err)
+ }
+ b := Load(dir, "")
+ if c := b.Cases["node-no-peers"]; c.Title != "Custom no peers" || c.Source != "user" {
+ t.Fatalf("override not applied: %+v", c)
+ }
+ bad := []byte("id: x1\ntitle: t\nseverity: low\nmatch: {all: [\"a == 1\"]}\nfix: [\"do it\"]\n")
+ if _, err := Validate(bad); err == nil {
+ t.Fatal("untagged fix step accepted")
+ }
+ mismatch := []byte("id: x2\ntitle: t\nseverity: low\nmatch: {all: [\"a == 1\"]}\nfix: [\"[read] x\"]\ntest: {signals: {a: 2}}\n")
+ if _, err := Validate(mismatch); err == nil {
+ t.Fatal("case not matching its own fixture accepted")
+ }
+}
+
+func TestSearch(t *testing.T) {
+ cs := Load("", "").Search("app hash mismatch", "cosmos-sdk", 3)
+ if len(cs) == 0 || cs[0].ID != "state-apphash-mismatch" {
+ t.Fatalf("search: %v", cs)
+ }
+}
+
+func TestExplainsAreKnownAndRootsFirst(t *testing.T) {
+ b := Load("", "")
+ for id, c := range b.Cases {
+ for _, x := range c.Explains {
+ if _, ok := b.Cases[x]; !ok {
+ t.Errorf("%s explains unknown case %s", id, x)
+ }
+ }
+ }
+ // a bad config.toml: crash loop, RPC down, jailed — the config is the root
+ s := Signals{"config.parse_error": true, "proc.running": false, "proc.restarts": 9.0, "proc.uptime_s": 7.0,
+ "logs.panic": 0.0, "logs.config_parse": 4.0, "node.reachable": false, "val.jailed": true, "val.tombstoned": false,
+ "val.jail_remaining_s": 20.0, "profile.role": "validator", "app.api_enable": true}
+ hits := b.Match(s, "cosmos-evm")
+ if hits[0].Case.ID != "cfg-parse-error" && hits[0].Case.ID != "node-exit-config-error" {
+ t.Fatalf("root = %s", hits[0].Case.ID)
+ }
+ for _, h := range hits {
+ if h.Case.ID == "val-jailed-downtime" && h.SymptomOf == "" {
+ t.Error("jail not marked as a symptom")
+ }
+ }
+ if last := hits[len(hits)-1]; last.Case.Kind != "advisory" {
+ t.Errorf("advisory not last: %s", last.Case.ID)
+ }
+}
diff --git a/internal/keys/keys.go b/internal/keys/keys.go
index dbac656..79dc69b 100644
--- a/internal/keys/keys.go
+++ b/internal/keys/keys.go
@@ -90,6 +90,19 @@ func ConsAddress(addr []byte, prefix string) (string, error) {
}
// Bech32ToHex converts a bech32 account address to 0x hex.
+// AccFromValoper converts a valoper address to its account address (same
+// bytes, the "valoper" suffix dropped from the prefix).
+func AccFromValoper(valoper string) (string, error) {
+ prefix, data, err := bech32.DecodeNoLimit(valoper)
+ if err != nil {
+ return "", err
+ }
+ if !strings.HasSuffix(prefix, "valoper") {
+ return "", fmt.Errorf("%s is not a valoper address", valoper)
+ }
+ return bech32.Encode(strings.TrimSuffix(prefix, "valoper"), data)
+}
+
func Bech32ToHex(addr string) (string, error) {
_, data, err := bech32.DecodeNoLimit(addr)
if err != nil {
diff --git a/internal/logscan/logscan.go b/internal/logscan/logscan.go
new file mode 100644
index 0000000..696e8d0
--- /dev/null
+++ b/internal/logscan/logscan.go
@@ -0,0 +1,175 @@
+// Package logscan classifies CometBFT / Cosmos SDK / Cosmos-EVM node log
+// lines into failure categories. Triage turns the counts into signals
+// (logs.); incident forensics shows sample lines per category.
+package logscan
+
+import (
+ "regexp"
+ "strings"
+)
+
+// Category is one class of log line.
+type Category struct {
+ Slug string // signal suffix: logs.
+ Name string // human label
+ Re *regexp.Regexp
+ // AnyLevel also counts info/debug lines; most categories only count
+ // warnings and errors, since healthy nodes log field names like
+ // appHash= or peer churn at info level all day.
+ AnyLevel bool
+}
+
+// Categories are checked in order; a line can fall in several.
+var Categories = []Category{
+ {"panic", "crash/panic", regexp.MustCompile(`(?i)\bpanic\b|fatal error|CONSENSUS FAILURE|segmentation`), false},
+ {"oom", "out of memory", regexp.MustCompile(`(?i)out of memory|oom.?kill|cannot allocate`), false},
+ {"disk_io", "disk full / IO", regexp.MustCompile(`(?i)no space left|disk quota|input/output error|read-only file system`), false},
+ {"peer_net", "peer loss / network", regexp.MustCompile(`(?i)connection refused|i/o timeout|dial tcp|stopping peer|no peers|failed to (dial|connect)|network is unreachable`), false},
+ {"privval", "signer / privval", regexp.MustCompile(`(?i)privval|remote signer|failed to sign|error signing|sign (vote|proposal)`), false},
+ {"apphash", "app hash / state", regexp.MustCompile(`(?i)wrong app ?hash|app ?hash (mismatch|does not match)|wrong block\.header\.apphash|wrong Block\.Header|state mismatch`), false},
+ {"clock", "clock / timing", regexp.MustCompile(`(?i)clock|time.*(skew|drift)|timed out waiting`), false},
+ {"too_many_files", "too many files", regexp.MustCompile(`(?i)too many open files`), false},
+ {"upgrade_needed", "upgrade halt", regexp.MustCompile(`UPGRADE "[^"]+" NEEDED`), true},
+ {"binary_early", "binary updated before upgrade", regexp.MustCompile(`(?i)BINARY UPDATED BEFORE TRIGGER|wrong app version|unknown upgrade|upgrade.*not found in plan`), true},
+ {"db_lock", "database locked", regexp.MustCompile(`(?i)resource temporarily unavailable|lock .*held|database is locked`), false},
+ {"db_corrupt", "database corruption", regexp.MustCompile(`(?i)corrupt|checksum mismatch|missing .*\.(sst|ldb)|manifest`), false},
+ {"height_regression", "privval height regression", regexp.MustCompile(`(?i)height regression|conflicting data|round regression|step regression`), true},
+ {"double_sign", "double sign", regexp.MustCompile(`(?i)double.?sign|duplicate ?vote ?evidence`), true},
+ {"mempool_full", "mempool full", regexp.MustCompile(`(?i)mempool is full`), false},
+ {"statesync_fail", "state sync failure", regexp.MustCompile(`(?i)state ?sync.*(fail|error|abort)|failed to (apply|restore|verify) snapshot|no available snapshots|no suitable snapshots`), false},
+ {"port_in_use", "port in use", regexp.MustCompile(`(?i)address already in use`), false},
+ {"config_parse", "config error", regexp.MustCompile(`(?i)error reading config|parsing config|failed to (load|parse|unmarshal|decode).*(config|toml)|toml: |unknown (field|flag)|invalid (config|minimum gas)`), false},
+ {"chain_mismatch", "chain / genesis mismatch", regexp.MustCompile(`(?i)different network|genesis.*(mismatch|doesn't match|hash)|wrong genesis|incompatible chain.?id|invalid chain.?id`), false},
+ {"startup_error", "startup error (process exits)", regexp.MustCompile(`^Error: `), true},
+ {"jail", "jail / slashing", regexp.MustCompile(`(?i)\bjail(ed|ing)?\b|liveness fault|tombston|slashing (and|validator)|\bslashed\b`), true},
+ {"evm_rpc", "EVM JSON-RPC errors", regexp.MustCompile(`(?i)json-?rpc.*(error|fail)|failed to start.*(json-?rpc|evm)|evm.*indexer.*(error|fail)`), false},
+ {"p2p_auth", "p2p handshake / auth", regexp.MustCompile(`(?i)auth failure|handshake.*fail|filtered|incompatible|peer.*rejected`), false},
+}
+
+// Result holds per-category counts and sample lines.
+type Result struct {
+ Lines int
+ Counts map[string]int // slug → matching lines
+ Samples map[string][]string // slug → up to MaxSamples lines
+ // UpgradeName is the plan name from an "UPGRADE "x" NEEDED" line.
+ UpgradeName string
+ // LastError is the most recent error-level line.
+ LastError string
+ // JailLines are jail/slashing lines (any validator's — callers filter
+ // by their consensus address), up to 50.
+ JailLines []string
+}
+
+// MaxSamples caps sample lines kept per category.
+var MaxSamples = 3
+
+var (
+ ansiRe = regexp.MustCompile(`\x1b\[[0-9;]*m`)
+ // info/debug level in CometBFT plain ("INF", "I[…]"), json and logfmt
+ // cobra usage output: flag docs, section headers
+ helpRe = regexp.MustCompile(`^(-\w, )?--[\w.-]+( |$)|^(Usage|Flags|Global Flags|Available Commands|Aliases|Examples):$`)
+ errRe = regexp.MustCompile(`^Error: |^(\S+\s+)?(ERR|ERROR|FTL)\s|^E\[\d{4}-|"level":"(error|fatal)"|\blevel=(error|fatal)\b|^panic: `)
+ infoRe = regexp.MustCompile(`^(\S+\s+)?(INF|DBG|TRC)\s|^[ID]\[\d{4}-|"level":"(info|debug|trace)"|\blevel=(info|debug|trace)\b`)
+)
+
+var upgradeRe = regexp.MustCompile(`UPGRADE "([^"]+)" NEEDED`)
+
+// Scan classifies every line of text.
+func Scan(text string) *Result {
+ r := &Result{Counts: map[string]int{}, Samples: map[string][]string{}}
+ for _, line := range strings.Split(text, "\n") {
+ if line = strings.TrimSpace(ansiRe.ReplaceAllString(line, "")); line == "" {
+ continue
+ }
+ if helpRe.MatchString(line) {
+ continue // a startup failure prints the whole --help: flag docs aren't events
+ }
+ r.Lines++
+ if errRe.MatchString(line) {
+ r.LastError = Clip(line)
+ }
+ for _, c := range classify(line) {
+ if c.Slug == "jail" && len(r.JailLines) < 50 {
+ r.JailLines = append(r.JailLines, Clip(line))
+ }
+ r.Counts[c.Slug]++
+ if len(r.Samples[c.Slug]) < MaxSamples {
+ r.Samples[c.Slug] = append(r.Samples[c.Slug], Clip(line))
+ }
+ }
+ if m := upgradeRe.FindStringSubmatch(line); m != nil {
+ r.UpgradeName = m[1]
+ }
+ }
+ return r
+}
+
+// Clean strips terminal color codes and surrounding space from a line.
+func Clean(line string) string { return strings.TrimSpace(ansiRe.ReplaceAllString(line, "")) }
+
+// Classify returns the categories one raw log line falls in.
+func Classify(line string) []Category {
+ return classify(strings.TrimSpace(ansiRe.ReplaceAllString(line, "")))
+}
+
+func classify(line string) []Category {
+ if helpRe.MatchString(line) {
+ return nil
+ }
+ quiet := infoRe.MatchString(line)
+ var out []Category
+ regression := false
+ for _, c := range Categories {
+ if (c.AnyLevel || !quiet) && c.Re.MatchString(line) {
+ out = append(out, c)
+ regression = regression || c.Slug == "height_regression"
+ }
+ }
+ if regression { // the signer refusing a regression is protection working, not a broken signer
+ kept := out[:0]
+ for _, c := range out {
+ if c.Slug != "privval" {
+ kept = append(kept, c)
+ }
+ }
+ out = kept
+ }
+ return out
+}
+
+var validatorAddrRe = regexp.MustCompile(`[a-z]+val(cons|oper)1[0-9a-z]+`)
+
+// OwnJailLine reports whether a jail/slashing line can be about our
+// validator (ids: its valcons and/or valoper address): it names one of
+// them, or names no validator at all. Lines about other validators are
+// dropped — every node logs every validator's jailing and slashing.
+func OwnJailLine(line string, ids ...string) bool {
+ known := false
+ for _, id := range ids {
+ if id != "" {
+ known = true
+ if strings.Contains(line, id) {
+ return true
+ }
+ }
+ }
+ return !known || !validatorAddrRe.MatchString(line)
+}
+
+// Name returns a category's label by slug.
+func Name(slug string) string {
+ for _, c := range Categories {
+ if c.Slug == slug {
+ return c.Name
+ }
+ }
+ return slug
+}
+
+// Clip shortens a log line for display.
+func Clip(s string) string {
+ if len(s) > 240 {
+ return s[:240] + "…"
+ }
+ return s
+}
diff --git a/internal/logscan/logscan_test.go b/internal/logscan/logscan_test.go
new file mode 100644
index 0000000..3251db8
--- /dev/null
+++ b/internal/logscan/logscan_test.go
@@ -0,0 +1,69 @@
+package logscan
+
+import "testing"
+
+func TestScanLevelsAndColors(t *testing.T) {
+ logs := "\x1b[90m5:30AM\x1b[0m \x1b[32mINF\x1b[0m Completed ABCI Handshake appHash=E3B0C4 appHeight=0 module=consensus\n" +
+ "5:31AM INF Stopping peer for error err=EOF module=p2p\n" +
+ "5:32AM ERR CONSENSUS FAILURE!!! err=\"wrong Block.Header.AppHash. Expected 1A, got 2B\"\n" +
+ "5:33AM INF slashing and jailing validator due to liveness fault height=50\n" +
+ "\x1b[90m5:34AM\x1b[0m \x1b[31mERR\x1b[0m UPGRADE \"v2\" NEEDED at height: 100: \n" +
+ "I[2026-10-06|21:41:13.000] dial tcp 1.2.3.4:26656: i/o timeout module=p2p\n" +
+ "E[2026-10-06|21:42:00.000] failed to write to db: no space left on device\n"
+ r := Scan(logs)
+ want := map[string]int{"apphash": 1, "panic": 1, "jail": 1, "upgrade_needed": 1, "peer_net": 0, "disk_io": 1}
+ for k, v := range want {
+ if r.Counts[k] != v {
+ t.Errorf("%s = %d, want %d", k, r.Counts[k], v)
+ }
+ }
+ if r.UpgradeName != "v2" {
+ t.Errorf("upgrade name %q", r.UpgradeName)
+ }
+}
+
+func TestScanIgnoresUsageText(t *testing.T) {
+ logs := "Error: While parsing config: toml: expected character ]\nUsage:\n evmd start [flags]\nFlags:\n" +
+ " --consensus.double_sign_check_height int how many blocks to look back\n" +
+ " --log_level string (trace|debug|info|warn|error|fatal|panic|disabled)\n" +
+ " -h, --help help for start\n"
+ r := Scan(logs)
+ if r.Counts["double_sign"] != 0 || r.Counts["panic"] != 0 {
+ t.Errorf("usage text classified: %v", r.Counts)
+ }
+ if r.Counts["config_parse"] != 1 || r.Counts["startup_error"] != 1 {
+ t.Errorf("startup error missed: %v", r.Counts)
+ }
+ if r.LastError != "Error: While parsing config: toml: expected character ]" {
+ t.Errorf("last error %q", r.LastError)
+ }
+}
+
+func TestOwnJailLine(t *testing.T) {
+ me := "cosmosvalcons1aaa"
+ if !OwnJailLine("INF validator jailed validator=cosmosvalcons1aaa", me) || OwnJailLine("INF validator jailed validator=cosmosvalcons1bbb", me) {
+ t.Fatal("address filter")
+ }
+ if OwnJailLine("INF validator slashed by slash factor validator=cosmosvaloper1other", me, "cosmosvaloper1mine") ||
+ !OwnJailLine("INF validator slashed validator=cosmosvaloper1mine", me, "cosmosvaloper1mine") {
+ t.Fatal("valoper filter")
+ }
+ if !OwnJailLine("INF slashing and jailing validator due to liveness fault height=5", me) {
+ t.Fatal("line without an address dropped")
+ }
+}
+
+func TestRegressionIsNotAPrivvalFault(t *testing.T) {
+ r := Scan(`5:58AM ERR failed signing vote err="error signing vote: step regression at height 1317 round 0. Got 2, last step 3" module=consensus`)
+ if r.Counts["privval"] != 0 || r.Counts["height_regression"] != 1 {
+ t.Fatalf("%v", r.Counts)
+ }
+}
+
+func TestJailPatternIgnoresStoreKeys(t *testing.T) {
+ r := Scan(`6:55AM INF Upgrading IAVL storage for faster queries store_key="KVStoreKey{0x400286ccf0, slashing}" version=1
+6:55AM INF validator jailed module=x/staking validator=cosmosvalcons1abc`)
+ if r.Counts["jail"] != 1 {
+ t.Fatalf("jail = %d", r.Counts["jail"])
+ }
+}
diff --git a/internal/mcp/server.go b/internal/mcp/server.go
index 843a9f8..82b6fce 100644
--- a/internal/mcp/server.go
+++ b/internal/mcp/server.go
@@ -135,8 +135,8 @@ func (s *Server) handle(ctx context.Context, raw []byte) *response {
func (s *Server) toolList() []map[string]any {
var out []map[string]any
for _, t := range s.Reg.All() {
- if toolkit.IsLongRunning(t) {
- continue // watchers never return from a tools/call
+ if toolkit.IsLongRunning(t) || toolkit.IsAgentOnly(t) || toolkit.IsOperatorOnly(t) {
+ continue // watchers never return; bash/read/edit are the client's own job
}
d := map[string]any{
"name": t.Name(),
@@ -186,6 +186,9 @@ func (s *Server) callTool(ctx context.Context, name string, args map[string]any)
if toolkit.IsLongRunning(t) {
return callResult{text: name + " is a long-running watcher — not callable over MCP; use the CLI"}, true
}
+ if toolkit.IsAgentOnly(t) || toolkit.IsOperatorOnly(t) {
+ return callResult{text: name + " is not exported over MCP"}, true
+ }
runCtx, cancel := toolkit.WithDeadline(s.Ctx, 120*time.Second)
defer func() { runCtx.Close(); cancel() }()
res, err := t.Run(runCtx, toolkit.Args(args))
diff --git a/internal/mcpclient/client.go b/internal/mcpclient/client.go
new file mode 100644
index 0000000..61968eb
--- /dev/null
+++ b/internal/mcpclient/client.go
@@ -0,0 +1,478 @@
+// Package mcpclient connects to Model Context Protocol servers (stdio or
+// streamable HTTP) so their tools can be offered to the agent.
+package mcpclient
+
+import (
+ "bufio"
+ "bytes"
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "net/http"
+ "os"
+ "os/exec"
+ "strings"
+ "sync"
+ "sync/atomic"
+ "time"
+
+ "github.com/abhijitkrm/cometcli/internal/settings"
+)
+
+const protocolVersion = "2025-06-18"
+
+// Tool is a tool a server offers.
+type Tool struct {
+ Name string `json:"name"`
+ Description string `json:"description"`
+ InputSchema map[string]any `json:"inputSchema"`
+ Annotations struct {
+ ReadOnlyHint bool `json:"readOnlyHint"`
+ DestructiveHint bool `json:"destructiveHint"`
+ } `json:"annotations"`
+}
+
+// Content is one item of a tool result.
+type Content struct {
+ Type string `json:"type"`
+ Text string `json:"text"`
+ MimeType string `json:"mimeType"`
+ Resource *struct {
+ URI string `json:"uri"`
+ Text string `json:"text"`
+ } `json:"resource"`
+}
+
+// CallResult is a tools/call result.
+type CallResult struct {
+ Content []Content `json:"content"`
+ IsError bool `json:"isError"`
+}
+
+// Text flattens the result for the model.
+func (r *CallResult) Text() string {
+ var parts []string
+ for _, c := range r.Content {
+ switch c.Type {
+ case "text":
+ parts = append(parts, c.Text)
+ case "resource":
+ if c.Resource != nil {
+ parts = append(parts, fmt.Sprintf("[resource %s]\n%s", c.Resource.URI, c.Resource.Text))
+ }
+ default:
+ parts = append(parts, fmt.Sprintf("[%s content (%s) omitted]", c.Type, c.MimeType))
+ }
+ }
+ return strings.Join(parts, "\n")
+}
+
+type rpcError struct {
+ Code int `json:"code"`
+ Message string `json:"message"`
+}
+
+type rpcMsg struct {
+ JSONRPC string `json:"jsonrpc"`
+ ID *json.RawMessage `json:"id,omitempty"`
+ Method string `json:"method,omitempty"`
+ Params any `json:"params,omitempty"`
+ Result json.RawMessage `json:"result,omitempty"`
+ Error *rpcError `json:"error,omitempty"`
+}
+
+type transport interface {
+ roundTrip(ctx context.Context, id int64, method string, params any) (json.RawMessage, error)
+ notify(ctx context.Context, method string, params any) error
+ close() error
+}
+
+// Client is one connected server.
+type Client struct {
+ Name string
+ Tools []Tool
+ tr transport
+ seq atomic.Int64
+}
+
+// Connect starts or dials a server, initializes it and lists its tools.
+func Connect(ctx context.Context, name string, cfg settings.MCPServer) (*Client, error) {
+ var tr transport
+ var err error
+ switch strings.ToLower(cfg.Type) {
+ case "", "stdio":
+ tr, err = newStdio(cfg)
+ case "http", "streamable-http", "streamablehttp":
+ tr = &httpTransport{url: expand(cfg.URL), headers: expandMap(cfg.Headers), hc: &http.Client{Timeout: 5 * time.Minute}}
+ case "sse":
+ return nil, fmt.Errorf("mcp %s: legacy SSE transport isn't supported — use the server's streamable HTTP endpoint", name)
+ default:
+ return nil, fmt.Errorf("mcp %s: unknown transport %q", name, cfg.Type)
+ }
+ if err != nil {
+ return nil, fmt.Errorf("mcp %s: %w", name, err)
+ }
+ c := &Client{Name: name, tr: tr}
+ if err := c.init(ctx); err != nil {
+ _ = tr.close()
+ return nil, fmt.Errorf("mcp %s: %w", name, err)
+ }
+ return c, nil
+}
+
+func (c *Client) call(ctx context.Context, method string, params any, out any) error {
+ raw, err := c.tr.roundTrip(ctx, c.seq.Add(1), method, params)
+ if err != nil {
+ return err
+ }
+ if out == nil {
+ return nil
+ }
+ return json.Unmarshal(raw, out)
+}
+
+func (c *Client) init(ctx context.Context) error {
+ err := c.call(ctx, "initialize", map[string]any{
+ "protocolVersion": protocolVersion,
+ "capabilities": map[string]any{},
+ "clientInfo": map[string]any{"name": "cometcli", "version": "1"},
+ }, nil)
+ if err != nil {
+ return fmt.Errorf("initialize: %w", err)
+ }
+ if err := c.tr.notify(ctx, "notifications/initialized", nil); err != nil {
+ return err
+ }
+ cursor := ""
+ for page := 0; page < 20; page++ {
+ var res struct {
+ Tools []Tool `json:"tools"`
+ NextCursor string `json:"nextCursor"`
+ }
+ var params any
+ if cursor != "" {
+ params = map[string]any{"cursor": cursor}
+ }
+ if err := c.call(ctx, "tools/list", params, &res); err != nil {
+ return fmt.Errorf("tools/list: %w", err)
+ }
+ c.Tools = append(c.Tools, res.Tools...)
+ if cursor = res.NextCursor; cursor == "" {
+ break
+ }
+ }
+ return nil
+}
+
+// CallTool invokes a tool.
+func (c *Client) CallTool(ctx context.Context, name string, args map[string]any) (*CallResult, error) {
+ if args == nil {
+ args = map[string]any{}
+ }
+ var res CallResult
+ if err := c.call(ctx, "tools/call", map[string]any{"name": name, "arguments": args}, &res); err != nil {
+ return nil, err
+ }
+ return &res, nil
+}
+
+// Close stops the server or ends the HTTP session.
+func (c *Client) Close() error { return c.tr.close() }
+
+// --- stdio -------------------------------------------------------------------
+
+type stdioTransport struct {
+ cmd *exec.Cmd
+ stdin io.WriteCloser
+ mu sync.Mutex // serializes writes
+ pmu sync.Mutex
+ wait map[string]chan rpcMsg
+ done chan struct{}
+ stderr *limitedBuffer
+ err error
+ waitOnce sync.Once
+}
+
+// reap waits for the process exactly once (Wait isn't concurrency-safe).
+func (t *stdioTransport) reap() { t.waitOnce.Do(func() { _ = t.cmd.Wait() }) }
+
+func newStdio(cfg settings.MCPServer) (*stdioTransport, error) {
+ if cfg.Command == "" {
+ return nil, fmt.Errorf("no command")
+ }
+ args := make([]string, len(cfg.Args))
+ for i, a := range cfg.Args {
+ args[i] = expand(a)
+ }
+ cmd := exec.Command(expand(cfg.Command), args...)
+ cmd.Env = os.Environ()
+ for k, v := range cfg.Env {
+ cmd.Env = append(cmd.Env, k+"="+expand(v))
+ }
+ stdin, err := cmd.StdinPipe()
+ if err != nil {
+ return nil, err
+ }
+ stdout, err := cmd.StdoutPipe()
+ if err != nil {
+ return nil, err
+ }
+ t := &stdioTransport{cmd: cmd, stdin: stdin, wait: map[string]chan rpcMsg{}, done: make(chan struct{}), stderr: &limitedBuffer{b: &bytes.Buffer{}, max: 8 << 10}}
+ cmd.Stderr = t.stderr
+ if err := cmd.Start(); err != nil {
+ return nil, err
+ }
+ go t.read(stdout)
+ return t, nil
+}
+
+func (t *stdioTransport) read(r io.Reader) {
+ defer close(t.done)
+ br := bufio.NewReaderSize(r, 1<<20)
+ for {
+ line, err := br.ReadBytes('\n')
+ if len(bytes.TrimSpace(line)) > 0 {
+ var m rpcMsg
+ if json.Unmarshal(line, &m) == nil {
+ t.dispatch(m)
+ }
+ }
+ if err != nil {
+ t.err = err
+ return
+ }
+ }
+}
+
+func (t *stdioTransport) dispatch(m rpcMsg) {
+ if m.ID == nil {
+ return // notification from the server
+ }
+ if m.Method != "" {
+ // a request from the server (roots/list, sampling…): unsupported
+ _ = t.write(rpcMsg{JSONRPC: "2.0", ID: m.ID, Error: &rpcError{Code: -32601, Message: "not supported by cometcli"}})
+ return
+ }
+ t.pmu.Lock()
+ ch := t.wait[string(*m.ID)]
+ delete(t.wait, string(*m.ID))
+ t.pmu.Unlock()
+ if ch != nil {
+ ch <- m
+ }
+}
+
+func (t *stdioTransport) write(m rpcMsg) error {
+ b, err := json.Marshal(m)
+ if err != nil {
+ return err
+ }
+ t.mu.Lock()
+ defer t.mu.Unlock()
+ _, err = t.stdin.Write(append(b, '\n'))
+ return err
+}
+
+func (t *stdioTransport) roundTrip(ctx context.Context, id int64, method string, params any) (json.RawMessage, error) {
+ rid := json.RawMessage(fmt.Sprint(id))
+ ch := make(chan rpcMsg, 1)
+ t.pmu.Lock()
+ t.wait[string(rid)] = ch
+ t.pmu.Unlock()
+ if err := t.write(rpcMsg{JSONRPC: "2.0", ID: &rid, Method: method, Params: params}); err != nil {
+ return nil, err
+ }
+ select {
+ case m := <-ch:
+ if m.Error != nil {
+ return nil, fmt.Errorf("%s (code %d)", m.Error.Message, m.Error.Code)
+ }
+ return m.Result, nil
+ case <-t.done:
+ t.reap() // stderr is fully copied once the process is reaped
+ msg := strings.TrimSpace(t.stderr.String())
+ if msg == "" && t.err != nil {
+ msg = t.err.Error()
+ }
+ return nil, fmt.Errorf("server exited: %s", lastLines(msg, 3))
+ case <-ctx.Done():
+ t.pmu.Lock()
+ delete(t.wait, string(rid))
+ t.pmu.Unlock()
+ return nil, ctx.Err()
+ }
+}
+
+func (t *stdioTransport) notify(_ context.Context, method string, params any) error {
+ return t.write(rpcMsg{JSONRPC: "2.0", Method: method, Params: params})
+}
+
+func (t *stdioTransport) close() error {
+ t.stdin.Close()
+ select {
+ case <-t.done:
+ case <-time.After(2 * time.Second):
+ _ = t.cmd.Process.Kill()
+ }
+ t.reap()
+ return nil
+}
+
+type limitedBuffer struct {
+ mu sync.Mutex
+ b *bytes.Buffer
+ max int
+}
+
+func (l *limitedBuffer) Write(p []byte) (int, error) {
+ l.mu.Lock()
+ defer l.mu.Unlock()
+ if room := l.max - l.b.Len(); room > 0 {
+ l.b.Write(p[:min(room, len(p))])
+ }
+ return len(p), nil
+}
+
+// String returns what was captured so far.
+func (l *limitedBuffer) String() string {
+ l.mu.Lock()
+ defer l.mu.Unlock()
+ return l.b.String()
+}
+
+func lastLines(s string, n int) string {
+ lines := strings.Split(s, "\n")
+ if len(lines) > n {
+ lines = lines[len(lines)-n:]
+ }
+ return strings.Join(lines, " | ")
+}
+
+// --- streamable HTTP -----------------------------------------------------------
+
+type httpTransport struct {
+ url string
+ headers map[string]string
+ hc *http.Client
+ session atomic.Value // string
+}
+
+func (h *httpTransport) post(ctx context.Context, m rpcMsg) (*http.Response, error) {
+ b, _ := json.Marshal(m)
+ req, err := http.NewRequestWithContext(ctx, "POST", h.url, bytes.NewReader(b))
+ if err != nil {
+ return nil, err
+ }
+ req.Header.Set("Content-Type", "application/json")
+ req.Header.Set("Accept", "application/json, text/event-stream")
+ req.Header.Set("MCP-Protocol-Version", protocolVersion)
+ if s, _ := h.session.Load().(string); s != "" {
+ req.Header.Set("Mcp-Session-Id", s)
+ }
+ for k, v := range h.headers {
+ req.Header.Set(k, v)
+ }
+ resp, err := h.hc.Do(req)
+ if err != nil {
+ return nil, err
+ }
+ if s := resp.Header.Get("Mcp-Session-Id"); s != "" {
+ h.session.Store(s)
+ }
+ if resp.StatusCode/100 != 2 {
+ defer resp.Body.Close()
+ body, _ := io.ReadAll(io.LimitReader(resp.Body, 2048))
+ return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
+ }
+ return resp, nil
+}
+
+func (h *httpTransport) roundTrip(ctx context.Context, id int64, method string, params any) (json.RawMessage, error) {
+ rid := json.RawMessage(fmt.Sprint(id))
+ resp, err := h.post(ctx, rpcMsg{JSONRPC: "2.0", ID: &rid, Method: method, Params: params})
+ if err != nil {
+ return nil, err
+ }
+ defer resp.Body.Close()
+ var m rpcMsg
+ if strings.HasPrefix(resp.Header.Get("Content-Type"), "text/event-stream") {
+ found := false
+ sc := bufio.NewScanner(resp.Body)
+ sc.Buffer(make([]byte, 64<<10), 16<<20)
+ var data []string
+ for sc.Scan() && !found {
+ line := sc.Text()
+ switch {
+ case strings.HasPrefix(line, "data:"):
+ data = append(data, strings.TrimSpace(strings.TrimPrefix(line, "data:")))
+ case line == "" && len(data) > 0:
+ var x rpcMsg
+ if json.Unmarshal([]byte(strings.Join(data, "\n")), &x) == nil && x.ID != nil && string(*x.ID) == string(rid) && x.Method == "" {
+ m, found = x, true
+ }
+ data = nil
+ }
+ }
+ if !found {
+ return nil, errors.New("stream ended without a response")
+ }
+ } else if err := json.NewDecoder(resp.Body).Decode(&m); err != nil {
+ return nil, err
+ }
+ if m.Error != nil {
+ return nil, fmt.Errorf("%s (code %d)", m.Error.Message, m.Error.Code)
+ }
+ return m.Result, nil
+}
+
+func (h *httpTransport) notify(ctx context.Context, method string, params any) error {
+ resp, err := h.post(ctx, rpcMsg{JSONRPC: "2.0", Method: method, Params: params})
+ if err != nil {
+ return err
+ }
+ resp.Body.Close()
+ return nil
+}
+
+func (h *httpTransport) close() error {
+ s, _ := h.session.Load().(string)
+ if s == "" {
+ return nil
+ }
+ req, err := http.NewRequest("DELETE", h.url, nil)
+ if err != nil {
+ return nil
+ }
+ req.Header.Set("Mcp-Session-Id", s)
+ ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
+ defer cancel()
+ if resp, err := h.hc.Do(req.WithContext(ctx)); err == nil {
+ resp.Body.Close()
+ }
+ return nil
+}
+
+// --- ${VAR} expansion ------------------------------------------------------------
+
+// expand substitutes ${VAR} and ${VAR:-default} from the environment.
+func expand(s string) string {
+ return os.Expand(s, func(k string) string {
+ if name, d, ok := strings.Cut(k, ":-"); ok {
+ if v := os.Getenv(name); v != "" {
+ return v
+ }
+ return d
+ }
+ return os.Getenv(k)
+ })
+}
+
+func expandMap(m map[string]string) map[string]string {
+ out := map[string]string{}
+ for k, v := range m {
+ out[k] = expand(v)
+ }
+ return out
+}
diff --git a/internal/mcpclient/client_test.go b/internal/mcpclient/client_test.go
new file mode 100644
index 0000000..d8e9eb5
--- /dev/null
+++ b/internal/mcpclient/client_test.go
@@ -0,0 +1,160 @@
+package mcpclient
+
+import (
+ "bufio"
+ "context"
+ "encoding/json"
+ "fmt"
+ "net/http"
+ "net/http/httptest"
+ "os"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/abhijitkrm/cometcli/internal/settings"
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+)
+
+// TestMain doubles as a tiny stdio MCP server when MCP_FAKE_SERVER=1.
+func TestMain(m *testing.M) {
+ if os.Getenv("MCP_FAKE_SERVER") == "1" {
+ fakeServer()
+ return
+ }
+ os.Exit(m.Run())
+}
+
+func respond(id json.RawMessage, method string, params map[string]any) any {
+ switch method {
+ case "initialize":
+ return map[string]any{"protocolVersion": protocolVersion, "capabilities": map[string]any{"tools": map[string]any{}}, "serverInfo": map[string]any{"name": "fake"}}
+ case "tools/list":
+ if params != nil && params["cursor"] == "p2" {
+ return map[string]any{"tools": []any{map[string]any{"name": "delete_dash", "description": "deletes", "inputSchema": map[string]any{"type": "object"}}}}
+ }
+ return map[string]any{"nextCursor": "p2", "tools": []any{map[string]any{"name": "echo", "description": "echoes", "inputSchema": map[string]any{"type": "object", "properties": map[string]any{"msg": map[string]any{"type": "string"}}}, "annotations": map[string]any{"readOnlyHint": true}}}}
+ case "tools/call":
+ args, _ := params["arguments"].(map[string]any)
+ return map[string]any{"content": []any{map[string]any{"type": "text", "text": fmt.Sprintf("echo: %v", args["msg"])}}}
+ }
+ return nil
+}
+
+func fakeServer() {
+ sc := bufio.NewScanner(os.Stdin)
+ for sc.Scan() {
+ var m struct {
+ ID json.RawMessage `json:"id"`
+ Method string `json:"method"`
+ Params map[string]any `json:"params"`
+ }
+ if json.Unmarshal(sc.Bytes(), &m) != nil || len(m.ID) == 0 {
+ continue
+ }
+ // a server→client request first, to check the client answers it
+ if m.Method == "tools/call" {
+ fmt.Println(`{"jsonrpc":"2.0","id":"srv-1","method":"roots/list"}`)
+ fmt.Println(`{"jsonrpc":"2.0","method":"notifications/message","params":{}}`)
+ }
+ b, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": m.ID, "result": respond(m.ID, m.Method, m.Params)})
+ fmt.Println(string(b))
+ }
+}
+
+func TestStdioServer(t *testing.T) {
+ exe, _ := os.Executable()
+ ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+ defer cancel()
+ c, err := Connect(ctx, "fake", settings.MCPServer{Command: exe, Args: []string{"-test.run=^$"}, Env: map[string]string{"MCP_FAKE_SERVER": "1"}})
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer c.Close()
+ if len(c.Tools) != 2 {
+ t.Fatalf("paginated tools = %+v", c.Tools)
+ }
+ res, err := c.CallTool(ctx, "echo", map[string]any{"msg": "hi"})
+ if err != nil || res.Text() != "echo: hi" {
+ t.Fatalf("call = %+v %v", res, err)
+ }
+
+ // adapter: names, tiers, schema, permission gate
+ echo, del := NewAgentTool(c, c.Tools[0]), NewAgentTool(c, c.Tools[1])
+ if echo.Name() != "mcp__fake__echo" || echo.Tier() != toolkit.TierDiagnose || del.Tier() != toolkit.TierLocalChange {
+ t.Fatalf("adapter: %s %s %s", echo.Name(), echo.Tier(), del.Tier())
+ }
+ if p, ok := del.Schema()["properties"].(map[string]any); !ok || p == nil {
+ t.Fatal("schema properties not normalized")
+ }
+ asked := 0
+ tc := &toolkit.Context{Context: ctx, AutoApproveBelow: toolkit.TierLocalChange,
+ Approver: func(*toolkit.Context, string, toolkit.Tier, map[string]any) (bool, error) { asked++; return false, nil }}
+ if r, err := echo.Run(tc, toolkit.Args{"msg": "x"}); err != nil || r.Text != "echo: x" || asked != 0 {
+ t.Fatalf("read-only tool: %v %v asked=%d", r, err, asked)
+ }
+ if _, err := del.Run(tc, toolkit.Args{}); err == nil || asked != 1 {
+ t.Fatalf("mutating tool must ask: %v asked=%d", err, asked)
+ }
+ tc.Rules, _ = toolkit.NewRules([]string{"mcp__fake"}, nil, nil)
+ if _, err := del.Run(tc, toolkit.Args{}); err != nil || asked != 1 {
+ t.Fatalf("server-wide allow rule ignored: %v asked=%d", err, asked)
+ }
+}
+
+func TestHTTPServerSSEAndSession(t *testing.T) {
+ var sawSession bool
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ if r.Method == "DELETE" {
+ return
+ }
+ var m struct {
+ ID json.RawMessage `json:"id"`
+ Method string `json:"method"`
+ Params map[string]any `json:"params"`
+ }
+ json.NewDecoder(r.Body).Decode(&m)
+ if r.Header.Get("Authorization") != "Bearer tok" {
+ w.WriteHeader(401)
+ return
+ }
+ if m.Method == "initialize" {
+ w.Header().Set("Mcp-Session-Id", "s-1")
+ } else if r.Header.Get("Mcp-Session-Id") == "s-1" {
+ sawSession = true
+ }
+ if len(m.ID) == 0 {
+ w.WriteHeader(202)
+ return
+ }
+ b, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": m.ID, "result": respond(m.ID, m.Method, m.Params)})
+ if m.Method == "tools/call" { // answer as an SSE stream
+ w.Header().Set("Content-Type", "text/event-stream")
+ fmt.Fprintf(w, "event: message\ndata: {\"jsonrpc\":\"2.0\",\"method\":\"notifications/progress\"}\n\ndata: %s\n\n", b)
+ return
+ }
+ w.Header().Set("Content-Type", "application/json")
+ w.Write(b)
+ }))
+ defer srv.Close()
+ t.Setenv("TOK", "tok")
+ ctx := context.Background()
+ c, err := Connect(ctx, "web", settings.MCPServer{Type: "http", URL: srv.URL, Headers: map[string]string{"Authorization": "Bearer ${TOK}"}})
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer c.Close()
+ res, err := c.CallTool(ctx, "echo", map[string]any{"msg": "sse"})
+ if err != nil || res.Text() != "echo: sse" || !sawSession {
+ t.Fatalf("call = %v %v session=%v", res, err, sawSession)
+ }
+}
+
+func TestStdioServerThatExits(t *testing.T) {
+ ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
+ defer cancel()
+ _, err := Connect(ctx, "bad", settings.MCPServer{Command: "sh", Args: []string{"-c", "echo 'boom: missing token' >&2; exit 1"}})
+ if err == nil || !strings.Contains(err.Error(), "boom: missing token") {
+ t.Fatalf("err = %v", err)
+ }
+}
diff --git a/internal/mcpclient/tool.go b/internal/mcpclient/tool.go
new file mode 100644
index 0000000..3344c84
--- /dev/null
+++ b/internal/mcpclient/tool.go
@@ -0,0 +1,103 @@
+package mcpclient
+
+import (
+ "fmt"
+ "regexp"
+ "strings"
+ "time"
+
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+)
+
+// AgentTool adapts one MCP tool to the registry. Its name is
+// mcp____; read-only tools (readOnlyHint) run without a
+// prompt, everything else asks, and rules like "mcp__github" or
+// "mcp__github__create_issue" apply.
+type AgentTool struct {
+ Client *Client
+ Tool Tool
+ name string
+}
+
+var unsafeName = regexp.MustCompile(`[^A-Za-z0-9_-]`)
+
+// ToolName builds the agent-facing name for a server tool.
+func ToolName(server, tool string) string {
+ n := "mcp__" + unsafeName.ReplaceAllString(server, "_") + "__" + unsafeName.ReplaceAllString(tool, "_")
+ if len(n) > 64 {
+ n = n[:64]
+ }
+ return n
+}
+
+// NewAgentTool wraps a server tool.
+func NewAgentTool(c *Client, t Tool) *AgentTool {
+ return &AgentTool{Client: c, Tool: t, name: ToolName(c.Name, t.Name)}
+}
+
+func (t *AgentTool) Name() string { return t.name }
+func (t *AgentTool) AgentOnly() bool { return true }
+func (t *AgentTool) OutputLimit() int { return 30_000 }
+func (t *AgentTool) Timeout(toolkit.Args) time.Duration {
+ return 5*time.Minute + 15*time.Minute // call + approval
+}
+
+func (t *AgentTool) Desc() string {
+ d := strings.TrimSpace(t.Tool.Description)
+ if i := strings.Index(d, "\n\n"); i > 0 && len(d) > 600 {
+ d = d[:i]
+ }
+ if len(d) > 600 {
+ d = d[:600] + "…"
+ }
+ return fmt.Sprintf("(MCP server %s) %s", t.Client.Name, d)
+}
+
+// Schema normalizes the server's input schema for strict validators.
+func (t *AgentTool) Schema() map[string]any {
+ s := map[string]any{}
+ for k, v := range t.Tool.InputSchema {
+ s[k] = v
+ }
+ s["type"] = "object"
+ if p, ok := s["properties"].(map[string]any); !ok || p == nil {
+ s["properties"] = map[string]any{}
+ }
+ if r, ok := s["required"].([]any); ok && len(r) == 0 {
+ delete(s, "required")
+ }
+ if s["required"] == nil {
+ delete(s, "required")
+ }
+ return s
+}
+
+func (t *AgentTool) Tier() toolkit.Tier {
+ if t.Tool.Annotations.ReadOnlyHint && !t.Tool.Annotations.DestructiveHint {
+ return toolkit.TierDiagnose
+ }
+ return toolkit.TierLocalChange
+}
+
+func (t *AgentTool) Run(c *toolkit.Context, a toolkit.Args) (*toolkit.Result, error) {
+ if err := c.Check(toolkit.Gate{
+ Request: toolkit.Request{Tool: t.name},
+ Tier: t.Tier(),
+ Prompt: fmt.Sprintf("call %s on MCP server %s", t.Tool.Name, t.Client.Name),
+ Detail: map[string]any{"arguments": fmt.Sprint(map[string]any(a))},
+ }); err != nil {
+ return nil, err
+ }
+ res, err := t.Client.CallTool(c, t.Tool.Name, a)
+ if err != nil {
+ return nil, err
+ }
+ text := res.Text()
+ if res.IsError {
+ return nil, fmt.Errorf("%s: %s", t.Tool.Name, text)
+ }
+ if text == "" {
+ text = "(no output)"
+ }
+ return &toolkit.Result{Text: text}, nil
+}
diff --git a/internal/redact/redact.go b/internal/redact/redact.go
index 53bb16b..1f55b6d 100644
--- a/internal/redact/redact.go
+++ b/internal/redact/redact.go
@@ -34,7 +34,7 @@ var (
// Text scrubs sensitive material from s.
func Text(s string) string {
s = mnemonicRe.ReplaceAllStringFunc(s, scrubMnemonic)
- s = hex64Re.ReplaceAllString(s, "[REDACTED_HEX]")
+ s = scrubHex64(s)
s = privValRe.ReplaceAllString(s, `"$1": "[REDACTED]"`)
s = keyValueRe.ReplaceAllString(s, `"value": "[REDACTED]"`)
s = kvSecretRe.ReplaceAllString(s, "[REDACTED_SECRET]")
@@ -44,6 +44,33 @@ func Text(s string) string {
return s
}
+// hashLabelRe ends with a label that marks the following hex as a hash
+// (tx, block, app hash): public data the operator needs, not a key.
+var hashLabelRe = regexp.MustCompile(`(?i)(hash["']?\s*(:|=)?\s*["']?|\btxs?\s+(get\s+)?(--hash\s+)?|\b(query|q)\s+tx\s+(--type[= ]hash\s+)?)$`)
+
+// scrubHex64 redacts 64-char hex (a private key's shape) unless it is
+// labelled as a hash: "tx hash: AB…", "txhash=…", "\"hash\": \"0x…\"".
+func scrubHex64(s string) string {
+ locs := hex64Re.FindAllStringIndex(s, -1)
+ if locs == nil {
+ return s
+ }
+ var b strings.Builder
+ last := 0
+ for _, l := range locs {
+ start := max(0, l[0]-24)
+ b.WriteString(s[last:l[0]])
+ if hashLabelRe.MatchString(s[start:l[0]]) {
+ b.WriteString(s[l[0]:l[1]])
+ } else {
+ b.WriteString("[REDACTED_HEX]")
+ }
+ last = l[1]
+ }
+ b.WriteString(s[last:])
+ return b.String()
+}
+
// mnemonicLen is the shortest BIP-39 mnemonic (128-bit entropy).
const mnemonicLen = 12
diff --git a/internal/redact/redact_test.go b/internal/redact/redact_test.go
index 437e4c8..83f1bb8 100644
--- a/internal/redact/redact_test.go
+++ b/internal/redact/redact_test.go
@@ -188,3 +188,27 @@ func TestHostOf(t *testing.T) {
}
}
}
+
+func TestHashesAreNotKeys(t *testing.T) {
+ h := "B240273F16D1F385FB9C4FC5CEB0820ECE4CC7C2C03B159FD22359CA7DAA014C"
+ for _, in := range []string{"tx hash: " + h, "txhash=" + h, `{"hash": "0x` + strings.ToLower(h) + `"}`, "app_hash=" + h, "TxHash " + h} {
+ if got := Text(in); !strings.Contains(strings.ToUpper(got), h) {
+ t.Errorf("hash redacted: %q → %q", in, got)
+ }
+ }
+ key := strings.ToLower(h)
+ for _, in := range []string{"key " + key, "priv=" + key, key, "export: 0x" + key} {
+ if got := Text(in); strings.Contains(got, key) {
+ t.Errorf("key not redacted: %q", got)
+ }
+ }
+}
+
+func TestTxQueryArgumentIsAHash(t *testing.T) {
+ h := "B240273F16D1F385FB9C4FC5CEB0820ECE4CC7C2C03B159FD22359CA7DAA014C"
+ for _, in := range []string{"evmd query tx " + h + " --node x", "evmd q tx " + h, "cometcli tx get " + h} {
+ if got := Text(in); !strings.Contains(got, h) {
+ t.Errorf("%q → %q", in, got)
+ }
+ }
+}
diff --git a/internal/runbook/runbook.go b/internal/runbook/runbook.go
index a7a2a30..6c71820 100644
--- a/internal/runbook/runbook.go
+++ b/internal/runbook/runbook.go
@@ -101,6 +101,20 @@ func Get(name string) (*Runbook, error) {
// Run executes a runbook against a context, printing progress.
func Run(rb *Runbook, c *toolkit.Context, reg *toolkit.Registry, out io.Writer) error {
+ // validate every step before running any: a typo in step 5 must not
+ // surface after steps 1-4 already changed things
+ for _, s := range rb.Steps {
+ if s.Manual != "" || s.Tool == "" {
+ continue
+ }
+ t, ok := reg.Get(s.Tool)
+ if !ok {
+ return fmt.Errorf("runbook %q step %q references unknown tool %q", rb.Name, s.Name, s.Tool)
+ }
+ if toolkit.IsOperatorOnly(t) && c.ToolName != "" {
+ return fmt.Errorf("runbook %q step %q uses %s, which handles key material — run the runbook yourself with `cometcli runbook run %s`", rb.Name, s.Name, s.Tool, rb.Name)
+ }
+ }
fmt.Fprintf(out, "▶ runbook: %s — %s\n\n", rb.Name, rb.Desc)
for i, s := range rb.Steps {
fmt.Fprintf(out, "── step %d/%d: %s\n", i+1, len(rb.Steps), s.Name)
@@ -114,10 +128,10 @@ func Run(rb *Runbook, c *toolkit.Context, reg *toolkit.Registry, out io.Writer)
}
continue
}
- t, ok := reg.Get(s.Tool)
- if !ok {
- return fmt.Errorf("runbook references unknown tool %q", s.Tool)
+ if s.Tool == "" {
+ continue // note-only step
}
+ t, _ := reg.Get(s.Tool)
res, err := t.Run(c, s.Args)
if err != nil {
if s.Optional {
diff --git a/internal/runbook/runbook_test.go b/internal/runbook/runbook_test.go
new file mode 100644
index 0000000..05c44ec
--- /dev/null
+++ b/internal/runbook/runbook_test.go
@@ -0,0 +1,102 @@
+package runbook
+
+import (
+ "bytes"
+ "context"
+ "errors"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+)
+
+type stub struct {
+ name string
+ err error
+ ran *[]string
+ op bool
+}
+
+func (s stub) Name() string { return s.name }
+func (s stub) Desc() string { return s.name }
+func (s stub) Schema() map[string]any { return toolkit.ObjSchema(nil) }
+func (s stub) Tier() toolkit.Tier { return toolkit.TierObserve }
+func (s stub) OperatorOnly() bool { return s.op }
+func (s stub) Run(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) {
+ *s.ran = append(*s.ran, s.name)
+ return &toolkit.Result{Text: s.name + " ok"}, s.err
+}
+
+func setup() (*toolkit.Registry, *[]string) {
+ var ran []string
+ reg := toolkit.NewRegistry()
+ reg.Register(stub{name: "a.ok", ran: &ran})
+ reg.Register(stub{name: "a.fail", ran: &ran, err: errors.New("boom")})
+ reg.Register(stub{name: "keys.add", ran: &ran, op: true})
+ return reg, &ran
+}
+
+func TestRunStepsOptionalAndManual(t *testing.T) {
+ reg, ran := setup()
+ asked := 0
+ c := &toolkit.Context{Context: context.Background(), Approver: func(*toolkit.Context, string, toolkit.Tier, map[string]any) (bool, error) { asked++; return true, nil }}
+ rb := &Runbook{Name: "r", Steps: []Step{
+ {Name: "one", Tool: "a.ok"},
+ {Name: "may fail", Tool: "a.fail", Optional: true},
+ {Name: "human", Manual: "restart the sentry"},
+ {Name: "note only", Note: "just a note"},
+ {Name: "two", Tool: "a.ok"},
+ }}
+ var out bytes.Buffer
+ if err := Run(rb, c, reg, &out); err != nil {
+ t.Fatal(err)
+ }
+ if strings.Join(*ran, ",") != "a.ok,a.fail,a.ok" || asked != 1 || !strings.Contains(out.String(), "✓ runbook complete") {
+ t.Fatalf("ran=%v asked=%d out=%s", *ran, asked, out.String())
+ }
+ *ran = nil
+ rb.Steps[1].Optional = false
+ if err := Run(rb, c, reg, &out); err == nil || !strings.Contains(err.Error(), `step "may fail" failed`) || len(*ran) != 2 {
+ t.Fatalf("required failure: %v ran=%v", err, *ran)
+ }
+}
+
+func TestRunValidatesBeforeRunningAnything(t *testing.T) {
+ reg, ran := setup()
+ c := &toolkit.Context{Context: context.Background()}
+ rb := &Runbook{Name: "r", Steps: []Step{{Name: "one", Tool: "a.ok"}, {Name: "typo", Tool: "a.okk"}}}
+ if err := Run(rb, c, reg, &bytes.Buffer{}); err == nil || len(*ran) != 0 {
+ t.Fatalf("unknown tool must fail before step 1 runs: %v ran=%v", err, *ran)
+ }
+ // key management steps: fine for the operator, refused for the agent
+ rb = &Runbook{Name: "keys", Steps: []Step{{Name: "add", Tool: "keys.add"}}}
+ if err := Run(rb, c, reg, &bytes.Buffer{}); err != nil {
+ t.Fatalf("operator run: %v", err)
+ }
+ c.ToolName = "runbook.run"
+ *ran = nil
+ if err := Run(rb, c, reg, &bytes.Buffer{}); err == nil || len(*ran) != 0 {
+ t.Fatalf("agent ran a key-management step: %v", err)
+ }
+}
+
+func TestCustomRunbooksLoad(t *testing.T) {
+ home := t.TempDir()
+ t.Setenv("COMETCLI_HOME", home)
+ os.MkdirAll(filepath.Join(home, "runbooks"), 0o700)
+ os.WriteFile(filepath.Join(home, "runbooks", "mine.yaml"), []byte("name: my-check\ndesc: checks\nsteps:\n - name: status\n tool: node.status\n - name: human\n manual: do it\n"), 0o600)
+ rb, err := GetAll("my-check")
+ if err != nil || len(rb.Steps) != 2 || rb.Steps[1].Manual != "do it" {
+ t.Fatalf("custom = %+v %v", rb, err)
+ }
+ if len(Builtins()) == 0 {
+ t.Fatal("no builtin runbooks")
+ }
+ for _, b := range Builtins() {
+ if _, err := GetAll(b.Name); err != nil {
+ t.Errorf("builtin %s: %v", b.Name, err)
+ }
+ }
+}
diff --git a/internal/scenario/jail_test.go b/internal/scenario/jail_test.go
new file mode 100644
index 0000000..7caceb7
--- /dev/null
+++ b/internal/scenario/jail_test.go
@@ -0,0 +1,251 @@
+// Package scenario runs whole incidents end to end: the real agent loop,
+// the real tool registry and the incident procedures, against a simulated
+// chain (fakenode gRPC + fake CometBFT RPC) that evolves while the agent
+// works. The model is scripted: it only reacts to what the tools return,
+// which proves the tools surface enough to drive the recovery.
+package scenario
+
+import (
+ "context"
+ "encoding/json"
+ "fmt"
+ "os"
+ "strings"
+ "sync"
+ "testing"
+ "time"
+
+ slashingv1beta1 "cosmossdk.io/api/cosmos/slashing/v1beta1"
+ stakingv1beta1 "cosmossdk.io/api/cosmos/staking/v1beta1"
+ "google.golang.org/protobuf/types/known/timestamppb"
+
+ "github.com/abhijitkrm/cometcli/internal/agent"
+ "github.com/abhijitkrm/cometcli/internal/client/host"
+ "github.com/abhijitkrm/cometcli/internal/config"
+ "github.com/abhijitkrm/cometcli/internal/testutil/fakenode"
+ "github.com/abhijitkrm/cometcli/internal/toolkit"
+ "github.com/abhijitkrm/cometcli/internal/tools"
+ "github.com/abhijitkrm/cometcli/internal/tools/waittool"
+ "github.com/abhijitkrm/cometcli/internal/tx"
+)
+
+func TestMain(m *testing.M) {
+ fakenode.MaybeRunFakeDocker()
+ waittool.MinInterval = 10 * time.Millisecond
+ tx.ConfirmTimeout, tx.ConfirmPoll = time.Second, 20*time.Millisecond
+ os.Exit(m.Run())
+}
+
+const jailLogs = `E[2026-10-06|21:41:13] Stopping peer for error err="read tcp 172.18.0.3:26656: i/o timeout" module=p2p
+E[2026-10-06|21:41:40] dial tcp 10.0.0.2:26656: connect: connection refused module=p2p
+INF slashing and jailing validator due to liveness fault height=162050 jailed_until=2026-10-06T21:54:31Z module=x/slashing
+`
+
+// scripted plays the model: each response depends only on the last
+// message, the way the /recover-jail procedure tells a model to act.
+type scripted struct {
+ t *testing.T
+ syncStart chan struct{} // closed when the model starts waiting for sync
+ mu sync.Mutex
+ calls []string
+ seen map[string]string // tool → last result text
+}
+
+func call(name string, args map[string]any) *agent.Response {
+ b, _ := json.Marshal(args)
+ return &agent.Response{Calls: []agent.Call{{ID: fmt.Sprintf("c%d", time.Now().UnixNano()), Name: name, Args: b}}}
+}
+
+func (s *scripted) Name() string { return "scripted" }
+func (s *scripted) Chat(_ context.Context, r *agent.Request) (*agent.Response, error) {
+ s.mu.Lock()
+ defer s.mu.Unlock()
+ last := r.Messages[len(r.Messages)-1]
+ if last.Role == "user" {
+ if !strings.Contains(last.Text, "val.jail-check") {
+ s.t.Errorf("first prompt isn't the recovery procedure: %.200s", last.Text)
+ }
+ s.calls = append(s.calls, "val.jail-check")
+ return call("val__jail-check", nil), nil
+ }
+ s.seen[last.ToolName] = last.Text
+ next := func(name string, args map[string]any) (*agent.Response, error) {
+ s.calls = append(s.calls, name)
+ return call(strings.ReplaceAll(name, ".", "__"), args), nil
+ }
+ fast := map[string]any{"interval": 0.02, "timeout": 20}
+ with := func(kv ...any) map[string]any {
+ m := map[string]any{}
+ for k, v := range fast {
+ m[k] = v
+ }
+ for i := 0; i < len(kv); i += 2 {
+ m[kv[i].(string)] = kv[i+1]
+ }
+ return m
+ }
+ switch last.ToolName {
+ case "val.jail-check":
+ if strings.Contains(last.Text, "TOMBSTONED") {
+ return &agent.Response{Text: "tombstoned — stopping", Done: true}, nil
+ }
+ if strings.Contains(last.Text, "node not synced") {
+ close(s.syncStart) // the simulated node starts catching up now
+ return next("wait.until", with("condition", "synced"))
+ }
+ return next("wait.until", with("condition", "unjailable"))
+ case "wait.until":
+ switch {
+ case strings.Contains(last.Text, "✓ synced"):
+ return next("wait.until", with("condition", "unjailable"))
+ case strings.Contains(last.Text, "✓ unjailable"):
+ return next("tool_search", map[string]any{"query": "select:val.unjail"})
+ case strings.Contains(last.Text, "✓ in-consensus"):
+ return next("val.consensus", map[string]any{"window": 20})
+ }
+ return &agent.Response{Text: "wait failed: " + last.Text, Done: true}, nil
+ case "tool_search":
+ return next("val.unjail", nil)
+ case "val.unjail":
+ switch {
+ case strings.Contains(last.Text, "tx hash"):
+ return next("wait.until", with("condition", "in-consensus", "window", 20, "min_signed_pct", 95))
+ case strings.Contains(last.Text, "raise the gas price"):
+ return next("val.unjail", map[string]any{"gas-price": "2000000000"})
+ }
+ return &agent.Response{Text: "unjail failed: " + last.Text, Done: true}, nil
+ case "val.consensus":
+ return &agent.Response{Text: "Recovered. " + strings.ReplaceAll(last.Text, "\n", "; "), Done: true}, nil
+ }
+ return &agent.Response{Text: "unexpected tool " + last.ToolName, Done: true}, nil
+}
+
+func TestJailRecoveryEndToEnd(t *testing.T) {
+ n := fakenode.Start(t, "primium-1")
+ chain := fakenode.StartComet(t)
+ p := n.Profile(t, "eth_secp256k1")
+ fakenode.WithComet(p, chain)
+ p.Service = config.Service{Type: "docker", Unit: "primium-validator0"}
+ p.Home = t.TempDir()
+ fakenode.InstallFakeDocker(t, "test", "")
+ fakenode.SetNodeLogs(t, jailLogs, "status=running restarts=2 oom_killed=false")
+
+ c := &toolkit.Context{Context: context.Background(), Profile: p, Cfg: &config.Config{Profiles: map[string]*config.Profile{p.Name: p}}}
+ c.SetHost(&host.Local{})
+ t.Cleanup(c.Close)
+ tb, err := c.Tx()
+ if err != nil {
+ t.Fatal(err)
+ }
+ valoper, _ := tb.ValAddress()
+ p.Metadata["valoper"] = valoper
+
+ // the incident: jailed for downtime, jail period ends shortly in
+ // chain time, the node is behind and syncing, and the node's minimum
+ // gas price was raised so the first unjail is rejected for its fee
+ n.Set(func(n *fakenode.Node) {
+ n.Validator = &stakingv1beta1.Validator{OperatorAddress: valoper, ConsensusPubkey: fakenode.ConsPubAny(),
+ Jailed: true, Status: stakingv1beta1.BondStatus_BOND_STATUS_UNBONDING, MinSelfDelegation: "1"}
+ n.SigningInfo = &slashingv1beta1.ValidatorSigningInfo{JailedUntil: timestamppb.New(time.Now().Add(20 * time.Second))}
+ n.CheckFailOnce = &struct {
+ Code uint32
+ Log string
+ }{13, "insufficient fee; got: 140000000000000adex required: 280000000000000adex"}
+ n.OnCommit = func(n *fakenode.Node, t fakenode.Tx) {
+ if t.Msgs[0].TypeUrl == "/cosmos.slashing.v1beta1.MsgUnjail" {
+ n.Validator.Jailed = false
+ n.Validator.Status = stakingv1beta1.BondStatus_BOND_STATUS_BONDED
+ chain.Set(func(c *fakenode.Comet) {
+ from := c.Height + 2
+ c.InSet, c.Signs = true, func(h int64) bool { return h >= from }
+ })
+ }
+ }
+ })
+ behind := time.Now().Add(-15 * time.Minute)
+ chain.Set(func(c *fakenode.Comet) { c.InSet, c.CatchingUp, c.LatestTimeOverride = false, true, &behind })
+ stop := make(chan struct{})
+ t.Cleanup(func() { close(stop) })
+ syncStart := make(chan struct{})
+ go func() { // the chain keeps producing blocks; the node catches up
+ select { // stays behind until the agent is waiting for it
+ case <-stop:
+ return
+ case <-syncStart:
+ }
+ for i := 1; ; i++ {
+ select {
+ case <-stop:
+ return
+ case <-time.After(15 * time.Millisecond):
+ }
+ chain.Set(func(c *fakenode.Comet) {
+ c.Height += 2
+ if c.CatchingUp {
+ lt := behind.Add(time.Duration(i) * 30 * time.Second)
+ c.LatestTimeOverride = <
+ if i >= 20 {
+ c.CatchingUp, c.LatestTimeOverride = false, nil
+ }
+ }
+ })
+ }
+ }()
+
+ reg := toolkit.NewRegistry()
+ tools.RegisterAll(reg)
+ p.Agent = config.AgentConf{Provider: "openai-compat", BaseURL: "http://127.0.0.1:1"}
+ a, err := agent.New(c, reg)
+ if err != nil {
+ t.Fatal(err)
+ }
+ model := &scripted{t: t, seen: map[string]string{}, syncStart: syncStart}
+ a.Provider = model
+ var txApprovals, signerChoices int
+ a.Ctx.Chooser = func(_ *toolkit.Context, _ string, opts []string) (int, error) {
+ signerChoices++
+ return 0, nil // cometcli keyring
+ }
+ a.Ctx.Approver = func(_ *toolkit.Context, _ string, tier toolkit.Tier, _ map[string]any) (bool, error) {
+ if tier == toolkit.TierOnChain {
+ txApprovals++
+ }
+ return true, nil
+ }
+
+ cmd, err := agent.RunCommand(a, a.Ctx, reg, "/recover-jail peers were flapping overnight")
+ if err != nil || !strings.Contains(cmd.Prompt, "peers were flapping overnight") {
+ t.Fatalf("/recover-jail: %v %v", cmd, err)
+ }
+ ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
+ defer cancel()
+ out, err := a.Run(ctx, cmd.Prompt)
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ want := "val.jail-check,wait.until,wait.until,tool_search,val.unjail,val.unjail,wait.until,val.consensus"
+ if got := strings.Join(model.calls, ","); got != want {
+ t.Fatalf("procedure:\n got %s\n want %s\nlast results: %v", got, want, model.seen)
+ }
+ jc := model.seen["val.jail-check"]
+ for _, w := range []string{"reason: downtime", "liveness fault", "peer loss / network", "restarts=2", "node not synced"} {
+ if !strings.Contains(jc, w) {
+ t.Errorf("jail-check missing %q:\n%s", w, jc)
+ }
+ }
+ bs, rej, _ := n.Snapshot()
+ if len(bs) != 1 || len(rej) != 1 || !strings.Contains(rej[0], "insufficient fee") || txApprovals != 2 {
+ t.Fatalf("broadcasts=%d rejected=%v approvals=%d", len(bs), rej, txApprovals)
+ }
+ if fee := bs[0].Fee.Amount[0].Amount; fee == "" || len(fee) < 15 {
+ t.Fatalf("retry fee = %v", bs[0].Fee)
+ }
+ if signerChoices != 2 {
+ t.Fatalf("signer asked %d times, want once per unjail attempt", signerChoices)
+ }
+ if !strings.Contains(out, "participating in consensus") || strings.Contains(out, "NOT participating") {
+ t.Fatalf("final: %s", out)
+ }
+ t.Logf("final answer: %s", out)
+}
diff --git a/internal/serve/serve.go b/internal/serve/serve.go
index 2cf1433..62fa9d0 100644
--- a/internal/serve/serve.go
+++ b/internal/serve/serve.go
@@ -378,5 +378,13 @@ func (s *Server) command(w http.ResponseWriter, r *http.Request) {
writeJSON(w, code, map[string]string{"error": err.Error()})
return
}
+ if res.Job != nil {
+ txt, err := res.Job(r.Context())
+ if err != nil {
+ writeJSON(w, http.StatusBadGateway, map[string]string{"error": err.Error()})
+ return
+ }
+ res.Text = txt
+ }
writeJSON(w, http.StatusOK, map[string]string{"text": res.Text, "prompt": res.Prompt})
}
diff --git a/internal/settings/markdown.go b/internal/settings/markdown.go
new file mode 100644
index 0000000..bc6489e
--- /dev/null
+++ b/internal/settings/markdown.go
@@ -0,0 +1,417 @@
+package settings
+
+import (
+ "fmt"
+ "io/fs"
+ "os"
+ "path/filepath"
+ "regexp"
+ "sort"
+ "strings"
+
+ "gopkg.in/yaml.v3"
+
+ "github.com/abhijitkrm/cometcli/internal/config"
+)
+
+// --- memory ----------------------------------------------------------------
+
+// Memory is one loaded COMET.md-style file.
+type Memory struct {
+ Path string
+ Scope string // user | project | local | node
+ Content string
+}
+
+// maxMemory bounds all memory text injected into the system prompt.
+const maxMemory = 40_000
+
+// LoadMemory reads, in order: ~/.cometcli/COMET.md; COMET.md (or
+// AGENTS.md when a directory has no COMET.md) and COMET.local.md in each
+// directory from the project root down to cwd; and in node mode
+// ~/.cometcli/memory/.md. "@path" lines import another file.
+func LoadMemory(root, cwd, profile string) []Memory {
+ var out []Memory
+ seen := map[string]bool{}
+ add := func(p, scope string) {
+ if seen[p] {
+ return
+ }
+ seen[p] = true
+ if b, err := os.ReadFile(p); err == nil && len(strings.TrimSpace(string(b))) > 0 {
+ out = append(out, Memory{Path: p, Scope: scope, Content: expandImports(string(b), filepath.Dir(p), 0)})
+ }
+ }
+ if d, err := config.Dir(); err == nil {
+ add(filepath.Join(d, "COMET.md"), "user")
+ }
+ for _, dir := range chain(root, cwd) {
+ if _, err := os.Stat(filepath.Join(dir, "COMET.md")); err == nil {
+ add(filepath.Join(dir, "COMET.md"), "project")
+ } else {
+ add(filepath.Join(dir, "AGENTS.md"), "project")
+ }
+ add(filepath.Join(dir, "COMET.local.md"), "local")
+ }
+ if profile != "" {
+ if p, err := NodeMemoryPath(profile); err == nil {
+ add(p, "node")
+ }
+ }
+ return out
+}
+
+// NodeMemoryPath is the per-profile memory file.
+func NodeMemoryPath(profile string) (string, error) {
+ return config.Path("memory", profile+".md")
+}
+
+// chain lists root, then each directory down to cwd (just cwd without a
+// root).
+func chain(root, cwd string) []string {
+ cwd = filepath.Clean(cwd)
+ if root == "" {
+ return []string{cwd}
+ }
+ rel, err := filepath.Rel(root, cwd)
+ if err != nil || strings.HasPrefix(rel, "..") {
+ return []string{cwd}
+ }
+ dirs := []string{root}
+ if rel == "." {
+ return dirs
+ }
+ d := root
+ for _, part := range strings.Split(rel, string(filepath.Separator)) {
+ d = filepath.Join(d, part)
+ dirs = append(dirs, d)
+ }
+ return dirs
+}
+
+var importRe = regexp.MustCompile(`(?m)^@(\S+)\s*$`)
+
+// keyFileRe matches files whose content must never enter a prompt.
+var keyFileRe = regexp.MustCompile(`priv_validator_key\.json|node_key\.json|mnemonic|keyring-|/\.ssh/|\.cometcli/keys`)
+
+func expandImports(s, base string, depth int) string {
+ if depth >= 3 {
+ return s
+ }
+ return importRe.ReplaceAllStringFunc(s, func(line string) string {
+ p := strings.TrimSpace(line[1:])
+ if strings.HasPrefix(p, "~/") {
+ home, _ := os.UserHomeDir()
+ p = filepath.Join(home, p[2:])
+ } else if !filepath.IsAbs(p) {
+ p = filepath.Join(base, p)
+ }
+ if keyFileRe.MatchString(strings.ToLower(p)) {
+ return "[import of key material refused: " + line[1:] + "]"
+ }
+ b, err := os.ReadFile(p)
+ if err != nil {
+ return line
+ }
+ return expandImports(string(b), filepath.Dir(p), depth+1)
+ })
+}
+
+// RenderMemory formats memory for the system prompt.
+func RenderMemory(ms []Memory) string {
+ if len(ms) == 0 {
+ return ""
+ }
+ var b strings.Builder
+ b.WriteString("MEMORY — standing instructions from the operator's COMET.md files; follow them:\n")
+ for _, m := range ms {
+ fmt.Fprintf(&b, "### %s (%s)\n%s\n", m.Path, m.Scope, strings.TrimSpace(m.Content))
+ }
+ s := b.String()
+ if len(s) > maxMemory {
+ s = s[:maxMemory] + "\n…[memory truncated — trim your COMET.md files]\n"
+ }
+ return s + "\n"
+}
+
+// AppendMemory adds a bullet to a memory file, creating it if needed.
+func AppendMemory(path, text string) error {
+ if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+ return err
+ }
+ f, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644)
+ if err != nil {
+ return err
+ }
+ defer f.Close()
+ if fi, _ := f.Stat(); fi != nil && fi.Size() > 0 {
+ if b, _ := os.ReadFile(path); len(b) > 0 && b[len(b)-1] != '\n' {
+ if _, err := f.WriteString("\n"); err != nil {
+ return err
+ }
+ }
+ }
+ _, err = f.WriteString("- " + strings.TrimSpace(text) + "\n")
+ return err
+}
+
+// --- frontmatter -------------------------------------------------------------
+
+// splitFrontmatter separates a leading "---" YAML block from the body.
+func splitFrontmatter(s string) (map[string]any, string) {
+ if !strings.HasPrefix(s, "---\n") && !strings.HasPrefix(s, "---\r\n") {
+ return map[string]any{}, s
+ }
+ rest := s[strings.Index(s, "\n")+1:]
+ end := strings.Index(rest, "\n---")
+ if end < 0 {
+ return map[string]any{}, s
+ }
+ meta := map[string]any{}
+ _ = yaml.Unmarshal([]byte(rest[:end]), &meta)
+ body := rest[end+4:]
+ body = strings.TrimPrefix(strings.TrimPrefix(body, "\r"), "\n")
+ return meta, body
+}
+
+func metaStr(m map[string]any, keys ...string) string {
+ for _, k := range keys {
+ if v, ok := m[k]; ok && v != nil {
+ return strings.TrimSpace(fmt.Sprint(v))
+ }
+ }
+ return ""
+}
+
+func metaList(m map[string]any, keys ...string) []string {
+ for _, k := range keys {
+ switch v := m[k].(type) {
+ case string:
+ var out []string
+ for _, p := range splitRules(v) {
+ if p != "" {
+ out = append(out, p)
+ }
+ }
+ return out
+ case []any:
+ var out []string
+ for _, x := range v {
+ out = append(out, strings.TrimSpace(fmt.Sprint(x)))
+ }
+ return out
+ }
+ }
+ return nil
+}
+
+// splitRules splits "Bash(git status:*), Read" on commas outside parens.
+func splitRules(s string) []string {
+ var out []string
+ depth, start := 0, 0
+ for i, c := range s {
+ switch c {
+ case '(':
+ depth++
+ case ')':
+ depth--
+ case ',':
+ if depth == 0 {
+ out = append(out, strings.TrimSpace(s[start:i]))
+ start = i + 1
+ }
+ }
+ }
+ return append(out, strings.TrimSpace(s[start:]))
+}
+
+// mdFiles collects *.md under dir, keyed by name: the relative path
+// without extension, subdirectories joined with ":".
+func mdFiles(dir string) map[string]string {
+ out := map[string]string{}
+ _ = filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
+ if err != nil || d.IsDir() || !strings.HasSuffix(p, ".md") {
+ return nil
+ }
+ rel, _ := filepath.Rel(dir, strings.TrimSuffix(p, ".md"))
+ out[strings.ReplaceAll(rel, string(filepath.Separator), ":")] = p
+ return nil
+ })
+ return out
+}
+
+// --- custom slash commands ---------------------------------------------------
+
+// Command is a markdown-defined slash command.
+type Command struct {
+ Name string
+ Description string
+ ArgHint string
+ AllowedTools []string // permission rules allowed while it runs
+ Body string
+ Path string
+ Scope string // user | project
+}
+
+// LoadCommands reads ~/.cometcli/commands and /.cometcli/commands;
+// a project command shadows a user one of the same name.
+func LoadCommands(root string) map[string]*Command {
+ out := map[string]*Command{}
+ load := func(dir, scope string) {
+ for name, p := range mdFiles(dir) {
+ raw, err := os.ReadFile(p)
+ if err != nil {
+ continue
+ }
+ meta, body := splitFrontmatter(string(raw))
+ c := &Command{
+ Name: name, Path: p, Scope: scope, Body: body,
+ Description: metaStr(meta, "description"),
+ ArgHint: metaStr(meta, "argument-hint", "argument_hint"),
+ AllowedTools: metaList(meta, "allowed-tools", "allowed_tools"),
+ }
+ if c.Description == "" {
+ c.Description = firstLine(body)
+ }
+ out[name] = c
+ }
+ }
+ if d, err := config.Dir(); err == nil {
+ load(filepath.Join(d, "commands"), "user")
+ }
+ if root != "" {
+ load(filepath.Join(root, ".cometcli", "commands"), "project")
+ }
+ return out
+}
+
+// SortedCommands returns commands by name.
+func SortedCommands(m map[string]*Command) []*Command {
+ out := make([]*Command, 0, len(m))
+ for _, c := range m {
+ out = append(out, c)
+ }
+ sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
+ return out
+}
+
+var (
+ bangRe = regexp.MustCompile("!`([^`]+)`")
+ atRe = regexp.MustCompile(`(^|\s)@([\w./~-][\w./~:-]*)`)
+ posRe = regexp.MustCompile(`\$([1-9])`)
+)
+
+// Expand fills the command template: $ARGUMENTS and $1..$9, then
+// !`cmd` (replaced by run's output) and @path (replaced by read's
+// content; unreadable paths are left as written).
+func (c *Command) Expand(args string, run func(cmd string) string, read func(path string) (string, bool)) string {
+ pos := shellSplit(args)
+ s := strings.ReplaceAll(c.Body, "$ARGUMENTS", args)
+ s = posRe.ReplaceAllStringFunc(s, func(m string) string {
+ i := int(m[1] - '1')
+ if i < len(pos) {
+ return pos[i]
+ }
+ return ""
+ })
+ if run != nil {
+ s = bangRe.ReplaceAllStringFunc(s, func(m string) string {
+ return run(bangRe.FindStringSubmatch(m)[1])
+ })
+ }
+ if read != nil {
+ s = atRe.ReplaceAllStringFunc(s, func(m string) string {
+ sub := atRe.FindStringSubmatch(m)
+ if body, ok := read(sub[2]); ok {
+ return sub[1] + "\n\n" + body + "\n\n"
+ }
+ return m
+ })
+ }
+ return strings.TrimSpace(s)
+}
+
+// shellSplit splits on spaces, honoring single and double quotes.
+func shellSplit(s string) []string {
+ var out []string
+ var cur strings.Builder
+ var q rune
+ in := false
+ for _, c := range s {
+ switch {
+ case q != 0:
+ if c == q {
+ q = 0
+ } else {
+ cur.WriteRune(c)
+ }
+ case c == '\'' || c == '"':
+ q, in = c, true
+ case c == ' ' || c == '\t':
+ if in {
+ out = append(out, cur.String())
+ cur.Reset()
+ in = false
+ }
+ default:
+ cur.WriteRune(c)
+ in = true
+ }
+ }
+ if in {
+ out = append(out, cur.String())
+ }
+ return out
+}
+
+func firstLine(s string) string {
+ for _, l := range strings.Split(s, "\n") {
+ if l = strings.TrimSpace(strings.TrimLeft(l, "# ")); l != "" {
+ if len(l) > 80 {
+ l = l[:80] + "…"
+ }
+ return l
+ }
+ }
+ return ""
+}
+
+// --- subagents ---------------------------------------------------------------
+
+// AgentDef is a markdown-defined subagent.
+type AgentDef struct {
+ Name string
+ Description string
+ Tools []string // tool names it may use (empty = the general tools)
+ Prompt string // its system prompt
+ Path string
+}
+
+// LoadAgents reads ~/.cometcli/agents and /.cometcli/agents.
+func LoadAgents(root string) map[string]*AgentDef {
+ out := map[string]*AgentDef{}
+ load := func(dir string) {
+ for name, p := range mdFiles(dir) {
+ raw, err := os.ReadFile(p)
+ if err != nil {
+ continue
+ }
+ meta, body := splitFrontmatter(string(raw))
+ if n := metaStr(meta, "name"); n != "" {
+ name = n
+ }
+ out[name] = &AgentDef{
+ Name: name, Path: p, Prompt: strings.TrimSpace(body),
+ Description: metaStr(meta, "description"),
+ Tools: metaList(meta, "tools"),
+ }
+ }
+ }
+ if d, err := config.Dir(); err == nil {
+ load(filepath.Join(d, "agents"))
+ }
+ if root != "" {
+ load(filepath.Join(root, ".cometcli", "agents"))
+ }
+ return out
+}
diff --git a/internal/settings/settings.go b/internal/settings/settings.go
new file mode 100644
index 0000000..7c72431
--- /dev/null
+++ b/internal/settings/settings.go
@@ -0,0 +1,328 @@
+// Package settings discovers and merges cometcli's file-based
+// extensibility: settings.json at user, project and local scope, COMET.md
+// memory files, custom slash commands, subagent definitions and MCP
+// server configs. Formats follow Claude Code's, so its configs carry over.
+//
+// Project-scope hooks and MCP servers execute commands, so they only take
+// effect once the project directory is trusted (`cometcli trust`).
+package settings
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "os"
+ "path/filepath"
+ "sort"
+ "strings"
+
+ "github.com/abhijitkrm/cometcli/internal/config"
+)
+
+// Scope says which file a setting came from.
+type Scope string
+
+const (
+ ScopeUser Scope = "user" // ~/.cometcli/settings.json
+ ScopeProject Scope = "project" // /.cometcli/settings.json
+ ScopeLocal Scope = "local" // /.cometcli/settings.local.json
+)
+
+// Permissions mirrors Claude Code's permissions block.
+type Permissions struct {
+ Allow []string `json:"allow,omitempty"`
+ Ask []string `json:"ask,omitempty"`
+ Deny []string `json:"deny,omitempty"`
+ DefaultMode string `json:"defaultMode,omitempty"`
+}
+
+// HookCommand is one command run for a hook event.
+type HookCommand struct {
+ Type string `json:"type"` // "command"
+ Command string `json:"command"`
+ Timeout int `json:"timeout,omitempty"` // seconds, default 60
+}
+
+// HookMatcher binds commands to tools whose name matches Matcher (a
+// regular expression, case-insensitive; "" or "*" matches all).
+type HookMatcher struct {
+ Matcher string `json:"matcher,omitempty"`
+ Hooks []HookCommand `json:"hooks"`
+ // Scope is where the matcher was defined (set on load).
+ Scope Scope `json:"-"`
+}
+
+// MCPServer is an MCP server definition (stdio or HTTP).
+type MCPServer struct {
+ Type string `json:"type,omitempty"` // stdio (default) | http
+ Command string `json:"command,omitempty"`
+ Args []string `json:"args,omitempty"`
+ Env map[string]string `json:"env,omitempty"`
+ URL string `json:"url,omitempty"`
+ Headers map[string]string `json:"headers,omitempty"`
+ // Scope is where the server was defined (set on load).
+ Scope Scope `json:"-"`
+}
+
+// File is one settings.json document.
+type File struct {
+ Model string `json:"model,omitempty"`
+ Effort string `json:"effort,omitempty"`
+ Permissions Permissions `json:"permissions,omitempty"`
+ Env map[string]string `json:"env,omitempty"`
+ Hooks map[string][]HookMatcher `json:"hooks,omitempty"`
+ MCPServers map[string]MCPServer `json:"mcpServers,omitempty"`
+}
+
+// Settings is the merged view for one session.
+type Settings struct {
+ Root string // project root ("" when none was found)
+ Trusted bool // project hooks/MCP servers may run
+ Model string
+ Effort string
+ Permissions Permissions
+ Env map[string]string
+ Hooks map[string][]HookMatcher
+ MCPServers map[string]MCPServer
+ // Sources lists the files that were read, in precedence order.
+ Sources []string
+ // Ignored explains project hooks/servers skipped for lack of trust.
+ Ignored []string
+}
+
+// FindRoot walks up from dir to the nearest directory holding a
+// .cometcli/ folder, a COMET.md or a .git, stopping at the home directory.
+func FindRoot(dir string) string {
+ home, _ := os.UserHomeDir()
+ d := filepath.Clean(dir)
+ for {
+ if d != home {
+ for _, marker := range []string{".cometcli", "COMET.md", ".git"} {
+ if _, err := os.Stat(filepath.Join(d, marker)); err == nil {
+ return d
+ }
+ }
+ }
+ parent := filepath.Dir(d)
+ if parent == d || d == home {
+ return ""
+ }
+ d = parent
+ }
+}
+
+// Load merges user, project and local settings for a session started in
+// cwd. Later scopes override scalar keys; permission rules, env, hooks
+// and MCP servers accumulate (a later server definition of the same name
+// wins).
+func Load(cwd string) (*Settings, error) {
+ s := &Settings{Root: FindRoot(cwd), Env: map[string]string{}, Hooks: map[string][]HookMatcher{}, MCPServers: map[string]MCPServer{}}
+ s.Trusted = s.Root != "" && IsTrusted(s.Root)
+ userDir, err := config.Dir()
+ if err != nil {
+ return nil, err
+ }
+ type src struct {
+ path string
+ scope Scope
+ }
+ srcs := []src{{filepath.Join(userDir, "settings.json"), ScopeUser}}
+ if s.Root != "" {
+ srcs = append(srcs,
+ src{filepath.Join(s.Root, ".cometcli", "settings.json"), ScopeProject},
+ src{filepath.Join(s.Root, ".cometcli", "settings.local.json"), ScopeLocal})
+ }
+ for _, sc := range srcs {
+ f, err := readFile(sc.path)
+ if err != nil {
+ return nil, err
+ }
+ if f == nil {
+ continue
+ }
+ s.Sources = append(s.Sources, sc.path)
+ s.apply(f, sc.scope)
+ }
+ // a project's .mcp.json (Claude Code's file) adds project servers
+ if s.Root != "" {
+ var mj struct {
+ MCPServers map[string]MCPServer `json:"mcpServers"`
+ }
+ p := filepath.Join(s.Root, ".mcp.json")
+ if raw, err := os.ReadFile(p); err == nil {
+ if err := json.Unmarshal(raw, &mj); err != nil {
+ return nil, fmt.Errorf("%s: %w", p, err)
+ }
+ s.Sources = append(s.Sources, p)
+ for n, srv := range mj.MCPServers {
+ s.addServer(n, srv, ScopeProject)
+ }
+ }
+ }
+ return s, nil
+}
+
+func readFile(p string) (*File, error) {
+ raw, err := os.ReadFile(p)
+ if errors.Is(err, os.ErrNotExist) {
+ return nil, nil
+ }
+ if err != nil {
+ return nil, err
+ }
+ var f File
+ if err := json.Unmarshal(raw, &f); err != nil {
+ return nil, fmt.Errorf("%s: %w", p, err)
+ }
+ return &f, nil
+}
+
+func (s *Settings) apply(f *File, scope Scope) {
+ if f.Model != "" {
+ s.Model = f.Model
+ }
+ if f.Effort != "" {
+ s.Effort = f.Effort
+ }
+ if f.Permissions.DefaultMode != "" {
+ s.Permissions.DefaultMode = f.Permissions.DefaultMode
+ }
+ s.Permissions.Allow = append(s.Permissions.Allow, f.Permissions.Allow...)
+ s.Permissions.Ask = append(s.Permissions.Ask, f.Permissions.Ask...)
+ s.Permissions.Deny = append(s.Permissions.Deny, f.Permissions.Deny...)
+ for k, v := range f.Env {
+ s.Env[k] = v
+ }
+ for ev, ms := range f.Hooks {
+ for _, m := range ms {
+ if scope != ScopeUser && !s.Trusted {
+ s.Ignored = append(s.Ignored, fmt.Sprintf("%s hook (%s settings)", ev, scope))
+ continue
+ }
+ m.Scope = scope
+ s.Hooks[ev] = append(s.Hooks[ev], m)
+ }
+ }
+ for n, srv := range f.MCPServers {
+ s.addServer(n, srv, scope)
+ }
+}
+
+func (s *Settings) addServer(name string, srv MCPServer, scope Scope) {
+ if scope != ScopeUser && !s.Trusted {
+ s.Ignored = append(s.Ignored, fmt.Sprintf("MCP server %q (%s)", name, scope))
+ return
+ }
+ srv.Scope = scope
+ s.MCPServers[name] = srv
+}
+
+// ApplyEnv exports the settings' env vars into this process (before
+// providers read their API keys). Existing variables are not overridden.
+func (s *Settings) ApplyEnv() {
+ for k, v := range s.Env {
+ if _, set := os.LookupEnv(k); !set {
+ _ = os.Setenv(k, v)
+ }
+ }
+}
+
+// --- trust -----------------------------------------------------------------
+
+func trustFile() (string, error) { return config.Path("trusted.json") }
+
+func trustedList() []string {
+ p, err := trustFile()
+ if err != nil {
+ return nil
+ }
+ raw, err := os.ReadFile(p)
+ if err != nil {
+ return nil
+ }
+ var dirs []string
+ _ = json.Unmarshal(raw, &dirs)
+ return dirs
+}
+
+// IsTrusted reports whether dir (or a parent of it) was trusted.
+func IsTrusted(dir string) bool {
+ dir = filepath.Clean(dir)
+ for _, t := range trustedList() {
+ if dir == t || strings.HasPrefix(dir, t+string(filepath.Separator)) {
+ return true
+ }
+ }
+ return false
+}
+
+// Trust marks dir as trusted (or untrusted when on is false).
+func Trust(dir string, on bool) error {
+ dir, err := filepath.Abs(dir)
+ if err != nil {
+ return err
+ }
+ var out []string
+ for _, t := range trustedList() {
+ if t != dir {
+ out = append(out, t)
+ }
+ }
+ if on {
+ out = append(out, dir)
+ }
+ sort.Strings(out)
+ raw, _ := json.MarshalIndent(out, "", " ")
+ p, err := trustFile()
+ if err != nil {
+ return err
+ }
+ return os.WriteFile(p, raw, 0o600)
+}
+
+// Trusted lists trusted directories.
+func Trusted() []string { return trustedList() }
+
+// --- user settings editing (cometcli mcp add/remove) -----------------------
+
+// EditFile loads, mutates and saves one settings file.
+func EditFile(path string, fn func(*File) error) error {
+ f, err := readFile(path)
+ if err != nil {
+ return err
+ }
+ if f == nil {
+ f = &File{}
+ }
+ // keep keys we don't model by round-tripping through a generic map
+ raw, _ := os.ReadFile(path)
+ var generic map[string]any
+ _ = json.Unmarshal(raw, &generic)
+ if err := fn(f); err != nil {
+ return err
+ }
+ b, _ := json.Marshal(f)
+ var mine map[string]any
+ _ = json.Unmarshal(b, &mine)
+ if generic == nil {
+ generic = map[string]any{}
+ }
+ for _, k := range []string{"model", "effort", "permissions", "env", "hooks", "mcpServers"} {
+ if v, ok := mine[k]; ok {
+ generic[k] = v
+ } else {
+ delete(generic, k)
+ }
+ }
+ out, err := json.MarshalIndent(generic, "", " ")
+ if err != nil {
+ return err
+ }
+ if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
+ return err
+ }
+ return os.WriteFile(path, append(out, '\n'), 0o600)
+}
+
+// UserPath is the user settings file.
+func UserPath() (string, error) { return config.Path("settings.json") }
diff --git a/internal/settings/settings_test.go b/internal/settings/settings_test.go
new file mode 100644
index 0000000..157a44d
--- /dev/null
+++ b/internal/settings/settings_test.go
@@ -0,0 +1,160 @@
+package settings
+
+import (
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+)
+
+func setup(t *testing.T) (home, proj string) {
+ t.Helper()
+ home = t.TempDir()
+ t.Setenv("HOME", home)
+ t.Setenv("COMETCLI_HOME", filepath.Join(home, ".cometcli"))
+ proj = filepath.Join(home, "work", "node-setup")
+ os.MkdirAll(filepath.Join(proj, ".cometcli", "commands", "ops"), 0o755)
+ os.MkdirAll(filepath.Join(proj, "run-validator"), 0o755)
+ os.MkdirAll(filepath.Join(home, ".cometcli"), 0o700)
+ return home, proj
+}
+
+func write(t *testing.T, p, s string) {
+ t.Helper()
+ os.MkdirAll(filepath.Dir(p), 0o755)
+ if err := os.WriteFile(p, []byte(s), 0o644); err != nil {
+ t.Fatal(err)
+ }
+}
+
+func TestLoadMergesScopesAndGatesTrust(t *testing.T) {
+ home, proj := setup(t)
+ write(t, filepath.Join(home, ".cometcli", "settings.json"), `{"model":"m-user","permissions":{"allow":["bash(git status:*)"]},
+ "env":{"FOO":"user","KEEP":"x"},"hooks":{"PreToolUse":[{"matcher":"bash","hooks":[{"type":"command","command":"true"}]}]},
+ "mcpServers":{"u":{"command":"u-srv"}}}`)
+ write(t, filepath.Join(proj, ".cometcli", "settings.json"), `{"effort":"low","permissions":{"deny":["bash(rm:*)"],"defaultMode":"acceptEdits"},
+ "hooks":{"Stop":[{"hooks":[{"type":"command","command":"evil"}]}]},"mcpServers":{"p":{"command":"p-srv"}}}`)
+ write(t, filepath.Join(proj, ".cometcli", "settings.local.json"), `{"model":"m-local","env":{"FOO":"local"}}`)
+ write(t, filepath.Join(proj, ".mcp.json"), `{"mcpServers":{"shared":{"type":"http","url":"https://x/mcp"}}}`)
+
+ s, err := Load(filepath.Join(proj, "run-validator"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if s.Root != proj || s.Model != "m-local" || s.Effort != "low" || s.Permissions.DefaultMode != "acceptEdits" {
+ t.Fatalf("merge: root=%s model=%s effort=%s mode=%s", s.Root, s.Model, s.Effort, s.Permissions.DefaultMode)
+ }
+ if len(s.Permissions.Allow) != 1 || len(s.Permissions.Deny) != 1 || s.Env["FOO"] != "local" {
+ t.Fatalf("accumulate: %+v %v", s.Permissions, s.Env)
+ }
+ // untrusted: project hooks and servers skipped, user ones kept
+ if len(s.Hooks["Stop"]) != 0 || len(s.Hooks["PreToolUse"]) != 1 {
+ t.Fatalf("hooks = %+v", s.Hooks)
+ }
+ if _, ok := s.MCPServers["p"]; ok {
+ t.Fatal("untrusted project server loaded")
+ }
+ if _, ok := s.MCPServers["u"]; !ok || len(s.Ignored) != 3 {
+ t.Fatalf("servers=%v ignored=%v", s.MCPServers, s.Ignored)
+ }
+
+ if err := Trust(proj, true); err != nil {
+ t.Fatal(err)
+ }
+ s, _ = Load(filepath.Join(proj, "run-validator"))
+ if !s.Trusted || len(s.Hooks["Stop"]) != 1 || s.MCPServers["shared"].URL != "https://x/mcp" || s.MCPServers["p"].Scope != ScopeProject {
+ t.Fatalf("trusted load: %+v", s)
+ }
+ Trust(proj, false)
+ if IsTrusted(proj) {
+ t.Fatal("trust not revoked")
+ }
+
+ os.Setenv("KEEP", "already")
+ s.ApplyEnv()
+ if os.Getenv("KEEP") != "already" || os.Getenv("FOO") != "local" {
+ t.Fatal("ApplyEnv overrode an existing var or missed one")
+ }
+ os.Unsetenv("FOO")
+}
+
+func TestEditFileKeepsUnknownKeys(t *testing.T) {
+ home, _ := setup(t)
+ p := filepath.Join(home, ".cometcli", "settings.json")
+ write(t, p, `{"statusLine":{"type":"command"},"model":"x"}`)
+ if err := EditFile(p, func(f *File) error {
+ f.MCPServers = map[string]MCPServer{"g": {Command: "grafana"}}
+ return nil
+ }); err != nil {
+ t.Fatal(err)
+ }
+ b, _ := os.ReadFile(p)
+ if !strings.Contains(string(b), "statusLine") || !strings.Contains(string(b), `"grafana"`) || !strings.Contains(string(b), `"model": "x"`) {
+ t.Fatalf("file = %s", b)
+ }
+}
+
+func TestMemoryOrderAndImports(t *testing.T) {
+ home, proj := setup(t)
+ write(t, filepath.Join(home, ".cometcli", "COMET.md"), "user rule")
+ write(t, filepath.Join(proj, "COMET.md"), "project rule\n@docs/ports.md\n@config/priv_validator_key.json")
+ write(t, filepath.Join(proj, "docs", "ports.md"), "p2p is 26656")
+ write(t, filepath.Join(proj, "run-validator", "AGENTS.md"), "validator dir rule")
+ write(t, filepath.Join(home, ".cometcli", "memory", "val01.md"), "val01 runs in docker")
+ ms := LoadMemory(proj, filepath.Join(proj, "run-validator"), "val01")
+ var scopes []string
+ for _, m := range ms {
+ scopes = append(scopes, m.Scope)
+ }
+ if strings.Join(scopes, ",") != "user,project,project,node" {
+ t.Fatalf("scopes = %v", scopes)
+ }
+ r := RenderMemory(ms)
+ for _, want := range []string{"user rule", "p2p is 26656", "validator dir rule", "val01 runs in docker", "import of key material refused"} {
+ if !strings.Contains(r, want) {
+ t.Errorf("memory missing %q", want)
+ }
+ }
+ p := filepath.Join(proj, "COMET.md")
+ AppendMemory(p, "always check peers first")
+ b, _ := os.ReadFile(p)
+ if !strings.HasSuffix(string(b), "\n- always check peers first\n") {
+ t.Fatalf("append = %q", b)
+ }
+}
+
+func TestCommandsLoadAndExpand(t *testing.T) {
+ home, proj := setup(t)
+ write(t, filepath.Join(home, ".cometcli", "commands", "status.md"), "user version")
+ write(t, filepath.Join(proj, ".cometcli", "commands", "status.md"), "---\ndescription: Node status\nargument-hint: [profile]\nallowed-tools: Bash(docker ps:*), Read\n---\nCheck $ARGUMENTS ($1 then $2).\nContainers: !`docker ps`\nSee @notes.md")
+ write(t, filepath.Join(proj, ".cometcli", "commands", "ops", "vote.md"), "# Vote on upgrade $1")
+ cs := LoadCommands(proj)
+ st := cs["status"]
+ if st == nil || st.Scope != "project" || st.Description != "Node status" || st.ArgHint != "[profile]" {
+ t.Fatalf("status = %+v", st)
+ }
+ if len(st.AllowedTools) != 2 || st.AllowedTools[0] != "Bash(docker ps:*)" {
+ t.Fatalf("allowed-tools = %q", st.AllowedTools)
+ }
+ if cs["ops:vote"] == nil || cs["ops:vote"].Description != "Vote on upgrade $1" {
+ t.Fatalf("namespaced = %+v", cs["ops:vote"])
+ }
+ out := st.Expand(`val01 "two words"`,
+ func(c string) string { return "[ran " + c + "]" },
+ func(p string) (string, bool) { return "NOTES", p == "notes.md" })
+ for _, want := range []string{`Check val01 "two words" (val01 then two words).`, "[ran docker ps]", ``, "NOTES"} {
+ if !strings.Contains(out, want) {
+ t.Errorf("expansion missing %q in %q", want, out)
+ }
+ }
+}
+
+func TestAgentsLoad(t *testing.T) {
+ _, proj := setup(t)
+ write(t, filepath.Join(proj, ".cometcli", "agents", "log-digger.md"), "---\nname: logs\ndescription: trawls logs\ntools: bash, grep, read\n---\nYou read logs and report anomalies.")
+ as := LoadAgents(proj)
+ d := as["logs"]
+ if d == nil || d.Description != "trawls logs" || len(d.Tools) != 3 || !strings.Contains(d.Prompt, "anomalies") {
+ t.Fatalf("agent = %+v", d)
+ }
+}
diff --git a/internal/testutil/fakenode/comet.go b/internal/testutil/fakenode/comet.go
new file mode 100644
index 0000000..2ab48c0
--- /dev/null
+++ b/internal/testutil/fakenode/comet.go
@@ -0,0 +1,177 @@
+package fakenode
+
+import (
+ "crypto/ed25519"
+ "crypto/sha256"
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "net/http/httptest"
+ "strconv"
+ "sync"
+ "testing"
+ "time"
+
+ cmted25519 "github.com/cometbft/cometbft/crypto/ed25519"
+ cmtjson "github.com/cometbft/cometbft/libs/json"
+ "github.com/cometbft/cometbft/p2p"
+ coretypes "github.com/cometbft/cometbft/rpc/core/types"
+ "github.com/cometbft/cometbft/types"
+ "google.golang.org/protobuf/encoding/protowire"
+ "google.golang.org/protobuf/types/known/anypb"
+)
+
+// Comet is a fake CometBFT JSON-RPC endpoint for a simulated chain the
+// test drives: height and block times advance as the test says, the node
+// can be catching up, and our validator signs (or not) per height.
+type Comet struct {
+ URL string
+
+ mu sync.Mutex
+ Height int64
+ BlockTime time.Duration // spacing of block times
+ Genesis time.Time // time of block 1
+ CatchingUp bool
+ // LatestTimeOverride, when set, is the latest block's time (a stalled
+ // or far-behind node).
+ LatestTimeOverride *time.Time
+ // Signs reports whether our validator signed height h (default: yes).
+ Signs func(h int64) bool
+ // InSet is whether our validator is in the active set.
+ InSet bool
+ Power int64
+ Peers int
+ // NodeKeyAddr, when set, is the consensus address the node reports
+ // signing with (a node running the wrong priv_validator_key).
+ NodeKeyAddr []byte
+}
+
+// ConsPub is the test validator's consensus key (ed25519, fixed seed).
+func ConsPub() ed25519.PublicKey {
+ return ed25519.NewKeyFromSeed(make([]byte, 32)).Public().(ed25519.PublicKey)
+}
+
+// ConsAddr is the CometBFT address of ConsPub (sha256(pub)[:20]).
+func ConsAddr() []byte {
+ s := sha256.Sum256(ConsPub())
+ return s[:20]
+}
+
+// ConsPubAny is ConsPub as the staking module stores it.
+func ConsPubAny() *anypb.Any {
+ var v []byte
+ v = protowire.AppendTag(v, 1, protowire.BytesType)
+ v = protowire.AppendBytes(v, ConsPub())
+ return &anypb.Any{TypeUrl: "/cosmos.crypto.ed25519.PubKey", Value: v}
+}
+
+// StartComet runs the fake RPC for the test's lifetime.
+func StartComet(t testing.TB) *Comet {
+ t.Helper()
+ c := &Comet{Height: 1000, BlockTime: time.Second, InSet: true, Power: 10, Peers: 3}
+ c.Genesis = time.Now().Add(-time.Duration(c.Height) * c.BlockTime)
+ srv := httptest.NewServer(http.HandlerFunc(c.handle))
+ t.Cleanup(srv.Close)
+ c.URL = srv.URL
+ return c
+}
+
+// Set mutates the chain under its lock.
+func (c *Comet) Set(fn func(c *Comet)) {
+ c.mu.Lock()
+ defer c.mu.Unlock()
+ fn(c)
+}
+
+// Advance adds n blocks (time moves with them).
+func (c *Comet) Advance(n int64) { c.Set(func(c *Comet) { c.Height += n }) }
+
+func (c *Comet) timeAt(h int64) time.Time {
+ if h == c.Height && c.LatestTimeOverride != nil {
+ return *c.LatestTimeOverride
+ }
+ return c.Genesis.Add(time.Duration(h) * c.BlockTime)
+}
+
+func (c *Comet) handle(w http.ResponseWriter, r *http.Request) {
+ body, _ := io.ReadAll(r.Body)
+ var req struct {
+ ID json.RawMessage `json:"id"`
+ Method string `json:"method"`
+ Params map[string]any `json:"params"`
+ }
+ _ = json.Unmarshal(body, &req)
+ c.mu.Lock()
+ res, err := c.result(req.Method, req.Params)
+ c.mu.Unlock()
+ w.Header().Set("Content-Type", "application/json")
+ if err != nil {
+ fmt.Fprintf(w, `{"jsonrpc":"2.0","id":%s,"error":{"code":-32603,"message":%q,"data":%q}}`, req.ID, err.Error(), err.Error())
+ return
+ }
+ raw, _ := cmtjson.Marshal(res)
+ fmt.Fprintf(w, `{"jsonrpc":"2.0","id":%s,"result":%s}`, req.ID, raw)
+}
+
+func (c *Comet) heightParam(p map[string]any) int64 {
+ switch v := p["height"].(type) {
+ case string:
+ h, _ := strconv.ParseInt(v, 10, 64)
+ return h
+ case float64:
+ return int64(v)
+ }
+ return c.Height
+}
+
+func (c *Comet) result(method string, p map[string]any) (any, error) {
+ addr := ConsAddr()
+ switch method {
+ case "health":
+ return &coretypes.ResultHealth{}, nil
+ case "status":
+ if c.NodeKeyAddr != nil {
+ addr = c.NodeKeyAddr
+ }
+ power := int64(0)
+ if c.InSet {
+ power = c.Power
+ }
+ return &coretypes.ResultStatus{
+ NodeInfo: p2p.DefaultNodeInfo{Network: "primium-1", Version: "0.39.3", Moniker: "validator-01"},
+ SyncInfo: coretypes.SyncInfo{LatestBlockHeight: c.Height, LatestBlockTime: c.timeAt(c.Height), CatchingUp: c.CatchingUp},
+ ValidatorInfo: coretypes.ValidatorInfo{Address: addr, PubKey: cmted25519.PubKey(ConsPub()), VotingPower: power},
+ }, nil
+ case "net_info":
+ return &coretypes.ResultNetInfo{Listening: true, NPeers: c.Peers}, nil
+ case "commit":
+ h := c.heightParam(p)
+ if h > c.Height {
+ return nil, fmt.Errorf("height %d must be less than or equal to the current blockchain height %d", h, c.Height)
+ }
+ flag := types.BlockIDFlagAbsent
+ if c.InSet && (c.Signs == nil || c.Signs(h)) {
+ flag = types.BlockIDFlagCommit
+ }
+ sigs := []types.CommitSig{{BlockIDFlag: types.BlockIDFlagCommit, ValidatorAddress: make([]byte, 20), Timestamp: c.timeAt(h), Signature: make([]byte, 64)}}
+ if c.InSet {
+ cs := types.CommitSig{BlockIDFlag: flag, ValidatorAddress: addr, Timestamp: c.timeAt(h)}
+ if flag == types.BlockIDFlagCommit {
+ cs.Signature = make([]byte, 64)
+ } else {
+ cs.ValidatorAddress, cs.Timestamp = nil, time.Time{}
+ }
+ sigs = append(sigs, cs)
+ }
+ hdr := &types.Header{ChainID: "primium-1", Height: h, Time: c.timeAt(h)}
+ return coretypes.NewResultCommit(hdr, &types.Commit{Height: h, Signatures: sigs}, true), nil
+ case "validators":
+ vals := []*types.Validator{types.NewValidator(cmted25519.GenPrivKeyFromSecret([]byte("other")).PubKey(), 100)}
+ if c.InSet {
+ vals = append(vals, types.NewValidator(cmted25519.PubKey(ConsPub()), c.Power))
+ }
+ return &coretypes.ResultValidators{BlockHeight: c.heightParam(p), Validators: vals, Count: len(vals), Total: len(vals)}, nil
+ }
+ return nil, fmt.Errorf("method %s not found", method)
+}
diff --git a/internal/testutil/fakenode/fakedocker.go b/internal/testutil/fakenode/fakedocker.go
new file mode 100644
index 0000000..e354a02
--- /dev/null
+++ b/internal/testutil/fakenode/fakedocker.go
@@ -0,0 +1,229 @@
+package fakenode
+
+import (
+ "bufio"
+ "encoding/base64"
+ "encoding/hex"
+ "encoding/json"
+ "fmt"
+ "os"
+ "path/filepath"
+ "regexp"
+ "strconv"
+ "strings"
+ "testing"
+
+ "github.com/decred/dcrd/dcrec/secp256k1/v4"
+ "github.com/decred/dcrd/dcrec/secp256k1/v4/ecdsa"
+ "golang.org/x/crypto/sha3"
+ "google.golang.org/protobuf/encoding/protojson"
+ "google.golang.org/protobuf/encoding/protowire"
+ "google.golang.org/protobuf/proto"
+ "google.golang.org/protobuf/reflect/protoregistry"
+ "google.golang.org/protobuf/types/known/anypb"
+
+ basev1beta1 "cosmossdk.io/api/cosmos/base/v1beta1"
+ signingv1beta1 "cosmossdk.io/api/cosmos/tx/signing/v1beta1"
+ txv1beta1 "cosmossdk.io/api/cosmos/tx/v1beta1"
+
+ "github.com/abhijitkrm/cometcli/internal/keys"
+)
+
+// TestKeyHex is the fixed test key every fake uses — never use it for real.
+const TestKeyHex = "4c0883a69102937d6231471b5dbb6204fe5129617082792ae468d01a3f362318"
+
+// TestAddress is the eth_secp256k1 cosmos address of TestKeyHex.
+func TestAddress() string {
+ priv, _ := hex.DecodeString(TestKeyHex)
+ pk := secp256k1.PrivKeyFromBytes(priv)
+ addr := keys.AddressFor(keys.AlgoEthSecp256k1, pk.PubKey())
+ k := keys.Key{Address: addr}
+ s, _ := k.Bech32("cosmos")
+ return s
+}
+
+// InstallFakeDocker puts a fake `docker` first on PATH: the test binary
+// itself, which must call MaybeRunFakeDocker from TestMain. keyring is
+// the backend the container's key lives in ("test" or "file"); password
+// is the file keyring's password.
+func InstallFakeDocker(t testing.TB, keyring, password string) {
+ t.Helper()
+ exe, err := os.Executable()
+ if err != nil {
+ t.Fatal(err)
+ }
+ dir := t.TempDir()
+ if err := os.Symlink(exe, filepath.Join(dir, "docker")); err != nil {
+ t.Fatal(err)
+ }
+ t.Setenv("PATH", dir+string(os.PathListSeparator)+os.Getenv("PATH"))
+ t.Setenv("FAKENODE_DOCKER", "1")
+ t.Setenv("FAKENODE_KEYRING", keyring)
+ t.Setenv("FAKENODE_PASSWORD", password)
+}
+
+// MaybeRunFakeDocker runs the fake when the process was started as the
+// fake docker, then exits. Call it first thing in TestMain.
+func MaybeRunFakeDocker() {
+ if os.Getenv("FAKENODE_DOCKER") != "1" || filepath.Base(os.Args[0]) != "docker" {
+ return
+ }
+ os.Exit(fakeDocker(os.Args[1:]))
+}
+
+var (
+ reKeyring = regexp.MustCompile(`--keyring-backend '?([a-z]+)'?`)
+ reB64 = regexp.MustCompile(`printf %s '([A-Za-z0-9+/=]+)'`)
+ reAccNum = regexp.MustCompile(`--account-number (\d+)`)
+ reSeq = regexp.MustCompile(`--sequence (\d+)`)
+ reChain = regexp.MustCompile(`--chain-id '([^']+)'`)
+)
+
+// fakeDocker emulates `docker exec -i sh -c 