Skip to content

feat(workflow): sessions in hosts teamai cannot detect name their tool with --tools (#1479) - #1480

Merged
SaulMoro merged 15 commits into
Tencent:mainfrom
hz89Dev:saulmoro/skill-session-tool
Oct 27, 2026
Merged

SaulMoro merged 15 commits into
Tencent:mainfrom
hz89Dev:saulmoro/skill-session-tool

Conversation

@hz89Dev

@hz89Dev hz89Dev commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Oh My Pi, Qoder, Kiro, ZCode, Trae, OpenClaw, Hermes and DeepSeek Harness set no session variable teamai reads. So a skill-driven workflow there could not resolve a step's model, and #1386's session judge never ran. --tools was refused with --driver skill, so the session had no way to say which tool it is. Now it names its tool with --tools default=<tool>, describe tells it when it has to, and the run keeps that tool to the end.

 teamai workflow start --describe <wf> --driver skill
+  says: "This coding session names no tool ...: describe the workflow again with --tools default=<tool>"
 teamai workflow start|resume ... --driver skill
-  --tools                    refused: "--tools is for --driver local"
+  --tools default=<tool>     the tool this session works in
 skill.json orchestrator
+  tool                       kept with the owner; runners resolve child models for it, done.score leases judging to it
  • Every command teamai prints for the session keeps --tools: the starts describe offers, the describe a refused start prints, and the resume a refused resume prints.
  • The tool moves with ownership: a takeover, or a session that comes to own an unowned run, works in its own tool. A resume with --tools repairs a run started without one.
  • Models resolve only for tools teamai installs agents for. Judging works for every tool.
  • A session whose host starts no child agent (h4) no longer gets the judge work. A judging-off event says why (no_child_agent).
  • features.sessionJudge in TOOLS records, per host, whether it starts a fresh subagent, with evidence. The README scorers cell shows ✓³ from it for Oh My Pi, Pi, OpenClaw, Hermes, DeepSeek Harness, Qoder, Qoder CN, Kiro, Trae, Trae CN and ZCode, in every README.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature causing existing behavior to change)
  • Documentation only
  • Refactor / internal cleanup

Test Plan

  • npx tsc --noEmit passes
  • npm run lint passes
  • npx vitest run passes: 17633 passed.
  • Added/updated tests for the change. Each new driver test was seen failing first.
  • npm run test:e2e -- src/__tests__/e2e/workflow-skill-claim.test.ts src/__tests__/e2e/score-external-driver.test.ts src/__tests__/e2e/workflow-session-1086.test.ts: 6 passed.

Real CLI, after npm run build, with no agent session variable set (Oh My Pi 18.12.6 on PATH). It runs a workflow whose one prompt step pins model alias strong and is scored by review-quality, from describe through done to the stored label:

# Before (base commit)
## $ teamai workflow start --describe review --driver skill
{"kind":"elsewhere","reason":"The coding session's tool has no confined judge teamai runs."}
## $ teamai workflow start --describe review --driver skill --tools default=omp
{"error":{"code":"invalid_arguments","message":"--tools is for --driver local","next":null}}
# After (this branch)
## $ teamai workflow start --describe review --driver skill
{"judging":{"kind":"elsewhere","reason":"This coding session names no tool."},"says":"- This coding session names no tool for its prompt steps and judging: describe the workflow again with --tools default=<tool>, the tool it works in, and start it from that reply.\n- review is not judged on this machine. This coding session names no tool."}
## $ teamai workflow start --describe review --driver skill --tools default=omp
{"judging":{"kind":"session","tool":"omp","reason":"The coding session's tool, omp, has no confined judge teamai runs."},"starts":["teamai workflow start review --driver skill --tools default=omp","teamai workflow start review --driver skill --tools default=omp --no-judge"]}
## dispatch of review, prepared by its detached runner
{"step":"review","model":{"by":"step","name":"anthropic/claude-opus-4-5","alias":"strong"}}
## skill.json orchestrator
{"tool":"omp","lastCallAt":"2026-10-27T20:36:27.318Z"}
## $ teamai workflow next <run> --driver skill   (after the review step reported)
{"status":"done","actions":[{"kind":"done","outcome":"succeeded","prs":[],"score":"teamai score --next --json '{\"run\":\"r2580524fd85fbbe2\",\"driver\":{\"kind\":\"skill\",\"machine\":\"<machine>\",\"workspace\":\"<root>/app\",\"can\":{\"judge\":{\"isolation\":\"best-effort\",\"access\":\"read\",\"tools\":[\"omp\"]}}},\"running\":[]}'"}]}
## $ <done.score>
{"subject":{"runId":"r2580524fd85fbbe2","step":"review","pass":1},"judge":{"tool":"omp","by":"tool_default","isolation":"best-effort","access":"read"},"submit":"teamai score --lease s55e5b013922bc/ab5f49714 --submit --stdin"}
## $ teamai score --lease s55e5b013922bc/ab5f49714 --submit --stdin   (from the lease cwd)
thorough
## stored label
{"scorer":"review-quality","label":"thorough","source":"workflow","subject":{"runId":"r2580524fd85fbbe2","step":"review","pass":1},"judge":{"kind":"model","tool":"omp","model":{"reported":"anthropic/claude-opus-4-5"},"isolation":"best-effort","access":"read"}}

Not run here: the other providers (gitlab, github) and agents (Codex, CodeBuddy, OpenCode). This change touches only the skill driver's session tool, so I leave those to CI. The ✓³ cells come from each host's docs or source, as each sessionJudge note says. None was run in that host.

Related Issues

Closes #1479

Notes for Reviewers

  • sessionJudge marks: Pi and OpenClaw are ◐. Bare Pi starts no subagent and answers h4, and an OpenClaw tool policy can remove sessions_spawn. DeepSeek Harness is ◐ because its minimal preset leaves the subagent tool out. JoyCode stays ?.
  • New session variables (HERMES_SESSION_ID, DSH_SESSION_ID) are left to [feat] Read the sessions of more tools, so they can be scored #1418. They also change invoke evidence, recall and teamai score's current session.
  • Rows R10, R11, R20, S1 and S4 of docs/dev/workflow-engine.md changed. The transition table test checks them cell by cell.

…dge its run

The session judges a skill run's scored steps in a fresh subagent, but a host
that answered h4 starts none, and done still offered the score --next literal.
Choosing h4 now drops the session judge and tells the person with a
judging-off no_child_agent event, so the work stays open for another driver.
Confined judging is unchanged.

A skill driver test that chooses h4 on a run its session would judge failed
without the change (no judging-off event) and passes with it.
… renders its agents

The skill driver kept the session's tool only when teamai renders agents for
it (isAgentToolId), so a Pi session, which sets PI_SESSION_ID, was unnamed and
its scored steps were never judged by the session. The session tool is now any
built-in ToolId. Only agent files and model aliases still need an agent tool:
a model-pinned step on another tool fails model_unavailable naming that tool.

A skill driver test with PI_SESSION_ID failed without the change (tool
undefined) and passes with it; the skill driver, transitions, preflight and
README matrix suites pass.
The judging-off event of a session whose host starts no child agent
carries failure code no_child_agent. The records test lists every code
teamai writes, and it did not list this one, so it failed.

Proof: npx vitest run src/__tests__/workflow-records.test.ts fails
before this change on no_child_agent, and passes after it.
…th --tools default=<tool>

The skill driver learns the session's tool from the session variable its
host sets. OpenClaw, Trae and other hosts set none that teamai reads, so
for them a step that pins a model failed with model_unavailable, and
describe judged nothing here.

workflow start and workflow resume with --driver skill now take
--tools default=<tool>, the existing option, and reject a role map or a
tool that teamai does not know. A declared tool wins over the one the
host names. skill.json records the tool beside the session that drives
the run. It is set at start and at a resume that creates skill.json, and
a takeover sets it to the new session's tool. The runner resolves a
child's model for that tool instead of the tool in its own environment.
Every start and describe command that describe offers, or that a refused
start prints, keeps the declared tool. Describe and the model failure
tell a session that names no tool to pass it.

Proof: new tests in workflow-skill-driver and workflow-skill-literals
were each seen failing for the intended reason, then passing:
- a declared tool resolves the step's model, and done leases the
  judging to it;
- a takeover hands the run to the new session's tool;
- the describe and start commands keep the tool;
- describe and the model failure name the option;
- a role map or an unknown tool is rejected.
tsc, lint, and the workflow tests pass.
Before this PR, the README scored a tool's skill-driven runs only when
the skill driver could name that tool from a session variable. Any
session can now name its tool with --tools default=<tool>. What remains
is whether the host can start a fresh subagent to judge in.

features.sessionJudge now records that, with its evidence:
- Qoder, Qoder CN: subagent transcripts in qodercli 1.1.78 (s).
- Kiro, ZCode: subagents in their own context, per the docs (d).
- Trae, Trae CN: the Task tool, from the docs and the shipped binaries
  (s).
- Oh My Pi: the task tool, which starts a new agent session (omp
  18.12.6, s).
- Hermes: delegate_task (s).
- OpenClaw: sessions_spawn (◐s), since a tool policy can remove it.
- DeepSeek Harness: the spawn backend (◐s), since the minimal preset
  leaves it out.
- WorkBuddy: CodeBuddy's engine (t).
- Pi stays without a mark (+s): bare Pi starts no subagent, and the
  example extension adds one.

The support page's Workflows column reads features.workflows, which
covers the local driver only, so it is now labelled as such. The README
workflows column is the skill driver's, so the README test now checks
that it follows the skills column. The README test no longer ties the
session judge cell to a session variable. The ten
rows with no session reader now show ✓³ in every README, and the
tool-support pages are regenerated.

Proof: npx vitest run src/__tests__/readme-matrix.test.ts failed on
those ten rows after the cells changed, and passed once the READMEs
followed. npx vitest run tool-support -u regenerated both pages.
…ithout asking

Describe answers a workflow with nothing to ask, attended or unattended,
with one start command. The declared tool must stay in it, as it does in
the options of a question. No test covered that path: a mutant that
dropped the tool there survived.

Proof: the mutant on the session literal in askFor is now killed by
npx vitest run src/__tests__/workflow-skill-driver.test.ts -t declared.
…ame it

A session in Oh My Pi, Qoder, Kiro or ZCode follows the workflow skill:
it describes without --tools and runs a start describe offers. Nothing
told it to name its tool unless a scored step was judged elsewhere, so
a modeled step failed mid-run with model_unavailable.

Describe now says so whenever the workflow has prompt or scored steps
and the session has no tool, and the workflow skill's Start step says
to describe again with --tools default=<tool>. The judging reason for
that case shrinks to the fact, as the new line carries the instruction.

Tested: a describe of a modeled, unscored workflow in a session with no
tool carries the line (seen failing first with no say at all).
A session that resumed with --tools default=<tool> and was refused got
a next literal without it: R20's --claim, R22's --confirmed, and the
busy retry. Running that literal claimed the run with no tool, so every
modeled step failed with model_unavailable and judging went elsewhere.

The resume call now carries its declared tool into the literals it
prints. R11, which answers a next on a run this machine does not drive,
tells a session with no tool to add --tools default=<tool> to the
resume it names, as next takes no --tools.

Tested: the R20 refusal of a declared resume names --tools before
--claim, R11 in a session with no tool says to add it (both seen
failing first), and the resume literal parses back with its tool.
… one

A run started before its session named a tool kept none, and a later
resume with --tools default=<tool> was ignored once its skill.json was
here, so its modeled steps could not resolve a model. S1 now fills the
run's tool from the calling session when the run has none, as it
already fills the owner.

Tested: a run started with no tool, resumed with --tools default=claude,
dispatches its step with the model resolved for claude (seen failing
first with no dispatch).
…me too

The failure fires inside a run that was started, resumed or taken over,
and a resume with --tools default=<tool> now gives the run its tool, so
the advice names both.
… readable

skill.json's orchestrator.tool must name a tool teamai knows. One a
newer teamai wrote, or one since renamed, made the whole state
unreadable, so the run answered run_unreadable. The field is now
dropped instead, and the next call fills it from its session.

Tested: a run whose skill.json names an unknown tool answers its turn
(seen failing first with run_unreadable).
…nstalls agents for it

The guide named OpenClaw and Trae as tools a session declares, and said
the steps' models resolve for a declared tool. teamai installs no agents
for either, so a modeled step there fails. The guide now names Oh My Pi
and Qoder, and both it and the --tools help say models resolve only for
a tool teamai installs agents for. commands.md is regenerated.
Pi sets PI_SESSION_ID, so its session judges its skill-driven runs like
any other once it can start a subagent. Bare Pi cannot and answers h4,
which turns judging off; the subagent example extension adds one. That
is ◐, not +, and the scorers cell follows it to ✓³ in every README.
The tool-support pages are regenerated.
S1 filled the run's owner and its tool independently. A run started by
a session with no session variable, then continued by a Claude Code
session, became Claude's while its runners still resolved child models
for the first session's tool. The tool now moves with ownership, and
is otherwise filled only for the owner or when neither session is known.

Tested: a run started unowned with --tools default=pi, then continued by
a claude session, dispatches its step with the model resolved for
claude (seen failing first with no dispatch).
… tool

done.score leased the run's judging to the tool of the session that
started it, kept in sessionJudge. After a takeover by a session of
another tool, or once another session came to own an unowned run, that
session got a score command naming the old tool, so its judge leases
were checked against a tool it does not run. done.score now names the
run's tool, which moves with its owner, and falls back to sessionJudge's
for a run an older teamai started.

Tested: after a pi session takes over a done claude run, done.score
leases judging to pi (seen failing first naming claude).
@github-actions

Copy link
Copy Markdown
  • [P1 blocking] An explicit resume tool cannot replace the stored tool — src/workflow/drivers/skill/transitions.ts:448: Start in Oh My Pi with --tools default=omp, then resume from Qoder with --tools default=qoder, with neither host setting a recognized session variable. The existing tool wins over input.tool, so child models and done.score still target Oh My Pi. An explicitly declared resume tool needs to override the previous tool.

  • [P1 blocking] Naming a missing tool on resume does not restore judging — src/workflow/drivers/skill/tick.ts:301: Start a scored prompt workflow without a detected tool, then resume with --tools default=omp before answering the host question. readiedRow discards the new judging decision because skill.json already exists; callRow fills only orchestrator.tool. The step can now run, but sessionJudge remains absent and completion prints no done.score, although judging was never opted out.

  • [P2 non-blocking] Command-only workflows bypass the no-subagent check — src/workflow/drivers/skill/transitions.ts:585: A command-only workflow with a best-effort scorer and --tools default=pi gets sessionJudge, but its host is none, so this h4 check never runs. Bare Pi consequently receives done.score requiring a fresh subagent it cannot start. Check judging capability independently of whether the workflow has prompt steps.

The PR description documents sufficient testing, including a representative real-CLI run after build. I ran, built, and installed nothing from the PR.

@SaulMoro SaulMoro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was generated by AI during triage.

Approving at 1f5c467. Every acceptance item in #1479 works.

Built CLI, in a host that sets no session variable (base = main bd3adf9b)

  • workflow start --describe --driver skill says to describe again with --tools default=<tool>. With it, the run dispatches with the step's model resolved for omp, done.score leases judging to omp, and the stored label has judge.tool: omp. Main refuses --tools with --driver skill.
  • The three --tools refusals, the takeover case and the h4 host (no child agent, no judge work) behave as the body says.
  • npx tsc --noEmit and npm run lint exit 0; the PR's 8 test files (237 tests) pass, also with a symlinked TMPDIR, and 15 of them fail on main. Every CI job passed at this head, including the three E2E shards.
  • Covered by unit tests only: the --claim resume keeping --tools, the judging-off event text.

Non-blocking

  • [P2 non-blocking] When neither session is known, a resume's --tools exits 0 but is ignored if the run already has a tool, so a run started with the wrong tool cannot be corrected. That follows #1479's "fills it when the run has none", but letting an explicit --tools replace the stored tool in callRow would allow the repair.
  • [P2 non-blocking] A workflow whose only scored step is a workflow: step with command children never asks the host question, so it skips the h4 check; bare Pi then gets a done.score it cannot act on. The lease lapses and stays open, so nothing is lost.
  • [P3 nit] A resume that names the tool does not turn session judging back on (describe had said so; main behaves the same).
  • [P3 nit] The describe instruction does not list which ids <tool> accepts (dsh, qoder-cn); the refusal for an unknown tool does.

The CI review's three findings reproduce; we rate them as above (the issue's own rule for the first, main's behavior for the second).

@SaulMoro
SaulMoro merged commit 0031758 into Tencent:main Oct 27, 2026
18 checks passed
@SaulMoro

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

@hz89Dev this merged with three findings from the CI review still open. We reproduced all three; could you fix them in a follow-up PR?

  1. An explicit resume tool does not replace the stored one (src/workflow/drivers/skill/transitions.ts:448). Start in Oh My Pi with --tools default=omp, resume from Qoder with --tools default=qoder, neither host setting a session variable: the resume exits 0, but child models and done.score still target Oh My Pi, so a run started with the wrong tool cannot be corrected. An explicit --tools should replace the stored tool in callRow.
  2. Naming the tool on resume does not restore judging (src/workflow/drivers/skill/tick.ts:301). Start a scored prompt workflow with no detected tool, then resume with --tools default=omp before answering the host question: the step runs, but sessionJudge stays absent and completion prints no done.score, although judging was never turned off.
  3. Command-only workflows skip the no-subagent check (src/workflow/drivers/skill/transitions.ts:585). A workflow whose only scored step is a workflow: step with command children never asks the host question, so the h4 check never runs and bare Pi gets a done.score it cannot act on. The judging check should not depend on the workflow having prompt steps.

@hz89Dev

hz89Dev commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in #1481, each with a test that failed before the fix:

  • A resume's --tools default=<tool> now replaces the run's tool, so child models and done.score follow it.
  • Judging can no longer be lost to a missing tool. skill.json does not record --no-judge, so a later resume cannot tell an opt-out from "no tool at start". Instead, describe asks a session that names no tool for one (ask.kind: tool), and a start or resume that would drive a run without the tool its models or judging need is refused until it names one.
  • A run its session judges asks the host question even without prompt steps, so h4 turns judging off. For such a run the answer decides only judging, so its commands still run as before.

SaulMoro pushed a commit that referenced this pull request Oct 28, 2026
… --tools replaces the run's (#1481)

Follow-up to #1480. Describe asks a session that names no tool for it, with a {tool} slot; a start or resume that needs a tool the session does not name is refused with tool_needed and a next that keeps the call's options; a resume's --tools replaces the tool the run had; a run of commands alone without judging needs no tool.

Refs #1479
Walternater pushed a commit to Walternater/teamai-cli that referenced this pull request Nov 2, 2026
…l with --tools (Tencent#1480)

A skill-driven workflow in a host that sets no session variable (Oh My Pi, Qoder, Kiro, ZCode, Trae, OpenClaw, Hermes, DeepSeek Harness) now names its tool with --tools default=<tool>. Describe says when it is needed, the run keeps the tool with its owner, child models resolve for it and session judging is leased to it. A host that starts no child agent gets no judge work.

Closes Tencent#1479
Walternater pushed a commit to Walternater/teamai-cli that referenced this pull request Nov 2, 2026
… --tools replaces the run's (Tencent#1481)

Follow-up to Tencent#1480. Describe asks a session that names no tool for it, with a {tool} slot; a start or resume that needs a tool the session does not name is refused with tool_needed and a next that keeps the call's options; a resume's --tools replaces the tool the run had; a run of commands alone without judging needs no tool.

Refs Tencent#1479
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feat] A coding session whose host names no tool names it for the skill driver

3 participants