feat(ai-studio): add agent-harness node for external coding-agent delegation

[tbrandenburg]

feat(ai-studio): add agent-harness node for external coding-agent delegation

Closes #147

---

What this adds

A new node type ai-studio/agent-harness that delegates a workflow step to an external autonomous coding-agent CLI/SDK — distinct from the existing ai-studio/ai-agent node, which performs a single bounded LLM completion.

	ai-studio/ai-agent	ai-studio/agent-harness (new)
Duration	Seconds	Minutes to tens of minutes
Execution	One generateText call	Long-lived external CLI/SDK session with its own tool loop
Side effects	None	Mutates files, runs shell commands, calls tools
Retries	maximumAttempts: 2	maximumAttempts: 1 (zero — side-effecting)
Timeout	10m	45m startToCloseTimeout + 5s heartbeatTimeout

v1 provider: GitHub Copilot. The config surface is designed for cross-provider parity, so adding a second provider is a new adapter, not a contract change.

---

Attribution

This implementation is derived from coleam00/Archon (MIT © 2025-2026 Cole Medin), which solved this problem thoroughly and well. The following modules are ports of Archon's work, adapted to workflowbuilder's architecture:

workflowbuilder module	Derived from Archon
agent-harness/types.ts	packages/providers/src/types.ts — provider contract, MessageChunk, TokenUsage, ProviderCapabilities, NodeConfig
agent-harness/credentials/delivery.ts	packages/core/src/credentials/delivery.ts — (vendor, credential) → { env, files } delivery model
agent-harness/providers/copilot/*	packages/providers/src/community/copilot/* — capabilities, config parsing, binary resolution, event bridge, provider
agent-harness/shared/idle-timeout.ts	packages/workflows/src/utils/idle-timeout.ts
agent-harness/shared/error-classification.ts	packages/workflows/src/executor-shared.ts — classifyError taxonomy
activities/agent-harness.ts (checkout guard, stream loop)	packages/workflows/src/dag-executor.ts
Node config field surface	packages/workflows/src/schemas/dag-node.ts + packages/web/.../NodeInspector.tsx (field checklist only)

Why adapted rather than depended upon: Archon is not published as a consumable library, and its execution model differs fundamentally from workflowbuilder's — it runs its own DAG executor in-process (no replay-determinism sandbox), ships as a Bun-compiled binary (driving an elaborate binary-resolution chain we don't need), and is a multi-tenant SaaS with an encrypted credential vault (vs. this single-tenant reference stack). The algorithms transfer; the packaging does not.

Full attribution and a per-file provenance table also live in apps/execution-worker/src/agent-harness/README.md.

---

Architecture

apps/execution-worker/src/agent-harness/   ← ported provider layer
  types.ts · errors.ts · registry.ts
  credentials/{delivery,catalog}.ts
  shared/{binary-resolution,run-config,idle-timeout,error-classification}.ts
  providers/copilot/{capabilities,config,binary-resolver,event-bridge,provider}.ts
apps/execution-worker/src/activities/agent-harness.ts   ← Temporal activity
apps/ai-studio/src/nodes/agent-harness/                 ← UI node (4 files)
apps/ai-studio/src/data/agent-harness-flow.ts           ← demo template

Deliberate deviations from the reference implementation

#	Reason	What changed
A1	runGraph is replay-deterministic and sandboxed	All I/O lives in the activity, never the graph runner
A2	Temporal cancellation requires heartbeating	Context.current().heartbeat() every 1s + heartbeatTimeout: '5s'
A3	Archon's planned manual process-group kill was not needed	Empirically verified: the Copilot SDK's own session.abort()/client.stop() cleanly kills the underlying process tree (4/4 clean kills in isolated testing, reconfirmed with zero orphans across every full-stack E2E run). No spawn(detached)+process.kill(-pid) workaround was implemented — see the agent-harness README's "SDK abort supersedes A3" section for the evidence and the documented fallback if a future SDK version regresses.
A4	Worker is normal Node/Docker, not a compiled binary	Plain top-level SDK import; binary resolution trimmed to env → config → PATH
A5	Single-tenant reference stack	Credential delivery ported faithfully; credential storage is worker env config, not a vault
A6	No codebase/project concept yet	Workdir from workflow context (context.variables.workdir), scratch temp dir fallback
A7	Host UI is JSONForms, not hand-rolled React	schema.ts/uischema.ts; a dedicated custom JSON-parsing control (originally planned as M8) turned out to be unnecessary — built-in TextArea controls were adequate for v1's output_format/agents JSON fields
A8	Naming	No Archon identifiers in source (verified via repo-wide grep)

---

Behaviour verified end-to-end

Real infrastructure (Postgres + Temporal via Docker), a real worker process, and the real copilot CLI (1.0.83) were used throughout — no mocks at the E2E layer.

• Demo workflow ("Agent Harness Demo": trigger → agent-harness → visualize) runs green end-to-end; the agent's real output (a written plan.md + summary) reaches the downstream visualize node
• Cancellation kills the subprocess — workflow/UI-driven cancellation completes in ~3-3.5s wall-clock, pgrep -x copilot empty and no native grandchild process survives, verified across 8+ separate cancellation runs (unit-level spike, direct-activity E2E, and full-stack UI-driven cancellation)
• Two concurrent runs complete correctly and in isolation (both a deliberate concurrent-trigger test and an earlier accidental double-run both completed correctly with no cross-contamination)
• Idle timeout (temporarily lowered) aborts a stalled/slow session cleanly, with correct partial-output salvage
• Missing/invalid credential → clear, actionable error ("Copilot authentication failed... Run copilot login (default), set COPILOT_GITHUB_TOKEN..."), classified permanent (not retried)
• mutatesCheckout: false violation → non-retryable failure, verified with a real git-initialized workdir and a real file write
• Unsupported config fields (mcp, skills, maxBudgetUsd) surface a provider warning rather than a silent drop (found and fixed a gap for maxBudgetUsd during the adversarial pass — it was previously silently ignored)
• Empty prompt fails fast (~20ms, classified permanent) instead of waiting out the idle timeout — found and fixed during the adversarial pass
• No scratch-dir or process leaks across repeated runs (/tmp scratch-dir count and pgrep -x copilot both verified stable across passes)
• Branch routing intact when wired downstream of a decision node

Two rounds of adversarial testing were run per the project's own testing discipline (see FINDINGS.md in the implementation's working notes) — the first round found and fixed 3 real issues (a lint-breaking numeric literal, the empty-prompt hang risk, and the silently-dropped maxBudgetUsd warning); the second full re-run of the entire checklist found zero new issues.

Screenshots

Captured locally during Playwright-MCP-driven E2E verification (property panel with all four config sections, out-of-scope fields correctly labeled "not yet supported"; a completed run showing real Copilot output in the visualize node). Available on request / can be attached to this PR directly via the GitHub UI if needed for review.

---

Breaking / notable changes

• packages/temporal: ActivityProfile gains an optional heartbeatTimeout key (additive; profile-validation previously rejected unknown keys). Changeset included (minor bump). Replay contract unaffected — activity options are not workflow-sandbox state.
• New optional env vars on the worker: COPILOT_GITHUB_TOKEN (falls back to ambient copilot login session if unset), COPILOT_CLI_PATH (overrides binary resolution).

v1 limitations (tracked in #147, deliberate)

• No automatic retries (maximumAttempts: 1) — the node is side-effecting
• No session persistence/resume — context beyond fresh renders in the UI but is not yet supported
• No live streaming of intermediate output — blocking, final-result-only (chunks are consumed internally but not fanned out to the UI)
• No structured-output enforcement — outputFormat renders but is not enforced
• No MCP / skills / sub-agent (agents) backend wiring — agents is schema-supported but needs a string→object parse step before it reaches the executor usably; mcp/skills are provider-capability-true but unwired, and surface a runtime warning if set
• Host-subprocess execution only (no container isolation)
• No human-in-the-loop approval gate (separate future issue)
• Provider-emitted warnings (unsupported-field notices) are currently only visible in worker debug logs, not in the ai-studio UI's execution log panel — flagged as a follow-up, not fixed here (touches shared, non-agent-harness UI code)
• Invalid model values are silently substituted by the Copilot SDK rather than failing or warning (upstream SDK behavior)

Test plan

pnpm check                                          # lint + typecheck + format (repo-wide)
pnpm -F @workflow-builder/execution-worker test     # 133 tests
pnpm -F @workflow-builder/ai-studio test
pnpm -F @workflowbuilder/temporal test               # 98 tests, incl. new heartbeatTimeout coverage
pnpm dev:ai-studio                                   # load "Agent Harness Demo", run it

Note: pnpm check currently fails on apps/docs (astro check, 4 pre-existing TypeScript errors unrelated to this change and present on main before this branch) — everything else (lint across all 14 workspaces, typecheck across all workspaces except docs, and format) is green.

Assumptions / follow-ups worth noting for reviewers

• context.variables.workdir is the convention this activity uses to resolve a real (non-scratch) working directory; no real caller populates it today, so production runs currently always execute in an ephemeral scratch tmpdir. Wiring a real "codebase" concept through to this convention is future work.
• The tools/skills UI fields (allowedTools/deniedTools/skills as comma-separated strings) don't yet have a conversion step to the backend's string[] shape — cosmetic/config-authoring gap, not a runtime crash risk, called out explicitly in the UI node's code comments.

[tbrandenburg]

E.g. working in the customer support triage:

[piotrblaszczyk]

Thanks for all of this. We haven't reviewed the implementations yet, so this is a status per issue rather than a verdict on the code.

Issue	Where we stand
#141 per-node-type task queue routing	A direction we want and it's on our roadmap. The smallest of the three and the one least tangled up in the rest.
#144 per-node model / provider	Per-node model is on the roadmap. Provider selection is largely superseded by #113, below.
#147 agent-harness	The one we can't answer quickly. Two questions are ours to settle first: it runs an external CLI as a host subprocess with no isolation, in the middle of an audit of exactly that surface; and the port derives from MIT-licensed Archon while this repo is Apache-2.0. Both are maintainer decisions, not review comments.

Something we owe you sooner: #113 has landed on main. It replaces the provider layer with the generic OpenAI-compatible provider driven by AI_BASE_URL, AI_API_KEY and AI_MODEL, and removes OPENROUTER_API_KEY entirely. All three of your PRs now conflict. You pushed commits on the 15th against a premise we had just removed, and we should have flagged it before that.

What stands out already: the two findings from your spike behind #147, that cancellation does nothing without ctx.heartbeat(), and that killing a wrapper can orphan a still-billing grandchild. Those matter to us independently of these PRs. Same for testing your own proposed fix and dropping it once the SDK turned out to handle the process tree.

Where we are: we're heads-down on several large pieces over the coming weeks (human-in-the-loop decisions, the design system, closing out a security review). We don't have the bandwidth to review a big change right now, and a shallow pass would be worse than none.

Rather than have you guess at our roadmap: would you be up for a short call? We'd like to hear what you're building and where this fits. The detail in these issues suggests you're running something real, and that's very interesting to us.

[ahmedibrahim085]

Thanks for this work. We run Workflow Builder on Temporal with long shell-backed steps (10–25 min each: test runs, an APK build, coding-agent calls), and we hit the same two problems your spike found: with only startToCloseTimeout, a crashed worker is noticed only when the cap runs out, and cancelling a run does not reach the running step.

Would it be possible to split the heartbeatTimeout part (the optional ActivityProfile key plus the activity context with heartbeat() and the cancellation signal, i.e. the temporal-node-activity-heartbeat-timeout changeset) into its own small PR? It is independent of the agent-harness node and of task-queue routing, so it could be reviewed without waiting on the isolation and licence questions around #147.

We are porting just that part into our copy in the meantime and are happy to share what we learn from running it.

[tbrandenburg]

@ahmedibrahim085 Thanks for sharing your use case — that split makes sense. I’ve opened #201 with the optional heartbeatTimeout support and a shell-backed example using Temporal’s native heartbeat and cancellation context. Real Temporal integration tests cover cancellation stopping the running process and heartbeat expiry; the package’s 172 tests pass, including replay checks. I’d be glad to hear what you learn from your port.

[tbrandenburg]

I have updated the branch against current main and resolved the conflicts, while retaining the newer AI Agent provider/structured-output changes and the human-decision flow. I also wired the agent-harness activity profile into the workflow bundle as well as the worker: a real Temporal integration test confirms the 45-minute start-to-close timeout, 5-second heartbeat timeout, and single attempt reach the scheduled activity. The Agent Harness demo template ID is now unique alongside Refund Review. Relevant validation passed: 595 AI Studio tests, 203 execution-worker tests, 176 Temporal tests, plus builds, lint, typechecks for affected workspaces, and both Compose configs. GitHub now reports the branch mergeable; it is awaiting maintainer review.