From 29dc8bc429e863c54447ed4427ab7fd328ca7f70 Mon Sep 17 00:00:00 2001 From: Christopher Date: Thu, 17 Sep 2026 14:48:03 +1000 Subject: [PATCH 01/44] docs(architecture): define coding execution gateway --- ...-agent-execution-through-an-a2a-gateway.md | 290 ++++++++++++++++++ 1 file changed, 290 insertions(+) create mode 100644 docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md new file mode 100644 index 00000000..c57f5fd3 --- /dev/null +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -0,0 +1,290 @@ +# ADR 0002: Serve coding-agent execution through an A2A gateway + +- Status: Accepted; implementation pending +- Date: 2026-09-17 + +## Context + +AllAgents already owns cross-client agent configuration, workspace knowledge, +plugins, hooks, MCP configuration, and launchers for Codex and other coding +agents. External systems also need to invoke those agents without importing +AllAgents internals or coupling to an interactive CLI process. + +The first planned consumer is AI Evals. Its +[ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) +removes AI Evals-owned coding workspaces in favor of a Promptfoo provider that +needs one remote coding-agent call to return output, usage, traces, file +changes, produced artifacts, failures, cleanup outcomes, and execution +provenance. Future clients may need the same execution boundary without +Promptfoo or evaluation semantics. + +A coding-agent execution is more than a model request. It includes immutable +source selection, repository acquisition, environment setup, credentials, +permissions, agent invocation, cancellation, evidence capture, process +termination, and cleanup. Those responsibilities need one public contract while +allowing materially different execution backends. + +The contract must not turn AllAgents into an evaluation harness. Dataset +expansion, repetition, assertions, scoring, experiment scheduling, and durable +evaluation Runs remain consumer concerns. + +## Decision + +### Add a separately deployable execution gateway + +AllAgents will provide a separately testable and deployable execution-gateway +entry point. It will not be coupled to an interactive CLI command lifecycle. + +The gateway owns: + +- authentication and authorization; +- stable Task and idempotency identity; +- deadline and cancellation propagation; +- execution-profile and backend selection; +- normalization of terminal output and evidence; +- protocol-level Task status and bounded retention; and +- enforcement of the coding-execution contract across every backend. + +The gateway is not an evaluator, grader, experiment scheduler, retry authority, +or durable evaluation Run ledger. It does not own a consumer's result store. + +### Keep the gateway separate from execution backends + +The gateway dispatches to peer execution backends. The initial design supports: + +- a direct Codex backend; and +- Agent-Conductor as an alternative backend. + +Using Codex does not require an Agent-Conductor hop. Additional backends may be +added only when they satisfy the same conformance contract. + +Execution backends own repository materialization, environment setup, agent +invocation, evidence collection, process termination, and cleanup. The gateway +must not execute evaluated agents or mount their writable repositories in the +gateway process. + +When deployed on Kubernetes, the gateway runs as its own Deployment and +ClusterIP Service, separate from consumers and execution workers. A direct +backend dispatches to a worker pool, per-invocation Job, or stronger sandbox. +Agent-Conductor remains a separate service. The protocol does not require one +worker topology. + +A separate gateway Pod is a service and failure boundary, not per-invocation +security isolation. Deployments requiring hostile-code or tenant isolation +must create or select a stronger execution boundary behind the gateway. + +### Profile A2A 1.0 instead of inventing an invocation API + +The external contract profiles the Linux Foundation +[Agent2Agent protocol](https://a2a-protocol.org/latest/specification/). The +initial profile requires the A2A 1.0 HTTP+JSON binding and retains Agent Card, +Message, Part, Task, Artifact, status, streaming, cancellation, security, and +error semantics. + +The profile narrows A2A for deterministic coding execution: + +- every accepted execution request creates exactly one addressable A2A Task; + direct-Message completion is not supported; +- the gateway implements all mandatory A2A core operations, including + `SendMessage`, `GetTask`, `ListTasks`, and `CancelTask`; when its Agent Card + advertises streaming, it also implements `SendStreamingMessage` and + `SubscribeToTask`; capability-gated operations retain their standard A2A + behavior instead of being replaced by bespoke `/v1/invocations`, `/v1/runs`, + or `/v1/trials` resources; +- terminal Task results use Artifacts for output and evidence rather than + relying on transient messages or stream events; and +- each versioned Agent Card advertises one mandatory AllAgents extension version + for source and runtime identity, traces, usage and cost, file changes, + produced artifacts, typed failures, cancellation and cleanup outcomes, + evidence completeness, and provenance. + +Generic A2A conformance is insufficient. The AllAgents extension and its +conformance fixtures define the coding-execution guarantees every backend must +satisfy. + +Breaking extension changes use a new extension URI and a versioned Agent Card +or service endpoint. During migration, the gateway keeps the old card, endpoint, +and required extension serviceable while consumers move to the new profile. +Each card requires exactly one extension version. Clients pin the card they +support; the gateway never silently falls back across incompatible versions. +Retiring an old profile is a separate coordinated compatibility decision, not a +lockstep deployment requirement. + +The AAIF +[agentgateway](https://github.com/agentgateway/agentgateway) project may be used +as traffic-policy infrastructure for A2A, MCP, or model calls. It is not the +AllAgents execution service or evidence schema. Documentation uses **AllAgents +execution gateway** where the distinction matters. + +### Keep adjacent protocols at their proper boundaries + +The [Agent Client Protocol](https://agentclientprotocol.com/) may be used behind +a backend adapter when a coding agent supports it. Its session, progress, tool, +permission, terminal, diff, usage, and cancellation semantics are useful +internally, but its stdio editor-to-agent protocol is not the external gateway +API. + +[Model Context Protocol](https://modelcontextprotocol.io/) remains a tool and +resource protocol inside an execution backend. It does not represent the whole +coding-agent execution. + +[Agent Format](https://agentformat.org/) may provide an optional static agent +manifest and vocabulary. It does not define the execution transport or prove +observed execution evidence. + +The archived IBM/BeeAI Agent Communication Protocol is superseded by A2A and +will not be adopted. + +### Separate trace propagation, span semantics, and durable evidence + +Gateway calls propagate +[W3C Trace Context](https://www.w3.org/TR/trace-context/) across HTTP and process +boundaries. AllAgents uses OpenTelemetry and OTLP for operational telemetry. +AllAgents-managed agent, model, and tool spans use +[OpenInference](https://arize-ai.github.io/openinference/) semantic conventions +where corresponding attributes exist; useful backend-native attributes may be +retained alongside them. Consumer-owned evaluator spans may join the propagated +trace without becoming gateway-owned. + +These standards are complementary: + +- W3C Trace Context propagates causal trace identity; +- OpenTelemetry and OTLP represent and transport live operational telemetry; +- OpenInference describes AI operations on OpenTelemetry spans; and +- the AllAgents A2A extension returns durable coding evidence and provenance. + +An external trace backend is not the sole durable result. Sampling, redaction, +transport loss, or retention policy must not erase the terminal facts needed by +a consumer. + +### Trial ATIF only as an optional trajectory Artifact + +The Harbor +[Agent Trajectory Interchange Format](https://github.com/harbor-framework/harbor/blob/main/rfcs/0001-trajectory-format.md) +may be returned as an optional, explicitly versioned A2A Artifact when a backend +can produce or truthfully normalize an ordered agent trajectory. It is not the +A2A transport, the OpenTelemetry trace, or the AllAgents evidence envelope. +Backend-native trajectories remain available when conversion would lose +information. + +An ATIF Artifact must declare its exact schema version and correlate its A2A +Task, OpenTelemetry trace, AllAgents invocation, and backend session identities +through the versioned AllAgents extension. Reasoning content is excluded by +default. Tool arguments, observations, and media follow explicit redaction, +size, and disclosure policy. Truncation or conversion loss is reported rather +than hidden. + +ATIF remains optional until its compatibility policy, specification, tooling, +and non-Harbor conformance mature enough for a required public-contract +capability. + +Harbor's task package, Job configuration, Job/Trial result models, hosted API, +artifact manifest, registry formats, and trial-directory layout will not become +the gateway contract. They remain Harbor-native formats that a future adapter +may preserve. Harbor's ASP `.asp.json` is a draft v0 sandbox proposal and is not +adopted by this decision. + +### Make execution provenance and cleanup explicit + +The gateway and selected backend are collectively responsible for: + +1. resolving and verifying immutable source identity; +2. acquiring or restoring source through the selected transport; +3. creating a clean or explicitly reusable working location; +4. running setup before the evaluated agent action; +5. applying permissions and execution isolation; +6. invoking the agent and propagating cancellation and deadlines; +7. capturing bounded output, usage, cost, file changes, checks, and artifact + references; +8. returning terminal status, evidence completeness, and provenance; and +9. terminating processes and releasing or retaining resources according to the + documented lifecycle. + +Source transport and runtime transport are independent. A backend may use one +immutable runtime image plus a separately digest-addressed source artifact; the +contract does not require source code to be baked into the runtime image. + +Credentials remain deployment policy. Requests must not embed deployment +credentials. The gateway authenticates callers, and the selected backend scopes +source and model credentials to the execution boundary without returning +secret-bearing paths or values. + +Retries must not multiply non-idempotent agent execution. Every request carries +a caller-scoped stable invocation key through the AllAgents extension. The +gateway binds the authenticated caller, invocation key, effective execution +profile, and request digest to the created Task for a documented retry-retention +window. An identical replay returns the original Task. Reusing the key with a +different request is rejected. Backend retry suppression remains an additional +safeguard; it does not replace gateway deduplication. + +### Keep evaluation commands out of scope + +This decision does not add `allagents eval`, benchmark authoring, assertions, +scoring, datasets, or experiment scheduling. A community evaluation wrapper and +an enterprise AI Evals wrapper may share this execution service in the future, +but their product and ownership model requires a separate decision. + +## Consequences + +- AllAgents becomes a service boundary in addition to a local CLI, but retains a + narrow coding-execution responsibility. +- Consumers depend on A2A 1.0 plus a versioned AllAgents extension, not + AllAgents TypeScript modules, CLI behavior, or workspace internals. +- Direct Codex and Agent-Conductor execution are interchangeable backends behind + one conformance suite. +- Gateway and execution workers scale and fail independently. +- The gateway can remain lightweight; physical isolation and resource policy + belong to the selected execution backend. +- A2A supplies discovery and lifecycle semantics. AllAgents supplies the + coding-specific evidence contract. +- W3C Trace Context, OpenTelemetry/OTLP, OpenInference, optional ATIF, and the + terminal evidence extension remain distinct layers rather than competing + universal formats. +- Implementations must preserve bounded native evidence whenever normalization + would lose information. + +## Rejected alternatives + +### Invent a bespoke invocation, run, or trial API + +Rejected because A2A already defines remote-agent discovery, Task lifecycle, +streaming, artifacts, cancellation, errors, and web security. Coding-specific +evidence belongs in a versioned A2A extension rather than a parallel transport. + +### Run agents in the gateway Pod + +Rejected because it couples control-plane availability and credentials to +mutable repository execution, prevents independent scaling, and mistakes a +service boundary for per-invocation isolation. + +### Make Agent-Conductor mandatory + +Rejected because a direct Codex adapter and Agent-Conductor are peer backends. +Mandatory indirection adds an ownership and failure boundary without improving +the public contract. + +### Use OpenInference instead of W3C Trace Context + +Rejected as a category error. W3C Trace Context propagates trace identity; +OpenInference supplies AI semantic conventions on OpenTelemetry spans. The +gateway uses both. + +### Use ATIF as the complete gateway result + +Rejected because ATIF represents an ordered agent trajectory, not remote Task +lifecycle, repository provenance, workspace changes, produced artifacts, +cleanup, authorization, or evidence completeness. + +### Adopt Harbor's Job or Trial API + +Rejected because Harbor's formats own benchmark orchestration, verification, +and persisted runner state. The AllAgents gateway executes one coding-agent +request and does not become an evaluation harness. + +## Reconsider when + +Revisit this decision if A2A standardizes the required coding-execution evidence +without an extension, if a stable cross-vendor execution protocol subsumes the +same lifecycle and provenance guarantees, or if operational evidence shows that +the gateway and backend boundary prevents required cancellation, isolation, or +result integrity. From 8b9a656597271a1071636b1f8f4c8b16ee7f7e09 Mon Sep 17 00:00:00 2001 From: Christopher Date: Fri, 18 Sep 2026 08:27:43 +1000 Subject: [PATCH 02/44] docs(architecture): refine gateway backend boundaries --- ...-agent-execution-through-an-a2a-gateway.md | 46 +++++--- .../agent-host-protocol-decision-inputs.md | 109 ++++++++++++++++++ 2 files changed, 138 insertions(+), 17 deletions(-) create mode 100644 docs/research/agent-host-protocol-decision-inputs.md diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index c57f5fd3..e798ef3e 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -50,13 +50,15 @@ or durable evaluation Run ledger. It does not own a consumer's result store. ### Keep the gateway separate from execution backends -The gateway dispatches to peer execution backends. The initial design supports: +The initial design supports three peer execution backends: -- a direct Codex backend; and -- Agent-Conductor as an alternative backend. +- Codex; +- OpenCode; and +- Pi. -Using Codex does not require an Agent-Conductor hop. Additional backends may be -added only when they satisfy the same conformance contract. +Each backend implements the same conformance contract. Provider-specific +process, session, permission, cancellation, and evidence behavior remains +behind its adapter. Execution backends own repository materialization, environment setup, agent invocation, evidence collection, process termination, and cleanup. The gateway @@ -64,10 +66,10 @@ must not execute evaluated agents or mount their writable repositories in the gateway process. When deployed on Kubernetes, the gateway runs as its own Deployment and -ClusterIP Service, separate from consumers and execution workers. A direct -backend dispatches to a worker pool, per-invocation Job, or stronger sandbox. -Agent-Conductor remains a separate service. The protocol does not require one -worker topology. +ClusterIP Service, separate from consumers and execution workers. A backend +dispatches to a worker pool, per-invocation Job, or stronger sandbox according +to the selected execution profile. The protocol does not require one worker +topology. A separate gateway Pod is a service and failure boundary, not per-invocation security isolation. Deployments requiring hostile-code or tenant isolation @@ -124,6 +126,15 @@ permission, terminal, diff, usage, and cancellation semantics are useful internally, but its stdio editor-to-agent protocol is not the external gateway API. +The [Agent Host Protocol](https://microsoft.github.io/agent-host-protocol/) +may be used behind a backend adapter when a host exposes it, or beside the +gateway if AllAgents later adds a collaborative multi-client session surface. +Its host-authoritative snapshots, actions, reconnection, tools, permissions, +and changesets solve live session synchronization; they do not replace A2A +Task identity, idempotency, authorization, terminal evidence, or retention. +The supporting research and implementation consequences are captured in the +[AHP decision inputs](../research/agent-host-protocol-decision-inputs.md). + [Model Context Protocol](https://modelcontextprotocol.io/) remains a tool and resource protocol inside an execution backend. It does not represent the whole coding-agent execution. @@ -230,8 +241,8 @@ but their product and ownership model requires a separate decision. narrow coding-execution responsibility. - Consumers depend on A2A 1.0 plus a versioned AllAgents extension, not AllAgents TypeScript modules, CLI behavior, or workspace internals. -- Direct Codex and Agent-Conductor execution are interchangeable backends behind - one conformance suite. +- Codex, OpenCode, and Pi are peer execution backends behind one conformance + suite. - Gateway and execution workers scale and fail independently. - The gateway can remain lightweight; physical isolation and resource policy belong to the selected execution backend. @@ -257,12 +268,6 @@ Rejected because it couples control-plane availability and credentials to mutable repository execution, prevents independent scaling, and mistakes a service boundary for per-invocation isolation. -### Make Agent-Conductor mandatory - -Rejected because a direct Codex adapter and Agent-Conductor are peer backends. -Mandatory indirection adds an ownership and failure boundary without improving -the public contract. - ### Use OpenInference instead of W3C Trace Context Rejected as a category error. W3C Trace Context propagates trace identity; @@ -281,6 +286,13 @@ Rejected because Harbor's formats own benchmark orchestration, verification, and persisted runner state. The AllAgents gateway executes one coding-agent request and does not become an evaluation harness. +### Replace A2A with the Agent Host Protocol + +Rejected because AHP explicitly targets synchronization of independent clients +around host-owned sessions, not agent-to-agent Task execution. Its reconnect +and changeset models do not supply caller-scoped idempotency, immutable source +handling, cleanup, complete terminal evidence, or bounded Task retention. + ## Reconsider when Revisit this decision if A2A standardizes the required coding-execution evidence diff --git a/docs/research/agent-host-protocol-decision-inputs.md b/docs/research/agent-host-protocol-decision-inputs.md new file mode 100644 index 00000000..a3ebebc9 --- /dev/null +++ b/docs/research/agent-host-protocol-decision-inputs.md @@ -0,0 +1,109 @@ +# Agent Host Protocol decision inputs for the execution gateway + +## Decision + +Keep A2A 1.0 plus the versioned AllAgents extension as the execution gateway's +northbound contract. Treat the Agent Host Protocol (AHP) as an optional future +protocol behind the gateway for a compatible backend or beside it for a +collaborative session client. + +AHP does not replace ADR 0002's Task identity, caller-scoped idempotency, +authorization, immutable source handling, cleanup, terminal evidence, or +bounded result retention. + +The initial backend set is Codex, OpenCode, and Pi. They are peer execution +adapters behind one conformance contract; provider-specific process, session, +permission, cancellation, and evidence behavior stays below that seam. + +This note records the AllAgents-specific consequences. The reusable research, +source inspection, and full protocol comparison live in the AI Research Wiki: + +- [Agent Host Protocol](https://github.com/tsoyang-org/ai-research-wiki/blob/main/entities/agent-host-protocol.md) +- [Agent Host Architecture](https://github.com/tsoyang-org/ai-research-wiki/blob/main/concepts/agent-host-architecture.md) +- [Agent Host Protocol vs Agent2Agent](https://github.com/tsoyang-org/ai-research-wiki/blob/main/comparisons/agent-host-protocol-vs-agent2agent.md) +- [VS Code Agent Host source note](https://github.com/tsoyang-org/ai-research-wiki/blob/main/raw/articles/vscode-agent-host-architecture.md) + +## Boundary + +| Concern | AllAgents A2A gateway | AHP host/session layer | +|---|---|---| +| Northbound consumer | AI Evals and future remote execution clients | IDE, browser, CLI, or collaborative operator client | +| Primary lifecycle | One addressable Task per accepted execution | Long-running session/chat with shared clients | +| Public identity | Agent Card, Message, Task, Artifact, invocation key | Host, client, channel, session, chat, turn, tool call | +| State | Task status, messages, artifacts, retention | Snapshots, ordered actions, reducers, reconnect | +| Authorization | Authenticate/authorize service caller | Endpoint/resource auth and tool confirmation | +| Cancellation | Cancel Task, abort backend, terminate, clean up, report terminal outcome | Cancel interactive turn and call provider-native abort | +| Evidence | Source, output, usage/cost, traces, file changes, artifacts, failures, cleanup, completeness, provenance | Live changesets and provider/session state | +| Isolation | Selected worker/backend boundary | Not supplied by the shared host process | + +The identities must be correlated rather than reused. At minimum retain the A2A +Task ID, AllAgents invocation key, backend execution/session ID, +provider-native thread/chat ID, and trace ID. + +## Adopt now + +1. Define one narrow backend adapter contract for create/invoke, progress, + permission decisions, cancellation, terminalization, evidence collection, + shutdown, native evidence passthrough, and explicit capabilities. +2. Keep gateway responsibilities separate from worker/backend responsibilities. + The gateway owns caller authorization, Task/idempotency identity, backend + selection, normalized results, cancellation propagation, and retention. + Workers own source materialization, provider processes, mutable workspaces, + evidence capture, process termination, and cleanup. +3. Propagate `CancelTask` and deadlines through the adapter to the + provider-native abort primitive, then persist terminal status and cleanup + outcome. Transport closure is not cancellation. +4. Separate caller authorization, execution permission policy, and + provider/resource credentials. +5. Combine normalized file operations with bounded provider-native + diffs/checkpoints/trajectories. Declare attribution limits and + incompleteness rather than treating the final working-tree diff as exact + agent causality. +6. Persist terminal facts independently of progress streams and telemetry. +7. Capability-gate backend behavior instead of inferring it from provider names + or software versions. +8. Keep Task lists and metadata bounded; store large logs, diffs, traces, and + produced artifacts behind references with size, redaction, and truncation + metadata. + +## Defer + +- An AHP backend adapter until a selected backend actually exposes AHP. +- An AHP server or multi-client reducer/reconciliation engine until a + collaborative session client is a product requirement. +- Client-contributed tools and customizations for unattended evaluation + profiles. +- Active-session reconnection beyond A2A Task lookup, subscription, and + terminal result retrieval. +- AHP local endpoint discovery, SSH host selection, and tunnel multiplexing. +- Generic changeset review/operation state. +- Long-lived session/chat catalogs and provider-native session adoption. + +## Reject + +- Replacing A2A with AHP for the execution gateway. +- Running evaluated agents or writable repositories in the gateway process. +- Treating AHP changesets as the complete AllAgents evidence envelope. +- Treating AHP action replay or session restoration as execution idempotency. +- Copying VS Code's local connection-token model as gateway authentication. +- Branching on provider names above the adapter boundary. +- Making a connected interactive client a hidden prerequisite for unattended + execution. + +## Evidence + +The conclusion is based on: + +- Microsoft's [Agent Host architecture article](https://code.visualstudio.com/blogs/2026/08/26/agent-host-architecture); +- the official [Agent Host Protocol documentation](https://microsoft.github.io/agent-host-protocol/); +- direct inspection of `microsoft/vscode` commit + [`046944034292b5479b4e9a50ad1a508033ffb64f`](https://github.com/microsoft/vscode/tree/046944034292b5479b4e9a50ad1a508033ffb64f), + whose generated registry identifies AHP `0.9.0`; and +- [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md). + +The inspected implementation demonstrates host-owned state/sequencing, +provider-neutral adapters for Copilot, Claude, and Codex, layered persistence, +bounded reconnect replay, provider-native cancellation, client-owned tools, +permission translation, Git checkpoint plus SDK edit evidence, and local/remote +host placement. These observations support the architecture seams above; they +do not supply the public execution guarantees retained by ADR 0002. From a7dfe48da0be3c74a5e1bc36f96812c29fc058ba Mon Sep 17 00:00:00 2001 From: Christopher Date: Fri, 18 Sep 2026 08:57:24 +1000 Subject: [PATCH 03/44] docs(plan): define execution gateway implementation --- ...0837-feat-coding-execution-gateway-plan.md | 658 ++++++++++++++++++ 1 file changed, 658 insertions(+) create mode 100644 docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md new file mode 100644 index 00000000..1d65837d --- /dev/null +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -0,0 +1,658 @@ +--- +title: "Coding-Agent Execution Gateway - Plan" +date: 2026-09-18 +deepened: 2026-09-18 +type: feat +artifact_contract: ce-unified-plan/v1 +artifact_readiness: implementation-ready +product_contract_source: ce-plan-bootstrap +execution: code +--- + +# Coding-Agent Execution Gateway - Plan + +## Goal Capsule + +- **Objective:** External systems can run Codex, OpenCode, or Pi against an immutable repository revision through one authenticated, cancellable, evidence-preserving remote contract. +- **Means:** Add a separately deployable A2A 1.0 gateway, a private worker protocol, and backend-neutral workers with three provider adapters (KTD1, KTD5, KTD7). +- **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) owns the public boundary. The A2A 1.0 specification owns core wire semantics. The versioned AllAgents extension owns coding-execution semantics. +- **Execution profile:** Build contract-first, then durable gateway state, worker lifecycle, provider adapters, packaging, and cross-backend conformance. Preserve the existing local CLI and Node 18 package compatibility. +- **Stop conditions:** Do not execute agents in the gateway process, accept mutable source identity, put deployment credentials in requests, treat streams or telemetry as terminal evidence, or add evaluation behavior. +- **Tail ownership:** The implementing workflow runs focused contract and lifecycle tests, the complete repository quality gates, isolated gateway/worker smoke tests, provider-specific credentialed smoke tests where credentials are available, and documentation validation. + +--- + +## Product Contract + +### Summary + +AllAgents gains a remote coding-execution service without becoming an evaluation framework. Callers use A2A Tasks and one required AllAgents extension. The gateway owns caller identity, idempotency, routing, status, cancellation, evidence normalization, and bounded retention. Separate workers own repository materialization, provider processes, mutable workspaces, evidence capture, termination, and cleanup. + +### Problem Frame + +AllAgents currently configures and launches coding clients but has no service boundary for external callers. AI Evals and future consumers would otherwise need to import AllAgents internals, drive interactive CLIs, or independently reimplement repository acquisition, permissions, cancellation, evidence, and cleanup. + +The three initial runtimes expose different programmatic contracts. Codex provides a TypeScript SDK over structured JSONL events, OpenCode provides a typed HTTP SDK and SSE event stream, and Pi provides a strict JSONL RPC mode. The public service must preserve one stable lifecycle without flattening provider-specific facts into false equivalence. + +### Actors + +- A1. **Gateway caller:** An authenticated service such as AI Evals that creates, observes, lists, cancels, and retrieves coding-execution Tasks. +- A2. **Execution gateway:** The A2A server that owns caller scope, Task identity, idempotency, routing, retention, and normalized results. +- A3. **Execution worker:** A separately deployed process that owns source materialization, one mutable workspace per invocation, provider execution, evidence capture, and cleanup. +- A4. **Backend adapter:** The Codex, OpenCode, or Pi integration that translates native events, cancellation, usage, failures, and evidence into the worker contract. +- A5. **Operator:** The person or deployment system that defines profiles, credentials, limits, retention, worker endpoints, and observability policy. + +### Key Decisions + +- **Profile A2A rather than creating a public invocation API.** The service keeps standard Agent Cards, Tasks, Artifacts, operations, errors, and capability negotiation. Governs R1-R4. +- **Keep execution outside the gateway process.** Mutable repositories and provider processes belong to workers. Governs R10-R16, R21-R22. +- **Keep evaluation outside AllAgents.** Dataset expansion, repetitions, assertions, scoring, retries, and durable evaluation Runs remain caller concerns. Governs R20. + +### Requirements + +**Public protocol and compatibility** + +- R1. The gateway implements A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, `GetTask`, `ListTasks`, and `CancelTask`; it implements streaming send and task subscription when the card advertises streaming. +- R2. Every valid new request returns exactly one addressable Task. Direct-Message completion and follow-up messages to an existing Task are unsupported. Non-streaming send honors A2A `returnImmediately`; streaming always emits the durable Task first. +- R3. Every request and terminal Task uses one required, versioned AllAgents coding-execution extension URI. Unsupported required extension versions fail without fallback. +- R4. Terminal output and execution evidence are retrievable as Task Artifacts for the configured retention window even when the original stream disconnects. Active subscription emits the current Task snapshot then future events without promising replay of missed progress; terminal subscription returns the standard unsupported-operation error and callers use `GetTask`. + +**Caller identity, Task identity, and retention** + +- R5. Every protocol operation authenticates the caller and scopes Task lookup, listing, subscription, cancellation, and artifact retrieval to that caller's tenant and principal before storage access can reveal resource existence. +- R6. Authentication, required-extension validation, request validation, source/profile authorization, quota admission, and deadline validation complete before Task creation. A caller-scoped invocation key, effective profile, authenticated owner, and canonical request digest then bind atomically to one Task; identical replay returns that Task and conflicting reuse is rejected without dispatch. +- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Task state and terminal Artifact metadata survive gateway restart; nonterminal Tasks that cannot be reattached settle failed once and stale worker events cannot overwrite them. +- R8. List operations implement all A2A filters, history bounds, page-size bounds, owner/query-bound cursor pagination, and descending status-update time. One immutable expiry logically hides the Task, claim, events, and artifacts before best-effort physical deletion; expired and unauthorized IDs are indistinguishable. +- R9. Small deployments work without an external database. The built-in durable store supports one gateway replica, enforces per-owner/global admission and storage quotas, and reserves capacity for cancellation and terminal settlement; multi-replica storage is outside this delivery. + +**Execution and policy** + +- R10. Codex, OpenCode, and Pi are peer backends behind one conformance contract. (session-settled: user-directed — chosen over an additional enterprise-only adapter: the open-source gateway supports the three named runtimes directly.) +- R11. A request selects a server-defined execution profile. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. +- R12. The only initial remote source form is a canonical credential-free HTTPS Git URL plus full commit object ID and optional repository-relative subdirectory. Acquisition revalidates destination policy for every connection, disables redirects and repository-controlled secondary fetch/exec features, uses hermetic Git configuration, and verifies that the fetched object is the requested commit before setup. +- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, and retained workspaces. +- R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. +- R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. +- R16. A worker creates a fresh invocation directory and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, and runs configured checks. It then proves all invocation descendants quiescent before final evidence/artifact capture and cleanup or explicit retention. + +**Evidence and observability** + +- R17. Every terminal result contains an integrity kernel: Task/source/profile/backend identities, action outcome, cancellation or failure classification, termination and cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. +- R18. Normalized file evidence distinguishes create, edit, delete, and rename where truthful. It preserves bounded provider-native diffs, events, or trajectories when normalization loses information and separately records truncation, redaction, attribution, original/captured size, and digest semantics. +- R19. Gateway and worker spans propagate W3C Trace Context and export OpenTelemetry data. Telemetry is operational evidence, not the only durable result. + +**Ownership and safety boundary** + +- R20. The gateway executes one coding request. It does not own eval configuration, datasets, repetition, scoring, retry policy, experiment scheduling, or a durable evaluation Run ledger. +- R21. The initial worker topology is one execution at a time for reviewed repositories inside one configured mutual-trust domain. Profiles that claim hostile-source or cross-tenant isolation are rejected until a stronger per-invocation UID, mount, PID, network, and credential boundary is configured. +- R22. Gateway admission and worker execution enforce profile limits for request rate, active/retained Tasks, subscriptions, stored bytes, source transfer/expansion, files/inodes, workspace bytes, CPU, memory, PIDs, network, phase deadlines, events, logs, and artifacts. Exhaustion is scoped to one invocation or owner and leaves capacity for terminalization and cleanup. + +### Key Flows + +- F1. **Admit, create, and stream an execution** + - **Actors:** A1, A2, A3, A4. + - **Trigger:** A caller sends a text Message with the required extension, immutable source, profile, invocation key, and deadline. + - **Steps:** Authenticate; validate and authorize the complete request; reserve quota; atomically claim idempotency and create a submitted Task; dispatch a fenced worker attempt; materialize and verify source; execute the selected backend; persist progress before emission; terminalize with Artifacts after quiescence and cleanup. + - **Outcome:** `returnImmediately: true` returns the durable current Task, false/unset waits for terminal state, and streaming starts with that Task before ordered updates. + - **Covered by:** R1-R22. +- F2. **Replay or reconnect to an invocation** + - **Actors:** A1, A2. + - **Trigger:** The owner repeats an invocation key or subscribes after a stream disconnect. + - **Steps:** Recompute the canonical digest; reject a conflict; return the existing Task; for active streaming replay/subscription emit its current snapshot then future events; for a terminal Task return it through send replay or `GetTask` without dispatch. + - **Outcome:** Retries do not multiply agent work, and reconnect never promises transient event replay. + - **Covered by:** R4, R6-R8. +- F3. **Cancel or time out an execution** + - **Actors:** A1, A2, A3, A4. + - **Trigger:** The caller invokes `CancelTask`, the effective deadline expires, or gateway shutdown claims cancellation. + - **Steps:** Atomically record the first cancellation source; if dispatch never occurred, prove no workspace exists; otherwise send one fenced worker cancel, invoke native abort, terminate descendants, capture termination-safe evidence, clean, and publish canceled only after verification. + - **Outcome:** Completion that wins first remains terminal and later cancel returns `TaskNotCancelableError`; cancellation that wins suppresses late provider success and fails instead of claiming canceled when termination or cleanup cannot be verified. + - **Covered by:** R7, R14, R16-R18. +- F4. **Recover from gateway or worker loss** + - **Actors:** A2, A3. + - **Trigger:** The gateway restarts with nonterminal Tasks, an acknowledgement is lost, or a worker crashes. + - **Steps:** Invalidate the attempt fence; settle each non-reattachable Task failed once; reject late events/results; stop renewing leases; let workers self-abort and clean. Record cleanup complete only when the worker/process boundary proves it; otherwise record unknown. + - **Outcome:** One Task has one terminal result, no ambiguous dispatch is retried automatically, and no stale worker can overwrite durable truth. + - **Covered by:** R7, R9, R14, R16-R18, R21-R22. +- F5. **Expire retained execution data** + - **Actors:** A1, A2. + - **Trigger:** The immutable Task expiry is reached. + - **Steps:** Atomically tombstone the complete ownership aggregate; stop authorizing Task and Artifact access; retry physical cleanup independently; permit the old invocation key to create a new Task only after logical expiry. + - **Outcome:** Expired, unknown, and unauthorized identifiers are indistinguishable and no Artifact outlives Task authorization. + - **Covered by:** R5-R9. + +### Acceptance Examples + +- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile and an exact Git SHA, when the caller streams a request, then one Task moves from submitted to working to completed and later `GetTask` returns the same output and evidence Artifacts. +- AE2. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with the same canonical request, then the gateway returns the original Task without a second worker dispatch. +- AE3. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with a different prompt, source, profile, or deadline, then the gateway rejects the request and leaves the original Task unchanged. +- AE4. **Covers R5.** Given a Task owned by caller A, when caller B lists Tasks, gets the Task, cancels it, subscribes, or requests an Artifact, then the gateway reveals no resource existence or content. +- AE5. **Covers R12, R16-R18.** Given a requested SHA that does not match the materialized repository, when the worker verifies source, then provider execution never starts and the Task fails with source-verification and cleanup evidence. +- AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. +- AE7. **Covers R10.** Given equivalent profiles and fixture runtime events for Codex, OpenCode, and Pi, when each completes the same repository mutation, then all three produce the same required normalized result fields while retaining distinct native evidence. +- AE8. **Covers R4, R7, R19.** Given a caller disconnects during work, when it subscribes again, then it receives the current Task and future updates without duplicate dispatch; telemetry loss does not affect later terminal lookup. +- AE9. **Covers R15.** Given a known capability denied by profile, the accepted Task becomes rejected after stop and cleanup; given an unknown permission type, it becomes failed as an adapter incompatibility without waiting for a client. +- AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. +- AE11. **Covers R6, R22.** Given invalid input or exhausted admission quota, the gateway returns a request/resource error and creates no Task; given capacity disappears after durable acceptance, the retained Task fails at dispatch and replay returns it without retry. +- AE12. **Covers R7, R14.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only safe operator telemetry. +- AE13. **Covers R8.** Given a Task reaches expiry while physical deletion fails, all Task and Artifact operations return the same not-found response and the invocation key can create a new Task. +- AE14. **Covers R21-R22.** Given a profile requests pooled hostile-source or cross-tenant execution, startup/admission rejects it; a reviewed single-trust-domain profile runs one bounded execution without exposing worker control credentials to the child environment. + +### Success Criteria + +- The official A2A JavaScript client can discover the card and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, replay, cancel, and expiry behavior against the built service. +- One conformance fixture passes unchanged through Codex, OpenCode, and Pi adapters. +- Admission, replay, fencing, cancellation races, restart recovery, authorization isolation, source hardening, quotas, and evidence integrity have deterministic integration coverage. +- The gateway image contains no coding-agent runtime and cannot access worker workspace roots. +- The initial worker runs one reviewed-trust-domain execution at a time and leaves no live descendant or retained workspace unless policy requests retention. + +### Scope Boundaries + +**In scope** + +- A2A 1.0 HTTP+JSON and SSE streaming. +- One versioned AllAgents coding-execution extension and one versioned private worker protocol. +- Codex, OpenCode, and Pi backends. +- Built-in bearer authentication with OIDC/JWT and static service-token modes. +- Single-replica durable file storage, authenticated Artifact retrieval, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. +- Reviewed repositories in one configured mutual-trust domain per worker deployment. + +**Deferred to follow-up work** + +- Multi-replica database-backed Task and idempotency storage. +- Kubernetes Job dispatch, queue brokers, autoscaling controllers, and stronger hostile-source or cross-tenant sandbox providers. +- Push-notification configuration, gRPC, JSON-RPC transport, and A2A extended Agent Cards. +- AHP server/client surfaces, long-lived interactive sessions, and client-contributed tools. +- Additional coding backends and provider-session restoration after gateway restart. +- Optional ATIF conversion after the format and tooling mature. + +**Outside this product's identity** + +- Evaluation authoring, datasets, assertions, grading, repetitions, experiment scheduling, and durable evaluation Runs. +- Caller-specific result projections such as Promptfoo `ProviderResponse` mapping. + +### Sources + +- [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) +- [AHP decision inputs](../research/agent-host-protocol-decision-inputs.md) +- [AI Evals ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) +- [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) +- [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) +- [Codex TypeScript SDK](https://github.com/openai/codex/tree/main/sdk/typescript) +- [OpenCode SDK and server](https://opencode.ai/docs/sdk/) +- [Pi RPC protocol](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/rpc.md) + +--- + +## Planning Contract + +### Key Technical Decisions + +- KTD1. **Use the official A2A JavaScript SDK behind an AllAgents request-handler decorator.** Pin a compatible A2A 1.x SDK. The decorator owns admission, canonical Task reservation, idempotent replay, stream snapshot selection, and cancellation routing before `DefaultRequestHandler` can allocate another Task or terminalize cancellation prematurely; the SDK retains standard transport/event mechanics. Governs R1-R8, R14. +- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes both versioned contracts, generated JSON Schemas, bounds, and fixtures. The worker protocol carries attempt identity, profile digest, dispatch acceptance, monotonic event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. +- KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates the invocation claim and submitted Task together, stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. +- KTD4. **Authenticate at HTTP ingress before A2A storage or dispatch.** Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. Unauthenticated mode is allowed only on a loopback listener. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Governs R5-R6, R13. +- KTD5. **Use fenced, separately deployable gateway and worker services.** The gateway owns A2A and durable results; the worker owns workspaces and provider processes. Every dispatch has a gateway-generated attempt ID, lease ID/epoch, short-lived capability, and event sequence. Workers idempotently accept duplicate delivery of the same attempt, reject conflicting attempts, and gateways ignore stale/out-of-order events and late terminal results. Governs R7, R10-R16, R21-R22. +- KTD6. **Make worker leases the orphan-execution fail-safe, not a replay mechanism.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry aborts and cleans the worker. Gateway restart invalidates old fences and records cleanup as unknown unless a process boundary proves it. Caller stream disconnect never affects the lease. Governs R7, R14, R16-R18. +- KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | opencode | pi` registry is the only production dispatch point. Governs R10, R14-R18, R21-R22. +- KTD8. **Use each provider's supported automation surface.** Codex uses `@openai/codex-sdk` streaming with `AbortSignal`; OpenCode uses its typed SDK against a worker-owned loopback server and session abort; Pi uses `pi --mode rpc --no-session` with a strict LF-delimited JSON parser, `agent_settled`, `get_session_stats`, and RPC abort. Governs R10, R14-R18. +- KTD9. **Make profiles the policy boundary.** Requests select a profile ID but cannot override backend credentials, executable paths, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. Profile digests enter idempotency and provenance. Governs R6, R11-R16, R21-R22. +- KTD10. **Capture Git and provider evidence as separate layers after quiescence.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded layer. Neither layer is promoted as exact causality when incomplete. Governs R16-R18. +- KTD11. **Treat Codex, OpenCode, and Pi as the complete initial backend set.** (session-settled: user-directed — chosen over adding an enterprise-only adapter: only the three named open-source gateway backends belong in this plan.) Governs R10. +- KTD12. **Separate terminal integrity from optional evidence bodies.** Identity, action outcome, failure/cancellation, termination, cleanup, Artifact index, completeness, and provenance must validate before terminal publication. Predictable budget truncation/redaction of logs, diffs, native events, or produced-file bodies may preserve completion with explicit metadata; capture failure that breaks the integrity kernel fails in the evidence phase. Governs R4, R17-R18. +- KTD13. **Harden Git acquisition as a network security boundary.** Accept canonical HTTPS origins only. Use hermetic Git configuration, disable redirects, proxies, helpers, hooks, filters, LFS smudge, submodule recursion, alternates, and non-HTTPS protocols. Revalidate normalized host/address policy for every connection, never forward credentials across origins, and verify the full object ID resolves to a commit fetched from the approved remote. Governs R12-R13, R22. +- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Provider/source credentials are absent from setup/check phases and child-visible worker control state. Stronger isolation is a separate sandbox-driver capability. Governs R16, R21-R22. +- KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. + +### High-Level Technical Design + +#### Component topology + +```mermaid +flowchart TB + Caller[Authenticated A2A caller] -->|HTTP+JSON / SSE| Gateway[execution-service gateway] + Gateway --> Auth[Auth, admission, profile policy] + Gateway --> Store[Generation-based Task and Artifact store] + Gateway -->|Fenced private protocol| Worker[Single-execution worker] + Worker --> Source[Hardened Git acquisition] + Worker --> Registry[Closed backend registry] + Registry --> Codex[Codex SDK] + Registry --> OpenCode[OpenCode SDK and server] + Registry --> Pi[Pi RPC process] + Worker --> Evidence[Quiesced checks, Git and native evidence] + Evidence -->|Bounded terminal result| Gateway + Gateway --> Telemetry[OpenTelemetry exporter] + Worker --> Telemetry +``` + +#### Admission, dispatch, and settlement sequence + +```mermaid +sequenceDiagram + participant C as Caller + participant G as Gateway decorator + participant S as Durable aggregate store + participant W as Worker + participant B as Backend adapter + + C->>G: SendMessage + required extension + G->>G: Authenticate, validate, authorize, quota, deadline + G->>S: Atomic claim + submitted Task + alt identical replay + S-->>G: Existing Task and current fence + G-->>C: Existing Task; follow active future events only + else new accepted Task + S-->>G: Task + attempt/lease fence + G->>W: Dispatch(attempt, fence, profile, source, deadline) + W-->>G: Accepted(attempt, fence) + W->>W: Materialize, verify, setup, baseline + W->>B: Invoke with isolated roots and policy + B-->>W: Progress, usage, native evidence + W-->>G: Sequenced fenced progress + G->>S: Compare-and-swap Task generation + opt cancellation or deadline wins + C->>G: CancelTask + G->>S: Persist cancellation intent once + G->>W: Fenced cancel + W->>B: Native abort + end + W->>W: Stop descendants, capture evidence, cleanup + W-->>G: Fenced terminal result + G->>S: Store blobs then atomically commit terminal manifest + G-->>C: Terminal status and Artifacts + end +``` + +#### Public A2A Task state + +```mermaid +stateDiagram-v2 + [*] --> Submitted: claim and Task committed + Submitted --> Working: worker accepts current fence + Submitted --> Canceled: cancellation proves no workspace exists + Submitted --> Failed: dispatch, restart, or source failure + Submitted --> Rejected: accepted policy refusal before work + Working --> Completed: integrity kernel and cleanup validate + Working --> Failed: provider, check, evidence, cleanup, crash, or restart failure + Working --> Rejected: known profile permission denial after stop and cleanup + Working --> Canceled: cancellation wins and stop/cleanup verify + Completed --> [*] + Failed --> [*] + Rejected --> [*] + Canceled --> [*] +``` + +Terminal states are immutable. Cancellation intent, termination, evidence capture, cleanup, and retention expiry are private record phases, not A2A Task states. + +#### Private execution-record phases + +```mermaid +stateDiagram-v2 + [*] --> Admitted + Admitted --> Dispatching + Dispatching --> Running: current fence accepted + Dispatching --> Terminalizing: dispatch rejected or unknown + Running --> CancelRequested: caller, deadline, shutdown, or lease expiry + Running --> Quiescing: provider and checks finish + CancelRequested --> Quiescing + Quiescing --> CapturingEvidence: descendants verified stopped + CapturingEvidence --> Cleaning + Cleaning --> Terminalizing + Terminalizing --> Retained + Retained --> Tombstoned: expiry + Tombstoned --> [*]: physical cleanup +``` + +### Output Structure + +```text +packages/execution-service/ + package.json + tsconfig.json + src/ + execution/ + contract.ts + extension-v1.ts + worker-protocol-v1.ts + errors.ts + profiles.ts + telemetry.ts + gateway/ + index.ts + config.ts + auth.ts + agent-card.ts + request-handler.ts + executor.ts + server.ts + worker-client.ts + store/ + gateway-repository.ts + file-gateway-repository.ts + worker/ + index.ts + config.ts + server.ts + lease.ts + workspace.ts + evidence.ts + adapters/ + types.ts + registry.ts + codex.ts + opencode.ts + pi.ts + tests/ + fixtures/execution/ + unit/execution/ + unit/gateway/ + unit/worker/ + e2e/execution-gateway.test.ts +containers/ + gateway.Dockerfile + worker.Dockerfile +examples/gateway/ + gateway.yaml + worker.yaml +docs/src/content/docs/ + guides/execution-gateway.mdx + reference/execution-gateway-configuration.mdx +``` + +### Configuration Contract + +- Gateway configuration defines listener/public URL, auth and canonical owner mapping, store/retention, admission and subscription quotas, low-space watermarks, Artifact limits, worker endpoints, internal capability secrets, and profiles. +- Each profile defines backend, worker route, allowed Git origins/addresses, provider/model settings, phase-specific environment allowlists, deterministic permissions, setup/check commands, artifact globs, effective deadline ceiling, trust class, resource limits, cleanup policy, and evidence budgets. +- Worker configuration fixes a private listener, one-execution concurrency, workspace root, lease grace, backend runtime constraints, trust domain, resource-control capability, and request/result limits. +- Configuration contains environment-variable names but never secret values. Startup resolves the complete graph, verifies that profile claims do not exceed deployment capabilities, and becomes ready only when store, workers, runtimes, quotas, and free-space reserves pass. + +### Error and Status Mapping + +| Condition | A2A result | Required extension detail | +|---|---|---| +| Authentication, malformed/unsupported extension, invalid source/profile, unauthorized policy, expired deadline, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | +| Identical invocation replay | Existing Task | No new Task, worker attempt, or quota reservation | +| Conflicting invocation key | Operation error; no new Task | Conflict code; existing Task unchanged | +| Worker capacity loss after acceptance | `TASK_STATE_FAILED` | `dispatch/capacity_exhausted`, retriable fact, no workspace created; gateway does not retry | +| Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | +| Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | +| Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | +| Source, setup, provider, check, mandatory evidence, worker crash, or infrastructure failure | `TASK_STATE_FAILED` | Typed phase, safe message, retriable fact, termination/cleanup/completeness | +| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors, native abort, termination, cleanup | +| Cancellation loses to terminal completion | Existing terminal Task / `TaskNotCancelableError` | No state mutation or second abort | +| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Output plus complete required evidence | +| Successful action with allowed bounded optional-evidence gap | `TASK_STATE_COMPLETED` | Per-dimension incomplete flag, reason, original/captured size, digest and redaction/truncation flags | +| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid and cleanup unknown unless proven | +| Retention expiry | Not found | Aggregate logically hidden before physical deletion; Artifact URL also invalid | + +### Phased Delivery + +1. Create the private Node 22 service package and freeze the public extension, worker protocol, profiles, fixtures, and error vocabulary. +2. Build authenticated durable A2A Task handling and fenced worker dispatch against a fake worker. +3. Build the single-execution worker lifecycle and hardened source/evidence handling against a fake adapter. +4. Add Codex, OpenCode, and Pi adapters in parallel, then compose them through the closed registry. +5. Package the services and run cross-backend, security, process, and A2A conformance before enabling a consumer. + +### System-Wide Impact + +- **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. +- **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. +- **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. +- **Security:** New review-critical surfaces are auth, owner-key derivation, source SSRF, admission/resource quotas, setup/check policy, phase-scoped secrets, internal fences, Artifact capture/serving, and reviewed-source trust enforcement. +- **Operations:** Gateway and worker health, readiness, quotas, low-space state, structured logs, traces, tombstone backlog, lease expiry, stale event rejection, and graceful shutdown need independent signals. +- **Consumers:** AI Evals can build its runner provider only after the Agent Card, extension schemas, and conformance fixtures are versioned and published. + +### Risks and Mitigations + +- **Provider API churn:** Pin exact compatible SDK/CLI versions in the service lockfile and worker image. Gate capabilities at startup and keep captured provider fixtures versioned. +- **False idempotency or stale settlement:** Claim Task/idempotency in one aggregate, use revision/fence compare-and-swap, sequence events, and fault-test duplicate delivery, cancellation races, restart, and late results. +- **Task/store corruption:** Publish immutable blobs and generations before one manifest switch; tombstone before deletion; validate owner tuples/manifests at startup; garbage-collect unreachable generations; document the one-replica limit. +- **Owner collision or path injection:** Hash a bounded canonical issuer/tenant/subject tuple, store and verify the tuple inside the owner aggregate, and use only server-generated opaque IDs in paths. +- **Orphan processes:** Combine explicit cancel, native abort, process-group termination, one-execution worker/container death, lease expiry, and quiescence proof before evidence capture. +- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral, origin-bound, and absent from repository config, process arguments, retained workspaces, logs, and errors. +- **Resource exhaustion:** Reserve per-owner/global gateway quota before claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. +- **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. +- **Evidence overclaim:** Enforce KTD12's integrity kernel and per-dimension completeness. Truncation and redaction remain independent facts. +- **Permission deadlock:** Initial profiles never prompt. Known requests resolve for one isolated invocation; unknown shapes fail closed as adapter incompatibility. +- **Trust-boundary overclaim:** Reject pooled hostile-source/cross-tenant profiles and state the reviewed mutual-trust boundary in config, readiness, Agent Card metadata, and docs. +- **Cross-platform drift:** Keep gateway/store tests cross-platform. State that worker execution and hardened evidence/source controls are Linux-only. + +### Assumptions + +- The first production deployment runs one gateway replica with persistent storage. Multi-replica transactional storage is deferred. +- Git over hardened HTTPS and exact commit object ID covers the initial consumer. Other source transports require a later extension version or capability. +- Setup and check commands are operator-controlled profile policy, not caller-supplied shell text. +- Initial repositories are reviewed inside one configured mutual-trust domain. Strong hostile-code or cross-tenant execution remains unavailable until a stronger sandbox driver exists. +- Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, OpenCode CLI 1.18.x with its compatible SDK, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. + +--- + +## Implementation Units + +### U1. Versioned public and worker contracts + +- **Goal:** Freeze the extension, profile vocabulary, private worker protocol, canonical digest input, result envelope, typed failures, and conformance fixtures before either service endpoint. +- **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12. +- **Dependencies:** None. +- **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. +- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas, one public extension URI, and one private protocol version. Include attempt/fence/lease identity, monotonic event sequence, accepted dispatch, renew/cancel, bounded terminal acknowledgement, public/private state separation, and integrity-kernel rules. Canonicalize caller input plus effective profile digest for idempotency. Generate checked-in JSON Schemas and fixtures from the same source. +- **Execution note:** Start with fixture-driven schema, framing, and digest tests. Observe failures for unknown versions, credential-bearing sources, mutable revisions, unsafe paths, invalid public states, stale fences, oversized records, and conflicting canonical inputs before implementing schemas. +- **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, and `src/core/native/types.ts` for safe error/provenance normalization. +- **Test scenarios:** + - A minimal valid request with text prompt, invocation key, profile, exact commit, and deadline parses and produces a stable digest across object-key ordering. + - Changing prompt, source object ID, profile ID/digest, artifact selection, or deadline changes the digest; trace IDs and transport metadata do not. + - A source URL with credentials, a branch/tag revision, absolute subdirectory, traversal, secret value, unknown backend, or unknown extension version is rejected safely. + - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. + - Worker fixtures reject missing/mismatched attempt IDs, lease epochs, profile digests, event sequence, bounds, and terminal acknowledgements. + - Completed, failed, canceled, and rejected results validate only with the integrity kernel; optional usage/native evidence gaps require explicit completeness reasons. + - File evidence accepts create/edit/delete/rename and rejects unsafe paths, duplicate identities, oversized inline content, and inconsistent before/after forms. +- **Verification:** Generated schemas are stable, public/private fixtures round-trip, digest vectors are cross-platform deterministic, and the private client/server fixture suite agrees before gateway or worker implementation. + +### U2. Authentication and durable gateway repository + +- **Goal:** Provide caller-scoped authentication, authorization, atomic Task/idempotency aggregates, Artifact storage, quota admission, pagination, restart fencing, logical expiry, and cleanup. +- **Requirements:** R4-R9, R13-R14, R17-R18, R22; AE2-AE4, AE6, AE8, AE10-AE13; KTD1, KTD3-KTD4, KTD12. +- **Dependencies:** U1. +- **Files:** `packages/execution-service/src/gateway/config.ts`, `packages/execution-service/src/gateway/auth.ts`, `packages/execution-service/src/gateway/store/gateway-repository.ts`, `packages/execution-service/src/gateway/store/file-gateway-repository.ts`, `packages/execution-service/tests/unit/gateway/auth.test.ts`, `packages/execution-service/tests/unit/gateway/file-gateway-repository.test.ts`. +- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Commit claim plus submitted Task in one manifest generation; publish immutable Artifact blobs before terminal manifest switch; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover and garbage-collect unreachable generations on startup. Reserve owner/global quotas before claims. Verify OIDC JWTs and constant-time static tokens before all repository access. +- **Execution note:** Implement concurrent-claim, transition-race, and crash-publication tests before request handling. Inject faults between blob, generation, manifest, tombstone, and cleanup operations. +- **Patterns to follow:** `src/core/marketplace.ts` and `src/core/profile/files.ts` for atomic publication/recovery, `src/core/mcp-http-stdio-proxy.ts` for private files and loopback safety, and the official A2A `TaskStore` owner-scoping contract. +- **Test scenarios:** + - Covers AE2-AE3. Concurrent identical claims create one aggregate; a conflicting digest returns conflict without dispatch permission. + - Covers AE4. Load/list/cancel/subscribe/Artifact lookup scopes before path/database access and gives unknown, unauthorized, and expired IDs indistinguishable behavior. + - Hostile/ambiguous issuer, tenant, subject, invocation key, Task ID, Artifact name, Unicode, case, delimiter, traversal, and Windows-reserved values cannot collide or become paths. + - All standard list filters, `historyLength`, page size 1-100, omitted Artifacts, ordering, total size, and always-present next token match A2A semantics. Tokens are owner/query-bound and reject malformed, swapped, or stale filters. + - Covers AE12. Terminal compare-and-swap wins once; stale fence, duplicate, and out-of-order updates cannot mutate the Task. + - Restart fails nonterminal Tasks once, invalidates fences, preserves terminal Tasks, and records cleanup unknown unless proven. + - Covers AE13. Exact expiry tombstones the aggregate before cleanup; failed deletion never restores visibility; same-key replay before expiry returns the old Task and after expiry creates a new Task. + - A crash between every aggregate publication step leaves either the prior or next valid manifest, never claim-without-Task or Task-with-missing-Artifact state. + - OIDC rejects wrong issuer, audience, signature, expiry, scope, tenant, and subject; static tokens and internal capabilities never appear in logs/errors. + - Quota-boundary races admit exactly the allowed count and preserve reserved capacity for cancel/terminal writes; low-space mode stops new claims without blocking settlement. + - Unauthenticated mode starts on loopback and refuses wildcard or non-loopback listeners. +- **Verification:** A fresh process retrieves prior records, fault recovery finds one valid aggregate generation, authorization cannot reveal neighboring owners, and expiry/quota behavior remains deterministic under concurrency. + +### U3. A2A gateway server and fenced worker client + +- **Goal:** Expose the accepted A2A profile while making admission, replay, streaming, lookup, worker fencing, failure, and cancellation use one durable state machine. +- **Requirements:** R1-R9, R11, R14-R15, R17-R22; F1-F5; AE1-AE4, AE6, AE8-AE13; KTD1-KTD7, KTD9, KTD12. +- **Dependencies:** U1, U2. +- **Files:** `packages/execution-service/src/gateway/agent-card.ts`, `packages/execution-service/src/gateway/request-handler.ts`, `packages/execution-service/src/gateway/executor.ts`, `packages/execution-service/src/gateway/server.ts`, `packages/execution-service/src/gateway/worker-client.ts`, `packages/execution-service/tests/unit/gateway/agent-card.test.ts`, `packages/execution-service/tests/unit/gateway/request-handler.test.ts`, `packages/execution-service/tests/unit/gateway/executor.test.ts`, `packages/execution-service/tests/e2e/gateway-fake-worker.test.ts`. +- **Approach:** Mount the official HTTP+JSON and Agent Card handlers behind auth. Put an AllAgents `A2ARequestHandler` decorator above `DefaultRequestHandler` so admission and canonical Task reservation happen first, identical replay bypasses new SDK Task/bus allocation, and cancellation waits for worker terminal evidence. Persist each public state before emission. Dispatch one fenced attempt, validate sequence/fence on every worker event, renew its lease, and atomically publish Artifact blobs plus terminal manifest. +- **Execution note:** Begin with an in-process fake worker and official A2A client. Prove operation errors versus accepted-Task failures, replay/subscribe behavior, fencing, cancellation races, and restart before adding providers. +- **Patterns to follow:** Official A2A sample `AgentExecutor`, `A2ARequestHandler`, `DefaultRequestHandler`, Express handlers, and cancellable-agent flow; `src/core/mcp-http-stdio-proxy.ts` for HTTP shutdown and loopback tests. +- **Test scenarios:** + - Covers AE1. `returnImmediately` true returns the submitted/working Task, false/unset waits for terminal state, and streaming starts with the same durable Task before ordered updates. + - Authentication, invalid extension/source/profile, expired deadline, and pre-claim quota failure return operation errors with no Task or worker request. + - Covers AE11. Capacity loss after acceptance fails the retained Task at `dispatch/capacity_exhausted`; ambiguous dispatch fails `dispatch_unknown`; neither is retried. + - Covers AE2-AE3. Identical send/stream replay returns the existing Task and follows only future events if active; conflict returns the documented operation error. + - Covers AE8. Active subscribe emits current snapshot then future events without missed-event replay; terminal subscribe errors and `GetTask` returns terminal truth. + - Covers AE4. Get/list/subscribe/cancel/Artifact endpoints apply owner authorization consistently. + - Covers AE6. Cancel in submitted/working, cancel versus accept/completion, caller versus deadline, duplicate cancel, and terminal cancel each produce one linearized outcome and at most one worker abort. + - Covers AE12. Duplicate, out-of-order, malformed, wrong-fence, and late terminal events cannot overwrite Task state; stale facts go only to safe telemetry. + - Known policy denial rejects only after stop/cleanup; unknown permission shape fails as adapter incompatibility. + - Caller SSE disconnect and telemetry exporter failure leave execution and terminal lookup intact. + - Graceful shutdown stops admission, claims cancellation for bounded active work, persists honest terminal state, and closes listeners. +- **Verification:** The official SDK client exercises every advertised operation against the built gateway and fake worker; persisted snapshots match streams while aggregate/fence invariants remain intact under races. + +### U4. Worker protocol and safe workspace lifecycle + +- **Goal:** Implement the single-execution worker, hardened immutable Git acquisition, profile enforcement, leases, isolated backend roots, resource controls, race-resistant evidence, termination, and cleanup independent of any provider. +- **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14. +- **Dependencies:** U1. +- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`. +- **Approach:** Authenticate and fence the private protocol, reserve the one execution before workspace creation, validate profile/deployment capability, and emit sequenced NDJSON. Acquire source under KTD13. Create separate workspace and backend config/data roots with a scrubbed phase-specific environment. Run setup, baseline, adapter, and checks under enforced budgets. Stop and verify the process group before descriptor-based regular-file evidence staging, then clean in `finally`. Lease expiry self-cancels. +- **Execution note:** Characterize every phase with a fake adapter, malicious fixtures, and disposable Git servers before real providers. Fault-inject dispatch acknowledgement, events, leases, acquisition, processes, evidence publication, and cleanup. +- **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, and `tests/helpers/env.ts` for isolated state. +- **Test scenarios:** + - Covers AE5. Exact object ID verifies; wrong/missing object, disallowed URL/host/address/port, credential-bearing URL, redirect, DNS rebinding, unsafe subdirectory, and fetch failure stop before adapter invocation. + - Repositories with LFS configuration/pointers, submodules, hooks, filters, alternates, proxy/helper config, or non-HTTPS secondary protocols cause no secondary connection or helper execution. + - Source credentials leave no repository config, process argument, child phase environment, log, error, evidence, or retained workspace trace. + - Setup changes establish the baseline; setup and checks receive no provider/control secrets; every backend gets disjoint invocation config/data roots with ambient selectors removed. + - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. + - Covers AE14. Concurrency above one and hostile/cross-tenant trust claims are rejected; worker control credentials are absent from child environment and configured filesystem roots. + - Source pack/tree/file/inode/path/sparse-file/disk limits and setup/provider/check CPU, memory, PID, network, phase-time, and workspace limits stop only the invocation and preserve worker health. + - Covers AE9. Known permissions receive one-invocation decisions; prompt-required profiles fail startup; unknown permission types fail the adapter. + - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps; capture I/O or malformed result that breaks the kernel fails the Task. + - Background swap attacks, links, mount crossings, FIFOs/devices/sockets, unstable files, and tampering between worker staging and gateway publication never expose external bytes or partial Artifacts. + - A worker crash before/after provider spawn reports cleanup complete only when the process/container boundary proves descendant death. +- **Verification:** A built worker mutates a disposable exact-SHA repository through the fake adapter and proves fenced dispatch, source hardening, phase isolation, budgets, quiescence, evidence integrity, and cleanup from its emitted result alone. + +### U5. Codex backend adapter + +- **Goal:** Run Codex through its supported TypeScript SDK while preserving structured progress, output, usage, file-change evidence, cancellation, and runtime identity. +- **Requirements:** R10-R22; AE1, AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. +- **Dependencies:** U4. +- **Files:** `packages/execution-service/src/worker/adapters/codex.ts`, `packages/execution-service/tests/unit/worker/adapters/codex.test.ts`, `packages/execution-service/tests/fixtures/execution/codex-events.jsonl`. +- **Approach:** Construct a fresh SDK thread in the invocation workspace with an isolated `CODEX_HOME` and scrubbed environment. Apply model, sandbox, network, approval, and writable-root settings only from the profile. Consume `runStreamed()` and pass an AbortSignal. Normalize agent messages, items, usage, failures, and file-change events while preserving the bounded native stream. +- **Execution note:** Drive the SDK through its executable override with a fixture Codex process before any credentialed smoke test. +- **Patterns to follow:** `src/core/profile/adapters/codex.ts` for root/environment isolation, `src/core/native/codex.ts` for version checks, and the SDK's `runStreamed`/AbortSignal contract. +- **Test scenarios:** + - A successful stream exposes thread ID, progress, final response, token usage, native file-change items, and terminal completion. + - Empty final response, turn failure, malformed JSONL, non-zero exit, unavailable runtime, and usage omission map to typed result/completeness fields. + - Covers AE6/AE12. Cancellation aborts the SDK process once; completion after cancel or stale fence cannot alter the selected terminal outcome. + - Profile sandbox, network, model, approval, working directory, and environment settings reach the SDK; caller input cannot override them. + - Two sequential invocations have disjoint `CODEX_HOME`, thread/session state, and writable roots; ambient selectors are removed. + - Native diffs and shared Git evidence coexist without claiming identical attribution. +- **Verification:** Fixture-driven tests cover every supported event and failure shape, followed by an isolated credentialed repository smoke test when Codex credentials are available. + +### U6. OpenCode backend adapter + +- **Goal:** Run OpenCode through its typed SDK and worker-owned loopback server while preserving session progress, output, usage/cost, diffs, permissions, cancellation, and disposal. +- **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. +- **Dependencies:** U4. +- **Files:** `packages/execution-service/src/worker/adapters/opencode.ts`, `packages/execution-service/tests/unit/worker/adapters/opencode.test.ts`, `packages/execution-service/tests/fixtures/execution/opencode-events.jsonl`. +- **Approach:** Start one loopback instance per invocation with isolated `OPENCODE_CONFIG`, `OPENCODE_CONFIG_DIR`, data/cache roots, scrubbed environment, profile configuration, and AbortSignal. Subscribe before prompting, create one session, resolve permission events from policy, collect message/session events and session diff, abort on cancellation, then delete the session and close the server in `finally`. Prevent the agent subprocess from reaching the worker control listener under the declared trust topology. +- **Execution note:** Inject SDK/server factories so protocol fixtures prove ordering and teardown without downloading or authenticating a real runtime. +- **Patterns to follow:** `src/core/profile/adapters/opencode.ts` for configuration/environment isolation and OpenCode's `createOpencode`, event subscription, session prompt/diff/abort APIs. +- **Test scenarios:** + - Successful execution collects text parts, assistant tokens/cost, session ID, events, and session diff before disposal. + - Subscription starts before prompt, ignores other session IDs, and finishes only after the target session becomes idle or errors. + - Covers AE6/AE12. Cancellation calls session abort once; late idle/completion cannot overwrite cancellation; server teardown remains idempotent. + - Covers AE9. Known permission events receive invocation-scoped `once`, `always`, or `reject` according to profile; `always` does not survive disposal and unknown types fail the adapter. + - Provider auth error, API error, aborted message, server-start timeout, SSE disconnect, and malformed SDK response map to typed failures. + - Sequential invocations have disjoint config/data/session roots; caller input cannot enable sharing, alter bind, select another project, or override provider/model/tools. +- **Verification:** Fixture tests prove session scoping, permission lifetime, cancellation races, and disposal, followed by an isolated credentialed repository smoke test when OpenCode credentials are available. + +### U7. Pi backend adapter + +- **Goal:** Run Pi through strict RPC mode while preserving settled completion, output, usage/cost, tool progress, cancellation, and process cleanup. +- **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. +- **Dependencies:** U4. +- **Files:** `packages/execution-service/src/worker/adapters/pi.ts`, `packages/execution-service/src/worker/adapters/pi-rpc.ts`, `packages/execution-service/tests/unit/worker/adapters/pi.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-rpc.test.ts`, `packages/execution-service/tests/fixtures/execution/pi-events.jsonl`. +- **Approach:** Spawn a supported Pi 0.85.x runtime with `--mode rpc --no-session`, an invocation-local `PI_CODING_AGENT_DIR`, profile model/provider, and scrubbed environment. Implement an LF-only JSONL parser rather than Node `readline`. Correlate responses, wait for `agent_settled`, read messages/stats, send RPC abort, and escalate process-group termination after the grace period. +- **Execution note:** Build parser and state-machine tests from captured RPC fixtures before process integration. +- **Patterns to follow:** `src/core/native/pi.ts` for version/trust checks, `src/core/profile/adapters/pi.ts` for root isolation, and the official Pi RPC framing/cancellation contract. +- **Test scenarios:** + - Successful prompt acceptance streams message/tool events, stops on `agent_settled`, retrieves final messages/stats, and reports session ID, usage, and cost. + - LF framing preserves `U+2028`/`U+2029` inside JSON strings, accepts CRLF by stripping trailing CR, handles partial/multiple chunks, and rejects oversized/malformed records. + - Covers AE6/AE12. Cancellation sends RPC abort once, waits for idle, then terminates the process group only after grace; late settled events cannot overwrite the terminal fence. + - Prompt rejection, agent error, aborted stop reason, retry/compaction sequence, premature exit, stderr overflow, and stats failure map truthfully. + - Sequential invocations have disjoint `PI_CODING_AGENT_DIR` and session state; caller input cannot send extension commands, steering/follow-up, arbitrary RPC commands, or override provider/model. +- **Verification:** Fixture and fake-process tests prove framing, correlation, settled completion, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when Pi credentials are available. + +### U8. Production registry, service packaging, and observability + +- **Goal:** Compose exactly three production adapters and package independently runnable gateway and worker services with safe startup, health, shutdown, tracing, and reproducible containers. +- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD11-KTD15. +- **Dependencies:** U3-U7. +- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. +- **Approach:** Register only Codex, OpenCode, and Pi through an explicit capability/availability map. Add gateway and worker entrypoints inside the private Node 22 workspace instead of the Node 18 CLI package. Validate config, store, workers, runtime pins, trust, quotas, and resource controls before readiness. Propagate `traceparent` and instrument every phase. Build a minimal gateway image with no provider runtimes and a one-execution worker image with exact runtime versions. +- **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. +- **Patterns to follow:** `src/core/profile/adapters/registry.ts` for explicit adapter composition, root package scripts for workspace delegation, `src/core/mcp-http-stdio-proxy.ts` for server lifecycle, `.github/workflows/ci.yml` for quality gates, and `.github/workflows/publish.yml` for immutable releases. +- **Test scenarios:** + - Registry exposes exactly Codex, OpenCode, and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects unknown backend IDs before workspace creation. + - Gateway and worker start from built service outputs, become ready only after dependencies pass, and stop gracefully on SIGTERM. + - Gateway readiness fails for malformed auth, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. + - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable resource enforcement, or unsupported backend runtime. + - Trace context enters through A2A, crosses the private call, and correlates result identities; exporter failure cannot change Task status. + - Gateway image contains no Codex, OpenCode, Pi, Git workspace, or provider credential material. + - Worker image pins all runtimes, confines one workspace/config root, enforces deployment limits, and completes fake-provider health smoke tests. + - Installing the root npm package on Node 18 does not load service dependencies; the private service workspace and containers enforce Node 22.19+. +- **Verification:** The registry dispatches every adapter through the same worker contract; built services and images pass lifecycle/security smoke tests; CI and publication bind immutable image tags to the release commit. + +### U9. Cross-backend conformance, documentation, and release evidence + +- **Goal:** Prove the public contract and operational workflow end to end and document deployment without leaking backend details into callers. +- **Requirements:** R1-R22; F1-F5; AE1-AE14. +- **Dependencies:** U1-U8. +- **Files:** `packages/execution-service/tests/e2e/execution-gateway.test.ts`, `packages/execution-service/tests/fixtures/execution/conformance-cases.ts`, `examples/gateway/gateway.yaml`, `examples/gateway/worker.yaml`, `docs/src/content/docs/guides/execution-gateway.mdx`, `docs/src/content/docs/reference/execution-gateway-configuration.mdx`, `README.md`, `CHANGELOG.md`. +- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases. Exercise gateway and worker as separate processes. Document extension/worker protocols, profiles, auth, trust boundary, storage/HA limits, source hardening, quotas, runtime requirements, cancellation races, evidence integrity, Artifact access, retention, observability, and troubleshooting. +- **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. +- **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, and Starlight guide/reference organization under `docs/src/content/docs/docs/`. +- **Test scenarios:** + - Covers AE1-AE14 through built services with a fake backend and official A2A client. + - The same mutation fixture passes through Codex, OpenCode, and Pi event fixtures and produces contract-equivalent normalized evidence. + - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; the one-execution worker serializes admitted work. + - Gateway restart, stream reconnect, lost dispatch acknowledgement, duplicate/out-of-order events, worker crash, lease expiry, cancellation race, provider failure, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome. + - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, control-endpoint probing, and secret-exfiltration fixtures are blocked within the documented reviewed-source boundary. + - Examples validate with production schemas and reference secrets only through environment variable names. + - Docs state one gateway replica, one execution per worker, reviewed mutual-trust sources, Node/runtime floors, and no hostile-code isolation claim. + - Opt-in real-provider smoke tests record backend/runtime versions and skip only when the named credential/runtime prerequisite is absent. +- **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor, the full suites and docs pass, the official A2A client exercises every advertised operation, and release evidence records each available real backend plus explicit skipped prerequisites. + +--- + +## Verification Contract + +| Gate | Applies to | Required evidence | +|---|---|---| +| Contract generation | U1 | Public extension and private worker schema generation report no drift; positive and negative fixtures pass. | +| Focused unit tests | U1-U8 | Active-unit tests pass with fault injection, state races, limits, cancellation, and cleanup. | +| Gateway/worker integration | U3-U4, U8-U9 | Built processes agree on fenced dispatch, sequencing, leases, Task persistence, Artifacts, shutdown, and cleanup. | +| Backend conformance | U5-U9 | One shared suite passes against Codex, OpenCode, and Pi adapters with fixture runtimes. | +| Credentialed provider smoke | U5-U7, U9 | Each available provider mutates a disposable exact-SHA repository; missing credentials/runtime are recorded as skipped prerequisites, never passing coverage. | +| A2A interoperability | U3, U9 | Official `@a2a-js/sdk` client passes immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, replay, cancel races, expiry, and owner isolation. | +| Security and abuse | U2-U4, U8-U9 | Malicious identity/source/artifact/resource fixtures prove auth-before-lookup, opaque owner keys, Git SSRF controls, phase-scoped secrets, quotas, quiescence, race-resistant capture, and trust-topology rejection. | +| Service packaging | U8-U9 | Root Node 18 install, private Node 22 build, gateway/worker smoke, and both container builds pass. | +| Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | +| Documentation | U9 | `bun run docs:build` passes and examples validate against current schemas. | + +The authoritative behavioral proof is the built-process E2E path with the official A2A client and a separately started worker. Unit tests alone do not prove protocol, durable aggregation, process isolation, fencing, cancellation, or cleanup integration. + +--- + +## Definition of Done + +### Global + +- Every R1-R22 requirement is implemented or explicitly shown in a passing conformance scenario. +- Public Agent Card/extension and private worker schemas are stable, generated from one source, and consumable without importing root AllAgents CLI modules. +- Codex, OpenCode, and Pi pass the same backend conformance suite and preserve bounded native evidence through the closed registry. +- Gateway and worker run as separate Node 22 processes/images; the Node 18 root CLI does not import service dependencies, and the gateway has no provider runtime or writable repository. +- Authentication precedes lookup, quota precedes Task creation, aggregate commits cannot split claims/Tasks/Artifacts, and terminal fences survive races and restart. +- Cancellation/deadlines reach one native abort, process termination, quiescence, evidence, and cleanup for all three backends. +- Source hardening, phase-scoped secrets, one-execution trust policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end. +- Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. +- Public documentation states supported topology, configuration, security boundary, storage/HA limitation, runtime pins, and deferred capabilities. +- Abandoned experiments, unused adapters, compatibility shims, generated scratch files, retained test workspaces, and stale documentation are removed. + +### Per unit + +- U1: Public/worker schemas, digest vectors, state/fence rules, typed failures, and fixtures are generated and stable. +- U2: Auth, opaque owner isolation, aggregate idempotency, CAS settlement, pagination, restart, quotas, Artifact access, tombstones, and cleanup pass fault injection. +- U3: Every advertised A2A operation agrees across stream and lookup while replay, fencing, and cancellation races preserve one Task. +- U4: Worker dispatch/source/setup/action/check/quiescence/evidence/cleanup lifecycle passes malicious and faulted disposable-repository scenarios. +- U5: Codex streaming, usage, native evidence, isolated roots, cancellation, and failure mapping pass adapter and applicable smoke verification. +- U6: OpenCode session/event/diff/permission isolation, abort, and disposal pass adapter and applicable smoke verification. +- U7: Pi strict JSONL framing, settled completion, stats, isolated roots, abort, and process cleanup pass adapter and applicable smoke verification. +- U8: Closed registry, Node-version separation, readiness, tracing, graceful shutdown, containers, and release artifacts work from built outputs. +- U9: Cross-backend E2E, A2A interoperability, abuse cases, examples, operator docs, changelog, and release evidence are complete. From 1f7644872b7a8ae5c9e242f9c3ab8ab30e83feeb Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Fri, 18 Sep 2026 16:51:02 +1000 Subject: [PATCH 04/44] docs(architecture): refine execution gateway scope --- ...-agent-execution-through-an-a2a-gateway.md | 100 +++++- ...0837-feat-coding-execution-gateway-plan.md | 288 +++++++++--------- 2 files changed, 234 insertions(+), 154 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index e798ef3e..1ed60865 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -50,15 +50,15 @@ or durable evaluation Run ledger. It does not own a consumer's result store. ### Keep the gateway separate from execution backends -The initial design supports three peer execution backends: +The initial design supports two execution backends, delivered in this order: -- Codex; -- OpenCode; and -- Pi. +1. Codex; and +2. Pi. Each backend implements the same conformance contract. Provider-specific -process, session, permission, cancellation, and evidence behavior remains -behind its adapter. +process, session, structured-output, cancellation, and evidence behavior +remains behind its adapter. OpenCode and other coding agents remain possible +follow-up adapters rather than part of the first delivery. Execution backends own repository materialization, environment setup, agent invocation, evidence collection, process termination, and cleanup. The gateway @@ -75,6 +75,63 @@ A separate gateway Pod is a service and failure boundary, not per-invocation security isolation. Deployments requiring hostile-code or tenant isolation must create or select a stronger execution boundary behind the gateway. +### Persist Task truth, not live provider execution + +The gateway durably stores Task identity, idempotency claims, terminal status, +Artifact metadata, and retained evidence. A provider execution itself is +ephemeral. The initial service does not checkpoint, reattach, resume, or +automatically replay an interrupted provider session. + +Gateway restart invalidates the active attempt fence and settles each +nonterminal Task failed once. A live worker that loses its lease aborts the +provider and cleans its invocation. If the worker process crashes, an external +supervisor terminates the complete execution boundary and the replacement +worker reaps or quarantines orphaned invocation roots before readiness. +Termination and filesystem cleanup are recorded separately and become complete +only when the responsible boundary proves them; otherwise the terminal record +says unknown. Durable execution and provider-session restoration require a +later decision backed by public provider guarantees. + +### Integrate providers directly + +The Codex adapter depends directly on `@openai/codex-sdk`; AllAgents does not +vendor or depend on Promptfoo's provider. Promptfoo's +[Codex provider](https://github.com/promptfoo/promptfoo/blob/main/src/providers/openai/codex-sdk.ts) +and +[tests](https://github.com/promptfoo/promptfoo/blob/main/test/providers/openai-codex-sdk.test.ts) +are characterization references for strict option mapping, minimal child +environment, working-directory validation, `AbortSignal`, structured output, +event normalization, and cleanup edge cases. + +AllAgents keeps only the gateway-owned subset: one fresh provider session per +Task, server-owned profile settings, bounded native evidence, typed failures, +and worker-proven process cleanup. It does not inherit Promptfoo configuration +layering, caching, pricing, eval retries, thread pools, or `ProviderResponse`. + +The extension defines `allagents.result-schema/v1` as a closed, bounded JSON +Schema Draft 2020-12 subset shared by admission, Codex, Pi, and terminal +validation. It requires an object root, requires every object schema to set +`additionalProperties: false`, lists every declared property in `required`, and +uses `null` unions for optional values. It allows only `type`, `properties`, +`required`, `additionalProperties` with the value `false`, `items`, `enum`, +`const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`, and rejects +remote references, format-dependent validation, and unknown keywords. The +extension version fixes byte, nesting, property, and enum limits. One shared +validator checks both the schema and the returned value, and the accepted schema +digest enters idempotency and provenance. Adapters cannot widen or narrow this +contract. + +Codex receives that schema through the SDK's per-turn `outputSchema`; Pi +implements the same terminal contract with an invocation-scoped terminating +tool. A successful structured request publishes exactly one Artifact named +`allagents.structured-result` with one A2A `Part` whose `data` field contains +the validated result object and whose `mediaType` is `application/json`. +Artifact metadata contains the result-schema version and digest. The Artifact +exists only for a valid result. The integrity +kernel always records `not_requested`, `not_produced`, `valid`, or `invalid`; +an earlier source, setup, provider, cancellation, or deadline outcome remains +the primary Task classification when no result could be produced. + ### Profile A2A 1.0 instead of inventing an invocation API The external contract profiles the Linux Foundation @@ -201,7 +258,7 @@ The gateway and selected backend are collectively responsible for: 1. resolving and verifying immutable source identity; 2. acquiring or restoring source through the selected transport; -3. creating a clean or explicitly reusable working location; +3. creating a fresh working location for one execution attempt; 4. running setup before the evaluated agent action; 5. applying permissions and execution isolation; 6. invoking the agent and propagating cancellation and deadlines; @@ -218,7 +275,9 @@ contract does not require source code to be baked into the runtime image. Credentials remain deployment policy. Requests must not embed deployment credentials. The gateway authenticates callers, and the selected backend scopes source and model credentials to the execution boundary without returning -secret-bearing paths or values. +secret-bearing paths or values. Provider and worker-control credentials must +also be absent from model-initiated command environments, tool output, retained +evidence, and repository-visible configuration. Retries must not multiply non-idempotent agent execution. Every request carries a caller-scoped stable invocation key through the AllAgents extension. The @@ -241,8 +300,14 @@ but their product and ownership model requires a separate decision. narrow coding-execution responsibility. - Consumers depend on A2A 1.0 plus a versioned AllAgents extension, not AllAgents TypeScript modules, CLI behavior, or workspace internals. -- Codex, OpenCode, and Pi are peer execution backends behind one conformance - suite. +- Codex and Pi are the initial execution backends behind one conformance suite; + Codex lands first and OpenCode is deferred. +- Durable Task and evidence records do not imply durable provider execution; + interrupted attempts fail rather than resume or replay. +- The result-schema subset, structured-result Artifact, and non-success result + states are public compatibility surface rather than adapter conventions. +- Reliable worker-crash cleanup requires an external execution supervisor and a + pre-readiness orphan-root reaper in addition to leases. - Gateway and execution workers scale and fail independently. - The gateway can remain lightweight; physical isolation and resource policy belong to the selected execution backend. @@ -293,6 +358,21 @@ around host-owned sessions, not agent-to-agent Task execution. Its reconnect and changeset models do not supply caller-scoped idempotency, immutable source handling, cleanup, complete terminal evidence, or bounded Task retention. +### Vendor Promptfoo's Codex provider + +Rejected because that provider includes Promptfoo-specific configuration +layering, caching, pricing, tracing, retry metadata, thread pooling, and result +mapping. AllAgents needs a smaller worker adapter against the Codex SDK and can +reuse Promptfoo's observable behavior as characterization evidence without +copying its implementation. + +### Treat provider session persistence as durable execution + +Rejected because a resumable provider thread does not prove workspace, +process, cancellation, evidence, or cleanup continuity across gateway or worker +failure. The initial service durably records failure and cleanup truth but does +not resume interrupted work. + ## Reconsider when Revisit this decision if A2A standardizes the required coding-execution evidence diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 1d65837d..60987efd 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -13,11 +13,11 @@ execution: code ## Goal Capsule -- **Objective:** External systems can run Codex, OpenCode, or Pi against an immutable repository revision through one authenticated, cancellable, evidence-preserving remote contract. -- **Means:** Add a separately deployable A2A 1.0 gateway, a private worker protocol, and backend-neutral workers with three provider adapters (KTD1, KTD5, KTD7). +- **Objective:** External systems can run Codex or Pi against an immutable repository revision through one authenticated, cancellable, evidence-preserving remote contract. +- **Means:** Add a separately deployable A2A 1.0 gateway, a private worker protocol, and backend-neutral workers with two direct provider adapters (KTD1, KTD5, KTD7-KTD8). - **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) owns the public boundary. The A2A 1.0 specification owns core wire semantics. The versioned AllAgents extension owns coding-execution semantics. -- **Execution profile:** Build contract-first, then durable gateway state, worker lifecycle, provider adapters, packaging, and cross-backend conformance. Preserve the existing local CLI and Node 18 package compatibility. -- **Stop conditions:** Do not execute agents in the gateway process, accept mutable source identity, put deployment credentials in requests, treat streams or telemetry as terminal evidence, or add evaluation behavior. +- **Execution profile:** Build contract-first, then durable Task/evidence state, worker lifecycle, Codex, Pi, packaging, and cross-backend conformance. Preserve the existing local CLI and Node 18 package compatibility. +- **Stop conditions:** Do not execute agents in the gateway process, accept mutable source identity, put deployment credentials in requests, treat streams or telemetry as terminal evidence, treat provider sessions as recovery checkpoints, vendor an evaluator's provider implementation, or add evaluation behavior. - **Tail ownership:** The implementing workflow runs focused contract and lifecycle tests, the complete repository quality gates, isolated gateway/worker smoke tests, provider-specific credentialed smoke tests where credentials are available, and documentation validation. --- @@ -32,20 +32,21 @@ AllAgents gains a remote coding-execution service without becoming an evaluation AllAgents currently configures and launches coding clients but has no service boundary for external callers. AI Evals and future consumers would otherwise need to import AllAgents internals, drive interactive CLIs, or independently reimplement repository acquisition, permissions, cancellation, evidence, and cleanup. -The three initial runtimes expose different programmatic contracts. Codex provides a TypeScript SDK over structured JSONL events, OpenCode provides a typed HTTP SDK and SSE event stream, and Pi provides a strict JSONL RPC mode. The public service must preserve one stable lifecycle without flattening provider-specific facts into false equivalence. +The two initial runtimes expose different programmatic contracts. Codex provides a TypeScript SDK over structured JSONL events and native per-turn `outputSchema`; Pi provides a strict JSONL RPC mode and invocation-scoped custom tools. The public service must preserve one stable lifecycle and structured-result contract without flattening provider-specific facts into false equivalence. ### Actors - A1. **Gateway caller:** An authenticated service such as AI Evals that creates, observes, lists, cancels, and retrieves coding-execution Tasks. - A2. **Execution gateway:** The A2A server that owns caller scope, Task identity, idempotency, routing, retention, and normalized results. - A3. **Execution worker:** A separately deployed process that owns source materialization, one mutable workspace per invocation, provider execution, evidence capture, and cleanup. -- A4. **Backend adapter:** The Codex, OpenCode, or Pi integration that translates native events, cancellation, usage, failures, and evidence into the worker contract. +- A4. **Backend adapter:** The Codex or Pi integration that translates native events, structured results, cancellation, usage, failures, and evidence into the worker contract. - A5. **Operator:** The person or deployment system that defines profiles, credentials, limits, retention, worker endpoints, and observability policy. ### Key Decisions - **Profile A2A rather than creating a public invocation API.** The service keeps standard Agent Cards, Tasks, Artifacts, operations, errors, and capability negotiation. Governs R1-R4. - **Keep execution outside the gateway process.** Mutable repositories and provider processes belong to workers. Governs R10-R16, R21-R22. +- **Persist Task truth, not live executions.** Accepted Task identity and terminal evidence survive restart; provider sessions do not resume or replay. Governs R7, R14, R16-R18. - **Keep evaluation outside AllAgents.** Dataset expansion, repetitions, assertions, scoring, retries, and durable evaluation Runs remain caller concerns. Governs R20. ### Requirements @@ -61,23 +62,23 @@ The three initial runtimes expose different programmatic contracts. Codex provid - R5. Every protocol operation authenticates the caller and scopes Task lookup, listing, subscription, cancellation, and artifact retrieval to that caller's tenant and principal before storage access can reveal resource existence. - R6. Authentication, required-extension validation, request validation, source/profile authorization, quota admission, and deadline validation complete before Task creation. A caller-scoped invocation key, effective profile, authenticated owner, and canonical request digest then bind atomically to one Task; identical replay returns that Task and conflicting reuse is rejected without dispatch. -- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Task state and terminal Artifact metadata survive gateway restart; nonterminal Tasks that cannot be reattached settle failed once and stale worker events cannot overwrite them. +- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Task state and terminal Artifact metadata survive gateway restart. Every nonterminal Task present at startup settles failed once, its old attempt fence is invalidated, and stale worker events cannot overwrite it; the initial service never resumes or automatically replays interrupted provider work. - R8. List operations implement all A2A filters, history bounds, page-size bounds, owner/query-bound cursor pagination, and descending status-update time. One immutable expiry logically hides the Task, claim, events, and artifacts before best-effort physical deletion; expired and unauthorized IDs are indistinguishable. - R9. Small deployments work without an external database. The built-in durable store supports one gateway replica, enforces per-owner/global admission and storage quotas, and reserves capacity for cancellation and terminal settlement; multi-replica storage is outside this delivery. **Execution and policy** -- R10. Codex, OpenCode, and Pi are peer backends behind one conformance contract. (session-settled: user-directed — chosen over an additional enterprise-only adapter: the open-source gateway supports the three named runtimes directly.) -- R11. A request selects a server-defined execution profile. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. +- R10. Codex and Pi are the complete initial backend set behind one conformance contract, delivered Codex first and Pi second. OpenCode is deferred. (session-settled: user-directed.) +- R11. A request selects a server-defined execution profile and may include one `allagents.result-schema/v1` schema for the terminal result: a bounded JSON Schema Draft 2020-12 subset with an object root, every object schema setting `additionalProperties: false`, every declared property listed in `required`, optional values represented by `null` unions, and only `type`, `properties`, `required`, `additionalProperties` with the value `false`, `items`, `enum`, `const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`. The extension version fixes byte, depth, property, and enum limits; admission rejects remote references, format-dependent validation, and unknown keywords; one shared validator governs schema admission and returned values. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. Requests cannot supply raw provider configuration. - R12. The only initial remote source form is a canonical credential-free HTTPS Git URL plus full commit object ID and optional repository-relative subdirectory. Acquisition revalidates destination policy for every connection, disables redirects and repository-controlled secondary fetch/exec features, uses hermetic Git configuration, and verifies that the fetched object is the requested commit before setup. -- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, and retained workspaces. +- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, and every model-initiated command or tool environment. - R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. - R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. -- R16. A worker creates a fresh invocation directory and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, and runs configured checks. It then proves all invocation descendants quiescent before final evidence/artifact capture and cleanup or explicit retention. +- R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves all invocation descendants quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. An external supervisor terminates the complete execution boundary when the worker process crashes, and a replacement worker reaps or quarantines orphaned roots before readiness. **Evidence and observability** -- R17. Every terminal result contains an integrity kernel: Task/source/profile/backend identities, action outcome, cancellation or failure classification, termination and cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. +- R17. Every terminal result contains an integrity kernel: Task/source/profile/backend identities, action outcome, a structured-result state of `not_requested`, `not_produced`, `valid`, or `invalid` plus reason and schema digest when requested, cancellation or failure classification, separate termination and filesystem-cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. A valid structured result is exactly one `allagents.structured-result` Artifact with one A2A `Part` whose `data` field contains the validated result object and whose `mediaType` is `application/json`; missing or invalid result data never publishes that Artifact. Pre-output source, setup, provider, cancellation, or deadline outcomes retain their primary Task classification and record `not_produced` secondarily. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. - R18. Normalized file evidence distinguishes create, edit, delete, and rename where truthful. It preserves bounded provider-native diffs, events, or trajectories when normalization loses information and separately records truncation, redaction, attribution, original/captured size, and digest semantics. - R19. Gateway and worker spans propagate W3C Trace Context and export OpenTelemetry data. Telemetry is operational evidence, not the only durable result. @@ -91,8 +92,8 @@ The three initial runtimes expose different programmatic contracts. Codex provid - F1. **Admit, create, and stream an execution** - **Actors:** A1, A2, A3, A4. - - **Trigger:** A caller sends a text Message with the required extension, immutable source, profile, invocation key, and deadline. - - **Steps:** Authenticate; validate and authorize the complete request; reserve quota; atomically claim idempotency and create a submitted Task; dispatch a fenced worker attempt; materialize and verify source; execute the selected backend; persist progress before emission; terminalize with Artifacts after quiescence and cleanup. + - **Trigger:** A caller sends a text Message with the required extension, immutable source, profile, invocation key, deadline, and optional bounded result schema. + - **Steps:** Authenticate; validate and authorize the complete request and result-schema subset; reserve quota; atomically claim idempotency and create a submitted Task; dispatch a fenced worker attempt; materialize and verify source; execute the selected backend; validate structured output with the shared validator when requested; persist progress before emission; terminalize with the fixed-name structured-result Artifact only for a valid result and with evidence Artifacts after quiescence and cleanup. - **Outcome:** `returnImmediately: true` returns the durable current Task, false/unset waits for terminal state, and streaming starts with that Task before ordered updates. - **Covered by:** R1-R22. - F2. **Replay or reconnect to an invocation** @@ -107,11 +108,11 @@ The three initial runtimes expose different programmatic contracts. Codex provid - **Steps:** Atomically record the first cancellation source; if dispatch never occurred, prove no workspace exists; otherwise send one fenced worker cancel, invoke native abort, terminate descendants, capture termination-safe evidence, clean, and publish canceled only after verification. - **Outcome:** Completion that wins first remains terminal and later cancel returns `TaskNotCancelableError`; cancellation that wins suppresses late provider success and fails instead of claiming canceled when termination or cleanup cannot be verified. - **Covered by:** R7, R14, R16-R18. -- F4. **Recover from gateway or worker loss** +- F4. **Settle after gateway or worker loss** - **Actors:** A2, A3. - - **Trigger:** The gateway restarts with nonterminal Tasks, an acknowledgement is lost, or a worker crashes. - - **Steps:** Invalidate the attempt fence; settle each non-reattachable Task failed once; reject late events/results; stop renewing leases; let workers self-abort and clean. Record cleanup complete only when the worker/process boundary proves it; otherwise record unknown. - - **Outcome:** One Task has one terminal result, no ambiguous dispatch is retried automatically, and no stale worker can overwrite durable truth. + - **Trigger:** The gateway restarts with nonterminal Tasks, an acknowledgement is lost, a live worker loses its lease, or a worker process crashes. + - **Steps:** Invalidate the attempt fence and settle every affected Task failed once without provider-session reattachment or automatic replay. A live worker that loses its lease self-aborts and cleans. On worker-process crash, the external supervisor terminates the complete execution boundary; the replacement worker proves termination, then reaps or quarantines orphaned roots before readiness. Reject late events/results and record termination and filesystem cleanup separately as complete only when the responsible boundary proves each outcome. + - **Outcome:** One Task has one terminal result, interrupted work is never presented as resumed, no stale worker can overwrite durable truth, and a crashed worker cannot leave an unowned process or reusable workspace. - **Covered by:** R7, R9, R14, R16-R18, R21-R22. - F5. **Expire retained execution data** - **Actors:** A1, A2. @@ -122,28 +123,28 @@ The three initial runtimes expose different programmatic contracts. Codex provid ### Acceptance Examples -- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile and an exact Git SHA, when the caller streams a request, then one Task moves from submitted to working to completed and later `GetTask` returns the same output and evidence Artifacts. +- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile, an exact Git SHA, and an optional result schema, when the caller streams a request, then one Task moves from submitted to working to completed and later `GetTask` returns the same validated output and evidence Artifacts. - AE2. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with the same canonical request, then the gateway returns the original Task without a second worker dispatch. - AE3. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with a different prompt, source, profile, or deadline, then the gateway rejects the request and leaves the original Task unchanged. - AE4. **Covers R5.** Given a Task owned by caller A, when caller B lists Tasks, gets the Task, cancels it, subscribes, or requests an Artifact, then the gateway reveals no resource existence or content. - AE5. **Covers R12, R16-R18.** Given a requested SHA that does not match the materialized repository, when the worker verifies source, then provider execution never starts and the Task fails with source-verification and cleanup evidence. - AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. -- AE7. **Covers R10.** Given equivalent profiles and fixture runtime events for Codex, OpenCode, and Pi, when each completes the same repository mutation, then all three produce the same required normalized result fields while retaining distinct native evidence. +- AE7. **Covers R10-R11, R17.** Given equivalent profiles, one accepted `allagents.result-schema/v1` schema, and fixture runtime events for Codex and Pi, when each completes the same repository mutation, then both validate with the same schema and validator, publish the same fixed-name structured-result Artifact containing one A2A `Part` with the validated `data` and `mediaType: application/json`, record the same integrity state, and produce the required normalized evidence fields while retaining distinct native evidence. - AE8. **Covers R4, R7, R19.** Given a caller disconnects during work, when it subscribes again, then it receives the current Task and future updates without duplicate dispatch; telemetry loss does not affect later terminal lookup. - AE9. **Covers R15.** Given a known capability denied by profile, the accepted Task becomes rejected after stop and cleanup; given an unknown permission type, it becomes failed as an adapter incompatibility without waiting for a client. - AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. - AE11. **Covers R6, R22.** Given invalid input or exhausted admission quota, the gateway returns a request/resource error and creates no Task; given capacity disappears after durable acceptance, the retained Task fails at dispatch and replay returns it without retry. -- AE12. **Covers R7, R14.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only safe operator telemetry. +- AE12. **Covers R7, R14, R16.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only safe operator telemetry. Given the worker is killed with live descendants and an invocation root, its supervisor terminates the execution boundary and the replacement worker reaps or quarantines the root before readiness without changing the failed Task. - AE13. **Covers R8.** Given a Task reaches expiry while physical deletion fails, all Task and Artifact operations return the same not-found response and the invocation key can create a new Task. - AE14. **Covers R21-R22.** Given a profile requests pooled hostile-source or cross-tenant execution, startup/admission rejects it; a reviewed single-trust-domain profile runs one bounded execution without exposing worker control credentials to the child environment. ### Success Criteria - The official A2A JavaScript client can discover the card and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, replay, cancel, and expiry behavior against the built service. -- One conformance fixture passes unchanged through Codex, OpenCode, and Pi adapters. -- Admission, replay, fencing, cancellation races, restart recovery, authorization isolation, source hardening, quotas, and evidence integrity have deterministic integration coverage. +- One conformance fixture passes unchanged through the Codex and Pi adapters. +- Admission, replay, fencing, cancellation races, restart terminalization without resume, supervised worker-crash cleanup, authorization isolation, source hardening, portable structured-result validation, quotas, and evidence integrity have deterministic integration coverage. - The gateway image contains no coding-agent runtime and cannot access worker workspace roots. -- The initial worker runs one reviewed-trust-domain execution at a time and leaves no live descendant or retained workspace unless policy requests retention. +- The initial worker runs one reviewed-trust-domain execution at a time, model-initiated tools receive no provider/control credentials, repository Pi extensions cannot auto-load, and no live descendant or reusable workspace survives a completed or crashed attempt. ### Scope Boundaries @@ -151,7 +152,7 @@ The three initial runtimes expose different programmatic contracts. Codex provid - A2A 1.0 HTTP+JSON and SSE streaming. - One versioned AllAgents coding-execution extension and one versioned private worker protocol. -- Codex, OpenCode, and Pi backends. +- Codex and Pi backends. - Built-in bearer authentication with OIDC/JWT and static service-token modes. - Single-replica durable file storage, authenticated Artifact retrieval, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. - Reviewed repositories in one configured mutual-trust domain per worker deployment. @@ -159,10 +160,11 @@ The three initial runtimes expose different programmatic contracts. Codex provid **Deferred to follow-up work** - Multi-replica database-backed Task and idempotency storage. +- Durable provider execution, checkpointing, provider-session restoration, and automatic replay after gateway or worker restart. +- OpenCode and additional coding backends. - Kubernetes Job dispatch, queue brokers, autoscaling controllers, and stronger hostile-source or cross-tenant sandbox providers. - Push-notification configuration, gRPC, JSON-RPC transport, and A2A extended Agent Cards. - AHP server/client surfaces, long-lived interactive sessions, and client-contributed tools. -- Additional coding backends and provider-session restoration after gateway restart. - Optional ATIF conversion after the format and tooling mature. **Outside this product's identity** @@ -178,8 +180,14 @@ The three initial runtimes expose different programmatic contracts. Codex provid - [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) - [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) - [Codex TypeScript SDK](https://github.com/openai/codex/tree/main/sdk/typescript) -- [OpenCode SDK and server](https://opencode.ai/docs/sdk/) +- [Codex configuration reference](https://developers.openai.com/codex/config-reference) +- [Promptfoo Codex provider documentation](https://github.com/promptfoo/promptfoo/blob/main/site/docs/providers/openai-codex-sdk.md) +- [Promptfoo Codex provider implementation](https://github.com/promptfoo/promptfoo/blob/main/src/providers/openai/codex-sdk.ts) +- [Promptfoo Codex provider tests](https://github.com/promptfoo/promptfoo/blob/main/test/providers/openai-codex-sdk.test.ts) - [Pi RPC protocol](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/rpc.md) +- [Pi CLI reference](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/README.md#cli-reference) +- [Pi extension API](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/extensions.md) +- [Pi provider credentials](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/providers.md) --- @@ -188,20 +196,20 @@ The three initial runtimes expose different programmatic contracts. Codex provid ### Key Technical Decisions - KTD1. **Use the official A2A JavaScript SDK behind an AllAgents request-handler decorator.** Pin a compatible A2A 1.x SDK. The decorator owns admission, canonical Task reservation, idempotent replay, stream snapshot selection, and cancellation routing before `DefaultRequestHandler` can allocate another Task or terminalize cancellation prematurely; the SDK retains standard transport/event mechanics. Governs R1-R8, R14. -- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes both versioned contracts, generated JSON Schemas, bounds, and fixtures. The worker protocol carries attempt identity, profile digest, dispatch acceptance, monotonic event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. +- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes both versioned contracts, generated JSON Schemas, bounds, and fixtures. The public contract carries the `allagents.result-schema/v1` closed subset, its canonical digest, four structured-result states, and the fixed `allagents.structured-result` Artifact containing one A2A `Part` with `data` and `mediaType: application/json`. The worker protocol carries attempt identity, profile digest, dispatch acceptance, monotonic event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. - KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates the invocation claim and submitted Task together, stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. - KTD4. **Authenticate at HTTP ingress before A2A storage or dispatch.** Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. Unauthenticated mode is allowed only on a loopback listener. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Governs R5-R6, R13. -- KTD5. **Use fenced, separately deployable gateway and worker services.** The gateway owns A2A and durable results; the worker owns workspaces and provider processes. Every dispatch has a gateway-generated attempt ID, lease ID/epoch, short-lived capability, and event sequence. Workers idempotently accept duplicate delivery of the same attempt, reject conflicting attempts, and gateways ignore stale/out-of-order events and late terminal results. Governs R7, R10-R16, R21-R22. -- KTD6. **Make worker leases the orphan-execution fail-safe, not a replay mechanism.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry aborts and cleans the worker. Gateway restart invalidates old fences and records cleanup as unknown unless a process boundary proves it. Caller stream disconnect never affects the lease. Governs R7, R14, R16-R18. -- KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | opencode | pi` registry is the only production dispatch point. Governs R10, R14-R18, R21-R22. -- KTD8. **Use each provider's supported automation surface.** Codex uses `@openai/codex-sdk` streaming with `AbortSignal`; OpenCode uses its typed SDK against a worker-owned loopback server and session abort; Pi uses `pi --mode rpc --no-session` with a strict LF-delimited JSON parser, `agent_settled`, `get_session_stats`, and RPC abort. Governs R10, R14-R18. -- KTD9. **Make profiles the policy boundary.** Requests select a profile ID but cannot override backend credentials, executable paths, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. Profile digests enter idempotency and provenance. Governs R6, R11-R16, R21-R22. +- KTD5. **Use fenced, separately deployable gateway and worker services.** The gateway owns A2A and durable Task/results truth; the worker owns ephemeral execution attempts, workspaces, and provider processes. Every dispatch has a gateway-generated attempt ID, lease ID/epoch, short-lived capability, and event sequence. Workers idempotently accept duplicate delivery of the same attempt, reject conflicting attempts, and gateways ignore stale/out-of-order events and late terminal results. Governs R7, R10-R16, R21-R22. +- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Worker-process exit makes the external supervisor terminate the complete execution boundary; before readiness the replacement worker proves termination and reaps or quarantines orphaned invocation roots. Termination and filesystem cleanup remain separate outcomes and are unknown until proved. The gateway never reattaches to or resumes a provider session. Caller stream disconnect never affects the lease. Governs R7, R14, R16-R18. +- KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, optional structured result, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, schema validation, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | pi` registry is the only production dispatch point. Governs R10-R11, R14-R18, R21-R22. +- KTD8. **Use each provider's supported automation surface directly.** Codex depends directly on pinned `@openai/codex-sdk`, creates one fresh thread per Task, passes `AbortSignal` and optional per-turn `outputSchema`, consumes streamed events, and applies a pinned shell-environment policy that excludes provider/control credentials from model-initiated commands. Pi uses `pi --mode rpc --no-session --no-extensions --no-builtin-tools` with a strict LF-delimited JSON parser, `agent_settled`, `get_session_stats`, RPC abort, an invocation-local credential store rather than credential environment variables, and one explicitly loaded worker-owned policy extension outside the repository. That extension supplies workspace-confined filesystem/command tools and the terminating result tool; no repository extension or unrestricted built-in tool loads. Promptfoo's Codex provider and tests are characterization references only; AllAgents neither vendors them nor inherits their config, cache, pricing, retry, thread-pool, or `ProviderResponse` concerns. Governs R10-R18. +- KTD9. **Make profiles the policy boundary.** Requests select a profile ID and may provide only an `allagents.result-schema/v1` schema. They cannot override backend credentials, executable paths, provider config, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. Profile digests and the canonical result-schema digest enter idempotency and provenance. Governs R6, R11-R16, R21-R22. - KTD10. **Capture Git and provider evidence as separate layers after quiescence.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded layer. Neither layer is promoted as exact causality when incomplete. Governs R16-R18. -- KTD11. **Treat Codex, OpenCode, and Pi as the complete initial backend set.** (session-settled: user-directed — chosen over adding an enterprise-only adapter: only the three named open-source gateway backends belong in this plan.) Governs R10. -- KTD12. **Separate terminal integrity from optional evidence bodies.** Identity, action outcome, failure/cancellation, termination, cleanup, Artifact index, completeness, and provenance must validate before terminal publication. Predictable budget truncation/redaction of logs, diffs, native events, or produced-file bodies may preserve completion with explicit metadata; capture failure that breaks the integrity kernel fails in the evidence phase. Governs R4, R17-R18. +- KTD11. **Treat Codex and Pi as the complete initial backend set.** Codex lands first; Pi lands second against the established contract; OpenCode is deferred. (session-settled: user-directed.) Governs R10. +- KTD12. **Separate terminal integrity from optional evidence bodies.** Identity, action outcome, the four-state structured-result record and fixed Artifact rule, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance must validate before terminal publication. A pre-output failure records `not_produced` without replacing its primary phase classification. Predictable budget truncation/redaction of logs, diffs, native events, or produced-file bodies may preserve completion with explicit metadata; capture failure that breaks the integrity kernel fails in the evidence phase. Governs R4, R17-R18. - KTD13. **Harden Git acquisition as a network security boundary.** Accept canonical HTTPS origins only. Use hermetic Git configuration, disable redirects, proxies, helpers, hooks, filters, LFS smudge, submodule recursion, alternates, and non-HTTPS protocols. Revalidate normalized host/address policy for every connection, never forward credentials across origins, and verify the full object ID resolves to a commit fetched from the approved remote. Governs R12-R13, R22. -- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Provider/source credentials are absent from setup/check phases and child-visible worker control state. Stronger isolation is a separate sandbox-driver capability. Governs R16, R21-R22. -- KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. +- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Provider/source credentials are absent from setup/check phases, model-initiated commands and tools, and child-visible worker control state. Pi disables repository extensions and built-in tools; only the worker-owned policy extension may load, and its replacement tools confine paths to the invocation workspace and spawn commands with the phase allowlist. Stronger isolation is a separate sandbox-driver capability. Governs R13, R16, R21-R22. +- KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, Codex SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. ### High-Level Technical Design @@ -216,7 +224,6 @@ flowchart TB Worker --> Source[Hardened Git acquisition] Worker --> Registry[Closed backend registry] Registry --> Codex[Codex SDK] - Registry --> OpenCode[OpenCode SDK and server] Registry --> Pi[Pi RPC process] Worker --> Evidence[Quiesced checks, Git and native evidence] Evidence -->|Bounded terminal result| Gateway @@ -312,6 +319,7 @@ packages/execution-service/ execution/ contract.ts extension-v1.ts + result-schema-v1.ts worker-protocol-v1.ts errors.ts profiles.ts @@ -332,6 +340,8 @@ packages/execution-service/ index.ts config.ts server.ts + supervisor.ts + reaper.ts lease.ts workspace.ts evidence.ts @@ -339,8 +349,8 @@ packages/execution-service/ types.ts registry.ts codex.ts - opencode.ts pi.ts + pi-policy-extension.ts tests/ fixtures/execution/ unit/execution/ @@ -362,7 +372,7 @@ docs/src/content/docs/ - Gateway configuration defines listener/public URL, auth and canonical owner mapping, store/retention, admission and subscription quotas, low-space watermarks, Artifact limits, worker endpoints, internal capability secrets, and profiles. - Each profile defines backend, worker route, allowed Git origins/addresses, provider/model settings, phase-specific environment allowlists, deterministic permissions, setup/check commands, artifact globs, effective deadline ceiling, trust class, resource limits, cleanup policy, and evidence budgets. -- Worker configuration fixes a private listener, one-execution concurrency, workspace root, lease grace, backend runtime constraints, trust domain, resource-control capability, and request/result limits. +- Worker configuration fixes a private listener, one-execution concurrency, workspace root, execution-supervisor mechanism, pre-readiness orphan policy, lease grace, backend runtime constraints, trust domain, resource-control capability, and request/result limits. - Configuration contains environment-variable names but never secret values. Startup resolves the complete graph, verifies that profile claims do not exceed deployment capabilities, and becomes ready only when store, workers, runtimes, quotas, and free-space reserves pass. ### Error and Status Mapping @@ -376,39 +386,43 @@ docs/src/content/docs/ | Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | | Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | | Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | -| Source, setup, provider, check, mandatory evidence, worker crash, or infrastructure failure | `TASK_STATE_FAILED` | Typed phase, safe message, retriable fact, termination/cleanup/completeness | -| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors, native abort, termination, cleanup | +| Source, setup, provider, check, mandatory evidence, worker crash, or infrastructure failure | `TASK_STATE_FAILED` | Typed primary phase, safe message, retriable fact, structured result `not_produced` when requested, separate termination/cleanup/completeness | +| Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` or `structured_result/invalid`, no structured-result Artifact | +| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors, native abort, structured result `not_produced` unless already valid, termination, cleanup | | Cancellation loses to terminal completion | Existing terminal Task / `TaskNotCancelableError` | No state mutation or second abort | -| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Output plus complete required evidence | +| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Output plus complete required evidence; a requested valid result uses the fixed-name Artifact with one A2A `Part` containing `data` and `mediaType: application/json` | | Successful action with allowed bounded optional-evidence gap | `TASK_STATE_COMPLETED` | Per-dimension incomplete flag, reason, original/captured size, digest and redaction/truncation flags | -| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid and cleanup unknown unless proven | +| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid, structured result `not_produced` unless already committed, and cleanup unknown unless proven | | Retention expiry | Not found | Aggregate logically hidden before physical deletion; Artifact URL also invalid | ### Phased Delivery -1. Create the private Node 22 service package and freeze the public extension, worker protocol, profiles, fixtures, and error vocabulary. -2. Build authenticated durable A2A Task handling and fenced worker dispatch against a fake worker. -3. Build the single-execution worker lifecycle and hardened source/evidence handling against a fake adapter. -4. Add Codex, OpenCode, and Pi adapters in parallel, then compose them through the closed registry. -5. Package the services and run cross-backend, security, process, and A2A conformance before enabling a consumer. +1. Create the private Node 22 service package and freeze the public extension, portable result-schema subset, structured-result Artifact, worker protocol, profiles, fixtures, and error vocabulary. +2. Build authenticated durable A2A Task handling and fenced worker dispatch against a fake worker; startup terminalizes interrupted Tasks without attempting provider reattachment. +3. Build the supervised single-execution worker lifecycle, pre-readiness orphan reaper, and hardened source/evidence handling against a fake adapter. +4. Add the direct Codex SDK adapter and prove structured output, cancellation, provider-credential exclusion from model commands, environment isolation, and native evidence. +5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing confined tools plus the terminating result tool. +6. Package the services and run cross-backend, security, process, and A2A conformance before enabling a consumer. ### System-Wide Impact - **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. - **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. - **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. -- **Security:** New review-critical surfaces are auth, owner-key derivation, source SSRF, admission/resource quotas, setup/check policy, phase-scoped secrets, internal fences, Artifact capture/serving, and reviewed-source trust enforcement. -- **Operations:** Gateway and worker health, readiness, quotas, low-space state, structured logs, traces, tombstone backlog, lease expiry, stale event rejection, and graceful shutdown need independent signals. +- **Security:** New review-critical surfaces are auth, owner-key derivation, source SSRF, admission/resource quotas, setup/check policy, provider-credential exclusion from model tools, Pi extension/tool replacement, phase-scoped secrets, internal fences, Artifact capture/serving, and reviewed-source trust enforcement. +- **Operations:** Gateway and worker health, readiness, quotas, low-space state, structured logs, traces, tombstone backlog, lease expiry, supervisor boundary health, orphan-root quarantine/reaping, stale event rejection, and graceful shutdown need independent signals. - **Consumers:** AI Evals can build its runner provider only after the Agent Card, extension schemas, and conformance fixtures are versioned and published. ### Risks and Mitigations -- **Provider API churn:** Pin exact compatible SDK/CLI versions in the service lockfile and worker image. Gate capabilities at startup and keep captured provider fixtures versioned. +- **Provider API churn:** Pin exact compatible SDK/CLI versions in the service lockfile and worker image. Gate capabilities at startup, keep captured provider fixtures versioned, and use Promptfoo's Codex tests as characterization input rather than vendored implementation. - **False idempotency or stale settlement:** Claim Task/idempotency in one aggregate, use revision/fence compare-and-swap, sequence events, and fault-test duplicate delivery, cancellation races, restart, and late results. - **Task/store corruption:** Publish immutable blobs and generations before one manifest switch; tombstone before deletion; validate owner tuples/manifests at startup; garbage-collect unreachable generations; document the one-replica limit. - **Owner collision or path injection:** Hash a bounded canonical issuer/tenant/subject tuple, store and verify the tuple inside the owner aggregate, and use only server-generated opaque IDs in paths. -- **Orphan processes:** Combine explicit cancel, native abort, process-group termination, one-execution worker/container death, lease expiry, and quiescence proof before evidence capture. -- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral, origin-bound, and absent from repository config, process arguments, retained workspaces, logs, and errors. +- **Orphan processes and roots:** Combine explicit cancel, native abort, process-group termination, one-execution supervisor/container death, lease expiry, pre-readiness orphan reaping or quarantine, and separate termination/filesystem proof before evidence or readiness. +- **False recovery claims:** Persist Task and evidence truth only. Startup fails active Tasks, invalidates fences, and relies on lease expiry or supervisor-boundary proof instead of resuming provider sessions. +- **Structured-output drift:** Admit only the versioned closed schema subset, include its canonical digest in idempotency/provenance, pass the exact accepted schema through each adapter's supported mechanism, validate with one shared validator, publish only the fixed Artifact shape, and fail rather than publish missing or invalid JSON. +- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral, origin-bound, and absent from repository config, process arguments, model-initiated command/tool environments, retained workspaces, logs, and errors; Pi repository extensions and unrestricted built-in tools never load, and policy tools cannot access Pi config/data roots. - **Resource exhaustion:** Reserve per-owner/global gateway quota before claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. - **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. - **Evidence overclaim:** Enforce KTD12's integrity kernel and per-dimension completeness. Truncation and redaction remain independent facts. @@ -422,7 +436,7 @@ docs/src/content/docs/ - Git over hardened HTTPS and exact commit object ID covers the initial consumer. Other source transports require a later extension version or capability. - Setup and check commands are operator-controlled profile policy, not caller-supplied shell text. - Initial repositories are reviewed inside one configured mutual-trust domain. Strong hostile-code or cross-tenant execution remains unavailable until a stronger sandbox driver exists. -- Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, OpenCode CLI 1.18.x with its compatible SDK, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. +- Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. --- @@ -434,16 +448,16 @@ docs/src/content/docs/ - **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12. - **Dependencies:** None. - **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. -- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas, one public extension URI, and one private protocol version. Include attempt/fence/lease identity, monotonic event sequence, accepted dispatch, renew/cancel, bounded terminal acknowledgement, public/private state separation, and integrity-kernel rules. Canonicalize caller input plus effective profile digest for idempotency. Generate checked-in JSON Schemas and fixtures from the same source. +- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas, one public extension URI, and one private protocol version. Define the exact `allagents.result-schema/v1` keyword allowlist and bounds, canonical schema digest, four structured-result states, fixed-name Artifact containing one A2A `Part` with `data` and `mediaType: application/json`, and shared schema/result validator. Include attempt/fence/lease identity, monotonic event sequence, accepted dispatch, renew/cancel, bounded terminal acknowledgement, public/private state separation, and integrity-kernel rules. Canonicalize caller input plus effective profile and result-schema digests for idempotency. Generate checked-in JSON Schemas and fixtures from the same source. - **Execution note:** Start with fixture-driven schema, framing, and digest tests. Observe failures for unknown versions, credential-bearing sources, mutable revisions, unsafe paths, invalid public states, stale fences, oversized records, and conflicting canonical inputs before implementing schemas. - **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, and `src/core/native/types.ts` for safe error/provenance normalization. - **Test scenarios:** - - A minimal valid request with text prompt, invocation key, profile, exact commit, and deadline parses and produces a stable digest across object-key ordering. - - Changing prompt, source object ID, profile ID/digest, artifact selection, or deadline changes the digest; trace IDs and transport metadata do not. - - A source URL with credentials, a branch/tag revision, absolute subdirectory, traversal, secret value, unknown backend, or unknown extension version is rejected safely. + - A minimal valid request with text prompt, invocation key, profile, exact commit, deadline, and optional `allagents.result-schema/v1` schema parses and produces a stable digest across object-key ordering. + - Changing prompt, source object ID, profile ID/digest, result schema, artifact selection, or deadline changes the digest; trace IDs and transport metadata do not. + - Unsupported keywords, remote references, non-object roots, object schemas that omit `additionalProperties: false`, undeclared optional properties, format-dependent validation, or schemas over byte/depth/property/enum limits are rejected before Task creation; every accepted schema validates identically in admission, worker, Codex forwarding, and Pi tool generation. - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. - Worker fixtures reject missing/mismatched attempt IDs, lease epochs, profile digests, event sequence, bounds, and terminal acknowledgements. - - Completed, failed, canceled, and rejected results validate only with the integrity kernel; optional usage/native evidence gaps require explicit completeness reasons. + - `not_requested`, `not_produced`, `valid`, and `invalid` cover success, pre-output failure, cancellation, missing output, and invalid output without replacing the primary Task classification; only `valid` permits one `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data`, `mediaType: application/json`, and a matching schema digest in Artifact metadata. - File evidence accepts create/edit/delete/rename and rejects unsafe paths, duplicate identities, oversized inline content, and inconsistent before/after forms. - **Verification:** Generated schemas are stable, public/private fixtures round-trip, digest vectors are cross-platform deterministic, and the private client/server fixture suite agrees before gateway or worker implementation. @@ -453,7 +467,7 @@ docs/src/content/docs/ - **Requirements:** R4-R9, R13-R14, R17-R18, R22; AE2-AE4, AE6, AE8, AE10-AE13; KTD1, KTD3-KTD4, KTD12. - **Dependencies:** U1. - **Files:** `packages/execution-service/src/gateway/config.ts`, `packages/execution-service/src/gateway/auth.ts`, `packages/execution-service/src/gateway/store/gateway-repository.ts`, `packages/execution-service/src/gateway/store/file-gateway-repository.ts`, `packages/execution-service/tests/unit/gateway/auth.test.ts`, `packages/execution-service/tests/unit/gateway/file-gateway-repository.test.ts`. -- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Commit claim plus submitted Task in one manifest generation; publish immutable Artifact blobs before terminal manifest switch; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover and garbage-collect unreachable generations on startup. Reserve owner/global quotas before claims. Verify OIDC JWTs and constant-time static tokens before all repository access. +- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Commit claim plus submitted Task in one manifest generation; publish immutable Artifact blobs before atomically switching the manifest to a new generation; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover and garbage-collect unreachable generations on startup. Startup recovery is a readiness barrier: invalidate every old fence and terminalize every nonterminal Task before admission, subscriptions, dispatch, or lease renewal begin. Reserve owner/global quotas before claims. Verify OIDC JWTs and constant-time static tokens before all repository access. - **Execution note:** Implement concurrent-claim, transition-race, and crash-publication tests before request handling. Inject faults between blob, generation, manifest, tombstone, and cleanup operations. - **Patterns to follow:** `src/core/marketplace.ts` and `src/core/profile/files.ts` for atomic publication/recovery, `src/core/mcp-http-stdio-proxy.ts` for private files and loopback safety, and the official A2A `TaskStore` owner-scoping contract. - **Test scenarios:** @@ -462,7 +476,7 @@ docs/src/content/docs/ - Hostile/ambiguous issuer, tenant, subject, invocation key, Task ID, Artifact name, Unicode, case, delimiter, traversal, and Windows-reserved values cannot collide or become paths. - All standard list filters, `historyLength`, page size 1-100, omitted Artifacts, ordering, total size, and always-present next token match A2A semantics. Tokens are owner/query-bound and reject malformed, swapped, or stale filters. - Covers AE12. Terminal compare-and-swap wins once; stale fence, duplicate, and out-of-order updates cannot mutate the Task. - - Restart fails nonterminal Tasks once, invalidates fences, preserves terminal Tasks, and records cleanup unknown unless proven. + - Restart, including repeated failure during startup recovery, completes the recovery barrier before serving: it fails every nonterminal Task once, invalidates fences, never renews an old lease or requests provider reattachment/replay, preserves terminal Tasks, and records cleanup unknown unless proven. - Covers AE13. Exact expiry tombstones the aggregate before cleanup; failed deletion never restores visibility; same-key replay before expiry returns the old Task and after expiry creates a new Task. - A crash between every aggregate publication step leaves either the prior or next valid manifest, never claim-without-Task or Task-with-missing-Artifact state. - OIDC rejects wrong issuer, audience, signature, expiry, scope, tenant, and subject; static tokens and internal capabilities never appear in logs/errors. @@ -495,117 +509,104 @@ docs/src/content/docs/ ### U4. Worker protocol and safe workspace lifecycle -- **Goal:** Implement the single-execution worker, hardened immutable Git acquisition, profile enforcement, leases, isolated backend roots, resource controls, race-resistant evidence, termination, and cleanup independent of any provider. +- **Goal:** Implement the supervised single-execution worker, hardened immutable Git acquisition, profile enforcement, leases, isolated backend roots, resource controls, race-resistant evidence, termination, and cleanup independent of any provider. - **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14. - **Dependencies:** U1. -- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`. -- **Approach:** Authenticate and fence the private protocol, reserve the one execution before workspace creation, validate profile/deployment capability, and emit sequenced NDJSON. Acquire source under KTD13. Create separate workspace and backend config/data roots with a scrubbed phase-specific environment. Run setup, baseline, adapter, and checks under enforced budgets. Stop and verify the process group before descriptor-based regular-file evidence staging, then clean in `finally`. Lease expiry self-cancels. +- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`. +- **Approach:** Authenticate and fence the private protocol, reserve the one execution before workspace creation, validate profile/deployment capability, and emit sequenced NDJSON. Run the worker server inside a deployment-approved supervisor boundary that kills all invocation descendants if the server exits. Before readiness, inspect invocation manifests, require supervisor proof that prior descendants are dead, and delete or quarantine orphaned roots; an unprovable root blocks reuse and reports degraded readiness. Acquire source under KTD13. Create separate workspace and backend config/data roots with a scrubbed phase-specific environment. Run setup, baseline, adapter, and checks under enforced budgets. Stop and verify the process group before descriptor-based regular-file evidence staging, then clean in `finally`. Lease expiry self-cancels. - **Execution note:** Characterize every phase with a fake adapter, malicious fixtures, and disposable Git servers before real providers. Fault-inject dispatch acknowledgement, events, leases, acquisition, processes, evidence publication, and cleanup. - **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, and `tests/helpers/env.ts` for isolated state. - **Test scenarios:** - Covers AE5. Exact object ID verifies; wrong/missing object, disallowed URL/host/address/port, credential-bearing URL, redirect, DNS rebinding, unsafe subdirectory, and fetch failure stop before adapter invocation. - Repositories with LFS configuration/pointers, submodules, hooks, filters, alternates, proxy/helper config, or non-HTTPS secondary protocols cause no secondary connection or helper execution. - Source credentials leave no repository config, process argument, child phase environment, log, error, evidence, or retained workspace trace. - - Setup changes establish the baseline; setup and checks receive no provider/control secrets; every backend gets disjoint invocation config/data roots with ambient selectors removed. + - Setup changes establish the baseline; setup, checks, and model-initiated tools receive no provider/control secrets; every backend gets disjoint invocation config/data roots with ambient selectors removed. - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. - Covers AE14. Concurrency above one and hostile/cross-tenant trust claims are rejected; worker control credentials are absent from child environment and configured filesystem roots. - Source pack/tree/file/inode/path/sparse-file/disk limits and setup/provider/check CPU, memory, PID, network, phase-time, and workspace limits stop only the invocation and preserve worker health. - Covers AE9. Known permissions receive one-invocation decisions; prompt-required profiles fail startup; unknown permission types fail the adapter. - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps; capture I/O or malformed result that breaks the kernel fails the Task. - Background swap attacks, links, mount crossings, FIFOs/devices/sockets, unstable files, and tampering between worker staging and gateway publication never expose external bytes or partial Artifacts. - - A worker crash before/after provider spawn reports cleanup complete only when the process/container boundary proves descendant death. -- **Verification:** A built worker mutates a disposable exact-SHA repository through the fake adapter and proves fenced dispatch, source hardening, phase isolation, budgets, quiescence, evidence integrity, and cleanup from its emitted result alone. + - Covers AE12. SIGKILL the worker before and after provider spawn with live descendants and a persistent invocation root; the supervisor proves descendant death, the replacement reaper deletes or quarantines the root before readiness, and the gateway retains one failed Task with separate termination and cleanup outcomes. +- **Verification:** A built supervised worker mutates a disposable exact-SHA repository through the fake adapter and proves fenced dispatch, source hardening, phase isolation, budgets, quiescence, evidence integrity, worker-crash containment, orphan-root handling, and cleanup from its emitted result plus supervisor proof. ### U5. Codex backend adapter -- **Goal:** Run Codex through its supported TypeScript SDK while preserving structured progress, output, usage, file-change evidence, cancellation, and runtime identity. +- **Goal:** Run Codex directly through its supported TypeScript SDK while preserving structured progress, validated output, usage, file-change evidence, cancellation, and runtime identity. - **Requirements:** R10-R22; AE1, AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. - **Dependencies:** U4. - **Files:** `packages/execution-service/src/worker/adapters/codex.ts`, `packages/execution-service/tests/unit/worker/adapters/codex.test.ts`, `packages/execution-service/tests/fixtures/execution/codex-events.jsonl`. -- **Approach:** Construct a fresh SDK thread in the invocation workspace with an isolated `CODEX_HOME` and scrubbed environment. Apply model, sandbox, network, approval, and writable-root settings only from the profile. Consume `runStreamed()` and pass an AbortSignal. Normalize agent messages, items, usage, failures, and file-change events while preserving the bounded native stream. -- **Execution note:** Drive the SDK through its executable override with a fixture Codex process before any credentialed smoke test. -- **Patterns to follow:** `src/core/profile/adapters/codex.ts` for root/environment isolation, `src/core/native/codex.ts` for version checks, and the SDK's `runStreamed`/AbortSignal contract. +- **Approach:** Depend directly on a pinned `@openai/codex-sdk` and fail worker readiness when it is unavailable or incompatible. Construct one fresh SDK thread in the invocation workspace with an isolated `CODEX_HOME` and a minimal allowlisted environment. Apply model, sandbox, network, approval, working-directory, writable-root, and a pinned `shell_environment_policy` only from the profile. The Codex runtime may receive its scoped provider credential, but model-initiated shell commands receive only named non-secret variables and never provider or worker-control credentials. Consume `runStreamed()`, pass the invocation `AbortSignal`, and pass the exact accepted result schema as per-turn `outputSchema`. Parse and validate the final JSON with the shared worker validator before publishing the canonical result Artifact. Normalize agent messages, items, usage, failures, and file-change events while preserving the bounded native stream. Never call `resumeThread`, pool threads, or reuse provider sessions after interruption. +- **Execution note:** Wrap the SDK behind an injectable factory and drive it through fixture events and its executable override before any credentialed smoke test. Use Promptfoo's provider and tests to enumerate observable edge cases, not as copied code or a runtime dependency. +- **Patterns to follow:** `src/core/profile/adapters/codex.ts` for root/environment isolation, `src/core/native/codex.ts` for version checks, the SDK's `startThread`/`runStreamed`/`AbortSignal`/`outputSchema` and shell-environment policy contracts, and Promptfoo's Codex provider tests for characterization of option forwarding, environment isolation, cancellation, structured output, and cleanup. - **Test scenarios:** - A successful stream exposes thread ID, progress, final response, token usage, native file-change items, and terminal completion. + - A structured request forwards the exact accepted schema to `outputSchema`; valid JSON becomes the fixed-name Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`, while missing, malformed, or schema-invalid output fails with a typed structured-result error. - Empty final response, turn failure, malformed JSONL, non-zero exit, unavailable runtime, and usage omission map to typed result/completeness fields. - - Covers AE6/AE12. Cancellation aborts the SDK process once; completion after cancel or stale fence cannot alter the selected terminal outcome. - - Profile sandbox, network, model, approval, working directory, and environment settings reach the SDK; caller input cannot override them. - - Two sequential invocations have disjoint `CODEX_HOME`, thread/session state, and writable roots; ambient selectors are removed. + - Covers AE6/AE12. A pre-aborted signal prevents start; in-flight cancellation aborts the SDK once; worker escalation proves descendant termination; completion after cancel or stale fence cannot alter the selected terminal outcome. + - Profile sandbox, network, model, approval, working directory, shell-environment policy, and environment settings reach the SDK; caller input cannot override them or supply raw Codex config. + - The Codex runtime receives only its scoped credential and minimal runtime environment. A model-initiated command that attempts to print provider/control credential names observes no values, and output, errors, and retained evidence contain none. + - Two sequential invocations create fresh threads with disjoint `CODEX_HOME`, session state, and writable roots; no resume or thread-persistence API is called. - Native diffs and shared Git evidence coexist without claiming identical attribution. -- **Verification:** Fixture-driven tests cover every supported event and failure shape, followed by an isolated credentialed repository smoke test when Codex credentials are available. +- **Verification:** Fixture-driven tests cover every supported event and failure shape, SDK option/schema/signal forwarding, environment isolation, fresh-thread behavior, and cleanup escalation, followed by an isolated credentialed repository smoke test when Codex credentials are available. -### U6. OpenCode backend adapter +### U6. Pi backend adapter -- **Goal:** Run OpenCode through its typed SDK and worker-owned loopback server while preserving session progress, output, usage/cost, diffs, permissions, cancellation, and disposal. +- **Goal:** Run Pi through strict RPC mode while preserving settled completion, schema-backed terminal output, usage/cost, tool progress, cancellation, and process cleanup. - **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. -- **Dependencies:** U4. -- **Files:** `packages/execution-service/src/worker/adapters/opencode.ts`, `packages/execution-service/tests/unit/worker/adapters/opencode.test.ts`, `packages/execution-service/tests/fixtures/execution/opencode-events.jsonl`. -- **Approach:** Start one loopback instance per invocation with isolated `OPENCODE_CONFIG`, `OPENCODE_CONFIG_DIR`, data/cache roots, scrubbed environment, profile configuration, and AbortSignal. Subscribe before prompting, create one session, resolve permission events from policy, collect message/session events and session diff, abort on cancellation, then delete the session and close the server in `finally`. Prevent the agent subprocess from reaching the worker control listener under the declared trust topology. -- **Execution note:** Inject SDK/server factories so protocol fixtures prove ordering and teardown without downloading or authenticating a real runtime. -- **Patterns to follow:** `src/core/profile/adapters/opencode.ts` for configuration/environment isolation and OpenCode's `createOpencode`, event subscription, session prompt/diff/abort APIs. -- **Test scenarios:** - - Successful execution collects text parts, assistant tokens/cost, session ID, events, and session diff before disposal. - - Subscription starts before prompt, ignores other session IDs, and finishes only after the target session becomes idle or errors. - - Covers AE6/AE12. Cancellation calls session abort once; late idle/completion cannot overwrite cancellation; server teardown remains idempotent. - - Covers AE9. Known permission events receive invocation-scoped `once`, `always`, or `reject` according to profile; `always` does not survive disposal and unknown types fail the adapter. - - Provider auth error, API error, aborted message, server-start timeout, SSE disconnect, and malformed SDK response map to typed failures. - - Sequential invocations have disjoint config/data/session roots; caller input cannot enable sharing, alter bind, select another project, or override provider/model/tools. -- **Verification:** Fixture tests prove session scoping, permission lifetime, cancellation races, and disposal, followed by an isolated credentialed repository smoke test when OpenCode credentials are available. - -### U7. Pi backend adapter - -- **Goal:** Run Pi through strict RPC mode while preserving settled completion, output, usage/cost, tool progress, cancellation, and process cleanup. -- **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. -- **Dependencies:** U4. -- **Files:** `packages/execution-service/src/worker/adapters/pi.ts`, `packages/execution-service/src/worker/adapters/pi-rpc.ts`, `packages/execution-service/tests/unit/worker/adapters/pi.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-rpc.test.ts`, `packages/execution-service/tests/fixtures/execution/pi-events.jsonl`. -- **Approach:** Spawn a supported Pi 0.85.x runtime with `--mode rpc --no-session`, an invocation-local `PI_CODING_AGENT_DIR`, profile model/provider, and scrubbed environment. Implement an LF-only JSONL parser rather than Node `readline`. Correlate responses, wait for `agent_settled`, read messages/stats, send RPC abort, and escalate process-group termination after the grace period. -- **Execution note:** Build parser and state-machine tests from captured RPC fixtures before process integration. -- **Patterns to follow:** `src/core/native/pi.ts` for version/trust checks, `src/core/profile/adapters/pi.ts` for root isolation, and the official Pi RPC framing/cancellation contract. +- **Dependencies:** U4, U5. +- **Files:** `packages/execution-service/src/worker/adapters/pi.ts`, `packages/execution-service/src/worker/adapters/pi-rpc.ts`, `packages/execution-service/src/worker/adapters/pi-policy-extension.ts`, `packages/execution-service/tests/unit/worker/adapters/pi.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-rpc.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-policy-extension.test.ts`, `packages/execution-service/tests/fixtures/execution/pi-events.jsonl`. +- **Approach:** Spawn a supported Pi 0.85.x runtime with `--mode rpc --no-session --no-extensions --no-builtin-tools`, an invocation-local `PI_CODING_AGENT_DIR`, profile model/provider, and scrubbed environment. Materialize the scoped provider credential only in Pi's supported invocation-local credential store with private permissions, not in the process environment. Explicitly load one worker-owned policy extension from outside the repository and verify the loaded extension/tool inventory before accepting work. The extension registers workspace-confined read/write/edit/search and sandboxed command tools plus the terminating result tool; command children receive only the phase allowlist and cannot access the Pi config/data roots. Implement an LF-only JSONL parser rather than Node `readline`. Correlate responses, wait for `agent_settled`, read messages/stats, send RPC abort, and escalate process-group termination after the grace period. For a structured request, generate the terminating tool from the exact accepted schema. The first observed tool call atomically claims the result candidate before validation: valid arguments produce the canonical Artifact; malformed or schema-invalid arguments fail the Task; later calls cannot replace the candidate. A prior cancel, deadline, or stale fence suppresses the call. Settling without a call fails as missing structured output. +- **Execution note:** Build parser, extension/tool-inventory, terminating-tool, policy-tool, and state-machine tests from captured RPC fixtures before process integration. Reuse the contract established by U5 rather than adding Pi-shaped public fields. +- **Patterns to follow:** `src/core/native/pi.ts` for version/trust checks, `src/core/profile/adapters/pi.ts` for root isolation, and the official Pi RPC framing, `--no-extensions` plus explicit `--extension`, `--no-builtin-tools`, custom-tool, credential-store, and cancellation contracts. - **Test scenarios:** - Successful prompt acceptance streams message/tool events, stops on `agent_settled`, retrieves final messages/stats, and reports session ID, usage, and cost. + - A structured request exposes only the invocation-scoped terminating tool in addition to the policy tools. The first observed call claims the candidate; valid arguments produce the fixed-name Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; an invalid first call fails without replacement; a later duplicate cannot replace the result; a cancel/deadline/fence that wins first suppresses the call; and settled completion without a call fails as missing output. - LF framing preserves `U+2028`/`U+2029` inside JSON strings, accepts CRLF by stripping trailing CR, handles partial/multiple chunks, and rejects oversized/malformed records. - - Covers AE6/AE12. Cancellation sends RPC abort once, waits for idle, then terminates the process group only after grace; late settled events cannot overwrite the terminal fence. + - Covers AE6/AE12. Cancellation sends RPC abort once, waits for idle, then terminates the process group only after grace; late settled or terminating-tool events cannot overwrite the terminal fence. - Prompt rejection, agent error, aborted stop reason, retry/compaction sequence, premature exit, stderr overflow, and stats failure map truthfully. - - Sequential invocations have disjoint `PI_CODING_AGENT_DIR` and session state; caller input cannot send extension commands, steering/follow-up, arbitrary RPC commands, or override provider/model. -- **Verification:** Fixture and fake-process tests prove framing, correlation, settled completion, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when Pi credentials are available. + - Sequential invocations have disjoint `PI_CODING_AGENT_DIR`, tool registration, and session state; the provider credential exists only in the private invocation-local store; policy tools cannot read that root and their command children observe no provider/control credential; repository `.pi/extensions` and unrestricted built-in tools do not load; and caller input cannot send extension commands, steering/follow-up, arbitrary RPC commands, or override provider/model. +- **Verification:** Fixture and fake-process tests prove framing, correlation, deterministic terminating-tool selection, shared validation, exact extension/tool inventory, workspace confinement, command-environment and credential isolation, settled completion, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when Pi credentials are available. -### U8. Production registry, service packaging, and observability +### U7. Production registry, service packaging, and observability -- **Goal:** Compose exactly three production adapters and package independently runnable gateway and worker services with safe startup, health, shutdown, tracing, and reproducible containers. +- **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with safe startup, health, shutdown, tracing, and reproducible containers. - **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD11-KTD15. -- **Dependencies:** U3-U7. -- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. -- **Approach:** Register only Codex, OpenCode, and Pi through an explicit capability/availability map. Add gateway and worker entrypoints inside the private Node 22 workspace instead of the Node 18 CLI package. Validate config, store, workers, runtime pins, trust, quotas, and resource controls before readiness. Propagate `traceparent` and instrument every phase. Build a minimal gateway image with no provider runtimes and a one-execution worker image with exact runtime versions. +- **Dependencies:** U3-U6. +- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. +- **Approach:** Register only Codex and Pi through an explicit capability/availability map. Add gateway and supervised worker entrypoints inside the private Node 22 workspace instead of the Node 18 CLI package. Pin both runtimes as direct service dependencies, validate config, store, workers, runtime availability, supervisor boundary, orphan roots, trust, quotas, and resource controls before readiness, and never discover a missing provider only after Task acceptance. Propagate `traceparent` and instrument every phase. Build a minimal gateway image with no provider runtimes and a one-execution worker image whose init/runtime kills the complete execution boundary when the worker server exits. - **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. - **Patterns to follow:** `src/core/profile/adapters/registry.ts` for explicit adapter composition, root package scripts for workspace delegation, `src/core/mcp-http-stdio-proxy.ts` for server lifecycle, `.github/workflows/ci.yml` for quality gates, and `.github/workflows/publish.yml` for immutable releases. - **Test scenarios:** - - Registry exposes exactly Codex, OpenCode, and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects unknown backend IDs before workspace creation. - - Gateway and worker start from built service outputs, become ready only after dependencies pass, and stop gracefully on SIGTERM. + - Registry exposes exactly Codex and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects OpenCode or unknown backend IDs before workspace creation. + - Gateway and supervised worker start from built service outputs, become ready only after dependencies and orphan recovery pass, and stop gracefully on SIGTERM. - Gateway readiness fails for malformed auth, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. - - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable resource enforcement, or unsupported backend runtime. + - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable supervisor/resource enforcement, unproved or unrecoverable orphan roots, or unavailable/incompatible Codex or Pi runtime. + - Killing the worker server while an adapter child and invocation root exist makes the supervisor kill the boundary; replacement readiness waits for root deletion or quarantine and never reuses it. - Trace context enters through A2A, crosses the private call, and correlates result identities; exporter failure cannot change Task status. - - Gateway image contains no Codex, OpenCode, Pi, Git workspace, or provider credential material. - - Worker image pins all runtimes, confines one workspace/config root, enforces deployment limits, and completes fake-provider health smoke tests. + - Gateway image contains no Codex, Pi, Git workspace, or provider credential material. + - Worker image pins both runtimes, confines one workspace/config root, excludes credentials from model tools, disables repository Pi extensions and unrestricted built-in tools, enforces deployment limits, and completes fake-provider health smoke tests. - Installing the root npm package on Node 18 does not load service dependencies; the private service workspace and containers enforce Node 22.19+. -- **Verification:** The registry dispatches every adapter through the same worker contract; built services and images pass lifecycle/security smoke tests; CI and publication bind immutable image tags to the release commit. +- **Verification:** The registry dispatches both adapters through the same worker contract; built services and images pass lifecycle/security smoke tests; CI and publication bind immutable image tags to the release commit. -### U9. Cross-backend conformance, documentation, and release evidence +### U8. Cross-backend conformance, documentation, and release evidence - **Goal:** Prove the public contract and operational workflow end to end and document deployment without leaking backend details into callers. - **Requirements:** R1-R22; F1-F5; AE1-AE14. -- **Dependencies:** U1-U8. +- **Dependencies:** U1-U7. - **Files:** `packages/execution-service/tests/e2e/execution-gateway.test.ts`, `packages/execution-service/tests/fixtures/execution/conformance-cases.ts`, `examples/gateway/gateway.yaml`, `examples/gateway/worker.yaml`, `docs/src/content/docs/guides/execution-gateway.mdx`, `docs/src/content/docs/reference/execution-gateway-configuration.mdx`, `README.md`, `CHANGELOG.md`. -- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases. Exercise gateway and worker as separate processes. Document extension/worker protocols, profiles, auth, trust boundary, storage/HA limits, source hardening, quotas, runtime requirements, cancellation races, evidence integrity, Artifact access, retention, observability, and troubleshooting. +- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases. Exercise gateway and supervised worker as separate processes. Document extension/worker protocols, the portable result-schema subset, fixed structured-result Artifact, four result states, profiles, auth, trust boundary, storage/HA limits, explicit lack of execution resume, worker-crash supervision/orphan recovery, source hardening, quotas, runtime requirements, cancellation races, evidence integrity, Artifact access, retention, observability, and troubleshooting. - **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. - **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, and Starlight guide/reference organization under `docs/src/content/docs/docs/`. - **Test scenarios:** - Covers AE1-AE14 through built services with a fake backend and official A2A client. - - The same mutation fixture passes through Codex, OpenCode, and Pi event fixtures and produces contract-equivalent normalized evidence. + - The same accepted schema, valid result, invalid result, missing result, and pre-output failure fixtures pass through Codex and Pi adapters with identical validator decisions, integrity states, and Artifact presence/shape while retaining distinct native evidence. - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; the one-execution worker serializes admitted work. - - Gateway restart, stream reconnect, lost dispatch acknowledgement, duplicate/out-of-order events, worker crash, lease expiry, cancellation race, provider failure, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome. - - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, control-endpoint probing, and secret-exfiltration fixtures are blocked within the documented reviewed-source boundary. + - Gateway restart, stream reconnect, lost dispatch acknowledgement, duplicate/out-of-order events, worker crash, lease expiry, cancellation race, provider failure, invalid structured output, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome without provider reattachment or replay. + - A worker SIGKILL with live descendants and an invocation root proves supervisor termination and pre-readiness deletion/quarantine; repeated gateway crashes cannot serve until startup terminalization completes. + - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, control-endpoint probing, provider-credential echo/read attempts, repository Pi extensions, unrestricted Pi built-in tools, and secret-exfiltration fixtures are blocked within the documented reviewed-source boundary. - Examples validate with production schemas and reference secrets only through environment variable names. - - Docs state one gateway replica, one execution per worker, reviewed mutual-trust sources, Node/runtime floors, and no hostile-code isolation claim. + - Docs state one gateway replica, one supervised execution per worker, reviewed mutual-trust sources, Node/runtime floors, ephemeral provider sessions, and no hostile-code isolation claim. - Opt-in real-provider smoke tests record backend/runtime versions and skip only when the named credential/runtime prerequisite is absent. - **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor, the full suites and docs pass, the official A2A client exercises every advertised operation, and release evidence records each available real backend plus explicit skipped prerequisites. @@ -616,15 +617,15 @@ docs/src/content/docs/ | Gate | Applies to | Required evidence | |---|---|---| | Contract generation | U1 | Public extension and private worker schema generation report no drift; positive and negative fixtures pass. | -| Focused unit tests | U1-U8 | Active-unit tests pass with fault injection, state races, limits, cancellation, and cleanup. | -| Gateway/worker integration | U3-U4, U8-U9 | Built processes agree on fenced dispatch, sequencing, leases, Task persistence, Artifacts, shutdown, and cleanup. | -| Backend conformance | U5-U9 | One shared suite passes against Codex, OpenCode, and Pi adapters with fixture runtimes. | -| Credentialed provider smoke | U5-U7, U9 | Each available provider mutates a disposable exact-SHA repository; missing credentials/runtime are recorded as skipped prerequisites, never passing coverage. | -| A2A interoperability | U3, U9 | Official `@a2a-js/sdk` client passes immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, replay, cancel races, expiry, and owner isolation. | -| Security and abuse | U2-U4, U8-U9 | Malicious identity/source/artifact/resource fixtures prove auth-before-lookup, opaque owner keys, Git SSRF controls, phase-scoped secrets, quotas, quiescence, race-resistant capture, and trust-topology rejection. | -| Service packaging | U8-U9 | Root Node 18 install, private Node 22 build, gateway/worker smoke, and both container builds pass. | +| Focused unit tests | U1-U7 | Active-unit tests pass with fault injection, state races, limits, cancellation, and cleanup. | +| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on fenced dispatch, sequencing, leases, Task persistence, Artifacts, shutdown, and cleanup. | +| Backend conformance | U5-U8 | One shared suite passes against Codex and Pi adapters with fixture runtimes, including identical acceptance and validation of the versioned result-schema subset, four result states, and fixed Artifact shape. | +| Credentialed provider smoke | U5-U6, U8 | Each available provider mutates a disposable exact-SHA repository; missing credentials/runtime are recorded as skipped prerequisites, never passing coverage. | +| A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, replay, cancel races, expiry, and owner isolation. | +| Security and abuse | U2-U4, U7-U8 | Malicious identity/source/artifact/resource fixtures prove auth-before-lookup, opaque owner keys, Git SSRF controls, phase-scoped secrets, provider-credential exclusion from model tools, disabled repository Pi extensions/built-ins, policy-tool confinement, quotas, quiescence, race-resistant capture, and trust-topology rejection. | +| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, worker-crash containment/orphan recovery, and both container builds pass. | | Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | -| Documentation | U9 | `bun run docs:build` passes and examples validate against current schemas. | +| Documentation | U8 | `bun run docs:build` passes and examples validate against current schemas. | The authoritative behavioral proof is the built-process E2E path with the official A2A client and a separately started worker. Unit tests alone do not prove protocol, durable aggregation, process isolation, fencing, cancellation, or cleanup integration. @@ -636,23 +637,22 @@ The authoritative behavioral proof is the built-process E2E path with the offici - Every R1-R22 requirement is implemented or explicitly shown in a passing conformance scenario. - Public Agent Card/extension and private worker schemas are stable, generated from one source, and consumable without importing root AllAgents CLI modules. -- Codex, OpenCode, and Pi pass the same backend conformance suite and preserve bounded native evidence through the closed registry. -- Gateway and worker run as separate Node 22 processes/images; the Node 18 root CLI does not import service dependencies, and the gateway has no provider runtime or writable repository. -- Authentication precedes lookup, quota precedes Task creation, aggregate commits cannot split claims/Tasks/Artifacts, and terminal fences survive races and restart. -- Cancellation/deadlines reach one native abort, process termination, quiescence, evidence, and cleanup for all three backends. -- Source hardening, phase-scoped secrets, one-execution trust policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end. +- Codex and Pi pass the same backend conformance suite, accept the same versioned result-schema subset, validate with the same shared validator, publish the same fixed structured-result Artifact shape and result states, and preserve bounded native evidence through the closed registry. +- Gateway and supervised worker run as separate Node 22 processes/images; the Node 18 root CLI does not import service dependencies, and the gateway has no provider runtime or writable repository. +- Authentication precedes lookup, quota precedes Task creation, aggregate commits cannot split claims/Tasks/Artifacts, startup recovery completes before serving, and terminal fences survive races and restart without claiming provider-session recovery. +- Cancellation/deadlines reach one native abort, process termination, quiescence, evidence, and cleanup for both backends; worker-process death triggers supervisor termination and pre-readiness orphan deletion or quarantine. +- Source hardening, phase-scoped secrets, provider-credential exclusion from model tools, disabled repository Pi extensions/unrestricted built-ins, Pi policy-tool confinement, one-execution trust policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end. - Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. -- Public documentation states supported topology, configuration, security boundary, storage/HA limitation, runtime pins, and deferred capabilities. +- Public documentation states supported topology, configuration, security boundary, storage/HA limitation, runtime pins, structured-result contract, worker-crash recovery, and deferred capabilities. - Abandoned experiments, unused adapters, compatibility shims, generated scratch files, retained test workspaces, and stale documentation are removed. ### Per unit -- U1: Public/worker schemas, digest vectors, state/fence rules, typed failures, and fixtures are generated and stable. -- U2: Auth, opaque owner isolation, aggregate idempotency, CAS settlement, pagination, restart, quotas, Artifact access, tombstones, and cleanup pass fault injection. +- U1: Public/worker schemas, result-schema subset, structured-result Artifact/states, digest vectors, fence rules, typed failures, and fixtures are generated and stable. +- U2: Auth, opaque owner isolation, aggregate idempotency, CAS settlement, pagination, startup recovery barrier, quotas, Artifact access, tombstones, and cleanup pass fault injection. - U3: Every advertised A2A operation agrees across stream and lookup while replay, fencing, and cancellation races preserve one Task. -- U4: Worker dispatch/source/setup/action/check/quiescence/evidence/cleanup lifecycle passes malicious and faulted disposable-repository scenarios. -- U5: Codex streaming, usage, native evidence, isolated roots, cancellation, and failure mapping pass adapter and applicable smoke verification. -- U6: OpenCode session/event/diff/permission isolation, abort, and disposal pass adapter and applicable smoke verification. -- U7: Pi strict JSONL framing, settled completion, stats, isolated roots, abort, and process cleanup pass adapter and applicable smoke verification. -- U8: Closed registry, Node-version separation, readiness, tracing, graceful shutdown, containers, and release artifacts work from built outputs. -- U9: Cross-backend E2E, A2A interoperability, abuse cases, examples, operator docs, changelog, and release evidence are complete. +- U4: Supervision, orphan recovery, worker dispatch/source/setup/action/check/quiescence/evidence/cleanup lifecycle pass malicious, crashed, and faulted disposable-repository scenarios. +- U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, usage, native evidence, minimal environment, model-tool credential exclusion, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. +- U6: Pi strict JSONL framing, deterministic terminating-tool output, exact policy-extension/tool inventory, disabled repository extensions and built-ins, credential-store and workspace confinement, settled completion, stats, isolated roots, abort, and process cleanup pass adapter and applicable smoke verification. +- U7: Closed registry, Node-version separation, runtime readiness, tracing, graceful shutdown, containers, and release artifacts work from built outputs. +- U8: Cross-backend E2E, A2A interoperability, abuse cases, examples, operator docs, changelog, and release evidence are complete. From dcab69364c00060afe3bb6c3a9a2c80a5b82c11d Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Fri, 18 Sep 2026 17:57:05 +1000 Subject: [PATCH 05/44] docs(architecture): harden execution gateway contract --- ...-agent-execution-through-an-a2a-gateway.md | 19 +- ...0837-feat-coding-execution-gateway-plan.md | 384 ++++++++++-------- .../agent-host-protocol-decision-inputs.md | 6 +- 3 files changed, 227 insertions(+), 182 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index 1ed60865..599d72e8 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -207,12 +207,23 @@ will not be adopted. Gateway calls propagate [W3C Trace Context](https://www.w3.org/TR/trace-context/) across HTTP and process -boundaries. AllAgents uses OpenTelemetry and OTLP for operational telemetry. +boundaries. AllAgents uses OpenTelemetry and OTLP for metadata-only operational +telemetry by default. An explicit allowlist limits structured logs and spans to +non-content operational metadata. Prompts and model outputs, tool arguments and +results, file bodies and source fragments, and secret-bearing attributes are +prohibited before export. A bounded filtering and redaction step must run before +any structured log or span processor so disallowed content cannot enter the +telemetry pipeline. + AllAgents-managed agent, model, and tool spans use [OpenInference](https://arize-ai.github.io/openinference/) semantic conventions -where corresponding attributes exist; useful backend-native attributes may be -retained alongside them. Consumer-owned evaluator spans may join the propagated -trace without becoming gateway-owned. +only for attributes that pass this allowlist. Backend-native attributes must +pass the same allowlist. Owner correlation is limited to an opaque identifier +appropriate for the telemetry operators' access; it does not expose caller +identity or grant access to a Task or Artifact. Telemetry access and retention +are governed separately from Task and Artifact access and retention. +Consumer-owned evaluator spans may join the propagated trace without becoming +gateway-owned. These standards are complementary: diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 60987efd..d373b358 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -55,14 +55,14 @@ The two initial runtimes expose different programmatic contracts. Codex provides - R1. The gateway implements A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, `GetTask`, `ListTasks`, and `CancelTask`; it implements streaming send and task subscription when the card advertises streaming. - R2. Every valid new request returns exactly one addressable Task. Direct-Message completion and follow-up messages to an existing Task are unsupported. Non-streaming send honors A2A `returnImmediately`; streaming always emits the durable Task first. -- R3. Every request and terminal Task uses one required, versioned AllAgents coding-execution extension URI. Unsupported required extension versions fail without fallback. +- R3. The public extension URI is `https://allagents.dev/a2a/extensions/coding-execution/v1`. The Agent Card advertises it as required; HTTP clients opt in with `A2A-Extensions`; each request sets `Message.extensions` to include the URI and puts the schema-defined request only at `Message.metadata[uri]`. Every terminal Task contains exactly one fixed-name `allagents.execution-integrity` Artifact whose `extensions` includes the URI and whose single `Part` contains the schema-defined integrity envelope in `data` with `mediaType: application/json`. The Task uses only the standard A2A fields and never adds `Task.extensions`. Unsupported or missing required extension versions fail without fallback. - R4. Terminal output and execution evidence are retrievable as Task Artifacts for the configured retention window even when the original stream disconnects. Active subscription emits the current Task snapshot then future events without promising replay of missed progress; terminal subscription returns the standard unsupported-operation error and callers use `GetTask`. **Caller identity, Task identity, and retention** -- R5. Every protocol operation authenticates the caller and scopes Task lookup, listing, subscription, cancellation, and artifact retrieval to that caller's tenant and principal before storage access can reveal resource existence. -- R6. Authentication, required-extension validation, request validation, source/profile authorization, quota admission, and deadline validation complete before Task creation. A caller-scoped invocation key, effective profile, authenticated owner, and canonical request digest then bind atomically to one Task; identical replay returns that Task and conflicting reuse is rejected without dispatch. -- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Task state and terminal Artifact metadata survive gateway restart. Every nonterminal Task present at startup settles failed once, its old attempt fence is invalidated, and stale worker events cannot overwrite it; the initial service never resumes or automatically replays interrupted provider work. +- R5. Every protocol operation authenticates the caller and scopes Task lookup, listing, subscription, cancellation, and artifact retrieval to that caller's tenant and principal before storage access can reveal resource existence. Production public ingress reaches the gateway through TLS terminated at the configured named trusted boundary; an unauthenticated loopback-only development listener is the sole plaintext exception. Remote gateway-worker links use mTLS or an explicitly configured equivalent authenticated encrypted overlay, while a same-host Unix socket is acceptable. The authenticated worker identity is bound to its route, capabilities, and attempt fence, and readiness fails for plaintext or identity-mismatched remote endpoints. +- R6. After authentication, required-extension checks, and bounded canonical parsing, the gateway first resolves the owner-scoped invocation claim. A retained claim compares the canonical caller request and result-schema digest against the originals and returns its existing Task only while its stored original effective-profile and schema bindings remain intact; a mismatch conflicts without dispatch. Current source/profile authorization, profile resolution/readiness, quota, and deadline checks apply only when atomically creating a new claim that binds the authenticated owner, canonical caller request digest, original effective-profile digest, original result-schema digest, and submitted Task. +- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Acceptance of the current worker fence moves a submitted Task to working before source materialization or setup, so a subsequent source/setup failure transitions from working to failed. Task state and terminal Artifact metadata survive gateway restart. Every nonterminal Task present at startup settles failed once, its old attempt fence is invalidated, and stale worker events cannot overwrite it; the initial service never resumes or automatically replays interrupted provider work. - R8. List operations implement all A2A filters, history bounds, page-size bounds, owner/query-bound cursor pagination, and descending status-update time. One immutable expiry logically hides the Task, claim, events, and artifacts before best-effort physical deletion; expired and unauthorized IDs are indistinguishable. - R9. Small deployments work without an external database. The built-in durable store supports one gateway replica, enforces per-owner/global admission and storage quotas, and reserves capacity for cancellation and terminal settlement; multi-replica storage is outside this delivery. @@ -71,48 +71,48 @@ The two initial runtimes expose different programmatic contracts. Codex provides - R10. Codex and Pi are the complete initial backend set behind one conformance contract, delivered Codex first and Pi second. OpenCode is deferred. (session-settled: user-directed.) - R11. A request selects a server-defined execution profile and may include one `allagents.result-schema/v1` schema for the terminal result: a bounded JSON Schema Draft 2020-12 subset with an object root, every object schema setting `additionalProperties: false`, every declared property listed in `required`, optional values represented by `null` unions, and only `type`, `properties`, `required`, `additionalProperties` with the value `false`, `items`, `enum`, `const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`. The extension version fixes byte, depth, property, and enum limits; admission rejects remote references, format-dependent validation, and unknown keywords; one shared validator governs schema admission and returned values. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. Requests cannot supply raw provider configuration. - R12. The only initial remote source form is a canonical credential-free HTTPS Git URL plus full commit object ID and optional repository-relative subdirectory. Acquisition revalidates destination policy for every connection, disables redirects and repository-controlled secondary fetch/exec features, uses hermetic Git configuration, and verifies that the fetched object is the requested commit before setup. -- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, and every model-initiated command or tool environment. +- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, structured logs/spans before processing or export, and every model-initiated command or tool environment. Credentialed profiles additionally require an OS-enforced provider/tool credential boundary: the credential-bearing provider runtime and model-invoked tools use distinct UID/process/mount policy that prevents tool access to provider processes, procfs entries, and backend config/data roots, or an equivalent credential broker keeps reusable credentials out of the agent runtime. Worker readiness fails when the declared boundary cannot be proved; environment filtering alone is not credential isolation. - R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. - R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. -- R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves all invocation descendants quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. An external supervisor terminates the complete execution boundary when the worker process crashes, and a replacement worker reaps or quarantines orphaned roots before readiness. +- R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves the complete invocation process set quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. If bounded termination escalation cannot prove quiescence, the worker persists termination as unknown/failed, poisons admission, and exits so the external supervisor destroys the complete process boundary; replacement readiness performs orphan recovery before accepting work. The same supervisor boundary handles a worker crash. **Evidence and observability** -- R17. Every terminal result contains an integrity kernel: Task/source/profile/backend identities, action outcome, a structured-result state of `not_requested`, `not_produced`, `valid`, or `invalid` plus reason and schema digest when requested, cancellation or failure classification, separate termination and filesystem-cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. A valid structured result is exactly one `allagents.structured-result` Artifact with one A2A `Part` whose `data` field contains the validated result object and whose `mediaType` is `application/json`; missing or invalid result data never publishes that Artifact. Pre-output source, setup, provider, cancellation, or deadline outcomes retain their primary Task classification and record `not_produced` secondarily. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. +- R17. Every terminal Task contains the required `allagents.execution-integrity` Artifact carrying an integrity kernel: Task/source/profile/backend identities, action outcome, a structured-result state of `not_requested`, `not_produced`, `valid`, or `invalid` plus reason and schema digest when requested, cancellation or failure classification, separate termination and filesystem-cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. A valid structured result is exactly one additional `allagents.structured-result` Artifact with one A2A `Part` whose `data` field contains the validated result object and whose `mediaType` is `application/json`; missing or invalid result data never publishes that Artifact. `not_produced` is legal only before a result candidate is produced. Once validation selects `valid` or `invalid`, later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the fixed structured-result Artifact while the later phase remains the primary Task failure classification. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. - R18. Normalized file evidence distinguishes create, edit, delete, and rename where truthful. It preserves bounded provider-native diffs, events, or trajectories when normalization loses information and separately records truncation, redaction, attribution, original/captured size, and digest semantics. -- R19. Gateway and worker spans propagate W3C Trace Context and export OpenTelemetry data. Telemetry is operational evidence, not the only durable result. +- R19. Gateway and worker calls propagate W3C Trace Context and export metadata-only OpenTelemetry data. One explicit pre-processor allowlist admits only bounded non-content operational metadata; OpenInference and backend-native attributes pass the same allowlist and bounded filtering/redaction before any structured log or span processor. Prompts, model outputs, tool arguments/results, file bodies, source fragments, and secret-bearing attributes are prohibited before export. Owner correlation uses only an opaque identifier appropriate to telemetry-operator access, never caller identity or Task/Artifact authorization. Telemetry access and retention are configured separately from Task and Artifact access and retention, and telemetry is neither durable result truth nor required for terminal lookup. **Ownership and safety boundary** - R20. The gateway executes one coding request. It does not own eval configuration, datasets, repetition, scoring, retry policy, experiment scheduling, or a durable evaluation Run ledger. -- R21. The initial worker topology is one execution at a time for reviewed repositories inside one configured mutual-trust domain. Profiles that claim hostile-source or cross-tenant isolation are rejected until a stronger per-invocation UID, mount, PID, network, and credential boundary is configured. +- R21. The initial worker topology is one execution at a time for reviewed repositories inside one configured mutual-trust domain. R13's narrow OS-enforced provider/tool credential boundary is required for credentialed profiles but does not claim hostile-source or cross-tenant isolation. Profiles making either stronger claim are rejected until a full per-invocation UID, mount, PID, network, and credential isolation boundary is configured. - R22. Gateway admission and worker execution enforce profile limits for request rate, active/retained Tasks, subscriptions, stored bytes, source transfer/expansion, files/inodes, workspace bytes, CPU, memory, PIDs, network, phase deadlines, events, logs, and artifacts. Exhaustion is scoped to one invocation or owner and leaves capacity for terminalization and cleanup. ### Key Flows - F1. **Admit, create, and stream an execution** - **Actors:** A1, A2, A3, A4. - - **Trigger:** A caller sends a text Message with the required extension, immutable source, profile, invocation key, deadline, and optional bounded result schema. - - **Steps:** Authenticate; validate and authorize the complete request and result-schema subset; reserve quota; atomically claim idempotency and create a submitted Task; dispatch a fenced worker attempt; materialize and verify source; execute the selected backend; validate structured output with the shared validator when requested; persist progress before emission; terminalize with the fixed-name structured-result Artifact only for a valid result and with evidence Artifacts after quiescence and cleanup. + - **Trigger:** A caller opts into `https://allagents.dev/a2a/extensions/coding-execution/v1` and sends a text Message whose `extensions` includes that URI and whose `metadata[uri]` contains the immutable source, profile, invocation key, deadline, and optional bounded result schema. + - **Steps:** Authenticate, check extension negotiation, and bounded-canonicalize the request; resolve an owner-scoped retained claim and return or conflict against its original request/profile/schema bindings before mutable admission checks. For a new claim only, validate current source/profile authorization, profile/readiness, quota, and deadline; atomically create the claim and submitted Task; dispatch a fenced worker attempt; accept the current fence and move the Task to working; materialize and verify source; execute the selected backend; validate structured output with the shared validator when requested; persist progress before emission; and terminalize with the required integrity Artifact plus the fixed-name structured-result Artifact only for a valid result after quiescence and cleanup. - **Outcome:** `returnImmediately: true` returns the durable current Task, false/unset waits for terminal state, and streaming starts with that Task before ordered updates. - **Covered by:** R1-R22. - F2. **Replay or reconnect to an invocation** - **Actors:** A1, A2. - **Trigger:** The owner repeats an invocation key or subscribes after a stream disconnect. - - **Steps:** Recompute the canonical digest; reject a conflict; return the existing Task; for active streaming replay/subscription emit its current snapshot then future events; for a terminal Task return it through send replay or `GetTask` without dispatch. + - **Steps:** After authentication and bounded canonical parsing, resolve the owner-scoped claim; compare the request and schema digest with the stored originals and verify the retained Task's original effective-profile/schema bindings without resolving the current profile. Reject a mismatch; otherwise return the existing Task before current authorization, quota, readiness, profile, or deadline checks. For active streaming replay/subscription emit its current snapshot then future events; for a terminal Task return it through send replay or `GetTask` without dispatch. - **Outcome:** Retries do not multiply agent work, and reconnect never promises transient event replay. - **Covered by:** R4, R6-R8. - F3. **Cancel or time out an execution** - **Actors:** A1, A2, A3, A4. - **Trigger:** The caller invokes `CancelTask`, the effective deadline expires, or gateway shutdown claims cancellation. - - **Steps:** Atomically record the first cancellation source; if dispatch never occurred, prove no workspace exists; otherwise send one fenced worker cancel, invoke native abort, terminate descendants, capture termination-safe evidence, clean, and publish canceled only after verification. - - **Outcome:** Completion that wins first remains terminal and later cancel returns `TaskNotCancelableError`; cancellation that wins suppresses late provider success and fails instead of claiming canceled when termination or cleanup cannot be verified. + - **Steps:** Atomically record the first cancellation source; send one revisioned fenced worker cancel even when dispatch delivery is unconfirmed, so an unseen attempt is tombstoned before any delayed dispatch can create a workspace. If work exists, invoke native abort, terminate descendants, capture termination-safe evidence, clean, and publish canceled only after verification. + - **Outcome:** Completion that wins first remains terminal and later cancel returns `TaskNotCancelableError`; cancellation that wins suppresses stale dispatch and late provider success. If bounded escalation cannot prove the complete invocation process set empty, the Task fails rather than claiming canceled, the worker poisons admission and exits, and its supervisor destroys the boundary. - **Covered by:** R7, R14, R16-R18. - F4. **Settle after gateway or worker loss** - **Actors:** A2, A3. - **Trigger:** The gateway restarts with nonterminal Tasks, an acknowledgement is lost, a live worker loses its lease, or a worker process crashes. - - **Steps:** Invalidate the attempt fence and settle every affected Task failed once without provider-session reattachment or automatic replay. A live worker that loses its lease self-aborts and cleans. On worker-process crash, the external supervisor terminates the complete execution boundary; the replacement worker proves termination, then reaps or quarantines orphaned roots before readiness. Reject late events/results and record termination and filesystem cleanup separately as complete only when the responsible boundary proves each outcome. - - **Outcome:** One Task has one terminal result, interrupted work is never presented as resumed, no stale worker can overwrite durable truth, and a crashed worker cannot leave an unowned process or reusable workspace. + - **Steps:** Invalidate the attempt fence and settle every affected Task failed once without provider-session reattachment or automatic replay. A live worker that loses its lease self-aborts and cleans. On worker-process crash or unproved quiescence after bounded escalation, poison admission and exit the worker so the external supervisor terminates the complete execution boundary; the replacement worker proves termination, then reaps or quarantines orphaned roots before readiness. Reject late events/results and record termination and filesystem cleanup separately as complete only when the responsible boundary proves each outcome. + - **Outcome:** One Task has one terminal result, interrupted work is never presented as resumed, no stale worker can overwrite durable truth, and a failed quiescence proof cannot leave the poisoned worker available for another reservation. - **Covered by:** R7, R9, R14, R16-R18, R21-R22. - F5. **Expire retained execution data** - **Actors:** A1, A2. @@ -124,27 +124,27 @@ The two initial runtimes expose different programmatic contracts. Codex provides ### Acceptance Examples - AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile, an exact Git SHA, and an optional result schema, when the caller streams a request, then one Task moves from submitted to working to completed and later `GetTask` returns the same validated output and evidence Artifacts. -- AE2. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with the same canonical request, then the gateway returns the original Task without a second worker dispatch. -- AE3. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with a different prompt, source, profile, or deadline, then the gateway rejects the request and leaves the original Task unchanged. +- AE2. **Covers R6.** Given a retained Task whose original absolute deadline has passed or whose profile is now disabled, changed, or no longer authorized for new work, when its owner reuses the invocation key with the same canonical request and result schema, then the gateway returns the original Task from its stored original bindings before mutable admission checks and makes no second worker dispatch. +- AE3. **Covers R6.** Given a retained Task, when its owner reuses the invocation key with a different prompt, source, profile ID, deadline, or result schema, or the stored original profile/schema binding is inconsistent, then the gateway rejects the request and leaves the original Task unchanged. - AE4. **Covers R5.** Given a Task owned by caller A, when caller B lists Tasks, gets the Task, cancels it, subscribes, or requests an Artifact, then the gateway reveals no resource existence or content. -- AE5. **Covers R12, R16-R18.** Given a requested SHA that does not match the materialized repository, when the worker verifies source, then provider execution never starts and the Task fails with source-verification and cleanup evidence. -- AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. -- AE7. **Covers R10-R11, R17.** Given equivalent profiles, one accepted `allagents.result-schema/v1` schema, and fixture runtime events for Codex and Pi, when each completes the same repository mutation, then both validate with the same schema and validator, publish the same fixed-name structured-result Artifact containing one A2A `Part` with the validated `data` and `mediaType: application/json`, record the same integrity state, and produce the required normalized evidence fields while retaining distinct native evidence. -- AE8. **Covers R4, R7, R19.** Given a caller disconnects during work, when it subscribes again, then it receives the current Task and future updates without duplicate dispatch; telemetry loss does not affect later terminal lookup. +- AE5. **Covers R12, R16-R18.** Given a requested SHA that does not match the materialized repository or setup fails, when the worker has already accepted the current fence, then provider execution never starts, the selected public trace is `Submitted -> Working -> Failed`, and the Task retains source/setup-failure and cleanup evidence. +- AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. Given cancel reaches a worker before its delayed dispatch, the worker tombstones the unseen attempt and the stale dispatch creates no workspace or provider process. +- AE7. **Covers R3, R10-R11, R17.** Given equivalent profiles, one accepted `allagents.result-schema/v1` schema, and fixture runtime events for Codex and Pi, when each completes the same repository mutation, then both publish the required fixed-name integrity Artifact at the schema-defined extension carrier, validate with the same schema and validator, publish the same fixed-name structured-result Artifact containing one A2A `Part` with the validated `data` and `mediaType: application/json`, record the same integrity state, and produce the required normalized evidence fields while retaining distinct native evidence. +- AE8. **Covers R4, R7, R19.** Given canary secrets and cross-owner content fragments in prompts, model output, tool arguments/results, source files, stale events, and errors, when agent, model, tool, stale-event, and error telemetry is processed, then the exporter receives only allowlisted bounded metadata plus the correct opaque owner correlation and receives none of those canaries, fragments, or raw caller identities. Given a caller or exporter disconnects during work, reconnect still returns the current Task and future updates without duplicate dispatch, and telemetry loss does not affect terminal lookup. - AE9. **Covers R15.** Given a known capability denied by profile, the accepted Task becomes rejected after stop and cleanup; given an unknown permission type, it becomes failed as an adapter incompatibility without waiting for a client. -- AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. +- AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. Given output validation has already selected `valid` or `invalid` and a later check or mandatory-evidence phase fails, the failed Task preserves that result state and a valid result preserves its one fixed structured-result Artifact; only a failure before candidate production records `not_produced`. - AE11. **Covers R6, R22.** Given invalid input or exhausted admission quota, the gateway returns a request/resource error and creates no Task; given capacity disappears after durable acceptance, the retained Task fails at dispatch and replay returns it without retry. -- AE12. **Covers R7, R14, R16.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only safe operator telemetry. Given the worker is killed with live descendants and an invocation root, its supervisor terminates the execution boundary and the replacement worker reaps or quarantines the root before readiness without changing the failed Task. +- AE12. **Covers R7, R14, R16.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only allowlisted metadata-only operator telemetry. Given bounded escalation cannot stop a descendant that starts a new session and ignores graceful signals, the worker persists termination unknown/failed, refuses another reservation, exits, and its supervisor destroys the boundary; replacement readiness performs orphan recovery without changing the failed Task. - AE13. **Covers R8.** Given a Task reaches expiry while physical deletion fails, all Task and Artifact operations return the same not-found response and the invocation key can create a new Task. -- AE14. **Covers R21-R22.** Given a profile requests pooled hostile-source or cross-tenant execution, startup/admission rejects it; a reviewed single-trust-domain profile runs one bounded execution without exposing worker control credentials to the child environment. +- AE14. **Covers R5, R13, R21-R22.** Given a production public listener or remote worker route lacks its configured trusted transport or authenticated peer identity, readiness fails; a same-host Unix worker socket is accepted. Given a credentialed reviewed-domain profile, model tools cannot inspect provider process environments, process listings, backend config/data roots, or exfiltrate provider/control credentials across the configured OS boundary. Hostile-source or cross-tenant claims remain rejected. ### Success Criteria -- The official A2A JavaScript client can discover the card and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, replay, cancel, and expiry behavior against the built service. +- The official A2A JavaScript client can discover the required extension, negotiate it through `A2A-Extensions`, use the standard Message and Artifact extension carriers, and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, retained replay, cancel, and expiry behavior against the built service without `Task.extensions`. - One conformance fixture passes unchanged through the Codex and Pi adapters. -- Admission, replay, fencing, cancellation races, restart terminalization without resume, supervised worker-crash cleanup, authorization isolation, source hardening, portable structured-result validation, quotas, and evidence integrity have deterministic integration coverage. +- Admission, retained replay, monotonic worker commands, fencing, acceptance-before-materialization source/setup failure, cancellation races, trace-order/fence/multiplicity constraints, failed-quiescence recycling, restart terminalization without resume, supervised worker-crash cleanup, trusted transports, authorization isolation, metadata-only telemetry export, OS-enforced provider/tool credential separation, source hardening, portable structured-result validation, quotas, and evidence integrity have deterministic integration coverage. - The gateway image contains no coding-agent runtime and cannot access worker workspace roots. -- The initial worker runs one reviewed-trust-domain execution at a time, model-initiated tools receive no provider/control credentials, repository Pi extensions cannot auto-load, and no live descendant or reusable workspace survives a completed or crashed attempt. +- The initial worker runs one reviewed-trust-domain execution at a time, model-initiated tools are OS-isolated from provider/control credentials, repository Pi extensions cannot auto-load, and no live descendant or reusable workspace survives a completed, failed-quiescence, or crashed attempt. ### Scope Boundaries @@ -153,9 +153,9 @@ The two initial runtimes expose different programmatic contracts. Codex provides - A2A 1.0 HTTP+JSON and SSE streaming. - One versioned AllAgents coding-execution extension and one versioned private worker protocol. - Codex and Pi backends. -- Built-in bearer authentication with OIDC/JWT and static service-token modes. -- Single-replica durable file storage, authenticated Artifact retrieval, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. -- Reviewed repositories in one configured mutual-trust domain per worker deployment. +- Built-in bearer authentication with OIDC/JWT and static service-token modes behind the named production TLS boundary. +- Single-replica durable file storage, authenticated Artifact retrieval, authenticated encrypted remote worker transport or same-host Unix sockets, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. +- Reviewed repositories in one configured mutual-trust domain per worker deployment, with the narrow OS-enforced provider/tool credential boundary required for credentialed profiles. **Deferred to follow-up work** @@ -188,6 +188,10 @@ The two initial runtimes expose different programmatic contracts. Codex provides - [Pi CLI reference](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/README.md#cli-reference) - [Pi extension API](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/extensions.md) - [Pi provider credentials](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/providers.md) +- [Buzz pure Kubernetes state classifier](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-backend-kubernetes/src/classify.rs) +- [Buzz non-secret intent fingerprint](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-backend-kubernetes/src/intent.rs) +- [Buzz conformance coverage checker](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-conformance/src/checker.rs) +- [Buzz bounded process-tree cancellation](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-dev-mcp/src/shell.rs) --- @@ -195,20 +199,20 @@ The two initial runtimes expose different programmatic contracts. Codex provides ### Key Technical Decisions -- KTD1. **Use the official A2A JavaScript SDK behind an AllAgents request-handler decorator.** Pin a compatible A2A 1.x SDK. The decorator owns admission, canonical Task reservation, idempotent replay, stream snapshot selection, and cancellation routing before `DefaultRequestHandler` can allocate another Task or terminalize cancellation prematurely; the SDK retains standard transport/event mechanics. Governs R1-R8, R14. -- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes both versioned contracts, generated JSON Schemas, bounds, and fixtures. The public contract carries the `allagents.result-schema/v1` closed subset, its canonical digest, four structured-result states, and the fixed `allagents.structured-result` Artifact containing one A2A `Part` with `data` and `mediaType: application/json`. The worker protocol carries attempt identity, profile digest, dispatch acceptance, monotonic event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. -- KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates the invocation claim and submitted Task together, stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. -- KTD4. **Authenticate at HTTP ingress before A2A storage or dispatch.** Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. Unauthenticated mode is allowed only on a loopback listener. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Governs R5-R6, R13. -- KTD5. **Use fenced, separately deployable gateway and worker services.** The gateway owns A2A and durable Task/results truth; the worker owns ephemeral execution attempts, workspaces, and provider processes. Every dispatch has a gateway-generated attempt ID, lease ID/epoch, short-lived capability, and event sequence. Workers idempotently accept duplicate delivery of the same attempt, reject conflicting attempts, and gateways ignore stale/out-of-order events and late terminal results. Governs R7, R10-R16, R21-R22. -- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Worker-process exit makes the external supervisor terminate the complete execution boundary; before readiness the replacement worker proves termination and reaps or quarantines orphaned invocation roots. Termination and filesystem cleanup remain separate outcomes and are unknown until proved. The gateway never reattaches to or resumes a provider session. Caller stream disconnect never affects the lease. Governs R7, R14, R16-R18. +- KTD1. **Use the official A2A JavaScript SDK behind an AllAgents request-handler decorator.** Pin a compatible A2A 1.x SDK. After authentication, required-extension checks, and bounded canonical parsing, the decorator resolves an owner-scoped retained claim before mutable admission; identical replay bypasses current profile/deadline/quota/readiness checks and any new SDK Task/bus allocation. New requests then pass mutable admission and canonical Task reservation. The decorator also owns stream snapshot selection and cancellation routing before `DefaultRequestHandler` can allocate another Task or terminalize cancellation prematurely; the SDK retains standard transport/event mechanics. Governs R1-R8, R14. +- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes `https://allagents.dev/a2a/extensions/coding-execution/v1`, its standard Agent Card/header/Message/Artifact negotiation, `Message.metadata[uri]` request location, and the single-Part `allagents.execution-integrity` Artifact data location; no schema or implementation adds `Task.extensions`. The public contract also carries the `allagents.result-schema/v1` closed subset, its canonical digest, four structured-result states, and the separate fixed `allagents.structured-result` Artifact. The worker protocol carries worker identity, attempt identity, profile digest, monotonic command revision and tombstone state, dispatch acceptance, event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. +- KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates a new invocation claim and submitted Task together after mutable admission, storing the canonical caller request and the original effective-profile and result-schema digests needed for retained replay. It stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. +- KTD4. **Authenticate at a named trusted HTTP ingress before A2A storage or dispatch.** Production traffic reaches the gateway through TLS terminated by the configured gateway or named trusted reverse-proxy boundary; plaintext is allowed only for an unauthenticated loopback development listener. Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Readiness rejects a production public URL whose trusted TLS boundary is absent or inconsistent. Governs R5-R6, R13. +- KTD5. **Use fenced, separately deployable gateway and worker services.** Remote gateway-worker routes use mTLS or an explicitly equivalent authenticated encrypted overlay; a same-host Unix socket is acceptable. The authenticated worker identity is pinned to the configured route/capability set, and every short-lived attempt capability is bound to that identity, attempt ID, lease ID/epoch, and fence. Each worker keeps one minimal durable monotonic command record scoped to its worker identity and lease: `Cancel(attempt, fence, revision)` tombstones even an unseen attempt, and `Dispatch` for a tombstoned or lower-revision attempt is rejected before workspace creation. Dispatch/cancel I/O conditionally verifies the persisted command/outbox revision immediately before any mutating or terminating effect. Duplicate delivery is idempotent; conflicting, stale, out-of-order, or identity-mismatched commands/events are rejected. Gateway and worker transition selectors remain pure and executors re-enter from persisted or freshly observed state. This record is worker-local fence state, not a new durable execution subsystem. Governs R5, R7, R10-R16, R21-R22. +- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Worker-process exit makes the external supervisor terminate the complete execution boundary. If bounded escalation cannot prove the complete invocation process set empty, the worker records termination unknown/failed, poisons admission, and exits rather than accepting another reservation; its supervisor destroys the boundary. Before readiness the replacement proves termination and reaps or quarantines orphaned invocation roots. Production readiness accepts a dedicated worker container process namespace under a minimal init/reaper as the baseline; a non-container deployment must prove an equivalent systemd/cgroup boundary. The gateway never reattaches to or resumes a provider session. Governs R7, R14, R16-R18. - KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, optional structured result, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, schema validation, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | pi` registry is the only production dispatch point. Governs R10-R11, R14-R18, R21-R22. -- KTD8. **Use each provider's supported automation surface directly.** Codex depends directly on pinned `@openai/codex-sdk`, creates one fresh thread per Task, passes `AbortSignal` and optional per-turn `outputSchema`, consumes streamed events, and applies a pinned shell-environment policy that excludes provider/control credentials from model-initiated commands. Pi uses `pi --mode rpc --no-session --no-extensions --no-builtin-tools` with a strict LF-delimited JSON parser, `agent_settled`, `get_session_stats`, RPC abort, an invocation-local credential store rather than credential environment variables, and one explicitly loaded worker-owned policy extension outside the repository. That extension supplies workspace-confined filesystem/command tools and the terminating result tool; no repository extension or unrestricted built-in tool loads. Promptfoo's Codex provider and tests are characterization references only; AllAgents neither vendors them nor inherits their config, cache, pricing, retry, thread-pool, or `ProviderResponse` concerns. Governs R10-R18. -- KTD9. **Make profiles the policy boundary.** Requests select a profile ID and may provide only an `allagents.result-schema/v1` schema. They cannot override backend credentials, executable paths, provider config, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. Profile digests and the canonical result-schema digest enter idempotency and provenance. Governs R6, R11-R16, R21-R22. -- KTD10. **Capture Git and provider evidence as separate layers after quiescence.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded layer. Neither layer is promoted as exact causality when incomplete. Governs R16-R18. +- KTD8. **Use each provider's supported automation surface directly behind the credential boundary.** Codex depends directly on pinned `@openai/codex-sdk`, creates one fresh thread per Task, passes `AbortSignal` and optional per-turn `outputSchema`, and consumes streamed events. Pi uses strict RPC with an invocation-local credential store and one explicitly loaded worker-owned policy extension; repository extensions and unrestricted built-ins never load. For either adapter, a credentialed provider runtime is separated from every model-invoked tool by the R13 OS-enforced UID/process/mount boundary or an equivalent credential broker; shell-environment filtering is defense in depth, not the boundary. Promptfoo's Codex provider and tests are characterization references only; AllAgents neither vendors them nor inherits their config, cache, pricing, retry, thread-pool, or `ProviderResponse` concerns. Governs R10-R18. +- KTD9. **Make profiles the new-admission policy boundary.** Requests select a profile ID and may provide only an `allagents.result-schema/v1` schema. They cannot override backend credentials, executable paths, provider config, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. For a new claim, resolve a versioned canonical `EffectiveProfileIntent`, compute its digest without resolved secrets or per-attempt state, and persist it with the canonical caller request and result-schema digest. Retained replay compares those stored original bindings and never substitutes or re-resolves the current profile. Governs R6, R11-R16, R21-R22. +- KTD10. **Keep durable evidence and operational telemetry as separate bounded layers.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded evidence layer; neither Git nor provider evidence is promoted as exact causality when incomplete. Telemetry is a third, non-durable metadata-only channel: one small shared pre-export sanitizer applies an explicit operational-metadata allowlist plus bounded filtering/redaction before every structured log or span processor, and only opaque owner correlation may cross the separately governed operator boundary. OpenInference and backend-native attributes receive no bypass. This is an export guard, not a telemetry framework or alternate evidence store. Governs R13, R16-R19. - KTD11. **Treat Codex and Pi as the complete initial backend set.** Codex lands first; Pi lands second against the established contract; OpenCode is deferred. (session-settled: user-directed.) Governs R10. -- KTD12. **Separate terminal integrity from optional evidence bodies.** Identity, action outcome, the four-state structured-result record and fixed Artifact rule, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance must validate before terminal publication. A pre-output failure records `not_produced` without replacing its primary phase classification. Predictable budget truncation/redaction of logs, diffs, native events, or produced-file bodies may preserve completion with explicit metadata; capture failure that breaks the integrity kernel fails in the evidence phase. Governs R4, R17-R18. +- KTD12. **Separate terminal integrity from optional evidence bodies.** The fixed `allagents.execution-integrity` Artifact validates identity, action outcome, the four-state structured-result record, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance before terminal publication. `not_produced` applies only before result-candidate production. Once validation selects `valid` or `invalid`, a later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the separate fixed structured-result Artifact while retaining the later phase as the primary Task failure. Predictable optional-body truncation/redaction may preserve completion; failure that breaks the integrity kernel fails in the evidence phase. Governs R3-R4, R17-R18. - KTD13. **Harden Git acquisition as a network security boundary.** Accept canonical HTTPS origins only. Use hermetic Git configuration, disable redirects, proxies, helpers, hooks, filters, LFS smudge, submodule recursion, alternates, and non-HTTPS protocols. Revalidate normalized host/address policy for every connection, never forward credentials across origins, and verify the full object ID resolves to a commit fetched from the approved remote. Governs R12-R13, R22. -- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Provider/source credentials are absent from setup/check phases, model-initiated commands and tools, and child-visible worker control state. Pi disables repository extensions and built-in tools; only the worker-owned policy extension may load, and its replacement tools confine paths to the invocation workspace and spawn commands with the phase allowlist. Stronger isolation is a separate sandbox-driver capability. Governs R13, R16, R21-R22. +- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Credentialed profiles still require R13's narrower OS-enforced provider/tool separation: model tools cannot inspect provider processes, procfs entries, or backend config/data roots, and readiness fails without that capability. Provider/source credentials are absent from setup/check phases and child-visible worker control state. Pi disables repository extensions and built-in tools; only the worker-owned policy extension may load. This credential boundary does not imply hostile-source or cross-tenant isolation; that stronger sandbox-driver capability remains deferred. Governs R13, R16, R21-R22. - KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, Codex SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. ### High-Level Technical Design @@ -217,10 +221,10 @@ The two initial runtimes expose different programmatic contracts. Codex provides ```mermaid flowchart TB - Caller[Authenticated A2A caller] -->|HTTP+JSON / SSE| Gateway[execution-service gateway] - Gateway --> Auth[Auth, admission, profile policy] + Caller[Authenticated A2A caller] -->|TLS at named trusted ingress| Gateway[execution-service gateway] + Gateway --> Auth[Auth, retained replay, new admission] Gateway --> Store[Generation-based Task and Artifact store] - Gateway -->|Fenced private protocol| Worker[Single-execution worker] + Gateway -->|mTLS/authenticated overlay or same-host Unix socket| Worker[Single-execution worker] Worker --> Source[Hardened Git acquisition] Worker --> Registry[Closed backend registry] Registry --> Codex[Codex SDK] @@ -241,31 +245,37 @@ sequenceDiagram participant W as Worker participant B as Backend adapter - C->>G: SendMessage + required extension - G->>G: Authenticate, validate, authorize, quota, deadline - G->>S: Atomic claim + submitted Task - alt identical replay - S-->>G: Existing Task and current fence - G-->>C: Existing Task; follow active future events only - else new accepted Task + C->>G: SendMessage + header/Message extension + metadata[uri] + G->>G: Authenticate, check extension, canonicalize within bounds + G->>S: Resolve owner-scoped invocation claim + alt retained identical replay + S-->>G: Existing Task + original request/profile/schema bindings + G-->>C: Existing Task before current admission checks + else conflicting retained claim + G-->>C: Conflict; existing Task unchanged + else no retained claim + G->>G: Current authorization, profile/readiness, quota, deadline + G->>S: Atomic new claim + submitted Task + original digests S-->>G: Task + attempt/lease fence - G->>W: Dispatch(attempt, fence, profile, source, deadline) + G->>W: Dispatch(attempt, fence, command revision) + W->>W: Verify command record before workspace creation W-->>G: Accepted(attempt, fence) W->>W: Materialize, verify, setup, baseline - W->>B: Invoke with isolated roots and policy + W->>B: Invoke with isolated roots and credential boundary B-->>W: Progress, usage, native evidence W-->>G: Sequenced fenced progress G->>S: Compare-and-swap Task generation opt cancellation or deadline wins C->>G: CancelTask G->>S: Persist cancellation intent once - G->>W: Fenced cancel + G->>W: Cancel(attempt, fence, newer command revision) + W->>W: Persist tombstone before effects W->>B: Native abort end W->>W: Stop descendants, capture evidence, cleanup W-->>G: Fenced terminal result G->>S: Store blobs then atomically commit terminal manifest - G-->>C: Terminal status and Artifacts + G-->>C: Terminal status and extension Artifacts end ``` @@ -276,10 +286,10 @@ stateDiagram-v2 [*] --> Submitted: claim and Task committed Submitted --> Working: worker accepts current fence Submitted --> Canceled: cancellation proves no workspace exists - Submitted --> Failed: dispatch, restart, or source failure + Submitted --> Failed: dispatch or restart failure Submitted --> Rejected: accepted policy refusal before work Working --> Completed: integrity kernel and cleanup validate - Working --> Failed: provider, check, evidence, cleanup, crash, or restart failure + Working --> Failed: source, setup, provider, check, evidence, cleanup, crash, or restart failure Working --> Rejected: known profile permission denial after stop and cleanup Working --> Canceled: cancellation wins and stop/cleanup verify Completed --> [*] @@ -296,12 +306,14 @@ Terminal states are immutable. Cancellation intent, termination, evidence captur stateDiagram-v2 [*] --> Admitted Admitted --> Dispatching - Dispatching --> Running: current fence accepted - Dispatching --> Terminalizing: dispatch rejected or unknown + Dispatching --> Running: current command revision accepted + Dispatching --> Terminalizing: dispatch rejected, tombstoned, or unknown Running --> CancelRequested: caller, deadline, shutdown, or lease expiry Running --> Quiescing: provider and checks finish CancelRequested --> Quiescing - Quiescing --> CapturingEvidence: descendants verified stopped + Quiescing --> CapturingEvidence: complete process set verified empty + Quiescing --> Poisoned: bounded escalation cannot prove empty + Poisoned --> [*]: persist unknown/failed and exit boundary CapturingEvidence --> Cleaning Cleaning --> Terminalizing Terminalizing --> Retained @@ -350,6 +362,7 @@ packages/execution-service/ registry.ts codex.ts pi.ts + pi-rpc.ts pi-policy-extension.ts tests/ fixtures/execution/ @@ -370,64 +383,69 @@ docs/src/content/docs/ ### Configuration Contract -- Gateway configuration defines listener/public URL, auth and canonical owner mapping, store/retention, admission and subscription quotas, low-space watermarks, Artifact limits, worker endpoints, internal capability secrets, and profiles. -- Each profile defines backend, worker route, allowed Git origins/addresses, provider/model settings, phase-specific environment allowlists, deterministic permissions, setup/check commands, artifact globs, effective deadline ceiling, trust class, resource limits, cleanup policy, and evidence budgets. -- Worker configuration fixes a private listener, one-execution concurrency, workspace root, execution-supervisor mechanism, pre-readiness orphan policy, lease grace, backend runtime constraints, trust domain, resource-control capability, and request/result limits. -- Configuration contains environment-variable names but never secret values. Startup resolves the complete graph, verifies that profile claims do not exceed deployment capabilities, and becomes ready only when store, workers, runtimes, quotas, and free-space reserves pass. +- Gateway configuration defines the listener/public URL, a named trusted TLS termination boundary for production ingress, auth and canonical owner mapping, store/retention, admission and subscription quotas, low-space watermarks, Artifact limits, worker routes, internal capability secrets, and profiles. Each remote worker route declares mTLS or an explicitly equivalent authenticated encrypted overlay, pinned worker identity/capabilities, and trust material; a same-host route may declare a Unix socket. Plaintext remote URLs are invalid. +- Each profile defines backend, worker route, allowed Git origins/addresses, provider/model settings, phase-specific environment allowlists, deterministic permissions, setup/check commands, artifact globs, effective deadline ceiling, trust class, resource limits, cleanup policy, evidence budgets, and the required provider/tool credential-boundary capability for credentialed execution. +- Worker configuration fixes a private listener, worker identity, one-execution concurrency, workspace root, minimal worker-local command-record location, execution-supervisor mechanism, pre-readiness orphan policy, lease grace, backend runtime constraints, trust domain, resource-control and provider/tool credential-boundary capabilities, and request/result limits. +- Production worker readiness requires authenticated route identity, protected remote transport or a same-host Unix socket, an enforceable credential boundary for every credentialed profile, and a supervisor that proves complete descendant termination and root ownership. The supported supervisor baseline is a dedicated worker container process namespace under a minimal init/reaper; bare-host deployment requires an equivalent systemd/cgroup mechanism. +- Telemetry configuration defines the OTLP destination, filtering/redaction bounds, opaque owner-correlation derivation, and telemetry-specific operator access and retention. The service version fixes the metadata allowlist; configuration cannot extend it to prompt/output/tool/source/file-body attributes, secret-bearing fields, raw caller identity, or unfiltered backend-native/OpenInference attribute passthrough. +- Configuration contains environment-variable names but never secret values. Startup resolves the complete graph and becomes ready only when trusted ingress, worker transports/identities, store, runtimes, quotas, free-space reserves, supervisor/orphan recovery, and declared profile capabilities pass. Any unprotected remote endpoint or unproved credential/supervisor boundary fails readiness. ### Error and Status Mapping | Condition | A2A result | Required extension detail | |---|---|---| -| Authentication, malformed/unsupported extension, invalid source/profile, unauthorized policy, expired deadline, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | -| Identical invocation replay | Existing Task | No new Task, worker attempt, or quota reservation | -| Conflicting invocation key | Operation error; no new Task | Conflict code; existing Task unchanged | +| New-admission authentication, malformed/unsupported extension carrier, invalid source/profile, unauthorized policy, expired deadline, current-profile/readiness failure, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | +| Identical retained invocation replay | Existing Task | Returned from stored original request/profile/schema bindings before current deadline, quota, authorization, readiness, or profile checks; no new Task, worker attempt, or quota reservation | +| Conflicting invocation key or inconsistent stored binding | Operation error; no new Task | Conflict code; existing Task unchanged | | Worker capacity loss after acceptance | `TASK_STATE_FAILED` | `dispatch/capacity_exhausted`, retriable fact, no workspace created; gateway does not retry | | Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | | Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | | Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | -| Source, setup, provider, check, mandatory evidence, worker crash, or infrastructure failure | `TASK_STATE_FAILED` | Typed primary phase, safe message, retriable fact, structured result `not_produced` when requested, separate termination/cleanup/completeness | -| Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` or `structured_result/invalid`, no structured-result Artifact | -| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors, native abort, structured result `not_produced` unless already valid, termination, cleanup | +| Failure before result-candidate production | `TASK_STATE_FAILED` | Typed primary source/setup/provider/dispatch/crash/infrastructure phase, safe message, retriable fact, requested structured result `not_produced`, separate termination/cleanup/completeness | +| Check, mandatory-evidence, cleanup, crash, or infrastructure failure after result validation | `TASK_STATE_FAILED` | Preserve selected `valid` or `invalid`; preserve exactly one fixed structured-result Artifact for `valid`; later phase remains primary failure | +| Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` with `not_produced`, or `structured_result/invalid` with `invalid`; no structured-result Artifact | +| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors and native abort; use `not_produced` only before a candidate, otherwise preserve `valid`/`invalid` and the valid Artifact; record termination and cleanup | | Cancellation loses to terminal completion | Existing terminal Task / `TaskNotCancelableError` | No state mutation or second abort | -| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Output plus complete required evidence; a requested valid result uses the fixed-name Artifact with one A2A `Part` containing `data` and `mediaType: application/json` | +| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Required extension integrity Artifact plus complete evidence; a requested valid result uses the separate fixed-name Artifact with one A2A `Part` containing `data` and `mediaType: application/json` | | Successful action with allowed bounded optional-evidence gap | `TASK_STATE_COMPLETED` | Per-dimension incomplete flag, reason, original/captured size, digest and redaction/truncation flags | -| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid, structured result `not_produced` unless already committed, and cleanup unknown unless proven | +| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid; use `not_produced` only before a candidate, otherwise preserve selected state and valid Artifact; cleanup unknown unless proven | | Retention expiry | Not found | Aggregate logically hidden before physical deletion; Artifact URL also invalid | ### Phased Delivery -1. Create the private Node 22 service package and freeze the public extension, portable result-schema subset, structured-result Artifact, worker protocol, profiles, fixtures, and error vocabulary. -2. Build authenticated durable A2A Task handling and fenced worker dispatch against a fake worker; startup terminalizes interrupted Tasks without attempting provider reattachment. -3. Build the supervised single-execution worker lifecycle, pre-readiness orphan reaper, and hardened source/evidence handling against a fake adapter. -4. Add the direct Codex SDK adapter and prove structured output, cancellation, provider-credential exclusion from model commands, environment isolation, and native evidence. -5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing confined tools plus the terminating result tool. -6. Package the services and run cross-backend, security, process, and A2A conformance before enabling a consumer. +1. Create the private Node 22 service package and freeze the public extension URI and standard carriers, integrity and structured-result Artifacts, portable result-schema subset, worker protocol including command revisions/tombstones, profiles, fixtures, and error vocabulary. +2. Build authenticated durable A2A Task handling, retained-claim-first replay, and trusted fenced worker dispatch against a fake worker; startup terminalizes interrupted Tasks without attempting provider reattachment. +3. Build the supervised single-execution worker lifecycle, monotonic command record, failed-quiescence boundary recycling, pre-readiness orphan reaper, OS credential boundary, and hardened source/evidence handling against a fake adapter. +4. Add the direct Codex SDK adapter and prove structured output, cancellation, OS-enforced provider/tool credential separation, and native evidence. +5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing OS-confined tools plus the terminating result tool. +6. Package the services and run cross-backend, transport, security, process, and A2A conformance before enabling a consumer. ### System-Wide Impact - **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. - **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. - **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. -- **Security:** New review-critical surfaces are auth, owner-key derivation, source SSRF, admission/resource quotas, setup/check policy, provider-credential exclusion from model tools, Pi extension/tool replacement, phase-scoped secrets, internal fences, Artifact capture/serving, and reviewed-source trust enforcement. -- **Operations:** Gateway and worker health, readiness, quotas, low-space state, structured logs, traces, tombstone backlog, lease expiry, supervisor boundary health, orphan-root quarantine/reaping, stale event rejection, and graceful shutdown need independent signals. +- **Security:** New review-critical surfaces are trusted public/private transports, auth, owner-key derivation, retained-replay ordering, source SSRF, admission/resource quotas, setup/check policy, OS-enforced provider/tool credential separation, Pi extension/tool replacement, phase-scoped secrets, metadata-only telemetry filtering and operator boundaries, internal fences and monotonic command records, Artifact capture/serving, and reviewed-source trust enforcement. +- **Operations:** Gateway and worker health, readiness, transport/peer identity, quotas, low-space state, allowlisted metadata-only structured logs/traces, telemetry-specific access/retention, command tombstones, lease expiry, poisoned-worker exit, supervisor boundary health, orphan-root quarantine/reaping, stale event rejection, and graceful shutdown need independent signals. - **Consumers:** AI Evals can build its runner provider only after the Agent Card, extension schemas, and conformance fixtures are versioned and published. ### Risks and Mitigations - **Provider API churn:** Pin exact compatible SDK/CLI versions in the service lockfile and worker image. Gate capabilities at startup, keep captured provider fixtures versioned, and use Promptfoo's Codex tests as characterization input rather than vendored implementation. -- **False idempotency or stale settlement:** Claim Task/idempotency in one aggregate, use revision/fence compare-and-swap, sequence events, and fault-test duplicate delivery, cancellation races, restart, and late results. +- **False idempotency or stale settlement:** Resolve owner-scoped retained claims before mutable admission and compare stored original request/profile/schema bindings. For new work, claim Task/idempotency in one aggregate, use revision/fence compare-and-swap, sequence events, and fault-test conflicts, cancellation races, restart, and late results. - **Task/store corruption:** Publish immutable blobs and generations before one manifest switch; tombstone before deletion; validate owner tuples/manifests at startup; garbage-collect unreachable generations; document the one-replica limit. - **Owner collision or path injection:** Hash a bounded canonical issuer/tenant/subject tuple, store and verify the tuple inside the owner aggregate, and use only server-generated opaque IDs in paths. -- **Orphan processes and roots:** Combine explicit cancel, native abort, process-group termination, one-execution supervisor/container death, lease expiry, pre-readiness orphan reaping or quarantine, and separate termination/filesystem proof before evidence or readiness. +- **Bearer interception or worker impersonation:** Require TLS at the named public ingress boundary and mTLS/equivalent authenticated encryption for remote worker routes, pin worker identity/capabilities, bind attempt capabilities to that identity and fence, and reject plaintext or wrong-peer readiness. +- **Orphan processes and roots:** Combine explicit cancel, native abort, process-set verification, one-execution supervisor/container death, lease expiry, and pre-readiness orphan reaping or quarantine. Failed quiescence poisons admission and exits the worker so the supervisor destroys the boundary; termination/filesystem outcomes remain separate. - **False recovery claims:** Persist Task and evidence truth only. Startup fails active Tasks, invalidates fences, and relies on lease expiry or supervisor-boundary proof instead of resuming provider sessions. -- **Structured-output drift:** Admit only the versioned closed schema subset, include its canonical digest in idempotency/provenance, pass the exact accepted schema through each adapter's supported mechanism, validate with one shared validator, publish only the fixed Artifact shape, and fail rather than publish missing or invalid JSON. -- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral, origin-bound, and absent from repository config, process arguments, model-initiated command/tool environments, retained workspaces, logs, and errors; Pi repository extensions and unrestricted built-in tools never load, and policy tools cannot access Pi config/data roots. -- **Resource exhaustion:** Reserve per-owner/global gateway quota before claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. +- **Structured-output drift:** Admit only the versioned closed schema subset, include its canonical digest in provenance and original claim bindings, pass the exact accepted schema through each adapter, validate with one shared validator, preserve an already selected result across later failures, and enforce the two fixed Artifact shapes. +- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral and origin-bound. Credentialed profiles also enforce the R13 OS provider/tool boundary or broker; environment filtering remains defense in depth. Pi repository extensions and unrestricted built-in tools never load. +- **Telemetry disclosure:** Apply KTD10's pre-export guard before every structured log/span processor and reject content or secret-bearing attributes rather than relying on exporter policy. Canary-secret and cross-owner-fragment tests cover agent, model, tool, stale-event, and error paths; telemetry operators receive only bounded metadata and opaque owner correlation under separate access and retention. +- **Resource exhaustion:** Reserve per-owner/global gateway quota only for new claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. - **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. - **Evidence overclaim:** Enforce KTD12's integrity kernel and per-dimension completeness. Truncation and redaction remain independent facts. - **Permission deadlock:** Initial profiles never prompt. Known requests resolve for one isolated invocation; unknown shapes fail closed as adapter incompatibility. -- **Trust-boundary overclaim:** Reject pooled hostile-source/cross-tenant profiles and state the reviewed mutual-trust boundary in config, readiness, Agent Card metadata, and docs. +- **Trust-boundary overclaim:** Enforce the narrow provider/tool credential boundary for credentialed profiles while rejecting pooled hostile-source/cross-tenant claims; state plainly that the former does not provide the latter. - **Cross-platform drift:** Keep gateway/store tests cross-platform. State that worker execution and hardened evidence/source controls are Linux-only. ### Assumptions @@ -435,7 +453,7 @@ docs/src/content/docs/ - The first production deployment runs one gateway replica with persistent storage. Multi-replica transactional storage is deferred. - Git over hardened HTTPS and exact commit object ID covers the initial consumer. Other source transports require a later extension version or capability. - Setup and check commands are operator-controlled profile policy, not caller-supplied shell text. -- Initial repositories are reviewed inside one configured mutual-trust domain. Strong hostile-code or cross-tenant execution remains unavailable until a stronger sandbox driver exists. +- Initial repositories are reviewed inside one configured mutual-trust domain. Credentialed profiles still enforce provider/tool credential separation, but that narrower boundary does not make hostile-code or cross-tenant execution available; those claims require a stronger sandbox driver. - Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. --- @@ -444,34 +462,35 @@ docs/src/content/docs/ ### U1. Versioned public and worker contracts -- **Goal:** Freeze the extension, profile vocabulary, private worker protocol, canonical digest input, result envelope, typed failures, and conformance fixtures before either service endpoint. +- **Goal:** Freeze the standard public extension carriers, integrity and structured-result Artifacts, profile vocabulary, private worker protocol including monotonic command state, original idempotency bindings, typed failures, and conformance fixtures before either service endpoint. - **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12. - **Dependencies:** None. -- **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. -- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas, one public extension URI, and one private protocol version. Define the exact `allagents.result-schema/v1` keyword allowlist and bounds, canonical schema digest, four structured-result states, fixed-name Artifact containing one A2A `Part` with `data` and `mediaType: application/json`, and shared schema/result validator. Include attempt/fence/lease identity, monotonic event sequence, accepted dispatch, renew/cancel, bounded terminal acknowledgement, public/private state separation, and integrity-kernel rules. Canonicalize caller input plus effective profile and result-schema digests for idempotency. Generate checked-in JSON Schemas and fixtures from the same source. +- **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/result-schema-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. +- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas and freeze `https://allagents.dev/a2a/extensions/coding-execution/v1`: required Agent Card advertisement, `A2A-Extensions` negotiation, `Message.extensions`, request data only at `Message.metadata[uri]`, and terminal integrity data only in the single Part of the fixed-name `allagents.execution-integrity` Artifact whose `extensions` contains the URI. Explicitly forbid `Task.extensions`. Define the portable result-schema subset, canonical caller/schema/profile digests, four result states, separate fixed `allagents.structured-result` Artifact, and shared validator. Define original claim bindings independently from mutable current policy. Add worker identity, attempt/fence/lease identity, monotonic command revision, unseen-attempt cancel tombstone, conditional effect revision, event sequence, terminal acknowledgement, and integrity rules. Generate checked-in schemas and fixtures from one source. - **Execution note:** Start with fixture-driven schema, framing, and digest tests. Observe failures for unknown versions, credential-bearing sources, mutable revisions, unsafe paths, invalid public states, stale fences, oversized records, and conflicting canonical inputs before implementing schemas. -- **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, and `src/core/native/types.ts` for safe error/provenance normalization. +- **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, `src/core/native/types.ts` for safe error/provenance normalization, and Buzz's structurally non-secret intent template for the narrow digest-input pattern. - **Test scenarios:** - - A minimal valid request with text prompt, invocation key, profile, exact commit, deadline, and optional `allagents.result-schema/v1` schema parses and produces a stable digest across object-key ordering. - - Changing prompt, source object ID, profile ID/digest, result schema, artifact selection, or deadline changes the digest; trace IDs and transport metadata do not. + - A minimal valid Message negotiates the exact URI in `A2A-Extensions`, includes it in `Message.extensions`, puts the bounded request only at `Message.metadata[uri]`, and produces a stable digest across object-key ordering; missing/mismatched carriers and any `Task.extensions` field are rejected. Every terminal fixture has exactly one `allagents.execution-integrity` Artifact with the URI in `Artifact.extensions` and the schema-defined envelope in its single `data` Part. + - Changing prompt, source object ID, profile ID, result schema, artifact selection, or deadline changes the canonical caller digest; trace IDs and transport metadata do not. The original effective-profile and result-schema digests are stored separately for retained replay. + - Rotating a resolved secret value, changing attempt/lease/trace identity, or changing a per-run path leaves the profile digest unchanged; changing a policy field or environment-variable name changes it, and the digest serializer cannot accept secret-bearing runtime state. - Unsupported keywords, remote references, non-object roots, object schemas that omit `additionalProperties: false`, undeclared optional properties, format-dependent validation, or schemas over byte/depth/property/enum limits are rejected before Task creation; every accepted schema validates identically in admission, worker, Codex forwarding, and Pi tool generation. - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. - - Worker fixtures reject missing/mismatched attempt IDs, lease epochs, profile digests, event sequence, bounds, and terminal acknowledgements. - - `not_requested`, `not_produced`, `valid`, and `invalid` cover success, pre-output failure, cancellation, missing output, and invalid output without replacing the primary Task classification; only `valid` permits one `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data`, `mediaType: application/json`, and a matching schema digest in Artifact metadata. + - Worker fixtures reject missing/mismatched worker identities, attempt IDs, lease epochs, profile digests, command revisions, conditional-effect revisions, event sequences, bounds, and terminal acknowledgements. Cancel for an unseen attempt persists a tombstone; tombstoned or lower-revision dispatch is invalid before workspace creation. + - `not_requested`, `not_produced`, `valid`, and `invalid` cover success and failure without replacing the primary Task classification. `not_produced` is accepted only before candidate production; a selected `valid` or `invalid` survives later check/evidence/infrastructure failure, and only `valid` permits exactly one separate `allagents.structured-result` Artifact with the matching schema digest. - File evidence accepts create/edit/delete/rename and rejects unsafe paths, duplicate identities, oversized inline content, and inconsistent before/after forms. - **Verification:** Generated schemas are stable, public/private fixtures round-trip, digest vectors are cross-platform deterministic, and the private client/server fixture suite agrees before gateway or worker implementation. ### U2. Authentication and durable gateway repository -- **Goal:** Provide caller-scoped authentication, authorization, atomic Task/idempotency aggregates, Artifact storage, quota admission, pagination, restart fencing, logical expiry, and cleanup. +- **Goal:** Provide caller-scoped authentication, trusted-ingress configuration, retained-claim-first idempotency aggregates with original bindings, Artifact storage, new-claim quota admission, pagination, restart fencing, logical expiry, and cleanup. - **Requirements:** R4-R9, R13-R14, R17-R18, R22; AE2-AE4, AE6, AE8, AE10-AE13; KTD1, KTD3-KTD4, KTD12. - **Dependencies:** U1. - **Files:** `packages/execution-service/src/gateway/config.ts`, `packages/execution-service/src/gateway/auth.ts`, `packages/execution-service/src/gateway/store/gateway-repository.ts`, `packages/execution-service/src/gateway/store/file-gateway-repository.ts`, `packages/execution-service/tests/unit/gateway/auth.test.ts`, `packages/execution-service/tests/unit/gateway/file-gateway-repository.test.ts`. -- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Commit claim plus submitted Task in one manifest generation; publish immutable Artifact blobs before atomically switching the manifest to a new generation; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover and garbage-collect unreachable generations on startup. Startup recovery is a readiness barrier: invalidate every old fence and terminalize every nonterminal Task before admission, subscriptions, dispatch, or lease renewal begin. Reserve owner/global quotas before claims. Verify OIDC JWTs and constant-time static tokens before all repository access. +- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Resolve a retained claim after authentication and bounded parsing, and compare its stored canonical caller request plus original effective-profile/result-schema digests without consulting mutable current policy. For new work only, reserve owner/global quota and commit the claim, original bindings, and submitted Task in one manifest generation. Publish immutable Artifact blobs before one manifest switch; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover unreachable generations; and complete startup recovery before serving. Verify OIDC/static tokens before repository access and validate the configured named TLS ingress boundary before readiness. - **Execution note:** Implement concurrent-claim, transition-race, and crash-publication tests before request handling. Inject faults between blob, generation, manifest, tombstone, and cleanup operations. - **Patterns to follow:** `src/core/marketplace.ts` and `src/core/profile/files.ts` for atomic publication/recovery, `src/core/mcp-http-stdio-proxy.ts` for private files and loopback safety, and the official A2A `TaskStore` owner-scoping contract. - **Test scenarios:** - - Covers AE2-AE3. Concurrent identical claims create one aggregate; a conflicting digest returns conflict without dispatch permission. + - Covers AE2-AE3. Concurrent identical new claims create one aggregate; a conflicting original request/schema binding returns conflict without dispatch permission. Identical retained replay still returns the existing Task after its deadline, quota, authorization, readiness, or current profile changes, while an inconsistent stored binding fails closed. - Covers AE4. Load/list/cancel/subscribe/Artifact lookup scopes before path/database access and gives unknown, unauthorized, and expired IDs indistinguishable behavior. - Hostile/ambiguous issuer, tenant, subject, invocation key, Task ID, Artifact name, Unicode, case, delimiter, traversal, and Windows-reserved values cannot collide or become paths. - All standard list filters, `historyLength`, page size 1-100, omitted Artifacts, ordering, total size, and always-present next token match A2A semantics. Tokens are owner/query-bound and reject malformed, swapped, or stale filters. @@ -479,29 +498,31 @@ docs/src/content/docs/ - Restart, including repeated failure during startup recovery, completes the recovery barrier before serving: it fails every nonterminal Task once, invalidates fences, never renews an old lease or requests provider reattachment/replay, preserves terminal Tasks, and records cleanup unknown unless proven. - Covers AE13. Exact expiry tombstones the aggregate before cleanup; failed deletion never restores visibility; same-key replay before expiry returns the old Task and after expiry creates a new Task. - A crash between every aggregate publication step leaves either the prior or next valid manifest, never claim-without-Task or Task-with-missing-Artifact state. - - OIDC rejects wrong issuer, audience, signature, expiry, scope, tenant, and subject; static tokens and internal capabilities never appear in logs/errors. - - Quota-boundary races admit exactly the allowed count and preserve reserved capacity for cancel/terminal writes; low-space mode stops new claims without blocking settlement. - - Unauthenticated mode starts on loopback and refuses wildcard or non-loopback listeners. + - OIDC rejects wrong issuer, audience, signature, expiry, scope, tenant, and subject; static tokens and internal capabilities never appear in logs/errors. Production readiness rejects missing/mismatched named TLS termination, while unauthenticated plaintext remains loopback-only. + - Quota-boundary races admit exactly the allowed new claims and preserve reserved capacity for cancel/terminal writes; low-space mode stops new claims without blocking retained replay or settlement. + - Unauthenticated mode starts only on loopback and refuses wildcard or non-loopback listeners. - **Verification:** A fresh process retrieves prior records, fault recovery finds one valid aggregate generation, authorization cannot reveal neighboring owners, and expiry/quota behavior remains deterministic under concurrency. ### U3. A2A gateway server and fenced worker client -- **Goal:** Expose the accepted A2A profile while making admission, replay, streaming, lookup, worker fencing, failure, and cancellation use one durable state machine. +- **Goal:** Expose the accepted A2A profile while making extension negotiation, retained replay, new admission, streaming, lookup, authenticated worker routing, monotonic worker commands, failure, and cancellation use one durable state machine. - **Requirements:** R1-R9, R11, R14-R15, R17-R22; F1-F5; AE1-AE4, AE6, AE8-AE13; KTD1-KTD7, KTD9, KTD12. - **Dependencies:** U1, U2. - **Files:** `packages/execution-service/src/gateway/agent-card.ts`, `packages/execution-service/src/gateway/request-handler.ts`, `packages/execution-service/src/gateway/executor.ts`, `packages/execution-service/src/gateway/server.ts`, `packages/execution-service/src/gateway/worker-client.ts`, `packages/execution-service/tests/unit/gateway/agent-card.test.ts`, `packages/execution-service/tests/unit/gateway/request-handler.test.ts`, `packages/execution-service/tests/unit/gateway/executor.test.ts`, `packages/execution-service/tests/e2e/gateway-fake-worker.test.ts`. -- **Approach:** Mount the official HTTP+JSON and Agent Card handlers behind auth. Put an AllAgents `A2ARequestHandler` decorator above `DefaultRequestHandler` so admission and canonical Task reservation happen first, identical replay bypasses new SDK Task/bus allocation, and cancellation waits for worker terminal evidence. Persist each public state before emission. Dispatch one fenced attempt, validate sequence/fence on every worker event, renew its lease, and atomically publish Artifact blobs plus terminal manifest. +- **Approach:** Mount the official HTTP+JSON and Agent Card handlers behind trusted ingress and auth. Advertise the exact required URI; validate `A2A-Extensions`, `Message.extensions`, and `Message.metadata[uri]`; and publish the integrity envelope only through the standard Artifact carrier, never `Task.extensions`. The `A2ARequestHandler` decorator authenticates and bounded-canonicalizes, resolves the owner-scoped retained claim, and returns or conflicts against stored original bindings before current profile/deadline/quota/readiness checks. New requests then pass mutable admission and canonical Task reservation. Keep transition selection pure and execute fenced I/O outside it. Dispatch revisioned commands over mTLS/equivalent authenticated encryption or a same-host Unix socket, binding worker identity/capability/fence; atomically publish Artifact blobs plus the terminal manifest. - **Execution note:** Begin with an in-process fake worker and official A2A client. Prove operation errors versus accepted-Task failures, replay/subscribe behavior, fencing, cancellation races, and restart before adding providers. -- **Patterns to follow:** Official A2A sample `AgentExecutor`, `A2ARequestHandler`, `DefaultRequestHandler`, Express handlers, and cancellable-agent flow; `src/core/mcp-http-stdio-proxy.ts` for HTTP shutdown and loopback tests. +- **Patterns to follow:** Official A2A sample `AgentExecutor`, `A2ARequestHandler`, `DefaultRequestHandler`, Express handlers, and cancellable-agent flow; `src/core/mcp-http-stdio-proxy.ts` for HTTP shutdown and loopback tests; and Buzz's pure classifier/I/O reconciler split for transition selection without adopting its Kubernetes model. - **Test scenarios:** - - Covers AE1. `returnImmediately` true returns the submitted/working Task, false/unset waits for terminal state, and streaming starts with the same durable Task before ordered updates. - - Authentication, invalid extension/source/profile, expired deadline, and pre-claim quota failure return operation errors with no Task or worker request. + - Covers AE1. Agent Card negotiation, `A2A-Extensions`, `Message.extensions`, `Message.metadata[uri]`, and the fixed integrity Artifact pass through the official client; missing/mismatched carriers and `Task.extensions` fail. `returnImmediately` and streaming expose the same durable Task. + - New-admission authentication, invalid extension/source/profile, expired deadline, current-profile/readiness failure, and pre-claim quota failure return operation errors with no Task or worker request. - Covers AE11. Capacity loss after acceptance fails the retained Task at `dispatch/capacity_exhausted`; ambiguous dispatch fails `dispatch_unknown`; neither is retried. - - Covers AE2-AE3. Identical send/stream replay returns the existing Task and follows only future events if active; conflict returns the documented operation error. + - Covers AE2-AE3. Identical send/stream replay returns the existing Task before mutable checks even after the stored deadline or current profile changes; changed request/schema or inconsistent original binding conflicts. - Covers AE8. Active subscribe emits current snapshot then future events without missed-event replay; terminal subscribe errors and `GetTask` returns terminal truth. - Covers AE4. Get/list/subscribe/cancel/Artifact endpoints apply owner authorization consistently. - - Covers AE6. Cancel in submitted/working, cancel versus accept/completion, caller versus deadline, duplicate cancel, and terminal cancel each produce one linearized outcome and at most one worker abort. - - Covers AE12. Duplicate, out-of-order, malformed, wrong-fence, and late terminal events cannot overwrite Task state; stale facts go only to safe telemetry. + - Covers AE6. Cancel in submitted/working, cancel versus accept/completion, caller versus deadline, duplicate cancel, and terminal cancel each produce one linearized outcome and at most one worker abort. If dispatch send is paused after selection and a newer cancel completes first, releasing the stale dispatch cannot create a workspace or provider process. + - Covers AE12. Duplicate, out-of-order, malformed, wrong-identity, wrong-revision, wrong-fence, and late terminal events cannot overwrite Task state; stale facts go only to allowlisted metadata-only telemetry with opaque owner correlation. + - A failed fenced effect or changed observation causes a durable re-read and reclassification; the executor never substitutes a fresher fence or revision into an effect selected from stale state. + - Plaintext remote workers, wrong certificates, wrong configured worker identity/capability, and replayed attempt capabilities fail before dispatch; mTLS/equivalent protected routes and same-host Unix sockets succeed. - Known policy denial rejects only after stop/cleanup; unknown permission shape fails as adapter incompatibility. - Caller SSE disconnect and telemetry exporter failure leave execution and terminal lookup intact. - Graceful shutdown stops admission, claims cancellation for bounded active work, persists honest terminal state, and closes listeners. @@ -509,26 +530,28 @@ docs/src/content/docs/ ### U4. Worker protocol and safe workspace lifecycle -- **Goal:** Implement the supervised single-execution worker, hardened immutable Git acquisition, profile enforcement, leases, isolated backend roots, resource controls, race-resistant evidence, termination, and cleanup independent of any provider. +- **Goal:** Implement the supervised single-execution worker with authenticated transport, a minimal monotonic command record, hardened immutable Git acquisition, OS-enforced credential separation, leases, isolated roots, resource controls, race-resistant evidence, failed-quiescence recycling, and cleanup independent of any provider. - **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14. - **Dependencies:** U1. - **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`. -- **Approach:** Authenticate and fence the private protocol, reserve the one execution before workspace creation, validate profile/deployment capability, and emit sequenced NDJSON. Run the worker server inside a deployment-approved supervisor boundary that kills all invocation descendants if the server exits. Before readiness, inspect invocation manifests, require supervisor proof that prior descendants are dead, and delete or quarantine orphaned roots; an unprovable root blocks reuse and reports degraded readiness. Acquire source under KTD13. Create separate workspace and backend config/data roots with a scrubbed phase-specific environment. Run setup, baseline, adapter, and checks under enforced budgets. Stop and verify the process group before descriptor-based regular-file evidence staging, then clean in `finally`. Lease expiry self-cancels. +- **Approach:** Authenticate the configured worker identity and fence every private command. Persist one minimal monotonic command record scoped to worker identity/lease before workspace creation: unseen-attempt cancel writes a tombstone, stale/lower-revision dispatch is rejected, and each dispatch/cancel effect conditionally rechecks the stored revision immediately before mutation. Reserve one execution only after that check. Validate profile/deployment and OS provider/tool credential-boundary capabilities, then emit sequenced NDJSON. Keep transition selection pure. Run inside a dedicated container process namespace under init/reaper or an equivalent systemd/cgroup boundary. After bounded termination escalation, prove the complete invocation process set empty; if proof fails, persist termination unknown/failed, poison admission, and exit so the supervisor destroys the boundary. Replacement readiness proves boundary termination and reaps/quarantines owned roots. Use KTD13 acquisition, separate roots, phase environments, budgets, and descriptor-safe evidence; clean in `finally`. - **Execution note:** Characterize every phase with a fake adapter, malicious fixtures, and disposable Git servers before real providers. Fault-inject dispatch acknowledgement, events, leases, acquisition, processes, evidence publication, and cleanup. -- **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, and `tests/helpers/env.ts` for isolated state. +- **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, `tests/helpers/env.ts` for isolated state, and Buzz's bounded process-group/job-object cancellation as a lifecycle characterization checklist rather than copied code. - **Test scenarios:** - - Covers AE5. Exact object ID verifies; wrong/missing object, disallowed URL/host/address/port, credential-bearing URL, redirect, DNS rebinding, unsafe subdirectory, and fetch failure stop before adapter invocation. + - Covers AE5. Exact object ID verifies; wrong/missing object, disallowed URL/host/address/port, credential-bearing URL, redirect, DNS rebinding, unsafe subdirectory, fetch failure, and setup failure stop before adapter invocation. After worker acceptance each such source/setup failure emits the selected `Submitted -> Working -> Failed` public trace. - Repositories with LFS configuration/pointers, submodules, hooks, filters, alternates, proxy/helper config, or non-HTTPS secondary protocols cause no secondary connection or helper execution. - Source credentials leave no repository config, process argument, child phase environment, log, error, evidence, or retained workspace trace. - - Setup changes establish the baseline; setup, checks, and model-initiated tools receive no provider/control secrets; every backend gets disjoint invocation config/data roots with ambient selectors removed. + - Setup changes establish the baseline; setup and checks receive no provider/control secrets. Credentialed provider runtimes and model tools run across the declared OS UID/process/mount boundary or broker, with disjoint config/data roots and ambient selectors removed. - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. - - Covers AE14. Concurrency above one and hostile/cross-tenant trust claims are rejected; worker control credentials are absent from child environment and configured filesystem roots. - - Source pack/tree/file/inode/path/sparse-file/disk limits and setup/provider/check CPU, memory, PID, network, phase-time, and workspace limits stop only the invocation and preserve worker health. + - Block dispatch after effect selection, complete a newer cancel for the unseen attempt, then release dispatch: the command tombstone/revision check rejects it before workspace or provider creation. Duplicate commands remain idempotent and all effects stay fence-bound. + - Covers AE12. A descendant calls `setsid`, ignores graceful signals, and survives per-process-group escalation during cancellation and normal completion; the worker records termination unknown/failed, refuses another reservation, exits, and replacement readiness reaps or quarantines the orphaned root after supervisor boundary destruction. + - Covers AE14. Concurrency above one and hostile/cross-tenant trust claims are rejected. Credentialed readiness fails without the narrow OS provider/tool boundary, and model tools cannot inspect provider/control process environments, procfs/process listings, or configured backend roots. + - Source pack/tree/file/inode/path/sparse-file/disk limits and setup/provider/check CPU, memory, PID, network, phase-time, and workspace limits stop only the invocation; failed quiescence recycles the worker rather than claiming it remains healthy. - Covers AE9. Known permissions receive one-invocation decisions; prompt-required profiles fail startup; unknown permission types fail the adapter. - - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps; capture I/O or malformed result that breaks the kernel fails the Task. + - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps. A valid or invalid result selected before a later check/evidence failure is preserved, including the valid Artifact; only a pre-candidate failure records `not_produced`. - Background swap attacks, links, mount crossings, FIFOs/devices/sockets, unstable files, and tampering between worker staging and gateway publication never expose external bytes or partial Artifacts. - - Covers AE12. SIGKILL the worker before and after provider spawn with live descendants and a persistent invocation root; the supervisor proves descendant death, the replacement reaper deletes or quarantines the root before readiness, and the gateway retains one failed Task with separate termination and cleanup outcomes. -- **Verification:** A built supervised worker mutates a disposable exact-SHA repository through the fake adapter and proves fenced dispatch, source hardening, phase isolation, budgets, quiescence, evidence integrity, worker-crash containment, orphan-root handling, and cleanup from its emitted result plus supervisor proof. + - SIGKILL before and after provider spawn proves supervisor descendant death and replacement root recovery; the gateway retains one failed Task with separate termination and cleanup outcomes. +- **Verification:** A built supervised worker mutates a disposable exact-SHA repository through the fake adapter and proves authenticated revisioned dispatch, unseen-cancel tombstones, source hardening, OS credential separation, budgets, result preservation, quiescence or poisoned-boundary exit, evidence integrity, worker-crash containment, orphan-root handling, and cleanup. ### U5. Codex backend adapter @@ -536,19 +559,18 @@ docs/src/content/docs/ - **Requirements:** R10-R22; AE1, AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. - **Dependencies:** U4. - **Files:** `packages/execution-service/src/worker/adapters/codex.ts`, `packages/execution-service/tests/unit/worker/adapters/codex.test.ts`, `packages/execution-service/tests/fixtures/execution/codex-events.jsonl`. -- **Approach:** Depend directly on a pinned `@openai/codex-sdk` and fail worker readiness when it is unavailable or incompatible. Construct one fresh SDK thread in the invocation workspace with an isolated `CODEX_HOME` and a minimal allowlisted environment. Apply model, sandbox, network, approval, working-directory, writable-root, and a pinned `shell_environment_policy` only from the profile. The Codex runtime may receive its scoped provider credential, but model-initiated shell commands receive only named non-secret variables and never provider or worker-control credentials. Consume `runStreamed()`, pass the invocation `AbortSignal`, and pass the exact accepted result schema as per-turn `outputSchema`. Parse and validate the final JSON with the shared worker validator before publishing the canonical result Artifact. Normalize agent messages, items, usage, failures, and file-change events while preserving the bounded native stream. Never call `resumeThread`, pool threads, or reuse provider sessions after interruption. +- **Approach:** Depend directly on pinned `@openai/codex-sdk` and fail readiness when the runtime or configured credential boundary is unavailable. Create one fresh SDK thread with isolated `CODEX_HOME`. The credential-bearing Codex runtime runs on the provider side of the declared UID/process/mount boundary or obtains credentials through the configured broker; model-invoked commands run on the tool side and cannot inspect provider procfs/process entries or config/data roots. A minimal allowlisted environment and pinned `shell_environment_policy` remain defense in depth. Apply profile model/sandbox/network/approval/path policy, pass `AbortSignal` and exact `outputSchema`, validate final JSON with the shared validator, normalize bounded events/evidence, and never resume or pool threads. Once validation selects `valid` or `invalid`, later check/evidence/infrastructure failure preserves that state and the valid Artifact. - **Execution note:** Wrap the SDK behind an injectable factory and drive it through fixture events and its executable override before any credentialed smoke test. Use Promptfoo's provider and tests to enumerate observable edge cases, not as copied code or a runtime dependency. - **Patterns to follow:** `src/core/profile/adapters/codex.ts` for root/environment isolation, `src/core/native/codex.ts` for version checks, the SDK's `startThread`/`runStreamed`/`AbortSignal`/`outputSchema` and shell-environment policy contracts, and Promptfoo's Codex provider tests for characterization of option forwarding, environment isolation, cancellation, structured output, and cleanup. - **Test scenarios:** - A successful stream exposes thread ID, progress, final response, token usage, native file-change items, and terminal completion. - - A structured request forwards the exact accepted schema to `outputSchema`; valid JSON becomes the fixed-name Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`, while missing, malformed, or schema-invalid output fails with a typed structured-result error. + - A structured request forwards the exact accepted schema to `outputSchema`; valid JSON selects `valid` and produces the fixed-name `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; malformed or schema-invalid output selects `invalid` without the Artifact and reports the typed validation error; missing output selects `not_produced` and reports the typed missing-output error. - Empty final response, turn failure, malformed JSONL, non-zero exit, unavailable runtime, and usage omission map to typed result/completeness fields. - Covers AE6/AE12. A pre-aborted signal prevents start; in-flight cancellation aborts the SDK once; worker escalation proves descendant termination; completion after cancel or stale fence cannot alter the selected terminal outcome. - - Profile sandbox, network, model, approval, working directory, shell-environment policy, and environment settings reach the SDK; caller input cannot override them or supply raw Codex config. - - The Codex runtime receives only its scoped credential and minimal runtime environment. A model-initiated command that attempts to print provider/control credential names observes no values, and output, errors, and retained evidence contain none. + - The credential-bearing Codex runtime receives only its scoped credential and minimal environment. Adversarial model commands probing parent/sibling environments, `/proc` and process listings, known or discovered `CODEX_HOME`/backend roots, and outbound secret exfiltration cannot recover provider/control credentials; readiness fails when this OS boundary or broker is unavailable. - Two sequential invocations create fresh threads with disjoint `CODEX_HOME`, session state, and writable roots; no resume or thread-persistence API is called. - Native diffs and shared Git evidence coexist without claiming identical attribution. -- **Verification:** Fixture-driven tests cover every supported event and failure shape, SDK option/schema/signal forwarding, environment isolation, fresh-thread behavior, and cleanup escalation, followed by an isolated credentialed repository smoke test when Codex credentials are available. +- **Verification:** Fixture-driven tests cover every supported event/failure shape, SDK option/schema/signal forwarding, OS-enforced provider/tool separation plus environment defense in depth, fresh-thread behavior, result-state preservation, and cleanup escalation, followed by an isolated credentialed repository smoke test when prerequisites are available. ### U6. Pi backend adapter @@ -556,59 +578,66 @@ docs/src/content/docs/ - **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. - **Dependencies:** U4, U5. - **Files:** `packages/execution-service/src/worker/adapters/pi.ts`, `packages/execution-service/src/worker/adapters/pi-rpc.ts`, `packages/execution-service/src/worker/adapters/pi-policy-extension.ts`, `packages/execution-service/tests/unit/worker/adapters/pi.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-rpc.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-policy-extension.test.ts`, `packages/execution-service/tests/fixtures/execution/pi-events.jsonl`. -- **Approach:** Spawn a supported Pi 0.85.x runtime with `--mode rpc --no-session --no-extensions --no-builtin-tools`, an invocation-local `PI_CODING_AGENT_DIR`, profile model/provider, and scrubbed environment. Materialize the scoped provider credential only in Pi's supported invocation-local credential store with private permissions, not in the process environment. Explicitly load one worker-owned policy extension from outside the repository and verify the loaded extension/tool inventory before accepting work. The extension registers workspace-confined read/write/edit/search and sandboxed command tools plus the terminating result tool; command children receive only the phase allowlist and cannot access the Pi config/data roots. Implement an LF-only JSONL parser rather than Node `readline`. Correlate responses, wait for `agent_settled`, read messages/stats, send RPC abort, and escalate process-group termination after the grace period. For a structured request, generate the terminating tool from the exact accepted schema. The first observed tool call atomically claims the result candidate before validation: valid arguments produce the canonical Artifact; malformed or schema-invalid arguments fail the Task; later calls cannot replace the candidate. A prior cancel, deadline, or stale fence suppresses the call. Settling without a call fails as missing structured output. +- **Approach:** Spawn supported Pi 0.85.x in strict RPC mode with an invocation-local `PI_CODING_AGENT_DIR`, no sessions/extensions/built-ins, and one explicit worker-owned policy extension. The credential store and provider runtime stay on the provider side of the configured UID/process/mount boundary or credential broker; policy/command tools run on the tool side and cannot inspect the provider process, procfs entries, or Pi config/data roots. Verify exact tool inventory before work. Implement bounded LF JSONL, correlation, settlement/stats, abort and escalation. For a structured request, generate the terminating tool from the accepted schema; the first call atomically claims and validates the candidate, later calls cannot replace it, and later checks/evidence failures preserve its `valid` or `invalid` state and valid Artifact. - **Execution note:** Build parser, extension/tool-inventory, terminating-tool, policy-tool, and state-machine tests from captured RPC fixtures before process integration. Reuse the contract established by U5 rather than adding Pi-shaped public fields. - **Patterns to follow:** `src/core/native/pi.ts` for version/trust checks, `src/core/profile/adapters/pi.ts` for root isolation, and the official Pi RPC framing, `--no-extensions` plus explicit `--extension`, `--no-builtin-tools`, custom-tool, credential-store, and cancellation contracts. - **Test scenarios:** - Successful prompt acceptance streams message/tool events, stops on `agent_settled`, retrieves final messages/stats, and reports session ID, usage, and cost. - - A structured request exposes only the invocation-scoped terminating tool in addition to the policy tools. The first observed call claims the candidate; valid arguments produce the fixed-name Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; an invalid first call fails without replacement; a later duplicate cannot replace the result; a cancel/deadline/fence that wins first suppresses the call; and settled completion without a call fails as missing output. + - A structured request exposes only the invocation-scoped terminating tool in addition to the policy tools. The first observed call claims the candidate; valid arguments select `valid` and produce the fixed-name `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; an invalid first call selects `invalid` without replacement or an Artifact; a later duplicate cannot replace the result; later check/evidence/infrastructure failure preserves the selected state and valid Artifact; a cancel/deadline/fence that wins first suppresses the call; and settled completion without a call selects `not_produced` as missing output. - LF framing preserves `U+2028`/`U+2029` inside JSON strings, accepts CRLF by stripping trailing CR, handles partial/multiple chunks, and rejects oversized/malformed records. - Covers AE6/AE12. Cancellation sends RPC abort once, waits for idle, then terminates the process group only after grace; late settled or terminating-tool events cannot overwrite the terminal fence. - Prompt rejection, agent error, aborted stop reason, retry/compaction sequence, premature exit, stderr overflow, and stats failure map truthfully. - - Sequential invocations have disjoint `PI_CODING_AGENT_DIR`, tool registration, and session state; the provider credential exists only in the private invocation-local store; policy tools cannot read that root and their command children observe no provider/control credential; repository `.pi/extensions` and unrestricted built-in tools do not load; and caller input cannot send extension commands, steering/follow-up, arbitrary RPC commands, or override provider/model. -- **Verification:** Fixture and fake-process tests prove framing, correlation, deterministic terminating-tool selection, shared validation, exact extension/tool inventory, workspace confinement, command-environment and credential isolation, settled completion, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when Pi credentials are available. + - Sequential invocations have disjoint `PI_CODING_AGENT_DIR`, tool registration, and session state. Adversarial policy/command tools probing parent/sibling environments, procfs/process listings, known or discovered Pi/backend roots, and outbound secret exfiltration cannot recover provider/control credentials; readiness fails without the OS boundary or broker. Repository `.pi/extensions` and unrestricted built-ins do not load, and caller input cannot override provider/model or issue arbitrary RPC/extension commands. +- **Verification:** Fixture and fake-process tests prove framing, correlation, terminating-tool selection, shared validation, exact tool inventory, OS-enforced provider/tool separation plus environment defense in depth, result-state preservation, settlement, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when prerequisites are available. ### U7. Production registry, service packaging, and observability -- **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with safe startup, health, shutdown, tracing, and reproducible containers. -- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD11-KTD15. +- **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with trusted transports, peer identity, credential-boundary and supervisor readiness, safe startup/shutdown, tracing, and reproducible containers. +- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD10-KTD15. - **Dependencies:** U3-U6. - **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. -- **Approach:** Register only Codex and Pi through an explicit capability/availability map. Add gateway and supervised worker entrypoints inside the private Node 22 workspace instead of the Node 18 CLI package. Pin both runtimes as direct service dependencies, validate config, store, workers, runtime availability, supervisor boundary, orphan roots, trust, quotas, and resource controls before readiness, and never discover a missing provider only after Task acceptance. Propagate `traceparent` and instrument every phase. Build a minimal gateway image with no provider runtimes and a one-execution worker image whose init/runtime kills the complete execution boundary when the worker server exits. +- **Approach:** Register only Codex and Pi. Add gateway and supervised worker entrypoints inside the private Node 22 workspace. Before readiness, validate named public TLS termination, every remote worker's mTLS/equivalent transport and pinned identity/capabilities, Unix-socket locality, store, runtimes, monotonic command storage, OS credential-boundary capability, supervisor boundary, orphan roots, trust, quotas, and resource controls. Propagate `traceparent`, then apply KTD10's small shared metadata allowlist and bounded filtering/redaction before any structured log/span processor or OTLP exporter; neither OpenInference nor backend-native attributes bypass it. Build a minimal gateway image with no provider runtime and a one-execution worker image whose init kills the complete boundary when the worker server exits, including poisoned failed-quiescence exit. - **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. - **Patterns to follow:** `src/core/profile/adapters/registry.ts` for explicit adapter composition, root package scripts for workspace delegation, `src/core/mcp-http-stdio-proxy.ts` for server lifecycle, `.github/workflows/ci.yml` for quality gates, and `.github/workflows/publish.yml` for immutable releases. - **Test scenarios:** - Registry exposes exactly Codex and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects OpenCode or unknown backend IDs before workspace creation. - - Gateway and supervised worker start from built service outputs, become ready only after dependencies and orphan recovery pass, and stop gracefully on SIGTERM. - - Gateway readiness fails for malformed auth, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. - - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable supervisor/resource enforcement, unproved or unrecoverable orphan roots, or unavailable/incompatible Codex or Pi runtime. - - Killing the worker server while an adapter child and invocation root exist makes the supervisor kill the boundary; replacement readiness waits for root deletion or quarantine and never reuses it. - - Trace context enters through A2A, crosses the private call, and correlates result identities; exporter failure cannot change Task status. - - Gateway image contains no Codex, Pi, Git workspace, or provider credential material. - - Worker image pins both runtimes, confines one workspace/config root, excludes credentials from model tools, disables repository Pi extensions and unrestricted built-in tools, enforces deployment limits, and completes fake-provider health smoke tests. + - Gateway and supervised worker start from built outputs, become ready only after trusted transport/identity, credential and supervisor boundaries, dependencies, and orphan recovery pass, and stop gracefully on SIGTERM. + - Gateway readiness fails for malformed auth, missing/mismatched named TLS termination, plaintext production public ingress, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. + - Worker-route readiness fails for plaintext remote URL, wrong/untrusted certificate, worker identity/capability mismatch, or replayed capability; mTLS/equivalent authenticated encryption and same-host Unix sockets pass. + - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable OS credential/supervisor/resource enforcement, unproved or unrecoverable orphan roots, or unavailable/incompatible Codex or Pi runtime. + - Killing or poisoning the worker server while an adapter child and invocation root exist makes the supervisor destroy the boundary; replacement readiness waits for root deletion/quarantine and never reuses it. + - Trace context crosses the authenticated private call and correlates result identities using only opaque owner correlation. Exporter probes for agent, model, tool, stale-event, and error spans contain allowlisted bounded metadata but no canary secret, prompt/output, tool argument/result, file body/source fragment, raw caller identity, or cross-owner fragment; exporter failure cannot change Task status. + - Gateway image contains no Codex, Pi, Git workspace, provider credential material, or worker trust private keys. + - Worker image pins both runtimes, enforces provider/tool UID/process/mount separation or the credential broker, confines one workspace/config root, disables repository Pi extensions and unrestricted built-ins, enforces deployment limits, and completes fake-provider security probes. - Installing the root npm package on Node 18 does not load service dependencies; the private service workspace and containers enforce Node 22.19+. -- **Verification:** The registry dispatches both adapters through the same worker contract; built services and images pass lifecycle/security smoke tests; CI and publication bind immutable image tags to the release commit. +- **Verification:** The registry dispatches both adapters through the same worker contract; built services and images pass lifecycle/security smoke tests; exporter-capture tests prove pre-processor metadata allowlisting, bounded redaction, opaque owner correlation, and canary/cross-owner exclusion across agent, model, tool, stale-event, and error spans; CI and publication bind immutable image tags to the release commit. ### U8. Cross-backend conformance, documentation, and release evidence -- **Goal:** Prove the public contract and operational workflow end to end and document deployment without leaking backend details into callers. +- **Goal:** Prove standard A2A extension carriers, retained replay, trusted transport, monotonic cancellation, truthful result preservation, credential separation, metadata-only telemetry, worker recycling, trace-order conformance, and the shared backend contract end to end without leaking backend details into callers. - **Requirements:** R1-R22; F1-F5; AE1-AE14. - **Dependencies:** U1-U7. - **Files:** `packages/execution-service/tests/e2e/execution-gateway.test.ts`, `packages/execution-service/tests/fixtures/execution/conformance-cases.ts`, `examples/gateway/gateway.yaml`, `examples/gateway/worker.yaml`, `docs/src/content/docs/guides/execution-gateway.mdx`, `docs/src/content/docs/reference/execution-gateway-configuration.mdx`, `README.md`, `CHANGELOG.md`. -- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases. Exercise gateway and supervised worker as separate processes. Document extension/worker protocols, the portable result-schema subset, fixed structured-result Artifact, four result states, profiles, auth, trust boundary, storage/HA limits, explicit lack of execution resume, worker-crash supervision/orphan recovery, source hardening, quotas, runtime requirements, cancellation races, evidence integrity, Artifact access, retention, observability, and troubleshooting. +- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases, with gateway and supervised worker as separate processes. Each race fixture declares attempt/fence correlation, required durable transitions and observed effects, required happens-before edges, maximum occurrence counts, and effects forbidden after terminalization. A deliberately small test-side checker evaluates those constraints against durable records plus observed worker/process outcomes without calling the production selector. It remains coverage protection—not TLA+, a model checker, event sourcing, or a second lifecycle implementation. Document the exact extension URI and legal Agent Card/header/Message/Artifact carriers, both fixed Artifacts and four result states, retained-replay ordering, worker transport/identity, monotonic command tombstones, failed-quiescence recycling, the narrow OS credential boundary, reviewed-domain limitation, metadata-only telemetry and its separate operator access/retention, storage/HA limits, lack of execution resume, source hardening, quotas, retention, and operations. - **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. -- **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, and Starlight guide/reference organization under `docs/src/content/docs/docs/`. +- **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, Starlight guide/reference organization under `docs/src/content/docs/`, and Buzz's required-critical-action coverage rule without importing its TLA+ model or production implementation. - **Test scenarios:** - Covers AE1-AE14 through built services with a fake backend and official A2A client. - - The same accepted schema, valid result, invalid result, missing result, and pre-output failure fixtures pass through Codex and Pi adapters with identical validator decisions, integrity states, and Artifact presence/shape while retaining distinct native evidence. - - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; the one-execution worker serializes admitted work. - - Gateway restart, stream reconnect, lost dispatch acknowledgement, duplicate/out-of-order events, worker crash, lease expiry, cancellation race, provider failure, invalid structured output, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome without provider reattachment or replay. - - A worker SIGKILL with live descendants and an invocation root proves supervisor termination and pre-readiness deletion/quarantine; repeated gateway crashes cannot serve until startup terminalization completes. - - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, control-endpoint probing, provider-credential echo/read attempts, repository Pi extensions, unrestricted Pi built-in tools, and secret-exfiltration fixtures are blocked within the documented reviewed-source boundary. - - Examples validate with production schemas and reference secrets only through environment variable names. - - Docs state one gateway replica, one supervised execution per worker, reviewed mutual-trust sources, Node/runtime floors, ephemeral provider sessions, and no hostile-code isolation claim. - - Opt-in real-provider smoke tests record backend/runtime versions and skip only when the named credential/runtime prerequisite is absent. -- **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor, the full suites and docs pass, the official A2A client exercises every advertised operation, and release evidence records each available real backend plus explicit skipped prerequisites. + - Agent Card required-extension advertisement, `A2A-Extensions`, `Message.extensions`, request `Message.metadata[uri]`, and the single fixed integrity Artifact carrier interoperate; missing/mismatched carriers and `Task.extensions` fail. + - Identical retained replay after deadline expiry, quota exhaustion, readiness loss, authorization change, or profile replacement returns the original Task; changed request/schema or inconsistent original bindings conflict. + - The same accepted schema, valid result, invalid result, missing result, and pre-output failure pass through Codex and Pi with identical decisions. A valid-result-then-check-failure and invalid-result-then-evidence-failure preserve the selected state and only the valid Artifact; `not_produced` remains pre-candidate only. + - Source mismatch and setup failure after worker acceptance produce the selected `Submitted -> Working -> Failed` trace; provider invocation never begins, and durable snapshots, streams, and conformance records agree. + - Pause dispatch after selection, complete unseen-attempt cancel, then release dispatch; the stale command creates no workspace/process. The small independent checker enforces each fixture's attempt/fence correlation, happens-before edges, maximum counts, and forbidden post-terminal effects. Deliberately bad traces that still contain every required action name fail for wrong order, wrong fence, duplicate-over-maximum effects, and an extra stale dispatch after terminalization. + - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; one worker serializes admitted work. + - Gateway restart, reconnect, ambiguous dispatch, duplicate/out-of-order commands/events, worker crash, lease expiry, cancellation, provider failure, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome without provider reattachment or replay. + - A child that calls `setsid` and ignores graceful signals forces termination unknown/failed, poisoned-worker exit, supervisor boundary destruction, and replacement orphan recovery before readiness; no next reservation is accepted by the poisoned worker. + - Public plaintext, wrong TLS boundary, private plaintext, wrong certificate/worker identity, and capability replay fail readiness/dispatch; configured TLS, mTLS/equivalent overlay, and same-host Unix socket cases pass. + - Both adapters block model-tool probes of parent/sibling environments, procfs/process listings, known/discovered backend roots, and network secret exfiltration under the OS credential boundary. Environment filtering alone is never accepted as proof, and hostile-source/cross-tenant claims remain rejected. + - End-to-end exporter capture repeats the agent/model/tool/stale-event/error canary and cross-owner probes, proving only bounded allowlisted metadata and opaque owner correlation cross the telemetry boundary while Task/Artifact access and retention remain independent. + - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, repository Pi extensions, and unrestricted built-ins remain blocked within the documented reviewed-source boundary. + - Examples validate with production schemas and use only secret variable names. Docs state the two transport boundaries, one gateway replica, one execution per worker, reviewed trust domain, narrow credential isolation versus deferred hostile-code isolation, metadata-only telemetry with its fixed pre-processor allowlist and separate operator access/retention, runtime floors, and ephemeral provider sessions. + - Opt-in real-provider smoke tests record backend/runtime and credential-boundary prerequisites, skipping only when a named prerequisite is absent. +- **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor; full suites and docs pass; the official A2A client exercises every advertised operation including the `Submitted -> Working -> Failed` source/setup path; exporter capture proves the telemetry canary/cross-owner contract; the independent checker rejects all-name-present traces with wrong order/fence/multiplicity or forbidden stale dispatch; and release evidence records each available real backend plus explicit skipped prerequisites. --- @@ -616,18 +645,20 @@ docs/src/content/docs/ | Gate | Applies to | Required evidence | |---|---|---| -| Contract generation | U1 | Public extension and private worker schema generation report no drift; positive and negative fixtures pass. | -| Focused unit tests | U1-U7 | Active-unit tests pass with fault injection, state races, limits, cancellation, and cleanup. | -| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on fenced dispatch, sequencing, leases, Task persistence, Artifacts, shutdown, and cleanup. | -| Backend conformance | U5-U8 | One shared suite passes against Codex and Pi adapters with fixture runtimes, including identical acceptance and validation of the versioned result-schema subset, four result states, and fixed Artifact shape. | -| Credentialed provider smoke | U5-U6, U8 | Each available provider mutates a disposable exact-SHA repository; missing credentials/runtime are recorded as skipped prerequisites, never passing coverage. | -| A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, replay, cancel races, expiry, and owner isolation. | -| Security and abuse | U2-U4, U7-U8 | Malicious identity/source/artifact/resource fixtures prove auth-before-lookup, opaque owner keys, Git SSRF controls, phase-scoped secrets, provider-credential exclusion from model tools, disabled repository Pi extensions/built-ins, policy-tool confinement, quotas, quiescence, race-resistant capture, and trust-topology rejection. | -| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, worker-crash containment/orphan recovery, and both container builds pass. | +| Contract generation | U1 | Exact extension URI/carriers, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | +| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, and cleanup. | +| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | +| Backend conformance | U5-U8 | One shared suite passes against Codex and Pi, including the versioned schema subset, four result states, valid/invalid preservation across later failure, integrity Artifact carrier, and structured-result Artifact rule. | +| Credentialed provider smoke | U5-U6, U8 | Each available provider mutates a disposable exact-SHA repository while adversarial tool probes cannot cross the OS credential boundary; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | +| A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes required-extension negotiation and legal carriers, immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, retained replay, cancel races, expiry, and owner isolation without `Task.extensions`. | +| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, Git SSRF controls, OS provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | +| Lifecycle trace conformance | U8 | The small test-side checker, independently of production selectors, validates attempt/fence correlation, required happens-before edges, maximum occurrence counts, and forbidden post-terminal effects against durable records plus observed worker/process outcomes; all-name-present bad traces fail for wrong order/fence/multiplicity and stale post-terminal dispatch. | +| Telemetry safety | U7-U8 | Exporter capture across agent, model, tool, stale-event, and error spans proves the pre-processor allowlist and bounded redaction exclude prompt/output/tool/source/file content, canary secrets, raw identities, and cross-owner fragments while retaining only bounded operational metadata and opaque owner correlation. | +| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both container builds pass. | | Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | | Documentation | U8 | `bun run docs:build` passes and examples validate against current schemas. | -The authoritative behavioral proof is the built-process E2E path with the official A2A client and a separately started worker. Unit tests alone do not prove protocol, durable aggregation, process isolation, fencing, cancellation, or cleanup integration. +The authoritative behavioral proof is the built-process E2E path with the official A2A client and a separately started worker. Unit tests alone do not prove extension carriers, retained-replay ordering, trusted transport, durable aggregation, monotonic worker commands, credential/process isolation, cancellation, boundary recycling, cleanup integration, or telemetry export safety. The deliberately small independent trace checker supplements that path only by rejecting ordering, fence, multiplicity, and post-terminal-effect violations; it is not a production lifecycle model. --- @@ -636,23 +667,26 @@ The authoritative behavioral proof is the built-process E2E path with the offici ### Global - Every R1-R22 requirement is implemented or explicitly shown in a passing conformance scenario. -- Public Agent Card/extension and private worker schemas are stable, generated from one source, and consumable without importing root AllAgents CLI modules. -- Codex and Pi pass the same backend conformance suite, accept the same versioned result-schema subset, validate with the same shared validator, publish the same fixed structured-result Artifact shape and result states, and preserve bounded native evidence through the closed registry. +- The exact required extension is advertised and negotiated through standard Agent Card/header/Message/Artifact surfaces; requests live only at `Message.metadata[uri]`, terminal integrity lives only in the fixed integrity Artifact, and no `Task.extensions` exists. +- Codex and Pi pass the same backend conformance suite, schema subset, and validator. Every terminal Task publishes the integrity Artifact; selected `valid`/`invalid` states survive later failures, and only `valid` publishes the separate fixed structured-result Artifact. - Gateway and supervised worker run as separate Node 22 processes/images; the Node 18 root CLI does not import service dependencies, and the gateway has no provider runtime or writable repository. -- Authentication precedes lookup, quota precedes Task creation, aggregate commits cannot split claims/Tasks/Artifacts, startup recovery completes before serving, and terminal fences survive races and restart without claiming provider-session recovery. -- Cancellation/deadlines reach one native abort, process termination, quiescence, evidence, and cleanup for both backends; worker-process death triggers supervisor termination and pre-readiness orphan deletion or quarantine. -- Source hardening, phase-scoped secrets, provider-credential exclusion from model tools, disabled repository Pi extensions/unrestricted built-ins, Pi policy-tool confinement, one-execution trust policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end. +- Authentication and bounded parsing precede owner-scoped retained lookup; identical replay uses stored original bindings before mutable admission, while current authorization/profile/readiness/deadline and quota apply only to atomic new claims. +- Production public ingress uses its named TLS boundary, remote worker routes authenticate and encrypt peers with worker identity/capability binding, and same-host Unix sockets are the only non-network alternative; unprotected remote endpoints fail readiness. +- Cancellation/deadlines use monotonic worker command tombstones and one native abort. Stale dispatch cannot create work, and failed quiescence poisons and exits the worker so supervisor destruction and replacement orphan recovery precede new admission. +- Source hardening, the OS-enforced provider/tool credential boundary, phase-scoped secrets, disabled repository Pi extensions/unrestricted built-ins, one-execution reviewed-domain policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end without claiming hostile-source/cross-tenant isolation. +- Metadata-only telemetry is filtered through the fixed allowlist and bounded redaction before processing/export; canary secrets, content, raw caller identities, and cross-owner fragments never reach exporters, and only opaque owner correlation crosses the separately governed operator boundary. +- Required source/setup and race traces satisfy attempt/fence, happens-before, maximum-count, and forbidden-post-terminal constraints in the independent test-side checker; all-name-present malformed traces fail without introducing a parallel lifecycle implementation. - Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. -- Public documentation states supported topology, configuration, security boundary, storage/HA limitation, runtime pins, structured-result contract, worker-crash recovery, and deferred capabilities. +- Public documentation states extension carriers, retained replay, trusted transports, credential versus hostile-code boundaries, metadata-only telemetry and its separate operator access/retention, topology, storage/HA limitation, runtime pins, result preservation, poisoned/crashed-worker recovery, and deferred capabilities. - Abandoned experiments, unused adapters, compatibility shims, generated scratch files, retained test workspaces, and stale documentation are removed. ### Per unit -- U1: Public/worker schemas, result-schema subset, structured-result Artifact/states, digest vectors, fence rules, typed failures, and fixtures are generated and stable. -- U2: Auth, opaque owner isolation, aggregate idempotency, CAS settlement, pagination, startup recovery barrier, quotas, Artifact access, tombstones, and cleanup pass fault injection. -- U3: Every advertised A2A operation agrees across stream and lookup while replay, fencing, and cancellation races preserve one Task. -- U4: Supervision, orphan recovery, worker dispatch/source/setup/action/check/quiescence/evidence/cleanup lifecycle pass malicious, crashed, and faulted disposable-repository scenarios. -- U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, usage, native evidence, minimal environment, model-tool credential exclusion, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. -- U6: Pi strict JSONL framing, deterministic terminating-tool output, exact policy-extension/tool inventory, disabled repository extensions and built-ins, credential-store and workspace confinement, settled completion, stats, isolated roots, abort, and process cleanup pass adapter and applicable smoke verification. -- U7: Closed registry, Node-version separation, runtime readiness, tracing, graceful shutdown, containers, and release artifacts work from built outputs. -- U8: Cross-backend E2E, A2A interoperability, abuse cases, examples, operator docs, changelog, and release evidence are complete. +- U1: Standard extension carriers, integrity/structured-result Artifact schemas, four result states, original claim digests, command revisions/tombstones, fence rules, typed failures, and fixtures are generated and stable. +- U2: Trusted ingress, auth, opaque owner isolation, retained-claim-first replay, original bindings, atomic new admission, CAS settlement, pagination, startup recovery, quotas, Artifact access, tombstones, and cleanup pass fault injection. +- U3: Every advertised A2A operation agrees across stream and lookup while extension negotiation, replay ordering, authenticated worker routes, fencing, monotonic cancellation, and races preserve one Task. +- U4: Worker command state, OS credential separation, supervision, poisoned-exit/orphan recovery, and dispatch/source/setup/action/check/quiescence/evidence/cleanup pass malicious, crashed, and faulted scenarios. +- U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, result preservation, OS credential separation, native evidence, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. +- U6: Pi strict RPC/framing, terminating result, exact policy tools, disabled repository extensions/built-ins, OS-isolated credential store/provider runtime, result preservation, settlement, stats, abort, and process cleanup pass verification. +- U7: Closed registry, trusted transport/identity readiness, credential/supervisor capability gating, poisoned-worker recycling, metadata-only pre-export telemetry controls, Node-version separation, tracing, shutdown, containers, and release artifacts work from built outputs. +- U8: Cross-backend E2E, standard A2A carriers, retained replay, selected source/setup transitions, independent race-trace constraints, telemetry canary/cross-owner probes, transport and credential abuse cases, command/quiescence races, examples, operator docs, changelog, and release evidence are complete. diff --git a/docs/research/agent-host-protocol-decision-inputs.md b/docs/research/agent-host-protocol-decision-inputs.md index a3ebebc9..113ea6fc 100644 --- a/docs/research/agent-host-protocol-decision-inputs.md +++ b/docs/research/agent-host-protocol-decision-inputs.md @@ -11,9 +11,9 @@ AHP does not replace ADR 0002's Task identity, caller-scoped idempotency, authorization, immutable source handling, cleanup, terminal evidence, or bounded result retention. -The initial backend set is Codex, OpenCode, and Pi. They are peer execution -adapters behind one conformance contract; provider-specific process, session, -permission, cancellation, and evidence behavior stays below that seam. +The initial backend set is Codex and Pi; OpenCode is deferred. They are peer +execution adapters behind one conformance contract; provider-specific process, +session, permission, cancellation, and evidence behavior stays below that seam. This note records the AllAgents-specific consequences. The reusable research, source inspection, and full protocol comparison live in the AI Research Wiki: From 048312d26b629fddcab8372dfa9f4f31711e4990 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Fri, 18 Sep 2026 20:31:43 +1000 Subject: [PATCH 06/44] docs(architecture): define registered workspace materializers --- ...-agent-execution-through-an-a2a-gateway.md | 175 +++++- ...0837-feat-coding-execution-gateway-plan.md | 585 ++++++++++++++++-- .../harbor-repository-materialization.md | 179 ++++++ 3 files changed, 855 insertions(+), 84 deletions(-) create mode 100644 docs/research/harbor-repository-materialization.md diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index 599d72e8..d8deb818 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -19,10 +19,12 @@ provenance. Future clients may need the same execution boundary without Promptfoo or evaluation semantics. A coding-agent execution is more than a model request. It includes immutable -source selection, repository acquisition, environment setup, credentials, -permissions, agent invocation, cancellation, evidence capture, process -termination, and cleanup. Those responsibilities need one public contract while -allowing materially different execution backends. +workspace selection, repository or snapshot acquisition, environment setup, +credentials, permissions, agent invocation, cancellation, evidence capture, +process termination, and cleanup. A workspace may contain multiple repositories +or be produced from a digest-pinned snapshot or organization-specific source. +Those responsibilities need one public contract while allowing materially +different execution backends and acquisition mechanisms. The contract must not turn AllAgents into an evaluation harness. Dataset expansion, repetition, assertions, scoring, experiment scheduling, and durable @@ -60,10 +62,10 @@ process, session, structured-output, cancellation, and evidence behavior remains behind its adapter. OpenCode and other coding agents remain possible follow-up adapters rather than part of the first delivery. -Execution backends own repository materialization, environment setup, agent +Execution workers own workspace materialization, environment setup, agent invocation, evidence collection, process termination, and cleanup. The gateway -must not execute evaluated agents or mount their writable repositories in the -gateway process. +must not execute evaluated agents, run materializer images, or mount writable +workspaces in the gateway process. When deployed on Kubernetes, the gateway runs as its own Deployment and ClusterIP Service, separate from consumers and execution workers. A backend @@ -75,6 +77,96 @@ A separate gateway Pod is a service and failure boundary, not per-invocation security isolation. Deployments requiring hostile-code or tenant isolation must create or select a stronger execution boundary behind the gateway. +### Make workspace materialization explicit and operator-registered + +The public extension represents one workspace as a closed discriminated union. +The allowed `kind` values and shapes are: + +1. `repositories`, with a bounded list of direct Git repositories, each with a + canonical credential-free HTTPS URL, full commit object ID, collision-free + relative destination, and optional repository-relative subdirectory; +2. `workspaceSnapshot`, with an OCI workspace snapshot referenced by manifest + digest and accompanied by the versioned AllAgents workspace manifest; or +3. `materializer`, with an operator-registered materializer ID, an expected + workspace-manifest digest, and bounded structured inputs. + +Fields from another union variant are invalid. + +The third mode supports organization-specific acquisition such as JFrog, +generated sources, or custom monorepo assembly without accepting executable +configuration from the caller. The request cannot supply a builder image, +Dockerfile, Compose file, shell command, credential, mutable image tag, network +policy, or output contract. + +Direct Git and OCI acquisition revalidate scheme, normalized host, resolved +address, port, and redirect policy for every connection. OCI foreign or +external layer URLs are rejected by default, and registry credentials are never +forwarded across origins. + +Each materializer ID is defined in an operator-owned deployment registry. The +gateway receives only its non-secret descriptor: ID, bounded input schema, +expected definition digest, expected output-manifest version, and required +worker capabilities. The worker receives the runtime definition, which +additionally pins an OCI image by digest and fixes credential handle names or +mount identities, allowed network destinations, resource and phase deadlines, +cache policy, and the OCI runner or sandbox capability. Credential values are +not part of either descriptor. + +The worker computes an algorithm-qualified definition digest over a versioned, +domain-separated canonical serialization of every non-secret, +behavior-affecting runtime field. The expected workspace-manifest digest +likewise identifies the canonical bytes of one declared manifest version. An +execution profile explicitly allows source modes and materializer IDs and +authorizes canonical Git repositories or namespaces, OCI namespaces, and +resource selectors inside structured materializer inputs. At readiness the +gateway matches its expected descriptor digest and profile against the +authenticated worker's computed digest and capabilities. At admission it +validates the selected ID, expected output digest, structured inputs, and +resource authorization; the worker resolves the same definition locally and +rejects missing, changed, or unsupported definitions before acquisition. + +Every source mode produces the same versioned workspace manifest. The manifest +separates worker-verified observations from materializer-attested claims and +records the verification method for each identity. It includes requested and +resolved commits or OCI digests, destinations, materializer identity and image +digest when applicable, normalized input and output digests, resulting tree +identities, and completeness. Materializer assertions are not described as +independently verified unless the worker or a configured trusted acquisition +service performs that verification. + +The worker materializes into a worker-owned staging directory under the same +filesystem publication root as the final workspace; readiness rejects a +cross-filesystem layout and publication never falls back to copy-then-delete. +After validating paths, file types, limits, identities, and the manifest, the +worker stops the materializer and removes its credential, process, mount, and +runner boundary. The validated host-owned staging tree remains. The worker then +atomically renames that tree into its final location before profile-owned setup +or any coding agent starts. + +The registered image is part of the deployment's trusted computing base. The +worker launches it through a configured OCI runner or sandbox in a boundary +separate from the agent runtime and never exposes that runner's control socket +to setup or model tools. Phase isolation prevents later code from receiving the +materializer's credentials or mounts, but it cannot make a malicious +operator-registered image safe from credentials intentionally given to it. +Operators must review and pin that image; deployments that do not trust it need +a credential broker or stronger acquisition service that never reveals reusable +credentials to the materializer. + +The canonical source request enters caller idempotency. The resolved +materializer definition digest enters the effective-profile binding, and both +the definition and output-manifest digests enter terminal provenance. New-claim +source authorization always runs before cache lookup. Cache metadata and keys +include the canonical source, materializer-definition digest, authorization +scope digest and revocation epoch, and configured trust domain. Reuse requires +manifest and content revalidation under the current authorization scope; +revocation advances the epoch and makes the old namespace unusable. + +This keeps Harbor's useful separation between content-addressed task acquisition +and environment execution without adopting task-owned opaque source. The +comparison is recorded in +[Harbor repository materialization lessons](../research/harbor-repository-materialization.md). + ### Persist Task truth, not live provider execution The gateway durably stores Task identity, idempotency claims, terminal status, @@ -265,30 +357,39 @@ adopted by this decision. ### Make execution provenance and cleanup explicit -The gateway and selected backend are collectively responsible for: - -1. resolving and verifying immutable source identity; -2. acquiring or restoring source through the selected transport; -3. creating a fresh working location for one execution attempt; -4. running setup before the evaluated agent action; -5. applying permissions and execution isolation; -6. invoking the agent and propagating cancellation and deadlines; -7. capturing bounded output, usage, cost, file changes, checks, and artifact - references; -8. returning terminal status, evidence completeness, and provenance; and -9. terminating processes and releasing or retaining resources according to the - documented lifecycle. - -Source transport and runtime transport are independent. A backend may use one -immutable runtime image plus a separately digest-addressed source artifact; the -contract does not require source code to be baked into the runtime image. +The gateway and selected worker are collectively responsible for: + +1. validating one canonical immutable workspace request and the selected + profile's exact source-resource or materializer authorization; +2. acquiring direct repositories, restoring a digest-pinned OCI snapshot, or + running the registered materializer in a phase-scoped boundary; +3. producing and validating the standard workspace manifest; +4. transferring the validated staging tree to worker ownership, destroying the + acquisition process/mount/credential boundary, and proving it gone; +5. atomically publishing the host-owned tree on the same filesystem; +6. running profile-owned setup before the evaluated agent action; +7. applying permissions and execution isolation; +8. invoking the agent and propagating cancellation and deadlines; +9. capturing bounded output, usage, cost, file changes, checks, artifact + references, workspace identity, and materializer provenance; +10. returning terminal status, evidence completeness, and provenance; and +11. terminating processes and releasing or retaining resources according to + the documented lifecycle. + +Source transport, materializer image, workspace snapshot, and harness runtime +are independent identities. A backend may use one immutable runtime image plus +a separately digest-addressed workspace artifact; the contract does not require +source code to be baked into the runtime image. Credentials remain deployment policy. Requests must not embed deployment -credentials. The gateway authenticates callers, and the selected backend scopes -source and model credentials to the execution boundary without returning -secret-bearing paths or values. Provider and worker-control credentials must -also be absent from model-initiated command environments, tool output, retained -evidence, and repository-visible configuration. +credentials. The gateway authenticates callers, and the selected worker scopes +source credentials to materialization and model credentials to provider +execution without returning secret-bearing paths or values. Materialization +credentials are absent from profile setup, the harness, model-initiated command +environments, tool output, retained evidence, and the published workspace. +Provider and worker-control credentials must likewise be absent from +model-initiated command environments, tool output, retained evidence, and +repository-visible configuration. Retries must not multiply non-idempotent agent execution. Every request carries a caller-scoped stable invocation key through the AllAgents extension. The @@ -322,6 +423,14 @@ but their product and ownership model requires a separate decision. - Gateway and execution workers scale and fail independently. - The gateway can remain lightweight; physical isolation and resource policy belong to the selected execution backend. +- Custom acquisition remains available without making caller-supplied code part + of the trust boundary: operators register digest-pinned materializers and + profiles decide which callers may select them. +- Direct Git, OCI snapshots, and registered materializers converge on one + validated workspace manifest and provenance contract. +- Deployments that enable external materializers must operate their image, + schema, credential, network, resource, and cache policies as worker + configuration. - A2A supplies discovery and lifecycle semantics. AllAgents supplies the coding-specific evidence contract. - W3C Trace Context, OpenTelemetry/OTLP, OpenInference, optional ATIF, and the @@ -362,6 +471,14 @@ Rejected because Harbor's formats own benchmark orchestration, verification, and persisted runner state. The AllAgents gateway executes one coding-agent request and does not become an evaluation harness. +### Let callers provide repository-acquisition code + +Rejected because a caller-selected image, Dockerfile, Compose file, or shell +script would turn request parsing into privileged code execution and would make +credential, network, provenance, and cache policy unreviewable. Callers may +select only source modes and materializer IDs explicitly registered and allowed +by the effective execution profile. + ### Replace A2A with the Agent Host Protocol Rejected because AHP explicitly targets synchronization of independent clients diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index d373b358..86c83707 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -13,11 +13,19 @@ execution: code ## Goal Capsule -- **Objective:** External systems can run Codex or Pi against an immutable repository revision through one authenticated, cancellable, evidence-preserving remote contract. +- **Objective:** External systems can run Codex or Pi against an immutable, + provenance-bearing workspace assembled from exact Git repositories, a + digest-pinned OCI snapshot, or an operator-registered materializer through + one authenticated, cancellable, evidence-preserving remote contract. - **Means:** Add a separately deployable A2A 1.0 gateway, a private worker protocol, and backend-neutral workers with two direct provider adapters (KTD1, KTD5, KTD7-KTD8). - **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) owns the public boundary. The A2A 1.0 specification owns core wire semantics. The versioned AllAgents extension owns coding-execution semantics. - **Execution profile:** Build contract-first, then durable Task/evidence state, worker lifecycle, Codex, Pi, packaging, and cross-backend conformance. Preserve the existing local CLI and Node 18 package compatibility. -- **Stop conditions:** Do not execute agents in the gateway process, accept mutable source identity, put deployment credentials in requests, treat streams or telemetry as terminal evidence, treat provider sessions as recovery checkpoints, vendor an evaluator's provider implementation, or add evaluation behavior. +- **Stop conditions:** Do not execute agents or materializers in the gateway + process, accept mutable source identity, accept caller-supplied acquisition + code or credentials, put deployment credentials in requests, treat streams + or telemetry as terminal evidence, treat provider sessions as recovery + checkpoints, vendor an evaluator's provider implementation, or add + evaluation behavior. - **Tail ownership:** The implementing workflow runs focused contract and lifecycle tests, the complete repository quality gates, isolated gateway/worker smoke tests, provider-specific credentialed smoke tests where credentials are available, and documentation validation. --- @@ -38,9 +46,15 @@ The two initial runtimes expose different programmatic contracts. Codex provides - A1. **Gateway caller:** An authenticated service such as AI Evals that creates, observes, lists, cancels, and retrieves coding-execution Tasks. - A2. **Execution gateway:** The A2A server that owns caller scope, Task identity, idempotency, routing, retention, and normalized results. -- A3. **Execution worker:** A separately deployed process that owns source materialization, one mutable workspace per invocation, provider execution, evidence capture, and cleanup. -- A4. **Backend adapter:** The Codex or Pi integration that translates native events, structured results, cancellation, usage, failures, and evidence into the worker contract. -- A5. **Operator:** The person or deployment system that defines profiles, credentials, limits, retention, worker endpoints, and observability policy. +- A3. **Execution worker:** A separately deployed process that owns registered + workspace materialization, one mutable workspace per invocation, provider + execution, evidence capture, and cleanup. +- A4. **Backend adapter:** The Codex or Pi integration that translates native + events, structured results, cancellation, usage, failures, and evidence into + the worker contract. +- A5. **Operator:** The person or deployment system that defines profiles, + materializer registrations, credentials, limits, retention, worker + endpoints, and observability policy. ### Key Decisions @@ -48,6 +62,10 @@ The two initial runtimes expose different programmatic contracts. Codex provides - **Keep execution outside the gateway process.** Mutable repositories and provider processes belong to workers. Governs R10-R16, R21-R22. - **Persist Task truth, not live executions.** Accepted Task identity and terminal evidence survive restart; provider sessions do not resume or replay. Governs R7, R14, R16-R18. - **Keep evaluation outside AllAgents.** Dataset expansion, repetitions, assertions, scoring, retries, and durable evaluation Runs remain caller concerns. Governs R20. +- **Make workspace acquisition explicit but extensible.** Requests select one + versioned workspace source mode; custom acquisition uses only + operator-registered, digest-pinned materializers allowed by the profile. + Governs R6, R11-R13, R16-R18, R21-R22. ### Requirements @@ -70,15 +88,78 @@ The two initial runtimes expose different programmatic contracts. Codex provides - R10. Codex and Pi are the complete initial backend set behind one conformance contract, delivered Codex first and Pi second. OpenCode is deferred. (session-settled: user-directed.) - R11. A request selects a server-defined execution profile and may include one `allagents.result-schema/v1` schema for the terminal result: a bounded JSON Schema Draft 2020-12 subset with an object root, every object schema setting `additionalProperties: false`, every declared property listed in `required`, optional values represented by `null` unions, and only `type`, `properties`, `required`, `additionalProperties` with the value `false`, `items`, `enum`, `const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`. The extension version fixes byte, depth, property, and enum limits; admission rejects remote references, format-dependent validation, and unknown keywords; one shared validator governs schema admission and returned values. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. Requests cannot supply raw provider configuration. -- R12. The only initial remote source form is a canonical credential-free HTTPS Git URL plus full commit object ID and optional repository-relative subdirectory. Acquisition revalidates destination policy for every connection, disables redirects and repository-controlled secondary fetch/exec features, uses hermetic Git configuration, and verifies that the fetched object is the requested commit before setup. +- R12. One request defines one workspace using exactly one closed source union: + `{ kind: "repositories", repositories: [...] }`, + `{ kind: "workspaceSnapshot", reference, workspaceManifestDigest }`, or + `{ kind: "materializer", materializerId, + expectedWorkspaceManifestDigest, inputs }`. Unknown kinds, fields from + another variant, and omitted variant fields fail admission. Repository + entries contain a canonical credential-free HTTPS Git URL, full commit object + ID, collision-free relative destination, and optional repository-relative + subdirectory. Snapshot references are digest-pinned OCI artifacts containing + the versioned workspace manifest. Materializer inputs are bounded by the + registered schema. Profiles explicitly allow source modes and materializer + IDs and authorize exact canonical Git repositories or namespaces, OCI + namespaces, and resource selectors inside materializer inputs. Callers cannot + supply builder images, Dockerfiles, Compose files, shell commands, + credentials, mutable image tags, network policy, or output contracts. Direct + Git revalidates destination policy for every connection, disables redirects + and repository-controlled secondary fetch/exec features, uses hermetic Git + configuration, fetches into an isolated object database from the approved + remote, and verifies that the checked-out commit equals the requested full + object ID. OCI acquisition rejects external or foreign layer URLs by default, + revalidates scheme, normalized host, resolved address, port, and redirects + for registry, authentication, manifest, and blob connections, never forwards + credentials across origins, and verifies every manifest and layer digest. A + materializer output must match the request's expected workspace-manifest + digest before publication. - R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, structured logs/spans before processing or export, and every model-initiated command or tool environment. Credentialed profiles additionally require an OS-enforced provider/tool credential boundary: the credential-bearing provider runtime and model-invoked tools use distinct UID/process/mount policy that prevents tool access to provider processes, procfs entries, and backend config/data roots, or an equivalent credential broker keeps reusable credentials out of the agent runtime. Worker readiness fails when the declared boundary cannot be proved; environment filtering alone is not credential isolation. + Materializer IDs are defined in an operator-owned deployment registry. The + gateway holds only the non-secret ID, bounded input schema, expected + definition digest, expected output-manifest version, and required worker + capabilities; the worker holds the runtime definition with the digest-pinned + image, credential handle names or mount identities, network destinations, + resource/deadline ceilings, cache policy, output-manifest version, and OCI + runner or sandbox capability. Credential values are excluded. The worker + derives an algorithm-qualified `sha256:<64 lowercase hex>` definition digest + from a versioned, domain-separated canonical serialization of every + non-secret behavior-affecting field and advertises it at readiness; the + gateway treats its copy only as the expected digest. The expected workspace + manifest and canonical materializer-input digests use the same + algorithm-qualified format with distinct domain separators and exact + versioned canonical JSON preimages. Readiness fails when computed and expected + descriptors differ across the authenticated route. Materialization + credentials exist only in that isolated phase and are not supplied to setup, + provider execution, or model tools. The registered image is operator-trusted + deployment code: phase isolation protects later phases but cannot make a + malicious registered image safe from credentials deliberately given to it. + Deployments requiring that stronger claim use a credential broker or + acquisition service that withholds reusable credentials. - R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. - R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. - R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves the complete invocation process set quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. If bounded termination escalation cannot prove quiescence, the worker persists termination as unknown/failed, poisons admission, and exits so the external supervisor destroys the complete process boundary; replacement readiness performs orphan recovery before accepting work. The same supervisor boundary handles a worker crash. + Every source mode materializes into a worker-owned staging directory under + the same filesystem publication root as the final workspace and produces the + same versioned workspace manifest. Readiness rejects cross-filesystem roots + and publication has no copy-then-delete fallback. The worker validates + repository or snapshot identities, destinations, paths, file types, limits, + materializer definition and image digests, output digest, provenance method, + and completeness. It then terminates the supervisor-owned acquisition + process, mount, runner, and credential boundary while preserving the + host-owned validated staging tree, proves that boundary gone, atomically + renames the tree into its final location, and only then starts setup. **Evidence and observability** - R17. Every terminal Task contains the required `allagents.execution-integrity` Artifact carrying an integrity kernel: Task/source/profile/backend identities, action outcome, a structured-result state of `not_requested`, `not_produced`, `valid`, or `invalid` plus reason and schema digest when requested, cancellation or failure classification, separate termination and filesystem-cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. A valid structured result is exactly one additional `allagents.structured-result` Artifact with one A2A `Part` whose `data` field contains the validated result object and whose `mediaType` is `application/json`; missing or invalid result data never publishes that Artifact. `not_produced` is legal only before a result candidate is produced. Once validation selects `valid` or `invalid`, later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the fixed structured-result Artifact while the later phase remains the primary Task failure classification. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. + Workspace provenance includes the source mode, requested and resolved + repository commits or OCI digests, destination map, workspace-manifest + digest, and, when applicable, materializer ID, computed definition digest, + image digest, canonical input digest, and output digest. It labels each field + as a worker-verified observation, trusted-service verification, or + materializer-attested claim and records the verification method; a custom + image's assertion is never reported as independently verified merely because + its output digest matched. - R18. Normalized file evidence distinguishes create, edit, delete, and rename where truthful. It preserves bounded provider-native diffs, events, or trajectories when normalization loses information and separately records truncation, redaction, attribution, original/captured size, and digest semantics. - R19. Gateway and worker calls propagate W3C Trace Context and export metadata-only OpenTelemetry data. One explicit pre-processor allowlist admits only bounded non-content operational metadata; OpenInference and backend-native attributes pass the same allowlist and bounded filtering/redaction before any structured log or span processor. Prompts, model outputs, tool arguments/results, file bodies, source fragments, and secret-bearing attributes are prohibited before export. Owner correlation uses only an opaque identifier appropriate to telemetry-operator access, never caller identity or Task/Artifact authorization. Telemetry access and retention are configured separately from Task and Artifact access and retention, and telemetry is neither durable result truth nor required for terminal lookup. @@ -87,6 +168,14 @@ The two initial runtimes expose different programmatic contracts. Codex provides - R20. The gateway executes one coding request. It does not own eval configuration, datasets, repetition, scoring, retry policy, experiment scheduling, or a durable evaluation Run ledger. - R21. The initial worker topology is one execution at a time for reviewed repositories inside one configured mutual-trust domain. R13's narrow OS-enforced provider/tool credential boundary is required for credentialed profiles but does not claim hostile-source or cross-tenant isolation. Profiles making either stronger claim are rejected until a full per-invocation UID, mount, PID, network, and credential isolation boundary is configured. - R22. Gateway admission and worker execution enforce profile limits for request rate, active/retained Tasks, subscriptions, stored bytes, source transfer/expansion, files/inodes, workspace bytes, CPU, memory, PIDs, network, phase deadlines, events, logs, and artifacts. Exhaustion is scoped to one invocation or owner and leaves capacity for terminalization and cleanup. + Materializer CPU, memory, PIDs, network, time, transfer, expansion, file, + inode, and workspace output count against the invocation's limits. New-claim + source authorization precedes every cache lookup. Cached outputs are reusable + only after manifest and content revalidation for the same canonical source, + materializer-definition digest, expected and actual output-manifest digests, + authorization-scope digest, source-authorization revocation epoch, and trust + domain. Revocation advances the epoch and makes the prior namespace + ineligible; the conservative default namespaces cache entries by owner. ### Key Flows @@ -123,26 +212,59 @@ The two initial runtimes expose different programmatic contracts. Codex provides ### Acceptance Examples -- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile, an exact Git SHA, and an optional result schema, when the caller streams a request, then one Task moves from submitted to working to completed and later `GetTask` returns the same validated output and evidence Artifacts. +- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile, one valid + immutable workspace source, and an optional result schema, when the caller + streams a request, then one Task moves from submitted to working to completed + and later `GetTask` returns the same validated output, workspace provenance, + and evidence Artifacts. - AE2. **Covers R6.** Given a retained Task whose original absolute deadline has passed or whose profile is now disabled, changed, or no longer authorized for new work, when its owner reuses the invocation key with the same canonical request and result schema, then the gateway returns the original Task from its stored original bindings before mutable admission checks and makes no second worker dispatch. - AE3. **Covers R6.** Given a retained Task, when its owner reuses the invocation key with a different prompt, source, profile ID, deadline, or result schema, or the stored original profile/schema binding is inconsistent, then the gateway rejects the request and leaves the original Task unchanged. - AE4. **Covers R5.** Given a Task owned by caller A, when caller B lists Tasks, gets the Task, cancels it, subscribes, or requests an Artifact, then the gateway reveals no resource existence or content. -- AE5. **Covers R12, R16-R18.** Given a requested SHA that does not match the materialized repository or setup fails, when the worker has already accepted the current fence, then provider execution never starts, the selected public trace is `Submitted -> Working -> Failed`, and the Task retains source/setup-failure and cleanup evidence. +- AE5. **Covers R12, R16-R18.** Given a wrong or missing Git commit, + conflicting repository destination, OCI digest or manifest mismatch, unknown + or profile-disallowed materializer, materializer definition/image drift, + produced workspace-manifest digest that differs from the request, malformed + materializer output, direct known-secret disclosure, or setup failure, when + the worker has already accepted the current fence, then provider execution + never starts, the selected public trace is + `Submitted -> Working -> Failed`, and the Task retains bounded + materialization/setup-failure and cleanup evidence. - AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. Given cancel reaches a worker before its delayed dispatch, the worker tombstones the unseen attempt and the stale dispatch creates no workspace or provider process. - AE7. **Covers R3, R10-R11, R17.** Given equivalent profiles, one accepted `allagents.result-schema/v1` schema, and fixture runtime events for Codex and Pi, when each completes the same repository mutation, then both publish the required fixed-name integrity Artifact at the schema-defined extension carrier, validate with the same schema and validator, publish the same fixed-name structured-result Artifact containing one A2A `Part` with the validated `data` and `mediaType: application/json`, record the same integrity state, and produce the required normalized evidence fields while retaining distinct native evidence. - AE8. **Covers R4, R7, R19.** Given canary secrets and cross-owner content fragments in prompts, model output, tool arguments/results, source files, stale events, and errors, when agent, model, tool, stale-event, and error telemetry is processed, then the exporter receives only allowlisted bounded metadata plus the correct opaque owner correlation and receives none of those canaries, fragments, or raw caller identities. Given a caller or exporter disconnects during work, reconnect still returns the current Task and future updates without duplicate dispatch, and telemetry loss does not affect terminal lookup. - AE9. **Covers R15.** Given a known capability denied by profile, the accepted Task becomes rejected after stop and cleanup; given an unknown permission type, it becomes failed as an adapter incompatibility without waiting for a client. - AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. Given output validation has already selected `valid` or `invalid` and a later check or mandatory-evidence phase fails, the failed Task preserves that result state and a valid result preserves its one fixed structured-result Artifact; only a failure before candidate production records `not_produced`. -- AE11. **Covers R6, R22.** Given invalid input or exhausted admission quota, the gateway returns a request/resource error and creates no Task; given capacity disappears after durable acceptance, the retained Task fails at dispatch and replay returns it without retry. +- AE11. **Covers R6, R11-R12, R22.** Given invalid input, an unknown or + profile-disallowed source mode/materializer, or exhausted admission quota, + the gateway returns a request/resource error and creates no Task; given + materializer availability or worker capacity disappears after durable + acceptance, the retained Task fails at dispatch or materialization and replay + returns it without retry. - AE12. **Covers R7, R14, R16.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only allowlisted metadata-only operator telemetry. Given bounded escalation cannot stop a descendant that starts a new session and ignores graceful signals, the worker persists termination unknown/failed, refuses another reservation, exits, and its supervisor destroys the boundary; replacement readiness performs orphan recovery without changing the failed Task. - AE13. **Covers R8.** Given a Task reaches expiry while physical deletion fails, all Task and Artifact operations return the same not-found response and the invocation key can create a new Task. -- AE14. **Covers R5, R13, R21-R22.** Given a production public listener or remote worker route lacks its configured trusted transport or authenticated peer identity, readiness fails; a same-host Unix worker socket is accepted. Given a credentialed reviewed-domain profile, model tools cannot inspect provider process environments, process listings, backend config/data roots, or exfiltrate provider/control credentials across the configured OS boundary. Hostile-source or cross-tenant claims remain rejected. +- AE14. **Covers R5, R13, R21-R22.** Given a production public listener or + remote worker route lacks its configured trusted transport or authenticated + peer identity, readiness fails; a same-host Unix worker socket is accepted. + Given a credentialed reviewed-domain profile, model tools cannot inspect + provider process environments, process listings, backend config/data roots, + or exfiltrate provider/control credentials across the configured OS + boundary. Given a registered materializer with source credentials, setup, + provider processes, and model tools have no access to its process, runner + control socket, credential environment/mounts, or staging root after + materialization, and direct known-secret canaries are absent from retained + logs, evidence, and published workspace files. The registered materializer + remains operator-trusted code; hostile-materializer, hostile-source, and + cross-tenant claims remain rejected without a stronger broker or sandbox. ### Success Criteria - The official A2A JavaScript client can discover the required extension, negotiate it through `A2A-Extensions`, use the standard Message and Artifact extension carriers, and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, retained replay, cancel, and expiry behavior against the built service without `Task.extensions`. - One conformance fixture passes unchanged through the Codex and Pi adapters. - Admission, retained replay, monotonic worker commands, fencing, acceptance-before-materialization source/setup failure, cancellation races, trace-order/fence/multiplicity constraints, failed-quiescence recycling, restart terminalization without resume, supervised worker-crash cleanup, trusted transports, authorization isolation, metadata-only telemetry export, OS-enforced provider/tool credential separation, source hardening, portable structured-result validation, quotas, and evidence integrity have deterministic integration coverage. +- Direct multi-repository Git, digest-pinned OCI snapshots, and a fake + digest-pinned registered materializer all produce the same validated + workspace manifest and terminal provenance contract before either backend + starts. - The gateway image contains no coding-agent runtime and cannot access worker workspace roots. - The initial worker runs one reviewed-trust-domain execution at a time, model-initiated tools are OS-isolated from provider/control credentials, repository Pi extensions cannot auto-load, and no live descendant or reusable workspace survives a completed, failed-quiescence, or crashed attempt. @@ -156,6 +278,9 @@ The two initial runtimes expose different programmatic contracts. Codex provides - Built-in bearer authentication with OIDC/JWT and static service-token modes behind the named production TLS boundary. - Single-replica durable file storage, authenticated Artifact retrieval, authenticated encrypted remote worker transport or same-host Unix sockets, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. - Reviewed repositories in one configured mutual-trust domain per worker deployment, with the narrow OS-enforced provider/tool credential boundary required for credentialed profiles. +- Direct multi-repository Git acquisition, digest-pinned OCI workspace + snapshots, and operator-registered digest-pinned materializers with + phase-scoped credentials and one standard workspace manifest. **Deferred to follow-up work** @@ -176,6 +301,7 @@ The two initial runtimes expose different programmatic contracts. Codex provides - [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) - [AHP decision inputs](../research/agent-host-protocol-decision-inputs.md) +- [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) - [AI Evals ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) - [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) - [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) @@ -204,15 +330,69 @@ The two initial runtimes expose different programmatic contracts. Codex provides - KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates a new invocation claim and submitted Task together after mutable admission, storing the canonical caller request and the original effective-profile and result-schema digests needed for retained replay. It stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. - KTD4. **Authenticate at a named trusted HTTP ingress before A2A storage or dispatch.** Production traffic reaches the gateway through TLS terminated by the configured gateway or named trusted reverse-proxy boundary; plaintext is allowed only for an unauthenticated loopback development listener. Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Readiness rejects a production public URL whose trusted TLS boundary is absent or inconsistent. Governs R5-R6, R13. - KTD5. **Use fenced, separately deployable gateway and worker services.** Remote gateway-worker routes use mTLS or an explicitly equivalent authenticated encrypted overlay; a same-host Unix socket is acceptable. The authenticated worker identity is pinned to the configured route/capability set, and every short-lived attempt capability is bound to that identity, attempt ID, lease ID/epoch, and fence. Each worker keeps one minimal durable monotonic command record scoped to its worker identity and lease: `Cancel(attempt, fence, revision)` tombstones even an unseen attempt, and `Dispatch` for a tombstoned or lower-revision attempt is rejected before workspace creation. Dispatch/cancel I/O conditionally verifies the persisted command/outbox revision immediately before any mutating or terminating effect. Duplicate delivery is idempotent; conflicting, stale, out-of-order, or identity-mismatched commands/events are rejected. Gateway and worker transition selectors remain pure and executors re-enter from persisted or freshly observed state. This record is worker-local fence state, not a new durable execution subsystem. Governs R5, R7, R10-R16, R21-R22. -- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Worker-process exit makes the external supervisor terminate the complete execution boundary. If bounded escalation cannot prove the complete invocation process set empty, the worker records termination unknown/failed, poisons admission, and exits rather than accepting another reservation; its supervisor destroys the boundary. Before readiness the replacement proves termination and reaps or quarantines orphaned invocation roots. Production readiness accepts a dedicated worker container process namespace under a minimal init/reaper as the baseline; a non-container deployment must prove an equivalent systemd/cgroup boundary. The gateway never reattaches to or resumes a provider session. Governs R7, R14, R16-R18. +- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Every external materializer launch creates a supervisor-owned runner resource labeled by worker, attempt, lease, and fence; worker-process exit makes the external supervisor terminate that resource and the complete execution boundary. If bounded escalation cannot prove the complete invocation process set empty, the worker records termination unknown/failed, poisons admission, and exits rather than accepting another reservation; its supervisor destroys the boundary. Before readiness the replacement proves termination and enumerates, destroys, or quarantines orphaned invocation roots, runner resources, credential mounts, and staging mounts; an unresolved resource keeps readiness false. Production readiness accepts a dedicated worker container process namespace under a minimal init/reaper as the baseline; a non-container deployment must prove an equivalent systemd/cgroup boundary. The gateway never reattaches to or resumes a provider session. Governs R7, R14, R16-R18. - KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, optional structured result, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, schema validation, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | pi` registry is the only production dispatch point. Governs R10-R11, R14-R18, R21-R22. - KTD8. **Use each provider's supported automation surface directly behind the credential boundary.** Codex depends directly on pinned `@openai/codex-sdk`, creates one fresh thread per Task, passes `AbortSignal` and optional per-turn `outputSchema`, and consumes streamed events. Pi uses strict RPC with an invocation-local credential store and one explicitly loaded worker-owned policy extension; repository extensions and unrestricted built-ins never load. For either adapter, a credentialed provider runtime is separated from every model-invoked tool by the R13 OS-enforced UID/process/mount boundary or an equivalent credential broker; shell-environment filtering is defense in depth, not the boundary. Promptfoo's Codex provider and tests are characterization references only; AllAgents neither vendors them nor inherits their config, cache, pricing, retry, thread-pool, or `ProviderResponse` concerns. Governs R10-R18. -- KTD9. **Make profiles the new-admission policy boundary.** Requests select a profile ID and may provide only an `allagents.result-schema/v1` schema. They cannot override backend credentials, executable paths, provider config, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. For a new claim, resolve a versioned canonical `EffectiveProfileIntent`, compute its digest without resolved secrets or per-attempt state, and persist it with the canonical caller request and result-schema digest. Retained replay compares those stored original bindings and never substitutes or re-resolves the current profile. Governs R6, R11-R16, R21-R22. +- KTD9. **Make profiles the new-admission policy boundary.** Requests select a + profile ID, one schema-defined workspace source mode, and optionally one + `allagents.result-schema/v1` schema. They cannot override backend or source + credentials, materializer definitions or images, executable paths, provider + config, setup/check commands, environment allowlists, permission rules, + trust class, resource limits, workspace retention, or evidence budgets. A + profile allowlists source modes and materializer IDs plus exact canonical Git + repository/namespace rules, OCI namespaces/signature rules, and resource + selectors for structured materializer inputs. New admission authorizes the + fully canonicalized resource and credential entitlement before cache lookup. + Resolve a versioned canonical `EffectiveProfileIntent` containing the + selected materializer definition digest, authorization-scope digest, and + source-authorization revocation epoch when applicable; compute its digest + without resolved secrets or per-attempt state and persist it with the + canonical caller request and result-schema digest. Retained replay compares + those stored original bindings and never substitutes or re-resolves current + policy. Governs R6, R11-R16, R21-R22. - KTD10. **Keep durable evidence and operational telemetry as separate bounded layers.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded evidence layer; neither Git nor provider evidence is promoted as exact causality when incomplete. Telemetry is a third, non-durable metadata-only channel: one small shared pre-export sanitizer applies an explicit operational-metadata allowlist plus bounded filtering/redaction before every structured log or span processor, and only opaque owner correlation may cross the separately governed operator boundary. OpenInference and backend-native attributes receive no bypass. This is an export guard, not a telemetry framework or alternate evidence store. Governs R13, R16-R19. - KTD11. **Treat Codex and Pi as the complete initial backend set.** Codex lands first; Pi lands second against the established contract; OpenCode is deferred. (session-settled: user-directed.) Governs R10. - KTD12. **Separate terminal integrity from optional evidence bodies.** The fixed `allagents.execution-integrity` Artifact validates identity, action outcome, the four-state structured-result record, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance before terminal publication. `not_produced` applies only before result-candidate production. Once validation selects `valid` or `invalid`, a later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the separate fixed structured-result Artifact while retaining the later phase as the primary Task failure. Predictable optional-body truncation/redaction may preserve completion; failure that breaks the integrity kernel fails in the evidence phase. Governs R3-R4, R17-R18. -- KTD13. **Harden Git acquisition as a network security boundary.** Accept canonical HTTPS origins only. Use hermetic Git configuration, disable redirects, proxies, helpers, hooks, filters, LFS smudge, submodule recursion, alternates, and non-HTTPS protocols. Revalidate normalized host/address policy for every connection, never forward credentials across origins, and verify the full object ID resolves to a commit fetched from the approved remote. Governs R12-R13, R22. +- KTD13. **Standardize and harden workspace materialization.** Define one + closed `kind`-discriminated workspace-source union and one output manifest; + reject unknown kinds and cross-variant fields. The built-in Git path accepts + canonical HTTPS repository identities and full commit IDs only, uses + hermetic Git configuration, disables redirects, proxies, helpers, hooks, + filters, LFS smudge, submodule recursion, alternates, and non-HTTPS + protocols, revalidates normalized host/address policy for every connection, + fetches into an isolated object database from the authorized remote, and + verifies the checked-out commit and resulting tree. The OCI path accepts + manifest digests, not tags; rejects foreign/external URLs by default; + revalidates scheme, host, resolved address, port, redirect, and credential + origin for every registry/auth/manifest/blob request; and verifies every + manifest/layer plus the embedded workspace manifest. The custom path accepts + a registered ID, expected workspace-manifest digest, and schema-validated, + resource-authorized inputs. + + The operator-owned registry splits a non-secret gateway descriptor from the + worker-only runtime definition. The worker computes a + `sha256:<64 lowercase hex>` digest over the versioned, domain-separated + canonical non-secret runtime definition; readiness compares that value with + the gateway's expected digest and capabilities. Distinct domain-separated + canonical JSON preimages define materializer input and workspace-manifest + digests. All paths stage in a worker-owned directory on the final + publication filesystem, validate destinations, links, file types, bounds, + identities, content, and manifest, then terminate the supervisor-owned + acquisition process/mount/credential/runner boundary while retaining the + validated host-owned tree. Only after proving the boundary gone does the + worker atomically rename the tree; no copy fallback exists. Provenance + distinguishes worker-verified observations, trusted-service verification, + and materializer-attested claims. The caller digest covers source kind, + expected output identity, and inputs; the effective-profile digest covers + materializer and authorization bindings; terminal provenance covers both and + the validated output. Governs R6, R11-R13, R16-R18, R21-R22. - KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Credentialed profiles still require R13's narrower OS-enforced provider/tool separation: model tools cannot inspect provider processes, procfs entries, or backend config/data roots, and readiness fails without that capability. Provider/source credentials are absent from setup/check phases and child-visible worker control state. Pi disables repository extensions and built-in tools; only the worker-owned policy extension may load. This credential boundary does not imply hostile-source or cross-tenant isolation; that stronger sandbox-driver capability remains deferred. Governs R13, R16, R21-R22. + An external materializer image is reviewed operator code in the deployment's + trusted computing base, not hostile caller code. Its runner or sandbox + control plane is never mounted into the workspace or exposed to setup, + providers, or model tools. A deployment that does not trust the registered + image with source credentials must use a broker or stronger acquisition + service and advertise that capability explicitly. - KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, Codex SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. ### High-Level Technical Design @@ -225,7 +405,10 @@ flowchart TB Gateway --> Auth[Auth, retained replay, new admission] Gateway --> Store[Generation-based Task and Artifact store] Gateway -->|mTLS/authenticated overlay or same-host Unix socket| Worker[Single-execution worker] - Worker --> Source[Hardened Git acquisition] + Worker --> Materialization[Workspace materializer registry] + Materialization --> Git[Hardened multi-repository Git] + Materialization --> OCI[Digest-pinned OCI snapshot] + Materialization --> Custom[Registered materializer image] Worker --> Registry[Closed backend registry] Registry --> Codex[Codex SDK] Registry --> Pi[Pi RPC process] @@ -260,7 +443,8 @@ sequenceDiagram G->>W: Dispatch(attempt, fence, command revision) W->>W: Verify command record before workspace creation W-->>G: Accepted(attempt, fence) - W->>W: Materialize, verify, setup, baseline + W->>W: Materialize into staging and validate workspace manifest + W->>W: Destroy acquisition boundary, publish atomically, setup, baseline W->>B: Invoke with isolated roots and credential boundary B-->>W: Progress, usage, native evidence W-->>G: Sequenced fenced progress @@ -356,6 +540,12 @@ packages/execution-service/ reaper.ts lease.ts workspace.ts + materializers/ + types.ts + registry.ts + git.ts + oci.ts + external.ts evidence.ts adapters/ types.ts @@ -383,10 +573,51 @@ docs/src/content/docs/ ### Configuration Contract -- Gateway configuration defines the listener/public URL, a named trusted TLS termination boundary for production ingress, auth and canonical owner mapping, store/retention, admission and subscription quotas, low-space watermarks, Artifact limits, worker routes, internal capability secrets, and profiles. Each remote worker route declares mTLS or an explicitly equivalent authenticated encrypted overlay, pinned worker identity/capabilities, and trust material; a same-host route may declare a Unix socket. Plaintext remote URLs are invalid. -- Each profile defines backend, worker route, allowed Git origins/addresses, provider/model settings, phase-specific environment allowlists, deterministic permissions, setup/check commands, artifact globs, effective deadline ceiling, trust class, resource limits, cleanup policy, evidence budgets, and the required provider/tool credential-boundary capability for credentialed execution. -- Worker configuration fixes a private listener, worker identity, one-execution concurrency, workspace root, minimal worker-local command-record location, execution-supervisor mechanism, pre-readiness orphan policy, lease grace, backend runtime constraints, trust domain, resource-control and provider/tool credential-boundary capabilities, and request/result limits. -- Production worker readiness requires authenticated route identity, protected remote transport or a same-host Unix socket, an enforceable credential boundary for every credentialed profile, and a supervisor that proves complete descendant termination and root ownership. The supported supervisor baseline is a dedicated worker container process namespace under a minimal init/reaper; bare-host deployment requires an equivalent systemd/cgroup mechanism. +- Gateway configuration defines the listener/public URL, a named trusted TLS + termination boundary for production ingress, auth and canonical owner + mapping, store/retention, admission and subscription quotas, low-space + watermarks, Artifact limits, worker routes, internal capability secrets, + non-secret materializer descriptors, and profiles. A descriptor contains the + materializer ID, bounded input schema, expected definition digest, expected + output-manifest version, and required worker capabilities. Each remote worker + route declares mTLS or an explicitly equivalent authenticated encrypted + overlay, pinned worker identity/capabilities, source modes and matching + materializer definition digests, and trust material; a same-host route may + declare a Unix socket. Plaintext remote URLs are invalid. +- Each profile defines backend, worker route, allowed workspace source modes, + allowed materializer IDs, exact Git repository or namespace rules, allowed + Git origins/addresses, OCI namespace/registry/signature policy, structured + materializer-input resource selectors, authorization-scope derivation and + source-authorization revocation epoch, provider/model settings, + phase-specific environment allowlists, deterministic permissions, + setup/check commands, artifact globs, effective deadline ceiling, trust + class, resource limits, cleanup policy, evidence budgets, and required + acquisition/provider/tool isolation capabilities. +- Worker configuration fixes a private listener, worker identity, + one-execution concurrency, one same-filesystem publication root containing + private staging and final workspace directories, a closed materializer + runtime registry, minimal worker-local command-record location, + execution-supervisor mechanism, pre-readiness orphan policy, lease grace, + backend runtime constraints, trust domain, resource-control and + credential-boundary capabilities, and request/result limits. Each external + materializer runtime entry matches the gateway descriptor's ID and expected + definition digest and additionally fixes a digest-pinned image, credential + handle names or mount identities, network destinations, resource/deadline + limits, cache policy, output version, and OCI runner or sandbox; it contains + no credential values. The worker derives, rather than trusts, the definition + digest from that complete non-secret runtime entry. +- Production worker readiness requires authenticated route identity, protected + remote transport or a same-host Unix socket, exact agreement between the + gateway's expected descriptor digest and the worker's computed runtime + definition digest, a same-filesystem staging/publication root with atomic + rename and no copy fallback, and an OCI materializer runner or sandbox that + assigns supervisor-owned attempt/lease/fence labels without exposing its + control plane to the workspace. It also requires an enforceable credential + boundary for every credentialed phase and a supervisor that proves complete + descendant termination and enumerates or destroys orphan runner resources, + credential mounts, staging mounts, and roots before readiness. The supported + worker baseline is a dedicated process namespace under a minimal init/reaper; + bare-host deployment requires an equivalent systemd/cgroup mechanism. - Telemetry configuration defines the OTLP destination, filtering/redaction bounds, opaque owner-correlation derivation, and telemetry-specific operator access and retention. The service version fixes the metadata allowlist; configuration cannot extend it to prompt/output/tool/source/file-body attributes, secret-bearing fields, raw caller identity, or unfiltered backend-native/OpenInference attribute passthrough. - Configuration contains environment-variable names but never secret values. Startup resolves the complete graph and becomes ready only when trusted ingress, worker transports/identities, store, runtimes, quotas, free-space reserves, supervisor/orphan recovery, and declared profile capabilities pass. Any unprotected remote endpoint or unproved credential/supervisor boundary fails readiness. @@ -394,14 +625,14 @@ docs/src/content/docs/ | Condition | A2A result | Required extension detail | |---|---|---| -| New-admission authentication, malformed/unsupported extension carrier, invalid source/profile, unauthorized policy, expired deadline, current-profile/readiness failure, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | +| New-admission authentication, malformed/unsupported extension carrier, invalid workspace source/profile, unknown or profile-disallowed materializer, unauthorized policy, expired deadline, current-profile/readiness failure, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | | Identical retained invocation replay | Existing Task | Returned from stored original request/profile/schema bindings before current deadline, quota, authorization, readiness, or profile checks; no new Task, worker attempt, or quota reservation | | Conflicting invocation key or inconsistent stored binding | Operation error; no new Task | Conflict code; existing Task unchanged | | Worker capacity loss after acceptance | `TASK_STATE_FAILED` | `dispatch/capacity_exhausted`, retriable fact, no workspace created; gateway does not retry | | Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | | Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | | Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | -| Failure before result-candidate production | `TASK_STATE_FAILED` | Typed primary source/setup/provider/dispatch/crash/infrastructure phase, safe message, retriable fact, requested structured result `not_produced`, separate termination/cleanup/completeness | +| Failure before result-candidate production | `TASK_STATE_FAILED` | Typed primary dispatch/materialization/setup/provider/crash/infrastructure phase, including manifest or materializer failure; safe message, retriable fact, requested structured result `not_produced`, separate termination/cleanup/completeness, and bounded workspace provenance | | Check, mandatory-evidence, cleanup, crash, or infrastructure failure after result validation | `TASK_STATE_FAILED` | Preserve selected `valid` or `invalid`; preserve exactly one fixed structured-result Artifact for `valid`; later phase remains primary failure | | Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` with `not_produced`, or `structured_result/invalid` with `invalid`; no structured-result Artifact | | Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors and native abort; use `not_produced` only before a candidate, otherwise preserve `valid`/`invalid` and the valid Artifact; record termination and cleanup | @@ -415,7 +646,11 @@ docs/src/content/docs/ 1. Create the private Node 22 service package and freeze the public extension URI and standard carriers, integrity and structured-result Artifacts, portable result-schema subset, worker protocol including command revisions/tombstones, profiles, fixtures, and error vocabulary. 2. Build authenticated durable A2A Task handling, retained-claim-first replay, and trusted fenced worker dispatch against a fake worker; startup terminalizes interrupted Tasks without attempting provider reattachment. -3. Build the supervised single-execution worker lifecycle, monotonic command record, failed-quiescence boundary recycling, pre-readiness orphan reaper, OS credential boundary, and hardened source/evidence handling against a fake adapter. +3. Build the supervised single-execution worker lifecycle, monotonic command + record, failed-quiescence boundary recycling, pre-readiness orphan reaper, + OS credential boundaries, the direct Git/OCI/registered-materializer + registry, and hardened workspace/evidence handling against fake + materializers and a fake backend. 4. Add the direct Codex SDK adapter and prove structured output, cancellation, OS-enforced provider/tool credential separation, and native evidence. 5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing OS-confined tools plus the terminating result tool. 6. Package the services and run cross-backend, transport, security, process, and A2A conformance before enabling a consumer. @@ -425,7 +660,15 @@ docs/src/content/docs/ - **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. - **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. - **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. -- **Security:** New review-critical surfaces are trusted public/private transports, auth, owner-key derivation, retained-replay ordering, source SSRF, admission/resource quotas, setup/check policy, OS-enforced provider/tool credential separation, Pi extension/tool replacement, phase-scoped secrets, metadata-only telemetry filtering and operator boundaries, internal fences and monotonic command records, Artifact capture/serving, and reviewed-source trust enforcement. +- **Security:** New review-critical surfaces are trusted public/private + transports, auth, owner-key derivation, retained-replay ordering, Git and OCI + source SSRF, materializer image supply chain, materializer input schemas, + phase-scoped source credentials and egress, workspace manifest validation, + admission/resource quotas, setup/check policy, OS-enforced provider/tool + credential separation, Pi extension/tool replacement, metadata-only + telemetry filtering and operator boundaries, internal fences and monotonic + command records, Artifact capture/serving, and reviewed-source trust + enforcement. - **Operations:** Gateway and worker health, readiness, transport/peer identity, quotas, low-space state, allowlisted metadata-only structured logs/traces, telemetry-specific access/retention, command tombstones, lease expiry, poisoned-worker exit, supervisor boundary health, orphan-root quarantine/reaping, stale event rejection, and graceful shutdown need independent signals. - **Consumers:** AI Evals can build its runner provider only after the Agent Card, extension schemas, and conformance fixtures are versioned and published. @@ -439,7 +682,15 @@ docs/src/content/docs/ - **Orphan processes and roots:** Combine explicit cancel, native abort, process-set verification, one-execution supervisor/container death, lease expiry, and pre-readiness orphan reaping or quarantine. Failed quiescence poisons admission and exits the worker so the supervisor destroys the boundary; termination/filesystem outcomes remain separate. - **False recovery claims:** Persist Task and evidence truth only. Startup fails active Tasks, invalidates fences, and relies on lease expiry or supervisor-boundary proof instead of resuming provider sessions. - **Structured-output drift:** Admit only the versioned closed schema subset, include its canonical digest in provenance and original claim bindings, pass the exact accepted schema through each adapter, validate with one shared validator, preserve an already selected result across later failures, and enforce the two fixed Artifact shapes. -- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral and origin-bound. Credentialed profiles also enforce the R13 OS provider/tool boundary or broker; environment filtering remains defense in depth. Pi repository extensions and unrestricted built-in tools never load. +- **Source SSRF, materializer compromise, or credential leakage:** Enforce + KTD13 for every source mode, connection, and phase. Pin external + materializer images and OCI snapshots by digest, validate their manifests, + isolate staging and acquisition processes, apply explicit egress and limits, + and atomically publish only validated outputs. Source credentials are + ephemeral, origin-bound, and removed before setup. Credentialed profiles also + enforce the R13 OS provider/tool boundary or broker; environment filtering + remains defense in depth. Pi repository extensions and unrestricted built-in + tools never load. - **Telemetry disclosure:** Apply KTD10's pre-export guard before every structured log/span processor and reject content or secret-bearing attributes rather than relying on exporter policy. Canary-secret and cross-owner-fragment tests cover agent, model, tool, stale-event, and error paths; telemetry operators receive only bounded metadata and opaque owner correlation under separate access and retention. - **Resource exhaustion:** Reserve per-owner/global gateway quota only for new claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. - **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. @@ -451,7 +702,10 @@ docs/src/content/docs/ ### Assumptions - The first production deployment runs one gateway replica with persistent storage. Multi-replica transactional storage is deferred. -- Git over hardened HTTPS and exact commit object ID covers the initial consumer. Other source transports require a later extension version or capability. +- The initial public extension supports direct multi-repository Git, + digest-pinned OCI workspace snapshots, and operator-registered materializers. + A deployment may enable only the source modes its worker route advertises; + direct hardened Git remains the required baseline. - Setup and check commands are operator-controlled profile policy, not caller-supplied shell text. - Initial repositories are reviewed inside one configured mutual-trust domain. Credentialed profiles still enforce provider/tool credential separation, but that narrower boundary does not make hostile-code or cross-tenant execution available; those claims require a stronger sandbox driver. - Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. @@ -462,17 +716,54 @@ docs/src/content/docs/ ### U1. Versioned public and worker contracts -- **Goal:** Freeze the standard public extension carriers, integrity and structured-result Artifacts, profile vocabulary, private worker protocol including monotonic command state, original idempotency bindings, typed failures, and conformance fixtures before either service endpoint. +- **Goal:** Freeze the standard public extension carriers, versioned workspace + source and manifest contracts, materializer/profile vocabulary, integrity and + structured-result Artifacts, private worker protocol including monotonic + command state, original idempotency bindings, typed failures, and conformance + fixtures before either service endpoint. - **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12. - **Dependencies:** None. - **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/result-schema-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. - **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas and freeze `https://allagents.dev/a2a/extensions/coding-execution/v1`: required Agent Card advertisement, `A2A-Extensions` negotiation, `Message.extensions`, request data only at `Message.metadata[uri]`, and terminal integrity data only in the single Part of the fixed-name `allagents.execution-integrity` Artifact whose `extensions` contains the URI. Explicitly forbid `Task.extensions`. Define the portable result-schema subset, canonical caller/schema/profile digests, four result states, separate fixed `allagents.structured-result` Artifact, and shared validator. Define original claim bindings independently from mutable current policy. Add worker identity, attempt/fence/lease identity, monotonic command revision, unseen-attempt cancel tombstone, conditional effect revision, event sequence, terminal acknowledgement, and integrity rules. Generate checked-in schemas and fixtures from one source. -- **Execution note:** Start with fixture-driven schema, framing, and digest tests. Observe failures for unknown versions, credential-bearing sources, mutable revisions, unsafe paths, invalid public states, stale fences, oversized records, and conflicting canonical inputs before implementing schemas. + The source contract is a strict `kind`-discriminated union for direct + repository lists, digest-pinned OCI snapshots, or a registered materializer + ID with an expected workspace-manifest digest and schema-validated structured + inputs; cross-variant fields are unrepresentable. Define the standard + workspace manifest, verification-method vocabulary, authorization scope and + revocation epoch, collision-safe destinations, and split gateway/worker + materializer descriptors. Define algorithm-qualified digest formats and + versioned, domain-separated canonical preimages for materializer definitions, + inputs, manifests, profiles, and caller requests; no public field can carry + acquisition code, image references, commands, credentials, or policy. +- **Execution note:** Start with fixture-driven schema, framing, and digest + tests. Observe failures for unknown versions, credential-bearing sources, + mutable revisions or image tags, duplicate/unsafe destinations, unknown or + disallowed materializers, invalid structured inputs or workspace manifests, + unsafe paths, invalid public states, stale fences, oversized records, and + conflicting canonical inputs before implementing schemas. - **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, `src/core/native/types.ts` for safe error/provenance normalization, and Buzz's structurally non-secret intent template for the narrow digest-input pattern. - **Test scenarios:** - A minimal valid Message negotiates the exact URI in `A2A-Extensions`, includes it in `Message.extensions`, puts the bounded request only at `Message.metadata[uri]`, and produces a stable digest across object-key ordering; missing/mismatched carriers and any `Task.extensions` field are rejected. Every terminal fixture has exactly one `allagents.execution-integrity` Artifact with the URI in `Artifact.extensions` and the schema-defined envelope in its single `data` Part. - Changing prompt, source object ID, profile ID, result schema, artifact selection, or deadline changes the canonical caller digest; trace IDs and transport metadata do not. The original effective-profile and result-schema digests are stored separately for retained replay. - - Rotating a resolved secret value, changing attempt/lease/trace identity, or changing a per-run path leaves the profile digest unchanged; changing a policy field or environment-variable name changes it, and the digest serializer cannot accept secret-bearing runtime state. + - Direct repositories are order-canonicalized without erasing destination + identity; duplicate destinations, mutable refs, unsafe subdirectories, and + ambiguous URL forms fail. The checked-out commit and tree match the + requested object from the authorized remote. OCI tags and external layer + URLs fail while allowed manifest digests pass. + - The source discriminator rejects unknown `kind` values, cross-variant + fields, and missing variant fields. Registered materializer inputs validate + against the operator schema and resource selectors, the request pins the + expected workspace-manifest digest, the worker-derived definition digest + changes the effective-profile digest, gateway and worker descriptors agree, + and caller-supplied image/command/credential fields are unrepresentable. + Fixed cross-language vectors prove algorithm-qualified, domain-separated + canonical digests and every non-secret runtime-field mutation changes the + definition digest while secret-value rotation does not. + - Rotating a resolved secret value, changing attempt/lease/trace identity, or + changing a per-run path leaves the profile digest unchanged; changing a + policy field, environment-variable name, authorization scope, or revocation + epoch changes it, and the digest serializer cannot accept secret-bearing + runtime state. - Unsupported keywords, remote references, non-object roots, object schemas that omit `additionalProperties: false`, undeclared optional properties, format-dependent validation, or schemas over byte/depth/property/enum limits are rejected before Task creation; every accepted schema validates identically in admission, worker, Codex forwarding, and Pi tool generation. - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. - Worker fixtures reject missing/mismatched worker identities, attempt IDs, lease epochs, profile digests, command revisions, conditional-effect revisions, event sequences, bounds, and terminal acknowledgements. Cancel for an unseen attempt persists a tombstone; tombstoned or lower-revision dispatch is invalid before workspace creation. @@ -530,17 +821,89 @@ docs/src/content/docs/ ### U4. Worker protocol and safe workspace lifecycle -- **Goal:** Implement the supervised single-execution worker with authenticated transport, a minimal monotonic command record, hardened immutable Git acquisition, OS-enforced credential separation, leases, isolated roots, resource controls, race-resistant evidence, failed-quiescence recycling, and cleanup independent of any provider. +- **Goal:** Implement the supervised single-execution worker with authenticated + transport, a minimal monotonic command record, a closed workspace + materializer registry for hardened multi-repository Git, digest-pinned OCI, + and operator-registered images, standard manifest validation, OS-enforced + credential separation, leases, isolated roots, resource controls, + race-resistant evidence, failed-quiescence recycling, and cleanup independent + of any provider. - **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14. - **Dependencies:** U1. -- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`. -- **Approach:** Authenticate the configured worker identity and fence every private command. Persist one minimal monotonic command record scoped to worker identity/lease before workspace creation: unseen-attempt cancel writes a tombstone, stale/lower-revision dispatch is rejected, and each dispatch/cancel effect conditionally rechecks the stored revision immediately before mutation. Reserve one execution only after that check. Validate profile/deployment and OS provider/tool credential-boundary capabilities, then emit sequenced NDJSON. Keep transition selection pure. Run inside a dedicated container process namespace under init/reaper or an equivalent systemd/cgroup boundary. After bounded termination escalation, prove the complete invocation process set empty; if proof fails, persist termination unknown/failed, poison admission, and exit so the supervisor destroys the boundary. Replacement readiness proves boundary termination and reaps/quarantines owned roots. Use KTD13 acquisition, separate roots, phase environments, budgets, and descriptor-safe evidence; clean in `finally`. -- **Execution note:** Characterize every phase with a fake adapter, malicious fixtures, and disposable Git servers before real providers. Fault-inject dispatch acknowledgement, events, leases, acquisition, processes, evidence publication, and cleanup. +- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/materializers/types.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/worker/materializers/git.ts`, `packages/execution-service/src/worker/materializers/oci.ts`, `packages/execution-service/src/worker/materializers/external.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/materializers.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`, `packages/execution-service/tests/fixtures/execution/fake-materializer.ts`. +- **Approach:** Authenticate the configured worker identity and fence every + private command. Persist one minimal monotonic command record scoped to worker + identity/lease before workspace creation: unseen-attempt cancel writes a + tombstone, stale/lower-revision dispatch is rejected, and each dispatch/cancel + effect conditionally rechecks the stored revision immediately before + mutation. Reserve one execution only after that check. Validate the fully + canonicalized source against profile resource policy before cache lookup; + bind cache entries to owner or authorization-scope digest, revocation epoch, + canonical source, definition digest, and expected/actual manifest digests. + Validate deployment, worker-computed materializer definition digest, and + acquisition plus provider/tool credential-boundary capabilities, then emit + sequenced NDJSON. Keep transition selection pure. Run inside a dedicated + container process namespace under init/reaper or an equivalent systemd/cgroup + boundary. Resolve only the closed KTD13 materializer registry. + + Launch an external materializer through the configured OCI runner or sandbox + as a supervisor-owned resource labeled by worker, attempt, lease, and fence, + with only schema-validated and resource-authorized inputs, its source + credentials, allowed egress, and a private host-owned staging mount under the + final publication root. Never expose gateway, provider, backend, + final-workspace roots, or the runner control socket. Treat the digest-pinned + image as operator-trusted deployment code. Verify the versioned output + manifest, request-pinned digest, content, immutable identities, and + verification-method labels. Terminate the acquisition process, credential + scope, mounts, and runner resource while retaining the validated host-owned + tree; prove that boundary gone before an atomic same-filesystem rename, + setup, and baseline. No copy fallback exists. After bounded termination + escalation, prove the complete invocation process set empty; if proof fails, + persist termination unknown/failed, poison admission, and exit so the + supervisor destroys the boundary. Replacement readiness enumerates and + destroys or quarantines orphan runner resources, credential/staging mounts, + and roots. Use separate phase environments, budgets, and descriptor-safe + evidence; clean in `finally`. +- **Execution note:** Characterize every phase with a fake adapter, fake + registered materializer, local OCI registry, malicious fixtures, and + disposable Git servers before real providers or private artifact systems. + Fault-inject dispatch acknowledgement, events, leases, every acquisition + mode, manifest publication, processes, evidence publication, and cleanup. - **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, `tests/helpers/env.ts` for isolated state, and Buzz's bounded process-group/job-object cancellation as a lifecycle characterization checklist rather than copied code. - **Test scenarios:** - - Covers AE5. Exact object ID verifies; wrong/missing object, disallowed URL/host/address/port, credential-bearing URL, redirect, DNS rebinding, unsafe subdirectory, fetch failure, and setup failure stop before adapter invocation. After worker acceptance each such source/setup failure emits the selected `Submitted -> Working -> Failed` public trace. - - Repositories with LFS configuration/pointers, submodules, hooks, filters, alternates, proxy/helper config, or non-HTTPS secondary protocols cause no secondary connection or helper execution. - - Source credentials leave no repository config, process argument, child phase environment, log, error, evidence, or retained workspace trace. + - Covers AE5. Every repository checkout matches its requested full object ID + and tree and destinations are disjoint; wrong/missing objects, disallowed + repository/namespace/URL/host/address/port, credential-bearing URLs, + redirects, DNS rebinding, unsafe subdirectories, fetch failure, and setup + failure stop before adapter invocation. After worker acceptance each + materialization/setup failure emits the selected + `Submitted -> Working -> Failed` public trace. + - Repositories with LFS configuration/pointers, submodules, hooks, filters, + alternates, proxy/helper config, or non-HTTPS secondary protocols cause no + secondary connection or helper execution. + - Source credentials leave no repository config, process argument, child + phase environment, log, error, evidence, or retained workspace trace. + - OCI tags, foreign/external URLs, cross-origin credential forwarding, + disallowed registry/auth/blob host/address/port, redirects, DNS rebinding, + manifest/layer mismatches, unsafe layers, missing workspace manifests, and + expansion-limit violations fail before publication. A valid digest-pinned + snapshot produces the same manifest contract as direct Git. + - Unknown, profile-disallowed, unpinned, or definition-drifted materializers; + unauthorized structured-input resources; gateway/worker descriptor + mismatch; missing expected workspace-manifest digest; schema-invalid + inputs; undeclared egress; malformed output manifests; and mismatched + expected/reported repository or output identities fail before setup. The + request cannot select an image or command. + - Materializer credential environments/mounts, process state, runner control + plane, and staging mounts are inaccessible to later phases. Literal secret + canaries in output or retained logs fail publication. This verifies phase + teardown, not safety from a malicious operator-registered image that + intentionally transforms a credential. Provenance labels its unverified + source assertions as materializer-attested. A cache hit occurs only after + current authorization and revalidates content plus the same owner or + authorization scope, revocation epoch, canonical source, + materializer-definition, expected-output, and actual output-manifest + digests inside the same trust domain. - Setup changes establish the baseline; setup and checks receive no provider/control secrets. Credentialed provider runtimes and model tools run across the declared OS UID/process/mount boundary or broker, with disjoint config/data roots and ambient selectors removed. - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. - Block dispatch after effect selection, complete a newer cancel for the unseen attempt, then release dispatch: the command tombstone/revision check rejects it before workspace or provider creation. Duplicate commands remain idempotent and all effects stay fence-bound. @@ -550,8 +913,22 @@ docs/src/content/docs/ - Covers AE9. Known permissions receive one-invocation decisions; prompt-required profiles fail startup; unknown permission types fail the adapter. - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps. A valid or invalid result selected before a later check/evidence failure is preserved, including the valid Artifact; only a pre-candidate failure records `not_produced`. - Background swap attacks, links, mount crossings, FIFOs/devices/sockets, unstable files, and tampering between worker staging and gateway publication never expose external bytes or partial Artifacts. - - SIGKILL before and after provider spawn proves supervisor descendant death and replacement root recovery; the gateway retains one failed Task with separate termination and cleanup outcomes. -- **Verification:** A built supervised worker mutates a disposable exact-SHA repository through the fake adapter and proves authenticated revisioned dispatch, unseen-cancel tombstones, source hardening, OS credential separation, budgets, result preservation, quiescence or poisoned-boundary exit, evidence integrity, worker-crash containment, orphan-root handling, and cleanup. + - SIGKILL during materialization and before or after provider spawn proves + supervisor-owned runner/process death, credential/staging mount removal, + and replacement root recovery before readiness; unresolved resources keep + readiness false, and the gateway retains one failed Task with separate + termination and cleanup outcomes. + - Cross-filesystem staging/publication configuration fails readiness. Faults + around the final rename expose either no final workspace or the complete + validated tree, never a copy fallback or partial publication. +- **Verification:** A built supervised worker materializes equivalent + workspaces through disposable exact-SHA repositories, a local digest-pinned + OCI snapshot, and a fake registered materializer; validates one standard + manifest; mutates each through the fake adapter; and proves authenticated + revisioned dispatch, unseen-cancel tombstones, acquisition hardening and + credential teardown, budgets, result preservation, quiescence or + poisoned-boundary exit, evidence integrity, worker-crash containment, + orphan-root handling, and cleanup. ### U5. Codex backend adapter @@ -595,16 +972,42 @@ docs/src/content/docs/ - **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with trusted transports, peer identity, credential-boundary and supervisor readiness, safe startup/shutdown, tracing, and reproducible containers. - **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD10-KTD15. - **Dependencies:** U3-U6. -- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. -- **Approach:** Register only Codex and Pi. Add gateway and supervised worker entrypoints inside the private Node 22 workspace. Before readiness, validate named public TLS termination, every remote worker's mTLS/equivalent transport and pinned identity/capabilities, Unix-socket locality, store, runtimes, monotonic command storage, OS credential-boundary capability, supervisor boundary, orphan roots, trust, quotas, and resource controls. Propagate `traceparent`, then apply KTD10's small shared metadata allowlist and bounded filtering/redaction before any structured log/span processor or OTLP exporter; neither OpenInference nor backend-native attributes bypass it. Build a minimal gateway image with no provider runtime and a one-execution worker image whose init kills the complete boundary when the worker server exits, including poisoned failed-quiescence exit. +- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/unit/worker/materializers/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. +- **Approach:** Register only Codex and Pi as backend adapters and register the + built-in Git/OCI materializers plus configured external materializers through + a separate closed registry. Add gateway and supervised worker entrypoints + inside the private Node 22 workspace. Before readiness, validate named public + TLS termination, every remote worker's mTLS/equivalent transport and pinned + identity/capabilities, Unix-socket locality, store, runtimes, matching + gateway/worker materializer definition digests, image digests, schemas, + credential names, egress, limits, and OCI runner/sandbox isolation, monotonic + command storage, acquisition and provider/tool credential-boundary + capabilities, + supervisor boundary, orphan roots, trust, quotas, and resource controls. + Propagate `traceparent`, then apply KTD10's small shared metadata allowlist and + bounded filtering/redaction before any structured log/span processor or OTLP + exporter; neither OpenInference nor backend-native attributes bypass it. + Build a minimal gateway image with no provider or materializer runtime and a + one-execution worker image whose init kills the complete boundary when the + worker server exits, including poisoned failed-quiescence exit. - **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. - **Patterns to follow:** `src/core/profile/adapters/registry.ts` for explicit adapter composition, root package scripts for workspace delegation, `src/core/mcp-http-stdio-proxy.ts` for server lifecycle, `.github/workflows/ci.yml` for quality gates, and `.github/workflows/publish.yml` for immutable releases. - **Test scenarios:** - Registry exposes exactly Codex and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects OpenCode or unknown backend IDs before workspace creation. + - Materializer registry exposes built-in Git and OCI plus only configured + external IDs, resolves every external image to the configured digest, + rejects duplicates/tags/unknown IDs, and cannot be influenced by request + image, command, credential, or policy fields. - Gateway and supervised worker start from built outputs, become ready only after trusted transport/identity, credential and supervisor boundaries, dependencies, and orphan recovery pass, and stop gracefully on SIGTERM. - Gateway readiness fails for malformed auth, missing/mismatched named TLS termination, plaintext production public ingress, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. - Worker-route readiness fails for plaintext remote URL, wrong/untrusted certificate, worker identity/capability mismatch, or replayed capability; mTLS/equivalent authenticated encryption and same-host Unix sockets pass. - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable OS credential/supervisor/resource enforcement, unproved or unrecoverable orphan roots, or unavailable/incompatible Codex or Pi runtime. + - Worker readiness fails when a profile allows a materializer absent from its + route, gateway and worker definition digests differ, an external image is + not digest-pinned, an input schema or output manifest version is + unsupported, named credentials or the OCI runner/sandbox are unavailable, + the runner control plane would be visible to the workspace, or configured + egress and resource enforcement cannot be provided. - Killing or poisoning the worker server while an adapter child and invocation root exist makes the supervisor destroy the boundary; replacement readiness waits for root deletion/quarantine and never reuses it. - Trace context crosses the authenticated private call and correlates result identities using only opaque owner correlation. Exporter probes for agent, model, tool, stale-event, and error spans contain allowlisted bounded metadata but no canary secret, prompt/output, tool argument/result, file body/source fragment, raw caller identity, or cross-owner fragment; exporter failure cannot change Task status. - Gateway image contains no Codex, Pi, Git workspace, provider credential material, or worker trust private keys. @@ -619,6 +1022,14 @@ docs/src/content/docs/ - **Dependencies:** U1-U7. - **Files:** `packages/execution-service/tests/e2e/execution-gateway.test.ts`, `packages/execution-service/tests/fixtures/execution/conformance-cases.ts`, `examples/gateway/gateway.yaml`, `examples/gateway/worker.yaml`, `docs/src/content/docs/guides/execution-gateway.mdx`, `docs/src/content/docs/reference/execution-gateway-configuration.mdx`, `README.md`, `CHANGELOG.md`. - **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases, with gateway and supervised worker as separate processes. Each race fixture declares attempt/fence correlation, required durable transitions and observed effects, required happens-before edges, maximum occurrence counts, and effects forbidden after terminalization. A deliberately small test-side checker evaluates those constraints against durable records plus observed worker/process outcomes without calling the production selector. It remains coverage protection—not TLA+, a model checker, event sourcing, or a second lifecycle implementation. Document the exact extension URI and legal Agent Card/header/Message/Artifact carriers, both fixed Artifacts and four result states, retained-replay ordering, worker transport/identity, monotonic command tombstones, failed-quiescence recycling, the narrow OS credential boundary, reviewed-domain limitation, metadata-only telemetry and its separate operator access/retention, storage/HA limits, lack of execution resume, source hardening, quotas, retention, and operations. + Document all three source modes, the exact source discriminator, canonical + repository/namespace and materializer-input authorization, multi-repository + destinations, the standard workspace manifest and verification-method + labels, materializer registration, worker-derived definition digests, + profile allowlisting, digest/profile/idempotency boundaries, same-filesystem + publication, supervisor-owned runner cleanup, acquisition credential + teardown, authorization-scoped cache reuse/revocation, source-mode + capabilities, and the prohibition on caller-supplied acquisition code. - **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. - **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, Starlight guide/reference organization under `docs/src/content/docs/`, and Buzz's required-critical-action coverage rule without importing its TLA+ model or production implementation. - **Test scenarios:** @@ -626,16 +1037,50 @@ docs/src/content/docs/ - Agent Card required-extension advertisement, `A2A-Extensions`, `Message.extensions`, request `Message.metadata[uri]`, and the single fixed integrity Artifact carrier interoperate; missing/mismatched carriers and `Task.extensions` fail. - Identical retained replay after deadline expiry, quota exhaustion, readiness loss, authorization change, or profile replacement returns the original Task; changed request/schema or inconsistent original bindings conflict. - The same accepted schema, valid result, invalid result, missing result, and pre-output failure pass through Codex and Pi with identical decisions. A valid-result-then-check-failure and invalid-result-then-evidence-failure preserve the selected state and only the valid Artifact; `not_produced` remains pre-candidate only. - - Source mismatch and setup failure after worker acceptance produce the selected `Submitted -> Working -> Failed` trace; provider invocation never begins, and durable snapshots, streams, and conformance records agree. + - Direct Git object/destination/authorization mismatch, OCI + digest/manifest/namespace mismatch, registered materializer + descriptor/input/resource/expected-output mismatch, direct known-secret + disclosure, and setup failure after worker acceptance produce the selected + `Submitted -> Working -> Failed` trace; provider invocation never begins, + and durable snapshots, streams, and conformance records agree. A policy + revocation before lookup cannot consume a previously populated cache entry. - Pause dispatch after selection, complete unseen-attempt cancel, then release dispatch; the stale command creates no workspace/process. The small independent checker enforces each fixture's attempt/fence correlation, happens-before edges, maximum counts, and forbidden post-terminal effects. Deliberately bad traces that still contain every required action name fail for wrong order, wrong fence, duplicate-over-maximum effects, and an extra stale dispatch after terminalization. - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; one worker serializes admitted work. - - Gateway restart, reconnect, ambiguous dispatch, duplicate/out-of-order commands/events, worker crash, lease expiry, cancellation, provider failure, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome without provider reattachment or replay. - - A child that calls `setsid` and ignores graceful signals forces termination unknown/failed, poisoned-worker exit, supervisor boundary destruction, and replacement orphan recovery before readiness; no next reservation is accepted by the poisoned worker. + - Gateway restart, reconnect, ambiguous dispatch, duplicate/out-of-order + commands/events, worker crash, lease expiry, cancellation, provider failure, + evidence truncation, logical expiry, and cleanup failure preserve one + truthful terminal outcome without provider reattachment or replay. + - SIGKILL during external materialization and before or after provider spawn + forces supervisor-owned runner/process death, credential/staging mount + removal, and replacement orphan recovery before readiness. A child that + calls `setsid` and ignores graceful signals forces termination + unknown/failed, poisoned-worker exit, supervisor boundary destruction, and + replacement orphan recovery; no poisoned worker accepts a next reservation. - Public plaintext, wrong TLS boundary, private plaintext, wrong certificate/worker identity, and capability replay fail readiness/dispatch; configured TLS, mTLS/equivalent overlay, and same-host Unix socket cases pass. - Both adapters block model-tool probes of parent/sibling environments, procfs/process listings, known/discovered backend roots, and network secret exfiltration under the OS credential boundary. Environment filtering alone is never accepted as proof, and hostile-source/cross-tenant claims remain rejected. - End-to-end exporter capture repeats the agent/model/tool/stale-event/error canary and cross-owner probes, proving only bounded allowlisted metadata and opaque owner correlation cross the telemetry boundary while Task/Artifact access and retention remain independent. - - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, repository Pi extensions, and unrestricted built-ins remain blocked within the documented reviewed-source boundary. - - Examples validate with production schemas and use only secret variable names. Docs state the two transport boundaries, one gateway replica, one execution per worker, reviewed trust domain, narrow credential isolation versus deferred hostile-code isolation, metadata-only telemetry with its fixed pre-processor allowlist and separate operator access/retention, runtime floors, and ephemeral provider sessions. + - Redirect/DNS-rebinding and unauthorized-resource cases cover every Git and + OCI registry/auth/manifest/blob connection. Secondary Git fetch, OCI tag, + foreign/external layer URL, cross-origin credential forwarding, + layer/manifest mismatch, unregistered or unpinned materializer, undeclared + materializer egress, malicious output manifest, resource exhaustion, + malicious file types/link swaps, repository Pi extensions, and unrestricted + built-ins remain blocked within the documented reviewed-source boundary. + - Same-filesystem publication succeeds by atomic rename; a cross-filesystem + staging root fails readiness and fault injection never observes a partial + final tree or copy fallback. Provenance distinguishes worker-verified and + trusted-service identities from materializer-attested claims. + - Examples validate with production schemas and use only secret variable + names. Docs state the three source modes and exact discriminator, standard + workspace manifest and provenance labels, materializer registry/profile + boundary, worker-derived definition digest, resource authorization and + cache-revocation boundary, prohibition on caller-supplied acquisition code, + same-filesystem publication, supervisor-owned runner cleanup, acquisition + credential teardown, two transport boundaries, one gateway replica, one + execution per worker, reviewed trust domain, narrow credential isolation + versus deferred hostile-code isolation, metadata-only telemetry with its + fixed pre-processor allowlist and separate operator access/retention, + runtime floors, and ephemeral provider sessions. - Opt-in real-provider smoke tests record backend/runtime and credential-boundary prerequisites, skipping only when a named prerequisite is absent. - **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor; full suites and docs pass; the official A2A client exercises every advertised operation including the `Submitted -> Working -> Failed` source/setup path; exporter capture proves the telemetry canary/cross-owner contract; the independent checker rejects all-name-present traces with wrong order/fence/multiplicity or forbidden stale dispatch; and release evidence records each available real backend plus explicit skipped prerequisites. @@ -645,16 +1090,16 @@ docs/src/content/docs/ | Gate | Applies to | Required evidence | |---|---|---| -| Contract generation | U1 | Exact extension URI/carriers, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | -| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, and cleanup. | -| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | +| Contract generation | U1 | Exact extension URI/carriers, closed workspace source discriminator and manifest, algorithm-qualified materializer/profile/input/output digest preimages and vectors, verification-method vocabulary, authorization-scope/revocation fields, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | +| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, source authorization/cache revocation, and cleanup. | +| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, direct Git/OCI/registered materialization, worker-derived registry digests, acquisition credential teardown, supervisor-owned materializer runners, same-filesystem atomic publication, workspace-manifest provenance labels, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | | Backend conformance | U5-U8 | One shared suite passes against Codex and Pi, including the versioned schema subset, four result states, valid/invalid preservation across later failure, integrity Artifact carrier, and structured-result Artifact rule. | -| Credentialed provider smoke | U5-U6, U8 | Each available provider mutates a disposable exact-SHA repository while adversarial tool probes cannot cross the OS credential boundary; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | +| Credentialed provider smoke | U4-U6, U8 | An available operator-trusted registered materializer and each available provider mutate a disposable immutable workspace while adversarial later-phase probes cannot directly access acquisition/provider credential environments, mounts, processes, roots, or runner control planes and literal canaries remain absent; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | | A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes required-extension negotiation and legal carriers, immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, retained replay, cancel races, expiry, and owner isolation without `Task.extensions`. | -| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, Git SSRF controls, OS provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | +| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, exact source-resource authorization, per-connection Git/OCI SSRF and credential-origin controls, authorization-scoped cache revocation, digest-pinned registered materializers, schema/manifest validation, truthful provenance labels, acquisition and provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | | Lifecycle trace conformance | U8 | The small test-side checker, independently of production selectors, validates attempt/fence correlation, required happens-before edges, maximum occurrence counts, and forbidden post-terminal effects against durable records plus observed worker/process outcomes; all-name-present bad traces fail for wrong order/fence/multiplicity and stale post-terminal dispatch. | | Telemetry safety | U7-U8 | Exporter capture across agent, model, tool, stale-event, and error spans proves the pre-processor allowlist and bounded redaction exclude prompt/output/tool/source/file content, canary secrets, raw identities, and cross-owner fragments while retaining only bounded operational metadata and opaque owner correlation. | -| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both container builds pass. | +| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, backend and materializer registry readiness, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both service container builds pass. | | Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | | Documentation | U8 | `bun run docs:build` passes and examples validate against current schemas. | @@ -673,7 +1118,17 @@ The authoritative behavioral proof is the built-process E2E path with the offici - Authentication and bounded parsing precede owner-scoped retained lookup; identical replay uses stored original bindings before mutable admission, while current authorization/profile/readiness/deadline and quota apply only to atomic new claims. - Production public ingress uses its named TLS boundary, remote worker routes authenticate and encrypt peers with worker identity/capability binding, and same-host Unix sockets are the only non-network alternative; unprotected remote endpoints fail readiness. - Cancellation/deadlines use monotonic worker command tombstones and one native abort. Stale dispatch cannot create work, and failed quiescence poisons and exits the worker so supervisor destruction and replacement orphan recovery precede new admission. -- Source hardening, the OS-enforced provider/tool credential boundary, phase-scoped secrets, disabled repository Pi extensions/unrestricted built-ins, one-execution reviewed-domain policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end without claiming hostile-source/cross-tenant isolation. +- Workspace source validation and exact resource authorization, + per-connection direct Git/OCI controls, worker-derived registered-materializer + digests, the standard workspace manifest and truthful provenance labels, + authorization-scoped cache revocation, same-filesystem atomic publication, + supervisor-owned runner cleanup, acquisition credential teardown, + OS-enforced provider/tool credential boundary, phase-scoped secrets, disabled + repository Pi extensions/unrestricted built-ins, one-execution + reviewed-domain policy, resource limits, Artifact race defenses, + completeness, provenance, and authenticated expiry are enforced end to end + without accepting caller acquisition code or claiming hostile-source or + cross-tenant isolation. - Metadata-only telemetry is filtered through the fixed allowlist and bounded redaction before processing/export; canary secrets, content, raw caller identities, and cross-owner fragments never reach exporters, and only opaque owner correlation crosses the separately governed operator boundary. - Required source/setup and race traces satisfy attempt/fence, happens-before, maximum-count, and forbidden-post-terminal constraints in the independent test-side checker; all-name-present malformed traces fail without introducing a parallel lifecycle implementation. - Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. @@ -682,11 +1137,31 @@ The authoritative behavioral proof is the built-process E2E path with the offici ### Per unit -- U1: Standard extension carriers, integrity/structured-result Artifact schemas, four result states, original claim digests, command revisions/tombstones, fence rules, typed failures, and fixtures are generated and stable. +- U1: Standard extension carriers, closed workspace source/manifest contracts, + algorithm-qualified materializer/profile/input/output digest preimages, + verification and authorization vocabulary, integrity/structured-result + Artifact schemas, four result states, original claim digests, command + revisions/tombstones, fence rules, typed failures, and fixtures are generated + and stable. - U2: Trusted ingress, auth, opaque owner isolation, retained-claim-first replay, original bindings, atomic new admission, CAS settlement, pagination, startup recovery, quotas, Artifact access, tombstones, and cleanup pass fault injection. - U3: Every advertised A2A operation agrees across stream and lookup while extension negotiation, replay ordering, authenticated worker routes, fencing, monotonic cancellation, and races preserve one Task. -- U4: Worker command state, OS credential separation, supervision, poisoned-exit/orphan recovery, and dispatch/source/setup/action/check/quiescence/evidence/cleanup pass malicious, crashed, and faulted scenarios. +- U4: Worker command state, exact source authorization, direct + Git/OCI/registered materialization, workspace-manifest validation and + provenance classification, authorization-scoped cache revocation, + same-filesystem atomic publication, acquisition and provider credential + separation, supervisor-owned runner cleanup, poisoned-exit/orphan recovery, + and dispatch/materialization/setup/action/check/quiescence/evidence/cleanup + pass malicious, crashed, and faulted scenarios. - U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, result preservation, OS credential separation, native evidence, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. - U6: Pi strict RPC/framing, terminating result, exact policy tools, disabled repository extensions/built-ins, OS-isolated credential store/provider runtime, result preservation, settlement, stats, abort, and process cleanup pass verification. -- U7: Closed registry, trusted transport/identity readiness, credential/supervisor capability gating, poisoned-worker recycling, metadata-only pre-export telemetry controls, Node-version separation, tracing, shutdown, containers, and release artifacts work from built outputs. -- U8: Cross-backend E2E, standard A2A carriers, retained replay, selected source/setup transitions, independent race-trace constraints, telemetry canary/cross-owner probes, transport and credential abuse cases, command/quiescence races, examples, operator docs, changelog, and release evidence are complete. +- U7: Closed backend and materializer registries, trusted + transport/identity/readiness, acquisition/provider credential and supervisor + capability gating, poisoned-worker recycling, metadata-only pre-export + telemetry controls, Node-version separation, tracing, shutdown, containers, + and release artifacts work from built outputs. +- U8: Cross-backend E2E, all three workspace source modes, standard manifest + provenance, acquisition credential teardown, standard A2A carriers, retained + replay, selected materialization/setup transitions, independent race-trace + constraints, telemetry canary/cross-owner probes, transport and credential + abuse cases, command/quiescence races, examples, operator docs, changelog, + and release evidence are complete. diff --git a/docs/research/harbor-repository-materialization.md b/docs/research/harbor-repository-materialization.md new file mode 100644 index 00000000..d4708d4c --- /dev/null +++ b/docs/research/harbor-repository-materialization.md @@ -0,0 +1,179 @@ +# Harbor repository materialization lessons + +## Decision + +Borrow Harbor's content-addressed package cache, sparse Git reads, staged publication, +and prebuilt-environment option. Do not copy its task model as the execution gateway's +workspace contract. + +Harbor does not expose a first-class, general-purpose "repositories in a workspace" +layer. It first downloads a Harbor **task package**. The task then defines an execution +environment with a Dockerfile, Compose file, or prebuilt image. Acquisition of the +repository the agent edits is therefore benchmark- and task-owned: it may be baked into +an image, cloned by a Dockerfile, copied as task content, or otherwise prepared by the +task author. + +For AllAgents, repository and workspace provenance must remain explicit in the public +execution request and terminal evidence. Custom acquisition should be an +operator-registered, digest-pinned materializer behind the worker protocol, not an +arbitrary caller-supplied image or setup script. + +## What Harbor fetches + +### Task packages from Git + +Harbor's `GitRepoRegistryClient` resolves a dataset registry ref, inspects the selected +commit's tree without checking out blobs, and identifies task directories containing +`task.toml`. When task content is requested, `TaskClient` groups requested task paths by +Git URL and performs one shallow, no-checkout clone per URL. It uses a blobless partial +clone where supported, configures sparse checkout for only the selected task paths, +fetches each requested commit at depth one, checks it out, and records the resolved +commit. + +This is efficient for a large repository containing many independent Harbor tasks. It +is not a mechanism for assembling several application repositories into one agent +workspace. + +Harbor also accepts an omitted commit or a mutable ref and resolves it to a commit. +That is convenient for an interactive local benchmark CLI, but it is weaker than the +AllAgents gateway requirement that an accepted request already name immutable source. + +### Task packages from the package registry + +Package-registry tasks are downloaded as tar archives into a cache keyed by the task's +content hash. A direct `sha256:` reference can hit that cache without registry +resolution. Dataset manifests likewise refer to task packages by SHA-256 digest. This +is the closest Harbor analogue to an OCI workspace snapshot: a content-addressed, +reusable input bundle. + +Before publishing a Git task directory, Harbor stages it in a temporary directory, +rejects source paths containing symlinks, materializes only relative symlinks that stay +inside the task root, rejects cycles and special entries, then replaces the target. +Those containment and publish-after-validation properties are useful for any cached +workspace artifact. + +### The repository edited by the agent + +Once the task package is present, Harbor asks the selected environment provider to +start the task's `environment/` definition. For Docker this can be: + +- `[environment].docker_image`; +- `environment/Dockerfile`; or +- `environment/docker-compose.yaml`. + +The task format deliberately leaves the environment flexible. Harbor builds the +Dockerfile/Compose definition or uses the prebuilt image, then runs the agent in that +environment. There is no core repository-source schema carrying URL, exact commit, +destination, and per-repository provenance. + +The Multi-SWE-bench adapter makes the distinction concrete. Each generated task uses +an upstream `mswebench/...:pr-...` base image that already contains the repository at +`/home/{repo_name}`. Its Dockerfile creates `/workspace/{repo_name}` as a symlink and +sets that as `WORKDIR`; Harbor itself never clones that application repository. + +## Lessons for the AllAgents execution gateway + +### Adopt + +1. **Separate descriptor acquisition from execution.** Resolve and validate immutable + inputs before starting the coding-agent runtime. +2. **Use content-addressed caches.** Key reusable workspace snapshots by a digest of + normalized source identities, materializer version/digest, setup policy, current + authorization scope, and revocation epoch rather than a mutable name. Reauthorize + before lookup and make an old epoch ineligible after revocation. +3. **Avoid downloading irrelevant content.** For Git-backed descriptor catalogs, + Harbor's tree-only discovery and sparse checkout are sound optimizations. For an + application repository, use partial/shallow acquisition only when it preserves the + required commit and evidence semantics. +4. **Stage, validate, then publish.** Materialize into a temporary location, enforce + path/link/type/size limits, verify every requested identity, and atomically expose + the completed workspace to the worker. +5. **Support prebuilt immutable artifacts.** A digest-pinned OCI workspace snapshot is + the scalable path for very large repositories and expensive setup. + +### Adapt + +Keep a first-class workspace manifest instead of hiding source inside an environment +image. Each materialized repository should retain at least: + +- canonical source URL or snapshot identity; +- requested and resolved immutable commit or OCI digest; +- destination path and optional source subdirectory; +- materializer identity and version/digest; +- resulting tree/content identity; +- cache hit/miss and completeness facts. + +Use three explicit source modes: + +1. **Direct Git repositories** for the normal case, each with an exact commit and + collision-free destination. +2. **OCI workspace snapshots** for large, preassembled workspaces, referenced by digest + rather than tag and accompanied by a signed/validated workspace manifest. +3. **Operator-registered materializers** for JFrog, unusual monorepos, generated source, + or organization-specific setup. A request selects a configured materializer ID, + pins the expected workspace-manifest digest, and supplies validated, + resource-authorized structured inputs. The operator configuration pins the builder + image by digest, the worker derives the non-secret definition digest, credentials are + scoped only to materialization, and the builder must produce the standard workspace + manifest before the agent starts. The builder is operator-trusted deployment code; + deployments that cannot grant that trust need a broker or stronger acquisition + service. + +This retains Harbor's useful task-owned flexibility without allowing a caller to choose +an arbitrary executable image or shell script inside the trusted worker. + +### Do not copy + +- Mutable Git refs, `HEAD`, image tags, or package `latest` as accepted execution + identities. +- Harbor's broad Git transport set (`http`, `ssh`, and `git` as well as HTTPS) at a + remote service boundary. The gateway should keep canonical credential-free HTTPS, + destination-policy revalidation, disabled redirects/helpers/filters/hooks/submodules, + and exact commit verification. +- A non-fatal Git LFS miss. If declared workspace content cannot be materialized, + preparation must fail before provider execution. +- Hashing a prebuilt image reference string as environment identity. Resolve and pin + the OCI manifest digest. +- Arbitrary task-authored Dockerfiles, Compose files, or public-network setup as caller + input. Harbor runs benchmark definitions trusted by the evaluator; the gateway + accepts remote service requests and has a different threat boundary. +- Treating a container image alone as sufficient provenance. An image can carry the + correct files while obscuring which repositories, commits, generator, and setup + produced them. + +## Recommended boundary + +The worker should execute a dedicated materialization phase before any harness starts: + +1. Validate the normalized workspace request, exact source-resource authorization, + configured materializer, and current authorization scope before any cache lookup. +2. Resolve phase-scoped source credentials without exposing them to setup, the model, + or later evidence. +3. Populate a worker-owned staging directory on the final publication filesystem or + pull and unpack a digest-pinned workspace snapshot there. +4. Verify repository commits, paths, limits, content, the expected manifest digest, + and the standard workspace manifest; distinguish worker-verified identities from + materializer-attested claims. +5. Stop the acquisition process, revoke credentials, remove its mounts and runner + resource, and retain only the validated host-owned staging tree. +6. Atomically rename that tree into the final workspace, record provenance, run + operator-owned setup, record the post-setup baseline, and only then launch the + harness-specific worker runtime. + +The practical conclusion is narrow: Harbor is strong evidence for content-addressed +input bundles and environment-provider indirection. It is not evidence for making +repository acquisition opaque or task-defined in the AllAgents public contract. + +## Primary sources + +Inspected Harbor commit +[`b83e7686999a18ba90a8603794d7d18d42cab010`](https://github.com/harbor-framework/harbor/tree/b83e7686999a18ba90a8603794d7d18d42cab010): + +- [`src/harbor/registry/client/git_repo.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/registry/client/git_repo.py) — ref resolution, tree-only discovery, and sparse registry checkout. +- [`src/harbor/tasks/client.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/tasks/client.py) — Git/local/package task acquisition, content-hash cache, LFS behavior, safe staging, and resolved commits. +- [`src/harbor/models/task/id.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/models/task/id.py) — Git, local, and package task identities. +- [`src/harbor/models/dataset/manifest.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/models/dataset/manifest.py) — digest-addressed dataset task references. +- [`docs/content/docs/tasks/index.mdx`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/docs/content/docs/tasks/index.mdx) — task structure and Docker image/Dockerfile/Compose environment contract. +- [`src/harbor/environments/definition.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/environments/definition.py) — environment selection and content identity. +- [`src/harbor/environments/docker/docker.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/environments/docker/docker.py) — prebuilt-image versus build behavior and container startup. +- [`adapters/multi-swe-bench/README.md`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/adapters/multi-swe-bench/README.md) and its [`environment/Dockerfile`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/adapters/multi-swe-bench/src/multi_swe_bench_adapter/task-template/environment/Dockerfile) — application repository supplied by an upstream prebuilt image rather than cloned by Harbor core. From 108174398d0fb1d1d67fd8d46edc07c441bee567 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Fri, 18 Sep 2026 21:57:48 +1000 Subject: [PATCH 07/44] docs(architecture): define GitHub source credentials --- ...-agent-execution-through-an-a2a-gateway.md | 109 ++- ...0837-feat-coding-execution-gateway-plan.md | 662 +++++++++++++++--- .../source-credential-broker-precedents.md | 260 +++++++ 3 files changed, 929 insertions(+), 102 deletions(-) create mode 100644 docs/research/source-credential-broker-precedents.md diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index d8deb818..f9357c13 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -149,24 +149,117 @@ separate from the agent runtime and never exposes that runner's control socket to setup or model tools. Phase isolation prevents later code from receiving the materializer's credentials or mounts, but it cannot make a malicious operator-registered image safe from credentials intentionally given to it. -Operators must review and pin that image; deployments that do not trust it need -a credential broker or stronger acquisition service that never reveals reusable -credentials to the materializer. +Operators must review and pin that image. A deployment that will not trust it +with credentials needs a separately versioned broker or central snapshot +protocol, which is deferred from the initial architecture. The canonical source request enters caller idempotency. The resolved materializer definition digest enters the effective-profile binding, and both the definition and output-manifest digests enter terminal provenance. New-claim source authorization always runs before cache lookup. Cache metadata and keys include the canonical source, materializer-definition digest, authorization -scope digest and revocation epoch, and configured trust domain. Reuse requires -manifest and content revalidation under the current authorization scope; -revocation advances the epoch and makes the old namespace unusable. +scope digest and revocation epoch, configured trust domain, and, for GitHub App +sources, the current installation-entitlement generation. Authenticated App +lifecycle webhooks and bounded control-plane reconciliation advance that +generation on uninstall, suspension, or repository-selection change. Unknown +or stale installation state fails cache authorization rather than reusing an +entry. Reuse also requires manifest and content revalidation under the current +authorization scope. Cache metadata preserves the original acquisition +provider metadata; terminal provenance distinguishes `cache_hit`, that original +provider, and the provider selected by current policy instead of claiming that +the current provider performed acquisition. Secret resolution and token minting +remain cache-miss-only. This keeps Harbor's useful separation between content-addressed task acquisition and environment execution without adopting task-owned opaque source. The comparison is recorded in [Harbor repository materialization lessons](../research/harbor-repository-materialization.md). +### Resolve GitHub source credentials from trusted deployment policy + +The public source request remains credential-free and does not select a +credential provider. The trusted acquisition boundary normalizes the repository +host and resolves a provider from operator configuration. `github.com` selects +the built-in GitHub source backend; GitHub Enterprise Server hosts require an +explicit host and API mapping because a custom hostname does not identify its +provider. The effective profile authorizes the canonical repository and provider +entitlement before cache lookup. + +For GitHub repositories, an ordered policy may prefer a GitHub App and permit a +local GitHub CLI fallback. The App provider is applicable only when trusted +operator configuration maps the requested repository to an installation ID; +`@octokit/auth-app` does not discover that mapping. It mints an installation +token scoped only to that repository, read-only contents permission, and its +GitHub expiry. A trusted-local CLI provider is pinned to one configured +non-secret account, which participates in its entitlement and effective-profile +digests. It may run only when no App installation mapping applies, as +`gh auth token --hostname --user `, with `GH_TOKEN`, +`GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` removed +from its environment. Failure to resolve the configured account fails that +provider. This is eligibility fallback, not authentication retry: after an App +provider is selected, configuration, authentication, minting, permission, +repository, rate-limit, or service failure terminates acquisition and never +falls through to the broader user identity. + +When AllAgents owns GitHub App token minting, a trusted control-plane +credential-provider component uses the focused `@octokit/auth-app` package +rather than implementing App JWT, clock-skew, expiry, and installation-token +renewal itself. Every cache-miss acquisition requests a fresh installation token +with auth-app cache bypass (`refresh: true`). Its remaining lifetime must be +strictly greater than the acquisition deadline plus the configured clock-skew +margin, and the delivery lease cannot outlive the token. Readiness rejects an +acquisition-phase ceiling that can exceed a fresh token's safe lifetime. Git +remains the repository transport; the full Octokit client is not required. + +The initial remote architecture is one central token-minter path. The +gateway/control-plane credential-lease controller is authoritative: an +authenticated worker requests credentials only for its active attempt and +fence; the controller rechecks the current command revision, lease epoch, +tombstone, and fence in durable dispatch state, then derives the +effective-profile digest, selected provider, host/API-mapping digest, +installation ID, canonical repository, operation, worker route and identity, +and expiry from durable dispatch and policy state. It issues a single-use, +non-durable grant/response and, when the minter is separate, requires its +configuration digest to agree with those selected bindings. Replay, worker +field substitution, stale command state, and configuration disagreement fail +closed. + +The authenticated delivery lease and channel bind that derived state to the +worker identity, attempt, lease epoch, command revision, fence, operation, and +expiry. Those bindings do not alter the bearer token: after delivery, the token +is enforceably scoped only by GitHub to the repository, read-only contents +permission, and token expiry. The remote worker never receives the App private +key, and only its one-shot acquisition child receives the token. A remote App +profile fails readiness when the central minter, authoritative lease controller, +fresh-token lifetime check, or authenticated non-durable delivery capability is +absent. A versioned central snapshot-delivery protocol is deferred and is not +an initial readiness alternative. + +The local GitHub CLI provider and remote lease path expose their resolved tokens +only to the one-shot acquisition process. Neither exposes credentials to setup, +the coding-agent runtime, model tools, repository configuration, process +arguments, logs, evidence, or the published workspace. Public failures use only +deterministic coarse source-auth code, safe reason, and retryability: +`source_auth_unavailable/no_eligible_provider`, +`source_auth_denied/installation_repository_denied`, and +`source_auth_failed` with `app_configuration_invalid`, +`app_authentication_failed`, `app_mint_failed`, or +`trusted_local_cli_failed` are not retryable; `source_auth_failed` with +`provider_rate_limited` or `provider_unavailable` is retryable. Provider, +installation, and account identifiers are non-secret but operator-only +provenance. Cache-hit provenance separately records `cache_hit`, the original +acquisition provider, and current policy selection. + +A standalone network broker is not required for trusted local execution: the +CLI provider may be a subprocess and a trusted co-located deployment may host +the App minter and lease controller inside its control plane. Remote routes +still use the same authenticated central-minter contract; the minter may be +split into a standalone service when private-key isolation, independent audit, +scaling, or blast-radius requirements demand it. + +The supporting precedents and trust-boundary analysis are recorded in +[Source credential broker precedents](../research/source-credential-broker-precedents.md). + ### Persist Task truth, not live provider execution The gateway durably stores Task identity, idempotency claims, terminal status, @@ -428,6 +521,10 @@ but their product and ownership model requires a separate decision. profiles decide which callers may select them. - Direct Git, OCI snapshots, and registered materializers converge on one validated workspace manifest and provenance contract. +- GitHub source credentials are selected by trusted host/profile policy rather + than caller input. GitHub App is preferred when applicable; GitHub CLI is a + local-only eligibility fallback and never masks an App authentication or + authorization failure. - Deployments that enable external materializers must operate their image, schema, credential, network, resource, and cache policies as worker configuration. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 86c83707..5f95d109 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -66,6 +66,13 @@ The two initial runtimes expose different programmatic contracts. Codex provides versioned workspace source mode; custom acquisition uses only operator-registered, digest-pinned materializers allowed by the profile. Governs R6, R11-R13, R16-R18, R21-R22. +- **Resolve source credentials from trusted host and profile policy.** Callers + never select a credential provider. For GitHub, prefer an applicable GitHub + App installation and permit GitHub CLI only as an explicit trusted-local + fallback when no installation applies; never fall back after a selected App + provider fails. When AllAgents owns App token minting, use + `@octokit/auth-app` rather than a custom minter. (session-settled: + user-directed.) Governs R6, R12-R13, R16, R21-R22. ### Requirements @@ -114,6 +121,60 @@ The two initial runtimes expose different programmatic contracts. Codex provides materializer output must match the request's expected workspace-manifest digest before publication. - R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, structured logs/spans before processing or export, and every model-initiated command or tool environment. Credentialed profiles additionally require an OS-enforced provider/tool credential boundary: the credential-bearing provider runtime and model-invoked tools use distinct UID/process/mount policy that prevents tool access to provider processes, procfs entries, and backend config/data roots, or an equivalent credential broker keeps reusable credentials out of the agent runtime. Worker readiness fails when the declared boundary cannot be proved; environment filtering alone is not credential isolation. + Source credential selection is server-side deployment policy, not caller + input. The acquisition boundary maps normalized repository hosts to source + backends; `github.com` selects the built-in GitHub backend, while GitHub + Enterprise Server hosts require explicit operator host/API mappings. Profiles + name an ordered provider policy and authorize its non-secret entitlement + before cache lookup. An App provider is applicable only when trusted operator + configuration maps the repository to an installation ID; auth-app does not + discover installations. Authenticated lifecycle webhooks plus bounded + reconciliation advance an installation-entitlement generation on uninstall, + suspension, or repository-selection change. Unknown or stale installation + state fails cache authorization. Cache metadata preserves the original + acquisition-provider metadata, while hit provenance separately records + `cache_hit`, that original provider, and current policy selection/entitlement. + Secret lookup and token minting remain cache-miss-only. + For a cache-miss GitHub acquisition, an applicable configured App installation + is preferred. Focused `@octokit/auth-app` minting uses `refresh: true` to + produce a fresh token scoped only to the authorized repository, read-only + contents permission, and GitHub expiry. Remaining lifetime must be strictly + greater than the acquisition deadline plus clock-skew margin, and the + credential lease cannot outlive the token. Readiness rejects an acquisition + ceiling that can exceed a fresh token's safe lifetime. A trusted-local + `github-cli` provider may run only when no App installation mapping applies; + it is pinned to a configured non-secret account included in entitlement and + effective-profile digests, invokes + `gh auth token --hostname --user ` without `GH_TOKEN`, + `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, or `GITHUB_ENTERPRISE_TOKEN`, and + fails if that account cannot be resolved. After App selection, no failure + falls through to `gh`. + The initial remote path requires a trusted central token minter and + authoritative gateway/control-plane credential-lease controller. The + authenticated worker requests only by active attempt and fence. From durable + dispatch and policy state, the controller rechecks active command revision, + lease epoch, tombstone, and fence, then derives the effective-profile digest, + selected provider, host/API-mapping digest, installation ID, canonical + repository, operation, worker route and identity, and expiry. It issues and + atomically consumes a single-use non-durable grant/response; a separately + deployed minter must agree with the selected configuration digest. Replay, + substitution, stale state, and controller/minter digest disagreement fail + closed. The authenticated lease/channel binds the derived repository and + provider state to worker identity, attempt, lease epoch, command revision, + fence, operation, and expiry; those are not token claims. The remote worker + never receives the App private key. Readiness fails without this complete + path. Versioned central snapshot delivery is deferred and is not an initial + readiness alternative. + Public failures expose only deterministic coarse code, safe reason, and + retryability; provider, installation, and account identifiers remain + operator-only. `source_auth_unavailable/no_eligible_provider`, + `source_auth_denied/installation_repository_denied`, + `source_auth_failed/app_configuration_invalid`, + `source_auth_failed/app_authentication_failed`, + `source_auth_failed/app_mint_failed`, and + `source_auth_failed/trusted_local_cli_failed` are not retryable. + `source_auth_failed/provider_rate_limited` and + `source_auth_failed/provider_unavailable` are retryable. Materializer IDs are defined in an operator-owned deployment registry. The gateway holds only the non-secret ID, bounded input schema, expected definition digest, expected output-manifest version, and required worker @@ -133,8 +194,8 @@ The two initial runtimes expose different programmatic contracts. Codex provides provider execution, or model tools. The registered image is operator-trusted deployment code: phase isolation protects later phases but cannot make a malicious registered image safe from credentials deliberately given to it. - Deployments requiring that stronger claim use a credential broker or - acquisition service that withholds reusable credentials. + Deployments requiring that stronger claim are deferred pending a separately + versioned broker or central snapshot-delivery protocol. - R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. - R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. - R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves the complete invocation process set quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. If bounded termination escalation cannot prove quiescence, the worker persists termination as unknown/failed, poisons admission, and exits so the external supervisor destroys the complete process boundary; replacement readiness performs orphan recovery before accepting work. The same supervisor boundary handles a worker crash. @@ -247,10 +308,30 @@ The two initial runtimes expose different programmatic contracts. Codex provides peer identity, readiness fails; a same-host Unix worker socket is accepted. Given a credentialed reviewed-domain profile, model tools cannot inspect provider process environments, process listings, backend config/data roots, - or exfiltrate provider/control credentials across the configured OS - boundary. Given a registered materializer with source credentials, setup, - provider processes, and model tools have no access to its process, runner - control socket, credential environment/mounts, or staging root after + or exfiltrate provider/control credentials across the configured OS boundary. + Given a GitHub repository, a trusted operator repository-to-installation + mapping wins over GitHub CLI; auth-app never discovers the installation. + Without a mapping, only an explicitly enabled trusted-local provider may + invoke the configured account through + `gh auth token --hostname --user ` with all four ambient + GitHub token variables absent. Any selected-App failure never invokes `gh`. + A remote worker requests a credential only by its active attempt/fence; the + controller derives all provider/repository/route bindings, rechecks current + command state, consumes one single-use grant, and rejects replay, + substitution, stale state, or minter configuration-digest disagreement. + The fresh token has repository/read-only/expiry scope only; the authenticated + lease carries worker, attempt, lease epoch, command revision, fence, + operation, and expiry bindings. Near-expiry cached auth-app output is bypassed + with `refresh: true`, lease expiry never exceeds token expiry, and an unsafe + acquisition ceiling fails readiness without CLI fallback. Authenticated App + lifecycle webhooks and bounded reconciliation invalidate old entitlement + generations; unknown or stale state cannot authorize a cache hit. Cache-hit + provenance distinguishes the original acquisition provider from current + policy selection. Every source-auth failure maps to the specified safe + code/reason/retryability, while provider, installation, and account identities + remain operator-only. Given a registered materializer with source credentials, + setup, provider processes, and model tools have no access to its process, + runner control socket, credential environment/mounts, or staging root after materialization, and direct known-secret canaries are absent from retained logs, evidence, and published workspace files. The registered materializer remains operator-trusted code; hostile-materializer, hostile-source, and @@ -291,6 +372,9 @@ The two initial runtimes expose different programmatic contracts. Codex provides - Push-notification configuration, gRPC, JSON-RPC transport, and A2A extended Agent Cards. - AHP server/client surfaces, long-lived interactive sessions, and client-contributed tools. - Optional ATIF conversion after the format and tooling mature. +- Versioned central source-snapshot acquisition and delivery; the initial + remote GitHub path is central token minting plus an authenticated non-durable + delivery lease. **Outside this product's identity** @@ -302,6 +386,9 @@ The two initial runtimes expose different programmatic contracts. Codex provides - [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) - [AHP decision inputs](../research/agent-host-protocol-decision-inputs.md) - [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) +- [GitHub source credential broker precedents](../research/source-credential-broker-precedents.md) +- [GitHub App installation access tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) +- [`@octokit/auth-app`](https://github.com/octokit/auth-app.js) - [AI Evals ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) - [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) - [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) @@ -344,12 +431,16 @@ The two initial runtimes expose different programmatic contracts. Codex provides selectors for structured materializer inputs. New admission authorizes the fully canonicalized resource and credential entitlement before cache lookup. Resolve a versioned canonical `EffectiveProfileIntent` containing the - selected materializer definition digest, authorization-scope digest, and - source-authorization revocation epoch when applicable; compute its digest - without resolved secrets or per-attempt state and persist it with the - canonical caller request and result-schema digest. Retained replay compares - those stored original bindings and never substitutes or re-resolves current - policy. Governs R6, R11-R16, R21-R22. + selected materializer definition digest, authorization-scope digest, + source-authorization revocation epoch, provider policy and pinned CLI account, + and current GitHub App entitlement generation when applicable; compute its + digest without resolved secrets or per-attempt state and persist it with the + canonical caller request and result-schema digest. Unknown or stale App + entitlement state fails cache authorization. Cache metadata retains original + acquisition-provider metadata, while cache-hit provenance records `cache_hit` + plus current policy selection separately. Retained replay compares stored + original bindings and never substitutes or re-resolves current policy. + Governs R6, R11-R16, R21-R22. - KTD10. **Keep durable evidence and operational telemetry as separate bounded layers.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded evidence layer; neither Git nor provider evidence is promoted as exact causality when incomplete. Telemetry is a third, non-durable metadata-only channel: one small shared pre-export sanitizer applies an explicit operational-metadata allowlist plus bounded filtering/redaction before every structured log or span processor, and only opaque owner correlation may cross the separately governed operator boundary. OpenInference and backend-native attributes receive no bypass. This is an export guard, not a telemetry framework or alternate evidence store. Governs R13, R16-R19. - KTD11. **Treat Codex and Pi as the complete initial backend set.** Codex lands first; Pi lands second against the established contract; OpenCode is deferred. (session-settled: user-directed.) Governs R10. - KTD12. **Separate terminal integrity from optional evidence bodies.** The fixed `allagents.execution-integrity` Artifact validates identity, action outcome, the four-state structured-result record, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance before terminal publication. `not_produced` applies only before result-candidate production. Once validation selects `valid` or `invalid`, a later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the separate fixed structured-result Artifact while retaining the later phase as the primary Task failure. Predictable optional-body truncation/redaction may preserve completion; failure that breaks the integrity kernel fails in the evidence phase. Governs R3-R4, R17-R18. @@ -357,11 +448,13 @@ The two initial runtimes expose different programmatic contracts. Codex provides closed `kind`-discriminated workspace-source union and one output manifest; reject unknown kinds and cross-variant fields. The built-in Git path accepts canonical HTTPS repository identities and full commit IDs only, uses - hermetic Git configuration, disables redirects, proxies, helpers, hooks, - filters, LFS smudge, submodule recursion, alternates, and non-HTTPS - protocols, revalidates normalized host/address policy for every connection, - fetches into an isolated object database from the authorized remote, and - verifies the checked-out commit and resulting tree. The OCI path accepts + hermetic Git configuration; disables inherited redirects, proxies, + credential helpers, hooks, filters, LFS smudge, submodule recursion, + alternates, and non-HTTPS protocols; injects only the KTD16-selected + one-shot credential channel; revalidates normalized host/address policy for + every connection; fetches into an isolated object database from the + authorized remote; and verifies the checked-out commit and resulting tree. + The OCI path accepts manifest digests, not tags; rejects foreign/external URLs by default; revalidates scheme, host, resolved address, port, redirect, and credential origin for every registry/auth/manifest/blob request; and verifies every @@ -391,9 +484,61 @@ The two initial runtimes expose different programmatic contracts. Codex provides trusted computing base, not hostile caller code. Its runner or sandbox control plane is never mounted into the workspace or exposed to setup, providers, or model tools. A deployment that does not trust the registered - image with source credentials must use a broker or stronger acquisition - service and advertise that capability explicitly. + image with source credentials is outside the initial trust model and must not + enable that registered materializer. Support requires the separately + versioned broker or central snapshot-delivery protocol deferred by R13. - KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, Codex SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. +- KTD16. **Resolve GitHub credentials through an authoritative ordered provider + registry and lease controller.** The caller supplies only a canonical + credential-free repository URL. The acquisition boundary maps `github.com` + to the built-in GitHub backend and requires explicit host/API mappings for + GitHub Enterprise Server. The profile supplies provider eligibility and + order, not secrets. A configured App provider is applicable only when trusted + operator configuration maps the authorized repository to an installation ID; + auth-app does not discover installations. A `github-cli` provider may follow + only in a trusted-local profile, only when no App mapping applies, and only + for its configured non-secret account. That account participates in + entitlement and effective-profile digests. Invoke + `gh auth token --hostname --user ` with `GH_TOKEN`, + `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` removed, + and fail when the configured account cannot be resolved. Selection is sticky: + App configuration, authentication, minting, authorization, rate-limit, or + service failure never falls through to the broader user identity. + + When AllAgents owns minting, its trusted central minter depends on focused + `@octokit/auth-app` rather than implementing App JWT, clock-skew, expiry, and + renewal; Git remains the transport and the full Octokit client is not added. + Every cache-miss acquisition uses `refresh: true` and accepts only a fresh + token whose remaining lifetime is strictly greater than the acquisition + deadline plus clock-skew margin. The token is scoped only to the repository, + read-only contents permission, and GitHub expiry. Lease expiry cannot exceed + token expiry, and readiness rejects an acquisition ceiling that can exceed a + fresh token's safe lifetime. + + The gateway/control-plane credential-lease controller, not the worker, is + authoritative. An authenticated worker request supplies only active attempt + and fence. The controller rechecks durable command revision, lease epoch, + tombstone, and fence and derives effective-profile digest, selected provider, + host/API-mapping digest, installation ID, canonical repository, operation, + worker route/identity, and expiry from durable dispatch and policy state. It + issues a single-use non-durable grant/response; a separate minter atomically + consumes the grant and must agree with the selected configuration digest. + Replay, substitution, stale command state, and digest disagreement fail + closed. The authenticated lease/channel binds the derived state to worker + identity, attempt, lease epoch, command revision, fence, operation, and + expiry. A remote worker never receives the App private key. Remote App + profiles fail readiness without that complete central path. A trusted + co-located deployment may keep the controller and minter in its control + plane. Versioned central snapshot delivery is deferred. + + Authenticated App lifecycle webhooks and bounded reconciliation advance an + installation-entitlement generation on uninstall, suspension, or + repository-selection change; unknown or stale state fails cache + authorization. Minting remains miss-only. Provenance distinguishes + `cache_hit`, original acquisition-provider metadata, and current policy + selection/entitlement. Public source-auth details contain only the specified + safe code, reason, and retryability; provider, installation, and account + identifiers are operator-only. Governs R6, R12-R13, R16, R21-R22. ### High-Level Technical Design @@ -405,8 +550,13 @@ flowchart TB Gateway --> Auth[Auth, retained replay, new admission] Gateway --> Store[Generation-based Task and Artifact store] Gateway -->|mTLS/authenticated overlay or same-host Unix socket| Worker[Single-execution worker] + Gateway --> LeaseController[Authoritative credential lease controller] + LeaseController --> AppMinter[Trusted GitHub App token minter] Worker --> Materialization[Workspace materializer registry] Materialization --> Git[Hardened multi-repository Git] + Git --> SourceCredentials[Source credential client] + SourceCredentials -->|Active attempt and fence| LeaseController + SourceCredentials --> GitHubCLI[Account-pinned trusted-local gh helper] Materialization --> OCI[Digest-pinned OCI snapshot] Materialization --> Custom[Registered materializer image] Worker --> Registry[Closed backend registry] @@ -427,6 +577,8 @@ sequenceDiagram participant S as Durable aggregate store participant W as Worker participant B as Backend adapter + participant L as Credential lease controller + participant M as App token minter C->>G: SendMessage + header/Message extension + metadata[uri] G->>G: Authenticate, check extension, canonicalize within bounds @@ -443,6 +595,14 @@ sequenceDiagram G->>W: Dispatch(attempt, fence, command revision) W->>W: Verify command record before workspace creation W-->>G: Accepted(attempt, fence) + opt private GitHub cache miss + W->>L: Request(active attempt, fence) + L->>S: Recheck command revision, lease epoch, tombstone, fence + L->>L: Derive profile/provider/mapping/install/repository/operation/route + L->>M: Single-use non-durable grant + configuration digest + M-->>L: Fresh repository/read-only token + GitHub expiry + L-->>W: Authenticated lease response bound to current command + end W->>W: Materialize into staging and validate workspace manifest W->>W: Destroy acquisition boundary, publish atomically, setup, baseline W->>B: Invoke with isolated roots and credential boundary @@ -520,6 +680,13 @@ packages/execution-service/ errors.ts profiles.ts telemetry.ts + source-credentials/ + contract.ts + registry.ts + github.ts + github-app-minter.ts + github-app-client.ts + github-cli.ts gateway/ index.ts config.ts @@ -559,6 +726,7 @@ packages/execution-service/ unit/execution/ unit/gateway/ unit/worker/ + unit/source-credentials/ e2e/execution-gateway.test.ts containers/ gateway.Dockerfile @@ -587,12 +755,39 @@ docs/src/content/docs/ - Each profile defines backend, worker route, allowed workspace source modes, allowed materializer IDs, exact Git repository or namespace rules, allowed Git origins/addresses, OCI namespace/registry/signature policy, structured - materializer-input resource selectors, authorization-scope derivation and - source-authorization revocation epoch, provider/model settings, - phase-specific environment allowlists, deterministic permissions, - setup/check commands, artifact globs, effective deadline ceiling, trust - class, resource limits, cleanup policy, evidence budgets, and required - acquisition/provider/tool isolation capabilities. + materializer-input resource selectors, authorization-scope derivation, + source-authorization revocation epoch, and applicable GitHub App + entitlement-generation authority, provider/model settings, phase-specific + environment allowlists, deterministic permissions, setup/check commands, + artifact globs, acquisition and effective deadline ceilings, clock-skew + margin, trust class, resource limits, cleanup policy, evidence budgets, and + required acquisition/provider/tool isolation capabilities. +- Source-credential configuration defines normalized-host backend mappings and + ordered provider entries. `github.com` has a built-in GitHub mapping; every + GitHub Enterprise Server hostname and API base URL is explicit. A + control-plane `github-app` entry references an App ID, private-key secret + handle, installation ID or deterministic repository-to-installation mapping, + requested read-only contents permission, entitlement-generation store, + authenticated lifecycle-webhook configuration, bounded reconciliation + interval and stale-state limit, fresh-token lifetime policy, and a + versioned non-secret provider/host/API-mapping configuration digest. The + worker receives no App private-key handle. A worker-local `github-cli` entry + contains no token, names one non-secret account/login included in entitlement + and effective-profile digests, and is valid only for an explicitly + trusted-local profile. It invokes the configured `gh` binary with + `auth token --hostname --user ` after removing `GH_TOKEN`, + `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN`. +- Every remote route using a GitHub App declares the authenticated central + token minter, authoritative credential-lease controller, and single-use + non-durable grant/response protocol. Startup rejects remote App profiles + without that complete path, `github-cli` on remote or multi-tenant routes, + missing central App secret handles, unsupported hosts, ambiguous + equal-priority providers, policies that allow runtime failure to trigger + identity fallback, or acquisition ceilings that can exceed a fresh token's + safe lifetime. Configuration and effective-profile digests include provider + IDs, order, host/API mappings, route capability, pinned CLI account, + non-secret entitlement policy, and mapping/configuration digest, but exclude + private keys, resolved tokens, lease payloads, and per-attempt state. - Worker configuration fixes a private listener, worker identity, one-execution concurrency, one same-filesystem publication root containing private staging and final workspace directories, a closed materializer @@ -618,6 +813,19 @@ docs/src/content/docs/ credential mounts, staging mounts, and roots before readiness. The supported worker baseline is a dedicated process namespace under a minimal init/reaper; bare-host deployment requires an equivalent systemd/cgroup mechanism. +- Remote App readiness additionally proves that the configured central minter + and gateway/control-plane lease controller authenticate the selected worker + route, agree on the selected provider/host/API-mapping configuration digest, + and support fresh `refresh: true` minting plus a single-use non-durable + grant/response. The controller must derive profile/provider/mapping/ + installation/repository/operation/route/identity/expiry from durable state, + recheck current command revision, lease epoch, tombstone, and fence, reject + replay or substitution, and bind delivery to worker identity, attempt, lease + epoch, command revision, fence, operation, and expiry. Readiness also proves + the acquisition ceiling plus clock-skew margin fits within a fresh token's + safe lifetime, lease expiry cannot exceed token expiry, token payloads never + persist in Task or command records, and delivery reaches only the acquisition + phase. The worker image and configuration contain no App private-key handle. - Telemetry configuration defines the OTLP destination, filtering/redaction bounds, opaque owner-correlation derivation, and telemetry-specific operator access and retention. The service version fixes the metadata allowlist; configuration cannot extend it to prompt/output/tool/source/file-body attributes, secret-bearing fields, raw caller identity, or unfiltered backend-native/OpenInference attribute passthrough. - Configuration contains environment-variable names but never secret values. Startup resolves the complete graph and becomes ready only when trusted ingress, worker transports/identities, store, runtimes, quotas, free-space reserves, supervisor/orphan recovery, and declared profile capabilities pass. Any unprotected remote endpoint or unproved credential/supervisor boundary fails readiness. @@ -632,6 +840,14 @@ docs/src/content/docs/ | Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | | Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | | Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | +| No eligible GitHub provider after acceptance | `TASK_STATE_FAILED` | `materialization/source_auth_unavailable`; safe reason `no_eligible_provider`; `retriable: false`; no provider identity in public detail | +| Selected installation does not cover the repository | `TASK_STATE_FAILED` | `materialization/source_auth_denied`; safe reason `installation_repository_denied`; `retriable: false`; installation identity is operator-only; no `gh` fallback | +| Selected App configuration is invalid | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_configuration_invalid`; `retriable: false`; operator-only provider detail; no `gh` fallback | +| Selected App authentication fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_authentication_failed`; `retriable: false`; operator-only provider detail; no `gh` fallback | +| Selected App token mint or fresh-lifetime validation fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_mint_failed`; `retriable: false`; operator-only provider detail; no `gh` fallback | +| Selected App provider is rate limited | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `provider_rate_limited`; `retriable: true`; no provider identity in public detail; no `gh` fallback | +| Selected App provider service is unavailable | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `provider_unavailable`; `retriable: true`; no provider identity in public detail; no `gh` fallback | +| Eligible trusted-local GitHub CLI provider fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `trusted_local_cli_failed`; `retriable: false`; configured account identity is operator-only | | Failure before result-candidate production | `TASK_STATE_FAILED` | Typed primary dispatch/materialization/setup/provider/crash/infrastructure phase, including manifest or materializer failure; safe message, retriable fact, requested structured result `not_produced`, separate termination/cleanup/completeness, and bounded workspace provenance | | Check, mandatory-evidence, cleanup, crash, or infrastructure failure after result validation | `TASK_STATE_FAILED` | Preserve selected `valid` or `invalid`; preserve exactly one fixed structured-result Artifact for `valid`; later phase remains primary failure | | Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` with `not_produced`, or `structured_result/invalid` with `invalid`; no structured-result Artifact | @@ -649,8 +865,9 @@ docs/src/content/docs/ 3. Build the supervised single-execution worker lifecycle, monotonic command record, failed-quiescence boundary recycling, pre-readiness orphan reaper, OS credential boundaries, the direct Git/OCI/registered-materializer - registry, and hardened workspace/evidence handling against fake - materializers and a fake backend. + registry, the central GitHub App minter/client and trusted-local GitHub CLI + source-credential registry using `@octokit/auth-app`, and hardened + workspace/evidence handling against fake materializers and a fake backend. 4. Add the direct Codex SDK adapter and prove structured output, cancellation, OS-enforced provider/tool credential separation, and native evidence. 5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing OS-confined tools plus the terminating result tool. 6. Package the services and run cross-backend, transport, security, process, and A2A conformance before enabling a consumer. @@ -658,6 +875,10 @@ docs/src/content/docs/ ### System-Wide Impact - **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. +- **Dependency surface:** `@octokit/auth-app` is private to the Node 22 + execution-service package and used only by the trusted control-plane GitHub + App minter. The root Node 18 CLI, remote worker, and acquisition subprocess + do not import the full Octokit client or hold App private-key material. - **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. - **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. - **Security:** New review-critical surfaces are trusted public/private @@ -691,6 +912,20 @@ docs/src/content/docs/ enforce the R13 OS provider/tool boundary or broker; environment filtering remains defense in depth. Pi repository extensions and unrestricted built-in tools never load. +- **Credential fallback, stale entitlement, or issuer-key escalation:** Treat + provider order as eligibility, not retry. Prefer only the operator-mapped + GitHub App installation, permit an account-pinned GitHub CLI provider only in + trusted-local profiles when no mapping applies, and fail closed after every + selected-App failure. Keep App private keys in the central minter. Make the + lease controller derive provider/repository/route state from the current + durable command, use one single-use grant, and reject replay, substitution, + stale fences, or controller/minter configuration-digest disagreement. Use a + fresh `refresh: true` token per cache-miss acquisition, bound its lifetime to + the acquisition deadline plus skew, and reject unsafe ceilings at readiness. + Authenticated lifecycle webhooks plus reconciliation advance entitlement + generations so stale/unknown App state cannot authorize cache reuse. Keep + tokens out of arguments, Git configuration, durable records, logs, evidence, + and later phases; public failures remain coarse and identities operator-only. - **Telemetry disclosure:** Apply KTD10's pre-export guard before every structured log/span processor and reject content or secret-bearing attributes rather than relying on exporter policy. Canary-secret and cross-owner-fragment tests cover agent, model, tool, stale-event, and error paths; telemetry operators receive only bounded metadata and opaque owner correlation under separate access and retention. - **Resource exhaustion:** Reserve per-owner/global gateway quota only for new claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. - **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. @@ -721,7 +956,7 @@ docs/src/content/docs/ structured-result Artifacts, private worker protocol including monotonic command state, original idempotency bindings, typed failures, and conformance fixtures before either service endpoint. -- **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12. +- **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12, KTD16. - **Dependencies:** None. - **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/result-schema-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. - **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas and freeze `https://allagents.dev/a2a/extensions/coding-execution/v1`: required Agent Card advertisement, `A2A-Extensions` negotiation, `Message.extensions`, request data only at `Message.metadata[uri]`, and terminal integrity data only in the single Part of the fixed-name `allagents.execution-integrity` Artifact whose `extensions` contains the URI. Explicitly forbid `Task.extensions`. Define the portable result-schema subset, canonical caller/schema/profile digests, four result states, separate fixed `allagents.structured-result` Artifact, and shared validator. Define original claim bindings independently from mutable current policy. Add worker identity, attempt/fence/lease identity, monotonic command revision, unseen-attempt cancel tombstone, conditional effect revision, event sequence, terminal acknowledgement, and integrity rules. Generate checked-in schemas and fixtures from one source. @@ -735,6 +970,28 @@ docs/src/content/docs/ versioned, domain-separated canonical preimages for materializer definitions, inputs, manifests, profiles, and caller requests; no public field can carry acquisition code, image references, commands, credentials, or policy. + Define normalized source-host/API mappings and ordered source-credential + provider policy as trusted profile/configuration fields. Public schemas cannot + select a provider. Effective-profile canonicalization includes provider IDs, + order, host/API mappings, mapping/configuration digest, pinned CLI account, + non-secret entitlement policy, and current App entitlement generation while + excluding App private keys, resolved tokens, local account tokens, and + per-attempt provider state. Define cache metadata that preserves original + acquisition-provider metadata and cache-hit provenance that separately names + `cache_hit` and current policy selection. + Define the private source-credential protocol separately from the durable + worker command record. A worker request contains only active attempt and + fence under its authenticated route. The controller response carries its + authoritative derivation of effective-profile digest, selected provider, + host/API-mapping digest, installation ID, repository, operation, worker + route/identity, lease epoch, command revision, and expiry, plus a single-use + non-durable grant/response state. The lease/channel binds all derived fields + and cannot outlive the token; the token schema expresses only repository, + read-only contents permission, and GitHub expiry. Define deterministic + source-auth code/reason/retryability enums and operator-only identity detail. + Token payloads are secret transport data: they are never part of public + schemas, canonical digests, Task storage, command records, events, logs, + errors, evidence, or provenance. - **Execution note:** Start with fixture-driven schema, framing, and digest tests. Observe failures for unknown versions, credential-bearing sources, mutable revisions or image tags, duplicate/unsafe destinations, unknown or @@ -761,9 +1018,34 @@ docs/src/content/docs/ definition digest while secret-value rotation does not. - Rotating a resolved secret value, changing attempt/lease/trace identity, or changing a per-run path leaves the profile digest unchanged; changing a - policy field, environment-variable name, authorization scope, or revocation - epoch changes it, and the digest serializer cannot accept secret-bearing - runtime state. + provider policy field, pinned CLI account, host/API mapping or mapping + digest, environment-variable name, authorization scope, revocation epoch, + or App entitlement generation changes it, and the digest serializer cannot + accept secret-bearing runtime state. + - Provider policy fixtures accept GitHub App followed by account-pinned + trusted-local GitHub CLI, reject CLI on remote or multi-tenant routes, + require explicit GitHub Enterprise Server host/API mappings, and reject + every public credential-provider field. Fixtures encode + `gh auth token --hostname --user ` and removal of all four + ambient GitHub token variables. Provider order, ID, host/API mapping, + pinned account, entitlement, or non-secret configuration digest changes + the effective-profile digest; private-key or token rotation does not. + - Private credential-request fixtures accept only active attempt and fence. + Controller-response fixtures carry worker identity/route, attempt, lease + epoch, command revision, fence, effective-profile/provider/mapping/ + installation/repository/operation bindings, and expiry; reject replay, + substitution, stale state, duplicate grant consumption, expiry after token + expiry, or controller/minter configuration-digest disagreement; and cannot + round-trip through durable command/Task serializers. Token fixtures contain + only repository/read-only/expiry scope. Remote App profile fixtures require + the complete central minter/controller capability, safe lifetime policy, + entitlement-generation authority, and no worker-side private-key handle. + - Cache metadata fixtures distinguish original acquisition-provider metadata, + `cache_hit`, and current policy selection/entitlement. Unknown or stale App + generations reject cache authorization. + - Exact source-auth fixtures cover every safe code/reason/retryability tuple + from the error table and prove provider, installation, and account + identifiers are absent from public detail but available to operators. - Unsupported keywords, remote references, non-object roots, object schemas that omit `additionalProperties: false`, undeclared optional properties, format-dependent validation, or schemas over byte/depth/property/enum limits are rejected before Task creation; every accepted schema validates identically in admission, worker, Codex forwarding, and Pi tool generation. - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. - Worker fixtures reject missing/mismatched worker identities, attempt IDs, lease epochs, profile digests, command revisions, conditional-effect revisions, event sequences, bounds, and terminal acknowledgements. Cancel for an unseen attempt persists a tombstone; tombstoned or lower-revision dispatch is invalid before workspace creation. @@ -824,28 +1106,69 @@ docs/src/content/docs/ - **Goal:** Implement the supervised single-execution worker with authenticated transport, a minimal monotonic command record, a closed workspace materializer registry for hardened multi-repository Git, digest-pinned OCI, - and operator-registered images, standard manifest validation, OS-enforced - credential separation, leases, isolated roots, resource controls, - race-resistant evidence, failed-quiescence recycling, and cleanup independent - of any provider. -- **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14. + and operator-registered images, trusted source-credential resolution, + standard manifest validation, OS-enforced credential separation, leases, + isolated roots, resource controls, race-resistant evidence, + failed-quiescence recycling, and cleanup independent of any provider. +- **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14, KTD16. - **Dependencies:** U1. -- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/materializers/types.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/worker/materializers/git.ts`, `packages/execution-service/src/worker/materializers/oci.ts`, `packages/execution-service/src/worker/materializers/external.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/materializers.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`, `packages/execution-service/tests/fixtures/execution/fake-materializer.ts`. +- **Files:** `packages/execution-service/src/source-credentials/contract.ts`, `packages/execution-service/src/source-credentials/registry.ts`, `packages/execution-service/src/source-credentials/github.ts`, `packages/execution-service/src/source-credentials/github-app-minter.ts`, `packages/execution-service/src/source-credentials/github-app-client.ts`, `packages/execution-service/src/source-credentials/github-cli.ts`, `packages/execution-service/src/source-credentials/lease-controller.ts`, `packages/execution-service/src/source-credentials/github-app-entitlements.ts`, `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/materializers/types.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/worker/materializers/git.ts`, `packages/execution-service/src/worker/materializers/oci.ts`, `packages/execution-service/src/worker/materializers/external.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/source-credentials/registry.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-minter.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-client.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-cli.test.ts`, `packages/execution-service/tests/unit/source-credentials/lease-controller.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-entitlements.test.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/materializers.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`, `packages/execution-service/tests/fixtures/execution/fake-materializer.ts`. - **Approach:** Authenticate the configured worker identity and fence every private command. Persist one minimal monotonic command record scoped to worker identity/lease before workspace creation: unseen-attempt cancel writes a tombstone, stale/lower-revision dispatch is rejected, and each dispatch/cancel effect conditionally rechecks the stored revision immediately before mutation. Reserve one execution only after that check. Validate the fully - canonicalized source against profile resource policy before cache lookup; - bind cache entries to owner or authorization-scope digest, revocation epoch, - canonical source, definition digest, and expected/actual manifest digests. + canonicalized source against profile resource policy before cache lookup. + Bind cache entries to owner or authorization-scope digest, revocation epoch, + current GitHub App entitlement generation when applicable, canonical source, + definition digest, expected/actual manifest digests, and original + acquisition-provider metadata. Authenticated App lifecycle webhooks plus a + bounded reconciler advance entitlement generation on uninstall, suspension, + and repository-selection changes; unknown or stale state fails cache + authorization. A hit revalidates current authorization and records + `cache_hit`, original acquisition provider, and current policy selection/ + entitlement separately. A hit never mints a token. Validate deployment, worker-computed materializer definition digest, and acquisition plus provider/tool credential-boundary capabilities, then emit sequenced NDJSON. Keep transition selection pure. Run inside a dedicated container process namespace under init/reaper or an equivalent systemd/cgroup boundary. Resolve only the closed KTD13 materializer registry. + Resolve direct-Git credentials through the closed source-credential registry + only after source authorization and a cache miss. Normalize the host, select + the configured backend, and evaluate providers in policy order. For GitHub, + trusted operator configuration resolves the installation ID; auth-app never + discovers it. The authenticated worker asks the gateway/control-plane + credential-lease controller only for its active attempt and fence. The + controller rechecks durable command revision, lease epoch, tombstone, and + fence; derives effective-profile digest, selected provider, host/API-mapping + digest, installation ID, canonical repository, operation, worker route/ + identity, and expiry; and issues a single-use non-durable grant/response. + A separate minter atomically consumes that grant and rejects a mismatched + configuration digest. Replay, field or provider substitution, and stale + state fail before minting. + + The control-plane minter uses focused `@octokit/auth-app` with + `refresh: true` for every acquisition. Accept only a fresh token scoped to the + authorized repository, read-only contents permission, and GitHub expiry, + with remaining lifetime strictly greater than the acquisition deadline plus + clock-skew margin; expire the lease no later than the token. The delivery + lease/channel—not the bearer token—binds worker identity, attempt, lease + epoch, command revision, fence, repository, operation, and expiry. The remote + worker never receives the App private key. + + Invoke `gh auth token --hostname --user ` only through the + account-pinned trusted-local provider when no App installation mapping + applies, after removing `GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, + and `GITHUB_ENTERPRISE_TOKEN`; fail if the configured account cannot be + resolved. Once App selection begins, every configuration, minting, access, + rate-limit, or service failure is terminal and never retries as the user. + Deliver the selected token through an ephemeral helper channel to the + one-shot Git acquisition process, never a URL, argument, repository config, + durable Task or command record, or later-phase environment. Tear down the + helper and release token references before publishing the workspace. + Launch an external materializer through the configured OCI runner or sandbox as a supervisor-owned resource labeled by worker, attempt, lease, and fence, with only schema-validated and resource-authorized inputs, its source @@ -880,9 +1203,37 @@ docs/src/content/docs/ `Submitted -> Working -> Failed` public trace. - Repositories with LFS configuration/pointers, submodules, hooks, filters, alternates, proxy/helper config, or non-HTTPS secondary protocols cause no - secondary connection or helper execution. + secondary connection or execution of repository, user, or system helpers; + only the KTD16-selected one-shot credential channel can run. - Source credentials leave no repository config, process argument, child phase environment, log, error, evidence, or retained workspace trace. + - GitHub credential resolution uses only the trusted operator + repository-to-installation mapping and never auth-app discovery. A remote + worker request contains only active attempt/fence; the fake controller + derives profile/provider/mapping/installation/repository/operation/route, + rechecks command revision, lease epoch, tombstone, and fence, and returns + one authenticated single-use non-durable lease response. Wrong worker, + replay, duplicate consumption, substituted repository/provider/operation, + stale command state, or controller/minter configuration-digest disagreement + fails before token delivery. The token carries only repository, + read-only-contents, and GitHub-expiry scope; the lease carries worker, + attempt, lease epoch, command revision, fence, operation, and delivery + expiry. The worker never receives the App private key. + - Auth-app is called with `refresh: true` for each acquisition. A cached + near-expiry token is bypassed, remaining lifetime must exceed acquisition + deadline plus clock-skew margin, lease expiry is capped by token expiry, and + an acquisition ceiling that can exceed a fresh token's safe lifetime fails + readiness. Every boundary failure remains terminal without invoking `gh`. + - A trusted-local profile invokes its fake CLI only when no installation + mapping applies, pins `--hostname --user `, removes all four + ambient GitHub token variables, and fails on account mismatch. Remote + profiles cannot select it. GitHub Enterprise Server works only through an + explicit host/API mapping. Provider ID, host, installation/account, and + selection reason appear only in operator provenance, never public detail. + - Table-driven failures assert the exact public safe code, reason, and + retryability for no provider, installation/repository denial, App + configuration/authentication/mint failure, rate limit, service outage, and + trusted-local CLI failure. No selected-App case invokes the CLI. - OCI tags, foreign/external URLs, cross-origin credential forwarding, disallowed registry/auth/blob host/address/port, redirects, DNS rebinding, manifest/layer mismatches, unsafe layers, missing workspace manifests, and @@ -899,11 +1250,16 @@ docs/src/content/docs/ canaries in output or retained logs fail publication. This verifies phase teardown, not safety from a malicious operator-registered image that intentionally transforms a credential. Provenance labels its unverified - source assertions as materializer-attested. A cache hit occurs only after - current authorization and revalidates content plus the same owner or - authorization scope, revocation epoch, canonical source, - materializer-definition, expected-output, and actual output-manifest - digests inside the same trust domain. + source assertions as materializer-attested. + - A cache hit occurs only after current authorization and revalidates content + plus the same owner or authorization scope, revocation epoch, canonical + source, materializer-definition, expected-output, actual output-manifest, + trust domain, and current App entitlement generation. Authenticated webhook + events and bounded reconciliation for uninstall, suspension, and repository + selection advance the generation; unknown, stale, or mismatched state + rejects reuse. Hit provenance records `cache_hit`, original acquisition + provider metadata, and current policy selection/entitlement separately, + including a hit after provider-policy change, and performs no mint. - Setup changes establish the baseline; setup and checks receive no provider/control secrets. Credentialed provider runtimes and model tools run across the declared OS UID/process/mount boundary or broker, with disjoint config/data roots and ambient selectors removed. - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. - Block dispatch after effect selection, complete a newer cancel for the unseen attempt, then release dispatch: the command tombstone/revision check rejects it before workspace or provider creation. Duplicate commands remain idempotent and all effects stay fence-bound. @@ -921,13 +1277,19 @@ docs/src/content/docs/ - Cross-filesystem staging/publication configuration fails readiness. Faults around the final rename expose either no final workspace or the complete validated tree, never a copy fallback or partial publication. -- **Verification:** A built supervised worker materializes equivalent - workspaces through disposable exact-SHA repositories, a local digest-pinned - OCI snapshot, and a fake registered materializer; validates one standard - manifest; mutates each through the fake adapter; and proves authenticated - revisioned dispatch, unseen-cancel tombstones, acquisition hardening and - credential teardown, budgets, result preservation, quiescence or - poisoned-boundary exit, evidence integrity, worker-crash containment, +- **Verification:** A built supervised worker, authoritative fake lease + controller, and fake central minter materialize equivalent workspaces through + disposable exact-SHA repositories, a local digest-pinned OCI snapshot, and a + fake registered materializer; validate one standard manifest; mutate each + through the fake adapter; and prove authenticated revisioned dispatch, + unseen-cancel tombstones, deterministic App-before-account-pinned-CLI + eligibility, remote App private-key exclusion, controller-derived single-use + lease delivery, repository/read/expiry-only token scope, replay/substitution/ + stale-state/config-digest rejection, fresh-token lifetime boundaries, + entitlement-generation cache revocation and truthful hit provenance, exact + source-auth mappings, no fallback after selected-App failure, acquisition + hardening and credential teardown, budgets, result preservation, quiescence + or poisoned-boundary exit, evidence integrity, worker-crash containment, orphan-root handling, and cleanup. ### U5. Codex backend adapter @@ -970,24 +1332,37 @@ docs/src/content/docs/ ### U7. Production registry, service packaging, and observability - **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with trusted transports, peer identity, credential-boundary and supervisor readiness, safe startup/shutdown, tracing, and reproducible containers. -- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD10-KTD15. +- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD10-KTD16. - **Dependencies:** U3-U6. -- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/unit/worker/materializers/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. +- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/gateway/github-app-webhook.ts`, `packages/execution-service/src/gateway/github-app-reconciler.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/gateway/github-app-webhook.test.ts`, `packages/execution-service/tests/unit/gateway/github-app-reconciler.test.ts`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/unit/worker/materializers/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. - **Approach:** Register only Codex and Pi as backend adapters and register the built-in Git/OCI materializers plus configured external materializers through a separate closed registry. Add gateway and supervised worker entrypoints - inside the private Node 22 workspace. Before readiness, validate named public - TLS termination, every remote worker's mTLS/equivalent transport and pinned - identity/capabilities, Unix-socket locality, store, runtimes, matching - gateway/worker materializer definition digests, image digests, schemas, - credential names, egress, limits, and OCI runner/sandbox isolation, monotonic - command storage, acquisition and provider/tool credential-boundary - capabilities, - supervisor boundary, orphan roots, trust, quotas, and resource controls. - Propagate `traceparent`, then apply KTD10's small shared metadata allowlist and - bounded filtering/redaction before any structured log/span processor or OTLP - exporter; neither OpenInference nor backend-native attributes bypass it. - Build a minimal gateway image with no provider or materializer runtime and a + inside the private Node 22 workspace. Register source credentials separately: + an authoritative gateway/control-plane lease controller, a trusted GitHub App + minter using focused `@octokit/auth-app`, its authenticated single-use + non-durable worker client, and an account-pinned GitHub CLI provider only on + trusted-local acquisition hosts. Wire authenticated GitHub App lifecycle + webhooks and bounded reconciliation to the durable entitlement-generation + store. Reject duplicate or ambiguous provider IDs, implicit enterprise host + detection, unpinned CLI accounts, ambient GitHub token variables, remote CLI + fallback, worker-side App private-key handles, fallback-on-error policy, and + provider/mapping configuration-digest disagreement. + Before readiness, validate named public TLS termination, every remote + worker's mTLS/equivalent transport and pinned identity/capabilities, the + complete central lease-controller/minter path for every remote App profile, + single-use grant consumption, fresh-token lifetime versus acquisition ceiling + and clock skew, current entitlement-generation authority, Unix-socket + locality, store, runtimes, matching gateway/worker materializer definition + digests, image digests, schemas, credential names, egress, limits, OCI + runner/sandbox isolation, monotonic command storage, acquisition and + provider/tool credential-boundary capabilities, supervisor boundary, orphan + roots, trust, quotas, and resource controls. Propagate `traceparent`, then + apply KTD10's small shared metadata allowlist and bounded filtering/redaction + before any structured log/span processor or OTLP exporter; neither + OpenInference nor backend-native attributes bypass it. Build a minimal + gateway/control-plane image with the lease controller and App minter but no + provider runtime, writable repository, or baked-in private key, and a one-execution worker image whose init kills the complete boundary when the worker server exits, including poisoned failed-quiescence exit. - **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. @@ -998,6 +1373,26 @@ docs/src/content/docs/ external IDs, resolves every external image to the configured digest, rejects duplicates/tags/unknown IDs, and cannot be influenced by request image, command, credential, or policy fields. + - Source-credential registry maps `github.com` and explicit enterprise + host/API pairs, selects only an operator-mapped App installation, and + permits GitHub CLI only for an account-pinned trusted-local no-mapping case. + The CLI invocation includes `--user` and no ambient GitHub token variables. + No request field can alter provider selection. Public output contains only + safe code/reason/retryability; operator provenance contains the non-secret + provider and installation/account identity. + - Remote App profiles fail readiness without the central minter and + authoritative lease controller, entitlement-generation webhook/ + reconciliation authority, safe fresh-token lifetime policy, single-use + grant support, matching provider/mapping configuration digest, or with an + App private-key handle in worker configuration. Runtime requests by active + attempt/fence derive every provider/repository/route field from current + durable state and reject replay, substitution, stale state, and duplicate + consumption. The same-host trusted case and a separately deployed minter + pass the same contract; neither uses snapshot delivery. + - Readiness rejects an acquisition ceiling that can exceed a fresh token's + safe lifetime. Near-expiry auth-app cache output is bypassed with + `refresh: true`, lease expiry is capped by token expiry, and failure never + falls through to the CLI. - Gateway and supervised worker start from built outputs, become ready only after trusted transport/identity, credential and supervisor boundaries, dependencies, and orphan recovery pass, and stop gracefully on SIGTERM. - Gateway readiness fails for malformed auth, missing/mismatched named TLS termination, plaintext production public ingress, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. - Worker-route readiness fails for plaintext remote URL, wrong/untrusted certificate, worker identity/capability mismatch, or replayed capability; mTLS/equivalent authenticated encryption and same-host Unix sockets pass. @@ -1010,10 +1405,26 @@ docs/src/content/docs/ egress and resource enforcement cannot be provided. - Killing or poisoning the worker server while an adapter child and invocation root exist makes the supervisor destroy the boundary; replacement readiness waits for root deletion/quarantine and never reuses it. - Trace context crosses the authenticated private call and correlates result identities using only opaque owner correlation. Exporter probes for agent, model, tool, stale-event, and error spans contain allowlisted bounded metadata but no canary secret, prompt/output, tool argument/result, file body/source fragment, raw caller identity, or cross-owner fragment; exporter failure cannot change Task status. - - Gateway image contains no Codex, Pi, Git workspace, provider credential material, or worker trust private keys. - - Worker image pins both runtimes, enforces provider/tool UID/process/mount separation or the credential broker, confines one workspace/config root, disables repository Pi extensions and unrestricted built-ins, enforces deployment limits, and completes fake-provider security probes. + - Gateway/control-plane images contain no Codex, Pi, Git workspace, coding + provider credentials, or baked-in GitHub App private key; the App key enters + only through its configured secret handle. Worker images and configuration + contain neither App issuer material nor user credential stores, pin both + coding runtimes, enforce provider/tool UID/process/mount separation or the + credential broker, confine one workspace/config root, disable repository Pi + extensions and unrestricted built-ins, enforce deployment limits, and + complete fake-provider security probes. - Installing the root npm package on Node 18 does not load service dependencies; the private service workspace and containers enforce Node 22.19+. -- **Verification:** The registry dispatches both adapters through the same worker contract; built services and images pass lifecycle/security smoke tests; exporter-capture tests prove pre-processor metadata allowlisting, bounded redaction, opaque owner correlation, and canary/cross-owner exclusion across agent, model, tool, stale-event, and error spans; CI and publication bind immutable image tags to the release commit. +- **Verification:** The registries dispatch both adapters and + source-credential providers through their respective contracts; built + services and images prove controller-authorized fresh App minting without + worker issuer material, single-use lease replay/staleness/config-digest + rejection, entitlement-generation webhook/reconciliation behavior, + account-pinned sanitized CLI eligibility, deterministic public failure + mapping with operator-only identities, and lifecycle/security behavior; + exporter-capture tests prove pre-processor metadata allowlisting, bounded + redaction, opaque owner correlation, and canary/cross-owner exclusion across + agent, model, tool, stale-event, and error spans; CI and publication bind + immutable image tags to the release commit. ### U8. Cross-backend conformance, documentation, and release evidence @@ -1030,6 +1441,16 @@ docs/src/content/docs/ publication, supervisor-owned runner cleanup, acquisition credential teardown, authorization-scoped cache reuse/revocation, source-mode capabilities, and the prohibition on caller-supplied acquisition code. + Document normalized host/API mapping, operator-owned + repository-to-installation mapping, GitHub App precedence and + repository/read/expiry token scope, account-pinned sanitized trusted-local + CLI eligibility, fail-closed selected-App behavior, focused + `@octokit/auth-app` ownership and `refresh: true`, central App private-key + custody, controller-derived single-use remote lease bindings, token/lease + lifetime rules, entitlement-generation webhooks/reconciliation and + cache-hit provenance, deterministic public source-auth mapping with + operator-only identities, remote readiness requirements, the one initial + token-minter path, and deferred versioned snapshot delivery. - **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. - **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, Starlight guide/reference organization under `docs/src/content/docs/`, and Buzz's required-critical-action coverage rule without importing its TLA+ model or production implementation. - **Test scenarios:** @@ -1043,7 +1464,24 @@ docs/src/content/docs/ disclosure, and setup failure after worker acceptance produce the selected `Submitted -> Working -> Failed` trace; provider invocation never begins, and durable snapshots, streams, and conformance records agree. A policy - revocation before lookup cannot consume a previously populated cache entry. + revocation, webhook-advanced entitlement generation, reconciliation result, + or unknown/stale App state before lookup cannot consume a previously + populated cache entry. A valid hit after provider-policy change records + `cache_hit`, original acquisition provider, and current selection separately + without minting. + - GitHub source cases prove operator-mapped App selection, account-pinned + sanitized trusted-local CLI selection only when no mapping applies, no CLI + invocation after any selected-App failure, explicit enterprise host/API + mapping, repository/read-only/expiry-only token narrowing, and an + authenticated controller-derived single-use lease carrying worker identity, + attempt, lease epoch, command revision, fence, repository, operation, and + expiry without worker issuer material. They reject replay, substitution, + stale state, duplicate grant consumption, and controller/minter + configuration-digest disagreement; bypass near-expiry auth-app cache output + with `refresh: true`; cap lease expiry by token expiry; fail readiness for an + unsafe acquisition ceiling; assert every safe source-auth + code/reason/retryability tuple and operator-only identity detail; and + publish a credential-free workspace. - Pause dispatch after selection, complete unseen-attempt cancel, then release dispatch; the stale command creates no workspace/process. The small independent checker enforces each fixture's attempt/fence correlation, happens-before edges, maximum counts, and forbidden post-terminal effects. Deliberately bad traces that still contain every required action name fail for wrong order, wrong fence, duplicate-over-maximum effects, and an extra stale dispatch after terminalization. - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; one worker serializes admitted work. - Gateway restart, reconnect, ambiguous dispatch, duplicate/out-of-order @@ -1074,7 +1512,11 @@ docs/src/content/docs/ names. Docs state the three source modes and exact discriminator, standard workspace manifest and provenance labels, materializer registry/profile boundary, worker-derived definition digest, resource authorization and - cache-revocation boundary, prohibition on caller-supplied acquisition code, + entitlement-generation cache-revocation boundary, cache-hit/original/current + provider provenance, prohibition on caller-supplied acquisition code, + one selected remote token-minter/lease path with deferred snapshot delivery, + account-pinned sanitized CLI invocation, token versus lease scope, fresh + token and readiness lifetime rules, deterministic source-auth mapping, same-filesystem publication, supervisor-owned runner cleanup, acquisition credential teardown, two transport boundaries, one gateway replica, one execution per worker, reviewed trust domain, narrow credential isolation @@ -1090,16 +1532,16 @@ docs/src/content/docs/ | Gate | Applies to | Required evidence | |---|---|---| -| Contract generation | U1 | Exact extension URI/carriers, closed workspace source discriminator and manifest, algorithm-qualified materializer/profile/input/output digest preimages and vectors, verification-method vocabulary, authorization-scope/revocation fields, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | -| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, source authorization/cache revocation, and cleanup. | -| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, direct Git/OCI/registered materialization, worker-derived registry digests, acquisition credential teardown, supervisor-owned materializer runners, same-filesystem atomic publication, workspace-manifest provenance labels, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | +| Contract generation | U1 | Exact extension URI/carriers, closed workspace source discriminator and manifest, algorithm-qualified materializer/profile/input/output digest preimages and vectors, verification-method vocabulary, authorization-scope/revocation/App-entitlement fields, cache-hit/original/current-provider provenance, worker request limited to active attempt/fence, controller-derived single-use non-durable lease fields and token-versus-lease scope, exact source-auth code/reason/retryability tuples, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | +| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, account-pinned sanitized CLI execution, fresh-token lifetime boundaries, entitlement-generation authorization/cache revocation, provenance states, exact source-auth failures, and cleanup. | +| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, direct Git/OCI/registered materialization, deterministic operator-mapped-App-before-account-pinned-CLI eligibility, central fresh App minting without worker issuer material, controller-derived single-use non-durable lease delivery, repository/read/expiry-only token scope, replay/substitution/stale/config-digest rejection, entitlement-generation cache revocation and hit provenance, exact failure mapping, fail-closed selected-App behavior, worker-derived registry digests, acquisition credential teardown, supervisor-owned materializer runners, same-filesystem atomic publication, workspace-manifest provenance labels, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | | Backend conformance | U5-U8 | One shared suite passes against Codex and Pi, including the versioned schema subset, four result states, valid/invalid preservation across later failure, integrity Artifact carrier, and structured-result Artifact rule. | -| Credentialed provider smoke | U4-U6, U8 | An available operator-trusted registered materializer and each available provider mutate a disposable immutable workspace while adversarial later-phase probes cannot directly access acquisition/provider credential environments, mounts, processes, roots, or runner control planes and literal canaries remain absent; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | +| Credentialed provider smoke | U4-U6, U8 | An available centrally held GitHub App, account-pinned trusted-local `gh` login, operator-trusted registered materializer, and each available coding provider mutate disposable immutable workspaces while provider selection follows policy. App acquisition proves `refresh: true`, minimum remaining lifetime, lease-at-or-before-token expiry, repository/read-only/expiry token scope, and no worker issuer material; local CLI smoke proves `--user` and sanitized ambient token variables. Adversarial later-phase probes cannot directly access acquisition/provider credential environments, mounts, processes, roots, or runner control planes and literal canaries remain absent; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | | A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes required-extension negotiation and legal carriers, immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, retained replay, cancel races, expiry, and owner isolation without `Task.extensions`. | -| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, exact source-resource authorization, per-connection Git/OCI SSRF and credential-origin controls, authorization-scoped cache revocation, digest-pinned registered materializers, schema/manifest validation, truthful provenance labels, acquisition and provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | +| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, exact source-resource authorization, per-connection Git/OCI SSRF and credential-origin controls, operator-mapped GitHub provider eligibility without identity escalation, central issuer-key custody, worker request limited to active attempt/fence, authoritative current-state derivation, single-use lease replay/substitution/staleness/config-digest rejection, repository/read/expiry-only token scope, fresh-token lifetime safety, fail-closed selected-App errors, account-pinned sanitized CLI use, exact public failure mappings with operator-only identities, webhook/reconciliation-driven entitlement cache revocation, truthful cache-hit provenance, digest-pinned registered materializers, schema/manifest validation, acquisition and provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | | Lifecycle trace conformance | U8 | The small test-side checker, independently of production selectors, validates attempt/fence correlation, required happens-before edges, maximum occurrence counts, and forbidden post-terminal effects against durable records plus observed worker/process outcomes; all-name-present bad traces fail for wrong order/fence/multiplicity and stale post-terminal dispatch. | | Telemetry safety | U7-U8 | Exporter capture across agent, model, tool, stale-event, and error spans proves the pre-processor allowlist and bounded redaction exclude prompt/output/tool/source/file content, canary secrets, raw identities, and cross-owner fragments while retaining only bounded operational metadata and opaque owner correlation. | -| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, backend and materializer registry readiness, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both service container builds pass. | +| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build with focused `@octokit/auth-app` and no full Octokit client, gateway/control-plane lease controller and minter plus supervised-worker smoke, entitlement webhook/reconciler, worker issuer-key exclusion, backend/materializer/source-credential registry readiness, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both service container builds pass. | | Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | | Documentation | U8 | `bun run docs:build` passes and examples validate against current schemas. | @@ -1119,16 +1561,27 @@ The authoritative behavioral proof is the built-process E2E path with the offici - Production public ingress uses its named TLS boundary, remote worker routes authenticate and encrypt peers with worker identity/capability binding, and same-host Unix sockets are the only non-network alternative; unprotected remote endpoints fail readiness. - Cancellation/deadlines use monotonic worker command tombstones and one native abort. Stale dispatch cannot create work, and failed quiescence poisons and exits the worker so supervisor destruction and replacement orphan recovery precede new admission. - Workspace source validation and exact resource authorization, - per-connection direct Git/OCI controls, worker-derived registered-materializer - digests, the standard workspace manifest and truthful provenance labels, - authorization-scoped cache revocation, same-filesystem atomic publication, + per-connection direct Git/OCI controls, trusted-policy GitHub provider + resolution with operator-mapped App precedence, account-pinned sanitized + local-only CLI eligibility, no selected-App failure fallback, central App + private-key custody, controller-derived single-use authenticated lease + delivery, repository/read-only/expiry-only token scope, current-command + recheck and replay/substitution/stale/config-digest rejection, fresh-token and + readiness lifetime bounds, deterministic public source-auth mapping with + operator-only identities, authenticated webhook/reconciliation-driven App + entitlement generations, fail-closed unknown/stale cache authorization, and + separate cache-hit/original-acquisition/current-selection provenance are + enforced end to end. The initial remote path is central token minting and + non-durable lease delivery; versioned snapshot delivery remains deferred. + Worker-derived registered-materializer digests, the standard workspace + manifest and truthful provenance labels, same-filesystem atomic publication, supervisor-owned runner cleanup, acquisition credential teardown, OS-enforced provider/tool credential boundary, phase-scoped secrets, disabled repository Pi extensions/unrestricted built-ins, one-execution reviewed-domain policy, resource limits, Artifact race defenses, - completeness, provenance, and authenticated expiry are enforced end to end - without accepting caller acquisition code or claiming hostile-source or - cross-tenant isolation. + completeness, provenance, and authenticated expiry are enforced without + accepting caller acquisition code or claiming hostile-source or cross-tenant + isolation. - Metadata-only telemetry is filtered through the fixed allowlist and bounded redaction before processing/export; canary secrets, content, raw caller identities, and cross-owner fragments never reach exporters, and only opaque owner correlation crosses the separately governed operator boundary. - Required source/setup and race traces satisfy attempt/fence, happens-before, maximum-count, and forbidden-post-terminal constraints in the independent test-side checker; all-name-present malformed traces fail without introducing a parallel lifecycle implementation. - Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. @@ -1139,27 +1592,44 @@ The authoritative behavioral proof is the built-process E2E path with the offici - U1: Standard extension carriers, closed workspace source/manifest contracts, algorithm-qualified materializer/profile/input/output digest preimages, - verification and authorization vocabulary, integrity/structured-result - Artifact schemas, four result states, original claim digests, command - revisions/tombstones, fence rules, typed failures, and fixtures are generated - and stable. + verification and authorization vocabulary, App entitlement generation and + cache provenance states, active-attempt/fence-only credential requests, + controller-derived single-use non-durable lease bindings, token-versus-lease + scope, exact source-auth code/reason/retryability tuples, + integrity/structured-result Artifact schemas, four result states, original + claim digests, command revisions/tombstones, fence rules, typed failures, and + fixtures are generated and stable. - U2: Trusted ingress, auth, opaque owner isolation, retained-claim-first replay, original bindings, atomic new admission, CAS settlement, pagination, startup recovery, quotas, Artifact access, tombstones, and cleanup pass fault injection. - U3: Every advertised A2A operation agrees across stream and lookup while extension negotiation, replay ordering, authenticated worker routes, fencing, monotonic cancellation, and races preserve one Task. - U4: Worker command state, exact source authorization, direct - Git/OCI/registered materialization, workspace-manifest validation and - provenance classification, authorization-scoped cache revocation, + Git/OCI/registered materialization, operator-mapped GitHub + App-before-account-pinned-CLI eligibility with sanitized invocation and + fail-closed selected-App errors, central fresh App minting without worker + issuer material, authoritative single-use lease delivery with + replay/substitution/stale/config-digest rejection, + repository/read-only/expiry-only token scope and safe lifetime boundaries, + deterministic public failure mapping with operator-only identities, + webhook/reconciliation-driven entitlement cache revocation and truthful hit + provenance, workspace-manifest validation and provenance classification, same-filesystem atomic publication, acquisition and provider credential separation, supervisor-owned runner cleanup, poisoned-exit/orphan recovery, and dispatch/materialization/setup/action/check/quiescence/evidence/cleanup pass malicious, crashed, and faulted scenarios. - U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, result preservation, OS credential separation, native evidence, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. - U6: Pi strict RPC/framing, terminating result, exact policy tools, disabled repository extensions/built-ins, OS-isolated credential store/provider runtime, result preservation, settlement, stats, abort, and process cleanup pass verification. -- U7: Closed backend and materializer registries, trusted - transport/identity/readiness, acquisition/provider credential and supervisor - capability gating, poisoned-worker recycling, metadata-only pre-export - telemetry controls, Node-version separation, tracing, shutdown, containers, - and release artifacts work from built outputs. -- U8: Cross-backend E2E, all three workspace source modes, standard manifest +- U7: Closed backend, materializer, and source-credential registries, focused + `@octokit/auth-app` packaging without a full Octokit or root-CLI dependency, + authoritative lease controller, central App private-key custody and fresh + minter readiness, entitlement webhook/reconciler, account-pinned sanitized + local CLI, trusted transport/identity/readiness, acquisition/provider + credential and supervisor capability gating, poisoned-worker recycling, + metadata-only pre-export telemetry controls, Node-version separation, + tracing, shutdown, containers, and release artifacts work from built outputs. +- U8: Cross-backend E2E, all three workspace source modes, central GitHub App + and account-pinned trusted-local CLI credential-selection cases, remote + issuer-key exclusion, controller-derived single-use lease delivery, + fresh-token lifetime, token-versus-lease scope, exact failure mappings, + entitlement-driven cache invalidation and hit provenance, standard manifest provenance, acquisition credential teardown, standard A2A carriers, retained replay, selected materialization/setup transitions, independent race-trace constraints, telemetry canary/cross-owner probes, transport and credential diff --git a/docs/research/source-credential-broker-precedents.md b/docs/research/source-credential-broker-precedents.md new file mode 100644 index 00000000..b1be9054 --- /dev/null +++ b/docs/research/source-credential-broker-precedents.md @@ -0,0 +1,260 @@ +# Source credential broker precedents + +## Decision + +The execution gateway does **not** need a mandatory standalone Git credential +broker for trusted local use. The settled local provider is an explicit, +account-pinned `gh auth token --hostname --user ` helper invoked +only when no App installation mapping applies. Its environment removes +`GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and +`GITHUB_ENTERPRISE_TOKEN`, and its token is exposed only to the one-shot +acquisition process. Git credential helpers and Git Credential Manager (GCM) +establish the process-boundary precedent, but arbitrary configured helpers are +not part of the selected implementation. A local helper is a broker in the +security sense; it is not a separately deployed network service. + +Remote or multi-tenant workers use one initial path: an authoritative +gateway/control-plane lease controller and trusted central token minter deliver +a fresh GitHub token over an authenticated, single-use, non-durable lease. The +GitHub bearer token is scoped only to the repository, read-only contents +permission, and GitHub expiry. Worker identity, attempt, lease epoch, command +revision, fence, operation, and delivery expiry are properties of the lease and +channel, not the token. Workers never inherit a person's credential helper, +credential store, SSH agent, or the App private key. A versioned central +snapshot-delivery protocol is deferred; it is not an alternative initial +readiness path. The minter may live inside the trusted control plane unless +private-key isolation, audit, scaling, or blast-radius requirements justify a +separate service process. + +## Precedents + +### Git credential helpers and Git Credential Manager + +**Trust boundary.** Git credential helpers are external programs. Git invokes a +configured helper through the shell, supplies an operation and credential context, +and stops consulting helpers after it has a username and a non-expired password +([Git `gitcredentials`](https://git-scm.com/docs/gitcredentials#Documentation/gitcredentials.txt-helper)). +The scriptable `git credential fill` interface sends the repository context on +standard input and returns the resolved username and password on standard output +([Git `git-credential`](https://git-scm.com/docs/git-credential#_typical_use_of_git_credential)). +Consequently, the Git/acquisition process receives the resulting bearer secret; +the helper is not a membrane that makes an untrusted caller safe. + +GCM is an implementation of this local contract, not a required remote service. +Its executable is a console application; on every invocation it reads Git's +request from standard input, retrieves or generates a credential, serializes the +credential to standard output, and terminates +([GCM architecture, “Command execution”](https://github.com/git-ecosystem/git-credential-manager/blob/main/docs/architecture.md#command-execution)). +Git calls it implicitly, and later Git commands reuse stored credentials or tokens +while they remain valid +([GCM README, “How to use”](https://github.com/git-ecosystem/git-credential-manager#how-to-use)). +GCM can put credentials in OS-controlled stores such as Windows Credential +Manager or macOS Keychain, use Secret Service or GPG-backed storage, use Git's +ephemeral cache, or disable its store entirely +([GCM credential stores](https://github.com/git-ecosystem/git-credential-manager/blob/main/docs/credstores.md)). + +**Lifetime.** Helper-process lifetime and credential lifetime are separate. GCM +exits after each request, while the selected store controls token persistence. +Git's built-in cache is an optional local daemon reachable over a Unix-domain +socket restricted to the current user; it forgets credentials after 900 seconds +by default or sooner if the daemon dies +([Git `git-credential-cache`](https://git-scm.com/docs/git-credential-cache#_description), +[options](https://git-scm.com/docs/git-credential-cache#_options)). This is a +local process/socket boundary, not a remotely reachable credential service. + +**Relevance.** Git helpers and GCM prove that a local credential provider can +be an on-demand process rather than a network service. AllAgents does not, +however, inherit or invoke an arbitrary configured helper chain. Its closed +provider registry permits only an explicit GitHub CLI provider pinned to a +configured non-secret account in a trusted-local profile, and only when no +configured GitHub App installation mapping applies. The helper invokes +`gh auth token --hostname --user ` without ambient GitHub token +variables. Its output reaches only the one-shot acquisition child; setup and +the coding harness inherit neither helper configuration nor the token. + +### SSH agent forwarding + +**Trust boundary.** `ssh-agent` holds private keys and exposes operations through +a Unix-domain socket. With forwarding, private keys and passphrases do not cross +the network; the SSH connection carries requests to the local agent and returns +the results +([OpenSSH `ssh-agent`](https://man.openbsd.org/ssh-agent#DESCRIPTION)). The +forwarded socket is nevertheless an authentication capability. OpenSSH warns +that anyone able to bypass the remote socket's permissions can use loaded +identities to authenticate even though they cannot extract the key material +([OpenSSH `ForwardAgent`](https://man.openbsd.org/ssh_config#ForwardAgent)). +GitHub gives the same operational warning: a trusted server can use the keys as +the user while the connection is established, so forwarding should be enabled +only for specifically trusted hosts +([GitHub, “Using SSH agent forwarding”](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/using-ssh-agent-forwarding#setting-up-ssh-agent-forwarding)). + +**Lifetime.** The remote forwarding capability lasts for the SSH connection. +The underlying identity may live longer: `ssh-agent` has no default maximum +identity lifetime unless configured, while `ssh-add -t` can impose one and +`ssh-add -c` can require confirmation for each use +([OpenSSH `ssh-agent -t`](https://man.openbsd.org/ssh-agent#t), +[OpenSSH `ssh-add`](https://man.openbsd.org/ssh-add#c)). + +**Relevance.** Agent forwarding is precedent for reusing a local identity +without copying the long-lived private key, but it is not selected for +AllAgents direct Git acquisition, which accepts canonical HTTPS repository URLs +only. A remote process with the forwarded socket could authenticate as the +user, so the socket must never reach a remote worker, setup code, or the +coding-agent runtime. + +### GitHub App installation tokens and Actions checkout + +**Trust boundary.** A GitHub App uses an RS256 JWT, created with the App private +key, to request an installation access token +([GitHub, “Generating a JSON Web Token”](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-json-web-token-jwt-for-a-github-app)). +The mint request can narrow the token to selected repositories and permissions, +and GitHub will not grant repositories or permissions beyond those already +granted to the installation +([GitHub, “Generating an installation access token”](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app#generating-an-installation-access-token)). +This separates high-value issuer material from the disposable credential handed +to a worker. + +GitHub Actions applies that model per job. GitHub creates a unique +`GITHUB_TOKEN` before each job; it is a GitHub App installation token limited to +the workflow repository, with permissions reducible through workflow policy +([GitHub Actions `GITHUB_TOKEN`](https://docs.github.com/en/actions/concepts/security/github_token#about-the-github_token)). +`actions/checkout` uses the token for Git commands, stores persisted credentials +in a separate file under `RUNNER_TEMP`, references that file from Git config, and +removes the references and file during post-job cleanup +([checkout README, v6 credential storage](https://github.com/actions/checkout#checkout-v6), +[checkout credential setup](https://github.com/actions/checkout/blob/main/src/git-auth-helper.ts#L329-L436), +[checkout credential cleanup](https://github.com/actions/checkout/blob/main/src/git-auth-helper.ts#L475-L510)). + +**Lifetime.** A normal GitHub App installation token expires after one hour +([GitHub installation token documentation](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app#generating-an-installation-access-token)). +The Actions token expires when its job finishes or at its effective maximum +lifetime; GitHub documents a six-hour maximum on GitHub-hosted runners and at +most 24 hours of refresh for longer self-hosted jobs +([GitHub Actions `GITHUB_TOKEN`](https://docs.github.com/en/actions/concepts/security/github_token#about-the-github_token)). +Checkout's credential file is a convenience capability inside that job, not a +long-term credential store, and its post-job deletion is defense in depth rather +than the token's revocation mechanism. + +**Relevance.** This is the closest production precedent for AllAgents: keep the +App private key at a trusted central minter, issue one fresh least-privilege +token for a particular repository acquisition, expose it only during that +phase, and remove its local material afterward. GitHub enforces repository, +read-only contents permission, and expiry; AllAgents separately enforces +attempt and operation bindings through its authenticated delivery lease. + +### BuildKit secret and SSH mounts + +**Trust boundary.** BuildKit distinguishes secret delivery from ordinary build +arguments and environment variables, which can persist in an image. A secret +mount makes a client-provided secret temporarily available only to a particular +build instruction; an SSH mount supplies an agent socket or key and is intended +for cases such as fetching private Git repositories +([Docker build secrets](https://docs.docker.com/build/building/secrets/#types-of-build-secrets)). +`RUN --mount=type=secret` makes the value available without baking it into the +image, while `RUN --mount=type=ssh` exposes SSH-agent access through a mounted +socket +([Dockerfile secret mount](https://docs.docker.com/reference/dockerfile/#run---mounttypesecret), +[Dockerfile SSH mount](https://docs.docker.com/reference/dockerfile/#run---mounttypessh)). + +The isolation guarantee is intentionally narrow. BuildKit states that secret +values must not be written to disk or included in cache checksums and that an +untrusted frontend cannot access forwarded SSH private keys; it also states that +a container explicitly run with a secret mount can read that secret +([BuildKit security boundary](https://github.com/moby/buildkit/blob/master/PROJECT.md#security-boundary)). +A mount therefore limits *where and when* a capability appears; it does not make +code within the mounted step trustworthy. + +**Lifetime.** The secret mount is available for the duration of its build +instruction, rather than becoming part of the resulting image +([Docker build secrets](https://docs.docker.com/build/building/secrets/#secret-mounts)). +When an agent socket is supplied, SSH access is available for the mounted +instruction without adding the private key to the image +([Dockerfile SSH mount](https://docs.docker.com/reference/dockerfile/#run---mounttypessh)). + +**Relevance.** AllAgents should copy the phase-scoping pattern, not necessarily +BuildKit itself: inject a token or agent capability only into the trusted source +acquisition operation, then tear down the mount/socket/environment before setup +or agent execution. Like BuildKit, this delivery mechanism does not mint +credentials and does not eliminate the need for a central issuer in production. + +## Recommendation for AllAgents + +### Local mode + +1. Resolve `github.com` through the built-in GitHub backend and require explicit + host/API mappings for GitHub Enterprise Server hostnames. +2. Prefer a configured GitHub App installation that trusted operator policy + maps to the authorized repository. Do not use `@octokit/auth-app` to discover + installations. If no installation mapping applies, a trusted-local profile + may invoke the explicit + `gh auth token --hostname --user ` provider pinned to a + configured non-secret account. Include that account in the entitlement and + effective-profile digests, remove `GH_TOKEN`, `GITHUB_TOKEN`, + `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` from the helper + environment, and fail if the configured account cannot be resolved. Do not + inherit an arbitrary Git helper/GCM chain or forward an SSH agent. +3. Treat provider order as eligibility, not retry. Once the App provider is + selected, configuration, authentication, minting, authorization, rate-limit, + or service failure terminates acquisition without falling through to the + user identity. +4. Give the resolved token only to the dedicated acquisition subprocess through + a temporary helper channel, remove that channel, terminate the child, and + publish only a credential-free verified workspace before setup or the coding + harness starts. +5. Do **not** require or auto-start an AllAgents network credential service for + trusted local execution. The explicit account-pinned provider subprocess is + sufficient. + +### Production remote or multi-tenant workers + +1. Put GitHub App issuer material in a trusted central token-minter component. + Trusted operator configuration, not auth-app discovery, maps the repository + to an installation ID. For every cache-miss acquisition, use focused + [`@octokit/auth-app`](https://github.com/octokit/auth-app.js) with + `refresh: true` to bypass its installation-token cache and mint a fresh token + narrowed to that repository and read-only contents permission. Require + remaining lifetime strictly greater than the acquisition deadline plus + clock-skew margin, expire the delivery lease no later than the token, and + fail readiness when the configured acquisition ceiling can exceed a fresh + token's safe lifetime. +2. Make the gateway/control-plane credential-lease controller authoritative. + The authenticated worker requests only by active attempt and fence. From + durable dispatch and policy state, the controller derives the + effective-profile digest, selected provider, host/API-mapping digest, + installation ID, repository, operation, worker route and identity, lease + epoch, command revision, and expiry. Immediately before issuance it rechecks + active command revision, tombstone, fence, and lease state. +3. Deliver one single-use, non-durable grant/response over the authenticated + acquisition channel. A separate minter must agree with the controller's + configuration digest and consume the grant atomically. Reject replay, + substituted fields or providers, stale command state, and configuration + disagreement. The bearer token itself remains scoped only by GitHub to the + repository, read-only contents permission, and expiry; worker, attempt, + fence, and operation bindings belong to the lease. +4. Advance a GitHub App entitlement generation from authenticated lifecycle + webhooks plus bounded reconciliation whenever an installation is uninstalled, + suspended, or changes repository selection. Unknown or stale installation + state fails cache authorization. Mint only on a cache miss. On a miss, record + the acquiring provider in operator provenance; on a hit, record `cache_hit`, + the cached original acquisition-provider metadata, and current policy + selection/entitlement binding separately. +5. Publish deterministic coarse failures: `source_auth_unavailable` / + `no_eligible_provider` (not retryable); `source_auth_denied` / + `installation_repository_denied` (not retryable); + `source_auth_failed` with `app_configuration_invalid`, + `app_authentication_failed`, or `app_mint_failed` (not retryable), + `provider_rate_limited` or `provider_unavailable` (retryable), or + `trusted_local_cli_failed` (not retryable). Keep provider, installation, and + account identifiers in operator-only provenance. +6. Never forward an operator's general SSH agent or reuse their desktop GCM + store in a remote worker. Those capabilities represent the person, not the + individual execution request. +7. Keep minting logically central even if it initially lives inside the trusted + gateway process. Split the minter into a standalone network service when + remote trust boundaries, private-key isolation, audit, scaling, or + blast-radius controls require it. A versioned central snapshot-delivery + protocol may be designed later, but is not part of the initial architecture. + +The resulting rule is: **local reuse may be subprocess-mediated; production +issuance must be centrally policy-mediated.** A process boundary is required in +both cases, but a standalone credential service is not. From 689fcd4cfba54e9bd9111b09d998c7266c00d889 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sat, 19 Sep 2026 12:39:27 +1000 Subject: [PATCH 08/44] docs(architecture): simplify execution gateway deployment --- ...-agent-execution-through-an-a2a-gateway.md | 957 +++---- ...0837-feat-coding-execution-gateway-plan.md | 2482 +++++++---------- .../agent-host-protocol-decision-inputs.md | 31 +- .../harbor-repository-materialization.md | 125 +- .../source-credential-broker-precedents.md | 180 +- 5 files changed, 1515 insertions(+), 2260 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index f9357c13..73afb298 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -2,606 +2,457 @@ - Status: Accepted; implementation pending - Date: 2026-09-17 +- Updated: 2026-09-19 ## Context -AllAgents already owns cross-client agent configuration, workspace knowledge, -plugins, hooks, MCP configuration, and launchers for Codex and other coding -agents. External systems also need to invoke those agents without importing -AllAgents internals or coupling to an interactive CLI process. - -The first planned consumer is AI Evals. Its -[ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) -removes AI Evals-owned coding workspaces in favor of a Promptfoo provider that -needs one remote coding-agent call to return output, usage, traces, file -changes, produced artifacts, failures, cleanup outcomes, and execution -provenance. Future clients may need the same execution boundary without -Promptfoo or evaluation semantics. - -A coding-agent execution is more than a model request. It includes immutable -workspace selection, repository or snapshot acquisition, environment setup, -credentials, permissions, agent invocation, cancellation, evidence capture, -process termination, and cleanup. A workspace may contain multiple repositories -or be produced from a digest-pinned snapshot or organization-specific source. -Those responsibilities need one public contract while allowing materially -different execution backends and acquisition mechanisms. +AllAgents already owns cross-client agent configuration, project workspace +knowledge, global profiles, plugins, hooks, MCP configuration, and generated +launchers. External systems also need to invoke those agents without importing +AllAgents internals or driving an interactive terminal. + +The first planned consumer is AI Evals. It needs one remote coding-agent call to +return terminal output, usage, traces, file changes, produced artifacts, +failures, cleanup outcomes, and execution provenance. Future trusted tools on a +private developer network may need the same execution boundary. + +The initial product is not a public multi-tenant control plane. Developers are +expected to run one gateway for one AllAgents project workspace and expose it on +loopback, a firewalled network, or a Tailscale network. Network reachability is +the trust and authorization boundary. + +A coding-agent execution still includes more than a model request. The gateway +must acquire an immutable workspace, select a configured agent target, contain +credentials to their required phases, propagate cancellation, collect evidence, +terminate descendants, and clean up. Those responsibilities need one public +contract even when the initial deployment remains a single trusted process. The contract must not turn AllAgents into an evaluation harness. Dataset -expansion, repetition, assertions, scoring, experiment scheduling, and durable +expansion, repetitions, assertions, scoring, experiment scheduling, and durable evaluation Runs remain consumer concerns. ## Decision -### Add a separately deployable execution gateway +### Add a trusted-network execution gateway -AllAgents will provide a separately testable and deployable execution-gateway -entry point. It will not be coupled to an interactive CLI command lifecycle. +AllAgents will provide an independently testable `allagents gateway serve` +entry point. It is separate from the interactive CLI command lifecycle but may +run as a single local service process that supervises acquisition and provider +child processes. -The gateway owns: +The gateway implements A2A 1.0 HTTP+JSON plus a required versioned AllAgents +coding-execution extension. It owns: -- authentication and authorization; - stable Task and idempotency identity; +- execution-target selection; +- workspace acquisition; - deadline and cancellation propagation; -- execution-profile and backend selection; -- normalization of terminal output and evidence; -- protocol-level Task status and bounded retention; and +- normalized terminal output and evidence; +- bounded Task and Artifact retention; and - enforcement of the coding-execution contract across every backend. The gateway is not an evaluator, grader, experiment scheduler, retry authority, -or durable evaluation Run ledger. It does not own a consumer's result store. - -### Keep the gateway separate from execution backends - -The initial design supports two execution backends, delivered in this order: - -1. Codex; and -2. Pi. - -Each backend implements the same conformance contract. Provider-specific -process, session, structured-output, cancellation, and evidence behavior -remains behind its adapter. OpenCode and other coding agents remain possible -follow-up adapters rather than part of the first delivery. - -Execution workers own workspace materialization, environment setup, agent -invocation, evidence collection, process termination, and cleanup. The gateway -must not execute evaluated agents, run materializer images, or mount writable -workspaces in the gateway process. - -When deployed on Kubernetes, the gateway runs as its own Deployment and -ClusterIP Service, separate from consumers and execution workers. A backend -dispatches to a worker pool, per-invocation Job, or stronger sandbox according -to the selected execution profile. The protocol does not require one worker -topology. - -A separate gateway Pod is a service and failure boundary, not per-invocation -security isolation. Deployments requiring hostile-code or tenant isolation -must create or select a stronger execution boundary behind the gateway. - -### Make workspace materialization explicit and operator-registered - -The public extension represents one workspace as a closed discriminated union. -The allowed `kind` values and shapes are: - -1. `repositories`, with a bounded list of direct Git repositories, each with a - canonical credential-free HTTPS URL, full commit object ID, collision-free - relative destination, and optional repository-relative subdirectory; -2. `workspaceSnapshot`, with an OCI workspace snapshot referenced by manifest - digest and accompanied by the versioned AllAgents workspace manifest; or -3. `materializer`, with an operator-registered materializer ID, an expected - workspace-manifest digest, and bounded structured inputs. - -Fields from another union variant are invalid. - -The third mode supports organization-specific acquisition such as JFrog, -generated sources, or custom monorepo assembly without accepting executable -configuration from the caller. The request cannot supply a builder image, -Dockerfile, Compose file, shell command, credential, mutable image tag, network -policy, or output contract. - -Direct Git and OCI acquisition revalidate scheme, normalized host, resolved -address, port, and redirect policy for every connection. OCI foreign or -external layer URLs are rejected by default, and registry credentials are never -forwarded across origins. - -Each materializer ID is defined in an operator-owned deployment registry. The -gateway receives only its non-secret descriptor: ID, bounded input schema, -expected definition digest, expected output-manifest version, and required -worker capabilities. The worker receives the runtime definition, which -additionally pins an OCI image by digest and fixes credential handle names or -mount identities, allowed network destinations, resource and phase deadlines, -cache policy, and the OCI runner or sandbox capability. Credential values are -not part of either descriptor. - -The worker computes an algorithm-qualified definition digest over a versioned, -domain-separated canonical serialization of every non-secret, -behavior-affecting runtime field. The expected workspace-manifest digest -likewise identifies the canonical bytes of one declared manifest version. An -execution profile explicitly allows source modes and materializer IDs and -authorizes canonical Git repositories or namespaces, OCI namespaces, and -resource selectors inside structured materializer inputs. At readiness the -gateway matches its expected descriptor digest and profile against the -authenticated worker's computed digest and capabilities. At admission it -validates the selected ID, expected output digest, structured inputs, and -resource authorization; the worker resolves the same definition locally and -rejects missing, changed, or unsupported definitions before acquisition. - -Every source mode produces the same versioned workspace manifest. The manifest -separates worker-verified observations from materializer-attested claims and -records the verification method for each identity. It includes requested and -resolved commits or OCI digests, destinations, materializer identity and image -digest when applicable, normalized input and output digests, resulting tree -identities, and completeness. Materializer assertions are not described as -independently verified unless the worker or a configured trusted acquisition -service performs that verification. - -The worker materializes into a worker-owned staging directory under the same -filesystem publication root as the final workspace; readiness rejects a -cross-filesystem layout and publication never falls back to copy-then-delete. -After validating paths, file types, limits, identities, and the manifest, the -worker stops the materializer and removes its credential, process, mount, and -runner boundary. The validated host-owned staging tree remains. The worker then -atomically renames that tree into its final location before profile-owned setup -or any coding agent starts. - -The registered image is part of the deployment's trusted computing base. The -worker launches it through a configured OCI runner or sandbox in a boundary -separate from the agent runtime and never exposes that runner's control socket -to setup or model tools. Phase isolation prevents later code from receiving the -materializer's credentials or mounts, but it cannot make a malicious -operator-registered image safe from credentials intentionally given to it. -Operators must review and pin that image. A deployment that will not trust it -with credentials needs a separately versioned broker or central snapshot -protocol, which is deferred from the initial architecture. - -The canonical source request enters caller idempotency. The resolved -materializer definition digest enters the effective-profile binding, and both -the definition and output-manifest digests enter terminal provenance. New-claim -source authorization always runs before cache lookup. Cache metadata and keys -include the canonical source, materializer-definition digest, authorization -scope digest and revocation epoch, configured trust domain, and, for GitHub App -sources, the current installation-entitlement generation. Authenticated App -lifecycle webhooks and bounded control-plane reconciliation advance that -generation on uninstall, suspension, or repository-selection change. Unknown -or stale installation state fails cache authorization rather than reusing an -entry. Reuse also requires manifest and content revalidation under the current -authorization scope. Cache metadata preserves the original acquisition -provider metadata; terminal provenance distinguishes `cache_hit`, that original -provider, and the provider selected by current policy instead of claiming that -the current provider performed acquisition. Secret resolution and token minting -remain cache-miss-only. - -This keeps Harbor's useful separation between content-addressed task acquisition -and environment execution without adopting task-owned opaque source. The -comparison is recorded in -[Harbor repository materialization lessons](../research/harbor-repository-materialization.md). - -### Resolve GitHub source credentials from trusted deployment policy - -The public source request remains credential-free and does not select a -credential provider. The trusted acquisition boundary normalizes the repository -host and resolves a provider from operator configuration. `github.com` selects -the built-in GitHub source backend; GitHub Enterprise Server hosts require an -explicit host and API mapping because a custom hostname does not identify its -provider. The effective profile authorizes the canonical repository and provider -entitlement before cache lookup. - -For GitHub repositories, an ordered policy may prefer a GitHub App and permit a -local GitHub CLI fallback. The App provider is applicable only when trusted -operator configuration maps the requested repository to an installation ID; -`@octokit/auth-app` does not discover that mapping. It mints an installation -token scoped only to that repository, read-only contents permission, and its -GitHub expiry. A trusted-local CLI provider is pinned to one configured -non-secret account, which participates in its entitlement and effective-profile -digests. It may run only when no App installation mapping applies, as -`gh auth token --hostname --user `, with `GH_TOKEN`, -`GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` removed -from its environment. Failure to resolve the configured account fails that -provider. This is eligibility fallback, not authentication retry: after an App -provider is selected, configuration, authentication, minting, permission, -repository, rate-limit, or service failure terminates acquisition and never -falls through to the broader user identity. - -When AllAgents owns GitHub App token minting, a trusted control-plane -credential-provider component uses the focused `@octokit/auth-app` package -rather than implementing App JWT, clock-skew, expiry, and installation-token -renewal itself. Every cache-miss acquisition requests a fresh installation token -with auth-app cache bypass (`refresh: true`). Its remaining lifetime must be -strictly greater than the acquisition deadline plus the configured clock-skew -margin, and the delivery lease cannot outlive the token. Readiness rejects an -acquisition-phase ceiling that can exceed a fresh token's safe lifetime. Git -remains the repository transport; the full Octokit client is not required. - -The initial remote architecture is one central token-minter path. The -gateway/control-plane credential-lease controller is authoritative: an -authenticated worker requests credentials only for its active attempt and -fence; the controller rechecks the current command revision, lease epoch, -tombstone, and fence in durable dispatch state, then derives the -effective-profile digest, selected provider, host/API-mapping digest, -installation ID, canonical repository, operation, worker route and identity, -and expiry from durable dispatch and policy state. It issues a single-use, -non-durable grant/response and, when the minter is separate, requires its -configuration digest to agree with those selected bindings. Replay, worker -field substitution, stale command state, and configuration disagreement fail -closed. - -The authenticated delivery lease and channel bind that derived state to the -worker identity, attempt, lease epoch, command revision, fence, operation, and -expiry. Those bindings do not alter the bearer token: after delivery, the token -is enforceably scoped only by GitHub to the repository, read-only contents -permission, and token expiry. The remote worker never receives the App private -key, and only its one-shot acquisition child receives the token. A remote App -profile fails readiness when the central minter, authoritative lease controller, -fresh-token lifetime check, or authenticated non-durable delivery capability is -absent. A versioned central snapshot-delivery protocol is deferred and is not -an initial readiness alternative. - -The local GitHub CLI provider and remote lease path expose their resolved tokens -only to the one-shot acquisition process. Neither exposes credentials to setup, -the coding-agent runtime, model tools, repository configuration, process -arguments, logs, evidence, or the published workspace. Public failures use only -deterministic coarse source-auth code, safe reason, and retryability: -`source_auth_unavailable/no_eligible_provider`, -`source_auth_denied/installation_repository_denied`, and -`source_auth_failed` with `app_configuration_invalid`, -`app_authentication_failed`, `app_mint_failed`, or -`trusted_local_cli_failed` are not retryable; `source_auth_failed` with -`provider_rate_limited` or `provider_unavailable` is retryable. Provider, -installation, and account identifiers are non-secret but operator-only -provenance. Cache-hit provenance separately records `cache_hit`, the original -acquisition provider, and current policy selection. - -A standalone network broker is not required for trusted local execution: the -CLI provider may be a subprocess and a trusted co-located deployment may host -the App minter and lease controller inside its control plane. Remote routes -still use the same authenticated central-minter contract; the minter may be -split into a standalone service when private-key isolation, independent audit, -scaling, or blast-radius requirements demand it. - -The supporting precedents and trust-boundary analysis are recorded in -[Source credential broker precedents](../research/source-credential-broker-precedents.md). +or durable evaluation Run ledger. + +### Trust the network boundary instead of adding application authentication + +The initial gateway has no application-level authentication or per-caller +authorization. It may bind to loopback, a specific interface, or `0.0.0.0`. +Loopback remains the default when no listen address is supplied, but an explicit +`0.0.0.0` binding is valid and requires no unsafe-mode flag. + +Every host able to reach the listener is equally trusted. Any reachable caller +may invoke every exposed target, list and retrieve retained Tasks and Artifacts, +and request cancellation. Task lookup and idempotency are deployment-wide, not +scoped to a caller identity. Operators must use Tailscale ACLs, host firewalls, +container networking, or equivalent network controls when the listener is not +loopback-only. + +TLS termination, OIDC, static bearer tokens, per-tenant ownership, and +multi-tenant information-hiding are deferred. They require a separate decision +when the service is exposed outside one trusted network boundary. + +### Use existing workspace files as the configuration authority + +The initial gateway has no `gateway.yaml` or `worker.yaml`. + +One gateway process serves one project workspace selected by `--workspace` or +the current directory. The project `.allagents/workspace.yaml` remains +canonical for repository identities, remote sources, destination paths, +default revisions, workspace files, plugins, and named OCI snapshot sources. + +The user `~/.allagents/workspace.yaml` remains canonical for global profiles and +launcher-backed execution targets. A launcher-bearing profile client is exposed +only when it explicitly declares: + +```yaml +profiles: + review: + clients: + - name: codex + launcher: codex-review + gateway: + expose: true +``` + +The public target ID is the launcher basename. Launcher names are already +portable and collision-checked across every user profile, while one profile may +contain several clients and therefore several launchers. Internally the target +resolves to exactly one `(profile, client)` pair. The gateway reserves built-in +target IDs, initially `codex` and `pi`; an exposed launcher whose portable +collision key matches a built-in ID is invalid. + +The built-in `codex` and `pi` targets remain available when their adapters are +ready. Explicit launcher-backed targets add configured variants such as +`codex-review` and `pi-tools`. Initially only Codex and Pi profile clients are +gateway-executable; other launcher-bearing clients become eligible only after a +reviewed adapter implements the common execution contract. + +The generated launcher file is a local UX artifact, not the remote execution +boundary. The gateway never discovers launchers from `PATH`, accepts a command, +executable path, arbitrary arguments, or environment overrides from a request, +or appends request data to a generated launcher. It resolves the profile through +its typed adapter and invokes the provider's supported automation surface. + +Process-level options use exact flags and environment variables for: + +- listener and workspace selection; +- a project-specific state-directory override; +- terminal Task retention and bounded Artifact/event storage; +- GitHub App identifiers and private-key file references; +- the configured GitHub CLI account; +- an OCI credential file or fixed credential-helper executable; and +- Codex and Pi auth-file handles. + +By default the state root is a deterministic child of +`~/.allagents/gateway/` keyed by the canonical project-workspace identity. The +store persists and verifies that identity and holds an exclusive lock for the +process lifetime. The root is current-user owned, private, symlink- and hard- +link-resistant, and disjoint from project, profile, and invocation roots. + +Secret values never belong in either workspace file. + +### Support direct repositories and OCI workspace snapshots + +Each request selects exactly one closed workspace source variant: + +1. `repositories`, which materializes the repositories declared by name in the + project workspace and accepts only optional revision overrides; or +2. `workspaceSnapshot`, which selects a named OCI snapshot repository declared + in the project workspace and supplies an immutable OCI manifest digest plus + the expected AllAgents workspace-manifest digest. + +Fields from another variant are invalid. The gateway does not fall back from an +OCI snapshot to Git repositories, or from Git repositories to a snapshot, after +a Task selects its source mode. + +For direct repositories, callers cannot override repository URLs or destination +paths. A revision override is keyed by a declared repository name. Branches and +tags may be accepted for developer convenience, but the gateway resolves and +records the full commit object ID before provider execution. Reproducibility- +sensitive callers should supply full commit IDs. + +For OCI snapshots, the project workspace declares the registry repository: + +```yaml +workspaceSnapshots: + evaluation: + repository: ghcr.io/entityprocess/allagents-workspaces +``` + +The request supplies the name `evaluation`, a `sha256:` OCI manifest digest, and +a `sha256:` workspace-manifest digest. The gateway constructs the full OCI +reference server-side. Callers cannot supply a registry host, repository name, +mutable tag, extraction destination, credential, or external-layer policy. + +Both modes produce the same versioned workspace manifest. It records requested +and resolved repository identities, destinations, acquisition kind, relevant +OCI manifest and layer digests, the workspace-manifest digest, completeness, +and whether each fact was independently verified or snapshot-attested. A commit +listed inside an OCI snapshot is not described as independently verified unless +the gateway separately verifies it against its Git remote. + +Acquisition occurs in a gateway-owned staging directory. The gateway validates +paths, collisions, file types, symlinks, layer and file counts, individual and +total sizes, digests, and the workspace manifest before atomically publishing +the invocation workspace. Absolute paths, traversal, device files, sockets, +escaping links, foreign or external OCI layers, and cross-origin credential +forwarding are rejected. + +### Resolve GitHub credentials with App-first eligibility fallback + +The source request is credential-free and never selects a credential provider. +For `github.com`, the gateway supports two trusted providers: + +1. a configured GitHub App; and +2. a configured GitHub CLI account. + +The App is preferred when it has an installation covering the configured +repository. Installation applicability has three outcomes: `eligible`, +`ineligible`, and `unknown`. The gateway discovers applicability through an +App-authenticated GitHub API client, or verifies an explicitly configured +installation ID against the repository. `@octokit/auth-app` handles App JWT and +installation-token authentication; it is not treated as the repository- +discovery policy by itself. + +For an eligible installation, the gateway requests a fresh repository-scoped +installation token for each acquisition and grants only required read +permissions. Acquisition receives at most 900 seconds or the shorter remaining +Task deadline. The token must remain valid beyond that sub-budget plus a +60-second clock-skew margin. + +GitHub CLI is an eligibility fallback only when the App is not configured or +applicability is positively `ineligible`. An `unknown` result caused by +configuration, authentication, rate-limit, permission, or service failure +terminates acquisition. The CLI provider invokes: + +```text +gh auth token --hostname github.com --user +``` + +with `GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and +`GITHUB_ENTERPRISE_TOKEN` removed from its environment. The configured account +is part of the acquisition-policy digest. + +After an App installation is selected, App configuration, authentication, +token minting, permission, repository-coverage, rate-limit, or service failure +terminates acquisition. The gateway never retries the same Task through the +broader GitHub CLI identity. + +Git receives credentials only through an invocation-scoped helper under +hermetic Git configuration. The gateway excludes system, global, and repository +credential helpers, Git Credential Manager, askpass, SSH agents, repository- +controlled secondary fetches, and executable Git configuration. Tokens never +appear in clone URLs, command arguments, Git configuration, logs, Tasks, +Artifacts, retained workspaces, profile setup, MCP processes, agent processes, +or model-invoked tools. The helper and token are destroyed before provider +execution. + +OCI credentials come from a configured auth-file or standard credential helper, +are scoped to snapshot acquisition, and are removed before publication. Public +registries require no credential configuration. + +### Integrate providers through typed adapters + +The initial backend registry contains Codex and Pi, delivered in that order. +Each adapter implements one behavior-focused contract for availability, +capabilities, invocation, progress, deterministic permission handling, abort, +terminal output, optional structured result, usage, native evidence, and +disposal. + +The Codex adapter depends directly on `@openai/codex-sdk`, creates one fresh +thread per Task, passes cancellation and optional output schema through the SDK, +and consumes structured events. + +The Pi adapter uses strict RPC mode with invocation-owned configuration and a +restricted policy extension. Repository extensions and unrestricted built-ins +are not loaded merely because they exist in acquired source. + +CLI-backed compatibility adapters may be added later when a client has a stable +machine protocol. Missing controls are reported honestly as capability gaps. +The gateway never scrapes a TUI or exposes arbitrary installed executables. +OMP is Pi-derived and is added only for demonstrated OMP-specific value beyond +direct Pi. + +Provider preparation is adapter-owned and typed. The gateway never executes +project or user `setup` shell entries as part of acquisition or invocation. +Validated profile settings, plugins, MCP declarations, and deterministic +workspace projections are applied through existing typed transforms. + +Provider control processes, MCP children, and model-invoked tools receive +distinct allowlisted environments and filesystem views. The provider control +process sees only its invocation-private auth channel; each MCP child sees only +its own resolved secrets; shell and other model-invoked tools see neither +provider nor MCP credentials. Every view excludes gateway state, operator home, +App keys, GitHub/OCI stores, acquisition helpers, unrelated adapter auth, and +the parent environment. A target is not ready unless its adapter can enforce +these separations. This credential/state isolation is required even though +general hostile-code sandboxing remains deferred. ### Persist Task truth, not live provider execution -The gateway durably stores Task identity, idempotency claims, terminal status, -Artifact metadata, and retained evidence. A provider execution itself is -ephemeral. The initial service does not checkpoint, reattach, resume, or -automatically replay an interrupted provider session. - -Gateway restart invalidates the active attempt fence and settles each -nonterminal Task failed once. A live worker that loses its lease aborts the -provider and cleans its invocation. If the worker process crashes, an external -supervisor terminates the complete execution boundary and the replacement -worker reaps or quarantines orphaned invocation roots before readiness. -Termination and filesystem cleanup are recorded separately and become complete -only when the responsible boundary proves them; otherwise the terminal record -says unknown. Durable execution and provider-session restoration require a -later decision backed by public provider guarantees. - -### Integrate providers directly - -The Codex adapter depends directly on `@openai/codex-sdk`; AllAgents does not -vendor or depend on Promptfoo's provider. Promptfoo's -[Codex provider](https://github.com/promptfoo/promptfoo/blob/main/src/providers/openai/codex-sdk.ts) -and -[tests](https://github.com/promptfoo/promptfoo/blob/main/test/providers/openai-codex-sdk.test.ts) -are characterization references for strict option mapping, minimal child -environment, working-directory validation, `AbortSignal`, structured output, -event normalization, and cleanup edge cases. - -AllAgents keeps only the gateway-owned subset: one fresh provider session per -Task, server-owned profile settings, bounded native evidence, typed failures, -and worker-proven process cleanup. It does not inherit Promptfoo configuration -layering, caching, pricing, eval retries, thread pools, or `ProviderResponse`. - -The extension defines `allagents.result-schema/v1` as a closed, bounded JSON -Schema Draft 2020-12 subset shared by admission, Codex, Pi, and terminal -validation. It requires an object root, requires every object schema to set -`additionalProperties: false`, lists every declared property in `required`, and -uses `null` unions for optional values. It allows only `type`, `properties`, -`required`, `additionalProperties` with the value `false`, `items`, `enum`, -`const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`, and rejects -remote references, format-dependent validation, and unknown keywords. The -extension version fixes byte, nesting, property, and enum limits. One shared -validator checks both the schema and the returned value, and the accepted schema -digest enters idempotency and provenance. Adapters cannot widen or narrow this -contract. - -Codex receives that schema through the SDK's per-turn `outputSchema`; Pi -implements the same terminal contract with an invocation-scoped terminating -tool. A successful structured request publishes exactly one Artifact named -`allagents.structured-result` with one A2A `Part` whose `data` field contains -the validated result object and whose `mediaType` is `application/json`. -Artifact metadata contains the result-schema version and digest. The Artifact -exists only for a valid result. The integrity -kernel always records `not_requested`, `not_produced`, `valid`, or `invalid`; -an earlier source, setup, provider, cancellation, or deadline outcome remains -the primary Task classification when no result could be produced. - -### Profile A2A 1.0 instead of inventing an invocation API - -The external contract profiles the Linux Foundation -[Agent2Agent protocol](https://a2a-protocol.org/latest/specification/). The -initial profile requires the A2A 1.0 HTTP+JSON binding and retains Agent Card, -Message, Part, Task, Artifact, status, streaming, cancellation, security, and -error semantics. - -The profile narrows A2A for deterministic coding execution: - -- every accepted execution request creates exactly one addressable A2A Task; - direct-Message completion is not supported; -- the gateway implements all mandatory A2A core operations, including - `SendMessage`, `GetTask`, `ListTasks`, and `CancelTask`; when its Agent Card - advertises streaming, it also implements `SendStreamingMessage` and - `SubscribeToTask`; capability-gated operations retain their standard A2A - behavior instead of being replaced by bespoke `/v1/invocations`, `/v1/runs`, - or `/v1/trials` resources; -- terminal Task results use Artifacts for output and evidence rather than - relying on transient messages or stream events; and -- each versioned Agent Card advertises one mandatory AllAgents extension version - for source and runtime identity, traces, usage and cost, file changes, - produced artifacts, typed failures, cancellation and cleanup outcomes, - evidence completeness, and provenance. - -Generic A2A conformance is insufficient. The AllAgents extension and its -conformance fixtures define the coding-execution guarantees every backend must -satisfy. - -Breaking extension changes use a new extension URI and a versioned Agent Card -or service endpoint. During migration, the gateway keeps the old card, endpoint, -and required extension serviceable while consumers move to the new profile. -Each card requires exactly one extension version. Clients pin the card they -support; the gateway never silently falls back across incompatible versions. -Retiring an old profile is a separate coordinated compatibility decision, not a -lockstep deployment requirement. - -The AAIF -[agentgateway](https://github.com/agentgateway/agentgateway) project may be used -as traffic-policy infrastructure for A2A, MCP, or model calls. It is not the -AllAgents execution service or evidence schema. Documentation uses **AllAgents -execution gateway** where the distinction matters. - -### Keep adjacent protocols at their proper boundaries - -The [Agent Client Protocol](https://agentclientprotocol.com/) may be used behind -a backend adapter when a coding agent supports it. Its session, progress, tool, -permission, terminal, diff, usage, and cancellation semantics are useful -internally, but its stdio editor-to-agent protocol is not the external gateway -API. - -The [Agent Host Protocol](https://microsoft.github.io/agent-host-protocol/) -may be used behind a backend adapter when a host exposes it, or beside the -gateway if AllAgents later adds a collaborative multi-client session surface. -Its host-authoritative snapshots, actions, reconnection, tools, permissions, -and changesets solve live session synchronization; they do not replace A2A -Task identity, idempotency, authorization, terminal evidence, or retention. -The supporting research and implementation consequences are captured in the -[AHP decision inputs](../research/agent-host-protocol-decision-inputs.md). - -[Model Context Protocol](https://modelcontextprotocol.io/) remains a tool and -resource protocol inside an execution backend. It does not represent the whole -coding-agent execution. - -[Agent Format](https://agentformat.org/) may provide an optional static agent -manifest and vocabulary. It does not define the execution transport or prove -observed execution evidence. - -The archived IBM/BeeAI Agent Communication Protocol is superseded by A2A and -will not be adopted. - -### Separate trace propagation, span semantics, and durable evidence - -Gateway calls propagate -[W3C Trace Context](https://www.w3.org/TR/trace-context/) across HTTP and process -boundaries. AllAgents uses OpenTelemetry and OTLP for metadata-only operational -telemetry by default. An explicit allowlist limits structured logs and spans to -non-content operational metadata. Prompts and model outputs, tool arguments and -results, file bodies and source fragments, and secret-bearing attributes are -prohibited before export. A bounded filtering and redaction step must run before -any structured log or span processor so disallowed content cannot enter the -telemetry pipeline. - -AllAgents-managed agent, model, and tool spans use -[OpenInference](https://arize-ai.github.io/openinference/) semantic conventions -only for attributes that pass this allowlist. Backend-native attributes must -pass the same allowlist. Owner correlation is limited to an opaque identifier -appropriate for the telemetry operators' access; it does not expose caller -identity or grant access to a Task or Artifact. Telemetry access and retention -are governed separately from Task and Artifact access and retention. -Consumer-owned evaluator spans may join the propagated trace without becoming -gateway-owned. - -These standards are complementary: - -- W3C Trace Context propagates causal trace identity; -- OpenTelemetry and OTLP represent and transport live operational telemetry; -- OpenInference describes AI operations on OpenTelemetry spans; and -- the AllAgents A2A extension returns durable coding evidence and provenance. - -An external trace backend is not the sole durable result. Sampling, redaction, -transport loss, or retention policy must not erase the terminal facts needed by -a consumer. - -### Trial ATIF only as an optional trajectory Artifact - -The Harbor -[Agent Trajectory Interchange Format](https://github.com/harbor-framework/harbor/blob/main/rfcs/0001-trajectory-format.md) -may be returned as an optional, explicitly versioned A2A Artifact when a backend -can produce or truthfully normalize an ordered agent trajectory. It is not the -A2A transport, the OpenTelemetry trace, or the AllAgents evidence envelope. -Backend-native trajectories remain available when conversion would lose -information. - -An ATIF Artifact must declare its exact schema version and correlate its A2A -Task, OpenTelemetry trace, AllAgents invocation, and backend session identities -through the versioned AllAgents extension. Reasoning content is excluded by -default. Tool arguments, observations, and media follow explicit redaction, -size, and disclosure policy. Truncation or conversion loss is reported rather -than hidden. - -ATIF remains optional until its compatibility policy, specification, tooling, -and non-Harbor conformance mature enough for a required public-contract -capability. - -Harbor's task package, Job configuration, Job/Trial result models, hosted API, -artifact manifest, registry formats, and trial-directory layout will not become -the gateway contract. They remain Harbor-native formats that a future adapter -may preserve. Harbor's ASP `.asp.json` is a draft v0 sandbox proposal and is not -adopted by this decision. - -### Make execution provenance and cleanup explicit - -The gateway and selected worker are collectively responsible for: - -1. validating one canonical immutable workspace request and the selected - profile's exact source-resource or materializer authorization; -2. acquiring direct repositories, restoring a digest-pinned OCI snapshot, or - running the registered materializer in a phase-scoped boundary; -3. producing and validating the standard workspace manifest; -4. transferring the validated staging tree to worker ownership, destroying the - acquisition process/mount/credential boundary, and proving it gone; -5. atomically publishing the host-owned tree on the same filesystem; -6. running profile-owned setup before the evaluated agent action; -7. applying permissions and execution isolation; -8. invoking the agent and propagating cancellation and deadlines; -9. capturing bounded output, usage, cost, file changes, checks, artifact - references, workspace identity, and materializer provenance; -10. returning terminal status, evidence completeness, and provenance; and -11. terminating processes and releasing or retaining resources according to - the documented lifecycle. - -Source transport, materializer image, workspace snapshot, and harness runtime -are independent identities. A backend may use one immutable runtime image plus -a separately digest-addressed workspace artifact; the contract does not require -source code to be baked into the runtime image. - -Credentials remain deployment policy. Requests must not embed deployment -credentials. The gateway authenticates callers, and the selected worker scopes -source credentials to materialization and model credentials to provider -execution without returning secret-bearing paths or values. Materialization -credentials are absent from profile setup, the harness, model-initiated command -environments, tool output, retained evidence, and the published workspace. -Provider and worker-control credentials must likewise be absent from -model-initiated command environments, tool output, retained evidence, and -repository-visible configuration. - -Retries must not multiply non-idempotent agent execution. Every request carries -a caller-scoped stable invocation key through the AllAgents extension. The -gateway binds the authenticated caller, invocation key, effective execution -profile, and request digest to the created Task for a documented retry-retention -window. An identical replay returns the original Task. Reusing the key with a -different request is rejected. Backend retry suppression remains an additional -safeguard; it does not replace gateway deduplication. +The gateway durably stores Task identity, the canonical request, idempotency +claim, selected target and source, effective configuration digest, terminal +status, Artifact metadata, and retained evidence under the configured state +directory. A provider session is not a durable recovery checkpoint. + +An identical idempotency replay returns the existing Task. Reusing the key with +a different canonical request conflicts. Because the initial service has no +caller identity, the idempotency namespace and Task visibility are gateway-wide. + +Terminal Task records, Artifacts, events, and invocation claims expire +atomically after the configured TTL. The retained-count limit never evicts an +unexpired Task; the gateway rejects new admission until expiry frees capacity. +State-store integrity or durability failure stops admission and prevents the +gateway from acknowledging creation or reporting terminal success. + +On gateway restart, interrupted nonterminal Tasks settle failed; provider work +is not resumed or automatically replayed. A new invocation may start fresh. + +### Make cancellation, evidence, and cleanup explicit + +The gateway supervises every acquisition and provider process set. Cancellation +first invokes the provider's native abort or protocol cancellation, then applies +bounded forced termination to the complete descendant set. + +Terminal cleanup evidence is recorded only after the supervisor proves the +complete invocation process set quiescent through an enforceable, invocation- +owned containment primitive. If the platform cannot provide that guarantee, the +gateway fails readiness rather than relying on best-effort process enumeration. +If termination or proof fails, the Task records termination as unknown or +failed and the gateway rejects new work. An unmanaged foreground gateway stays +alive with poisoned readiness and continues reaping while printing the stable +containment identifier and platform recovery command. It may exit with a +nonempty set only after a validated external manager accepts cleanup ownership. + +On startup the gateway identifies every interrupted invocation's containment +set and proves it empty before binding or advertising readiness. It may +quarantine a stale filesystem root only after process quiescence is proven. A +reaping or proof failure terminalizes the Task with unknown/failed termination, +keeps readiness false, and enters the same managed or unmanaged recovery path. + +Terminal evidence distinguishes: + +- agent output; +- optional validated structured result; +- requested and resolved repository or OCI identities; +- pre- and post-execution Git state where applicable; +- produced artifacts; +- usage and bounded provider-native evidence; +- cancellation and termination outcomes; and +- workspace cleanup outcome. + +Credentials, raw secret-bearing paths, and unrestricted prompt, output, tool, +source, or file contents are excluded from operational logs. + +### Profile A2A instead of inventing an invocation API + +The gateway uses A2A Agent Cards, Messages, Tasks, Artifacts, operations, errors, +streaming, and cancellation. The Agent Card declares the AllAgents coding- +execution extension as required. Every operation that creates, returns, lists, +subscribes to, or mutates profiled Tasks or Artifacts activates +`https://allagents.dev/a2a/extensions/coding-execution/v1` through the +`A2A-Extensions` header. Unsupported calls receive the standard A2A extension- +support error, and responses echo the activated URI. + +The versioned extension carries the invocation key, execution target, closed +workspace source, bounded deadline, and optional bounded result schema in its +own strict `Message.metadata` member without rejecting unrelated A2A metadata. +Every terminal Task has one fixed-name, versioned integrity Artifact plus zero +or more produced Artifacts. Breaking extension versions receive versioned cards +and endpoints rather than silent fallback. + +The Agent Card advertises built-in and explicitly exposed launcher-backed +targets through an allowlisted capability projection. It does not publish local +paths, commands, arguments, environment selectors, credentials, exact source +authorization details, or transient worker state. + +ACP, app-server, SDK, and RPC protocols remain backend implementation details. +W3C Trace Context may propagate correlation through HTTP and child-process +boundaries. OpenTelemetry and provider-native evidence remain optional, +separate layers; neither replaces durable Task evidence. ### Keep evaluation commands out of scope This decision does not add `allagents eval`, benchmark authoring, assertions, -scoring, datasets, or experiment scheduling. A community evaluation wrapper and -an enterprise AI Evals wrapper may share this execution service in the future, -but their product and ownership model requires a separate decision. +scoring, datasets, repetitions, experiment scheduling, or automatic execution +retry. Consumers own those concerns. ## Consequences -- AllAgents becomes a service boundary in addition to a local CLI, but retains a - narrow coding-execution responsibility. -- Consumers depend on A2A 1.0 plus a versioned AllAgents extension, not - AllAgents TypeScript modules, CLI behavior, or workspace internals. -- Codex and Pi are the initial execution backends behind one conformance suite; - Codex lands first and OpenCode is deferred. -- Durable Task and evidence records do not imply durable provider execution; - interrupted attempts fail rather than resume or replay. -- The result-schema subset, structured-result Artifact, and non-success result - states are public compatibility surface rather than adapter conventions. -- Reliable worker-crash cleanup requires an external execution supervisor and a - pre-readiness orphan-root reaper in addition to leases. -- Gateway and execution workers scale and fail independently. -- The gateway can remain lightweight; physical isolation and resource policy - belong to the selected execution backend. -- Custom acquisition remains available without making caller-supplied code part - of the trust boundary: operators register digest-pinned materializers and - profiles decide which callers may select them. -- Direct Git, OCI snapshots, and registered materializers converge on one - validated workspace manifest and provenance contract. -- GitHub source credentials are selected by trusted host/profile policy rather - than caller input. GitHub App is preferred when applicable; GitHub CLI is a - local-only eligibility fallback and never masks an App authentication or - authorization failure. -- Deployments that enable external materializers must operate their image, - schema, credential, network, resource, and cache policies as worker - configuration. -- A2A supplies discovery and lifecycle semantics. AllAgents supplies the - coding-specific evidence contract. -- W3C Trace Context, OpenTelemetry/OTLP, OpenInference, optional ATIF, and the - terminal evidence extension remain distinct layers rather than competing - universal formats. -- Implementations must preserve bounded native evidence whenever normalization - would lose information. +- Developers can start one endpoint with `allagents gateway serve` and use + loopback, `0.0.0.0`, a specific interface, Tailscale, or firewall policy. +- There is no application authentication, per-caller authorization, tenant + isolation, `gateway.yaml`, `worker.yaml`, remote worker protocol, or required + Kubernetes deployment in the initial product. +- Project and user workspace files remain the sole declaration authority for + source identities and exposed profile launchers. +- Network reachability grants access to every exposed target and retained Task. + Operators must treat network policy as the authorization boundary. +- GitHub App credentials support private repositories without forcing every + developer to use one identity; GitHub CLI remains a local eligibility + fallback when no App installation applies. +- Direct repositories and digest-pinned OCI snapshots converge on one validated + workspace manifest and evidence contract. +- The gateway process remains a meaningful API and lifecycle boundary, but not + a hostile-code sandbox. Strong multi-tenant isolation remains future work. +- Codex and Pi share one conformance suite while retaining bounded native + evidence and honest capability differences. +- A future deployment configuration becomes justified only when the product + needs multiple worker routes, tenants, credential policies, custom + materializers, centralized storage, or other operator-selected variants. ## Rejected alternatives -### Invent a bespoke invocation, run, or trial API +### Define a second profile registry in `gateway.yaml` -Rejected because A2A already defines remote-agent discovery, Task lifecycle, -streaming, artifacts, cancellation, errors, and web security. Coding-specific -evidence belongs in a versioned A2A extension rather than a parallel transport. +Rejected because global profiles and launcher identities already belong to +`~/.allagents/workspace.yaml`. A second profile map would drift in client, +model, plugin, MCP, and launcher configuration. -### Run agents in the gateway Pod +### Require application authentication for every deployment -Rejected because it couples control-plane availability and credentials to -mutable repository execution, prevents independent scaling, and mistakes a -service boundary for per-invocation isolation. +Rejected for the initial trusted-network product. It would add caller identity, +tenant scoping, token lifecycle, and ingress configuration before the expected +users need those boundaries. Tailscale ACLs and firewalls are the initial access +control. -### Use OpenInference instead of W3C Trace Context +### Restrict the listener to loopback -Rejected as a category error. W3C Trace Context propagates trace identity; -OpenInference supplies AI semantic conventions on OpenTelemetry spans. The -gateway uses both. +Rejected because developers need to expose the endpoint through Tailscale, +containers, VMs, and private networks. Explicit `0.0.0.0` binding is supported; +the operator owns the surrounding network policy. -### Use ATIF as the complete gateway result +### Execute generated launcher files as the remote protocol -Rejected because ATIF represents an ordered agent trajectory, not remote Task -lifecycle, repository provenance, workspace changes, produced artifacts, -cleanup, authorization, or evidence completeness. +Rejected because local launchers intentionally preserve cwd and append local +caller arguments. Remote requests must resolve a typed profile adapter and can +never control commands or argv. -### Adopt Harbor's Job or Trial API +### Let callers provide repository URLs or OCI repositories -Rejected because Harbor's formats own benchmark orchestration, verification, -and persisted runner state. The AllAgents gateway executes one coding-agent -request and does not become an evaluation harness. +Rejected because workspace configuration already defines trusted source +identities and destinations. Requests may select declared names and immutable +revisions or digests, not introduce new origins. -### Let callers provide repository-acquisition code +### Fall back from a selected GitHub App after runtime failure -Rejected because a caller-selected image, Dockerfile, Compose file, or shell -script would turn request parsing into privileged code execution and would make -credential, network, provenance, and cache policy unreviewable. Callers may -select only source modes and materializer IDs explicitly registered and allowed -by the effective execution profile. +Rejected because it would silently change identity and authorization scope after +selection. GitHub CLI fallback applies only when the App is ineligible. -### Replace A2A with the Agent Host Protocol +### Use mutable OCI tags -Rejected because AHP explicitly targets synchronization of independent clients -around host-owned sessions, not agent-to-agent Task execution. Its reconnect -and changeset models do not supply caller-scoped idempotency, immutable source -handling, cleanup, complete terminal evidence, or bounded Task retention. +Rejected because the same request could produce different workspaces. Snapshot +selection requires an OCI manifest digest and expected workspace-manifest +digest. -### Vendor Promptfoo's Codex provider +### Treat provider sessions as durable execution -Rejected because that provider includes Promptfoo-specific configuration -layering, caching, pricing, tracing, retry metadata, thread pooling, and result -mapping. AllAgents needs a smaller worker adapter against the Codex SDK and can -reuse Promptfoo's observable behavior as characterization evidence without -copying its implementation. +Rejected because a resumable provider thread does not prove workspace, +process, cancellation, evidence, or cleanup continuity across gateway restart. -### Treat provider session persistence as durable execution +### Invent a bespoke invocation API -Rejected because a resumable provider thread does not prove workspace, -process, cancellation, evidence, or cleanup continuity across gateway or worker -failure. The initial service durably records failure and cleanup truth but does -not resume interrupted work. +Rejected because A2A already supplies discovery, Task lifecycle, streaming, +Artifacts, cancellation, and errors. Coding-specific evidence belongs in a +versioned extension. + +### Adopt an evaluator's Job or Trial API + +Rejected because benchmark orchestration, verification, and persisted evaluation +state remain consumer concerns. The gateway executes one coding-agent Task. ## Reconsider when -Revisit this decision if A2A standardizes the required coding-execution evidence -without an extension, if a stable cross-vendor execution protocol subsumes the -same lifecycle and provenance guarantees, or if operational evidence shows that -the gateway and backend boundary prevents required cancellation, isolation, or -result integrity. +Revisit this decision when any of these become requirements: + +- callers outside one trusted network must share the endpoint; +- per-caller Task privacy, authorization, or audit identity is required; +- multiple gateway replicas need transactional shared storage; +- execution must route among remote worker pools or hostile-code sandboxes; +- custom materializers are needed beyond direct Git and OCI snapshots; +- multiple GitHub hosts, Apps, CLI accounts, or ordered credential policies need + declarative configuration; +- A2A standardizes the required coding-execution evidence without an extension; + or +- a stable cross-vendor automation protocol subsumes the backend adapter seam. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 5f95d109..4793ba0c 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,7 +1,7 @@ --- title: "Coding-Agent Execution Gateway - Plan" date: 2026-09-18 -deepened: 2026-09-18 +updated: 2026-09-19 type: feat artifact_contract: ce-unified-plan/v1 artifact_readiness: implementation-ready @@ -13,20 +13,29 @@ execution: code ## Goal Capsule -- **Objective:** External systems can run Codex or Pi against an immutable, - provenance-bearing workspace assembled from exact Git repositories, a - digest-pinned OCI snapshot, or an operator-registered materializer through - one authenticated, cancellable, evidence-preserving remote contract. -- **Means:** Add a separately deployable A2A 1.0 gateway, a private worker protocol, and backend-neutral workers with two direct provider adapters (KTD1, KTD5, KTD7-KTD8). -- **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) owns the public boundary. The A2A 1.0 specification owns core wire semantics. The versioned AllAgents extension owns coding-execution semantics. -- **Execution profile:** Build contract-first, then durable Task/evidence state, worker lifecycle, Codex, Pi, packaging, and cross-backend conformance. Preserve the existing local CLI and Node 18 package compatibility. -- **Stop conditions:** Do not execute agents or materializers in the gateway - process, accept mutable source identity, accept caller-supplied acquisition - code or credentials, put deployment credentials in requests, treat streams - or telemetry as terminal evidence, treat provider sessions as recovery - checkpoints, vendor an evaluator's provider implementation, or add - evaluation behavior. -- **Tail ownership:** The implementing workflow runs focused contract and lifecycle tests, the complete repository quality gates, isolated gateway/worker smoke tests, provider-specific credentialed smoke tests where credentials are available, and documentation validation. +- **Objective:** A developer can run one trusted-network A2A endpoint for one + AllAgents workspace and invoke built-in or explicitly exposed profile targets + against either declared Git repositories or a digest-pinned OCI workspace + snapshot. +- **Means:** Add `allagents gateway serve`, a private execution-service package, + a bounded durable Task store, direct Codex and Pi adapters, GitHub App and + GitHub CLI acquisition providers, OCI snapshot acquisition, and one supervised + invocation lifecycle. +- **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) + owns the public and trust boundaries. Project and user `workspace.yaml` files + own source and profile declarations. A2A 1.0 owns core wire semantics. +- **Execution order:** Capture a red built-CLI E2E for the missing gateway; + freeze schemas and configuration projection; implement the Task store, A2A + server, acquisition, supervisor, Codex, and Pi; run a final implementation + review and fix important findings; then run the green built-CLI E2E, + repository gates, and documentation validation. +- **Stop conditions:** Do not add application authentication, `gateway.yaml`, + `worker.yaml`, remote worker routing, caller-supplied URLs or commands, + mutable OCI tags, selected-provider failure fallback, evaluation behavior, or + automatic execution retry. +- **Tail ownership:** The implementing workflow runs focused contract and + lifecycle tests, provider fixture tests, the repository quality gates, a + built-CLI trusted-network smoke test, and documentation validation. --- @@ -34,377 +43,519 @@ execution: code ### Summary -AllAgents gains a remote coding-execution service without becoming an evaluation framework. Callers use A2A Tasks and one required AllAgents extension. The gateway owns caller identity, idempotency, routing, status, cancellation, evidence normalization, and bounded retention. Separate workers own repository materialization, provider processes, mutable workspaces, evidence capture, termination, and cleanup. +AllAgents gains a single-workspace coding-execution service without becoming an +evaluation framework or multi-tenant platform. Callers use A2A Tasks and one +required AllAgents extension. Network reachability is authorization. The +service resolves configured targets and sources from existing workspace files, +acquires a fresh invocation workspace, invokes Codex or Pi through a typed +adapter, and retains bounded terminal evidence. ### Problem Frame -AllAgents currently configures and launches coding clients but has no service boundary for external callers. AI Evals and future consumers would otherwise need to import AllAgents internals, drive interactive CLIs, or independently reimplement repository acquisition, permissions, cancellation, evidence, and cleanup. +AllAgents configures and launches coding clients but has no service boundary for +trusted tools such as AI Evals. Those tools would otherwise import AllAgents +internals, drive interactive CLIs, or duplicate profile resolution, repository +acquisition, credential handling, cancellation, evidence capture, and cleanup. -The two initial runtimes expose different programmatic contracts. Codex provides a TypeScript SDK over structured JSONL events and native per-turn `outputSchema`; Pi provides a strict JSONL RPC mode and invocation-scoped custom tools. The public service must preserve one stable lifecycle and structured-result contract without flattening provider-specific facts into false equivalence. +Developers expect a process they can start in a workspace and expose on +loopback, `0.0.0.0`, a private interface, or Tailscale. They do not need an +application authentication stack, Kubernetes control plane, remote worker +registry, or another profile configuration file for the initial use case. ### Actors -- A1. **Gateway caller:** An authenticated service such as AI Evals that creates, observes, lists, cancels, and retrieves coding-execution Tasks. -- A2. **Execution gateway:** The A2A server that owns caller scope, Task identity, idempotency, routing, retention, and normalized results. -- A3. **Execution worker:** A separately deployed process that owns registered - workspace materialization, one mutable workspace per invocation, provider - execution, evidence capture, and cleanup. -- A4. **Backend adapter:** The Codex or Pi integration that translates native - events, structured results, cancellation, usage, failures, and evidence into - the worker contract. -- A5. **Operator:** The person or deployment system that defines profiles, - materializer registrations, credentials, limits, retention, worker - endpoints, and observability policy. +- A1. **Trusted-network caller:** Any process able to reach the endpoint. All + callers have the same authority and Task visibility. +- A2. **Execution gateway:** The A2A server and invocation supervisor. It owns + deployment-wide Task identity, acquisition, routing, status, cancellation, + evidence, retention, and cleanup. +- A3. **Backend adapter:** The Codex or Pi implementation translating native + automation events and cancellation into the common contract. +- A4. **Operator/developer:** The person who selects the project workspace, + exposes profile launchers, supplies process flags and credential handles, and + controls network access. +- A5. **GitHub/OCI source:** The remote content service used only during the + acquisition phase. ### Key Decisions -- **Profile A2A rather than creating a public invocation API.** The service keeps standard Agent Cards, Tasks, Artifacts, operations, errors, and capability negotiation. Governs R1-R4. -- **Keep execution outside the gateway process.** Mutable repositories and provider processes belong to workers. Governs R10-R16, R21-R22. -- **Persist Task truth, not live executions.** Accepted Task identity and terminal evidence survive restart; provider sessions do not resume or replay. Governs R7, R14, R16-R18. -- **Keep evaluation outside AllAgents.** Dataset expansion, repetitions, assertions, scoring, retries, and durable evaluation Runs remain caller concerns. Governs R20. -- **Make workspace acquisition explicit but extensible.** Requests select one - versioned workspace source mode; custom acquisition uses only - operator-registered, digest-pinned materializers allowed by the profile. - Governs R6, R11-R13, R16-R18, R21-R22. -- **Resolve source credentials from trusted host and profile policy.** Callers - never select a credential provider. For GitHub, prefer an applicable GitHub - App installation and permit GitHub CLI only as an explicit trusted-local - fallback when no installation applies; never fall back after a selected App - provider fails. When AllAgents owns App token minting, use - `@octokit/auth-app` rather than a custom minter. (session-settled: - user-directed.) Governs R6, R12-R13, R16, R21-R22. +- **Use A2A rather than inventing an invocation API.** Standard Agent Cards, + Tasks, Artifacts, errors, streaming, and cancellation remain the public + lifecycle. Governs R1-R3. +- **Treat the network as the trust boundary.** The initial service has no + application authentication or caller ownership. Explicit `0.0.0.0` binding is + valid. (session-settled: user-directed.) Governs R4-R5. +- **Reuse workspace configuration.** Project `workspace.yaml` owns sources; + user `workspace.yaml` owns profiles, launchers, and exposure. There is no + `gateway.yaml`. (session-settled: user-directed.) Governs R6-R8, R18. +- **Support two acquisition modes.** Direct declared repositories and named, + digest-pinned OCI workspace snapshots converge on one manifest and evidence + contract. (session-settled: user-directed.) Governs R9-R11. +- **Use App-first GitHub credential eligibility.** Prefer an applicable GitHub + App; use a configured `gh` account only when no App installation applies; + never fall back after selected-App failure. (session-settled: user-directed.) + Governs R10-R11. +- **Keep a typed backend seam.** Codex SDK and Pi RPC are the complete initial + backend set. Launcher-backed profiles resolve through those adapters rather + than executing generated wrapper files. Governs R7-R8, R12-R15. +- **Persist Task truth, not provider sessions.** Restart settles interrupted + work failed; it never resumes or automatically replays provider execution. + Governs R5, R13-R16. +- **Keep evaluation outside AllAgents.** Consumers own datasets, repetitions, + scoring, assertions, and evaluation Runs. Governs R17. ### Requirements -**Public protocol and compatibility** - -- R1. The gateway implements A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, `GetTask`, `ListTasks`, and `CancelTask`; it implements streaming send and task subscription when the card advertises streaming. -- R2. Every valid new request returns exactly one addressable Task. Direct-Message completion and follow-up messages to an existing Task are unsupported. Non-streaming send honors A2A `returnImmediately`; streaming always emits the durable Task first. -- R3. The public extension URI is `https://allagents.dev/a2a/extensions/coding-execution/v1`. The Agent Card advertises it as required; HTTP clients opt in with `A2A-Extensions`; each request sets `Message.extensions` to include the URI and puts the schema-defined request only at `Message.metadata[uri]`. Every terminal Task contains exactly one fixed-name `allagents.execution-integrity` Artifact whose `extensions` includes the URI and whose single `Part` contains the schema-defined integrity envelope in `data` with `mediaType: application/json`. The Task uses only the standard A2A fields and never adds `Task.extensions`. Unsupported or missing required extension versions fail without fallback. -- R4. Terminal output and execution evidence are retrievable as Task Artifacts for the configured retention window even when the original stream disconnects. Active subscription emits the current Task snapshot then future events without promising replay of missed progress; terminal subscription returns the standard unsupported-operation error and callers use `GetTask`. - -**Caller identity, Task identity, and retention** - -- R5. Every protocol operation authenticates the caller and scopes Task lookup, listing, subscription, cancellation, and artifact retrieval to that caller's tenant and principal before storage access can reveal resource existence. Production public ingress reaches the gateway through TLS terminated at the configured named trusted boundary; an unauthenticated loopback-only development listener is the sole plaintext exception. Remote gateway-worker links use mTLS or an explicitly configured equivalent authenticated encrypted overlay, while a same-host Unix socket is acceptable. The authenticated worker identity is bound to its route, capabilities, and attempt fence, and readiness fails for plaintext or identity-mismatched remote endpoints. -- R6. After authentication, required-extension checks, and bounded canonical parsing, the gateway first resolves the owner-scoped invocation claim. A retained claim compares the canonical caller request and result-schema digest against the originals and returns its existing Task only while its stored original effective-profile and schema bindings remain intact; a mismatch conflicts without dispatch. Current source/profile authorization, profile resolution/readiness, quota, and deadline checks apply only when atomically creating a new claim that binds the authenticated owner, canonical caller request digest, original effective-profile digest, original result-schema digest, and submitted Task. -- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Acceptance of the current worker fence moves a submitted Task to working before source materialization or setup, so a subsequent source/setup failure transitions from working to failed. Task state and terminal Artifact metadata survive gateway restart. Every nonterminal Task present at startup settles failed once, its old attempt fence is invalidated, and stale worker events cannot overwrite it; the initial service never resumes or automatically replays interrupted provider work. -- R8. List operations implement all A2A filters, history bounds, page-size bounds, owner/query-bound cursor pagination, and descending status-update time. One immutable expiry logically hides the Task, claim, events, and artifacts before best-effort physical deletion; expired and unauthorized IDs are indistinguishable. -- R9. Small deployments work without an external database. The built-in durable store supports one gateway replica, enforces per-owner/global admission and storage quotas, and reserves capacity for cancellation and terminal settlement; multi-replica storage is outside this delivery. - -**Execution and policy** - -- R10. Codex and Pi are the complete initial backend set behind one conformance contract, delivered Codex first and Pi second. OpenCode is deferred. (session-settled: user-directed.) -- R11. A request selects a server-defined execution profile and may include one `allagents.result-schema/v1` schema for the terminal result: a bounded JSON Schema Draft 2020-12 subset with an object root, every object schema setting `additionalProperties: false`, every declared property listed in `required`, optional values represented by `null` unions, and only `type`, `properties`, `required`, `additionalProperties` with the value `false`, `items`, `enum`, `const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`. The extension version fixes byte, depth, property, and enum limits; admission rejects remote references, format-dependent validation, and unknown keywords; one shared validator governs schema admission and returned values. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. Requests cannot supply raw provider configuration. -- R12. One request defines one workspace using exactly one closed source union: - `{ kind: "repositories", repositories: [...] }`, - `{ kind: "workspaceSnapshot", reference, workspaceManifestDigest }`, or - `{ kind: "materializer", materializerId, - expectedWorkspaceManifestDigest, inputs }`. Unknown kinds, fields from - another variant, and omitted variant fields fail admission. Repository - entries contain a canonical credential-free HTTPS Git URL, full commit object - ID, collision-free relative destination, and optional repository-relative - subdirectory. Snapshot references are digest-pinned OCI artifacts containing - the versioned workspace manifest. Materializer inputs are bounded by the - registered schema. Profiles explicitly allow source modes and materializer - IDs and authorize exact canonical Git repositories or namespaces, OCI - namespaces, and resource selectors inside materializer inputs. Callers cannot - supply builder images, Dockerfiles, Compose files, shell commands, - credentials, mutable image tags, network policy, or output contracts. Direct - Git revalidates destination policy for every connection, disables redirects - and repository-controlled secondary fetch/exec features, uses hermetic Git - configuration, fetches into an isolated object database from the approved - remote, and verifies that the checked-out commit equals the requested full - object ID. OCI acquisition rejects external or foreign layer URLs by default, - revalidates scheme, normalized host, resolved address, port, and redirects - for registry, authentication, manifest, and blob connections, never forwards - credentials across origins, and verifies every manifest and layer digest. A - materializer output must match the request's expected workspace-manifest - digest before publication. -- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, structured logs/spans before processing or export, and every model-initiated command or tool environment. Credentialed profiles additionally require an OS-enforced provider/tool credential boundary: the credential-bearing provider runtime and model-invoked tools use distinct UID/process/mount policy that prevents tool access to provider processes, procfs entries, and backend config/data roots, or an equivalent credential broker keeps reusable credentials out of the agent runtime. Worker readiness fails when the declared boundary cannot be proved; environment filtering alone is not credential isolation. - Source credential selection is server-side deployment policy, not caller - input. The acquisition boundary maps normalized repository hosts to source - backends; `github.com` selects the built-in GitHub backend, while GitHub - Enterprise Server hosts require explicit operator host/API mappings. Profiles - name an ordered provider policy and authorize its non-secret entitlement - before cache lookup. An App provider is applicable only when trusted operator - configuration maps the repository to an installation ID; auth-app does not - discover installations. Authenticated lifecycle webhooks plus bounded - reconciliation advance an installation-entitlement generation on uninstall, - suspension, or repository-selection change. Unknown or stale installation - state fails cache authorization. Cache metadata preserves the original - acquisition-provider metadata, while hit provenance separately records - `cache_hit`, that original provider, and current policy selection/entitlement. - Secret lookup and token minting remain cache-miss-only. - For a cache-miss GitHub acquisition, an applicable configured App installation - is preferred. Focused `@octokit/auth-app` minting uses `refresh: true` to - produce a fresh token scoped only to the authorized repository, read-only - contents permission, and GitHub expiry. Remaining lifetime must be strictly - greater than the acquisition deadline plus clock-skew margin, and the - credential lease cannot outlive the token. Readiness rejects an acquisition - ceiling that can exceed a fresh token's safe lifetime. A trusted-local - `github-cli` provider may run only when no App installation mapping applies; - it is pinned to a configured non-secret account included in entitlement and - effective-profile digests, invokes - `gh auth token --hostname --user ` without `GH_TOKEN`, - `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, or `GITHUB_ENTERPRISE_TOKEN`, and - fails if that account cannot be resolved. After App selection, no failure - falls through to `gh`. - The initial remote path requires a trusted central token minter and - authoritative gateway/control-plane credential-lease controller. The - authenticated worker requests only by active attempt and fence. From durable - dispatch and policy state, the controller rechecks active command revision, - lease epoch, tombstone, and fence, then derives the effective-profile digest, - selected provider, host/API-mapping digest, installation ID, canonical - repository, operation, worker route and identity, and expiry. It issues and - atomically consumes a single-use non-durable grant/response; a separately - deployed minter must agree with the selected configuration digest. Replay, - substitution, stale state, and controller/minter digest disagreement fail - closed. The authenticated lease/channel binds the derived repository and - provider state to worker identity, attempt, lease epoch, command revision, - fence, operation, and expiry; those are not token claims. The remote worker - never receives the App private key. Readiness fails without this complete - path. Versioned central snapshot delivery is deferred and is not an initial - readiness alternative. - Public failures expose only deterministic coarse code, safe reason, and - retryability; provider, installation, and account identifiers remain - operator-only. `source_auth_unavailable/no_eligible_provider`, - `source_auth_denied/installation_repository_denied`, - `source_auth_failed/app_configuration_invalid`, - `source_auth_failed/app_authentication_failed`, - `source_auth_failed/app_mint_failed`, and - `source_auth_failed/trusted_local_cli_failed` are not retryable. - `source_auth_failed/provider_rate_limited` and - `source_auth_failed/provider_unavailable` are retryable. - Materializer IDs are defined in an operator-owned deployment registry. The - gateway holds only the non-secret ID, bounded input schema, expected - definition digest, expected output-manifest version, and required worker - capabilities; the worker holds the runtime definition with the digest-pinned - image, credential handle names or mount identities, network destinations, - resource/deadline ceilings, cache policy, output-manifest version, and OCI - runner or sandbox capability. Credential values are excluded. The worker - derives an algorithm-qualified `sha256:<64 lowercase hex>` definition digest - from a versioned, domain-separated canonical serialization of every - non-secret behavior-affecting field and advertises it at readiness; the - gateway treats its copy only as the expected digest. The expected workspace - manifest and canonical materializer-input digests use the same - algorithm-qualified format with distinct domain separators and exact - versioned canonical JSON preimages. Readiness fails when computed and expected - descriptors differ across the authenticated route. Materialization - credentials exist only in that isolated phase and are not supplied to setup, - provider execution, or model tools. The registered image is operator-trusted - deployment code: phase isolation protects later phases but cannot make a - malicious registered image safe from credentials deliberately given to it. - Deployments requiring that stronger claim are deferred pending a separately - versioned broker or central snapshot-delivery protocol. -- R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. -- R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. -- R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves the complete invocation process set quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. If bounded termination escalation cannot prove quiescence, the worker persists termination as unknown/failed, poisons admission, and exits so the external supervisor destroys the complete process boundary; replacement readiness performs orphan recovery before accepting work. The same supervisor boundary handles a worker crash. - Every source mode materializes into a worker-owned staging directory under - the same filesystem publication root as the final workspace and produces the - same versioned workspace manifest. Readiness rejects cross-filesystem roots - and publication has no copy-then-delete fallback. The worker validates - repository or snapshot identities, destinations, paths, file types, limits, - materializer definition and image digests, output digest, provenance method, - and completeness. It then terminates the supervisor-owned acquisition - process, mount, runner, and credential boundary while preserving the - host-owned validated staging tree, proves that boundary gone, atomically - renames the tree into its final location, and only then starts setup. - -**Evidence and observability** - -- R17. Every terminal Task contains the required `allagents.execution-integrity` Artifact carrying an integrity kernel: Task/source/profile/backend identities, action outcome, a structured-result state of `not_requested`, `not_produced`, `valid`, or `invalid` plus reason and schema digest when requested, cancellation or failure classification, separate termination and filesystem-cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. A valid structured result is exactly one additional `allagents.structured-result` Artifact with one A2A `Part` whose `data` field contains the validated result object and whose `mediaType` is `application/json`; missing or invalid result data never publishes that Artifact. `not_produced` is legal only before a result candidate is produced. Once validation selects `valid` or `invalid`, later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the fixed structured-result Artifact while the later phase remains the primary Task failure classification. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. - Workspace provenance includes the source mode, requested and resolved - repository commits or OCI digests, destination map, workspace-manifest - digest, and, when applicable, materializer ID, computed definition digest, - image digest, canonical input digest, and output digest. It labels each field - as a worker-verified observation, trusted-service verification, or - materializer-attested claim and records the verification method; a custom - image's assertion is never reported as independently verified merely because - its output digest matched. -- R18. Normalized file evidence distinguishes create, edit, delete, and rename where truthful. It preserves bounded provider-native diffs, events, or trajectories when normalization loses information and separately records truncation, redaction, attribution, original/captured size, and digest semantics. -- R19. Gateway and worker calls propagate W3C Trace Context and export metadata-only OpenTelemetry data. One explicit pre-processor allowlist admits only bounded non-content operational metadata; OpenInference and backend-native attributes pass the same allowlist and bounded filtering/redaction before any structured log or span processor. Prompts, model outputs, tool arguments/results, file bodies, source fragments, and secret-bearing attributes are prohibited before export. Owner correlation uses only an opaque identifier appropriate to telemetry-operator access, never caller identity or Task/Artifact authorization. Telemetry access and retention are configured separately from Task and Artifact access and retention, and telemetry is neither durable result truth nor required for terminal lookup. - -**Ownership and safety boundary** - -- R20. The gateway executes one coding request. It does not own eval configuration, datasets, repetition, scoring, retry policy, experiment scheduling, or a durable evaluation Run ledger. -- R21. The initial worker topology is one execution at a time for reviewed repositories inside one configured mutual-trust domain. R13's narrow OS-enforced provider/tool credential boundary is required for credentialed profiles but does not claim hostile-source or cross-tenant isolation. Profiles making either stronger claim are rejected until a full per-invocation UID, mount, PID, network, and credential isolation boundary is configured. -- R22. Gateway admission and worker execution enforce profile limits for request rate, active/retained Tasks, subscriptions, stored bytes, source transfer/expansion, files/inodes, workspace bytes, CPU, memory, PIDs, network, phase deadlines, events, logs, and artifacts. Exhaustion is scoped to one invocation or owner and leaves capacity for terminalization and cleanup. - Materializer CPU, memory, PIDs, network, time, transfer, expansion, file, - inode, and workspace output count against the invocation's limits. New-claim - source authorization precedes every cache lookup. Cached outputs are reusable - only after manifest and content revalidation for the same canonical source, - materializer-definition digest, expected and actual output-manifest digests, - authorization-scope digest, source-authorization revocation epoch, and trust - domain. Revocation advances the epoch and makes the prior namespace - ineligible; the conservative default namespaces cache entries by owner. +**Public protocol** + +- R1. Implement A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, + `GetTask`, `ListTasks`, `CancelTask`, streaming send, and active Task + subscription when advertised. The Agent Card declares + `https://allagents.dev/a2a/extensions/coding-execution/v1` with + `required: true`. Every operation that creates, returns, lists, subscribes to, + or mutates profiled Tasks or Artifacts must include + `A2A-Extensions: https://allagents.dev/a2a/extensions/coding-execution/v1`; + responses echo the activated URI, and unsupported calls receive A2A + `ExtensionSupportRequiredError`. +- R2. Generate a strict versioned request schema from Zod and place it only at + `Message.metadata[extensionUri]`. Strict objects reject every unlisted member. + V1 uses these wire scalars: + - `InvocationKey` matches `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`. + - `ConfigName` and `TargetId` match + `^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`. + - `RevisionText` is NFC UTF-8, 1-255 bytes, with no U+0000-U+001F or U+007F. + - `Digest` matches `^sha256:[0-9a-f]{64}$`. + The request object is exactly: + `version: "1"`; `invocationKey: InvocationKey`; `target: TargetId`; `source`, + one of `{ kind: "repositories", revisions?: Record }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, + digest: Digest, workspaceManifestDigest: Digest }`; optional + `deadlineSeconds` (integer 1-3600, default 1800); and optional + `resultSchema: { version: "1", schema: SchemaNode }`. + + A `SchemaNode` is exactly one branch below. `description` is optional NFC + UTF-8 of at most 1024 bytes. Scalar `enum` arrays contain 1-128 canonically + distinct values of the node's type; string enum values are at most 4096 UTF-8 + bytes, numbers are finite, integers are JSON safe integers, and null permits + only `[null]`. + - null or boolean: `{ type, description?, enum? }`; + - string: `{ type: "string", description?, enum?, minLength?, maxLength? }`, + where lengths are integers 0-1,048,576 Unicode scalar values and minimum + does not exceed maximum; + - number or integer: + `{ type, description?, enum?, minimum?, maximum? }`, where number bounds + are finite, integer bounds are JSON safe integers, and minimum does not + exceed maximum; + - array: + `{ type: "array", description?, items: SchemaNode, minItems?, maxItems? }`, + where item bounds are integers 0-4096 and minimum does not exceed maximum; + - object: + `{ type: "object", description?, properties, required?, + additionalProperties: false, minProperties?, maxProperties? }`, where + `properties` is a strict record of 0-256 `ConfigName` keys, + `required` is a unique subset of those keys, property bounds are integers + 0-256, and minimum does not exceed maximum. + No pattern dialect exists in v1. References, unions/combinators, conditionals, + formats, defaults, coercion, non-finite numbers, duplicate canonical enum + values, and unknown keywords are rejected. The canonical result schema is at + most 64 KiB, 256 nodes, and 32 levels deep. Repository revision count cannot + exceed declared repositories. The Message contains exactly one `TextPart` + whose UTF-8 prompt is 1 byte to 1 MiB; other Part kinds are rejected. Only the + extension-owned metadata object is strict; unrelated A2A metadata and other + activated-extension keys are preserved or ignored according to A2A. + Canonicalization materializes defaults, normalizes extension strings to UTF-8 + NFC, sorts record keys, and hashes RFC 8785 extension JSON plus prompt bytes. + Do not add `Task.extensions` or backend-specific public fields. +- R3. One valid new request creates one addressable Task. Follow-up messages to + an existing Task are unsupported. Every terminal Task has exactly one + integrity Artifact plus zero or more produced Artifacts. The integrity + Artifact has `artifactId` and `name` equal to + `allagents.execution-integrity` and one `DataPart` whose strict + `allagents.execution-integrity/v1` object has the following normative wire + shape. `SafeUInt` is an integer 0-9,007,199,254,740,991; `ShortText` is valid + UTF-8 of at most 4096 bytes; `ArtifactId` matches + `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`; and `MediaType` is a valid RFC 6838 + media type of at most 255 ASCII bytes. + - `version` is the literal `"1"`; `taskId` is a lowercase canonical UUIDv7; + and `target` is `TargetId`. + - `sourceIdentity` is either + `{ kind: "repositories", repositories }` or + `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, + workspaceManifestDigest: Digest, repositories }`. + `repositories` contains 1-64 unique strict entries + `{ name: ConfigName, canonicalUrl: string, requestedRevision?: + RevisionText, resolvedCommit: string, verification: + "independentlyVerified" | "snapshotAttested" }`; `canonicalUrl` is a + canonical HTTPS URL of at most 2048 bytes, and `resolvedCommit` matches + `^[0-9a-f]{40}$`. + - optional `workspaceManifestDigest` is `Digest`. + - `terminalOutput` is `{ text, truncated }`, where `text` is valid UTF-8 of at + most 1 MiB and `truncated` is boolean. + - optional `usage` is a strict object with optional `inputTokens`, + `outputTokens`, `cachedInputTokens`, and `totalTokens` `SafeUInt` fields, + plus optional `provider` containing 0-64 `ConfigName: SafeUInt` counters. + - `producedArtifacts` contains 0-128 strict entries + `{ artifactId: ArtifactId, name?: ShortText, mediaType?: MediaType, + size: SafeUInt, digest: Digest }`; each references one additional A2A + Artifact. + - `evidence` is `{ items, complete, truncated }`, where the booleans have + their literal JSON meaning and `items` contains 0-256 strict entries + `{ kind, artifactId?, digest?, summary? }`. `kind` is one of + `gitState | providerTrace | fileChanges | usage | cancellation | + termination | cleanup`; `artifactId` and `digest` use the aliases above, + `summary` is `ShortText`, and at least one of those three optional members + is present. + - `termination` is `{ status: "clean" | "failed" | "unknown", + reason?: ShortText }`; `cleanup` is + `{ workspace: "removed" | "retained" | "failed", reason?: ShortText }`. + - optional `failure` is `{ code, message, retryable, cause }`, where `code` + is one stable code from the error table below, `cause` is one member of the + closed cause union defined below that table, `message` is `ShortText`, and + `retryable` is that row's fresh-invocation value. + - `result` is exactly one of + `{ status: "valid", value }`, + `{ status: "invalid", errors }`, or + `{ status: "notProduced", reason }`. `value` is JSON that validates against + the requested `SchemaNode`, serializes to at most 1 MiB, and is used only + when a result schema was requested. `errors` contains 1-64 strict + `{ path, keyword, message }` entries: `path` is an RFC 6901 JSON Pointer of + at most 1024 bytes, `keyword` is one of the v1 `SchemaNode` member names, + and `message` is `ShortText`. `reason` is one of + `notRequested | providerDidNotReturn | providerFailed | cancelled | + deadlineExceeded | invalidProviderPayload`. + + Failure, rejection, and cancellation retain every available field without + implying a valid result. Task records, Artifacts, events, and claims expire + atomically after the configured TTL; expired keys may create new Tasks. The + gateway never evicts unexpired Tasks to satisfy the retained-count limit: it + rejects new admission until expiry frees capacity. + +**Trust, identity, and Task storage** + +- R4. Do not authenticate application callers. Allow loopback, specific-address, + and explicit `0.0.0.0` listeners. Every reachable caller may create, list, + retrieve, subscribe to, cancel, and fetch Artifacts for every Task. Document + Tailscale ACLs, firewalls, or equivalent network controls as the authorization + boundary. +- R5. Idempotency and Task visibility are deployment-wide. Atomically and + durably bind an invocation key to the canonical request, selected target, + source identity, optional result-schema digest, deadline, and effective + configuration digest before acknowledging Task creation. Identical replay + returns the existing Task; a changed request conflicts. The project-specific + state root persists the canonical workspace identity and holds an exclusive + process lock. The root and all state files must be current-user owned, use + `0700`/`0600`-equivalent permissions, be disjoint from project, profile, and + invocation roots, and be opened descriptor-relatively without following + symlinks or accepting hard-linked files. Startup verifies those invariants, + store integrity, and workspace identity; terminalizes interrupted Tasks + failed; and never resumes provider work. Store open, corruption, write, + rename, or fsync failure stops admission, aborts and contains active work, + prevents terminal success, and exits only after quiescence or a validated + external manager accepts cleanup ownership. + +**Workspace and target configuration** + +- R6. One gateway process serves one project workspace selected by `--workspace` + or cwd. Parse its `.allagents/workspace.yaml` through the authoritative project + schema. Repository names, URLs, destinations, default revisions, workspace + projection, plugins, and named OCI snapshot repositories come only from that + declaration. +- R7. Parse `~/.allagents/workspace.yaml` through the authoritative user schema. + Built-in `codex` and `pi` targets are available when ready. A launcher-bearing + profile client adds a target only when `gateway.expose: true`. Its public ID is + the globally collision-checked launcher basename and resolves to exactly one + `(profile, client)` pair. Built-in IDs are reserved under the same portable + collision key; colliding exposure is a configuration error. Initially only + Codex and Pi profile clients are executable. +- R8. A request selects a declared target, may set the bounded + `deadlineSeconds`, and may provide one bounded result schema. The overall + deadline covers acquisition, publication, typed preparation, provider + execution, and evidence collection. Acquisition receives + `min(900 seconds, remaining overall deadline)`; exceeding that sub-budget + fails before provider execution. Overall expiry initiates abort and bounded + forced termination. Cleanup then uses its own fixed bounded budget and the + R16 fail-closed quiescence rule. A request cannot provide or override backend, + executable path, command, argv, environment, profile settings, plugins, MCP + servers, repository URLs, destination paths, credential provider, setup + behavior, or permission policy. Readiness rejects missing, partial, drifted, + unsupported, or declaration-missing exposed profiles. + +**Workspace acquisition** + +- R9. Use exactly one closed source union: + - `{ kind: "repositories", revisions?: Record }`; or + - `{ kind: "workspaceSnapshot", snapshot, digest, workspaceManifestDigest }`. + Unknown variants, cross-variant fields, undeclared names, mutable snapshot + references, malformed digests, and destination overrides fail admission. + Source-mode failure never falls through to the other mode. +- R10. Repository mode materializes every configured repository required by the + selected project workspace. Caller revisions may override only a declared + repository's default revision. Canonicalize HTTPS GitHub origins, resolve and + record full commits before provider execution, use hermetic Git configuration, + disable redirects and repository-controlled secondary fetch/exec features, + verify checkout identities, and reject path collisions or escapes. +- R11. Snapshot mode maps `snapshot` to a declared OCI repository and constructs + `@` server-side. Validate registry origin, OCI manifest + and layer digests, expected workspace-manifest digest, paths, symlinks, file + types, file/layer counts, individual and total sizes, and manifest + completeness in staging before atomic publication. Reject external or foreign + layers and cross-origin credential forwarding. The common workspace manifest + distinguishes independently verified Git facts from snapshot-attested facts. + +**Credential selection and containment** + +- R12. Repository requests never carry credentials or select providers. For + `github.com`, determine configured-App applicability as + `eligible | ineligible | unknown` through an App-authenticated GitHub API + client, or verify an explicit installation ID against the repository. For + `eligible`, mint a fresh repository-scoped, read-only installation token + through `@octokit/auth-app` and require remaining lifetime greater than the + R8 acquisition sub-budget plus a 60-second clock-skew margin. Use the + configured GitHub CLI account only when the App is absent or applicability is + positively `ineligible`. An `unknown` result or any selected-App + configuration, authentication, minting, permission, repository, rate-limit, + or service failure terminates acquisition without `gh` fallback. Run + `gh auth token --hostname github.com --user ` with ambient token + variables removed. Deliver either token only through an invocation-scoped Git + credential helper and destroy it before typed preparation or provider + execution. OCI credentials likewise exist only during snapshot acquisition. + +**Execution, evidence, and cleanup** + +- R13. Keep one closed `codex | pi` backend registry and one behavior-focused + interface covering availability, capabilities, invocation, progress, + deterministic permission handling, abort, terminal output, optional structured + result, usage, bounded native evidence, and disposal. Profile targets resolve + adapter-owned configuration directly; never execute generated launchers, + discover executables as targets from `PATH`, scrape a TUI, or append public + input to argv. +- R14. Codex uses pinned `@openai/codex-sdk`, one fresh thread per Task, + `AbortSignal`, streamed events, optional native `outputSchema`, and an + operator-selected Codex auth-file handle. Pi uses strict RPC, invocation-owned + configuration, an operator-selected Pi auth-file handle, and one restricted + policy extension; repository extensions and unrestricted built-ins do not + auto-load. The gateway copies only the selected adapter's required auth + material into an invocation-private, read-only control-process view and + removes it during cleanup. +- R15. Acquire into a private staging root and atomically publish the invocation + workspace. Run only adapter-owned typed preparation that projects validated + project/profile settings, plugins, and MCP declarations through existing + deterministic transforms; never execute project or user `setup` entries or + other configured shell commands. Enforce distinct process views: + - the provider control process receives only its invocation workspace, + minimum non-secret profile configuration, and adapter auth channel; + - each MCP child receives only its own resolved secret references; and + - model-invoked shell/tools receive the workspace and no provider or MCP + credentials. + All views exclude gateway state, operator home, App keys, GitHub/OCI stores, + source helpers, unrelated adapter credentials, and the parent environment. + Fail readiness for a target when its adapter cannot enforce those separations. + Acquisition credentials and mounts are absent first. Treat the mutated + workspace as untrusted during evidence collection: use descriptor-relative + no-follow reads; reject hard links, special/sparse files, path replacement, + out-of-root targets, and `.git` gitdir/core.worktree/alternates escapes; and + run Git inspection with hermetic configuration that disables hooks, filters, + drivers, fsmonitor, pagers, helpers, and external commands. +- R16. Supervise the complete acquisition/provider descendant set inside an + invocation-owned OS containment primitive whose membership children cannot + escape. Fail readiness when the platform cannot enforce and inspect that + boundary. Cancellation persists intent with an atomic state transition before + native abort, then applies bounded forced termination. A terminal commit that + wins first makes later cancellation not cancelable; cancellation intent that + wins settles cancelled after quiescence. Terminal cleanup evidence requires + proof that the containment set is empty. Failure records termination + unknown/failed and rejects admission. An unmanaged foreground gateway remains + alive with poisoned readiness and continues reaping; it prints the stable + containment identifier and platform recovery command. It may exit with a + nonempty set only after a validated external manager accepts ownership. Startup + proves every interrupted set empty before it may quarantine stale roots or + advertise readiness. Graceful shutdown stops admission atomically, persists + shutdown/cancellation intent, drains or aborts active work within a bounded + grace period, proves quiescence, settles once, and only then exits. + +**Scope and configuration** + +- R17. Do not add evaluation commands, datasets, assertions, scoring, + repetitions, experiment scheduling, or automatic Task retry. +- R18. Do not add `gateway.yaml` or `worker.yaml`. Process configuration uses + the exact CLI flags and environment variables in the Configuration Contract + for listener, workspace, state/retention, GitHub, OCI, and Codex/Pi auth-file + handles. Secret values never enter workspace files, requests, logs, Tasks, + Artifacts, retained workspaces, or model-invoked tool environments. ### Key Flows -- F1. **Admit, create, and stream an execution** - - **Actors:** A1, A2, A3, A4. - - **Trigger:** A caller opts into `https://allagents.dev/a2a/extensions/coding-execution/v1` and sends a text Message whose `extensions` includes that URI and whose `metadata[uri]` contains the immutable source, profile, invocation key, deadline, and optional bounded result schema. - - **Steps:** Authenticate, check extension negotiation, and bounded-canonicalize the request; resolve an owner-scoped retained claim and return or conflict against its original request/profile/schema bindings before mutable admission checks. For a new claim only, validate current source/profile authorization, profile/readiness, quota, and deadline; atomically create the claim and submitted Task; dispatch a fenced worker attempt; accept the current fence and move the Task to working; materialize and verify source; execute the selected backend; validate structured output with the shared validator when requested; persist progress before emission; and terminalize with the required integrity Artifact plus the fixed-name structured-result Artifact only for a valid result after quiescence and cleanup. - - **Outcome:** `returnImmediately: true` returns the durable current Task, false/unset waits for terminal state, and streaming starts with that Task before ordered updates. - - **Covered by:** R1-R22. -- F2. **Replay or reconnect to an invocation** - - **Actors:** A1, A2. - - **Trigger:** The owner repeats an invocation key or subscribes after a stream disconnect. - - **Steps:** After authentication and bounded canonical parsing, resolve the owner-scoped claim; compare the request and schema digest with the stored originals and verify the retained Task's original effective-profile/schema bindings without resolving the current profile. Reject a mismatch; otherwise return the existing Task before current authorization, quota, readiness, profile, or deadline checks. For active streaming replay/subscription emit its current snapshot then future events; for a terminal Task return it through send replay or `GetTask` without dispatch. - - **Outcome:** Retries do not multiply agent work, and reconnect never promises transient event replay. - - **Covered by:** R4, R6-R8. -- F3. **Cancel or time out an execution** - - **Actors:** A1, A2, A3, A4. - - **Trigger:** The caller invokes `CancelTask`, the effective deadline expires, or gateway shutdown claims cancellation. - - **Steps:** Atomically record the first cancellation source; send one revisioned fenced worker cancel even when dispatch delivery is unconfirmed, so an unseen attempt is tombstoned before any delayed dispatch can create a workspace. If work exists, invoke native abort, terminate descendants, capture termination-safe evidence, clean, and publish canceled only after verification. - - **Outcome:** Completion that wins first remains terminal and later cancel returns `TaskNotCancelableError`; cancellation that wins suppresses stale dispatch and late provider success. If bounded escalation cannot prove the complete invocation process set empty, the Task fails rather than claiming canceled, the worker poisons admission and exits, and its supervisor destroys the boundary. - - **Covered by:** R7, R14, R16-R18. -- F4. **Settle after gateway or worker loss** - - **Actors:** A2, A3. - - **Trigger:** The gateway restarts with nonterminal Tasks, an acknowledgement is lost, a live worker loses its lease, or a worker process crashes. - - **Steps:** Invalidate the attempt fence and settle every affected Task failed once without provider-session reattachment or automatic replay. A live worker that loses its lease self-aborts and cleans. On worker-process crash or unproved quiescence after bounded escalation, poison admission and exit the worker so the external supervisor terminates the complete execution boundary; the replacement worker proves termination, then reaps or quarantines orphaned roots before readiness. Reject late events/results and record termination and filesystem cleanup separately as complete only when the responsible boundary proves each outcome. - - **Outcome:** One Task has one terminal result, interrupted work is never presented as resumed, no stale worker can overwrite durable truth, and a failed quiescence proof cannot leave the poisoned worker available for another reservation. - - **Covered by:** R7, R9, R14, R16-R18, R21-R22. -- F5. **Expire retained execution data** - - **Actors:** A1, A2. - - **Trigger:** The immutable Task expiry is reached. - - **Steps:** Atomically tombstone the complete ownership aggregate; stop authorizing Task and Artifact access; retry physical cleanup independently; permit the old invocation key to create a new Task only after logical expiry. - - **Outcome:** Expired, unknown, and unauthorized identifiers are indistinguishable and no Artifact outlives Task authorization. - - **Covered by:** R5-R9. +- F1. **Start and advertise** + 1. Resolve cwd or `--workspace`, user workspace, project-specific state root, + retention limits, listen address, source credentials, and provider auth + handles. + 2. Validate state-root ownership, permissions, links, disjointness, and + workspace identity; acquire the exclusive lock; validate repositories, + snapshots, target namespace, backend availability, profile state, + containment, separate provider/MCP/tool views, and credential handles. + 3. Reconcile interrupted Tasks and prove every stale containment set empty + before quarantining filesystem roots. + 4. Bind the requested address, including `0.0.0.0` when explicit, and publish + one Agent Card whose required extension and allowlisted targets match the + validated configuration. + +- F2. **Acquire repositories and execute** + 1. Negotiate the required extension and validate the strict request, one text + prompt, target, repository-name/revision map, result schema, deadline, and + deployment-wide idempotency claim. + 2. Durably commit the claim and Task before acknowledgment; create the + invocation containment and staging root. + 3. For each declared repository, classify App applicability, select App or + `gh` only by eligibility, resolve the revision, fetch hermetically, verify + the commit, and remove credentials. + 4. Publish the complete workspace, run typed preparation, invoke the isolated + adapter, validate any structured result, collect evidence through safe + reads, terminate descendants, clean up, and settle the Task once. + +- F3. **Acquire an OCI snapshot and execute** + 1. Resolve the named snapshot repository and digest-pinned reference. + 2. Authenticate if required, pull and verify the OCI manifest and layers, + extract safely, and validate the workspace-manifest digest. + 3. Remove registry credentials, publish atomically, run typed preparation, + invoke the isolated adapter, collect safe evidence, clean up, and settle. + +- F4. **Cancel** + 1. Atomically persist cancellation intent if the Task remains cancelable. + 2. Abort acquisition or provider work, escalate within the bounded termination + budget, prove containment quiescence, preserve partial evidence, clean up, + and settle cancelled. + 3. Repeated cancellation while intent is pending does not re-signal work. + Cancellation after any terminal state returns A2A + `TaskNotCancelableError`. + +- F5. **Shut down** + 1. Stop new admission before signaling active work. + 2. Persist shutdown cancellation intent, abort and escalate, drain evidence, + prove quiescence, and settle the accepted Task once. + 3. Exit only after durable settlement and empty containment. If proof fails, + unmanaged mode remains alive, not ready, and continues reaping while + printing the platform recovery command. Managed mode may exit only after + its validated external manager accepts containment ownership. ### Acceptance Examples -- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile, one valid - immutable workspace source, and an optional result schema, when the caller - streams a request, then one Task moves from submitted to working to completed - and later `GetTask` returns the same validated output, workspace provenance, - and evidence Artifacts. -- AE2. **Covers R6.** Given a retained Task whose original absolute deadline has passed or whose profile is now disabled, changed, or no longer authorized for new work, when its owner reuses the invocation key with the same canonical request and result schema, then the gateway returns the original Task from its stored original bindings before mutable admission checks and makes no second worker dispatch. -- AE3. **Covers R6.** Given a retained Task, when its owner reuses the invocation key with a different prompt, source, profile ID, deadline, or result schema, or the stored original profile/schema binding is inconsistent, then the gateway rejects the request and leaves the original Task unchanged. -- AE4. **Covers R5.** Given a Task owned by caller A, when caller B lists Tasks, gets the Task, cancels it, subscribes, or requests an Artifact, then the gateway reveals no resource existence or content. -- AE5. **Covers R12, R16-R18.** Given a wrong or missing Git commit, - conflicting repository destination, OCI digest or manifest mismatch, unknown - or profile-disallowed materializer, materializer definition/image drift, - produced workspace-manifest digest that differs from the request, malformed - materializer output, direct known-secret disclosure, or setup failure, when - the worker has already accepted the current fence, then provider execution - never starts, the selected public trace is - `Submitted -> Working -> Failed`, and the Task retains bounded - materialization/setup-failure and cleanup evidence. -- AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. Given cancel reaches a worker before its delayed dispatch, the worker tombstones the unseen attempt and the stale dispatch creates no workspace or provider process. -- AE7. **Covers R3, R10-R11, R17.** Given equivalent profiles, one accepted `allagents.result-schema/v1` schema, and fixture runtime events for Codex and Pi, when each completes the same repository mutation, then both publish the required fixed-name integrity Artifact at the schema-defined extension carrier, validate with the same schema and validator, publish the same fixed-name structured-result Artifact containing one A2A `Part` with the validated `data` and `mediaType: application/json`, record the same integrity state, and produce the required normalized evidence fields while retaining distinct native evidence. -- AE8. **Covers R4, R7, R19.** Given canary secrets and cross-owner content fragments in prompts, model output, tool arguments/results, source files, stale events, and errors, when agent, model, tool, stale-event, and error telemetry is processed, then the exporter receives only allowlisted bounded metadata plus the correct opaque owner correlation and receives none of those canaries, fragments, or raw caller identities. Given a caller or exporter disconnects during work, reconnect still returns the current Task and future updates without duplicate dispatch, and telemetry loss does not affect terminal lookup. -- AE9. **Covers R15.** Given a known capability denied by profile, the accepted Task becomes rejected after stop and cleanup; given an unknown permission type, it becomes failed as an adapter incompatibility without waiting for a client. -- AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. Given output validation has already selected `valid` or `invalid` and a later check or mandatory-evidence phase fails, the failed Task preserves that result state and a valid result preserves its one fixed structured-result Artifact; only a failure before candidate production records `not_produced`. -- AE11. **Covers R6, R11-R12, R22.** Given invalid input, an unknown or - profile-disallowed source mode/materializer, or exhausted admission quota, - the gateway returns a request/resource error and creates no Task; given - materializer availability or worker capacity disappears after durable - acceptance, the retained Task fails at dispatch or materialization and replay - returns it without retry. -- AE12. **Covers R7, R14, R16.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only allowlisted metadata-only operator telemetry. Given bounded escalation cannot stop a descendant that starts a new session and ignores graceful signals, the worker persists termination unknown/failed, refuses another reservation, exits, and its supervisor destroys the boundary; replacement readiness performs orphan recovery without changing the failed Task. -- AE13. **Covers R8.** Given a Task reaches expiry while physical deletion fails, all Task and Artifact operations return the same not-found response and the invocation key can create a new Task. -- AE14. **Covers R5, R13, R21-R22.** Given a production public listener or - remote worker route lacks its configured trusted transport or authenticated - peer identity, readiness fails; a same-host Unix worker socket is accepted. - Given a credentialed reviewed-domain profile, model tools cannot inspect - provider process environments, process listings, backend config/data roots, - or exfiltrate provider/control credentials across the configured OS boundary. - Given a GitHub repository, a trusted operator repository-to-installation - mapping wins over GitHub CLI; auth-app never discovers the installation. - Without a mapping, only an explicitly enabled trusted-local provider may - invoke the configured account through - `gh auth token --hostname --user ` with all four ambient - GitHub token variables absent. Any selected-App failure never invokes `gh`. - A remote worker requests a credential only by its active attempt/fence; the - controller derives all provider/repository/route bindings, rechecks current - command state, consumes one single-use grant, and rejects replay, - substitution, stale state, or minter configuration-digest disagreement. - The fresh token has repository/read-only/expiry scope only; the authenticated - lease carries worker, attempt, lease epoch, command revision, fence, - operation, and expiry bindings. Near-expiry cached auth-app output is bypassed - with `refresh: true`, lease expiry never exceeds token expiry, and an unsafe - acquisition ceiling fails readiness without CLI fallback. Authenticated App - lifecycle webhooks and bounded reconciliation invalidate old entitlement - generations; unknown or stale state cannot authorize a cache hit. Cache-hit - provenance distinguishes the original acquisition provider from current - policy selection. Every source-auth failure maps to the specified safe - code/reason/retryability, while provider, installation, and account identities - remain operator-only. Given a registered materializer with source credentials, - setup, provider processes, and model tools have no access to its process, - runner control socket, credential environment/mounts, or staging root after - materialization, and direct known-secret canaries are absent from retained - logs, evidence, and published workspace files. The registered materializer - remains operator-trusted code; hostile-materializer, hostile-source, and - cross-tenant claims remain rejected without a stronger broker or sandbox. +- AE1. A caller on a permitted Tailscale or firewalled network discovers the + gateway bound to `0.0.0.0`, selects `codex-review`, and receives one durable + Task without presenting an application credential. +- AE2. Any reachable caller can list, retrieve, cancel, and fetch Artifacts for + a Task created by another reachable caller; documentation states this shared + trust model without implying tenant privacy. +- AE3. A launcher-bearing Codex profile without `gateway.expose: true` is absent + from discovery and rejected when selected. An exposed but drifted profile + fails readiness/new admission. +- AE4. A multi-client profile exposes `codex-review` and `pi-review` as distinct + targets. Both resolve through adapters; neither generated wrapper is executed. +- AE5. Repository mode accepts declared names and revision overrides, rejects an + undeclared name or URL override, and records the resolved full commits. +- AE6. An applicable GitHub App mints a fresh repository-scoped token whose + lifetime exceeds the acquisition sub-budget plus skew. A repository with no + applicable installation uses the configured `gh` account. Unknown App + applicability, auth, or mint failure does not fall through to `gh`. +- AE7. Snapshot mode accepts a declared snapshot name and matching OCI/workspace + digests, rejects mutable tags, traversal, foreign layers, digest mismatch, or + undeclared registry repositories, and publishes only after full validation. +- AE8. Repository and snapshot modes produce the same workspace-manifest shape, + while OCI-contained commit identities remain marked snapshot-attested unless + independently verified. +- AE9. Identical invocation-key replay returns the original Task. Reusing the key + with a changed target, source, prompt, or result schema conflicts. +- AE10. Cancellation during Git, OCI pull, Codex, or Pi terminates the complete + process set and records cleanup. Unproved quiescence poisons readiness; an + unmanaged foreground process stays alive and reaps, while managed exit + requires accepted external cleanup ownership. +- AE11. Restart turns interrupted Tasks into one terminal failure and never + resumes a provider session. Terminal Tasks and Artifacts remain retrievable + until expiry. +- AE12. A valid structured result survives later check or evidence failure as a + valid result with an overall failed Task; invalid or absent results are never + published as valid. +- AE13. An exposed launcher named `codex`, `pi`, or a portable case-equivalent + fails configuration compilation instead of shadowing a built-in target. +- AE14. Two gateways for different workspaces use distinct private state roots; + a second process for the same root fails the exclusive lock. Wrong-owner, + permissive, linked, hard-linked, or overlapping roots fail startup. Store + fault injection cannot acknowledge an uncommitted Task or false success. +- AE15. Deadline expiry during Git, OCI, preparation, Codex, Pi, or evidence + initiates one abort/termination path and retains truthful partial evidence. +- AE16. Repeated cancel while cancellation is pending is idempotent; cancel + after cancelled, completed, failed, or rejected returns + `TaskNotCancelableError`. +- AE17. A workspace containing `setup` shell entries never executes them through + gateway acquisition or startup. Built-in Codex/Pi authenticate through their + selected private control-process auth views; model-invoked tools cannot read + provider or MCP secrets, operator stores, or gateway state. +- AE18. Evidence collection rejects a provider-created escaping link, hard link, + special file, sparse-file abuse, or `.git` indirection and runs Git inspection + without repository-controlled execution hooks. +- AE19. The 1001st unexpired retained Task is rejected with + `retention_capacity_exhausted`; no retained Task is evicted before TTL. +- AE20. Unrelated Message metadata survives request processing. Every profiled + A2A operation requires activation, and a terminal Task may contain the single + integrity Artifact plus referenced produced Artifacts. ### Success Criteria -- The official A2A JavaScript client can discover the required extension, negotiate it through `A2A-Extensions`, use the standard Message and Artifact extension carriers, and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, retained replay, cancel, and expiry behavior against the built service without `Task.extensions`. -- One conformance fixture passes unchanged through the Codex and Pi adapters. -- Admission, retained replay, monotonic worker commands, fencing, acceptance-before-materialization source/setup failure, cancellation races, trace-order/fence/multiplicity constraints, failed-quiescence recycling, restart terminalization without resume, supervised worker-crash cleanup, trusted transports, authorization isolation, metadata-only telemetry export, OS-enforced provider/tool credential separation, source hardening, portable structured-result validation, quotas, and evidence integrity have deterministic integration coverage. -- Direct multi-repository Git, digest-pinned OCI snapshots, and a fake - digest-pinned registered materializer all produce the same validated - workspace manifest and terminal provenance contract before either backend - starts. -- The gateway image contains no coding-agent runtime and cannot access worker workspace roots. -- The initial worker runs one reviewed-trust-domain execution at a time, model-initiated tools are OS-isolated from provider/control credentials, repository Pi extensions cannot auto-load, and no live descendant or reusable workspace survives a completed, failed-quiescence, or crashed attempt. +- `allagents gateway serve` starts from a real workspace with no deployment YAML. +- Explicit loopback, private-interface, and `0.0.0.0` listeners work. +- The official A2A client exercises required-extension negotiation, send, + stream, get, list, subscribe, replay, cancel, terminal cancel errors, Artifact + retrieval, and expiry. +- Built-in Codex/Pi and exposed profile targets pass one conformance suite, + including reserved-ID collisions. +- Direct Git and OCI snapshot fixtures produce equivalent validated workspace + manifests and truthful provenance. +- GitHub App eligibility, unknown failure, no-installation `gh` fallback, + selected-App failure, token lifetime, containment, and OCI credential cleanup + are proven end to end. +- No request can supply a command, executable, URL, destination, credential, + mutable OCI tag, backend override, or arbitrary environment value. +- State-store fault, deadline, cancellation-race, shutdown, descendant escape, + unsafe evidence, and stale-root scenarios fail closed. +- The built CLI passes a trusted-network smoke test against project and user + workspaces created under `/tmp/`. ### Scope Boundaries **In scope** -- A2A 1.0 HTTP+JSON and SSE streaming. -- One versioned AllAgents coding-execution extension and one versioned private worker protocol. -- Codex and Pi backends. -- Built-in bearer authentication with OIDC/JWT and static service-token modes behind the named production TLS boundary. -- Single-replica durable file storage, authenticated Artifact retrieval, authenticated encrypted remote worker transport or same-host Unix sockets, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. -- Reviewed repositories in one configured mutual-trust domain per worker deployment, with the narrow OS-enforced provider/tool credential boundary required for credentialed profiles. -- Direct multi-repository Git acquisition, digest-pinned OCI workspace - snapshots, and operator-registered digest-pinned materializers with - phase-scoped credentials and one standard workspace manifest. - -**Deferred to follow-up work** - -- Multi-replica database-backed Task and idempotency storage. -- Durable provider execution, checkpointing, provider-session restoration, and automatic replay after gateway or worker restart. -- OpenCode and additional coding backends. -- Kubernetes Job dispatch, queue brokers, autoscaling controllers, and stronger hostile-source or cross-tenant sandbox providers. -- Push-notification configuration, gRPC, JSON-RPC transport, and A2A extended Agent Cards. -- AHP server/client surfaces, long-lived interactive sessions, and client-contributed tools. -- Optional ATIF conversion after the format and tooling mature. -- Versioned central source-snapshot acquisition and delivery; the initial - remote GitHub path is central token minting plus an authenticated non-durable - delivery lease. - -**Outside this product's identity** - -- Evaluation authoring, datasets, assertions, grading, repetitions, experiment scheduling, and durable evaluation Runs. -- Caller-specific result projections such as Promptfoo `ProviderResponse` mapping. +- A2A 1.0 HTTP+JSON and the required AllAgents extension. +- One process and one active invocation at a time initially. +- Built-in and exposed profile-backed Codex/Pi targets. +- Direct declared Git repositories and named OCI workspace snapshots. +- GitHub App and configured GitHub CLI acquisition credentials. +- Local durable Task/evidence storage, cancellation, cleanup, and provenance. +- Listen addresses including `0.0.0.0`. + +**Out of scope** + +- Application authentication, tenant isolation, caller-private Tasks, and public + Internet hardening. +- `gateway.yaml`, `worker.yaml`, remote workers, mTLS worker links, Kubernetes + routing, autoscaling, and multiple gateway replicas. +- Caller-provided repository or registry origins, mutable OCI tags, custom + materializers, Dockerfiles, Compose files, or acquisition commands. +- GitHub Enterprise Server and multiple ordered Apps/accounts in the initial + delivery. +- OpenCode, Claude, Copilot, OMP, arbitrary CLI, and TUI adapters. +- Evaluation orchestration and automatic retries. ### Sources - [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) - [AHP decision inputs](../research/agent-host-protocol-decision-inputs.md) - [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) -- [GitHub source credential broker precedents](../research/source-credential-broker-precedents.md) -- [GitHub App installation access tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) -- [`@octokit/auth-app`](https://github.com/octokit/auth-app.js) -- [AI Evals ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) -- [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) -- [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) -- [Codex TypeScript SDK](https://github.com/openai/codex/tree/main/sdk/typescript) -- [Codex configuration reference](https://developers.openai.com/codex/config-reference) -- [Promptfoo Codex provider documentation](https://github.com/promptfoo/promptfoo/blob/main/site/docs/providers/openai-codex-sdk.md) -- [Promptfoo Codex provider implementation](https://github.com/promptfoo/promptfoo/blob/main/src/providers/openai/codex-sdk.ts) -- [Promptfoo Codex provider tests](https://github.com/promptfoo/promptfoo/blob/main/test/providers/openai-codex-sdk.test.ts) -- [Pi RPC protocol](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/rpc.md) -- [Pi CLI reference](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/README.md#cli-reference) -- [Pi extension API](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/extensions.md) -- [Pi provider credentials](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/providers.md) -- [Buzz pure Kubernetes state classifier](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-backend-kubernetes/src/classify.rs) -- [Buzz non-secret intent fingerprint](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-backend-kubernetes/src/intent.rs) -- [Buzz conformance coverage checker](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-conformance/src/checker.rs) -- [Buzz bounded process-tree cancellation](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-dev-mcp/src/shell.rs) +- [Source credential broker precedents](../research/source-credential-broker-precedents.md) +- [A2A 1.0 specification](https://a2a-protocol.org/latest/specification/) +- [OpenAI Codex SDK](https://developers.openai.com/codex/sdk/) +- [OpenAI Codex app-server](https://developers.openai.com/codex/app-server/) +- [GitHub App installation tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) +- [Git credential helpers](https://git-scm.com/docs/gitcredentials) +- [OCI Image Specification](https://github.com/opencontainers/image-spec) --- @@ -412,1119 +563,447 @@ The two initial runtimes expose different programmatic contracts. Codex provides ### Key Technical Decisions -- KTD1. **Use the official A2A JavaScript SDK behind an AllAgents request-handler decorator.** Pin a compatible A2A 1.x SDK. After authentication, required-extension checks, and bounded canonical parsing, the decorator resolves an owner-scoped retained claim before mutable admission; identical replay bypasses current profile/deadline/quota/readiness checks and any new SDK Task/bus allocation. New requests then pass mutable admission and canonical Task reservation. The decorator also owns stream snapshot selection and cancellation routing before `DefaultRequestHandler` can allocate another Task or terminalize cancellation prematurely; the SDK retains standard transport/event mechanics. Governs R1-R8, R14. -- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes `https://allagents.dev/a2a/extensions/coding-execution/v1`, its standard Agent Card/header/Message/Artifact negotiation, `Message.metadata[uri]` request location, and the single-Part `allagents.execution-integrity` Artifact data location; no schema or implementation adds `Task.extensions`. The public contract also carries the `allagents.result-schema/v1` closed subset, its canonical digest, four structured-result states, and the separate fixed `allagents.structured-result` Artifact. The worker protocol carries worker identity, attempt identity, profile digest, monotonic command revision and tombstone state, dispatch acceptance, event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. -- KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates a new invocation claim and submitted Task together after mutable admission, storing the canonical caller request and the original effective-profile and result-schema digests needed for retained replay. It stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. -- KTD4. **Authenticate at a named trusted HTTP ingress before A2A storage or dispatch.** Production traffic reaches the gateway through TLS terminated by the configured gateway or named trusted reverse-proxy boundary; plaintext is allowed only for an unauthenticated loopback development listener. Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Readiness rejects a production public URL whose trusted TLS boundary is absent or inconsistent. Governs R5-R6, R13. -- KTD5. **Use fenced, separately deployable gateway and worker services.** Remote gateway-worker routes use mTLS or an explicitly equivalent authenticated encrypted overlay; a same-host Unix socket is acceptable. The authenticated worker identity is pinned to the configured route/capability set, and every short-lived attempt capability is bound to that identity, attempt ID, lease ID/epoch, and fence. Each worker keeps one minimal durable monotonic command record scoped to its worker identity and lease: `Cancel(attempt, fence, revision)` tombstones even an unseen attempt, and `Dispatch` for a tombstoned or lower-revision attempt is rejected before workspace creation. Dispatch/cancel I/O conditionally verifies the persisted command/outbox revision immediately before any mutating or terminating effect. Duplicate delivery is idempotent; conflicting, stale, out-of-order, or identity-mismatched commands/events are rejected. Gateway and worker transition selectors remain pure and executors re-enter from persisted or freshly observed state. This record is worker-local fence state, not a new durable execution subsystem. Governs R5, R7, R10-R16, R21-R22. -- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Every external materializer launch creates a supervisor-owned runner resource labeled by worker, attempt, lease, and fence; worker-process exit makes the external supervisor terminate that resource and the complete execution boundary. If bounded escalation cannot prove the complete invocation process set empty, the worker records termination unknown/failed, poisons admission, and exits rather than accepting another reservation; its supervisor destroys the boundary. Before readiness the replacement proves termination and enumerates, destroys, or quarantines orphaned invocation roots, runner resources, credential mounts, and staging mounts; an unresolved resource keeps readiness false. Production readiness accepts a dedicated worker container process namespace under a minimal init/reaper as the baseline; a non-container deployment must prove an equivalent systemd/cgroup boundary. The gateway never reattaches to or resumes a provider session. Governs R7, R14, R16-R18. -- KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, optional structured result, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, schema validation, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | pi` registry is the only production dispatch point. Governs R10-R11, R14-R18, R21-R22. -- KTD8. **Use each provider's supported automation surface directly behind the credential boundary.** Codex depends directly on pinned `@openai/codex-sdk`, creates one fresh thread per Task, passes `AbortSignal` and optional per-turn `outputSchema`, and consumes streamed events. Pi uses strict RPC with an invocation-local credential store and one explicitly loaded worker-owned policy extension; repository extensions and unrestricted built-ins never load. For either adapter, a credentialed provider runtime is separated from every model-invoked tool by the R13 OS-enforced UID/process/mount boundary or an equivalent credential broker; shell-environment filtering is defense in depth, not the boundary. Promptfoo's Codex provider and tests are characterization references only; AllAgents neither vendors them nor inherits their config, cache, pricing, retry, thread-pool, or `ProviderResponse` concerns. Governs R10-R18. -- KTD9. **Make profiles the new-admission policy boundary.** Requests select a - profile ID, one schema-defined workspace source mode, and optionally one - `allagents.result-schema/v1` schema. They cannot override backend or source - credentials, materializer definitions or images, executable paths, provider - config, setup/check commands, environment allowlists, permission rules, - trust class, resource limits, workspace retention, or evidence budgets. A - profile allowlists source modes and materializer IDs plus exact canonical Git - repository/namespace rules, OCI namespaces/signature rules, and resource - selectors for structured materializer inputs. New admission authorizes the - fully canonicalized resource and credential entitlement before cache lookup. - Resolve a versioned canonical `EffectiveProfileIntent` containing the - selected materializer definition digest, authorization-scope digest, - source-authorization revocation epoch, provider policy and pinned CLI account, - and current GitHub App entitlement generation when applicable; compute its - digest without resolved secrets or per-attempt state and persist it with the - canonical caller request and result-schema digest. Unknown or stale App - entitlement state fails cache authorization. Cache metadata retains original - acquisition-provider metadata, while cache-hit provenance records `cache_hit` - plus current policy selection separately. Retained replay compares stored - original bindings and never substitutes or re-resolves current policy. - Governs R6, R11-R16, R21-R22. -- KTD10. **Keep durable evidence and operational telemetry as separate bounded layers.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded evidence layer; neither Git nor provider evidence is promoted as exact causality when incomplete. Telemetry is a third, non-durable metadata-only channel: one small shared pre-export sanitizer applies an explicit operational-metadata allowlist plus bounded filtering/redaction before every structured log or span processor, and only opaque owner correlation may cross the separately governed operator boundary. OpenInference and backend-native attributes receive no bypass. This is an export guard, not a telemetry framework or alternate evidence store. Governs R13, R16-R19. -- KTD11. **Treat Codex and Pi as the complete initial backend set.** Codex lands first; Pi lands second against the established contract; OpenCode is deferred. (session-settled: user-directed.) Governs R10. -- KTD12. **Separate terminal integrity from optional evidence bodies.** The fixed `allagents.execution-integrity` Artifact validates identity, action outcome, the four-state structured-result record, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance before terminal publication. `not_produced` applies only before result-candidate production. Once validation selects `valid` or `invalid`, a later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the separate fixed structured-result Artifact while retaining the later phase as the primary Task failure. Predictable optional-body truncation/redaction may preserve completion; failure that breaks the integrity kernel fails in the evidence phase. Governs R3-R4, R17-R18. -- KTD13. **Standardize and harden workspace materialization.** Define one - closed `kind`-discriminated workspace-source union and one output manifest; - reject unknown kinds and cross-variant fields. The built-in Git path accepts - canonical HTTPS repository identities and full commit IDs only, uses - hermetic Git configuration; disables inherited redirects, proxies, - credential helpers, hooks, filters, LFS smudge, submodule recursion, - alternates, and non-HTTPS protocols; injects only the KTD16-selected - one-shot credential channel; revalidates normalized host/address policy for - every connection; fetches into an isolated object database from the - authorized remote; and verifies the checked-out commit and resulting tree. - The OCI path accepts - manifest digests, not tags; rejects foreign/external URLs by default; - revalidates scheme, host, resolved address, port, redirect, and credential - origin for every registry/auth/manifest/blob request; and verifies every - manifest/layer plus the embedded workspace manifest. The custom path accepts - a registered ID, expected workspace-manifest digest, and schema-validated, - resource-authorized inputs. - - The operator-owned registry splits a non-secret gateway descriptor from the - worker-only runtime definition. The worker computes a - `sha256:<64 lowercase hex>` digest over the versioned, domain-separated - canonical non-secret runtime definition; readiness compares that value with - the gateway's expected digest and capabilities. Distinct domain-separated - canonical JSON preimages define materializer input and workspace-manifest - digests. All paths stage in a worker-owned directory on the final - publication filesystem, validate destinations, links, file types, bounds, - identities, content, and manifest, then terminate the supervisor-owned - acquisition process/mount/credential/runner boundary while retaining the - validated host-owned tree. Only after proving the boundary gone does the - worker atomically rename the tree; no copy fallback exists. Provenance - distinguishes worker-verified observations, trusted-service verification, - and materializer-attested claims. The caller digest covers source kind, - expected output identity, and inputs; the effective-profile digest covers - materializer and authorization bindings; terminal provenance covers both and - the validated output. Governs R6, R11-R13, R16-R18, R21-R22. -- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Credentialed profiles still require R13's narrower OS-enforced provider/tool separation: model tools cannot inspect provider processes, procfs entries, or backend config/data roots, and readiness fails without that capability. Provider/source credentials are absent from setup/check phases and child-visible worker control state. Pi disables repository extensions and built-in tools; only the worker-owned policy extension may load. This credential boundary does not imply hostile-source or cross-tenant isolation; that stronger sandbox-driver capability remains deferred. Governs R13, R16, R21-R22. - An external materializer image is reviewed operator code in the deployment's - trusted computing base, not hostile caller code. Its runner or sandbox - control plane is never mounted into the workspace or exposed to setup, - providers, or model tools. A deployment that does not trust the registered - image with source credentials is outside the initial trust model and must not - enable that registered materializer. Support requires the separately - versioned broker or central snapshot-delivery protocol deferred by R13. -- KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, Codex SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. -- KTD16. **Resolve GitHub credentials through an authoritative ordered provider - registry and lease controller.** The caller supplies only a canonical - credential-free repository URL. The acquisition boundary maps `github.com` - to the built-in GitHub backend and requires explicit host/API mappings for - GitHub Enterprise Server. The profile supplies provider eligibility and - order, not secrets. A configured App provider is applicable only when trusted - operator configuration maps the authorized repository to an installation ID; - auth-app does not discover installations. A `github-cli` provider may follow - only in a trusted-local profile, only when no App mapping applies, and only - for its configured non-secret account. That account participates in - entitlement and effective-profile digests. Invoke - `gh auth token --hostname --user ` with `GH_TOKEN`, - `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` removed, - and fail when the configured account cannot be resolved. Selection is sticky: - App configuration, authentication, minting, authorization, rate-limit, or - service failure never falls through to the broader user identity. - - When AllAgents owns minting, its trusted central minter depends on focused - `@octokit/auth-app` rather than implementing App JWT, clock-skew, expiry, and - renewal; Git remains the transport and the full Octokit client is not added. - Every cache-miss acquisition uses `refresh: true` and accepts only a fresh - token whose remaining lifetime is strictly greater than the acquisition - deadline plus clock-skew margin. The token is scoped only to the repository, - read-only contents permission, and GitHub expiry. Lease expiry cannot exceed - token expiry, and readiness rejects an acquisition ceiling that can exceed a - fresh token's safe lifetime. - - The gateway/control-plane credential-lease controller, not the worker, is - authoritative. An authenticated worker request supplies only active attempt - and fence. The controller rechecks durable command revision, lease epoch, - tombstone, and fence and derives effective-profile digest, selected provider, - host/API-mapping digest, installation ID, canonical repository, operation, - worker route/identity, and expiry from durable dispatch and policy state. It - issues a single-use non-durable grant/response; a separate minter atomically - consumes the grant and must agree with the selected configuration digest. - Replay, substitution, stale command state, and digest disagreement fail - closed. The authenticated lease/channel binds the derived state to worker - identity, attempt, lease epoch, command revision, fence, operation, and - expiry. A remote worker never receives the App private key. Remote App - profiles fail readiness without that complete central path. A trusted - co-located deployment may keep the controller and minter in its control - plane. Versioned central snapshot delivery is deferred. - - Authenticated App lifecycle webhooks and bounded reconciliation advance an - installation-entitlement generation on uninstall, suspension, or - repository-selection change; unknown or stale state fails cache - authorization. Minting remains miss-only. Provenance distinguishes - `cache_hit`, original acquisition-provider metadata, and current policy - selection/entitlement. Public source-auth details contain only the specified - safe code, reason, and retryability; provider, installation, and account - identifiers are operator-only. Governs R6, R12-R13, R16, R21-R22. +- KTD1. **Use the official A2A JavaScript SDK behind a small AllAgents request + decorator.** The decorator validates the required extension and canonical + request, resolves the deployment-wide retained claim, performs new-admission + checks, and preserves standard Task/Artifact carriers. +- KTD2. **Generate public, Task-store, workspace-manifest, and adapter contracts + from canonical Zod schemas.** Keep the result-schema subset shared across + Codex and Pi and forbid backend-specific public fields. +- KTD3. **Use a single-process supervisor, not a remote worker protocol.** One + service owns Task state, staging, publication, backend child processes, + evidence, termination, and cleanup. Child processes remain contained behind + an invocation lifecycle boundary. +- KTD4. **Make application authentication intentionally absent.** All Tasks and + Artifacts share one deployment namespace. The listener accepts explicit + `0.0.0.0`; network controls are external. (session-settled: user-directed.) +- KTD5. **Compile configuration from existing workspace files.** Add + `workspaceSnapshots` to the project schema and `gateway.expose` to strict + profile-client schemas. Resolve the public launcher ID to one profile/client. + Add no deployment YAML. (session-settled: user-directed.) +- KTD6. **Keep source input name-based and closed.** Repository requests carry + only declared-name revisions; snapshot requests carry only a declared snapshot + name and immutable digests. Compute one canonical source identity for + idempotency and provenance. +- KTD7. **Use direct acquisition implementations.** Git runs with hermetic config + and an invocation credential helper. OCI pulls through a library or fixed + non-shell client interface that validates registry redirects and digests and + extracts without trusting archive paths. +- KTD8. **Select GitHub credentials by three-way eligibility.** Discover + repository coverage with an App-authenticated GitHub API client or verify an + explicit installation ID. Only positive `ineligible` permits the configured + `gh` account; `unknown` and selected-provider failure are terminal. + (session-settled: user-directed.) +- KTD9. **Keep one behavior-focused `codex | pi` adapter registry.** Direct + targets and exposed profile targets resolve to the same adapter types and + conformance tests; profile context modifies server-owned configuration, never + the public command line. Provider control, each MCP child, and model-invoked + tools receive separate secret scopes and filesystem/environment views. +- KTD10. **Store one immutable terminal Task generation.** A private, + project-specific locked local store supports one process, durable atomic + idempotency claim plus Task creation, monotonic status, bounded + events/Artifacts, no eviction before TTL, atomic expiry, and startup + terminalization. State paths are ownership/mode/link/disjointness checked. + Integrity or durability failure stops admission and prevents false success. +- KTD11. **Use enforceable invocation containment.** The platform implementation + owns a non-escapable descendant set, drains stdout/stderr, and covers + credential helpers, Git/OCI, MCP, and provider processes. Failure to inspect + or prove an empty set fails or poisons readiness. An unmanaged gateway remains + alive to reap and expose recovery instructions; managed exit requires an + external manager that already accepted containment ownership. +- KTD12. **Keep durable evidence separate and collect it defensively.** Evidence + retains bounded source, Git, provider, result, artifact, and cleanup facts. + Post-execution workspace reads are descriptor-relative and no-follow; Git + metadata indirections and repository-controlled execution are rejected. + Structured logs remain metadata-only and never retain secrets or unrestricted + request/output/file bodies. ### High-Level Technical Design -#### Component topology - ```mermaid flowchart TB - Caller[Authenticated A2A caller] -->|TLS at named trusted ingress| Gateway[execution-service gateway] - Gateway --> Auth[Auth, retained replay, new admission] - Gateway --> Store[Generation-based Task and Artifact store] - Gateway -->|mTLS/authenticated overlay or same-host Unix socket| Worker[Single-execution worker] - Gateway --> LeaseController[Authoritative credential lease controller] - LeaseController --> AppMinter[Trusted GitHub App token minter] - Worker --> Materialization[Workspace materializer registry] - Materialization --> Git[Hardened multi-repository Git] - Git --> SourceCredentials[Source credential client] - SourceCredentials -->|Active attempt and fence| LeaseController - SourceCredentials --> GitHubCLI[Account-pinned trusted-local gh helper] - Materialization --> OCI[Digest-pinned OCI snapshot] - Materialization --> Custom[Registered materializer image] - Worker --> Registry[Closed backend registry] - Registry --> Codex[Codex SDK] - Registry --> Pi[Pi RPC process] - Worker --> Evidence[Quiesced checks, Git and native evidence] - Evidence -->|Bounded terminal result| Gateway - Gateway --> Telemetry[OpenTelemetry exporter] - Worker --> Telemetry -``` - -#### Admission, dispatch, and settlement sequence - -```mermaid -sequenceDiagram - participant C as Caller - participant G as Gateway decorator - participant S as Durable aggregate store - participant W as Worker - participant B as Backend adapter - participant L as Credential lease controller - participant M as App token minter - - C->>G: SendMessage + header/Message extension + metadata[uri] - G->>G: Authenticate, check extension, canonicalize within bounds - G->>S: Resolve owner-scoped invocation claim - alt retained identical replay - S-->>G: Existing Task + original request/profile/schema bindings - G-->>C: Existing Task before current admission checks - else conflicting retained claim - G-->>C: Conflict; existing Task unchanged - else no retained claim - G->>G: Current authorization, profile/readiness, quota, deadline - G->>S: Atomic new claim + submitted Task + original digests - S-->>G: Task + attempt/lease fence - G->>W: Dispatch(attempt, fence, command revision) - W->>W: Verify command record before workspace creation - W-->>G: Accepted(attempt, fence) - opt private GitHub cache miss - W->>L: Request(active attempt, fence) - L->>S: Recheck command revision, lease epoch, tombstone, fence - L->>L: Derive profile/provider/mapping/install/repository/operation/route - L->>M: Single-use non-durable grant + configuration digest - M-->>L: Fresh repository/read-only token + GitHub expiry - L-->>W: Authenticated lease response bound to current command - end - W->>W: Materialize into staging and validate workspace manifest - W->>W: Destroy acquisition boundary, publish atomically, setup, baseline - W->>B: Invoke with isolated roots and credential boundary - B-->>W: Progress, usage, native evidence - W-->>G: Sequenced fenced progress - G->>S: Compare-and-swap Task generation - opt cancellation or deadline wins - C->>G: CancelTask - G->>S: Persist cancellation intent once - G->>W: Cancel(attempt, fence, newer command revision) - W->>W: Persist tombstone before effects - W->>B: Native abort - end - W->>W: Stop descendants, capture evidence, cleanup - W-->>G: Fenced terminal result - G->>S: Store blobs then atomically commit terminal manifest - G-->>C: Terminal status and extension Artifacts - end -``` - -#### Public A2A Task state - -```mermaid -stateDiagram-v2 - [*] --> Submitted: claim and Task committed - Submitted --> Working: worker accepts current fence - Submitted --> Canceled: cancellation proves no workspace exists - Submitted --> Failed: dispatch or restart failure - Submitted --> Rejected: accepted policy refusal before work - Working --> Completed: integrity kernel and cleanup validate - Working --> Failed: source, setup, provider, check, evidence, cleanup, crash, or restart failure - Working --> Rejected: known profile permission denial after stop and cleanup - Working --> Canceled: cancellation wins and stop/cleanup verify - Completed --> [*] - Failed --> [*] - Rejected --> [*] - Canceled --> [*] + C[Trusted-network A2A caller] --> G[Gateway server] + G --> S[Local Task store] + G --> W[Workspace compiler] + W --> PW[Project workspace.yaml] + W --> UW[User workspace.yaml] + G --> A[Acquisition supervisor] + A --> Git[Declared Git repositories] + A --> OCI[Named OCI snapshot] + A --> P[Atomically published invocation workspace] + G --> R[Closed adapter registry] + R --> Codex[Codex SDK] + R --> Pi[Pi RPC] + Codex --> E[Evidence and cleanup] + Pi --> E + E --> S ``` -Terminal states are immutable. Cancellation intent, termination, evidence capture, cleanup, and retention expiry are private record phases, not A2A Task states. - -#### Private execution-record phases +### Configuration Contract -```mermaid -stateDiagram-v2 - [*] --> Admitted - Admitted --> Dispatching - Dispatching --> Running: current command revision accepted - Dispatching --> Terminalizing: dispatch rejected, tombstoned, or unknown - Running --> CancelRequested: caller, deadline, shutdown, or lease expiry - Running --> Quiescing: provider and checks finish - CancelRequested --> Quiescing - Quiescing --> CapturingEvidence: complete process set verified empty - Quiescing --> Poisoned: bounded escalation cannot prove empty - Poisoned --> [*]: persist unknown/failed and exit boundary - CapturingEvidence --> Cleaning - Cleaning --> Terminalizing - Terminalizing --> Retained - Retained --> Tombstoned: expiry - Tombstoned --> [*]: physical cleanup +No `gateway.yaml` or `worker.yaml` is introduced. + +**CLI flags and environment** + +| Concern | CLI | Environment | Default | +|---|---|---|---| +| Listener | `--listen` | `ALLAGENTS_GATEWAY_LISTEN` | `127.0.0.1:4732` | +| Project workspace | `--workspace` | `ALLAGENTS_GATEWAY_WORKSPACE` | cwd | +| State directory | `--state-dir` | `ALLAGENTS_GATEWAY_STATE_DIR` | `~/.allagents/gateway/` | +| Terminal Task TTL | `--task-ttl` | `ALLAGENTS_GATEWAY_TASK_TTL` | `24h` | +| Retained Task limit | `--max-retained-tasks` | `ALLAGENTS_GATEWAY_MAX_RETAINED_TASKS` | `1000` | +| Per-Task retained bytes | `--max-task-bytes` | `ALLAGENTS_GATEWAY_MAX_TASK_BYTES` | `64MiB` | +| GitHub App ID | `--github-app-id` | `ALLAGENTS_GITHUB_APP_ID` | unset | +| App private key file | `--github-app-private-key-file` | `ALLAGENTS_GITHUB_APP_PRIVATE_KEY_FILE` | unset | +| App installation ID | `--github-app-installation-id` | `ALLAGENTS_GITHUB_APP_INSTALLATION_ID` | discovered/unset | +| GitHub CLI account | `--github-cli-account` | `ALLAGENTS_GITHUB_CLI_ACCOUNT` | unset | +| OCI auth file | `--oci-auth-file` | `ALLAGENTS_OCI_AUTH_FILE` | unset | +| OCI credential helper | `--oci-credential-helper` | `ALLAGENTS_OCI_CREDENTIAL_HELPER` | unset | +| Codex auth file | `--codex-auth-file` | `ALLAGENTS_CODEX_AUTH_FILE` | supported Codex default if safe | +| Pi auth file | `--pi-auth-file` | `ALLAGENTS_PI_AUTH_FILE` | supported Pi default if safe | + +Precedence is CLI over environment over default. Credential options name file +handles, accounts, or IDs, never secret values. Auth files must be regular, +current-user/root-owned, non-hard-linked, and no broader than `0600`. Setting +both OCI options is a startup error. The OCI helper value is one absolute +executable path with no arguments; it must be current-user/root-owned and not +group/world-writable. The gateway implements Docker credential-helper `get` +directly, without a shell: argv is exactly `[helperPath, "get"]`; stdin is the +canonical registry origin `https://[:nondefault-port]` plus one +newline; and an exit-zero stdout must be one UTF-8 JSON object with exactly +nonempty string fields `Username` and `Secret`, each at most 64 KiB. Stdout over +128 KiB, a timeout, nonzero exit, signal, malformed UTF-8/JSON, an unknown +member, or an empty credential fails acquisition with +`source_auth_oci_failed`; stderr is bounded, treated as secret-bearing, and not +placed in logs or evidence. The helper is invoked once per registry origin and +its credential is scoped to that origin and destroyed after acquisition. +Provider defaults are eligible only when their resolved auth files pass the +same checks; otherwise the target is not ready. The gateway projects only the +selected provider auth into its control-process view. + +The derived workspace ID is a stable digest of the canonical project-workspace +path and is verified against store metadata. Retention includes Task records, +Artifacts, events, and invocation-key claims; expiry is atomic. When the +unexpired Task-count limit is reached, new admission fails rather than evicting +retained Tasks. + +**Project workspace additions** + +```yaml +repositories: + - name: allagents + source: https://github.com/EntityProcess/allagents.git + path: allagents + branch: main + +workspaceSnapshots: + evaluation: + repository: ghcr.io/entityprocess/allagents-workspaces ``` -### Output Structure - -```text -packages/execution-service/ - package.json - tsconfig.json - src/ - execution/ - contract.ts - extension-v1.ts - result-schema-v1.ts - worker-protocol-v1.ts - errors.ts - profiles.ts - telemetry.ts - source-credentials/ - contract.ts - registry.ts - github.ts - github-app-minter.ts - github-app-client.ts - github-cli.ts - gateway/ - index.ts - config.ts - auth.ts - agent-card.ts - request-handler.ts - executor.ts - server.ts - worker-client.ts - store/ - gateway-repository.ts - file-gateway-repository.ts - worker/ - index.ts - config.ts - server.ts - supervisor.ts - reaper.ts - lease.ts - workspace.ts - materializers/ - types.ts - registry.ts - git.ts - oci.ts - external.ts - evidence.ts - adapters/ - types.ts - registry.ts - codex.ts - pi.ts - pi-rpc.ts - pi-policy-extension.ts - tests/ - fixtures/execution/ - unit/execution/ - unit/gateway/ - unit/worker/ - unit/source-credentials/ - e2e/execution-gateway.test.ts -containers/ - gateway.Dockerfile - worker.Dockerfile -examples/gateway/ - gateway.yaml - worker.yaml -docs/src/content/docs/ - guides/execution-gateway.mdx - reference/execution-gateway-configuration.mdx +Snapshot names use the portable profile-name vocabulary. Repositories must have +unique stable names for remote acquisition. Snapshot repository values contain +only scheme/host/repository identity and never tags, digests, credentials, or +extraction paths. + +**User workspace additions** + +```yaml +profiles: + review: + clients: + - name: codex + launcher: codex-review + gateway: + expose: true ``` -### Configuration Contract - -- Gateway configuration defines the listener/public URL, a named trusted TLS - termination boundary for production ingress, auth and canonical owner - mapping, store/retention, admission and subscription quotas, low-space - watermarks, Artifact limits, worker routes, internal capability secrets, - non-secret materializer descriptors, and profiles. A descriptor contains the - materializer ID, bounded input schema, expected definition digest, expected - output-manifest version, and required worker capabilities. Each remote worker - route declares mTLS or an explicitly equivalent authenticated encrypted - overlay, pinned worker identity/capabilities, source modes and matching - materializer definition digests, and trust material; a same-host route may - declare a Unix socket. Plaintext remote URLs are invalid. -- Each profile defines backend, worker route, allowed workspace source modes, - allowed materializer IDs, exact Git repository or namespace rules, allowed - Git origins/addresses, OCI namespace/registry/signature policy, structured - materializer-input resource selectors, authorization-scope derivation, - source-authorization revocation epoch, and applicable GitHub App - entitlement-generation authority, provider/model settings, phase-specific - environment allowlists, deterministic permissions, setup/check commands, - artifact globs, acquisition and effective deadline ceilings, clock-skew - margin, trust class, resource limits, cleanup policy, evidence budgets, and - required acquisition/provider/tool isolation capabilities. -- Source-credential configuration defines normalized-host backend mappings and - ordered provider entries. `github.com` has a built-in GitHub mapping; every - GitHub Enterprise Server hostname and API base URL is explicit. A - control-plane `github-app` entry references an App ID, private-key secret - handle, installation ID or deterministic repository-to-installation mapping, - requested read-only contents permission, entitlement-generation store, - authenticated lifecycle-webhook configuration, bounded reconciliation - interval and stale-state limit, fresh-token lifetime policy, and a - versioned non-secret provider/host/API-mapping configuration digest. The - worker receives no App private-key handle. A worker-local `github-cli` entry - contains no token, names one non-secret account/login included in entitlement - and effective-profile digests, and is valid only for an explicitly - trusted-local profile. It invokes the configured `gh` binary with - `auth token --hostname --user ` after removing `GH_TOKEN`, - `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN`. -- Every remote route using a GitHub App declares the authenticated central - token minter, authoritative credential-lease controller, and single-use - non-durable grant/response protocol. Startup rejects remote App profiles - without that complete path, `github-cli` on remote or multi-tenant routes, - missing central App secret handles, unsupported hosts, ambiguous - equal-priority providers, policies that allow runtime failure to trigger - identity fallback, or acquisition ceilings that can exceed a fresh token's - safe lifetime. Configuration and effective-profile digests include provider - IDs, order, host/API mappings, route capability, pinned CLI account, - non-secret entitlement policy, and mapping/configuration digest, but exclude - private keys, resolved tokens, lease payloads, and per-attempt state. -- Worker configuration fixes a private listener, worker identity, - one-execution concurrency, one same-filesystem publication root containing - private staging and final workspace directories, a closed materializer - runtime registry, minimal worker-local command-record location, - execution-supervisor mechanism, pre-readiness orphan policy, lease grace, - backend runtime constraints, trust domain, resource-control and - credential-boundary capabilities, and request/result limits. Each external - materializer runtime entry matches the gateway descriptor's ID and expected - definition digest and additionally fixes a digest-pinned image, credential - handle names or mount identities, network destinations, resource/deadline - limits, cache policy, output version, and OCI runner or sandbox; it contains - no credential values. The worker derives, rather than trusts, the definition - digest from that complete non-secret runtime entry. -- Production worker readiness requires authenticated route identity, protected - remote transport or a same-host Unix socket, exact agreement between the - gateway's expected descriptor digest and the worker's computed runtime - definition digest, a same-filesystem staging/publication root with atomic - rename and no copy fallback, and an OCI materializer runner or sandbox that - assigns supervisor-owned attempt/lease/fence labels without exposing its - control plane to the workspace. It also requires an enforceable credential - boundary for every credentialed phase and a supervisor that proves complete - descendant termination and enumerates or destroys orphan runner resources, - credential mounts, staging mounts, and roots before readiness. The supported - worker baseline is a dedicated process namespace under a minimal init/reaper; - bare-host deployment requires an equivalent systemd/cgroup mechanism. -- Remote App readiness additionally proves that the configured central minter - and gateway/control-plane lease controller authenticate the selected worker - route, agree on the selected provider/host/API-mapping configuration digest, - and support fresh `refresh: true` minting plus a single-use non-durable - grant/response. The controller must derive profile/provider/mapping/ - installation/repository/operation/route/identity/expiry from durable state, - recheck current command revision, lease epoch, tombstone, and fence, reject - replay or substitution, and bind delivery to worker identity, attempt, lease - epoch, command revision, fence, operation, and expiry. Readiness also proves - the acquisition ceiling plus clock-skew margin fits within a fresh token's - safe lifetime, lease expiry cannot exceed token expiry, token payloads never - persist in Task or command records, and delivery reaches only the acquisition - phase. The worker image and configuration contain no App private-key handle. -- Telemetry configuration defines the OTLP destination, filtering/redaction bounds, opaque owner-correlation derivation, and telemetry-specific operator access and retention. The service version fixes the metadata allowlist; configuration cannot extend it to prompt/output/tool/source/file-body attributes, secret-bearing fields, raw caller identity, or unfiltered backend-native/OpenInference attribute passthrough. -- Configuration contains environment-variable names but never secret values. Startup resolves the complete graph and becomes ready only when trusted ingress, worker transports/identities, store, runtimes, quotas, free-space reserves, supervisor/orphan recovery, and declared profile capabilities pass. Any unprotected remote endpoint or unproved credential/supervisor boundary fails readiness. +The nested object is strict and initially contains only `expose: true`. Absence +means not exposed. Exposure requires a launcher, an initial supported backend, +and a healthy installed profile with matching declaration digest. ### Error and Status Mapping -| Condition | A2A result | Required extension detail | +| Condition | Stable code and A2A outcome | Fresh-invocation retryable | |---|---|---| -| New-admission authentication, malformed/unsupported extension carrier, invalid workspace source/profile, unknown or profile-disallowed materializer, unauthorized policy, expired deadline, current-profile/readiness failure, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | -| Identical retained invocation replay | Existing Task | Returned from stored original request/profile/schema bindings before current deadline, quota, authorization, readiness, or profile checks; no new Task, worker attempt, or quota reservation | -| Conflicting invocation key or inconsistent stored binding | Operation error; no new Task | Conflict code; existing Task unchanged | -| Worker capacity loss after acceptance | `TASK_STATE_FAILED` | `dispatch/capacity_exhausted`, retriable fact, no workspace created; gateway does not retry | -| Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | -| Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | -| Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | -| No eligible GitHub provider after acceptance | `TASK_STATE_FAILED` | `materialization/source_auth_unavailable`; safe reason `no_eligible_provider`; `retriable: false`; no provider identity in public detail | -| Selected installation does not cover the repository | `TASK_STATE_FAILED` | `materialization/source_auth_denied`; safe reason `installation_repository_denied`; `retriable: false`; installation identity is operator-only; no `gh` fallback | -| Selected App configuration is invalid | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_configuration_invalid`; `retriable: false`; operator-only provider detail; no `gh` fallback | -| Selected App authentication fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_authentication_failed`; `retriable: false`; operator-only provider detail; no `gh` fallback | -| Selected App token mint or fresh-lifetime validation fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_mint_failed`; `retriable: false`; operator-only provider detail; no `gh` fallback | -| Selected App provider is rate limited | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `provider_rate_limited`; `retriable: true`; no provider identity in public detail; no `gh` fallback | -| Selected App provider service is unavailable | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `provider_unavailable`; `retriable: true`; no provider identity in public detail; no `gh` fallback | -| Eligible trusted-local GitHub CLI provider fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `trusted_local_cli_failed`; `retriable: false`; configured account identity is operator-only | -| Failure before result-candidate production | `TASK_STATE_FAILED` | Typed primary dispatch/materialization/setup/provider/crash/infrastructure phase, including manifest or materializer failure; safe message, retriable fact, requested structured result `not_produced`, separate termination/cleanup/completeness, and bounded workspace provenance | -| Check, mandatory-evidence, cleanup, crash, or infrastructure failure after result validation | `TASK_STATE_FAILED` | Preserve selected `valid` or `invalid`; preserve exactly one fixed structured-result Artifact for `valid`; later phase remains primary failure | -| Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` with `not_produced`, or `structured_result/invalid` with `invalid`; no structured-result Artifact | -| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors and native abort; use `not_produced` only before a candidate, otherwise preserve `valid`/`invalid` and the valid Artifact; record termination and cleanup | -| Cancellation loses to terminal completion | Existing terminal Task / `TaskNotCancelableError` | No state mutation or second abort | -| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Required extension integrity Artifact plus complete evidence; a requested valid result uses the separate fixed-name Artifact with one A2A `Part` containing `data` and `mediaType: application/json` | -| Successful action with allowed bounded optional-evidence gap | `TASK_STATE_COMPLETED` | Per-dimension incomplete flag, reason, original/captured size, digest and redaction/truncation flags | -| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid; use `not_produced` only before a candidate, otherwise preserve selected state and valid Artifact; cleanup unknown unless proven | -| Retention expiry | Not found | Aggregate logically hidden before physical deletion; Artifact URL also invalid | +| Missing required extension | A2A `ExtensionSupportRequiredError`; no Task | No | +| Malformed request, source, digest, schema, prompt, or unknown target/source | `invalid_execution_request` in A2A `InvalidParamsError.data`; no Task | No | +| Invocation-key conflict | `invocation_key_conflict` in A2A `InvalidParamsError.data`; no new Task | No | +| Identical retained invocation replay | Existing Task and Artifacts | N/A | +| Cancel after terminal state | A2A `TaskNotCancelableError` | No | +| Retained Task capacity exhausted | `retention_capacity_exhausted` in A2A `InternalError.data`; no Task | Yes, after expiry | +| Runtime capacity unavailable after acceptance | `execution_capacity_unavailable`; failed Task | Yes | +| App absent/ineligible and configured `gh` succeeds | Continue with recorded provider class | N/A | +| App applicability unknown | `source_auth_applicability_unknown`; failed Task; no fallback | Yes for rate-limit/service causes only | +| Selected App config/auth/mint failure | `source_auth_failed`; failed Task; no fallback | No | +| Selected App permission/repository denial | `source_auth_denied`; failed Task; no fallback | No | +| Selected App rate limit | `source_auth_rate_limited`; failed Task; no fallback | Yes | +| Selected App service failure | `source_auth_unavailable`; failed Task; no fallback | Yes | +| `gh` account missing or token resolution fails | `source_auth_unavailable`; failed Task | No | +| Git revision/identity failure | `source_git_identity_invalid`; failed Task | No | +| Git transport failure | `source_git_unavailable`; failed Task | Yes | +| OCI helper timeout, process, protocol, or credential failure | `source_auth_oci_failed`; failed Task; no fallback | No | +| OCI auth/digest/manifest/extraction validation failure | `source_snapshot_invalid`; failed Task; no Git fallback | No | +| OCI registry service failure | `source_snapshot_unavailable`; failed Task; no Git fallback | Yes | +| Deadline expires | `execution_deadline_exceeded`; abort/terminate; failed Task | Yes | +| Known provider permission denial | `execution_permission_denied`; rejected Task | No | +| Unknown provider protocol or result shape | `provider_protocol_invalid`; failed Task | No | +| Cancellation with proven quiescence | `execution_cancelled`; cancelled Task | No | +| Termination or cleanup cannot be proven | `execution_quiescence_unknown`; failed Task; readiness poisoned | No | +| State store durability/integrity failure | `task_store_failed`; stop admission; abort/contain; no success | No | +| Restart finds interrupted Task | `gateway_restarted`; failed Task; no provider resume | Yes as a new invocation | +| Retention expiry | A2A `TaskNotFoundError` | Yes as a new invocation | + +Accepted-Task failures use the integrity Artifact's strict `failure` object with +`code`, safe `message`, table-defined `retryable`, and one closed cause from +`validation | capacity | sourceAuth | sourceGit | sourceSnapshot | deadline | +permission | providerProtocol | cancellation | termination | stateStore | +restart`. Admission failures use the exact A2A error type in the table with the +same stable code and retryability in safe `data`. Retryability describes whether +a caller may create a fresh invocation; it never enables automatic Task retry +or fallback. Provider identifiers, credentials, paths, and raw upstream +messages never enter either carrier. ### Phased Delivery -1. Create the private Node 22 service package and freeze the public extension URI and standard carriers, integrity and structured-result Artifacts, portable result-schema subset, worker protocol including command revisions/tombstones, profiles, fixtures, and error vocabulary. -2. Build authenticated durable A2A Task handling, retained-claim-first replay, and trusted fenced worker dispatch against a fake worker; startup terminalizes interrupted Tasks without attempting provider reattachment. -3. Build the supervised single-execution worker lifecycle, monotonic command - record, failed-quiescence boundary recycling, pre-readiness orphan reaper, - OS credential boundaries, the direct Git/OCI/registered-materializer - registry, the central GitHub App minter/client and trusted-local GitHub CLI - source-credential registry using `@octokit/auth-app`, and hardened - workspace/evidence handling against fake materializers and a fake backend. -4. Add the direct Codex SDK adapter and prove structured output, cancellation, OS-enforced provider/tool credential separation, and native evidence. -5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing OS-confined tools plus the terminating result tool. -6. Package the services and run cross-backend, transport, security, process, and A2A conformance before enabling a consumer. +1. Build the current CLI and record the red E2E showing that + `allagents gateway serve` is unavailable. Record the exact `/tmp/` workspace + setup, command, and observed failure. +2. Freeze workspace additions, public extension, common manifests, result + schema, errors, and fixtures. +3. Build the deployment-wide Task store and unauthenticated A2A server against + a fake adapter. +4. Add repository and OCI acquisition with credential containment and manifest + validation. +5. Add the invocation supervisor, execution containment, safe evidence, and + shared backend contract. +6. Add Codex, then Pi, against the same conformance suite. +7. Run final implementation review and fix important correctness, security, + contract, reliability, DRY, and coverage findings. +8. Run the green built-CLI `/tmp/` E2E, repository quality gates, user + documentation, and release evidence. ### System-Wide Impact -- **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. -- **Dependency surface:** `@octokit/auth-app` is private to the Node 22 - execution-service package and used only by the trusted control-plane GitHub - App minter. The root Node 18 CLI, remote worker, and acquisition subprocess - do not import the full Octokit client or hold App private-key material. -- **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. -- **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. -- **Security:** New review-critical surfaces are trusted public/private - transports, auth, owner-key derivation, retained-replay ordering, Git and OCI - source SSRF, materializer image supply chain, materializer input schemas, - phase-scoped source credentials and egress, workspace manifest validation, - admission/resource quotas, setup/check policy, OS-enforced provider/tool - credential separation, Pi extension/tool replacement, metadata-only - telemetry filtering and operator boundaries, internal fences and monotonic - command records, Artifact capture/serving, and reviewed-source trust - enforcement. -- **Operations:** Gateway and worker health, readiness, transport/peer identity, quotas, low-space state, allowlisted metadata-only structured logs/traces, telemetry-specific access/retention, command tombstones, lease expiry, poisoned-worker exit, supervisor boundary health, orphan-root quarantine/reaping, stale event rejection, and graceful shutdown need independent signals. -- **Consumers:** AI Evals can build its runner provider only after the Agent Card, extension schemas, and conformance fixtures are versioned and published. +- **Package surface:** Add a private execution-service package and the public + `allagents gateway serve` command. Preserve existing profile and sync commands. +- **Schema surface:** Extend project workspace schemas with named snapshots and + user profile-client schemas with explicit exposure. Regenerate versioned JSON + Schemas and update configuration docs. +- **Dependency surface:** Add the official A2A SDK, pinned Codex SDK, + `@octokit/auth-app`, and a focused OCI client/extraction implementation to the + private execution package. +- **State surface:** Add a bounded gateway state root and per-invocation staging, + publication, evidence, and cleanup roots. Do not alter existing profile state. +- **Security surface:** The network is the authorization boundary. Source + credentials are phase-scoped; acquired code and agent tools never receive App, + `gh`, or OCI credentials. +- **Compatibility:** Existing workspace files remain valid because new fields are + optional. Older binaries reject the new strict nested profile field, so docs + must state the minimum supporting version. ### Risks and Mitigations -- **Provider API churn:** Pin exact compatible SDK/CLI versions in the service lockfile and worker image. Gate capabilities at startup, keep captured provider fixtures versioned, and use Promptfoo's Codex tests as characterization input rather than vendored implementation. -- **False idempotency or stale settlement:** Resolve owner-scoped retained claims before mutable admission and compare stored original request/profile/schema bindings. For new work, claim Task/idempotency in one aggregate, use revision/fence compare-and-swap, sequence events, and fault-test conflicts, cancellation races, restart, and late results. -- **Task/store corruption:** Publish immutable blobs and generations before one manifest switch; tombstone before deletion; validate owner tuples/manifests at startup; garbage-collect unreachable generations; document the one-replica limit. -- **Owner collision or path injection:** Hash a bounded canonical issuer/tenant/subject tuple, store and verify the tuple inside the owner aggregate, and use only server-generated opaque IDs in paths. -- **Bearer interception or worker impersonation:** Require TLS at the named public ingress boundary and mTLS/equivalent authenticated encryption for remote worker routes, pin worker identity/capabilities, bind attempt capabilities to that identity and fence, and reject plaintext or wrong-peer readiness. -- **Orphan processes and roots:** Combine explicit cancel, native abort, process-set verification, one-execution supervisor/container death, lease expiry, and pre-readiness orphan reaping or quarantine. Failed quiescence poisons admission and exits the worker so the supervisor destroys the boundary; termination/filesystem outcomes remain separate. -- **False recovery claims:** Persist Task and evidence truth only. Startup fails active Tasks, invalidates fences, and relies on lease expiry or supervisor-boundary proof instead of resuming provider sessions. -- **Structured-output drift:** Admit only the versioned closed schema subset, include its canonical digest in provenance and original claim bindings, pass the exact accepted schema through each adapter, validate with one shared validator, preserve an already selected result across later failures, and enforce the two fixed Artifact shapes. -- **Source SSRF, materializer compromise, or credential leakage:** Enforce - KTD13 for every source mode, connection, and phase. Pin external - materializer images and OCI snapshots by digest, validate their manifests, - isolate staging and acquisition processes, apply explicit egress and limits, - and atomically publish only validated outputs. Source credentials are - ephemeral, origin-bound, and removed before setup. Credentialed profiles also - enforce the R13 OS provider/tool boundary or broker; environment filtering - remains defense in depth. Pi repository extensions and unrestricted built-in - tools never load. -- **Credential fallback, stale entitlement, or issuer-key escalation:** Treat - provider order as eligibility, not retry. Prefer only the operator-mapped - GitHub App installation, permit an account-pinned GitHub CLI provider only in - trusted-local profiles when no mapping applies, and fail closed after every - selected-App failure. Keep App private keys in the central minter. Make the - lease controller derive provider/repository/route state from the current - durable command, use one single-use grant, and reject replay, substitution, - stale fences, or controller/minter configuration-digest disagreement. Use a - fresh `refresh: true` token per cache-miss acquisition, bound its lifetime to - the acquisition deadline plus skew, and reject unsafe ceilings at readiness. - Authenticated lifecycle webhooks plus reconciliation advance entitlement - generations so stale/unknown App state cannot authorize cache reuse. Keep - tokens out of arguments, Git configuration, durable records, logs, evidence, - and later phases; public failures remain coarse and identities operator-only. -- **Telemetry disclosure:** Apply KTD10's pre-export guard before every structured log/span processor and reject content or secret-bearing attributes rather than relying on exporter policy. Canary-secret and cross-owner-fragment tests cover agent, model, tool, stale-event, and error paths; telemetry operators receive only bounded metadata and opaque owner correlation under separate access and retention. -- **Resource exhaustion:** Reserve per-owner/global gateway quota only for new claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. -- **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. -- **Evidence overclaim:** Enforce KTD12's integrity kernel and per-dimension completeness. Truncation and redaction remain independent facts. -- **Permission deadlock:** Initial profiles never prompt. Known requests resolve for one isolated invocation; unknown shapes fail closed as adapter incompatibility. -- **Trust-boundary overclaim:** Enforce the narrow provider/tool credential boundary for credentialed profiles while rejecting pooled hostile-source/cross-tenant claims; state plainly that the former does not provide the latter. -- **Cross-platform drift:** Keep gateway/store tests cross-platform. State that worker execution and hardened evidence/source controls are Linux-only. +- **Accidental network exposure:** Binding `0.0.0.0` is intentional and allowed; + startup output and docs state that every reachable host has full authority. +- **Profile identity drift:** Derive targets only from current validated user + declarations and matching installed state; never resurrect declaration-missing + launchers from retained profile state. +- **Credential leakage:** Use fresh App tokens or one configured `gh` account, + invocation-only helpers, hermetic Git, and credential teardown before + publication. Separate provider-control, per-MCP, and model-tool views prevent + one secret scope from reading another. +- **Identity-changing fallback:** Classify App applicability as eligible, + ineligible, or unknown; only positive ineligibility permits `gh`. +- **OCI archive abuse:** Require immutable digests, configured repositories, + bounded extraction, path/type/link validation, and manifest verification. +- **Untrusted acquired code:** General hostile-code sandboxing is not claimed, + but model-invoked tools cannot reach provider/MCP/operator credentials or + gateway state. Project/user setup shell commands are never automatic. +- **Evidence-time attacks:** Treat the mutated workspace as untrusted, use + descriptor-relative no-follow reads, reject Git metadata indirection, and + disable repository-controlled Git execution features. +- **Provider/API churn:** Pin compatible SDK/CLI versions and retain versioned + native fixtures plus one adapter conformance suite. +- **Orphaned processes:** Require a platform containment primitive whose + descendants cannot escape; fail readiness when unavailable. On uncertain + quiescence, unmanaged mode stays alive to reap and managed mode exits only + after cleanup ownership transfer. +- **Store corruption or disclosure:** Validate ownership, modes, links, root + disjointness, lock, and workspace identity. Integrity/durability failure stops + admission and prevents terminal success. ### Assumptions -- The first production deployment runs one gateway replica with persistent storage. Multi-replica transactional storage is deferred. -- The initial public extension supports direct multi-repository Git, - digest-pinned OCI workspace snapshots, and operator-registered materializers. - A deployment may enable only the source modes its worker route advertises; - direct hardened Git remains the required baseline. -- Setup and check commands are operator-controlled profile policy, not caller-supplied shell text. -- Initial repositories are reviewed inside one configured mutual-trust domain. Credentialed profiles still enforce provider/tool credential separation, but that narrower boundary does not make hostile-code or cross-tenant execution available; those claims require a stronger sandbox driver. -- Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. +- The initial deployment is one gateway process and one active invocation. +- Every network peer able to connect is trusted with all exposed targets and + retained Tasks. +- The selected project workspace is operator-controlled and uses supported + repository/snapshot declarations. +- GitHub.com is the only authenticated Git host in the initial delivery. +- OCI snapshots use registries reachable through HTTPS and immutable manifests. +- Codex and Pi automation surfaces remain compatible with the pinned versions. +- Supported platforms provide a non-escapable invocation containment strategy; + the gateway fails readiness where that invariant cannot be met. --- ## Implementation Units -### U1. Versioned public and worker contracts - -- **Goal:** Freeze the standard public extension carriers, versioned workspace - source and manifest contracts, materializer/profile vocabulary, integrity and - structured-result Artifacts, private worker protocol including monotonic - command state, original idempotency bindings, typed failures, and conformance - fixtures before either service endpoint. -- **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12, KTD16. -- **Dependencies:** None. -- **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/result-schema-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. -- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas and freeze `https://allagents.dev/a2a/extensions/coding-execution/v1`: required Agent Card advertisement, `A2A-Extensions` negotiation, `Message.extensions`, request data only at `Message.metadata[uri]`, and terminal integrity data only in the single Part of the fixed-name `allagents.execution-integrity` Artifact whose `extensions` contains the URI. Explicitly forbid `Task.extensions`. Define the portable result-schema subset, canonical caller/schema/profile digests, four result states, separate fixed `allagents.structured-result` Artifact, and shared validator. Define original claim bindings independently from mutable current policy. Add worker identity, attempt/fence/lease identity, monotonic command revision, unseen-attempt cancel tombstone, conditional effect revision, event sequence, terminal acknowledgement, and integrity rules. Generate checked-in schemas and fixtures from one source. - The source contract is a strict `kind`-discriminated union for direct - repository lists, digest-pinned OCI snapshots, or a registered materializer - ID with an expected workspace-manifest digest and schema-validated structured - inputs; cross-variant fields are unrepresentable. Define the standard - workspace manifest, verification-method vocabulary, authorization scope and - revocation epoch, collision-safe destinations, and split gateway/worker - materializer descriptors. Define algorithm-qualified digest formats and - versioned, domain-separated canonical preimages for materializer definitions, - inputs, manifests, profiles, and caller requests; no public field can carry - acquisition code, image references, commands, credentials, or policy. - Define normalized source-host/API mappings and ordered source-credential - provider policy as trusted profile/configuration fields. Public schemas cannot - select a provider. Effective-profile canonicalization includes provider IDs, - order, host/API mappings, mapping/configuration digest, pinned CLI account, - non-secret entitlement policy, and current App entitlement generation while - excluding App private keys, resolved tokens, local account tokens, and - per-attempt provider state. Define cache metadata that preserves original - acquisition-provider metadata and cache-hit provenance that separately names - `cache_hit` and current policy selection. - Define the private source-credential protocol separately from the durable - worker command record. A worker request contains only active attempt and - fence under its authenticated route. The controller response carries its - authoritative derivation of effective-profile digest, selected provider, - host/API-mapping digest, installation ID, repository, operation, worker - route/identity, lease epoch, command revision, and expiry, plus a single-use - non-durable grant/response state. The lease/channel binds all derived fields - and cannot outlive the token; the token schema expresses only repository, - read-only contents permission, and GitHub expiry. Define deterministic - source-auth code/reason/retryability enums and operator-only identity detail. - Token payloads are secret transport data: they are never part of public - schemas, canonical digests, Task storage, command records, events, logs, - errors, evidence, or provenance. -- **Execution note:** Start with fixture-driven schema, framing, and digest - tests. Observe failures for unknown versions, credential-bearing sources, - mutable revisions or image tags, duplicate/unsafe destinations, unknown or - disallowed materializers, invalid structured inputs or workspace manifests, - unsafe paths, invalid public states, stale fences, oversized records, and - conflicting canonical inputs before implementing schemas. -- **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, `src/core/native/types.ts` for safe error/provenance normalization, and Buzz's structurally non-secret intent template for the narrow digest-input pattern. -- **Test scenarios:** - - A minimal valid Message negotiates the exact URI in `A2A-Extensions`, includes it in `Message.extensions`, puts the bounded request only at `Message.metadata[uri]`, and produces a stable digest across object-key ordering; missing/mismatched carriers and any `Task.extensions` field are rejected. Every terminal fixture has exactly one `allagents.execution-integrity` Artifact with the URI in `Artifact.extensions` and the schema-defined envelope in its single `data` Part. - - Changing prompt, source object ID, profile ID, result schema, artifact selection, or deadline changes the canonical caller digest; trace IDs and transport metadata do not. The original effective-profile and result-schema digests are stored separately for retained replay. - - Direct repositories are order-canonicalized without erasing destination - identity; duplicate destinations, mutable refs, unsafe subdirectories, and - ambiguous URL forms fail. The checked-out commit and tree match the - requested object from the authorized remote. OCI tags and external layer - URLs fail while allowed manifest digests pass. - - The source discriminator rejects unknown `kind` values, cross-variant - fields, and missing variant fields. Registered materializer inputs validate - against the operator schema and resource selectors, the request pins the - expected workspace-manifest digest, the worker-derived definition digest - changes the effective-profile digest, gateway and worker descriptors agree, - and caller-supplied image/command/credential fields are unrepresentable. - Fixed cross-language vectors prove algorithm-qualified, domain-separated - canonical digests and every non-secret runtime-field mutation changes the - definition digest while secret-value rotation does not. - - Rotating a resolved secret value, changing attempt/lease/trace identity, or - changing a per-run path leaves the profile digest unchanged; changing a - provider policy field, pinned CLI account, host/API mapping or mapping - digest, environment-variable name, authorization scope, revocation epoch, - or App entitlement generation changes it, and the digest serializer cannot - accept secret-bearing runtime state. - - Provider policy fixtures accept GitHub App followed by account-pinned - trusted-local GitHub CLI, reject CLI on remote or multi-tenant routes, - require explicit GitHub Enterprise Server host/API mappings, and reject - every public credential-provider field. Fixtures encode - `gh auth token --hostname --user ` and removal of all four - ambient GitHub token variables. Provider order, ID, host/API mapping, - pinned account, entitlement, or non-secret configuration digest changes - the effective-profile digest; private-key or token rotation does not. - - Private credential-request fixtures accept only active attempt and fence. - Controller-response fixtures carry worker identity/route, attempt, lease - epoch, command revision, fence, effective-profile/provider/mapping/ - installation/repository/operation bindings, and expiry; reject replay, - substitution, stale state, duplicate grant consumption, expiry after token - expiry, or controller/minter configuration-digest disagreement; and cannot - round-trip through durable command/Task serializers. Token fixtures contain - only repository/read-only/expiry scope. Remote App profile fixtures require - the complete central minter/controller capability, safe lifetime policy, - entitlement-generation authority, and no worker-side private-key handle. - - Cache metadata fixtures distinguish original acquisition-provider metadata, - `cache_hit`, and current policy selection/entitlement. Unknown or stale App - generations reject cache authorization. - - Exact source-auth fixtures cover every safe code/reason/retryability tuple - from the error table and prove provider, installation, and account - identifiers are absent from public detail but available to operators. - - Unsupported keywords, remote references, non-object roots, object schemas that omit `additionalProperties: false`, undeclared optional properties, format-dependent validation, or schemas over byte/depth/property/enum limits are rejected before Task creation; every accepted schema validates identically in admission, worker, Codex forwarding, and Pi tool generation. - - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. - - Worker fixtures reject missing/mismatched worker identities, attempt IDs, lease epochs, profile digests, command revisions, conditional-effect revisions, event sequences, bounds, and terminal acknowledgements. Cancel for an unseen attempt persists a tombstone; tombstoned or lower-revision dispatch is invalid before workspace creation. - - `not_requested`, `not_produced`, `valid`, and `invalid` cover success and failure without replacing the primary Task classification. `not_produced` is accepted only before candidate production; a selected `valid` or `invalid` survives later check/evidence/infrastructure failure, and only `valid` permits exactly one separate `allagents.structured-result` Artifact with the matching schema digest. - - File evidence accepts create/edit/delete/rename and rejects unsafe paths, duplicate identities, oversized inline content, and inconsistent before/after forms. -- **Verification:** Generated schemas are stable, public/private fixtures round-trip, digest vectors are cross-platform deterministic, and the private client/server fixture suite agrees before gateway or worker implementation. - -### U2. Authentication and durable gateway repository - -- **Goal:** Provide caller-scoped authentication, trusted-ingress configuration, retained-claim-first idempotency aggregates with original bindings, Artifact storage, new-claim quota admission, pagination, restart fencing, logical expiry, and cleanup. -- **Requirements:** R4-R9, R13-R14, R17-R18, R22; AE2-AE4, AE6, AE8, AE10-AE13; KTD1, KTD3-KTD4, KTD12. -- **Dependencies:** U1. -- **Files:** `packages/execution-service/src/gateway/config.ts`, `packages/execution-service/src/gateway/auth.ts`, `packages/execution-service/src/gateway/store/gateway-repository.ts`, `packages/execution-service/src/gateway/store/file-gateway-repository.ts`, `packages/execution-service/tests/unit/gateway/auth.test.ts`, `packages/execution-service/tests/unit/gateway/file-gateway-repository.test.ts`. -- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Resolve a retained claim after authentication and bounded parsing, and compare its stored canonical caller request plus original effective-profile/result-schema digests without consulting mutable current policy. For new work only, reserve owner/global quota and commit the claim, original bindings, and submitted Task in one manifest generation. Publish immutable Artifact blobs before one manifest switch; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover unreachable generations; and complete startup recovery before serving. Verify OIDC/static tokens before repository access and validate the configured named TLS ingress boundary before readiness. -- **Execution note:** Implement concurrent-claim, transition-race, and crash-publication tests before request handling. Inject faults between blob, generation, manifest, tombstone, and cleanup operations. -- **Patterns to follow:** `src/core/marketplace.ts` and `src/core/profile/files.ts` for atomic publication/recovery, `src/core/mcp-http-stdio-proxy.ts` for private files and loopback safety, and the official A2A `TaskStore` owner-scoping contract. -- **Test scenarios:** - - Covers AE2-AE3. Concurrent identical new claims create one aggregate; a conflicting original request/schema binding returns conflict without dispatch permission. Identical retained replay still returns the existing Task after its deadline, quota, authorization, readiness, or current profile changes, while an inconsistent stored binding fails closed. - - Covers AE4. Load/list/cancel/subscribe/Artifact lookup scopes before path/database access and gives unknown, unauthorized, and expired IDs indistinguishable behavior. - - Hostile/ambiguous issuer, tenant, subject, invocation key, Task ID, Artifact name, Unicode, case, delimiter, traversal, and Windows-reserved values cannot collide or become paths. - - All standard list filters, `historyLength`, page size 1-100, omitted Artifacts, ordering, total size, and always-present next token match A2A semantics. Tokens are owner/query-bound and reject malformed, swapped, or stale filters. - - Covers AE12. Terminal compare-and-swap wins once; stale fence, duplicate, and out-of-order updates cannot mutate the Task. - - Restart, including repeated failure during startup recovery, completes the recovery barrier before serving: it fails every nonterminal Task once, invalidates fences, never renews an old lease or requests provider reattachment/replay, preserves terminal Tasks, and records cleanup unknown unless proven. - - Covers AE13. Exact expiry tombstones the aggregate before cleanup; failed deletion never restores visibility; same-key replay before expiry returns the old Task and after expiry creates a new Task. - - A crash between every aggregate publication step leaves either the prior or next valid manifest, never claim-without-Task or Task-with-missing-Artifact state. - - OIDC rejects wrong issuer, audience, signature, expiry, scope, tenant, and subject; static tokens and internal capabilities never appear in logs/errors. Production readiness rejects missing/mismatched named TLS termination, while unauthenticated plaintext remains loopback-only. - - Quota-boundary races admit exactly the allowed new claims and preserve reserved capacity for cancel/terminal writes; low-space mode stops new claims without blocking retained replay or settlement. - - Unauthenticated mode starts only on loopback and refuses wildcard or non-loopback listeners. -- **Verification:** A fresh process retrieves prior records, fault recovery finds one valid aggregate generation, authorization cannot reveal neighboring owners, and expiry/quota behavior remains deterministic under concurrency. - -### U3. A2A gateway server and fenced worker client - -- **Goal:** Expose the accepted A2A profile while making extension negotiation, retained replay, new admission, streaming, lookup, authenticated worker routing, monotonic worker commands, failure, and cancellation use one durable state machine. -- **Requirements:** R1-R9, R11, R14-R15, R17-R22; F1-F5; AE1-AE4, AE6, AE8-AE13; KTD1-KTD7, KTD9, KTD12. -- **Dependencies:** U1, U2. -- **Files:** `packages/execution-service/src/gateway/agent-card.ts`, `packages/execution-service/src/gateway/request-handler.ts`, `packages/execution-service/src/gateway/executor.ts`, `packages/execution-service/src/gateway/server.ts`, `packages/execution-service/src/gateway/worker-client.ts`, `packages/execution-service/tests/unit/gateway/agent-card.test.ts`, `packages/execution-service/tests/unit/gateway/request-handler.test.ts`, `packages/execution-service/tests/unit/gateway/executor.test.ts`, `packages/execution-service/tests/e2e/gateway-fake-worker.test.ts`. -- **Approach:** Mount the official HTTP+JSON and Agent Card handlers behind trusted ingress and auth. Advertise the exact required URI; validate `A2A-Extensions`, `Message.extensions`, and `Message.metadata[uri]`; and publish the integrity envelope only through the standard Artifact carrier, never `Task.extensions`. The `A2ARequestHandler` decorator authenticates and bounded-canonicalizes, resolves the owner-scoped retained claim, and returns or conflicts against stored original bindings before current profile/deadline/quota/readiness checks. New requests then pass mutable admission and canonical Task reservation. Keep transition selection pure and execute fenced I/O outside it. Dispatch revisioned commands over mTLS/equivalent authenticated encryption or a same-host Unix socket, binding worker identity/capability/fence; atomically publish Artifact blobs plus the terminal manifest. -- **Execution note:** Begin with an in-process fake worker and official A2A client. Prove operation errors versus accepted-Task failures, replay/subscribe behavior, fencing, cancellation races, and restart before adding providers. -- **Patterns to follow:** Official A2A sample `AgentExecutor`, `A2ARequestHandler`, `DefaultRequestHandler`, Express handlers, and cancellable-agent flow; `src/core/mcp-http-stdio-proxy.ts` for HTTP shutdown and loopback tests; and Buzz's pure classifier/I/O reconciler split for transition selection without adopting its Kubernetes model. -- **Test scenarios:** - - Covers AE1. Agent Card negotiation, `A2A-Extensions`, `Message.extensions`, `Message.metadata[uri]`, and the fixed integrity Artifact pass through the official client; missing/mismatched carriers and `Task.extensions` fail. `returnImmediately` and streaming expose the same durable Task. - - New-admission authentication, invalid extension/source/profile, expired deadline, current-profile/readiness failure, and pre-claim quota failure return operation errors with no Task or worker request. - - Covers AE11. Capacity loss after acceptance fails the retained Task at `dispatch/capacity_exhausted`; ambiguous dispatch fails `dispatch_unknown`; neither is retried. - - Covers AE2-AE3. Identical send/stream replay returns the existing Task before mutable checks even after the stored deadline or current profile changes; changed request/schema or inconsistent original binding conflicts. - - Covers AE8. Active subscribe emits current snapshot then future events without missed-event replay; terminal subscribe errors and `GetTask` returns terminal truth. - - Covers AE4. Get/list/subscribe/cancel/Artifact endpoints apply owner authorization consistently. - - Covers AE6. Cancel in submitted/working, cancel versus accept/completion, caller versus deadline, duplicate cancel, and terminal cancel each produce one linearized outcome and at most one worker abort. If dispatch send is paused after selection and a newer cancel completes first, releasing the stale dispatch cannot create a workspace or provider process. - - Covers AE12. Duplicate, out-of-order, malformed, wrong-identity, wrong-revision, wrong-fence, and late terminal events cannot overwrite Task state; stale facts go only to allowlisted metadata-only telemetry with opaque owner correlation. - - A failed fenced effect or changed observation causes a durable re-read and reclassification; the executor never substitutes a fresher fence or revision into an effect selected from stale state. - - Plaintext remote workers, wrong certificates, wrong configured worker identity/capability, and replayed attempt capabilities fail before dispatch; mTLS/equivalent protected routes and same-host Unix sockets succeed. - - Known policy denial rejects only after stop/cleanup; unknown permission shape fails as adapter incompatibility. - - Caller SSE disconnect and telemetry exporter failure leave execution and terminal lookup intact. - - Graceful shutdown stops admission, claims cancellation for bounded active work, persists honest terminal state, and closes listeners. -- **Verification:** The official SDK client exercises every advertised operation against the built gateway and fake worker; persisted snapshots match streams while aggregate/fence invariants remain intact under races. - -### U4. Worker protocol and safe workspace lifecycle - -- **Goal:** Implement the supervised single-execution worker with authenticated - transport, a minimal monotonic command record, a closed workspace - materializer registry for hardened multi-repository Git, digest-pinned OCI, - and operator-registered images, trusted source-credential resolution, - standard manifest validation, OS-enforced credential separation, leases, - isolated roots, resource controls, race-resistant evidence, - failed-quiescence recycling, and cleanup independent of any provider. -- **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14, KTD16. -- **Dependencies:** U1. -- **Files:** `packages/execution-service/src/source-credentials/contract.ts`, `packages/execution-service/src/source-credentials/registry.ts`, `packages/execution-service/src/source-credentials/github.ts`, `packages/execution-service/src/source-credentials/github-app-minter.ts`, `packages/execution-service/src/source-credentials/github-app-client.ts`, `packages/execution-service/src/source-credentials/github-cli.ts`, `packages/execution-service/src/source-credentials/lease-controller.ts`, `packages/execution-service/src/source-credentials/github-app-entitlements.ts`, `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/materializers/types.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/worker/materializers/git.ts`, `packages/execution-service/src/worker/materializers/oci.ts`, `packages/execution-service/src/worker/materializers/external.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/source-credentials/registry.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-minter.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-client.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-cli.test.ts`, `packages/execution-service/tests/unit/source-credentials/lease-controller.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-entitlements.test.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/materializers.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`, `packages/execution-service/tests/fixtures/execution/fake-materializer.ts`. -- **Approach:** Authenticate the configured worker identity and fence every - private command. Persist one minimal monotonic command record scoped to worker - identity/lease before workspace creation: unseen-attempt cancel writes a - tombstone, stale/lower-revision dispatch is rejected, and each dispatch/cancel - effect conditionally rechecks the stored revision immediately before - mutation. Reserve one execution only after that check. Validate the fully - canonicalized source against profile resource policy before cache lookup. - Bind cache entries to owner or authorization-scope digest, revocation epoch, - current GitHub App entitlement generation when applicable, canonical source, - definition digest, expected/actual manifest digests, and original - acquisition-provider metadata. Authenticated App lifecycle webhooks plus a - bounded reconciler advance entitlement generation on uninstall, suspension, - and repository-selection changes; unknown or stale state fails cache - authorization. A hit revalidates current authorization and records - `cache_hit`, original acquisition provider, and current policy selection/ - entitlement separately. A hit never mints a token. - Validate deployment, worker-computed materializer definition digest, and - acquisition plus provider/tool credential-boundary capabilities, then emit - sequenced NDJSON. Keep transition selection pure. Run inside a dedicated - container process namespace under init/reaper or an equivalent systemd/cgroup - boundary. Resolve only the closed KTD13 materializer registry. - - Resolve direct-Git credentials through the closed source-credential registry - only after source authorization and a cache miss. Normalize the host, select - the configured backend, and evaluate providers in policy order. For GitHub, - trusted operator configuration resolves the installation ID; auth-app never - discovers it. The authenticated worker asks the gateway/control-plane - credential-lease controller only for its active attempt and fence. The - controller rechecks durable command revision, lease epoch, tombstone, and - fence; derives effective-profile digest, selected provider, host/API-mapping - digest, installation ID, canonical repository, operation, worker route/ - identity, and expiry; and issues a single-use non-durable grant/response. - A separate minter atomically consumes that grant and rejects a mismatched - configuration digest. Replay, field or provider substitution, and stale - state fail before minting. - - The control-plane minter uses focused `@octokit/auth-app` with - `refresh: true` for every acquisition. Accept only a fresh token scoped to the - authorized repository, read-only contents permission, and GitHub expiry, - with remaining lifetime strictly greater than the acquisition deadline plus - clock-skew margin; expire the lease no later than the token. The delivery - lease/channel—not the bearer token—binds worker identity, attempt, lease - epoch, command revision, fence, repository, operation, and expiry. The remote - worker never receives the App private key. - - Invoke `gh auth token --hostname --user ` only through the - account-pinned trusted-local provider when no App installation mapping - applies, after removing `GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, - and `GITHUB_ENTERPRISE_TOKEN`; fail if the configured account cannot be - resolved. Once App selection begins, every configuration, minting, access, - rate-limit, or service failure is terminal and never retries as the user. - Deliver the selected token through an ephemeral helper channel to the - one-shot Git acquisition process, never a URL, argument, repository config, - durable Task or command record, or later-phase environment. Tear down the - helper and release token references before publishing the workspace. - - Launch an external materializer through the configured OCI runner or sandbox - as a supervisor-owned resource labeled by worker, attempt, lease, and fence, - with only schema-validated and resource-authorized inputs, its source - credentials, allowed egress, and a private host-owned staging mount under the - final publication root. Never expose gateway, provider, backend, - final-workspace roots, or the runner control socket. Treat the digest-pinned - image as operator-trusted deployment code. Verify the versioned output - manifest, request-pinned digest, content, immutable identities, and - verification-method labels. Terminate the acquisition process, credential - scope, mounts, and runner resource while retaining the validated host-owned - tree; prove that boundary gone before an atomic same-filesystem rename, - setup, and baseline. No copy fallback exists. After bounded termination - escalation, prove the complete invocation process set empty; if proof fails, - persist termination unknown/failed, poison admission, and exit so the - supervisor destroys the boundary. Replacement readiness enumerates and - destroys or quarantines orphan runner resources, credential/staging mounts, - and roots. Use separate phase environments, budgets, and descriptor-safe - evidence; clean in `finally`. -- **Execution note:** Characterize every phase with a fake adapter, fake - registered materializer, local OCI registry, malicious fixtures, and - disposable Git servers before real providers or private artifact systems. - Fault-inject dispatch acknowledgement, events, leases, every acquisition - mode, manifest publication, processes, evidence publication, and cleanup. -- **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, `tests/helpers/env.ts` for isolated state, and Buzz's bounded process-group/job-object cancellation as a lifecycle characterization checklist rather than copied code. -- **Test scenarios:** - - Covers AE5. Every repository checkout matches its requested full object ID - and tree and destinations are disjoint; wrong/missing objects, disallowed - repository/namespace/URL/host/address/port, credential-bearing URLs, - redirects, DNS rebinding, unsafe subdirectories, fetch failure, and setup - failure stop before adapter invocation. After worker acceptance each - materialization/setup failure emits the selected - `Submitted -> Working -> Failed` public trace. - - Repositories with LFS configuration/pointers, submodules, hooks, filters, - alternates, proxy/helper config, or non-HTTPS secondary protocols cause no - secondary connection or execution of repository, user, or system helpers; - only the KTD16-selected one-shot credential channel can run. - - Source credentials leave no repository config, process argument, child - phase environment, log, error, evidence, or retained workspace trace. - - GitHub credential resolution uses only the trusted operator - repository-to-installation mapping and never auth-app discovery. A remote - worker request contains only active attempt/fence; the fake controller - derives profile/provider/mapping/installation/repository/operation/route, - rechecks command revision, lease epoch, tombstone, and fence, and returns - one authenticated single-use non-durable lease response. Wrong worker, - replay, duplicate consumption, substituted repository/provider/operation, - stale command state, or controller/minter configuration-digest disagreement - fails before token delivery. The token carries only repository, - read-only-contents, and GitHub-expiry scope; the lease carries worker, - attempt, lease epoch, command revision, fence, operation, and delivery - expiry. The worker never receives the App private key. - - Auth-app is called with `refresh: true` for each acquisition. A cached - near-expiry token is bypassed, remaining lifetime must exceed acquisition - deadline plus clock-skew margin, lease expiry is capped by token expiry, and - an acquisition ceiling that can exceed a fresh token's safe lifetime fails - readiness. Every boundary failure remains terminal without invoking `gh`. - - A trusted-local profile invokes its fake CLI only when no installation - mapping applies, pins `--hostname --user `, removes all four - ambient GitHub token variables, and fails on account mismatch. Remote - profiles cannot select it. GitHub Enterprise Server works only through an - explicit host/API mapping. Provider ID, host, installation/account, and - selection reason appear only in operator provenance, never public detail. - - Table-driven failures assert the exact public safe code, reason, and - retryability for no provider, installation/repository denial, App - configuration/authentication/mint failure, rate limit, service outage, and - trusted-local CLI failure. No selected-App case invokes the CLI. - - OCI tags, foreign/external URLs, cross-origin credential forwarding, - disallowed registry/auth/blob host/address/port, redirects, DNS rebinding, - manifest/layer mismatches, unsafe layers, missing workspace manifests, and - expansion-limit violations fail before publication. A valid digest-pinned - snapshot produces the same manifest contract as direct Git. - - Unknown, profile-disallowed, unpinned, or definition-drifted materializers; - unauthorized structured-input resources; gateway/worker descriptor - mismatch; missing expected workspace-manifest digest; schema-invalid - inputs; undeclared egress; malformed output manifests; and mismatched - expected/reported repository or output identities fail before setup. The - request cannot select an image or command. - - Materializer credential environments/mounts, process state, runner control - plane, and staging mounts are inaccessible to later phases. Literal secret - canaries in output or retained logs fail publication. This verifies phase - teardown, not safety from a malicious operator-registered image that - intentionally transforms a credential. Provenance labels its unverified - source assertions as materializer-attested. - - A cache hit occurs only after current authorization and revalidates content - plus the same owner or authorization scope, revocation epoch, canonical - source, materializer-definition, expected-output, actual output-manifest, - trust domain, and current App entitlement generation. Authenticated webhook - events and bounded reconciliation for uninstall, suspension, and repository - selection advance the generation; unknown, stale, or mismatched state - rejects reuse. Hit provenance records `cache_hit`, original acquisition - provider metadata, and current policy selection/entitlement separately, - including a hit after provider-policy change, and performs no mint. - - Setup changes establish the baseline; setup and checks receive no provider/control secrets. Credentialed provider runtimes and model tools run across the declared OS UID/process/mount boundary or broker, with disjoint config/data roots and ambient selectors removed. - - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. - - Block dispatch after effect selection, complete a newer cancel for the unseen attempt, then release dispatch: the command tombstone/revision check rejects it before workspace or provider creation. Duplicate commands remain idempotent and all effects stay fence-bound. - - Covers AE12. A descendant calls `setsid`, ignores graceful signals, and survives per-process-group escalation during cancellation and normal completion; the worker records termination unknown/failed, refuses another reservation, exits, and replacement readiness reaps or quarantines the orphaned root after supervisor boundary destruction. - - Covers AE14. Concurrency above one and hostile/cross-tenant trust claims are rejected. Credentialed readiness fails without the narrow OS provider/tool boundary, and model tools cannot inspect provider/control process environments, procfs/process listings, or configured backend roots. - - Source pack/tree/file/inode/path/sparse-file/disk limits and setup/provider/check CPU, memory, PID, network, phase-time, and workspace limits stop only the invocation; failed quiescence recycles the worker rather than claiming it remains healthy. - - Covers AE9. Known permissions receive one-invocation decisions; prompt-required profiles fail startup; unknown permission types fail the adapter. - - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps. A valid or invalid result selected before a later check/evidence failure is preserved, including the valid Artifact; only a pre-candidate failure records `not_produced`. - - Background swap attacks, links, mount crossings, FIFOs/devices/sockets, unstable files, and tampering between worker staging and gateway publication never expose external bytes or partial Artifacts. - - SIGKILL during materialization and before or after provider spawn proves - supervisor-owned runner/process death, credential/staging mount removal, - and replacement root recovery before readiness; unresolved resources keep - readiness false, and the gateway retains one failed Task with separate - termination and cleanup outcomes. - - Cross-filesystem staging/publication configuration fails readiness. Faults - around the final rename expose either no final workspace or the complete - validated tree, never a copy fallback or partial publication. -- **Verification:** A built supervised worker, authoritative fake lease - controller, and fake central minter materialize equivalent workspaces through - disposable exact-SHA repositories, a local digest-pinned OCI snapshot, and a - fake registered materializer; validate one standard manifest; mutate each - through the fake adapter; and prove authenticated revisioned dispatch, - unseen-cancel tombstones, deterministic App-before-account-pinned-CLI - eligibility, remote App private-key exclusion, controller-derived single-use - lease delivery, repository/read/expiry-only token scope, replay/substitution/ - stale-state/config-digest rejection, fresh-token lifetime boundaries, - entitlement-generation cache revocation and truthful hit provenance, exact - source-auth mappings, no fallback after selected-App failure, acquisition - hardening and credential teardown, budgets, result preservation, quiescence - or poisoned-boundary exit, evidence integrity, worker-crash containment, - orphan-root handling, and cleanup. +### U1. Workspace, extension, and manifest contracts + +- **Goal:** Freeze configuration additions and all versioned public/private data + contracts before runtime implementation. +- **Requirements:** R1, R2, R3, R6, R7, R8, R9, R18; AE3, AE4, AE5, AE7, + AE8, AE9, AE13, AE16, AE20; KTD1, KTD2, KTD5, KTD6. +- **Files:** `src/models/workspace-config.ts`, schema generation tests and + generated public schemas, `packages/execution-service/src/contracts/*`, + `packages/execution-service/tests/unit/contracts/*`, configuration docs. +- **Approach:** Add strict named `workspaceSnapshots` and nested profile-client + `gateway.expose`; preserve project/user scope and reserve built-in IDs. Define + activation on every profiled operation, the extension-owned request envelope, + other-metadata behavior, exact result-schema grammar, source union, deadline, + integrity and produced Artifacts, stable failures, workspace manifest, and + canonical digest preimages from Zod. +- **Execution note:** Start with fixtures that reject missing activation, cross- + variant/unknown extension fields, extra Message Parts, undeclared names, + mutable snapshot references, malformed digests, invalid deadlines, exposure + without launcher, built-in collisions, and unsupported clients while + preserving unrelated metadata. +- **Verification:** Focused workspace-schema and contract tests; generated schema + drift check; representative YAML and wire examples parse through runtime + schemas; canonicalization and Artifact-cardinality fixtures pass. + +### U2. Deployment-wide Task store and A2A server + +- **Goal:** Serve the A2A lifecycle without application authentication and keep + durable deployment-wide Task/idempotency truth. +- **Requirements:** R1, R2, R3, R4, R5, R8, R16, R17, R18; AE1, AE2, AE9, + AE11, AE14, AE15, AE16, AE19, AE20; KTD1, KTD2, KTD3, KTD4, KTD10. +- **Files:** execution-service Task repository, Agent Card, request handler, + server, pagination/retention, CLI gateway command, and focused tests. +- **Approach:** Implement flags/env precedence, private link-safe project state + and lock, loopback default, explicit `0.0.0.0`, startup integrity/ + reconciliation, per-operation extension negotiation, durable atomic + claim+Task creation, monotonic terminal settlement, bounded events/Artifacts, + no early eviction, atomic expiry, global listing/cancellation, deadline + handling, and fail-closed graceful shutdown. +- **Execution note:** Prove with the official A2A client that one caller can read + and cancel another caller's Task; this is expected behavior. Fault-inject + unsafe state paths plus open/write/rename/fsync boundaries before + acknowledgment, cancellation intent, Artifact, and terminal settlement. +- **Verification:** A2A discovery/send/stream/get/list/subscribe/cancel/replay/ + expiry integration tests on loopback and `0.0.0.0`; state-path, retained- + capacity, store-fault, competing-lock, deadline, shutdown, and restart tests. + +### U3. Git and OCI workspace acquisition + +- **Goal:** Materialize declared repository sets and named OCI snapshots into the + same validated invocation workspace contract. +- **Requirements:** R6, R9, R10, R11, R12, R15, R16, R18; AE5, AE6, AE7, + AE8, AE10, AE15, AE17, AE18; KTD6, KTD7, KTD8, KTD11. +- **Files:** acquisition coordinator, Git transport, GitHub provider selection, + OCI client/extractor, workspace-manifest validator, staging/publication helper, + fixtures and tests. +- **Approach:** Resolve name-based source requests from project workspace. + Implement hermetic Git and full-commit verification. Classify App + applicability as eligible/ineligible/unknown, mint a fresh token with adequate + lifetime, permit `gh` only for positive ineligibility, and use temporary + credential helpers. Pull digest-pinned OCI manifests from declared + repositories, validate every layer and extraction boundary, validate the + expected workspace-manifest digest, and publish atomically. Tear down every + acquisition credential before typed preparation. +- **Execution note:** Use local Git remotes and a local OCI test registry/fixture; + prove unknown/selected-App failures do not call `gh`, token lifetime is + enforced, and snapshot failures never invoke Git fallback. +- **Verification:** Focused three-way provider-selection tests, Git integration + tests including branch/tag resolution, OCI digest/path/limit tests, credential + leak scans, and equivalent manifest output across both acquisition modes. + +### U4. Invocation supervisor and backend contract + +- **Goal:** Run one invocation through acquisition, adapter execution, evidence, + cancellation, descendant termination, and cleanup with truthful terminal + outcomes. +- **Requirements:** R3, R5, R8, R13, R14, R15, R16; AE9, AE10, AE11, AE12, + AE14, AE15, AE16, AE17, AE18; KTD3, KTD9, KTD10, KTD11, KTD12. +- **Files:** backend interface/registry, typed preparation, provider/MCP/tool + secret-view compiler, invocation state machine, platform containment, evidence + collector, result validator, cleanup/reaper, fake adapter, and lifecycle tests. +- **Approach:** Resolve targets to typed adapter context; never use generated + launchers or workspace setup commands. Project validated configuration through + deterministic transforms. Give provider control, each MCP child, and model + tools separate minimal views; enforce deadlines; track the non-escapable + containment set; preserve result states; collect evidence through safe reads; + and require quiescence before cleanup success. Startup reconciles Tasks and + containment before roots. Unmanaged poisoned mode continues reaping; managed + exit proves cleanup ownership transfer. +- **Execution note:** Build the fake adapter first and fault-inject every + boundary: cancellation/terminal races, deadline, shutdown, child escape, + cross-scope provider/MCP/tool secret reads, unsafe state reads, output + truncation, malicious evidence, valid-result-then-evidence-failure, unknown + cleanup, and manager handoff. +- **Verification:** Deterministic lifecycle, containment, separate-secret-view, + preparation, and evidence tests plus one real child-process smoke fixture per + supported platform strategy. ### U5. Codex backend adapter -- **Goal:** Run Codex directly through its supported TypeScript SDK while preserving structured progress, validated output, usage, file-change evidence, cancellation, and runtime identity. -- **Requirements:** R10-R22; AE1, AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. -- **Dependencies:** U4. -- **Files:** `packages/execution-service/src/worker/adapters/codex.ts`, `packages/execution-service/tests/unit/worker/adapters/codex.test.ts`, `packages/execution-service/tests/fixtures/execution/codex-events.jsonl`. -- **Approach:** Depend directly on pinned `@openai/codex-sdk` and fail readiness when the runtime or configured credential boundary is unavailable. Create one fresh SDK thread with isolated `CODEX_HOME`. The credential-bearing Codex runtime runs on the provider side of the declared UID/process/mount boundary or obtains credentials through the configured broker; model-invoked commands run on the tool side and cannot inspect provider procfs/process entries or config/data roots. A minimal allowlisted environment and pinned `shell_environment_policy` remain defense in depth. Apply profile model/sandbox/network/approval/path policy, pass `AbortSignal` and exact `outputSchema`, validate final JSON with the shared validator, normalize bounded events/evidence, and never resume or pool threads. Once validation selects `valid` or `invalid`, later check/evidence/infrastructure failure preserves that state and the valid Artifact. -- **Execution note:** Wrap the SDK behind an injectable factory and drive it through fixture events and its executable override before any credentialed smoke test. Use Promptfoo's provider and tests to enumerate observable edge cases, not as copied code or a runtime dependency. -- **Patterns to follow:** `src/core/profile/adapters/codex.ts` for root/environment isolation, `src/core/native/codex.ts` for version checks, the SDK's `startThread`/`runStreamed`/`AbortSignal`/`outputSchema` and shell-environment policy contracts, and Promptfoo's Codex provider tests for characterization of option forwarding, environment isolation, cancellation, structured output, and cleanup. -- **Test scenarios:** - - A successful stream exposes thread ID, progress, final response, token usage, native file-change items, and terminal completion. - - A structured request forwards the exact accepted schema to `outputSchema`; valid JSON selects `valid` and produces the fixed-name `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; malformed or schema-invalid output selects `invalid` without the Artifact and reports the typed validation error; missing output selects `not_produced` and reports the typed missing-output error. - - Empty final response, turn failure, malformed JSONL, non-zero exit, unavailable runtime, and usage omission map to typed result/completeness fields. - - Covers AE6/AE12. A pre-aborted signal prevents start; in-flight cancellation aborts the SDK once; worker escalation proves descendant termination; completion after cancel or stale fence cannot alter the selected terminal outcome. - - The credential-bearing Codex runtime receives only its scoped credential and minimal environment. Adversarial model commands probing parent/sibling environments, `/proc` and process listings, known or discovered `CODEX_HOME`/backend roots, and outbound secret exfiltration cannot recover provider/control credentials; readiness fails when this OS boundary or broker is unavailable. - - Two sequential invocations create fresh threads with disjoint `CODEX_HOME`, session state, and writable roots; no resume or thread-persistence API is called. - - Native diffs and shared Git evidence coexist without claiming identical attribution. -- **Verification:** Fixture-driven tests cover every supported event/failure shape, SDK option/schema/signal forwarding, OS-enforced provider/tool separation plus environment defense in depth, fresh-thread behavior, result-state preservation, and cleanup escalation, followed by an isolated credentialed repository smoke test when prerequisites are available. +- **Goal:** Run built-in and profile-backed Codex targets through the supported + SDK while preserving structured progress, result, usage, cancellation, and + native evidence. +- **Requirements:** R7, R8, R13, R14, R15, R16; AE1, AE3, AE4, AE10, AE12, + AE15, AE17, AE18; KTD9, KTD11, KTD12. +- **Files:** Codex adapter, profile-context and auth bridge, fixtures, + conformance and optional credentialed smoke tests. +- **Approach:** Pin the SDK; create one fresh thread per Task; pass cwd, typed + profile configuration, abort signal, optional output schema, and the private + Codex control-process auth view inside containment. Keep Codex-invoked tools + outside that auth view; normalize events/usage; bound evidence; dispose fully. +- **Execution note:** Characterize the pinned SDK and its tool-sandbox/auth + separation with captured fixtures before implementing normalization. Do not + import Promptfoo provider code. +- **Verification:** Shared adapter conformance, deadline, auth-isolation, and + tool-secret-denial fixtures plus an opt-in credentialed smoke case. ### U6. Pi backend adapter -- **Goal:** Run Pi through strict RPC mode while preserving settled completion, schema-backed terminal output, usage/cost, tool progress, cancellation, and process cleanup. -- **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. -- **Dependencies:** U4, U5. -- **Files:** `packages/execution-service/src/worker/adapters/pi.ts`, `packages/execution-service/src/worker/adapters/pi-rpc.ts`, `packages/execution-service/src/worker/adapters/pi-policy-extension.ts`, `packages/execution-service/tests/unit/worker/adapters/pi.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-rpc.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-policy-extension.test.ts`, `packages/execution-service/tests/fixtures/execution/pi-events.jsonl`. -- **Approach:** Spawn supported Pi 0.85.x in strict RPC mode with an invocation-local `PI_CODING_AGENT_DIR`, no sessions/extensions/built-ins, and one explicit worker-owned policy extension. The credential store and provider runtime stay on the provider side of the configured UID/process/mount boundary or credential broker; policy/command tools run on the tool side and cannot inspect the provider process, procfs entries, or Pi config/data roots. Verify exact tool inventory before work. Implement bounded LF JSONL, correlation, settlement/stats, abort and escalation. For a structured request, generate the terminating tool from the accepted schema; the first call atomically claims and validates the candidate, later calls cannot replace it, and later checks/evidence failures preserve its `valid` or `invalid` state and valid Artifact. -- **Execution note:** Build parser, extension/tool-inventory, terminating-tool, policy-tool, and state-machine tests from captured RPC fixtures before process integration. Reuse the contract established by U5 rather than adding Pi-shaped public fields. -- **Patterns to follow:** `src/core/native/pi.ts` for version/trust checks, `src/core/profile/adapters/pi.ts` for root isolation, and the official Pi RPC framing, `--no-extensions` plus explicit `--extension`, `--no-builtin-tools`, custom-tool, credential-store, and cancellation contracts. -- **Test scenarios:** - - Successful prompt acceptance streams message/tool events, stops on `agent_settled`, retrieves final messages/stats, and reports session ID, usage, and cost. - - A structured request exposes only the invocation-scoped terminating tool in addition to the policy tools. The first observed call claims the candidate; valid arguments select `valid` and produce the fixed-name `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; an invalid first call selects `invalid` without replacement or an Artifact; a later duplicate cannot replace the result; later check/evidence/infrastructure failure preserves the selected state and valid Artifact; a cancel/deadline/fence that wins first suppresses the call; and settled completion without a call selects `not_produced` as missing output. - - LF framing preserves `U+2028`/`U+2029` inside JSON strings, accepts CRLF by stripping trailing CR, handles partial/multiple chunks, and rejects oversized/malformed records. - - Covers AE6/AE12. Cancellation sends RPC abort once, waits for idle, then terminates the process group only after grace; late settled or terminating-tool events cannot overwrite the terminal fence. - - Prompt rejection, agent error, aborted stop reason, retry/compaction sequence, premature exit, stderr overflow, and stats failure map truthfully. - - Sequential invocations have disjoint `PI_CODING_AGENT_DIR`, tool registration, and session state. Adversarial policy/command tools probing parent/sibling environments, procfs/process listings, known or discovered Pi/backend roots, and outbound secret exfiltration cannot recover provider/control credentials; readiness fails without the OS boundary or broker. Repository `.pi/extensions` and unrestricted built-ins do not load, and caller input cannot override provider/model or issue arbitrary RPC/extension commands. -- **Verification:** Fixture and fake-process tests prove framing, correlation, terminating-tool selection, shared validation, exact tool inventory, OS-enforced provider/tool separation plus environment defense in depth, result-state preservation, settlement, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when prerequisites are available. - -### U7. Production registry, service packaging, and observability - -- **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with trusted transports, peer identity, credential-boundary and supervisor readiness, safe startup/shutdown, tracing, and reproducible containers. -- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD10-KTD16. -- **Dependencies:** U3-U6. -- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/gateway/github-app-webhook.ts`, `packages/execution-service/src/gateway/github-app-reconciler.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/gateway/github-app-webhook.test.ts`, `packages/execution-service/tests/unit/gateway/github-app-reconciler.test.ts`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/unit/worker/materializers/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. -- **Approach:** Register only Codex and Pi as backend adapters and register the - built-in Git/OCI materializers plus configured external materializers through - a separate closed registry. Add gateway and supervised worker entrypoints - inside the private Node 22 workspace. Register source credentials separately: - an authoritative gateway/control-plane lease controller, a trusted GitHub App - minter using focused `@octokit/auth-app`, its authenticated single-use - non-durable worker client, and an account-pinned GitHub CLI provider only on - trusted-local acquisition hosts. Wire authenticated GitHub App lifecycle - webhooks and bounded reconciliation to the durable entitlement-generation - store. Reject duplicate or ambiguous provider IDs, implicit enterprise host - detection, unpinned CLI accounts, ambient GitHub token variables, remote CLI - fallback, worker-side App private-key handles, fallback-on-error policy, and - provider/mapping configuration-digest disagreement. - Before readiness, validate named public TLS termination, every remote - worker's mTLS/equivalent transport and pinned identity/capabilities, the - complete central lease-controller/minter path for every remote App profile, - single-use grant consumption, fresh-token lifetime versus acquisition ceiling - and clock skew, current entitlement-generation authority, Unix-socket - locality, store, runtimes, matching gateway/worker materializer definition - digests, image digests, schemas, credential names, egress, limits, OCI - runner/sandbox isolation, monotonic command storage, acquisition and - provider/tool credential-boundary capabilities, supervisor boundary, orphan - roots, trust, quotas, and resource controls. Propagate `traceparent`, then - apply KTD10's small shared metadata allowlist and bounded filtering/redaction - before any structured log/span processor or OTLP exporter; neither - OpenInference nor backend-native attributes bypass it. Build a minimal - gateway/control-plane image with the lease controller and App minter but no - provider runtime, writable repository, or baked-in private key, and a - one-execution worker image whose init kills the complete boundary when the - worker server exits, including poisoned failed-quiescence exit. -- **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. -- **Patterns to follow:** `src/core/profile/adapters/registry.ts` for explicit adapter composition, root package scripts for workspace delegation, `src/core/mcp-http-stdio-proxy.ts` for server lifecycle, `.github/workflows/ci.yml` for quality gates, and `.github/workflows/publish.yml` for immutable releases. -- **Test scenarios:** - - Registry exposes exactly Codex and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects OpenCode or unknown backend IDs before workspace creation. - - Materializer registry exposes built-in Git and OCI plus only configured - external IDs, resolves every external image to the configured digest, - rejects duplicates/tags/unknown IDs, and cannot be influenced by request - image, command, credential, or policy fields. - - Source-credential registry maps `github.com` and explicit enterprise - host/API pairs, selects only an operator-mapped App installation, and - permits GitHub CLI only for an account-pinned trusted-local no-mapping case. - The CLI invocation includes `--user` and no ambient GitHub token variables. - No request field can alter provider selection. Public output contains only - safe code/reason/retryability; operator provenance contains the non-secret - provider and installation/account identity. - - Remote App profiles fail readiness without the central minter and - authoritative lease controller, entitlement-generation webhook/ - reconciliation authority, safe fresh-token lifetime policy, single-use - grant support, matching provider/mapping configuration digest, or with an - App private-key handle in worker configuration. Runtime requests by active - attempt/fence derive every provider/repository/route field from current - durable state and reject replay, substitution, stale state, and duplicate - consumption. The same-host trusted case and a separately deployed minter - pass the same contract; neither uses snapshot delivery. - - Readiness rejects an acquisition ceiling that can exceed a fresh token's - safe lifetime. Near-expiry auth-app cache output is bypassed with - `refresh: true`, lease expiry is capped by token expiry, and failure never - falls through to the CLI. - - Gateway and supervised worker start from built outputs, become ready only after trusted transport/identity, credential and supervisor boundaries, dependencies, and orphan recovery pass, and stop gracefully on SIGTERM. - - Gateway readiness fails for malformed auth, missing/mismatched named TLS termination, plaintext production public ingress, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. - - Worker-route readiness fails for plaintext remote URL, wrong/untrusted certificate, worker identity/capability mismatch, or replayed capability; mTLS/equivalent authenticated encryption and same-host Unix sockets pass. - - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable OS credential/supervisor/resource enforcement, unproved or unrecoverable orphan roots, or unavailable/incompatible Codex or Pi runtime. - - Worker readiness fails when a profile allows a materializer absent from its - route, gateway and worker definition digests differ, an external image is - not digest-pinned, an input schema or output manifest version is - unsupported, named credentials or the OCI runner/sandbox are unavailable, - the runner control plane would be visible to the workspace, or configured - egress and resource enforcement cannot be provided. - - Killing or poisoning the worker server while an adapter child and invocation root exist makes the supervisor destroy the boundary; replacement readiness waits for root deletion/quarantine and never reuses it. - - Trace context crosses the authenticated private call and correlates result identities using only opaque owner correlation. Exporter probes for agent, model, tool, stale-event, and error spans contain allowlisted bounded metadata but no canary secret, prompt/output, tool argument/result, file body/source fragment, raw caller identity, or cross-owner fragment; exporter failure cannot change Task status. - - Gateway/control-plane images contain no Codex, Pi, Git workspace, coding - provider credentials, or baked-in GitHub App private key; the App key enters - only through its configured secret handle. Worker images and configuration - contain neither App issuer material nor user credential stores, pin both - coding runtimes, enforce provider/tool UID/process/mount separation or the - credential broker, confine one workspace/config root, disable repository Pi - extensions and unrestricted built-ins, enforce deployment limits, and - complete fake-provider security probes. - - Installing the root npm package on Node 18 does not load service dependencies; the private service workspace and containers enforce Node 22.19+. -- **Verification:** The registries dispatch both adapters and - source-credential providers through their respective contracts; built - services and images prove controller-authorized fresh App minting without - worker issuer material, single-use lease replay/staleness/config-digest - rejection, entitlement-generation webhook/reconciliation behavior, - account-pinned sanitized CLI eligibility, deterministic public failure - mapping with operator-only identities, and lifecycle/security behavior; - exporter-capture tests prove pre-processor metadata allowlisting, bounded - redaction, opaque owner correlation, and canary/cross-owner exclusion across - agent, model, tool, stale-event, and error spans; CI and publication bind - immutable image tags to the release commit. - -### U8. Cross-backend conformance, documentation, and release evidence - -- **Goal:** Prove standard A2A extension carriers, retained replay, trusted transport, monotonic cancellation, truthful result preservation, credential separation, metadata-only telemetry, worker recycling, trace-order conformance, and the shared backend contract end to end without leaking backend details into callers. -- **Requirements:** R1-R22; F1-F5; AE1-AE14. -- **Dependencies:** U1-U7. -- **Files:** `packages/execution-service/tests/e2e/execution-gateway.test.ts`, `packages/execution-service/tests/fixtures/execution/conformance-cases.ts`, `examples/gateway/gateway.yaml`, `examples/gateway/worker.yaml`, `docs/src/content/docs/guides/execution-gateway.mdx`, `docs/src/content/docs/reference/execution-gateway-configuration.mdx`, `README.md`, `CHANGELOG.md`. -- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases, with gateway and supervised worker as separate processes. Each race fixture declares attempt/fence correlation, required durable transitions and observed effects, required happens-before edges, maximum occurrence counts, and effects forbidden after terminalization. A deliberately small test-side checker evaluates those constraints against durable records plus observed worker/process outcomes without calling the production selector. It remains coverage protection—not TLA+, a model checker, event sourcing, or a second lifecycle implementation. Document the exact extension URI and legal Agent Card/header/Message/Artifact carriers, both fixed Artifacts and four result states, retained-replay ordering, worker transport/identity, monotonic command tombstones, failed-quiescence recycling, the narrow OS credential boundary, reviewed-domain limitation, metadata-only telemetry and its separate operator access/retention, storage/HA limits, lack of execution resume, source hardening, quotas, retention, and operations. - Document all three source modes, the exact source discriminator, canonical - repository/namespace and materializer-input authorization, multi-repository - destinations, the standard workspace manifest and verification-method - labels, materializer registration, worker-derived definition digests, - profile allowlisting, digest/profile/idempotency boundaries, same-filesystem - publication, supervisor-owned runner cleanup, acquisition credential - teardown, authorization-scoped cache reuse/revocation, source-mode - capabilities, and the prohibition on caller-supplied acquisition code. - Document normalized host/API mapping, operator-owned - repository-to-installation mapping, GitHub App precedence and - repository/read/expiry token scope, account-pinned sanitized trusted-local - CLI eligibility, fail-closed selected-App behavior, focused - `@octokit/auth-app` ownership and `refresh: true`, central App private-key - custody, controller-derived single-use remote lease bindings, token/lease - lifetime rules, entitlement-generation webhooks/reconciliation and - cache-hit provenance, deterministic public source-auth mapping with - operator-only identities, remote readiness requirements, the one initial - token-minter path, and deferred versioned snapshot delivery. -- **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. -- **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, Starlight guide/reference organization under `docs/src/content/docs/`, and Buzz's required-critical-action coverage rule without importing its TLA+ model or production implementation. -- **Test scenarios:** - - Covers AE1-AE14 through built services with a fake backend and official A2A client. - - Agent Card required-extension advertisement, `A2A-Extensions`, `Message.extensions`, request `Message.metadata[uri]`, and the single fixed integrity Artifact carrier interoperate; missing/mismatched carriers and `Task.extensions` fail. - - Identical retained replay after deadline expiry, quota exhaustion, readiness loss, authorization change, or profile replacement returns the original Task; changed request/schema or inconsistent original bindings conflict. - - The same accepted schema, valid result, invalid result, missing result, and pre-output failure pass through Codex and Pi with identical decisions. A valid-result-then-check-failure and invalid-result-then-evidence-failure preserve the selected state and only the valid Artifact; `not_produced` remains pre-candidate only. - - Direct Git object/destination/authorization mismatch, OCI - digest/manifest/namespace mismatch, registered materializer - descriptor/input/resource/expected-output mismatch, direct known-secret - disclosure, and setup failure after worker acceptance produce the selected - `Submitted -> Working -> Failed` trace; provider invocation never begins, - and durable snapshots, streams, and conformance records agree. A policy - revocation, webhook-advanced entitlement generation, reconciliation result, - or unknown/stale App state before lookup cannot consume a previously - populated cache entry. A valid hit after provider-policy change records - `cache_hit`, original acquisition provider, and current selection separately - without minting. - - GitHub source cases prove operator-mapped App selection, account-pinned - sanitized trusted-local CLI selection only when no mapping applies, no CLI - invocation after any selected-App failure, explicit enterprise host/API - mapping, repository/read-only/expiry-only token narrowing, and an - authenticated controller-derived single-use lease carrying worker identity, - attempt, lease epoch, command revision, fence, repository, operation, and - expiry without worker issuer material. They reject replay, substitution, - stale state, duplicate grant consumption, and controller/minter - configuration-digest disagreement; bypass near-expiry auth-app cache output - with `refresh: true`; cap lease expiry by token expiry; fail readiness for an - unsafe acquisition ceiling; assert every safe source-auth - code/reason/retryability tuple and operator-only identity detail; and - publish a credential-free workspace. - - Pause dispatch after selection, complete unseen-attempt cancel, then release dispatch; the stale command creates no workspace/process. The small independent checker enforces each fixture's attempt/fence correlation, happens-before edges, maximum counts, and forbidden post-terminal effects. Deliberately bad traces that still contain every required action name fail for wrong order, wrong fence, duplicate-over-maximum effects, and an extra stale dispatch after terminalization. - - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; one worker serializes admitted work. - - Gateway restart, reconnect, ambiguous dispatch, duplicate/out-of-order - commands/events, worker crash, lease expiry, cancellation, provider failure, - evidence truncation, logical expiry, and cleanup failure preserve one - truthful terminal outcome without provider reattachment or replay. - - SIGKILL during external materialization and before or after provider spawn - forces supervisor-owned runner/process death, credential/staging mount - removal, and replacement orphan recovery before readiness. A child that - calls `setsid` and ignores graceful signals forces termination - unknown/failed, poisoned-worker exit, supervisor boundary destruction, and - replacement orphan recovery; no poisoned worker accepts a next reservation. - - Public plaintext, wrong TLS boundary, private plaintext, wrong certificate/worker identity, and capability replay fail readiness/dispatch; configured TLS, mTLS/equivalent overlay, and same-host Unix socket cases pass. - - Both adapters block model-tool probes of parent/sibling environments, procfs/process listings, known/discovered backend roots, and network secret exfiltration under the OS credential boundary. Environment filtering alone is never accepted as proof, and hostile-source/cross-tenant claims remain rejected. - - End-to-end exporter capture repeats the agent/model/tool/stale-event/error canary and cross-owner probes, proving only bounded allowlisted metadata and opaque owner correlation cross the telemetry boundary while Task/Artifact access and retention remain independent. - - Redirect/DNS-rebinding and unauthorized-resource cases cover every Git and - OCI registry/auth/manifest/blob connection. Secondary Git fetch, OCI tag, - foreign/external layer URL, cross-origin credential forwarding, - layer/manifest mismatch, unregistered or unpinned materializer, undeclared - materializer egress, malicious output manifest, resource exhaustion, - malicious file types/link swaps, repository Pi extensions, and unrestricted - built-ins remain blocked within the documented reviewed-source boundary. - - Same-filesystem publication succeeds by atomic rename; a cross-filesystem - staging root fails readiness and fault injection never observes a partial - final tree or copy fallback. Provenance distinguishes worker-verified and - trusted-service identities from materializer-attested claims. - - Examples validate with production schemas and use only secret variable - names. Docs state the three source modes and exact discriminator, standard - workspace manifest and provenance labels, materializer registry/profile - boundary, worker-derived definition digest, resource authorization and - entitlement-generation cache-revocation boundary, cache-hit/original/current - provider provenance, prohibition on caller-supplied acquisition code, - one selected remote token-minter/lease path with deferred snapshot delivery, - account-pinned sanitized CLI invocation, token versus lease scope, fresh - token and readiness lifetime rules, deterministic source-auth mapping, - same-filesystem publication, supervisor-owned runner cleanup, acquisition - credential teardown, two transport boundaries, one gateway replica, one - execution per worker, reviewed trust domain, narrow credential isolation - versus deferred hostile-code isolation, metadata-only telemetry with its - fixed pre-processor allowlist and separate operator access/retention, - runtime floors, and ephemeral provider sessions. - - Opt-in real-provider smoke tests record backend/runtime and credential-boundary prerequisites, skipping only when a named prerequisite is absent. -- **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor; full suites and docs pass; the official A2A client exercises every advertised operation including the `Submitted -> Working -> Failed` source/setup path; exporter capture proves the telemetry canary/cross-owner contract; the independent checker rejects all-name-present traces with wrong order/fence/multiplicity or forbidden stale dispatch; and release evidence records each available real backend plus explicit skipped prerequisites. +- **Goal:** Run built-in and profile-backed Pi targets through strict RPC with the + same public lifecycle and honest capability reporting. +- **Requirements:** R7, R8, R13, R14, R15, R16; AE3, AE4, AE10, AE12, AE15, + AE17, AE18; KTD9, KTD11, KTD12. +- **Files:** Pi adapter, RPC parser, restricted policy extension, profile-context + and auth bridge, fixtures, conformance and optional credentialed smoke tests. +- **Approach:** Launch Pi with typed invocation configuration, its private + control-process auth view, strict JSONL RPC, explicit allowed tools/extensions, + per-MCP secret views, deterministic permissions, event validation, deadline/ + cancellation escalation, and settled completion. Pi-invoked tools receive no + provider or MCP credentials. Repository extensions and unrestricted built-ins + remain disabled. +- **Execution note:** Reuse the adapter contract exactly; record Pi-specific facts + as bounded native evidence rather than public schema branches. +- **Verification:** Shared adapter conformance, malformed/unknown RPC, deadline, + auth/MCP/tool-secret denial, and an opt-in credentialed smoke case. + +### U7. End-to-end delivery and documentation + +- **Goal:** Prove the built CLI and document the trusted-network operating model, + workspace configuration, credentials, sources, and risks. +- **Requirements:** R1-R18; F1-F5; AE1-AE20. +- **Files:** gateway guide/reference, configuration reference, README, CHANGELOG, + real example project/user workspaces, E2E fixtures, release evidence. +- **Approach:** After the final implementation review is resolved, build the CLI; + create project and user workspaces under `/tmp/`; expose Codex/Pi fixture + targets; serve on loopback and `0.0.0.0`; acquire from local Git and OCI + fixtures; run the official A2A client through negotiation, success, replay, + cancellation, deadline, shutdown, restart, and expiry. Document that network + reachability grants full authority and App/OCI secrets are process inputs, not + YAML. +- **Execution note:** The green smoke test must exercise the same built command + and `/tmp/` workspace shape as the recorded red E2E, not a test-only server. +- **Verification:** `bun run build`, focused and full tests, typecheck, lint, docs + build, schema drift check, and exact red/green E2E commands/results recorded in + the PR description. --- @@ -1532,106 +1011,65 @@ docs/src/content/docs/ | Gate | Applies to | Required evidence | |---|---|---| -| Contract generation | U1 | Exact extension URI/carriers, closed workspace source discriminator and manifest, algorithm-qualified materializer/profile/input/output digest preimages and vectors, verification-method vocabulary, authorization-scope/revocation/App-entitlement fields, cache-hit/original/current-provider provenance, worker request limited to active attempt/fence, controller-derived single-use non-durable lease fields and token-versus-lease scope, exact source-auth code/reason/retryability tuples, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | -| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, account-pinned sanitized CLI execution, fresh-token lifetime boundaries, entitlement-generation authorization/cache revocation, provenance states, exact source-auth failures, and cleanup. | -| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, direct Git/OCI/registered materialization, deterministic operator-mapped-App-before-account-pinned-CLI eligibility, central fresh App minting without worker issuer material, controller-derived single-use non-durable lease delivery, repository/read/expiry-only token scope, replay/substitution/stale/config-digest rejection, entitlement-generation cache revocation and hit provenance, exact failure mapping, fail-closed selected-App behavior, worker-derived registry digests, acquisition credential teardown, supervisor-owned materializer runners, same-filesystem atomic publication, workspace-manifest provenance labels, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | -| Backend conformance | U5-U8 | One shared suite passes against Codex and Pi, including the versioned schema subset, four result states, valid/invalid preservation across later failure, integrity Artifact carrier, and structured-result Artifact rule. | -| Credentialed provider smoke | U4-U6, U8 | An available centrally held GitHub App, account-pinned trusted-local `gh` login, operator-trusted registered materializer, and each available coding provider mutate disposable immutable workspaces while provider selection follows policy. App acquisition proves `refresh: true`, minimum remaining lifetime, lease-at-or-before-token expiry, repository/read-only/expiry token scope, and no worker issuer material; local CLI smoke proves `--user` and sanitized ambient token variables. Adversarial later-phase probes cannot directly access acquisition/provider credential environments, mounts, processes, roots, or runner control planes and literal canaries remain absent; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | -| A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes required-extension negotiation and legal carriers, immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, retained replay, cancel races, expiry, and owner isolation without `Task.extensions`. | -| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, exact source-resource authorization, per-connection Git/OCI SSRF and credential-origin controls, operator-mapped GitHub provider eligibility without identity escalation, central issuer-key custody, worker request limited to active attempt/fence, authoritative current-state derivation, single-use lease replay/substitution/staleness/config-digest rejection, repository/read/expiry-only token scope, fresh-token lifetime safety, fail-closed selected-App errors, account-pinned sanitized CLI use, exact public failure mappings with operator-only identities, webhook/reconciliation-driven entitlement cache revocation, truthful cache-hit provenance, digest-pinned registered materializers, schema/manifest validation, acquisition and provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | -| Lifecycle trace conformance | U8 | The small test-side checker, independently of production selectors, validates attempt/fence correlation, required happens-before edges, maximum occurrence counts, and forbidden post-terminal effects against durable records plus observed worker/process outcomes; all-name-present bad traces fail for wrong order/fence/multiplicity and stale post-terminal dispatch. | -| Telemetry safety | U7-U8 | Exporter capture across agent, model, tool, stale-event, and error spans proves the pre-processor allowlist and bounded redaction exclude prompt/output/tool/source/file content, canary secrets, raw identities, and cross-owner fragments while retaining only bounded operational metadata and opaque owner correlation. | -| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build with focused `@octokit/auth-app` and no full Octokit client, gateway/control-plane lease controller and minter plus supervised-worker smoke, entitlement webhook/reconciler, worker issuer-key exclusion, backend/materializer/source-credential registry readiness, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both service container builds pass. | -| Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | -| Documentation | U8 | `bun run docs:build` passes and examples validate against current schemas. | - -The authoritative behavioral proof is the built-process E2E path with the official A2A client and a separately started worker. Unit tests alone do not prove extension carriers, retained-replay ordering, trusted transport, durable aggregation, monotonic worker commands, credential/process isolation, cancellation, boundary recycling, cleanup integration, or telemetry export safety. The deliberately small independent trace checker supplements that path only by rejecting ordering, fence, multiplicity, and post-terminal-effect violations; it is not a production lifecycle model. - ---- +| Workspace schema | U1 | Project/user parsing, strict nested fields, built-in collision rules, generated-schema drift | +| Public contract | U1-U2 | Official A2A client, every-operation activation, metadata preservation, exact request/result/Artifact/error/canonicalization fixtures | +| Trusted-network model | U2, U7 | Loopback and `0.0.0.0`; shared Task visibility/cancellation; docs warning | +| Durable Task lifecycle | U2, U4 | Private safe state paths, lock, durable claim+Task, no early eviction, store faults, races, restart, atomic expiry | +| Repository acquisition | U3 | Declared-name revision resolution, hermetic Git, full commits, three-way App/`gh` eligibility and sub-budget | +| OCI acquisition | U3 | Declared repository, manifest/layer/workspace digests, safe extraction, no fallback | +| Credential and state isolation | U3-U7 | Separate provider/MCP/tool views; teardown; no cross-scope secrets, operator home, or state root | +| Supervisor lifecycle | U4 | Deadline, shutdown, cancellation races, non-escapable containment, unmanaged recovery, manager handoff, stale-root proof | +| Safe evidence | U4-U6 | Descriptor-relative no-follow reads; links/special files/Git indirection rejected; hermetic Git | +| Backend conformance | U4-U6 | Same suite for fake, Codex, and Pi; profile and built-in variants | +| Structured result | U1, U4-U6 | Exact subset and envelope, valid/invalid/not-produced states, Artifact cardinality, no false publication | +| Repository quality | All | Build, focused/full tests, typecheck, lint, schema check, docs build | +| Built CLI E2E | U7 | Recorded red then green built command under `/tmp/`, both sources, auth isolation, replay/cancel/deadline/shutdown/restart | ## Definition of Done ### Global -- Every R1-R22 requirement is implemented or explicitly shown in a passing conformance scenario. -- The exact required extension is advertised and negotiated through standard Agent Card/header/Message/Artifact surfaces; requests live only at `Message.metadata[uri]`, terminal integrity lives only in the fixed integrity Artifact, and no `Task.extensions` exists. -- Codex and Pi pass the same backend conformance suite, schema subset, and validator. Every terminal Task publishes the integrity Artifact; selected `valid`/`invalid` states survive later failures, and only `valid` publishes the separate fixed structured-result Artifact. -- Gateway and supervised worker run as separate Node 22 processes/images; the Node 18 root CLI does not import service dependencies, and the gateway has no provider runtime or writable repository. -- Authentication and bounded parsing precede owner-scoped retained lookup; identical replay uses stored original bindings before mutable admission, while current authorization/profile/readiness/deadline and quota apply only to atomic new claims. -- Production public ingress uses its named TLS boundary, remote worker routes authenticate and encrypt peers with worker identity/capability binding, and same-host Unix sockets are the only non-network alternative; unprotected remote endpoints fail readiness. -- Cancellation/deadlines use monotonic worker command tombstones and one native abort. Stale dispatch cannot create work, and failed quiescence poisons and exits the worker so supervisor destruction and replacement orphan recovery precede new admission. -- Workspace source validation and exact resource authorization, - per-connection direct Git/OCI controls, trusted-policy GitHub provider - resolution with operator-mapped App precedence, account-pinned sanitized - local-only CLI eligibility, no selected-App failure fallback, central App - private-key custody, controller-derived single-use authenticated lease - delivery, repository/read-only/expiry-only token scope, current-command - recheck and replay/substitution/stale/config-digest rejection, fresh-token and - readiness lifetime bounds, deterministic public source-auth mapping with - operator-only identities, authenticated webhook/reconciliation-driven App - entitlement generations, fail-closed unknown/stale cache authorization, and - separate cache-hit/original-acquisition/current-selection provenance are - enforced end to end. The initial remote path is central token minting and - non-durable lease delivery; versioned snapshot delivery remains deferred. - Worker-derived registered-materializer digests, the standard workspace - manifest and truthful provenance labels, same-filesystem atomic publication, - supervisor-owned runner cleanup, acquisition credential teardown, - OS-enforced provider/tool credential boundary, phase-scoped secrets, disabled - repository Pi extensions/unrestricted built-ins, one-execution - reviewed-domain policy, resource limits, Artifact race defenses, - completeness, provenance, and authenticated expiry are enforced without - accepting caller acquisition code or claiming hostile-source or cross-tenant - isolation. -- Metadata-only telemetry is filtered through the fixed allowlist and bounded redaction before processing/export; canary secrets, content, raw caller identities, and cross-owner fragments never reach exporters, and only opaque owner correlation crosses the separately governed operator boundary. -- Required source/setup and race traces satisfy attempt/fence, happens-before, maximum-count, and forbidden-post-terminal constraints in the independent test-side checker; all-name-present malformed traces fail without introducing a parallel lifecycle implementation. -- Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. -- Public documentation states extension carriers, retained replay, trusted transports, credential versus hostile-code boundaries, metadata-only telemetry and its separate operator access/retention, topology, storage/HA limitation, runtime pins, result preservation, poisoned/crashed-worker recovery, and deferred capabilities. -- Abandoned experiments, unused adapters, compatibility shims, generated scratch files, retained test workspaces, and stale documentation are removed. +- Every R1-R18 requirement is implemented or explicitly demonstrated by a + passing acceptance scenario. +- The gateway starts with no `gateway.yaml` or `worker.yaml`, defaults to + loopback, and accepts explicit `0.0.0.0`. +- Network reachability is the only caller trust boundary; Task visibility and + idempotency are deployment-wide and documented accurately. +- Project workspace declarations own repositories and named OCI snapshot + repositories; user workspace declarations own profile launcher exposure; + built-in target IDs cannot be shadowed. +- The A2A card, every-operation activation header, metadata preservation, strict + request and result-schema grammar, exact error mapping, integrity/produced + Artifacts, canonicalization, retention capacity, and cancellation semantics + pass official-client contract fixtures. +- Repository and OCI modes produce one validated workspace-manifest contract, + never fall back across source modes, and retain truthful provenance. +- GitHub App eligibility/unknown state, acquisition sub-budget, no-installation + `gh` fallback, selected-App failure, OCI auth containment, and pre-provider + source-credential teardown are proven. +- Typed preparation never runs workspace setup shell commands. Built-in and + profile targets authenticate through private provider-control views; every MCP + child is secret-scoped; model tools cannot reach provider/MCP/operator + credentials or gateway state. +- Deadline, cancellation/terminal races, shutdown, result states, safe private + state paths, retention capacity, store failure, descendant quiescence, + managed/unmanaged recovery, safe evidence, and cleanup pass fault tests. +- Evaluation behavior, public-Internet authentication, remote workers, custom + materializers, and multi-tenant policy remain absent. ### Per unit -- U1: Standard extension carriers, closed workspace source/manifest contracts, - algorithm-qualified materializer/profile/input/output digest preimages, - verification and authorization vocabulary, App entitlement generation and - cache provenance states, active-attempt/fence-only credential requests, - controller-derived single-use non-durable lease bindings, token-versus-lease - scope, exact source-auth code/reason/retryability tuples, - integrity/structured-result Artifact schemas, four result states, original - claim digests, command revisions/tombstones, fence rules, typed failures, and - fixtures are generated and stable. -- U2: Trusted ingress, auth, opaque owner isolation, retained-claim-first replay, original bindings, atomic new admission, CAS settlement, pagination, startup recovery, quotas, Artifact access, tombstones, and cleanup pass fault injection. -- U3: Every advertised A2A operation agrees across stream and lookup while extension negotiation, replay ordering, authenticated worker routes, fencing, monotonic cancellation, and races preserve one Task. -- U4: Worker command state, exact source authorization, direct - Git/OCI/registered materialization, operator-mapped GitHub - App-before-account-pinned-CLI eligibility with sanitized invocation and - fail-closed selected-App errors, central fresh App minting without worker - issuer material, authoritative single-use lease delivery with - replay/substitution/stale/config-digest rejection, - repository/read-only/expiry-only token scope and safe lifetime boundaries, - deterministic public failure mapping with operator-only identities, - webhook/reconciliation-driven entitlement cache revocation and truthful hit - provenance, workspace-manifest validation and provenance classification, - same-filesystem atomic publication, acquisition and provider credential - separation, supervisor-owned runner cleanup, poisoned-exit/orphan recovery, - and dispatch/materialization/setup/action/check/quiescence/evidence/cleanup - pass malicious, crashed, and faulted scenarios. -- U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, result preservation, OS credential separation, native evidence, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. -- U6: Pi strict RPC/framing, terminating result, exact policy tools, disabled repository extensions/built-ins, OS-isolated credential store/provider runtime, result preservation, settlement, stats, abort, and process cleanup pass verification. -- U7: Closed backend, materializer, and source-credential registries, focused - `@octokit/auth-app` packaging without a full Octokit or root-CLI dependency, - authoritative lease controller, central App private-key custody and fresh - minter readiness, entitlement webhook/reconciler, account-pinned sanitized - local CLI, trusted transport/identity/readiness, acquisition/provider - credential and supervisor capability gating, poisoned-worker recycling, - metadata-only pre-export telemetry controls, Node-version separation, - tracing, shutdown, containers, and release artifacts work from built outputs. -- U8: Cross-backend E2E, all three workspace source modes, central GitHub App - and account-pinned trusted-local CLI credential-selection cases, remote - issuer-key exclusion, controller-derived single-use lease delivery, - fresh-token lifetime, token-versus-lease scope, exact failure mappings, - entitlement-driven cache invalidation and hit provenance, standard manifest - provenance, acquisition credential teardown, standard A2A carriers, retained - replay, selected materialization/setup transitions, independent race-trace - constraints, telemetry canary/cross-owner probes, transport and credential - abuse cases, command/quiescence races, examples, operator docs, changelog, - and release evidence are complete. +- U1: Runtime and generated schemas agree; invalid negotiation, request, + source/exposure/collision/configuration fixtures fail at expected paths. +- U2: Official A2A operations, global replay/visibility, project locks, store + faults, listeners, deadline, shutdown, restart, and retention pass. +- U3: Git and OCI fixtures pass; three-way provider eligibility, token lifetime, + and all no-fallback rules are observed; leak scans are clean. +- U4: Fake-adapter lifecycle proves terminal monotonicity, typed preparation, + isolation, containment, bounded/safe evidence, deadline/cancellation/shutdown, + poisoning, and cleanup. +- U5: Codex passes shared conformance and optional credentialed smoke evidence is + recorded when credentials exist. +- U6: Pi passes the same conformance and malformed RPC cannot produce success. +- U7: Final review is resolved; built CLI red/green E2E under `/tmp/`, complete + repository gates, schemas, docs, and reproducible PR instructions are complete. diff --git a/docs/research/agent-host-protocol-decision-inputs.md b/docs/research/agent-host-protocol-decision-inputs.md index 113ea6fc..e49d39f8 100644 --- a/docs/research/agent-host-protocol-decision-inputs.md +++ b/docs/research/agent-host-protocol-decision-inputs.md @@ -7,9 +7,9 @@ northbound contract. Treat the Agent Host Protocol (AHP) as an optional future protocol behind the gateway for a compatible backend or beside it for a collaborative session client. -AHP does not replace ADR 0002's Task identity, caller-scoped idempotency, -authorization, immutable source handling, cleanup, terminal evidence, or -bounded result retention. +AHP does not replace ADR 0002's deployment-wide Task identity and idempotency, +network trust boundary, immutable source handling, cleanup, terminal evidence, +or bounded result retention. The initial backend set is Codex and Pi; OpenCode is deferred. They are peer execution adapters behind one conformance contract; provider-specific process, @@ -27,14 +27,14 @@ source inspection, and full protocol comparison live in the AI Research Wiki: | Concern | AllAgents A2A gateway | AHP host/session layer | |---|---|---| -| Northbound consumer | AI Evals and future remote execution clients | IDE, browser, CLI, or collaborative operator client | +| Northbound consumer | AI Evals and future trusted-network execution clients | IDE, browser, CLI, or collaborative operator client | | Primary lifecycle | One addressable Task per accepted execution | Long-running session/chat with shared clients | | Public identity | Agent Card, Message, Task, Artifact, invocation key | Host, client, channel, session, chat, turn, tool call | -| State | Task status, messages, artifacts, retention | Snapshots, ordered actions, reducers, reconnect | -| Authorization | Authenticate/authorize service caller | Endpoint/resource auth and tool confirmation | +| State | Deployment-wide Task status, messages, artifacts, retention | Snapshots, ordered actions, reducers, reconnect | +| Authorization | Network reachability; no application caller identity | Endpoint/resource auth and tool confirmation | | Cancellation | Cancel Task, abort backend, terminate, clean up, report terminal outcome | Cancel interactive turn and call provider-native abort | | Evidence | Source, output, usage/cost, traces, file changes, artifacts, failures, cleanup, completeness, provenance | Live changesets and provider/session state | -| Isolation | Selected worker/backend boundary | Not supplied by the shared host process | +| Isolation | Single-process supervisor with invocation-owned child containment | Not supplied by the shared host process | The identities must be correlated rather than reused. At minimum retain the A2A Task ID, AllAgents invocation key, backend execution/session ID, @@ -45,16 +45,17 @@ provider-native thread/chat ID, and trace ID. 1. Define one narrow backend adapter contract for create/invoke, progress, permission decisions, cancellation, terminalization, evidence collection, shutdown, native evidence passthrough, and explicit capabilities. -2. Keep gateway responsibilities separate from worker/backend responsibilities. - The gateway owns caller authorization, Task/idempotency identity, backend - selection, normalized results, cancellation propagation, and retention. - Workers own source materialization, provider processes, mutable workspaces, - evidence capture, process termination, and cleanup. +2. Keep A2A and backend responsibilities separate inside one gateway service. + The A2A layer owns deployment-wide Task/idempotency identity, backend + selection, normalized results, cancellation propagation, and retention. The + invocation supervisor owns source acquisition, provider child processes, + mutable workspaces, evidence capture, process termination, and cleanup. 3. Propagate `CancelTask` and deadlines through the adapter to the provider-native abort primitive, then persist terminal status and cleanup outcome. Transport closure is not cancellation. -4. Separate caller authorization, execution permission policy, and - provider/resource credentials. +4. Separate network authorization, execution permission policy, and + provider/resource credentials. The initial gateway has no application caller + identity. 5. Combine normalized file operations with bounded provider-native diffs/checkpoints/trajectories. Declare attribution limits and incompleteness rather than treating the final working-tree diff as exact @@ -76,6 +77,8 @@ provider-native thread/chat ID, and trace ID. - Active-session reconnection beyond A2A Task lookup, subscription, and terminal result retrieval. - AHP local endpoint discovery, SSH host selection, and tunnel multiplexing. +- Remote worker ownership, routing, and session transport until the initial + single-process gateway needs a separate execution host. - Generic changeset review/operation state. - Long-lived session/chat catalogs and provider-native session adoption. diff --git a/docs/research/harbor-repository-materialization.md b/docs/research/harbor-repository-materialization.md index d4708d4c..38b6be3f 100644 --- a/docs/research/harbor-repository-materialization.md +++ b/docs/research/harbor-repository-materialization.md @@ -13,10 +13,10 @@ repository the agent edits is therefore benchmark- and task-owned: it may be bak an image, cloned by a Dockerfile, copied as task content, or otherwise prepared by the task author. -For AllAgents, repository and workspace provenance must remain explicit in the public -execution request and terminal evidence. Custom acquisition should be an -operator-registered, digest-pinned materializer behind the worker protocol, not an -arbitrary caller-supplied image or setup script. +For AllAgents, repository and workspace provenance must remain explicit in the +public execution request and terminal evidence. The initial gateway supports +only declared Git repositories and named digest-pinned OCI workspace snapshots; +custom materializers remain deferred. ## What Harbor fetches @@ -34,9 +34,11 @@ This is efficient for a large repository containing many independent Harbor task is not a mechanism for assembling several application repositories into one agent workspace. -Harbor also accepts an omitted commit or a mutable ref and resolves it to a commit. -That is convenient for an interactive local benchmark CLI, but it is weaker than the -AllAgents gateway requirement that an accepted request already name immutable source. +Harbor also accepts an omitted commit or a mutable ref and resolves it to a +commit. AllAgents permits a caller to override a declared repository with a +branch, tag, or commit for developer convenience, but resolves and records the +full commit before provider execution. Reproducibility-sensitive callers use a +full commit; OCI snapshots remain digest-pinned at admission. ### Task packages from the package registry @@ -77,10 +79,10 @@ sets that as `WORKDIR`; Harbor itself never clones that application repository. 1. **Separate descriptor acquisition from execution.** Resolve and validate immutable inputs before starting the coding-agent runtime. -2. **Use content-addressed caches.** Key reusable workspace snapshots by a digest of - normalized source identities, materializer version/digest, setup policy, current - authorization scope, and revocation epoch rather than a mutable name. Reauthorize - before lookup and make an old epoch ineligible after revocation. +2. **Use content-addressed snapshot caches.** Key reusable OCI workspace + snapshots by their immutable OCI and workspace-manifest digests. Direct Git + mode resolves revisions independently and records the resulting commits. + Reauthorize every remote acquisition. 3. **Avoid downloading irrelevant content.** For Git-backed descriptor catalogs, Harbor's tree-only discovery and sparse checkout are sound optimizations. For an application repository, use partial/shallow acquisition only when it preserves the @@ -93,43 +95,40 @@ sets that as `WORKDIR`; Harbor itself never clones that application repository. ### Adapt -Keep a first-class workspace manifest instead of hiding source inside an environment -image. Each materialized repository should retain at least: +Keep a first-class workspace manifest instead of hiding source inside an +environment image. Each materialized repository or snapshot should retain at +least: -- canonical source URL or snapshot identity; -- requested and resolved immutable commit or OCI digest; +- canonical source URL or configured snapshot identity; +- requested revision and resolved commit, or OCI manifest digest; - destination path and optional source subdirectory; -- materializer identity and version/digest; -- resulting tree/content identity; -- cache hit/miss and completeness facts. - -Use three explicit source modes: - -1. **Direct Git repositories** for the normal case, each with an exact commit and - collision-free destination. -2. **OCI workspace snapshots** for large, preassembled workspaces, referenced by digest - rather than tag and accompanied by a signed/validated workspace manifest. -3. **Operator-registered materializers** for JFrog, unusual monorepos, generated source, - or organization-specific setup. A request selects a configured materializer ID, - pins the expected workspace-manifest digest, and supplies validated, - resource-authorized structured inputs. The operator configuration pins the builder - image by digest, the worker derives the non-secret definition digest, credentials are - scoped only to materialization, and the builder must produce the standard workspace - manifest before the agent starts. The builder is operator-trusted deployment code; - deployments that cannot grant that trust need a broker or stronger acquisition - service. - -This retains Harbor's useful task-owned flexibility without allowing a caller to choose -an arbitrary executable image or shell script inside the trusted worker. +- acquisition implementation identity; +- resulting tree/content identity; and +- completeness and verification-versus-attestation facts. + +Use exactly two initial source modes: + +1. **Direct declared Git repositories** for the normal case. A request selects + configured repository names and may override only their revisions. The + gateway resolves and records full commits and enforces collision-free + destinations. +2. **Named OCI workspace snapshots** for large, preassembled workspaces. The + project workspace declares the repository; the request supplies immutable + OCI and workspace-manifest digests. + +Both modes produce the same standard workspace manifest. Neither mode falls +through to the other after admission. ### Do not copy -- Mutable Git refs, `HEAD`, image tags, or package `latest` as accepted execution - identities. -- Harbor's broad Git transport set (`http`, `ssh`, and `git` as well as HTTPS) at a - remote service boundary. The gateway should keep canonical credential-free HTTPS, - destination-policy revalidation, disabled redirects/helpers/filters/hooks/submodules, - and exact commit verification. +- Unresolved mutable Git refs as terminal execution identities. Branch and tag + overrides are valid only when the gateway resolves and records a full commit + before provider execution. +- Mutable OCI tags or package `latest` as accepted snapshot identities. +- Harbor's broad Git transport set (`http`, `ssh`, and `git` as well as HTTPS) at + a service boundary. The gateway keeps canonical credential-free HTTPS, + destination-policy revalidation, disabled redirects/helpers/filters/hooks/ + submodules, and full-commit verification. - A non-fatal Git LFS miss. If declared workspace content cannot be materialized, preparation must fail before provider execution. - Hashing a prebuilt image reference string as environment identity. Resolve and pin @@ -143,25 +142,31 @@ an arbitrary executable image or shell script inside the trusted worker. ## Recommended boundary -The worker should execute a dedicated materialization phase before any harness starts: - -1. Validate the normalized workspace request, exact source-resource authorization, - configured materializer, and current authorization scope before any cache lookup. -2. Resolve phase-scoped source credentials without exposing them to setup, the model, - or later evidence. -3. Populate a worker-owned staging directory on the final publication filesystem or - pull and unpack a digest-pinned workspace snapshot there. -4. Verify repository commits, paths, limits, content, the expected manifest digest, - and the standard workspace manifest; distinguish worker-verified identities from - materializer-attested claims. -5. Stop the acquisition process, revoke credentials, remove its mounts and runner - resource, and retain only the validated host-owned staging tree. +The gateway supervisor executes a dedicated acquisition phase before any +provider starts: + +1. Validate the normalized source request and its declared repository or + snapshot identities before any network access. +2. Resolve phase-scoped source credentials without exposing them to typed + provider preparation, the model, or later evidence collection. +3. Populate a gateway-owned staging directory on the final publication + filesystem, or pull and unpack a digest-pinned workspace snapshot there. +4. Verify repository commits, paths, limits, content, the expected manifest + digest, and the standard workspace manifest; distinguish gateway-verified + identities from snapshot-attested claims. +5. Stop acquisition processes, revoke credentials, remove helpers and mounts, + and retain only the validated credential-free staging tree. 6. Atomically rename that tree into the final workspace, record provenance, run - operator-owned setup, record the post-setup baseline, and only then launch the - harness-specific worker runtime. + adapter-owned typed preparation, record the baseline, and only then launch + the provider runtime. Project or user `setup` shell commands are not run. + +Operator-registered materializers, custom builders, and third source variants +are deferred until direct Git and OCI snapshots cannot satisfy a demonstrated +deployment need. Adding one requires a new decision for trust, configuration, +credential, provenance, and isolation boundaries. -The practical conclusion is narrow: Harbor is strong evidence for content-addressed -input bundles and environment-provider indirection. It is not evidence for making +The practical conclusion is narrow: Harbor is strong evidence for content- +addressed input bundles and staged publication. It is not evidence for making repository acquisition opaque or task-defined in the AllAgents public contract. ## Primary sources diff --git a/docs/research/source-credential-broker-precedents.md b/docs/research/source-credential-broker-precedents.md index b1be9054..f2c69f5b 100644 --- a/docs/research/source-credential-broker-precedents.md +++ b/docs/research/source-credential-broker-precedents.md @@ -2,29 +2,29 @@ ## Decision -The execution gateway does **not** need a mandatory standalone Git credential -broker for trusted local use. The settled local provider is an explicit, -account-pinned `gh auth token --hostname --user ` helper invoked -only when no App installation mapping applies. Its environment removes -`GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and -`GITHUB_ENTERPRISE_TOKEN`, and its token is exposed only to the one-shot -acquisition process. Git credential helpers and Git Credential Manager (GCM) -establish the process-boundary precedent, but arbitrary configured helpers are -not part of the selected implementation. A local helper is a broker in the -security sense; it is not a separately deployed network service. - -Remote or multi-tenant workers use one initial path: an authoritative -gateway/control-plane lease controller and trusted central token minter deliver -a fresh GitHub token over an authenticated, single-use, non-durable lease. The -GitHub bearer token is scoped only to the repository, read-only contents -permission, and GitHub expiry. Worker identity, attempt, lease epoch, command -revision, fence, operation, and delivery expiry are properties of the lease and -channel, not the token. Workers never inherit a person's credential helper, -credential store, SSH agent, or the App private key. A versioned central -snapshot-delivery protocol is deferred; it is not an alternative initial -readiness path. The minter may live inside the trusted control plane unless -private-key isolation, audit, scaling, or blast-radius requirements justify a -separate service process. +The execution gateway does **not** need a standalone Git credential broker for +the initial trusted-network deployment. It supports two in-process trusted +providers for `github.com`: a configured GitHub App and a configured, +account-pinned `gh auth token --hostname github.com --user ` fallback. + +The App is preferred whenever an App-authenticated repository-coverage check +proves an installation eligible. `gh` is considered only when the App is absent +or coverage is positively ineligible; unknown discovery, authentication, +permission, rate-limit, or service failures fail closed. Ambient `GH_TOKEN`, +`GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` are removed +from the CLI helper environment. + +Either token is exposed only to the one-shot acquisition process through an +invocation-scoped Git credential helper. The helper, token, and acquisition +process are gone before adapter preparation or provider execution. Git +credential helpers and Git Credential Manager establish the process-boundary +precedent, but arbitrary configured helpers are not part of the selected +implementation. A local helper is a broker in the security sense; it is not a +separately deployed network service. + +Central token minters, authenticated delivery leases, remote workers, and +multi-tenant credential policy are deferred until ADR 0002's deployment +boundary is reconsidered. ## Precedents @@ -65,12 +65,13 @@ local process/socket boundary, not a remotely reachable credential service. **Relevance.** Git helpers and GCM prove that a local credential provider can be an on-demand process rather than a network service. AllAgents does not, however, inherit or invoke an arbitrary configured helper chain. Its closed -provider registry permits only an explicit GitHub CLI provider pinned to a -configured non-secret account in a trusted-local profile, and only when no -configured GitHub App installation mapping applies. The helper invokes -`gh auth token --hostname --user ` without ambient GitHub token -variables. Its output reaches only the one-shot acquisition child; setup and -the coding harness inherit neither helper configuration nor the token. +provider registry permits only the selected GitHub App token or an explicit +GitHub CLI provider pinned to a configured non-secret account when App +eligibility is positively absent. The CLI invokes +`gh auth token --hostname github.com --user ` without ambient GitHub +token variables. Its output reaches only the one-shot acquisition child; +adapter preparation and the coding runtime inherit neither helper configuration +nor token. ### SSH agent forwarding @@ -136,11 +137,11 @@ long-term credential store, and its post-job deletion is defense in depth rather than the token's revocation mechanism. **Relevance.** This is the closest production precedent for AllAgents: keep the -App private key at a trusted central minter, issue one fresh least-privilege +App private key in the trusted gateway process, issue one fresh least-privilege token for a particular repository acquisition, expose it only during that phase, and remove its local material afterward. GitHub enforces repository, -read-only contents permission, and expiry; AllAgents separately enforces -attempt and operation bindings through its authenticated delivery lease. +read-only contents permission, and expiry; the gateway separately binds the +acquisition to the retained Task and effective configuration digest. ### BuildKit secret and SSH mounts @@ -179,82 +180,39 @@ credentials and does not eliminate the need for a central issuer in production. ## Recommendation for AllAgents -### Local mode - -1. Resolve `github.com` through the built-in GitHub backend and require explicit - host/API mappings for GitHub Enterprise Server hostnames. -2. Prefer a configured GitHub App installation that trusted operator policy - maps to the authorized repository. Do not use `@octokit/auth-app` to discover - installations. If no installation mapping applies, a trusted-local profile - may invoke the explicit - `gh auth token --hostname --user ` provider pinned to a - configured non-secret account. Include that account in the entitlement and - effective-profile digests, remove `GH_TOKEN`, `GITHUB_TOKEN`, - `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` from the helper - environment, and fail if the configured account cannot be resolved. Do not - inherit an arbitrary Git helper/GCM chain or forward an SSH agent. -3. Treat provider order as eligibility, not retry. Once the App provider is - selected, configuration, authentication, minting, authorization, rate-limit, - or service failure terminates acquisition without falling through to the - user identity. -4. Give the resolved token only to the dedicated acquisition subprocess through - a temporary helper channel, remove that channel, terminate the child, and - publish only a credential-free verified workspace before setup or the coding - harness starts. -5. Do **not** require or auto-start an AllAgents network credential service for - trusted local execution. The explicit account-pinned provider subprocess is - sufficient. - -### Production remote or multi-tenant workers - -1. Put GitHub App issuer material in a trusted central token-minter component. - Trusted operator configuration, not auth-app discovery, maps the repository - to an installation ID. For every cache-miss acquisition, use focused - [`@octokit/auth-app`](https://github.com/octokit/auth-app.js) with - `refresh: true` to bypass its installation-token cache and mint a fresh token - narrowed to that repository and read-only contents permission. Require - remaining lifetime strictly greater than the acquisition deadline plus - clock-skew margin, expire the delivery lease no later than the token, and - fail readiness when the configured acquisition ceiling can exceed a fresh - token's safe lifetime. -2. Make the gateway/control-plane credential-lease controller authoritative. - The authenticated worker requests only by active attempt and fence. From - durable dispatch and policy state, the controller derives the - effective-profile digest, selected provider, host/API-mapping digest, - installation ID, repository, operation, worker route and identity, lease - epoch, command revision, and expiry. Immediately before issuance it rechecks - active command revision, tombstone, fence, and lease state. -3. Deliver one single-use, non-durable grant/response over the authenticated - acquisition channel. A separate minter must agree with the controller's - configuration digest and consume the grant atomically. Reject replay, - substituted fields or providers, stale command state, and configuration - disagreement. The bearer token itself remains scoped only by GitHub to the - repository, read-only contents permission, and expiry; worker, attempt, - fence, and operation bindings belong to the lease. -4. Advance a GitHub App entitlement generation from authenticated lifecycle - webhooks plus bounded reconciliation whenever an installation is uninstalled, - suspended, or changes repository selection. Unknown or stale installation - state fails cache authorization. Mint only on a cache miss. On a miss, record - the acquiring provider in operator provenance; on a hit, record `cache_hit`, - the cached original acquisition-provider metadata, and current policy - selection/entitlement binding separately. -5. Publish deterministic coarse failures: `source_auth_unavailable` / - `no_eligible_provider` (not retryable); `source_auth_denied` / - `installation_repository_denied` (not retryable); - `source_auth_failed` with `app_configuration_invalid`, - `app_authentication_failed`, or `app_mint_failed` (not retryable), - `provider_rate_limited` or `provider_unavailable` (retryable), or - `trusted_local_cli_failed` (not retryable). Keep provider, installation, and - account identifiers in operator-only provenance. -6. Never forward an operator's general SSH agent or reuse their desktop GCM - store in a remote worker. Those capabilities represent the person, not the - individual execution request. -7. Keep minting logically central even if it initially lives inside the trusted - gateway process. Split the minter into a standalone network service when - remote trust boundaries, private-key isolation, audit, scaling, or - blast-radius controls require it. A versioned central snapshot-delivery - protocol may be designed later, but is not part of the initial architecture. - -The resulting rule is: **local reuse may be subprocess-mediated; production -issuance must be centrally policy-mediated.** A process boundary is required in -both cases, but a standalone credential service is not. +### Initial trusted-network gateway + +1. Resolve only canonical `github.com` HTTPS origins in the initial delivery. +2. Determine App applicability through an App-authenticated GitHub API client, + or verify an explicitly configured installation ID against the repository. + Model the result as `eligible`, `ineligible`, or `unknown`. +3. For `eligible`, use focused + [`@octokit/auth-app`](https://github.com/octokit/auth-app.js) authentication + and mint a fresh token narrowed to the repository and read-only contents. + Require remaining lifetime greater than the gateway's at-most-900-second + acquisition sub-budget plus a 60-second clock-skew margin. +4. For a missing App or proven `ineligible`, a trusted local deployment may use + the configured `gh auth token --hostname github.com --user ` + provider. Include the account in the acquisition-policy digest and strip + ambient token variables. An `unknown` App result never falls through. +5. Treat provider order as eligibility, not retry. Once App is selected, + configuration, authentication, minting, authorization, repository coverage, + rate-limit, or service failure terminates acquisition. +6. Give the resolved token only to the dedicated acquisition subprocess through + a temporary helper channel. Remove the channel and terminate the process + before atomically publishing the credential-free verified workspace. +7. Do not require or auto-start a network credential service. Keep App issuer + material and GitHub/OCI auth stores inaccessible to the adapter process and + model-invoked tools. + +### Deferred remote or multi-tenant deployment + +A future deployment may require a central token minter, authenticated single-use +delivery leases, entitlement generations, revocation reconciliation, worker +identity, fencing, and a snapshot-delivery protocol. Those mechanisms are not +part of the selected single-process architecture. They require a separate +decision when remote workers or tenant isolation become product requirements. + +The resulting initial rule is: **credential reuse is acquisition-subprocess- +mediated and ends before provider execution.** Remote or multi-tenant issuance +policy remains deferred; a standalone credential service is not required now. From 4f9036a9d53afc8a005b11cca033743c4c851f91 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sat, 19 Sep 2026 13:15:52 +1000 Subject: [PATCH 09/44] docs(architecture): define Promptfoo gateway consumption --- ...-agent-execution-through-an-a2a-gateway.md | 96 +++++++- ...0837-feat-coding-execution-gateway-plan.md | 206 ++++++++++++++++-- 2 files changed, 271 insertions(+), 31 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index 73afb298..b66c36c5 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -165,12 +165,14 @@ a `sha256:` workspace-manifest digest. The gateway constructs the full OCI reference server-side. Callers cannot supply a registry host, repository name, mutable tag, extraction destination, credential, or external-layer policy. -Both modes produce the same versioned workspace manifest. It records requested -and resolved repository identities, destinations, acquisition kind, relevant -OCI manifest and layer digests, the workspace-manifest digest, completeness, -and whether each fact was independently verified or snapshot-attested. A commit -listed inside an OCI snapshot is not described as independently verified unless -the gateway separately verifies it against its Git remote. +Both modes produce the same versioned, wire-visible workspace manifest. It +records declared logical repository names, requested revisions, resolved +commits, acquisition kind, relevant OCI manifest and layer digests, the +workspace-manifest digest, completeness, and whether each fact was independently +verified or snapshot-attested. It omits Git URLs, OCI repository origins, and +destination paths. A commit listed inside an OCI snapshot is not described as +independently verified unless the gateway separately verifies it against its +Git remote. Acquisition occurs in a gateway-owned staging directory. The gateway validates paths, collisions, file types, symlinks, layer and file counts, individual and @@ -179,6 +181,76 @@ the invocation workspace. Absolute paths, traversal, device files, sockets, escaping links, foreign or external OCI layers, and cross-origin credential forwarding are rejected. +### Consume the gateway from Promptfoo through an AI Evals provider + +Rejecting caller-supplied origins does not prevent AI Evals from selecting a +workspace in Promptfoo YAML. The two files have different ownership: + +- the AllAgents project workspace is the operator-controlled catalog that maps + repository and snapshot names to Git URLs, destinations, and OCI repositories; +- the Promptfoo configuration selects a target and source mode. Repository mode + materializes the complete configured repository set and may override + revisions by declared repository name. Snapshot mode selects one declared + snapshot name and supplies immutable digests. + +AI Evals owns a Promptfoo +[custom JavaScript/TypeScript provider](https://www.promptfoo.dev/docs/providers/custom-api/). +It implements `ApiProvider`: its constructor receives `ProviderOptions`, retains +`options.id`, validates `options.config`, and exposes `id()`. +`callApi(prompt, context, options)` reads bounded test variables from +`context.vars` and cancellation from `options?.abortSignal`. The provider +translates one `callApi` into one A2A Task: it creates an invocation key, puts +the prompt in the single `TextPart`, puts the target and closed source union in +the required extension metadata, waits or streams to terminal, and returns +output, normalized token usage, and logical provenance in Promptfoo's +`ProviderResponse`. + +For example, AI Evals can define two provider instances without sending either +origin over the wire: + +```yaml +providers: + - id: file://./providers/allagents-a2a.ts + label: codex-direct + config: + endpoint: http://allagents-gateway.tailnet:4732 + target: codex + source: + kind: repositories + revisions: + allagents: 0123456789abcdef0123456789abcdef01234567 + + - id: file://./providers/allagents-a2a.ts + label: codex-evaluation-snapshot + config: + endpoint: http://allagents-gateway.tailnet:4732 + target: codex + source: + kind: workspaceSnapshot + snapshot: evaluation + digest: sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef + workspaceManifestDigest: sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789 +``` + +The first provider materializes the complete configured repository set and uses +the `allagents` key only to override that repository's revision. The second +provider's `evaluation` key resolves to the declared +`ghcr.io/entityprocess/allagents-workspaces` repository. + +Static provider config fixes the source kind and logical names. The only +per-test object is `context.vars.allagentsSource`: repository mode accepts +revision overrides only for statically listed names and only as full lowercase +40-hex commits; snapshot mode accepts only replacement OCI and workspace- +manifest `sha256:` digests. Missing leaves retain static values. A URL, +destination, mutable revision, credential, command, unknown member, or changed +source kind/name fails before submission. After Task acceptance, the provider's +bounded deadline or `options?.abortSignal` sends `CancelTask`. It maps gateway +input, output, cached-input, and total token counts to Promptfoo's `prompt`, +`completion`, `cached`, and `total` fields respectively; other usage and Task/ +Artifact evidence stays in metadata without origins. The provider belongs in AI +Evals. AllAgents exposes the A2A contract and consumer documentation without +taking a runtime dependency on Promptfoo. + ### Resolve GitHub credentials with App-first eligibility fallback The source request is credential-free and never selects a credential provider. @@ -315,7 +387,8 @@ Terminal evidence distinguishes: - agent output; - optional validated structured result; -- requested and resolved repository or OCI identities; +- logical repository names, requested revisions, resolved commits, or snapshot + names and digests, never source origins or destinations; - pre- and post-execution Git state where applicable; - produced artifacts; - usage and bounded provider-native evidence; @@ -367,6 +440,10 @@ retry. Consumers own those concerns. Kubernetes deployment in the initial product. - Project and user workspace files remain the sole declaration authority for source identities and exposed profile launchers. +- AI Evals can express the configured repository set with named revision + overrides, or select a prebuilt image through a snapshot handle, in Promptfoo + YAML. Its custom provider translates that closed source choice to A2A and + keeps raw origins under AllAgents operator control. - Network reachability grants access to every exposed target and retained Task. Operators must treat network policy as the authorization boundary. - GitHub App credentials support private repositories without forcing every @@ -412,8 +489,9 @@ never control commands or argv. ### Let callers provide repository URLs or OCI repositories Rejected because workspace configuration already defines trusted source -identities and destinations. Requests may select declared names and immutable -revisions or digests, not introduce new origins. +identities and destinations. Repository requests materialize the configured set +and may override revisions by declared name; snapshot requests select a declared +name and immutable digests. Neither variant introduces a new origin. ### Fall back from a selected GitHub App after runtime failure diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 4793ba0c..07ec97e9 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -15,12 +15,14 @@ execution: code - **Objective:** A developer can run one trusted-network A2A endpoint for one AllAgents workspace and invoke built-in or explicitly exposed profile targets - against either declared Git repositories or a digest-pinned OCI workspace - snapshot. + against either the complete configured Git repository set, with optional + named revision overrides, or a digest-pinned OCI workspace snapshot. AI Evals + can configure either source mode in Promptfoo YAML through a custom provider + without sending origins. - **Means:** Add `allagents gateway serve`, a private execution-service package, a bounded durable Task store, direct Codex and Pi adapters, GitHub App and - GitHub CLI acquisition providers, OCI snapshot acquisition, and one supervised - invocation lifecycle. + GitHub CLI acquisition providers, OCI snapshot acquisition, one supervised + invocation lifecycle, and a documented Promptfoo provider contract. - **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) owns the public and trust boundaries. Project and user `workspace.yaml` files own source and profile declarations. A2A 1.0 owns core wire semantics. @@ -48,7 +50,10 @@ evaluation framework or multi-tenant platform. Callers use A2A Tasks and one required AllAgents extension. Network reachability is authorization. The service resolves configured targets and sources from existing workspace files, acquires a fresh invocation workspace, invokes Codex or Pi through a typed -adapter, and retains bounded terminal evidence. +adapter, and retains bounded terminal evidence. AI Evals consumes that boundary +through its own Promptfoo custom provider: evaluation YAML supplies named +revision overrides for the configured repository set, or one snapshot handle +and immutable digests, while AllAgents retains origin and credential authority. ### Problem Frame @@ -65,7 +70,8 @@ registry, or another profile configuration file for the initial use case. ### Actors - A1. **Trusted-network caller:** Any process able to reach the endpoint. All - callers have the same authority and Task visibility. + callers have the same authority and Task visibility. The first caller is an + AI Evals-owned Promptfoo custom provider that maps one `callApi` to one Task. - A2. **Execution gateway:** The A2A server and invocation supervisor. It owns deployment-wide Task identity, acquisition, routing, status, cancellation, evidence, retention, and cleanup. @@ -103,6 +109,10 @@ registry, or another profile configuration file for the initial use case. Governs R5, R13-R16. - **Keep evaluation outside AllAgents.** Consumers own datasets, repetitions, scoring, assertions, and evaluation Runs. Governs R17. +- **Bridge Promptfoo at the consumer boundary.** AI Evals owns a custom provider + that maps Promptfoo YAML and test variables to the closed A2A source modes and + maps terminal Tasks back to `ProviderResponse`. AllAgents owns no Promptfoo + runtime behavior. Governs R19. ### Requirements @@ -183,11 +193,14 @@ registry, or another profile configuration file for the initial use case. `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, workspaceManifestDigest: Digest, repositories }`. `repositories` contains 1-64 unique strict entries - `{ name: ConfigName, canonicalUrl: string, requestedRevision?: - RevisionText, resolvedCommit: string, verification: - "independentlyVerified" | "snapshotAttested" }`; `canonicalUrl` is a - canonical HTTPS URL of at most 2048 bytes, and `resolvedCommit` matches - `^[0-9a-f]{40}$`. + `{ name: ConfigName, requestedRevision?: RevisionText, + resolvedCommit: string, verification: + "independentlyVerified" | "snapshotAttested" }`; `resolvedCommit` matches + `^[0-9a-f]{40}$`. Gateway-generated source identity, workspace-manifest + fields, evidence metadata, and provider-added metadata never contain Git + URLs, OCI repository origins, or destination paths. This guarantee does not + inspect or sanitize opaque caller prompts, provider terminal output, or + produced-Artifact payloads. - optional `workspaceManifestDigest` is `Digest`. - `terminalOutput` is `{ text, truncated }`, where `text` is valid UTF-8 of at most 1 MiB and `truncated` is boolean. @@ -382,6 +395,28 @@ registry, or another profile configuration file for the initial use case. for listener, workspace, state/retention, GitHub, OCI, and Codex/Pi auth-file handles. Secret values never enter workspace files, requests, logs, Tasks, Artifacts, retained workspaces, or model-invoked tool environments. +- R19. Document AI Evals consumption through a Promptfoo custom + JavaScript/TypeScript provider implementing Promptfoo's `ApiProvider`. + `constructor(options: ProviderOptions)` retains `options.id`, validates + `options.config`, and `id()` returns the retained ID. Static config contains + the gateway endpoint, target ID, and exactly one closed source mode: + repository mode materializes the complete configured repository set and + carries only an optional revision map keyed by declared repository name; + snapshot mode carries one declared snapshot name with OCI and workspace- + manifest digests. `callApi(prompt, context, options)` may apply the exact + `context.vars.allagentsSource` leaf overrides defined below. Dynamic + repository revisions must be full lowercase 40-hex commit IDs; dynamic + snapshot values must be full lowercase `sha256:` digests. Source kind, + snapshot name, and repository origins never vary per test. Unknown members, + revision names absent from static config, URLs, destinations, tags, + credentials, commands, and permission policy fail before submission. + `options?.abortSignal` and the provider's bounded deadline both invoke A2A + `CancelTask` after acceptance. One `callApi` creates one A2A Task and maps + terminal output, usage, Task/Artifact IDs, structured result, and logical + provenance into `ProviderResponse`; admission or terminal failure maps to + `error`. AI Evals owns the provider implementation. AllAgents publishes the + protocol and YAML examples without importing Promptfoo provider code or adding + Promptfoo as a runtime dependency. ### Key Flows @@ -437,6 +472,24 @@ registry, or another profile configuration file for the initial use case. printing the platform recovery command. Managed mode may exit only after its validated external manager accepts containment ownership. +- F6. **Invoke from Promptfoo** + 1. Promptfoo constructs the AI Evals-owned TypeScript provider with + `ProviderOptions`; the provider retains the ID and validates + `options.config` containing the private-network endpoint, target, and one + closed source-mode object. + 2. `callApi(prompt, context, options)` applies only valid + `context.vars.allagentsSource` leaf overrides, creates one invocation key, + and sends one A2A Message with the prompt and required extension. + 3. The provider waits or streams until terminal. Its deadline or + `options?.abortSignal` sends `CancelTask` once after acceptance and waits + for the same terminal cleanup path. + 4. It returns terminal text or validated structured output in + `ProviderResponse.output`; maps `inputTokens -> prompt`, + `outputTokens -> completion`, `cachedInputTokens -> cached`, and + `totalTokens -> total`; and puts other usage plus Task, Artifact, logical + source, termination, and cleanup facts in `metadata`. Admission or terminal + execution failure returns `error`. + ### Acceptance Examples - AE1. A caller on a permitted Tailscale or firewalled network discovers the @@ -497,6 +550,17 @@ registry, or another profile configuration file for the initial use case. - AE20. Unrelated Message metadata survives request processing. Every profiled A2A operation requires activation, and a terminal Task may contain the single integrity Artifact plus referenced produced Artifacts. +- AE21. The AI Evals Promptfoo provider loads one repository-mode and one + snapshot-mode YAML instance. Repository mode materializes the complete + configured set and sends only optional revision overrides keyed by declared + name; snapshot mode sends one declared name and immutable digests. Neither + request source metadata nor response source-identity metadata contains a Git + URL, OCI repository, or destination. + Both calls return scorable `ProviderResponse.output`, the exact normalized + token mapping, and Task/Artifact/logical-provenance metadata. Per-test + repository overrides accept only full commits. An unknown variable member, + mutable revision, origin, destination, or undeclared name fails before + submission. ### Success Criteria @@ -505,6 +569,9 @@ registry, or another profile configuration file for the initial use case. - The official A2A client exercises required-extension negotiation, send, stream, get, list, subscribe, replay, cancel, terminal cancel errors, Artifact retrieval, and expiry. +- An AI Evals-style Promptfoo custom-provider fixture consumes representative + YAML for both source modes and maps a terminal Task to `ProviderResponse` + without adding Promptfoo to the AllAgents runtime. - Built-in Codex/Pi and exposed profile targets pass one conformance suite, including reserved-ID collisions. - Direct Git and OCI snapshot fixtures produce equivalent validated workspace @@ -722,6 +789,87 @@ The nested object is strict and initially contains only `expose: true`. Absence means not exposed. Exposure requires a launcher, an initial supported backend, and a healthy installed profile with matching declaration digest. +**Promptfoo custom-provider consumption** + +AI Evals implements Promptfoo's +[`ApiProvider`](https://www.promptfoo.dev/docs/providers/custom-api/) in +TypeScript. Its `constructor(options: ProviderOptions)` stores +`options.id ?? "allagents-a2a"` and validates `options.config`; `id()` returns +that stored value. Its +`callApi(prompt, context, options)` uses `context.vars` for test data and +`options?.abortSignal` for request cancellation. + +Static YAML defines the source mode and every logical name: + +```yaml +providers: + - id: file://./providers/allagents-a2a.ts + label: codex-direct + config: + endpoint: http://allagents-gateway.tailnet:4732 + target: codex + source: + kind: repositories + revisions: + allagents: 0123456789abcdef0123456789abcdef01234567 + + - id: file://./providers/allagents-a2a.ts + label: codex-evaluation-snapshot + config: + endpoint: http://allagents-gateway.tailnet:4732 + target: codex + source: + kind: workspaceSnapshot + snapshot: evaluation + digest: sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef + workspaceManifestDigest: sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789 + +tests: + - description: direct repositories at an exact commit + providers: [codex-direct] + vars: + allagentsSource: + revisions: + allagents: fedcba9876543210fedcba9876543210fedcba98 + + - description: immutable prebuilt workspace + providers: [codex-evaluation-snapshot] + vars: + allagentsSource: + digest: sha256:fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210 + workspaceManifestDigest: sha256:6789abcdef0123456789abcdef0123456789abcdef0123456789abcdef012345 +``` + +`allagents` is a declared repository name used only as a revision-override key; +repository mode still materializes the complete configured set. `evaluation` is +the logical snapshot handle. The provider sends the source mode, optional named +revisions, and immutable digests, not +`https://github.com/EntityProcess/allagents.git` or +`ghcr.io/entityprocess/allagents-workspaces`. The gateway resolves origins and +credentials server-side and omits them from A2A source-identity responses. + +`context.vars.allagentsSource` is the only per-test override. In repository mode +it may contain exactly `revisions`, whose keys must already exist in static +`config.source.revisions` and whose values are full lowercase 40-hex commits. +In snapshot mode it may contain exactly `digest` and/or +`workspaceManifestDigest`, both full lowercase `sha256:` digests. Present leaves +replace static leaves; absent leaves retain static values. Source kind, +repository-name allowlist, and snapshot name remain static. Unknown members, +mutable revisions, origins, destinations, credentials, and commands fail before +A2A submission. + +Each `callApi` creates one invocation key and A2A Task. The provider sends +`CancelTask` when its bounded deadline or `options?.abortSignal` fires after +acceptance. It returns terminal text or the validated structured result as +`ProviderResponse.output`. It maps gateway usage exactly as +`inputTokens -> tokenUsage.prompt`, `outputTokens -> tokenUsage.completion`, +`cachedInputTokens -> tokenUsage.cached`, and +`totalTokens -> tokenUsage.total`; provider-specific counters stay in +`metadata`. Task ID, Artifact references, logical source identity, termination, +and cleanup evidence also remain in `metadata`, without origins or destination +paths. Admission and terminal failures use `ProviderResponse.error`. This +provider is AI Evals code; AllAgents has no Promptfoo runtime dependency. + ### Error and Status Mapping | Condition | Stable code and A2A outcome | Fresh-invocation retryable | @@ -988,22 +1136,29 @@ messages never enter either carrier. ### U7. End-to-end delivery and documentation - **Goal:** Prove the built CLI and document the trusted-network operating model, - workspace configuration, credentials, sources, and risks. -- **Requirements:** R1-R18; F1-F5; AE1-AE20. + workspace configuration, credentials, sources, Promptfoo consumption, and + risks. +- **Requirements:** R1-R19; F1-F6; AE1-AE21. - **Files:** gateway guide/reference, configuration reference, README, CHANGELOG, - real example project/user workspaces, E2E fixtures, release evidence. + real example project/user workspaces, AI Evals-style Promptfoo YAML and custom- + provider contract fixture, E2E fixtures, release evidence. - **Approach:** After the final implementation review is resolved, build the CLI; create project and user workspaces under `/tmp/`; expose Codex/Pi fixture targets; serve on loopback and `0.0.0.0`; acquire from local Git and OCI fixtures; run the official A2A client through negotiation, success, replay, - cancellation, deadline, shutdown, restart, and expiry. Document that network - reachability grants full authority and App/OCI secrets are process inputs, not - YAML. + cancellation, deadline, shutdown, restart, and expiry. Run a minimal custom- + provider fixture through one configured-repository-set invocation and one + named-snapshot invocation, proving Promptfoo configuration carries only + source mode, named revision overrides, snapshot handle, and digests while the + gateway resolves origins. Document that network reachability grants full + authority and App/OCI secrets are process inputs, not YAML. - **Execution note:** The green smoke test must exercise the same built command and `/tmp/` workspace shape as the recorded red E2E, not a test-only server. + The consumer fixture models AI Evals but remains test/documentation code; the + AllAgents runtime does not import Promptfoo. - **Verification:** `bun run build`, focused and full tests, typecheck, lint, docs - build, schema drift check, and exact red/green E2E commands/results recorded in - the PR description. + build, schema drift check, custom-provider contract fixture, and exact + red/green E2E commands/results recorded in the PR description. --- @@ -1024,12 +1179,13 @@ messages never enter either carrier. | Structured result | U1, U4-U6 | Exact subset and envelope, valid/invalid/not-produced states, Artifact cardinality, no false publication | | Repository quality | All | Build, focused/full tests, typecheck, lint, schema check, docs build | | Built CLI E2E | U7 | Recorded red then green built command under `/tmp/`, both sources, auth isolation, replay/cancel/deadline/shutdown/restart | +| Promptfoo consumption | U7 | AI Evals-style YAML for both source modes; request source metadata and gateway-generated response provenance omit origins; terminal Task maps to `ProviderResponse` | ## Definition of Done ### Global -- Every R1-R18 requirement is implemented or explicitly demonstrated by a +- Every R1-R19 requirement is implemented or explicitly demonstrated by a passing acceptance scenario. - The gateway starts with no `gateway.yaml` or `worker.yaml`, defaults to loopback, and accepts explicit `0.0.0.0`. @@ -1042,6 +1198,11 @@ messages never enter either carrier. request and result-schema grammar, exact error mapping, integrity/produced Artifacts, canonicalization, retention capacity, and cancellation semantics pass official-client contract fixtures. +- AI Evals-style Promptfoo YAML selects repository mode with optional named + revision overrides, or snapshot mode with one logical handle and immutable + digests. The custom-provider fixture maps one `callApi` to one Task, propagates + cancellation, normalizes usage, and returns output, Artifacts, and logical + provenance without sending origins or adding a Promptfoo runtime dependency. - Repository and OCI modes produce one validated workspace-manifest contract, never fall back across source modes, and retain truthful provenance. - GitHub App eligibility/unknown state, acquisition sub-budget, no-installation @@ -1071,5 +1232,6 @@ messages never enter either carrier. - U5: Codex passes shared conformance and optional credentialed smoke evidence is recorded when credentials exist. - U6: Pi passes the same conformance and malformed RPC cannot produce success. -- U7: Final review is resolved; built CLI red/green E2E under `/tmp/`, complete - repository gates, schemas, docs, and reproducible PR instructions are complete. +- U7: Final review is resolved; built CLI red/green E2E under `/tmp/`, Promptfoo + custom-provider contract fixture, complete repository gates, schemas, docs, + and reproducible PR instructions are complete. From fc15c2b041fe4a443b1276fb2aa05050e43ccda2 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sat, 19 Sep 2026 15:13:01 +1000 Subject: [PATCH 10/44] docs(architecture): align execution gateway contracts --- ...-agent-execution-through-an-a2a-gateway.md | 404 ++-- ...0837-feat-coding-execution-gateway-plan.md | 1657 +++++++++++------ 2 files changed, 1309 insertions(+), 752 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index b66c36c5..22aacddc 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -40,8 +40,8 @@ entry point. It is separate from the interactive CLI command lifecycle but may run as a single local service process that supervises acquisition and provider child processes. -The gateway implements A2A 1.0 HTTP+JSON plus a required versioned AllAgents -coding-execution extension. It owns: +The gateway implements A2A protocol version `1.0` over the `HTTP+JSON` binding +plus a required versioned AllAgents coding-execution extension. It owns: - stable Task and idempotency identity; - execution-target selection; @@ -59,18 +59,31 @@ or durable evaluation Run ledger. The initial gateway has no application-level authentication or per-caller authorization. It may bind to loopback, a specific interface, or `0.0.0.0`. Loopback remains the default when no listen address is supplied, but an explicit -`0.0.0.0` binding is valid and requires no unsafe-mode flag. - -Every host able to reach the listener is equally trusted. Any reachable caller -may invoke every exposed target, list and retrieve retained Tasks and Artifacts, -and request cancellation. Task lookup and idempotency are deployment-wide, not +`0.0.0.0` binding is valid and requires no unsafe-mode flag. The Agent Card +advertises a separate absolute interface URL; non-loopback listeners require +that value explicitly because a wildcard bind address is not routable. +Production interfaces use HTTPS; direct HTTP is limited to loopback development. + +Every external host able to reach the listener is equally trusted. Any reachable +caller may invoke every available target, including built-in and gateway-enabled +profile targets; list or retrieve retained Tasks and their Artifacts; and +request cancellation. Task lookup and idempotency are deployment-wide, not scoped to a caller identity. Operators must use Tailscale ACLs, host firewalls, container networking, or equivalent network controls when the listener is not loopback-only. -TLS termination, OIDC, static bearer tokens, per-tenant ownership, and -multi-tenant information-hiding are deferred. They require a separate decision -when the service is exposed outside one trusted network boundary. +Invocation descendants are not network peers. Provider, MCP, and model-tool +processes run in role-specific network namespaces that cannot route to host +loopback, any gateway bind or advertised address, ingress proxies, or operator +management networks. Provider and MCP egress is default-deny except for +destinations compiled from adapter and MCP configuration; model tools receive no +network unless an explicit adapter policy grants the same constrained egress. + +Gateway-managed TLS termination, OIDC, static bearer tokens, per-tenant +ownership, and multi-tenant information-hiding are deferred. Production clients +reach the advertised HTTPS interface through operator-managed termination or an +encrypted private overlay. Application authentication requires a separate +decision when the service leaves one trusted network boundary. ### Use existing workspace files as the configuration authority @@ -82,8 +95,8 @@ canonical for repository identities, remote sources, destination paths, default revisions, workspace files, plugins, and named OCI snapshot sources. The user `~/.allagents/workspace.yaml` remains canonical for global profiles and -launcher-backed execution targets. A launcher-bearing profile client is exposed -only when it explicitly declares: +launcher-backed execution targets. A launcher-bearing profile client is gateway- +enabled only when it explicitly declares: ```yaml profiles: @@ -92,15 +105,15 @@ profiles: - name: codex launcher: codex-review gateway: - expose: true + enabled: true ``` The public target ID is the launcher basename. Launcher names are already portable and collision-checked across every user profile, while one profile may contain several clients and therefore several launchers. Internally the target resolves to exactly one `(profile, client)` pair. The gateway reserves built-in -target IDs, initially `codex` and `pi`; an exposed launcher whose portable -collision key matches a built-in ID is invalid. +target IDs, initially `codex` and `pi`; a gateway-enabled launcher whose +portable collision key matches a built-in ID is invalid. The built-in `codex` and `pi` targets remain available when their adapters are ready. Explicit launcher-backed targets add configured variants such as @@ -116,21 +129,25 @@ its typed adapter and invokes the provider's supported automation surface. Process-level options use exact flags and environment variables for: -- listener and workspace selection; +- listener, advertised-interface URL, and workspace selection; - a project-specific state-directory override; - terminal Task retention and bounded Artifact/event storage; - GitHub App identifiers and private-key file references; - the configured GitHub CLI account; -- an OCI credential file or fixed credential-helper executable; and +- a strict Docker-auth file or fixed Docker credential-helper executable; and - Codex and Pi auth-file handles. By default the state root is a deterministic child of `~/.allagents/gateway/` keyed by the canonical project-workspace identity. The -store persists and verifies that identity and holds an exclusive lock for the -process lifetime. The root is current-user owned, private, symlink- and hard- -link-resistant, and disjoint from project, profile, and invocation roots. - -Secret values never belong in either workspace file. +packaged Rust helper owns a private SQLite store in WAL/full-synchronization mode +through a descriptor-rooted VFS. Every database, WAL, SHM, journal, and temporary +file open uses `openat2` beneath/no-symlink resolution and rejects hard links. +The store persists claims, Tasks, one execution lease, internal outcome intent, +events, bounded Artifact bytes, containment identity, and expiry transactions. +It verifies workspace identity and holds an exclusive process-lifetime lock. +The root is current-user owned, private, link-resistant, and disjoint from +project, profile, and invocation roots. The listener exposes metadata-only +`/healthz` and `/readyz`; readiness is false whenever admission is unsafe. ### Support direct repositories and OCI workspace snapshots @@ -152,18 +169,43 @@ tags may be accepted for developer convenience, but the gateway resolves and records the full commit object ID before provider execution. Reproducibility- sensitive callers should supply full commit IDs. -For OCI snapshots, the project workspace declares the registry repository: +For OCI snapshots, the project workspace declares the registry repository and +any exact cross-origin layer-blob redirect hosts: ```yaml workspaceSnapshots: evaluation: repository: ghcr.io/entityprocess/allagents-workspaces + layerRedirectHosts: + - pkg-containers.githubusercontent.com ``` -The request supplies the name `evaluation`, a `sha256:` OCI manifest digest, and -a `sha256:` workspace-manifest digest. The gateway constructs the full OCI -reference server-side. Callers cannot supply a registry host, repository name, -mutable tag, extraction destination, credential, or external-layer policy. +The request supplies the name `evaluation`, a `sha256:` OCI image-manifest +digest, and a `sha256:` workspace-manifest digest. The gateway constructs the +full OCI reference server-side. Callers cannot supply a registry host, +repository name, mutable tag, extraction destination, credential, platform +selector, redirect host, or external-layer policy. + +V1 accepts only an OCI Image Manifest directly at the requested digest; image +indexes, descriptor URLs or embedded data, non-distributable layers, and +unknown media types are rejected. Its config is the RFC 8785 canonical +`application/vnd.allagents.workspace-manifest.v1+json` object and must match the +requested workspace-manifest digest. The gateway verifies the manifest body, +config, and every distributable tar/gzip/zstd layer descriptor before decoding, +then applies layers in manifest order with OCI whiteout and opaque-whiteout +semantics. + +Registry metadata remains same-origin. A cross-origin redirect is allowed only +for a layer-blob `GET` or `HEAD` to an exact operator-declared +`layerRedirectHosts` entry; an absent allowlist rejects it. Every bounded HTTPS +hop strips authorization, cookies, and client credentials, rejects URL +credentials, resolves and validates every address at connection time, and +rejects mixed answers, rebinding, downgrade, and unapproved destinations. +Loopback, link-local, private, reserved, or other non-global addresses are +permitted only when their exact host is the source's operator-declared +repository host or layer-redirect host. Token, manifest, and config redirects +remain same-origin. Descriptor size and digest verification remains mandatory +after redirects. Both modes produce the same versioned, wire-visible workspace manifest. It records declared logical repository names, requested revisions, resolved @@ -176,10 +218,10 @@ Git remote. Acquisition occurs in a gateway-owned staging directory. The gateway validates paths, collisions, file types, symlinks, layer and file counts, individual and -total sizes, digests, and the workspace manifest before atomically publishing -the invocation workspace. Absolute paths, traversal, device files, sockets, -escaping links, foreign or external OCI layers, and cross-origin credential -forwarding are rejected. +total compressed and expanded sizes, digests, and the workspace manifest before +atomically publishing the invocation workspace. Absolute paths, traversal, +device files, sockets, escaping links, foreign or external OCI layers, and +unapproved cross-origin access are rejected. ### Consume the gateway from Promptfoo through an AI Evals provider @@ -195,25 +237,40 @@ workspace in Promptfoo YAML. The two files have different ownership: AI Evals owns a Promptfoo [custom JavaScript/TypeScript provider](https://www.promptfoo.dev/docs/providers/custom-api/). -It implements `ApiProvider`: its constructor receives `ProviderOptions`, retains -`options.id`, validates `options.config`, and exposes `id()`. -`callApi(prompt, context, options)` reads bounded test variables from -`context.vars` and cancellation from `options?.abortSignal`. The provider -translates one `callApi` into one A2A Task: it creates an invocation key, puts -the prompt in the single `TextPart`, puts the target and closed source union in -the required extension metadata, waits or streams to terminal, and returns -output, normalized token usage, and logical provenance in Promptfoo's -`ProviderResponse`. +It implements `ApiProvider`: its constructor receives `ProviderOptions`, +requires and retains a nonempty `options.id`, validates `options.config`, and +exposes `id()`. +`callApi(prompt, context?, options?)` reads bounded test variables from +`context?.vars` when present and cancellation from `options?.abortSignal`. The +provider translates one `callApi` into one A2A Task: it creates and retains a +high-entropy invocation key, sends one Message whose sole Part has `text` set, +declares the extension in `Message.extensions`, puts the target and closed source +union in the matching metadata member, and calls `SendMessage` with +`returnImmediately: true`. It captures the Task ID and follows terminal state +through `SubscribeToTask`, with `GetTask` and bounded resubscription for races or +disconnects. It returns output, normalized token usage, stable failure metadata, +and logical provenance in Promptfoo's `ProviderResponse`. For example, AI Evals can define two provider instances without sending either origin over the wire: ```yaml +prompts: + - file://./prompts/coding-task.txt + +sharing: false +evaluateOptions: + maxConcurrency: 1 + cache: false +commandLineOptions: + write: false + share: false + providers: - id: file://./providers/allagents-a2a.ts label: codex-direct config: - endpoint: http://allagents-gateway.tailnet:4732 + endpoint: https://allagents-gateway.example.internal target: codex source: kind: repositories @@ -223,7 +280,7 @@ providers: - id: file://./providers/allagents-a2a.ts label: codex-evaluation-snapshot config: - endpoint: http://allagents-gateway.tailnet:4732 + endpoint: https://allagents-gateway.example.internal target: codex source: kind: workspaceSnapshot @@ -235,21 +292,32 @@ providers: The first provider materializes the complete configured repository set and uses the `allagents` key only to override that repository's revision. The second provider's `evaluation` key resolves to the declared -`ghcr.io/entityprocess/allagents-workspaces` repository. +`ghcr.io/entityprocess/allagents-workspaces` repository. The gateway enforces +one active invocation transactionally. Promptfoo keeps `maxConcurrency: 1` to +avoid predictably creating failed capacity Tasks; other trusted callers need no +external queue for correctness. The no-cache/no-write/no-share values are secure +defaults for confidential prompts and outputs; consumers may enable persistence +or sharing only after applying their own retention, access, destination, and +redaction policy. Static provider config fixes the source kind and logical names. The only -per-test object is `context.vars.allagentsSource`: repository mode accepts +per-test object is `context?.vars?.allagentsSource`: repository mode accepts revision overrides only for statically listed names and only as full lowercase 40-hex commits; snapshot mode accepts only replacement OCI and workspace- -manifest `sha256:` digests. Missing leaves retain static values. A URL, -destination, mutable revision, credential, command, unknown member, or changed -source kind/name fails before submission. After Task acceptance, the provider's -bounded deadline or `options?.abortSignal` sends `CancelTask`. It maps gateway -input, output, cached-input, and total token counts to Promptfoo's `prompt`, -`completion`, `cached`, and `total` fields respectively; other usage and Task/ -Artifact evidence stays in metadata without origins. The provider belongs in AI -Evals. AllAgents exposes the A2A contract and consumer documentation without -taking a runtime dependency on Promptfoo. +manifest `sha256:` digests. Missing context or leaves retain static values. A +URL, destination, mutable revision, credential, command, unknown member, or +changed source kind/name fails before submission. After Task acceptance, the +provider's bounded deadline or `options?.abortSignal` sends one `CancelTask` +using a fresh cleanup signal rather than the already aborted request signal. +It maps gateway input, output, cached-input, and total token counts to +Promptfoo's `prompt`, `completion`, `cached`, and `total` fields respectively. +Safe stable failure +code, retryability, accepted Task ID, other usage, and logical Task/Artifact +evidence stay in metadata without origins. Opaque prompts, terminal output, +structured results, native evidence, and produced-Artifact payloads remain +unredacted sensitive data. The provider belongs in AI Evals. AllAgents exposes +the A2A contract and consumer documentation without taking a runtime dependency +on Promptfoo. ### Resolve GitHub credentials with App-first eligibility fallback @@ -261,17 +329,20 @@ For `github.com`, the gateway supports two trusted providers: The App is preferred when it has an installation covering the configured repository. Installation applicability has three outcomes: `eligible`, -`ineligible`, and `unknown`. The gateway discovers applicability through an -App-authenticated GitHub API client, or verifies an explicitly configured -installation ID against the repository. `@octokit/auth-app` handles App JWT and -installation-token authentication; it is not treated as the repository- -discovery policy by itself. - -For an eligible installation, the gateway requests a fresh repository-scoped -installation token for each acquisition and grants only required read -permissions. Acquisition receives at most 900 seconds or the shorter remaining -Task deadline. The token must remain valid beyond that sub-budget plus a -60-second clock-skew margin. +`ineligible`, and `unknown`. An App-authenticated lookup that returns coverage +is eligible. A 404 is ineligible only after the configured GitHub CLI identity +independently proves that the repository exists; an uncorroborated 404 or any +authentication, permission, rate-limit, timeout, or service ambiguity is +unknown. An explicitly configured installation ID must positively verify +repository coverage. + +For an eligible installation, the gateway bypasses the SDK token cache and +requests a fresh repository-scoped, read-only token for each acquisition. It +validates repository selection, permissions, creation time, and expiry. +Acquisition receives at most 900 seconds or the shorter remaining Task deadline, +and the token must remain valid beyond that sub-budget plus a 60-second clock- +skew margin. The gateway revokes the token after acquisition; unconfirmed +revocation fails before provider execution. GitHub CLI is an eligibility fallback only when the App is not configured or applicability is positively `ineligible`. An `unknown` result caused by @@ -287,9 +358,9 @@ with `GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and is part of the acquisition-policy digest. After an App installation is selected, App configuration, authentication, -token minting, permission, repository-coverage, rate-limit, or service failure -terminates acquisition. The gateway never retries the same Task through the -broader GitHub CLI identity. +token minting or validation, permission, repository coverage, revocation, +rate-limit, or service failure terminates acquisition. The gateway never retries +the same Task through the broader GitHub CLI identity. Git receives credentials only through an invocation-scoped helper under hermetic Git configuration. The gateway excludes system, global, and repository @@ -300,9 +371,10 @@ Artifacts, retained workspaces, profile setup, MCP processes, agent processes, or model-invoked tools. The helper and token are destroyed before provider execution. -OCI credentials come from a configured auth-file or standard credential helper, -are scoped to snapshot acquisition, and are removed before publication. Public -registries require no credential configuration. +OCI credentials come from either a strict Docker-auth subset that cannot name +executables or a fixed Docker credential helper using its standard `get` +protocol. They are scoped to snapshot acquisition and removed before +publication. Public registries require no credential configuration. ### Integrate providers through typed adapters @@ -312,9 +384,11 @@ capabilities, invocation, progress, deterministic permission handling, abort, terminal output, optional structured result, usage, native evidence, and disposal. -The Codex adapter depends directly on `@openai/codex-sdk`, creates one fresh -thread per Task, passes cancellation and optional output schema through the SDK, -and consumes structured events. +The Codex adapter depends directly on pinned `@openai/codex-sdk`, creates one +fresh thread per Task, passes cancellation, and consumes structured events. It +uses native `outputSchema` only for schemas supported by the pinned Structured +Outputs contract; other valid public schemas use explicit JSON guidance and the +same gateway-side validator used by every backend. The Pi adapter uses strict RPC mode with invocation-owned configuration and a restricted policy extension. Repository extensions and unrestricted built-ins @@ -332,56 +406,88 @@ Validated profile settings, plugins, MCP declarations, and deterministic workspace projections are applied through existing typed transforms. Provider control processes, MCP children, and model-invoked tools receive -distinct allowlisted environments and filesystem views. The provider control -process sees only its invocation-private auth channel; each MCP child sees only -its own resolved secrets; shell and other model-invoked tools see neither -provider nor MCP credentials. Every view excludes gateway state, operator home, -App keys, GitHub/OCI stores, acquisition helpers, unrelated adapter auth, and -the parent environment. A target is not ready unless its adapter can enforce -these separations. This credential/state isolation is required even though -general hostile-code sandboxing remains deferred. +distinct allowlisted filesystem, environment, descriptor, secret, and network +views. The provider control process sees only its invocation-private auth +channel; each MCP child sees only its own resolved secrets; shell and other +model-invoked tools see neither provider nor MCP credentials. Every view excludes +gateway state, operator home, App keys, GitHub/OCI stores, acquisition helpers, +unrelated adapter auth, the parent environment, gateway endpoints, host +loopback, and management networks. + +The pinned backend must expose a non-bypassable synchronous hook that delegates +every MCP and model-tool spawn to the Rust helper. The helper enters the role's +mount and network namespaces, replaces the environment, closes every +non-allowlisted descriptor, and only then executes untrusted code. Codex or Pi +is unavailable when its pinned surface can bypass that hook. This enforced +credential/state/network boundary is required even though general hostile-code +sandboxing remains deferred. ### Persist Task truth, not live provider execution The gateway durably stores Task identity, the canonical request, idempotency -claim, selected target and source, effective configuration digest, terminal -status, Artifact metadata, and retained evidence under the configured state -directory. A provider session is not a durable recovery checkpoint. - -An identical idempotency replay returns the existing Task. Reusing the key with -a different canonical request conflicts. Because the initial service has no -caller identity, the idempotency namespace and Task visibility are gateway-wide. - -Terminal Task records, Artifacts, events, and invocation claims expire -atomically after the configured TTL. The retained-count limit never evicts an +claim, selected target and source, effective configuration digest, one execution +lease, internal outcome intent, Artifact bytes, retained evidence, and +containment identity under the configured state directory. The official A2A +HTTP+JSON transport wraps an AllAgents-owned request handler; typed transactions +execute in the Rust helper's descriptor-rooted SQLite VFS. One transaction +arbitrates `createOrReplay`, UUIDv7 Task creation, execution-lease acquisition, +and containment binding; another atomically settles terminal status, result or +failure, evidence, Artifacts, cleanup, and lease release. A provider session is +not a durable recovery checkpoint. + +At most one Task holds the execution lease from acquisition through final +evidence collection. A second otherwise-valid request settles failed with +`execution_capacity_unavailable`. Its transient empty containment set is +destroyed after settlement without releasing the start gate or launching a +helper child. An identical idempotency replay returns the existing Task. Reusing +the key with a different canonical request conflicts. Clients generate at least +128 bits of randomness +once per logical invocation and reuse the same key plus request after an +ambiguous transport failure. Because the initial service has no caller identity, +the idempotency namespace and Task visibility are gateway-wide. + +Terminal Task records, Artifacts, events, and invocation claims expire in one +transaction after the configured TTL. The retained-count limit never evicts an unexpired Task; the gateway rejects new admission until expiry frees capacity. -State-store integrity or durability failure stops admission and prevents the -gateway from acknowledging creation or reporting terminal success. +State-store integrity, VFS, helper protocol, or durability failure stops +admission and prevents the gateway from acknowledging creation or reporting +terminal success. -On gateway restart, interrupted nonterminal Tasks settle failed; provider work -is not resumed or automatically replayed. A new invocation may start fresh. +On gateway restart, interrupted nonterminal Tasks settle failed only after +containment reconciliation; provider work is not resumed or automatically +replayed. A new invocation may start fresh. ### Make cancellation, evidence, and cleanup explicit -The gateway supervises every acquisition and provider process set. Cancellation -first invokes the provider's native abort or protocol cancellation, then applies -bounded forced termination to the complete descendant set. - -Terminal cleanup evidence is recorded only after the supervisor proves the -complete invocation process set quiescent through an enforceable, invocation- -owned containment primitive. If the platform cannot provide that guarantee, the -gateway fails readiness rather than relying on best-effort process enumeration. -If termination or proof fails, the Task records termination as unknown or -failed and the gateway rejects new work. An unmanaged foreground gateway stays -alive with poisoned readiness and continues reaping while printing the stable -containment identifier and platform recovery command. It may exit with a -nonempty set only after a validated external manager accepts cleanup ownership. - -On startup the gateway identifies every interrupted invocation's containment -set and proves it empty before binding or advertising readiness. It may -quarantine a stale filesystem root only after process quiescence is proven. A -reaping or proof failure terminalizes the Task with unknown/failed termination, -keeps readiness false, and enters the same managed or unmanaged recovery path. +The gateway supervises every acquisition and provider process set. The helper +allocates a stable empty containment set behind a start gate. The Task, +execution lease, and containment identifier commit durably before the helper may +release that gate or execute any child; a failed commit destroys the empty set. + +One durable compare-and-set arbitrates provider terminal outcome, caller +cancellation, deadline, and shutdown as an internal outcome intent while the +externally visible Task remains nonterminal. The winning intent owns the stable +result or failure code and drives one idempotent abort and quiescence path. +Cancellation first invokes the provider's native abort or protocol cancellation, +then applies bounded forced termination to the complete descendant set. + +Live provider events are bounded while execution runs. Filesystem, Git, and +produced-Artifact evidence is read only after the supervisor proves the complete +invocation process set quiescent through its invocation-owned containment. +Only then does one transaction atomically publish terminal status, the integrity +Artifact, bounded evidence, result or failure, produced Artifacts, termination, +cleanup, and lease release. If quiescence cannot be proven, that transaction +settles `execution_quiescence_unknown` without verified filesystem evidence. +The gateway rejects new work and stays alive with poisoned readiness while +continuing to reap; later recovery changes only internal recovery/readiness +state, never the settled Task. + +On startup the helper enumerates the entire project-owned containment namespace, +including unknown identifiers, and proves every set empty before binding, +releasing a retained lease, quarantining stale roots, or advertising readiness. +It prints the stable containment identifier and platform recovery command for +any nonempty set. An unsupported platform fails before binding rather than +relying on process enumeration. Terminal evidence distinguishes: @@ -401,24 +507,33 @@ source, or file contents are excluded from operational logs. ### Profile A2A instead of inventing an invocation API The gateway uses A2A Agent Cards, Messages, Tasks, Artifacts, operations, errors, -streaming, and cancellation. The Agent Card declares the AllAgents coding- -execution extension as required. Every operation that creates, returns, lists, -subscribes to, or mutates profiled Tasks or Artifacts activates -`https://allagents.dev/a2a/extensions/coding-execution/v1` through the -`A2A-Extensions` header. Unsupported calls receive the standard A2A extension- -support error, and responses echo the activated URI. - -The versioned extension carries the invocation key, execution target, closed +streaming, and cancellation. Its Agent Card advertises one interface with +`protocolBinding: "HTTP+JSON"`, `protocolVersion: "1.0"`, and +`capabilities.streaming: true`. The coding- +execution `AgentExtension` is required and has strict +`params: { targets: TargetId[] }`, populated from ready built-in and explicitly +gateway-enabled launcher-backed targets. It does not publish paths, commands, +arguments, environment selectors, credentials, exact source authorization +details, or transient worker state. + +Every A2A HTTP+JSON request carries `A2A-Version: 1.0`. Every operation that +creates, returns, lists, subscribes to, or mutates profiled Tasks or Artifacts +also activates `https://allagents.dev/a2a/extensions/coding-execution/v1` +through `A2A-Extensions`. Missing activation receives +`ExtensionSupportRequiredError`; an unsupported protocol version receives +`VersionNotSupportedError`. Unsuccessful HTTP responses use the A2A +`google.rpc.Status` JSON envelope with typed `google.rpc.ErrorInfo` details; +validation also uses `google.rpc.BadRequest`, never JSON-RPC error carriers. + +The published versioned extension specification defines Agent Card params, +activation, request/idempotency/replay, errors, and terminal Task/Artifact +schemas. Its request carries the invocation key, execution target, closed workspace source, bounded deadline, and optional bounded result schema in its own strict `Message.metadata` member without rejecting unrelated A2A metadata. -Every terminal Task has one fixed-name, versioned integrity Artifact plus zero -or more produced Artifacts. Breaking extension versions receive versioned cards -and endpoints rather than silent fallback. - -The Agent Card advertises built-in and explicitly exposed launcher-backed -targets through an allowlisted capability projection. It does not publish local -paths, commands, arguments, environment selectors, credentials, exact source -authorization details, or transient worker state. +The request Message lists the URI in `Message.extensions`. Every terminal Task +has one fixed-name, versioned integrity Artifact whose `Artifact.extensions` +lists the URI, plus zero or more produced Artifacts. Breaking extension versions +receive versioned cards and endpoints rather than silent fallback. ACP, app-server, SDK, and RPC protocols remain backend implementation details. W3C Trace Context may propagate correlation through HTTP and child-process @@ -439,22 +554,33 @@ retry. Consumers own those concerns. isolation, `gateway.yaml`, `worker.yaml`, remote worker protocol, or required Kubernetes deployment in the initial product. - Project and user workspace files remain the sole declaration authority for - source identities and exposed profile launchers. + source identities and gateway-enabled profile launchers. - AI Evals can express the configured repository set with named revision overrides, or select a prebuilt image through a snapshot handle, in Promptfoo YAML. Its custom provider translates that closed source choice to A2A and keeps raw origins under AllAgents operator control. -- Network reachability grants access to every exposed target and retained Task. - Operators must treat network policy as the authorization boundary. +- External network reachability grants access to every available target, + including built-in and gateway-enabled profile targets, plus every retained + Task. Operators treat network policy as the authorization boundary; invocation + descendants are isolated from that boundary and host-management networks. +- One durable execution lease enforces one active invocation independent of + consumer concurrency settings. - GitHub App credentials support private repositories without forcing every - developer to use one identity; GitHub CLI remains a local eligibility - fallback when no App installation applies. + developer to use one identity; GitHub CLI remains a local eligibility fallback + only when no App installation applies. - Direct repositories and digest-pinned OCI snapshots converge on one validated - workspace manifest and evidence contract. -- The gateway process remains a meaningful API and lifecycle boundary, but not - a hostile-code sandbox. Strong multi-tenant isolation remains future work. + workspace manifest and evidence contract. OCI metadata remains same-origin; + only layer blobs may redirect to exact operator-approved hosts. +- Gateway v1 execution is supported on Linux x64/arm64 with the packaged state/ + security helper, descriptor-rooted SQLite VFS, cgroup v2, mount/network + namespaces, nftables, pidfds, and safe-file operations. Matching helper + packages publish and verify before the root package; unsupported hosts or + missing capabilities fail before binding rather than degrading containment. +- The gateway process remains a meaningful API and lifecycle boundary, but not a + hostile-code sandbox. Strong multi-tenant isolation remains future work. - Codex and Pi share one conformance suite while retaining bounded native - evidence and honest capability differences. + evidence and honest capability differences. A backend is unavailable unless + its pinned surface can delegate every MCP/tool spawn through the helper. - A future deployment configuration becomes justified only when the product needs multiple worker routes, tenants, credential policies, custom materializers, centralized storage, or other operator-selected variants. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 07ec97e9..b2e14039 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -14,8 +14,8 @@ execution: code ## Goal Capsule - **Objective:** A developer can run one trusted-network A2A endpoint for one - AllAgents workspace and invoke built-in or explicitly exposed profile targets - against either the complete configured Git repository set, with optional + AllAgents workspace and invoke built-in or explicitly gateway-enabled profile + targets against either the complete configured Git repository set, with optional named revision overrides, or a digest-pinned OCI workspace snapshot. AI Evals can configure either source mode in Promptfoo YAML through a custom provider without sending origins. @@ -28,9 +28,9 @@ execution: code own source and profile declarations. A2A 1.0 owns core wire semantics. - **Execution order:** Capture a red built-CLI E2E for the missing gateway; freeze schemas and configuration projection; implement the Task store, A2A - server, acquisition, supervisor, Codex, and Pi; run a final implementation - review and fix important findings; then run the green built-CLI E2E, - repository gates, and documentation validation. + server, supervisor/helper, acquisition, Codex, and Pi; run a final + implementation review and fix important findings; then run the green built- + CLI E2E, repository gates, and documentation validation. - **Stop conditions:** Do not add application authentication, `gateway.yaml`, `worker.yaml`, remote worker routing, caller-supplied URLs or commands, mutable OCI tags, selected-provider failure fallback, evaluation behavior, or @@ -78,8 +78,8 @@ registry, or another profile configuration file for the initial use case. - A3. **Backend adapter:** The Codex or Pi implementation translating native automation events and cancellation into the common contract. - A4. **Operator/developer:** The person who selects the project workspace, - exposes profile launchers, supplies process flags and credential handles, and - controls network access. + gateway-enables profile launchers, supplies process flags and credential + handles, and controls network access. - A5. **GitHub/OCI source:** The remote content service used only during the acquisition phase. @@ -92,18 +92,18 @@ registry, or another profile configuration file for the initial use case. application authentication or caller ownership. Explicit `0.0.0.0` binding is valid. (session-settled: user-directed.) Governs R4-R5. - **Reuse workspace configuration.** Project `workspace.yaml` owns sources; - user `workspace.yaml` owns profiles, launchers, and exposure. There is no - `gateway.yaml`. (session-settled: user-directed.) Governs R6-R8, R18. + user `workspace.yaml` owns profiles, launchers, and gateway enablement. There + is no `gateway.yaml`. (session-settled: user-directed.) Governs R6-R8, R18. - **Support two acquisition modes.** Direct declared repositories and named, digest-pinned OCI workspace snapshots converge on one manifest and evidence contract. (session-settled: user-directed.) Governs R9-R11. - **Use App-first GitHub credential eligibility.** Prefer an applicable GitHub App; use a configured `gh` account only when no App installation applies; never fall back after selected-App failure. (session-settled: user-directed.) - Governs R10-R11. + Governs R12. - **Keep a typed backend seam.** Codex SDK and Pi RPC are the complete initial backend set. Launcher-backed profiles resolve through those adapters rather - than executing generated wrapper files. Governs R7-R8, R12-R15. + than executing generated wrapper files. Governs R7-R8, R13-R15. - **Persist Task truth, not provider sessions.** Restart settles interrupted work failed; it never resumes or automatically replays provider execution. Governs R5, R13-R16. @@ -120,16 +120,29 @@ registry, or another profile configuration file for the initial use case. - R1. Implement A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, `GetTask`, `ListTasks`, `CancelTask`, streaming send, and active Task - subscription when advertised. The Agent Card declares + subscription. Every A2A request carries `A2A-Version: 1.0`; another version + receives `VersionNotSupportedError`. The Agent Card advertises exactly one + absolute interface URL with `protocolBinding: "HTTP+JSON"`, + `protocolVersion: "1.0"`, and `capabilities.streaming: true`. It declares `https://allagents.dev/a2a/extensions/coding-execution/v1` with - `required: true`. Every operation that creates, returns, lists, subscribes to, - or mutates profiled Tasks or Artifacts must include - `A2A-Extensions: https://allagents.dev/a2a/extensions/coding-execution/v1`; - responses echo the activated URI, and unsupported calls receive A2A + `required: true` and strict `params: { targets: TargetId[] }`, populated from + ready built-in and gateway-enabled profile targets. Production interface URLs + use HTTPS; direct HTTP is limited to loopback development. Every operation + that creates, returns, lists, subscribes to, or mutates profiled Tasks or + Artifacts includes that URI in `A2A-Extensions`; missing activation receives `ExtensionSupportRequiredError`. + + Honor both `SendMessageConfiguration.returnImmediately` modes. `ListTasks` + implements every standard filter, cursor pagination, `pageSize` 1-100 with a + default no greater than 50, descending status-timestamp order, and required + `tasks`, `nextPageToken`, `pageSize`, and `totalSize` fields. + `nextPageToken` is present and empty on the final page. With the default + `includeArtifacts: false`, each returned Task omits `artifacts`; `true` + includes the field. - R2. Generate a strict versioned request schema from Zod and place it only at - `Message.metadata[extensionUri]`. Strict objects reject every unlisted member. - V1 uses these wire scalars: + `Message.metadata[extensionUri]`; the Message also lists `extensionUri` in + `Message.extensions`. Strict objects reject every unlisted member. V1 uses + these wire scalars: - `InvocationKey` matches `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`. - `ConfigName` and `TargetId` match `^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`. @@ -169,38 +182,53 @@ registry, or another profile configuration file for the initial use case. formats, defaults, coercion, non-finite numbers, duplicate canonical enum values, and unknown keywords are rejected. The canonical result schema is at most 64 KiB, 256 nodes, and 32 levels deep. Repository revision count cannot - exceed declared repositories. The Message contains exactly one `TextPart` - whose UTF-8 prompt is 1 byte to 1 MiB; other Part kinds are rejected. Only the - extension-owned metadata object is strict; unrelated A2A metadata and other - activated-extension keys are preserved or ignored according to A2A. - Canonicalization materializes defaults, normalizes extension strings to UTF-8 - NFC, sorts record keys, and hashes RFC 8785 extension JSON plus prompt bytes. - Do not add `Task.extensions` or backend-specific public fields. + exceed declared repositories. The Message contains exactly one `Part` with + `text` set to a UTF-8 prompt of 1 byte to 1 MiB; other Part content fields are + rejected. Only the extension-owned metadata object is strict; unrelated A2A + metadata and other activated-extension keys are preserved or ignored + according to A2A. Canonicalization materializes defaults, normalizes extension + strings to UTF-8 NFC, sorts record keys, and hashes RFC 8785 extension JSON + plus prompt bytes. Do not add `Task.extensions` or backend-specific public + fields. + + A client generates an opaque invocation key with at least 128 bits of + randomness once per logical execution, durably reuses that key and identical + canonical request after an ambiguous transport failure, and creates a new key + only for intentionally new execution. A2A `messageId` remains Message identity + and does not replace the extension idempotency key. - R3. One valid new request creates one addressable Task. Follow-up messages to an existing Task are unsupported. Every terminal Task has exactly one integrity Artifact plus zero or more produced Artifacts. The integrity Artifact has `artifactId` and `name` equal to - `allagents.execution-integrity` and one `DataPart` whose strict - `allagents.execution-integrity/v1` object has the following normative wire - shape. `SafeUInt` is an integer 0-9,007,199,254,740,991; `ShortText` is valid - UTF-8 of at most 4096 bytes; `ArtifactId` matches + `allagents.execution-integrity`, lists `extensionUri` in + `Artifact.extensions`, and has one `Part` with `data` set to the strict + `allagents.execution-integrity/v1` object and `mediaType: + "application/json"`. Its `taskId` equals the enclosing A2A `Task.id`. + `SafeUInt` is an integer 0-9,007,199,254,740,991; `ShortText` is valid UTF-8 + of at most 4096 bytes; `ArtifactId` matches `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`; and `MediaType` is a valid RFC 6838 media type of at most 255 ASCII bytes. - `version` is the literal `"1"`; `taskId` is a lowercase canonical UUIDv7; and `target` is `TargetId`. - `sourceIdentity` is either - `{ kind: "repositories", repositories }` or + `{ kind: "repositories", complete, repositories }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, - workspaceManifestDigest: Digest, repositories }`. - `repositories` contains 1-64 unique strict entries + workspaceManifestDigest: Digest, layerDigests, complete, repositories }`. + `complete` is boolean. `layerDigests` contains 0-64 `Digest` values in + manifest order. `repositories` contains 0-64 unique strict entries `{ name: ConfigName, requestedRevision?: RevisionText, resolvedCommit: string, verification: "independentlyVerified" | "snapshotAttested" }`; `resolvedCommit` matches - `^[0-9a-f]{40}$`. Gateway-generated source identity, workspace-manifest - fields, evidence metadata, and provider-added metadata never contain Git - URLs, OCI repository origins, or destination paths. This guarantee does not - inspect or sanitize opaque caller prompts, provider terminal output, or - produced-Artifact payloads. + `^[0-9a-f]{40}$`. Before provider execution, `complete` must be true, + `repositories` must exactly match the configured catalog, and snapshot + identities must include every layer digest. Failed acquisition records only + verified members and sets `complete` false. + - Gateway-generated source identity, workspace-manifest fields, evidence + metadata, and provider-added metadata never contain Git URLs, OCI repository + origins, or destination paths. This guarantee applies only to + gateway-managed credentials and generated metadata; it does not inspect or + sanitize opaque prompts, terminal output, structured results, native + evidence, or produced-Artifact payloads. - optional `workspaceManifestDigest` is `Digest`. - `terminalOutput` is `{ text, truncated }`, where `text` is valid UTF-8 of at most 1 MiB and `truncated` is boolean. @@ -210,7 +238,7 @@ registry, or another profile configuration file for the initial use case. - `producedArtifacts` contains 0-128 strict entries `{ artifactId: ArtifactId, name?: ShortText, mediaType?: MediaType, size: SafeUInt, digest: Digest }`; each references one additional A2A - Artifact. + Artifact embedded in the Task. - `evidence` is `{ items, complete, truncated }`, where the booleans have their literal JSON meaning and `items` contains 0-256 strict entries `{ kind, artifactId?, digest?, summary? }`. `kind` is one of @@ -234,7 +262,7 @@ registry, or another profile configuration file for the initial use case. `{ path, keyword, message }` entries: `path` is an RFC 6901 JSON Pointer of at most 1024 bytes, `keyword` is one of the v1 `SchemaNode` member names, and `message` is `ShortText`. `reason` is one of - `notRequested | providerDidNotReturn | providerFailed | cancelled | + `notRequested | providerDidNotReturn | providerFailed | canceled | deadlineExceeded | invalidProviderPayload`. Failure, rejection, and cancellation retain every available field without @@ -247,43 +275,63 @@ registry, or another profile configuration file for the initial use case. - R4. Do not authenticate application callers. Allow loopback, specific-address, and explicit `0.0.0.0` listeners. Every reachable caller may create, list, - retrieve, subscribe to, cancel, and fetch Artifacts for every Task. Document - Tailscale ACLs, firewalls, or equivalent network controls as the authorization - boundary. + retrieve, subscribe to, and cancel every Task. Artifacts are retrieved only + inside Tasks through `GetTask` or `ListTasks(includeArtifacts: true)`; v1 adds + no separate Artifact endpoint. Document Tailscale ACLs, firewalls, or + equivalent network controls as the authorization boundary. - R5. Idempotency and Task visibility are deployment-wide. Atomically and durably bind an invocation key to the canonical request, selected target, source identity, optional result-schema digest, deadline, and effective - configuration digest before acknowledging Task creation. Identical replay - returns the existing Task; a changed request conflicts. The project-specific - state root persists the canonical workspace identity and holds an exclusive - process lock. The root and all state files must be current-user owned, use - `0700`/`0600`-equivalent permissions, be disjoint from project, profile, and - invocation roots, and be opened descriptor-relatively without following - symlinks or accepting hard-linked files. Startup verifies those invariants, - store integrity, and workspace identity; terminalizes interrupted Tasks - failed; and never resumes provider work. Store open, corruption, write, - rename, or fsync failure stops admission, aborts and contains active work, - prevents terminal success, and exits only after quiescence or a validated - external manager accepts cleanup ownership. + configuration digest before acknowledging Task creation. One transactional + `createOrReplay` operation arbitrates competing requests. Identical replay + returns the existing Task; a changed request conflicts. Status and terminal + settlement are monotonic. The project-specific state root persists the + canonical workspace identity and holds an exclusive process lock. The root + and all state files must be current-user owned, use `0700`/`0600`-equivalent + permissions, be disjoint from project, profile, and invocation roots, and be + opened descriptor-relatively without following symlinks or accepting hard- + linked files. Startup verifies those invariants, store integrity, and + workspace identity; terminalizes interrupted Tasks failed; and never resumes + provider work. A durable commit fsyncs every changed file and affected + containing directory before acknowledgment. Store open, corruption, write, + transaction, rename, or fsync failure stops admission, aborts and contains + active work, prevents terminal success, and keeps the process alive with + poisoned readiness until the containment set is proven empty. **Workspace and target configuration** - R6. One gateway process serves one project workspace selected by `--workspace` or cwd. Parse its `.allagents/workspace.yaml` through the authoritative project - schema. Repository names, URLs, destinations, default revisions, workspace - projection, plugins, and named OCI snapshot repositories come only from that - declaration. + schema, then compile a gateway-only repository catalog without tightening + ordinary workspace parsing. Derive each logical name from `name` or the + existing path-basename fallback; require 1-64 unique `ConfigName` values, + collision-free normalized destinations, and one supported canonical GitHub + origin resolved through existing `source`/`repo` semantics. Repository names, + origins, destinations, default revisions, workspace projection, plugins, and + named OCI snapshot repositories come only from that declaration. A catalog + failure makes the gateway not ready. - R7. Parse `~/.allagents/workspace.yaml` through the authoritative user schema. Built-in `codex` and `pi` targets are available when ready. A launcher-bearing - profile client adds a target only when `gateway.expose: true`. Its public ID is - the globally collision-checked launcher basename and resolves to exactly one - `(profile, client)` pair. Built-in IDs are reserved under the same portable - collision key; colliding exposure is a configuration error. Initially only + profile client adds a target only when `gateway.enabled: true`. Its public ID + is the globally collision-checked launcher basename and resolves to exactly + one `(profile, client)` pair. Built-in IDs are reserved under the same portable + collision key; colliding enablement is a configuration error. Initially only Codex and Pi profile clients are executable. - R8. A request selects a declared target, may set the bounded - `deadlineSeconds`, and may provide one bounded result schema. The overall - deadline covers acquisition, publication, typed preparation, provider - execution, and evidence collection. Acquisition receives + `deadlineSeconds`, and may provide one bounded result schema. The gateway owns + one durable execution lease covering acquisition through final evidence + collection. Admission claims that lease transactionally before launching any + helper child; at most one Task may hold it. A second otherwise-valid request + is accepted as a Task and settles failed with + `execution_capacity_unavailable`. It may transiently allocate one empty, + start-gated containment set, but never releases the gate, starts acquisition, + or executes a child and must destroy that set after the failure settlement. + Lease identity is stored with the Task, survives restart, and is released only + by the final settlement transaction or startup reconciliation after the + recorded containment set is proven empty. + + The overall deadline covers acquisition, publication, typed preparation, + provider execution, and evidence collection. Acquisition receives `min(900 seconds, remaining overall deadline)`; exceeding that sub-budget fails before provider execution. Overall expiry initiates abort and bounded forced termination. Cleanup then uses its own fixed bounded budget and the @@ -291,7 +339,7 @@ registry, or another profile configuration file for the initial use case. executable path, command, argv, environment, profile settings, plugins, MCP servers, repository URLs, destination paths, credential provider, setup behavior, or permission policy. Readiness rejects missing, partial, drifted, - unsupported, or declaration-missing exposed profiles. + unsupported, or declaration-missing gateway-enabled profiles. **Workspace acquisition** @@ -301,37 +349,68 @@ registry, or another profile configuration file for the initial use case. Unknown variants, cross-variant fields, undeclared names, mutable snapshot references, malformed digests, and destination overrides fail admission. Source-mode failure never falls through to the other mode. -- R10. Repository mode materializes every configured repository required by the - selected project workspace. Caller revisions may override only a declared - repository's default revision. Canonicalize HTTPS GitHub origins, resolve and - record full commits before provider execution, use hermetic Git configuration, - disable redirects and repository-controlled secondary fetch/exec features, - verify checkout identities, and reject path collisions or escapes. +- R10. Repository mode materializes every entry in the compiled gateway catalog. + Caller revisions may override only a declared repository's default revision. + Canonicalize HTTPS GitHub origins, resolve and record full commits before + provider execution, use hermetic Git configuration, disable redirects and + repository-controlled secondary fetch/exec features, verify checkout + identities, and reject path collisions or escapes. - R11. Snapshot mode maps `snapshot` to a declared OCI repository and constructs - `@` server-side. Validate registry origin, OCI manifest - and layer digests, expected workspace-manifest digest, paths, symlinks, file - types, file/layer counts, individual and total sizes, and manifest - completeness in staging before atomic publication. Reject external or foreign - layers and cross-origin credential forwarding. The common workspace manifest + `@` server-side. V1 accepts only + `application/vnd.oci.image.manifest.v1+json` with `schemaVersion: 2` directly + at the requested digest. Reject image indexes, nested indexes, descriptor + `urls` or embedded `data`, non-distributable layers, unknown media types, and + more than 64 layers. The config descriptor must use + `application/vnd.allagents.workspace-manifest.v1+json`; its digest must equal + `workspaceManifestDigest`, and its bytes are RFC 8785 canonical JSON. Accepted + layer media types are the OCI distributable tar, gzip, and zstd variants. + Verify the raw manifest body and every config/layer descriptor size and digest + while streaming, before decoding. Apply layers base-to-top with OCI whiteout + and opaque-whiteout semantics. + + The workspace manifest must contain every compiled project repository exactly + once at its operator-declared destination; reject missing, extra, renamed, + misplaced, or duplicate repositories and undeclared generated content. Apply + these fixed v1 ceilings across all processed layers, including overwritten or + whiteouted content: 4 MiB manifest, 4 MiB config, 2 GiB total compressed + layer bytes, 8 GiB total expanded bytes, 250,000 entries, 1 GiB per regular + file, 4096 UTF-8 bytes and 128 components per path, and 1 MiB per PAX or other + extended header. Abort before crossing a limit. Validate paths, collisions, + file types, modes, links, and manifest completeness in staging before atomic + publication. Reject absolute or traversing paths, devices, sockets, sparse + files, escaping links, credentials in redirect URLs, unapproved cross-origin + redirects, and external layers. Cross-origin redirects are limited to + layer-blob `GET`/`HEAD` requests and exact operator-declared + `layerRedirectHosts`; token, manifest, and config requests remain same-origin. + Private or otherwise non-global destinations are permitted only when the exact + host is the source's declared repository host or a declared layer-redirect + host, with per-hop rebinding checks. The common workspace manifest distinguishes independently verified Git facts from snapshot-attested facts. **Credential selection and containment** - R12. Repository requests never carry credentials or select providers. For - `github.com`, determine configured-App applicability as - `eligible | ineligible | unknown` through an App-authenticated GitHub API - client, or verify an explicit installation ID against the repository. For - `eligible`, mint a fresh repository-scoped, read-only installation token - through `@octokit/auth-app` and require remaining lifetime greater than the - R8 acquisition sub-budget plus a 60-second clock-skew margin. Use the - configured GitHub CLI account only when the App is absent or applicability is - positively `ineligible`. An `unknown` result or any selected-App - configuration, authentication, minting, permission, repository, rate-limit, - or service failure terminates acquisition without `gh` fallback. Run - `gh auth token --hostname github.com --user ` with ambient token + `github.com`, a configured App lookup returning installation coverage is + `eligible`. A 404 is `ineligible` only after the repository's existence is + independently proven through the configured GitHub CLI identity; an + uncorroborated 404, 401, 403, 429, timeout, or 5xx is `unknown`. An explicit + installation ID is eligible only after positive repository-coverage + verification. For `eligible`, call `@octokit/auth-app` with `refresh: true` + and the exact repository selection to mint a new read-only installation token + for every acquisition. Validate its repository selection, permissions, + creation time, and expiry, and require remaining lifetime greater than the R8 + acquisition sub-budget plus a 60-second clock-skew margin. + + Use the configured GitHub CLI account only when the App is absent or + applicability is positively `ineligible`. An `unknown` result or any + selected-App configuration, authentication, minting, permission, repository, + rate-limit, or service failure terminates acquisition without `gh` fallback. + Run `gh auth token --hostname github.com --user ` with ambient token variables removed. Deliver either token only through an invocation-scoped Git - credential helper and destroy it before typed preparation or provider - execution. OCI credentials likewise exist only during snapshot acquisition. + credential helper. Revoke an App token after acquisition and fail before + provider execution if revocation cannot be confirmed; destroy all local token + material before typed preparation. OCI credentials likewise exist only during + snapshot acquisition. **Execution, evidence, and cleanup** @@ -343,48 +422,92 @@ registry, or another profile configuration file for the initial use case. discover executables as targets from `PATH`, scrape a TUI, or append public input to argv. - R14. Codex uses pinned `@openai/codex-sdk`, one fresh thread per Task, - `AbortSignal`, streamed events, optional native `outputSchema`, and an - operator-selected Codex auth-file handle. Pi uses strict RPC, invocation-owned - configuration, an operator-selected Pi auth-file handle, and one restricted - policy extension; repository extensions and unrestricted built-ins do not - auto-load. The gateway copies only the selected adapter's required auth - material into an invocation-private, read-only control-process view and - removes it during cleanup. + `AbortSignal`, streamed events, and an operator-selected Codex auth-file + handle. It passes native `outputSchema` only when the public schema has an + object root, every object's `required` set equals its property set, nesting is + at most 10 levels, and every keyword is supported by the pinned model/API. + Other valid public schemas use explicit JSON prompt guidance plus the common + gateway-side validator without a native schema. Pi uses strict RPC, + invocation-owned configuration, an operator-selected Pi auth-file handle, and + one restricted policy extension; repository extensions and unrestricted + built-ins do not auto-load. The gateway copies only the selected adapter's + required auth material into an invocation-private, read-only control-process + view and removes it during cleanup. - R15. Acquire into a private staging root and atomically publish the invocation workspace. Run only adapter-owned typed preparation that projects validated project/profile settings, plugins, and MCP declarations through existing deterministic transforms; never execute project or user `setup` entries or - other configured shell commands. Enforce distinct process views: + other configured shell commands. + + Enforce distinct process views: - the provider control process receives only its invocation workspace, minimum non-secret profile configuration, and adapter auth channel; - each MCP child receives only its own resolved secret references; and - model-invoked shell/tools receive the workspace and no provider or MCP credentials. - All views exclude gateway state, operator home, App keys, GitHub/OCI stores, - source helpers, unrelated adapter credentials, and the parent environment. - Fail readiness for a target when its adapter cannot enforce those separations. - Acquisition credentials and mounts are absent first. Treat the mutated - workspace as untrusted during evidence collection: use descriptor-relative - no-follow reads; reject hard links, special/sparse files, path replacement, - out-of-root targets, and `.git` gitdir/core.worktree/alternates escapes; and - run Git inspection with hermetic configuration that disables hooks, filters, - drivers, fsmonitor, pagers, helpers, and external commands. + + The pinned backend must expose one non-bypassable synchronous spawn hook for + every MCP and model-tool process. The hook delegates execution to the security + helper, which enters the role-specific mount and network namespaces, replaces + the environment, closes every non-allowlisted descriptor, and only then + executes untrusted code. A backend that can spawn any tool without this hook + is not a v1 target and fails readiness; conformance fixtures alone cannot waive + that requirement. All views exclude gateway state, operator home, App keys, + GitHub/OCI stores, source helpers, unrelated adapter credentials, and the + parent environment. + + Invocation network namespaces cannot route to host loopback, any gateway bind + or advertised address, operator management networks, or ingress proxies. + Provider and MCP egress is default-deny except for role-specific destinations + compiled from adapter and MCP configuration; every resolved address is checked + at connection time, and gateway/host-management destinations remain denied + even when a hostname resolves to them. Model tools receive no network unless + the adapter's explicit policy grants similarly constrained egress. Fail target + readiness unless all filesystem, credential, descriptor, and network + separations are enforceable. + + Acquisition credentials and mounts are absent first. Capture bounded provider + events while the process is live. After the provider reports terminal, abort + and terminate its complete containment set and prove it empty before reading + Git state, hashing or copying files, or describing produced Artifacts as + verified. Treat the mutated workspace as untrusted: use descriptor-relative + no-follow reads; revalidate identity and size after open; reject hard links, + special/sparse files, path replacement, out-of-root targets, and `.git` + gitdir/core.worktree/alternates escapes; and run Git inspection with hermetic + configuration that disables hooks, filters, drivers, fsmonitor, pagers, + helpers, and external commands. If quiescence cannot be proven, retain only + truthful partial process evidence; do not publish filesystem evidence or + produced Artifacts as verified. - R16. Supervise the complete acquisition/provider descendant set inside an invocation-owned OS containment primitive whose membership children cannot - escape. Fail readiness when the platform cannot enforce and inspect that - boundary. Cancellation persists intent with an atomic state transition before - native abort, then applies bounded forced termination. A terminal commit that - wins first makes later cancellation not cancelable; cancellation intent that - wins settles cancelled after quiescence. Terminal cleanup evidence requires - proof that the containment set is empty. Failure records termination - unknown/failed and rejects admission. An unmanaged foreground gateway remains - alive with poisoned readiness and continues reaping; it prints the stable - containment identifier and platform recovery command. It may exit with a - nonempty set only after a validated external manager accepts ownership. Startup - proves every interrupted set empty before it may quarantine stale roots or - advertise readiness. Graceful shutdown stops admission atomically, persists - shutdown/cancellation intent, drains or aborts active work within a bounded - grace period, proves quiescence, settles once, and only then exits. + escape. Allocate its stable identifier and empty set first, then commit that + identity with the Task and execution lease before the helper may release its + start gate or execute any child. A failed commit destroys the still-empty set. + Startup enumerates the entire project-owned containment namespace, reconciles + both recorded and unknown identifiers, and refuses readiness while any + unknown or nonempty set remains. + + One durable compare-and-set arbitrates provider terminal outcome, caller + cancellation, overall deadline, and shutdown as an internal `outcomeIntent` + while the externally visible Task remains nonterminal. The winning intent + owns the stable result or failure code and drives one idempotent abort and + quiescence path. Only after quiescence, safe evidence collection, produced- + Artifact verification, and cleanup does one settlement transaction atomically + write terminal Task status, result/failure, bounded evidence, exactly one + integrity Artifact, produced Artifacts, termination outcome, lease release, + and cleanup outcome. + + Cancellation intent persists before native abort, followed by bounded forced + termination. If quiescence cannot be proven, settle once with + `execution_quiescence_unknown`, no verified filesystem evidence, and immutable + unknown/failed termination; reject admission, keep readiness false, and leave + the process alive to continue reaping. Later recovery changes only internal + recovery/readiness state, never the settled Task. Print the stable containment + identifier and platform recovery command. Startup proves every interrupted set + empty before it may quarantine stale roots or advertise readiness. Graceful + shutdown stops admission atomically, commits shutdown intent, drains or aborts + active work within a bounded grace period, follows the same settlement path, + and only then exits. **Scope and configuration** @@ -392,29 +515,44 @@ registry, or another profile configuration file for the initial use case. repetitions, experiment scheduling, or automatic Task retry. - R18. Do not add `gateway.yaml` or `worker.yaml`. Process configuration uses the exact CLI flags and environment variables in the Configuration Contract - for listener, workspace, state/retention, GitHub, OCI, and Codex/Pi auth-file - handles. Secret values never enter workspace files, requests, logs, Tasks, - Artifacts, retained workspaces, or model-invoked tool environments. + for listener, advertised interface URL, workspace, state/retention, GitHub, + OCI, and Codex/Pi auth-file handles. The listener also exposes unauthenticated + metadata-only `/healthz` and `/readyz` endpoints outside A2A: liveness returns + 200 while the process can serve; readiness returns 200 only while new + admission is safe and otherwise 503. They reveal no targets, sources, paths, + or failure details and do not require A2A headers. Gateway code never copies + acquisition or provider credential values into generated workspace files, + requests, logs, Task/Artifact metadata, retained workspaces, or model-tool + environments. This is not a redaction guarantee for opaque prompts, provider + output, structured results, native evidence, or produced-Artifact payloads. - R19. Document AI Evals consumption through a Promptfoo custom JavaScript/TypeScript provider implementing Promptfoo's `ApiProvider`. - `constructor(options: ProviderOptions)` retains `options.id`, validates - `options.config`, and `id()` returns the retained ID. Static config contains - the gateway endpoint, target ID, and exactly one closed source mode: - repository mode materializes the complete configured repository set and - carries only an optional revision map keyed by declared repository name; - snapshot mode carries one declared snapshot name with OCI and workspace- - manifest digests. `callApi(prompt, context, options)` may apply the exact - `context.vars.allagentsSource` leaf overrides defined below. Dynamic - repository revisions must be full lowercase 40-hex commit IDs; dynamic - snapshot values must be full lowercase `sha256:` digests. Source kind, - snapshot name, and repository origins never vary per test. Unknown members, - revision names absent from static config, URLs, destinations, tags, - credentials, commands, and permission policy fail before submission. - `options?.abortSignal` and the provider's bounded deadline both invoke A2A - `CancelTask` after acceptance. One `callApi` creates one A2A Task and maps - terminal output, usage, Task/Artifact IDs, structured result, and logical - provenance into `ProviderResponse`; admission or terminal failure maps to - `error`. AI Evals owns the provider implementation. AllAgents publishes the + `constructor(options: ProviderOptions)` requires and retains a nonempty + `options.id`, validates `options.config`, and `id()` returns that ID. Static + config contains the + gateway endpoint, target ID, and exactly one closed source mode: repository + mode materializes the complete configured repository set and carries only an + optional revision map keyed by declared repository name; snapshot mode carries + one declared snapshot name with OCI and workspace-manifest digests. + `callApi(prompt, context?, options?)` may apply the exact + `context?.vars?.allagentsSource` leaf overrides defined below; missing context + means no override. Dynamic repository revisions must be full lowercase + 40-hex commit IDs; dynamic snapshot values must be full lowercase `sha256:` + digests. Source kind, snapshot name, and repository origins never vary per + test. Unknown members, revision names absent from static config, URLs, + destinations, tags, credentials, commands, and permission policy fail before + submission. + + The provider sends `SendMessage` with `configuration.returnImmediately: true`, + captures the accepted Task ID, and calls `SubscribeToTask`; a terminal-before- + subscribe race or broken stream falls back to `GetTask` and resubscription + within the same deadline. A deadline or `options?.abortSignal` issues exactly + one `CancelTask` with a fresh bounded cleanup signal rather than the already + aborted request signal. One `callApi` creates one A2A Task and maps terminal + output, usage, Task/Artifact IDs, structured result, and logical provenance + into `ProviderResponse`. Admission or terminal failure maps a safe human + message to `error` and stable `code`, `retryable`, and accepted `taskId` to + `metadata`. AI Evals owns the provider implementation. AllAgents publishes the protocol and YAML examples without importing Promptfoo provider code or adding Promptfoo as a runtime dependency. @@ -422,169 +560,220 @@ registry, or another profile configuration file for the initial use case. - F1. **Start and advertise** 1. Resolve cwd or `--workspace`, user workspace, project-specific state root, - retention limits, listen address, source credentials, and provider auth + retention limits, listen address, advertised interface URL, source + credentials, and provider auth handles. + 2. Validate state-root ownership, permissions, links, disjointness, workspace + identity, compiled repository catalog, snapshots, target namespace, backend + availability, profile state, Linux containment/helper availability, + provider/MCP/tool mount, descriptor, and network views, and credential handles. - 2. Validate state-root ownership, permissions, links, disjointness, and - workspace identity; acquire the exclusive lock; validate repositories, - snapshots, target namespace, backend availability, profile state, - containment, separate provider/MCP/tool views, and credential handles. - 3. Reconcile interrupted Tasks and prove every stale containment set empty - before quarantining filesystem roots. - 4. Bind the requested address, including `0.0.0.0` when explicit, and publish - one Agent Card whose required extension and allowlisted targets match the + 3. Enumerate the entire project-owned containment namespace. Reconcile + recorded and unknown identifiers and prove every set empty before + quarantining filesystem roots or releasing a retained execution lease. + 4. Bind the requested address, including `0.0.0.0` when explicit; serve + metadata-only health/readiness probes; and publish one Agent Card whose + absolute interface URL, required extension, and target allowlist match the validated configuration. - F2. **Acquire repositories and execute** - 1. Negotiate the required extension and validate the strict request, one text - prompt, target, repository-name/revision map, result schema, deadline, and - deployment-wide idempotency claim. - 2. Durably commit the claim and Task before acknowledgment; create the - invocation containment and staging root. - 3. For each declared repository, classify App applicability, select App or - `gh` only by eligibility, resolve the revision, fetch hermetically, verify - the commit, and remove credentials. + 1. Negotiate A2A version and the required extension, then validate the strict + request, one text Part, target, repository-name/revision map, result schema, + deadline, and deployment-wide idempotency claim. + 2. Ask the helper to allocate a stable empty containment set behind a start + gate. In one transaction, create or replay the claim and Task, acquire the + execution lease, and bind the containment identifier before acknowledgment. + Capacity failure settles the Task with `execution_capacity_unavailable`, + then destroys the empty set without releasing the gate or launching a + child. Commit failure likewise destroys the empty set. + 3. Release the start gate. For each declared repository, classify App + applicability, select App or `gh` only by eligibility, resolve the revision, + fetch hermetically, verify the commit, revoke an App token, and remove every + acquisition credential. 4. Publish the complete workspace, run typed preparation, invoke the isolated - adapter, validate any structured result, collect evidence through safe - reads, terminate descendants, clean up, and settle the Task once. + adapter, and validate any structured result while capturing live events. + Terminate and prove the containment set empty before safe filesystem/Git + evidence reads and produced-Artifact verification. Atomically settle the + terminal Task, evidence, Artifacts, cleanup, and lease release. - F3. **Acquire an OCI snapshot and execute** - 1. Resolve the named snapshot repository and digest-pinned reference. - 2. Authenticate if required, pull and verify the OCI manifest and layers, - extract safely, and validate the workspace-manifest digest. + 1. Perform the same version/extension validation, gated empty-containment + allocation, and atomic claim+Task+lease+containment commit as F2. + 2. Resolve the named snapshot repository and digest-pinned reference. + Authenticate if required; pull and verify the direct image manifest, + workspace-manifest config blob, and distributable layers; apply changesets + in order; enforce all limits; and validate the exact project catalog. 3. Remove registry credentials, publish atomically, run typed preparation, - invoke the isolated adapter, collect safe evidence, clean up, and settle. + invoke the isolated adapter, and capture live events. Terminate and prove + quiescence before safe filesystem evidence and verified produced Artifacts, + then perform the same atomic settlement and lease release as F2. - F4. **Cancel** 1. Atomically persist cancellation intent if the Task remains cancelable. 2. Abort acquisition or provider work, escalate within the bounded termination - budget, prove containment quiescence, preserve partial evidence, clean up, - and settle cancelled. + budget, prove containment quiescence, preserve truthful partial evidence, + clean up, and settle canceled. 3. Repeated cancellation while intent is pending does not re-signal work. Cancellation after any terminal state returns A2A `TaskNotCancelableError`. - F5. **Shut down** 1. Stop new admission before signaling active work. - 2. Persist shutdown cancellation intent, abort and escalate, drain evidence, + 2. Persist shutdown intent, abort and escalate, drain live process evidence, prove quiescence, and settle the accepted Task once. 3. Exit only after durable settlement and empty containment. If proof fails, - unmanaged mode remains alive, not ready, and continues reaping while - printing the platform recovery command. Managed mode may exit only after - its validated external manager accepts containment ownership. + remain alive, not ready, and continue reaping while printing the stable + containment identifier and platform recovery command. - F6. **Invoke from Promptfoo** 1. Promptfoo constructs the AI Evals-owned TypeScript provider with `ProviderOptions`; the provider retains the ID and validates `options.config` containing the private-network endpoint, target, and one closed source-mode object. - 2. `callApi(prompt, context, options)` applies only valid - `context.vars.allagentsSource` leaf overrides, creates one invocation key, - and sends one A2A Message with the prompt and required extension. - 3. The provider waits or streams until terminal. Its deadline or - `options?.abortSignal` sends `CancelTask` once after acceptance and waits - for the same terminal cleanup path. - 4. It returns terminal text or validated structured output in - `ProviderResponse.output`; maps `inputTokens -> prompt`, + 2. `callApi(prompt, context?, options?)` applies only valid + `context?.vars?.allagentsSource` leaf overrides, creates and retains one + high-entropy invocation key, and sends one A2A Message with + `configuration.returnImmediately: true`. + 3. After receiving the Task ID, subscribe to terminal updates. Resolve a + terminal-before-subscribe or disconnected-stream race through `GetTask` + and bounded resubscription. Deadline or abort sends `CancelTask` once with + a fresh cleanup signal. + 4. Return terminal text or validated structured output in + `ProviderResponse.output`; map `inputTokens -> prompt`, `outputTokens -> completion`, `cachedInputTokens -> cached`, and - `totalTokens -> total`; and puts other usage plus Task, Artifact, logical - source, termination, and cleanup facts in `metadata`. Admission or terminal - execution failure returns `error`. + `totalTokens -> total`; and put other usage plus Task, Artifact, logical + source, termination, cleanup, and stable failure facts in `metadata`. + Admission or terminal failure returns a safe `error`. ### Acceptance Examples - AE1. A caller on a permitted Tailscale or firewalled network discovers the - gateway bound to `0.0.0.0`, selects `codex-review`, and receives one durable - Task without presenting an application credential. -- AE2. Any reachable caller can list, retrieve, cancel, and fetch Artifacts for - a Task created by another reachable caller; documentation states this shared - trust model without implying tenant privacy. -- AE3. A launcher-bearing Codex profile without `gateway.expose: true` is absent - from discovery and rejected when selected. An exposed but drifted profile - fails readiness/new admission. -- AE4. A multi-client profile exposes `codex-review` and `pi-review` as distinct - targets. Both resolve through adapters; neither generated wrapper is executed. + gateway through its configured HTTPS interface URL while it is bound to + `0.0.0.0`, selects `codex-review`, and receives one durable Task without an + application credential. +- AE2. Any reachable caller can list, retrieve, and cancel a Task created by + another reachable caller and inspect its embedded Artifacts through `GetTask` + or `ListTasks(includeArtifacts: true)`; documentation states this shared trust + model without implying tenant privacy. +- AE3. A launcher-bearing Codex profile without `gateway.enabled: true` is + absent from discovery and rejected when selected. An enabled but drifted + profile fails readiness/new admission. +- AE4. A multi-client profile gateway-enables `codex-review` and `pi-review` as + distinct targets. Both resolve through adapters; neither generated wrapper is + executed. - AE5. Repository mode accepts declared names and revision overrides, rejects an undeclared name or URL override, and records the resolved full commits. -- AE6. An applicable GitHub App mints a fresh repository-scoped token whose - lifetime exceeds the acquisition sub-budget plus skew. A repository with no - applicable installation uses the configured `gh` account. Unknown App - applicability, auth, or mint failure does not fall through to `gh`. -- AE7. Snapshot mode accepts a declared snapshot name and matching OCI/workspace - digests, rejects mutable tags, traversal, foreign layers, digest mismatch, or - undeclared registry repositories, and publishes only after full validation. -- AE8. Repository and snapshot modes produce the same workspace-manifest shape, - while OCI-contained commit identities remain marked snapshot-attested unless - independently verified. -- AE9. Identical invocation-key replay returns the original Task. Reusing the key - with a changed target, source, prompt, or result schema conflicts. +- AE6. An applicable GitHub App bypasses its token cache, mints a new + repository-scoped read-only token with adequate lifetime, validates the token, + and revokes it after acquisition. A corroborated existing repository with no + applicable installation uses the configured `gh` account. An uncorroborated + 404, unknown applicability, auth, mint, validation, or revocation failure does + not fall through to `gh` or start the provider. +- AE7. Snapshot mode accepts a direct image manifest with matching manifest, + config/workspace, and layer digests; applies gzip/zstd layers and whiteouts in + order; and enforces every fixed limit. Same-origin metadata redirects work; + only layer requests may cross origin to an exact declared host, with + credentials stripped and every resolved address checked. Mutable tags, + indexes, unknown/non-distributable media, descriptor URLs/data, traversal, + foreign layers, digest/size mismatch, malformed whiteouts, undeclared + repositories, redirect loops/rebinding, non-global destinations not declared + for that source, and unapproved origins fail. +- AE8. Repository and snapshot modes produce the same workspace-manifest shape + and exact compiled repository set/layout. OCI-contained commit identities are + snapshot-attested unless independently verified; source identity includes + completeness and ordered layer digests without origins. +- AE9. Identical invocation-key replay, including after a lost response, returns + the original Task. Reusing the key with a changed target, source, prompt, or + result schema conflicts; separate high-entropy keys create separate Tasks. - AE10. Cancellation during Git, OCI pull, Codex, or Pi terminates the complete - process set and records cleanup. Unproved quiescence poisons readiness; an - unmanaged foreground process stays alive and reaps, while managed exit - requires accepted external cleanup ownership. -- AE11. Restart turns interrupted Tasks into one terminal failure and never - resumes a provider session. Terminal Tasks and Artifacts remain retrievable - until expiry. + process set and records cleanup. Unproved quiescence poisons readiness; the + gateway stays alive, rejects admission, and continues reaping until empty. +- AE11. Kill fixtures before and after empty-containment creation, durable + Task/lease/containment binding, child clone, start-gate release, and response + acknowledgment leave no unrecorded live set. Restart enumerates the full + project-owned namespace, refuses unknown/nonempty sets, turns interrupted + Tasks into one terminal failure, never resumes a provider session, and keeps + terminal Tasks and embedded Artifacts retrievable until expiry. - AE12. A valid structured result survives later check or evidence failure as a valid result with an overall failed Task; invalid or absent results are never published as valid. -- AE13. An exposed launcher named `codex`, `pi`, or a portable case-equivalent - fails configuration compilation instead of shadowing a built-in target. +- AE13. A gateway-enabled launcher named `codex`, `pi`, or a portable case- + equivalent fails configuration compilation instead of shadowing a built-in + target. - AE14. Two gateways for different workspaces use distinct private state roots; a second process for the same root fails the exclusive lock. Wrong-owner, - permissive, linked, hard-linked, or overlapping roots fail startup. Store - fault injection cannot acknowledge an uncommitted Task or false success. -- AE15. Deadline expiry during Git, OCI, preparation, Codex, Pi, or evidence - initiates one abort/termination path and retains truthful partial evidence. + permissive, linked, hard-linked, or overlapping roots fail startup. The real + helper VFS rejects database, WAL, SHM, journal, temporary-file, symlink, + hard-link, and rename-swap attacks. Process-kill fixtures at transaction, file + sync, directory sync, and response boundaries recover either the complete old + or new generation and never lose an acknowledged Task or publish false + success. +- AE15. Barrier-controlled provider-terminal, caller-cancel, deadline, and + shutdown races durably select one internal intent and one abort/quiescence + path during Git, OCI, preparation, Codex, Pi, or evidence. Subscribers observe + no terminal Task until one transaction writes status, integrity Artifact, + bounded evidence, result/failure, termination, cleanup, and lease release. + Later reaping changes only internal readiness/recovery state. - AE16. Repeated cancel while cancellation is pending is idempotent; cancel - after cancelled, completed, failed, or rejected returns + after canceled, completed, failed, or rejected returns `TaskNotCancelableError`. - AE17. A workspace containing `setup` shell entries never executes them through - gateway acquisition or startup. Built-in Codex/Pi authenticate through their - selected private control-process auth views; model-invoked tools cannot read - provider or MCP secrets, operator stores, or gateway state. -- AE18. Evidence collection rejects a provider-created escaping link, hard link, - special file, sparse-file abuse, or `.git` indirection and runs Git inspection - without repository-controlled execution hooks. + gateway acquisition or startup. Real Codex/Pi child and grandchild tool paths + are helper-mediated: filesystem, environment, inherited descriptor, `/proc`, + and magic-link probes cannot read provider/MCP secrets, operator stores, or + gateway state. Agent Card, Task operations, host loopback, bind/advertised + addresses, ingress, and management-network probes fail from every invocation + role; only compiled role egress succeeds. +- AE18. Evidence is collected only after containment quiescence. An escaping + link, hard link, special file, sparse-file abuse, replaced inode, or `.git` + indirection is rejected and Git inspection runs without repository-controlled + execution. Unknown quiescence produces no verified filesystem Artifact. - AE19. The 1001st unexpired retained Task is rejected with - `retention_capacity_exhausted`; no retained Task is evicted before TTL. -- AE20. Unrelated Message metadata survives request processing. Every profiled - A2A operation requires activation, and a terminal Task may contain the single - integrity Artifact plus referenced produced Artifacts. -- AE21. The AI Evals Promptfoo provider loads one repository-mode and one - snapshot-mode YAML instance. Repository mode materializes the complete - configured set and sends only optional revision overrides keyed by declared - name; snapshot mode sends one declared name and immutable digests. Neither - request source metadata nor response source-identity metadata contains a Git - URL, OCI repository, or destination. - Both calls return scorable `ProviderResponse.output`, the exact normalized - token mapping, and Task/Artifact/logical-provenance metadata. Per-test - repository overrides accept only full commits. An unknown variable member, - mutable revision, origin, destination, or undeclared name fails before - submission. + `retention_capacity_exhausted`; no retained Task is evicted before TTL. While + one Task holds the execution lease, a barrier-controlled second request + settles `execution_capacity_unavailable` and launches no helper child; races + and restart never produce two lease holders. +- AE20. Official HTTP+JSON client fixtures send `A2A-Version: 1.0`, exercise + required-extension activation and both `SendMessage` modes, preserve unrelated + metadata, verify standard `google.rpc.Status` errors, and cover every + `ListTasks` filter, cursor, order, response field, and artifact-inclusion rule. + A terminal Task contains one extension-marked integrity Artifact plus + referenced produced Artifacts using unified Parts. +- AE21. The AI Evals Promptfoo fixture has a top-level prompt and disables + sharing, caching, result writes, and concurrency above one. It loads one + repository-mode and one snapshot-mode provider, sends only closed logical + source data, retains one invocation key across ambiguous retries, and cancels + an accepted Task on abort. Both calls return scorable output, normalized token + usage, and Task/Artifact/logical-provenance metadata. Safe failure metadata + includes code, retryability, and accepted Task ID. Calls with omitted context + work; unknown variables, mutable revisions, origins, destinations, or + undeclared names fail before submission. ### Success Criteria - `allagents gateway serve` starts from a real workspace with no deployment YAML. -- Explicit loopback, private-interface, and `0.0.0.0` listeners work. -- The official A2A client exercises required-extension negotiation, send, - stream, get, list, subscribe, replay, cancel, terminal cancel errors, Artifact - retrieval, and expiry. -- An AI Evals-style Promptfoo custom-provider fixture consumes representative - YAML for both source modes and maps a terminal Task to `ProviderResponse` - without adding Promptfoo to the AllAgents runtime. -- Built-in Codex/Pi and exposed profile targets pass one conformance suite, - including reserved-ID collisions. +- Explicit loopback, private-interface, and `0.0.0.0` listeners work with a + distinct valid advertised interface URL; health/readiness reflect admission. +- The official A2A client exercises version and extension negotiation, both send + modes, stream, get, complete list/pagination semantics, subscribe, replay, + cancel, terminal cancel errors, Task-embedded Artifacts, standard HTTP+JSON + errors, and expiry. +- An AI Evals-style Promptfoo custom-provider fixture consumes secure-default + YAML for both source modes, propagates post-acceptance cancellation, and maps a + terminal Task to `ProviderResponse` without adding Promptfoo to the AllAgents + runtime. +- Built-in Codex/Pi and gateway-enabled profile targets pass one conformance + suite, including reserved-ID collisions and Codex native-schema gating. - Direct Git and OCI snapshot fixtures produce equivalent validated workspace - manifests and truthful provenance. -- GitHub App eligibility, unknown failure, no-installation `gh` fallback, - selected-App failure, token lifetime, containment, and OCI credential cleanup - are proven end to end. + manifests and truthful complete provenance. +- GitHub App eligibility, 404 ambiguity, unknown failure, no-installation `gh` + fallback, fresh-token validation/revocation, OCI authentication and challenge + handling, and pre-provider credential teardown are proven end to end. - No request can supply a command, executable, URL, destination, credential, mutable OCI tag, backend override, or arbitrary environment value. -- State-store fault, deadline, cancellation-race, shutdown, descendant escape, - unsafe evidence, and stale-root scenarios fail closed. -- The built CLI passes a trusted-network smoke test against project and user - workspaces created under `/tmp/`. +- State-store crash, deadline/cancellation/terminal/shutdown race, descendant + escape, unsafe evidence, and stale-root scenarios fail closed. +- The bundled CLI and packaged Linux helper pass a trusted-network smoke test + against project and user workspaces created under `/tmp/`. ### Scope Boundaries @@ -592,7 +781,7 @@ registry, or another profile configuration file for the initial use case. - A2A 1.0 HTTP+JSON and the required AllAgents extension. - One process and one active invocation at a time initially. -- Built-in and exposed profile-backed Codex/Pi targets. +- Built-in and gateway-enabled profile-backed Codex/Pi targets. - Direct declared Git repositories and named OCI workspace snapshots. - GitHub App and configured GitHub CLI acquisition credentials. - Local durable Task/evidence storage, cancellation, cleanup, and provenance. @@ -610,6 +799,8 @@ registry, or another profile configuration file for the initial use case. delivery. - OpenCode, Claude, Copilot, OMP, arbitrary CLI, and TUI adapters. - Evaluation orchestration and automatic retries. +- Non-Linux gateway execution in v1; ordinary AllAgents CLI behavior remains + cross-platform. ### Sources @@ -617,12 +808,19 @@ registry, or another profile configuration file for the initial use case. - [AHP decision inputs](../research/agent-host-protocol-decision-inputs.md) - [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) - [Source credential broker precedents](../research/source-credential-broker-precedents.md) -- [A2A 1.0 specification](https://a2a-protocol.org/latest/specification/) +- [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) +- [A2A extension guide](https://a2a-protocol.org/latest/topics/extensions/) +- [Promptfoo custom providers](https://www.promptfoo.dev/docs/providers/custom-api/) +- [Promptfoo configuration reference](https://github.com/promptfoo/promptfoo/blob/main/site/docs/configuration/reference.md) - [OpenAI Codex SDK](https://developers.openai.com/codex/sdk/) -- [OpenAI Codex app-server](https://developers.openai.com/codex/app-server/) +- [OpenAI structured outputs](https://developers.openai.com/api/docs/guides/structured-outputs/) - [GitHub App installation tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) - [Git credential helpers](https://git-scm.com/docs/gitcredentials) +- [Docker credential stores](https://docs.docker.com/reference/cli/docker/login/#credential-stores) +- [Node.js SQLite API](https://nodejs.org/docs/latest-v22.x/api/sqlite.html) - [OCI Image Specification](https://github.com/opencontainers/image-spec) +- [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec) +- [Linux cgroup v2](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html) --- @@ -630,59 +828,89 @@ registry, or another profile configuration file for the initial use case. ### Key Technical Decisions -- KTD1. **Use the official A2A JavaScript SDK behind a small AllAgents request - decorator.** The decorator validates the required extension and canonical - request, resolves the deployment-wide retained claim, performs new-admission - checks, and preserves standard Task/Artifact carriers. -- KTD2. **Generate public, Task-store, workspace-manifest, and adapter contracts - from canonical Zod schemas.** Keep the result-schema subset shared across - Codex and Pi and forbid backend-specific public fields. +- KTD1. **Use the official A2A JavaScript SDK transport around an + AllAgents-owned request handler.** Do not use `DefaultRequestHandler` or its + non-transactional `TaskStore` seam. Implement the SDK's request-handler + interface so AllAgents controls UUIDv7 creation, atomic `createOrReplay`, + monotonic settlement, listing, retention, expiry, and HTTP+JSON error details + while retaining standard Task/Artifact carriers. +- KTD2. **Generate and publish the extension and storage contracts from canonical + Zod schemas.** The versioned extension specification at its declared URI + defines Agent Card params, activation, Message metadata/extensions, request, + Task/Artifact, idempotency, error, replay, examples, and versioning. Generate + public JSON Schemas, Task-store, workspace-manifest, and adapter types from the + same source. Keep backend-specific fields private. - KTD3. **Use a single-process supervisor, not a remote worker protocol.** One service owns Task state, staging, publication, backend child processes, evidence, termination, and cleanup. Child processes remain contained behind an invocation lifecycle boundary. - KTD4. **Make application authentication intentionally absent.** All Tasks and Artifacts share one deployment namespace. The listener accepts explicit - `0.0.0.0`; network controls are external. (session-settled: user-directed.) -- KTD5. **Compile configuration from existing workspace files.** Add - `workspaceSnapshots` to the project schema and `gateway.expose` to strict - profile-client schemas. Resolve the public launcher ID to one profile/client. + `0.0.0.0`; network controls are external. Bind and advertised interface URL + are distinct. (session-settled: user-directed.) +- KTD5. **Compile gateway configuration from existing workspace files.** Add + `workspaceSnapshots` to the project schema and `gateway.enabled` to strict + profile-client schemas. A gateway-only compiler normalizes the project + repository catalog and resolves each public launcher ID to one profile/client. Add no deployment YAML. (session-settled: user-directed.) - KTD6. **Keep source input name-based and closed.** Repository requests carry only declared-name revisions; snapshot requests carry only a declared snapshot name and immutable digests. Compute one canonical source identity for idempotency and provenance. -- KTD7. **Use direct acquisition implementations.** Git runs with hermetic config - and an invocation credential helper. OCI pulls through a library or fixed - non-shell client interface that validates registry redirects and digests and - extracts without trusting archive paths. -- KTD8. **Select GitHub credentials by three-way eligibility.** Discover - repository coverage with an App-authenticated GitHub API client or verify an - explicit installation ID. Only positive `ineligible` permits the configured - `gh` account; `unknown` and selected-provider failure are terminal. - (session-settled: user-directed.) +- KTD7. **Freeze direct Git and OCI acquisition profiles.** Git runs with + hermetic config and an invocation credential helper. An AllAgents-owned + minimal OCI Distribution client in the Rust helper uses pinned `reqwest` + (rustls, redirects and ambient proxies disabled), `tar`, `flate2`, and `zstd` + crates for streaming pull, bounded authentication, decoding, and changeset + application behind the helper's typed protocol. The client implements only the + v1 direct-image manifest/config/layer profile, RFC 8785 workspace-manifest + config, fixed extraction limits, and explicit Distribution-Spec authentication + and redirect policy. A test-only deterministic reference packer produces the + conformance fixture that freezes the format. +- KTD8. **Select GitHub credentials by provable three-way eligibility.** App + lookup 200 is eligible; 404 is ineligible only with independent repository- + existence proof; all ambiguous outcomes are unknown. Fresh App tokens bypass + SDK cache, are validated and revoked, and only positive ineligibility permits + the configured `gh` account. (session-settled: user-directed.) - KTD9. **Keep one behavior-focused `codex | pi` adapter registry.** Direct - targets and exposed profile targets resolve to the same adapter types and - conformance tests; profile context modifies server-owned configuration, never - the public command line. Provider control, each MCP child, and model-invoked - tools receive separate secret scopes and filesystem/environment views. -- KTD10. **Store one immutable terminal Task generation.** A private, - project-specific locked local store supports one process, durable atomic - idempotency claim plus Task creation, monotonic status, bounded - events/Artifacts, no eviction before TTL, atomic expiry, and startup - terminalization. State paths are ownership/mode/link/disjointness checked. - Integrity or durability failure stops admission and prevents false success. -- KTD11. **Use enforceable invocation containment.** The platform implementation - owns a non-escapable descendant set, drains stdout/stderr, and covers - credential helpers, Git/OCI, MCP, and provider processes. Failure to inspect - or prove an empty set fails or poisons readiness. An unmanaged gateway remains - alive to reap and expose recovery instructions; managed exit requires an - external manager that already accepted containment ownership. -- KTD12. **Keep durable evidence separate and collect it defensively.** Evidence - retains bounded source, Git, provider, result, artifact, and cleanup facts. - Post-execution workspace reads are descriptor-relative and no-follow; Git - metadata indirections and repository-controlled execution are rejected. - Structured logs remain metadata-only and never retain secrets or unrestricted + targets and gateway-enabled profile targets resolve to the same adapter types + and conformance tests; profile context modifies server-owned configuration, + never the public command line. A target is ready only when its pinned backend + exposes a non-bypassable spawn hook through which the helper launches every + MCP and model-tool process with separate filesystem, environment, descriptor, + secret, and network views. +- KTD10. **Put durable Task truth behind the Rust helper's SQLite VFS.** The + helper owns the single process-lifetime SQLite connection and exposes typed + transactional store operations; TypeScript never opens the database by path. + A small audited VFS roots every database, WAL, SHM, journal, and temporary-file + open beneath a preopened private state-directory descriptor with `openat2` + beneath/no-symlink checks, rejects hard links, and fsyncs files and containing + directories. SQLite uses WAL, foreign keys, and `synchronous=FULL`. Claims, + Tasks, events, bounded Artifact bytes, execution lease, containment identity, + internal outcome intent, and expiry live in transactional tables. + `createOrReplay`, lease acquisition, and terminal settlement are transactions; + acknowledge only committed state. Crash recovery yields a complete old or new + generation, never a mixed or missing acknowledged Task. Integrity, VFS, helper + protocol, or durability failure stops admission and prevents false success. +- KTD11. **Package one enforceable Linux security and state helper.** V1 supports + Linux x64/arm64 with cgroup v2, `clone3(CLONE_INTO_CGROUP)`, pidfds, `openat2` + beneath/no-symlink resolution, mount and network namespaces, and nftables + through a small audited Rust helper distributed in platform-specific optional + packages. Its typed inherited-pipe protocol owns SQLite operations, creates + empty containment behind a durable start gate, atomically launches and tracks + the complete acquisition/provider descendant set, mediates every MCP/tool + spawn, builds role-specific filesystem/environment/descriptor/network views, + terminates and waits for membership, and performs safe file operations. + Missing kernel features, delegated cgroup/network access, helper package, + backend spawn mediation, or protocol compatibility fails before binding; + there is no weaker fallback. A poisoned process remains alive to reap until + the set is empty. +- KTD12. **Capture live events, then collect durable filesystem evidence only + after quiescence.** Evidence retains bounded source, Git, provider, result, + Artifact, and cleanup facts. Post-execution workspace reads use the helper's + descriptor-relative no-follow handles, revalidate identity/size, and reject + Git metadata indirections or repository-controlled execution. Structured logs + remain metadata-only and never retain secrets or unrestricted request/output/file bodies. ### High-Level Technical Design @@ -690,19 +918,22 @@ registry, or another profile configuration file for the initial use case. ```mermaid flowchart TB C[Trusted-network A2A caller] --> G[Gateway server] - G --> S[Local Task store] + G --> H[Linux security and state helper] + H --> S[SQLite Task store] G --> W[Workspace compiler] W --> PW[Project workspace.yaml] W --> UW[User workspace.yaml] G --> A[Acquisition supervisor] A --> Git[Declared Git repositories] A --> OCI[Named OCI snapshot] + A --> H A --> P[Atomically published invocation workspace] G --> R[Closed adapter registry] R --> Codex[Codex SDK] R --> Pi[Pi RPC] - Codex --> E[Evidence and cleanup] - Pi --> E + Codex --> H + Pi --> H + H --> E[Quiescence then evidence and cleanup] E --> S ``` @@ -715,44 +946,91 @@ No `gateway.yaml` or `worker.yaml` is introduced. | Concern | CLI | Environment | Default | |---|---|---|---| | Listener | `--listen` | `ALLAGENTS_GATEWAY_LISTEN` | `127.0.0.1:4732` | +| Advertised interface URL | `--advertise-url` | `ALLAGENTS_GATEWAY_ADVERTISE_URL` | `http://127.0.0.1:4732` only with the default loopback listener; otherwise required | | Project workspace | `--workspace` | `ALLAGENTS_GATEWAY_WORKSPACE` | cwd | | State directory | `--state-dir` | `ALLAGENTS_GATEWAY_STATE_DIR` | `~/.allagents/gateway/` | | Terminal Task TTL | `--task-ttl` | `ALLAGENTS_GATEWAY_TASK_TTL` | `24h` | | Retained Task limit | `--max-retained-tasks` | `ALLAGENTS_GATEWAY_MAX_RETAINED_TASKS` | `1000` | | Per-Task retained bytes | `--max-task-bytes` | `ALLAGENTS_GATEWAY_MAX_TASK_BYTES` | `64MiB` | -| GitHub App ID | `--github-app-id` | `ALLAGENTS_GITHUB_APP_ID` | unset | -| App private key file | `--github-app-private-key-file` | `ALLAGENTS_GITHUB_APP_PRIVATE_KEY_FILE` | unset | -| App installation ID | `--github-app-installation-id` | `ALLAGENTS_GITHUB_APP_INSTALLATION_ID` | discovered/unset | -| GitHub CLI account | `--github-cli-account` | `ALLAGENTS_GITHUB_CLI_ACCOUNT` | unset | -| OCI auth file | `--oci-auth-file` | `ALLAGENTS_OCI_AUTH_FILE` | unset | -| OCI credential helper | `--oci-credential-helper` | `ALLAGENTS_OCI_CREDENTIAL_HELPER` | unset | -| Codex auth file | `--codex-auth-file` | `ALLAGENTS_CODEX_AUTH_FILE` | supported Codex default if safe | -| Pi auth file | `--pi-auth-file` | `ALLAGENTS_PI_AUTH_FILE` | supported Pi default if safe | - -Precedence is CLI over environment over default. Credential options name file -handles, accounts, or IDs, never secret values. Auth files must be regular, -current-user/root-owned, non-hard-linked, and no broader than `0600`. Setting -both OCI options is a startup error. The OCI helper value is one absolute -executable path with no arguments; it must be current-user/root-owned and not -group/world-writable. The gateway implements Docker credential-helper `get` -directly, without a shell: argv is exactly `[helperPath, "get"]`; stdin is the -canonical registry origin `https://[:nondefault-port]` plus one -newline; and an exit-zero stdout must be one UTF-8 JSON object with exactly -nonempty string fields `Username` and `Secret`, each at most 64 KiB. Stdout over -128 KiB, a timeout, nonzero exit, signal, malformed UTF-8/JSON, an unknown -member, or an empty credential fails acquisition with -`source_auth_oci_failed`; stderr is bounded, treated as secret-bearing, and not -placed in logs or evidence. The helper is invoked once per registry origin and -its credential is scoped to that origin and destroyed after acquisition. -Provider defaults are eligible only when their resolved auth files pass the -same checks; otherwise the target is not ready. The gateway projects only the -selected provider auth into its control-process view. +| GitHub App ID | `--github-app-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_ID` | unset | +| App private key file | `--github-app-private-key-file` | `ALLAGENTS_GATEWAY_GITHUB_APP_PRIVATE_KEY_FILE` | unset | +| App installation ID | `--github-app-installation-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_INSTALLATION_ID` | discovered/unset | +| GitHub CLI account | `--github-cli-account` | `ALLAGENTS_GATEWAY_GITHUB_CLI_ACCOUNT` | unset | +| OCI auth file | `--oci-auth-file` | `ALLAGENTS_GATEWAY_OCI_AUTH_FILE` | unset | +| OCI credential helper | `--oci-credential-helper` | `ALLAGENTS_GATEWAY_OCI_CREDENTIAL_HELPER` | unset | +| Codex auth file | `--codex-auth-file` | `ALLAGENTS_GATEWAY_CODEX_AUTH_FILE` | supported Codex default if safe | +| Pi auth file | `--pi-auth-file` | `ALLAGENTS_GATEWAY_PI_AUTH_FILE` | supported Pi default if safe | + +Precedence is CLI over environment over default. The advertised value is the +absolute URL placed in `AgentCard.supportedInterfaces`; wildcard hosts are +invalid, non-loopback listeners require an explicit value, and production uses +HTTPS. Credential options name file handles, accounts, or IDs, never secret +values. + +The Linux helper resolves every key/auth/helper path from a verified root with +`openat2(RESOLVE_BENEATH | RESOLVE_NO_SYMLINKS | RESOLVE_NO_MAGICLINKS)`, rejects +group/world-writable parent directories and linked or non-regular leaves, opens +with close-on-exec/no-follow, and verifies owner, mode, link count, device, and +inode with `fstat` after open. Consumers read the verified descriptor rather than +reopening the path. The helper executes a credential-helper binary from that +verified inode; a path or inode swap fails. Auth leaves are current-user/root- +owned, have one link, and are no broader than `0600`; helper leaves are +current-user/root-owned and not group/world-writable. + +Setting both OCI options is a startup error. `--oci-auth-file` accepts at most +1 MiB of strict UTF-8 Docker-config JSON containing only `auths`. Each key is the +exact registry lookup key below and each strict entry contains exactly one of: +bounded base64 `auth` decoding to `username:secret`, or bounded nonempty +`identitytoken`. `credsStore`, `credHelpers`, proxy/plugin fields, unknown +members, commands, and duplicate keys are rejected; nothing named by the file +is executed. Credential selection is exact-key only. + +The fixed OCI helper receives argv `[helperPath, "get"]` without a shell. Stdin +is the raw Docker lookup key plus newline: lowercase `host[:nondefault-port]` +except Docker Hub, which uses `https://index.docker.io/v1/`. Exit-zero stdout is +one UTF-8 JSON object with required nonempty `Username` and `Secret` strings and +optional `ServerURL`, each at most 64 KiB. `ServerURL`, when present, must equal +the lookup key; `Username: ""` classifies `Secret` as an identity token. +Stdout over 128 KiB, timeout, nonzero exit, signal, malformed UTF-8/JSON, unknown +member, mismatch, or empty credential fails with `source_auth_oci_failed`. +Stderr is bounded, treated as secret-bearing, and never logged or retained. + +Registry access starts anonymously. Accept at most one well-formed HTTPS Bearer +challenge and one authenticated retry per request, with one token refresh after +an in-budget 401. Scope must exactly equal +`repository::pull`; service is bounded, +passed only as data, and must match the registry service +(`registry.docker.io` for Docker Hub). +Credentialed token exchange is allowed only at a same-origin HTTPS realm +or the exact Docker Hub realm `https://auth.docker.io/token`; other realms are +anonymous-only. Do not request offline access or accept refresh tokens. Validate +token type and bounded expiry. + +Redirect handling is manual and limited to three HTTPS hops. Same-origin +redirects are permitted. A cross-origin redirect is permitted only for a +layer-blob `GET`/`HEAD` when the destination's normalized `host[:port]` exactly +matches that snapshot source's `layerRedirectHosts`; token, manifest, and config +requests reject it. Every hop rejects URL credentials, strips authorization, +cookies, and client credentials, resolves DNS afresh, validates every A/AAAA +address, and connects to a validated address with the original hostname used +for Host/SNI. Loopback, link-local, multicast, unspecified, RFC1918, ULA, CGNAT, +and other non-global destinations are rejected unless that exact host is +operator-approved for the source. Redirect loops, downgrade, mixed approved and +unapproved answers, and rebinding fail. Final descriptor bytes still must match +size and digest. + +Credentials are invoked once per registry lookup key, scoped to that origin and +repository pull, zeroed after use, and destroyed before publication. + +Provider defaults are eligible only when their resolved auth files pass the same +descriptor checks; otherwise the target is not ready. The gateway projects only +the selected provider auth into its control-process view. The derived workspace ID is a stable digest of the canonical project-workspace -path and is verified against store metadata. Retention includes Task records, -Artifacts, events, and invocation-key claims; expiry is atomic. When the -unexpired Task-count limit is reached, new admission fails rather than evicting -retained Tasks. +path and is verified against SQLite metadata. Retention includes Task records, +Artifact bytes, events, and invocation-key claims; expiry is transactional. When +the unexpired Task-count limit is reached, new admission fails rather than +evicting retained Tasks. **Project workspace additions** @@ -766,12 +1044,15 @@ repositories: workspaceSnapshots: evaluation: repository: ghcr.io/entityprocess/allagents-workspaces + layerRedirectHosts: + - pkg-containers.githubusercontent.com ``` Snapshot names use the portable profile-name vocabulary. Repositories must have unique stable names for remote acquisition. Snapshot repository values contain -only scheme/host/repository identity and never tags, digests, credentials, or -extraction paths. +only scheme/host/repository identity and an optional exact +`layerRedirectHosts` allowlist; never tags, digests, credentials, or extraction +paths. An absent allowlist rejects cross-origin layer redirects. **User workspace additions** @@ -782,31 +1063,44 @@ profiles: - name: codex launcher: codex-review gateway: - expose: true + enabled: true ``` -The nested object is strict and initially contains only `expose: true`. Absence -means not exposed. Exposure requires a launcher, an initial supported backend, -and a healthy installed profile with matching declaration digest. +The nested object is strict and initially contains only `enabled: true`. +Absence or `false` keeps the client unavailable through the gateway. Enablement +requires a launcher, an initial supported backend, and a healthy installed +profile with matching declaration digest. **Promptfoo custom-provider consumption** AI Evals implements Promptfoo's [`ApiProvider`](https://www.promptfoo.dev/docs/providers/custom-api/) in -TypeScript. Its `constructor(options: ProviderOptions)` stores -`options.id ?? "allagents-a2a"` and validates `options.config`; `id()` returns -that stored value. Its -`callApi(prompt, context, options)` uses `context.vars` for test data and -`options?.abortSignal` for request cancellation. +TypeScript. Its `constructor(options: ProviderOptions)` requires and stores a +nonempty `options.id`, validates `options.config`, and `id()` returns that +stored value. +`callApi(prompt, context?, options?)` reads +`context?.vars?.allagentsSource` when present and +`options?.abortSignal` for cancellation. Static YAML defines the source mode and every logical name: ```yaml +prompts: + - file://./prompts/coding-task.txt + +sharing: false +evaluateOptions: + maxConcurrency: 1 + cache: false +commandLineOptions: + write: false + share: false + providers: - id: file://./providers/allagents-a2a.ts label: codex-direct config: - endpoint: http://allagents-gateway.tailnet:4732 + endpoint: https://allagents-gateway.example.internal target: codex source: kind: repositories @@ -816,7 +1110,7 @@ providers: - id: file://./providers/allagents-a2a.ts label: codex-evaluation-snapshot config: - endpoint: http://allagents-gateway.tailnet:4732 + endpoint: https://allagents-gateway.example.internal target: codex source: kind: workspaceSnapshot @@ -840,6 +1134,13 @@ tests: workspaceManifestDigest: sha256:6789abcdef0123456789abcdef0123456789abcdef0123456789abcdef012345 ``` +The gateway enforces one active invocation transactionally. Promptfoo keeps +`maxConcurrency: 1` to avoid predictably creating failed capacity Tasks; other +trusted callers need no external queue for correctness. Disabling cache, local +result writes, and sharing is the safe baseline for confidential prompts and +opaque provider output. Consumers may enable persistence or sharing only after +defining their own access, retention, destination, and redaction policy. + `allagents` is a declared repository name used only as a revision-override key; repository mode still materializes the complete configured set. `evaluation` is the logical snapshot handle. The provider sends the source mode, optional named @@ -848,8 +1149,8 @@ revisions, and immutable digests, not `ghcr.io/entityprocess/allagents-workspaces`. The gateway resolves origins and credentials server-side and omits them from A2A source-identity responses. -`context.vars.allagentsSource` is the only per-test override. In repository mode -it may contain exactly `revisions`, whose keys must already exist in static +`context?.vars?.allagentsSource` is the only per-test override. In repository +mode it may contain exactly `revisions`, whose keys must already exist in static `config.source.revisions` and whose values are full lowercase 40-hex commits. In snapshot mode it may contain exactly `digest` and/or `workspaceManifestDigest`, both full lowercase `sha256:` digests. Present leaves @@ -858,32 +1159,44 @@ repository-name allowlist, and snapshot name remain static. Unknown members, mutable revisions, origins, destinations, credentials, and commands fail before A2A submission. -Each `callApi` creates one invocation key and A2A Task. The provider sends -`CancelTask` when its bounded deadline or `options?.abortSignal` fires after -acceptance. It returns terminal text or the validated structured result as -`ProviderResponse.output`. It maps gateway usage exactly as +Each `callApi` creates one high-entropy invocation key and sends `SendMessage` +with `returnImmediately: true`, then follows the accepted Task through +`SubscribeToTask`, `GetTask`, and bounded resubscription. An abort or deadline +sends one `CancelTask` with a fresh cleanup signal. Ambiguous submission retry +reuses the same key and request. The provider returns terminal text or validated +structured result as `ProviderResponse.output`. It maps gateway usage exactly as `inputTokens -> tokenUsage.prompt`, `outputTokens -> tokenUsage.completion`, `cachedInputTokens -> tokenUsage.cached`, and -`totalTokens -> tokenUsage.total`; provider-specific counters stay in +`totalTokens -> tokenUsage.total`; provider-specific counters remain in `metadata`. Task ID, Artifact references, logical source identity, termination, -and cleanup evidence also remain in `metadata`, without origins or destination -paths. Admission and terminal failures use `ProviderResponse.error`. This -provider is AI Evals code; AllAgents has no Promptfoo runtime dependency. +cleanup, and stable failure `code`/`retryable`/accepted `taskId` also remain in +`metadata`, without origins or destination paths. Admission and terminal +failures use a safe `ProviderResponse.error`. This provider is AI Evals code; +AllAgents has no Promptfoo runtime dependency. ### Error and Status Mapping -| Condition | Stable code and A2A outcome | Fresh-invocation retryable | +Every unsuccessful HTTP response has `Content-Type: application/json` and the +A2A 1.0 `google.rpc.Status` JSON shape under `error`. Standard A2A errors include +`google.rpc.ErrorInfo` with domain `a2a-protocol.org` and the specified uppercase +reason. Custom admission errors include `google.rpc.ErrorInfo` with domain +`allagents.dev`, uppercase stable-code reason, and string metadata `code`, +`retryable`, and optional `taskId`; field validation also includes +`google.rpc.BadRequest`. No HTTP+JSON response uses JSON-RPC `.data`. + +| Condition | Stable code and A2A/HTTP+JSON outcome | Fresh-invocation retryable | |---|---|---| -| Missing required extension | A2A `ExtensionSupportRequiredError`; no Task | No | -| Malformed request, source, digest, schema, prompt, or unknown target/source | `invalid_execution_request` in A2A `InvalidParamsError.data`; no Task | No | -| Invocation-key conflict | `invocation_key_conflict` in A2A `InvalidParamsError.data`; no new Task | No | -| Identical retained invocation replay | Existing Task and Artifacts | N/A | -| Cancel after terminal state | A2A `TaskNotCancelableError` | No | -| Retained Task capacity exhausted | `retention_capacity_exhausted` in A2A `InternalError.data`; no Task | Yes, after expiry | +| Unsupported A2A version | HTTP 400 A2A `VersionNotSupportedError`; no Task | No | +| Missing required extension | HTTP 400 A2A `ExtensionSupportRequiredError`; no Task | No | +| Malformed request, source, digest, schema, prompt, or unknown target/source | HTTP 400 `INVALID_ARGUMENT`; `invalid_execution_request`; no Task | No | +| Invocation-key conflict | HTTP 409 `ALREADY_EXISTS`; `invocation_key_conflict`; no new Task | No | +| Identical retained invocation replay | Existing Task with embedded Artifacts | N/A | +| Cancel after terminal state | HTTP 400 A2A `TaskNotCancelableError` | No | +| Retained Task capacity exhausted | HTTP 429 `RESOURCE_EXHAUSTED`; `retention_capacity_exhausted`; `Retry-After`; no Task | Yes, after expiry | | Runtime capacity unavailable after acceptance | `execution_capacity_unavailable`; failed Task | Yes | | App absent/ineligible and configured `gh` succeeds | Continue with recorded provider class | N/A | | App applicability unknown | `source_auth_applicability_unknown`; failed Task; no fallback | Yes for rate-limit/service causes only | -| Selected App config/auth/mint failure | `source_auth_failed`; failed Task; no fallback | No | +| Selected App config/auth/mint/validation/revocation failure | `source_auth_failed`; failed Task; no fallback | No | | Selected App permission/repository denial | `source_auth_denied`; failed Task; no fallback | No | | Selected App rate limit | `source_auth_rate_limited`; failed Task; no fallback | Yes | | Selected App service failure | `source_auth_unavailable`; failed Task; no fallback | Yes | @@ -891,108 +1204,144 @@ provider is AI Evals code; AllAgents has no Promptfoo runtime dependency. | Git revision/identity failure | `source_git_identity_invalid`; failed Task | No | | Git transport failure | `source_git_unavailable`; failed Task | Yes | | OCI helper timeout, process, protocol, or credential failure | `source_auth_oci_failed`; failed Task; no fallback | No | -| OCI auth/digest/manifest/extraction validation failure | `source_snapshot_invalid`; failed Task; no Git fallback | No | +| OCI auth/challenge/digest/manifest/extraction validation failure | `source_snapshot_invalid`; failed Task; no Git fallback | No | | OCI registry service failure | `source_snapshot_unavailable`; failed Task; no Git fallback | Yes | | Deadline expires | `execution_deadline_exceeded`; abort/terminate; failed Task | Yes | | Known provider permission denial | `execution_permission_denied`; rejected Task | No | | Unknown provider protocol or result shape | `provider_protocol_invalid`; failed Task | No | -| Cancellation with proven quiescence | `execution_cancelled`; cancelled Task | No | +| Cancellation with proven quiescence | `execution_canceled`; canceled Task | No | | Termination or cleanup cannot be proven | `execution_quiescence_unknown`; failed Task; readiness poisoned | No | | State store durability/integrity failure | `task_store_failed`; stop admission; abort/contain; no success | No | | Restart finds interrupted Task | `gateway_restarted`; failed Task; no provider resume | Yes as a new invocation | -| Retention expiry | A2A `TaskNotFoundError` | Yes as a new invocation | +| Retention expiry | HTTP 404 A2A `TaskNotFoundError` | Yes as a new invocation | Accepted-Task failures use the integrity Artifact's strict `failure` object with `code`, safe `message`, table-defined `retryable`, and one closed cause from `validation | capacity | sourceAuth | sourceGit | sourceSnapshot | deadline | permission | providerProtocol | cancellation | termination | stateStore | -restart`. Admission failures use the exact A2A error type in the table with the -same stable code and retryability in safe `data`. Retryability describes whether -a caller may create a fresh invocation; it never enables automatic Task retry -or fallback. Provider identifiers, credentials, paths, and raw upstream -messages never enter either carrier. +restart`. Retryability says whether a caller may create a fresh invocation; it +never enables automatic Task retry or provider/source fallback. Promptfoo copies +only the safe code, retryability, and accepted Task ID into metadata. Provider +identifiers, credentials, paths, and raw upstream messages enter neither +carrier. ### Phased Delivery 1. Build the current CLI and record the red E2E showing that `allagents gateway serve` is unavailable. Record the exact `/tmp/` workspace setup, command, and observed failure. -2. Freeze workspace additions, public extension, common manifests, result - schema, errors, and fixtures. -3. Build the deployment-wide Task store and unauthenticated A2A server against - a fake adapter. -4. Add repository and OCI acquisition with credential containment and manifest - validation. -5. Add the invocation supervisor, execution containment, safe evidence, and - shared backend contract. +2. Freeze workspace additions, the published extension, snapshot format, + common manifests, result schema, errors, packaging, and fixtures. Establish + the Rust helper protocol, safe SQLite VFS, platform packages, and ordered + release pipeline first. +3. Build the SQLite Task store through the helper, AllAgents A2A request handler, + HTTP+JSON server, minimal backend interface/registry, and fake adapter. +4. Extend the packaged helper with invocation supervision, execution + containment, spawn mediation, role-specific network/secret views, safe + evidence, and terminal arbitration around the fake adapter. +5. Add repository and OCI acquisition through the supervisor/helper with + credential containment and manifest validation. 6. Add Codex, then Pi, against the same conformance suite. 7. Run final implementation review and fix important correctness, security, contract, reliability, DRY, and coverage findings. -8. Run the green built-CLI `/tmp/` E2E, repository quality gates, user - documentation, and release evidence. +8. Run the green bundled-CLI `/tmp/` E2E, clean-registry install smoke, + repository quality gates, user documentation, and release evidence. ### System-Wide Impact -- **Package surface:** Add a private execution-service package and the public - `allagents gateway serve` command. Preserve existing profile and sync commands. -- **Schema surface:** Extend project workspace schemas with named snapshots and - user profile-client schemas with explicit exposure. Regenerate versioned JSON - Schemas and update configuration docs. -- **Dependency surface:** Add the official A2A SDK, pinned Codex SDK, - `@octokit/auth-app`, and a focused OCI client/extraction implementation to the - private execution package. -- **State surface:** Add a bounded gateway state root and per-invocation staging, - publication, evidence, and cleanup roots. Do not alter existing profile state. -- **Security surface:** The network is the authorization boundary. Source - credentials are phase-scoped; acquired code and agent tools never receive App, - `gh`, or OCI credentials. -- **Compatibility:** Existing workspace files remain valid because new fields are - optional. Older binaries reject the new strict nested profile field, so docs - must state the minimum supporting version. +- **Package surface:** Declare a root Bun workspace; add private + `packages/execution-service` and Rust `packages/execution-helper`; add the + service as a root `workspace:*` development dependency; distribute Linux + x64/arm64 helper binaries through versioned platform-specific optional + packages; and bundle the service into published `dist/index.js`. The release + scripts and Publish workflow version matching helper packages and root + dependency ranges, publish and verify both platform packages first, and + publish `allagents` only after their registry metadata and checksums resolve. + Add public `allagents gateway serve` without changing existing profile and + sync commands. Root build, typecheck, tests, and clean-registry install smoke + include the private workspace service and resolved helper binary. +- **Schema surface:** Extend project workspace schemas with named snapshots, + exact layer-redirect hosts, and user profile-client schemas with explicit + gateway enablement. Publish the versioned extension specification and + generated JSON Schemas; update configuration docs. +- **Dependency surface:** Put the official A2A SDK, pinned Codex SDK, and + `@octokit/auth-app` in the private service package. Pin SQLite, the custom VFS + bindings, `reqwest` with rustls, `tar`, `flate2`, and `zstd` in the Rust helper + lockfile together with the Rust toolchain/helper protocol; check helper release + checksums. +- **State surface:** Add one bounded SQLite gateway state root and per-invocation + staging, publication, evidence, and cleanup roots. Do not alter profile state. +- **Security surface:** The network is the caller authorization boundary. Source + credentials are phase-scoped; helper-mediated process and network views keep + acquired code and agent tools from App, `gh`, OCI, provider, MCP, operator, and + gateway credentials/state. Helper absence or capability loss fails closed. +- **Compatibility:** Existing workspace files remain valid because new fields + are optional. Gateway startup applies stricter repository-catalog rules. + Older binaries reject the new strict nested profile field, so docs state the + minimum supporting version. ### Risks and Mitigations - **Accidental network exposure:** Binding `0.0.0.0` is intentional and allowed; - startup output and docs state that every reachable host has full authority. + require a distinct advertised URL, use HTTPS in production, and state in + startup output/docs that every reachable host has full authority. - **Profile identity drift:** Derive targets only from current validated user declarations and matching installed state; never resurrect declaration-missing launchers from retained profile state. -- **Credential leakage:** Use fresh App tokens or one configured `gh` account, - invocation-only helpers, hermetic Git, and credential teardown before - publication. Separate provider-control, per-MCP, and model-tool views prevent - one secret scope from reading another. +- **Credential leakage or path swap:** Use fresh validated/revoked App tokens or + one configured `gh` account, descriptor-bound credential handles, hermetic + Git, strict Docker auth/helper protocols, and credential teardown before + publication. Non-bypassable helper spawn mediation replaces environments, + closes descriptors, and enters role-specific mount/network namespaces before + every MCP or model-tool exec; a backend lacking that hook is unavailable. - **Identity-changing fallback:** Classify App applicability as eligible, - ineligible, or unknown; only positive ineligibility permits `gh`. -- **OCI archive abuse:** Require immutable digests, configured repositories, - bounded extraction, path/type/link validation, and manifest verification. -- **Untrusted acquired code:** General hostile-code sandboxing is not claimed, - but model-invoked tools cannot reach provider/MCP/operator credentials or - gateway state. Project/user setup shell commands are never automatic. -- **Evidence-time attacks:** Treat the mutated workspace as untrusted, use - descriptor-relative no-follow reads, reject Git metadata indirection, and - disable repository-controlled Git execution features. -- **Provider/API churn:** Pin compatible SDK/CLI versions and retain versioned - native fixtures plus one adapter conformance suite. -- **Orphaned processes:** Require a platform containment primitive whose - descendants cannot escape; fail readiness when unavailable. On uncertain - quiescence, unmanaged mode stays alive to reap and managed mode exits only - after cleanup ownership transfer. -- **Store corruption or disclosure:** Validate ownership, modes, links, root - disjointness, lock, and workspace identity. Integrity/durability failure stops - admission and prevents terminal success. + ineligible, or unknown; require repository-existence proof for 404 + ineligibility; only positive ineligibility permits `gh`. +- **OCI registry/archive abuse:** Require immutable digests, a closed + manifest/config/layer profile, same-origin metadata, exact operator-approved + layer-redirect hosts with per-hop address validation, changeset semantics, + fixed extraction limits, safe paths/types/links, and exact project-catalog + manifest verification. +- **Untrusted acquired code:** General hostile-code sandboxing beyond the + declared Linux process/network namespace and secret boundary is not claimed. + Invocation routes deny gateway, host loopback, and management networks; + provider/MCP egress is allowlisted; and model tools cannot reach provider/MCP/ + operator credentials or gateway state. Project/user setup shell commands are + never automatic. +- **Evidence-time attacks:** Prove containment empty first, then use + descriptor-relative no-follow reads with identity/size revalidation, reject + Git metadata indirection, and disable repository-controlled Git execution. +- **Provider/API churn:** Pin compatible SDK/CLI/model versions and retain + versioned native fixtures plus one adapter conformance suite. Gate Codex native + schemas to the pinned Structured Outputs subset and backend availability to a + proven non-bypassable spawn hook. +- **Orphaned processes:** Persist a stable empty containment identity before + start-gate release; enumerate the full project-owned cgroup namespace on + startup. On uncertain quiescence, stay alive, reject admission, and continue + reaping until empty without mutating the settled Task. +- **Store corruption or disclosure:** Route SQLite and all sidecars through the + helper's descriptor-rooted no-follow VFS with full synchronization and + transactions; validate ownership, modes, links, root disjointness, lock, and + workspace identity. Integrity/durability failure stops admission and prevents + terminal success. ### Assumptions -- The initial deployment is one gateway process and one active invocation. -- Every network peer able to connect is trusted with all exposed targets and - retained Tasks. -- The selected project workspace is operator-controlled and uses supported - repository/snapshot declarations. +- The initial deployment is one gateway process and one transactionally enforced + active invocation. +- Every external network peer able to connect is trusted with all available + targets, including built-ins and gateway-enabled profiles, and all retained + Tasks. Invocation descendants are deliberately unable to reach that network + boundary. +- The selected project workspace is operator-controlled and compiles to 1-64 + uniquely named GitHub repositories with collision-free destinations. - GitHub.com is the only authenticated Git host in the initial delivery. -- OCI snapshots use registries reachable through HTTPS and immutable manifests. -- Codex and Pi automation surfaces remain compatible with the pinned versions. -- Supported platforms provide a non-escapable invocation containment strategy; - the gateway fails readiness where that invariant cannot be met. +- OCI snapshots use HTTPS registries and the frozen v1 direct-image format. +- Codex and Pi are available only when their pinned automation surfaces support + non-bypassable helper-mediated tool and MCP spawning. +- Gateway v1 execution supports Linux x64/arm64 hosts with cgroup v2, `clone3`, + pidfds, `openat2`, mount/network namespaces, nftables, and delegated + permissions. --- @@ -1000,100 +1349,145 @@ messages never enter either carrier. ### U1. Workspace, extension, and manifest contracts -- **Goal:** Freeze configuration additions and all versioned public/private data - contracts before runtime implementation. -- **Requirements:** R1, R2, R3, R6, R7, R8, R9, R18; AE3, AE4, AE5, AE7, - AE8, AE9, AE13, AE16, AE20; KTD1, KTD2, KTD5, KTD6. -- **Files:** `src/models/workspace-config.ts`, schema generation tests and - generated public schemas, `packages/execution-service/src/contracts/*`, - `packages/execution-service/tests/unit/contracts/*`, configuration docs. -- **Approach:** Add strict named `workspaceSnapshots` and nested profile-client - `gateway.expose`; preserve project/user scope and reserve built-in IDs. Define - activation on every profiled operation, the extension-owned request envelope, - other-metadata behavior, exact result-schema grammar, source union, deadline, - integrity and produced Artifacts, stable failures, workspace manifest, and - canonical digest preimages from Zod. -- **Execution note:** Start with fixtures that reject missing activation, cross- - variant/unknown extension fields, extra Message Parts, undeclared names, - mutable snapshot references, malformed digests, invalid deadlines, exposure - without launcher, built-in collisions, and unsupported clients while - preserving unrelated metadata. -- **Verification:** Focused workspace-schema and contract tests; generated schema - drift check; representative YAML and wire examples parse through runtime - schemas; canonicalization and Artifact-cardinality fixtures pass. +- **Goal:** Freeze configuration, packaging, safe state primitives, and every + versioned public/private contract before runtime implementation. +- **Requirements:** R1, R2, R3, R5, R6, R7, R8, R9, R11, R18; AE3, AE4, AE5, + AE7, AE8, AE9, AE13, AE14, AE16, AE20; KTD1, KTD2, KTD5, KTD6, KTD7, + KTD10, KTD11. +- **Files:** root `package.json`/build/typecheck configuration, + `packages/execution-service/package.json` and TypeScript config, Rust + `packages/execution-helper`, Linux x64/arm64 optional packages, typed helper + protocol, SQLite schema/migrations and descriptor-rooted VFS, + `scripts/release.ts`, `scripts/publish.ts`, `.github/workflows/publish.yml`, + `src/models/workspace-config.ts`, schema generation tests and generated public + schemas, execution-service contracts, + `docs/src/pages/a2a/extensions/coding-execution/v1.astro` at the exact + declared URI plus a generated schema asset beneath that route, a versioned + snapshot-format specification, deterministic reference packer/conformance + fixtures, and configuration docs. +- **Approach:** Declare the Bun workspace and root `workspace:*` development + edge so the private service is installed, checked, and bundled. Establish the + helper protocol and audited SQLite VFS before the server store client. Version + helper packages with matching root optional-dependency ranges; publish and + verify both platform packages before the root package. Verify a clean registry + install resolves the matching helper binary and checksum and that the packed + root manifest contains no workspace protocol. + + Add strict named `workspaceSnapshots` with exact layer-redirect hosts and + nested profile-client `gateway.enabled`; preserve ordinary project/user + parsing while compiling gateway repository and target catalogs. Publish Agent + Card params, version/header activation, Message metadata/extensions, unified + Parts, exact result-schema grammar, source union, deadline, idempotency/replay, + HTTP+JSON errors, integrity/produced Artifacts, workspace manifest, OCI media/ + change-set/limit profile, and canonical digest preimages from Zod. +- **Execution note:** Start with independent wire fixtures that use only the + published extension specification. Reject missing version/activation, cross- + variant/unknown fields, extra Message Parts, undeclared names, mutable + snapshot references, malformed digests, invalid deadlines, incomplete or + mismatched manifests, gateway enablement without launcher, built-in + collisions, and unsupported clients while preserving unrelated metadata. + Fault-inject database/WAL/SHM link and rename swaps through the real VFS. +- **Verification:** Focused workspace-schema, packaging, helper VFS, and + contract tests; generated schema/spec drift checks; representative YAML, HTTP + errors, and wire examples parse through runtime schemas; snapshot conformance, + canonicalization, Artifact-cardinality, clean-registry install, matching + helper version/checksum, and ordered publish dry-run fixtures pass. ### U2. Deployment-wide Task store and A2A server - **Goal:** Serve the A2A lifecycle without application authentication and keep - durable deployment-wide Task/idempotency truth. -- **Requirements:** R1, R2, R3, R4, R5, R8, R16, R17, R18; AE1, AE2, AE9, - AE11, AE14, AE15, AE16, AE19, AE20; KTD1, KTD2, KTD3, KTD4, KTD10. -- **Files:** execution-service Task repository, Agent Card, request handler, - server, pagination/retention, CLI gateway command, and focused tests. -- **Approach:** Implement flags/env precedence, private link-safe project state - and lock, loopback default, explicit `0.0.0.0`, startup integrity/ - reconciliation, per-operation extension negotiation, durable atomic - claim+Task creation, monotonic terminal settlement, bounded events/Artifacts, - no early eviction, atomic expiry, global listing/cancellation, deadline - handling, and fail-closed graceful shutdown. -- **Execution note:** Prove with the official A2A client that one caller can read - and cancel another caller's Task; this is expected behavior. Fault-inject - unsafe state paths plus open/write/rename/fsync boundaries before - acknowledgment, cancellation intent, Artifact, and terminal settlement. -- **Verification:** A2A discovery/send/stream/get/list/subscribe/cancel/replay/ - expiry integration tests on loopback and `0.0.0.0`; state-path, retained- - capacity, store-fault, competing-lock, deadline, shutdown, and restart tests. - -### U3. Git and OCI workspace acquisition + durable deployment-wide Task/idempotency truth behind a fake backend. +- **Requirements:** R1, R2, R3, R4, R5, R8, R13, R16, R17, R18; AE1, AE2, AE9, + AE11, AE14, AE15, AE16, AE19, AE20; KTD1, KTD2, KTD3, KTD4, KTD9, KTD10. +- **Files:** typed Task-store client, Agent Card, AllAgents request handler, + HTTP+JSON/SSE server, pagination/retention, minimal backend interface and + registry, fake adapter, health/readiness, CLI gateway command, focused tests. +- **Approach:** Implement flags/env precedence, bind/advertised-URL separation, + private project state and lock, helper-owned SQLite full-sync transactions, + startup integrity and full containment-namespace reconciliation, A2A version + and extension negotiation, exact `SendMessage` modes and `ListTasks` + semantics, standard/custom `google.rpc.Status` errors, durable + `createOrReplay`, one execution lease, internal outcome intent plus atomic + terminal settlement, bounded events/Artifact bytes, no early eviction, + transactional expiry, global listing/cancellation, deadline handling, and + fail-closed graceful shutdown against the fake adapter. +- **Execution note:** Prove with the official A2A client that one external caller + can read and cancel another caller's Task; this is expected behavior. Kill + subprocesses after transaction write/sync/commit/response boundaries and + fault-inject helper/VFS I/O, capacity races, cancellation intent, Artifact, + and terminal settlement. +- **Verification:** A2A discovery/send modes/stream/get/full list/subscribe/ + cancel/replay/expiry and HTTP-error integration tests on loopback plus explicit + `0.0.0.0`/advertised URL; health/readiness, state-path, retained and active + capacity, crash/store-fault, competing-lock, deadline, shutdown, and restart + tests. + +### U3. Invocation supervisor and backend contract + +- **Goal:** Run one fake-backed invocation through containment, typed + preparation, evidence, terminal arbitration, and cleanup with truthful + outcomes before real acquisition/adapters. +- **Requirements:** R3, R5, R8, R13, R14, R15, R16; AE9, AE10, AE11, AE12, + AE14, AE15, AE16, AE17, AE18; KTD3, KTD9, KTD10, KTD11, KTD12. +- **Files:** security/state helper extensions, provider/MCP/tool view and egress + compiler, invocation state machine, containment/start-gate controller, spawn + broker, typed preparation, evidence collector, result validator, + cleanup/reaper, and lifecycle tests. +- **Approach:** Extend the U1 helper to allocate an empty cgroup with a stable ID + and start gate, commit Task+lease+containment before release, and enumerate + recorded and unknown cgroups on startup. Launch every child into the cgroup; + mediate every backend MCP/tool spawn; enter role-specific mount and network + namespaces; replace environments; close descriptors; apply nftables egress + policy; use pidfds for termination/wait; and expose safe file operations. + Resolve targets through U2's typed fake adapter; never execute generated + launchers or setup commands. Commit one internal intent across provider, + cancel, deadline, and shutdown; capture live events; prove quiescence before + filesystem evidence; atomically settle status, evidence, Artifacts, cleanup, + and lease release; remain alive to reap when poisoned without mutating the + settled Task. +- **Execution note:** Fault-inject every boundary: capacity races and restart; + process death before/after empty-set creation, Task binding, child clone, and + start-gate release; pairwise and three-way outcome races; child fork/escape; + helper protocol/version/package mismatch; provider/MCP/tool attempts to reach + Agent Card, ListTasks, GetTask, SendMessage, CancelTask, host loopback, and + management networks; environment/path/inherited-FD/`/proc`/magic-link secret + reads by real child and grandchild processes; output truncation, malicious + evidence, valid-result-then-evidence-failure, and unknown cleanup. +- **Verification:** Deterministic lifecycle, single execution lease, helper + packaging/checksum, cgroup/pidfd/mount/network namespace containment, + non-bypassable spawn mediation, separate secret/descriptor/egress views, + typed preparation, safe-file/evidence, unknown-cgroup reconciliation, and + poison/reaping tests plus real child-process smoke on Linux x64/arm64 CI. + +### U4. Git and OCI workspace acquisition - **Goal:** Materialize declared repository sets and named OCI snapshots into the - same validated invocation workspace contract. + same validated invocation workspace through the U3 security helper. - **Requirements:** R6, R9, R10, R11, R12, R15, R16, R18; AE5, AE6, AE7, - AE8, AE10, AE15, AE17, AE18; KTD6, KTD7, KTD8, KTD11. + AE8, AE10, AE15, AE17, AE18; KTD6, KTD7, KTD8, KTD11, KTD12. - **Files:** acquisition coordinator, Git transport, GitHub provider selection, - OCI client/extractor, workspace-manifest validator, staging/publication helper, - fixtures and tests. -- **Approach:** Resolve name-based source requests from project workspace. - Implement hermetic Git and full-commit verification. Classify App - applicability as eligible/ineligible/unknown, mint a fresh token with adequate - lifetime, permit `gh` only for positive ineligibility, and use temporary - credential helpers. Pull digest-pinned OCI manifests from declared - repositories, validate every layer and extraction boundary, validate the - expected workspace-manifest digest, and publish atomically. Tear down every - acquisition credential before typed preparation. -- **Execution note:** Use local Git remotes and a local OCI test registry/fixture; - prove unknown/selected-App failures do not call `gh`, token lifetime is - enforced, and snapshot failures never invoke Git fallback. -- **Verification:** Focused three-way provider-selection tests, Git integration - tests including branch/tag resolution, OCI digest/path/limit tests, credential - leak scans, and equivalent manifest output across both acquisition modes. - -### U4. Invocation supervisor and backend contract - -- **Goal:** Run one invocation through acquisition, adapter execution, evidence, - cancellation, descendant termination, and cleanup with truthful terminal - outcomes. -- **Requirements:** R3, R5, R8, R13, R14, R15, R16; AE9, AE10, AE11, AE12, - AE14, AE15, AE16, AE17, AE18; KTD3, KTD9, KTD10, KTD11, KTD12. -- **Files:** backend interface/registry, typed preparation, provider/MCP/tool - secret-view compiler, invocation state machine, platform containment, evidence - collector, result validator, cleanup/reaper, fake adapter, and lifecycle tests. -- **Approach:** Resolve targets to typed adapter context; never use generated - launchers or workspace setup commands. Project validated configuration through - deterministic transforms. Give provider control, each MCP child, and model - tools separate minimal views; enforce deadlines; track the non-escapable - containment set; preserve result states; collect evidence through safe reads; - and require quiescence before cleanup success. Startup reconciles Tasks and - containment before roots. Unmanaged poisoned mode continues reaping; managed - exit proves cleanup ownership transfer. -- **Execution note:** Build the fake adapter first and fault-inject every - boundary: cancellation/terminal races, deadline, shutdown, child escape, - cross-scope provider/MCP/tool secret reads, unsafe state reads, output - truncation, malicious evidence, valid-result-then-evidence-failure, unknown - cleanup, and manager handoff. -- **Verification:** Deterministic lifecycle, containment, separate-secret-view, - preparation, and evidence tests plus one real child-process smoke fixture per - supported platform strategy. + strict Docker-auth/helper resolver, OCI Distribution client and changeset + applier, workspace-manifest validator, staging/publication helper, fixtures and + tests. +- **Approach:** Resolve name-based requests from the compiled project catalog. + Implement hermetic Git and full-commit verification. Apply the exact App + eligibility proof table, bypass token cache, validate/revoke each fresh token, + permit `gh` only for positive ineligibility, and use descriptor-bound temporary + helpers. Implement anonymous-first bounded Bearer authentication, redirect/ + credential-origin rules, the frozen direct-image media profile, streaming + descriptor verification, gzip/zstd changeset and whiteout semantics, all + extraction ceilings, exact project-manifest validation, and atomic publication. + Tear down every acquisition credential before typed preparation. +- **Execution note:** Use local Git remotes and a local OCI registry plus the U1 + producer fixture. Prove ambiguous/selected-App failures never call `gh`, two + sequential acquisitions mint distinct tokens, token validation/revocation and + lifetime are enforced, helper/auth-file swaps fail, and snapshot failure never + invokes Git fallback. +- **Verification:** Three-way provider-selection and real-response fixture tests; + Git branch/tag/full-commit integration; GHCR/Docker Hub helper fixtures; + malicious realm/scope/downgrade/redirect tests; OCI index/media/digest/size/ + limit/order/whiteout/path/catalog fixtures; credential leak scans; equivalent + complete manifest output across both acquisition modes. ### U5. Codex backend adapter @@ -1104,15 +1498,22 @@ messages never enter either carrier. AE15, AE17, AE18; KTD9, KTD11, KTD12. - **Files:** Codex adapter, profile-context and auth bridge, fixtures, conformance and optional credentialed smoke tests. -- **Approach:** Pin the SDK; create one fresh thread per Task; pass cwd, typed - profile configuration, abort signal, optional output schema, and the private - Codex control-process auth view inside containment. Keep Codex-invoked tools - outside that auth view; normalize events/usage; bound evidence; dispose fully. -- **Execution note:** Characterize the pinned SDK and its tool-sandbox/auth - separation with captured fixtures before implementing normalization. Do not - import Promptfoo provider code. -- **Verification:** Shared adapter conformance, deadline, auth-isolation, and - tool-secret-denial fixtures plus an opt-in credentialed smoke case. +- **Approach:** Pin SDK/model compatibility and first prove a non-bypassable + synchronous hook that delegates every MCP and model-tool spawn to the U3 + helper. If the pinned Codex surface can bypass that hook, Codex is unavailable + in v1 rather than relying on an asserted view. Create one fresh thread per + Task; pass cwd, typed profile configuration, abort signal, and the private + Codex control-process auth view inside containment. Pass native `outputSchema` + only for the pinned Structured Outputs subset; otherwise add JSON guidance and + use the common terminal validator. Normalize events/usage, bound evidence, and + dispose fully. +- **Execution note:** Characterize the pinned SDK/model's spawn, schema, tool- + sandbox, auth, abort, and event behavior with captured fixtures before + normalization. Do not import Promptfoo provider code. +- **Verification:** Shared adapter conformance, real SDK child/grandchild spawn + mediation, filesystem/environment/inherited-FD/`/proc` credential denial, + gateway/host-network denial, native-schema and validated-fallback paths, + deadline, and an opt-in credentialed smoke case. ### U6. Pi backend adapter @@ -1122,43 +1523,51 @@ messages never enter either carrier. AE17, AE18; KTD9, KTD11, KTD12. - **Files:** Pi adapter, RPC parser, restricted policy extension, profile-context and auth bridge, fixtures, conformance and optional credentialed smoke tests. -- **Approach:** Launch Pi with typed invocation configuration, its private - control-process auth view, strict JSONL RPC, explicit allowed tools/extensions, - per-MCP secret views, deterministic permissions, event validation, deadline/ - cancellation escalation, and settled completion. Pi-invoked tools receive no - provider or MCP credentials. Repository extensions and unrestricted built-ins - remain disabled. -- **Execution note:** Reuse the adapter contract exactly; record Pi-specific facts - as bounded native evidence rather than public schema branches. -- **Verification:** Shared adapter conformance, malformed/unknown RPC, deadline, - auth/MCP/tool-secret denial, and an opt-in credentialed smoke case. +- **Approach:** First prove strict RPC exposes a non-bypassable synchronous hook + that delegates every MCP and model-tool spawn to the U3 helper. If Pi can + bypass that hook, Pi is unavailable in v1. Launch Pi with typed invocation + configuration, its private control-process auth view, strict JSONL RPC, + explicit allowed tools/extensions, per-MCP secret declarations, + deterministic permissions, event validation, deadline/cancellation + escalation, and settled completion. Repository extensions and unrestricted + built-ins remain disabled. +- **Execution note:** Characterize and pin Pi's spawn/RPC contract; record + Pi-specific facts as bounded native evidence rather than public schema + branches. +- **Verification:** Shared adapter conformance, real RPC child/grandchild spawn + mediation, filesystem/environment/inherited-FD/`/proc` provider/MCP secret + denial, gateway/host-network denial, malformed/unknown RPC, deadline, and an + opt-in credentialed smoke case. ### U7. End-to-end delivery and documentation -- **Goal:** Prove the built CLI and document the trusted-network operating model, - workspace configuration, credentials, sources, Promptfoo consumption, and - risks. +- **Goal:** Prove the bundled/packed CLI and document the trusted-network + operating model, Linux requirements, workspace configuration, credentials, + sources, Promptfoo consumption, and risks. - **Requirements:** R1-R19; F1-F6; AE1-AE21. -- **Files:** gateway guide/reference, configuration reference, README, CHANGELOG, - real example project/user workspaces, AI Evals-style Promptfoo YAML and custom- - provider contract fixture, E2E fixtures, release evidence. -- **Approach:** After the final implementation review is resolved, build the CLI; - create project and user workspaces under `/tmp/`; expose Codex/Pi fixture - targets; serve on loopback and `0.0.0.0`; acquire from local Git and OCI - fixtures; run the official A2A client through negotiation, success, replay, - cancellation, deadline, shutdown, restart, and expiry. Run a minimal custom- - provider fixture through one configured-repository-set invocation and one - named-snapshot invocation, proving Promptfoo configuration carries only - source mode, named revision overrides, snapshot handle, and digests while the - gateway resolves origins. Document that network reachability grants full - authority and App/OCI secrets are process inputs, not YAML. -- **Execution note:** The green smoke test must exercise the same built command - and `/tmp/` workspace shape as the recorded red E2E, not a test-only server. - The consumer fixture models AI Evals but remains test/documentation code; the - AllAgents runtime does not import Promptfoo. -- **Verification:** `bun run build`, focused and full tests, typecheck, lint, docs - build, schema drift check, custom-provider contract fixture, and exact - red/green E2E commands/results recorded in the PR description. +- **Files:** published extension and snapshot-format pages, gateway guide/ + reference, configuration reference, README, CHANGELOG, real project/user + workspaces, AI Evals-style Promptfoo YAML and custom-provider contract fixture, + E2E fixtures, packed-install smoke, release evidence. +- **Approach:** After final implementation review, build and pack the CLI plus + both helper packages; install in a clean Linux environment; create project and + user workspaces under `/tmp/`; gateway-enable fixture targets; serve on + loopback and `0.0.0.0` with a valid advertised URL; exercise health/readiness; + acquire local Git and OCI fixtures; and run an independently generated + official A2A client through version/extension negotiation, errors, both send + modes, complete listing, success, replay, cancellation, deadline, shutdown, + restart, and expiry. Run the custom-provider fixture through repository and + snapshot invocations with secure Promptfoo defaults, proving requests contain + only logical source data while the gateway resolves origins. Document full + network-peer authority, sensitive opaque payloads, and process-only secrets. +- **Execution note:** Green smoke uses the same built command and `/tmp/` + workspace shape as red E2E, never a test-only server. The consumer fixture is + AI Evals-style test/documentation code; AllAgents runtime does not import + Promptfoo. +- **Verification:** `bun run build`, packed-install/helper checksum smoke, + focused and full tests, typecheck, lint, docs build, extension/schema drift, + custom-provider contract fixture, and exact red/green commands/results in the + PR description. --- @@ -1166,20 +1575,20 @@ messages never enter either carrier. | Gate | Applies to | Required evidence | |---|---|---| -| Workspace schema | U1 | Project/user parsing, strict nested fields, built-in collision rules, generated-schema drift | -| Public contract | U1-U2 | Official A2A client, every-operation activation, metadata preservation, exact request/result/Artifact/error/canonicalization fixtures | -| Trusted-network model | U2, U7 | Loopback and `0.0.0.0`; shared Task visibility/cancellation; docs warning | -| Durable Task lifecycle | U2, U4 | Private safe state paths, lock, durable claim+Task, no early eviction, store faults, races, restart, atomic expiry | -| Repository acquisition | U3 | Declared-name revision resolution, hermetic Git, full commits, three-way App/`gh` eligibility and sub-budget | -| OCI acquisition | U3 | Declared repository, manifest/layer/workspace digests, safe extraction, no fallback | -| Credential and state isolation | U3-U7 | Separate provider/MCP/tool views; teardown; no cross-scope secrets, operator home, or state root | -| Supervisor lifecycle | U4 | Deadline, shutdown, cancellation races, non-escapable containment, unmanaged recovery, manager handoff, stale-root proof | -| Safe evidence | U4-U6 | Descriptor-relative no-follow reads; links/special files/Git indirection rejected; hermetic Git | -| Backend conformance | U4-U6 | Same suite for fake, Codex, and Pi; profile and built-in variants | -| Structured result | U1, U4-U6 | Exact subset and envelope, valid/invalid/not-produced states, Artifact cardinality, no false publication | -| Repository quality | All | Build, focused/full tests, typecheck, lint, schema check, docs build | -| Built CLI E2E | U7 | Recorded red then green built command under `/tmp/`, both sources, auth isolation, replay/cancel/deadline/shutdown/restart | -| Promptfoo consumption | U7 | AI Evals-style YAML for both source modes; request source metadata and gateway-generated response provenance omit origins; terminal Task maps to `ProviderResponse` | +| Workspace/package schema | U1 | Root workspace install/build edge; ordered helper-package publication and clean-registry resolution; project/user parsing; compiled repository/target catalogs; generated schema/spec drift | +| Public contract | U1-U2 | Independent official HTTP+JSON client; card interface/params/streaming capability; A2A version and every-operation extension headers; unified Parts; both send modes; complete listing; metadata; `google.rpc.Status`; request/result/Artifact/canonicalization fixtures | +| Trusted-network model | U2-U3, U7 | Loopback and `0.0.0.0` with distinct advertised URL; HTTPS docs; shared external Task visibility/cancellation; invocation-to-gateway and host-network denial; metadata-only health/readiness | +| Durable Task lifecycle | U1-U3 | Descriptor-rooted SQLite VFS/full-sync transactions; private state/lock; create-or-replay; one execution lease; internal outcome intent and atomic terminal settlement; no early eviction; crash/store faults; restart; transactional expiry | +| Repository acquisition | U4 | Compiled-name resolution, hermetic Git, commits, 200/404/ambiguous App eligibility, cache bypass, token validation/revocation, `gh` fallback and sub-budget | +| OCI acquisition | U4 | Strict Docker auth/helper; exact layer-redirect allowlist and per-hop address checks; Bearer origin policy; direct-image/config/layer media; descriptor verification; changesets/whiteouts; fixed limits; exact project catalog; no fallback | +| Linux helper and isolation | U1, U3-U7 | x64/arm64 packages/checksums; kernel/cgroup readiness; gated durable containment; full namespace enumeration; pidfd termination; openat2 path/VFS handles; non-bypassable spawn mediation; separate mount/environment/descriptor/network views | +| Supervisor lifecycle | U3 | Capacity races; pre/post-gate crash points; provider/cancel/deadline/shutdown intent races; live-event capture; atomic evidence settlement; poison/readiness/reaping; unknown-set proof | +| Safe evidence | U3-U6 | Descriptor-relative reads with identity/size recheck; links/special/sparse/replaced files and Git indirection rejected; no verified FS evidence before quiescence | +| Backend conformance | U2-U3, U5-U6 | Same lifecycle suite for fake, Codex, and Pi; real child/grandchild spawn mediation; credential and gateway-network denial; profile and built-in variants | +| Structured result | U1, U3, U5-U6 | Public grammar, Codex native-subset gate and fallback, valid/invalid/not-produced states, Artifact cardinality, no false publication | +| Repository quality | All | Build, clean-registry install, focused/full tests, typecheck, lint, schema/spec checks, docs build | +| Bundled CLI E2E | U7 | Recorded red then green command under `/tmp/`, both sources, advertised URL/probes, auth and network isolation, capacity, replay/cancel/deadline/shutdown/restart | +| Promptfoo consumption | U7 | Secure-default AI Evals YAML for both modes; optional context; nonblocking acceptance/subscription/cancel; source/provenance omit origins; output/usage/error metadata mapping | ## Definition of Done @@ -1188,50 +1597,72 @@ messages never enter either carrier. - Every R1-R19 requirement is implemented or explicitly demonstrated by a passing acceptance scenario. - The gateway starts with no `gateway.yaml` or `worker.yaml`, defaults to - loopback, and accepts explicit `0.0.0.0`. -- Network reachability is the only caller trust boundary; Task visibility and - idempotency are deployment-wide and documented accurately. -- Project workspace declarations own repositories and named OCI snapshot - repositories; user workspace declarations own profile launcher exposure; - built-in target IDs cannot be shadowed. -- The A2A card, every-operation activation header, metadata preservation, strict - request and result-schema grammar, exact error mapping, integrity/produced - Artifacts, canonicalization, retention capacity, and cancellation semantics - pass official-client contract fixtures. -- AI Evals-style Promptfoo YAML selects repository mode with optional named - revision overrides, or snapshot mode with one logical handle and immutable - digests. The custom-provider fixture maps one `callApi` to one Task, propagates - cancellation, normalizes usage, and returns output, Artifacts, and logical - provenance without sending origins or adding a Promptfoo runtime dependency. -- Repository and OCI modes produce one validated workspace-manifest contract, - never fall back across source modes, and retain truthful provenance. -- GitHub App eligibility/unknown state, acquisition sub-budget, no-installation - `gh` fallback, selected-App failure, OCI auth containment, and pre-provider - source-credential teardown are proven. -- Typed preparation never runs workspace setup shell commands. Built-in and - profile targets authenticate through private provider-control views; every MCP - child is secret-scoped; model tools cannot reach provider/MCP/operator - credentials or gateway state. -- Deadline, cancellation/terminal races, shutdown, result states, safe private - state paths, retention capacity, store failure, descendant quiescence, - managed/unmanaged recovery, safe evidence, and cleanup pass fault tests. + loopback HTTP, accepts explicit `0.0.0.0`, requires a separate advertised URL + off default loopback, documents production HTTPS, and exposes truthful + metadata-only health/readiness. +- Network reachability is the only external caller trust boundary; Task + visibility and idempotency are deployment-wide. Invocation descendants cannot + reach that boundary, host loopback, or management networks. +- Project workspace declarations compile to the exact repository/snapshot + catalog; user declarations own profile launcher gateway enablement; built-in + target IDs cannot be shadowed. +- The published extension, Agent Card interface/params, A2A version and + activation headers, unified Parts, both send modes, full ListTasks behavior, + metadata preservation, strict schemas, HTTP+JSON errors, embedded Artifacts, + canonicalization, retention, and cancellation pass independent official-client + fixtures. +- Secure-default AI Evals Promptfoo YAML selects repository mode with optional + named revision overrides or snapshot mode with one handle and immutable + digests. The provider maps one optional-context `callApi` to one nonblocking + Task, retains its high-entropy key across ambiguous retry, propagates + cancellation with a fresh cleanup signal, normalizes usage, and returns safe + error metadata and logical provenance without origins or runtime dependency. +- Git and OCI modes produce one complete workspace-manifest contract. OCI v1 + uses the direct-image/config/layer profile, exact project catalog, descriptor + verification, same-origin metadata, operator-approved layer redirect hosts + with per-hop address validation, changeset semantics, and fixed extraction + ceilings. Source modes never fall back and provenance never overclaims + verification. +- App eligibility and ambiguous 404 handling, acquisition sub-budget, positive- + ineligibility `gh` fallback, fresh token cache bypass/validation/revocation, + strict Docker auth/helper and registry challenge policy, and pre-provider + credential teardown are proven. +- Typed preparation never runs workspace setup commands. The packaged Linux + helper durably binds containment before releasing any child, enumerates + unknown cgroups, and mediates every MCP/tool exec into role-specific mount, + environment, descriptor, credential, and network views. Real Codex/Pi + child/grandchild tests prove model tools cannot reach provider/MCP/operator + credentials, gateway state, or the gateway/host-management network; a backend + without non-bypassable spawn mediation is unavailable. +- The descriptor-rooted SQLite VFS, execution lease, pre/post-start-gate crash + boundaries, internal outcome-intent races, atomic terminal evidence + settlement, result states, state-path safety, descendant quiescence, readiness + poisoning/reaping, immutable terminal Tasks, and cleanup pass fault tests. - Evaluation behavior, public-Internet authentication, remote workers, custom - materializers, and multi-tenant policy remain absent. + materializers, non-Linux gateway execution, and multi-tenant policy remain + absent. ### Per unit -- U1: Runtime and generated schemas agree; invalid negotiation, request, - source/exposure/collision/configuration fixtures fail at expected paths. -- U2: Official A2A operations, global replay/visibility, project locks, store - faults, listeners, deadline, shutdown, restart, and retention pass. -- U3: Git and OCI fixtures pass; three-way provider eligibility, token lifetime, - and all no-fallback rules are observed; leak scans are clean. -- U4: Fake-adapter lifecycle proves terminal monotonicity, typed preparation, - isolation, containment, bounded/safe evidence, deadline/cancellation/shutdown, - poisoning, and cleanup. -- U5: Codex passes shared conformance and optional credentialed smoke evidence is - recorded when credentials exist. +- U1: Root workspace packaging, ordered helper-platform publication, clean- + registry resolution, safe SQLite VFS, runtime/generated schemas, published + extension and snapshot format, producer fixture, and invalid negotiation/ + source/enablement/collision/configuration fixtures agree. +- U2: Official HTTP+JSON operations, version/extension/error/list semantics, + global replay/visibility, helper-owned SQLite locks/crashes, execution lease, + listeners/advertised URL, probes, deadline, shutdown, restart, and retention + pass against the fake backend. +- U3: The fake lifecycle proves gated durable containment, full namespace + reconciliation, atomic terminal settlement, typed preparation, spawn-mediated + secret/descriptor/network views, safe evidence ordering, poisoning, reaping, + and cleanup on Linux x64/arm64. +- U4: Git and OCI fixtures pass; App eligibility/cache bypass/token + validation/revocation, Docker credential/challenge and layer-redirect rules, + changesets, limits, exact catalog, and no-fallback rules are observed; leak + scans are clean. +- U5: Codex passes shared conformance and both schema paths; optional + credentialed smoke evidence is recorded when credentials exist. - U6: Pi passes the same conformance and malformed RPC cannot produce success. -- U7: Final review is resolved; built CLI red/green E2E under `/tmp/`, Promptfoo - custom-provider contract fixture, complete repository gates, schemas, docs, - and reproducible PR instructions are complete. +- U7: Final review is resolved; bundled and packed CLI red/green E2E under + `/tmp/`, Promptfoo fixture, complete repository gates, published schemas/specs, + docs, and reproducible PR instructions are complete. From 768f35353a1edbd9b0cd3a993d3b9851c4ace2ec Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sun, 20 Sep 2026 13:50:25 +1000 Subject: [PATCH 11/44] docs(architecture): define reusable execution workspaces --- ...-agent-execution-through-an-a2a-gateway.md | 603 ++-- ...0837-feat-coding-execution-gateway-plan.md | 2429 +++++++++++------ 2 files changed, 1959 insertions(+), 1073 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index 22aacddc..1d78419a 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -2,7 +2,7 @@ - Status: Accepted; implementation pending - Date: 2026-09-17 -- Updated: 2026-09-19 +- Updated: 2026-09-20 ## Context @@ -22,10 +22,12 @@ loopback, a firewalled network, or a Tailscale network. Network reachability is the trust and authorization boundary. A coding-agent execution still includes more than a model request. The gateway -must acquire an immutable workspace, select a configured agent target, contain -credentials to their required phases, propagate cancellation, collect evidence, -terminate descendants, and clean up. Those responsibilities need one public -contract even when the initial deployment remains a single trusted process. +must acquire or reuse an immutable workspace base, select a configured agent +target, propagate cancellation, collect bounded evidence, and clean up. +The trusted CI job, VM, or container that runs the gateway is the execution and +secret boundary: provider code and model-invoked tools run with that runner's +authority. The gateway owns one public contract for the lifecycle without +claiming hostile-code containment inside that boundary. The contract must not turn AllAgents into an evaluation harness. Dataset expansion, repetitions, assertions, scoring, experiment scheduling, and durable @@ -35,10 +37,12 @@ evaluation Runs remain consumer concerns. ### Add a trusted-network execution gateway -AllAgents will provide an independently testable `allagents gateway serve` -entry point. It is separate from the interactive CLI command lifecycle but may -run as a single local service process that supervises acquisition and provider -child processes. +AllAgents will provide an independently testable, separately installed +`allagents-gateway serve` entry point. It runs as one Bun service process that +supervises phase-scoped acquisition and host provider processes. The ordinary +`allagents` CLI does not contain or depend on the gateway. A convenience +dispatcher may locate and execute a separately installed compatible +`allagents-gateway`, but it must not download or embed gateway artifacts. The gateway implements A2A protocol version `1.0` over the `HTTP+JSON` binding plus a required versioned AllAgents coding-execution extension. It owns: @@ -54,6 +58,56 @@ plus a required versioned AllAgents coding-execution extension. It owns: The gateway is not an evaluator, grader, experiment scheduler, retry authority, or durable evaluation Run ledger. +### Use one TypeScript/Bun workspace with narrow package boundaries + +The gateway and CLI are TypeScript products in one Bun workspace monorepo. The +private root package owns orchestration only. `apps/cli` publishes `allagents`; +`apps/gateway` publishes `allagents-gateway`; and `apps/acquirer` is built only +as a digest-pinned GHCR image, never as an npm package. Shared code is limited to +three justified packages: + +- `packages/workspace-config` owns the project and user workspace projections + consumed by the CLI and gateway; +- `packages/execution-contracts` owns the A2A coding-execution wire contract and + portable validation; and +- `packages/acquisition-contracts` owns the typed request and manifest exchanged + with the acquisition image. + +Generated, language-portable contract fixtures live under `contracts/`. The +repository does not introduce speculative `core`, `common`, native platform, or +provider-sharing packages. A package is added only for an already-demonstrated +ownership boundary. + +This architecture follows from the deployment boundary. V1 runs on one trusted +Linux CI runner, the gateway and official provider automation surfaces are +available in TypeScript, and Docker is needed only for untrusted repository and +OCI materialization. Adding another gateway implementation runtime and custom +in-job security layer would increase release and operational surface without +creating a boundary inside the already-trusted CI job. + +### Keep independent CLI and gateway release trains + +The `allagents` CLI and `allagents-gateway` have independent versions, tags, and +release triggers. A CLI release publishes only `apps/cli`; installing it fetches +neither the gateway package nor the acquisition image. + +A gateway release first builds the multi-architecture `apps/acquirer` image, +pushes it to GHCR, records the immutable image-index digest and each supported +architecture's manifest digest, and verifies acquisition against those exact +digests. It then packs the exact `apps/gateway` npm tarball and runs package and +registry conformance with that tarball and those image digests. Only after both +artifacts pass does the workflow publish `allagents-gateway`. It does not +publish `allagents`. + +Compatibility is a versioned contract, not equal npm versions. +`allagents-gateway compatibility --format json` reports the product, gateway +version, build identity, acquisition image digest, and supported A2A, +coding-extension, workspace, execution-contract, acquisition-contract, and +snapshot versions. The +optional CLI dispatcher may launch a separately installed gateway only when the +required contract-version intersections are non-empty. Compatibility does not +depend on target-specific npm wrappers or embedded native binaries. + ### Trust the network boundary instead of adding application authentication The initial gateway has no application-level authentication or per-caller @@ -72,12 +126,12 @@ scoped to a caller identity. Operators must use Tailscale ACLs, host firewalls, container networking, or equivalent network controls when the listener is not loopback-only. -Invocation descendants are not network peers. Provider, MCP, and model-tool -processes run in role-specific network namespaces that cannot route to host -loopback, any gateway bind or advertised address, ingress proxies, or operator -management networks. Provider and MCP egress is default-deny except for -destinations compiled from adapter and MCP configuration; model tools receive no -network unless an explicit adapter policy grants the same constrained egress. +Provider processes, MCP servers, and model-invoked tools are not separate +network principals. They execute on the same trusted CI runner as the gateway +and may exercise the authority available to that job. Operators must provision +the runner accordingly and must not rely on AllAgents to isolate host secrets, +the gateway listener, management networks, or arbitrary repository code from +model-invoked tools. Gateway-managed TLS termination, OIDC, static bearer tokens, per-tenant ownership, and multi-tenant information-hiding are deferred. Production clients @@ -131,23 +185,23 @@ Process-level options use exact flags and environment variables for: - listener, advertised-interface URL, and workspace selection; - a project-specific state-directory override; -- terminal Task retention and bounded Artifact/event storage; +- disjoint immutable-base cache and per-Task runtime/workspace roots; +- workspace materialization policy plus Task and cache retention limits; - GitHub App identifiers and private-key file references; -- the configured GitHub CLI account; -- a strict Docker-auth file or fixed Docker credential-helper executable; and -- Codex and Pi auth-file handles. +- the configured GitHub CLI account; and +- a strict Docker-auth file or fixed Docker credential-helper executable used + only for acquisition. By default the state root is a deterministic child of `~/.allagents/gateway/` keyed by the canonical project-workspace identity. The -packaged Rust helper owns a private SQLite store in WAL/full-synchronization mode -through a descriptor-rooted VFS. Every database, WAL, SHM, journal, and temporary -file open uses `openat2` beneath/no-symlink resolution and rejects hard links. -The store persists claims, Tasks, one execution lease, internal outcome intent, -events, bounded Artifact bytes, containment identity, and expiry transactions. -It verifies workspace identity and holds an exclusive process-lifetime lock. -The root is current-user owned, private, link-resistant, and disjoint from -project, profile, and invocation roots. The listener exposes metadata-only -`/healthz` and `/readyz`; readiness is false whenever admission is unsafe. +gateway owns an ordinary private Bun SQLite database with transactions, WAL +mode, and full synchronization. It persists claims, Tasks, one execution lease, +internal outcome intent, events, bounded Artifact bytes, and expiry state. The +gateway verifies workspace identity and holds an exclusive process-lifetime +lock. The root is current-user owned, private, and disjoint from project, +profile, and invocation roots. Standard Bun SQLite APIs are the entire storage +layer. The listener exposes metadata-only `/healthz` and `/readyz`; readiness is +false whenever admission is unsafe. ### Support direct repositories and OCI workspace snapshots @@ -169,8 +223,8 @@ tags may be accepted for developer convenience, but the gateway resolves and records the full commit object ID before provider execution. Reproducibility- sensitive callers should supply full commit IDs. -For OCI snapshots, the project workspace declares the registry repository and -any exact cross-origin layer-blob redirect hosts: +For OCI snapshots, the project workspace declares an operator-selected OCI +Distribution repository and any exact cross-origin layer-blob redirect hosts: ```yaml workspaceSnapshots: @@ -180,6 +234,26 @@ workspaceSnapshots: - pkg-containers.githubusercontent.com ``` +The repository field is registry-neutral. V1 must pull AllAgents-formatted +workspace snapshots from Docker Hub, GHCR, JFrog Artifactory/JFrog Container +Registry, and compatible private OCI Distribution registries. Registry choice +does not change the snapshot media types, digest requirements, extraction +rules, or caller-visible source contract. + +Registry conformance is tiered. Every pull request runs local Distribution +fixtures and a live public, digest-pinned GHCR pull through the exact gateway +package under test and the exact acquisition image index and architecture +manifest digests built for that pull request. A release workflow additionally +tests least-privilege authenticated GHCR and a digest-pinned disposable JFrog +Container Registry over HTTPS with a private CA and pull-only identity. Those +release checks install the exact gateway npm tarball and use the exact +multi-architecture acquisition image index and per-architecture manifests +intended for publication, for every architecture the registry and runner +support, without rebuilding either artifact. A report for another commit, +package, image digest, architecture manifest, build identity, or compatibility +output is rejected. Docker Hub behavior remains covered by protocol fixtures to +avoid public rate-limit dependence in pull-request CI. + The request supplies the name `evaluation`, a `sha256:` OCI image-manifest digest, and a `sha256:` workspace-manifest digest. The gateway constructs the full OCI reference server-side. Callers cannot supply a registry host, @@ -216,12 +290,74 @@ destination paths. A commit listed inside an OCI snapshot is not described as independently verified unless the gateway separately verifies it against its Git remote. -Acquisition occurs in a gateway-owned staging directory. The gateway validates -paths, collisions, file types, symlinks, layer and file counts, individual and -total compressed and expanded sizes, digests, and the workspace manifest before -atomically publishing the invocation workspace. Absolute paths, traversal, -device files, sockets, escaping links, foreign or external OCI layers, and -unapproved cross-origin access are rejected. +For a source without a reusable validated base, the host gateway creates a +staging directory and bind-mounts only that directory into the digest-pinned +acquisition image. The acquisition container receives only the selected +repository or registry credential plus the strict network, redirect, size, +file-count, and archive policy needed for that source. The host resolves GitHub +App eligibility and mints any installation token; the container never receives +the App private key, host home directory, provider authentication state, or +Docker socket. The image contains and downloads no Codex, Pi, or other coding +harness. + +The container materializes the repository or OCI source into staging, emits the +typed acquisition manifest, and exits. The gateway removes it before provider +execution, validates the manifest plus paths, collisions, file types, symlinks, +layer and file counts, individual and total compressed and expanded sizes, and +digests, then atomically promotes staging to a validated base. Absolute paths, +traversal, device files, sockets, escaping links, foreign or external OCI +layers, and unapproved cross-origin access are rejected. Every non-publication +path removes staging. Docker has no role after acquisition completes. + +The base-cache key binds the acquisition-contract version, compiled catalog and +layout digest, and immutable source identity: every effective repository commit, +or the OCI manifest and workspace-manifest digests. A repository request is +reusable only when every effective revision is a full commit ID. Mutable +branch or tag requests instead receive a non-reusable Task-owned base that is +removed during settlement or reconciliation. Cache hits mint no credential and +start no acquisition container. Active Tasks pin reusable bases; bounded cache +eviction removes only unpinned entries. + +The request optionally selects `workspaceAccess: "readOnly" | "readWrite"` and +defaults to `readWrite`. A read-only Task resolves its provider cwd directly +inside its validated base; exact immutable requests may share a reusable cached +base, while mutable branch or tag requests own a non-reusable base. Every Task +receives a private runtime directory for temporary, home, provider-state, and +evidence files. The gateway disables optional Git locks and asks the adapter for +its native read-only policy when available. It does not inspect the prompt or +add a per-Task mount, chmod pass, or full-tree verification. Read-only is a +cooperative contract and best-effort provider control, not a hostile-code +boundary; the consumer remains responsible for giving the Task work that does +not require project writes. A violating provider can contaminate a cached base +and later Tasks; the operator must evict that entry before reuse. + +A read-write Task receives a unique writable view under +`//workspace`. The host materializer prefers a +filesystem block clone, falls back to rootless OverlayFS on supported Linux +hosts, and supports an explicit ordinary-copy backend for portability. It never +uses hard links for writable files. The selected materializer is operator +configuration, not request input. After evidence collection, normal settlement +unmounts when needed and removes the Task-owned view plus any non-reusable base; +a non-settling provider retains them with the poisoned execution lease until +reconciliation. + +For either access mode, the provider cwd is resolved from an optional logical +`workingDirectory` selector: + +- `{ kind: "workspaceRoot" }` selects the effective workspace root and is the + default; or +- `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }` + selects a declared repository and an optional validated directory beneath it. + +The caller never supplies an absolute path, configured destination, materializer, +cache key, or physical workspace name. The gateway maps the repository name +through the compiled catalog, resolves the optional relative path, and requires +the result to be an existing directory whose resolved path remains beneath the +selected repository root. The logical selector and access mode are part of the +canonical request, idempotency identity, and integrity evidence. +Gateway-generated structured metadata and operational logs never contain the +physical path; opaque terminal output, native evidence, and produced Artifact +payloads are not sanitized and may contain it. ### Consume the gateway from Promptfoo through an AI Evals provider @@ -240,14 +376,17 @@ AI Evals owns a Promptfoo It implements `ApiProvider`: its constructor receives `ProviderOptions`, requires and retains a nonempty `options.id`, validates `options.config`, and exposes `id()`. -`callApi(prompt, context?, options?)` reads bounded test variables from -`context?.vars` when present and cancellation from `options?.abortSignal`. The -provider translates one `callApi` into one A2A Task: it creates and retains a -high-entropy invocation key, sends one Message whose sole Part has `text` set, -declares the extension in `Message.extensions`, puts the target and closed source -union in the matching metadata member, and calls `SendMessage` with -`returnImmediately: true`. It captures the Task ID and follows terminal state -through `SubscribeToTask`, with `GetTask` and bounded resubscription for races or +`callApi(prompt, context?, options?)` reads bounded source, working-directory, +and workspace-access test variables from `context?.vars` when present and +cancellation from `options?.abortSignal`. The provider translates one `callApi` +into one A2A Task: it creates and retains a high-entropy invocation key, resolves +the effective logical working-directory selector and `readOnly | readWrite` +access mode, sends one Message whose sole Part has `text` set, declares the +extension in `Message.extensions`, puts the target, closed source union, logical +working directory, and access mode in the matching metadata member, and calls +`SendMessage` with `returnImmediately: true`. +It captures the Task ID and follows terminal state through `SubscribeToTask`, +with `GetTask` and bounded resubscription for races or disconnects. It returns output, normalized token usage, stable failure metadata, and logical provenance in Promptfoo's `ProviderResponse`. @@ -272,6 +411,10 @@ providers: config: endpoint: https://allagents-gateway.example.internal target: codex + workingDirectory: + kind: repository + repository: allagents + workspaceAccess: readOnly source: kind: repositories revisions: @@ -282,11 +425,24 @@ providers: config: endpoint: https://allagents-gateway.example.internal target: codex + workingDirectory: + kind: repository + repository: allagents + workspaceAccess: readWrite source: kind: workspaceSnapshot snapshot: evaluation digest: sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef workspaceManifestDigest: sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789 + +tests: + - description: gateway package trial + providers: [codex-direct] + vars: + allagentsWorkingDirectory: + kind: repository + repository: allagents + path: apps/gateway ``` The first provider materializes the complete configured repository set and uses @@ -300,20 +456,26 @@ defaults for confidential prompts and outputs; consumers may enable persistence or sharing only after applying their own retention, access, destination, and redaction policy. -Static provider config fixes the source kind and logical names. The only -per-test object is `context?.vars?.allagentsSource`: repository mode accepts -revision overrides only for statically listed names and only as full lowercase -40-hex commits; snapshot mode accepts only replacement OCI and workspace- -manifest `sha256:` digests. Missing context or leaves retain static values. A -URL, destination, mutable revision, credential, command, unknown member, or -changed source kind/name fails before submission. After Task acceptance, the -provider's bounded deadline or `options?.abortSignal` sends one `CancelTask` -using a fresh cleanup signal rather than the already aborted request signal. +Static provider config fixes the source kind and logical names and may define a +default logical `workingDirectory` and `workspaceAccess`; absent values default +to `{ kind: "workspaceRoot" }` and `readWrite`. Per-test +`context?.vars?.allagentsWorkingDirectory` may replace the selector, while +`context?.vars?.allagentsWorkspaceAccess` may replace the access mode with the +exact string `readOnly` or `readWrite`. Separate read-only trials may share one +immutable physical base and cwd. Read-write trials receive distinct Task-owned +writable views even when their logical selectors are equal. +`context?.vars?.allagentsSource` remains limited to revision or digest leaves. +Missing variables retain static values. Absolute paths, `.` or `..` segments, +configured destinations, unknown repositories, URLs, mutable revisions, +credentials, commands, materializer choices, and unknown members fail before +provider execution. After Task acceptance, the provider's bounded deadline or +`options?.abortSignal` sends one `CancelTask` using a fresh cleanup signal +rather than the already aborted request signal. It maps gateway input, output, cached-input, and total token counts to Promptfoo's `prompt`, `completion`, `cached`, and `total` fields respectively. -Safe stable failure -code, retryability, accepted Task ID, other usage, and logical Task/Artifact -evidence stay in metadata without origins. Opaque prompts, terminal output, +Safe stable failure code, retryability, accepted Task ID, other usage, logical +working directory, and Task/Artifact evidence stay in metadata without origins, +configured destinations, or physical paths. Opaque prompts, terminal output, structured results, native evidence, and produced-Artifact payloads remain unredacted sensitive data. The provider belongs in AI Evals. AllAgents exposes the A2A contract and consumer documentation without taking a runtime dependency @@ -363,131 +525,154 @@ rate-limit, or service failure terminates acquisition. The gateway never retries the same Task through the broader GitHub CLI identity. Git receives credentials only through an invocation-scoped helper under -hermetic Git configuration. The gateway excludes system, global, and repository -credential helpers, Git Credential Manager, askpass, SSH agents, repository- -controlled secondary fetches, and executable Git configuration. Tokens never -appear in clone URLs, command arguments, Git configuration, logs, Tasks, -Artifacts, retained workspaces, profile setup, MCP processes, agent processes, -or model-invoked tools. The helper and token are destroyed before provider -execution. +hermetic Git configuration inside the acquisition container. The gateway +excludes system, global, and repository credential helpers, Git Credential +Manager, askpass, SSH agents, repository-controlled secondary fetches, and +executable Git configuration. Tokens never appear in clone URLs, command +arguments, Git configuration, logs, Tasks, Artifacts, or retained workspaces. +The helper and token are destroyed and the acquisition container is removed +before provider execution. OCI credentials come from either a strict Docker-auth subset that cannot name executables or a fixed Docker credential helper using its standard `get` -protocol. They are scoped to snapshot acquisition and removed before -publication. Public registries require no credential configuration. - -### Integrate providers through typed adapters +protocol. Acquisition supports anonymous pulls plus same-origin Basic and +Distribution Bearer challenge flows required by the declared registry, with the +documented Docker Hub token-service exception. Any other cross-origin Bearer +realm is rejected before a request is sent. System roots may be supplemented by +an operator map from exact registry `host[:port]` keys to verified PEM bundles; +each bundle is trusted only for connections to its key. Credentials and custom +trust material are scoped to snapshot acquisition and removed before +publication. Public registries require no credential or custom-CA +configuration. + +### Integrate host providers through narrow typed adapters The initial backend registry contains Codex and Pi, delivered in that order. -Each adapter implements one behavior-focused contract for availability, +Each adapter implements a narrow AllAgents-owned contract for availability, capabilities, invocation, progress, deterministic permission handling, abort, terminal output, optional structured result, usage, native evidence, and -disposal. - -The Codex adapter depends directly on pinned `@openai/codex-sdk`, creates one -fresh thread per Task, passes cancellation, and consumes structured events. It -uses native `outputSchema` only for schemas supported by the pinned Structured -Outputs contract; other valid public schemas use explicit JSON guidance and the -same gateway-side validator used by every backend. - -The Pi adapter uses strict RPC mode with invocation-owned configuration and a -restricted policy extension. Repository extensions and unrestricted built-ins -are not loaded merely because they exist in acquired source. - -CLI-backed compatibility adapters may be added later when a client has a stable -machine protocol. Missing controls are reported honestly as capability gaps. -The gateway never scrapes a TUI or exposes arbitrary installed executables. -OMP is Pi-derived and is added only for demonstrated OMP-specific value beyond -direct Pi. +disposal. The gateway does not adopt AI SDK Harnesses or make a third-party +cross-provider abstraction part of its execution contract. + +The Codex adapter uses a pinned `@openai/codex-sdk` release first. App-server is +permitted only when a required, demonstrated capability is absent from that +SDK; convenience or speculative parity is not enough. Native `outputSchema` is +used only for schemas supported by the pinned Structured Outputs contract; +other valid public schemas use explicit JSON guidance and the same gateway-side +validator used by every backend. The adapter does not scrape a TUI or use an +unstable bridge merely to preserve the target name. + +The Pi adapter uses a pinned, supported RPC or package surface with +invocation-owned configuration and a restricted policy extension. Repository +extensions and unrestricted built-ins are not loaded merely because they exist +in acquired source. OMP remains out of the initial registry and is added only +for demonstrated OMP-specific value beyond direct Pi. + +Provider runtimes are installed and pinned as part of the CI runner or gateway +installation; the gateway never downloads them per request. An operator may +select a globally installed binary override only when an exact version and +capability compatibility probe succeeds. Missing controls are reported honestly +as capability gaps, and the gateway never exposes arbitrary installed +executables. + +Codex and Pi execute bare metal, directly on the same trusted Linux CI runner as +the gateway. For a read-only Task, cwd resolves inside its validated base, +shared only when reusable; for a read-write Task, cwd resolves inside the Task's +unique writable view. When +explicit API credentials are absent, Codex reuses the runner's existing +`CODEX_HOME` and ChatGPT login, and Pi reuses its existing supported host +authentication. The gateway references those host paths in place; it does not +copy, mount, or import OAuth files. + +Each provider process receives an explicitly constructed environment containing +only the invocation configuration, selected provider settings, and required +host identity, executable, home, and authentication paths. This reduces +accidental ambient-variable leakage but is not an isolation or secret- +containment claim: MCP servers, provider descendants, and model-invoked tools +may exercise the same CI-job authority and reach secrets available to that +runner. The CI job, VM, or container must therefore be provisioned as the +security boundary. Provider preparation is adapter-owned and typed. The gateway never executes project or user `setup` shell entries as part of acquisition or invocation. Validated profile settings, plugins, MCP declarations, and deterministic workspace projections are applied through existing typed transforms. -Provider control processes, MCP children, and model-invoked tools receive -distinct allowlisted filesystem, environment, descriptor, secret, and network -views. The provider control process sees only its invocation-private auth -channel; each MCP child sees only its own resolved secrets; shell and other -model-invoked tools see neither provider nor MCP credentials. Every view excludes -gateway state, operator home, App keys, GitHub/OCI stores, acquisition helpers, -unrelated adapter auth, the parent environment, gateway endpoints, host -loopback, and management networks. - -The pinned backend must expose a non-bypassable synchronous hook that delegates -every MCP and model-tool spawn to the Rust helper. The helper enters the role's -mount and network namespaces, replaces the environment, closes every -non-allowlisted descriptor, and only then executes untrusted code. Codex or Pi -is unavailable when its pinned surface can bypass that hook. This enforced -credential/state/network boundary is required even though general hostile-code -sandboxing remains deferred. - ### Persist Task truth, not live provider execution The gateway durably stores Task identity, the canonical request, idempotency claim, selected target and source, effective configuration digest, one execution -lease, internal outcome intent, Artifact bytes, retained evidence, and -containment identity under the configured state directory. The official A2A -HTTP+JSON transport wraps an AllAgents-owned request handler; typed transactions -execute in the Rust helper's descriptor-rooted SQLite VFS. One transaction -arbitrates `createOrReplay`, UUIDv7 Task creation, execution-lease acquisition, -and containment binding; another atomically settles terminal status, result or +lease, internal outcome intent, Artifact bytes, retained evidence, cleanup +outcome, and expiry state under the configured state directory. AllAgents-owned +TypeScript handlers expose the A2A contract and use ordinary private Bun SQLite +ownership and transactions with full synchronization. One transaction +arbitrates `createOrReplay`, UUIDv7 Task creation, and execution-lease +acquisition. Normal terminal settlement atomically writes status, result or failure, evidence, Artifacts, cleanup, and lease release. A provider session is not a durable recovery checkpoint. At most one Task holds the execution lease from acquisition through final evidence collection. A second otherwise-valid request settles failed with -`execution_capacity_unavailable`. Its transient empty containment set is -destroyed after settlement without releasing the start gate or launching a -helper child. An identical idempotency replay returns the existing Task. Reusing -the key with a different canonical request conflicts. Clients generate at least -128 bits of randomness -once per logical invocation and reuse the same key plus request after an -ambiguous transport failure. Because the initial service has no caller identity, -the idempotency namespace and Task visibility are gateway-wide. +`execution_capacity_unavailable` without launching an acquisition container or +provider process. An identical idempotency replay returns the existing Task. +Reusing the key with a different canonical request conflicts. Clients generate +at least 128 bits of randomness once per logical invocation and reuse the same +key plus request after an ambiguous transport failure. Because the initial +service has no caller identity, the idempotency namespace and Task visibility +are gateway-wide. Terminal Task records, Artifacts, events, and invocation claims expire in one transaction after the configured TTL. The retained-count limit never evicts an unexpired Task; the gateway rejects new admission until expiry frees capacity. -State-store integrity, VFS, helper protocol, or durability failure stops -admission and prevents the gateway from acknowledging creation or reporting -terminal success. +State-store integrity or durability failure stops admission and prevents the +gateway from acknowledging creation or reporting terminal success. -On gateway restart, interrupted nonterminal Tasks settle failed only after -containment reconciliation; provider work is not resumed or automatically -replayed. A new invocation may start fresh. +On gateway restart, interrupted nonterminal Tasks settle failed; provider work +is not resumed or automatically replayed. Admission resumes only after any +recorded acquisition container is gone and the recorded provider process group +is confirmed absent. Otherwise the gateway remains unready with the lease held. ### Make cancellation, evidence, and cleanup explicit -The gateway supervises every acquisition and provider process set. The helper -allocates a stable empty containment set behind a start gate. The Task, -execution lease, and containment identifier commit durably before the helper may -release that gate or execute any child; a failed commit destroys the empty set. - One durable compare-and-set arbitrates provider terminal outcome, caller cancellation, deadline, and shutdown as an internal outcome intent while the externally visible Task remains nonterminal. The winning intent owns the stable -result or failure code and drives one idempotent abort and quiescence path. -Cancellation first invokes the provider's native abort or protocol cancellation, -then applies bounded forced termination to the complete descendant set. +result or failure code and drives one idempotent cancellation and settlement +path. + +On Linux, each direct provider process starts in its own process group. +Cancellation first invokes the provider's supported graceful abort, then sends +`SIGTERM` to the process group after a bounded grace period, and finally sends +`SIGKILL` after a second bounded period. This is best-effort lifecycle control, +not containment: descendants can deliberately detach or escape the group. CI +runner teardown is the final orphan boundary. Cancellation during acquisition +stops and removes the acquisition container and unpublished staging; provider +execution never occurs in that container. Live provider events are bounded while execution runs. Filesystem, Git, and -produced-Artifact evidence is read only after the supervisor proves the complete -invocation process set quiescent through its invocation-owned containment. -Only then does one transaction atomically publish terminal status, the integrity -Artifact, bounded evidence, result or failure, produced Artifacts, termination, -cleanup, and lease release. If quiescence cannot be proven, that transaction -settles `execution_quiescence_unknown` without verified filesystem evidence. -The gateway rejects new work and stays alive with poisoned readiness while -continuing to reap; later recovery changes only internal recovery/readiness -state, never the settled Task. - -On startup the helper enumerates the entire project-owned containment namespace, -including unknown identifiers, and proves every set empty before binding, -releasing a retained lease, quarantining stale roots, or advertising readiness. -It prints the stable containment identifier and platform recovery command for -any nonempty set. An unsupported platform fails before binding rather than -relying on process enumeration. +produced-Artifact evidence is collected only after the direct provider process +has settled and the configured process-group escalation has completed. The +gateway does not claim to prove full descendant quiescence. A settled read-only +Task removes its private runtime and any non-reusable base; it retains only a +reusable cached base. A settled read-write Task removes its writable view and +any non-reusable base after evidence collection. One transaction then atomically +publishes terminal status, the integrity Artifact, bounded evidence, result or +failure, produced Artifacts, observed termination and cleanup outcomes, and +lease release. Task-owned cleanup failure publishes `workspace_cleanup_failed` +and retains an internal cleanup record for reconciliation. If the direct process +does not settle after final escalation, the gateway instead publishes +`execution_termination_failed` without filesystem, Git, or produced-Artifact +evidence; retains any Task-owned runtime, writable view, non-reusable base, and +the lease; stops admission; and remains unready until runner teardown and +startup reconciliation confirm the recorded process group is absent and clean +the retained state. +Evidence describes only what the gateway actually observed; +escaped descendants and uncertain cleanup are never upgraded to verified +outcomes. + +V1 supports trusted Linux CI runners and one active invocation. Other operating +systems and concurrent execution require a separate lifecycle design rather +than silent degradation. Terminal evidence distinguishes: @@ -528,8 +713,9 @@ validation also uses `google.rpc.BadRequest`, never JSON-RPC error carriers. The published versioned extension specification defines Agent Card params, activation, request/idempotency/replay, errors, and terminal Task/Artifact schemas. Its request carries the invocation key, execution target, closed -workspace source, bounded deadline, and optional bounded result schema in its -own strict `Message.metadata` member without rejecting unrelated A2A metadata. +workspace source, logical working-directory selector, bounded deadline, and +optional bounded result schema in its own strict `Message.metadata` member +without rejecting unrelated A2A metadata. The request Message lists the URI in `Message.extensions`. Every terminal Task has one fixed-name, versioned integrity Artifact whose `Artifact.extensions` lists the URI, plus zero or more produced Artifacts. Breaking extension versions @@ -548,8 +734,9 @@ retry. Consumers own those concerns. ## Consequences -- Developers can start one endpoint with `allagents gateway serve` and use - loopback, `0.0.0.0`, a specific interface, Tailscale, or firewall policy. +- Developers who explicitly install the gateway package can start one endpoint + with `allagents-gateway serve` and use loopback, `0.0.0.0`, a specific + interface, Tailscale, or firewall policy. - There is no application authentication, per-caller authorization, tenant isolation, `gateway.yaml`, `worker.yaml`, remote worker protocol, or required Kubernetes deployment in the initial product. @@ -561,26 +748,38 @@ retry. Consumers own those concerns. keeps raw origins under AllAgents operator control. - External network reachability grants access to every available target, including built-in and gateway-enabled profile targets, plus every retained - Task. Operators treat network policy as the authorization boundary; invocation - descendants are isolated from that boundary and host-management networks. -- One durable execution lease enforces one active invocation independent of - consumer concurrency settings. + Task. Operators treat network policy as authorization and must restrict the + systems and secrets available to the trusted CI runner; AllAgents does not + isolate provider or model-tool descendants within that runner. +- One durable SQLite execution lease enforces one active invocation independent + of consumer concurrency settings. - GitHub App credentials support private repositories without forcing every developer to use one identity; GitHub CLI remains a local eligibility fallback only when no App installation applies. - Direct repositories and digest-pinned OCI snapshots converge on one validated - workspace manifest and evidence contract. OCI metadata remains same-origin; - only layer blobs may redirect to exact operator-approved hosts. -- Gateway v1 execution is supported on Linux x64/arm64 with the packaged state/ - security helper, descriptor-rooted SQLite VFS, cgroup v2, mount/network - namespaces, nftables, pidfds, and safe-file operations. Matching helper - packages publish and verify before the root package; unsupported hosts or - missing capabilities fail before binding rather than degrading containment. -- The gateway process remains a meaningful API and lifecycle boundary, but not a - hostile-code sandbox. Strong multi-tenant isolation remains future work. -- Codex and Pi share one conformance suite while retaining bounded native - evidence and honest capability differences. A backend is unavailable unless - its pinned surface can delegate every MCP/tool spawn through the helper. + immutable-base manifest and evidence contract. Immutable source identities may + reuse a cached base; OCI metadata remains same-origin and only layer blobs may + redirect to exact operator-approved hosts. +- Read-only Tasks may share that base and physical cwd while keeping private + runtime state. Read-write Tasks receive disposable independent writable views + through the selected copy-on-write or copy materializer. +- Docker is a short-lived base-acquisition boundary only when no reusable + validated base exists. The container receives staging plus source credentials, + emits a typed manifest, and is removed before Codex or Pi starts on the host + runner. +- The private Bun workspace root orchestrates `apps/cli`, `apps/gateway`, the + image-only `apps/acquirer`, and the three contract/configuration packages. + CLI-only installs fetch neither the gateway package nor acquisition image. +- CLI and gateway versions and releases remain independent. Gateway releases + verify the exact npm tarball and the exact digest-pinned multi-architecture + acquisition image before publishing. +- Codex and Pi use pinned supported automation surfaces and existing host + authentication through narrow adapters. Explicit environment construction + reduces accidental leakage but cannot hide runner secrets from model-invoked + tools. +- Linux process-group escalation provides bounded best-effort cancellation. + Runner teardown remains the final orphan boundary, and evidence never claims + full descendant quiescence. - A future deployment configuration becomes justified only when the product needs multiple worker routes, tenants, credential policies, custom materializers, centralized storage, or other operator-selected variants. @@ -619,6 +818,21 @@ identities and destinations. Repository requests materialize the configured set and may override revisions by declared name; snapshot requests select a declared name and immutable digests. Neither variant introduces a new origin. +### Let callers provide a host cwd + +Rejected because an absolute or configured destination path would let a caller +select unrelated host content and bypass gateway-owned acquisition. Promptfoo +gets the required runtime control through a logical workspace-root or declared- +repository selector; the gateway maps it into the access-appropriate reusable +or Task-owned base or writable view according to `workspaceAccess`. + +### Always allocate a unique full workspace + +Rejected because read-only Tasks have no mutable project state to isolate, and +copying a large immutable workspace for every trial wastes transfer, storage, +and I/O. They share one validated base. Writable Tasks isolate only their +changes through a disposable copy-on-write view or explicit portable copy. + ### Fall back from a selected GitHub App after runtime failure Rejected because it would silently change identity and authorization scope after @@ -646,14 +860,69 @@ versioned extension. Rejected because benchmark orchestration, verification, and persisted evaluation state remain consumer concerns. The gateway executes one coding-agent Task. +### Build v1 around Rust and kernel containment + +Rejected because the trusted CI job is already the execution boundary. A Rust +gateway plus custom cgroups, pidfds, `openat2` VFS behavior, namespaces, +`nftables`, or spawn mediation would add implementation and release risk without +isolating model-invoked tools from secrets available to that job. Reconsider +native or stronger containment only if hostile-code or in-job secret isolation +becomes a product requirement. + +### Adopt AI SDK Harnesses as the backend abstraction + +Rejected because AllAgents needs a small contract tailored to its A2A Task, +evidence, cancellation, and profile semantics. Depending on a broad +cross-provider abstraction would enlarge the compatibility surface without +removing the need to understand the official Codex and Pi automation APIs. + +### Run shared host provider daemons + +Rejected because a long-lived daemon introduces cross-invocation state, +ownership, cancellation, and authentication ambiguity. V1 starts one direct +provider process for the one active Task and treats provider sessions as +ephemeral. + +### Run providers in per-invocation containers + +Rejected because official Codex and Pi automation should reuse the trusted +runner's existing installation and authentication. Copying or mounting OAuth +state into a provider container complicates ownership without creating a +security boundary against model tools. Docker remains limited to acquisition. + +### Trust ambient unversioned provider binaries + +Rejected because PATH discovery can silently change behavior between runs. +Pinned SDK, RPC, or package surfaces are the default; a global binary override +must pass exact version and capability probes, and runtimes are never downloaded +per request. + +### Couple CLI and gateway versions or publish them together + +Rejected because the products have different dependencies and release cadence. +Compatibility is explicit at the contract boundary; gateway-only work must not +force a CLI release, and CLI-only installation must not fetch gateway or +acquisition artifacts. + +### Bundle the gateway into every CLI installation + +Rejected because plugin/skill-only users do not need the A2A server, SQLite, +provider adapters, or acquisition image. The gateway ships as the separately +installed `allagents-gateway` npm package, and its release independently binds +the digest-pinned GHCR acquisition image. + ## Reconsider when Revisit this decision when any of these become requirements: - callers outside one trusted network must share the endpoint; - per-caller Task privacy, authorization, or audit identity is required; +- provider or model-tool code must be isolated from runner secrets or treated as + hostile inside the execution environment; - multiple gateway replicas need transactional shared storage; -- execution must route among remote worker pools or hostile-code sandboxes; +- more than one active invocation or shared provider daemons are required; +- execution must route among remote worker pools or sandboxes; +- non-Linux runners need equivalent lifecycle and cancellation semantics; - custom materializers are needed beyond direct Git and OCI snapshots; - multiple GitHub hosts, Apps, CLI accounts, or ordered credential policies need declarative configuration; diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index b2e14039..d6db4a43 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,7 +1,7 @@ --- title: "Coding-Agent Execution Gateway - Plan" date: 2026-09-18 -updated: 2026-09-19 +updated: 2026-09-20 type: feat artifact_contract: ce-unified-plan/v1 artifact_readiness: implementation-ready @@ -13,31 +13,49 @@ execution: code ## Goal Capsule -- **Objective:** A developer can run one trusted-network A2A endpoint for one - AllAgents workspace and invoke built-in or explicitly gateway-enabled profile - targets against either the complete configured Git repository set, with optional - named revision overrides, or a digest-pinned OCI workspace snapshot. AI Evals - can configure either source mode in Promptfoo YAML through a custom provider - without sending origins. -- **Means:** Add `allagents gateway serve`, a private execution-service package, - a bounded durable Task store, direct Codex and Pi adapters, GitHub App and - GitHub CLI acquisition providers, OCI snapshot acquisition, one supervised - invocation lifecycle, and a documented Promptfoo provider contract. +- **Objective:** A developer can install and run one trusted-network A2A + endpoint for one AllAgents workspace and invoke built-in or explicitly + gateway-enabled profile targets against either the complete configured Git + repository set, with optional named revision overrides, or a digest-pinned + OCI workspace snapshot. AI Evals can configure either source mode in + Promptfoo YAML through a custom provider without sending origins. +- **Means:** Convert the repository to a private Bun workspace monorepo with + independently released `allagents` and `allagents-gateway` applications, + versioned workspace/execution/acquisition contract packages, generated + portable fixtures under `contracts/`, a bounded SQLite Task store, direct + Codex and Pi host-process adapters, and one digest-pinned acquisition image + used only when no reusable validated base exists. Read-only Tasks share a + reusable base or own a non-reusable base for mutable revisions; read-write + Tasks receive disposable writable views through an automatic block-clone/ + OverlayFS materializer with an explicit portable copy backend. + Providers still run bare metal on the trusted Linux CI runner. - **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) - owns the public and trust boundaries. Project and user `workspace.yaml` files - own source and profile declarations. A2A 1.0 owns core wire semantics. -- **Execution order:** Capture a red built-CLI E2E for the missing gateway; - freeze schemas and configuration projection; implement the Task store, A2A - server, supervisor/helper, acquisition, Codex, and Pi; run a final - implementation review and fix important findings; then run the green built- - CLI E2E, repository gates, and documentation validation. -- **Stop conditions:** Do not add application authentication, `gateway.yaml`, - `worker.yaml`, remote worker routing, caller-supplied URLs or commands, - mutable OCI tags, selected-provider failure fallback, evaluation behavior, or - automatic execution retry. -- **Tail ownership:** The implementing workflow runs focused contract and - lifecycle tests, provider fixture tests, the repository quality gates, a - built-CLI trusted-network smoke test, and documentation validation. + owns the public, trust, runtime, and packaging boundaries. Project and user + `workspace.yaml` files own source and profile declarations. A2A 1.0 owns core + wire semantics. The CI job, VM, or deployment container is the only + operational execution and isolation boundary. AllAgents does not claim that + boundary contains hostile code or hides job secrets from model-invoked tools; + it owns process lifecycle and truthful evidence only. +- **Execution order:** Capture red CLI-only and standalone-gateway package + smokes; complete the Bun monorepo, A2A SDK, provider-surface, process-group, + Docker-acquirer, package, and release feasibility gate; freeze schemas, + generated fixtures, configuration projection, SQLite ownership, and release + binding; implement the Task store and A2A server, host supervisor, Docker-only + acquisition, Codex, and Pi; run final review; then run green packed-package, + exact-image, registry-conformance, repository, and documentation gates. +- **Stop conditions:** Stop dependent production work if the pinned A2A surface + cannot implement the required public protocol or if the acquisition-container + boundary and exact image/package release binding are infeasible. A missing + Codex or Pi capability makes that target unavailable rather than changing the + A2A, trust, source, or Task contracts. Do not add application authentication, + deployment YAML, remote workers, caller-supplied origins/commands, mutable OCI + tags, provider fallback, evaluation behavior, automatic retries, per-provider + Docker, or containment claims. +- **Tail ownership:** The implementing workflow runs focused contract, + lifecycle, provider-environment, process-group, acquisition-boundary, and + release-binding tests; repository quality gates; packed CLI/gateway and exact + acquisition-image smoke tests; registry conformance; and documentation + validation. --- @@ -49,11 +67,13 @@ AllAgents gains a single-workspace coding-execution service without becoming an evaluation framework or multi-tenant platform. Callers use A2A Tasks and one required AllAgents extension. Network reachability is authorization. The service resolves configured targets and sources from existing workspace files, -acquires a fresh invocation workspace, invokes Codex or Pi through a typed -adapter, and retains bounded terminal evidence. AI Evals consumes that boundary -through its own Promptfoo custom provider: evaluation YAML supplies named -revision overrides for the configured repository set, or one snapshot handle -and immutable digests, while AllAgents retains origin and credential authority. +acquires or reuses an immutable base, shares it for read-only Tasks, creates an +independent disposable view for read-write Tasks, invokes Codex or Pi through a +typed adapter, and retains bounded terminal evidence. AI Evals consumes that +boundary through its own Promptfoo custom provider: evaluation YAML supplies +named revision overrides for the configured repository set, or one snapshot +handle and immutable digests, while AllAgents retains origin and credential +authority. ### Problem Frame @@ -97,13 +117,21 @@ registry, or another profile configuration file for the initial use case. - **Support two acquisition modes.** Direct declared repositories and named, digest-pinned OCI workspace snapshots converge on one manifest and evidence contract. (session-settled: user-directed.) Governs R9-R11. +- **Share immutable bases; isolate writes.** `workspaceAccess` defaults to + `readWrite`. Read-only Tasks may reuse one validated physical base and cwd + with Task-private runtime state; read-write Tasks receive unique disposable + writable views. Reflink/block clone is preferred, rootless OverlayFS is the + Linux fallback, and an explicit copy backend preserves portability. + (session-settled: user-directed.) Governs R2-R3, R5, R8-R11, R15-R16, R18-R19. - **Use App-first GitHub credential eligibility.** Prefer an applicable GitHub App; use a configured `gh` account only when no App installation applies; never fall back after selected-App failure. (session-settled: user-directed.) Governs R12. -- **Keep a typed backend seam.** Codex SDK and Pi RPC are the complete initial - backend set. Launcher-backed profiles resolve through those adapters rather - than executing generated wrapper files. Governs R7-R8, R13-R15. +- **Keep a narrow typed backend seam.** Pinned supported Codex and Pi package or + RPC surfaces are the complete initial backend set. Launcher-backed profiles + resolve through AllAgents-owned adapters and execute on the gateway host + rather than through generated wrapper files or the acquisition container. + Governs R7-R8, R13-R15. - **Persist Task truth, not provider sessions.** Restart settles interrupted work failed; it never resumes or automatically replays provider execution. Governs R5, R13-R16. @@ -139,20 +167,27 @@ registry, or another profile configuration file for the initial use case. `nextPageToken` is present and empty on the final page. With the default `includeArtifacts: false`, each returned Task omits `artifacts`; `true` includes the field. -- R2. Generate a strict versioned request schema from Zod and place it only at - `Message.metadata[extensionUri]`; the Message also lists `extensionUri` in - `Message.extensions`. Strict objects reject every unlisted member. V1 uses - these wire scalars: +- R2. Generate a strict versioned request schema from the canonical domain type + and place it only at `Message.metadata[extensionUri]`; the Message also lists + `extensionUri` in `Message.extensions`. Strict objects reject every unlisted + member. V1 uses these wire scalars: - `InvocationKey` matches `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`. - `ConfigName` and `TargetId` match `^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`. - `RevisionText` is NFC UTF-8, 1-255 bytes, with no U+0000-U+001F or U+007F. - `Digest` matches `^sha256:[0-9a-f]{64}$`. + - `RelativeDirectory` is NFC UTF-8 of 1-1024 bytes containing 1-32 + slash-separated segments. Each segment is 1-255 bytes, is neither `.` nor + `..`, and contains no slash, backslash, U+0000-U+001F, or U+007F. The request object is exactly: `version: "1"`; `invocationKey: InvocationKey`; `target: TargetId`; `source`, one of `{ kind: "repositories", revisions?: Record }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, workspaceManifestDigest: Digest }`; optional + `workingDirectory`, one of `{ kind: "workspaceRoot" }` or + `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }`, + defaulting to `{ kind: "workspaceRoot" }`; optional `workspaceAccess`, one of + `"readOnly" | "readWrite"`, defaulting to `"readWrite"`; optional `deadlineSeconds` (integer 1-3600, default 1800); and optional `resultSchema: { version: "1", schema: SchemaNode }`. @@ -210,6 +245,14 @@ registry, or another profile configuration file for the initial use case. media type of at most 255 ASCII bytes. - `version` is the literal `"1"`; `taskId` is a lowercase canonical UUIDv7; and `target` is `TargetId`. + - `workingDirectory` is the effective logical selector from the request: + `{ kind: "workspaceRoot" }` or + `{ kind: "repository", repository: ConfigName, + path?: RelativeDirectory }`. It never contains a physical path or configured + repository destination. + - `workspaceAccess` is the effective `"readOnly" | "readWrite"` value. + `readOnly` is a consumer-selected cooperative contract with best-effort + provider-policy enforcement, not hostile-code containment. - `sourceIdentity` is either `{ kind: "repositories", complete, repositories }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, @@ -246,9 +289,17 @@ registry, or another profile configuration file for the initial use case. termination | cleanup`; `artifactId` and `digest` use the aliases above, `summary` is `ShortText`, and at least one of those three optional members is present. + `complete` means every configured bounded evidence category was attempted + after the direct provider process settled; it never means every descendant + was enumerated or quiescent. - `termination` is `{ status: "clean" | "failed" | "unknown", - reason?: ShortText }`; `cleanup` is - `{ workspace: "removed" | "retained" | "failed", reason?: ShortText }`. + reason?: ShortText }` and reports the direct provider/process-group + observation only. + - `cleanup` is + `{ workspace: "shared" | "removed" | "retained" | "failed", + reason?: ShortText }`. `shared` means a read-only Task removed its private + runtime state while retaining a reusable cached base; `removed` means every + Task-owned runtime, writable view, and non-reusable base was removed. - optional `failure` is `{ code, message, retryable, cause }`, where `code` is one stable code from the error table below, `cause` is one member of the closed cause union defined below that table, `message` is `ShortText`, and @@ -281,22 +332,33 @@ registry, or another profile configuration file for the initial use case. equivalent network controls as the authorization boundary. - R5. Idempotency and Task visibility are deployment-wide. Atomically and durably bind an invocation key to the canonical request, selected target, - source identity, optional result-schema digest, deadline, and effective - configuration digest before acknowledging Task creation. One transactional + source identity, effective logical working directory, effective workspace + access, optional result-schema digest, deadline, and effective configuration + digest before acknowledging Task creation. One transactional `createOrReplay` operation arbitrates competing requests. Identical replay returns the existing Task; a changed request conflicts. Status and terminal settlement are monotonic. The project-specific state root persists the - canonical workspace identity and holds an exclusive process lock. The root - and all state files must be current-user owned, use `0700`/`0600`-equivalent - permissions, be disjoint from project, profile, and invocation roots, and be - opened descriptor-relatively without following symlinks or accepting hard- - linked files. Startup verifies those invariants, store integrity, and - workspace identity; terminalizes interrupted Tasks failed; and never resumes - provider work. A durable commit fsyncs every changed file and affected - containing directory before acknowledgment. Store open, corruption, write, - transaction, rename, or fsync failure stops admission, aborts and contains - active work, prevents terminal success, and keeps the process alive with - poisoned readiness until the containment set is proven empty. + canonical workspace identity and holds an exclusive process lock. + + The Bun gateway privately owns one SQLite database through `bun:sqlite`. + Claims, Tasks, events, bounded Artifact bytes, the single execution lease, + internal outcome intent, acquisition-container identity, staging and + transient-base identity, direct provider process-group identity, and expiry + live in ordinary transactional tables. Enable foreign keys, use WAL where + supported, set `synchronous=FULL`, and acknowledge only committed + transactions. The current user owns the state root and database with + `0700`/`0600`-equivalent permissions; the root is disjoint from project, + staging, publication, profile, and provider-auth roots. No custom SQLite VFS + or native file primitive is introduced. Startup verifies the root, lock, + schema/integrity, and workspace identity; terminalizes interrupted Tasks + failed; removes recorded acquisition containers and orphan staging; and + attempts to terminate recorded provider process groups. It releases a stale + or termination-poisoned lease only after the container is gone, the recorded + process group is confirmed absent, and recorded Task-owned runtime, view, and + transient-base cleanup has completed. Otherwise readiness remains false and + admission stays stopped. Provider work is never resumed. + Store open, corruption, write, transaction, or synchronization failure stops + admission and prevents terminal success. **Workspace and target configuration** @@ -317,29 +379,63 @@ registry, or another profile configuration file for the initial use case. one `(profile, client)` pair. Built-in IDs are reserved under the same portable collision key; colliding enablement is a configuration error. Initially only Codex and Pi profile clients are executable. -- R8. A request selects a declared target, may set the bounded - `deadlineSeconds`, and may provide one bounded result schema. The gateway owns - one durable execution lease covering acquisition through final evidence - collection. Admission claims that lease transactionally before launching any - helper child; at most one Task may hold it. A second otherwise-valid request - is accepted as a Task and settles failed with - `execution_capacity_unavailable`. It may transiently allocate one empty, - start-gated containment set, but never releases the gate, starts acquisition, - or executes a child and must destroy that set after the failure settlement. - Lease identity is stored with the Task, survives restart, and is released only - by the final settlement transaction or startup reconciliation after the - recorded containment set is proven empty. - - The overall deadline covers acquisition, publication, typed preparation, - provider execution, and evidence collection. Acquisition receives +- R8. A request selects a declared target, may select one logical + `workingDirectory`, may select `workspaceAccess`, may set the bounded + `deadlineSeconds`, and may provide one bounded result schema. `workspaceAccess` + defaults to `readWrite`; it is never inferred from prompt text. + + A read-only Task resolves its cwd directly inside its validated base. An exact + immutable request may share a reusable cached base; a mutable branch or tag + request owns a non-reusable base for the Task lifetime. The Task receives a + private `//runtime` for temporary, home, + provider-state, and evidence files. Provider environments disable optional + Git locks, and adapters request native read-only policy when available. The + gateway does not inspect prompts or add a per-Task mount, chmod pass, or full- + tree verification. The consumer remains responsible for assigning work that + does not require project mutation. + + A read-write Task receives a unique writable view at + `//workspace`. The configured host materializer + prefers a filesystem block clone, falls back to rootless OverlayFS on + supported Linux hosts, and supports an explicit ordinary-copy backend for + portability. Writable files never use hard links. Normal settlement removes + the writable view and any non-reusable base after evidence collection; non- + settling execution retains them with the poisoned lease until verified + reconciliation. + + `{ kind: "workspaceRoot" }` selects the effective base or Task view. A + repository selector maps its declared name through the compiled catalog, + appends only the validated `RelativeDirectory`, resolves links without escape, + and must name an existing directory beneath that repository. Absolute paths, + configured destinations, undeclared repositories, non-directories, and + escaping resolutions fail before provider start. Gateway-generated requests, + structured Task/Artifact metadata, and operational logs never contain the + physical path. Opaque terminal output, native evidence, and produced-Artifact + payloads are not sanitized and may contain it. + + The gateway owns one durable execution lease covering base acquisition or + lookup through final evidence collection. Admission claims that lease + transactionally before starting an acquisition container or provider process; + at most one Task may hold it. A second otherwise-valid request is accepted as + a Task and settles failed with `execution_capacity_unavailable` without + creating a container or process. Lease identity is stored with the Task and + survives gateway restart. Normal final settlement releases it; a provider- + termination failure retains it until verified startup reconciliation confirms + the recorded process group is absent. + + The overall deadline covers cache lookup, Docker acquisition on miss, + publication, optional materialization, typed preparation, bare-metal provider + execution, and evidence collection. Acquisition receives `min(900 seconds, remaining overall deadline)`; exceeding that sub-budget - fails before provider execution. Overall expiry initiates abort and bounded - forced termination. Cleanup then uses its own fixed bounded budget and the - R16 fail-closed quiescence rule. A request cannot provide or override backend, - executable path, command, argv, environment, profile settings, plugins, MCP - servers, repository URLs, destination paths, credential provider, setup - behavior, or permission policy. Readiness rejects missing, partial, drifted, - unsupported, or declaration-missing gateway-enabled profiles. + removes the acquisition container and fails before provider execution. + Overall expiry initiates adapter abort and Linux process-group escalation. + Cleanup then uses its own fixed bounded budget. A request cannot provide or + override backend, executable path, command, argv, environment, provider home, + profile settings, plugins, MCP servers, repository URLs, destination paths, + credential provider, setup behavior, provider permission policy, materializer, + cache key, Docker options, image reference, or mounts. Readiness rejects + missing, partial, drifted, unsupported, or declaration-missing gateway-enabled + profiles. **Workspace acquisition** @@ -356,7 +452,10 @@ registry, or another profile configuration file for the initial use case. repository-controlled secondary fetch/exec features, verify checkout identities, and reject path collisions or escapes. - R11. Snapshot mode maps `snapshot` to a declared OCI repository and constructs - `@` server-side. V1 accepts only + `@` server-side. The same digest-pull contract must + interoperate with Docker Hub, GHCR, JFrog Artifactory/JFrog Container + Registry, and compatible private OCI Distribution registries; registry choice + does not alter the accepted snapshot format. V1 accepts only `application/vnd.oci.image.manifest.v1+json` with `schemaVersion: 2` directly at the requested digest. Reject image indexes, nested indexes, descriptor `urls` or embedded `data`, non-distributable layers, unknown media types, and @@ -368,24 +467,38 @@ registry, or another profile configuration file for the initial use case. while streaming, before decoding. Apply layers base-to-top with OCI whiteout and opaque-whiteout semantics. - The workspace manifest must contain every compiled project repository exactly - once at its operator-declared destination; reject missing, extra, renamed, - misplaced, or duplicate repositories and undeclared generated content. Apply - these fixed v1 ceilings across all processed layers, including overwritten or - whiteouted content: 4 MiB manifest, 4 MiB config, 2 GiB total compressed - layer bytes, 8 GiB total expanded bytes, 250,000 entries, 1 GiB per regular - file, 4096 UTF-8 bytes and 128 components per path, and 1 MiB per PAX or other - extended header. Abort before crossing a limit. Validate paths, collisions, - file types, modes, links, and manifest completeness in staging before atomic - publication. Reject absolute or traversing paths, devices, sockets, sparse - files, escaping links, credentials in redirect URLs, unapproved cross-origin - redirects, and external layers. Cross-origin redirects are limited to - layer-blob `GET`/`HEAD` requests and exact operator-declared - `layerRedirectHosts`; token, manifest, and config requests remain same-origin. - Private or otherwise non-global destinations are permitted only when the exact - host is the source's declared repository host or a declared layer-redirect - host, with per-hop rebinding checks. The common workspace manifest - distinguishes independently verified Git facts from snapshot-attested facts. + The wire-visible workspace manifest contains every compiled project + repository exactly once by logical name and omits destination paths. After + applying layers, the gateway uses the compiled operator catalog to verify that + each listed repository exists at its configured destination and that no + repository is missing, extra, renamed, misplaced, duplicated, or accompanied + by undeclared generated content. Apply these fixed v1 ceilings across all + processed layers, including overwritten or whiteouted content: 4 MiB manifest, + 4 MiB config, 8 GiB total compressed layer bytes, 32 GiB total expanded bytes, + 500,000 entries, 4 GiB per regular file, 4096 UTF-8 bytes and 128 components + per path, and 1 MiB per PAX or other extended header. Abort before crossing a + limit. Validate paths, collisions, file types, modes, links, and the compiled + filesystem layout in staging before atomic publication. Reject absolute or + traversing paths, devices, sockets, sparse files, escaping links, credentials + in redirect URLs, unapproved cross-origin redirects, and external layers. + Cross-origin redirects are limited to layer-blob `GET`/`HEAD` requests and + exact operator-declared `layerRedirectHosts`; token, manifest, and config + requests remain same-origin. Private or otherwise non-global destinations are + permitted only when the exact host is the source's declared repository host + or a declared layer-redirect host, with per-hop rebinding checks. The common + path-free workspace manifest distinguishes independently verified Git facts + from snapshot-attested facts; compiled destinations remain private validation + inputs. + + After host validation, atomically promote staging to a validated base. An OCI + identity is reusable under a gateway-owned cache key containing its manifest + and workspace-manifest digests. A repository identity is reusable only when + every effective revision is a full commit ID. Its cache key also binds the + acquisition-contract version, compiled catalog/layout digest, and every + commit. Branch and tag requests instead receive a non-reusable Task-owned base + and never populate or reuse a cache entry. A valid cache hit starts no + acquisition container and resolves no source credential. Active Tasks pin a + reusable base; bounded eviction removes only unpinned cache entries. **Credential selection and containment** @@ -395,119 +508,148 @@ registry, or another profile configuration file for the initial use case. independently proven through the configured GitHub CLI identity; an uncorroborated 404, 401, 403, 429, timeout, or 5xx is `unknown`. An explicit installation ID is eligible only after positive repository-coverage - verification. For `eligible`, call `@octokit/auth-app` with `refresh: true` - and the exact repository selection to mint a new read-only installation token - for every acquisition. Validate its repository selection, permissions, - creation time, and expiry, and require remaining lifetime greater than the R8 - acquisition sub-budget plus a 60-second clock-skew margin. + verification. For `eligible`, the host gateway creates the App JWT from the + configured private key, discovers and verifies installation coverage, and + mints a new repository-scoped read-only installation token for every cache- + miss acquisition. Credentials are never cached. Validate the token's + repository selection, + permissions, creation time, and expiry, and require remaining lifetime greater + than the R8 acquisition sub-budget plus a 60-second clock-skew margin. Only + the resulting installation token enters the acquisition container; the App + private key remains on the host. Use the configured GitHub CLI account only when the App is absent or applicability is positively `ineligible`. An `unknown` result or any selected-App configuration, authentication, minting, permission, repository, rate-limit, or service failure terminates acquisition without `gh` fallback. - Run `gh auth token --hostname github.com --user ` with ambient token - variables removed. Deliver either token only through an invocation-scoped Git - credential helper. Revoke an App token after acquisition and fail before - provider execution if revocation cannot be confirmed; destroy all local token - material before typed preparation. OCI credentials likewise exist only during - snapshot acquisition. + Resolve `gh auth token --hostname github.com --user ` on the host + with ambient token variables removed, then inject only the selected + invocation-scoped source credential into the acquisition container. Revoke an + App token after acquisition and fail before provider execution if revocation + cannot be confirmed. OCI acquisition accepts anonymous pulls or exact- + registry credentials from the strict Docker-auth/helper boundary and supports + same-origin Basic and Distribution Bearer challenges, the documented Docker + Hub token service, and an operator-supplied exact-host CA-bundle map. The + acquisition container receives source-only credentials and trust material; + they are destroyed with the container before provider preparation. **Execution, evidence, and cleanup** -- R13. Keep one closed `codex | pi` backend registry and one behavior-focused - interface covering availability, capabilities, invocation, progress, - deterministic permission handling, abort, terminal output, optional structured - result, usage, bounded native evidence, and disposal. Profile targets resolve +- R13. Keep one closed `codex | pi` backend registry behind a narrow + AllAgents-owned TypeScript interface covering availability, capabilities, + invocation, progress, deterministic permission handling, abort, direct + process settlement, terminal output, optional structured result, usage, + bounded native evidence, and disposal. The gateway owns contract + normalization rather than adopting AI SDK Harnesses. Profile targets resolve adapter-owned configuration directly; never execute generated launchers, - discover executables as targets from `PATH`, scrape a TUI, or append public - input to argv. -- R14. Codex uses pinned `@openai/codex-sdk`, one fresh thread per Task, - `AbortSignal`, streamed events, and an operator-selected Codex auth-file - handle. It passes native `outputSchema` only when the public schema has an - object root, every object's `required` set equals its property set, nesting is - at most 10 levels, and every keyword is supported by the pinned model/API. - Other valid public schemas use explicit JSON prompt guidance plus the common - gateway-side validator without a native schema. Pi uses strict RPC, - invocation-owned configuration, an operator-selected Pi auth-file handle, and - one restricted policy extension; repository extensions and unrestricted - built-ins do not auto-load. The gateway copies only the selected adapter's - required auth material into an invocation-private, read-only control-process - view and removes it during cleanup. -- R15. Acquire into a private staging root and atomically publish the invocation - workspace. Run only adapter-owned typed preparation that projects validated - project/profile settings, plugins, and MCP declarations through existing - deterministic transforms; never execute project or user `setup` entries or - other configured shell commands. - - Enforce distinct process views: - - the provider control process receives only its invocation workspace, - minimum non-secret profile configuration, and adapter auth channel; - - each MCP child receives only its own resolved secret references; and - - model-invoked shell/tools receive the workspace and no provider or MCP - credentials. - - The pinned backend must expose one non-bypassable synchronous spawn hook for - every MCP and model-tool process. The hook delegates execution to the security - helper, which enters the role-specific mount and network namespaces, replaces - the environment, closes every non-allowlisted descriptor, and only then - executes untrusted code. A backend that can spawn any tool without this hook - is not a v1 target and fails readiness; conformance fixtures alone cannot waive - that requirement. All views exclude gateway state, operator home, App keys, - GitHub/OCI stores, source helpers, unrelated adapter credentials, and the - parent environment. - - Invocation network namespaces cannot route to host loopback, any gateway bind - or advertised address, operator management networks, or ingress proxies. - Provider and MCP egress is default-deny except for role-specific destinations - compiled from adapter and MCP configuration; every resolved address is checked - at connection time, and gateway/host-management destinations remain denied - even when a hostname resolves to them. Model tools receive no network unless - the adapter's explicit policy grants similarly constrained egress. Fail target - readiness unless all filesystem, credential, descriptor, and network - separations are enforceable. - - Acquisition credentials and mounts are absent first. Capture bounded provider - events while the process is live. After the provider reports terminal, abort - and terminate its complete containment set and prove it empty before reading - Git state, hashing or copying files, or describing produced Artifacts as - verified. Treat the mutated workspace as untrusted: use descriptor-relative - no-follow reads; revalidate identity and size after open; reject hard links, - special/sparse files, path replacement, out-of-root targets, and `.git` - gitdir/core.worktree/alternates escapes; and run Git inspection with hermetic - configuration that disables hooks, filters, drivers, fsmonitor, pagers, - helpers, and external commands. If quiescence cannot be proven, retain only - truthful partial process evidence; do not publish filesystem evidence or - produced Artifacts as verified. -- R16. Supervise the complete acquisition/provider descendant set inside an - invocation-owned OS containment primitive whose membership children cannot - escape. Allocate its stable identifier and empty set first, then commit that - identity with the Task and execution lease before the helper may release its - start gate or execute any child. A failed commit destroys the still-empty set. - Startup enumerates the entire project-owned containment namespace, reconciles - both recorded and unknown identifiers, and refuses readiness while any - unknown or nonempty set remains. - - One durable compare-and-set arbitrates provider terminal outcome, caller + discover arbitrary executables as targets from `PATH`, scrape a TUI, append + public input to argv, or download a provider runtime per request. A configured + globally installed binary override is eligible only after an exact version + and protocol compatibility probe. +- R14. Codex uses a pinned `@openai/codex-sdk` directly from the Bun gateway. + Codex app-server is allowed only if U0 demonstrates a required capability + absent from that pinned SDK; the reason and tested protocol version must then + be recorded. Each Task receives one fresh SDK execution context, streamed + events, native cancellation, and an explicitly constructed child environment. + When API credentials are absent, preserve the existing host `CODEX_HOME` and + ChatGPT login in place; do not copy, mount, parse, or import OAuth files. + Pass native `outputSchema` only when the public schema has an object root, + every object's `required` set equals its property set, nesting is at most 10 + levels, and every keyword is supported by the pinned SDK/model. Other valid + public schemas use explicit JSON guidance plus the common gateway-side + validator. + + Pi uses a pinned supported package/RPC surface, invocation-owned + configuration, one restricted policy extension, and the existing host Pi + authentication location. Repository extensions and unrestricted built-ins do + not auto-load. Do not copy, mount, parse, or import Pi authentication files. + Both adapters preserve only required host identity, authentication paths, + executable lookup, locale, certificate, and proxy settings in an explicit + environment allowlist. This reduces accidental environment leakage; it is + not a secret-isolation guarantee because model-invoked tools run with the same + CI-job authority. +- R15. Start a fresh Docker container only when a request has no reusable + validated base, including a cache miss or a non-reusable branch/tag request. + Probe Docker and the exact digest-pinned `apps/acquirer` image at that point. + A repository-mode probe failure is `source_git_unavailable`; a snapshot-mode + probe failure is `source_snapshot_unavailable`. The gateway creates private + staging and starts the image with that directory as its only writable bind + mount. The container receives the canonical acquisition request, compiled + catalog, strict network/size/archive policy, source-only GitHub or OCI + credentials, and only required exact-host CA material. It receives no GitHub + App private key, host home, provider home, Docker socket, gateway database, + published base, unrelated credential, Codex, Pi, or other coding harness. It + never downloads a coding harness. Docker network access is limited to source + endpoints required by the selected Git or OCI mode. + + The acquirer writes content beneath staging and emits one typed manifest + through the bind mount, then exits. The host gateway waits for exit, removes + the container, destroys source credentials, validates the manifest and tree + against the compiled catalog and fixed limits, and atomically promotes staging + to a reusable cache entry or non-reusable Task-owned base. Every cancellation, + deadline, validation failure, or other non-publication path removes staging + idempotently; cleanup uncertainty fails `source_cleanup_failed` and stops + admission. Source-mode failure never falls through. + + A read-only Task uses the base directly plus Task-private runtime state. + Adapter-owned preparation for that mode must keep project files unchanged and + place invocation configuration outside the base. A read-write Task first + receives a unique block-cloned, overlaid, or copied view; typed preparation + may then project validated project/profile settings, plugins, and MCP + declarations into that view. Project or user `setup` entries and other + configured shell commands never run automatically. + + Codex and Pi execute as direct host processes on the same trusted Linux CI + runner as the gateway. The adapter receives the access-appropriate resolved + cwd selected by the logical `workingDirectory`, Task-private runtime paths, + and the effective access mode; it never receives a caller-supplied physical + path or materializer choice. Provider execution never reuses the acquisition + container and never creates a per-invocation provider container. The CI job, + VM, or deployment container is the isolation boundary. AllAgents does not + claim containment of hostile repository code, network access by model tools, + or provider/MCP/operator secrets from those tools. Capture bounded provider + events while the direct provider process is live. Collect filesystem/Git + evidence only after that direct process settles and process-group termination + attempts finish; phrase the evidence as observed after direct-process + settlement, never as proof that every descendant is quiescent. Run Git + inspection with hermetic configuration that disables hooks, filters, drivers, + fsmonitor, pagers, helpers, optional locks, and external commands. +- R16. On trusted Linux runners, start each direct provider in a new process + group and persist its leader PID plus Linux process-start marker with the Task + and execution lease before recording provider execution as started. One + durable compare-and-set arbitrates provider terminal outcome, caller cancellation, overall deadline, and shutdown as an internal `outcomeIntent` - while the externally visible Task remains nonterminal. The winning intent - owns the stable result or failure code and drives one idempotent abort and - quiescence path. Only after quiescence, safe evidence collection, produced- - Artifact verification, and cleanup does one settlement transaction atomically - write terminal Task status, result/failure, bounded evidence, exactly one - integrity Artifact, produced Artifacts, termination outcome, lease release, - and cleanup outcome. - - Cancellation intent persists before native abort, followed by bounded forced - termination. If quiescence cannot be proven, settle once with - `execution_quiescence_unknown`, no verified filesystem evidence, and immutable - unknown/failed termination; reject admission, keep readiness false, and leave - the process alive to continue reaping. Later recovery changes only internal - recovery/readiness state, never the settled Task. Print the stable containment - identifier and platform recovery command. Startup proves every interrupted set - empty before it may quarantine stale roots or advertise readiness. Graceful - shutdown stops admission atomically, commits shutdown intent, drains or aborts - active work within a bounded grace period, follows the same settlement path, - and only then exits. + while the external Task remains nonterminal. The winning intent owns the + stable result or failure code and drives one idempotent abort path: request + graceful adapter abort, wait the configured grace period, send `SIGTERM` to + the process group, then `SIGKILL` after the forced-termination period. + + After the direct provider process has settled and bounded evidence collection + finishes, cleanup removes a read-only Task's private runtime state, unmounts + and removes a read-write Task's writable view, and removes any non-reusable + base. One transaction then writes terminal Task status, result/failure, + bounded evidence, exactly one integrity Artifact, produced Artifacts, observed + termination, cleanup outcome, and lease release. Any Task-owned cleanup + failure settles `workspace_cleanup_failed` with + `cleanup.workspace: "failed"` and retains its internal cleanup record for + startup or operator repair; admission stops when an active mount or uncertain + writable view remains. If the direct process does not settle after `SIGKILL`, + one transaction instead writes a failed Task with + `execution_termination_failed`, live provider evidence, and observed + termination, but no filesystem, Git, or produced-Artifact evidence. It retains + the Task runtime, any writable view or non-reusable base, and the lease; makes + readiness false; and stops admission. Startup may release that poisoned lease + only after the recorded process group is confirmed absent following runner + teardown and retained Task-owned state is reconciled; otherwise it remains + unready. + + Repeated cancellation while intent is pending does not re-signal work. + Startup never resumes a session. Gateway shutdown stops admission, commits + shutdown intent, performs the same escalation and settlement rules, and + exits. CI runner teardown is the final orphan boundary. AllAgents does not use + cgroups, pidfds, namespaces, nftables, `openat2`, a native platform layer, or + non-bypassable spawn mediation, and does not claim complete descendant + enumeration or hostile-code containment. **Scope and configuration** @@ -515,33 +657,40 @@ registry, or another profile configuration file for the initial use case. repetitions, experiment scheduling, or automatic Task retry. - R18. Do not add `gateway.yaml` or `worker.yaml`. Process configuration uses the exact CLI flags and environment variables in the Configuration Contract - for listener, advertised interface URL, workspace, state/retention, GitHub, - OCI, and Codex/Pi auth-file handles. The listener also exposes unauthenticated - metadata-only `/healthz` and `/readyz` endpoints outside A2A: liveness returns - 200 while the process can serve; readiness returns 200 only while new - admission is safe and otherwise 503. They reveal no targets, sources, paths, - or failure details and do not require A2A headers. Gateway code never copies - acquisition or provider credential values into generated workspace files, - requests, logs, Task/Artifact metadata, retained workspaces, or model-tool - environments. This is not a redaction guarantee for opaque prompts, provider - output, structured results, native evidence, or produced-Artifact payloads. + for listener, advertised interface URL, workspace, state/retention, + immutable-base cache, workspace materializer, acquisition image and Docker + access, GitHub/OCI source credentials, provider executable overrides, provider + home/auth paths, and process-group timeouts. The listener also exposes + unauthenticated metadata-only `/healthz` and `/readyz` endpoints outside A2A: + liveness returns 200 while the process can serve; readiness returns 200 only + while new admission is safe and otherwise 503. They reveal no targets, + sources, paths, or failure details and do not require A2A headers. Gateway code + never copies acquisition credential values into generated workspace files, + requests, logs, Task/Artifact metadata, retained Task views, cache entries, or + provider environments. This is not a redaction or isolation guarantee for + opaque prompts, provider/tool output, structured results, inherited host + authentication, native evidence, or produced-Artifact payloads. - R19. Document AI Evals consumption through a Promptfoo custom JavaScript/TypeScript provider implementing Promptfoo's `ApiProvider`. `constructor(options: ProviderOptions)` requires and retains a nonempty `options.id`, validates `options.config`, and `id()` returns that ID. Static - config contains the - gateway endpoint, target ID, and exactly one closed source mode: repository - mode materializes the complete configured repository set and carries only an - optional revision map keyed by declared repository name; snapshot mode carries - one declared snapshot name with OCI and workspace-manifest digests. + config contains the gateway endpoint, target ID, optional default logical + `workingDirectory`, optional `workspaceAccess` defaulting to `readWrite`, and + exactly one closed source mode: repository mode materializes the complete + configured repository set and carries only an optional revision map keyed by + declared repository name; snapshot mode carries one declared snapshot name + with OCI and workspace-manifest digests. `callApi(prompt, context?, options?)` may apply the exact - `context?.vars?.allagentsSource` leaf overrides defined below; missing context - means no override. Dynamic repository revisions must be full lowercase - 40-hex commit IDs; dynamic snapshot values must be full lowercase `sha256:` - digests. Source kind, snapshot name, and repository origins never vary per - test. Unknown members, revision names absent from static config, URLs, - destinations, tags, credentials, commands, and permission policy fail before - submission. + `context?.vars?.allagentsSource` leaf overrides, may replace the default + selector through `context?.vars?.allagentsWorkingDirectory`, and may replace + access through `context?.vars?.allagentsWorkspaceAccess`; missing context + retains static values. Dynamic source values remain limited as defined below. + The working-directory variable is exactly `{ kind: "workspaceRoot" }` or + `{ kind: "repository", repository: ConfigName, + path?: RelativeDirectory }`; access is exactly `readOnly` or `readWrite`. + Unknown members, invalid relative paths, URLs, physical or configured + destination paths, credentials, commands, Docker options, materializer + choices, and provider permission policy fail before provider execution. The provider sends `SendMessage` with `configuration.returnImmediately: true`, captures the accepted Task ID, and calls `SubscribeToTask`; a terminal-before- @@ -560,78 +709,112 @@ registry, or another profile configuration file for the initial use case. - F1. **Start and advertise** 1. Resolve cwd or `--workspace`, user workspace, project-specific state root, - retention limits, listen address, advertised interface URL, source - credentials, and provider auth handles. - 2. Validate state-root ownership, permissions, links, disjointness, workspace - identity, compiled repository catalog, snapshots, target namespace, backend - availability, profile state, Linux containment/helper availability, - provider/MCP/tool mount, descriptor, and network views, and credential - handles. - 3. Enumerate the entire project-owned containment namespace. Reconcile - recorded and unknown identifiers and prove every set empty before - quarantining filesystem roots or releasing a retained execution lease. + disjoint immutable-base cache and invocation roots, cache/task retention, + workspace materializer, listener, advertised URL, digest-pinned acquisition + image, Docker endpoint, source credentials, provider homes, and configured + provider executable overrides. + 2. Validate the SQLite state root, cache/invocation roots, workspace identity, + materializer policy, and static acquisition-image reference; compile + repository, snapshot, and target catalogs; verify Codex SDK and Pi RPC/ + package compatibility; and check any global binary override exactly. Do not + contact Docker or the acquisition registry at startup. + 3. Reconcile interrupted Tasks by removing any recorded acquisition container + and orphan staging, terminating any recorded Linux provider process group, + and marking the Task failed without resuming it. Release the durable lease + only after container removal, confirmed process-group absence, and cleanup + of recorded Task-owned runtime, view, and non-reusable base; otherwise keep + readiness false and the lease poisoned. 4. Bind the requested address, including `0.0.0.0` when explicit; serve metadata-only health/readiness probes; and publish one Agent Card whose absolute interface URL, required extension, and target allowlist match the validated configuration. -- F2. **Acquire repositories and execute** +- F2. **Acquire or reuse repositories and execute** 1. Negotiate A2A version and the required extension, then validate the strict - request, one text Part, target, repository-name/revision map, result schema, - deadline, and deployment-wide idempotency claim. - 2. Ask the helper to allocate a stable empty containment set behind a start - gate. In one transaction, create or replay the claim and Task, acquire the - execution lease, and bind the containment identifier before acknowledgment. - Capacity failure settles the Task with `execution_capacity_unavailable`, - then destroys the empty set without releasing the gate or launching a - child. Commit failure likewise destroys the empty set. - 3. Release the start gate. For each declared repository, classify App - applicability, select App or `gh` only by eligibility, resolve the revision, - fetch hermetically, verify the commit, revoke an App token, and remove every - acquisition credential. - 4. Publish the complete workspace, run typed preparation, invoke the isolated - adapter, and validate any structured result while capturing live events. - Terminate and prove the containment set empty before safe filesystem/Git - evidence reads and produced-Artifact verification. Atomically settle the - terminal Task, evidence, Artifacts, cleanup, and lease release. - -- F3. **Acquire an OCI snapshot and execute** - 1. Perform the same version/extension validation, gated empty-containment - allocation, and atomic claim+Task+lease+containment commit as F2. - 2. Resolve the named snapshot repository and digest-pinned reference. - Authenticate if required; pull and verify the direct image manifest, - workspace-manifest config blob, and distributable layers; apply changesets - in order; enforce all limits; and validate the exact project catalog. - 3. Remove registry credentials, publish atomically, run typed preparation, - invoke the isolated adapter, and capture live events. Terminate and prove - quiescence before safe filesystem evidence and verified produced Artifacts, - then perform the same atomic settlement and lease release as F2. + request, one text Part, target, repository-name/revision map, logical + working-directory selector, workspace access, result schema, deadline, and + deployment-wide idempotency claim. + 2. In one SQLite transaction, create or replay the claim and Task and acquire + the execution lease before starting work. Capacity failure settles the Task + with `execution_capacity_unavailable` and launches neither Docker nor a + provider. + 3. When every effective revision is a full commit, derive the immutable-base + key and pin a matching validated cache entry. On a miss or for mutable + branch/tag revisions, classify App applicability and mint a fresh + installation token or resolve the configured `gh` token only according to + eligibility. Probe Docker and the exact digest-pinned acquisition image, + then start it with only private staging, compiled request/policy, and the + selected token. The container fetches hermetically, verifies full commits, + writes the typed manifest, exits, and is removed. + 4. On acquisition, revoke any App token, destroy source credentials, validate + the manifest/staging on the host, and atomically promote it to either a + reusable cache entry or a non-reusable Task-owned base. A cache hit performs + none of those acquisition, Docker, or credential operations. Every + non-publication path removes staging. + 5. For `readOnly`, resolve the logical cwd directly in the base and create only + Task-private runtime state. For `readWrite`, create the unique writable view + through the selected materializer, then resolve cwd in that view. Run + access-appropriate typed preparation and start the adapter there as a direct + host process group with explicit environment and existing host auth. + Validate structured results while capturing bounded live events. + 6. After the direct provider process settles and cancellation escalation + finishes, collect bounded truthful evidence; remove private runtime, any + writable view, and any non-reusable base; unpin a reusable base; and + atomically settle the Task, Artifacts, observed termination, cleanup, and + lease. If the process does not settle after `SIGKILL`, publish + `execution_termination_failed` without filesystem/Git evidence, retain + Task-owned state and the lease for runner teardown, make readiness false, + and stop admission until startup reconciliation confirms the process group + absent and cleans retained state. + +- F3. **Acquire or reuse an OCI snapshot and execute** + 1. Perform the same version/extension validation and atomic + claim+Task+lease transaction as F2. + 2. Pin a cache entry matching the named snapshot, manifest digest, workspace- + manifest digest, catalog/layout digest, and acquisition-contract version. + On a miss, probe Docker and the exact digest-pinned acquisition image, then + start it with the digest-pinned reference, staging mount, exact-host + registry credentials/CA material, and frozen network/archive policy. Pull + and verify the direct image manifest, workspace-manifest config, and + distributable layers; apply changesets in order; enforce all limits; and + emit the typed manifest. + 3. On a miss, remove the container and registry material, validate and publish + the immutable base on the host, or remove staging on every non-publication + path. Then select the read-only shared base or read-write Task view and + settle through the same bare-metal and cleanup path as F2. Provider + execution never occurs in the acquisition container. - F4. **Cancel** 1. Atomically persist cancellation intent if the Task remains cancelable. - 2. Abort acquisition or provider work, escalate within the bounded termination - budget, prove containment quiescence, preserve truthful partial evidence, - clean up, and settle canceled. + 2. For acquisition, stop and remove the Docker container, source material, and + unpublished staging. For provider work, request graceful adapter abort, + then escalate to process-group `SIGTERM` and `SIGKILL` within bounded + periods. Preserve only observed, bounded evidence; clean up and settle + canceled or failed according to the durable winning intent. 3. Repeated cancellation while intent is pending does not re-signal work. Cancellation after any terminal state returns A2A `TaskNotCancelableError`. - F5. **Shut down** 1. Stop new admission before signaling active work. - 2. Persist shutdown intent, abort and escalate, drain live process evidence, - prove quiescence, and settle the accepted Task once. - 3. Exit only after durable settlement and empty containment. If proof fails, - remain alive, not ready, and continue reaping while printing the stable - containment identifier and platform recovery command. + 2. Persist shutdown intent, remove active acquisition Docker work or escalate + the direct provider process group, collect evidence only after the direct + provider settles, and settle the accepted Task once. + 3. Exit after the bounded settlement and cleanup path. Document that CI runner + teardown is the final orphan boundary and that gateway shutdown does not + prove every model-tool descendant is gone. - F6. **Invoke from Promptfoo** 1. Promptfoo constructs the AI Evals-owned TypeScript provider with `ProviderOptions`; the provider retains the ID and validates - `options.config` containing the private-network endpoint, target, and one - closed source-mode object. + `options.config` containing the private-network endpoint, target, optional + default logical working directory, optional default workspace access, and + one closed source-mode object. 2. `callApi(prompt, context?, options?)` applies only valid - `context?.vars?.allagentsSource` leaf overrides, creates and retains one - high-entropy invocation key, and sends one A2A Message with + `context?.vars?.allagentsSource` leaves and optional strict + `context?.vars?.allagentsWorkingDirectory` and + `context?.vars?.allagentsWorkspaceAccess` replacements, creates and retains + one high-entropy invocation key, and sends one A2A Message with `configuration.returnImmediately: true`. 3. After receiving the Task ID, subscribe to terminal updates. Resolve a terminal-before-subscribe or disconnected-stream race through `GetTask` @@ -662,12 +845,14 @@ registry, or another profile configuration file for the initial use case. executed. - AE5. Repository mode accepts declared names and revision overrides, rejects an undeclared name or URL override, and records the resolved full commits. -- AE6. An applicable GitHub App bypasses its token cache, mints a new - repository-scoped read-only token with adequate lifetime, validates the token, - and revokes it after acquisition. A corroborated existing repository with no - applicable installation uses the configured `gh` account. An uncorroborated - 404, unknown applicability, auth, mint, validation, or revocation failure does - not fall through to `gh` or start the provider. +- AE6. On a base-acquisition miss, including a mutable branch/tag request, an + applicable GitHub App bypasses its token cache, mints a repository-scoped + read-only token with adequate lifetime, validates and revokes it after + acquisition. A cache hit resolves no source credential. + A corroborated existing repository with no applicable installation uses the + configured `gh` account. An uncorroborated 404, unknown applicability, auth, + mint, validation, or revocation failure does not fall through to `gh` or start + the provider. - AE7. Snapshot mode accepts a direct image manifest with matching manifest, config/workspace, and layer digests; applies gzip/zstd layers and whiteouts in order; and enforces every fixed limit. Same-origin metadata redirects work; @@ -677,22 +862,38 @@ registry, or another profile configuration file for the initial use case. foreign layers, digest/size mismatch, malformed whiteouts, undeclared repositories, redirect loops/rebinding, non-global destinations not declared for that source, and unapproved origins fail. -- AE8. Repository and snapshot modes produce the same workspace-manifest shape - and exact compiled repository set/layout. OCI-contained commit identities are - snapshot-attested unless independently verified; source identity includes - completeness and ordered layer digests without origins. +- AE8. Repository and snapshot modes produce the same path-free wire-visible + workspace-manifest shape and logical repository set. The gateway separately + validates the acquired base against the exact compiled private destinations. + OCI-contained commit identities are snapshot-attested unless independently + verified; source identity includes completeness and ordered layer digests + without origins. One hundred Tasks using the same immutable identity perform + one full acquisition while the entry remains cached and pinned correctly. + A branch or tag request acquires a non-reusable Task-owned base, never enters + the reusable cache, and removes that base during settlement or reconciliation. - AE9. Identical invocation-key replay, including after a lost response, returns - the original Task. Reusing the key with a changed target, source, prompt, or - result schema conflicts; separate high-entropy keys create separate Tasks. -- AE10. Cancellation during Git, OCI pull, Codex, or Pi terminates the complete - process set and records cleanup. Unproved quiescence poisons readiness; the - gateway stays alive, rejects admission, and continues reaping until empty. -- AE11. Kill fixtures before and after empty-containment creation, durable - Task/lease/containment binding, child clone, start-gate release, and response - acknowledgment leave no unrecorded live set. Restart enumerates the full - project-owned namespace, refuses unknown/nonempty sets, turns interrupted - Tasks into one terminal failure, never resumes a provider session, and keeps - terminal Tasks and embedded Artifacts retrievable until expiry. + the original Task. Reusing the key with changed target, source, prompt, logical + working directory, workspace access, or result schema conflicts. Separate + read-only Tasks may share one physical base and cwd while keeping private + runtime state. Separate read-write Tasks receive independent writable views + even when their logical working-directory selectors are equal. +- AE10. Cancellation during a Git or OCI base acquisition stops and removes the + acquisition container and unpublished staging. Cancellation during Codex or + Pi requests graceful abort, then sends process-group `SIGTERM` and `SIGKILL` + on schedule. + The Task records observed termination and cleanup without claiming complete + descendant quiescence. If the direct process does not settle, the gateway + retains the Task's runtime and any writable view plus the lease, omits + filesystem/Git evidence, stops admission, and remains unready until post- + teardown startup reconciliation confirms the group absent. +- AE11. Kill fixtures before and after durable Task/lease creation, acquisition- + container start, provider process-group recording, and response + acknowledgment leave one recoverable SQLite truth. Restart removes the + recorded acquisition container, orphan staging, and safe Task-owned state; + best-effort terminates the recorded process group; and turns the interrupted + Task into one terminal failure without resuming a provider session. It retains + the lease and stays unready unless provider absence and required cleanup are + confirmed. Terminal Tasks and Artifacts remain until expiry. - AE12. A valid structured result survives later check or evidence failure as a valid result with an overall failed Task; invalid or absent results are never published as valid. @@ -701,56 +902,63 @@ registry, or another profile configuration file for the initial use case. target. - AE14. Two gateways for different workspaces use distinct private state roots; a second process for the same root fails the exclusive lock. Wrong-owner, - permissive, linked, hard-linked, or overlapping roots fail startup. The real - helper VFS rejects database, WAL, SHM, journal, temporary-file, symlink, - hard-link, and rename-swap attacks. Process-kill fixtures at transaction, file - sync, directory sync, and response boundaries recover either the complete old - or new generation and never lose an acknowledged Task or publish false - success. + permissive, linked, or overlapping roots fail startup. Ordinary Bun SQLite + transactions with foreign keys and `synchronous=FULL` recover a committed + Task/claim/lease generation after process-kill fixtures and never acknowledge + an uncommitted Task or publish false success; no custom VFS is required. - AE15. Barrier-controlled provider-terminal, caller-cancel, deadline, and - shutdown races durably select one internal intent and one abort/quiescence - path during Git, OCI, preparation, Codex, Pi, or evidence. Subscribers observe - no terminal Task until one transaction writes status, integrity Artifact, - bounded evidence, result/failure, termination, cleanup, and lease release. - Later reaping changes only internal readiness/recovery state. + shutdown races durably select one internal intent and one abort path during + Docker acquisition, preparation, Codex, Pi, or evidence. Normal settlement + writes status, integrity Artifact, bounded evidence, result/failure, observed + termination, cleanup, and lease release in one transaction. The + `execution_termination_failed` exception writes the terminal failure without + filesystem/Git evidence and deliberately retains the poisoned lease. - AE16. Repeated cancel while cancellation is pending is idempotent; cancel after canceled, completed, failed, or rejected returns `TaskNotCancelableError`. -- AE17. A workspace containing `setup` shell entries never executes them through - gateway acquisition or startup. Real Codex/Pi child and grandchild tool paths - are helper-mediated: filesystem, environment, inherited descriptor, `/proc`, - and magic-link probes cannot read provider/MCP secrets, operator stores, or - gateway state. Agent Card, Task operations, host loopback, bind/advertised - addresses, ingress, and management-network probes fail from every invocation - role; only compiled role egress succeeds. -- AE18. Evidence is collected only after containment quiescence. An escaping - link, hard link, special file, sparse-file abuse, replaced inode, or `.git` - indirection is rejected and Git inspection runs without repository-controlled - execution. Unknown quiescence produces no verified filesystem Artifact. +- AE17. A workspace containing `setup` shell entries never executes them during + acquisition or startup. The acquisition image receives only staging, + source-only credentials, exact source network policy, and archive limits; it + receives no host home, Docker socket, gateway state, provider auth, Codex, Pi, + or coding harness. Codex and Pi run afterward as direct host processes with + explicit environments that preserve required host identity/auth paths and + omit unrelated ambient values. +- AE18. Evidence collection starts only after the direct provider process has + settled and process-group escalation has completed. Git inspection disables + repository-controlled execution, and the integrity Artifact distinguishes + observed direct-process termination and cleanup from full descendant + quiescence. Documentation explicitly states that AllAgents provides no + hostile-code or model-tool secret-isolation guarantee. - AE19. The 1001st unexpired retained Task is rejected with `retention_capacity_exhausted`; no retained Task is evicted before TTL. While one Task holds the execution lease, a barrier-controlled second request - settles `execution_capacity_unavailable` and launches no helper child; races - and restart never produce two lease holders. + settles `execution_capacity_unavailable` and launches no acquisition or + provider child; races and restart never produce two lease holders. - AE20. Official HTTP+JSON client fixtures send `A2A-Version: 1.0`, exercise required-extension activation and both `SendMessage` modes, preserve unrelated metadata, verify standard `google.rpc.Status` errors, and cover every `ListTasks` filter, cursor, order, response field, and artifact-inclusion rule. - A terminal Task contains one extension-marked integrity Artifact plus - referenced produced Artifacts using unified Parts. + A terminal Task contains one extension-marked integrity Artifact with its + effective logical working directory, workspace access, and referenced + produced Artifacts using unified Parts. - AE21. The AI Evals Promptfoo fixture has a top-level prompt and disables sharing, caching, result writes, and concurrency above one. It loads one repository-mode and one snapshot-mode provider, sends only closed logical - source data, retains one invocation key across ambiguous retries, and cancels - an accepted Task on abort. Both calls return scorable output, normalized token + source, working-directory, and workspace-access data, replaces cwd and access + per trial through `allagentsWorkingDirectory` and + `allagentsWorkspaceAccess`, retains one invocation key across ambiguous + retries, and cancels an accepted Task on abort. Two read-only trials for the + same immutable source share the validated base; two read-write trials receive + independent disposable views. Both return scorable output, normalized token usage, and Task/Artifact/logical-provenance metadata. Safe failure metadata includes code, retryability, and accepted Task ID. Calls with omitted context - work; unknown variables, mutable revisions, origins, destinations, or - undeclared names fail before submission. + work; unknown variables, invalid or escaping relative directories, physical + paths, mutable revisions, origins, destinations, materializer choices, or + undeclared names fail before provider execution. ### Success Criteria -- `allagents gateway serve` starts from a real workspace with no deployment YAML. +- `allagents-gateway serve` starts from a real workspace with no deployment YAML. - Explicit loopback, private-interface, and `0.0.0.0` listeners work with a distinct valid advertised interface URL; health/readiness reflect admission. - The official A2A client exercises version and extension negotiation, both send @@ -758,33 +966,49 @@ registry, or another profile configuration file for the initial use case. cancel, terminal cancel errors, Task-embedded Artifacts, standard HTTP+JSON errors, and expiry. - An AI Evals-style Promptfoo custom-provider fixture consumes secure-default - YAML for both source modes, propagates post-acceptance cancellation, and maps a + YAML for both source modes, selects a logical cwd and access mode per trial, + proves shared-base reuse for read-only trials and independent disposable views + for read-write trials, propagates post-acceptance cancellation, and maps a terminal Task to `ProviderResponse` without adding Promptfoo to the AllAgents runtime. -- Built-in Codex/Pi and gateway-enabled profile targets pass one conformance - suite, including reserved-ID collisions and Codex native-schema gating. -- Direct Git and OCI snapshot fixtures produce equivalent validated workspace - manifests and truthful complete provenance. -- GitHub App eligibility, 404 ambiguity, unknown failure, no-installation `gh` - fallback, fresh-token validation/revocation, OCI authentication and challenge - handling, and pre-provider credential teardown are proven end to end. -- No request can supply a command, executable, URL, destination, credential, - mutable OCI tag, backend override, or arbitrary environment value. -- State-store crash, deadline/cancellation/terminal/shutdown race, descendant - escape, unsafe evidence, and stale-root scenarios fail closed. -- The bundled CLI and packaged Linux helper pass a trusted-network smoke test - against project and user workspaces created under `/tmp/`. +- Built-in Codex/Pi and gateway-enabled profile targets pass one backend + conformance suite, including reserved-ID collisions, existing-host-auth + behavior, explicit environment construction, cancellation escalation, and + Codex native-schema gating. +- Direct Git and OCI snapshot acquisition in the exact digest-pinned image + produces equivalent typed manifests and truthful provenance; repeated + immutable requests reuse one validated base and the image is removed before + provider execution. GitHub App eligibility, 404 ambiguity, unknown failure, + no-installation `gh` fallback, base-acquisition token validation/revocation, OCI + authentication/challenge handling, staging validation, and pre-provider + credential teardown are proven end to end. +- No request can supply a command, executable, URL, physical cwd, configured + destination, credential, mutable OCI tag, backend or materializer override, + arbitrary environment value, Docker option, image reference, or mount. +- SQLite crash/race/restart, acquisition-container cleanup, host provider + process-group cancellation, and truthful post-settlement evidence scenarios + pass without claiming complete descendant containment. +- The independently packaged Bun gateway passes a trusted-network smoke against + project and user workspaces under `/tmp/`; a CLI-only install fetches neither + the gateway package nor the acquisition image. ### Scope Boundaries **In scope** - A2A 1.0 HTTP+JSON and the required AllAgents extension. -- One process and one active invocation at a time initially. -- Built-in and gateway-enabled profile-backed Codex/Pi targets. -- Direct declared Git repositories and named OCI workspace snapshots. +- One gateway process and one active invocation at a time initially. +- Built-in and gateway-enabled profile-backed Codex/Pi host execution. +- Docker-only acquisition of direct declared Git repositories and named OCI + workspace snapshots when no reusable validated base exists. +- Reusable immutable bases and Task-private runtime state for read-only + execution; non-reusable Task-owned bases for mutable revisions; unique + disposable writable views for read-write execution. +- Logical workspace-root or declared-repository-relative provider cwd plus + explicit `readOnly | readWrite` access selected at runtime. - GitHub App and configured GitHub CLI acquisition credentials. -- Local durable Task/evidence storage, cancellation, cleanup, and provenance. +- Local durable Task/evidence storage, bounded base caching, process-group + cancellation, materialization cleanup, and provenance. - Listen addresses including `0.0.0.0`. **Out of scope** @@ -793,13 +1017,17 @@ registry, or another profile configuration file for the initial use case. Internet hardening. - `gateway.yaml`, `worker.yaml`, remote workers, mTLS worker links, Kubernetes routing, autoscaling, and multiple gateway replicas. -- Caller-provided repository or registry origins, mutable OCI tags, custom - materializers, Dockerfiles, Compose files, or acquisition commands. +- Caller-provided physical workspaces/cwds, repository or registry origins, + mutable OCI tags, custom materializers, Dockerfiles, Compose files, or + acquisition commands. - GitHub Enterprise Server and multiple ordered Apps/accounts in the initial delivery. - OpenCode, Claude, Copilot, OMP, arbitrary CLI, and TUI adapters. - Evaluation orchestration and automatic retries. -- Non-Linux gateway execution in v1; ordinary AllAgents CLI behavior remains +- Per-provider containers; cgroups, pidfds, namespaces, nftables, `openat2`, a + native platform layer, non-bypassable spawn mediation, hostile-code + containment, and secret isolation from model-invoked tools. +- Non-Linux gateway execution in v1; ordinary `allagents` CLI behavior remains cross-platform. ### Sources @@ -810,6 +1038,9 @@ registry, or another profile configuration file for the initial use case. - [Source credential broker precedents](../research/source-credential-broker-precedents.md) - [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) - [A2A extension guide](https://a2a-protocol.org/latest/topics/extensions/) +- [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) +- [Bun workspaces](https://bun.sh/docs/install/workspaces) +- [Bun SQLite](https://bun.sh/docs/api/sqlite) - [Promptfoo custom providers](https://www.promptfoo.dev/docs/providers/custom-api/) - [Promptfoo configuration reference](https://github.com/promptfoo/promptfoo/blob/main/site/docs/configuration/reference.md) - [OpenAI Codex SDK](https://developers.openai.com/codex/sdk/) @@ -817,10 +1048,15 @@ registry, or another profile configuration file for the initial use case. - [GitHub App installation tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) - [Git credential helpers](https://git-scm.com/docs/gitcredentials) - [Docker credential stores](https://docs.docker.com/reference/cli/docker/login/#credential-stores) -- [Node.js SQLite API](https://nodejs.org/docs/latest-v22.x/api/sqlite.html) - [OCI Image Specification](https://github.com/opencontainers/image-spec) - [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec) -- [Linux cgroup v2](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html) +- [GitHub Container registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry) +- [JFrog Artifactory Docker repositories](https://jfrog.com/help/r/jfrog-artifactory-documentation/docker-repositories) +- [JFrog Container Registry image](https://hub.docker.com/r/jfrog/artifactory-jcr) +- [Docker OverlayFS storage driver](https://docs.docker.com/engine/storage/drivers/overlayfs-driver/) +- [Docker VFS copy fallback](https://docs.docker.com/engine/storage/drivers/vfs-driver/) +- [Windows ReFS block cloning](https://learn.microsoft.com/en-us/windows-server/storage/refs/block-cloning) +- [GitHub-hosted runners](https://docs.github.com/en/actions/reference/runners/github-hosted-runners) --- @@ -828,22 +1064,30 @@ registry, or another profile configuration file for the initial use case. ### Key Technical Decisions -- KTD1. **Use the official A2A JavaScript SDK transport around an - AllAgents-owned request handler.** Do not use `DefaultRequestHandler` or its - non-transactional `TaskStore` seam. Implement the SDK's request-handler - interface so AllAgents controls UUIDv7 creation, atomic `createOrReplay`, - monotonic settlement, listing, retention, expiry, and HTTP+JSON error details - while retaining standard Task/Artifact carriers. -- KTD2. **Generate and publish the extension and storage contracts from canonical - Zod schemas.** The versioned extension specification at its declared URI - defines Agent Card params, activation, Message metadata/extensions, request, - Task/Artifact, idempotency, error, replay, examples, and versioning. Generate - public JSON Schemas, Task-store, workspace-manifest, and adapter types from the - same source. Keep backend-specific fields private. -- KTD3. **Use a single-process supervisor, not a remote worker protocol.** One - service owns Task state, staging, publication, backend child processes, - evidence, termination, and cleanup. Child processes remain contained behind - an invocation lifecycle boundary. +- KTD1. **Gate the official A2A JavaScript SDK in the shipped Bun server + direction before adopting it.** Pin the exact SDK version and prove Agent Card + discovery, both send modes, streaming, Task get/list/cancel, resubscription, + extension negotiation, metadata preservation, and HTTP error envelopes by + driving the production gateway server with an independent official client + fixture. Implement the SDK's public request-handler seam while AllAgents owns + UUIDv7 creation, atomic `createOrReplay`, monotonic settlement, listing, + retention, expiry, and HTTP+JSON error details. Do not use an SDK default store + as the transaction boundary or replace A2A with a bespoke protocol. +- KTD2. **Keep contracts portable and generated from narrow TypeScript + packages.** `packages/workspace-config` owns project/user parsing and compiled + catalogs; `packages/execution-contracts` owns A2A extension, request, + Task/Artifact, idempotency, result, error, adapter, and evidence schemas; + `packages/acquisition-contracts` owns acquisition requests, typed manifests, + OCI snapshot rules, and fixed limits. Check generated JSON Schemas and golden + accepted/rejected examples into `contracts/` for the host gateway, acquirer + image, public docs, and consumer fixtures. Do not create `core`, `common`, or + a speculative shared package. +- KTD3. **Use one gateway supervisor, not a remote worker protocol.** The Bun + gateway owns Task state, immutable-base caching, Task runtime/view + materialization, provider child processes, evidence, termination, and cleanup. + It creates an ephemeral Docker container only when no reusable validated base + exists, removes it before provider execution, and launches Codex/Pi directly + on the trusted host in Linux process groups. - KTD4. **Make application authentication intentionally absent.** All Tasks and Artifacts share one deployment namespace. The listener accepts explicit `0.0.0.0`; network controls are external. Bind and advertised interface URL @@ -853,88 +1097,161 @@ registry, or another profile configuration file for the initial use case. profile-client schemas. A gateway-only compiler normalizes the project repository catalog and resolves each public launcher ID to one profile/client. Add no deployment YAML. (session-settled: user-directed.) -- KTD6. **Keep source input name-based and closed.** Repository requests carry - only declared-name revisions; snapshot requests carry only a declared snapshot - name and immutable digests. Compute one canonical source identity for - idempotency and provenance. -- KTD7. **Freeze direct Git and OCI acquisition profiles.** Git runs with - hermetic config and an invocation credential helper. An AllAgents-owned - minimal OCI Distribution client in the Rust helper uses pinned `reqwest` - (rustls, redirects and ambient proxies disabled), `tar`, `flate2`, and `zstd` - crates for streaming pull, bounded authentication, decoding, and changeset - application behind the helper's typed protocol. The client implements only the - v1 direct-image manifest/config/layer profile, RFC 8785 workspace-manifest - config, fixed extraction limits, and explicit Distribution-Spec authentication - and redirect policy. A test-only deterministic reference packer produces the - conformance fixture that freezes the format. -- KTD8. **Select GitHub credentials by provable three-way eligibility.** App - lookup 200 is eligible; 404 is ineligible only with independent repository- - existence proof; all ambiguous outcomes are unknown. Fresh App tokens bypass - SDK cache, are validated and revoked, and only positive ineligibility permits - the configured `gh` account. (session-settled: user-directed.) +- KTD6. **Keep source, cwd, and access input logical and closed.** Repository + requests carry only declared-name revisions; snapshot requests carry only a + declared snapshot name and immutable digests. Working-directory requests + select only the effective workspace root or a declared repository plus a + bounded relative directory. `workspaceAccess` is exactly `readOnly` or + `readWrite`. Read-only Tasks may share the immutable base; read-write Tasks + receive Task-ID-derived views. The gateway never accepts or returns a caller + path or materializer choice. Include canonical source identity, logical cwd, + and access in idempotency and provenance. +- KTD7. **Freeze Docker-only base acquisition.** Build `apps/acquirer` once as a + multi-architecture GHCR image and select it by manifest digest. Each request + without a reusable validated base starts a fresh container with one staging + bind mount, a typed acquisition request, source-only credentials, strict + source network policy, fixed size/archive limits, no host home, and no Docker + socket. + The image owns hermetic Git plus the minimal OCI Distribution client + and implements only the v1 direct-image manifest/config/layer profile, + RFC 8785 workspace-manifest config, explicit authentication/redirect rules, + streaming digest checks, and changeset application. It emits a typed manifest, + exits, and is removed before host validation/publication. It contains and + downloads no Codex, Pi, or other coding harness. A deterministic producer + fixture freezes the format. +- KTD8. **Select GitHub credentials by provable three-way eligibility.** On a + base-acquisition miss, App lookup 200 is eligible; 404 is ineligible only with + independent repository-existence proof; all ambiguous outcomes are unknown. + Fresh App tokens bypass credential cache, are validated and revoked, and only + positive ineligibility permits the configured `gh` account. The selected token + enters only the acquisition container; base-cache hits resolve no credential. + (session-settled: user-directed.) - KTD9. **Keep one behavior-focused `codex | pi` adapter registry.** Direct - targets and gateway-enabled profile targets resolve to the same adapter types - and conformance tests; profile context modifies server-owned configuration, - never the public command line. A target is ready only when its pinned backend - exposes a non-bypassable spawn hook through which the helper launches every - MCP and model-tool process with separate filesystem, environment, descriptor, - secret, and network views. -- KTD10. **Put durable Task truth behind the Rust helper's SQLite VFS.** The - helper owns the single process-lifetime SQLite connection and exposes typed - transactional store operations; TypeScript never opens the database by path. - A small audited VFS roots every database, WAL, SHM, journal, and temporary-file - open beneath a preopened private state-directory descriptor with `openat2` - beneath/no-symlink checks, rejects hard links, and fsyncs files and containing - directories. SQLite uses WAL, foreign keys, and `synchronous=FULL`. Claims, - Tasks, events, bounded Artifact bytes, execution lease, containment identity, - internal outcome intent, and expiry live in transactional tables. - `createOrReplay`, lease acquisition, and terminal settlement are transactions; - acknowledge only committed state. Crash recovery yields a complete old or new - generation, never a mixed or missing acknowledged Task. Integrity, VFS, helper - protocol, or durability failure stops admission and prevents false success. -- KTD11. **Package one enforceable Linux security and state helper.** V1 supports - Linux x64/arm64 with cgroup v2, `clone3(CLONE_INTO_CGROUP)`, pidfds, `openat2` - beneath/no-symlink resolution, mount and network namespaces, and nftables - through a small audited Rust helper distributed in platform-specific optional - packages. Its typed inherited-pipe protocol owns SQLite operations, creates - empty containment behind a durable start gate, atomically launches and tracks - the complete acquisition/provider descendant set, mediates every MCP/tool - spawn, builds role-specific filesystem/environment/descriptor/network views, - terminates and waits for membership, and performs safe file operations. - Missing kernel features, delegated cgroup/network access, helper package, - backend spawn mediation, or protocol compatibility fails before binding; - there is no weaker fallback. A poisoned process remains alive to reap until - the set is empty. -- KTD12. **Capture live events, then collect durable filesystem evidence only - after quiescence.** Evidence retains bounded source, Git, provider, result, - Artifact, and cleanup facts. Post-execution workspace reads use the helper's - descriptor-relative no-follow handles, revalidate identity/size, and reject - Git metadata indirections or repository-controlled execution. Structured logs - remain metadata-only and never retain secrets or unrestricted - request/output/file bodies. + targets and gateway-enabled profile targets resolve to the same narrow + AllAgents-owned TypeScript adapter contract and conformance suite; profile + context modifies server-owned configuration, never public argv. Codex uses + pinned `@openai/codex-sdk` first; app-server is allowed only for a proven + required SDK gap. Pi uses a pinned supported package/RPC surface. Neither + adapter downloads runtimes per request or adopts AI SDK Harnesses. A global + binary override requires an exact compatibility probe. +- KTD10. **Keep durable Task truth inside ordinary Bun SQLite ownership.** The + gateway holds the process-lifetime `bun:sqlite` connection, private state + root, and exclusive lock. SQLite uses foreign keys, transactional + `createOrReplay`/lease/settlement/expiry operations, WAL where supported, and + `synchronous=FULL`; acknowledge only committed state. Claims, Tasks, events, + bounded Artifact bytes, execution lease, acquisition-container/staging/ + transient-base identity, provider process-group identity, internal outcome + intent, and expiry live in tables. Startup integrity or durability failure + stops admission and prevents false success. Do not build a custom VFS or + native file layer. +- KTD11. **Treat the trusted Linux CI job as the provider isolation boundary.** + The gateway uses Docker only when no reusable validated base exists. Codex and + Pi run bare metal with the same CI-job authority as the gateway and existing + host + auth. Read-only is a consumer-selected cooperative contract with private + runtime state, optional-lock suppression, and native provider policy where + available; it is not hostile-code containment. + Construct provider environments explicitly to preserve required identity/auth + paths while omitting unrelated ambient values, but do not claim this protects + secrets from model-invoked tools. Linux cancellation is adapter abort, then + process-group `SIGTERM`, then `SIGKILL`; runner teardown is the final orphan + boundary. Do not add cgroups, pidfds, `openat2`, namespaces, nftables, native + containment packages, per-provider Docker, or spawn mediation. +- KTD12. **Capture live events, then collect bounded evidence after the direct + provider settles.** Evidence retains bounded source, Git, provider, result, + Artifact, observed termination, and cleanup facts. Git inspection disables + repository-controlled execution. Evidence and docs must not turn process- + group termination into a claim that all descendants are quiescent or that + model-tool output is redacted. +- KTD13. **Use one private Bun workspace without coupling releases.** The root + package is private orchestration. `apps/cli` publishes `allagents`; + `apps/gateway` publishes `allagents-gateway`; `apps/acquirer` is never + published to npm and ships only as a digest-pinned multi-architecture GHCR + image. Shared packages are limited to `packages/workspace-config`, + `packages/execution-contracts`, and `packages/acquisition-contracts`; + generated portable fixtures live under `contracts/`. + + CLI and gateway have independent versions, tags, changelogs, triggers, npm + tarballs, and release jobs. A CLI-only install resolves neither the gateway nor + the acquisition image. A gateway release first builds the acquisition image + once for the exact commit, resolves and records its multi-architecture + manifest plus supported platform digests, runs package and registry checks + against those exact immutable artifacts, and only then publishes the exact + `allagents-gateway` npm tarball. A gateway-only release never publishes + `allagents`; no Rust, Cargo, native binary, or platform npm package exists. +- KTD14. **Use tiered OCI registry conformance bound to exact release + artifacts.** Every pull request runs a local Distribution fixture and a live + public digest-pinned GHCR snapshot pull through the exact acquirer image. A + reusable release workflow adds authenticated least-privilege GHCR and pinned + private-CA JFrog Artifactory/JCR coverage. + + The callable workflow receives the exact gateway npm tarball, acquisition + multi-architecture manifest digest, per-platform image digests where the + registry supports them, build commit, and expected compatibility output; it + never rebuilds either artifact. Reports record the tested commit, npm tarball + digest, acquisition manifest/platform digests, architecture, image/registry + identity, auth mode, snapshot descriptor digests, and compatibility output, + including partial evidence on red paths. They cover valid anonymous and + authenticated pulls plus wrong credentials, insufficient permissions, digest + mismatch, missing/wrong CA, invalid media, and repository-path failures. The + gateway release must verify GHCR and JFrog against those exact artifacts + before npm publication; the JFrog target need not run on every pull request. + +### Package compatibility contract + +`allagents-gateway compatibility --format json` emits one strict, versioned +object containing `product: "allagents-gateway"`, `gatewayVersion`, +`buildCommit`, `runtime: "bun"`, the pinned acquisition image repository and +multi-architecture manifest digest, supported acquisition platforms/digests, +and supported A2A, coding-extension, workspace, execution-contract, acquisition- +contract, and snapshot versions. The packed npm tarball, clean-install smoke, +registry workflow, and release workflow consume this same object. + +An optional `allagents gateway ...` dispatcher locates but never installs the +separate gateway. It accepts independent CLI and gateway versions only when the +product identity and required contract-version ranges intersect; otherwise it +prints a clear install/upgrade error and does not start the service. The gateway +rejects an acquisition image whose manifest digest, platform digest, build +identity, or acquisition-contract version differs from its release metadata. +Golden fixtures cover exact matches, supported CLI/gateway version skew, +unsupported contract versions, wrong image manifests/platforms, divergent npm +tarball or image build identities, and newest/oldest supported pairs. There are +no platform npm packages or native-binary compatibility checks. ### High-Level Technical Design ```mermaid flowchart TB - C[Trusted-network A2A caller] --> G[Gateway server] - G --> H[Linux security and state helper] - H --> S[SQLite Task store] - G --> W[Workspace compiler] + C[Trusted-network A2A caller] --> G[Bun gateway host process] + G --> S[Bun SQLite Task store] + G --> W[workspace-config compiler] W --> PW[Project workspace.yaml] W --> UW[User workspace.yaml] - G --> A[Acquisition supervisor] + G --> BL[Reusable immutable-base lookup] + BL -->|hit| RB[Validated reusable base and pin] + BL -->|miss or mutable revision| D[Docker acquisition coordinator] + D --> A[Digest-pinned acquirer container] A --> Git[Declared Git repositories] A --> OCI[Named OCI snapshot] - A --> H - A --> P[Atomically published invocation workspace] - G --> R[Closed adapter registry] - R --> Codex[Codex SDK] - R --> Pi[Pi RPC] - Codex --> H - Pi --> H - H --> E[Quiescence then evidence and cleanup] - E --> S + A --> ST[Staging plus typed manifest] + ST --> V[Host validation and atomic base promotion] + V -->|exact identity| RB + V -->|mutable revision| TB[Task-owned transient base] + RB --> RO[Read-only base plus private runtime] + TB --> RO + RB --> M[Block clone or rootless OverlayFS or copy] + TB --> M + M --> RW[Task-owned writable view] + RO --> WD[Logical cwd resolver] + RW --> WD + WD --> R[Closed host adapter registry] + R --> Codex[Pinned Codex SDK] + R --> Pi[Pinned Pi RPC/package] + Codex --> PG[Linux provider process group] + Pi --> PG + PG --> E[Direct-process settlement then bounded evidence] + E --> C[Remove Task runtime, view, and transient base] + C --> S ``` ### Configuration Contract @@ -949,88 +1266,117 @@ No `gateway.yaml` or `worker.yaml` is introduced. | Advertised interface URL | `--advertise-url` | `ALLAGENTS_GATEWAY_ADVERTISE_URL` | `http://127.0.0.1:4732` only with the default loopback listener; otherwise required | | Project workspace | `--workspace` | `ALLAGENTS_GATEWAY_WORKSPACE` | cwd | | State directory | `--state-dir` | `ALLAGENTS_GATEWAY_STATE_DIR` | `~/.allagents/gateway/` | +| Invocation workspace root | `--invocation-root` | `ALLAGENTS_GATEWAY_INVOCATION_ROOT` | `~/.allagents/gateway-workspaces/` | +| Immutable-base cache root | `--base-cache-dir` | `ALLAGENTS_GATEWAY_BASE_CACHE_DIR` | `~/.allagents/gateway-cache/` | +| Immutable-base cache budget | `--base-cache-max-bytes` | `ALLAGENTS_GATEWAY_BASE_CACHE_MAX_BYTES` | `64GiB` | +| Workspace materializer | `--workspace-materializer` | `ALLAGENTS_GATEWAY_WORKSPACE_MATERIALIZER` | `auto` (`auto | cow | copy`) | +| Automatic copy ceiling | `--max-auto-copy-bytes` | `ALLAGENTS_GATEWAY_MAX_AUTO_COPY_BYTES` | `1GiB` | | Terminal Task TTL | `--task-ttl` | `ALLAGENTS_GATEWAY_TASK_TTL` | `24h` | | Retained Task limit | `--max-retained-tasks` | `ALLAGENTS_GATEWAY_MAX_RETAINED_TASKS` | `1000` | | Per-Task retained bytes | `--max-task-bytes` | `ALLAGENTS_GATEWAY_MAX_TASK_BYTES` | `64MiB` | +| Acquisition image | `--acquisition-image` | `ALLAGENTS_GATEWAY_ACQUISITION_IMAGE` | release-embedded `ghcr.io/.../allagents-acquirer@sha256:` | +| Docker endpoint | `--docker-host` | `ALLAGENTS_GATEWAY_DOCKER_HOST` | existing local Docker context/socket | +| Docker acquisition network | `--acquisition-network` | `ALLAGENTS_GATEWAY_ACQUISITION_NETWORK` | release-documented acquisition-only network | +| Acquisition timeout | `--acquisition-timeout` | `ALLAGENTS_GATEWAY_ACQUISITION_TIMEOUT` | `900s`, capped by remaining Task deadline | | GitHub App ID | `--github-app-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_ID` | unset | | App private key file | `--github-app-private-key-file` | `ALLAGENTS_GATEWAY_GITHUB_APP_PRIVATE_KEY_FILE` | unset | | App installation ID | `--github-app-installation-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_INSTALLATION_ID` | discovered/unset | | GitHub CLI account | `--github-cli-account` | `ALLAGENTS_GATEWAY_GITHUB_CLI_ACCOUNT` | unset | | OCI auth file | `--oci-auth-file` | `ALLAGENTS_GATEWAY_OCI_AUTH_FILE` | unset | | OCI credential helper | `--oci-credential-helper` | `ALLAGENTS_GATEWAY_OCI_CREDENTIAL_HELPER` | unset | -| Codex auth file | `--codex-auth-file` | `ALLAGENTS_GATEWAY_CODEX_AUTH_FILE` | supported Codex default if safe | -| Pi auth file | `--pi-auth-file` | `ALLAGENTS_GATEWAY_PI_AUTH_FILE` | supported Pi default if safe | - -Precedence is CLI over environment over default. The advertised value is the -absolute URL placed in `AgentCard.supportedInterfaces`; wildcard hosts are -invalid, non-loopback listeners require an explicit value, and production uses -HTTPS. Credential options name file handles, accounts, or IDs, never secret -values. - -The Linux helper resolves every key/auth/helper path from a verified root with -`openat2(RESOLVE_BENEATH | RESOLVE_NO_SYMLINKS | RESOLVE_NO_MAGICLINKS)`, rejects -group/world-writable parent directories and linked or non-regular leaves, opens -with close-on-exec/no-follow, and verifies owner, mode, link count, device, and -inode with `fstat` after open. Consumers read the verified descriptor rather than -reopening the path. The helper executes a credential-helper binary from that -verified inode; a path or inode swap fails. Auth leaves are current-user/root- -owned, have one link, and are no broader than `0600`; helper leaves are -current-user/root-owned and not group/world-writable. - -Setting both OCI options is a startup error. `--oci-auth-file` accepts at most -1 MiB of strict UTF-8 Docker-config JSON containing only `auths`. Each key is the -exact registry lookup key below and each strict entry contains exactly one of: -bounded base64 `auth` decoding to `username:secret`, or bounded nonempty -`identitytoken`. `credsStore`, `credHelpers`, proxy/plugin fields, unknown -members, commands, and duplicate keys are rejected; nothing named by the file -is executed. Credential selection is exact-key only. - -The fixed OCI helper receives argv `[helperPath, "get"]` without a shell. Stdin -is the raw Docker lookup key plus newline: lowercase `host[:nondefault-port]` -except Docker Hub, which uses `https://index.docker.io/v1/`. Exit-zero stdout is -one UTF-8 JSON object with required nonempty `Username` and `Secret` strings and -optional `ServerURL`, each at most 64 KiB. `ServerURL`, when present, must equal -the lookup key; `Username: ""` classifies `Secret` as an identity token. -Stdout over 128 KiB, timeout, nonzero exit, signal, malformed UTF-8/JSON, unknown -member, mismatch, or empty credential fails with `source_auth_oci_failed`. -Stderr is bounded, treated as secret-bearing, and never logged or retained. - -Registry access starts anonymously. Accept at most one well-formed HTTPS Bearer -challenge and one authenticated retry per request, with one token refresh after -an in-budget 401. Scope must exactly equal -`repository::pull`; service is bounded, -passed only as data, and must match the registry service -(`registry.docker.io` for Docker Hub). -Credentialed token exchange is allowed only at a same-origin HTTPS realm -or the exact Docker Hub realm `https://auth.docker.io/token`; other realms are -anonymous-only. Do not request offline access or accept refresh tokens. Validate -token type and bounded expiry. - -Redirect handling is manual and limited to three HTTPS hops. Same-origin -redirects are permitted. A cross-origin redirect is permitted only for a -layer-blob `GET`/`HEAD` when the destination's normalized `host[:port]` exactly -matches that snapshot source's `layerRedirectHosts`; token, manifest, and config -requests reject it. Every hop rejects URL credentials, strips authorization, -cookies, and client credentials, resolves DNS afresh, validates every A/AAAA -address, and connects to a validated address with the original hostname used -for Host/SNI. Loopback, link-local, multicast, unspecified, RFC1918, ULA, CGNAT, -and other non-global destinations are rejected unless that exact host is -operator-approved for the source. Redirect loops, downgrade, mixed approved and -unapproved answers, and rebinding fail. Final descriptor bytes still must match -size and digest. - -Credentials are invoked once per registry lookup key, scoped to that origin and -repository pull, zeroed after use, and destroyed before publication. - -Provider defaults are eligible only when their resolved auth files pass the same -descriptor checks; otherwise the target is not ready. The gateway projects only -the selected provider auth into its control-process view. +| OCI CA bundle map | `--oci-ca-bundle-map` | `ALLAGENTS_GATEWAY_OCI_CA_BUNDLE_MAP` | system roots only | +| Codex home | `--codex-home` | `ALLAGENTS_GATEWAY_CODEX_HOME`, then `CODEX_HOME` | existing supported host Codex home | +| Codex binary override | `--codex-bin` | `ALLAGENTS_GATEWAY_CODEX_BIN` | pinned SDK-managed surface; unset | +| Pi home | `--pi-home` | `ALLAGENTS_GATEWAY_PI_HOME` | existing supported host Pi home | +| Pi binary override | `--pi-bin` | `ALLAGENTS_GATEWAY_PI_BIN` | pinned package/RPC surface; unset | +| Graceful abort period | `--abort-grace` | `ALLAGENTS_GATEWAY_ABORT_GRACE` | `10s` | +| SIGTERM period | `--term-grace` | `ALLAGENTS_GATEWAY_TERM_GRACE` | `10s` | +| Final cleanup period | `--cleanup-timeout` | `ALLAGENTS_GATEWAY_CLEANUP_TIMEOUT` | `30s` | + +Precedence is CLI over gateway-specific environment over provider-standard +environment over default. For Codex this is `--codex-home`, +`ALLAGENTS_GATEWAY_CODEX_HOME`, then the ordinary `CODEX_HOME` identity +location. The advertised value is the absolute URL placed in +`AgentCard.supportedInterfaces`; wildcard hosts are invalid, non-loopback +listeners require an explicit value, and production uses HTTPS. The acquisition +image must be a full `repository@sha256:` reference; tags are rejected. +The gateway verifies that the local platform resolves to the release-recorded +platform digest before starting acquisition. + +Docker is a base-acquisition dependency only when no reusable validated base +exists. The configured endpoint must support creating, waiting for, stopping, +and removing a container plus bind-mounting gateway-created staging. A validated +cache hit does not contact Docker or resolve a source credential. The gateway +never passes the +Docker socket into the container. The acquisition network is preconfigured by +the operator to reach only declared Git/OCI source hosts and required auth/ +redirect hosts; the gateway supplies the stricter per-request host policy to the +acquirer. No Docker flag, mount, network, image, or environment override is +accepted from A2A. + +Credential and CA paths are resolved on the trusted host, must be current-user +owned regular files with private permissions, and are read only for acquisition. +Setting both OCI credential options is a startup error. `--oci-auth-file` +accepts at most 1 MiB of strict UTF-8 Docker-config JSON containing only +`auths`; each exact registry key contains one bounded `auth` or +`identitytoken`. `credsStore`, `credHelpers`, proxy/plugin fields, commands, +duplicate keys, and unknown members are rejected. + +The fixed OCI helper receives argv `[helperPath, "get"]` without a shell and the +raw exact Docker lookup key on stdin. Exit-zero stdout is one bounded strict JSON +object with nonempty `Username` and `Secret` plus optional matching `ServerURL`. +Timeout, nonzero exit, signal, malformed output, mismatch, or empty credentials +fails with `source_auth_oci_failed`; stderr is secret-bearing and never logged. + +`--oci-ca-bundle-map` names a bounded strict JSON file mapping exact normalized +`host[:port]` keys to private PEM CA files. Only the bundle for the exact +registry, token service, or declared layer-redirect host augments system roots; +there is no insecure-TLS switch. Registry access begins anonymously and accepts +only bounded same-origin Basic or Distribution Bearer behavior plus the +documented Docker Hub token service. Cross-origin redirects remain limited to +layer `GET`/`HEAD` requests for exact declared hosts, with credentials stripped +and every hop checked. The gateway passes only the selected source credential +and exact CA material into the acquisition container and destroys both before +provider execution. + +Provider homes are never copied, mounted into Docker, parsed by AllAgents, or +imported into another store. The direct Codex/Pi host process receives the +selected home path and required host identity/auth environment in place. +Binary overrides are absolute host paths and must pass the pinned adapter's +exact version/protocol probe at readiness; they are not request-selectable. +The explicit provider environment starts from an allowlist rather than the +gateway's complete environment, but this is leakage reduction, not isolation. + +The immutable-base cache and invocation roots are current-user owned, private, +and disjoint from state, project, profile, provider-auth, and each other. +Acquisition writes a unique directory under `/.staging`; host +validation completes before an atomic same-filesystem rename to either the final +cache-key directory or `/transient/` for a non-reusable +base. Active Task references pin reusable entries. Least-recently-used eviction +enforces the byte budget and removes only unpinned reusable bases. Every non- +publication path removes its staging directory, and startup reconciles orphan +staging and recorded transient bases before readiness. + +For read-only access, every Task owns +`//runtime`; its cwd resolves in a reusable cached base +or its non-reusable transient base. For read-write access, the Task also owns +`//workspace`. `auto` probes same-filesystem block +clone first, then rootless OverlayFS on Linux, then ordinary copy only when the +base does not exceed `--max-auto-copy-bytes`. `cow` requires block clone or +rootless OverlayFS and fails readiness when neither is available. `copy` is the +explicit portable, higher-I/O backend and may exceed the automatic copy ceiling. +The explicit `copy` backend has no Linux-only filesystem requirement, but it +does not by itself make the v1 gateway available on Windows; process lifecycle +and cancellation remain Linux-only in this plan. Startup logs the selected +capabilities without paths. No mode uses writable hard links. Startup rejects +overlapping roots and stale mounts it cannot safely reconcile. The derived workspace ID is a stable digest of the canonical project-workspace path and is verified against SQLite metadata. Retention includes Task records, -Artifact bytes, events, and invocation-key claims; expiry is transactional. When -the unexpired Task-count limit is reached, new admission fails rather than -evicting retained Tasks. +Artifact bytes, events, and invocation-key claims; expiry is transactional. Task +expiry does not evict a pinned base, and base eviction does not remove retained +Task metadata. When the unexpired Task-count limit is reached, new admission +fails rather than evicting retained Tasks. **Project workspace additions** @@ -1046,13 +1392,25 @@ workspaceSnapshots: repository: ghcr.io/entityprocess/allagents-workspaces layerRedirectHosts: - pkg-containers.githubusercontent.com + enterprise: + repository: company.jfrog.io/docker-local/allagents-workspaces ``` Snapshot names use the portable profile-name vocabulary. Repositories must have -unique stable names for remote acquisition. Snapshot repository values contain -only scheme/host/repository identity and an optional exact -`layerRedirectHosts` allowlist; never tags, digests, credentials, or extraction -paths. An absent allowlist rejects cross-origin layer redirects. +unique stable names for remote acquisition. Non-Docker-Hub repository values +contain only an exact registry `host[:port]/repository-path` identity and an +optional exact `layerRedirectHosts` allowlist; never tags, digests, credentials, +or extraction paths. + +Docker Hub uses only the canonical declaration +`docker.io//` with an explicit namespace. The gateway +maps that declaration to API origin `https://registry-1.docker.io`, Docker +credential lookup key `https://index.docker.io/v1/`, Bearer service +`registry.docker.io`, and token realm `https://auth.docker.io/token`; +`index.docker.io` and `registry-1.docker.io` declarations are rejected as +aliases. GHCR, JFrog Artifactory/JCR, and compatible private OCI registries keep +their declared exact host. An absent allowlist rejects cross-origin layer +redirects. **User workspace additions** @@ -1079,10 +1437,13 @@ TypeScript. Its `constructor(options: ProviderOptions)` requires and stores a nonempty `options.id`, validates `options.config`, and `id()` returns that stored value. `callApi(prompt, context?, options?)` reads -`context?.vars?.allagentsSource` when present and +`context?.vars?.allagentsSource`, +`context?.vars?.allagentsWorkingDirectory`, and +`context?.vars?.allagentsWorkspaceAccess` when present, plus `options?.abortSignal` for cancellation. -Static YAML defines the source mode and every logical name: +Static YAML defines the source mode, logical names, and optional default logical +working directory and workspace access: ```yaml prompts: @@ -1102,6 +1463,10 @@ providers: config: endpoint: https://allagents-gateway.example.internal target: codex + workingDirectory: + kind: repository + repository: allagents + workspaceAccess: readOnly source: kind: repositories revisions: @@ -1112,6 +1477,10 @@ providers: config: endpoint: https://allagents-gateway.example.internal target: codex + workingDirectory: + kind: repository + repository: allagents + workspaceAccess: readWrite source: kind: workspaceSnapshot snapshot: evaluation @@ -1125,6 +1494,11 @@ tests: allagentsSource: revisions: allagents: fedcba9876543210fedcba9876543210fedcba98 + allagentsWorkingDirectory: + kind: repository + repository: allagents + path: apps/gateway + allagentsWorkspaceAccess: readOnly - description: immutable prebuilt workspace providers: [codex-evaluation-snapshot] @@ -1132,6 +1506,11 @@ tests: allagentsSource: digest: sha256:fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210 workspaceManifestDigest: sha256:6789abcdef0123456789abcdef0123456789abcdef0123456789abcdef012345 + allagentsWorkingDirectory: + kind: repository + repository: allagents + path: apps/gateway + allagentsWorkspaceAccess: readWrite ``` The gateway enforces one active invocation transactionally. Promptfoo keeps @@ -1149,29 +1528,43 @@ revisions, and immutable digests, not `ghcr.io/entityprocess/allagents-workspaces`. The gateway resolves origins and credentials server-side and omits them from A2A source-identity responses. -`context?.vars?.allagentsSource` is the only per-test override. In repository +`context?.vars?.allagentsSource` remains limited to source leaves. In repository mode it may contain exactly `revisions`, whose keys must already exist in static -`config.source.revisions` and whose values are full lowercase 40-hex commits. -In snapshot mode it may contain exactly `digest` and/or +`config.source.revisions` and whose values are full lowercase 40-hex commits. In +snapshot mode it may contain exactly `digest` and/or `workspaceManifestDigest`, both full lowercase `sha256:` digests. Present leaves replace static leaves; absent leaves retain static values. Source kind, -repository-name allowlist, and snapshot name remain static. Unknown members, -mutable revisions, origins, destinations, credentials, and commands fail before -A2A submission. +repository-name allowlist, and snapshot name remain static. + +`context?.vars?.allagentsWorkingDirectory` replaces the complete static +selector for that trial. It is exactly `workspaceRoot` or a declared repository +name plus an optional `RelativeDirectory`; the gateway performs catalog and +post-acquisition directory validation. `allagentsWorkspaceAccess` replaces the +static access value with exactly `readOnly` or `readWrite`. Missing access +defaults to `readWrite`. Neither variable accepts an absolute path, configured +destination, materializer, cache key, `.` or `..` segment, backslash, symlink +escape, or non-directory. Unknown members, mutable revisions, origins, +destinations, credentials, and commands fail before provider execution. Each `callApi` creates one high-entropy invocation key and sends `SendMessage` with `returnImmediately: true`, then follows the accepted Task through -`SubscribeToTask`, `GetTask`, and bounded resubscription. An abort or deadline +`SubscribeToTask`, `GetTask`, and bounded resubscription. A read-only Task may +share its immutable physical base and cwd with other Tasks while keeping private +runtime state; a read-write Task receives a unique disposable writable view. The +caller chooses neither physical path nor materializer. An abort or deadline sends one `CancelTask` with a fresh cleanup signal. Ambiguous submission retry -reuses the same key and request. The provider returns terminal text or validated -structured result as `ProviderResponse.output`. It maps gateway usage exactly as +reuses the same key, canonical request, Task, base/view, cwd, and access mode. +The provider returns terminal text or validated structured result as +`ProviderResponse.output`. It maps gateway usage exactly as `inputTokens -> tokenUsage.prompt`, `outputTokens -> tokenUsage.completion`, `cachedInputTokens -> tokenUsage.cached`, and `totalTokens -> tokenUsage.total`; provider-specific counters remain in -`metadata`. Task ID, Artifact references, logical source identity, termination, -cleanup, and stable failure `code`/`retryable`/accepted `taskId` also remain in -`metadata`, without origins or destination paths. Admission and terminal -failures use a safe `ProviderResponse.error`. This provider is AI Evals code; +`metadata`. Task ID, Artifact references, logical source identity, logical +working directory, workspace access, termination, cleanup, and stable failure +`code`/`retryable`/accepted `taskId` also remain in metadata, without origins, +configured destinations, or physical paths. +Admission and terminal failures use a safe `ProviderResponse.error`. This +provider is AI Evals code; AllAgents has no Promptfoo runtime dependency. ### Error and Status Mapping @@ -1188,12 +1581,15 @@ reason. Custom admission errors include `google.rpc.ErrorInfo` with domain |---|---|---| | Unsupported A2A version | HTTP 400 A2A `VersionNotSupportedError`; no Task | No | | Missing required extension | HTTP 400 A2A `ExtensionSupportRequiredError`; no Task | No | -| Malformed request, source, digest, schema, prompt, or unknown target/source | HTTP 400 `INVALID_ARGUMENT`; `invalid_execution_request`; no Task | No | +| Malformed request, source, working-directory selector, workspace access, digest, schema, prompt, or unknown target/source/repository | HTTP 400 `INVALID_ARGUMENT`; `invalid_execution_request`; no Task | No | | Invocation-key conflict | HTTP 409 `ALREADY_EXISTS`; `invocation_key_conflict`; no new Task | No | | Identical retained invocation replay | Existing Task with embedded Artifacts | N/A | | Cancel after terminal state | HTTP 400 A2A `TaskNotCancelableError` | No | | Retained Task capacity exhausted | HTTP 429 `RESOURCE_EXHAUSTED`; `retention_capacity_exhausted`; `Retry-After`; no Task | Yes, after expiry | | Runtime capacity unavailable after acceptance | `execution_capacity_unavailable`; failed Task | Yes | +| Valid logical cwd resolves to a missing, non-directory, or escaping path after acquisition | `execution_working_directory_invalid`; failed Task; no provider start; no physical path returned | No | +| Required copy-on-write materializer unavailable, or `auto` would copy above its ceiling | `workspace_materialization_unavailable`; failed Task; no provider start | No | +| Task-private runtime, non-reusable base, or writable-view creation/removal fails | `workspace_cleanup_failed`; failed Task; `cleanup.workspace: "failed"`; retain cleanup record; stop admission if an active mount or uncertain writable view remains | Yes only as a fresh invocation after operator repair | | App absent/ineligible and configured `gh` succeeds | Continue with recorded provider class | N/A | | App applicability unknown | `source_auth_applicability_unknown`; failed Task; no fallback | Yes for rate-limit/service causes only | | Selected App config/auth/mint/validation/revocation failure | `source_auth_failed`; failed Task; no fallback | No | @@ -1209,15 +1605,17 @@ reason. Custom admission errors include `google.rpc.ErrorInfo` with domain | Deadline expires | `execution_deadline_exceeded`; abort/terminate; failed Task | Yes | | Known provider permission denial | `execution_permission_denied`; rejected Task | No | | Unknown provider protocol or result shape | `provider_protocol_invalid`; failed Task | No | -| Cancellation with proven quiescence | `execution_canceled`; canceled Task | No | -| Termination or cleanup cannot be proven | `execution_quiescence_unknown`; failed Task; readiness poisoned | No | -| State store durability/integrity failure | `task_store_failed`; stop admission; abort/contain; no success | No | +| Cancellation after acquisition removal or direct provider settlement | `execution_canceled`; canceled Task | No | +| Acquisition container or unpublished staging cannot be removed | `source_cleanup_failed`; failed Task; stop admission | No | +| Direct provider does not settle after abort/`SIGTERM`/`SIGKILL` | `execution_termination_failed`; failed Task; no filesystem/Git evidence; retain Task-owned runtime, view, or non-reusable base and lease; stop admission until post-teardown reconciliation | No | +| State store durability/integrity failure | `task_store_failed`; stop admission; request active-work abort; no success | No | | Restart finds interrupted Task | `gateway_restarted`; failed Task; no provider resume | Yes as a new invocation | | Retention expiry | HTTP 404 A2A `TaskNotFoundError` | Yes as a new invocation | Accepted-Task failures use the integrity Artifact's strict `failure` object with `code`, safe `message`, table-defined `retryable`, and one closed cause from -`validation | capacity | sourceAuth | sourceGit | sourceSnapshot | deadline | +`validation | capacity | sourceAuth | sourceGit | sourceSnapshot | sourceCleanup | +workingDirectory | workspaceMaterialization | workspaceCleanup | deadline | permission | providerProtocol | cancellation | termination | stateStore | restart`. Retryability says whether a caller may create a fresh invocation; it never enables automatic Task retry or provider/source fallback. Promptfoo copies @@ -1227,347 +1625,507 @@ carrier. ### Phased Delivery -1. Build the current CLI and record the red E2E showing that - `allagents gateway serve` is unavailable. Record the exact `/tmp/` workspace - setup, command, and observed failure. -2. Freeze workspace additions, the published extension, snapshot format, - common manifests, result schema, errors, packaging, and fixtures. Establish - the Rust helper protocol, safe SQLite VFS, platform packages, and ordered - release pipeline first. -3. Build the SQLite Task store through the helper, AllAgents A2A request handler, - HTTP+JSON server, minimal backend interface/registry, and fake adapter. -4. Extend the packaged helper with invocation supervision, execution - containment, spawn mediation, role-specific network/secret views, safe - evidence, and terminal arbitration around the fake adapter. -5. Add repository and OCI acquisition through the supervisor/helper with - credential containment and manifest validation. -6. Add Codex, then Pi, against the same conformance suite. -7. Run final implementation review and fix important correctness, security, +1. In a clean `/tmp/` npm prefix, install the current `allagents` package and + record the red E2E showing that `allagents-gateway serve` is unavailable and + that no acquisition image is fetched. +2. Execute U0 as a bounded feasibility gate: establish the private Bun + workspace layout; prove the shipped A2A server with the official JavaScript + client; pin and probe Codex SDK and Pi RPC/package surfaces; characterize + explicit provider environments and Linux process groups; build and run the + digest-pinned acquisition image for both supported architectures; and prove + independent CLI/gateway packaging plus exact release binding. +3. Freeze workspace additions, published extension, snapshot format, execution + and acquisition contracts, generated portable fixtures, error vocabulary, + SQLite schema/transactions, compatibility output, and release manifest. +4. Build the Bun SQLite Task store, AllAgents A2A request handler, HTTP+JSON/SSE + server, minimal backend interface/registry, and fake adapter. +5. Add direct host-process supervision, explicit environment construction, + read-only runtime separation, read-write materialization, process-group + cancellation, typed preparation, bounded evidence, terminal arbitration, + restart handling, and cleanup around the fake adapter. +6. Add Docker-only Git and OCI acquisition for requests without reusable bases, + with source-only credentials, strict mount/network/archive limits, typed + manifest emission, host validation, reusable-cache or non-reusable-base + publication, pinning, cleanup, reuse, and eviction. +7. Add Codex through the pinned SDK, then Pi through the pinned supported + package/RPC surface, against the same conformance suite. +8. Run final implementation review and fix important correctness, security, contract, reliability, DRY, and coverage findings. -8. Run the green bundled-CLI `/tmp/` E2E, clean-registry install smoke, - repository quality gates, user documentation, and release evidence. +9. Run green packed CLI/gateway `/tmp/` smokes, exact multi-architecture + acquirer-image tests, public GHCR conformance, exact-release authenticated + GHCR/JFrog conformance, repository quality gates, user documentation, and + release evidence. ### System-Wide Impact -- **Package surface:** Declare a root Bun workspace; add private - `packages/execution-service` and Rust `packages/execution-helper`; add the - service as a root `workspace:*` development dependency; distribute Linux - x64/arm64 helper binaries through versioned platform-specific optional - packages; and bundle the service into published `dist/index.js`. The release - scripts and Publish workflow version matching helper packages and root - dependency ranges, publish and verify both platform packages first, and - publish `allagents` only after their registry metadata and checksums resolve. - Add public `allagents gateway serve` without changing existing profile and - sync commands. Root build, typecheck, tests, and clean-registry install smoke - include the private workspace service and resolved helper binary. -- **Schema surface:** Extend project workspace schemas with named snapshots, - exact layer-redirect hosts, and user profile-client schemas with explicit - gateway enablement. Publish the versioned extension specification and - generated JSON Schemas; update configuration docs. -- **Dependency surface:** Put the official A2A SDK, pinned Codex SDK, and - `@octokit/auth-app` in the private service package. Pin SQLite, the custom VFS - bindings, `reqwest` with rustls, `tar`, `flate2`, and `zstd` in the Rust helper - lockfile together with the Rust toolchain/helper protocol; check helper release - checksums. -- **State surface:** Add one bounded SQLite gateway state root and per-invocation - staging, publication, evidence, and cleanup roots. Do not alter profile state. -- **Security surface:** The network is the caller authorization boundary. Source - credentials are phase-scoped; helper-mediated process and network views keep - acquired code and agent tools from App, `gh`, OCI, provider, MCP, operator, and - gateway credentials/state. Helper absence or capability loss fails closed. +- **Package surface:** Convert the root to private Bun workspace orchestration. + `apps/cli` publishes `allagents`; `apps/gateway` publishes + `allagents-gateway`; `apps/acquirer` publishes no npm package and builds only + the digest-pinned GHCR image. The ordinary CLI has no gateway dependency. + GitHub Actions has independent CLI and gateway release triggers. Gateway + release builds and verifies the acquisition image first, then publishes the + exact npm tarball; a gateway-only run never publishes the CLI. +- **Runtime surface:** `apps/gateway` owns gateway behavior end to end. Docker + exists only at the base-acquisition boundary when no reusable validated base + exists. Codex and Pi execute directly on the trusted Linux runner with + existing host + authentication. Packages share contracts and configuration, not generic + implementation helpers; do not add `core`, `common`, native IPC, or dual + implementations. +- **Schema surface:** `packages/workspace-config` extends project schemas with + named snapshots/exact redirect hosts and user profile-client schemas with + gateway enablement. `packages/execution-contracts` and + `packages/acquisition-contracts` generate versioned JSON Schemas and fixtures + under `contracts/`; update extension, snapshot-format, and configuration docs. +- **Dependency surface:** Pin Bun, the official A2A JavaScript SDK, + `@openai/codex-sdk`, the supported Pi package/RPC dependency, and the minimal + Git/OCI/archive dependencies used by `apps/acquirer` in the Bun lockfile. + Minimize dependencies per workspace and scan both the npm tarball and image. +- **State surface:** Add one bounded private Bun SQLite state root, one bounded + immutable-base cache, and per-Task runtime plus optional writable-view roots. + Do not alter provider profile or authentication state. +- **Security surface:** Network reachability authorizes callers. The acquisition + container has staging, source-only credentials, and strict source policy but + no host home or Docker socket. Provider execution has trusted CI-job + authority; explicit environments reduce accidental leakage but do not isolate + secrets or hostile code from model tools. - **Compatibility:** Existing workspace files remain valid because new fields - are optional. Gateway startup applies stricter repository-catalog rules. - Older binaries reject the new strict nested profile field, so docs state the - minimum supporting version. + are optional; request access defaults to `readWrite`. Gateway startup applies + stricter catalog, cache-root, materializer, and provider-readiness rules. + CLI/gateway version skew is governed by contract ranges; gateway/image + compatibility is exact by manifest digest and acquisition-contract version. ### Risks and Mitigations +- **A2A or provider-surface immaturity:** Pin exact JavaScript package versions + and run U0 wire/provider probes before production units. If the Codex SDK + lacks a required capability, document proof before selecting pinned app-server; + if neither works, the target is unavailable rather than silently scraped. +- **Contract drift:** Generate public and private schemas plus accepted/rejected + fixtures from the three narrow packages and run drift checks in the gateway, + acquirer, docs, and consumer fixtures. +- **Install-size regression:** Keep CLI and gateway workspace dependency graphs + separate, report packed/installed sizes, enforce budgets, and fail CLI-only + smoke if it resolves the gateway or acquisition image. - **Accidental network exposure:** Binding `0.0.0.0` is intentional and allowed; require a distinct advertised URL, use HTTPS in production, and state in - startup output/docs that every reachable host has full authority. + startup output/docs that every reachable peer has full authority. - **Profile identity drift:** Derive targets only from current validated user declarations and matching installed state; never resurrect declaration-missing launchers from retained profile state. -- **Credential leakage or path swap:** Use fresh validated/revoked App tokens or - one configured `gh` account, descriptor-bound credential handles, hermetic - Git, strict Docker auth/helper protocols, and credential teardown before - publication. Non-bypassable helper spawn mediation replaces environments, - closes descriptors, and enters role-specific mount/network namespaces before - every MCP or model-tool exec; a backend lacking that hook is unavailable. +- **Working-directory escape or mutable cross-trial reuse:** Accept only the + closed logical selector and `RelativeDirectory` grammar, resolve through the + compiled catalog, and require an existing directory beneath the selected + repository. Read-only Tasks share only the gateway-managed immutable base and + keep private runtime state; read-write views derive from Task IDs. Never expose + or accept a resolved host path. +- **Read-only contract violated by the prompt or provider:** Do not inspect + prompts or claim a sandbox. Disable optional Git locks, request native provider + read-only policy when available, isolate runtime writes, and document that + consumers must choose `readWrite` when project mutation is required. Do not + add a per-Task mount, chmod traversal, or full-tree verification in v1. A + violating provider can contaminate the base and later Tasks; the operator must + evict that entry before reuse. +- **Large workspace duplication or unsupported copy-on-write:** Acquire each + immutable identity once, pin shared bases, prefer block clone, fall back to + rootless OverlayFS, and retain explicit `copy` for portability. `auto` refuses + a full copy above its byte ceiling; startup reports capabilities, and CI users + provision enough disk or choose a larger/self-hosted runner. +- **Base-cache corruption or unbounded growth:** Bind keys to immutable source, + catalog/layout, and acquisition-contract identity; publish atomically; keep + roots private; pin active entries; evict only unpinned least-recently-used + entries under a byte budget; and stop admission on detected metadata or + filesystem inconsistency. This is trusted-runner state, not a hostile-process + integrity boundary. +- **Acquisition credential leakage:** Mount only staging, inject only the + selected source credential and exact-host CA material, never mount host home + or Docker socket, remove the container before provider execution, and scan + the manifest/staging/logs for gateway-managed credential values. - **Identity-changing fallback:** Classify App applicability as eligible, ineligible, or unknown; require repository-existence proof for 404 ineligibility; only positive ineligibility permits `gh`. - **OCI registry/archive abuse:** Require immutable digests, a closed - manifest/config/layer profile, same-origin metadata, exact operator-approved - layer-redirect hosts with per-hop address validation, changeset semantics, - fixed extraction limits, safe paths/types/links, and exact project-catalog - manifest verification. -- **Untrusted acquired code:** General hostile-code sandboxing beyond the - declared Linux process/network namespace and secret boundary is not claimed. - Invocation routes deny gateway, host loopback, and management networks; - provider/MCP egress is allowlisted; and model tools cannot reach provider/MCP/ - operator credentials or gateway state. Project/user setup shell commands are - never automatic. -- **Evidence-time attacks:** Prove containment empty first, then use - descriptor-relative no-follow reads with identity/size revalidation, reject - Git metadata indirection, and disable repository-controlled Git execution. -- **Provider/API churn:** Pin compatible SDK/CLI/model versions and retain - versioned native fixtures plus one adapter conformance suite. Gate Codex native - schemas to the pinned Structured Outputs subset and backend availability to a - proven non-bypassable spawn hook. -- **Orphaned processes:** Persist a stable empty containment identity before - start-gate release; enumerate the full project-owned cgroup namespace on - startup. On uncertain quiescence, stay alive, reject admission, and continue - reaping until empty without mutating the settled Task. -- **Store corruption or disclosure:** Route SQLite and all sidecars through the - helper's descriptor-rooted no-follow VFS with full synchronization and - transactions; validate ownership, modes, links, root disjointness, lock, and - workspace identity. Integrity/durability failure stops admission and prevents - terminal success. + manifest/config/layer profile, exact host/redirect policy, changeset + semantics, fixed extraction limits, safe paths/types/links, and exact catalog + validation inside the image and again at the host publication boundary. +- **Untrusted provider execution:** The acquired workspace and model tools run + with the same authority as the trusted CI job. Mitigate by using ephemeral + runners or an operator-managed VM/container boundary, least-privilege CI + credentials, explicit provider environments, no automatic setup commands, + and clear documentation. Do not describe AllAgents as a sandbox. +- **Evidence overclaim:** Collect only after the direct provider process settles, + keep evidence bounded, record process-group signals and observed cleanup, and + explicitly avoid claiming full descendant quiescence or output redaction. +- **Provider/API churn:** Pin SDK/package/protocol/model compatibility, require + exact probes for binary overrides, retain native fixtures, and share one + adapter conformance suite. Never download a provider runtime per request. +- **Orphaned processes:** Use a new Linux process group per direct provider, + persist the leader identity, escalate abort to `SIGTERM` and `SIGKILL`, and + rely on CI runner teardown as the final orphan boundary. +- **Store corruption or disclosure:** Use a current-user private state root, + exclusive gateway lock, ordinary Bun SQLite transactions, foreign keys, + `synchronous=FULL`, integrity checks, and bounded data. Integrity/durability + failure stops admission and prevents false success. +- **Artifact mismatch:** Bind every release report to the exact gateway npm + tarball digest and acquisition manifest/platform digests. Reject rebuilt, + mutable-tagged, wrong-commit, or contract-incompatible substitutes. ### Assumptions - The initial deployment is one gateway process and one transactionally enforced - active invocation. + active invocation on a trusted Linux CI runner. - Every external network peer able to connect is trusted with all available - targets, including built-ins and gateway-enabled profiles, and all retained - Tasks. Invocation descendants are deliberately unable to reach that network - boundary. + targets and retained Tasks. +- The CI job, VM, or deployment container is the isolation boundary. AllAgents + does not isolate hostile repository code, provider credentials, MCP secrets, + or host network access from model-invoked tools. - The selected project workspace is operator-controlled and compiles to 1-64 uniquely named GitHub repositories with collision-free destinations. - GitHub.com is the only authenticated Git host in the initial delivery. -- OCI snapshots use HTTPS registries and the frozen v1 direct-image format. -- Codex and Pi are available only when their pinned automation surfaces support - non-bypassable helper-mediated tool and MCP spawning. -- Gateway v1 execution supports Linux x64/arm64 hosts with cgroup v2, `clone3`, - pidfds, `openat2`, mount/network namespaces, nftables, and delegated - permissions. +- OCI snapshots use HTTPS Docker Hub, GHCR, JFrog Artifactory/JCR, or compatible + private OCI registries and the frozen v1 direct-image format. +- Docker is available solely for base-acquisition containers when no reusable + validated base exists, and the operator-provided acquisition network enforces + the deployment's source egress boundary. Immutable repository cache reuse + requires full commit IDs. +- Codex and Pi are installed or provided by pinned workspace dependencies before + gateway start and can reuse their existing host authentication locations. +- The implementation units after U0 assume the Bun/A2A/provider/process/acquirer + feasibility gates passed. A failed provider probe disables that target; a + failed architecture or release-binding gate stops the affected release rather + than introducing Rust, native platform packages, or a split runtime. --- ## Implementation Units -### U1. Workspace, extension, and manifest contracts - -- **Goal:** Freeze configuration, packaging, safe state primitives, and every - versioned public/private contract before runtime implementation. -- **Requirements:** R1, R2, R3, R5, R6, R7, R8, R9, R11, R18; AE3, AE4, AE5, - AE7, AE8, AE9, AE13, AE14, AE16, AE20; KTD1, KTD2, KTD5, KTD6, KTD7, - KTD10, KTD11. -- **Files:** root `package.json`/build/typecheck configuration, - `packages/execution-service/package.json` and TypeScript config, Rust - `packages/execution-helper`, Linux x64/arm64 optional packages, typed helper - protocol, SQLite schema/migrations and descriptor-rooted VFS, - `scripts/release.ts`, `scripts/publish.ts`, `.github/workflows/publish.yml`, - `src/models/workspace-config.ts`, schema generation tests and generated public - schemas, execution-service contracts, - `docs/src/pages/a2a/extensions/coding-execution/v1.astro` at the exact - declared URI plus a generated schema asset beneath that route, a versioned - snapshot-format specification, deterministic reference packer/conformance - fixtures, and configuration docs. -- **Approach:** Declare the Bun workspace and root `workspace:*` development - edge so the private service is installed, checked, and bundled. Establish the - helper protocol and audited SQLite VFS before the server store client. Version - helper packages with matching root optional-dependency ranges; publish and - verify both platform packages before the root package. Verify a clean registry - install resolves the matching helper binary and checksum and that the packed - root manifest contains no workspace protocol. - - Add strict named `workspaceSnapshots` with exact layer-redirect hosts and - nested profile-client `gateway.enabled`; preserve ordinary project/user - parsing while compiling gateway repository and target catalogs. Publish Agent - Card params, version/header activation, Message metadata/extensions, unified - Parts, exact result-schema grammar, source union, deadline, idempotency/replay, - HTTP+JSON errors, integrity/produced Artifacts, workspace manifest, OCI media/ - change-set/limit profile, and canonical digest preimages from Zod. -- **Execution note:** Start with independent wire fixtures that use only the - published extension specification. Reject missing version/activation, cross- - variant/unknown fields, extra Message Parts, undeclared names, mutable - snapshot references, malformed digests, invalid deadlines, incomplete or - mismatched manifests, gateway enablement without launcher, built-in - collisions, and unsupported clients while preserving unrelated metadata. - Fault-inject database/WAL/SHM link and rename swaps through the real VFS. -- **Verification:** Focused workspace-schema, packaging, helper VFS, and - contract tests; generated schema/spec drift checks; representative YAML, HTTP - errors, and wire examples parse through runtime schemas; snapshot conformance, - canonicalization, Artifact-cardinality, clean-registry install, matching - helper version/checksum, and ordered publish dry-run fixtures pass. +### U0. Bun monorepo, provider, process, and acquirer feasibility + +- **Goal:** Prove the settled Bun architecture can preserve A2A behavior, + independent distribution, supported provider control, Linux cancellation, + read-only shared-base execution, read-write materialization, and exact + acquisition-image release binding before production implementation. +- **Requirements:** R1-R2, R8, R13-R16, R18; AE8-AE11, AE17-AE18, AE20; + KTD1-KTD3, KTD6-KTD7, KTD9, KTD11-KTD14. +- **Files:** private root `package.json`/`bun.lock`, `apps/cli`, + `apps/gateway`, `apps/acquirer`, the three named `packages/` workspaces, + representative generated fixtures under `contracts/`, acquisition Dockerfile/ + image metadata, provider and workspace-materializer feasibility probes, + process-group probe, independent CLI/gateway pack scripts, and gateway release + workflow skeleton. +- **Approach:** Move the existing CLI into `apps/cli` without changing its + public package or behavior. Establish `apps/gateway` as the separately packed + Bun executable package and `apps/acquirer` as image-only code. Pin the + official A2A JavaScript SDK and drive a minimal production-direction server + through every required operation. Probe `@openai/codex-sdk` for invocation, + events, native abort, usage, structured-output support, and existing + `CODEX_HOME` behavior; consider app-server only when a named required + capability is proven absent. Probe the supported Pi package/RPC surface for + invocation, events, abort, usage, and existing host auth. Prove exact + compatibility rejection for global binary overrides. + + Run a real Linux child in a new process group and demonstrate graceful abort, + `SIGTERM`, and `SIGKILL` escalation plus the limit that unrelated/escaped + descendants are not proven gone. Prove one immutable base can serve repeated + read-only Tasks with private runtime state; probe block cloning and rootless + OverlayFS; verify independent writable changes and removal; and prove explicit + copy behavior plus the automatic copy ceiling. Build the acquirer image for + every supported architecture, run it with only a staging mount and synthetic + source secret, verify typed manifest output and container removal, and prove + the image has no provider runtime or Docker socket. Pack CLI and gateway + separately, prove a CLI-only install fetches neither gateway nor image, and + define the immutable gateway-tarball/acquisition-manifest release record. +- **Execution note:** U0 is a feasibility gate, not partial production + scaffolding. Do not paper over missing SDK/RPC behavior with TUI scraping, + AI SDK Harnesses, per-request downloads, per-provider Docker, or native + containment machinery. A missing provider capability disables that provider; + failed A2A, process, acquisition, or release-binding feasibility returns the + affected design for revision before dependent units. +- **Verification:** Official JavaScript client fixtures pass against the Bun + server; provider probes record exact pinned versions and auth-path behavior; + shared read-only base, private runtime, reflink, rootless-overlay, explicit + copy, cleanup, environment, and process-group probes pass on Linux; multi- + architecture image manifests/digests are recorded and the image boundary + rejects extra mounts/credentials/network; independent packed CLI/gateway + installs and compatibility fixtures pass; CLI-only installation fetches + neither gateway nor acquisition image. + +### U1. Workspace packages, contracts, SQLite, and release foundation + +- **Goal:** Freeze the monorepo ownership, workspace configuration, public and + acquisition contracts, ordinary SQLite transactions, and exact release + artifact binding before runtime implementation. +- **Requirements:** R1-R3, R5-R9, R11-R12, R18; AE3-AE9, AE13-AE14, AE16, + AE20; KTD1-KTD2, KTD5-KTD8, KTD10, KTD13-KTD14. +- **Files:** `packages/workspace-config`, `packages/execution-contracts`, + `packages/acquisition-contracts`, generated `contracts/` schemas and golden + examples, gateway SQLite schema/migrations, release scripts/workflows, + deterministic snapshot producer/conformance fixture, published extension and + snapshot-format assets, and configuration docs. +- **Approach:** Move authoritative project/user parsing and gateway catalog + compilation into `workspace-config`; add strict named `workspaceSnapshots`, + exact redirect hosts, and nested `gateway.enabled` without changing ordinary + CLI behavior. Define execution contracts for Agent Card params, version/header + activation, Message metadata/extensions, unified Parts, source union, logical + working-directory union and relative-path grammar, workspace access/default, + result-schema grammar, deadline, idempotency/replay, materialization errors, + HTTP errors, integrity/produced Artifacts, adapter events/results, and + evidence. Define acquisition contracts for the closed request, path-free typed + manifest, immutable-base cache key, private compiled-layout checks, OCI + media/change-set profile, fixed limits, and canonical digests. Generate + portable accepted/rejected fixtures beneath `contracts/`. + + Add private `bun:sqlite` ownership with foreign keys, WAL where supported, + `synchronous=FULL`, migrations, one execution lease, `createOrReplay`, + immutable-base metadata and active pins, internal outcome intent, atomic + settlement, transactional expiry, and recorded acquisition-container/provider- + process identities. Establish independent CLI/gateway versions and release + triggers. The gateway release record binds the exact npm tarball digest to the + acquirer multi-architecture manifest and supported platform digests; the image + is verified before npm publication. +- **Execution note:** Do not add `core`, `common`, a custom VFS, native file + primitives, native/platform npm packages, or runtime compatibility shims. + Start with external wire/manifest fixtures and stable rejection codes. Fault + SQLite transactions and process exit around commit/acknowledgment boundaries, + not filesystem attacks the ordinary SQLite contract does not claim to defeat. +- **Verification:** Workspace parsing/catalog fixtures, generated-schema drift, + wire/manifest accepted/rejected examples, canonicalization, Artifact + cardinality, SQLite commit/replay/lease/settlement/expiry/crash fixtures, + independent package versions, CLI-only and gateway clean-registry installs, + compatibility skew/image-mismatch matrix, exact tarball/image release record, + and idempotent absent/identical/divergent publication fixtures pass. ### U2. Deployment-wide Task store and A2A server - **Goal:** Serve the A2A lifecycle without application authentication and keep durable deployment-wide Task/idempotency truth behind a fake backend. -- **Requirements:** R1, R2, R3, R4, R5, R8, R13, R16, R17, R18; AE1, AE2, AE9, - AE11, AE14, AE15, AE16, AE19, AE20; KTD1, KTD2, KTD3, KTD4, KTD9, KTD10. -- **Files:** typed Task-store client, Agent Card, AllAgents request handler, - HTTP+JSON/SSE server, pagination/retention, minimal backend interface and - registry, fake adapter, health/readiness, CLI gateway command, focused tests. -- **Approach:** Implement flags/env precedence, bind/advertised-URL separation, - private project state and lock, helper-owned SQLite full-sync transactions, - startup integrity and full containment-namespace reconciliation, A2A version - and extension negotiation, exact `SendMessage` modes and `ListTasks` - semantics, standard/custom `google.rpc.Status` errors, durable - `createOrReplay`, one execution lease, internal outcome intent plus atomic - terminal settlement, bounded events/Artifact bytes, no early eviction, - transactional expiry, global listing/cancellation, deadline handling, and - fail-closed graceful shutdown against the fake adapter. -- **Execution note:** Prove with the official A2A client that one external caller - can read and cancel another caller's Task; this is expected behavior. Kill - subprocesses after transaction write/sync/commit/response boundaries and - fault-inject helper/VFS I/O, capacity races, cancellation intent, Artifact, - and terminal settlement. -- **Verification:** A2A discovery/send modes/stream/get/full list/subscribe/ - cancel/replay/expiry and HTTP-error integration tests on loopback plus explicit - `0.0.0.0`/advertised URL; health/readiness, state-path, retained and active - capacity, crash/store-fault, competing-lock, deadline, shutdown, and restart - tests. - -### U3. Invocation supervisor and backend contract - -- **Goal:** Run one fake-backed invocation through containment, typed - preparation, evidence, terminal arbitration, and cleanup with truthful - outcomes before real acquisition/adapters. -- **Requirements:** R3, R5, R8, R13, R14, R15, R16; AE9, AE10, AE11, AE12, - AE14, AE15, AE16, AE17, AE18; KTD3, KTD9, KTD10, KTD11, KTD12. -- **Files:** security/state helper extensions, provider/MCP/tool view and egress - compiler, invocation state machine, containment/start-gate controller, spawn - broker, typed preparation, evidence collector, result validator, - cleanup/reaper, and lifecycle tests. -- **Approach:** Extend the U1 helper to allocate an empty cgroup with a stable ID - and start gate, commit Task+lease+containment before release, and enumerate - recorded and unknown cgroups on startup. Launch every child into the cgroup; - mediate every backend MCP/tool spawn; enter role-specific mount and network - namespaces; replace environments; close descriptors; apply nftables egress - policy; use pidfds for termination/wait; and expose safe file operations. - Resolve targets through U2's typed fake adapter; never execute generated - launchers or setup commands. Commit one internal intent across provider, - cancel, deadline, and shutdown; capture live events; prove quiescence before - filesystem evidence; atomically settle status, evidence, Artifacts, cleanup, - and lease release; remain alive to reap when poisoned without mutating the - settled Task. -- **Execution note:** Fault-inject every boundary: capacity races and restart; - process death before/after empty-set creation, Task binding, child clone, and - start-gate release; pairwise and three-way outcome races; child fork/escape; - helper protocol/version/package mismatch; provider/MCP/tool attempts to reach - Agent Card, ListTasks, GetTask, SendMessage, CancelTask, host loopback, and - management networks; environment/path/inherited-FD/`/proc`/magic-link secret - reads by real child and grandchild processes; output truncation, malicious - evidence, valid-result-then-evidence-failure, and unknown cleanup. -- **Verification:** Deterministic lifecycle, single execution lease, helper - packaging/checksum, cgroup/pidfd/mount/network namespace containment, - non-bypassable spawn mediation, separate secret/descriptor/egress views, - typed preparation, safe-file/evidence, unknown-cgroup reconciliation, and - poison/reaping tests plus real child-process smoke on Linux x64/arm64 CI. - -### U4. Git and OCI workspace acquisition - -- **Goal:** Materialize declared repository sets and named OCI snapshots into the - same validated invocation workspace through the U3 security helper. -- **Requirements:** R6, R9, R10, R11, R12, R15, R16, R18; AE5, AE6, AE7, - AE8, AE10, AE15, AE17, AE18; KTD6, KTD7, KTD8, KTD11, KTD12. -- **Files:** acquisition coordinator, Git transport, GitHub provider selection, - strict Docker-auth/helper resolver, OCI Distribution client and changeset - applier, workspace-manifest validator, staging/publication helper, fixtures and - tests. -- **Approach:** Resolve name-based requests from the compiled project catalog. - Implement hermetic Git and full-commit verification. Apply the exact App - eligibility proof table, bypass token cache, validate/revoke each fresh token, - permit `gh` only for positive ineligibility, and use descriptor-bound temporary - helpers. Implement anonymous-first bounded Bearer authentication, redirect/ - credential-origin rules, the frozen direct-image media profile, streaming - descriptor verification, gzip/zstd changeset and whiteout semantics, all - extraction ceilings, exact project-manifest validation, and atomic publication. - Tear down every acquisition credential before typed preparation. -- **Execution note:** Use local Git remotes and a local OCI registry plus the U1 - producer fixture. Prove ambiguous/selected-App failures never call `gh`, two - sequential acquisitions mint distinct tokens, token validation/revocation and - lifetime are enforced, helper/auth-file swaps fail, and snapshot failure never - invokes Git fallback. -- **Verification:** Three-way provider-selection and real-response fixture tests; - Git branch/tag/full-commit integration; GHCR/Docker Hub helper fixtures; - malicious realm/scope/downgrade/redirect tests; OCI index/media/digest/size/ - limit/order/whiteout/path/catalog fixtures; credential leak scans; equivalent - complete manifest output across both acquisition modes. - -### U5. Codex backend adapter - -- **Goal:** Run built-in and profile-backed Codex targets through the supported - SDK while preserving structured progress, result, usage, cancellation, and - native evidence. -- **Requirements:** R7, R8, R13, R14, R15, R16; AE1, AE3, AE4, AE10, AE12, - AE15, AE17, AE18; KTD9, KTD11, KTD12. -- **Files:** Codex adapter, profile-context and auth bridge, fixtures, - conformance and optional credentialed smoke tests. -- **Approach:** Pin SDK/model compatibility and first prove a non-bypassable - synchronous hook that delegates every MCP and model-tool spawn to the U3 - helper. If the pinned Codex surface can bypass that hook, Codex is unavailable - in v1 rather than relying on an asserted view. Create one fresh thread per - Task; pass cwd, typed profile configuration, abort signal, and the private - Codex control-process auth view inside containment. Pass native `outputSchema` - only for the pinned Structured Outputs subset; otherwise add JSON guidance and - use the common terminal validator. Normalize events/usage, bound evidence, and - dispose fully. -- **Execution note:** Characterize the pinned SDK/model's spawn, schema, tool- - sandbox, auth, abort, and event behavior with captured fixtures before - normalization. Do not import Promptfoo provider code. -- **Verification:** Shared adapter conformance, real SDK child/grandchild spawn - mediation, filesystem/environment/inherited-FD/`/proc` credential denial, - gateway/host-network denial, native-schema and validated-fallback paths, - deadline, and an opt-in credentialed smoke case. - -### U6. Pi backend adapter - -- **Goal:** Run built-in and profile-backed Pi targets through strict RPC with the - same public lifecycle and honest capability reporting. -- **Requirements:** R7, R8, R13, R14, R15, R16; AE3, AE4, AE10, AE12, AE15, - AE17, AE18; KTD9, KTD11, KTD12. -- **Files:** Pi adapter, RPC parser, restricted policy extension, profile-context - and auth bridge, fixtures, conformance and optional credentialed smoke tests. -- **Approach:** First prove strict RPC exposes a non-bypassable synchronous hook - that delegates every MCP and model-tool spawn to the U3 helper. If Pi can - bypass that hook, Pi is unavailable in v1. Launch Pi with typed invocation - configuration, its private control-process auth view, strict JSONL RPC, - explicit allowed tools/extensions, per-MCP secret declarations, - deterministic permissions, event validation, deadline/cancellation - escalation, and settled completion. Repository extensions and unrestricted - built-ins remain disabled. -- **Execution note:** Characterize and pin Pi's spawn/RPC contract; record - Pi-specific facts as bounded native evidence rather than public schema - branches. -- **Verification:** Shared adapter conformance, real RPC child/grandchild spawn - mediation, filesystem/environment/inherited-FD/`/proc` provider/MCP secret - denial, gateway/host-network denial, malformed/unknown RPC, deadline, and an - opt-in credentialed smoke case. +- **Requirements:** R1-R5, R8, R13, R16-R18; AE1-AE2, AE9, AE11-AE16, + AE19-AE20; KTD1-KTD4, KTD9-KTD10. +- **Files:** `apps/gateway` Task-store module, Agent Card, A2A request handler, + HTTP+JSON/SSE server, pagination/retention, backend registry/fake adapter, + health/readiness, `allagents-gateway` command, and focused integration tests. +- **Approach:** Implement flags/environment precedence, bind/advertised-URL + separation, private state/lock, SQLite transactions, startup integrity and + interrupted-Task reconciliation, A2A version/extension negotiation, exact + `SendMessage` modes and `ListTasks` semantics, standard/custom + `google.rpc.Status` errors, durable `createOrReplay`, one execution lease, + internal outcome intent plus atomic terminal settlement, bounded + events/Artifact bytes, no early eviction, transactional expiry, deployment- + wide listing/cancellation, deadline handling, and graceful shutdown against a + fake adapter. +- **Execution note:** Use an independent official JavaScript A2A client to prove + one external caller can read and cancel another caller's Task; that is expected + trusted-network behavior. Kill gateway subprocesses around SQLite transaction, + commit, acknowledgment, cancellation-intent, Artifact, and settlement + boundaries. Do not add caller ownership or an application credential. +- **Verification:** Discovery, both send modes, stream/get/full list/subscribe/ + cancel/replay/expiry, HTTP errors, loopback and explicit + `0.0.0.0`/advertised URL, probes, retained/active capacity, competing lock, + SQLite crash/fault, deadline, shutdown, restart, and fake-backend tests pass. + +### U3. Host process supervisor and backend contract + +- **Goal:** Run fake-backed direct host invocations through shared read-only and + independent read-write workspace selection, logical cwd resolution, typed + preparation, explicit environment construction, process-group cancellation, + evidence, terminal arbitration, and cleanup with truthful limits before real + adapters. +- **Requirements:** R3, R5, R8, R13-R16, R18; AE8-AE12, AE14-AE18; + KTD3, KTD6, KTD9-KTD12. +- **Files:** `apps/gateway` backend types/registry, immutable-base manager, + workspace materializer, provider environment builder, Linux process-group + supervisor, invocation state machine, typed preparation, evidence collector, + result validator, cleanup/restart reconciliation, fake process fixtures, and + lifecycle tests. +- **Approach:** Define the minimal adapter contract for availability, + capabilities, access-aware invoke/events, graceful abort, direct-process + settlement, result/usage/evidence, and disposal. Resolve fake targets without + executing generated launchers or setup commands. For read-only, resolve cwd in + immutable base and allocate private runtime state. For read-write, materialize + a Task-ID-derived view via block clone, rootless OverlayFS, + or explicit copy. Resolve workspace-root and repository-relative selectors, + reject missing/non-directory/escaping paths, and pass only the effective cwd, + runtime paths, and access mode to the adapter. Start each direct provider in a + new process group, persist its leader PID and process-start marker before + marking execution started, and build its environment from a reviewed allowlist + that preserves required host identity/auth paths. Commit one internal intent + across provider terminal, cancel, deadline, and shutdown. Escalate adapter + abort to process-group `SIGTERM` and `SIGKILL`; capture bounded live events; + collect filesystem/Git evidence only after the direct process settles; remove + Task runtime or writable view; and atomically settle status, evidence, + Artifacts, observed termination, cleanup, and lease release. The non-settling + path emits only termination failure and live evidence, retains Task-owned + state plus lease, and blocks admission until verified reconciliation. +- **Execution note:** Fixtures must distinguish what AllAgents observes from what + it cannot guarantee. Exercise child and grandchild processes, including one + that escapes or outlives the direct process, and assert the gateway never + labels process-group cleanup as complete descendant quiescence. The CI runner + teardown is the final orphan boundary. No cgroups, pidfds, namespaces, + nftables, `openat2`, spawn broker, provider container, or isolation claim. +- **Verification:** Deterministic lifecycle; shared-base reuse without shared + runtime state; adapter-native read-only policy where available; independent + reflink, rootless-overlay, and copy views; automatic copy ceiling; cwd + resolution and escape rejection; single lease; explicit environment + inclusion/exclusion; + required host-auth preservation; binary-override compatibility rejection; + graceful/TERM/KILL timing; cancellation/deadline/shutdown races; poisoned- + lease behavior; post-teardown reconciliation; evidence ordering; result + states; cleanup outcomes; and truthful orphan-limit fixtures pass on trusted + Linux CI. + +### U4. Docker-only Git and OCI immutable-base acquisition + +- **Goal:** Materialize declared repository sets and named OCI snapshots when no + reusable validated base exists, validate and promote bases on the host, and + prove reuse and non-reusable cleanup without placing providers in Docker. +- **Requirements:** R6, R8-R12, R15-R16, R18; AE5-AE11, AE15, AE17-AE19; + KTD3, KTD6-KTD8, KTD10-KTD14. +- **Files:** `apps/acquirer` Git/OCI implementations and entrypoint, + `packages/acquisition-contracts`, `apps/gateway` Docker coordinator, + immutable-base cache/pin/eviction manager and host staging/manifest validator, + deterministic producer fixture, local/GHCR/JFrog fixtures, reusable registry- + conformance workflow, and focused tests. +- **Approach:** Derive cacheability and keys from the compiled catalog, + acquisition-contract version, layout digest, and immutable source identity. + A valid hit pins the base and starts no container or credential flow. On a + miss, the host creates private staging, resolves only the selected source + credential, and starts the exact digest-pinned image with staging as its sole + writable bind, no host home, no Docker socket, and per-request source policy. + In repository mode the host coordinator implements the App eligibility table, + mints and injects only the fresh repository-scoped token, validates/revokes it, + selects `gh` only for positive ineligibility, and never falls back after + selected-provider failure. Branch/tag requests bypass reusable bases. In + snapshot mode implement anonymous-first bounded Basic/Bearer authentication, + canonical Docker Hub normalization, exact-host CA/realm/redirect rules, + direct-image media profile, streaming digest verification, gzip/zstd + changesets/whiteouts, fixed limits, and path-free manifest/private layout + checks. + + The acquirer emits only typed manifest and staging content, then exits. The + gateway removes it, destroys source material, validates manifest, limits, and + exact catalog again on the host, and atomically publishes the base. Private- + root, pinning, budgeted unpinned-LRU eviction, and + restart fixtures cover cache lifecycle. Image probes prove no Codex/Pi/harness, + provider auth, host home, gateway state, or Docker control reaches acquisition. + Snapshot failure never invokes Git fallback. +- **Execution note:** Every PR runs local Git, local Distribution, and live + public digest-pinned GHCR against the exact built image. Release conformance + reuses the exact gateway npm tarball plus multi-architecture acquisition + manifest/platform digests without rebuilding. Authenticated GHCR uses least- + privilege pull credentials; pinned JFrog JCR uses HTTPS/private CA/private + repository/pull-only identity. Run platform-specific cases only where the + registry/runner supports that architecture and record coverage explicitly. +- **Verification:** App three-way selection, base-acquisition token lifetime/ + validation/revocation, cache-hit no-credential/no-container behavior, `gh` + fallback, Git revisions, immutable key invalidation, pin/eviction/restart, + non-reusable branch/tag base cleanup, Docker mount/env/network/credential/limit + enforcement, container and orphan-staging removal, + host revalidation/atomic publication, OCI auth/realm/redirect/CA/media/digest/ + size/whiteout/path/catalog cases, clean leak scans, equivalent typed manifests, + and exact local/public GHCR/authenticated GHCR/private-CA JFrog reports pass. + +### U5. Codex SDK adapter + +- **Goal:** Run built-in and profile-backed Codex targets on the trusted host + through the pinned SDK while preserving progress, result, usage, cancellation, + existing authentication, and truthful evidence. +- **Requirements:** R7-R8, R13-R16, R18; AE1, AE3-AE4, AE10-AE12, + AE15, AE17-AE18; KTD9, KTD11-KTD12. +- **Files:** `apps/gateway` Codex adapter, typed profile projection, environment + policy, SDK fixtures, shared conformance tests, and optional credentialed + smoke tests. +- **Approach:** Use pinned `@openai/codex-sdk` first. Create one fresh execution + context per Task; pass the resolved cwd, access mode, Task-private runtime + paths, and typed profile settings. For `readOnly`, request the native read-only + policy when supported and keep preparation outside the base. Preserve the + existing host `CODEX_HOME`/ChatGPT login when API credentials are absent; + stream/normalize events and usage; connect native abort to U3; bound evidence; + and dispose. + Use app-server only if U0 recorded a specific required SDK gap and pin/probe + its protocol. + Pass native `outputSchema` only for the supported Structured Outputs subset; + otherwise add JSON guidance and use the common terminal validator. +- **Execution note:** Characterize pinned SDK/model auth, abort, event, tool, and + schema behavior before normalization. Provider and model tools retain trusted + CI-job authority; tests inspect the explicit environment but make no hostile- + code, network, or secret-isolation claim. Do not import Promptfoo or AI SDK + Harnesses and do not download Codex per request. +- **Verification:** Shared adapter conformance; built-in/profile targets; + existing `CODEX_HOME` and API-credential paths; environment allowlist; exact + override probe; event/usage/result normalization; graceful/TERM/KILL + cancellation; native-schema and validated-fallback paths; deadline; malformed + provider payload; and opt-in credentialed smoke pass outside Docker. + +### U6. Pi RPC adapter + +- **Goal:** Run built-in and profile-backed Pi targets on the trusted host through + the pinned supported package/RPC surface with the same public lifecycle and + honest capability reporting. +- **Requirements:** R7-R8, R13-R16, R18; AE3-AE4, AE10-AE12, AE15, + AE17-AE18; KTD9, KTD11-KTD12. +- **Files:** `apps/gateway` Pi adapter/RPC parser, restricted policy extension, + typed profile projection, environment policy, fixtures, shared conformance + tests, and optional credentialed smoke tests. +- **Approach:** Launch Pi directly in the resolved cwd with access mode, + Task-private runtime paths, typed invocation configuration, existing host Pi + authentication location, strict RPC, explicit supported tools/extensions, + deterministic permissions, validated events, bounded evidence, and U3 + cancellation escalation. Request a native read-only policy when supported. + Never copy, mount, parse, or import Pi auth. + Repository extensions and unrestricted built-ins remain disabled. A global + Pi binary override must pass the exact pinned version/protocol probe. +- **Execution note:** Characterize and pin Pi's RPC/auth/abort/event contract. + Pi-specific facts remain bounded native evidence rather than public schema + branches. Model tools retain trusted CI-job authority; do not claim the + explicit environment isolates provider/MCP/operator secrets. +- **Verification:** Shared adapter conformance; built-in/profile targets; + existing host auth; environment allowlist; exact override probe; strict + malformed/unknown RPC rejection; event/usage/result normalization; graceful/ + TERM/KILL cancellation; deadline; and opt-in credentialed smoke pass outside + Docker. Malformed RPC can never produce success. ### U7. End-to-end delivery and documentation -- **Goal:** Prove the bundled/packed CLI and document the trusted-network - operating model, Linux requirements, workspace configuration, credentials, - sources, Promptfoo consumption, and risks. -- **Requirements:** R1-R19; F1-F6; AE1-AE21. -- **Files:** published extension and snapshot-format pages, gateway guide/ - reference, configuration reference, README, CHANGELOG, real project/user - workspaces, AI Evals-style Promptfoo YAML and custom-provider contract fixture, - E2E fixtures, packed-install smoke, release evidence. -- **Approach:** After final implementation review, build and pack the CLI plus - both helper packages; install in a clean Linux environment; create project and - user workspaces under `/tmp/`; gateway-enable fixture targets; serve on - loopback and `0.0.0.0` with a valid advertised URL; exercise health/readiness; - acquire local Git and OCI fixtures; and run an independently generated - official A2A client through version/extension negotiation, errors, both send - modes, complete listing, success, replay, cancellation, deadline, shutdown, - restart, and expiry. Run the custom-provider fixture through repository and - snapshot invocations with secure Promptfoo defaults, proving requests contain - only logical source data while the gateway resolves origins. Document full - network-peer authority, sensitive opaque payloads, and process-only secrets. -- **Execution note:** Green smoke uses the same built command and `/tmp/` - workspace shape as red E2E, never a test-only server. The consumer fixture is - AI Evals-style test/documentation code; AllAgents runtime does not import - Promptfoo. -- **Verification:** `bun run build`, packed-install/helper checksum smoke, - focused and full tests, typecheck, lint, docs build, extension/schema drift, - custom-provider contract fixture, and exact red/green commands/results in the - PR description. +- **Goal:** Prove independently released Bun CLI/gateway packages and the exact + acquisition image, then document the trusted-network and trusted-runner model, + workspace/source configuration, host auth, registry coverage, Promptfoo + consumption, installation, release ordering, and limits. +- **Requirements:** R1-R19; F1-F6; AE1-AE21; KTD1-KTD14. +- **Files:** published extension/snapshot-format pages, gateway guide/reference, + configuration reference, README, CHANGELOGs, real project/user workspaces, + AI Evals-style Promptfoo YAML/provider contract fixture, E2E fixtures, + CLI-only and gateway packed-install smokes, acquisition-image release record, + GHCR/JFrog reports, size/SBOM evidence, and independent release evidence. +- **Approach:** After final review, pack `apps/cli` and `apps/gateway` + independently without publishing. Prove CLI-only installation resolves + neither gateway nor image; install the gateway tarball in a clean trusted + Linux environment with Docker and pre-existing Codex/Pi host auth. Create + project/user workspaces under `/tmp/`; serve on loopback and `0.0.0.0`; test + probes and the complete A2A lifecycle; acquire local Git/OCI plus live registry + fixtures through the exact image; and run Codex/Pi on the host. Exercise the + Promptfoo consumer fixture in both source modes with per-trial logical cwd and + workspace access. Prove read-only Tasks reuse one immutable base without + shared runtime state, read-write Tasks receive independent disposable views, + and requests carry only logical source, cwd, and access data. + + Run registry workflows with the exact gateway tarball, acquisition manifest, + supported platform digests, and build commit. Gateway publication is blocked + until the image has passed required GHCR/JFrog conformance. Document that + network peers have full Task authority, providers/model tools have CI-job + authority, explicit environments are not isolation, evidence follows only + direct-process settlement, Docker is acquisition-only, and ephemeral runner + teardown is the final orphan boundary. +- **Execution note:** Green smoke uses the release-candidate npm tarball and + exact acquisition image artifacts, never a checkout rebuild. The consumer + fixture is AI Evals-owned test/documentation code; AllAgents runtime does not + import Promptfoo. +- **Verification:** CLI-only/gateway clean installs and sizes, independent + release dry runs, compatibility/image mismatch fixtures, local Distribution + and public digest-pinned GHCR on every PR, authenticated GHCR and private-CA + JFrog release conformance against exact artifacts, complete A2A/Task/provider/ + acquisition E2E, Promptfoo contract fixture, Bun typecheck/lint/test/build, + dependency/image scans, generated contract/docs drift, docs build, and exact + red/green commands/results in the PR description. --- @@ -1575,94 +2133,153 @@ carrier. | Gate | Applies to | Required evidence | |---|---|---| -| Workspace/package schema | U1 | Root workspace install/build edge; ordered helper-package publication and clean-registry resolution; project/user parsing; compiled repository/target catalogs; generated schema/spec drift | -| Public contract | U1-U2 | Independent official HTTP+JSON client; card interface/params/streaming capability; A2A version and every-operation extension headers; unified Parts; both send modes; complete listing; metadata; `google.rpc.Status`; request/result/Artifact/canonicalization fixtures | -| Trusted-network model | U2-U3, U7 | Loopback and `0.0.0.0` with distinct advertised URL; HTTPS docs; shared external Task visibility/cancellation; invocation-to-gateway and host-network denial; metadata-only health/readiness | -| Durable Task lifecycle | U1-U3 | Descriptor-rooted SQLite VFS/full-sync transactions; private state/lock; create-or-replay; one execution lease; internal outcome intent and atomic terminal settlement; no early eviction; crash/store faults; restart; transactional expiry | -| Repository acquisition | U4 | Compiled-name resolution, hermetic Git, commits, 200/404/ambiguous App eligibility, cache bypass, token validation/revocation, `gh` fallback and sub-budget | -| OCI acquisition | U4 | Strict Docker auth/helper; exact layer-redirect allowlist and per-hop address checks; Bearer origin policy; direct-image/config/layer media; descriptor verification; changesets/whiteouts; fixed limits; exact project catalog; no fallback | -| Linux helper and isolation | U1, U3-U7 | x64/arm64 packages/checksums; kernel/cgroup readiness; gated durable containment; full namespace enumeration; pidfd termination; openat2 path/VFS handles; non-bypassable spawn mediation; separate mount/environment/descriptor/network views | -| Supervisor lifecycle | U3 | Capacity races; pre/post-gate crash points; provider/cancel/deadline/shutdown intent races; live-event capture; atomic evidence settlement; poison/readiness/reaping; unknown-set proof | -| Safe evidence | U3-U6 | Descriptor-relative reads with identity/size recheck; links/special/sparse/replaced files and Git indirection rejected; no verified FS evidence before quiescence | -| Backend conformance | U2-U3, U5-U6 | Same lifecycle suite for fake, Codex, and Pi; real child/grandchild spawn mediation; credential and gateway-network denial; profile and built-in variants | -| Structured result | U1, U3, U5-U6 | Public grammar, Codex native-subset gate and fallback, valid/invalid/not-produced states, Artifact cardinality, no false publication | -| Repository quality | All | Build, clean-registry install, focused/full tests, typecheck, lint, schema/spec checks, docs build | -| Bundled CLI E2E | U7 | Recorded red then green command under `/tmp/`, both sources, advertised URL/probes, auth and network isolation, capacity, replay/cancel/deadline/shutdown/restart | -| Promptfoo consumption | U7 | Secure-default AI Evals YAML for both modes; optional context; nonblocking acceptance/subscription/cancel; source/provenance omit origins; output/usage/error metadata mapping | +| Bun architecture feasibility | U0 | Exact A2A JavaScript SDK pin and official-client server-direction operations; pinned Codex SDK and Pi RPC/package probes; existing host-auth behavior; shared read-only base/private runtime; reflink, rootless-overlay, and copy probes; Linux abort/TERM/KILL process-group probe with truthful descendant limit; exact multi-architecture acquirer image; independent packed CLI/gateway installs; immutable tarball/image binding | +| Bun repository quality | U0-U7 | One lockfile; private root orchestration; workspace-scoped typecheck/lint/test/build; dependency and image scans; generated-contract drift; minimized runtime dependencies; packed and installed size budgets | +| Package and release separation | U0-U1, U7 | Independent `allagents` and `allagents-gateway` versions/tags/triggers/tarballs; image-first gateway release; exact npm tarball plus acquisition manifest/platform digests; idempotent publication; CLI-only install fetches neither gateway nor image; gateway-only release never publishes the CLI | +| Workspace and contract packages | U0-U1 | Only `workspace-config`, `execution-contracts`, and `acquisition-contracts` shared packages; generated portable `contracts/` fixtures; normalized catalogs/defaults/order/collision keys/stable errors; project/user parsing; schema/spec drift; no `core`/`common` | +| Public contract | U0-U2 | Official JavaScript client against the Bun gateway; card interface/params/streaming; A2A version and extension headers; unified Parts; logical cwd and workspace-access schema/default/canonicalization/integrity evidence; both send modes; complete listing; metadata; `google.rpc.Status`; request/result/Artifact fixtures | +| Trusted-network and runner model | U2-U7 | Loopback and `0.0.0.0` with distinct advertised URL; HTTPS docs; shared external Task visibility/cancellation; trusted Linux CI job/VM/deployment container as provider isolation boundary; read-only described as cooperative best-effort; explicit no-hostile-code/no-secret-isolation wording; metadata-only probes | +| Durable Task lifecycle | U1-U3 | Private gateway-owned `bun:sqlite`; foreign keys and `synchronous=FULL`; transactions for create/replay, base pins, lease, intent, settlement, and expiry; no early eviction; process-kill/store faults; lock/restart/interrupted-Task reconciliation | +| Acquisition-container boundary | U0, U4, U7 | One fresh container when no reusable validated base exists and none on cache hit; exact digest-pinned image; staging-only writable bind; selected repository/registry credentials and CA material only; no App private key, host home, Docker socket, gateway state, provider auth, Codex, Pi, or harness downloads; strict source network/size/archive policy; typed manifest; exit/removal before host validation and provider execution; orphan-staging cleanup | +| Immutable-base cache | U0-U4, U7 | Key binds acquisition contract, compiled catalog/layout, and immutable source; exact commit/digest reuse; branch/tag bypass into non-reusable Task-owned bases; atomic promotion; active pins; unpinned LRU byte-budget eviction; one acquisition across 100 identical read-only trials; no cached credentials; transient-base cleanup | +| Repository acquisition | U0, U4 | Compiled-name resolution; hermetic Git/full commits; App 200/404/ambiguous eligibility; fresh base-acquisition token validation/revocation; cache-hit no credential; `gh` only after positive ineligibility; acquisition sub-budget; no provider start on failure | +| OCI acquisition | U4 | Canonical Docker Hub plus GHCR/JFrog/private-registry matrix; exact-key auth/helper/CA; bounded Basic/Bearer; redirect/rebinding policy; direct-image/config/layer media; descriptor verification; path-free manifest/private layout; changesets/whiteouts; fixed limits; no fallback | +| Registry and exact-artifact conformance | U4, U7 | Local Distribution and public digest-pinned GHCR on every PR; authenticated GHCR and private-CA JFrog release targets; exact gateway npm tarball plus acquisition multi-architecture manifest/platform digests without rebuild; positive/negative auth/permission/CA/media/path cases; explicit architecture coverage | +| Host supervisor lifecycle | U3 | One active lease; reusable-base read-only/private-runtime and non-reusable-base paths; independent writable views and cleanup; provider PID/start identity persisted before started state; explicit environment allowlist and host auth paths; cancel/deadline/shutdown races; graceful abort then TERM/KILL; non-settling failure retains Task-owned state/lease and blocks readiness; verified post-teardown reconciliation | +| Truthful bounded evidence | U3-U6 | Live bounded events; collection only after the direct provider settles and escalation finishes; hermetic Git inspection; observed termination/cleanup recorded; no claim of full descendant quiescence, hostile-code containment, secret isolation, or opaque-output redaction | +| Backend conformance | U0, U2-U3, U5-U6 | Narrow access-aware AllAgents adapter contract; same lifecycle suite for fake, Codex SDK, and Pi RPC/package; built-in/profile variants; reusable or non-reusable read-only bases and independent read-write views; resolved logical cwd/runtime/access passed to providers; existing host auth; exact binary override probes; direct host execution outside acquisition Docker; no AI SDK Harnesses or per-request runtime download | +| Structured result | U1, U3, U5-U6 | Public grammar; Codex native-subset gate and validated fallback; valid/invalid/not-produced states; Artifact cardinality; malformed provider/RPC payload cannot publish success | +| Repository quality | All | Bun install/typecheck/lint/test/build; focused and full suites; clean-registry packed installs; dependency/image audit; generated schema/spec checks; docs build | +| Packaged gateway E2E | U7 | Recorded red/green `/tmp/` commands; explicit gateway install; exact acquisition image; Git/local OCI/GHCR/JFrog sources; base reuse/materialization/cleanup; advertised URL/probes; capacity/replay/cancel/deadline/shutdown/restart; host Codex/Pi auth; truthful trust documentation | +| Promptfoo consumption | U7 | Secure-default AI Evals YAML for both source modes; optional context; per-trial `allagentsWorkingDirectory` and `allagentsWorkspaceAccess`; shared-base read-only trials; independent disposable read-write views; nonblocking acceptance/subscription/cancel; logical source/cwd/access provenance without origins or physical paths; output/usage/error metadata mapping; no AllAgents Promptfoo runtime dependency | ## Definition of Done ### Global - Every R1-R19 requirement is implemented or explicitly demonstrated by a - passing acceptance scenario. -- The gateway starts with no `gateway.yaml` or `worker.yaml`, defaults to - loopback HTTP, accepts explicit `0.0.0.0`, requires a separate advertised URL - off default loopback, documents production HTTPS, and exposes truthful + passing acceptance scenario; F1-F6 and AE1-AE21 agree with the implementation + and error table. +- U0's Bun/A2A/provider/process/acquirer/package gate passes before dependent + units. The private root, three apps, three named packages, and generated + `contracts/` fixtures are the complete shared layout; no speculative shared + package, native sidecar, or split runtime remains. +- `allagents` and `allagents-gateway` remain independently versioned and + released. A CLI-only install fetches neither gateway nor acquisition image. + Gateway release builds/verifies the exact acquisition image and registry + reports before publishing the bound npm tarball. +- The gateway starts without `gateway.yaml` or `worker.yaml`, defaults to + loopback HTTP, accepts explicit `0.0.0.0`, requires a distinct advertised URL + away from default loopback, documents production HTTPS, and exposes truthful metadata-only health/readiness. -- Network reachability is the only external caller trust boundary; Task - visibility and idempotency are deployment-wide. Invocation descendants cannot - reach that boundary, host loopback, or management networks. +- Network reachability is the external caller authorization boundary; Task + visibility and idempotency are deployment-wide. Provider execution uses the + trusted Linux CI job/VM/deployment-container boundary and existing host auth. + Documentation explicitly says AllAgents does not contain hostile repository + code or isolate provider/MCP/operator secrets from model-invoked tools. - Project workspace declarations compile to the exact repository/snapshot catalog; user declarations own profile launcher gateway enablement; built-in - target IDs cannot be shadowed. + IDs cannot be shadowed. Requests may select only the effective workspace root + or a declared repository plus a bounded relative directory and may select only + `readOnly | readWrite` access. They cannot supply physical/configured + destination paths, origins, credentials, commands, provider environments, + materializers, cache keys, Docker images/options/mounts, or provider permission + policy. - The published extension, Agent Card interface/params, A2A version and - activation headers, unified Parts, both send modes, full ListTasks behavior, - metadata preservation, strict schemas, HTTP+JSON errors, embedded Artifacts, - canonicalization, retention, and cancellation pass independent official-client - fixtures. + activation headers, logical cwd and workspace-access unions/defaults, unified + Parts, both send modes, full `ListTasks`, metadata preservation, strict + schemas, HTTP+JSON errors, embedded Artifacts, canonicalization, retention, + and cancellation pass official-client fixtures. - Secure-default AI Evals Promptfoo YAML selects repository mode with optional - named revision overrides or snapshot mode with one handle and immutable - digests. The provider maps one optional-context `callApi` to one nonblocking - Task, retains its high-entropy key across ambiguous retry, propagates - cancellation with a fresh cleanup signal, normalizes usage, and returns safe - error metadata and logical provenance without origins or runtime dependency. -- Git and OCI modes produce one complete workspace-manifest contract. OCI v1 - uses the direct-image/config/layer profile, exact project catalog, descriptor - verification, same-origin metadata, operator-approved layer redirect hosts - with per-hop address validation, changeset semantics, and fixed extraction - ceilings. Source modes never fall back and provenance never overclaims + named revisions or snapshot mode with immutable digests and can replace the + logical cwd and access per trial. Each `callApi` maps to one nonblocking Task; + read-only Tasks may share the immutable base and physical cwd, while read- + write Tasks receive independent disposable views. Ambiguous retries retain the + same key, base/view, cwd, and access. The provider propagates cancellation, + normalizes usage, and returns safe failure/logical provenance without origins, + configured destinations, physical paths, or an AllAgents Promptfoo dependency. +- Every request without a reusable validated base starts the exact digest-pinned + image with staging as its only writable bind plus source-only credentials and + strict policy. A valid cache hit starts no container and resolves no + credential. The image has + no host home, Docker socket, gateway state, provider auth, Codex, Pi, or + harness download path. It emits a typed manifest, exits, and is removed before + host validation, immutable-base publication, typed preparation, or provider + execution. Git and OCI modes produce one path-free manifest contract while the + host validates exact private destinations. OCI v1 remains registry-neutral + across Docker Hub, GHCR, JFrog, and compatible private registries with + immutable digests, descriptor verification, exact redirect/auth/CA rules, + changesets, fixed limits, no cross-mode fallback, and truthful source verification. -- App eligibility and ambiguous 404 handling, acquisition sub-budget, positive- - ineligibility `gh` fallback, fresh token cache bypass/validation/revocation, - strict Docker auth/helper and registry challenge policy, and pre-provider - credential teardown are proven. -- Typed preparation never runs workspace setup commands. The packaged Linux - helper durably binds containment before releasing any child, enumerates - unknown cgroups, and mediates every MCP/tool exec into role-specific mount, - environment, descriptor, credential, and network views. Real Codex/Pi - child/grandchild tests prove model tools cannot reach provider/MCP/operator - credentials, gateway state, or the gateway/host-management network; a backend - without non-bypassable spawn mediation is unavailable. -- The descriptor-rooted SQLite VFS, execution lease, pre/post-start-gate crash - boundaries, internal outcome-intent races, atomic terminal evidence - settlement, result states, state-path safety, descendant quiescence, readiness - poisoning/reaping, immutable terminal Tasks, and cleanup pass fault tests. -- Evaluation behavior, public-Internet authentication, remote workers, custom - materializers, non-Linux gateway execution, and multi-tenant policy remain - absent. +- App eligibility/ambiguous 404 handling, base-acquisition fresh token + validation/revocation, cache-hit credential avoidance, positive-ineligibility + selection, OCI auth/challenges, exact-host CA, and acquisition credential + teardown pass. Local Distribution and public digest-pinned GHCR run on every + PR; authenticated GHCR and private-CA JFrog release reports match the exact + gateway tarball, acquisition manifest, supported platform digests, commit, and + compatibility output. +- Codex uses pinned `@openai/codex-sdk` first and existing `CODEX_HOME`/ChatGPT + login when API credentials are absent; app-server is used only for a recorded + SDK capability gap. Pi uses its pinned supported package/RPC surface and + existing host auth. No OAuth/auth files are copied, mounted, parsed, or + imported, and no provider runtime is downloaded per request. +- Direct providers start in Linux process groups with explicit environments that + preserve required identity/auth paths. Read-only Tasks use shared immutable + bases with private runtime state and best-effort provider policy; read-write + Tasks use independent disposable block-cloned, overlaid, or copied views. + Cancellation escalates adapter abort to `SIGTERM` to `SIGKILL`. Evidence is + bounded and begins only after the direct provider settles. A non-settling + provider publishes no filesystem/Git evidence, retains its Task-owned runtime + and any writable view plus the lease, and blocks readiness until verified + post-teardown reconciliation. Evidence reports observed termination/cleanup, + not full descendant quiescence; CI runner teardown is the final orphan + boundary. +- Ordinary private Bun SQLite ownership, foreign keys, full synchronization, + create/replay, base pins, one lease, internal outcome races, atomic settlement, + immutable terminal Tasks, expiry, crash/restart reconciliation, cache + eviction, and cleanup pass fault tests without a custom VFS or native file + layer. +- Evaluation behavior, public-Internet authentication, remote workers, caller- + selected custom materializers, per-provider Docker, native containment + primitives, non-Linux gateway execution, and multi-tenant policy remain absent. ### Per unit -- U1: Root workspace packaging, ordered helper-platform publication, clean- - registry resolution, safe SQLite VFS, runtime/generated schemas, published - extension and snapshot format, producer fixture, and invalid negotiation/ - source/enablement/collision/configuration fixtures agree. -- U2: Official HTTP+JSON operations, version/extension/error/list semantics, - global replay/visibility, helper-owned SQLite locks/crashes, execution lease, - listeners/advertised URL, probes, deadline, shutdown, restart, and retention - pass against the fake backend. -- U3: The fake lifecycle proves gated durable containment, full namespace - reconciliation, atomic terminal settlement, typed preparation, spawn-mediated - secret/descriptor/network views, safe evidence ordering, poisoning, reaping, - and cleanup on Linux x64/arm64. -- U4: Git and OCI fixtures pass; App eligibility/cache bypass/token - validation/revocation, Docker credential/challenge and layer-redirect rules, - changesets, limits, exact catalog, and no-fallback rules are observed; leak - scans are clean. -- U5: Codex passes shared conformance and both schema paths; optional - credentialed smoke evidence is recorded when credentials exist. -- U6: Pi passes the same conformance and malformed RPC cannot produce success. -- U7: Final review is resolved; bundled and packed CLI red/green E2E under - `/tmp/`, Promptfoo fixture, complete repository gates, published schemas/specs, - docs, and reproducible PR instructions are complete. +- U0: Bun workspace layout, official-client A2A server direction, pinned Codex + SDK/Pi surface probes, host-auth behavior, shared read-only/private-runtime and + read-write materializer probes, explicit environment/process-group + feasibility, multi-architecture acquirer image, independent packed installs, + and exact tarball/image release binding all pass. +- U1: Three narrow packages and generated fixtures, workspace additions, + execution/acquisition contracts including logical cwd, workspace access, + relative-path grammar, base-cache identity, and materialization errors, Bun + SQLite transactions/pins, published extension/snapshot format, independent + versions, compatibility matrix, and image-first release fixtures agree. +- U2: Official-client operations, version/extension/error/list semantics, + logical cwd/access defaults/canonicalization/integrity evidence, deployment- + wide replay/visibility, SQLite lock/crash/lease behavior, listeners/advertised + URL/probes, deadline/shutdown/restart/retention, and fake backend pass. +- U3: The fake lifecycle proves shared immutable-base read-only execution with + private runtime state, independent read-write views across every configured + materializer, cwd resolution/escape rejection, explicit provider environments, + required host-auth preservation, process-group abort/TERM/KILL, outcome races, + atomic settlement, evidence ordering, restart cleanup, and truthful orphan + limitations on Linux. +- U4: Git/OCI fixtures, App/`gh` selection, cache hit/miss/key/pin/eviction, + exact acquisition image boundary, staging-only mount, source credentials/ + network/limits, typed manifest, host revalidation/publication, local/public/ + authenticated GHCR, private-CA JFrog, exact manifest/platform digests, no + fallback, and leak scans pass. +- U5: Codex passes shared access-aware conformance and both schema paths through + the pinned SDK or documented required app-server fallback, receives resolved + cwd/runtime/access, reuses existing host auth, runs outside Docker, and records + optional credentialed smoke evidence. +- U6: Pi passes the same host-process conformance through pinned RPC/package + support with resolved cwd/runtime/access, reuses existing auth, and malformed + RPC cannot produce success. +- U7: Final review is resolved; CLI-only/gateway packed smokes and `/tmp/` E2E, + independent release/size/SBOM evidence, public GHCR on every PR, + authenticated GHCR/private-CA JFrog exact-artifact reports, Promptfoo + per-trial logical-cwd/access shared-base/read-write-view fixture, repository + gates, published schemas/specs, truthful threat-model docs, and reproducible PR + instructions are complete. From 0b276209283b18e6bd8c86d7b57a761bc81980b9 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 21 Sep 2026 16:37:16 +1000 Subject: [PATCH 12/44] docs(gateway): clarify execution gateway decision Focus the ADR on user and operator impact while keeping implementation plumbing in the linked plan. --- ...-agent-execution-through-an-a2a-gateway.md | 1130 +++-------------- 1 file changed, 209 insertions(+), 921 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index 1d78419a..4d087379 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -2,930 +2,218 @@ - Status: Accepted; implementation pending - Date: 2026-09-17 -- Updated: 2026-09-20 - -## Context - -AllAgents already owns cross-client agent configuration, project workspace -knowledge, global profiles, plugins, hooks, MCP configuration, and generated -launchers. External systems also need to invoke those agents without importing -AllAgents internals or driving an interactive terminal. - -The first planned consumer is AI Evals. It needs one remote coding-agent call to -return terminal output, usage, traces, file changes, produced artifacts, -failures, cleanup outcomes, and execution provenance. Future trusted tools on a -private developer network may need the same execution boundary. - -The initial product is not a public multi-tenant control plane. Developers are -expected to run one gateway for one AllAgents project workspace and expose it on -loopback, a firewalled network, or a Tailscale network. Network reachability is -the trust and authorization boundary. - -A coding-agent execution still includes more than a model request. The gateway -must acquire or reuse an immutable workspace base, select a configured agent -target, propagate cancellation, collect bounded evidence, and clean up. -The trusted CI job, VM, or container that runs the gateway is the execution and -secret boundary: provider code and model-invoked tools run with that runner's -authority. The gateway owns one public contract for the lifecycle without -claiming hostile-code containment inside that boundary. - -The contract must not turn AllAgents into an evaluation harness. Dataset -expansion, repetitions, assertions, scoring, experiment scheduling, and durable -evaluation Runs remain consumer concerns. +- Updated: 2026-09-21 ## Decision -### Add a trusted-network execution gateway - -AllAgents will provide an independently testable, separately installed -`allagents-gateway serve` entry point. It runs as one Bun service process that -supervises phase-scoped acquisition and host provider processes. The ordinary -`allagents` CLI does not contain or depend on the gateway. A convenience -dispatcher may locate and execute a separately installed compatible -`allagents-gateway`, but it must not download or embed gateway artifacts. - -The gateway implements A2A protocol version `1.0` over the `HTTP+JSON` binding -plus a required versioned AllAgents coding-execution extension. It owns: - -- stable Task and idempotency identity; -- execution-target selection; -- workspace acquisition; -- deadline and cancellation propagation; -- normalized terminal output and evidence; -- bounded Task and Artifact retention; and -- enforcement of the coding-execution contract across every backend. - -The gateway is not an evaluator, grader, experiment scheduler, retry authority, -or durable evaluation Run ledger. - -### Use one TypeScript/Bun workspace with narrow package boundaries - -The gateway and CLI are TypeScript products in one Bun workspace monorepo. The -private root package owns orchestration only. `apps/cli` publishes `allagents`; -`apps/gateway` publishes `allagents-gateway`; and `apps/acquirer` is built only -as a digest-pinned GHCR image, never as an npm package. Shared code is limited to -three justified packages: - -- `packages/workspace-config` owns the project and user workspace projections - consumed by the CLI and gateway; -- `packages/execution-contracts` owns the A2A coding-execution wire contract and - portable validation; and -- `packages/acquisition-contracts` owns the typed request and manifest exchanged - with the acquisition image. - -Generated, language-portable contract fixtures live under `contracts/`. The -repository does not introduce speculative `core`, `common`, native platform, or -provider-sharing packages. A package is added only for an already-demonstrated -ownership boundary. - -This architecture follows from the deployment boundary. V1 runs on one trusted -Linux CI runner, the gateway and official provider automation surfaces are -available in TypeScript, and Docker is needed only for untrusted repository and -OCI materialization. Adding another gateway implementation runtime and custom -in-job security layer would increase release and operational surface without -creating a boundary inside the already-trusted CI job. - -### Keep independent CLI and gateway release trains - -The `allagents` CLI and `allagents-gateway` have independent versions, tags, and -release triggers. A CLI release publishes only `apps/cli`; installing it fetches -neither the gateway package nor the acquisition image. - -A gateway release first builds the multi-architecture `apps/acquirer` image, -pushes it to GHCR, records the immutable image-index digest and each supported -architecture's manifest digest, and verifies acquisition against those exact -digests. It then packs the exact `apps/gateway` npm tarball and runs package and -registry conformance with that tarball and those image digests. Only after both -artifacts pass does the workflow publish `allagents-gateway`. It does not -publish `allagents`. - -Compatibility is a versioned contract, not equal npm versions. -`allagents-gateway compatibility --format json` reports the product, gateway -version, build identity, acquisition image digest, and supported A2A, -coding-extension, workspace, execution-contract, acquisition-contract, and -snapshot versions. The -optional CLI dispatcher may launch a separately installed gateway only when the -required contract-version intersections are non-empty. Compatibility does not -depend on target-specific npm wrappers or embedded native binaries. - -### Trust the network boundary instead of adding application authentication - -The initial gateway has no application-level authentication or per-caller -authorization. It may bind to loopback, a specific interface, or `0.0.0.0`. -Loopback remains the default when no listen address is supplied, but an explicit -`0.0.0.0` binding is valid and requires no unsafe-mode flag. The Agent Card -advertises a separate absolute interface URL; non-loopback listeners require -that value explicitly because a wildcard bind address is not routable. -Production interfaces use HTTPS; direct HTTP is limited to loopback development. - -Every external host able to reach the listener is equally trusted. Any reachable -caller may invoke every available target, including built-in and gateway-enabled -profile targets; list or retrieve retained Tasks and their Artifacts; and -request cancellation. Task lookup and idempotency are deployment-wide, not -scoped to a caller identity. Operators must use Tailscale ACLs, host firewalls, -container networking, or equivalent network controls when the listener is not -loopback-only. - -Provider processes, MCP servers, and model-invoked tools are not separate -network principals. They execute on the same trusted CI runner as the gateway -and may exercise the authority available to that job. Operators must provision -the runner accordingly and must not rely on AllAgents to isolate host secrets, -the gateway listener, management networks, or arbitrary repository code from -model-invoked tools. - -Gateway-managed TLS termination, OIDC, static bearer tokens, per-tenant -ownership, and multi-tenant information-hiding are deferred. Production clients -reach the advertised HTTPS interface through operator-managed termination or an -encrypted private overlay. Application authentication requires a separate -decision when the service leaves one trusted network boundary. - -### Use existing workspace files as the configuration authority - -The initial gateway has no `gateway.yaml` or `worker.yaml`. - -One gateway process serves one project workspace selected by `--workspace` or -the current directory. The project `.allagents/workspace.yaml` remains -canonical for repository identities, remote sources, destination paths, -default revisions, workspace files, plugins, and named OCI snapshot sources. - -The user `~/.allagents/workspace.yaml` remains canonical for global profiles and -launcher-backed execution targets. A launcher-bearing profile client is gateway- -enabled only when it explicitly declares: - -```yaml -profiles: - review: - clients: - - name: codex - launcher: codex-review - gateway: - enabled: true -``` - -The public target ID is the launcher basename. Launcher names are already -portable and collision-checked across every user profile, while one profile may -contain several clients and therefore several launchers. Internally the target -resolves to exactly one `(profile, client)` pair. The gateway reserves built-in -target IDs, initially `codex` and `pi`; a gateway-enabled launcher whose -portable collision key matches a built-in ID is invalid. - -The built-in `codex` and `pi` targets remain available when their adapters are -ready. Explicit launcher-backed targets add configured variants such as -`codex-review` and `pi-tools`. Initially only Codex and Pi profile clients are -gateway-executable; other launcher-bearing clients become eligible only after a -reviewed adapter implements the common execution contract. - -The generated launcher file is a local UX artifact, not the remote execution -boundary. The gateway never discovers launchers from `PATH`, accepts a command, -executable path, arbitrary arguments, or environment overrides from a request, -or appends request data to a generated launcher. It resolves the profile through -its typed adapter and invokes the provider's supported automation surface. - -Process-level options use exact flags and environment variables for: - -- listener, advertised-interface URL, and workspace selection; -- a project-specific state-directory override; -- disjoint immutable-base cache and per-Task runtime/workspace roots; -- workspace materialization policy plus Task and cache retention limits; -- GitHub App identifiers and private-key file references; -- the configured GitHub CLI account; and -- a strict Docker-auth file or fixed Docker credential-helper executable used - only for acquisition. - -By default the state root is a deterministic child of -`~/.allagents/gateway/` keyed by the canonical project-workspace identity. The -gateway owns an ordinary private Bun SQLite database with transactions, WAL -mode, and full synchronization. It persists claims, Tasks, one execution lease, -internal outcome intent, events, bounded Artifact bytes, and expiry state. The -gateway verifies workspace identity and holds an exclusive process-lifetime -lock. The root is current-user owned, private, and disjoint from project, -profile, and invocation roots. Standard Bun SQLite APIs are the entire storage -layer. The listener exposes metadata-only `/healthz` and `/readyz`; readiness is -false whenever admission is unsafe. - -### Support direct repositories and OCI workspace snapshots - -Each request selects exactly one closed workspace source variant: - -1. `repositories`, which materializes the repositories declared by name in the - project workspace and accepts only optional revision overrides; or -2. `workspaceSnapshot`, which selects a named OCI snapshot repository declared - in the project workspace and supplies an immutable OCI manifest digest plus - the expected AllAgents workspace-manifest digest. - -Fields from another variant are invalid. The gateway does not fall back from an -OCI snapshot to Git repositories, or from Git repositories to a snapshot, after -a Task selects its source mode. - -For direct repositories, callers cannot override repository URLs or destination -paths. A revision override is keyed by a declared repository name. Branches and -tags may be accepted for developer convenience, but the gateway resolves and -records the full commit object ID before provider execution. Reproducibility- -sensitive callers should supply full commit IDs. - -For OCI snapshots, the project workspace declares an operator-selected OCI -Distribution repository and any exact cross-origin layer-blob redirect hosts: - -```yaml -workspaceSnapshots: - evaluation: - repository: ghcr.io/entityprocess/allagents-workspaces - layerRedirectHosts: - - pkg-containers.githubusercontent.com -``` - -The repository field is registry-neutral. V1 must pull AllAgents-formatted -workspace snapshots from Docker Hub, GHCR, JFrog Artifactory/JFrog Container -Registry, and compatible private OCI Distribution registries. Registry choice -does not change the snapshot media types, digest requirements, extraction -rules, or caller-visible source contract. - -Registry conformance is tiered. Every pull request runs local Distribution -fixtures and a live public, digest-pinned GHCR pull through the exact gateway -package under test and the exact acquisition image index and architecture -manifest digests built for that pull request. A release workflow additionally -tests least-privilege authenticated GHCR and a digest-pinned disposable JFrog -Container Registry over HTTPS with a private CA and pull-only identity. Those -release checks install the exact gateway npm tarball and use the exact -multi-architecture acquisition image index and per-architecture manifests -intended for publication, for every architecture the registry and runner -support, without rebuilding either artifact. A report for another commit, -package, image digest, architecture manifest, build identity, or compatibility -output is rejected. Docker Hub behavior remains covered by protocol fixtures to -avoid public rate-limit dependence in pull-request CI. - -The request supplies the name `evaluation`, a `sha256:` OCI image-manifest -digest, and a `sha256:` workspace-manifest digest. The gateway constructs the -full OCI reference server-side. Callers cannot supply a registry host, -repository name, mutable tag, extraction destination, credential, platform -selector, redirect host, or external-layer policy. - -V1 accepts only an OCI Image Manifest directly at the requested digest; image -indexes, descriptor URLs or embedded data, non-distributable layers, and -unknown media types are rejected. Its config is the RFC 8785 canonical -`application/vnd.allagents.workspace-manifest.v1+json` object and must match the -requested workspace-manifest digest. The gateway verifies the manifest body, -config, and every distributable tar/gzip/zstd layer descriptor before decoding, -then applies layers in manifest order with OCI whiteout and opaque-whiteout -semantics. - -Registry metadata remains same-origin. A cross-origin redirect is allowed only -for a layer-blob `GET` or `HEAD` to an exact operator-declared -`layerRedirectHosts` entry; an absent allowlist rejects it. Every bounded HTTPS -hop strips authorization, cookies, and client credentials, rejects URL -credentials, resolves and validates every address at connection time, and -rejects mixed answers, rebinding, downgrade, and unapproved destinations. -Loopback, link-local, private, reserved, or other non-global addresses are -permitted only when their exact host is the source's operator-declared -repository host or layer-redirect host. Token, manifest, and config redirects -remain same-origin. Descriptor size and digest verification remains mandatory -after redirects. - -Both modes produce the same versioned, wire-visible workspace manifest. It -records declared logical repository names, requested revisions, resolved -commits, acquisition kind, relevant OCI manifest and layer digests, the -workspace-manifest digest, completeness, and whether each fact was independently -verified or snapshot-attested. It omits Git URLs, OCI repository origins, and -destination paths. A commit listed inside an OCI snapshot is not described as -independently verified unless the gateway separately verifies it against its -Git remote. - -For a source without a reusable validated base, the host gateway creates a -staging directory and bind-mounts only that directory into the digest-pinned -acquisition image. The acquisition container receives only the selected -repository or registry credential plus the strict network, redirect, size, -file-count, and archive policy needed for that source. The host resolves GitHub -App eligibility and mints any installation token; the container never receives -the App private key, host home directory, provider authentication state, or -Docker socket. The image contains and downloads no Codex, Pi, or other coding -harness. - -The container materializes the repository or OCI source into staging, emits the -typed acquisition manifest, and exits. The gateway removes it before provider -execution, validates the manifest plus paths, collisions, file types, symlinks, -layer and file counts, individual and total compressed and expanded sizes, and -digests, then atomically promotes staging to a validated base. Absolute paths, -traversal, device files, sockets, escaping links, foreign or external OCI -layers, and unapproved cross-origin access are rejected. Every non-publication -path removes staging. Docker has no role after acquisition completes. - -The base-cache key binds the acquisition-contract version, compiled catalog and -layout digest, and immutable source identity: every effective repository commit, -or the OCI manifest and workspace-manifest digests. A repository request is -reusable only when every effective revision is a full commit ID. Mutable -branch or tag requests instead receive a non-reusable Task-owned base that is -removed during settlement or reconciliation. Cache hits mint no credential and -start no acquisition container. Active Tasks pin reusable bases; bounded cache -eviction removes only unpinned entries. - -The request optionally selects `workspaceAccess: "readOnly" | "readWrite"` and -defaults to `readWrite`. A read-only Task resolves its provider cwd directly -inside its validated base; exact immutable requests may share a reusable cached -base, while mutable branch or tag requests own a non-reusable base. Every Task -receives a private runtime directory for temporary, home, provider-state, and -evidence files. The gateway disables optional Git locks and asks the adapter for -its native read-only policy when available. It does not inspect the prompt or -add a per-Task mount, chmod pass, or full-tree verification. Read-only is a -cooperative contract and best-effort provider control, not a hostile-code -boundary; the consumer remains responsible for giving the Task work that does -not require project writes. A violating provider can contaminate a cached base -and later Tasks; the operator must evict that entry before reuse. - -A read-write Task receives a unique writable view under -`//workspace`. The host materializer prefers a -filesystem block clone, falls back to rootless OverlayFS on supported Linux -hosts, and supports an explicit ordinary-copy backend for portability. It never -uses hard links for writable files. The selected materializer is operator -configuration, not request input. After evidence collection, normal settlement -unmounts when needed and removes the Task-owned view plus any non-reusable base; -a non-settling provider retains them with the poisoned execution lease until -reconciliation. - -For either access mode, the provider cwd is resolved from an optional logical -`workingDirectory` selector: - -- `{ kind: "workspaceRoot" }` selects the effective workspace root and is the - default; or -- `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }` - selects a declared repository and an optional validated directory beneath it. - -The caller never supplies an absolute path, configured destination, materializer, -cache key, or physical workspace name. The gateway maps the repository name -through the compiled catalog, resolves the optional relative path, and requires -the result to be an existing directory whose resolved path remains beneath the -selected repository root. The logical selector and access mode are part of the -canonical request, idempotency identity, and integrity evidence. -Gateway-generated structured metadata and operational logs never contain the -physical path; opaque terminal output, native evidence, and produced Artifact -payloads are not sanitized and may contain it. - -### Consume the gateway from Promptfoo through an AI Evals provider - -Rejecting caller-supplied origins does not prevent AI Evals from selecting a -workspace in Promptfoo YAML. The two files have different ownership: - -- the AllAgents project workspace is the operator-controlled catalog that maps - repository and snapshot names to Git URLs, destinations, and OCI repositories; -- the Promptfoo configuration selects a target and source mode. Repository mode - materializes the complete configured repository set and may override - revisions by declared repository name. Snapshot mode selects one declared - snapshot name and supplies immutable digests. - -AI Evals owns a Promptfoo -[custom JavaScript/TypeScript provider](https://www.promptfoo.dev/docs/providers/custom-api/). -It implements `ApiProvider`: its constructor receives `ProviderOptions`, -requires and retains a nonempty `options.id`, validates `options.config`, and -exposes `id()`. -`callApi(prompt, context?, options?)` reads bounded source, working-directory, -and workspace-access test variables from `context?.vars` when present and -cancellation from `options?.abortSignal`. The provider translates one `callApi` -into one A2A Task: it creates and retains a high-entropy invocation key, resolves -the effective logical working-directory selector and `readOnly | readWrite` -access mode, sends one Message whose sole Part has `text` set, declares the -extension in `Message.extensions`, puts the target, closed source union, logical -working directory, and access mode in the matching metadata member, and calls -`SendMessage` with `returnImmediately: true`. -It captures the Task ID and follows terminal state through `SubscribeToTask`, -with `GetTask` and bounded resubscription for races or -disconnects. It returns output, normalized token usage, stable failure metadata, -and logical provenance in Promptfoo's `ProviderResponse`. - -For example, AI Evals can define two provider instances without sending either -origin over the wire: - -```yaml -prompts: - - file://./prompts/coding-task.txt - -sharing: false -evaluateOptions: - maxConcurrency: 1 - cache: false -commandLineOptions: - write: false - share: false - -providers: - - id: file://./providers/allagents-a2a.ts - label: codex-direct - config: - endpoint: https://allagents-gateway.example.internal - target: codex - workingDirectory: - kind: repository - repository: allagents - workspaceAccess: readOnly - source: - kind: repositories - revisions: - allagents: 0123456789abcdef0123456789abcdef01234567 - - - id: file://./providers/allagents-a2a.ts - label: codex-evaluation-snapshot - config: - endpoint: https://allagents-gateway.example.internal - target: codex - workingDirectory: - kind: repository - repository: allagents - workspaceAccess: readWrite - source: - kind: workspaceSnapshot - snapshot: evaluation - digest: sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef - workspaceManifestDigest: sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789 - -tests: - - description: gateway package trial - providers: [codex-direct] - vars: - allagentsWorkingDirectory: - kind: repository - repository: allagents - path: apps/gateway -``` - -The first provider materializes the complete configured repository set and uses -the `allagents` key only to override that repository's revision. The second -provider's `evaluation` key resolves to the declared -`ghcr.io/entityprocess/allagents-workspaces` repository. The gateway enforces -one active invocation transactionally. Promptfoo keeps `maxConcurrency: 1` to -avoid predictably creating failed capacity Tasks; other trusted callers need no -external queue for correctness. The no-cache/no-write/no-share values are secure -defaults for confidential prompts and outputs; consumers may enable persistence -or sharing only after applying their own retention, access, destination, and -redaction policy. - -Static provider config fixes the source kind and logical names and may define a -default logical `workingDirectory` and `workspaceAccess`; absent values default -to `{ kind: "workspaceRoot" }` and `readWrite`. Per-test -`context?.vars?.allagentsWorkingDirectory` may replace the selector, while -`context?.vars?.allagentsWorkspaceAccess` may replace the access mode with the -exact string `readOnly` or `readWrite`. Separate read-only trials may share one -immutable physical base and cwd. Read-write trials receive distinct Task-owned -writable views even when their logical selectors are equal. -`context?.vars?.allagentsSource` remains limited to revision or digest leaves. -Missing variables retain static values. Absolute paths, `.` or `..` segments, -configured destinations, unknown repositories, URLs, mutable revisions, -credentials, commands, materializer choices, and unknown members fail before -provider execution. After Task acceptance, the provider's bounded deadline or -`options?.abortSignal` sends one `CancelTask` using a fresh cleanup signal -rather than the already aborted request signal. -It maps gateway input, output, cached-input, and total token counts to -Promptfoo's `prompt`, `completion`, `cached`, and `total` fields respectively. -Safe stable failure code, retryability, accepted Task ID, other usage, logical -working directory, and Task/Artifact evidence stay in metadata without origins, -configured destinations, or physical paths. Opaque prompts, terminal output, -structured results, native evidence, and produced-Artifact payloads remain -unredacted sensitive data. The provider belongs in AI Evals. AllAgents exposes -the A2A contract and consumer documentation without taking a runtime dependency -on Promptfoo. - -### Resolve GitHub credentials with App-first eligibility fallback - -The source request is credential-free and never selects a credential provider. -For `github.com`, the gateway supports two trusted providers: - -1. a configured GitHub App; and -2. a configured GitHub CLI account. - -The App is preferred when it has an installation covering the configured -repository. Installation applicability has three outcomes: `eligible`, -`ineligible`, and `unknown`. An App-authenticated lookup that returns coverage -is eligible. A 404 is ineligible only after the configured GitHub CLI identity -independently proves that the repository exists; an uncorroborated 404 or any -authentication, permission, rate-limit, timeout, or service ambiguity is -unknown. An explicitly configured installation ID must positively verify -repository coverage. - -For an eligible installation, the gateway bypasses the SDK token cache and -requests a fresh repository-scoped, read-only token for each acquisition. It -validates repository selection, permissions, creation time, and expiry. -Acquisition receives at most 900 seconds or the shorter remaining Task deadline, -and the token must remain valid beyond that sub-budget plus a 60-second clock- -skew margin. The gateway revokes the token after acquisition; unconfirmed -revocation fails before provider execution. - -GitHub CLI is an eligibility fallback only when the App is not configured or -applicability is positively `ineligible`. An `unknown` result caused by -configuration, authentication, rate-limit, permission, or service failure -terminates acquisition. The CLI provider invokes: - -```text -gh auth token --hostname github.com --user -``` - -with `GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and -`GITHUB_ENTERPRISE_TOKEN` removed from its environment. The configured account -is part of the acquisition-policy digest. - -After an App installation is selected, App configuration, authentication, -token minting or validation, permission, repository coverage, revocation, -rate-limit, or service failure terminates acquisition. The gateway never retries -the same Task through the broader GitHub CLI identity. - -Git receives credentials only through an invocation-scoped helper under -hermetic Git configuration inside the acquisition container. The gateway -excludes system, global, and repository credential helpers, Git Credential -Manager, askpass, SSH agents, repository-controlled secondary fetches, and -executable Git configuration. Tokens never appear in clone URLs, command -arguments, Git configuration, logs, Tasks, Artifacts, or retained workspaces. -The helper and token are destroyed and the acquisition container is removed -before provider execution. - -OCI credentials come from either a strict Docker-auth subset that cannot name -executables or a fixed Docker credential helper using its standard `get` -protocol. Acquisition supports anonymous pulls plus same-origin Basic and -Distribution Bearer challenge flows required by the declared registry, with the -documented Docker Hub token-service exception. Any other cross-origin Bearer -realm is rejected before a request is sent. System roots may be supplemented by -an operator map from exact registry `host[:port]` keys to verified PEM bundles; -each bundle is trusted only for connections to its key. Credentials and custom -trust material are scoped to snapshot acquisition and removed before -publication. Public registries require no credential or custom-CA -configuration. - -### Integrate host providers through narrow typed adapters - -The initial backend registry contains Codex and Pi, delivered in that order. -Each adapter implements a narrow AllAgents-owned contract for availability, -capabilities, invocation, progress, deterministic permission handling, abort, -terminal output, optional structured result, usage, native evidence, and -disposal. The gateway does not adopt AI SDK Harnesses or make a third-party -cross-provider abstraction part of its execution contract. - -The Codex adapter uses a pinned `@openai/codex-sdk` release first. App-server is -permitted only when a required, demonstrated capability is absent from that -SDK; convenience or speculative parity is not enough. Native `outputSchema` is -used only for schemas supported by the pinned Structured Outputs contract; -other valid public schemas use explicit JSON guidance and the same gateway-side -validator used by every backend. The adapter does not scrape a TUI or use an -unstable bridge merely to preserve the target name. - -The Pi adapter uses a pinned, supported RPC or package surface with -invocation-owned configuration and a restricted policy extension. Repository -extensions and unrestricted built-ins are not loaded merely because they exist -in acquired source. OMP remains out of the initial registry and is added only -for demonstrated OMP-specific value beyond direct Pi. - -Provider runtimes are installed and pinned as part of the CI runner or gateway -installation; the gateway never downloads them per request. An operator may -select a globally installed binary override only when an exact version and -capability compatibility probe succeeds. Missing controls are reported honestly -as capability gaps, and the gateway never exposes arbitrary installed -executables. - -Codex and Pi execute bare metal, directly on the same trusted Linux CI runner as -the gateway. For a read-only Task, cwd resolves inside its validated base, -shared only when reusable; for a read-write Task, cwd resolves inside the Task's -unique writable view. When -explicit API credentials are absent, Codex reuses the runner's existing -`CODEX_HOME` and ChatGPT login, and Pi reuses its existing supported host -authentication. The gateway references those host paths in place; it does not -copy, mount, or import OAuth files. - -Each provider process receives an explicitly constructed environment containing -only the invocation configuration, selected provider settings, and required -host identity, executable, home, and authentication paths. This reduces -accidental ambient-variable leakage but is not an isolation or secret- -containment claim: MCP servers, provider descendants, and model-invoked tools -may exercise the same CI-job authority and reach secrets available to that -runner. The CI job, VM, or container must therefore be provisioned as the -security boundary. - -Provider preparation is adapter-owned and typed. The gateway never executes -project or user `setup` shell entries as part of acquisition or invocation. -Validated profile settings, plugins, MCP declarations, and deterministic -workspace projections are applied through existing typed transforms. - -### Persist Task truth, not live provider execution - -The gateway durably stores Task identity, the canonical request, idempotency -claim, selected target and source, effective configuration digest, one execution -lease, internal outcome intent, Artifact bytes, retained evidence, cleanup -outcome, and expiry state under the configured state directory. AllAgents-owned -TypeScript handlers expose the A2A contract and use ordinary private Bun SQLite -ownership and transactions with full synchronization. One transaction -arbitrates `createOrReplay`, UUIDv7 Task creation, and execution-lease -acquisition. Normal terminal settlement atomically writes status, result or -failure, evidence, Artifacts, cleanup, and lease release. A provider session is -not a durable recovery checkpoint. - -At most one Task holds the execution lease from acquisition through final -evidence collection. A second otherwise-valid request settles failed with -`execution_capacity_unavailable` without launching an acquisition container or -provider process. An identical idempotency replay returns the existing Task. -Reusing the key with a different canonical request conflicts. Clients generate -at least 128 bits of randomness once per logical invocation and reuse the same -key plus request after an ambiguous transport failure. Because the initial -service has no caller identity, the idempotency namespace and Task visibility -are gateway-wide. - -Terminal Task records, Artifacts, events, and invocation claims expire in one -transaction after the configured TTL. The retained-count limit never evicts an -unexpired Task; the gateway rejects new admission until expiry frees capacity. -State-store integrity or durability failure stops admission and prevents the -gateway from acknowledging creation or reporting terminal success. - -On gateway restart, interrupted nonterminal Tasks settle failed; provider work -is not resumed or automatically replayed. Admission resumes only after any -recorded acquisition container is gone and the recorded provider process group -is confirmed absent. Otherwise the gateway remains unready with the lease held. - -### Make cancellation, evidence, and cleanup explicit - -One durable compare-and-set arbitrates provider terminal outcome, caller -cancellation, deadline, and shutdown as an internal outcome intent while the -externally visible Task remains nonterminal. The winning intent owns the stable -result or failure code and drives one idempotent cancellation and settlement -path. - -On Linux, each direct provider process starts in its own process group. -Cancellation first invokes the provider's supported graceful abort, then sends -`SIGTERM` to the process group after a bounded grace period, and finally sends -`SIGKILL` after a second bounded period. This is best-effort lifecycle control, -not containment: descendants can deliberately detach or escape the group. CI -runner teardown is the final orphan boundary. Cancellation during acquisition -stops and removes the acquisition container and unpublished staging; provider -execution never occurs in that container. - -Live provider events are bounded while execution runs. Filesystem, Git, and -produced-Artifact evidence is collected only after the direct provider process -has settled and the configured process-group escalation has completed. The -gateway does not claim to prove full descendant quiescence. A settled read-only -Task removes its private runtime and any non-reusable base; it retains only a -reusable cached base. A settled read-write Task removes its writable view and -any non-reusable base after evidence collection. One transaction then atomically -publishes terminal status, the integrity Artifact, bounded evidence, result or -failure, produced Artifacts, observed termination and cleanup outcomes, and -lease release. Task-owned cleanup failure publishes `workspace_cleanup_failed` -and retains an internal cleanup record for reconciliation. If the direct process -does not settle after final escalation, the gateway instead publishes -`execution_termination_failed` without filesystem, Git, or produced-Artifact -evidence; retains any Task-owned runtime, writable view, non-reusable base, and -the lease; stops admission; and remains unready until runner teardown and -startup reconciliation confirm the recorded process group is absent and clean -the retained state. -Evidence describes only what the gateway actually observed; -escaped descendants and uncertain cleanup are never upgraded to verified -outcomes. - -V1 supports trusted Linux CI runners and one active invocation. Other operating -systems and concurrent execution require a separate lifecycle design rather -than silent degradation. - -Terminal evidence distinguishes: - -- agent output; -- optional validated structured result; -- logical repository names, requested revisions, resolved commits, or snapshot - names and digests, never source origins or destinations; -- pre- and post-execution Git state where applicable; -- produced artifacts; -- usage and bounded provider-native evidence; -- cancellation and termination outcomes; and -- workspace cleanup outcome. - -Credentials, raw secret-bearing paths, and unrestricted prompt, output, tool, -source, or file contents are excluded from operational logs. - -### Profile A2A instead of inventing an invocation API - -The gateway uses A2A Agent Cards, Messages, Tasks, Artifacts, operations, errors, -streaming, and cancellation. Its Agent Card advertises one interface with -`protocolBinding: "HTTP+JSON"`, `protocolVersion: "1.0"`, and -`capabilities.streaming: true`. The coding- -execution `AgentExtension` is required and has strict -`params: { targets: TargetId[] }`, populated from ready built-in and explicitly -gateway-enabled launcher-backed targets. It does not publish paths, commands, -arguments, environment selectors, credentials, exact source authorization -details, or transient worker state. - -Every A2A HTTP+JSON request carries `A2A-Version: 1.0`. Every operation that -creates, returns, lists, subscribes to, or mutates profiled Tasks or Artifacts -also activates `https://allagents.dev/a2a/extensions/coding-execution/v1` -through `A2A-Extensions`. Missing activation receives -`ExtensionSupportRequiredError`; an unsupported protocol version receives -`VersionNotSupportedError`. Unsuccessful HTTP responses use the A2A -`google.rpc.Status` JSON envelope with typed `google.rpc.ErrorInfo` details; -validation also uses `google.rpc.BadRequest`, never JSON-RPC error carriers. - -The published versioned extension specification defines Agent Card params, -activation, request/idempotency/replay, errors, and terminal Task/Artifact -schemas. Its request carries the invocation key, execution target, closed -workspace source, logical working-directory selector, bounded deadline, and -optional bounded result schema in its own strict `Message.metadata` member -without rejecting unrelated A2A metadata. -The request Message lists the URI in `Message.extensions`. Every terminal Task -has one fixed-name, versioned integrity Artifact whose `Artifact.extensions` -lists the URI, plus zero or more produced Artifacts. Breaking extension versions -receive versioned cards and endpoints rather than silent fallback. - -ACP, app-server, SDK, and RPC protocols remain backend implementation details. -W3C Trace Context may propagate correlation through HTTP and child-process -boundaries. OpenTelemetry and provider-native evidence remain optional, -separate layers; neither replaces durable Task evidence. - -### Keep evaluation commands out of scope - -This decision does not add `allagents eval`, benchmark authoring, assertions, -scoring, datasets, repetitions, experiment scheduling, or automatic execution -retry. Consumers own those concerns. - -## Consequences - -- Developers who explicitly install the gateway package can start one endpoint - with `allagents-gateway serve` and use loopback, `0.0.0.0`, a specific - interface, Tailscale, or firewall policy. -- There is no application authentication, per-caller authorization, tenant - isolation, `gateway.yaml`, `worker.yaml`, remote worker protocol, or required - Kubernetes deployment in the initial product. -- Project and user workspace files remain the sole declaration authority for - source identities and gateway-enabled profile launchers. -- AI Evals can express the configured repository set with named revision - overrides, or select a prebuilt image through a snapshot handle, in Promptfoo - YAML. Its custom provider translates that closed source choice to A2A and - keeps raw origins under AllAgents operator control. -- External network reachability grants access to every available target, - including built-in and gateway-enabled profile targets, plus every retained - Task. Operators treat network policy as authorization and must restrict the - systems and secrets available to the trusted CI runner; AllAgents does not - isolate provider or model-tool descendants within that runner. -- One durable SQLite execution lease enforces one active invocation independent - of consumer concurrency settings. -- GitHub App credentials support private repositories without forcing every - developer to use one identity; GitHub CLI remains a local eligibility fallback - only when no App installation applies. -- Direct repositories and digest-pinned OCI snapshots converge on one validated - immutable-base manifest and evidence contract. Immutable source identities may - reuse a cached base; OCI metadata remains same-origin and only layer blobs may - redirect to exact operator-approved hosts. -- Read-only Tasks may share that base and physical cwd while keeping private - runtime state. Read-write Tasks receive disposable independent writable views - through the selected copy-on-write or copy materializer. -- Docker is a short-lived base-acquisition boundary only when no reusable - validated base exists. The container receives staging plus source credentials, - emits a typed manifest, and is removed before Codex or Pi starts on the host - runner. -- The private Bun workspace root orchestrates `apps/cli`, `apps/gateway`, the - image-only `apps/acquirer`, and the three contract/configuration packages. - CLI-only installs fetch neither the gateway package nor acquisition image. -- CLI and gateway versions and releases remain independent. Gateway releases - verify the exact npm tarball and the exact digest-pinned multi-architecture - acquisition image before publishing. -- Codex and Pi use pinned supported automation surfaces and existing host - authentication through narrow adapters. Explicit environment construction - reduces accidental leakage but cannot hide runner secrets from model-invoked - tools. -- Linux process-group escalation provides bounded best-effort cancellation. - Runner teardown remains the final orphan boundary, and evidence never claims - full descendant quiescence. -- A future deployment configuration becomes justified only when the product - needs multiple worker routes, tenants, credential policies, custom - materializers, centralized storage, or other operator-selected variants. - -## Rejected alternatives - -### Define a second profile registry in `gateway.yaml` - -Rejected because global profiles and launcher identities already belong to -`~/.allagents/workspace.yaml`. A second profile map would drift in client, -model, plugin, MCP, and launcher configuration. - -### Require application authentication for every deployment - -Rejected for the initial trusted-network product. It would add caller identity, -tenant scoping, token lifecycle, and ingress configuration before the expected -users need those boundaries. Tailscale ACLs and firewalls are the initial access -control. - -### Restrict the listener to loopback - -Rejected because developers need to expose the endpoint through Tailscale, -containers, VMs, and private networks. Explicit `0.0.0.0` binding is supported; -the operator owns the surrounding network policy. - -### Execute generated launcher files as the remote protocol - -Rejected because local launchers intentionally preserve cwd and append local -caller arguments. Remote requests must resolve a typed profile adapter and can -never control commands or argv. - -### Let callers provide repository URLs or OCI repositories - -Rejected because workspace configuration already defines trusted source -identities and destinations. Repository requests materialize the configured set -and may override revisions by declared name; snapshot requests select a declared -name and immutable digests. Neither variant introduces a new origin. - -### Let callers provide a host cwd - -Rejected because an absolute or configured destination path would let a caller -select unrelated host content and bypass gateway-owned acquisition. Promptfoo -gets the required runtime control through a logical workspace-root or declared- -repository selector; the gateway maps it into the access-appropriate reusable -or Task-owned base or writable view according to `workspaceAccess`. - -### Always allocate a unique full workspace - -Rejected because read-only Tasks have no mutable project state to isolate, and -copying a large immutable workspace for every trial wastes transfer, storage, -and I/O. They share one validated base. Writable Tasks isolate only their -changes through a disposable copy-on-write view or explicit portable copy. - -### Fall back from a selected GitHub App after runtime failure - -Rejected because it would silently change identity and authorization scope after -selection. GitHub CLI fallback applies only when the App is ineligible. - -### Use mutable OCI tags - -Rejected because the same request could produce different workspaces. Snapshot -selection requires an OCI manifest digest and expected workspace-manifest -digest. - -### Treat provider sessions as durable execution - -Rejected because a resumable provider thread does not prove workspace, -process, cancellation, evidence, or cleanup continuity across gateway restart. - -### Invent a bespoke invocation API - -Rejected because A2A already supplies discovery, Task lifecycle, streaming, -Artifacts, cancellation, and errors. Coding-specific evidence belongs in a -versioned extension. - -### Adopt an evaluator's Job or Trial API - -Rejected because benchmark orchestration, verification, and persisted evaluation -state remain consumer concerns. The gateway executes one coding-agent Task. - -### Build v1 around Rust and kernel containment - -Rejected because the trusted CI job is already the execution boundary. A Rust -gateway plus custom cgroups, pidfds, `openat2` VFS behavior, namespaces, -`nftables`, or spawn mediation would add implementation and release risk without -isolating model-invoked tools from secrets available to that job. Reconsider -native or stronger containment only if hostile-code or in-job secret isolation -becomes a product requirement. - -### Adopt AI SDK Harnesses as the backend abstraction - -Rejected because AllAgents needs a small contract tailored to its A2A Task, -evidence, cancellation, and profile semantics. Depending on a broad -cross-provider abstraction would enlarge the compatibility surface without -removing the need to understand the official Codex and Pi automation APIs. - -### Run shared host provider daemons - -Rejected because a long-lived daemon introduces cross-invocation state, -ownership, cancellation, and authentication ambiguity. V1 starts one direct -provider process for the one active Task and treats provider sessions as -ephemeral. - -### Run providers in per-invocation containers - -Rejected because official Codex and Pi automation should reuse the trusted -runner's existing installation and authentication. Copying or mounting OAuth -state into a provider container complicates ownership without creating a -security boundary against model tools. Docker remains limited to acquisition. - -### Trust ambient unversioned provider binaries - -Rejected because PATH discovery can silently change behavior between runs. -Pinned SDK, RPC, or package surfaces are the default; a global binary override -must pass exact version and capability probes, and runtimes are never downloaded -per request. - -### Couple CLI and gateway versions or publish them together - -Rejected because the products have different dependencies and release cadence. -Compatibility is explicit at the contract boundary; gateway-only work must not -force a CLI release, and CLI-only installation must not fetch gateway or -acquisition artifacts. - -### Bundle the gateway into every CLI installation - -Rejected because plugin/skill-only users do not need the A2A server, SQLite, -provider adapters, or acquisition image. The gateway ships as the separately -installed `allagents-gateway` npm package, and its release independently binds -the digest-pinned GHCR acquisition image. +AllAgents will provide a separately installed gateway that lets trusted tools +start a configured Codex or Pi run remotely and receive its output, usage, file +changes, artifacts, source provenance, and cleanup outcome through A2A. + +AI Evals is the first consumer. The gateway executes one coding-agent run; it +does not own datasets, scoring, assertions, scheduling, retries, or durable +evaluation records. + +Version one serves one AllAgents project workspace on one trusted Linux runner. +Network access controls who can use it, and the runner is the execution and +secret boundary. This is not a sandbox for hostile code or model-invoked tools. + +Implementation details live in the +[coding-agent execution gateway plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). +This ADR records the decisions and their impact. + +## What changes for users + +- **CLI users:** installing `allagents` does not install or start the gateway. +- **Operators:** install `allagents-gateway`, select one existing workspace, and + run `allagents-gateway serve`. Existing project and user workspace files remain + the source of truth. +- **Callers:** choose a configured target, declared workspace source, logical + working directory, `readOnly` or `readWrite` access, a bounded deadline, and an + optional result schema. They cannot provide repository URLs, host paths, + commands, credentials, or environment overrides. +- **AI Evals:** owns the Promptfoo provider and evaluation behavior. AllAgents + owns the gateway contract and documentation. + +## Main flow + +1. The operator starts the gateway for one project workspace. +2. A caller sends an A2A Message with one invocation key. Reusing that key with + the same canonical request returns the same Task; a new key starts a new run. +3. The gateway reuses or prepares a workspace from declared Git repositories or + an immutable OCI snapshot. +4. Codex or Pi runs on the trusted host against a validated read-only base, + shared only for exact immutable requests, or a private disposable read-write + workspace. +5. The gateway streams progress, handles cancellation, records observed evidence, + cleans up Task-owned state, and frees the single execution slot. Uncertainty + about source cleanup, an active mount, or a writable Task view stops admission. + If it cannot confirm provider termination, it retains the slot and stays + unready until teardown confirms the process is gone. + +## Important consequences + +### Network reachability grants full access + +The gateway has no application login, caller identity, tenant isolation, or +per-caller privacy. Loopback is the default, but operators may expose it on a +private interface or `0.0.0.0`. + +Every reachable caller can invoke every available target, inspect every retained +Task and Artifact, and request cancellation. Non-loopback exposure requires +operator-managed HTTPS and network access controls such as Tailscale ACLs, +firewalls, or container networking. Direct HTTP is limited to loopback use. + +Providers and model-invoked tools may use the runner's credentials, secrets, and +network access. Explicit environments reduce accidental leakage but do not +create isolation. Application authentication and multi-tenant ownership are +deferred until the service must leave one trusted network. + +### Callers choose logical work, not infrastructure + +The gateway reuses existing workspace configuration; it does not add +`gateway.yaml` or `worker.yaml`. Built-in Codex and Pi targets are available when +ready. Profile-backed targets require explicit gateway enablement. + +A request chooses either the complete configured repository set, with optional +revision overrides by declared name, or one declared OCI snapshot selected by +immutable digests. It never falls back between those modes. The operator owns +origins, destinations, credentials, and registry policy. Credential routing +prefers a proven applicable GitHub App and uses the configured `gh` account only +when the App is absent or positively ineligible. Ambiguity or failure after +selection never falls back to a broader identity. + +The caller selects the workspace root or a directory beneath a declared +repository, never a physical host path. Invalid or escaping paths fail before +provider execution. + +### Read-only is an optimization, not a security boundary + +`readWrite` is the default and gives each Task a private disposable workspace. +`readOnly` uses a validated base: exact immutable requests may share a reusable +base, while mutable revisions get a Task-owned, non-reusable base. Every Task +still gets disposable private provider, temporary, and evidence state. + +Read-only enforcement is cooperative. A provider that writes anyway can +contaminate the shared workspace and later Tasks; the operator must then evict +that workspace before reuse. + +Workspace preparation is isolated from provider execution and receives only the +source credential and network access it needs. That credential and preparation +environment are gone before Codex or Pi starts. + +### Providers run directly on the trusted host + +Codex and Pi use supported, pinned integrations and existing host authentication. +The gateway does not run workspace `setup` shell entries, download a provider +runtime for each request, execute generated launcher files remotely, or expose +arbitrary installed executables. + +If a supported integration lacks a required control, that target is unavailable +rather than silently weakening the public contract. + +### One Task runs at a time + +One execution slot covers workspace preparation, provider execution, evidence, +and cleanup. A second valid request becomes a failed Task with +`execution_capacity_unavailable`; it starts no workspace or provider work. + +Task identity, retries, and visibility are shared across the gateway. Reusing an +invocation key for a different request conflicts. + +Terminal Tasks and evidence are retained within configured limits. Unexpired +Tasks are not deleted to make room for new work. Prompts, output, structured +results, native evidence, and produced Artifacts remain sensitive and +unredacted. +Operational logs exclude credential values, secret-bearing paths, and +unrestricted prompt, output, tool, source, and file content. + +### A2A provides the lifecycle; AllAgents defines coding execution + +A2A 1.0 over HTTP+JSON provides discovery, Messages, Tasks, streaming, Artifacts, +cancellation, and errors. A required, versioned AllAgents extension adds targets, +workspace selection, idempotency, provenance, and evidence. Callers send the A2A +version and activate the extension on every operation that creates, returns, +lists, subscribes to, or mutates profiled Tasks or Artifacts. Missing extension +support and unsupported versions use standard A2A errors. Breaking changes use a +new extension version rather than silent fallback. + +The known conformance question is authentication. A2A 1.0 says servers +authenticate requests and authorization-scope Task operations, while this design +has no application identity and treats every reachable caller as one authority +domain. Agent Card security declarations are optional, so the anonymous case is +not explicit. The implementation feasibility gate must resolve this before the +gateway claims full A2A 1.0 conformance. If it cannot, this ADR must be amended; +the implementation must not silently add authentication or weaken the +conformance claim. + +### The gateway remains a separate product + +`allagents` and `allagents-gateway` have independent versions and release +cadence. Installing the CLI fetches neither the gateway nor its workspace- +preparation image. Compatibility comes from versioned contracts, not matching +package versions. + +Every terminal Task has exactly one versioned, extension-marked +execution-integrity Artifact, plus any produced Artifacts, even after failure or +cancellation. Consumers remain +responsible for evaluation workflows, retries, retention, sharing, and redaction. + +## Failure behavior + +- **Busy:** the new Task fails with `execution_capacity_unavailable`; no work + starts. +- **Malformed source or working-directory input:** admission fails with + `invalid_execution_request`; no Task is created. +- **Accepted source, credential, or logical-directory resolution then fails:** + the Task fails with the corresponding stable error code and does not switch + source mode or credential identity. +- **Gateway restart:** interrupted Tasks fail. Provider work is not resumed or + automatically replayed. +- **Provider cannot be stopped:** the Task reports + `execution_termination_failed` with live-provider and observed termination + evidence, but no filesystem, Git, or produced-Artifact evidence. It retains + its workspace and execution slot and leaves the gateway unready until teardown + confirms the process is gone. +- **Workspace cleanup fails:** the Task reports `workspace_cleanup_failed` and + retains enough state for later cleanup. +- **Durable Task state is unsafe:** the gateway stops admitting work and does not + acknowledge creation or report success it cannot preserve. + +Evidence describes only what the gateway observed. It never presents uncertain +termination, cleanup, provenance, or file state as verified. + +## Deliberate limits + +Version one deliberately avoids: + +- application authentication, tenants, caller-private Tasks, and public-Internet + hardening because the initial product assumes one trusted network; +- queues, concurrent execution, replicas, remote workers, shared provider + daemons, and resumed provider sessions because one durable Task lifecycle is + the initial boundary; +- caller-provided origins, physical host paths, configured destinations, + commands, credentials, environments, or materializers because the gateway is + not a remote shell; +- hostile-code containment and per-provider containers because the runner is + already the execution and secret boundary; +- a bespoke or evaluator-specific API because A2A already owns the remote Task + lifecycle; +- a second configuration registry because workspace files already own sources + and profiles; and +- bundling or version-locking the gateway with the CLI because most CLI users do + not need the service and the products have different release cadence. ## Reconsider when -Revisit this decision when any of these become requirements: - -- callers outside one trusted network must share the endpoint; -- per-caller Task privacy, authorization, or audit identity is required; -- provider or model-tool code must be isolated from runner secrets or treated as - hostile inside the execution environment; -- multiple gateway replicas need transactional shared storage; -- more than one active invocation or shared provider daemons are required; -- execution must route among remote worker pools or sandboxes; -- non-Linux runners need equivalent lifecycle and cancellation semantics; -- custom materializers are needed beyond direct Git and OCI snapshots; -- multiple GitHub hosts, Apps, CLI accounts, or ordered credential policies need - declarative configuration; -- A2A standardizes the required coding-execution evidence without an extension; - or -- a stable cross-vendor automation protocol subsumes the backend adapter seam. +Revisit this decision when: + +- callers outside one trusted network must share the service; +- callers need private Tasks, distinct authorization, or audit identity; +- provider or model-tool code must be isolated from runner secrets; +- the gateway needs concurrency, replicas, shared daemons, or remote workers; +- non-Linux runners, new source materializers, or richer credential routing are + required; +- A2A standardizes the coding-execution fields now carried by the AllAgents + extension; or +- a stable cross-vendor protocol replaces the Codex/Pi adapter seam. From a0894a6b57b0507e96079c4ec438221b56ab28cd Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Tue, 22 Sep 2026 07:19:01 +1000 Subject: [PATCH 13/44] docs(architecture): define harness execution contract --- ...-agent-execution-through-an-a2a-gateway.md | 259 +- ...0837-feat-coding-execution-gateway-plan.md | 2576 +++++++++++------ 2 files changed, 1833 insertions(+), 1002 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index 4d087379..cebd114f 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -10,9 +10,9 @@ AllAgents will provide a separately installed gateway that lets trusted tools start a configured Codex or Pi run remotely and receive its output, usage, file changes, artifacts, source provenance, and cleanup outcome through A2A. -AI Evals is the first consumer. The gateway executes one coding-agent run; it -does not own datasets, scoring, assertions, scheduling, retries, or durable -evaluation records. +AI Evals is the first consumer. The gateway executes one coding-agent turn at a +time; it does not own datasets, scoring, assertions, scheduling, retries, or +durable evaluation records. Version one serves one AllAgents project workspace on one trusted Linux runner. Network access controls who can use it, and the runner is the execution and @@ -29,45 +29,58 @@ This ADR records the decisions and their impact. run `allagents-gateway serve`. Existing project and user workspace files remain the source of truth. - **Callers:** choose a configured target, declared workspace source, logical - working directory, `readOnly` or `readWrite` access, a bounded deadline, and an - optional result schema. They cannot provide repository URLs, host paths, - commands, credentials, or environment overrides. -- **AI Evals:** owns the Promptfoo provider and evaluation behavior. AllAgents - owns the gateway contract and documentation. + working directory, `readOnly` or `readWrite` access, a bounded deadline, + optional result schema, and one-shot/start/resume session mode. They cannot + provide repository URLs, host paths, commands, credentials, or environment + overrides. +- **AI Evals:** owns the Promptfoo provider, turn chaining, and evaluation + behavior. AllAgents preserves provider/workspace session continuity and + reports native cached-input usage, but does not guarantee a model cache hit. ## Main flow 1. The operator starts the gateway for one project workspace. -2. A caller sends an A2A Message with one invocation key. Reusing that key with - the same canonical request returns the same Task; a new key starts a new run. +2. A caller sends an A2A Message with one invocation key for one turn. It may run + one-shot, start a session, or resume the exact head Task in an existing + context. Reusing the key with the same canonical request returns the same + Task; each intentional new turn uses a new key and immutable Task. 3. The gateway reuses or prepares a workspace from declared Git repositories or - an immutable OCI snapshot. -4. Codex or Pi runs on the trusted host against a validated read-only base, - shared only for exact immutable requests, or a private disposable read-write - workspace. -5. The gateway streams progress, handles cancellation, records observed evidence, - cleans up Task-owned state, and frees the single execution slot. Uncertainty - about source cleanup, an active mount, or a writable Task view stops admission. - If it cannot confirm provider termination, it retains the slot and stays - unready until teardown confirms the process is gone. + an immutable OCI snapshot. A resumed read-write session derives a per-turn + candidate from its last committed workspace generation. +4. Codex or Pi runs on the trusted host against a validated read-only base or the + private candidate. A session resumes the provider-native conversation. +5. The gateway streams progress, handles cancellation, and records observed + evidence. A successful continuing turn atomically advances the provider + checkpoint, committed workspace generation, and session head; a one-shot or + closing turn cleans up. Cleanup/checkpoint uncertainty, a live provider + process group, or an observed escaped/outliving descendant retains the + execution slot and makes the gateway unready. Process groups do not prove that + an unobserved hostile descendant cannot escape the trusted runner boundary. ## Important consequences ### Network reachability grants full access The gateway has no application login, caller identity, tenant isolation, or -per-caller privacy. Loopback is the default, but operators may expose it on a -private interface or `0.0.0.0`. +per-caller privacy. Loopback is the default. Version one supports two enforceable +private topologies: +- bind loopback HTTP and expose it only through a private HTTPS terminator such + as Tailscale Serve; or +- bind one specific loopback, RFC 1918, RFC 4193, link-local, or RFC 6598 + address and serve TLS itself from configured certificate/key files. + +Wildcard and public-address listeners are rejected. The advertised URL is +loopback HTTP for local development or private HTTPS for either remote topology. Every reachable caller can invoke every available target, inspect every retained -Task and Artifact, and request cancellation. Non-loopback exposure requires -operator-managed HTTPS and network access controls such as Tailscale ACLs, -firewalls, or container networking. Direct HTTP is limited to loopback use. +Task and Artifact, resume every retained conversation, and request cancellation. +Operators must also enforce Tailscale ACLs, private firewall rules, or equivalent +network policy. Version one must not be exposed to the public Internet. Providers and model-invoked tools may use the runner's credentials, secrets, and network access. Explicit environments reduce accidental leakage but do not -create isolation. Application authentication and multi-tenant ownership are -deferred until the service must leave one trusted network. +create isolation. Public-Internet exposure requires application authentication, +authorization, and an amended ADR before deployment. ### Callers choose logical work, not infrastructure @@ -89,10 +102,12 @@ provider execution. ### Read-only is an optimization, not a security boundary -`readWrite` is the default and gives each Task a private disposable workspace. -`readOnly` uses a validated base: exact immutable requests may share a reusable -base, while mutable revisions get a Task-owned, non-reusable base. Every Task -still gets disposable private provider, temporary, and evidence state. +`readWrite` is the default. A one-shot Task gets a disposable workspace. A +read-write session keeps an immutable committed generation and runs each turn in +a candidate that becomes committed only with the provider checkpoint/session +head. `readOnly` uses a validated base: exact immutable requests may share one, +while mutable revisions get a Task/session-owned non-reusable base. Every +one-shot Task/session still gets private provider, temporary, and evidence state. Read-only enforcement is cooperative. A provider that writes anyway can contaminate the shared workspace and later Tasks; the operator must then evict @@ -104,22 +119,31 @@ environment are gone before Codex or Pi starts. ### Providers run directly on the trusted host -Codex and Pi use supported, pinned integrations and existing host authentication. -The gateway does not run workspace `setup` shell entries, download a provider -runtime for each request, execute generated launcher files remotely, or expose -arbitrary installed executables. +Codex and Pi use supported, pinned integrations. Mutable provider/session state +lives in Task/session-private storage. Existing host authentication is reusable +only when the pinned integration can reference it separately without copying, +mounting, parsing, or writing conversation state into the auth location. +Otherwise that auth/target/mode combination is unavailable. The gateway does not +run workspace `setup` shell entries, download a provider runtime for each +request, execute generated launcher files remotely, or expose arbitrary +installed executables. It never silently weakens its advertised contract. -If a supported integration lacks a required control, that target is unavailable -rather than silently weakening the public contract. +### One gateway-controlled Task runs at a time -### One Task runs at a time +One execution slot covers workspace preparation, one session turn, evidence, and +cleanup/checkpointing. A second valid request becomes a failed Task with +`execution_capacity_unavailable`; it starts no gateway-controlled workspace or +provider work. Sessions never run concurrent turns. -One execution slot covers workspace preparation, provider execution, evidence, -and cleanup. A second valid request becomes a failed Task with -`execution_capacity_unavailable`; it starts no workspace or provider work. +This is an admission and supervision invariant, not hostile-process containment. +An observed escaped/outliving descendant retains the slot until verified gone or +runner teardown. An unobserved descendant can outlive a turn because version one +does not provide a non-bypassable process boundary. Task identity, retries, and visibility are shared across the gateway. Reusing an -invocation key for a different request conflicts. +invocation key for a different request conflicts. Each turn is a new immutable +Task; the session persists conversation and committed workspace state between +turns. Terminal Tasks and evidence are retained within configured limits. Unexpired Tasks are not deleted to make room for new work. Prompts, output, structured @@ -128,24 +152,80 @@ unredacted. Operational logs exclude credential values, secret-bearing paths, and unrestricted prompt, output, tool, source, and file content. -### A2A provides the lifecycle; AllAgents defines coding execution - -A2A 1.0 over HTTP+JSON provides discovery, Messages, Tasks, streaming, Artifacts, -cancellation, and errors. A required, versioned AllAgents extension adds targets, -workspace selection, idempotency, provenance, and evidence. Callers send the A2A -version and activate the extension on every operation that creates, returns, -lists, subscribes to, or mutates profiled Tasks or Artifacts. Missing extension -support and unsupported versions use standard A2A errors. Breaking changes use a -new extension version rather than silent fallback. - -The known conformance question is authentication. A2A 1.0 says servers -authenticate requests and authorization-scope Task operations, while this design -has no application identity and treats every reachable caller as one authority -domain. Agent Card security declarations are optional, so the anonymous case is -not explicit. The implementation feasibility gate must resolve this before the -gateway claims full A2A 1.0 conformance. If it cannot, this ADR must be amended; -the implementation must not silently add authentication or weaken the -conformance claim. +### The Harness Execution Contract owns semantics; A2A is the first binding + +The caller may be an evaluation runner, chat platform, application, or another +agent. The shared domain is therefore not agent-to-agent collaboration or one +vendor's HTTP shape; it is configured harness execution. + +AllAgents defines a transport-neutral, versioned Harness Execution Contract (HEC). +Its Core conformance class covers one turn: configured target selection, +idempotency, deadlines, ordered progress, normalized tool-call/result trajectory, +cancellation, terminal result, usage, stable failures, and artifacts. The +version-one Sessions extension adds a durable session identity, ordered turns, +provider-native conversation resumption, a retained workspace, expiry, and +close-after-turn cleanup. One-shot Core invocation remains available when a +caller does not request a session. + +Protocol bindings map that contract onto an existing transport without changing +its semantics. Core owns neutral outcomes such as `timedOut` and the cancel +dispositions `accepted | alreadyTerminal`; each binding maps those outcomes to +its own state and response vocabulary. Every binding must pass the same Core +semantic vectors plus its own wire-conformance cases. A2A 1.0 over HTTP+JSON is +the first binding: discovery uses an Agent Card, one turn becomes one Message +and one immutable Task, ordered execution events become Task updates and +Artifacts, and A2A owns streaming, retrieval, cancellation, and transport +errors. A2A `contextId` identifies the durable session; a resumed turn uses the +same context and references the prior terminal Task. An A2A Client may be an +application or an agent; neither side needs autonomous multi-agent behavior. + +The AllAgents coding-workspace extension remains separate from Core and +Sessions. It defines configured Git/OCI sources, logical working directories, +access mode, provenance, produced files, integrity, and cleanup. A session pins +those inputs and retains its private provider state and workspace until +close-after-turn, expiry, or verified operator cleanup. Version one uses the +non-dereferenceable Profile Extension identifier +`urn:allagents:a2a:profile:coding-execution:v1`, which composes the HEC A2A +binding, Sessions, and the coding-workspace extension. This URN identifies a +contract; it is not a network endpoint. Their schemas and conformance groups +remain independently validatable. Breaking changes use a new URN rather than +silent fallback. + +This is a contract and standard candidate, not a claimed neutral standard. +AllAgents should describe it as a standard only after independent +implementations, multiple bindings, executable cross-binding conformance, and +neutral governance exist. + +The known A2A-binding conformance question is authentication. A2A 1.0 says +servers authenticate requests and authorization-scope Task operations, while +this design has no application identity and treats every reachable caller as +one authority domain. Agent Card security declarations are optional, so the +anonymous case is not explicit. The implementation feasibility gate must +resolve this before the gateway claims full A2A 1.0 conformance. If it cannot, +this ADR must be amended; the implementation must not silently add +authentication or weaken the conformance claim. + +### UHP informs a future Responses binding but is not the version-one wire + +The Unified Harness Protocol (UHP) is a close semantic match for +application-to-harness execution. Its Responses-compatible request shape, +ordered tool-call/result output, durable session with conversation and working +directory continuity, exact `previous_response_id` predecessor chaining, +lifecycle vocabulary, and executable conformance suite are design inputs for +Core, Sessions, and a possible future Responses/UHP binding. + +UHP is not adopted as the version-one wire because its conformant core also +assumes application authentication, principal scoping, and concurrent work +across sessions. Those are appropriate for a hosted, multi-user harness router +but conflict with this gateway's trusted-private-network and one-execution-slot +boundary. UHP also leaves the AllAgents-specific Git/OCI acquisition and +integrity evidence contract to an extension, so it would not eliminate the +domain contract this project must own. + +Version one exposes only the A2A binding. The implementation may reuse UHP +semantics, not UHP wire claims. It must not advertise UHP compatibility without +implementing a defined binding and passing the applicable UHP and +Harness Execution Contract conformance suites. ### The gateway remains a separate product @@ -154,10 +234,11 @@ cadence. Installing the CLI fetches neither the gateway nor its workspace- preparation image. Compatibility comes from versioned contracts, not matching package versions. -Every terminal Task has exactly one versioned, extension-marked -execution-integrity Artifact, plus any produced Artifacts, even after failure or -cancellation. Consumers remain -responsible for evaluation workflows, retries, retention, sharing, and redaction. +Every terminal Task has exactly one versioned Core outcome Artifact, one ordered +normalized Core execution-trajectory Artifact, one AllAgents workspace- +integrity Artifact, and any produced Artifacts, even after failure or +cancellation. Consumers remain responsible for evaluation workflows, retries, +retention, sharing, and redaction. ## Failure behavior @@ -168,17 +249,20 @@ responsible for evaluation workflows, retries, retention, sharing, and redaction - **Accepted source, credential, or logical-directory resolution then fails:** the Task fails with the corresponding stable error code and does not switch source mode or credential identity. -- **Gateway restart:** interrupted Tasks fail. Provider work is not resumed or - automatically replayed. +- **Gateway restart:** interrupted Tasks fail and are never replayed. A retained + session remains resumable only after the gateway confirms the recorded process + group and observed descendants are gone, discards any uncommitted workspace + candidate, and verifies the prior provider/workspace checkpoint pair; + otherwise it becomes non-resumable pending cleanup. - **Provider cannot be stopped:** the Task reports `execution_termination_failed` with live-provider and observed termination - evidence, but no filesystem, Git, or produced-Artifact evidence. It retains - its workspace and execution slot and leaves the gateway unready until teardown - confirms the process is gone. + evidence, but no filesystem, Git, or produced-Artifact evidence. A live process + group or observed outliving descendant retains its workspace/execution slot + and leaves the gateway unready until verified disappearance or runner teardown. - **Workspace cleanup fails:** the Task reports `workspace_cleanup_failed` and retains enough state for later cleanup. -- **Durable Task state is unsafe:** the gateway stops admitting work and does not - acknowledge creation or report success it cannot preserve. +- **Durable gateway state is unsafe:** the gateway stops admitting work and does + not acknowledge creation or report success it cannot preserve. Evidence describes only what the gateway observed. It never presents uncertain termination, cleanup, provenance, or file state as verified. @@ -188,17 +272,20 @@ termination, cleanup, provenance, or file state as verified. Version one deliberately avoids: - application authentication, tenants, caller-private Tasks, and public-Internet - hardening because the initial product assumes one trusted network; -- queues, concurrent execution, replicas, remote workers, shared provider - daemons, and resumed provider sessions because one durable Task lifecycle is - the initial boundary; + exposure because version one is restricted to one trusted private network; +- queues, concurrent admission, replicas, remote workers, and shared provider + daemons because the gateway supervises one admitted turn at a time; - caller-provided origins, physical host paths, configured destinations, commands, credentials, environments, or materializers because the gateway is not a remote shell; - hostile-code containment and per-provider containers because the runner is - already the execution and secret boundary; -- a bespoke or evaluator-specific API because A2A already owns the remote Task - lifecycle; + already the execution/secret boundary. An unobserved escaped descendant can + overlap a later admitted turn; operators needing OS-wide exclusivity must use + an ephemeral runner boundary or wait for a future containment design; +- a bespoke or evaluator-specific API because HEC owns harness semantics and A2A + already owns the version-one remote Task and multi-turn context lifecycle; +- a second wire binding because version one proves Core and Sessions through + A2A before adding Responses/UHP; - a second configuration registry because workspace files already own sources and profiles; and - bundling or version-locking the gateway with the CLI because most CLI users do @@ -208,12 +295,22 @@ Version one deliberately avoids: Revisit this decision when: -- callers outside one trusted network must share the service; -- callers need private Tasks, distinct authorization, or audit identity; +- callers outside one trusted private network must share the service, which + requires application authentication and authorization before exposure; - provider or model-tool code must be isolated from runner secrets; +- OS-wide turn exclusivity is required, which needs a non-bypassable provider + lifecycle boundary before the gateway may admit a later turn; - the gateway needs concurrency, replicas, shared daemons, or remote workers; - non-Linux runners, new source materializers, or richer credential routing are required; -- A2A standardizes the coding-execution fields now carried by the AllAgents - extension; or +- another independent implementation needs the Harness Execution Contract or a + second binding, at which point both must pass the shared Core conformance + vectors without changing Core semantics; +- session branching or concurrent turns, which require explicit fork semantics + beyond the version-one linear session history; +- A2A standardizes equivalent portable harness-execution semantics that should + replace or upstream the AllAgents binding; +- UHP gains neutral multi-vendor governance, several independent conformant + implementations, and a one-shot conformance class suitable for a + Responses/UHP binding; or - a stable cross-vendor protocol replaces the Codex/Pi adapter seam. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index d6db4a43..ee7dbf30 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,7 +1,7 @@ --- title: "Coding-Agent Execution Gateway - Plan" date: 2026-09-18 -updated: 2026-09-20 +updated: 2026-09-21 type: feat artifact_contract: ce-unified-plan/v1 artifact_readiness: implementation-ready @@ -22,33 +22,40 @@ execution: code - **Means:** Convert the repository to a private Bun workspace monorepo with independently released `allagents` and `allagents-gateway` applications, versioned workspace/execution/acquisition contract packages, generated - portable fixtures under `contracts/`, a bounded SQLite Task store, direct - Codex and Pi host-process adapters, and one digest-pinned acquisition image - used only when no reusable validated base exists. Read-only Tasks share a - reusable base or own a non-reusable base for mutable revisions; read-write - Tasks receive disposable writable views through an automatic block-clone/ - OverlayFS materializer with an explicit portable copy backend. + portable fixtures under `contracts/`, a bounded SQLite Task/session store, + direct Codex and Pi host-process adapters, and one digest-pinned acquisition + image used only when no reusable validated base exists. One-shot read-only + Tasks share a reusable base or own a non-reusable base for mutable revisions; + sessions pin a base and retain private runtime/workspace state; one-shot + read-write Tasks receive disposable writable views through an automatic + block-clone/OverlayFS materializer with an explicit portable copy backend. Providers still run bare metal on the trusted Linux CI runner. - **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) - owns the public, trust, runtime, and packaging boundaries. Project and user - `workspace.yaml` files own source and profile declarations. A2A 1.0 owns core - wire semantics. The CI job, VM, or deployment container is the only - operational execution and isolation boundary. AllAgents does not claim that - boundary contains hostile code or hides job secrets from model-invoked tools; - it owns process lifecycle and truthful evidence only. + owns the wire, trust, runtime, and packaging boundaries. Project and user + `workspace.yaml` files own source and profile declarations. The + transport-neutral Harness Execution Contract (HEC) Core and Sessions own + deterministic per-turn and continuation semantics; their A2A binding owns the + version-one wire mapping; the AllAgents coding-workspace extension owns + Git/OCI workspace semantics. + The CI job, VM, or deployment container is the only operational execution + and isolation boundary. AllAgents does not claim that boundary contains + hostile code or hides job secrets from model-invoked tools; it owns process + lifecycle and truthful evidence only. - **Execution order:** Capture red CLI-only and standalone-gateway package smokes; complete the Bun monorepo, A2A SDK, provider-surface, process-group, Docker-acquirer, package, and release feasibility gate; freeze schemas, generated fixtures, configuration projection, SQLite ownership, and release - binding; implement the Task store and A2A server, host supervisor, Docker-only - acquisition, Codex, and Pi; run final review; then run green packed-package, - exact-image, registry-conformance, repository, and documentation gates. + binding; implement the Task/session store and A2A server, host supervisor, + Docker-only acquisition, Codex, and Pi; run final review; then run green packed- + package, exact-image, registry-conformance, repository, and documentation + gates. - **Stop conditions:** Stop dependent production work if the pinned A2A surface - cannot implement the required public protocol or if the acquisition-container - boundary and exact image/package release binding are infeasible. A missing - Codex or Pi capability makes that target unavailable rather than changing the - A2A, trust, source, or Task contracts. Do not add application authentication, - deployment YAML, remote workers, caller-supplied origins/commands, mutable OCI + cannot implement the required private-network protocol or if the acquisition- + container boundary and exact image/package release binding are infeasible. + A missing Codex or Pi capability makes that target unavailable rather than + changing the A2A, trust, source, or Task/session contracts. Do not add + application authentication, deployment YAML, remote workers, caller-supplied + origins/commands, mutable OCI tags, provider fallback, evaluation behavior, automatic retries, per-provider Docker, or containment claims. - **Tail ownership:** The implementing workflow runs focused contract, @@ -64,16 +71,22 @@ execution: code ### Summary AllAgents gains a single-workspace coding-execution service without becoming an -evaluation framework or multi-tenant platform. Callers use A2A Tasks and one -required AllAgents extension. Network reachability is authorization. The -service resolves configured targets and sources from existing workspace files, -acquires or reuses an immutable base, shares it for read-only Tasks, creates an -independent disposable view for read-write Tasks, invokes Codex or Pi through a -typed adapter, and retains bounded terminal evidence. AI Evals consumes that -boundary through its own Promptfoo custom provider: evaluation YAML supplies -named revision overrides for the configured repository set, or one snapshot -handle and immutable digests, while AllAgents retains origin and credential -authority. +evaluation framework or multi-tenant platform. A transport-neutral Harness +Execution Contract defines one turn's invocation, ordered tool trajectory, +results, usage, cancellation, failures, and artifacts plus durable resumable +Sessions for evaluation runners, chat platforms, applications, and other agents. +The gateway exposes its first binding through A2A Tasks and one required +AllAgents coding-execution Profile Extension. That profile composes the A2A +binding, Sessions, and the AllAgents-specific coding-workspace extension while +preserving separate schemas and conformance groups. +Network reachability is authorization. The service resolves configured targets +and sources from existing workspace files, acquires or reuses an immutable base, +shares it for read-only Tasks, creates an independent disposable view for +read-write Tasks, invokes Codex or Pi through a typed adapter, and retains +bounded terminal evidence. AI Evals consumes that boundary through its own +Promptfoo custom provider: evaluation YAML supplies named revision overrides +for the configured repository set, or one snapshot handle and immutable +digests, while AllAgents retains origin and credential authority. ### Problem Frame @@ -83,16 +96,20 @@ internals, drive interactive CLIs, or duplicate profile resolution, repository acquisition, credential handling, cancellation, evidence capture, and cleanup. Developers expect a process they can start in a workspace and expose on -loopback, `0.0.0.0`, a private interface, or Tailscale. They do not need an -application authentication stack, Kubernetes control plane, remote worker -registry, or another profile configuration file for the initial use case. +loopback or a trusted private network such as Tailscale. Remote access uses +either a loopback backend behind a private TLS terminator or one specific private +TLS listener; wildcard/public listeners are rejected. Version one is never a +public-Internet service. They do not need an application +authentication stack, Kubernetes control plane, remote worker registry, or +another profile configuration file for the initial use case. ### Actors -- A1. **Trusted-network caller:** Any process able to reach the endpoint. All - callers have the same authority and Task visibility. The first caller is an - AI Evals-owned Promptfoo custom provider that maps one `callApi` to one Task. -- A2. **Execution gateway:** The A2A server and invocation supervisor. It owns +- A1. **Trusted-network caller:** An evaluation runner, chat platform, + application, or agent able to reach the endpoint. All callers have the same + authority and Task visibility. The first caller is an AI Evals-owned + Promptfoo custom provider that maps one `callApi` to one execution. +- A2. **Execution gateway:** The A2A binding and invocation supervisor. It owns deployment-wide Task identity, acquisition, routing, status, cancellation, evidence, retention, and cleanup. - A3. **Backend adapter:** The Codex or Pi implementation translating native @@ -105,12 +122,21 @@ registry, or another profile configuration file for the initial use case. ### Key Decisions -- **Use A2A rather than inventing an invocation API.** Standard Agent Cards, - Tasks, Artifacts, errors, streaming, and cancellation remain the public - lifecycle. Governs R1-R3. -- **Treat the network as the trust boundary.** The initial service has no - application authentication or caller ownership. Explicit `0.0.0.0` binding is - valid. (session-settled: user-directed.) Governs R4-R5. +- **Define one harness contract with A2A as its first binding.** The + transport-neutral Harness Execution Contract Core owns one turn's invocation, + idempotency, ordered progress and tool trajectory, cancellation, result, + usage, stable failures, and artifacts. The version-one Sessions extension owns + durable linear conversation identity, provider-native resumption, retained + workspace state, expiry, and close-after-turn cleanup. Standard A2A Agent + Cards, Messages, Tasks, context IDs, task references, Artifacts, errors, + streaming, and cancellation carry those semantics. An evaluation runner, chat + platform, application, or agent uses the same contracts; Promptfoo does not + masquerade as an autonomous agent. Governs R1-R3. +- **Treat the private network as the trust boundary.** The initial service has + no application authentication or caller ownership. Remote access requires a + loopback backend behind private TLS ingress or a specific private-address TLS + listener; wildcard/public listeners and public exposure are prohibited. + Governs R4-R5. - **Reuse workspace configuration.** Project `workspace.yaml` owns sources; user `workspace.yaml` owns profiles, launchers, and gateway enablement. There is no `gateway.yaml`. (session-settled: user-directed.) Governs R6-R8, R18. @@ -118,11 +144,14 @@ registry, or another profile configuration file for the initial use case. digest-pinned OCI workspace snapshots converge on one manifest and evidence contract. (session-settled: user-directed.) Governs R9-R11. - **Share immutable bases; isolate writes.** `workspaceAccess` defaults to - `readWrite`. Read-only Tasks may reuse one validated physical base and cwd - with Task-private runtime state; read-write Tasks receive unique disposable - writable views. Reflink/block clone is preferred, rootless OverlayFS is the - Linux fallback, and an explicit copy backend preserves portability. - (session-settled: user-directed.) Governs R2-R3, R5, R8-R11, R15-R16, R18-R19. + `readWrite`. One-shot read-only Tasks and read-only sessions may reuse one + validated physical base; each receives private runtime state. One-shot + read-write Tasks receive disposable writable views. A read-write session + retains an immutable committed generation and runs each turn in a private + candidate that becomes the next generation only at atomic settlement. + Reflink/block clone is preferred, rootless OverlayFS is the Linux fallback, + and an explicit copy backend preserves portability. Governs R2-R3, R5, + R8-R11, R15-R16, R18-R19. - **Use App-first GitHub credential eligibility.** Prefer an applicable GitHub App; use a configured `gh` account only when no App installation applies; never fall back after selected-App failure. (session-settled: user-directed.) @@ -132,46 +161,124 @@ registry, or another profile configuration file for the initial use case. resolve through AllAgents-owned adapters and execute on the gateway host rather than through generated wrapper files or the acquisition container. Governs R7-R8, R13-R15. -- **Persist Task truth, not provider sessions.** Restart settles interrupted - work failed; it never resumes or automatically replays provider execution. - Governs R5, R13-R16. +- **Persist Task truth and resumable session checkpoints.** Restart settles an + interrupted turn failed and never replays it. An idle session resumes only + from its last committed provider/workspace checkpoint; an uncertain checkpoint + is non-resumable pending cleanup. Governs R3, R5, R8, R13-R16. - **Keep evaluation outside AllAgents.** Consumers own datasets, repetitions, scoring, assertions, and evaluation Runs. Governs R17. - **Bridge Promptfoo at the consumer boundary.** AI Evals owns a custom provider - that maps Promptfoo YAML and test variables to the closed A2A source modes and - maps terminal Tasks back to `ProviderResponse`. AllAgents owns no Promptfoo - runtime behavior. Governs R19. + that maps Promptfoo YAML and test variables to closed A2A source/session modes + and maps terminal Tasks plus session/cache metadata back to + `ProviderResponse`. AllAgents owns no Promptfoo runtime behavior. Governs R19. ### Requirements -**Public protocol** +**Private-network protocol** - R1. Implement A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, `GetTask`, `ListTasks`, `CancelTask`, streaming send, and active Task subscription. Every A2A request carries `A2A-Version: 1.0`; another version receives `VersionNotSupportedError`. The Agent Card advertises exactly one - absolute interface URL with `protocolBinding: "HTTP+JSON"`, - `protocolVersion: "1.0"`, and `capabilities.streaming: true`. It declares - `https://allagents.dev/a2a/extensions/coding-execution/v1` with - `required: true` and strict `params: { targets: TargetId[] }`, populated from - ready built-in and gateway-enabled profile targets. Production interface URLs - use HTTPS; direct HTTP is limited to loopback development. Every operation - that creates, returns, lists, subscribes to, or mutates profiled Tasks or - Artifacts includes that URI in `A2A-Extensions`; missing activation receives + absolute private interface URL with `protocolBinding: "HTTP+JSON"`, + `protocolVersion: "1.0"`, and `capabilities.streaming: true`. It declares the + coding-execution Profile Extension identifier + `urn:allagents:a2a:profile:coding-execution:v1` with `required: true`. Strict + `params` contains a sorted `targets` array whose entries have + `id: TargetId` and + `sessionModes: ("oneShot" | "start" | "resume")[]`. `oneShot` is always + present; `start` and `resume` are advertised together only when the target's + adapter supports native durable continuation. Targets and modes outside these + entries fail before admission. The Agent Card uses + `defaultInputModes: ["text/plain"]`, `defaultOutputModes: ["text/plain", + "application/json", "application/octet-stream"]`, and exactly one stable + skill: `id: "coding-execution"`, `name: "Coding execution"`, a + profile-defined description, tags `["coding", "harness-execution"]`, and an + example that produces a profile-shaped Task. Extension params advertise ready + targets and their usable continuation modes only. + V1 callers obtain configured logical repository and snapshot selectors + from their workspace or consumer configuration; the Agent Card does not + publish origins, credentials, destinations, physical paths, or snapshot + selection state. + + Remote interface URLs use HTTPS and resolve only to loopback, RFC 1918, + RFC 4193, link-local, or RFC 6598 addresses such as Tailscale's + `100.64.0.0/10`; direct HTTP is limited to a loopback listener. Startup rejects + wildcard/public listener addresses, public URL literals, advertised hostnames + with any public address, and a private direct listener without TLS key/cert. + Every operation that creates, returns, lists, subscribes to, or + mutates profiled Tasks or Artifacts includes the Profile URN in + `A2A-Extensions`; missing activation receives `ExtensionSupportRequiredError`. + Check the transport-neutral Harness Execution Contract Core and Sessions + specifications into `docs/contracts/harness-execution-v1.md` with identifiers + `urn:allagents:harness-execution:core:v1` and + `urn:allagents:harness-execution:sessions:v1`. Core defines one turn's + invocation, target selection, idempotency, deadline, ordered progress and + normalized tool-call/result trajectory, cancellation, terminal result, usage, + portable failures, and artifacts without importing A2A or UHP types. Sessions + defines linear continuation and durable provider/workspace checkpoints without + importing A2A types. The required Profile URN identifies the normative A2A + binding plus Sessions and the AllAgents coding-workspace extension, not an + undocumented metadata convention. These URNs are non-dereferenceable contract + identifiers, never gateway or public documentation endpoints. The binding + defines permitted A2A values, one-Message/one-Task-per-turn constraints, + schemas, context/task-reference mapping, state and event mapping, errors, + examples, versioning, and executable conformance cases. V1 uses one required + A2A Profile URN because this gateway requires Core, Sessions capability, and + the workspace extension; it exposes no second wire protocol. + Honor both `SendMessageConfiguration.returnImmediately` modes. `ListTasks` implements every standard filter, cursor pagination, `pageSize` 1-100 with a default no greater than 50, descending status-timestamp order, and required `tasks`, `nextPageToken`, `pageSize`, and `totalSize` fields. `nextPageToken` is present and empty on the final page. With the default `includeArtifacts: false`, each returned Task omits `artifacts`; `true` - includes the field. -- R2. Generate a strict versioned request schema from the canonical domain type - and place it only at `Message.metadata[extensionUri]`; the Message also lists - `extensionUri` in `Message.extensions`. Strict objects reject every unlisted - member. V1 uses these wire scalars: + includes the field. A response never exceeds the configured serialized-byte + limit. `ListTasks` may return fewer Tasks than `pageSize` when the next whole + Task would cross that limit; it never splits a Task, and its cursor resumes at + the first omitted Task. The validated per-Task/response invariant guarantees + one complete Task fits `GetTask` and a nonempty list page. +- R2. Define the exact transport-neutral `HarnessInvocationV1` DTO as + `{ version: "1", invocationKey, target, prompt, deadlineSeconds, + resultSchema? }`. The prompt is part of Core even though a binding may carry it + outside its control object. Define the separate Sessions input as exactly one + of `{ mode: "oneShot" }`, `{ mode: "start", closeAfterTurn }`, or + `{ mode: "resume", sessionId, previousTaskId, closeAfterTurn }`; both booleans + default to false. Its terminal turn projection is exactly one of + `{ mode: "oneShot", state: "none" }`, + `{ mode: "start" | "resume", sessionId, state: "idle", headTaskId }`, + `{ mode: "start" | "resume", sessionId, state: "closed" }`, or + `{ mode: "start" | "resume", sessionId, state: "notResumable" }`. An idle + `headTaskId` is the just-terminal Task and the only Task valid for the next + resume. Provider-checkpoint/workspace generation may independently remain the + prior committed pair after a safely failed or canceled turn. + Define the separate AllAgents workspace input as + `{ source, workingDirectory, workspaceAccess }`. Core imports neither + Sessions nor workspace types. + + The A2A binding projects exactly one Message text Part to Core `prompt`; maps + the remaining Core fields, Sessions `mode` as `sessionMode`, + `closeAfterTurn`, and the workspace fields to the flat object at + `Message.metadata[profileUri]`; and maps Sessions `sessionId` to + `Message.contextId` plus `previousTaskId` to the sole + `Message.referenceTaskIds` member. Each contract module owns a disjoint named + property set and its module-local required members but does not close the + shared metadata object. On a terminal Task, the binding writes the exact + Sessions terminal projection to strict `Task.metadata[profileUri].session`; + it is absent from nonterminal Tasks and events. `sessionId` equals the Task + context ID, and any idle `headTaskId` names the exact terminal Task accepted + for the next resume. + The composed A2A schema alone declares the complete + flat property set, unions required members, materializes cross-module + defaults, and sets `additionalProperties: false`; duplicate property ownership + or incompatible constraints fail generation. Strict nested objects reject + every unlisted member. V1 uses these wire scalars: - `InvocationKey` matches `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`. + - `ContextId` is NFC UTF-8 of 1-256 bytes with no U+0000-U+001F or U+007F; + gateway-generated session IDs are lowercase canonical UUIDv7 values. + - `TaskId` is the gateway-generated lowercase canonical UUIDv7 Task ID. - `ConfigName` and `TargetId` match `^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`. - `RevisionText` is NFC UTF-8, 1-255 bytes, with no U+0000-U+001F or U+007F. @@ -179,14 +286,17 @@ registry, or another profile configuration file for the initial use case. - `RelativeDirectory` is NFC UTF-8 of 1-1024 bytes containing 1-32 slash-separated segments. Each segment is 1-255 bytes, is neither `.` nor `..`, and contains no slash, backslash, U+0000-U+001F, or U+007F. - The request object is exactly: - `version: "1"`; `invocationKey: InvocationKey`; `target: TargetId`; `source`, - one of `{ kind: "repositories", revisions?: Record }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, - digest: Digest, workspaceManifestDigest: Digest }`; optional - `workingDirectory`, one of `{ kind: "workspaceRoot" }` or - `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }`, - defaulting to `{ kind: "workspaceRoot" }`; optional `workspaceAccess`, one of + The A2A profile metadata object is exactly: + `version: "1"`; `invocationKey: InvocationKey`; `target: TargetId`; + optional `sessionMode: "oneShot" | "start" | "resume"` defaulting to + `"oneShot"`; optional `closeAfterTurn: boolean` defaulting to false and + forbidden for `oneShot`; `source`, one of `{ kind: "repositories", + revisions?: Record }` or + `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, + workspaceManifestDigest: Digest }`; optional `workingDirectory`, one of + `{ kind: "workspaceRoot" }` or `{ kind: "repository", + repository: ConfigName, path?: RelativeDirectory }`, defaulting to + `{ kind: "workspaceRoot" }`; optional `workspaceAccess`, one of `"readOnly" | "readWrite"`, defaulting to `"readWrite"`; optional `deadlineSeconds` (integer 1-3600, default 1800); and optional `resultSchema: { version: "1", schema: SchemaNode }`. @@ -216,148 +326,335 @@ registry, or another profile configuration file for the initial use case. No pattern dialect exists in v1. References, unions/combinators, conditionals, formats, defaults, coercion, non-finite numbers, duplicate canonical enum values, and unknown keywords are rejected. The canonical result schema is at - most 64 KiB, 256 nodes, and 32 levels deep. Repository revision count cannot - exceed declared repositories. The Message contains exactly one `Part` with - `text` set to a UTF-8 prompt of 1 byte to 1 MiB; other Part content fields are - rejected. Only the extension-owned metadata object is strict; unrelated A2A - metadata and other activated-extension keys are preserved or ignored - according to A2A. Canonicalization materializes defaults, normalizes extension - strings to UTF-8 NFC, sorts record keys, and hashes RFC 8785 extension JSON - plus prompt bytes. Do not add `Task.extensions` or backend-specific public - fields. + most 64 KiB, 256 nodes, and 32 levels deep. Repository revision count cannot exceed declared repositories. + The Message contains exactly one `Part` with `text` set to a UTF-8 Core + `prompt` of 1 byte to 1 MiB; other Part content fields are rejected. Only the + profile-owned metadata object is strict; unrelated A2A metadata and other + activated-extension keys are preserved or ignored according to A2A. + `oneShot` and `start` forbid client `contextId` and `referenceTaskIds`; + `resume` requires a `ContextId` and exactly one `TaskId` reference. + Canonicalization materializes defaults, normalizes profile strings to UTF-8 + NFC, sorts record keys, and hashes the Core Invocation excluding + `invocationKey`, the Sessions and workspace inputs, and the complete A2A + session projection: mode, close-after-turn value, supplied context + presence/value, and ordered task references. Generated context IDs are not + hashed. Do not add `Task.extensions` or backend-specific public fields. A client generates an opaque invocation key with at least 128 bits of - randomness once per logical execution, durably reuses that key and identical + randomness once per logical turn, durably reuses that key and identical canonical request after an ambiguous transport failure, and creates a new key - only for intentionally new execution. A2A `messageId` remains Message identity - and does not replace the extension idempotency key. -- R3. One valid new request creates one addressable Task. Follow-up messages to - an existing Task are unsupported. Every terminal Task has exactly one - integrity Artifact plus zero or more produced Artifacts. The integrity - Artifact has `artifactId` and `name` equal to - `allagents.execution-integrity`, lists `extensionUri` in - `Artifact.extensions`, and has one `Part` with `data` set to the strict - `allagents.execution-integrity/v1` object and `mediaType: - "application/json"`. Its `taskId` equals the enclosing A2A `Task.id`. - `SafeUInt` is an integer 0-9,007,199,254,740,991; `ShortText` is valid UTF-8 - of at most 4096 bytes; `ArtifactId` matches + only for an intentionally new turn. A replay with any changed session + projection conflicts. A2A `messageId` remains Message identity and does not + replace the profile idempotency key. +- R3. One valid new turn creates one addressable immutable Task. `oneShot` and + `start` atomically generate and persist a lowercase canonical UUIDv7 context + ID with the absent-context invocation claim; only `start` creates a durable + session under that ID. Replays return the persisted generated value. `resume` + requires an idle, unexpired, resumable session at `Message.contextId`, the + session's exact current head Task as its sole `referenceTaskIds` member, and + the same target, canonical source, logical cwd, access mode, provider identity, + and configuration digest. A stale head, changed pinned input, active turn, + expired session, or poisoned checkpoint fails respectively with + `session_head_mismatch`, `session_configuration_mismatch`, `session_busy`, + `session_expired`, or `session_not_resumable`, and creates no Task. + + Every session turn and event uses the session context ID. Terminal Tasks remain + immutable; continuation always creates a new Task in the same context. V1 + serializes a linear session history and rejects branching or concurrent turns. + + HEC Core defines seven execution states: `submitted`, `working`, `completed`, + `failed`, `timedOut`, `canceled`, and `rejected`. A cancellation command + returns the neutral disposition `accepted | alreadyTerminal`; the latter never + changes the outcome. The A2A binding maps `submitted` and `working` to + `TASK_STATE_SUBMITTED` and `TASK_STATE_WORKING`; `completed`, `canceled`, and + `rejected` to their same-named A2A states; and both `failed` and `timedOut` to + `TASK_STATE_FAILED`. A2A maps `alreadyTerminal` to + `TaskNotCancelableError`. A Message addressed to an existing Task ID returns + `UnsupportedOperationError` whether that Task is active or terminal; a session + continuation instead creates a new Task using the same context ID and the + prior Task as a reference. The binding never emits + `TASK_STATE_INPUT_REQUIRED`, `TASK_STATE_AUTH_REQUIRED`, or + `TASK_STATE_UNSPECIFIED`. + + Core owns one durable ordered execution-event stream. `SafeUInt` is an integer + 0-9,007,199,254,740,991; `ShortText` is valid UTF-8 of at most 4096 bytes; + `TrajectoryText` is valid UTF-8 of at most 65,536 bytes; `ArtifactId` matches `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`; and `MediaType` is a valid RFC 6838 - media type of at most 255 ASCII bytes. - - `version` is the literal `"1"`; `taskId` is a lowercase canonical UUIDv7; - and `target` is `TargetId`. - - `workingDirectory` is the effective logical selector from the request: - `{ kind: "workspaceRoot" }` or + media type of at most 255 ASCII bytes. `TrajectoryPayload` is exactly + `{ kind: "json", value }` or `{ kind: "text", text: TrajectoryText }`. + JSON `value` has at most 32 levels, contains no non-finite number, and its + RFC 8785 encoding is at most 65,536 bytes. Native structured values use the + JSON branch; native strings or values without a lossless JSON representation + use the text branch. Equivalent structured values therefore normalize to the + same bytes. + + `HarnessExecutionEventV1` is exactly one gapless `sequence` from zero plus: + - `{ type: "progress", phase, message?: ShortText }`, where `phase` is + `accepted | acquiring | preparing | executing | collecting | cleaning`; + - `{ type: "toolCall", callId: ArtifactId, name: ShortText, + arguments: TrajectoryPayload }`; or + - `{ type: "toolResult", callId: ArtifactId, + status: "completed" | "failed" | "canceled", output: TrajectoryPayload }`. + + Tool-call IDs are unique. A result follows exactly one matching call, and a + call has at most one result. A normally completed execution with a complete + trajectory has exactly one result for every call. An interrupted call emits a + `canceled` result when the adapter observed that outcome; otherwise it remains + unmatched and forces `complete: false`. Duplicate event delivery is + idempotent only when its sequence and canonical payload are identical; + divergence fails `provider_protocol_invalid`. + + Each Core event is transactionally appended before publication. The gateway + computes `terminalTailReserveBytes` from the generated maximum compact + encodings of every non-droppable terminal field: a 1 MiB valid result, maximal + bounded Core/composed failure records, maximal required source/provenance + identity, the three reserved Artifact envelopes, all terminal events, and + response envelopes. Truncatable output/evidence and optional produced + Artifacts are excluded. Every nonterminal append projects both the durable + event record and its duplicate in the terminal trajectory plus that reserve; + it commits only when the projected footprint remains within + `max-task-bytes`. Otherwise the gateway atomically + records trajectory truncation and publishes neither that event nor later + nonterminal events; provider execution continues. The A2A binding emits one + nonterminal `TaskStatusUpdateEvent` per committed event, stores the exact Core + event at `event.metadata[profileUri].executionEvent`, uses + `TASK_STATE_SUBMITTED` only for `phase: "accepted"` and + `TASK_STATE_WORKING` otherwise, and preserves the Task context ID. Replay and + resubscription emit only committed events in sequence. Tool events are live + progress and also form the terminal normalized trajectory. + + Every terminal Task has exactly one Core outcome Artifact, one Core execution- + trajectory Artifact, one AllAgents workspace-integrity Artifact, and zero or + more produced Artifacts. Atomic settlement commits the terminal Task, every + terminal `TaskArtifactUpdateEvent`, and the terminal + `TaskStatusUpdateEvent` in the same transaction as those Artifacts. Publishers + emit only committed events: every Artifact update precedes the terminal + status update, whose terminal `TaskState` makes it the final stream item, and + the stream then closes. + + The Core outcome Artifact has `artifactId` and `name` equal to + `harness.execution-outcome`, lists `profileUri` in `Artifact.extensions`, and + has one `Part` with `data` set to the strict + `harness-execution-outcome/v1` object and media type + `application/vnd.allagents.harness-outcome+json`. The payload is a strict + discriminated union on terminal `state`; every branch also contains + `version: "1"`, `target: TargetId`, + `terminalOutput: { text, truncated }`, optional `usage`, + `executionTrajectory`, and `result`. Terminal output is UTF-8 at most 1 MiB. + Usage is a strict object with optional `inputTokens`, `outputTokens`, + `cachedInputTokens`, and `totalTokens` `SafeUInt` fields plus optional + `provider` containing 0-64 `ConfigName: SafeUInt` counters. + `executionTrajectory` is + `{ artifactId: "harness.execution-trajectory", eventCount: SafeUInt, + complete, truncated, digest: Digest }`. + + The state branches are closed: + - `completed` forbids `failure` and permits only valid result or + `notProduced` with `notRequested | providerDidNotReturn`; + - `canceled` requires `execution_canceled`/`cancellation` and + `notProduced: canceled`; + - `timedOut` requires `execution_deadline_exceeded`/`deadline` and + `notProduced: deadlineExceeded`; + - `rejected` requires `execution_permission_denied`/`permission` and + `notProduced: rejected`; and + - `failed` requires one other Core failure/result pair from the normative + `core-outcome-matrix/v1`. + + Core-only `failure` is `{ code, message, retryable, cause }`; codes are + HEC-owned error-table rows or `execution_extension_failed`; causes are + `validation | capacity | deadline | permission | providerProtocol | + cancellation | termination | stateStore | restart | extension`; `message` is + `ShortText`. Retryability matches the Core matrix row except + `execution_extension_failed`, whose Core schema admits either boolean. + `result` is exactly `{ status: "valid", value }`, + `{ status: "invalid", errors }`, or + `{ status: "notProduced", reason }`. `value` validates against the requested + `SchemaNode`, serializes to at most 1 MiB, and appears only when requested. + `errors` contains 1-64 strict `{ path, keyword, message }` entries; `path` is + an RFC 6901 JSON Pointer at most 1024 bytes, `keyword` is a v1 `SchemaNode` + member, and `message` is `ShortText`. Reasons are + `notRequested | providerDidNotReturn | providerFailed | extensionFailed | + rejected | canceled | deadlineExceeded | invalidProviderPayload | + retentionLimitExceeded`. + + The checked-in, hand-authored matrix has rows for every Core failure/result + combination and explicitly enumerates terminal state, result status/reason, + and retryability. Its `execution_extension_failed` rows admit both retryability + values; the composed profile requires equality with the selected workspace + failure row. Generated JSON Schema is a `oneOf` over the matrix. An extension + failure before a Core execution result is determined requires + `notProduced: extensionFailed`; a later extension settlement failure preserves + the already determined result, + including a valid result or cancellation/deadline reason. Every unlisted + combination is rejected, including completed+failure, canceled+provider + failure, or timedOut without deadline failure. + + The Core trajectory Artifact has `artifactId` and `name` equal to + `harness.execution-trajectory`, lists `profileUri` in `Artifact.extensions`, + and has exactly one `Part` with `data` set to the strict + `harness-execution-trajectory/v1` object and media type + `application/vnd.allagents.harness-trajectory+json`. Its payload is + `{ version: "1", eventCount, complete, truncated, digest, events }`. + `events` is the longest whole-event prefix, at most 4096 events, that fits the + retained-byte bound; middle or earlier events are never sampled or dropped. + `eventCount` equals its length. `truncated` is true exactly when an observed + event was omitted by a count or byte bound. `complete` is true only when the + adapter asserts full event observability, no event was omitted, and every + interrupted call is represented as above. `digest` is SHA-256 over RFC 8785 + bytes of the same payload with `digest` omitted. The A2A binding requires the + Core outcome's `executionTrajectory` fields to equal the unique trajectory + Artifact's `artifactId`, `eventCount`, `complete`, `truncated`, and `digest` + exactly; any mismatch fails composed validation. Native provider traces remain + optional evidence and never replace this Core record. + + The AllAgents workspace-integrity Artifact has `artifactId` and `name` equal + to `allagents.workspace-integrity`, lists `profileUri` in + `Artifact.extensions`, and has one `Part` with `data` set to the strict + `allagents.workspace-integrity/v1` object and media type + `application/vnd.allagents.workspace-integrity+json`. Its `taskId` equals the + enclosing A2A Task ID. The payload owns: + - `version: "1"` and `taskId`; + - effective logical `workingDirectory`, exactly `{ kind: "workspaceRoot" }` or `{ kind: "repository", repository: ConfigName, - path?: RelativeDirectory }`. It never contains a physical path or configured - repository destination. - - `workspaceAccess` is the effective `"readOnly" | "readWrite"` value. - `readOnly` is a consumer-selected cooperative contract with best-effort - provider-policy enforcement, not hostile-code containment. - - `sourceIdentity` is either + path?: RelativeDirectory }`, and effective + `workspaceAccess: "readOnly" | "readWrite"`. No physical or configured + destination path is present. `readOnly` is cooperative best-effort provider + policy, not hostile-code containment; + - `sourceIdentity`, either `{ kind: "repositories", complete, repositories }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, workspaceManifestDigest: Digest, layerDigests, complete, repositories }`. - `complete` is boolean. `layerDigests` contains 0-64 `Digest` values in - manifest order. `repositories` contains 0-64 unique strict entries + `layerDigests` has 0-64 `Digest` values in manifest order. + `repositories` has 0-64 unique strict entries `{ name: ConfigName, requestedRevision?: RevisionText, resolvedCommit: string, verification: "independentlyVerified" | "snapshotAttested" }`; `resolvedCommit` matches - `^[0-9a-f]{40}$`. Before provider execution, `complete` must be true, - `repositories` must exactly match the configured catalog, and snapshot - identities must include every layer digest. Failed acquisition records only - verified members and sets `complete` false. - - Gateway-generated source identity, workspace-manifest fields, evidence - metadata, and provider-added metadata never contain Git URLs, OCI repository - origins, or destination paths. This guarantee applies only to - gateway-managed credentials and generated metadata; it does not inspect or - sanitize opaque prompts, terminal output, structured results, native - evidence, or produced-Artifact payloads. - - optional `workspaceManifestDigest` is `Digest`. - - `terminalOutput` is `{ text, truncated }`, where `text` is valid UTF-8 of at - most 1 MiB and `truncated` is boolean. - - optional `usage` is a strict object with optional `inputTokens`, - `outputTokens`, `cachedInputTokens`, and `totalTokens` `SafeUInt` fields, - plus optional `provider` containing 0-64 `ConfigName: SafeUInt` counters. - - `producedArtifacts` contains 0-128 strict entries + `^[0-9a-f]{40}$`. Before provider execution, `complete` is true, repositories + exactly match the configured catalog, and snapshot identities include every + layer digest. Failed acquisition records only verified members and sets + `complete: false`; + - optional `workspaceManifestDigest: Digest`; + - `producedArtifacts`, 0-128 strict entries `{ artifactId: ArtifactId, name?: ShortText, mediaType?: MediaType, - size: SafeUInt, digest: Digest }`; each references one additional A2A - Artifact embedded in the Task. - - `evidence` is `{ items, complete, truncated }`, where the booleans have - their literal JSON meaning and `items` contains 0-256 strict entries - `{ kind, artifactId?, digest?, summary? }`. `kind` is one of + size: SafeUInt, digest: Digest }`. IDs are unique, entries sort by ID, and + their ID set exactly equals `Task.artifacts` minus the three reserved + Artifacts. Each referenced produced Artifact lists `profileUri`, mirrors the + optional name and media type, and has exactly one `Part.raw` containing the + base64 file bytes; other Part content and metadata are absent. `size` and + SHA-256 `digest` cover the exact decoded bytes; + - `evidence: { items, complete, truncated }`, whose 0-256 strict items are + `{ kind, artifactId?, digest?, summary? }` with kind `gitState | providerTrace | fileChanges | usage | cancellation | - termination | cleanup`; `artifactId` and `digest` use the aliases above, - `summary` is `ShortText`, and at least one of those three optional members - is present. - `complete` means every configured bounded evidence category was attempted - after the direct provider process settled; it never means every descendant - was enumerated or quiescent. - - `termination` is `{ status: "clean" | "failed" | "unknown", - reason?: ShortText }` and reports the direct provider/process-group - observation only. - - `cleanup` is - `{ workspace: "shared" | "removed" | "retained" | "failed", - reason?: ShortText }`. `shared` means a read-only Task removed its private - runtime state while retaining a reusable cached base; `removed` means every - Task-owned runtime, writable view, and non-reusable base was removed. - - optional `failure` is `{ code, message, retryable, cause }`, where `code` - is one stable code from the error table below, `cause` is one member of the - closed cause union defined below that table, `message` is `ShortText`, and - `retryable` is that row's fresh-invocation value. - - `result` is exactly one of - `{ status: "valid", value }`, - `{ status: "invalid", errors }`, or - `{ status: "notProduced", reason }`. `value` is JSON that validates against - the requested `SchemaNode`, serializes to at most 1 MiB, and is used only - when a result schema was requested. `errors` contains 1-64 strict - `{ path, keyword, message }` entries: `path` is an RFC 6901 JSON Pointer of - at most 1024 bytes, `keyword` is one of the v1 `SchemaNode` member names, - and `message` is `ShortText`. `reason` is one of - `notRequested | providerDidNotReturn | providerFailed | canceled | - deadlineExceeded | invalidProviderPayload`. - - Failure, rejection, and cancellation retain every available field without - implying a valid result. Task records, Artifacts, events, and claims expire - atomically after the configured TTL; expired keys may create new Tasks. The - gateway never evicts unexpired Tasks to satisfy the retained-count limit: it - rejects new admission until expiry frees capacity. + termination | cleanup`, at least one optional member present, and aliases as + above. `complete` means each configured bounded category was attempted after + the direct provider settled, never that every descendant was quiescent; + - `termination: { status: "clean" | "failed" | "unknown", + reason?: ShortText }`, reporting only direct provider/process-group + observation; + - `cleanup: { workspace: "shared" | "removed" | "retained" | "failed", + reason?: ShortText }`. `shared` retains only a reusable base after removing + private runtime; `removed` means all Task/session-owned runtime, writable + view, and non-reusable base state is gone; and + - optional workspace-only `failure: { code, message, retryable, cause }`, + whose code/cause is one AllAgents workspace-extension row in the error table. + It never admits a HEC-owned failure code. + + The module-local workspace payload contains no Core Artifact reference. The + composed profile alone requires the three reserved Artifacts and, for a + workspace failure, pairs exact workspace `failure` with Core + `execution_extension_failed`, cause `extension`, and the same retryability. + A non-extension Core-origin failure leaves workspace `failure` absent. + + Gateway-generated source identity, workspace-manifest fields, evidence + metadata, and provider-added metadata never contain Git URLs, OCI repository + origins, or destination paths. This does not inspect or sanitize opaque + prompts, output, result values, trajectory text, native evidence, or produced + bytes. + + Failure, rejection, and cancellation retain every available bounded field + without implying a valid result. A Task's retained footprint is the maximum + of: (a) the sum of exact UTF-8 bytes of compact canonical JSON for its claim, + Task, event, and Artifact payloads, counting raw Artifact data at its base64 + wire size; (b) its exact serialized `GetTask` HTTP+JSON response; and (c) its + exact serialized one-Task `ListTasks(includeArtifacts: true)` response. The + production serializer computes those values after JSON escaping and envelope + fields, before every commit. + + Collectors retain the longest event prefix, truncate stream-like fields, or + omit a whole produced Artifact to remain within `max-task-bytes`; they never + retain a partial produced file. An otherwise valid structured result that + cannot fit becomes `result.status: "notProduced"`, reason + `retentionLimitExceeded`, and terminal failure + `execution_result_too_large`. Reconciled terminal Task records, Artifacts, + events, and claims expire atomically after their configured TTL, releasing the + actual retained charge. A poisoned/unreconciled Task is ineligible for expiry, + is charged only by its active `max-task-bytes` reservation, and starts its TTL + only when reconciliation atomically replaces that reservation with its exact + footprint. **Trust, identity, and Task storage** -- R4. Do not authenticate application callers. Allow loopback, specific-address, - and explicit `0.0.0.0` listeners. Every reachable caller may create, list, - retrieve, subscribe to, and cancel every Task. Artifacts are retrieved only - inside Tasks through `GetTask` or `ListTasks(includeArtifacts: true)`; v1 adds - no separate Artifact endpoint. Document Tailscale ACLs, firewalls, or - equivalent network controls as the authorization boundary. -- R5. Idempotency and Task visibility are deployment-wide. Atomically and - durably bind an invocation key to the canonical request, selected target, - source identity, effective logical working directory, effective workspace - access, optional result-schema digest, deadline, and effective configuration - digest before acknowledging Task creation. One transactional - `createOrReplay` operation arbitrates competing requests. Identical replay - returns the existing Task; a changed request conflicts. Status and terminal - settlement are monotonic. The project-specific state root persists the - canonical workspace identity and holds an exclusive process lock. +- R4. Do not authenticate application callers because version one is restricted + to one trusted private network. Allow exactly: loopback HTTP, optionally + exposed through a private HTTPS terminator; or native HTTPS bound to one + specific loopback/RFC 1918/RFC 4193/link-local/RFC 6598 address. Native + off-loopback TLS requires configured certificate/key files. Wildcard and + public listeners are rejected before binding. The advertised URL is loopback + HTTP only for local use and otherwise private HTTPS. Tailscale ACLs, private + firewall rules, or equivalent controls remain required. Every reachable caller + may create, list, retrieve, subscribe to, resume, and cancel every Task/session. + Artifacts are retrieved only inside Tasks through `GetTask` or + `ListTasks(includeArtifacts: true)`; v1 adds no separate Artifact endpoint. + Startup/docs state that reachability is authorization and public exposure + requires application authentication, authorization, and an amended ADR. +- R5. Idempotency, Task visibility, and session visibility are deployment-wide. + One transactional `createOrReplay` operation validates and locks the selected + session state, then atomically writes the invocation claim, Task, initial + `accepted` event, effective context ID, and byte reservation. A `start` also + creates the session row; a `resume` compare-and-sets the exact idle head to the + new active Task. The claim binds the key to the canonical Core request, + Sessions input, workspace input, full A2A session projection, selected + target/source/cwd/access, optional result-schema digest, deadline, and + effective configuration digest before acknowledging Task creation. The Task + reservation is exactly `max-task-bytes`. + Admission succeeds only when retained Task count, + `settledFootprints + activeReservations + maxTaskBytes`, retained session count, + and prior-plus-candidate session byte reservations fit. Replay returns the + existing Task/reservations; a changed request conflicts. Settlement uses the + R16 prepared record, then atomically replaces the Task reservation with its + footprint and either sets the idle lineage head to the current terminal Task + plus the selected committed provider-checkpoint/workspace-generation pair, or + records close/poison state. Task expiry releases only its footprint and never + a live session pair. Status, terminal + settlement, and head/pointer advancement are monotonic. The project state root + persists canonical workspace identity and holds one process lock. The Bun gateway privately owns one SQLite database through `bun:sqlite`. - Claims, Tasks, events, bounded Artifact bytes, the single execution lease, - internal outcome intent, acquisition-container identity, staging and - transient-base identity, direct provider process-group identity, and expiry - live in ordinary transactional tables. Enable foreign keys, use WAL where - supported, set `synchronous=FULL`, and acknowledge only committed - transactions. The current user owns the state root and database with - `0700`/`0600`-equivalent permissions; the root is disjoint from project, - staging, publication, profile, and provider-auth roots. No custom SQLite VFS + Ordinary tables hold claims, Tasks, events, Sessions, ordered turns, opaque + provider checkpoints, committed/candidate/superseded workspace-generation + identities, prepared settlement/cleanup records, pinned configuration/source/ + workspace digests, session byte reservations/charges, retained Task charges, + bounded Artifact bytes, the execution lease, outcome intent, acquisition- + container/staging/transient-base identity, provider process-group identity, + observed outliving-descendant identity, and expiry. + Enable foreign keys, use WAL where supported, set `synchronous=FULL`, and + acknowledge only committed transactions. The current user owns the state root + and database with `0700`/`0600`-equivalent permissions; the root is disjoint + from project, staging, publication, profile, and provider-auth roots. No custom SQLite VFS or native file primitive is introduced. Startup verifies the root, lock, schema/integrity, and workspace identity; terminalizes interrupted Tasks failed; removes recorded acquisition containers and orphan staging; and - attempts to terminate recorded provider process groups. It releases a stale - or termination-poisoned lease only after the container is gone, the recorded - process group is confirmed absent, and recorded Task-owned runtime, view, and - transient-base cleanup has completed. Otherwise readiness remains false and - admission stays stopped. Provider work is never resumed. - Store open, corruption, write, transaction, or synchronization failure stops + attempts to terminate recorded provider process groups. An unresolved App mint + intent or revocation-pending token requires recorded container/local-secret + absence and an atomically durable expiry-backed tombstone before the + interrupted Task terminalizes. + The gateway releases a stale/poisoned lease only after the container and + recorded process group are gone, every observed outliving descendant is gone, + candidate/committed/superseded workspace and other owned state reconcile to the + database pointer, and any revocation tombstone is durable. Otherwise readiness + remains false. An interrupted turn is never resumed/replayed; an idle session + may resume only from its last committed provider-checkpoint/workspace- + generation pair. Store open/corruption/write/synchronization failure stops admission and prevents terminal success. **Workspace and target configuration** @@ -381,29 +678,47 @@ registry, or another profile configuration file for the initial use case. Codex and Pi profile clients are executable. - R8. A request selects a declared target, may select one logical `workingDirectory`, may select `workspaceAccess`, may set the bounded - `deadlineSeconds`, and may provide one bounded result schema. `workspaceAccess` - defaults to `readWrite`; it is never inferred from prompt text. - - A read-only Task resolves its cwd directly inside its validated base. An exact - immutable request may share a reusable cached base; a mutable branch or tag - request owns a non-reusable base for the Task lifetime. The Task receives a - private `//runtime` for temporary, home, - provider-state, and evidence files. Provider environments disable optional - Git locks, and adapters request native read-only policy when available. The - gateway does not inspect prompts or add a per-Task mount, chmod pass, or full- - tree verification. The consumer remains responsible for assigning work that - does not require project mutation. - - A read-write Task receives a unique writable view at - `//workspace`. The configured host materializer + `deadlineSeconds`, may provide one bounded result schema, and chooses + `oneShot | start | resume`. `workspaceAccess` defaults to `readWrite`; it is + never inferred from prompt text. A session start pins target, canonical + source, logical cwd, access mode, provider identity, and effective + configuration. Resume must repeat those values exactly. + + A one-shot read-only Task resolves its cwd directly inside its validated base. + An exact immutable request may share a reusable cached base; a mutable branch + or tag request owns a non-reusable base for the Task lifetime. The Task + receives a private `/tasks//runtime` for temporary, + home, provider-state, and evidence files. A read-only session instead pins its + base and retains private `/sessions//runtime` + state between turns. Provider environments disable optional Git locks, and + adapters request native read-only policy when available. The gateway does not + inspect prompts or add a mount, chmod pass, or full-tree verification. The + consumer remains responsible for assigning read-only work that does not + require project mutation. + + A one-shot read-write Task receives a unique writable view at + `/tasks//workspace`. A read-write session never + mutates its last committed view in place. Each turn materializes + `/sessions//candidates/` from the pinned + base (start) or committed view (resume), runs the provider only in that + candidate, and keeps the prior view until settlement. Committed immutable + generations live under + `/sessions//workspaces/`, with the + current generation named only by the SQLite session row. The materializer prefers a filesystem block clone, falls back to rootless OverlayFS on - supported Linux hosts, and supports an explicit ordinary-copy backend for - portability. Writable files never use hard links. Normal settlement removes - the writable view and any non-reusable base after evidence collection; non- - settling execution retains them with the poisoned lease until verified - reconciliation. - - `{ kind: "workspaceRoot" }` selects the effective base or Task view. A + supported Linux, and supports ordinary copy; writable files never use hard + links. Admission reserves both prior and candidate charges against per-session + and aggregate limits. Successful continuing settlement atomically advances + the provider checkpoint, workspace-generation pointer, and head before + removing the prior generation. Crash before that commit discards the + candidate and leaves the prior pair; crash after it keeps the new pair and + reconciles the recorded old generation. One-shot settlement removes its view. + Session `closeAfterTurn` or idle expiry removes all generations, candidates, + provider state, and non-reusable base only after no active turn remains, then + releases the session charge. Uncertain cleanup poisons and retains the session. + + `{ kind: "workspaceRoot" }` selects the effective base, Task view, or session + view. A repository selector maps its declared name through the compiled catalog, appends only the validated `RelativeDirectory`, resolves links without escape, and must name an existing directory beneath that repository. Absolute paths, @@ -414,14 +729,14 @@ registry, or another profile configuration file for the initial use case. payloads are not sanitized and may contain it. The gateway owns one durable execution lease covering base acquisition or - lookup through final evidence collection. Admission claims that lease - transactionally before starting an acquisition container or provider process; - at most one Task may hold it. A second otherwise-valid request is accepted as - a Task and settles failed with `execution_capacity_unavailable` without - creating a container or process. Lease identity is stored with the Task and - survives gateway restart. Normal final settlement releases it; a provider- - termination failure retains it until verified startup reconciliation confirms - the recorded process group is absent. + lookup through final evidence/cleanup. Admission claims it transactionally + before starting a container or provider; at most one gateway-controlled Task + may hold it. A second valid request settles failed with + `execution_capacity_unavailable` without creating a container/process. Lease + identity survives restart. Normal settlement releases it only after the + recorded process group is absent, no observed escaped/outliving descendant + remains, and required cleanup/checkpoint reconciliation succeeds. Failure + retains the lease and readiness stays false. The overall deadline covers cache lookup, Docker acquisition on miss, publication, optional materialization, typed preparation, bare-metal provider @@ -495,10 +810,11 @@ registry, or another profile configuration file for the initial use case. and workspace-manifest digests. A repository identity is reusable only when every effective revision is a full commit ID. Its cache key also binds the acquisition-contract version, compiled catalog/layout digest, and every - commit. Branch and tag requests instead receive a non-reusable Task-owned base - and never populate or reuse a cache entry. A valid cache hit starts no - acquisition container and resolves no source credential. Active Tasks pin a - reusable base; bounded eviction removes only unpinned cache entries. + commit. Branch and tag requests instead receive a non-reusable Task- or + session-owned base and never populate or reuse a cache entry. A valid cache hit + starts no acquisition container and resolves no source credential. Active + Tasks/sessions pin a reusable base; bounded eviction removes only unpinned + cache entries. **Credential selection and containment** @@ -508,15 +824,21 @@ registry, or another profile configuration file for the initial use case. independently proven through the configured GitHub CLI identity; an uncorroborated 404, 401, 403, 429, timeout, or 5xx is `unknown`. An explicit installation ID is eligible only after positive repository-coverage - verification. For `eligible`, the host gateway creates the App JWT from the - configured private key, discovers and verifies installation coverage, and - mints a new repository-scoped read-only installation token for every cache- - miss acquisition. Credentials are never cached. Validate the token's - repository selection, - permissions, creation time, and expiry, and require remaining lifetime greater - than the R8 acquisition sub-budget plus a 60-second clock-skew margin. Only - the resulting installation token enters the acquisition container; the App - private key remains on the host. + verification. For `eligible`, the host gateway creates the App JWT and + verifies installation coverage. Before the external mint request, it durably + records a non-secret mint intent with conservative possible-token expiry + `now + mintRequestTimeout + 1 hour + 60 seconds`. The gateway enforces and + aborts the external request at `mintRequestTimeout`; the bound covers a token + minted at the last permitted instant, GitHub.com's one-hour lifetime, and + clock skew. It then mints one repository-scoped read-only installation token + for the cache-miss acquisition and atomically replaces the intent with the + returned non-secret issue time, expiry, and `revocationPending: true`. A + definitive no-token response may clear the intent; any crash or ambiguous + mint outcome leaves it for reconciliation. Credentials are never cached. + Validate a returned token's repository selection, permissions, creation time, + and expiry, and require remaining lifetime greater than the R8 acquisition + sub-budget plus a 60-second clock-skew margin. Only the installation token + enters the acquisition container; the App private key remains on the host. Use the configured GitHub CLI account only when the App is absent or applicability is positively `ineligible`. An `unknown` result or any @@ -524,9 +846,24 @@ registry, or another profile configuration file for the initial use case. rate-limit, or service failure terminates acquisition without `gh` fallback. Resolve `gh auth token --hostname github.com --user ` on the host with ambient token variables removed, then inject only the selected - invocation-scoped source credential into the acquisition container. Revoke an - App token after acquisition and fail before provider execution if revocation - cannot be confirmed. OCI acquisition accepts anonymous pulls or exact- + invocation-scoped source credential into the acquisition container. Every + non-crash exit after an App token is minted—including cancellation, deadline, + shutdown, validation failure, and successful acquisition—runs one idempotent + revoke-and-confirm path; the Task does not terminally settle or report source + cleanup complete until that path finishes. A revocation failure becomes + `source_auth_failed` and prevents provider execution. + + A persisted unresolved mint intent or revocation-pending token is reconciled + before Task terminalization. Startup removes any recorded container and local + credential state, atomically persists an expiry-backed tombstone through the + conservative or actual expiry, and settles the interrupted Task + `TASK_STATE_FAILED` with `gateway_restarted` plus evidence that revocation is + unconfirmed. The tombstone is not cascade-deleted with Task expiry and is + removed only after its own recorded expiry. Once the container and local + secret are confirmed absent and the tombstone is durable, the stale execution + lease may be released and fresh App-backed acquisition may proceed with a new + token; the gateway never claims the possible or known old token was revoked. + OCI acquisition accepts anonymous pulls or exact- registry credentials from the strict Docker-auth/helper boundary and supports same-origin Basic and Distribution Bearer challenges, the documented Docker Hub token service, and an operator-supplied exact-host CA-bundle map. The @@ -536,38 +873,58 @@ registry, or another profile configuration file for the initial use case. **Execution, evidence, and cleanup** - R13. Keep one closed `codex | pi` backend registry behind a narrow - AllAgents-owned TypeScript interface covering availability, capabilities, - invocation, progress, deterministic permission handling, abort, direct - process settlement, terminal output, optional structured result, usage, - bounded native evidence, and disposal. The gateway owns contract - normalization rather than adopting AI SDK Harnesses. Profile targets resolve - adapter-owned configuration directly; never execute generated launchers, - discover arbitrary executables as targets from `PATH`, scrape a TUI, append - public input to argv, or download a provider runtime per request. A configured - globally installed binary override is eligible only after an exact version - and protocol compatibility probe. -- R14. Codex uses a pinned `@openai/codex-sdk` directly from the Bun gateway. - Codex app-server is allowed only if U0 demonstrates a required capability - absent from that pinned SDK; the reason and tested protocol version must then - be recorded. Each Task receives one fresh SDK execution context, streamed - events, native cancellation, and an explicitly constructed child environment. - When API credentials are absent, preserve the existing host `CODEX_HOME` and - ChatGPT login in place; do not copy, mount, parse, or import OAuth files. - Pass native `outputSchema` only when the public schema has an object root, - every object's `required` set equals its property set, nesting is at most 10 - levels, and every keyword is supported by the pinned SDK/model. Other valid - public schemas use explicit JSON guidance plus the common gateway-side - validator. - - Pi uses a pinned supported package/RPC surface, invocation-owned - configuration, one restricted policy extension, and the existing host Pi - authentication location. Repository extensions and unrestricted built-ins do - not auto-load. Do not copy, mount, parse, or import Pi authentication files. - Both adapters preserve only required host identity, authentication paths, - executable lookup, locale, certificate, and proxy settings in an explicit - environment allowlist. This reduces accidental environment leakage; it is - not a secret-isolation guarantee because model-invoked tools run with the same - CI-job authority. + AllAgents-owned TypeScript interface covering availability, per-mode + capability advertisement, start/resume/checkpoint/dispose, invocation, + ordered normalized events, deterministic permission handling, abort, process + settlement, output/structured result, native usage/cache counters, evidence, + and disposal. Provider session handles are opaque, private, and accepted only + from their creating adapter. Each adapter preserves native event order, + reports observability gaps, supplies native call identity when available, and + directs all mutable conversation/runtime state to the Task/session-private + root. It may reference host authentication only through a pinned, + provider-supported auth input distinct from that mutable state root. U0 proves + this split for every advertised auth/mode combination; otherwise the target or + session modes remain unavailable. The gateway never copies, mounts, parses, or + imports provider auth files to synthesize the split. + + The gateway derives stable invocation-local call IDs when needed, normalizes + payloads, chooses the retained prefix, and computes trajectory `complete`, + `truncated`, and digest. Neither layer invents events or cache hits. Profile + targets resolve through adapter-owned configuration; never discover arbitrary + executables from `PATH`, scrape a TUI, append public input to argv, or download + a provider runtime per request. A global binary override is eligible only + after an exact version/protocol probe. +- R14. Codex uses pinned `@openai/codex-sdk` directly. Start uses + `startThread()`, resume uses `resumeThread()` with the last committed opaque + thread ID, and each Task is one SDK `run()` turn. App-server is allowed only + when U0 proves a named SDK gap and records the tested protocol. Each Task gets + streamed events, native cancellation, an explicit environment, and a private + state home under its Task/session runtime. API credentials may pass through + the explicit auth allowlist. Existing ChatGPT login is supported only if the + pinned public integration can reference its host auth location separately + while keeping thread/session writes in the private state home; otherwise that + auth/target combination is not ready. Pass native `outputSchema` only when the + public schema has an object root, every object's `required` set equals its + property set, nesting is at most 10 levels, and every keyword is supported by + the pinned SDK/model. Other valid schemas use JSON guidance plus gateway + validation. + + Pi uses a pinned supported package/RPC surface under the same auth/state split, + with explicit create/resume/checkpoint/dispose, invocation-owned + configuration, and one restricted policy extension. If its pinned public + surface cannot separately reference host auth, root mutable state privately, + and durably resume by opaque stable handle, only the modes that pass those + probes are advertised; transcript replay is never a substitute. Repository + extensions and unrestricted built-ins do not auto-load. + + Both adapters preserve only required auth references, executable lookup, + locale, certificates, and proxy settings in an explicit environment allowlist. + This reduces accidental leakage; it is not secret isolation because model + tools retain CI-job authority. A resumed session appends to provider-native + history with pinned target/model/tool configuration, preserving the longest + eligible prompt prefix. This improves cache eligibility but never guarantees a + hit: routing, model rules, prefix length, and TTL remain external. Report only + native `cachedInputTokens`; never infer savings. - R15. Start a fresh Docker container only when a request has no reusable validated base, including a cache miss or a non-reusable branch/tag request. Probe Docker and the exact digest-pinned `apps/acquirer` image at that point. @@ -586,24 +943,31 @@ registry, or another profile configuration file for the initial use case. through the bind mount, then exits. The host gateway waits for exit, removes the container, destroys source credentials, validates the manifest and tree against the compiled catalog and fixed limits, and atomically promotes staging - to a reusable cache entry or non-reusable Task-owned base. Every cancellation, - deadline, validation failure, or other non-publication path removes staging - idempotently; cleanup uncertainty fails `source_cleanup_failed` and stops - admission. Source-mode failure never falls through. - - A read-only Task uses the base directly plus Task-private runtime state. - Adapter-owned preparation for that mode must keep project files unchanged and - place invocation configuration outside the base. A read-write Task first - receives a unique block-cloned, overlaid, or copied view; typed preparation - may then project validated project/profile settings, plugins, and MCP - declarations into that view. Project or user `setup` entries and other - configured shell commands never run automatically. + to a reusable cache entry or non-reusable Task/session-owned base. Every + cancellation, deadline, validation failure, or other non-publication path + removes staging idempotently. Cleanup uncertainty writes + `source_cleanup_failed` and terminal Artifacts/events within the active + reservation, but retains the acquisition identity, staging, reservation, + reconciliation record, and lease; disables expiry/readiness; and stops + admission. Verified startup/operator cleanup uses the same atomic + reservation-to-footprint, lease-release, and TTL-start rule as other poisoned + turns. Source-mode failure never falls through. + + A read-only one-shot uses the base plus Task-private runtime; a read-only + session uses the base plus session-private runtime. Adapter preparation keeps + project files unchanged and places invocation config outside the base. A + read-write one-shot receives a unique view; each read-write session turn + receives its R8 candidate from the committed generation. Typed preparation may + project validated project/profile settings, plugins, and MCP declarations into + that writable view/candidate. Project/user `setup` entries and other shell + commands never run automatically. Codex and Pi execute as direct host processes on the same trusted Linux CI - runner as the gateway. The adapter receives the access-appropriate resolved - cwd selected by the logical `workingDirectory`, Task-private runtime paths, - and the effective access mode; it never receives a caller-supplied physical - path or materializer choice. Provider execution never reuses the acquisition + runner as the gateway. The adapter receives resolved cwd, effective access, + Task/session-private runtime and mutable state, plus only a supported separate + host-auth reference; it never receives a caller-supplied physical path or + materializer choice. Provider execution never + reuses the acquisition container and never creates a per-invocation provider container. The CI job, VM, or deployment container is the isolation boundary. AllAgents does not claim containment of hostile repository code, network access by model tools, @@ -620,36 +984,66 @@ registry, or another profile configuration file for the initial use case. durable compare-and-set arbitrates provider terminal outcome, caller cancellation, overall deadline, and shutdown as an internal `outcomeIntent` while the external Task remains nonterminal. The winning intent owns the - stable result or failure code and drives one idempotent abort path: request - graceful adapter abort, wait the configured grace period, send `SIGTERM` to - the process group, then `SIGKILL` after the forced-termination period. - + execution-result facts and drives one idempotent abort path: request graceful + adapter abort, wait the configured grace period, send `SIGTERM` to the process + group, then `SIGKILL` after the forced-termination period. After the direct provider process has settled and bounded evidence collection - finishes, cleanup removes a read-only Task's private runtime state, unmounts - and removes a read-write Task's writable view, and removes any non-reusable - base. One transaction then writes terminal Task status, result/failure, - bounded evidence, exactly one integrity Artifact, produced Artifacts, observed - termination, cleanup outcome, and lease release. Any Task-owned cleanup - failure settles `workspace_cleanup_failed` with - `cleanup.workspace: "failed"` and retains its internal cleanup record for - startup or operator repair; admission stops when an active mount or uncertain - writable view remains. If the direct process does not settle after `SIGKILL`, - one transaction instead writes a failed Task with - `execution_termination_failed`, live provider evidence, and observed - termination, but no filesystem, Git, or produced-Artifact evidence. It retains - the Task runtime, any writable view or non-reusable base, and the lease; makes - readiness false; and stops admission. Startup may release that poisoned lease - only after the recorded process group is confirmed absent following runner - teardown and retained Task-owned state is reconciled; otherwise it remains - unready. - - Repeated cancellation while intent is pending does not re-signal work. - Startup never resumes a session. Gateway shutdown stops admission, commits - shutdown intent, performs the same escalation and settlement rules, and - exits. CI runner teardown is the final orphan boundary. AllAgents does not use + finishes, a one-shot or `closeAfterTurn` path removes private runtime state, + all applicable workspace views, any non-reusable base, and the provider + checkpoint. A continuing read-only session verifies its provider checkpoint; + a continuing read-write session additionally verifies the completed candidate + view and its measured charge while the prior committed generation remains + untouched. + + Settlement persists a prepared record naming the exact prior/candidate + workspace generations and provider checkpoint. One SQLite commit writes the + terminal Task, three reserved Artifacts, produced Artifacts, bounded evidence, + observed termination, Task footprint, terminal events, and either session + removal or an idle lineage head naming the current Task plus the selected + committed provider checkpoint/workspace-generation pointer. The pair may be + newly committed or the provably unchanged prior pair. It also records any + superseded generation for cleanup. The gateway removes that recorded state + and releases the lease only in a final transaction after cleanup succeeds. + Startup deterministically completes a committed cleanup + intent or discards an uncommitted candidate; it never pairs a prior provider + checkpoint with candidate workspace bytes. + + Settlement failure overrides the external execution outcome. If required + Task/session cleanup, provider checkpointing, candidate verification, or + pointer advancement fails or remains uncertain, the failed Task records Core + `execution_extension_failed`, the truthful execution result, exact workspace/ + session failure, cleanup evidence, all three reserved Artifacts, and terminal + events. The session becomes non-resumable unless the adapter proves the prior + provider checkpoint unchanged and the database still points at the untouched + prior workspace generation. When that proof succeeds, the session is idle + with this failed Task as lineage head and the prior checkpoint/workspace pair. + Unreconciled reservations, cleanup record, lease, + and affected Task/session state remain non-expiring while readiness is false. + + If the direct process does not settle after `SIGKILL`, the failed Task records + `execution_termination_failed`, incomplete trajectory, and live-provider/ + termination evidence, but no filesystem/Git or produced-Artifact evidence. + It retains all prior/candidate state, reservation, reconciliation record, and + lease with expiry/readiness disabled. + + Startup/operator repair releases the lease only after the recorded process + group is absent, every observed outliving descendant is gone, and all + retained state is reconciled. It may restore only a provably unchanged + provider checkpoint plus the database-selected committed workspace + generation; otherwise it cleans up and closes the session. Repeated + cancellation does not re-signal work. A canceled continuing session remains + open only when `closeAfterTurn` is false and that same checkpoint/workspace + proof succeeds; its lineage head still advances to the canceled Task. Startup + never resumes an interrupted turn. Gateway shutdown + stops admission, commits shutdown intent, performs the same escalation and + settlement, and exits. + + CI runner teardown is the final orphan boundary. AllAgents does not use cgroups, pidfds, namespaces, nftables, `openat2`, a native platform layer, or - non-bypassable spawn mediation, and does not claim complete descendant - enumeration or hostile-code containment. + non-bypassable spawn mediation. It admits only one gateway-controlled turn at + a time but does not claim complete descendant enumeration: any observed + escaped/outliving descendant retains the lease and readiness remains false + until runner teardown or verified disappearance. **Scope and configuration** @@ -666,26 +1060,42 @@ registry, or another profile configuration file for the initial use case. while new admission is safe and otherwise 503. They reveal no targets, sources, paths, or failure details and do not require A2A headers. Gateway code never copies acquisition credential values into generated workspace files, - requests, logs, Task/Artifact metadata, retained Task views, cache entries, or - provider environments. This is not a redaction or isolation guarantee for + requests, logs, Task/Artifact metadata, retained Task/session views, cache + entries, or provider environments. This is not a redaction or isolation + guarantee for opaque prompts, provider/tool output, structured results, inherited host authentication, native evidence, or produced-Artifact payloads. - R19. Document AI Evals consumption through a Promptfoo custom JavaScript/TypeScript provider implementing Promptfoo's `ApiProvider`. `constructor(options: ProviderOptions)` requires and retains a nonempty `options.id`, validates `options.config`, and `id()` returns that ID. Static - config contains the gateway endpoint, target ID, optional default logical - `workingDirectory`, optional `workspaceAccess` defaulting to `readWrite`, and - exactly one closed source mode: repository mode materializes the complete - configured repository set and carries only an optional revision map keyed by - declared repository name; snapshot mode carries one declared snapshot name - with OCI and workspace-manifest digests. + config contains the private gateway endpoint, target ID, optional default + logical `workingDirectory`, optional `workspaceAccess` defaulting to + `readWrite`, and exactly one closed source mode: repository mode materializes + the complete configured repository set and carries only an optional revision + map keyed by declared repository name; snapshot mode carries one declared + snapshot name with OCI and workspace-manifest digests. `callApi(prompt, context?, options?)` may apply the exact `context?.vars?.allagentsSource` leaf overrides, may replace the default - selector through `context?.vars?.allagentsWorkingDirectory`, and may replace - access through `context?.vars?.allagentsWorkspaceAccess`; missing context - retains static values. Dynamic source values remain limited as defined below. - The working-directory variable is exactly `{ kind: "workspaceRoot" }` or + selector through `context?.vars?.allagentsWorkingDirectory`, may replace + access through `context?.vars?.allagentsWorkspaceAccess`, and may select + `{ mode: "oneShot" }`, `{ mode: "start", closeAfterTurn? }`, or + `{ mode: "resume", sessionId, previousTaskId, closeAfterTurn? }` through + `context?.vars?.allagentsSession`. An absent session value means `oneShot`; + missing context otherwise retains static values. + For executable Promptfoo multi-turn tests, the provider also accepts + `allagentsConversation: { id: ConfigName, action: "start" | "continue" | + "close" }`, mutually exclusive with `allagentsSession`. It keeps an + instance-local map from conversation ID to the last terminal projection. + `start` requires no entry; `continue` and `close` require an idle entry and + send its exact `sessionId`/`headTaskId`, with `close` setting + `closeAfterTurn`. An idle terminal projection atomically replaces the map + entry; `closed` or `notResumable` removes it. Interleaved IDs remain isolated, + and a second in-flight call for one ID fails locally. Explicit + `allagentsSession` remains the crash-recovery/manual handoff using IDs stored + by the evaluator. + Dynamic source values remain limited as defined below. The working-directory + variable is exactly `{ kind: "workspaceRoot" }` or `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }`; access is exactly `readOnly` or `readWrite`. Unknown members, invalid relative paths, URLs, physical or configured @@ -693,16 +1103,20 @@ registry, or another profile configuration file for the initial use case. choices, and provider permission policy fail before provider execution. The provider sends `SendMessage` with `configuration.returnImmediately: true`, - captures the accepted Task ID, and calls `SubscribeToTask`; a terminal-before- - subscribe race or broken stream falls back to `GetTask` and resubscription - within the same deadline. A deadline or `options?.abortSignal` issues exactly - one `CancelTask` with a fresh bounded cleanup signal rather than the already - aborted request signal. One `callApi` creates one A2A Task and maps terminal - output, usage, Task/Artifact IDs, structured result, and logical provenance - into `ProviderResponse`. Admission or terminal failure maps a safe human - message to `error` and stable `code`, `retryable`, and accepted `taskId` to - `metadata`. AI Evals owns the provider implementation. AllAgents publishes the - protocol and YAML examples without importing Promptfoo provider code or adding + captures the accepted Task and context IDs, and calls `SubscribeToTask`; a + terminal-before-subscribe race or broken stream falls back to `GetTask` and + resubscription within the same deadline. A resume sends the retained context + ID and head Task as its sole task reference. A deadline or + `options?.abortSignal` issues exactly one `CancelTask` with a fresh bounded + cleanup signal rather than the aborted request signal. One `callApi` creates + one A2A Task and maps terminal output, usage, Task/Artifact IDs, structured + result, logical provenance, and provider-reported cached input tokens into + `ProviderResponse`; the validated + `Task.metadata[profileUri].session` projection is copied unchanged to + `ProviderResponse.metadata.session`. Admission/terminal failure maps a safe + human message, stable `code`, `retryable`, accepted `taskId`, and any terminal + session projection. AI Evals owns chaining/provider code. AllAgents publishes + the protocol and YAML examples without importing Promptfoo provider code or Promptfoo as a runtime dependency. ### Key Flows @@ -711,86 +1125,92 @@ registry, or another profile configuration file for the initial use case. 1. Resolve cwd or `--workspace`, user workspace, project-specific state root, disjoint immutable-base cache and invocation roots, cache/task retention, workspace materializer, listener, advertised URL, digest-pinned acquisition - image, Docker endpoint, source credentials, provider homes, and configured - provider executable overrides. + image, Docker endpoint, source credentials, provider auth locations, and + configured provider executable overrides. 2. Validate the SQLite state root, cache/invocation roots, workspace identity, materializer policy, and static acquisition-image reference; compile repository, snapshot, and target catalogs; verify Codex SDK and Pi RPC/ package compatibility; and check any global binary override exactly. Do not contact Docker or the acquisition registry at startup. - 3. Reconcile interrupted Tasks by removing any recorded acquisition container + 3. Reconcile interrupted turns by removing any recorded acquisition container and orphan staging, terminating any recorded Linux provider process group, - and marking the Task failed without resuming it. Release the durable lease - only after container removal, confirmed process-group absence, and cleanup - of recorded Task-owned runtime, view, and non-reusable base; otherwise keep - readiness false and the lease poisoned. - 4. Bind the requested address, including `0.0.0.0` when explicit; serve - metadata-only health/readiness probes; and publish one Agent Card whose - absolute interface URL, required extension, and target allowlist match the - validated configuration. - -- F2. **Acquire or reuse repositories and execute** + and reconciling recorded Task/session-owned runtime, view, non-reusable + base, and provider checkpoint state. When an App mint intent is unresolved + or token revocation was pending, confirm the container and local secret are + absent and atomically persist the conservative- or exact-expiry tombstone. + Terminalize the Task failed and release the durable lease only after those + conditions hold; restore only a provably unchanged committed session + checkpoint, otherwise poison the session. Keep readiness false while + uncertainty remains. + 4. Bind loopback HTTP or one specific private address with native TLS; reject + wildcard/public binds, invalid TLS files, and public advertised resolution + before listening. Serve metadata-only probes and publish one Agent Card + whose absolute interface URL, Profile URN, and target/mode allowlist match + the validated direct-listener or loopback-plus-private-proxy topology. + +- F2. **Acquire or reuse repositories and execute one turn** 1. Negotiate A2A version and the required extension, then validate the strict - request, one text Part, target, repository-name/revision map, logical - working-directory selector, workspace access, result schema, deadline, and - deployment-wide idempotency claim. - 2. In one SQLite transaction, create or replay the claim and Task and acquire - the execution lease before starting work. Capacity failure settles the Task - with `execution_capacity_unavailable` and launches neither Docker nor a - provider. - 3. When every effective revision is a full commit, derive the immutable-base - key and pin a matching validated cache entry. On a miss or for mutable - branch/tag revisions, classify App applicability and mint a fresh - installation token or resolve the configured `gh` token only according to - eligibility. Probe Docker and the exact digest-pinned acquisition image, - then start it with only private staging, compiled request/policy, and the - selected token. The container fetches hermetically, verifies full commits, - writes the typed manifest, exits, and is removed. + request, one text Part, target, session mode/context/head reference, + repository-name/revision map, logical working-directory selector, workspace + access, result schema, deadline, and deployment-wide idempotency claim. + 2. In one SQLite transaction, create or replay the claim/Task, create or lock + the session/head when selected, reserve bytes, and acquire the execution + lease before starting work. Capacity failure settles the Task; `start` + closes its new session, while `resume` advances the lineage head to this + failed Task but preserves the prior committed checkpoint/workspace pair. + The matching terminal projection is reported; no Docker/provider launches. + 3. For a session resume, use its pinned base/runtime, committed workspace + generation when writable, and provider checkpoint. Otherwise, when every + effective revision is a full commit, derive the immutable-base key and pin + a matching validated cache entry. On a miss or for mutable branch/tag + revisions, select source credentials, probe Docker and the exact + digest-pinned acquisition image, and run acquisition with only private + staging, compiled request/policy, and selected credential. 4. On acquisition, revoke any App token, destroy source credentials, validate - the manifest/staging on the host, and atomically promote it to either a - reusable cache entry or a non-reusable Task-owned base. A cache hit performs - none of those acquisition, Docker, or credential operations. Every - non-publication path removes staging. - 5. For `readOnly`, resolve the logical cwd directly in the base and create only - Task-private runtime state. For `readWrite`, create the unique writable view - through the selected materializer, then resolve cwd in that view. Run - access-appropriate typed preparation and start the adapter there as a direct - host process group with explicit environment and existing host auth. + the manifest/staging on the host, and atomically promote it to a reusable + cache entry or non-reusable Task/session-owned base. A cache hit performs no + acquisition, Docker, or credential operation. + 5. For read-write sessions, reserve/materialize a per-turn candidate from the + base or committed generation. Resolve logical cwd in the read-only base, + one-shot view, or candidate; create Task/session-private runtime; run typed + preparation; and start/resume the adapter as a direct host process group + with an explicit environment and supported separated auth/state paths. Validate structured results while capturing bounded live events. - 6. After the direct provider process settles and cancellation escalation - finishes, collect bounded truthful evidence; remove private runtime, any - writable view, and any non-reusable base; unpin a reusable base; and - atomically settle the Task, Artifacts, observed termination, cleanup, and - lease. If the process does not settle after `SIGKILL`, publish - `execution_termination_failed` without filesystem/Git evidence, retain - Task-owned state and the lease for runner teardown, make readiness false, - and stop admission until startup reconciliation confirms the process group - absent and cleans retained state. - -- F3. **Acquire or reuse an OCI snapshot and execute** - 1. Perform the same version/extension validation and atomic - claim+Task+lease transaction as F2. - 2. Pin a cache entry matching the named snapshot, manifest digest, workspace- - manifest digest, catalog/layout digest, and acquisition-contract version. - On a miss, probe Docker and the exact digest-pinned acquisition image, then - start it with the digest-pinned reference, staging mount, exact-host - registry credentials/CA material, and frozen network/archive policy. Pull - and verify the direct image manifest, workspace-manifest config, and - distributable layers; apply changesets in order; enforce all limits; and - emit the typed manifest. + 6. After direct-process settlement and cancellation escalation, collect + bounded truthful evidence. For a continuing session, verify/checkpoint the + provider handle and retain measured private state; otherwise remove private + runtime/view/non-reusable base and dispose the handle. Atomically settle the + Task, Artifacts, observed termination, cleanup/checkpoint, session head/ + close state, charges, and lease. A nonsettling process or uncertain + checkpoint/cleanup retains owned state and the lease, poisons the session, + makes readiness false, and stops admission pending verified reconciliation. + +- F3. **Acquire or reuse an OCI snapshot and execute one turn** + 1. Perform the same version/extension/session validation and atomic + claim+Task+session-head+lease transaction as F2. + 2. For a resume, use the pinned session base/runtime/view. Otherwise pin a + cache entry matching the named snapshot, manifest digest, workspace-manifest + digest, catalog/layout digest, and acquisition-contract version. On a miss, + probe Docker and the exact digest-pinned acquisition image, then start it + with the digest-pinned reference, staging mount, exact-host registry + credentials/CA material, and frozen network/archive policy. Pull and verify + the direct image manifest, workspace-manifest config, and distributable + layers; apply changesets in order; enforce all limits; and emit the typed + manifest. 3. On a miss, remove the container and registry material, validate and publish the immutable base on the host, or remove staging on every non-publication - path. Then select the read-only shared base or read-write Task view and - settle through the same bare-metal and cleanup path as F2. Provider + path. Then select the one-shot or session-owned read-only/private writable + state and settle/checkpoint through the same bare-metal path as F2. Provider execution never occurs in the acquisition container. - F4. **Cancel** 1. Atomically persist cancellation intent if the Task remains cancelable. - 2. For acquisition, stop and remove the Docker container, source material, and + 2. For acquisition, stop/remove the Docker container, source material, and unpublished staging. For provider work, request graceful adapter abort, - then escalate to process-group `SIGTERM` and `SIGKILL` within bounded - periods. Preserve only observed, bounded evidence; clean up and settle - canceled or failed according to the durable winning intent. + then escalate to process-group `SIGTERM`/`SIGKILL` within bounded periods. + Preserve only observed bounded evidence; settle the Task according to the + winning intent; retain the session only if the adapter proves a consistent + checkpoint, otherwise poison it pending verified cleanup. 3. Repeated cancellation while intent is pending does not re-signal work. Cancellation after any terminal state returns A2A `TaskNotCancelableError`. @@ -807,32 +1227,41 @@ registry, or another profile configuration file for the initial use case. - F6. **Invoke from Promptfoo** 1. Promptfoo constructs the AI Evals-owned TypeScript provider with `ProviderOptions`; the provider retains the ID and validates - `options.config` containing the private-network endpoint, target, optional - default logical working directory, optional default workspace access, and - one closed source-mode object. - 2. `callApi(prompt, context?, options?)` applies only valid - `context?.vars?.allagentsSource` leaves and optional strict - `context?.vars?.allagentsWorkingDirectory` and - `context?.vars?.allagentsWorkspaceAccess` replacements, creates and retains - one high-entropy invocation key, and sends one A2A Message with - `configuration.returnImmediately: true`. - 3. After receiving the Task ID, subscribe to terminal updates. Resolve a - terminal-before-subscribe or disconnected-stream race through `GetTask` - and bounded resubscription. Deadline or abort sends `CancelTask` once with - a fresh cleanup signal. - 4. Return terminal text or validated structured output in - `ProviderResponse.output`; map `inputTokens -> prompt`, + `options.config` containing the private endpoint, target, optional default + logical working directory/access, and one closed source-mode object. + 2. `callApi(prompt, context?, options?)` applies only valid source/cwd/access + replacements and either strict `allagentsSession` or + `allagentsConversation`. The latter resolves start/continue/close through + the provider's conversation-ID map. The call creates and retains one + high-entropy invocation key per turn and sends one A2A Message with + `configuration.returnImmediately: true`; resume carries the stored context + ID and exact prior head Task reference. + 3. After receiving the Task/context IDs, subscribe to terminal updates. + Resolve a terminal-before-subscribe or disconnected-stream race through + `GetTask` and bounded resubscription. Deadline/abort sends `CancelTask` once + with a fresh cleanup signal. + 4. Validate the terminal Sessions projection and update/remove the selected + conversation-map entry before returning. Put terminal text or structured + output in `ProviderResponse.output`; map `inputTokens -> prompt`, `outputTokens -> completion`, `cachedInputTokens -> cached`, and - `totalTokens -> total`; and put other usage plus Task, Artifact, logical - source, termination, cleanup, and stable failure facts in `metadata`. - Admission or terminal failure returns a safe `error`. + `totalTokens -> total`; and put other usage plus Task/session, Artifact, + logical source, termination, cleanup, and stable failure facts in + `metadata`. Admission/terminal failure returns a safe `error`. + 5. A two-turn fixture uses one explicit conversation ID with `start`, then + `close`; the provider turns the second call into resume with the first + terminal projection's exact context/head. Interleaved fixtures use distinct + IDs. Explicit session IDs support recovery across provider-process loss. + Cached-token usage is observation, never a guaranteed hit. ### Acceptance Examples -- AE1. A caller on a permitted Tailscale or firewalled network discovers the - gateway through its configured HTTPS interface URL while it is bound to - `0.0.0.0`, selects `codex-review`, and receives one durable Task without an - application credential. +- AE1. A caller on permitted Tailscale/private routing discovers either a native + specific-private-address TLS listener or a private HTTPS terminator whose + backend is loopback-only, selects `codex-review`, and receives one durable Task + without an application credential. Startup fixtures reject wildcard/public + binds, a private direct bind without TLS key/cert, a public URL literal, a DNS + name with any public address, and an `http:` remote advertised URL before + listening. Contract URNs are never fetched as endpoints. - AE2. Any reachable caller can list, retrieve, and cancel a Task created by another reachable caller and inspect its embedded Artifacts through `GetTask` or `ListTasks(includeArtifacts: true)`; documentation states this shared trust @@ -846,13 +1275,18 @@ registry, or another profile configuration file for the initial use case. - AE5. Repository mode accepts declared names and revision overrides, rejects an undeclared name or URL override, and records the resolved full commits. - AE6. On a base-acquisition miss, including a mutable branch/tag request, an - applicable GitHub App bypasses its token cache, mints a repository-scoped - read-only token with adequate lifetime, validates and revokes it after - acquisition. A cache hit resolves no source credential. + applicable GitHub App bypasses its token cache, persists a conservative-expiry + pre-mint intent, mints a repository-scoped read-only token with adequate + lifetime, replaces the intent with the exact expiry, validates the token, and + revokes it after acquisition. A cache hit resolves no source credential. A corroborated existing repository with no applicable installation uses the configured `gh` account. An uncorroborated 404, unknown applicability, auth, mint, validation, or revocation failure does not fall through to `gh` or start - the provider. + the provider. Process-kill barriers before the mint request, after an + ambiguous/successful mint response, and before revocation confirmation remove + the acquisition container and local secret on restart, fail the Task with + `gateway_restarted`, persist the conservative- or exact-expiry tombstone, and + never report confirmed revocation. - AE7. Snapshot mode accepts a direct image manifest with matching manifest, config/workspace, and layer digests; applies gzip/zstd layers and whiteouts in order; and enforces every fixed limit. Same-origin metadata redirects work; @@ -869,31 +1303,46 @@ registry, or another profile configuration file for the initial use case. verified; source identity includes completeness and ordered layer digests without origins. One hundred Tasks using the same immutable identity perform one full acquisition while the entry remains cached and pinned correctly. - A branch or tag request acquires a non-reusable Task-owned base, never enters - the reusable cache, and removes that base during settlement or reconciliation. + A branch or tag request acquires a non-reusable Task/session-owned base, never + enters the reusable cache, and removes that base during one-shot/closing + settlement or reconciliation. - AE9. Identical invocation-key replay, including after a lost response, returns the original Task. Reusing the key with changed target, source, prompt, logical - working directory, workspace access, or result schema conflicts. Separate - read-only Tasks may share one physical base and cwd while keeping private - runtime state. Separate read-write Tasks receive independent writable views - even when their logical working-directory selectors are equal. -- AE10. Cancellation during a Git or OCI base acquisition stops and removes the - acquisition container and unpublished staging. Cancellation during Codex or - Pi requests graceful abort, then sends process-group `SIGTERM` and `SIGKILL` - on schedule. - The Task records observed termination and cleanup without claiming complete - descendant quiescence. If the direct process does not settle, the gateway - retains the Task's runtime and any writable view plus the lease, omits - filesystem/Git evidence, stops admission, and remains unready until post- - teardown startup reconciliation confirms the group absent. -- AE11. Kill fixtures before and after durable Task/lease creation, acquisition- - container start, provider process-group recording, and response - acknowledgment leave one recoverable SQLite truth. Restart removes the - recorded acquisition container, orphan staging, and safe Task-owned state; - best-effort terminates the recorded process group; and turns the interrupted - Task into one terminal failure without resuming a provider session. It retains - the lease and stays unready unless provider absence and required cleanup are - confirmed. Terminal Tasks and Artifacts remain until expiry. + working directory, workspace access, result schema, session mode, context ID, + prior Task, or close-after-turn value conflicts. Separate one-shot read-only + Tasks may share one physical base while keeping private runtime. Separate + one-shot read-write Tasks receive independent views. A resumed read-write + session pins the same base/prior committed generation and exact head while + creating a new Task-specific candidate. +- AE10. Cancellation during Git/OCI acquisition stops and removes the container + and unpublished staging. Cancellation during Codex/Pi requests graceful abort, + then process-group `SIGTERM` and `SIGKILL` on schedule. The Task records + observed termination/cleanup without claiming descendant quiescence. Any + observed escaped/outliving descendant retains the lease until verified gone + or runner teardown. Cancellation leaves a non-closing session open only when + the adapter proves a consistent provider checkpoint paired with the unchanged + committed workspace generation; a closing session is removed after verified + cleanup. Otherwise it poisons the session. Uncertain acquisition cleanup, + checkpointing, candidate cleanup, or provider settlement retains the active + reservation, reconciliation record, affected Task/session state, and lease; + stops admission; stays unready/non-expiring across configured TTLs; and + reconciles before releasing the lease or restoring/closing the session. +- AE11. Kill fixtures before/after Task/session-head/lease creation, mutable + workspace candidate creation, durable pre-mint App intent, token response, + exact-expiry replacement, revocation confirmation, acquisition-container + start, provider process-group recording, provider checkpoint, prepared + settlement, workspace-generation pointer commit, old-generation cleanup, + terminal transaction, and response acknowledgment leave one recoverable + SQLite truth. Restart removes recorded containers/staging, terminates the + recorded process group, waits on any observed outliving descendant, persists + required revocation tombstones, and terminalizes the interrupted Task without + replay. It discards an uncommitted candidate or retains the database-selected + committed generation and restores only its provably unchanged provider + checkpoint; otherwise it poisons/retains the session. Lease/readiness remain + held/false until provider/observed-descendant absence, credential destruction, + tombstone durability, and owned-state reconciliation are confirmed. + Terminal Tasks/Artifacts remain until Task expiry independently of a live + session. - AE12. A valid structured result survives later check or evidence failure as a valid result with an overall failed Task; invalid or absent results are never published as valid. @@ -909,13 +1358,19 @@ registry, or another profile configuration file for the initial use case. - AE15. Barrier-controlled provider-terminal, caller-cancel, deadline, and shutdown races durably select one internal intent and one abort path during Docker acquisition, preparation, Codex, Pi, or evidence. Normal settlement - writes status, integrity Artifact, bounded evidence, result/failure, observed - termination, cleanup, and lease release in one transaction. The - `execution_termination_failed` exception writes the terminal failure without - filesystem/Git evidence and deliberately retains the poisoned lease. + writes the Task, three reserved Artifacts, produced Artifacts, bounded + evidence, result/failure, observed termination, cleanup, exact footprint, + terminal Artifact events, terminal status event, and lease release in one + transaction; replay after a crash includes that terminal event sequence. + Nonsettling-provider and uncertain cleanup/checkpoint cases retain the + reservation, reconciliation record, Task/session-owned state, and poisoned + lease; crossing configured Task/session TTL cannot delete either before + reconciliation. Verified repair atomically replaces the reservation with the + exact footprint, releases the lease, and restores a provable checkpoint or + closes the session. - AE16. Repeated cancel while cancellation is pending is idempotent; cancel - after canceled, completed, failed, or rejected returns - `TaskNotCancelableError`. + after canceled, completed, failed, timed out, or rejected returns + `TaskNotCancelableError` without changing the Core `alreadyTerminal` outcome. - AE17. A workspace containing `setup` shell entries never executes them during acquisition or startup. The acquisition image receives only staging, source-only credentials, exact source network policy, and archive limits; it @@ -925,56 +1380,91 @@ registry, or another profile configuration file for the initial use case. omit unrelated ambient values. - AE18. Evidence collection starts only after the direct provider process has settled and process-group escalation has completed. Git inspection disables - repository-controlled execution, and the integrity Artifact distinguishes - observed direct-process termination and cleanup from full descendant + repository-controlled execution, and the workspace-integrity Artifact + distinguishes observed direct-process termination and cleanup from full quiescence. Documentation explicitly states that AllAgents provides no hostile-code or model-tool secret-isolation guarantee. -- AE19. The 1001st unexpired retained Task is rejected with - `retention_capacity_exhausted`; no retained Task is evicted before TTL. While - one Task holds the execution lease, a barrier-controlled second request - settles `execution_capacity_unavailable` and launches no acquisition or - provider child; races and restart never produce two lease holders. +- AE19. The 1001st unexpired retained Task and any admission for which + `settledFootprints + activeReservations + maxTaskBytes` exceeds the aggregate + limit are rejected with `retention_capacity_exhausted`; no retained Task is + evicted before TTL. Boundary fixtures grow an accepted Task until its projected + event-plus-trajectory bytes reach the terminal-tail reserve, prove the next + event is not committed or published, then settle a maximal 1 MiB valid result + followed by workspace-cleanup failure without dropping that result or + exceeding the reservation. The transaction replaces the reservation with a + measured terminal footprint containing all mandatory Artifacts/events. + `ListTasks` stops before its exact serialized response budget, returns a cursor + to the first omitted Task, and never splits a Task. Invalid budget + relationships fail startup. While one Task holds the execution lease, a + barrier-controlled second request settles `execution_capacity_unavailable` + and launches no acquisition or provider child; races and restart never produce two lease holders. - AE20. Official HTTP+JSON client fixtures send `A2A-Version: 1.0`, exercise required-extension activation and both `SendMessage` modes, preserve unrelated metadata, verify standard `google.rpc.Status` errors, and cover every - `ListTasks` filter, cursor, order, response field, and artifact-inclusion rule. - A terminal Task contains one extension-marked integrity Artifact with its - effective logical working directory, workspace access, and referenced + `ListTasks` filter, cursor, order, byte-budget, response-field, and artifact- + inclusion rule. They exercise oneShot/start/resume projection, server-generated + context IDs, exact prior-Task references, and immutable Task-per-turn behavior. + A terminal Task contains one Core outcome, one ordered Core execution + trajectory, one AllAgents workspace-integrity Artifact, and referenced produced Artifacts using unified Parts. - AE21. The AI Evals Promptfoo fixture has a top-level prompt and disables - sharing, caching, result writes, and concurrency above one. It loads one - repository-mode and one snapshot-mode provider, sends only closed logical - source, working-directory, and workspace-access data, replaces cwd and access - per trial through `allagentsWorkingDirectory` and - `allagentsWorkspaceAccess`, retains one invocation key across ambiguous - retries, and cancels an accepted Task on abort. Two read-only trials for the - same immutable source share the validated base; two read-write trials receive - independent disposable views. Both return scorable output, normalized token - usage, and Task/Artifact/logical-provenance metadata. Safe failure metadata - includes code, retryability, and accepted Task ID. Calls with omitted context - work; unknown variables, invalid or escaping relative directories, physical - paths, mutable revisions, origins, destinations, materializer choices, or - undeclared names fail before provider execution. + sharing, Promptfoo result caching, result writes, and concurrency above one. + It loads both source modes, sends only closed logical inputs, retains one + invocation key across ambiguous retries, and cancels an accepted Task on + abort. One-shot read-only trials share a base; read-write trials use disposable + views. Two interleaved explicit conversation IDs each complete start then + close: the provider stores each terminal projection, sends that ID's exact + returned context/head on turn two, observes turn-one conversation/workspace + changes, and removes the map entry/session after close. A provider restart + resumes once through explicit `allagentsSession` IDs stored by the evaluator. + Native cached-input tokens are reported when present; no cache hit is required. + Safe failure metadata includes code, retryability, accepted Task ID, and + session state. Calls with + omitted context work; unknown variables, stale heads, changed pinned inputs, + invalid or escaping relative directories, physical paths, mutable revisions, + origins, destinations, materializer choices, or undeclared names fail before + provider execution. ### Success Criteria - `allagents-gateway serve` starts from a real workspace with no deployment YAML. -- Explicit loopback, private-interface, and `0.0.0.0` listeners work with a - distinct valid advertised interface URL; health/readiness reflect admission. -- The official A2A client exercises version and extension negotiation, both send - modes, stream, get, complete list/pagination semantics, subscribe, replay, - cancel, terminal cancel errors, Task-embedded Artifacts, standard HTTP+JSON - errors, and expiry. +- Loopback HTTP, loopback behind private HTTPS ingress, and native + specific-private-address TLS listeners work with the matching advertised URL; + wildcard/public binds and public URL resolution fail startup; probes reflect + admission. +- The official A2A client exercises version and required-profile negotiation, + both send modes, stream, get, complete list/pagination semantics, subscribe, + replay, cancel, terminal cancel errors, Task-embedded Artifacts, standard + HTTP+JSON errors, and expiry. Hand-authored HEC Core vectors independently + exercise the complete invocation DTO, neutral states and cancel dispositions, + the closed state/failure/result/retryability outcome matrix and negative + combinations, including retryable and non-retryable extension failures, + ordered progress/tool events, canonical structured payloads, deterministic + prefix truncation, exact outcome/trajectory descriptor linkage, + result/usage/outcome settlement, idempotency, deadlines, + cancellation/provider/deadline races, and portable failure retryability + through a transport-neutral test adapter and the A2A binding. Sessions vectors + independently exercise oneShot/start/resume/close, exact predecessor, + linearization, pinning, expiry, capacity, restart/poison handling, immutable + committed workspace generations, per-turn candidates, crash before/after + provider/workspace pointer commit, and truthful cache usage. Binding vectors + exercise session/context ID, prior-Task and terminal-session projection, Core- + to-A2A state/cancel/event/Artifact/error mapping, list/response budgets, replay, + and unknown-field rejection. Workspace/composed-profile vectors exercise + module composition, produced-Artifact bytes, integrity, and cleanup. - An AI Evals-style Promptfoo custom-provider fixture consumes secure-default - YAML for both source modes, selects a logical cwd and access mode per trial, - proves shared-base reuse for read-only trials and independent disposable views - for read-write trials, propagates post-acceptance cancellation, and maps a - terminal Task to `ProviderResponse` without adding Promptfoo to the AllAgents - runtime. + YAML for both source modes, applies logical cwd/access per trial, proves + one-shot base/view behavior, isolates two interleaved explicit conversation + IDs, uses each stored terminal projection for exact resume, closes both, + recovers once through explicit evaluator-stored session IDs after provider + restart, propagates post-acceptance cancellation, reports provider-native + cached-input usage without guaranteeing a hit, and maps each terminal Task to + `ProviderResponse` without adding Promptfoo to the AllAgents runtime. - Built-in Codex/Pi and gateway-enabled profile targets pass one backend - conformance suite, including reserved-ID collisions, existing-host-auth - behavior, explicit environment construction, cancellation escalation, and - Codex native-schema gating. + conformance suite, including reserved-ID collisions, host-auth/private-state + separation, explicit environments, per-mode advertisement, one-shot and + start/resume/checkpoint/dispose, cancellation escalation, native cache-usage + truthfulness, and Codex native-schema gating. - Direct Git and OCI snapshot acquisition in the exact digest-pinned image produces equivalent typed manifests and truthful provenance; repeated immutable requests reuse one validated base and the image is removed before @@ -996,25 +1486,29 @@ registry, or another profile configuration file for the initial use case. **In scope** -- A2A 1.0 HTTP+JSON and the required AllAgents extension. -- One gateway process and one active invocation at a time initially. +- The transport-neutral Harness Execution Contract Core and Sessions, their A2A + 1.0 HTTP+JSON binding, the AllAgents coding-workspace extension, and their one + required composed coding-execution Profile Extension. +- One gateway process and one active gateway-controlled turn at a time initially. - Built-in and gateway-enabled profile-backed Codex/Pi host execution. - Docker-only acquisition of direct declared Git repositories and named OCI workspace snapshots when no reusable validated base exists. -- Reusable immutable bases and Task-private runtime state for read-only - execution; non-reusable Task-owned bases for mutable revisions; unique - disposable writable views for read-write execution. +- Reusable immutable bases with Task/session-private runtime state for read-only + execution; non-reusable Task/session-owned bases for mutable revisions; + disposable one-shot writable views and retained session-private writable + views for read-write execution. - Logical workspace-root or declared-repository-relative provider cwd plus explicit `readOnly | readWrite` access selected at runtime. - GitHub App and configured GitHub CLI acquisition credentials. -- Local durable Task/evidence storage, bounded base caching, process-group - cancellation, materialization cleanup, and provenance. -- Listen addresses including `0.0.0.0`. +- Local durable Task/session/evidence storage, bounded base caching, + process-group cancellation, checkpoint/workspace cleanup, and provenance. +- Loopback and trusted-private-network access through either loopback/private-TLS + ingress or a native specific-private-address TLS listener. **Out of scope** -- Application authentication, tenant isolation, caller-private Tasks, and public - Internet hardening. +- Application authentication, tenant isolation, caller-private Tasks, and all + public-Internet exposure. - `gateway.yaml`, `worker.yaml`, remote workers, mTLS worker links, Kubernetes routing, autoscaling, and multiple gateway replicas. - Caller-provided physical workspaces/cwds, repository or registry origins, @@ -1023,6 +1517,9 @@ registry, or another profile configuration file for the initial use case. - GitHub Enterprise Server and multiple ordered Apps/accounts in the initial delivery. - OpenCode, Claude, Copilot, OMP, arbitrary CLI, and TUI adapters. +- A Responses/UHP binding, session branching, and concurrent turns; version one + implements only linear resumable Sessions through A2A, while any second + transport or fork semantics remain future independently versioned work. - Evaluation orchestration and automatic retries. - Per-provider containers; cgroups, pidfds, namespaces, nftables, `openat2`, a native platform layer, non-bypassable spawn mediation, hostile-code @@ -1037,13 +1534,21 @@ registry, or another profile configuration file for the initial use case. - [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) - [Source credential broker precedents](../research/source-credential-broker-precedents.md) - [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) +- [A2A life of a Task and multi-turn contexts](https://a2a-protocol.org/v1.0.0/topics/life-of-a-task/) - [A2A extension guide](https://a2a-protocol.org/latest/topics/extensions/) - [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) +- [Unified Harness Protocol](https://unifiedharnessprotocol.org/) +- [UHP Sessions](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/sessions.md) +- [UHP governance](https://github.com/HarnessRouter/harnessrouter/blob/76c0d0a55682f953ca10c44bdd0645a4475a4ebd/protocol/GOVERNANCE.md) +- [UHP implementations](https://github.com/HarnessRouter/harnessrouter/blob/76c0d0a55682f953ca10c44bdd0645a4475a4ebd/protocol/IMPLEMENTATIONS.md) - [Bun workspaces](https://bun.sh/docs/install/workspaces) - [Bun SQLite](https://bun.sh/docs/api/sqlite) - [Promptfoo custom providers](https://www.promptfoo.dev/docs/providers/custom-api/) - [Promptfoo configuration reference](https://github.com/promptfoo/promptfoo/blob/main/site/docs/configuration/reference.md) -- [OpenAI Codex SDK](https://developers.openai.com/codex/sdk/) +- [OpenAI Codex SDK thread continuation](https://developers.openai.com/codex/codex-sdk) +- [OpenAI Codex app-server thread/turn model](https://developers.openai.com/codex/app-server) +- [OpenAI conversation state](https://developers.openai.com/api/docs/guides/conversation-state) +- [OpenAI prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching) - [OpenAI structured outputs](https://developers.openai.com/api/docs/guides/structured-outputs/) - [GitHub App installation tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) - [Git credential helpers](https://git-scm.com/docs/gitcredentials) @@ -1064,34 +1569,48 @@ registry, or another profile configuration file for the initial use case. ### Key Technical Decisions -- KTD1. **Gate the official A2A JavaScript SDK in the shipped Bun server - direction before adopting it.** Pin the exact SDK version and prove Agent Card - discovery, both send modes, streaming, Task get/list/cancel, resubscription, - extension negotiation, metadata preservation, and HTTP error envelopes by - driving the production gateway server with an independent official client - fixture. Implement the SDK's public request-handler seam while AllAgents owns - UUIDv7 creation, atomic `createOrReplay`, monotonic settlement, listing, - retention, expiry, and HTTP+JSON error details. Do not use an SDK default store - as the transaction boundary or replace A2A with a bespoke protocol. -- KTD2. **Keep contracts portable and generated from narrow TypeScript - packages.** `packages/workspace-config` owns project/user parsing and compiled - catalogs; `packages/execution-contracts` owns A2A extension, request, - Task/Artifact, idempotency, result, error, adapter, and evidence schemas; +- KTD1. **Gate the official A2A JavaScript SDK and the first Harness Execution + Contract binding in the shipped Bun server direction.** Pin the exact SDK + version and prove Agent Card discovery, both send modes, streaming, Task + get/list/cancel, resubscription, required-profile negotiation, metadata + preservation, and HTTP error envelopes by driving the production gateway + server with an independent official client fixture. Run the transport-neutral + Core semantic vectors against the A2A binding, then run binding-specific + one-Message/one-Task, state/event, schema, context-ID, Artifact, and standard + error conformance. Implement the SDK's public request-handler seam while + AllAgents owns UUIDv7 creation, atomic `createOrReplay`, monotonic settlement, + listing, retention, expiry, and HTTP+JSON error details. Do not use an SDK + default store as the transaction boundary, fork A2A core types, or expose a + second version-one wire protocol. +- KTD2. **Keep the harness contract transport-neutral inside one narrow + TypeScript package.** `packages/workspace-config` owns project/user parsing and + compiled catalogs. `packages/execution-contracts` owns the transport-neutral + Harness Execution Contract Core and Sessions types/semantic vectors, the A2A + binding, the AllAgents coding-workspace extension, composed request/Task/ + Artifact schemas, adapter contracts, and evidence schemas. Core types import + no Sessions, A2A, UHP, workspace, provider, or evaluator types. Sessions imports + Core identity/outcome types but no A2A types. The binding, Sessions, and + workspace groups compose one required v1 A2A Profile URN because the gateway + supports one-shot and resumable execution. A future binding must pass the same + Core and Sessions semantic vectors plus its own wire suite. `packages/acquisition-contracts` owns acquisition requests, typed manifests, - OCI snapshot rules, and fixed limits. Check generated JSON Schemas and golden - accepted/rejected examples into `contracts/` for the host gateway, acquirer - image, public docs, and consumer fixtures. Do not create `core`, `common`, or - a speculative shared package. + OCI snapshot rules, and fixed limits. Check the normative Core, Sessions, + A2A-binding, and workspace-extension text, generated JSON Schemas, + hand-authored conformance vectors, and golden accepted/rejected examples into + `contracts/` for the host gateway, acquirer image, public docs, and consumer + fixtures. Do not create a generic `core`/`common` package, a second v1 wire + endpoint, or a speculative shared package. - KTD3. **Use one gateway supervisor, not a remote worker protocol.** The Bun gateway owns Task state, immutable-base caching, Task runtime/view materialization, provider child processes, evidence, termination, and cleanup. It creates an ephemeral Docker container only when no reusable validated base exists, removes it before provider execution, and launches Codex/Pi directly on the trusted host in Linux process groups. -- KTD4. **Make application authentication intentionally absent.** All Tasks and - Artifacts share one deployment namespace. The listener accepts explicit - `0.0.0.0`; network controls are external. Bind and advertised interface URL - are distinct. (session-settled: user-directed.) +- KTD4. **Make application authentication intentionally absent.** All Tasks, + sessions, and Artifacts share one deployment namespace. Remote service uses + loopback behind private TLS ingress or native TLS on one specific private + address; wildcard/public listeners and public exposure are prohibited. Bind + and advertised interface URL remain distinct. (session-settled: user-directed.) - KTD5. **Compile gateway configuration from existing workspace files.** Add `workspaceSnapshots` to the project schema and `gateway.enabled` to strict profile-client schemas. A gateway-only compiler normalizes the project @@ -1102,10 +1621,12 @@ registry, or another profile configuration file for the initial use case. declared snapshot name and immutable digests. Working-directory requests select only the effective workspace root or a declared repository plus a bounded relative directory. `workspaceAccess` is exactly `readOnly` or - `readWrite`. Read-only Tasks may share the immutable base; read-write Tasks - receive Task-ID-derived views. The gateway never accepts or returns a caller - path or materializer choice. Include canonical source identity, logical cwd, - and access in idempotency and provenance. + `readWrite`. One-shot read-only Tasks may share the immutable base and one-shot + read-write Tasks receive Task-ID-derived views; sessions pin the base and + retain session-private runtime/view state between turns. The gateway never + accepts or returns a caller path or materializer choice. Include canonical + source identity, logical cwd, access, and session projection in idempotency and + provenance. - KTD7. **Freeze Docker-only base acquisition.** Build `apps/acquirer` once as a multi-architecture GHCR image and select it by manifest digest. Each request without a reusable validated base starts a fresh container with one staging @@ -1128,22 +1649,25 @@ registry, or another profile configuration file for the initial use case. (session-settled: user-directed.) - KTD9. **Keep one behavior-focused `codex | pi` adapter registry.** Direct targets and gateway-enabled profile targets resolve to the same narrow - AllAgents-owned TypeScript adapter contract and conformance suite; profile - context modifies server-owned configuration, never public argv. Codex uses - pinned `@openai/codex-sdk` first; app-server is allowed only for a proven - required SDK gap. Pi uses a pinned supported package/RPC surface. Neither - adapter downloads runtimes per request or adopts AI SDK Harnesses. A global - binary override requires an exact compatibility probe. -- KTD10. **Keep durable Task truth inside ordinary Bun SQLite ownership.** The - gateway holds the process-lifetime `bun:sqlite` connection, private state - root, and exclusive lock. SQLite uses foreign keys, transactional - `createOrReplay`/lease/settlement/expiry operations, WAL where supported, and - `synchronous=FULL`; acknowledge only committed state. Claims, Tasks, events, - bounded Artifact bytes, execution lease, acquisition-container/staging/ - transient-base identity, provider process-group identity, internal outcome - intent, and expiry live in tables. Startup integrity or durability failure - stops admission and prevents false success. Do not build a custom VFS or - native file layer. + AllAgents-owned TypeScript adapter and one-shot/Session conformance suite; + profile context modifies server-owned configuration, never public argv. Codex + uses pinned `@openai/codex-sdk` `startThread`/`resumeThread` first; app-server + is allowed only for a proven required SDK gap. Pi uses a pinned supported + package/RPC surface and advertises Session capability only after exact + create/resume/checkpoint/dispose probes. Neither adapter downloads runtimes per + request, replays transcript text as fake resumption, or adopts AI SDK + Harnesses. A global binary override requires an exact compatibility probe. +- KTD10. **Keep durable Task and Session truth inside ordinary Bun SQLite + ownership.** The gateway holds the process-lifetime `bun:sqlite` connection, + private state root, and exclusive lock. SQLite uses foreign keys, + transactional `createOrReplay`/session-head/lease/settlement/checkpoint/expiry + operations, WAL where supported, and `synchronous=FULL`; acknowledge only + committed state. Claims, Tasks, Sessions, turns, events, opaque provider + checkpoint handles, pinned workspace/config digests, session charges, bounded + Artifact bytes, execution lease, acquisition-container/staging/transient-base + identity, provider process-group identity, internal outcome intent, and expiry + live in tables. Startup integrity or durability failure stops admission and + prevents false success. Do not build a custom VFS or native file layer. - KTD11. **Treat the trusted Linux CI job as the provider isolation boundary.** The gateway uses Docker only when no reusable validated base exists. Codex and Pi run bare metal with the same CI-job authority as the gateway and existing @@ -1203,9 +1727,11 @@ registry, or another profile configuration file for the initial use case. object containing `product: "allagents-gateway"`, `gatewayVersion`, `buildCommit`, `runtime: "bun"`, the pinned acquisition image repository and multi-architecture manifest digest, supported acquisition platforms/digests, -and supported A2A, coding-extension, workspace, execution-contract, acquisition- -contract, and snapshot versions. The packed npm tarball, clean-install smoke, -registry workflow, and release workflow consume this same object. +and supported Harness Execution Contract Core, Sessions, A2A binding/profile, +coding-workspace extension, workspace, execution-contract, +acquisition-contract, and snapshot versions. The packed npm tarball, +clean-install smoke, registry workflow, and release workflow consume this same +object. An optional `allagents gateway ...` dispatcher locates but never installs the separate gateway. It accepts independent CLI and gateway versions only when the @@ -1222,8 +1748,16 @@ no platform npm packages or native-binary compatibility checks. ```mermaid flowchart TB - C[Trusted-network A2A caller] --> G[Bun gateway host process] - G --> S[Bun SQLite Task store] + ER[Evaluation runner] --> AB[A2A binding] + CP[Chat platform] --> AB + AG[Another agent] --> AB + AB --> HC[Harness Execution Contract Core] + AB --> HS[Harness Execution Contract Sessions] + AB --> CW[AllAgents coding-workspace extension] + HC --> G[Bun gateway host process] + HS --> G + CW --> G + G --> S[Bun SQLite Task and session store] G --> W[workspace-config compiler] W --> PW[Project workspace.yaml] W --> UW[User workspace.yaml] @@ -1236,21 +1770,21 @@ flowchart TB A --> ST[Staging plus typed manifest] ST --> V[Host validation and atomic base promotion] V -->|exact identity| RB - V -->|mutable revision| TB[Task-owned transient base] + V -->|mutable revision| TB[Task or session-owned transient base] RB --> RO[Read-only base plus private runtime] TB --> RO RB --> M[Block clone or rootless OverlayFS or copy] TB --> M - M --> RW[Task-owned writable view] + M --> RW[Task or session-owned writable view] RO --> WD[Logical cwd resolver] RW --> WD WD --> R[Closed host adapter registry] - R --> Codex[Pinned Codex SDK] - R --> Pi[Pinned Pi RPC/package] - Codex --> PG[Linux provider process group] + R --> Codex[Pinned Codex SDK thread] + R --> Pi[Pinned Pi RPC/package session] + Codex --> PG[Linux provider process group per turn] Pi --> PG PG --> E[Direct-process settlement then bounded evidence] - E --> C[Remove Task runtime, view, and transient base] + E --> C[Checkpoint session or remove one-shot/session state] C --> S ``` @@ -1262,8 +1796,10 @@ No `gateway.yaml` or `worker.yaml` is introduced. | Concern | CLI | Environment | Default | |---|---|---|---| -| Listener | `--listen` | `ALLAGENTS_GATEWAY_LISTEN` | `127.0.0.1:4732` | -| Advertised interface URL | `--advertise-url` | `ALLAGENTS_GATEWAY_ADVERTISE_URL` | `http://127.0.0.1:4732` only with the default loopback listener; otherwise required | +| Listener | `--listen` | `ALLAGENTS_GATEWAY_LISTEN` | `127.0.0.1:4732`; IP literal only; no wildcard | +| Advertised interface URL | `--advertise-url` | `ALLAGENTS_GATEWAY_ADVERTISE_URL` | `http://127.0.0.1:4732` only with the default listener; otherwise required | +| Native TLS certificate | `--tls-cert-file` | `ALLAGENTS_GATEWAY_TLS_CERT_FILE` | unset; required with key for a specific non-loopback listener | +| Native TLS private key | `--tls-key-file` | `ALLAGENTS_GATEWAY_TLS_KEY_FILE` | unset; required with certificate for a specific non-loopback listener | | Project workspace | `--workspace` | `ALLAGENTS_GATEWAY_WORKSPACE` | cwd | | State directory | `--state-dir` | `ALLAGENTS_GATEWAY_STATE_DIR` | `~/.allagents/gateway/` | | Invocation workspace root | `--invocation-root` | `ALLAGENTS_GATEWAY_INVOCATION_ROOT` | `~/.allagents/gateway-workspaces/` | @@ -1273,7 +1809,13 @@ No `gateway.yaml` or `worker.yaml` is introduced. | Automatic copy ceiling | `--max-auto-copy-bytes` | `ALLAGENTS_GATEWAY_MAX_AUTO_COPY_BYTES` | `1GiB` | | Terminal Task TTL | `--task-ttl` | `ALLAGENTS_GATEWAY_TASK_TTL` | `24h` | | Retained Task limit | `--max-retained-tasks` | `ALLAGENTS_GATEWAY_MAX_RETAINED_TASKS` | `1000` | +| Idle session TTL | `--session-ttl` | `ALLAGENTS_GATEWAY_SESSION_TTL` | `24h`, refreshed after each committed turn | +| Retained session limit | `--max-retained-sessions` | `ALLAGENTS_GATEWAY_MAX_RETAINED_SESSIONS` | `100` | +| Per-session live bytes | `--max-session-bytes` | `ALLAGENTS_GATEWAY_MAX_SESSION_BYTES` | `10GiB` | +| Aggregate retained session bytes | `--max-retained-session-bytes` | `ALLAGENTS_GATEWAY_MAX_RETAINED_SESSION_BYTES` | `100GiB` | | Per-Task retained bytes | `--max-task-bytes` | `ALLAGENTS_GATEWAY_MAX_TASK_BYTES` | `64MiB` | +| Aggregate retained bytes | `--max-retained-bytes` | `ALLAGENTS_GATEWAY_MAX_RETAINED_BYTES` | `1GiB` | +| Serialized response bytes | `--max-response-bytes` | `ALLAGENTS_GATEWAY_MAX_RESPONSE_BYTES` | `96MiB` | | Acquisition image | `--acquisition-image` | `ALLAGENTS_GATEWAY_ACQUISITION_IMAGE` | release-embedded `ghcr.io/.../allagents-acquirer@sha256:` | | Docker endpoint | `--docker-host` | `ALLAGENTS_GATEWAY_DOCKER_HOST` | existing local Docker context/socket | | Docker acquisition network | `--acquisition-network` | `ALLAGENTS_GATEWAY_ACQUISITION_NETWORK` | release-documented acquisition-only network | @@ -1281,25 +1823,53 @@ No `gateway.yaml` or `worker.yaml` is introduced. | GitHub App ID | `--github-app-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_ID` | unset | | App private key file | `--github-app-private-key-file` | `ALLAGENTS_GATEWAY_GITHUB_APP_PRIVATE_KEY_FILE` | unset | | App installation ID | `--github-app-installation-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_INSTALLATION_ID` | discovered/unset | +| GitHub App mint request timeout | `--github-app-mint-timeout` | `ALLAGENTS_GATEWAY_GITHUB_APP_MINT_TIMEOUT` | `30s` | | GitHub CLI account | `--github-cli-account` | `ALLAGENTS_GATEWAY_GITHUB_CLI_ACCOUNT` | unset | | OCI auth file | `--oci-auth-file` | `ALLAGENTS_GATEWAY_OCI_AUTH_FILE` | unset | | OCI credential helper | `--oci-credential-helper` | `ALLAGENTS_GATEWAY_OCI_CREDENTIAL_HELPER` | unset | | OCI CA bundle map | `--oci-ca-bundle-map` | `ALLAGENTS_GATEWAY_OCI_CA_BUNDLE_MAP` | system roots only | -| Codex home | `--codex-home` | `ALLAGENTS_GATEWAY_CODEX_HOME`, then `CODEX_HOME` | existing supported host Codex home | +| Codex host auth location | `--codex-auth-home` | `ALLAGENTS_GATEWAY_CODEX_AUTH_HOME`, then `CODEX_HOME` | existing host location; usable only through a proven separate auth input | | Codex binary override | `--codex-bin` | `ALLAGENTS_GATEWAY_CODEX_BIN` | pinned SDK-managed surface; unset | -| Pi home | `--pi-home` | `ALLAGENTS_GATEWAY_PI_HOME` | existing supported host Pi home | +| Pi host auth location | `--pi-auth-home` | `ALLAGENTS_GATEWAY_PI_AUTH_HOME` | unset; usable only through a proven separate auth input | | Pi binary override | `--pi-bin` | `ALLAGENTS_GATEWAY_PI_BIN` | pinned package/RPC surface; unset | | Graceful abort period | `--abort-grace` | `ALLAGENTS_GATEWAY_ABORT_GRACE` | `10s` | | SIGTERM period | `--term-grace` | `ALLAGENTS_GATEWAY_TERM_GRACE` | `10s` | | Final cleanup period | `--cleanup-timeout` | `ALLAGENTS_GATEWAY_CLEANUP_TIMEOUT` | `30s` | +`--max-task-bytes` is valid from `8MiB` through `512MiB` and must exceed the +generated `terminalTailReserveBytes`. `--max-retained-bytes` must be at least +`--max-task-bytes`, and `--max-response-bytes` must be at least +`--max-task-bytes`; startup rejects any other relationship. The per-Task +footprint measures the exact serialized one-Task response including its +envelope, base64, and JSON escaping. A page adds Tasks only while exact +production serialization remains within `--max-response-bytes`. +`--max-session-bytes` measures provider state, private runtime, pinned +non-reusable base, committed workspace generation, and any reserved candidate/ +superseded generation. Admission reserves the projected candidate before +materialization; an over-limit turn creates no provider process. +`--max-retained-session-bytes` is at least `--max-session-bytes`. Session expiry +uses the same verified cleanup/poison rule as `closeAfterTurn`; it never silently +abandons retained state. +The GitHub App mint request timeout is 1-120 seconds and is part of the pre-mint +conservative-expiry calculation. + Precedence is CLI over gateway-specific environment over provider-standard -environment over default. For Codex this is `--codex-home`, -`ALLAGENTS_GATEWAY_CODEX_HOME`, then the ordinary `CODEX_HOME` identity -location. The advertised value is the absolute URL placed in -`AgentCard.supportedInterfaces`; wildcard hosts are invalid, non-loopback -listeners require an explicit value, and production uses HTTPS. The acquisition -image must be a full `repository@sha256:` reference; tags are rejected. +environment over default. `--codex-auth-home`, +`ALLAGENTS_GATEWAY_CODEX_AUTH_HOME`, then ordinary `CODEX_HOME` select only the +host auth source; the child process receives a Task/session-private mutable state +home, never that auth path as its writable provider home. The advertised value +is the absolute URL placed in `AgentCard.supportedInterfaces`. `--listen` accepts +one IP literal plus port and rejects `0.0.0.0`, `::`, and public addresses. +Specific non-loopback listeners require both readable TLS files and private +HTTPS advertised resolution; the gateway itself serves TLS. A loopback listener +may advertise matching loopback HTTP or private HTTPS through an operator TLS +terminator whose only backend is that loopback socket. Startup rejects public +URL literals, any public DNS answer, unresolved hosts, mismatched TLS options, +and remote `http:` URLs. TLS inputs must be readable regular PEM files; the +private key is current-user owned and not group/world accessible, the certificate +and key must match, and both parse before binding. The acquisition image must be +a full +`repository@sha256:` reference; tags are rejected. The gateway verifies that the local platform resolves to the release-recorded platform digest before starting acquisition. @@ -1339,10 +1909,11 @@ and every hop checked. The gateway passes only the selected source credential and exact CA material into the acquisition container and destroys both before provider execution. -Provider homes are never copied, mounted into Docker, parsed by AllAgents, or -imported into another store. The direct Codex/Pi host process receives the -selected home path and required host identity/auth environment in place. -Binary overrides are absolute host paths and must pass the pinned adapter's +Provider auth locations are never copied, mounted into Docker, parsed by +AllAgents, imported into another store, or used as the mutable provider-state +home. The direct Codex/Pi process receives a private state root and only the +pinned adapter's supported separate auth reference. Binary overrides are +absolute host paths and must pass the pinned adapter's exact version/protocol probe at readiness; they are not request-selectable. The explicit provider environment starts from an allowlist rather than the gateway's complete environment, but this is leakage reduction, not isolation. @@ -1350,23 +1921,25 @@ gateway's complete environment, but this is leakage reduction, not isolation. The immutable-base cache and invocation roots are current-user owned, private, and disjoint from state, project, profile, provider-auth, and each other. Acquisition writes a unique directory under `/.staging`; host -validation completes before an atomic same-filesystem rename to either the final -cache-key directory or `/transient/` for a non-reusable -base. Active Task references pin reusable entries. Least-recently-used eviction -enforces the byte budget and removes only unpinned reusable bases. Every non- -publication path removes its staging directory, and startup reconciles orphan -staging and recorded transient bases before readiness. - -For read-only access, every Task owns -`//runtime`; its cwd resolves in a reusable cached base -or its non-reusable transient base. For read-write access, the Task also owns -`//workspace`. `auto` probes same-filesystem block -clone first, then rootless OverlayFS on Linux, then ordinary copy only when the -base does not exceed `--max-auto-copy-bytes`. `cow` requires block clone or -rootless OverlayFS and fails readiness when neither is available. `copy` is the -explicit portable, higher-I/O backend and may exceed the automatic copy ceiling. -The explicit `copy` backend has no Linux-only filesystem requirement, but it -does not by itself make the v1 gateway available on Windows; process lifecycle +validation completes before an atomic same-filesystem rename to the final +cache-key directory, `/transient/tasks/`, or +`/transient/sessions/`. Active Task/session +references pin reusable entries. Least-recently-used eviction enforces the byte +budget and removes only unpinned reusable bases. Every non-publication path +removes its staging directory, and startup reconciles orphan staging and +recorded transient bases before readiness. + +One-shot state lives only at +`/tasks//{runtime,workspace?}`. Retained session state +lives only at +`/sessions//{runtime,workspaces,candidates}`. +Read-only cwd resolves in the pinned +reusable or non-reusable base. `auto` probes same-filesystem block clone first, +then rootless OverlayFS on Linux, then ordinary copy only when the base does not +exceed `--max-auto-copy-bytes`. `cow` requires block clone or rootless OverlayFS +and fails readiness when neither is available. `copy` is the explicit portable, +higher-I/O backend and may exceed the automatic copy ceiling. It does not by +itself make the v1 gateway available on Windows; process lifecycle and cancellation remain Linux-only in this plan. Startup logs the selected capabilities without paths. No mode uses writable hard links. Startup rejects overlapping roots and stale mounts it cannot safely reconcile. @@ -1438,8 +2011,10 @@ nonempty `options.id`, validates `options.config`, and `id()` returns that stored value. `callApi(prompt, context?, options?)` reads `context?.vars?.allagentsSource`, -`context?.vars?.allagentsWorkingDirectory`, and -`context?.vars?.allagentsWorkspaceAccess` when present, plus +`context?.vars?.allagentsWorkingDirectory`, +`context?.vars?.allagentsWorkspaceAccess`, +`context?.vars?.allagentsSession`, and +`context?.vars?.allagentsConversation` when present, plus `options?.abortSignal` for cancellation. Static YAML defines the source mode, logical names, and optional default logical @@ -1513,9 +2088,10 @@ tests: allagentsWorkspaceAccess: readWrite ``` -The gateway enforces one active invocation transactionally. Promptfoo keeps -`maxConcurrency: 1` to avoid predictably creating failed capacity Tasks; other -trusted callers need no external queue for correctness. Disabling cache, local +The gateway admits one active gateway-controlled invocation transactionally. +Promptfoo keeps `maxConcurrency: 1` to avoid predictably creating failed +capacity Tasks; other trusted callers need no external queue for admission +correctness. Disabling cache, local result writes, and sharing is the safe baseline for confidential prompts and opaque provider output. Consumers may enable persistence or sharing only after defining their own access, retention, destination, and redaction policy. @@ -1548,24 +2124,25 @@ destinations, credentials, and commands fail before provider execution. Each `callApi` creates one high-entropy invocation key and sends `SendMessage` with `returnImmediately: true`, then follows the accepted Task through -`SubscribeToTask`, `GetTask`, and bounded resubscription. A read-only Task may -share its immutable physical base and cwd with other Tasks while keeping private -runtime state; a read-write Task receives a unique disposable writable view. The -caller chooses neither physical path nor materializer. An abort or deadline -sends one `CancelTask` with a fresh cleanup signal. Ambiguous submission retry -reuses the same key, canonical request, Task, base/view, cwd, and access mode. +`SubscribeToTask`, `GetTask`, and bounded resubscription. `oneShot` receives +Task-private runtime and a disposable writable view when needed. `start` stores +the returned context/head; `resume` sends that exact context/head and receives a +new Task while retaining the session's provider/workspace state. The final +configured turn sets `closeAfterTurn`. The caller chooses neither physical path +nor materializer. An abort/deadline sends one `CancelTask` with a fresh cleanup +signal. Ambiguous submission retry reuses the same key, canonical request, Task, +base/view, cwd, access, and session projection. The provider returns terminal text or validated structured result as `ProviderResponse.output`. It maps gateway usage exactly as `inputTokens -> tokenUsage.prompt`, `outputTokens -> tokenUsage.completion`, `cachedInputTokens -> tokenUsage.cached`, and `totalTokens -> tokenUsage.total`; provider-specific counters remain in -`metadata`. Task ID, Artifact references, logical source identity, logical -working directory, workspace access, termination, cleanup, and stable failure -`code`/`retryable`/accepted `taskId` also remain in metadata, without origins, -configured destinations, or physical paths. -Admission and terminal failures use a safe `ProviderResponse.error`. This -provider is AI Evals code; -AllAgents has no Promptfoo runtime dependency. +`metadata`. Task ID, session context/head, Artifact references, logical source +identity, logical working directory, workspace access, termination, cleanup, +and stable failure `code`/`retryable`/accepted `taskId` also remain in metadata, +without origins, configured destinations, or physical paths. Admission and +terminal failures use a safe `ProviderResponse.error`. This provider is AI Evals +code; AllAgents has no Promptfoo runtime dependency. ### Error and Status Mapping @@ -1581,15 +2158,21 @@ reason. Custom admission errors include `google.rpc.ErrorInfo` with domain |---|---|---| | Unsupported A2A version | HTTP 400 A2A `VersionNotSupportedError`; no Task | No | | Missing required extension | HTTP 400 A2A `ExtensionSupportRequiredError`; no Task | No | -| Malformed request, source, working-directory selector, workspace access, digest, schema, prompt, or unknown target/source/repository | HTTP 400 `INVALID_ARGUMENT`; `invalid_execution_request`; no Task | No | +| Malformed request, context ID, source, working-directory selector, workspace access, digest, schema, prompt, or unknown target/source/repository | HTTP 400 `INVALID_ARGUMENT`; `invalid_execution_request`; no Task | No | | Invocation-key conflict | HTTP 409 `ALREADY_EXISTS`; `invocation_key_conflict`; no new Task | No | | Identical retained invocation replay | Existing Task with embedded Artifacts | N/A | +| Follow-up Message to an active or terminal Task | HTTP 400 A2A `UnsupportedOperationError`; existing Task unchanged | No | +| Resume names an unknown or expired session | HTTP 404 `NOT_FOUND`; `session_expired`; no Task | No; start a new session | +| Resume names a stale/non-head prior Task or changes pinned target/source/cwd/access/provider configuration | HTTP 409 `FAILED_PRECONDITION`; `session_head_mismatch` or `session_configuration_mismatch`; no Task | No; refresh head or start a new session | +| Resume targets an active session turn | HTTP 409 `FAILED_PRECONDITION`; `session_busy`; no Task | Yes, after the active turn settles | +| Session checkpoint is poisoned or cleanup is unresolved | HTTP 409 `FAILED_PRECONDITION`; `session_not_resumable`; no Task | No, until verified operator reconciliation | | Cancel after terminal state | HTTP 400 A2A `TaskNotCancelableError` | No | -| Retained Task capacity exhausted | HTTP 429 `RESOURCE_EXHAUSTED`; `retention_capacity_exhausted`; `Retry-After`; no Task | Yes, after expiry | +| Retained Task/session count or aggregate byte capacity exhausted | HTTP 429 `RESOURCE_EXHAUSTED`; `retention_capacity_exhausted`; `Retry-After`; no Task | Yes, after Task/session expiry | | Runtime capacity unavailable after acceptance | `execution_capacity_unavailable`; failed Task | Yes | +| A composed extension reports failure | Core `execution_extension_failed`; failed Task; exact extension failure remains in its Artifact | Per paired extension row | | Valid logical cwd resolves to a missing, non-directory, or escaping path after acquisition | `execution_working_directory_invalid`; failed Task; no provider start; no physical path returned | No | | Required copy-on-write materializer unavailable, or `auto` would copy above its ceiling | `workspace_materialization_unavailable`; failed Task; no provider start | No | -| Task-private runtime, non-reusable base, or writable-view creation/removal fails | `workspace_cleanup_failed`; failed Task; `cleanup.workspace: "failed"`; retain cleanup record; stop admission if an active mount or uncertain writable view remains | Yes only as a fresh invocation after operator repair | +| Task/session-private runtime, non-reusable base, provider checkpoint, or writable-view creation/removal fails | Workspace/session `workspace_cleanup_failed`; Core `execution_extension_failed`; failed non-expiring Task; poison the session and retain reservation, cleanup record, owned state, and lease until verified repair | Yes only as a fresh invocation after operator repair | | App absent/ineligible and configured `gh` succeeds | Continue with recorded provider class | N/A | | App applicability unknown | `source_auth_applicability_unknown`; failed Task; no fallback | Yes for rate-limit/service causes only | | Selected App config/auth/mint/validation/revocation failure | `source_auth_failed`; failed Task; no fallback | No | @@ -1602,26 +2185,30 @@ reason. Custom admission errors include `google.rpc.ErrorInfo` with domain | OCI helper timeout, process, protocol, or credential failure | `source_auth_oci_failed`; failed Task; no fallback | No | | OCI auth/challenge/digest/manifest/extraction validation failure | `source_snapshot_invalid`; failed Task; no Git fallback | No | | OCI registry service failure | `source_snapshot_unavailable`; failed Task; no Git fallback | Yes | -| Deadline expires | `execution_deadline_exceeded`; abort/terminate; failed Task | Yes | +| Deadline expires | Core `timedOut`; `execution_deadline_exceeded`; abort/terminate; A2A failed Task | Yes | | Known provider permission denial | `execution_permission_denied`; rejected Task | No | | Unknown provider protocol or result shape | `provider_protocol_invalid`; failed Task | No | -| Cancellation after acquisition removal or direct provider settlement | `execution_canceled`; canceled Task | No | -| Acquisition container or unpublished staging cannot be removed | `source_cleanup_failed`; failed Task; stop admission | No | -| Direct provider does not settle after abort/`SIGTERM`/`SIGKILL` | `execution_termination_failed`; failed Task; no filesystem/Git evidence; retain Task-owned runtime, view, or non-reusable base and lease; stop admission until post-teardown reconciliation | No | -| State store durability/integrity failure | `task_store_failed`; stop admission; request active-work abort; no success | No | -| Restart finds interrupted Task | `gateway_restarted`; failed Task; no provider resume | Yes as a new invocation | -| Retention expiry | HTTP 404 A2A `TaskNotFoundError` | Yes as a new invocation | - -Accepted-Task failures use the integrity Artifact's strict `failure` object with -`code`, safe `message`, table-defined `retryable`, and one closed cause from -`validation | capacity | sourceAuth | sourceGit | sourceSnapshot | sourceCleanup | -workingDirectory | workspaceMaterialization | workspaceCleanup | deadline | -permission | providerProtocol | cancellation | termination | stateStore | -restart`. Retryability says whether a caller may create a fresh invocation; it -never enables automatic Task retry or provider/source fallback. Promptfoo copies -only the safe code, retryability, and accepted Task ID into metadata. Provider -identifiers, credentials, paths, and raw upstream messages enter neither -carrier. +| Valid structured result exceeds retained footprint after optional evidence truncation | `execution_result_too_large`; failed Task; `result.reason: "retentionLimitExceeded"` | No | +| Cancellation after acquisition removal or direct provider settlement, with successful termination and cleanup | Core `canceled`; `execution_canceled`; A2A canceled Task | No | +| Acquisition container or unpublished staging cannot be removed | Workspace `source_cleanup_failed`; Core `execution_extension_failed`; failed non-expiring Task; retain reservation/reconciliation record/lease; stop admission until verified repair | No | +| Direct provider does not settle after abort/`SIGTERM`/`SIGKILL` | `execution_termination_failed`; failed non-expiring Task; no filesystem/Git evidence; retain reservation, Task/session-owned state, reconciliation record, and lease until post-teardown verification | No | +| State store durability/integrity failure | `state_store_failed`; stop admission; request active-work abort; no success | No | +| Restart finds interrupted Task | `gateway_restarted`; failed Task; persist any required App-token revocation tombstone before releasing the lease; never replay the turn; restore only a provably unchanged prior session checkpoint, otherwise poison the session | Yes as a new turn only when the session remains resumable | +| Task retention expiry | HTTP 404 A2A `TaskNotFoundError`; does not delete a live session checkpoint | Yes as a new invocation | +| Session idle expiry | Verified provider/workspace cleanup, then `session_expired` on resume; cleanup uncertainty poisons and retains the session | No; start a new session | + +Accepted-Task Core-origin failures use the Core outcome Artifact's strict +`failure` and HEC-only cause union. Accepted workspace/source failures instead +pair Core `execution_extension_failed`/`extension` with the workspace-integrity +Artifact's exact code, safe message, retryability, and one closed cause from +`sourceAuth | sourceGit | sourceSnapshot | sourceCleanup | workingDirectory | +workspaceMaterialization | workspaceCleanup`. The composed schema and +conformance matrix own this pairing; neither module imports the other's codes. +Retryability says whether a caller may create a fresh invocation; it never +enables automatic Task retry or provider/source fallback. Promptfoo copies the +workspace failure when present, otherwise the Core failure, plus retryability +and accepted Task ID. Provider identifiers, credentials, paths, and raw upstream +messages enter neither Artifact nor Promptfoo metadata. ### Phased Delivery @@ -1629,16 +2216,23 @@ carrier. record the red E2E showing that `allagents-gateway serve` is unavailable and that no acquisition image is fetched. 2. Execute U0 as a bounded feasibility gate: establish the private Bun - workspace layout; prove the shipped A2A server with the official JavaScript - client; pin and probe Codex SDK and Pi RPC/package surfaces; characterize - explicit provider environments and Linux process groups; build and run the - digest-pinned acquisition image for both supported architectures; and prove - independent CLI/gateway packaging plus exact release binding. -3. Freeze workspace additions, published extension, snapshot format, execution - and acquisition contracts, generated portable fixtures, error vocabulary, - SQLite schema/transactions, compatibility output, and release manifest. -4. Build the Bun SQLite Task store, AllAgents A2A request handler, HTTP+JSON/SSE - server, minimal backend interface/registry, and fake adapter. + workspace layout; prove Harness Execution Contract Core and Sessions + semantics, the shipped A2A binding and required coding-workspace extension, + and anonymous trusted-network conformance direction with the official + JavaScript client; prove a minimal Promptfoo custom provider can activate the + profile, consume a terminal Task, and resume a second turn; pin and probe + Codex SDK and Pi RPC/package one-shot/session surfaces; characterize explicit + provider environments and Linux process groups; build/run the digest-pinned + acquisition image for both supported architectures; and prove independent + CLI/gateway packaging plus exact release binding. +3. Freeze the normative Harness Execution Contract Core and Sessions, A2A + binding, AllAgents coding-workspace extension, their composed v1 Profile URN + and wire schema, independent module/binding/extension conformance vectors, + workspace additions, snapshot format, execution/acquisition contracts, error + vocabulary, SQLite schema/transactions, compatibility output, and release + manifest. +4. Build the Bun SQLite Task/session store, AllAgents A2A request handler, + HTTP+JSON/SSE server, minimal backend interface/registry, and fake adapter. 5. Add direct host-process supervision, explicit environment construction, read-only runtime separation, read-write materialization, process-group cancellation, typed preparation, bounded evidence, terminal arbitration, @@ -1682,8 +2276,8 @@ carrier. Git/OCI/archive dependencies used by `apps/acquirer` in the Bun lockfile. Minimize dependencies per workspace and scan both the npm tarball and image. - **State surface:** Add one bounded private Bun SQLite state root, one bounded - immutable-base cache, and per-Task runtime plus optional writable-view roots. - Do not alter provider profile or authentication state. + immutable-base cache, and Task/session-private runtime plus optional writable + view roots. Do not alter provider profile or authentication state. - **Security surface:** Network reachability authorizes callers. The acquisition container has staging, source-only credentials, and strict source policy but no host home or Docker socket. Provider execution has trusted CI-job @@ -1697,19 +2291,36 @@ carrier. ### Risks and Mitigations -- **A2A or provider-surface immaturity:** Pin exact JavaScript package versions - and run U0 wire/provider probes before production units. If the Codex SDK - lacks a required capability, document proof before selecting pinned app-server; - if neither works, the target is unavailable rather than silently scraped. -- **Contract drift:** Generate public and private schemas plus accepted/rejected - fixtures from the three narrow packages and run drift checks in the gateway, - acquirer, docs, and consumer fixtures. +- **A2A, binding, or provider-surface immaturity:** Pin exact JavaScript package + versions and run U0 wire/provider probes before production units. The gate + must prove that applications, agents, and evaluation runners can use the same + Harness Execution Contract semantics through the A2A binding, and that + trusted-network anonymous operation can make an accurate A2A 1.0 conformance + claim. If authentication semantics make that impossible, amend the ADR before + production work; do not silently add identity or weaken the claim. If the + Codex SDK lacks a required capability, document proof before selecting pinned + app-server; if neither works, the target is unavailable rather than silently + scraped. +- **Harness-contract fragmentation:** Publish one normative Core with schemas, + examples, and executable semantic vectors; keep the A2A binding and AllAgents + coding-workspace extension separate, then expose only their one composed v1 + URI. Add no UHP/Responses or bespoke fallback wire in v1. Future bindings must + pass the same Core vectors rather than redefine lifecycle semantics. +- **Premature standard claim:** Describe HEC as a contract and standard + candidate until multiple independent implementations, multiple bindings, + neutral governance, and cross-binding conformance exist. +- **Contract drift:** Generate binding, extension, and composed schemas plus + accepted/rejected fixtures from the three narrow packages and run drift + checks in the gateway, acquirer, docs, and consumer fixtures. Keep Core + semantic vectors hand-authored and independent from generated types. - **Install-size regression:** Keep CLI and gateway workspace dependency graphs separate, report packed/installed sizes, enforce budgets, and fail CLI-only smoke if it resolves the gateway or acquisition image. -- **Accidental network exposure:** Binding `0.0.0.0` is intentional and allowed; - require a distinct advertised URL, use HTTPS in production, and state in - startup output/docs that every reachable peer has full authority. +- **Accidental network exposure:** Reject wildcard/public binds. Require either a + loopback-only backend behind private HTTPS ingress or a native TLS listener on + one specific private address, plus private advertised resolution and external + ACLs. Startup/docs state that every reachable peer has full authority. Public + exposure requires application authentication and an amended ADR first. - **Profile identity drift:** Derive targets only from current validated user declarations and matching installed state; never resurrect declaration-missing launchers from retained profile state. @@ -1759,9 +2370,11 @@ carrier. - **Provider/API churn:** Pin SDK/package/protocol/model compatibility, require exact probes for binary overrides, retain native fixtures, and share one adapter conformance suite. Never download a provider runtime per request. -- **Orphaned processes:** Use a new Linux process group per direct provider, - persist the leader identity, escalate abort to `SIGTERM` and `SIGKILL`, and - rely on CI runner teardown as the final orphan boundary. +- **Orphaned processes:** Use a new Linux process group per provider, persist its + leader identity, escalate abort to `SIGTERM`/`SIGKILL`, and retain lease/unready + state for any observed escaped/outliving descendant until verified gone or + runner teardown. An unobserved descendant may overlap a later admitted turn; + operators requiring OS-wide exclusivity must use an ephemeral runner boundary. - **Store corruption or disclosure:** Use a current-user private state root, exclusive gateway lock, ordinary Bun SQLite transactions, foreign keys, `synchronous=FULL`, integrity checks, and bounded data. Integrity/durability @@ -1789,7 +2402,8 @@ carrier. the deployment's source egress boundary. Immutable repository cache reuse requires full commit IDs. - Codex and Pi are installed or provided by pinned workspace dependencies before - gateway start and can reuse their existing host authentication locations. + gateway start. Existing host authentication is reusable only when the pinned + adapter proves a public auth reference separate from private mutable state. - The implementation units after U0 assume the Bun/A2A/provider/process/acquirer feasibility gates passed. A failed provider probe disables that target; a failed architecture or release-binding gate stops the affected release rather @@ -1799,13 +2413,15 @@ carrier. ## Implementation Units -### U0. Bun monorepo, provider, process, and acquirer feasibility +### U0. Harness-contract, Bun, provider, process, and acquirer feasibility -- **Goal:** Prove the settled Bun architecture can preserve A2A behavior, - independent distribution, supported provider control, Linux cancellation, - read-only shared-base execution, read-write materialization, and exact +- **Goal:** Prove the settled Bun architecture can preserve the + transport-neutral Harness Execution Contract Core and Sessions semantics + through their first A2A binding, independent distribution, supported provider + create/resume/checkpoint control, Linux cancellation, read-only shared-base + execution, retained session workspace, read-write materialization, and exact acquisition-image release binding before production implementation. -- **Requirements:** R1-R2, R8, R13-R16, R18; AE8-AE11, AE17-AE18, AE20; +- **Requirements:** R1-R3, R8, R13-R16, R18; AE8-AE11, AE17-AE18, AE20-AE21; KTD1-KTD3, KTD6-KTD7, KTD9, KTD11-KTD14. - **Files:** private root `package.json`/`bun.lock`, `apps/cli`, `apps/gateway`, `apps/acquirer`, the three named `packages/` workspaces, @@ -1817,19 +2433,37 @@ carrier. public package or behavior. Establish `apps/gateway` as the separately packed Bun executable package and `apps/acquirer` as image-only code. Pin the official A2A JavaScript SDK and drive a minimal production-direction server - through every required operation. Probe `@openai/codex-sdk` for invocation, - events, native abort, usage, structured-output support, and existing - `CODEX_HOME` behavior; consider app-server only when a named required - capability is proven absent. Probe the supported Pi package/RPC surface for - invocation, events, abort, usage, and existing host auth. Prove exact - compatibility rejection for global binary overrides. + through every required operation. Add a small transport-neutral in-memory + adapter and representative Core, Sessions, A2A-binding, workspace-extension, + and composed-profile schemas. Run the same hand-authored Core vectors through + the in-memory adapter and A2A binding to prove per-turn lifecycle, ordered + progress/tool trajectory, result/usage/failure settlement, cancellation, and + artifacts are binding-independent. Prove Sessions start/resume/close, exact + predecessor chaining, linearization, pinned configuration, restart handling, + and workspace continuity independently, then prove the A2A context/task- + reference mapping. Separately prove required profile activation, + one-Message/one-Task-per-turn behavior, A2A state/event/Artifact mapping, and + standard errors. Drive two turns through the same server via a throwaway + Promptfoo custom provider to prove an application can resume context and + observe provider-reported cache usage without a second server protocol. + Resolve whether anonymous trusted-private-network operation can claim A2A 1.0 + conformance before dependent work. + Probe `@openai/codex-sdk` for start/continue/resume, opaque thread identity, + events, native abort, usage/cache reporting, structured-output support, and + private state-home/host-auth separation; consider app-server only when a named + required capability is proven absent. Probe the supported Pi package/RPC + surface for equivalent create/resume/checkpoint/dispose, events, abort, usage, + and auth/state separation. Prove exact compatibility rejection for global + binary overrides. Run a real Linux child in a new process group and demonstrate graceful abort, - `SIGTERM`, and `SIGKILL` escalation plus the limit that unrelated/escaped - descendants are not proven gone. Prove one immutable base can serve repeated - read-only Tasks with private runtime state; probe block cloning and rootless - OverlayFS; verify independent writable changes and removal; and prove explicit - copy behavior plus the automatic copy ceiling. Build the acquirer image for + `SIGTERM`, and `SIGKILL`; an escaped/outliving fixture must retain the lease + and readiness=false until verified disappearance. Also prove the explicit + limit that unobserved descendants are not contained. Prove one immutable base + can serve repeated read-only Tasks with private runtime state; probe block + cloning and rootless OverlayFS; verify independent writable changes/removal; + and prove explicit copy behavior plus the automatic copy ceiling. Build the + acquirer image for every supported architecture, run it with only a staging mount and synthetic source secret, verify typed manifest output and container removal, and prove the image has no provider runtime or Docker socket. Pack CLI and gateway @@ -1841,39 +2475,68 @@ carrier. containment machinery. A missing provider capability disables that provider; failed A2A, process, acquisition, or release-binding feasibility returns the affected design for revision before dependent units. -- **Verification:** Official JavaScript client fixtures pass against the Bun - server; provider probes record exact pinned versions and auth-path behavior; - shared read-only base, private runtime, reflink, rootless-overlay, explicit - copy, cleanup, environment, and process-group probes pass on Linux; multi- - architecture image manifests/digests are recorded and the image boundary +- **Verification:** Hand-authored Core and Sessions semantic vectors pass + unchanged through the in-memory adapter and A2A binding; official JavaScript + client and representative binding/workspace/composed-profile conformance + fixtures pass against the Bun server. The Promptfoo probe completes two + terminal Tasks under one conversation ID: turn two sends turn one's returned + context ID and exact head reference and proves conversation/workspace + continuity before close. The A2A authentication conformance result is + recorded; provider probes record exact pinned versions and auth/state-path + behavior; shared read-only base, private runtime, reflink, rootless-overlay, + explicit copy, cleanup, environment, and process-group probes pass on Linux; + multi-architecture image manifests/digests are recorded and the image boundary rejects extra mounts/credentials/network; independent packed CLI/gateway installs and compatibility fixtures pass; CLI-only installation fetches neither gateway nor acquisition image. ### U1. Workspace packages, contracts, SQLite, and release foundation -- **Goal:** Freeze the monorepo ownership, workspace configuration, public and - acquisition contracts, ordinary SQLite transactions, and exact release - artifact binding before runtime implementation. +- **Goal:** Freeze the monorepo ownership, workspace configuration, Harness + Execution Contract Core, Sessions, and first binding, acquisition contracts, + ordinary SQLite transactions, and exact release artifact binding before + runtime implementation. - **Requirements:** R1-R3, R5-R9, R11-R12, R18; AE3-AE9, AE13-AE14, AE16, - AE20; KTD1-KTD2, KTD5-KTD8, KTD10, KTD13-KTD14. + AE20-AE21; KTD1-KTD2, KTD5-KTD8, KTD10, KTD13-KTD14. - **Files:** `packages/workspace-config`, `packages/execution-contracts`, - `packages/acquisition-contracts`, generated `contracts/` schemas and golden - examples, gateway SQLite schema/migrations, release scripts/workflows, - deterministic snapshot producer/conformance fixture, published extension and - snapshot-format assets, and configuration docs. + `packages/acquisition-contracts`, normative Core/binding/extension text, + generated `contracts/` schemas/conformance vectors/golden examples, gateway + SQLite schema/migrations, release scripts/workflows, deterministic snapshot + producer/conformance fixture, snapshot-format assets, and configuration docs. - **Approach:** Move authoritative project/user parsing and gateway catalog compilation into `workspace-config`; add strict named `workspaceSnapshots`, exact redirect hosts, and nested `gateway.enabled` without changing ordinary - CLI behavior. Define execution contracts for Agent Card params, version/header - activation, Message metadata/extensions, unified Parts, source union, logical - working-directory union and relative-path grammar, workspace access/default, - result-schema grammar, deadline, idempotency/replay, materialization errors, - HTTP errors, integrity/produced Artifacts, adapter events/results, and - evidence. Define acquisition contracts for the closed request, path-free typed - manifest, immutable-base cache key, private compiled-layout checks, OCI - media/change-set profile, fixed limits, and canonical digests. Generate - portable accepted/rejected fixtures beneath `contracts/`. + CLI behavior. Define four separately versioned contract modules in + `execution-contracts`: the transport-neutral Harness Execution Contract Core, + its Sessions extension, its A2A binding, and the AllAgents coding-workspace + extension; compose binding, Sessions, and workspace extension into one + required v1 A2A Profile URN and request. Core owns one turn's invocation, + target, deadline, idempotency/replay, ordered progress and tool-call/result + trajectory, cancellation, result, usage, portable failure codes, and artifacts + without importing Sessions, A2A, UHP, workspace, provider, or evaluator types. + Sessions owns oneShot/start/resume, durable identity, ordered linear turns, + exact predecessor, checkpoint/expiry/close states, and portable session errors. + The A2A binding owns Agent Card parameters and skill semantics, version/header + activation, Message/Task/state/event/Artifact mapping, context-ID and prior- + Task mapping, and standard error mapping. The coding-workspace extension owns + source, logical working directory and relative-path grammar, workspace access/ + default, workspace failure codes, provenance, produced files, integrity, and + cleanup evidence. Each module exposes an independently validatable schema and + conformance group; only the composed v1 profile owns cross-module pinning, the + merged failure union, and simultaneous Core outcome/trajectory plus AllAgents + workspace-integrity Artifact requirements. + + Generate strict binding, extension, and composed wire schemas from canonical + types. Keep Core semantic vectors and normative prose hand-authored as an + independent oracle; run the same vectors through an abstract in-memory adapter + and the A2A binding. Check the Core, binding, extension, schemas, vectors, and + accepted/rejected fixtures beneath `contracts/`, and include a deliberate + contract-perturbation fixture that proves drift between generated types and + the normative contract turns CI red. + + Define acquisition contracts for the closed request, path-free typed manifest, + immutable-base cache key, private compiled-layout checks, OCI media/change-set + profile, fixed limits, and canonical digests. Add private `bun:sqlite` ownership with foreign keys, WAL where supported, `synchronous=FULL`, migrations, one execution lease, `createOrReplay`, @@ -1883,95 +2546,117 @@ carrier. triggers. The gateway release record binds the exact npm tarball digest to the acquirer multi-architecture manifest and supported platform digests; the image is verified before npm publication. -- **Execution note:** Do not add `core`, `common`, a custom VFS, native file - primitives, native/platform npm packages, or runtime compatibility shims. - Start with external wire/manifest fixtures and stable rejection codes. Fault +- **Execution note:** Do not add catch-all `core` or `common` packages, a custom + VFS, native file primitives, native/platform npm packages, or runtime + compatibility shims. Start with external wire/manifest fixtures and stable + rejection codes. Fault SQLite transactions and process exit around commit/acknowledgment boundaries, not filesystem attacks the ordinary SQLite contract does not claim to defeat. -- **Verification:** Workspace parsing/catalog fixtures, generated-schema drift, - wire/manifest accepted/rejected examples, canonicalization, Artifact - cardinality, SQLite commit/replay/lease/settlement/expiry/crash fixtures, - independent package versions, CLI-only and gateway clean-registry installs, - compatibility skew/image-mismatch matrix, exact tarball/image release record, - and idempotent absent/identical/divergent publication fixtures pass. - -### U2. Deployment-wide Task store and A2A server +- **Verification:** Workspace parsing/catalog fixtures; Core and Sessions + semantic vectors through both the in-memory adapter and A2A binding; binding- + specific context/session ID generation, exact prior-Task projection, + state/cancel/event/Artifact mapping, and byte budgets; binding/extension/ + composed-schema drift; hand-authored lifecycle/composition conformance; full + Core outcome matrix, extension retryability pairings, and rejected cross- + products; Sessions start/resume/close, head linearization, pinning, expiry, + capacity, and poison/reconciliation; canonical ordered execution trajectories + plus negative descriptor-linkage cases; ambiguous/identical/divergent replay; + deadline/cancellation/race/expiry behavior; produced-Artifact bytes and + descriptor linkage; wire/manifest accepted/rejected examples; deliberate + contract perturbation; canonicalization; Artifact cardinality; SQLite Task/ + session commit/replay/head/lease/settlement/expiry/crash and revocation- + tombstone fixtures; independent package versions; CLI-only and gateway clean- + registry installs; compatibility skew/image-mismatch matrix; exact tarball/ + image release record; and idempotent absent/identical/divergent publication + fixtures pass. + +### U2. Deployment-wide Task/session store and A2A server - **Goal:** Serve the A2A lifecycle without application authentication and keep - durable deployment-wide Task/idempotency truth behind a fake backend. + durable deployment-wide Task, idempotency, and linear session truth behind a + fake backend. - **Requirements:** R1-R5, R8, R13, R16-R18; AE1-AE2, AE9, AE11-AE16, - AE19-AE20; KTD1-KTD4, KTD9-KTD10. -- **Files:** `apps/gateway` Task-store module, Agent Card, A2A request handler, - HTTP+JSON/SSE server, pagination/retention, backend registry/fake adapter, - health/readiness, `allagents-gateway` command, and focused integration tests. -- **Approach:** Implement flags/environment precedence, bind/advertised-URL - separation, private state/lock, SQLite transactions, startup integrity and - interrupted-Task reconciliation, A2A version/extension negotiation, exact - `SendMessage` modes and `ListTasks` semantics, standard/custom - `google.rpc.Status` errors, durable `createOrReplay`, one execution lease, - internal outcome intent plus atomic terminal settlement, bounded - events/Artifact bytes, no early eviction, transactional expiry, deployment- - wide listing/cancellation, deadline handling, and graceful shutdown against a - fake adapter. + AE19-AE21; KTD1-KTD4, KTD9-KTD10. +- **Files:** `apps/gateway` Task/session-store module, Agent Card, A2A request + handler, HTTP+JSON/SSE server, pagination/retention, backend registry/fake + adapter, health/readiness, `allagents-gateway` command, and focused integration + tests. +- **Approach:** Implement flags/environment precedence, private bind/advertised- + URL validation, private state/lock, SQLite transactions, startup integrity and + interrupted-turn reconciliation, A2A version/profile negotiation, exact + `SendMessage` modes and `ListTasks` semantics, one immutable Task per turn, + Sessions start/resume/close and exact-head linearization, lifecycle/state/event + constraints, standard/custom `google.rpc.Status` errors, durable + `createOrReplay`, one execution lease, internal outcome intent plus atomic + terminal/checkpoint settlement, bounded events/Artifact/session-workspace + bytes, no early eviction, transactional Task/session expiry, deployment-wide + listing/cancellation, deadline handling, and graceful shutdown against a fake + adapter. - **Execution note:** Use an independent official JavaScript A2A client to prove one external caller can read and cancel another caller's Task; that is expected trusted-network behavior. Kill gateway subprocesses around SQLite transaction, commit, acknowledgment, cancellation-intent, Artifact, and settlement boundaries. Do not add caller ownership or an application credential. - **Verification:** Discovery, both send modes, stream/get/full list/subscribe/ - cancel/replay/expiry, HTTP errors, loopback and explicit - `0.0.0.0`/advertised URL, probes, retained/active capacity, competing lock, - SQLite crash/fault, deadline, shutdown, restart, and fake-backend tests pass. + cancel/replay/Task expiry, start/resume/close/session expiry, stale-head and + concurrent-turn rejection, HTTP errors, loopback/direct-private-TLS/ + loopback-proxy advertised URL combinations, wildcard/public listener and URL + rejection, TLS-file validation, probes, retained/active capacity, competing + lock, SQLite crash/fault, deadline, shutdown, restart, and fake-backend tests + pass. ### U3. Host process supervisor and backend contract -- **Goal:** Run fake-backed direct host invocations through shared read-only and - independent read-write workspace selection, logical cwd resolution, typed - preparation, explicit environment construction, process-group cancellation, - evidence, terminal arbitration, and cleanup with truthful limits before real - adapters. -- **Requirements:** R3, R5, R8, R13-R16, R18; AE8-AE12, AE14-AE18; +- **Goal:** Run fake-backed direct host turns through shared read-only, + one-shot/private read-write, and session candidate/committed workspace + selection, logical cwd resolution, typed preparation, explicit environment + construction, provider checkpointing, process-group cancellation, evidence, + arbitration, and cleanup with truthful limits before real adapters. +- **Requirements:** R3, R5, R8, R13-R16, R18; AE8-AE12, AE14-AE21; KTD3, KTD6, KTD9-KTD12. - **Files:** `apps/gateway` backend types/registry, immutable-base manager, - workspace materializer, provider environment builder, Linux process-group - supervisor, invocation state machine, typed preparation, evidence collector, - result validator, cleanup/restart reconciliation, fake process fixtures, and - lifecycle tests. + workspace/session materializer, provider environment builder, Linux process- + group supervisor, invocation/session state machines, typed preparation, + evidence collector, result validator, cleanup/restart reconciliation, fake + process fixtures, and lifecycle tests. - **Approach:** Define the minimal adapter contract for availability, - capabilities, access-aware invoke/events, graceful abort, direct-process - settlement, result/usage/evidence, and disposal. Resolve fake targets without - executing generated launchers or setup commands. For read-only, resolve cwd in - immutable base and allocate private runtime state. For read-write, materialize - a Task-ID-derived view via block clone, rootless OverlayFS, - or explicit copy. Resolve workspace-root and repository-relative selectors, - reject missing/non-directory/escaping paths, and pass only the effective cwd, - runtime paths, and access mode to the adapter. Start each direct provider in a - new process group, persist its leader PID and process-start marker before - marking execution started, and build its environment from a reviewed allowlist - that preserves required host identity/auth paths. Commit one internal intent - across provider terminal, cancel, deadline, and shutdown. Escalate adapter - abort to process-group `SIGTERM` and `SIGKILL`; capture bounded live events; - collect filesystem/Git evidence only after the direct process settles; remove - Task runtime or writable view; and atomically settle status, evidence, - Artifacts, observed termination, cleanup, and lease release. The non-settling - path emits only termination failure and live evidence, retains Task-owned - state plus lease, and blocks admission until verified reconciliation. -- **Execution note:** Fixtures must distinguish what AllAgents observes from what - it cannot guarantee. Exercise child and grandchild processes, including one - that escapes or outlives the direct process, and assert the gateway never - labels process-group cleanup as complete descendant quiescence. The CI runner - teardown is the final orphan boundary. No cgroups, pidfds, namespaces, - nftables, `openat2`, spawn broker, provider container, or isolation claim. -- **Verification:** Deterministic lifecycle; shared-base reuse without shared - runtime state; adapter-native read-only policy where available; independent - reflink, rootless-overlay, and copy views; automatic copy ceiling; cwd - resolution and escape rejection; single lease; explicit environment - inclusion/exclusion; - required host-auth preservation; binary-override compatibility rejection; - graceful/TERM/KILL timing; cancellation/deadline/shutdown races; poisoned- - lease behavior; post-teardown reconciliation; evidence ordering; result - states; cleanup outcomes; and truthful orphan-limit fixtures pass on trusted - Linux CI. + one-shot/session capabilities, start/resume/checkpoint/dispose, access-aware + invoke/events, graceful abort, direct-process settlement, result/usage/cache + evidence, and disposal. Resolve fake targets without executing generated + launchers or setup commands. For read-only, resolve cwd in the immutable base + and allocate Task- or session-private runtime state. For read-write, + materialize a Task- or session-ID-derived view via block clone, rootless + OverlayFS, or explicit copy. Resolve workspace-root and repository-relative + selectors, reject missing/non-directory/escaping paths, and pass only the + effective cwd, runtime paths, and access mode to the adapter. Start each turn's + direct provider in a new process group, persist its leader PID/process-start + marker before marking execution started, and build its environment from a + reviewed allowlist preserving required host identity/auth paths. Commit one + internal intent across provider terminal, cancel, deadline, and shutdown. + Escalate adapter abort to process-group `SIGTERM`/`SIGKILL`; capture bounded + live events; collect filesystem/Git evidence only after direct-process + settlement; then checkpoint a retained session or remove one-shot/closing + state using the R16 prepared-settlement/cleanup protocol. A non-settling, + observed-outliving, or uncertain checkpoint/candidate path emits truthful + failure/evidence, retains the lease, and blocks admission until reconciliation. +- **Execution note:** Fixtures distinguish observation from guarantee. Exercise + child/grandchild processes, including one that escapes/outlives the direct + process; it must retain lease/readiness state until the fixture exits. The + gateway never labels process-group cleanup as complete descendant quiescence; + an unobserved escape remains outside the v1 guarantee. CI runner teardown is + the final orphan boundary. No cgroups, pidfds, namespaces, nftables, `openat2`, + spawn broker, provider container, or isolation claim. +- **Verification:** Deterministic per-turn/session lifecycle; shared-base reuse + without shared runtime; one-shot views and read-write session prior/candidate/ + committed generations across reflink/rootless-overlay/copy; candidate byte + reservations; crash before/after workspace-pointer commit; adapter-native + read-only policy where available; cwd/escape rejection; exact session head and + provider/workspace checkpoint pair; one gateway-controlled lease/turn; + auth/private-state environment separation; binary-override compatibility; + abort/TERM/KILL timing; cancellation/deadline/shutdown races; observed + escaped-descendant lease retention; poisoned Task/session behavior; + post-teardown reconciliation; evidence ordering; result states; cleanup + outcomes; and truthful unobserved-orphan/cache-limit fixtures pass on Linux. ### U4. Docker-only Git and OCI immutable-base acquisition @@ -2027,61 +2712,72 @@ carrier. ### U5. Codex SDK adapter - **Goal:** Run built-in and profile-backed Codex targets on the trusted host - through the pinned SDK while preserving progress, result, usage, cancellation, - existing authentication, and truthful evidence. + through the pinned SDK while preserving one-shot and resumable thread + semantics, progress, result, usage/cache reporting, cancellation, existing + authentication, and truthful evidence. - **Requirements:** R7-R8, R13-R16, R18; AE1, AE3-AE4, AE10-AE12, - AE15, AE17-AE18; KTD9, KTD11-KTD12. + AE15, AE17-AE21; KTD9, KTD11-KTD12. - **Files:** `apps/gateway` Codex adapter, typed profile projection, environment - policy, SDK fixtures, shared conformance tests, and optional credentialed - smoke tests. -- **Approach:** Use pinned `@openai/codex-sdk` first. Create one fresh execution - context per Task; pass the resolved cwd, access mode, Task-private runtime - paths, and typed profile settings. For `readOnly`, request the native read-only - policy when supported and keep preparation outside the base. Preserve the - existing host `CODEX_HOME`/ChatGPT login when API credentials are absent; - stream/normalize events and usage; connect native abort to U3; bound evidence; - and dispose. - Use app-server only if U0 recorded a specific required SDK gap and pin/probe - its protocol. - Pass native `outputSchema` only for the supported Structured Outputs subset; - otherwise add JSON guidance and use the common terminal validator. + policy, SDK thread/checkpoint fixtures, shared conformance tests, and optional + credentialed smoke tests. +- **Approach:** Use pinned `@openai/codex-sdk` first. Call `startThread()` for a + session start, `resumeThread(threadId)` for each later turn, and a fresh thread + for one-shot execution; pass resolved cwd, access mode, Task/session-private + runtime/state home, and typed profile settings. Persist only the opaque thread + ID after successful turn settlement. For `readOnly`, request native read-only + policy when supported and keep preparation outside the base. Pass API + credentials through the explicit allowlist, or reference an existing ChatGPT + login only through the U0-proven separate auth mechanism; never point mutable + SDK thread state at the host auth home. Stream/normalize events and native + usage including cached input; connect abort to U3; bound evidence; and dispose + private state on close. Use app-server only if U0 recorded a specific required + SDK gap and pin/probe its protocol. Pass native `outputSchema` only for the + supported Structured Outputs subset; otherwise add JSON guidance and use the + common terminal validator. - **Execution note:** Characterize pinned SDK/model auth, abort, event, tool, and schema behavior before normalization. Provider and model tools retain trusted CI-job authority; tests inspect the explicit environment but make no hostile- code, network, or secret-isolation claim. Do not import Promptfoo or AI SDK Harnesses and do not download Codex per request. -- **Verification:** Shared adapter conformance; built-in/profile targets; - existing `CODEX_HOME` and API-credential paths; environment allowlist; exact - override probe; event/usage/result normalization; graceful/TERM/KILL - cancellation; native-schema and validated-fallback paths; deadline; malformed - provider payload; and opt-in credentialed smoke pass outside Docker. +- **Verification:** Shared adapter conformance; built-in/profile targets; API + credentials and any advertised ChatGPT-login path each prove host-auth/private- + state separation; start/two-turn resume/close, exact opaque checkpoint, + conversation/workspace continuity, close cleanup, native cached-input + reporting without a required hit, environment allowlist, exact override probe, + event/usage/result normalization, graceful/TERM/KILL cancellation, native- + schema and validated-fallback paths, deadline, malformed provider payload, and + opt-in credentialed smoke pass outside Docker. ### U6. Pi RPC adapter - **Goal:** Run built-in and profile-backed Pi targets on the trusted host through - the pinned supported package/RPC surface with the same public lifecycle and - honest capability reporting. + the pinned supported package/RPC surface with the same one-shot/session + lifecycle and honest capability/cache reporting. - **Requirements:** R7-R8, R13-R16, R18; AE3-AE4, AE10-AE12, AE15, - AE17-AE18; KTD9, KTD11-KTD12. + AE17-AE21; KTD9, KTD11-KTD12. - **Files:** `apps/gateway` Pi adapter/RPC parser, restricted policy extension, - typed profile projection, environment policy, fixtures, shared conformance - tests, and optional credentialed smoke tests. -- **Approach:** Launch Pi directly in the resolved cwd with access mode, - Task-private runtime paths, typed invocation configuration, existing host Pi - authentication location, strict RPC, explicit supported tools/extensions, - deterministic permissions, validated events, bounded evidence, and U3 - cancellation escalation. Request a native read-only policy when supported. - Never copy, mount, parse, or import Pi auth. - Repository extensions and unrestricted built-ins remain disabled. A global - Pi binary override must pass the exact pinned version/protocol probe. -- **Execution note:** Characterize and pin Pi's RPC/auth/abort/event contract. - Pi-specific facts remain bounded native evidence rather than public schema - branches. Model tools retain trusted CI-job authority; do not claim the + typed profile projection, environment policy, session checkpoint fixtures, + shared conformance tests, and optional credentialed smoke tests. +- **Approach:** Launch Pi directly in resolved cwd with access mode, + Task/session-private runtime/state paths, typed invocation configuration, + strict RPC, explicit supported tools/extensions, deterministic permissions, + validated events, bounded evidence, and U3 cancellation escalation. Reference + a host Pi auth location only through the U0-proven separate mechanism. Use the + pinned native create/resume/checkpoint/dispose surface for Sessions; advertise + only modes whose auth/state/checkpoint combination passes. Request native + read-only policy when supported. Never copy, mount, parse, or import Pi auth. + Repository extensions and unrestricted built-ins remain disabled. A global Pi + binary override must pass the exact pinned version/protocol probe. +- **Execution note:** Characterize and pin Pi's RPC/auth/session/abort/event + contract. Pi-specific facts remain bounded native evidence rather than public + schema branches. Model tools retain trusted CI-job authority; do not claim the explicit environment isolates provider/MCP/operator secrets. - **Verification:** Shared adapter conformance; built-in/profile targets; - existing host auth; environment allowlist; exact override probe; strict + host-auth/private-state separation; advertised modes match one-shot/create/ + resume/checkpoint/dispose probes; two-turn continuity and close when available; + native cache usage only; environment allowlist; exact override probe; strict malformed/unknown RPC rejection; event/usage/result normalization; graceful/ - TERM/KILL cancellation; deadline; and opt-in credentialed smoke pass outside + TERM/KILL cancellation; deadline; and opt-in credentialed smoke outside Docker. Malformed RPC can never produce success. ### U7. End-to-end delivery and documentation @@ -2091,7 +2787,7 @@ carrier. workspace/source configuration, host auth, registry coverage, Promptfoo consumption, installation, release ordering, and limits. - **Requirements:** R1-R19; F1-F6; AE1-AE21; KTD1-KTD14. -- **Files:** published extension/snapshot-format pages, gateway guide/reference, +- **Files:** published profile/snapshot-format pages, gateway guide/reference, configuration reference, README, CHANGELOGs, real project/user workspaces, AI Evals-style Promptfoo YAML/provider contract fixture, E2E fixtures, CLI-only and gateway packed-install smokes, acquisition-image release record, @@ -2099,10 +2795,11 @@ carrier. - **Approach:** After final review, pack `apps/cli` and `apps/gateway` independently without publishing. Prove CLI-only installation resolves neither gateway nor image; install the gateway tarball in a clean trusted - Linux environment with Docker and pre-existing Codex/Pi host auth. Create - project/user workspaces under `/tmp/`; serve on loopback and `0.0.0.0`; test - probes and the complete A2A lifecycle; acquire local Git/OCI plus live registry - fixtures through the exact image; and run Codex/Pi on the host. Exercise the + Linux environment with Docker and pre-existing supported Codex/Pi host auth. + Create project/user workspaces under `/tmp/`; serve on loopback and a specific + private TLS address, plus a loopback-only fake private-TLS proxy; reject + wildcard/public listeners; test probes and the complete A2A lifecycle; acquire + local Git/OCI plus live registry fixtures through the exact image; and run Promptfoo consumer fixture in both source modes with per-trial logical cwd and workspace access. Prove read-only Tasks reuse one immutable base without shared runtime state, read-write Tasks receive independent disposable views, @@ -2110,11 +2807,14 @@ carrier. Run registry workflows with the exact gateway tarball, acquisition manifest, supported platform digests, and build commit. Gateway publication is blocked - until the image has passed required GHCR/JFrog conformance. Document that - network peers have full Task authority, providers/model tools have CI-job - authority, explicit environments are not isolation, evidence follows only - direct-process settlement, Docker is acquisition-only, and ephemeral runner - teardown is the final orphan boundary. + until the image has passed required GHCR/JFrog conformance. Publish the + transport-neutral HEC Core and Sessions contracts, A2A binding, + coding-workspace extension, composed Profile URN, and conformance entry points + without claiming a neutral standard or Responses/UHP binding. Document that + network peers have full Task/session authority, providers/model tools have + CI-job authority, explicit environments are not isolation, evidence follows + only direct-process settlement, Docker is acquisition-only, and ephemeral + runner teardown is the final orphan boundary. - **Execution note:** Green smoke uses the release-candidate npm tarball and exact acquisition image artifacts, never a checkout rebuild. The consumer fixture is AI Evals-owned test/documentation code; AllAgents runtime does not @@ -2133,25 +2833,25 @@ carrier. | Gate | Applies to | Required evidence | |---|---|---| -| Bun architecture feasibility | U0 | Exact A2A JavaScript SDK pin and official-client server-direction operations; pinned Codex SDK and Pi RPC/package probes; existing host-auth behavior; shared read-only base/private runtime; reflink, rootless-overlay, and copy probes; Linux abort/TERM/KILL process-group probe with truthful descendant limit; exact multi-architecture acquirer image; independent packed CLI/gateway installs; immutable tarball/image binding | +| Bun architecture feasibility | U0 | Exact A2A SDK pin and official-client operations; HEC Core/Sessions vectors through an in-memory adapter/A2A binding; two-turn Promptfoo context/head handoff; pinned Codex/Pi mode, checkpoint, auth/private-state, and cache probes; shared read-only base/private runtime; read-write prior/candidate/commit/crash probes across materializers; abort/TERM/KILL plus observed-outliving lease retention and truthful unobserved-descendant limit; multi-architecture acquirer image; independent packed installs; immutable tarball/image binding | | Bun repository quality | U0-U7 | One lockfile; private root orchestration; workspace-scoped typecheck/lint/test/build; dependency and image scans; generated-contract drift; minimized runtime dependencies; packed and installed size budgets | | Package and release separation | U0-U1, U7 | Independent `allagents` and `allagents-gateway` versions/tags/triggers/tarballs; image-first gateway release; exact npm tarball plus acquisition manifest/platform digests; idempotent publication; CLI-only install fetches neither gateway nor image; gateway-only release never publishes the CLI | -| Workspace and contract packages | U0-U1 | Only `workspace-config`, `execution-contracts`, and `acquisition-contracts` shared packages; generated portable `contracts/` fixtures; normalized catalogs/defaults/order/collision keys/stable errors; project/user parsing; schema/spec drift; no `core`/`common` | -| Public contract | U0-U2 | Official JavaScript client against the Bun gateway; card interface/params/streaming; A2A version and extension headers; unified Parts; logical cwd and workspace-access schema/default/canonicalization/integrity evidence; both send modes; complete listing; metadata; `google.rpc.Status`; request/result/Artifact fixtures | -| Trusted-network and runner model | U2-U7 | Loopback and `0.0.0.0` with distinct advertised URL; HTTPS docs; shared external Task visibility/cancellation; trusted Linux CI job/VM/deployment container as provider isolation boundary; read-only described as cooperative best-effort; explicit no-hostile-code/no-secret-isolation wording; metadata-only probes | -| Durable Task lifecycle | U1-U3 | Private gateway-owned `bun:sqlite`; foreign keys and `synchronous=FULL`; transactions for create/replay, base pins, lease, intent, settlement, and expiry; no early eviction; process-kill/store faults; lock/restart/interrupted-Task reconciliation | +| Workspace and contract packages | U0-U1 | Only `workspace-config`, `execution-contracts`, and `acquisition-contracts` shared packages; separately versioned HEC Core, Sessions, A2A binding, and coding-workspace extension inside `execution-contracts`; generated portable `contracts/` fixtures; normalized catalogs/defaults/order/collision keys/stable errors; project/user parsing; schema/spec drift; no catch-all `core`/`common` package | +| Wire contract | U0-U2 | Official JavaScript client against the Bun gateway; normative transport-neutral HEC Core per-turn Invocation, Sessions start/resume/close/checkpoint semantics, neutral state/cancel outcomes, ordered execution-event stream, Core outcome/trajectory, A2A binding, coding-workspace extension/integrity, and one composed Profile URN; shared Core/Sessions vectors plus binding/extension/composed conformance; A2A card/version/profile, context/session and prior-Task mapping, one-Message/one-Task-per-turn, unified Parts, three reserved Artifacts, both send modes, exact byte-bounded listing, state/cancel/event/Artifact/error projection, metadata, and request/result fixtures | +| Trusted-network and runner model | U2-U7 | Loopback HTTP, loopback behind private HTTPS ingress, and native TLS on a specific private address; wildcard/public listener and public advertised-address rejection; TLS file and topology validation; external ACL enforcement; shared external Task/session visibility and cancellation; trusted Linux CI job/VM/deployment container as provider isolation boundary; one gateway-controlled turn with truthful escaped-descendant limit; read-only cooperative best-effort; explicit no-hostile-code/no-secret-isolation wording; metadata-only probes | +| Durable Task and session lifecycle | U1-U3 | Private gateway-owned `bun:sqlite`; foreign keys and `synchronous=FULL`; transactions for create/replay, exact head compare-and-set, Task/session and prior/candidate byte reservations, base pins, lease, intent, prepared settlement, terminal Task/Artifacts/events, provider-checkpoint plus workspace-generation advancement, superseded-generation cleanup, close, and expiry; crash before/after pointer commit; poisoned state retains resources until reconciliation; no early eviction; process/store faults and restart/tombstone reconciliation | | Acquisition-container boundary | U0, U4, U7 | One fresh container when no reusable validated base exists and none on cache hit; exact digest-pinned image; staging-only writable bind; selected repository/registry credentials and CA material only; no App private key, host home, Docker socket, gateway state, provider auth, Codex, Pi, or harness downloads; strict source network/size/archive policy; typed manifest; exit/removal before host validation and provider execution; orphan-staging cleanup | -| Immutable-base cache | U0-U4, U7 | Key binds acquisition contract, compiled catalog/layout, and immutable source; exact commit/digest reuse; branch/tag bypass into non-reusable Task-owned bases; atomic promotion; active pins; unpinned LRU byte-budget eviction; one acquisition across 100 identical read-only trials; no cached credentials; transient-base cleanup | -| Repository acquisition | U0, U4 | Compiled-name resolution; hermetic Git/full commits; App 200/404/ambiguous eligibility; fresh base-acquisition token validation/revocation; cache-hit no credential; `gh` only after positive ineligibility; acquisition sub-budget; no provider start on failure | +| Immutable-base cache | U0-U4, U7 | Key binds acquisition contract, compiled catalog/layout, and immutable source; exact commit/digest reuse; branch/tag bypass into non-reusable Task/session-owned bases; atomic promotion; active Task/session pins; unpinned LRU byte-budget eviction; one acquisition across repeated immutable read-only trials/turns; no cached credentials; transient-base cleanup | +| Repository acquisition | U0, U4 | Compiled-name resolution; hermetic Git/full commits; App 200/404/ambiguous eligibility; durable conservative-expiry pre-mint intent; exact-expiry token validation/revocation; crash or ambiguous response at every mint/revocation boundary; cache-hit no credential; `gh` only after positive ineligibility; acquisition sub-budget; no provider start on failure | | OCI acquisition | U4 | Canonical Docker Hub plus GHCR/JFrog/private-registry matrix; exact-key auth/helper/CA; bounded Basic/Bearer; redirect/rebinding policy; direct-image/config/layer media; descriptor verification; path-free manifest/private layout; changesets/whiteouts; fixed limits; no fallback | | Registry and exact-artifact conformance | U4, U7 | Local Distribution and public digest-pinned GHCR on every PR; authenticated GHCR and private-CA JFrog release targets; exact gateway npm tarball plus acquisition multi-architecture manifest/platform digests without rebuild; positive/negative auth/permission/CA/media/path cases; explicit architecture coverage | -| Host supervisor lifecycle | U3 | One active lease; reusable-base read-only/private-runtime and non-reusable-base paths; independent writable views and cleanup; provider PID/start identity persisted before started state; explicit environment allowlist and host auth paths; cancel/deadline/shutdown races; graceful abort then TERM/KILL; non-settling failure retains Task-owned state/lease and blocks readiness; verified post-teardown reconciliation | -| Truthful bounded evidence | U3-U6 | Live bounded events; collection only after the direct provider settles and escalation finishes; hermetic Git inspection; observed termination/cleanup recorded; no claim of full descendant quiescence, hostile-code containment, secret isolation, or opaque-output redaction | -| Backend conformance | U0, U2-U3, U5-U6 | Narrow access-aware AllAgents adapter contract; same lifecycle suite for fake, Codex SDK, and Pi RPC/package; built-in/profile variants; reusable or non-reusable read-only bases and independent read-write views; resolved logical cwd/runtime/access passed to providers; existing host auth; exact binary override probes; direct host execution outside acquisition Docker; no AI SDK Harnesses or per-request runtime download | -| Structured result | U1, U3, U5-U6 | Public grammar; Codex native-subset gate and validated fallback; valid/invalid/not-produced states; Artifact cardinality; malformed provider/RPC payload cannot publish success | +| Host supervisor lifecycle | U3 | One active gateway-controlled lease/turn; reusable-base read-only/private-runtime and non-reusable-base paths; one-shot views versus immutable committed session generations plus per-turn candidates; provider PID/start identity before started state; explicit environment and separated host-auth/private-state paths; cancel/deadline/shutdown races; abort then TERM/KILL; prepared settlement and checkpoint/workspace-pointer commit; cleanup before lease release; non-settling or observed-outliving failure retains Task/session state, lease, and unready status; truthful unobserved-descendant limit | +| Truthful bounded evidence | U3-U6 | Durable live ordered Core progress/tool events; canonical JSON/text payloads; deterministic prefix truncation; one result per call and interrupted-call rules; truthful complete/truncated markers; collection only after direct provider settlement/escalation; hermetic Git inspection; observed termination/cleanup; no claim of full descendant quiescence, hostile-code containment, secret isolation, opaque-output redaction, unobserved events, or inferred cache hits | +| Backend conformance | U0, U2-U3, U5-U6 | Narrow access-aware adapter contract; HEC lifecycle/event/trajectory and Sessions create/resume/checkpoint/dispose suites for fake/Codex/Pi; per-target mode advertisement; private mutable state with proven separate host-auth reference; read-only bases/private runtime, one-shot views, and session candidate/committed generations; resolved cwd/access; native cached-token reporting only; exact binary probes; host execution outside acquisition Docker; no transcript replay, AI SDK Harnesses, or per-request download | +| Structured result | U1, U3, U5-U6 | Public grammar; Codex native-subset gate and validated fallback; valid/invalid/not-produced/retention-limit states; three reserved Artifact cardinalities; oversized or malformed provider/RPC result cannot publish success | | Repository quality | All | Bun install/typecheck/lint/test/build; focused and full suites; clean-registry packed installs; dependency/image audit; generated schema/spec checks; docs build | -| Packaged gateway E2E | U7 | Recorded red/green `/tmp/` commands; explicit gateway install; exact acquisition image; Git/local OCI/GHCR/JFrog sources; base reuse/materialization/cleanup; advertised URL/probes; capacity/replay/cancel/deadline/shutdown/restart; host Codex/Pi auth; truthful trust documentation | -| Promptfoo consumption | U7 | Secure-default AI Evals YAML for both source modes; optional context; per-trial `allagentsWorkingDirectory` and `allagentsWorkspaceAccess`; shared-base read-only trials; independent disposable read-write views; nonblocking acceptance/subscription/cancel; logical source/cwd/access provenance without origins or physical paths; output/usage/error metadata mapping; no AllAgents Promptfoo runtime dependency | +| Packaged gateway E2E | U7 | Recorded red/green `/tmp/` commands; exact gateway/image artifacts; Git/local OCI/GHCR/JFrog; one-shot reuse/cleanup; read-write candidate commit/crash recovery; two-turn session continuity/close/expiry; direct-private TLS and loopback/private-proxy topology plus wildcard/public rejection; capacity/replay/cancel/deadline/shutdown/restart; separated Codex/Pi auth/private state; truthful trust/cache docs | +| Promptfoo consumption | U7 | Secure-default AI Evals YAML for both source modes; optional context; strict logical override/session inputs; provider-managed `allagentsConversation` map keyed by explicit ID with isolated interleaving and start/continue/close; manual `allagentsSession` crash recovery; two terminal Tasks with exact returned context/head; one-shot base/view behavior; nonblocking subscribe/cancel; provenance and output/usage/native-cache/error metadata; no AllAgents Promptfoo dependency | ## Definition of Done @@ -2160,23 +2860,26 @@ carrier. - Every R1-R19 requirement is implemented or explicitly demonstrated by a passing acceptance scenario; F1-F6 and AE1-AE21 agree with the implementation and error table. -- U0's Bun/A2A/provider/process/acquirer/package gate passes before dependent - units. The private root, three apps, three named packages, and generated - `contracts/` fixtures are the complete shared layout; no speculative shared - package, native sidecar, or split runtime remains. +- U0's HEC Core/Sessions/A2A-binding, Bun, provider, process, acquirer, and + package gates + pass before dependent units. The private root, three apps, three named + packages, and generated `contracts/` fixtures are the complete shared layout; + no speculative shared package, native sidecar, or split runtime remains. - `allagents` and `allagents-gateway` remain independently versioned and released. A CLI-only install fetches neither gateway nor acquisition image. Gateway release builds/verifies the exact acquisition image and registry reports before publishing the bound npm tarball. -- The gateway starts without `gateway.yaml` or `worker.yaml`, defaults to - loopback HTTP, accepts explicit `0.0.0.0`, requires a distinct advertised URL - away from default loopback, documents production HTTPS, and exposes truthful - metadata-only health/readiness. -- Network reachability is the external caller authorization boundary; Task - visibility and idempotency are deployment-wide. Provider execution uses the - trusted Linux CI job/VM/deployment-container boundary and existing host auth. - Documentation explicitly says AllAgents does not contain hostile repository - code or isolate provider/MCP/operator secrets from model-invoked tools. +- The gateway starts without `gateway.yaml` or `worker.yaml` and defaults to + loopback HTTP. Remote service is loopback-only behind private HTTPS ingress or + native TLS on one specific private address. Startup rejects wildcard/public + binds, missing/mismatched TLS inputs, public advertised resolution, and remote + HTTP; health/readiness remain metadata-only. +- Network reachability within that private boundary authorizes callers; Task and + session visibility/idempotency are deployment-wide. Provider execution uses + the trusted Linux CI job/VM/deployment-container boundary. Host authentication + is referenced only through an adapter-proven path separate from private + mutable provider state. Documentation says AllAgents does not contain hostile + repository code or isolate provider/MCP/operator secrets from model tools. - Project workspace declarations compile to the exact repository/snapshot catalog; user declarations own profile launcher gateway enablement; built-in IDs cannot be shadowed. Requests may select only the effective workspace root @@ -2185,19 +2888,31 @@ carrier. destination paths, origins, credentials, commands, provider environments, materializers, cache keys, Docker images/options/mounts, or provider permission policy. -- The published extension, Agent Card interface/params, A2A version and - activation headers, logical cwd and workspace-access unions/defaults, unified - Parts, both send modes, full `ListTasks`, metadata preservation, strict - schemas, HTTP+JSON errors, embedded Artifacts, canonicalization, retention, - and cancellation pass official-client fixtures. +- The published transport-neutral HEC Core Invocation, neutral state/cancel + outcomes, ordered canonical progress/tool events, deterministic trajectory, + outcome schema, Sessions extension, A2A binding, AllAgents coding-workspace + extension, and composed v1 Profile URN pass common Core/Session suites plus + official-client, binding-specific, extension, and composed-profile suites. + Binding tests own Agent Card/version/activation, context-ID/session and prior- + Task mapping, one-Message/one-Task-per-turn, Core-to-A2A state/cancel/event/ + Artifact/error mapping, unified Parts, three reserved Artifacts, both send + modes, exact byte-bounded `ListTasks`, metadata, canonicalization, retention, + and cancellation. Sessions tests own start/resume/close, exact head, + linearization, pinned configuration, expiry, restart/poison handling, and + cache-usage truthfulness. Workspace tests own logical cwd/access, source/ + provenance, candidate/committed session generations, produced bytes, + integrity, and cleanup. + The release calls HEC a contract/standard candidate, not a neutral standard, + and exposes no Responses/UHP binding in v1. - Secure-default AI Evals Promptfoo YAML selects repository mode with optional - named revisions or snapshot mode with immutable digests and can replace the - logical cwd and access per trial. Each `callApi` maps to one nonblocking Task; - read-only Tasks may share the immutable base and physical cwd, while read- - write Tasks receive independent disposable views. Ambiguous retries retain the - same key, base/view, cwd, and access. The provider propagates cancellation, - normalizes usage, and returns safe failure/logical provenance without origins, - configured destinations, physical paths, or an AllAgents Promptfoo dependency. + named revisions or snapshot mode with immutable digests, can replace logical + cwd/access per trial, and can start/resume/close a linear session. Each + `callApi` maps to one nonblocking Task; resumed Tasks retain provider context + and the session workspace. Ambiguous retries retain the same invocation key, + context/head, base/view, cwd, and access. The provider propagates cancellation, + normalizes usage including native cached-input tokens, and returns safe + failure/logical provenance without origins, configured destinations, physical + paths, or an AllAgents Promptfoo dependency. - Every request without a reusable validated base starts the exact digest-pinned image with staging as its only writable bind plus source-only credentials and strict policy. A valid cache hit starts no container and resolves no @@ -2211,75 +2926,94 @@ carrier. immutable digests, descriptor verification, exact redirect/auth/CA rules, changesets, fixed limits, no cross-mode fallback, and truthful source verification. -- App eligibility/ambiguous 404 handling, base-acquisition fresh token - validation/revocation, cache-hit credential avoidance, positive-ineligibility - selection, OCI auth/challenges, exact-host CA, and acquisition credential - teardown pass. Local Distribution and public digest-pinned GHCR run on every - PR; authenticated GHCR and private-CA JFrog release reports match the exact - gateway tarball, acquisition manifest, supported platform digests, commit, and - compatibility output. -- Codex uses pinned `@openai/codex-sdk` first and existing `CODEX_HOME`/ChatGPT - login when API credentials are absent; app-server is used only for a recorded - SDK capability gap. Pi uses its pinned supported package/RPC surface and - existing host auth. No OAuth/auth files are copied, mounted, parsed, or - imported, and no provider runtime is downloaded per request. -- Direct providers start in Linux process groups with explicit environments that - preserve required identity/auth paths. Read-only Tasks use shared immutable - bases with private runtime state and best-effort provider policy; read-write - Tasks use independent disposable block-cloned, overlaid, or copied views. - Cancellation escalates adapter abort to `SIGTERM` to `SIGKILL`. Evidence is - bounded and begins only after the direct provider settles. A non-settling - provider publishes no filesystem/Git evidence, retains its Task-owned runtime - and any writable view plus the lease, and blocks readiness until verified - post-teardown reconciliation. Evidence reports observed termination/cleanup, - not full descendant quiescence; CI runner teardown is the final orphan - boundary. -- Ordinary private Bun SQLite ownership, foreign keys, full synchronization, - create/replay, base pins, one lease, internal outcome races, atomic settlement, - immutable terminal Tasks, expiry, crash/restart reconciliation, cache - eviction, and cleanup pass fault tests without a custom VFS or native file - layer. -- Evaluation behavior, public-Internet authentication, remote workers, caller- - selected custom materializers, per-provider Docker, native containment +- App eligibility/ambiguous 404 handling, durable pre-mint intent, + exact-expiry token validation/revocation, ambiguous-response and crash + tombstones through every mint/revocation boundary, cache-hit credential + avoidance, positive-ineligibility selection, OCI auth/challenges, exact-host + CA, and acquisition credential teardown pass. Local Distribution and public + digest-pinned GHCR run on every PR; authenticated GHCR and private-CA JFrog + release reports match the exact gateway tarball, acquisition manifest, + supported platform digests, commit, and compatibility output. +- Codex uses pinned `@openai/codex-sdk` first; app-server is used only for a + recorded SDK capability gap. Codex/Pi advertise only auth/mode combinations + that prove host-auth references are separate from Task/session-private mutable + state. Auth files are never copied, mounted, parsed, or imported, and provider + runtimes are never downloaded per request. +- Direct providers start in Linux process groups with explicit environments and + separated auth/private-state paths. One-shot read-only Tasks and read-only + sessions use shared immutable bases with private runtime; one-shot read-write + Tasks use disposable views; read-write sessions use immutable committed + generations plus per-turn candidates. Cancellation escalates adapter abort to + `SIGTERM`/`SIGKILL`. Evidence starts only after the direct provider settles. A + non-settling provider or observed outliving descendant publishes no + filesystem/Git evidence, retains Task/session state plus the lease, and blocks + readiness until verified disappearance/reconciliation or runner teardown. + Evidence reports observed cleanup, not descendant containment. +- Private Bun SQLite ownership, foreign keys, and full synchronization cover + create/replay, base pins, one lease, outcome races, prepared Task/Artifact/ + event settlement, provider-checkpoint/workspace-generation advancement, + old-generation cleanup, terminal-tail/aggregate reservations, immutable Tasks, + poison-exempt expiry, crash around every pointer/cleanup boundary, restart, + mint/revocation tombstones, retained-count/per-Task/aggregate/response/session + byte limits, cache eviction, and cleanup fault tests without a custom VFS or + native file layer. +- Evaluation orchestration, public-Internet authentication/exposure, remote + workers, Responses/UHP binding, session branching/concurrent turns, + caller-selected custom materializers, per-provider Docker, native containment primitives, non-Linux gateway execution, and multi-tenant policy remain absent. ### Per unit -- U0: Bun workspace layout, official-client A2A server direction, pinned Codex - SDK/Pi surface probes, host-auth behavior, shared read-only/private-runtime and - read-write materializer probes, explicit environment/process-group - feasibility, multi-architecture acquirer image, independent packed installs, - and exact tarball/image release binding all pass. +- U0: HEC Core/Sessions vectors pass through an in-memory adapter/A2A binding; + official-client, composed-profile, and two-turn Promptfoo context/head probes + pass; anonymous-conformance is recorded; pinned Codex/Pi mode/checkpoint, + auth/private-state, and native-cache probes pass; read-only/private-runtime and + read-write prior/candidate/commit/crash materializer probes pass; process-group + escalation plus observed-outliving lease retention and truthful unobserved- + descendant limits pass; multi-architecture image, independent packed installs, + and exact tarball/image release binding pass. - U1: Three narrow packages and generated fixtures, workspace additions, - execution/acquisition contracts including logical cwd, workspace access, - relative-path grammar, base-cache identity, and materialization errors, Bun - SQLite transactions/pins, published extension/snapshot format, independent - versions, compatibility matrix, and image-first release fixtures agree. -- U2: Official-client operations, version/extension/error/list semantics, - logical cwd/access defaults/canonicalization/integrity evidence, deployment- - wide replay/visibility, SQLite lock/crash/lease behavior, listeners/advertised - URL/probes, deadline/shutdown/restart/retention, and fake backend pass. -- U3: The fake lifecycle proves shared immutable-base read-only execution with - private runtime state, independent read-write views across every configured - materializer, cwd resolution/escape rejection, explicit provider environments, - required host-auth preservation, process-group abort/TERM/KILL, outcome races, - atomic settlement, evidence ordering, restart cleanup, and truthful orphan - limitations on Linux. -- U4: Git/OCI fixtures, App/`gh` selection, cache hit/miss/key/pin/eviction, - exact acquisition image boundary, staging-only mount, source credentials/ - network/limits, typed manifest, host revalidation/publication, local/public/ - authenticated GHCR, private-CA JFrog, exact manifest/platform digests, no - fallback, and leak scans pass. -- U5: Codex passes shared access-aware conformance and both schema paths through - the pinned SDK or documented required app-server fallback, receives resolved - cwd/runtime/access, reuses existing host auth, runs outside Docker, and records - optional credentialed smoke evidence. -- U6: Pi passes the same host-process conformance through pinned RPC/package - support with resolved cwd/runtime/access, reuses existing auth, and malformed - RPC cannot produce success. -- U7: Final review is resolved; CLI-only/gateway packed smokes and `/tmp/` E2E, - independent release/size/SBOM evidence, public GHCR on every PR, - authenticated GHCR/private-CA JFrog exact-artifact reports, Promptfoo - per-trial logical-cwd/access shared-base/read-write-view fixture, repository - gates, published schemas/specs, truthful threat-model docs, and reproducible PR - instructions are complete. + normative HEC Core/Sessions/A2A-binding/coding-workspace-extension text and + separate conformance groups, composed v1 wire contract, Core Invocation/state/ + cancel/event/outcome/trajectory semantics, Sessions identity/head/checkpoint/ + expiry/close semantics, binding-owned context IDs and prior-Task references, + three reserved Artifacts, execution/acquisition contracts including logical + cwd, workspace access, relative-path grammar, base-cache identity, and + materialization errors, Bun SQLite Task/session transactions/reservations and + byte budgets, published contract/snapshot format, independent versions, + compatibility matrix, and image-first release fixtures agree. +- U2: Official-client operations, version/profile/error/list and response-budget + semantics, Core/Sessions-to-A2A state/cancel/event/context/head/Artifact + mapping, logical cwd/access defaults/canonicalization/workspace integrity, + deployment-wide replay/visibility, linear start/resume/close/expiry and + stale-head/busy rejection, SQLite lock/crash/lease/reservation behavior, + listeners/advertised URL/probes, deadline/shutdown/restart/retention, and fake + backend pass. +- U3: The fake lifecycle proves shared immutable-base read-only/private-runtime + execution, one-shot views, and read-write session prior/candidate/committed + generations across every materializer; candidate reservations; cwd/escape + rejection; separated auth/private-state environments; durable events/ + trajectory; process-group escalation and observed-outliving lease retention; + outcome races; prepared Task/Artifact/event settlement plus atomic provider- + checkpoint/workspace-pointer advancement; crashes before/after pointer commit; + cleanup/poison/restart behavior; byte bounds; and truthful orphan limits. +- U4: Git/OCI fixtures, App/`gh` selection, pre-mint intent and mint/revocation + tombstone reconciliation, cache hit/miss/key/pin/eviction, exact acquisition + image boundary, staging-only mount, source credentials/network/limits, typed + manifest, host revalidation/publication, local/public/authenticated GHCR, + private-CA JFrog, exact manifest/platform digests, no fallback, and leak scans + pass. +- U5: Codex passes shared one-shot/session, access, event, cancellation, + checkpoint, cache-reporting, and private-state/host-auth separation conformance + through the pinned SDK or documented app-server fallback, receives resolved + cwd/runtime/access, runs outside Docker, and records optional credentialed + smoke evidence. +- U6: Pi passes the same host-process/auth-state conformance through the pinned + RPC/package surface and cannot publish success from malformed RPC. Each mode is + advertised only when its one-shot/create/resume/checkpoint/dispose probes pass. +- U7: Final review is resolved; packed smokes and `/tmp/` E2E, independent + release/size/SBOM evidence, public GHCR, authenticated GHCR/private-CA JFrog + exact-artifact reports, Promptfoo one-shot and isolated/interleaved two-turn + conversation-map plus explicit-session recovery fixtures, logical cwd/access, + truthful cached-input usage, repository gates, published schemas/specs, + threat-model docs, and reproducible PR instructions are complete. From 03564c7baa457fafd0bc181335a6e2ee01bfe87c Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Tue, 22 Sep 2026 12:30:03 +1000 Subject: [PATCH 14/44] docs(architecture): adopt UHP through HarnessRouter --- .../0002-adopt-uhp-through-harnessrouter.md | 308 ++ ...-agent-execution-through-an-a2a-gateway.md | 316 -- ...0837-feat-coding-execution-gateway-plan.md | 3678 ++++------------- .../agent-host-protocol-decision-inputs.md | 6 +- 4 files changed, 1059 insertions(+), 3249 deletions(-) create mode 100644 docs/decisions/0002-adopt-uhp-through-harnessrouter.md delete mode 100644 docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md new file mode 100644 index 00000000..04f1c921 --- /dev/null +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -0,0 +1,308 @@ +# ADR 0002: Adopt UHP through HarnessRouter with an AllAgents workspace materializer + +- Status: Accepted; implementation pending +- Date: 2026-09-17 +- Updated: 2026-09-21 +- Supersedes: the original A2A/HEC execution-gateway design recorded by this ADR + +## Decision + +AllAgents will use the Unified Harness Protocol (UHP) `2026-09-12` through a +pinned HarnessRouter Community Edition deployment for remote Codex execution. +Pi remains capability-gated until a real probe proves a HarnessRouter-supported +custom-provider format against `codex-lb`. + +AllAgents will not implement an A2A gateway, a separate Harness Execution +Contract, provider adapters, or its own task/session engine. HarnessRouter owns +authentication, UHP request and response semantics, streaming, cancellation, +idempotency, session continuity, per-session workspaces, agent execution, usage, +and artifacts. + +The initial deployment will use a narrow AllAgents-maintained HarnessRouter fork. +The fork adds a generic pre-turn workspace-materializer hook. An AllAgents +materializer behind that hook interprets a namespaced workspace descriptor, +acquires the configured Git repository set or an immutable OCI workspace +snapshot, and returns normalized provenance before HarnessRouter launches the +agent. + +The fork is a delivery mechanism, not a new protocol. All fork changes must be +structured for a later upstream contribution. Delivery does not depend on +upstream acceptance or timing. + +Implementation details live in the +[coding-agent execution gateway plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). + +## Topology + +```mermaid +flowchart TB + CLIENT[Promptfoo or another UHP client] + GATEWAY[Forked HarnessRouter gateway] + RUNNER[HarnessRouter runner] + MATERIALIZER[AllAgents workspace materializer] + LB[codex-lb] + MODEL[Model provider] + + CLIENT -->|UHP + allagents.workspace| GATEWAY + GATEWAY -->|runner /materialize| RUNNER + RUNNER -->|generic pre-turn hook| MATERIALIZER + MATERIALIZER -->|prepared staging + provenance| RUNNER + RUNNER -->|invocation-scoped broker credential| GATEWAY + GATEWAY -->|long-lived codex-lb API key| LB + LB -->|provider OAuth and routing| MODEL +``` + +Promptfoo authenticates to HarnessRouter with a HarnessRouter API key. +HarnessRouter runs in brokered sandbox-auth mode: the gateway holds a separate +`codex-lb` API key and gives the agent only an invocation-scoped broker +credential. `codex-lb` owns provider OAuth, account selection, continuation +affinity, and provider routing. Neither the `codex-lb` key nor provider OAuth +credentials enter Promptfoo, request metadata, the materializer, or the agent +workspace. + +## Protocol boundary + +UHP is the sole execution wire contract. Version one uses its Responses-shaped +request, ordered streaming events, `previous_response_id` continuation, +cancellation, files, artifacts, usage, lifecycle, and error semantics. +HarnessRouter's UHP conformance suite is the protocol oracle. + +AllAgents adds one namespaced request extension: + +```json +{ + "metadata": { + "allagents.workspace": { + "version": "1", + "source": { + "kind": "repositories", + "revisions": { + "api": "main" + } + }, + "workingDirectory": { + "kind": "repository", + "repository": "api", + "path": "packages/service" + } + } + } +} +``` + +The extension identifies configured sources by logical name. Callers cannot +supply repository or registry origins, host paths, credentials, commands, +environment variables, materializer executables, or Docker options. + +The materializer returns a strict result containing the resolved source +identity, logical working directory, workspace-manifest digest, and bounded +failure information. HarnessRouter returns that result in namespaced response +metadata. It does not expose configured origins, physical paths, or credentials. + +Ordinary UHP input files remain supported. HarnessRouter applies the materialized +workspace first and request input files second, so explicit attachments may +overlay source files deterministically. + +## Workspace and session semantics + +The workspace descriptor is accepted only when creating the first response in a +session. HarnessRouter binds its canonical digest and resolved provenance to the +session before agent execution. + +A continuation uses `previous_response_id` and the same HarnessRouter session +workspace. It must omit the workspace descriptor; the session's pinned +descriptor and provenance remain authoritative. A new source revision or +working directory requires a new session. + +The materializer runs before the first agent turn and never depends on the model +reading a prompt, calling an MCP tool, or extracting an archive. Source +acquisition failure starts no agent process and never falls through to another +source mode or credential identity. + +Workspaces are writable and private to the HarnessRouter session. Version one +does not add the former `readOnly` optimization or copy-on-write generations. +The fork extends HarnessRouter's existing workspace checkpoint with a durable +pre-agent materialization state and nested-repository collection metadata. + +Completed session state survives a HarnessRouter restart when its documented +durable data volume is preserved. An in-flight agent process does not survive +whole-container termination. Interrupted turns fail and are not replayed +automatically; a later continuation is allowed only when HarnessRouter reports +the session resumable. + +## Fork boundary + +The HarnessRouter fork is limited to the workspace-integration seam and the +custom Codex-provider rendering needed by `codex-lb`. The workspace seam: + +1. recognizes one configured, bounded metadata key on the first UHP response; +2. treats its JSON value as opaque, canonicalizes it, and binds it to the session; +3. calls a dedicated runner materialization endpoint once, before provider + selection or fallback; +4. invokes the configured materializer executable and validates its typed envelope; +5. publishes and checkpoints the prepared workspace before any agent starts; +6. allows a symlink-safe logical working directory beneath the session root while + preserving session-UID isolation; +7. preserves nested repository Git state and collects their produced files; +8. persists and returns bounded hook metadata through streaming, terminal, + retrieval, and idempotent-replay response paths; +9. rejects the workspace key on continuations; +10. applies ordinary input files only after successful materialization; and +11. strips every configured materializer-only environment name from agent children. + +The generic fork layer does not understand the AllAgents descriptor. It enforces +only the configured key, JSON/size bounds, immutable first-turn binding, hook +envelope, lifecycle, and response namespace. The external AllAgents executable +owns schema/default validation, workspace configuration, Git/OCI acquisition, +credential selection, filesystem policy, and provenance. + +The fork must preserve stock behavior for requests without the configured key +and must continue to pass upstream UHP conformance. The separate provider seam +only renders the documented `codex-lb` Codex identity and OpenAI-auth capability +fields. The maintained patch series is pinned to an upstream commit, covered by +focused integration tests, and kept free of unrelated changes. The intended +upstream contributions are these generic integration fixes, not the +AllAgents-specific descriptor schema. + +## Source authority and credentials + +The project `workspace.yaml` remains the source of truth for logical repository +names, origins, non-root destinations, and default revisions. Repository +execution names are explicit `name` values or the portable basename of `path`; +duplicate names or destinations, root destinations, and local/originless entries +make execution preflight fail. Its schema gains a strict `workspaceSnapshots` +catalog and environment-variable credential references; secret values remain +deployment-only. HarnessRouter owns harness/model/provider configuration. The +user workspace does not become a second source catalog, and no `gateway.yaml` or +caller-controlled registry is introduced. + +Repository mode acquires the complete declared repository set, with optional +revision overrides by logical name. It accepts only a bounded ref-name grammar, +rejects option-like or refspec-shaped values, resolves advertised refs to full +commits before agent execution, fetches by verified object ID, and records those +commits in provenance. + +Snapshot mode accepts only a configured OCI repository plus immutable manifest +and workspace-manifest digests. It verifies manifest, config, layer sizes and +digests, applies OCI whiteouts, validates the resulting declared workspace +layout, and records the ordered layer digests. + +Source credentials are selected server-side and exist only for the +materialization subprocess. The materializer must use hermetic Git/registry +configuration, prevent credentials from being persisted in Git configuration +or remote URLs, remove temporary credential state before returning, and emit no +secret value. The fork removes every configured materializer-only environment +name from every agent child independent of the variable's spelling. The agent +process receives neither the acquisition credential nor the acquisition +environment. + +## Trust and deployment + +HarnessRouter API authentication is mandatory even on a private network. +Operators should still bind it to loopback or a private network and enforce +Tailscale ACLs, firewall policy, or equivalent controls. Version one is not a +public multi-tenant service. + +HarnessRouter CE provides per-session operating-system identities and workspace +directories, not a hostile-code sandbox. Agent tools may access capabilities +available to their runner environment. Operators requiring stronger isolation +must place the complete HarnessRouter deployment inside an ephemeral VM or +equivalent boundary. + +The deployment uses a pinned custom HarnessRouter image containing: + +- the pinned HarnessRouter CE revision plus the reviewed patch series; +- the AllAgents materializer executable and its pinned runtime; +- the source-acquisition tools required by the accepted Git/OCI contract; and +- pinned HarnessRouter-supported Codex and optional Pi versions. + +`codex-lb` remains a separate service with proxy API-key authentication enabled. +The private deployment sets both `HARNESS_PUBLIC_BASE_URL` and the +runner-reachable `HARNESS_GATEWAY_URL` to the same gateway address; “public” here +means the base advertised across the private deployment, not Internet exposure. +Codex uses the `codex-lb` Responses endpoint with the provider identity and +OpenAI-auth capability fields required for `/responses` and +`/responses/compact`; readiness proves both plus a resumed turn through the +exact container network. Pi is advertised only if a release-gating probe proves +a separate HarnessRouter-supported custom format and `codex-lb` endpoint; +failure disables Pi rather than exposing a direct provider credential or adding +another proxy. + +## Failure behavior + +- **Invalid extension:** the AllAgents hook rejects it before acquisition. +- **Extension on a continuation:** reject without changing session state. +- **Unknown logical source or working directory:** fail before network access. +- **Source authentication or acquisition failure:** remove partial workspace + state, return a stable materializer failure, and start no agent or provider + fallback. +- **Materializer timeout or crash:** terminate the hook, remove partial source + state, return failure, and start no agent. +- **Agent cancellation or timeout:** use HarnessRouter's UHP lifecycle and + cancellation behavior. +- **HarnessRouter restart:** preserve completed state from the durable volume; + fail interrupted turns without automatic replay. +- **Provider failure:** return HarnessRouter's normalized UHP failure without + source fallback or provider-credential leakage. + +Failures report only verified provenance. Partial acquisition never appears as a +complete workspace identity. + +## Consequences + +This decision deletes substantial custom scope: + +- no A2A server or Agent Card; +- no HEC schemas, bindings, or conformance suite; +- no AllAgents Task/session SQLite store; +- no custom SSE lifecycle or cancellation protocol; +- no direct Codex SDK or Pi RPC adapters; +- no custom process supervisor or artifact store; +- no separate `allagents-gateway` npm product; and +- no Promptfoo-specific runtime in AllAgents. + +AllAgents instead owns the smaller differentiated surface: workspace selection, +deterministic Git/OCI materialization, source credentials, provenance, the +HarnessRouter integration patch, and deployment documentation. + +The cost is a temporary fork and custom image. The fork must be rebased and +tested against upstream releases until the generic hook is accepted or an +equivalent supported extension exists. + +## Alternatives rejected + +- **Custom A2A/HEC gateway:** duplicates mature UHP/HarnessRouter session, + streaming, cancellation, authentication, artifact, and provider behavior. +- **Thin adapter in front of stock HarnessRouter:** avoids a fork but introduces + another network service and makes source acquisition a client-side concern. +- **Put the descriptor in the prompt:** lets the model control acquisition and + is not deterministic or safe. +- **Expose acquisition as an MCP tool:** depends on the model choosing to call it + and runs too late to define the initial working directory. +- **Upload every source file as UHP input files:** works for small regular-file + snapshots, but loses exact symlink, mode, and OCI layer semantics and moves + repository acquisition to every caller. +- **Wait for upstream before delivery:** makes the product schedule depend on a + project we do not maintain. + +## Deliberate limits + +Version one does not add evaluation datasets, scoring, assertions, automatic +retries, session branching, concurrent turns within one session, caller-supplied +origins, public multi-tenancy, arbitrary materializer commands, mutable OCI +tags, transparent source-mode fallback, or guaranteed provider prompt-cache +hits. + +## Reconsider when + +Revisit this decision when: + +- upstream HarnessRouter accepts the generic materializer hook or exposes an + equivalent supported extension; +- the maintained patch grows beyond the narrow integration boundary; +- HarnessRouter changes or removes required UHP/session/provider behavior; +- exact per-turn workspace rollback becomes a product requirement; +- source acquisition must run in a stronger isolation boundary; +- callers require a public multi-tenant authorization model; or +- a second independent UHP implementation offers a materially smaller and more + stable integration surface. diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md deleted file mode 100644 index cebd114f..00000000 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ /dev/null @@ -1,316 +0,0 @@ -# ADR 0002: Serve coding-agent execution through an A2A gateway - -- Status: Accepted; implementation pending -- Date: 2026-09-17 -- Updated: 2026-09-21 - -## Decision - -AllAgents will provide a separately installed gateway that lets trusted tools -start a configured Codex or Pi run remotely and receive its output, usage, file -changes, artifacts, source provenance, and cleanup outcome through A2A. - -AI Evals is the first consumer. The gateway executes one coding-agent turn at a -time; it does not own datasets, scoring, assertions, scheduling, retries, or -durable evaluation records. - -Version one serves one AllAgents project workspace on one trusted Linux runner. -Network access controls who can use it, and the runner is the execution and -secret boundary. This is not a sandbox for hostile code or model-invoked tools. - -Implementation details live in the -[coding-agent execution gateway plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). -This ADR records the decisions and their impact. - -## What changes for users - -- **CLI users:** installing `allagents` does not install or start the gateway. -- **Operators:** install `allagents-gateway`, select one existing workspace, and - run `allagents-gateway serve`. Existing project and user workspace files remain - the source of truth. -- **Callers:** choose a configured target, declared workspace source, logical - working directory, `readOnly` or `readWrite` access, a bounded deadline, - optional result schema, and one-shot/start/resume session mode. They cannot - provide repository URLs, host paths, commands, credentials, or environment - overrides. -- **AI Evals:** owns the Promptfoo provider, turn chaining, and evaluation - behavior. AllAgents preserves provider/workspace session continuity and - reports native cached-input usage, but does not guarantee a model cache hit. - -## Main flow - -1. The operator starts the gateway for one project workspace. -2. A caller sends an A2A Message with one invocation key for one turn. It may run - one-shot, start a session, or resume the exact head Task in an existing - context. Reusing the key with the same canonical request returns the same - Task; each intentional new turn uses a new key and immutable Task. -3. The gateway reuses or prepares a workspace from declared Git repositories or - an immutable OCI snapshot. A resumed read-write session derives a per-turn - candidate from its last committed workspace generation. -4. Codex or Pi runs on the trusted host against a validated read-only base or the - private candidate. A session resumes the provider-native conversation. -5. The gateway streams progress, handles cancellation, and records observed - evidence. A successful continuing turn atomically advances the provider - checkpoint, committed workspace generation, and session head; a one-shot or - closing turn cleans up. Cleanup/checkpoint uncertainty, a live provider - process group, or an observed escaped/outliving descendant retains the - execution slot and makes the gateway unready. Process groups do not prove that - an unobserved hostile descendant cannot escape the trusted runner boundary. - -## Important consequences - -### Network reachability grants full access - -The gateway has no application login, caller identity, tenant isolation, or -per-caller privacy. Loopback is the default. Version one supports two enforceable -private topologies: - -- bind loopback HTTP and expose it only through a private HTTPS terminator such - as Tailscale Serve; or -- bind one specific loopback, RFC 1918, RFC 4193, link-local, or RFC 6598 - address and serve TLS itself from configured certificate/key files. - -Wildcard and public-address listeners are rejected. The advertised URL is -loopback HTTP for local development or private HTTPS for either remote topology. -Every reachable caller can invoke every available target, inspect every retained -Task and Artifact, resume every retained conversation, and request cancellation. -Operators must also enforce Tailscale ACLs, private firewall rules, or equivalent -network policy. Version one must not be exposed to the public Internet. - -Providers and model-invoked tools may use the runner's credentials, secrets, and -network access. Explicit environments reduce accidental leakage but do not -create isolation. Public-Internet exposure requires application authentication, -authorization, and an amended ADR before deployment. - -### Callers choose logical work, not infrastructure - -The gateway reuses existing workspace configuration; it does not add -`gateway.yaml` or `worker.yaml`. Built-in Codex and Pi targets are available when -ready. Profile-backed targets require explicit gateway enablement. - -A request chooses either the complete configured repository set, with optional -revision overrides by declared name, or one declared OCI snapshot selected by -immutable digests. It never falls back between those modes. The operator owns -origins, destinations, credentials, and registry policy. Credential routing -prefers a proven applicable GitHub App and uses the configured `gh` account only -when the App is absent or positively ineligible. Ambiguity or failure after -selection never falls back to a broader identity. - -The caller selects the workspace root or a directory beneath a declared -repository, never a physical host path. Invalid or escaping paths fail before -provider execution. - -### Read-only is an optimization, not a security boundary - -`readWrite` is the default. A one-shot Task gets a disposable workspace. A -read-write session keeps an immutable committed generation and runs each turn in -a candidate that becomes committed only with the provider checkpoint/session -head. `readOnly` uses a validated base: exact immutable requests may share one, -while mutable revisions get a Task/session-owned non-reusable base. Every -one-shot Task/session still gets private provider, temporary, and evidence state. - -Read-only enforcement is cooperative. A provider that writes anyway can -contaminate the shared workspace and later Tasks; the operator must then evict -that workspace before reuse. - -Workspace preparation is isolated from provider execution and receives only the -source credential and network access it needs. That credential and preparation -environment are gone before Codex or Pi starts. - -### Providers run directly on the trusted host - -Codex and Pi use supported, pinned integrations. Mutable provider/session state -lives in Task/session-private storage. Existing host authentication is reusable -only when the pinned integration can reference it separately without copying, -mounting, parsing, or writing conversation state into the auth location. -Otherwise that auth/target/mode combination is unavailable. The gateway does not -run workspace `setup` shell entries, download a provider runtime for each -request, execute generated launcher files remotely, or expose arbitrary -installed executables. It never silently weakens its advertised contract. - -### One gateway-controlled Task runs at a time - -One execution slot covers workspace preparation, one session turn, evidence, and -cleanup/checkpointing. A second valid request becomes a failed Task with -`execution_capacity_unavailable`; it starts no gateway-controlled workspace or -provider work. Sessions never run concurrent turns. - -This is an admission and supervision invariant, not hostile-process containment. -An observed escaped/outliving descendant retains the slot until verified gone or -runner teardown. An unobserved descendant can outlive a turn because version one -does not provide a non-bypassable process boundary. - -Task identity, retries, and visibility are shared across the gateway. Reusing an -invocation key for a different request conflicts. Each turn is a new immutable -Task; the session persists conversation and committed workspace state between -turns. - -Terminal Tasks and evidence are retained within configured limits. Unexpired -Tasks are not deleted to make room for new work. Prompts, output, structured -results, native evidence, and produced Artifacts remain sensitive and -unredacted. -Operational logs exclude credential values, secret-bearing paths, and -unrestricted prompt, output, tool, source, and file content. - -### The Harness Execution Contract owns semantics; A2A is the first binding - -The caller may be an evaluation runner, chat platform, application, or another -agent. The shared domain is therefore not agent-to-agent collaboration or one -vendor's HTTP shape; it is configured harness execution. - -AllAgents defines a transport-neutral, versioned Harness Execution Contract (HEC). -Its Core conformance class covers one turn: configured target selection, -idempotency, deadlines, ordered progress, normalized tool-call/result trajectory, -cancellation, terminal result, usage, stable failures, and artifacts. The -version-one Sessions extension adds a durable session identity, ordered turns, -provider-native conversation resumption, a retained workspace, expiry, and -close-after-turn cleanup. One-shot Core invocation remains available when a -caller does not request a session. - -Protocol bindings map that contract onto an existing transport without changing -its semantics. Core owns neutral outcomes such as `timedOut` and the cancel -dispositions `accepted | alreadyTerminal`; each binding maps those outcomes to -its own state and response vocabulary. Every binding must pass the same Core -semantic vectors plus its own wire-conformance cases. A2A 1.0 over HTTP+JSON is -the first binding: discovery uses an Agent Card, one turn becomes one Message -and one immutable Task, ordered execution events become Task updates and -Artifacts, and A2A owns streaming, retrieval, cancellation, and transport -errors. A2A `contextId` identifies the durable session; a resumed turn uses the -same context and references the prior terminal Task. An A2A Client may be an -application or an agent; neither side needs autonomous multi-agent behavior. - -The AllAgents coding-workspace extension remains separate from Core and -Sessions. It defines configured Git/OCI sources, logical working directories, -access mode, provenance, produced files, integrity, and cleanup. A session pins -those inputs and retains its private provider state and workspace until -close-after-turn, expiry, or verified operator cleanup. Version one uses the -non-dereferenceable Profile Extension identifier -`urn:allagents:a2a:profile:coding-execution:v1`, which composes the HEC A2A -binding, Sessions, and the coding-workspace extension. This URN identifies a -contract; it is not a network endpoint. Their schemas and conformance groups -remain independently validatable. Breaking changes use a new URN rather than -silent fallback. - -This is a contract and standard candidate, not a claimed neutral standard. -AllAgents should describe it as a standard only after independent -implementations, multiple bindings, executable cross-binding conformance, and -neutral governance exist. - -The known A2A-binding conformance question is authentication. A2A 1.0 says -servers authenticate requests and authorization-scope Task operations, while -this design has no application identity and treats every reachable caller as -one authority domain. Agent Card security declarations are optional, so the -anonymous case is not explicit. The implementation feasibility gate must -resolve this before the gateway claims full A2A 1.0 conformance. If it cannot, -this ADR must be amended; the implementation must not silently add -authentication or weaken the conformance claim. - -### UHP informs a future Responses binding but is not the version-one wire - -The Unified Harness Protocol (UHP) is a close semantic match for -application-to-harness execution. Its Responses-compatible request shape, -ordered tool-call/result output, durable session with conversation and working -directory continuity, exact `previous_response_id` predecessor chaining, -lifecycle vocabulary, and executable conformance suite are design inputs for -Core, Sessions, and a possible future Responses/UHP binding. - -UHP is not adopted as the version-one wire because its conformant core also -assumes application authentication, principal scoping, and concurrent work -across sessions. Those are appropriate for a hosted, multi-user harness router -but conflict with this gateway's trusted-private-network and one-execution-slot -boundary. UHP also leaves the AllAgents-specific Git/OCI acquisition and -integrity evidence contract to an extension, so it would not eliminate the -domain contract this project must own. - -Version one exposes only the A2A binding. The implementation may reuse UHP -semantics, not UHP wire claims. It must not advertise UHP compatibility without -implementing a defined binding and passing the applicable UHP and -Harness Execution Contract conformance suites. - -### The gateway remains a separate product - -`allagents` and `allagents-gateway` have independent versions and release -cadence. Installing the CLI fetches neither the gateway nor its workspace- -preparation image. Compatibility comes from versioned contracts, not matching -package versions. - -Every terminal Task has exactly one versioned Core outcome Artifact, one ordered -normalized Core execution-trajectory Artifact, one AllAgents workspace- -integrity Artifact, and any produced Artifacts, even after failure or -cancellation. Consumers remain responsible for evaluation workflows, retries, -retention, sharing, and redaction. - -## Failure behavior - -- **Busy:** the new Task fails with `execution_capacity_unavailable`; no work - starts. -- **Malformed source or working-directory input:** admission fails with - `invalid_execution_request`; no Task is created. -- **Accepted source, credential, or logical-directory resolution then fails:** - the Task fails with the corresponding stable error code and does not switch - source mode or credential identity. -- **Gateway restart:** interrupted Tasks fail and are never replayed. A retained - session remains resumable only after the gateway confirms the recorded process - group and observed descendants are gone, discards any uncommitted workspace - candidate, and verifies the prior provider/workspace checkpoint pair; - otherwise it becomes non-resumable pending cleanup. -- **Provider cannot be stopped:** the Task reports - `execution_termination_failed` with live-provider and observed termination - evidence, but no filesystem, Git, or produced-Artifact evidence. A live process - group or observed outliving descendant retains its workspace/execution slot - and leaves the gateway unready until verified disappearance or runner teardown. -- **Workspace cleanup fails:** the Task reports `workspace_cleanup_failed` and - retains enough state for later cleanup. -- **Durable gateway state is unsafe:** the gateway stops admitting work and does - not acknowledge creation or report success it cannot preserve. - -Evidence describes only what the gateway observed. It never presents uncertain -termination, cleanup, provenance, or file state as verified. - -## Deliberate limits - -Version one deliberately avoids: - -- application authentication, tenants, caller-private Tasks, and public-Internet - exposure because version one is restricted to one trusted private network; -- queues, concurrent admission, replicas, remote workers, and shared provider - daemons because the gateway supervises one admitted turn at a time; -- caller-provided origins, physical host paths, configured destinations, - commands, credentials, environments, or materializers because the gateway is - not a remote shell; -- hostile-code containment and per-provider containers because the runner is - already the execution/secret boundary. An unobserved escaped descendant can - overlap a later admitted turn; operators needing OS-wide exclusivity must use - an ephemeral runner boundary or wait for a future containment design; -- a bespoke or evaluator-specific API because HEC owns harness semantics and A2A - already owns the version-one remote Task and multi-turn context lifecycle; -- a second wire binding because version one proves Core and Sessions through - A2A before adding Responses/UHP; -- a second configuration registry because workspace files already own sources - and profiles; and -- bundling or version-locking the gateway with the CLI because most CLI users do - not need the service and the products have different release cadence. - -## Reconsider when - -Revisit this decision when: - -- callers outside one trusted private network must share the service, which - requires application authentication and authorization before exposure; -- provider or model-tool code must be isolated from runner secrets; -- OS-wide turn exclusivity is required, which needs a non-bypassable provider - lifecycle boundary before the gateway may admit a later turn; -- the gateway needs concurrency, replicas, shared daemons, or remote workers; -- non-Linux runners, new source materializers, or richer credential routing are - required; -- another independent implementation needs the Harness Execution Contract or a - second binding, at which point both must pass the shared Core conformance - vectors without changing Core semantics; -- session branching or concurrent turns, which require explicit fork semantics - beyond the version-one linear session history; -- A2A standardizes equivalent portable harness-execution semantics that should - replace or upstream the AllAgents binding; -- UHP gains neutral multi-vendor governance, several independent conformant - implementations, and a one-shot conformance class suitable for a - Responses/UHP binding; or -- a stable cross-vendor protocol replaces the Codex/Pi adapter seam. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index ee7dbf30..600d5e67 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,5 +1,5 @@ --- -title: "Coding-Agent Execution Gateway - Plan" +title: "UHP Coding-Agent Execution through HarnessRouter - Plan" date: 2026-09-18 updated: 2026-09-21 type: feat @@ -9,60 +9,47 @@ product_contract_source: ce-plan-bootstrap execution: code --- -# Coding-Agent Execution Gateway - Plan +# UHP Coding-Agent Execution through HarnessRouter - Plan ## Goal Capsule -- **Objective:** A developer can install and run one trusted-network A2A - endpoint for one AllAgents workspace and invoke built-in or explicitly - gateway-enabled profile targets against either the complete configured Git - repository set, with optional named revision overrides, or a digest-pinned - OCI workspace snapshot. AI Evals can configure either source mode in - Promptfoo YAML through a custom provider without sending origins. -- **Means:** Convert the repository to a private Bun workspace monorepo with - independently released `allagents` and `allagents-gateway` applications, - versioned workspace/execution/acquisition contract packages, generated - portable fixtures under `contracts/`, a bounded SQLite Task/session store, - direct Codex and Pi host-process adapters, and one digest-pinned acquisition - image used only when no reusable validated base exists. One-shot read-only - Tasks share a reusable base or own a non-reusable base for mutable revisions; - sessions pin a base and retain private runtime/workspace state; one-shot - read-write Tasks receive disposable writable views through an automatic - block-clone/OverlayFS materializer with an explicit portable copy backend. - Providers still run bare metal on the trusted Linux CI runner. -- **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) - owns the wire, trust, runtime, and packaging boundaries. Project and user - `workspace.yaml` files own source and profile declarations. The - transport-neutral Harness Execution Contract (HEC) Core and Sessions own - deterministic per-turn and continuation semantics; their A2A binding owns the - version-one wire mapping; the AllAgents coding-workspace extension owns - Git/OCI workspace semantics. - The CI job, VM, or deployment container is the only operational execution - and isolation boundary. AllAgents does not claim that boundary contains - hostile code or hides job secrets from model-invoked tools; it owns process - lifecycle and truthful evidence only. -- **Execution order:** Capture red CLI-only and standalone-gateway package - smokes; complete the Bun monorepo, A2A SDK, provider-surface, process-group, - Docker-acquirer, package, and release feasibility gate; freeze schemas, - generated fixtures, configuration projection, SQLite ownership, and release - binding; implement the Task/session store and A2A server, host supervisor, - Docker-only acquisition, Codex, and Pi; run final review; then run green packed- - package, exact-image, registry-conformance, repository, and documentation - gates. -- **Stop conditions:** Stop dependent production work if the pinned A2A surface - cannot implement the required private-network protocol or if the acquisition- - container boundary and exact image/package release binding are infeasible. - A missing Codex or Pi capability makes that target unavailable rather than - changing the A2A, trust, source, or Task/session contracts. Do not add - application authentication, deployment YAML, remote workers, caller-supplied - origins/commands, mutable OCI - tags, provider fallback, evaluation behavior, automatic retries, per-provider - Docker, or containment claims. -- **Tail ownership:** The implementing workflow runs focused contract, - lifecycle, provider-environment, process-group, acquisition-boundary, and - release-binding tests; repository quality gates; packed CLI/gateway and exact - acquisition-image smoke tests; registry conformance; and documentation - validation. +- **Objective:** Let Promptfoo and other authenticated UHP clients run Codex + against a configured AllAgents Git workspace or immutable OCI workspace + snapshot, continue the same conversation and writable workspace with + `previous_response_id`, and receive source provenance, output, usage, and + artifacts. Pi is optional and advertised only after its `codex-lb` route + passes a release-gating probe. +- **Means:** Deploy a pinned HarnessRouter CE fork. Preserve HarnessRouter's UHP, + authentication, session, streaming, cancellation, artifact, and agent-runner + behavior. Add a generic first-turn materializer boundary, nested working + directory support, durable materialization state, and nested-repository + checkpoint/collection support. Implement Git/OCI semantics in a separate + AllAgents executable. Route Codex through HarnessRouter's credential broker to + `codex-lb`; keep the long-lived `codex-lb` key in the gateway and provider OAuth + in `codex-lb`. +- **Authority:** [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) + owns the protocol, fork, trust, workspace, and provider-routing decisions. + UHP `2026-09-12` and HarnessRouter's conformance suite own execution-wire + behavior. The project `workspace.yaml` owns logical Git/OCI sources and + environment-variable credential references; deployment secrets provide the + values; HarnessRouter configuration owns harness/model/provider targets. The + namespaced AllAgents extension owns acquisition and provenance semantics. +- **Execution order:** Prove the fork seam, brokered Codex route, materialization + state machine, checkpoint integration, and optional Pi route against a real + HarnessRouter runner; freeze the generic hook and AllAgents contracts; + implement Git then OCI acquisition; run Promptfoo one-shot and continuation + E2E; complete release, fork-maintenance, and upstream-ready documentation. +- **Stop conditions:** Stop before production implementation if materialization + cannot complete and durably checkpoint before provider dispatch, if nested Git + workspaces cannot be collected without corrupting HarnessRouter checkpoints, + if either source credentials or the long-lived `codex-lb` key reach the agent, + or if the fork cannot preserve stock UHP behavior and conformance. Do not fall + back to prompt instructions, an MCP acquisition tool, client-side repository + upload, A2A/HEC, owner-trust provider keys, or a new task/session engine. +- **Tail ownership:** Implementation owns focused tests in both repositories, + upstream UHP conformance, built-image smoke tests, exact Git/OCI E2E, + two-turn Promptfoo success and failure verification, credential leak checks, + documentation, and a clean upstreamable HarnessRouter patch series. --- @@ -70,2950 +57,777 @@ execution: code ### Summary -AllAgents gains a single-workspace coding-execution service without becoming an -evaluation framework or multi-tenant platform. A transport-neutral Harness -Execution Contract defines one turn's invocation, ordered tool trajectory, -results, usage, cancellation, failures, and artifacts plus durable resumable -Sessions for evaluation runners, chat platforms, applications, and other agents. -The gateway exposes its first binding through A2A Tasks and one required -AllAgents coding-execution Profile Extension. That profile composes the A2A -binding, Sessions, and the AllAgents-specific coding-workspace extension while -preserving separate schemas and conformance groups. -Network reachability is authorization. The service resolves configured targets -and sources from existing workspace files, acquires or reuses an immutable base, -shares it for read-only Tasks, creates an independent disposable view for -read-write Tasks, invokes Codex or Pi through a typed adapter, and retains -bounded terminal evidence. AI Evals consumes that boundary through its own -Promptfoo custom provider: evaluation YAML supplies named revision overrides -for the configured repository set, or one snapshot handle and immutable -digests, while AllAgents retains origin and credential authority. +AllAgents uses HarnessRouter as the execution gateway instead of building one. +HarnessRouter exposes UHP, authenticates callers, creates and persists sessions, +streams events, runs Codex and capability-gated Pi, handles cancellation and +idempotency, and returns output, usage, and artifacts. + +The missing product-specific capability is deterministic workspace acquisition +before the first agent turn. A focused HarnessRouter fork calls a generic +materializer boundary after allocating the session workspace but before provider +selection. The AllAgents executable validates the product-specific descriptor, +writes and validates staging, and returns path-free provenance. The runner +publishes staging, initializes HarnessRouter and nested-repository checkpoints, +persists a pre-agent checkpoint, and only then permits provider dispatch. + +A continuation supplies `previous_response_id`, omits the workspace extension, +and uses HarnessRouter's current native conversation and writable session +workspace. A different revision, snapshot, or working directory requires a new +session. AllAgents does not add stricter predecessor-head or branching semantics +beyond HarnessRouter's UHP behavior. ### Problem Frame -AllAgents configures and launches coding clients but has no service boundary for -trusted tools such as AI Evals. Those tools would otherwise import AllAgents -internals, drive interactive CLIs, or duplicate profile resolution, repository -acquisition, credential handling, cancellation, evidence capture, and cleanup. +Stock HarnessRouter already implements the expensive generic execution concerns. +Rebuilding those concerns behind A2A would add a second protocol, lifecycle, +session store, process supervisor, artifact model, provider integration, and +conformance burden without differentiating AllAgents. -Developers expect a process they can start in a workspace and expose on -loopback or a trusted private network such as Tailscale. Remote access uses -either a loopback backend behind a private TLS terminator or one specific private -TLS listener; wildcard/public listeners are rejected. Version one is never a -public-Internet service. They do not need an application -authentication stack, Kubernetes control plane, remote worker registry, or -another profile configuration file for the initial use case. +Stock HarnessRouter does not expose a documented generic pre-turn seam for a +server-side Git/OCI descriptor. It does not copy arbitrary request metadata into +response metadata or forward it to ordinary Codex/Pi runners. Input files are +written before the agent and support nested paths, but client-side expansion +loses exact symlink, mode, Git-history, and OCI layer semantics and makes every +caller responsible for acquisition. The temporary fork closes those seams. ### Actors -- A1. **Trusted-network caller:** An evaluation runner, chat platform, - application, or agent able to reach the endpoint. All callers have the same - authority and Task visibility. The first caller is an AI Evals-owned - Promptfoo custom provider that maps one `callApi` to one execution. -- A2. **Execution gateway:** The A2A binding and invocation supervisor. It owns - deployment-wide Task identity, acquisition, routing, status, cancellation, - evidence, retention, and cleanup. -- A3. **Backend adapter:** The Codex or Pi implementation translating native - automation events and cancellation into the common contract. -- A4. **Operator/developer:** The person who selects the project workspace, - gateway-enables profile launchers, supplies process flags and credential - handles, and controls network access. -- A5. **GitHub/OCI source:** The remote content service used only during the - acquisition phase. +- **A1. UHP caller:** Promptfoo or another application holding a HarnessRouter + API key. It chooses a configured HarnessRouter harness ID and model, prompt, + initial workspace descriptor, and optional continuation predecessor. +- **A2. HarnessRouter gateway:** Authenticates and validates UHP, owns response + and session identity, treats the configured workspace metadata value as + bounded opaque JSON, drives materialization before provider fallback, and + returns hook metadata on every response path. +- **A3. AllAgents materializer:** A non-network executable invoked before the + first turn. It validates the AllAgents descriptor, reads the mounted project + workspace configuration, acquires Git or OCI sources into staging, validates + the tree, and returns provenance. +- **A4. HarnessRouter runner:** Owns the per-session operating-system identity, + publication, checkpoints, nested-repository collection, input files, safe + nested cwd, and Codex/Pi process. +- **A5. `codex-lb`:** Exposes the Codex Responses endpoint, accepts the + gateway-held API key through HarnessRouter's broker, owns provider OAuth and + account routing, and preserves continuation affinity. +- **A6. Operator:** Pins and deploys the custom image, mounts durable data and + project workspace configuration, supplies deployment-only source credential + values through the allowlisted environment, and controls private-network access. ### Key Decisions -- **Define one harness contract with A2A as its first binding.** The - transport-neutral Harness Execution Contract Core owns one turn's invocation, - idempotency, ordered progress and tool trajectory, cancellation, result, - usage, stable failures, and artifacts. The version-one Sessions extension owns - durable linear conversation identity, provider-native resumption, retained - workspace state, expiry, and close-after-turn cleanup. Standard A2A Agent - Cards, Messages, Tasks, context IDs, task references, Artifacts, errors, - streaming, and cancellation carry those semantics. An evaluation runner, chat - platform, application, or agent uses the same contracts; Promptfoo does not - masquerade as an autonomous agent. Governs R1-R3. -- **Treat the private network as the trust boundary.** The initial service has - no application authentication or caller ownership. Remote access requires a - loopback backend behind private TLS ingress or a specific private-address TLS - listener; wildcard/public listeners and public exposure are prohibited. - Governs R4-R5. -- **Reuse workspace configuration.** Project `workspace.yaml` owns sources; - user `workspace.yaml` owns profiles, launchers, and gateway enablement. There - is no `gateway.yaml`. (session-settled: user-directed.) Governs R6-R8, R18. -- **Support two acquisition modes.** Direct declared repositories and named, - digest-pinned OCI workspace snapshots converge on one manifest and evidence - contract. (session-settled: user-directed.) Governs R9-R11. -- **Share immutable bases; isolate writes.** `workspaceAccess` defaults to - `readWrite`. One-shot read-only Tasks and read-only sessions may reuse one - validated physical base; each receives private runtime state. One-shot - read-write Tasks receive disposable writable views. A read-write session - retains an immutable committed generation and runs each turn in a private - candidate that becomes the next generation only at atomic settlement. - Reflink/block clone is preferred, rootless OverlayFS is the Linux fallback, - and an explicit copy backend preserves portability. Governs R2-R3, R5, - R8-R11, R15-R16, R18-R19. -- **Use App-first GitHub credential eligibility.** Prefer an applicable GitHub - App; use a configured `gh` account only when no App installation applies; - never fall back after selected-App failure. (session-settled: user-directed.) - Governs R12. -- **Keep a narrow typed backend seam.** Pinned supported Codex and Pi package or - RPC surfaces are the complete initial backend set. Launcher-backed profiles - resolve through AllAgents-owned adapters and execute on the gateway host - rather than through generated wrapper files or the acquisition container. - Governs R7-R8, R13-R15. -- **Persist Task truth and resumable session checkpoints.** Restart settles an - interrupted turn failed and never replays it. An idle session resumes only - from its last committed provider/workspace checkpoint; an uncertain checkpoint - is non-resumable pending cleanup. Governs R3, R5, R8, R13-R16. -- **Keep evaluation outside AllAgents.** Consumers own datasets, repetitions, - scoring, assertions, and evaluation Runs. Governs R17. -- **Bridge Promptfoo at the consumer boundary.** AI Evals owns a custom provider - that maps Promptfoo YAML and test variables to closed A2A source/session modes - and maps terminal Tasks plus session/cache metadata back to - `ProviderResponse`. AllAgents owns no Promptfoo runtime behavior. Governs R19. +- **Use UHP, not A2A or HEC.** UHP `2026-09-12` is the only northbound execution + contract. HarnessRouter conformance is authoritative. +- **Fork narrowly and upstream later.** Delivery uses an AllAgents-maintained + fork. The upstreamable layer is a configured opaque-metadata key, immutable + first-turn binding, typed command envelope, durable pre-provider lifecycle, + safe nested cwd, and checkpoint/collection integration. It contains no + AllAgents Git/OCI schema logic. Upstream acceptance is not critical-path. +- **Run the AllAgents component behind HarnessRouter.** The materializer is a + subprocess hook, not another HTTP gateway and not a custom agent backend. +- **Materialize once per extension-bearing session.** An extension-bearing + initial request creates and checkpoints the workspace. Continuations must omit + the extension and reuse the session through `previous_response_id`. +- **Keep source authority server-side.** Callers select logical source names and + revisions but cannot send origins, credentials, host paths, commands, or + Docker options. +- **Broker provider access.** Promptfoo uses a HarnessRouter API key. The gateway + keeps the `codex-lb` key and mints an invocation-scoped broker credential for + the agent. Only `codex-lb` handles provider OAuth. +- **Preserve stock UHP requests.** Requests without the configured metadata key + behave exactly as upstream. +- **Use a custom image, not an `allagents-gateway` package.** The image combines a + pinned HarnessRouter revision, reviewed patch series, pinned agent runtimes, + and the AllAgents materializer executable. ### Requirements -**Private-network protocol** - -- R1. Implement A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, - `GetTask`, `ListTasks`, `CancelTask`, streaming send, and active Task - subscription. Every A2A request carries `A2A-Version: 1.0`; another version - receives `VersionNotSupportedError`. The Agent Card advertises exactly one - absolute private interface URL with `protocolBinding: "HTTP+JSON"`, - `protocolVersion: "1.0"`, and `capabilities.streaming: true`. It declares the - coding-execution Profile Extension identifier - `urn:allagents:a2a:profile:coding-execution:v1` with `required: true`. Strict - `params` contains a sorted `targets` array whose entries have - `id: TargetId` and - `sessionModes: ("oneShot" | "start" | "resume")[]`. `oneShot` is always - present; `start` and `resume` are advertised together only when the target's - adapter supports native durable continuation. Targets and modes outside these - entries fail before admission. The Agent Card uses - `defaultInputModes: ["text/plain"]`, `defaultOutputModes: ["text/plain", - "application/json", "application/octet-stream"]`, and exactly one stable - skill: `id: "coding-execution"`, `name: "Coding execution"`, a - profile-defined description, tags `["coding", "harness-execution"]`, and an - example that produces a profile-shaped Task. Extension params advertise ready - targets and their usable continuation modes only. - V1 callers obtain configured logical repository and snapshot selectors - from their workspace or consumer configuration; the Agent Card does not - publish origins, credentials, destinations, physical paths, or snapshot - selection state. - - Remote interface URLs use HTTPS and resolve only to loopback, RFC 1918, - RFC 4193, link-local, or RFC 6598 addresses such as Tailscale's - `100.64.0.0/10`; direct HTTP is limited to a loopback listener. Startup rejects - wildcard/public listener addresses, public URL literals, advertised hostnames - with any public address, and a private direct listener without TLS key/cert. - Every operation that creates, returns, lists, subscribes to, or - mutates profiled Tasks or Artifacts includes the Profile URN in - `A2A-Extensions`; missing activation receives - `ExtensionSupportRequiredError`. - - Check the transport-neutral Harness Execution Contract Core and Sessions - specifications into `docs/contracts/harness-execution-v1.md` with identifiers - `urn:allagents:harness-execution:core:v1` and - `urn:allagents:harness-execution:sessions:v1`. Core defines one turn's - invocation, target selection, idempotency, deadline, ordered progress and - normalized tool-call/result trajectory, cancellation, terminal result, usage, - portable failures, and artifacts without importing A2A or UHP types. Sessions - defines linear continuation and durable provider/workspace checkpoints without - importing A2A types. The required Profile URN identifies the normative A2A - binding plus Sessions and the AllAgents coding-workspace extension, not an - undocumented metadata convention. These URNs are non-dereferenceable contract - identifiers, never gateway or public documentation endpoints. The binding - defines permitted A2A values, one-Message/one-Task-per-turn constraints, - schemas, context/task-reference mapping, state and event mapping, errors, - examples, versioning, and executable conformance cases. V1 uses one required - A2A Profile URN because this gateway requires Core, Sessions capability, and - the workspace extension; it exposes no second wire protocol. - - Honor both `SendMessageConfiguration.returnImmediately` modes. `ListTasks` - implements every standard filter, cursor pagination, `pageSize` 1-100 with a - default no greater than 50, descending status-timestamp order, and required - `tasks`, `nextPageToken`, `pageSize`, and `totalSize` fields. - `nextPageToken` is present and empty on the final page. With the default - `includeArtifacts: false`, each returned Task omits `artifacts`; `true` - includes the field. A response never exceeds the configured serialized-byte - limit. `ListTasks` may return fewer Tasks than `pageSize` when the next whole - Task would cross that limit; it never splits a Task, and its cursor resumes at - the first omitted Task. The validated per-Task/response invariant guarantees - one complete Task fits `GetTask` and a nonempty list page. -- R2. Define the exact transport-neutral `HarnessInvocationV1` DTO as - `{ version: "1", invocationKey, target, prompt, deadlineSeconds, - resultSchema? }`. The prompt is part of Core even though a binding may carry it - outside its control object. Define the separate Sessions input as exactly one - of `{ mode: "oneShot" }`, `{ mode: "start", closeAfterTurn }`, or - `{ mode: "resume", sessionId, previousTaskId, closeAfterTurn }`; both booleans - default to false. Its terminal turn projection is exactly one of - `{ mode: "oneShot", state: "none" }`, - `{ mode: "start" | "resume", sessionId, state: "idle", headTaskId }`, - `{ mode: "start" | "resume", sessionId, state: "closed" }`, or - `{ mode: "start" | "resume", sessionId, state: "notResumable" }`. An idle - `headTaskId` is the just-terminal Task and the only Task valid for the next - resume. Provider-checkpoint/workspace generation may independently remain the - prior committed pair after a safely failed or canceled turn. - Define the separate AllAgents workspace input as - `{ source, workingDirectory, workspaceAccess }`. Core imports neither - Sessions nor workspace types. - - The A2A binding projects exactly one Message text Part to Core `prompt`; maps - the remaining Core fields, Sessions `mode` as `sessionMode`, - `closeAfterTurn`, and the workspace fields to the flat object at - `Message.metadata[profileUri]`; and maps Sessions `sessionId` to - `Message.contextId` plus `previousTaskId` to the sole - `Message.referenceTaskIds` member. Each contract module owns a disjoint named - property set and its module-local required members but does not close the - shared metadata object. On a terminal Task, the binding writes the exact - Sessions terminal projection to strict `Task.metadata[profileUri].session`; - it is absent from nonterminal Tasks and events. `sessionId` equals the Task - context ID, and any idle `headTaskId` names the exact terminal Task accepted - for the next resume. - The composed A2A schema alone declares the complete - flat property set, unions required members, materializes cross-module - defaults, and sets `additionalProperties: false`; duplicate property ownership - or incompatible constraints fail generation. Strict nested objects reject - every unlisted member. V1 uses these wire scalars: - - `InvocationKey` matches `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`. - - `ContextId` is NFC UTF-8 of 1-256 bytes with no U+0000-U+001F or U+007F; - gateway-generated session IDs are lowercase canonical UUIDv7 values. - - `TaskId` is the gateway-generated lowercase canonical UUIDv7 Task ID. - - `ConfigName` and `TargetId` match - `^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`. - - `RevisionText` is NFC UTF-8, 1-255 bytes, with no U+0000-U+001F or U+007F. - - `Digest` matches `^sha256:[0-9a-f]{64}$`. - - `RelativeDirectory` is NFC UTF-8 of 1-1024 bytes containing 1-32 - slash-separated segments. Each segment is 1-255 bytes, is neither `.` nor - `..`, and contains no slash, backslash, U+0000-U+001F, or U+007F. - The A2A profile metadata object is exactly: - `version: "1"`; `invocationKey: InvocationKey`; `target: TargetId`; - optional `sessionMode: "oneShot" | "start" | "resume"` defaulting to - `"oneShot"`; optional `closeAfterTurn: boolean` defaulting to false and - forbidden for `oneShot`; `source`, one of `{ kind: "repositories", - revisions?: Record }` or +#### UHP, authentication, and routing + +- **R1.** Pin HarnessRouter CE to a reviewed upstream commit and UHP version + `2026-09-12`. The deployment must pass the applicable upstream conformance + suite without weakening, replacing, or reinterpreting stock UHP behavior. +- **R2.** Require a HarnessRouter API key for every execution endpoint. Bind the + service to loopback or a private interface and document the remaining need for + Tailscale ACLs, firewall policy, or equivalent network controls. +- **R3.** HarnessRouter deployment configuration owns stable harness IDs and + their backend, model allowlist, and provider integration. Requests select a + harness with stock `metadata.harness_id` and a model with `model`; AllAgents + profiles are not projected into this catalog. Codex uses a custom + `api_format: "responses"` integration pointed at the `codex-lb` + `/backend-api/codex` base. Its generated provider block must use + `name = "openai"` and `requires_openai_auth = true`, including remote + compaction. Pi is advertised only when a separate real probe proves a + HarnessRouter-supported Pi custom format against a documented `codex-lb` + endpoint; otherwise Pi is unavailable without fallback. +- **R4.** Set `HR_SANDBOX_TRUST` to a non-owner broker mode and configure both + `HARNESS_PUBLIC_BASE_URL` and runner-reachable `HARNESS_GATEWAY_URL` to the same + private gateway address so stock broker eligibility and runner routing agree. + The gateway holds the long-lived `codex-lb` key; the runner receives only a + turn credential and broker URL. Preserve HarnessRouter streaming, + cancellation, idempotency, files, artifacts, usage, errors, completed-session + persistence, and per-session workspace/UID isolation. Do not duplicate those + capabilities in AllAgents. + +#### Workspace extension and hook + +- **R5.** On an initial response request, accept one optional JSON object of at + most 64 KiB and 32 levels at `metadata["allagents.workspace"]`. + HarnessRouter checks only those generic bounds, canonicalizes the opaque value + with RFC 8785, and binds its digest to the new session. A continuation must + omit this key. The AllAgents hook validates the exact v1 object + `{ version: "1", source, workingDirectory? }`; `source` is exactly + `{ kind: "repositories", revisions?: Record }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, - workspaceManifestDigest: Digest }`; optional `workingDirectory`, one of - `{ kind: "workspaceRoot" }` or `{ kind: "repository", - repository: ConfigName, path?: RelativeDirectory }`, defaulting to - `{ kind: "workspaceRoot" }`; optional `workspaceAccess`, one of - `"readOnly" | "readWrite"`, defaulting to `"readWrite"`; optional - `deadlineSeconds` (integer 1-3600, default 1800); and optional - `resultSchema: { version: "1", schema: SchemaNode }`. - - A `SchemaNode` is exactly one branch below. `description` is optional NFC - UTF-8 of at most 1024 bytes. Scalar `enum` arrays contain 1-128 canonically - distinct values of the node's type; string enum values are at most 4096 UTF-8 - bytes, numbers are finite, integers are JSON safe integers, and null permits - only `[null]`. - - null or boolean: `{ type, description?, enum? }`; - - string: `{ type: "string", description?, enum?, minLength?, maxLength? }`, - where lengths are integers 0-1,048,576 Unicode scalar values and minimum - does not exceed maximum; - - number or integer: - `{ type, description?, enum?, minimum?, maximum? }`, where number bounds - are finite, integer bounds are JSON safe integers, and minimum does not - exceed maximum; - - array: - `{ type: "array", description?, items: SchemaNode, minItems?, maxItems? }`, - where item bounds are integers 0-4096 and minimum does not exceed maximum; - - object: - `{ type: "object", description?, properties, required?, - additionalProperties: false, minProperties?, maxProperties? }`, where - `properties` is a strict record of 0-256 `ConfigName` keys, - `required` is a unique subset of those keys, property bounds are integers - 0-256, and minimum does not exceed maximum. - No pattern dialect exists in v1. References, unions/combinators, conditionals, - formats, defaults, coercion, non-finite numbers, duplicate canonical enum - values, and unknown keywords are rejected. The canonical result schema is at - most 64 KiB, 256 nodes, and 32 levels deep. Repository revision count cannot exceed declared repositories. - The Message contains exactly one `Part` with `text` set to a UTF-8 Core - `prompt` of 1 byte to 1 MiB; other Part content fields are rejected. Only the - profile-owned metadata object is strict; unrelated A2A metadata and other - activated-extension keys are preserved or ignored according to A2A. - `oneShot` and `start` forbid client `contextId` and `referenceTaskIds`; - `resume` requires a `ContextId` and exactly one `TaskId` reference. - Canonicalization materializes defaults, normalizes profile strings to UTF-8 - NFC, sorts record keys, and hashes the Core Invocation excluding - `invocationKey`, the Sessions and workspace inputs, and the complete A2A - session projection: mode, close-after-turn value, supplied context - presence/value, and ordered task references. Generated context IDs are not - hashed. Do not add `Task.extensions` or backend-specific public fields. - - A client generates an opaque invocation key with at least 128 bits of - randomness once per logical turn, durably reuses that key and identical - canonical request after an ambiguous transport failure, and creates a new key - only for an intentionally new turn. A replay with any changed session - projection conflicts. A2A `messageId` remains Message identity and does not - replace the profile idempotency key. -- R3. One valid new turn creates one addressable immutable Task. `oneShot` and - `start` atomically generate and persist a lowercase canonical UUIDv7 context - ID with the absent-context invocation claim; only `start` creates a durable - session under that ID. Replays return the persisted generated value. `resume` - requires an idle, unexpired, resumable session at `Message.contextId`, the - session's exact current head Task as its sole `referenceTaskIds` member, and - the same target, canonical source, logical cwd, access mode, provider identity, - and configuration digest. A stale head, changed pinned input, active turn, - expired session, or poisoned checkpoint fails respectively with - `session_head_mismatch`, `session_configuration_mismatch`, `session_busy`, - `session_expired`, or `session_not_resumable`, and creates no Task. - - Every session turn and event uses the session context ID. Terminal Tasks remain - immutable; continuation always creates a new Task in the same context. V1 - serializes a linear session history and rejects branching or concurrent turns. - - HEC Core defines seven execution states: `submitted`, `working`, `completed`, - `failed`, `timedOut`, `canceled`, and `rejected`. A cancellation command - returns the neutral disposition `accepted | alreadyTerminal`; the latter never - changes the outcome. The A2A binding maps `submitted` and `working` to - `TASK_STATE_SUBMITTED` and `TASK_STATE_WORKING`; `completed`, `canceled`, and - `rejected` to their same-named A2A states; and both `failed` and `timedOut` to - `TASK_STATE_FAILED`. A2A maps `alreadyTerminal` to - `TaskNotCancelableError`. A Message addressed to an existing Task ID returns - `UnsupportedOperationError` whether that Task is active or terminal; a session - continuation instead creates a new Task using the same context ID and the - prior Task as a reference. The binding never emits - `TASK_STATE_INPUT_REQUIRED`, `TASK_STATE_AUTH_REQUIRED`, or - `TASK_STATE_UNSPECIFIED`. - - Core owns one durable ordered execution-event stream. `SafeUInt` is an integer - 0-9,007,199,254,740,991; `ShortText` is valid UTF-8 of at most 4096 bytes; - `TrajectoryText` is valid UTF-8 of at most 65,536 bytes; `ArtifactId` matches - `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`; and `MediaType` is a valid RFC 6838 - media type of at most 255 ASCII bytes. `TrajectoryPayload` is exactly - `{ kind: "json", value }` or `{ kind: "text", text: TrajectoryText }`. - JSON `value` has at most 32 levels, contains no non-finite number, and its - RFC 8785 encoding is at most 65,536 bytes. Native structured values use the - JSON branch; native strings or values without a lossless JSON representation - use the text branch. Equivalent structured values therefore normalize to the - same bytes. - - `HarnessExecutionEventV1` is exactly one gapless `sequence` from zero plus: - - `{ type: "progress", phase, message?: ShortText }`, where `phase` is - `accepted | acquiring | preparing | executing | collecting | cleaning`; - - `{ type: "toolCall", callId: ArtifactId, name: ShortText, - arguments: TrajectoryPayload }`; or - - `{ type: "toolResult", callId: ArtifactId, - status: "completed" | "failed" | "canceled", output: TrajectoryPayload }`. - - Tool-call IDs are unique. A result follows exactly one matching call, and a - call has at most one result. A normally completed execution with a complete - trajectory has exactly one result for every call. An interrupted call emits a - `canceled` result when the adapter observed that outcome; otherwise it remains - unmatched and forces `complete: false`. Duplicate event delivery is - idempotent only when its sequence and canonical payload are identical; - divergence fails `provider_protocol_invalid`. - - Each Core event is transactionally appended before publication. The gateway - computes `terminalTailReserveBytes` from the generated maximum compact - encodings of every non-droppable terminal field: a 1 MiB valid result, maximal - bounded Core/composed failure records, maximal required source/provenance - identity, the three reserved Artifact envelopes, all terminal events, and - response envelopes. Truncatable output/evidence and optional produced - Artifacts are excluded. Every nonterminal append projects both the durable - event record and its duplicate in the terminal trajectory plus that reserve; - it commits only when the projected footprint remains within - `max-task-bytes`. Otherwise the gateway atomically - records trajectory truncation and publishes neither that event nor later - nonterminal events; provider execution continues. The A2A binding emits one - nonterminal `TaskStatusUpdateEvent` per committed event, stores the exact Core - event at `event.metadata[profileUri].executionEvent`, uses - `TASK_STATE_SUBMITTED` only for `phase: "accepted"` and - `TASK_STATE_WORKING` otherwise, and preserves the Task context ID. Replay and - resubscription emit only committed events in sequence. Tool events are live - progress and also form the terminal normalized trajectory. - - Every terminal Task has exactly one Core outcome Artifact, one Core execution- - trajectory Artifact, one AllAgents workspace-integrity Artifact, and zero or - more produced Artifacts. Atomic settlement commits the terminal Task, every - terminal `TaskArtifactUpdateEvent`, and the terminal - `TaskStatusUpdateEvent` in the same transaction as those Artifacts. Publishers - emit only committed events: every Artifact update precedes the terminal - status update, whose terminal `TaskState` makes it the final stream item, and - the stream then closes. - - The Core outcome Artifact has `artifactId` and `name` equal to - `harness.execution-outcome`, lists `profileUri` in `Artifact.extensions`, and - has one `Part` with `data` set to the strict - `harness-execution-outcome/v1` object and media type - `application/vnd.allagents.harness-outcome+json`. The payload is a strict - discriminated union on terminal `state`; every branch also contains - `version: "1"`, `target: TargetId`, - `terminalOutput: { text, truncated }`, optional `usage`, - `executionTrajectory`, and `result`. Terminal output is UTF-8 at most 1 MiB. - Usage is a strict object with optional `inputTokens`, `outputTokens`, - `cachedInputTokens`, and `totalTokens` `SafeUInt` fields plus optional - `provider` containing 0-64 `ConfigName: SafeUInt` counters. - `executionTrajectory` is - `{ artifactId: "harness.execution-trajectory", eventCount: SafeUInt, - complete, truncated, digest: Digest }`. - - The state branches are closed: - - `completed` forbids `failure` and permits only valid result or - `notProduced` with `notRequested | providerDidNotReturn`; - - `canceled` requires `execution_canceled`/`cancellation` and - `notProduced: canceled`; - - `timedOut` requires `execution_deadline_exceeded`/`deadline` and - `notProduced: deadlineExceeded`; - - `rejected` requires `execution_permission_denied`/`permission` and - `notProduced: rejected`; and - - `failed` requires one other Core failure/result pair from the normative - `core-outcome-matrix/v1`. - - Core-only `failure` is `{ code, message, retryable, cause }`; codes are - HEC-owned error-table rows or `execution_extension_failed`; causes are - `validation | capacity | deadline | permission | providerProtocol | - cancellation | termination | stateStore | restart | extension`; `message` is - `ShortText`. Retryability matches the Core matrix row except - `execution_extension_failed`, whose Core schema admits either boolean. - `result` is exactly `{ status: "valid", value }`, - `{ status: "invalid", errors }`, or - `{ status: "notProduced", reason }`. `value` validates against the requested - `SchemaNode`, serializes to at most 1 MiB, and appears only when requested. - `errors` contains 1-64 strict `{ path, keyword, message }` entries; `path` is - an RFC 6901 JSON Pointer at most 1024 bytes, `keyword` is a v1 `SchemaNode` - member, and `message` is `ShortText`. Reasons are - `notRequested | providerDidNotReturn | providerFailed | extensionFailed | - rejected | canceled | deadlineExceeded | invalidProviderPayload | - retentionLimitExceeded`. - - The checked-in, hand-authored matrix has rows for every Core failure/result - combination and explicitly enumerates terminal state, result status/reason, - and retryability. Its `execution_extension_failed` rows admit both retryability - values; the composed profile requires equality with the selected workspace - failure row. Generated JSON Schema is a `oneOf` over the matrix. An extension - failure before a Core execution result is determined requires - `notProduced: extensionFailed`; a later extension settlement failure preserves - the already determined result, - including a valid result or cancellation/deadline reason. Every unlisted - combination is rejected, including completed+failure, canceled+provider - failure, or timedOut without deadline failure. - - The Core trajectory Artifact has `artifactId` and `name` equal to - `harness.execution-trajectory`, lists `profileUri` in `Artifact.extensions`, - and has exactly one `Part` with `data` set to the strict - `harness-execution-trajectory/v1` object and media type - `application/vnd.allagents.harness-trajectory+json`. Its payload is - `{ version: "1", eventCount, complete, truncated, digest, events }`. - `events` is the longest whole-event prefix, at most 4096 events, that fits the - retained-byte bound; middle or earlier events are never sampled or dropped. - `eventCount` equals its length. `truncated` is true exactly when an observed - event was omitted by a count or byte bound. `complete` is true only when the - adapter asserts full event observability, no event was omitted, and every - interrupted call is represented as above. `digest` is SHA-256 over RFC 8785 - bytes of the same payload with `digest` omitted. The A2A binding requires the - Core outcome's `executionTrajectory` fields to equal the unique trajectory - Artifact's `artifactId`, `eventCount`, `complete`, `truncated`, and `digest` - exactly; any mismatch fails composed validation. Native provider traces remain - optional evidence and never replace this Core record. - - The AllAgents workspace-integrity Artifact has `artifactId` and `name` equal - to `allagents.workspace-integrity`, lists `profileUri` in - `Artifact.extensions`, and has one `Part` with `data` set to the strict - `allagents.workspace-integrity/v1` object and media type - `application/vnd.allagents.workspace-integrity+json`. Its `taskId` equals the - enclosing A2A Task ID. The payload owns: - - `version: "1"` and `taskId`; - - effective logical `workingDirectory`, exactly `{ kind: "workspaceRoot" }` or - `{ kind: "repository", repository: ConfigName, - path?: RelativeDirectory }`, and effective - `workspaceAccess: "readOnly" | "readWrite"`. No physical or configured - destination path is present. `readOnly` is cooperative best-effort provider - policy, not hostile-code containment; - - `sourceIdentity`, either - `{ kind: "repositories", complete, repositories }` or - `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, - workspaceManifestDigest: Digest, layerDigests, complete, repositories }`. - `layerDigests` has 0-64 `Digest` values in manifest order. - `repositories` has 0-64 unique strict entries - `{ name: ConfigName, requestedRevision?: RevisionText, - resolvedCommit: string, verification: - "independentlyVerified" | "snapshotAttested" }`; `resolvedCommit` matches - `^[0-9a-f]{40}$`. Before provider execution, `complete` is true, repositories - exactly match the configured catalog, and snapshot identities include every - layer digest. Failed acquisition records only verified members and sets - `complete: false`; - - optional `workspaceManifestDigest: Digest`; - - `producedArtifacts`, 0-128 strict entries - `{ artifactId: ArtifactId, name?: ShortText, mediaType?: MediaType, - size: SafeUInt, digest: Digest }`. IDs are unique, entries sort by ID, and - their ID set exactly equals `Task.artifacts` minus the three reserved - Artifacts. Each referenced produced Artifact lists `profileUri`, mirrors the - optional name and media type, and has exactly one `Part.raw` containing the - base64 file bytes; other Part content and metadata are absent. `size` and - SHA-256 `digest` cover the exact decoded bytes; - - `evidence: { items, complete, truncated }`, whose 0-256 strict items are - `{ kind, artifactId?, digest?, summary? }` with kind - `gitState | providerTrace | fileChanges | usage | cancellation | - termination | cleanup`, at least one optional member present, and aliases as - above. `complete` means each configured bounded category was attempted after - the direct provider settled, never that every descendant was quiescent; - - `termination: { status: "clean" | "failed" | "unknown", - reason?: ShortText }`, reporting only direct provider/process-group - observation; - - `cleanup: { workspace: "shared" | "removed" | "retained" | "failed", - reason?: ShortText }`. `shared` retains only a reusable base after removing - private runtime; `removed` means all Task/session-owned runtime, writable - view, and non-reusable base state is gone; and - - optional workspace-only `failure: { code, message, retryable, cause }`, - whose code/cause is one AllAgents workspace-extension row in the error table. - It never admits a HEC-owned failure code. - - The module-local workspace payload contains no Core Artifact reference. The - composed profile alone requires the three reserved Artifacts and, for a - workspace failure, pairs exact workspace `failure` with Core - `execution_extension_failed`, cause `extension`, and the same retryability. - A non-extension Core-origin failure leaves workspace `failure` absent. - - Gateway-generated source identity, workspace-manifest fields, evidence - metadata, and provider-added metadata never contain Git URLs, OCI repository - origins, or destination paths. This does not inspect or sanitize opaque - prompts, output, result values, trajectory text, native evidence, or produced - bytes. - - Failure, rejection, and cancellation retain every available bounded field - without implying a valid result. A Task's retained footprint is the maximum - of: (a) the sum of exact UTF-8 bytes of compact canonical JSON for its claim, - Task, event, and Artifact payloads, counting raw Artifact data at its base64 - wire size; (b) its exact serialized `GetTask` HTTP+JSON response; and (c) its - exact serialized one-Task `ListTasks(includeArtifacts: true)` response. The - production serializer computes those values after JSON escaping and envelope - fields, before every commit. - - Collectors retain the longest event prefix, truncate stream-like fields, or - omit a whole produced Artifact to remain within `max-task-bytes`; they never - retain a partial produced file. An otherwise valid structured result that - cannot fit becomes `result.status: "notProduced"`, reason - `retentionLimitExceeded`, and terminal failure - `execution_result_too_large`. Reconciled terminal Task records, Artifacts, - events, and claims expire atomically after their configured TTL, releasing the - actual retained charge. A poisoned/unreconciled Task is ineligible for expiry, - is charged only by its active `max-task-bytes` reservation, and starts its TTL - only when reconciliation atomically replaces that reservation with its exact - footprint. - -**Trust, identity, and Task storage** - -- R4. Do not authenticate application callers because version one is restricted - to one trusted private network. Allow exactly: loopback HTTP, optionally - exposed through a private HTTPS terminator; or native HTTPS bound to one - specific loopback/RFC 1918/RFC 4193/link-local/RFC 6598 address. Native - off-loopback TLS requires configured certificate/key files. Wildcard and - public listeners are rejected before binding. The advertised URL is loopback - HTTP only for local use and otherwise private HTTPS. Tailscale ACLs, private - firewall rules, or equivalent controls remain required. Every reachable caller - may create, list, retrieve, subscribe to, resume, and cancel every Task/session. - Artifacts are retrieved only inside Tasks through `GetTask` or - `ListTasks(includeArtifacts: true)`; v1 adds no separate Artifact endpoint. - Startup/docs state that reachability is authorization and public exposure - requires application authentication, authorization, and an amended ADR. -- R5. Idempotency, Task visibility, and session visibility are deployment-wide. - One transactional `createOrReplay` operation validates and locks the selected - session state, then atomically writes the invocation claim, Task, initial - `accepted` event, effective context ID, and byte reservation. A `start` also - creates the session row; a `resume` compare-and-sets the exact idle head to the - new active Task. The claim binds the key to the canonical Core request, - Sessions input, workspace input, full A2A session projection, selected - target/source/cwd/access, optional result-schema digest, deadline, and - effective configuration digest before acknowledging Task creation. The Task - reservation is exactly `max-task-bytes`. - Admission succeeds only when retained Task count, - `settledFootprints + activeReservations + maxTaskBytes`, retained session count, - and prior-plus-candidate session byte reservations fit. Replay returns the - existing Task/reservations; a changed request conflicts. Settlement uses the - R16 prepared record, then atomically replaces the Task reservation with its - footprint and either sets the idle lineage head to the current terminal Task - plus the selected committed provider-checkpoint/workspace-generation pair, or - records close/poison state. Task expiry releases only its footprint and never - a live session pair. Status, terminal - settlement, and head/pointer advancement are monotonic. The project state root - persists canonical workspace identity and holds one process lock. - - The Bun gateway privately owns one SQLite database through `bun:sqlite`. - Ordinary tables hold claims, Tasks, events, Sessions, ordered turns, opaque - provider checkpoints, committed/candidate/superseded workspace-generation - identities, prepared settlement/cleanup records, pinned configuration/source/ - workspace digests, session byte reservations/charges, retained Task charges, - bounded Artifact bytes, the execution lease, outcome intent, acquisition- - container/staging/transient-base identity, provider process-group identity, - observed outliving-descendant identity, and expiry. - Enable foreign keys, use WAL where supported, set `synchronous=FULL`, and - acknowledge only committed transactions. The current user owns the state root - and database with `0700`/`0600`-equivalent permissions; the root is disjoint - from project, staging, publication, profile, and provider-auth roots. No custom SQLite VFS - or native file primitive is introduced. Startup verifies the root, lock, - schema/integrity, and workspace identity; terminalizes interrupted Tasks - failed; removes recorded acquisition containers and orphan staging; and - attempts to terminate recorded provider process groups. An unresolved App mint - intent or revocation-pending token requires recorded container/local-secret - absence and an atomically durable expiry-backed tombstone before the - interrupted Task terminalizes. - The gateway releases a stale/poisoned lease only after the container and - recorded process group are gone, every observed outliving descendant is gone, - candidate/committed/superseded workspace and other owned state reconcile to the - database pointer, and any revocation tombstone is durable. Otherwise readiness - remains false. An interrupted turn is never resumed/replayed; an idle session - may resume only from its last committed provider-checkpoint/workspace- - generation pair. Store open/corruption/write/synchronization failure stops - admission and prevents terminal success. - -**Workspace and target configuration** - -- R6. One gateway process serves one project workspace selected by `--workspace` - or cwd. Parse its `.allagents/workspace.yaml` through the authoritative project - schema, then compile a gateway-only repository catalog without tightening - ordinary workspace parsing. Derive each logical name from `name` or the - existing path-basename fallback; require 1-64 unique `ConfigName` values, - collision-free normalized destinations, and one supported canonical GitHub - origin resolved through existing `source`/`repo` semantics. Repository names, - origins, destinations, default revisions, workspace projection, plugins, and - named OCI snapshot repositories come only from that declaration. A catalog - failure makes the gateway not ready. -- R7. Parse `~/.allagents/workspace.yaml` through the authoritative user schema. - Built-in `codex` and `pi` targets are available when ready. A launcher-bearing - profile client adds a target only when `gateway.enabled: true`. Its public ID - is the globally collision-checked launcher basename and resolves to exactly - one `(profile, client)` pair. Built-in IDs are reserved under the same portable - collision key; colliding enablement is a configuration error. Initially only - Codex and Pi profile clients are executable. -- R8. A request selects a declared target, may select one logical - `workingDirectory`, may select `workspaceAccess`, may set the bounded - `deadlineSeconds`, may provide one bounded result schema, and chooses - `oneShot | start | resume`. `workspaceAccess` defaults to `readWrite`; it is - never inferred from prompt text. A session start pins target, canonical - source, logical cwd, access mode, provider identity, and effective - configuration. Resume must repeat those values exactly. - - A one-shot read-only Task resolves its cwd directly inside its validated base. - An exact immutable request may share a reusable cached base; a mutable branch - or tag request owns a non-reusable base for the Task lifetime. The Task - receives a private `/tasks//runtime` for temporary, - home, provider-state, and evidence files. A read-only session instead pins its - base and retains private `/sessions//runtime` - state between turns. Provider environments disable optional Git locks, and - adapters request native read-only policy when available. The gateway does not - inspect prompts or add a mount, chmod pass, or full-tree verification. The - consumer remains responsible for assigning read-only work that does not - require project mutation. - - A one-shot read-write Task receives a unique writable view at - `/tasks//workspace`. A read-write session never - mutates its last committed view in place. Each turn materializes - `/sessions//candidates/` from the pinned - base (start) or committed view (resume), runs the provider only in that - candidate, and keeps the prior view until settlement. Committed immutable - generations live under - `/sessions//workspaces/`, with the - current generation named only by the SQLite session row. The materializer - prefers a filesystem block clone, falls back to rootless OverlayFS on - supported Linux, and supports ordinary copy; writable files never use hard - links. Admission reserves both prior and candidate charges against per-session - and aggregate limits. Successful continuing settlement atomically advances - the provider checkpoint, workspace-generation pointer, and head before - removing the prior generation. Crash before that commit discards the - candidate and leaves the prior pair; crash after it keeps the new pair and - reconciles the recorded old generation. One-shot settlement removes its view. - Session `closeAfterTurn` or idle expiry removes all generations, candidates, - provider state, and non-reusable base only after no active turn remains, then - releases the session charge. Uncertain cleanup poisons and retains the session. - - `{ kind: "workspaceRoot" }` selects the effective base, Task view, or session - view. A - repository selector maps its declared name through the compiled catalog, - appends only the validated `RelativeDirectory`, resolves links without escape, - and must name an existing directory beneath that repository. Absolute paths, - configured destinations, undeclared repositories, non-directories, and - escaping resolutions fail before provider start. Gateway-generated requests, - structured Task/Artifact metadata, and operational logs never contain the - physical path. Opaque terminal output, native evidence, and produced-Artifact - payloads are not sanitized and may contain it. - - The gateway owns one durable execution lease covering base acquisition or - lookup through final evidence/cleanup. Admission claims it transactionally - before starting a container or provider; at most one gateway-controlled Task - may hold it. A second valid request settles failed with - `execution_capacity_unavailable` without creating a container/process. Lease - identity survives restart. Normal settlement releases it only after the - recorded process group is absent, no observed escaped/outliving descendant - remains, and required cleanup/checkpoint reconciliation succeeds. Failure - retains the lease and readiness stays false. - - The overall deadline covers cache lookup, Docker acquisition on miss, - publication, optional materialization, typed preparation, bare-metal provider - execution, and evidence collection. Acquisition receives - `min(900 seconds, remaining overall deadline)`; exceeding that sub-budget - removes the acquisition container and fails before provider execution. - Overall expiry initiates adapter abort and Linux process-group escalation. - Cleanup then uses its own fixed bounded budget. A request cannot provide or - override backend, executable path, command, argv, environment, provider home, - profile settings, plugins, MCP servers, repository URLs, destination paths, - credential provider, setup behavior, provider permission policy, materializer, - cache key, Docker options, image reference, or mounts. Readiness rejects - missing, partial, drifted, unsupported, or declaration-missing gateway-enabled - profiles. - -**Workspace acquisition** - -- R9. Use exactly one closed source union: - - `{ kind: "repositories", revisions?: Record }`; or - - `{ kind: "workspaceSnapshot", snapshot, digest, workspaceManifestDigest }`. - Unknown variants, cross-variant fields, undeclared names, mutable snapshot - references, malformed digests, and destination overrides fail admission. - Source-mode failure never falls through to the other mode. -- R10. Repository mode materializes every entry in the compiled gateway catalog. - Caller revisions may override only a declared repository's default revision. - Canonicalize HTTPS GitHub origins, resolve and record full commits before - provider execution, use hermetic Git configuration, disable redirects and - repository-controlled secondary fetch/exec features, verify checkout - identities, and reject path collisions or escapes. -- R11. Snapshot mode maps `snapshot` to a declared OCI repository and constructs - `@` server-side. The same digest-pull contract must - interoperate with Docker Hub, GHCR, JFrog Artifactory/JFrog Container - Registry, and compatible private OCI Distribution registries; registry choice - does not alter the accepted snapshot format. V1 accepts only - `application/vnd.oci.image.manifest.v1+json` with `schemaVersion: 2` directly - at the requested digest. Reject image indexes, nested indexes, descriptor - `urls` or embedded `data`, non-distributable layers, unknown media types, and - more than 64 layers. The config descriptor must use - `application/vnd.allagents.workspace-manifest.v1+json`; its digest must equal - `workspaceManifestDigest`, and its bytes are RFC 8785 canonical JSON. Accepted - layer media types are the OCI distributable tar, gzip, and zstd variants. - Verify the raw manifest body and every config/layer descriptor size and digest - while streaming, before decoding. Apply layers base-to-top with OCI whiteout - and opaque-whiteout semantics. - - The wire-visible workspace manifest contains every compiled project - repository exactly once by logical name and omits destination paths. After - applying layers, the gateway uses the compiled operator catalog to verify that - each listed repository exists at its configured destination and that no - repository is missing, extra, renamed, misplaced, duplicated, or accompanied - by undeclared generated content. Apply these fixed v1 ceilings across all - processed layers, including overwritten or whiteouted content: 4 MiB manifest, - 4 MiB config, 8 GiB total compressed layer bytes, 32 GiB total expanded bytes, - 500,000 entries, 4 GiB per regular file, 4096 UTF-8 bytes and 128 components - per path, and 1 MiB per PAX or other extended header. Abort before crossing a - limit. Validate paths, collisions, file types, modes, links, and the compiled - filesystem layout in staging before atomic publication. Reject absolute or - traversing paths, devices, sockets, sparse files, escaping links, credentials - in redirect URLs, unapproved cross-origin redirects, and external layers. - Cross-origin redirects are limited to layer-blob `GET`/`HEAD` requests and - exact operator-declared `layerRedirectHosts`; token, manifest, and config - requests remain same-origin. Private or otherwise non-global destinations are - permitted only when the exact host is the source's declared repository host - or a declared layer-redirect host, with per-hop rebinding checks. The common - path-free workspace manifest distinguishes independently verified Git facts - from snapshot-attested facts; compiled destinations remain private validation - inputs. - - After host validation, atomically promote staging to a validated base. An OCI - identity is reusable under a gateway-owned cache key containing its manifest - and workspace-manifest digests. A repository identity is reusable only when - every effective revision is a full commit ID. Its cache key also binds the - acquisition-contract version, compiled catalog/layout digest, and every - commit. Branch and tag requests instead receive a non-reusable Task- or - session-owned base and never populate or reuse a cache entry. A valid cache hit - starts no acquisition container and resolves no source credential. Active - Tasks/sessions pin a reusable base; bounded eviction removes only unpinned - cache entries. - -**Credential selection and containment** - -- R12. Repository requests never carry credentials or select providers. For - `github.com`, a configured App lookup returning installation coverage is - `eligible`. A 404 is `ineligible` only after the repository's existence is - independently proven through the configured GitHub CLI identity; an - uncorroborated 404, 401, 403, 429, timeout, or 5xx is `unknown`. An explicit - installation ID is eligible only after positive repository-coverage - verification. For `eligible`, the host gateway creates the App JWT and - verifies installation coverage. Before the external mint request, it durably - records a non-secret mint intent with conservative possible-token expiry - `now + mintRequestTimeout + 1 hour + 60 seconds`. The gateway enforces and - aborts the external request at `mintRequestTimeout`; the bound covers a token - minted at the last permitted instant, GitHub.com's one-hour lifetime, and - clock skew. It then mints one repository-scoped read-only installation token - for the cache-miss acquisition and atomically replaces the intent with the - returned non-secret issue time, expiry, and `revocationPending: true`. A - definitive no-token response may clear the intent; any crash or ambiguous - mint outcome leaves it for reconciliation. Credentials are never cached. - Validate a returned token's repository selection, permissions, creation time, - and expiry, and require remaining lifetime greater than the R8 acquisition - sub-budget plus a 60-second clock-skew margin. Only the installation token - enters the acquisition container; the App private key remains on the host. - - Use the configured GitHub CLI account only when the App is absent or - applicability is positively `ineligible`. An `unknown` result or any - selected-App configuration, authentication, minting, permission, repository, - rate-limit, or service failure terminates acquisition without `gh` fallback. - Resolve `gh auth token --hostname github.com --user ` on the host - with ambient token variables removed, then inject only the selected - invocation-scoped source credential into the acquisition container. Every - non-crash exit after an App token is minted—including cancellation, deadline, - shutdown, validation failure, and successful acquisition—runs one idempotent - revoke-and-confirm path; the Task does not terminally settle or report source - cleanup complete until that path finishes. A revocation failure becomes - `source_auth_failed` and prevents provider execution. - - A persisted unresolved mint intent or revocation-pending token is reconciled - before Task terminalization. Startup removes any recorded container and local - credential state, atomically persists an expiry-backed tombstone through the - conservative or actual expiry, and settles the interrupted Task - `TASK_STATE_FAILED` with `gateway_restarted` plus evidence that revocation is - unconfirmed. The tombstone is not cascade-deleted with Task expiry and is - removed only after its own recorded expiry. Once the container and local - secret are confirmed absent and the tombstone is durable, the stale execution - lease may be released and fresh App-backed acquisition may proceed with a new - token; the gateway never claims the possible or known old token was revoked. - OCI acquisition accepts anonymous pulls or exact- - registry credentials from the strict Docker-auth/helper boundary and supports - same-origin Basic and Distribution Bearer challenges, the documented Docker - Hub token service, and an operator-supplied exact-host CA-bundle map. The - acquisition container receives source-only credentials and trust material; - they are destroyed with the container before provider preparation. - -**Execution, evidence, and cleanup** - -- R13. Keep one closed `codex | pi` backend registry behind a narrow - AllAgents-owned TypeScript interface covering availability, per-mode - capability advertisement, start/resume/checkpoint/dispose, invocation, - ordered normalized events, deterministic permission handling, abort, process - settlement, output/structured result, native usage/cache counters, evidence, - and disposal. Provider session handles are opaque, private, and accepted only - from their creating adapter. Each adapter preserves native event order, - reports observability gaps, supplies native call identity when available, and - directs all mutable conversation/runtime state to the Task/session-private - root. It may reference host authentication only through a pinned, - provider-supported auth input distinct from that mutable state root. U0 proves - this split for every advertised auth/mode combination; otherwise the target or - session modes remain unavailable. The gateway never copies, mounts, parses, or - imports provider auth files to synthesize the split. - - The gateway derives stable invocation-local call IDs when needed, normalizes - payloads, chooses the retained prefix, and computes trajectory `complete`, - `truncated`, and digest. Neither layer invents events or cache hits. Profile - targets resolve through adapter-owned configuration; never discover arbitrary - executables from `PATH`, scrape a TUI, append public input to argv, or download - a provider runtime per request. A global binary override is eligible only - after an exact version/protocol probe. -- R14. Codex uses pinned `@openai/codex-sdk` directly. Start uses - `startThread()`, resume uses `resumeThread()` with the last committed opaque - thread ID, and each Task is one SDK `run()` turn. App-server is allowed only - when U0 proves a named SDK gap and records the tested protocol. Each Task gets - streamed events, native cancellation, an explicit environment, and a private - state home under its Task/session runtime. API credentials may pass through - the explicit auth allowlist. Existing ChatGPT login is supported only if the - pinned public integration can reference its host auth location separately - while keeping thread/session writes in the private state home; otherwise that - auth/target combination is not ready. Pass native `outputSchema` only when the - public schema has an object root, every object's `required` set equals its - property set, nesting is at most 10 levels, and every keyword is supported by - the pinned SDK/model. Other valid schemas use JSON guidance plus gateway - validation. - - Pi uses a pinned supported package/RPC surface under the same auth/state split, - with explicit create/resume/checkpoint/dispose, invocation-owned - configuration, and one restricted policy extension. If its pinned public - surface cannot separately reference host auth, root mutable state privately, - and durably resume by opaque stable handle, only the modes that pass those - probes are advertised; transcript replay is never a substitute. Repository - extensions and unrestricted built-ins do not auto-load. - - Both adapters preserve only required auth references, executable lookup, - locale, certificates, and proxy settings in an explicit environment allowlist. - This reduces accidental leakage; it is not secret isolation because model - tools retain CI-job authority. A resumed session appends to provider-native - history with pinned target/model/tool configuration, preserving the longest - eligible prompt prefix. This improves cache eligibility but never guarantees a - hit: routing, model rules, prefix length, and TTL remain external. Report only - native `cachedInputTokens`; never infer savings. -- R15. Start a fresh Docker container only when a request has no reusable - validated base, including a cache miss or a non-reusable branch/tag request. - Probe Docker and the exact digest-pinned `apps/acquirer` image at that point. - A repository-mode probe failure is `source_git_unavailable`; a snapshot-mode - probe failure is `source_snapshot_unavailable`. The gateway creates private - staging and starts the image with that directory as its only writable bind - mount. The container receives the canonical acquisition request, compiled - catalog, strict network/size/archive policy, source-only GitHub or OCI - credentials, and only required exact-host CA material. It receives no GitHub - App private key, host home, provider home, Docker socket, gateway database, - published base, unrelated credential, Codex, Pi, or other coding harness. It - never downloads a coding harness. Docker network access is limited to source - endpoints required by the selected Git or OCI mode. - - The acquirer writes content beneath staging and emits one typed manifest - through the bind mount, then exits. The host gateway waits for exit, removes - the container, destroys source credentials, validates the manifest and tree - against the compiled catalog and fixed limits, and atomically promotes staging - to a reusable cache entry or non-reusable Task/session-owned base. Every - cancellation, deadline, validation failure, or other non-publication path - removes staging idempotently. Cleanup uncertainty writes - `source_cleanup_failed` and terminal Artifacts/events within the active - reservation, but retains the acquisition identity, staging, reservation, - reconciliation record, and lease; disables expiry/readiness; and stops - admission. Verified startup/operator cleanup uses the same atomic - reservation-to-footprint, lease-release, and TTL-start rule as other poisoned - turns. Source-mode failure never falls through. - - A read-only one-shot uses the base plus Task-private runtime; a read-only - session uses the base plus session-private runtime. Adapter preparation keeps - project files unchanged and places invocation config outside the base. A - read-write one-shot receives a unique view; each read-write session turn - receives its R8 candidate from the committed generation. Typed preparation may - project validated project/profile settings, plugins, and MCP declarations into - that writable view/candidate. Project/user `setup` entries and other shell - commands never run automatically. - - Codex and Pi execute as direct host processes on the same trusted Linux CI - runner as the gateway. The adapter receives resolved cwd, effective access, - Task/session-private runtime and mutable state, plus only a supported separate - host-auth reference; it never receives a caller-supplied physical path or - materializer choice. Provider execution never - reuses the acquisition - container and never creates a per-invocation provider container. The CI job, - VM, or deployment container is the isolation boundary. AllAgents does not - claim containment of hostile repository code, network access by model tools, - or provider/MCP/operator secrets from those tools. Capture bounded provider - events while the direct provider process is live. Collect filesystem/Git - evidence only after that direct process settles and process-group termination - attempts finish; phrase the evidence as observed after direct-process - settlement, never as proof that every descendant is quiescent. Run Git - inspection with hermetic configuration that disables hooks, filters, drivers, - fsmonitor, pagers, helpers, optional locks, and external commands. -- R16. On trusted Linux runners, start each direct provider in a new process - group and persist its leader PID plus Linux process-start marker with the Task - and execution lease before recording provider execution as started. One - durable compare-and-set arbitrates provider terminal outcome, caller - cancellation, overall deadline, and shutdown as an internal `outcomeIntent` - while the external Task remains nonterminal. The winning intent owns the - execution-result facts and drives one idempotent abort path: request graceful - adapter abort, wait the configured grace period, send `SIGTERM` to the process - group, then `SIGKILL` after the forced-termination period. - After the direct provider process has settled and bounded evidence collection - finishes, a one-shot or `closeAfterTurn` path removes private runtime state, - all applicable workspace views, any non-reusable base, and the provider - checkpoint. A continuing read-only session verifies its provider checkpoint; - a continuing read-write session additionally verifies the completed candidate - view and its measured charge while the prior committed generation remains - untouched. - - Settlement persists a prepared record naming the exact prior/candidate - workspace generations and provider checkpoint. One SQLite commit writes the - terminal Task, three reserved Artifacts, produced Artifacts, bounded evidence, - observed termination, Task footprint, terminal events, and either session - removal or an idle lineage head naming the current Task plus the selected - committed provider checkpoint/workspace-generation pointer. The pair may be - newly committed or the provably unchanged prior pair. It also records any - superseded generation for cleanup. The gateway removes that recorded state - and releases the lease only in a final transaction after cleanup succeeds. - Startup deterministically completes a committed cleanup - intent or discards an uncommitted candidate; it never pairs a prior provider - checkpoint with candidate workspace bytes. - - Settlement failure overrides the external execution outcome. If required - Task/session cleanup, provider checkpointing, candidate verification, or - pointer advancement fails or remains uncertain, the failed Task records Core - `execution_extension_failed`, the truthful execution result, exact workspace/ - session failure, cleanup evidence, all three reserved Artifacts, and terminal - events. The session becomes non-resumable unless the adapter proves the prior - provider checkpoint unchanged and the database still points at the untouched - prior workspace generation. When that proof succeeds, the session is idle - with this failed Task as lineage head and the prior checkpoint/workspace pair. - Unreconciled reservations, cleanup record, lease, - and affected Task/session state remain non-expiring while readiness is false. - - If the direct process does not settle after `SIGKILL`, the failed Task records - `execution_termination_failed`, incomplete trajectory, and live-provider/ - termination evidence, but no filesystem/Git or produced-Artifact evidence. - It retains all prior/candidate state, reservation, reconciliation record, and - lease with expiry/readiness disabled. - - Startup/operator repair releases the lease only after the recorded process - group is absent, every observed outliving descendant is gone, and all - retained state is reconciled. It may restore only a provably unchanged - provider checkpoint plus the database-selected committed workspace - generation; otherwise it cleans up and closes the session. Repeated - cancellation does not re-signal work. A canceled continuing session remains - open only when `closeAfterTurn` is false and that same checkpoint/workspace - proof succeeds; its lineage head still advances to the canceled Task. Startup - never resumes an interrupted turn. Gateway shutdown - stops admission, commits shutdown intent, performs the same escalation and - settlement, and exits. - - CI runner teardown is the final orphan boundary. AllAgents does not use - cgroups, pidfds, namespaces, nftables, `openat2`, a native platform layer, or - non-bypassable spawn mediation. It admits only one gateway-controlled turn at - a time but does not claim complete descendant enumeration: any observed - escaped/outliving descendant retains the lease and readiness remains false - until runner teardown or verified disappearance. - -**Scope and configuration** - -- R17. Do not add evaluation commands, datasets, assertions, scoring, - repetitions, experiment scheduling, or automatic Task retry. -- R18. Do not add `gateway.yaml` or `worker.yaml`. Process configuration uses - the exact CLI flags and environment variables in the Configuration Contract - for listener, advertised interface URL, workspace, state/retention, - immutable-base cache, workspace materializer, acquisition image and Docker - access, GitHub/OCI source credentials, provider executable overrides, provider - home/auth paths, and process-group timeouts. The listener also exposes - unauthenticated metadata-only `/healthz` and `/readyz` endpoints outside A2A: - liveness returns 200 while the process can serve; readiness returns 200 only - while new admission is safe and otherwise 503. They reveal no targets, - sources, paths, or failure details and do not require A2A headers. Gateway code - never copies acquisition credential values into generated workspace files, - requests, logs, Task/Artifact metadata, retained Task/session views, cache - entries, or provider environments. This is not a redaction or isolation - guarantee for - opaque prompts, provider/tool output, structured results, inherited host - authentication, native evidence, or produced-Artifact payloads. -- R19. Document AI Evals consumption through a Promptfoo custom - JavaScript/TypeScript provider implementing Promptfoo's `ApiProvider`. - `constructor(options: ProviderOptions)` requires and retains a nonempty - `options.id`, validates `options.config`, and `id()` returns that ID. Static - config contains the private gateway endpoint, target ID, optional default - logical `workingDirectory`, optional `workspaceAccess` defaulting to - `readWrite`, and exactly one closed source mode: repository mode materializes - the complete configured repository set and carries only an optional revision - map keyed by declared repository name; snapshot mode carries one declared - snapshot name with OCI and workspace-manifest digests. - `callApi(prompt, context?, options?)` may apply the exact - `context?.vars?.allagentsSource` leaf overrides, may replace the default - selector through `context?.vars?.allagentsWorkingDirectory`, may replace - access through `context?.vars?.allagentsWorkspaceAccess`, and may select - `{ mode: "oneShot" }`, `{ mode: "start", closeAfterTurn? }`, or - `{ mode: "resume", sessionId, previousTaskId, closeAfterTurn? }` through - `context?.vars?.allagentsSession`. An absent session value means `oneShot`; - missing context otherwise retains static values. - For executable Promptfoo multi-turn tests, the provider also accepts - `allagentsConversation: { id: ConfigName, action: "start" | "continue" | - "close" }`, mutually exclusive with `allagentsSession`. It keeps an - instance-local map from conversation ID to the last terminal projection. - `start` requires no entry; `continue` and `close` require an idle entry and - send its exact `sessionId`/`headTaskId`, with `close` setting - `closeAfterTurn`. An idle terminal projection atomically replaces the map - entry; `closed` or `notResumable` removes it. Interleaved IDs remain isolated, - and a second in-flight call for one ID fails locally. Explicit - `allagentsSession` remains the crash-recovery/manual handoff using IDs stored - by the evaluator. - Dynamic source values remain limited as defined below. The working-directory - variable is exactly `{ kind: "workspaceRoot" }` or - `{ kind: "repository", repository: ConfigName, - path?: RelativeDirectory }`; access is exactly `readOnly` or `readWrite`. - Unknown members, invalid relative paths, URLs, physical or configured - destination paths, credentials, commands, Docker options, materializer - choices, and provider permission policy fail before provider execution. - - The provider sends `SendMessage` with `configuration.returnImmediately: true`, - captures the accepted Task and context IDs, and calls `SubscribeToTask`; a - terminal-before-subscribe race or broken stream falls back to `GetTask` and - resubscription within the same deadline. A resume sends the retained context - ID and head Task as its sole task reference. A deadline or - `options?.abortSignal` issues exactly one `CancelTask` with a fresh bounded - cleanup signal rather than the aborted request signal. One `callApi` creates - one A2A Task and maps terminal output, usage, Task/Artifact IDs, structured - result, logical provenance, and provider-reported cached input tokens into - `ProviderResponse`; the validated - `Task.metadata[profileUri].session` projection is copied unchanged to - `ProviderResponse.metadata.session`. Admission/terminal failure maps a safe - human message, stable `code`, `retryable`, accepted `taskId`, and any terminal - session projection. AI Evals owns chaining/provider code. AllAgents publishes - the protocol and YAML examples without importing Promptfoo provider code or - Promptfoo as a runtime dependency. + workspaceManifestDigest: Digest }`; `workingDirectory` is exactly + `{ kind: "workspaceRoot" }` or + `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }`. +- **R6.** The AllAgents hook expands omitted `revisions` to `{}` and omitted + `workingDirectory` to `{ kind: "workspaceRoot" }`; an omitted repository + `path` remains absent and an empty path is invalid. It NFC-normalizes strings, + sorts maps, rejects unknown fields, and hashes RFC 8785 bytes as the effective + descriptor digest. Project-config default refs affect resolved provenance, not + this request digest. Accepted/rejected fixtures prove omitted and + explicit-default forms canonicalize identically. +- **R7.** Add one runner-side `/materialize` operation outside the provider + candidate loop. It invokes a configured executable directly without a shell + after fresh-session hydrate and before `/turn`. Pass at most 128 KiB on stdin, + accept at most 1 MiB on stdout and 64 KiB on stderr, and use the smaller of + 900 seconds or the remaining UHP deadline. The generic request contains the + opaque metadata value, session workspace and fixed sibling staging roots, + project configuration root, and deadline. Source secret values arrive only in + a configured allowlisted child environment; the runner subtracts every name + in that allowlist from all agent child environments regardless of its spelling. + The typed result is `completed` with effective relative cwd and bounded public + metadata or `failed` with stable code, safe message, and retryability. A + materializer failure terminalizes the UHP response and never enters provider + fallback. +- **R8.** Persist a CAS-protected session materialization state: + `unbound -> materializing -> ready` or `failed`. The runner validates successful + staging, publishes it with a recoverable same-filesystem rename protocol, + writes a descriptor/provenance marker, initializes the HarnessRouter root + checkpoint and nested-repository collection baselines, and returns + `published` with the checkpoint digest. Before provider dispatch, the gateway + stores that digest and public metadata and CASes the session to `ready`. On + restart in `materializing`, reconcile to `ready` only when the workspace + marker and durable checkpoint match the bound descriptor; otherwise mark the + session non-resumable, remove or quarantine the workspace, and never replay + acquisition. Provider fallback sees `ready` state only and cannot invoke the + hook. Ordinary input files are applied only afterward. + A validated logical cwd may be the root or a symlink-safe descendant; the + runner derives UID isolation from the session root and rejects cross-session + or escaping paths. + +#### Source acquisition and provenance + +- **R9.** Parse the project `workspace.yaml` through its authoritative schema. + Add strict project-only `workspaceSnapshots` entries: + `{ name: ConfigName, repository: OciRepository, + workspaceManifestMediaType: MediaType, executionCredential?: "${ENV_VAR}" }`. + `OciRepository` is a normalized `registry-host/repository-path` with no scheme, + tag, digest, userinfo, query, or fragment. Reject unknown fields, literal + secrets, and duplicate snapshot names; snapshot entries do not merge with user + configuration. Add the same optional environment-reference field to + execution-eligible repositories. Secret values remain deployment-only. A + repository's logical name is explicit `name` or the portable basename of + normalized `path`. Execution-eligible Git destinations must be unique, + non-empty, non-root relative child paths so their `.git` directories cannot + collide with HarnessRouter's root checkpoint repository. Reuse one shared + source resolver: `source` as a supported HTTPS URL is complete when `repo` is + absent; otherwise `source` names the supported host/provider and `repo` names + its repository. Conflicting forms, local/originless entries, duplicate + repository names, escaping destinations, and unsupported schemes make + materializer preflight fail. HarnessRouter owns harness/model/provider targets, + and the user workspace is not a materialization catalog. +- **R10.** Repository mode materializes every execution-eligible declared + repository. Optional revisions override only matching logical names; otherwise + use configured `branch`, then the remote symbolic HEAD. `RevisionText` is at + most 255 ASCII bytes and is either a full 40-hex object ID or a + `git-check-ref-format`-equivalent ref name. Reject leading dashes, whitespace + and controls, refspec colons, glob metacharacters, traversal-like components, + `@{`, and `.lock` components. Resolve a validated full ref, or an unambiguous + shorthand under `refs/heads/` or `refs/tags/`, with `ls-remote`; accept object + IDs only when advertised. Subsequent fetch/checkout commands receive only the + verified object ID with explicit end-of-options handling, never caller text. + Allow only argument-vector HTTPS Git operations to exact configured hosts, with + no URL credentials, query, fragment, or redirects. Use an isolated HOME plus + `GIT_CONFIG_NOSYSTEM=1`, no global config, empty credential helper, disabled + hooks, `protocol.file.allow=never`, `protocol.ext.allow=never`, and no + submodule recursion, Git LFS hydration, or configured clean/smudge filters. + Preserve each repository's `.git` directory for the coding agent. Failure never + falls through to snapshot mode or another credential identity. +- **R11.** Snapshot mode constructs a server-side immutable OCI reference from + the selected snapshot's configured repository and caller-provided digest. + Accept only a direct OCI image manifest with at most 64 distributable + tar/gzip/zstd layers and the entry's configured workspace-manifest media type; + redirects may not change registry authority. Verify manifest, config, layer + size and digest before use; apply OCI whiteouts; limit manifest and config to + 4 MiB each, total compressed layers to 8 GiB, expanded bytes to 32 GiB, + entries to 500,000, one regular file to 4 GiB, paths to 4096 UTF-8 bytes and + 128 components, and one PAX/extended header to 1 MiB. Reject devices, sockets, + traversal, escaping links, sparse files, unknown or foreign layers, mutable + tags, and undeclared output. Validate the final logical repository catalog and + workspace-manifest digest in staging. Snapshot repository roots need not + contain `.git`; after publication the runner creates private collection + baselines from the verified trees so later produced-file reporting remains + truthful. +- **R12.** Extend HarnessRouter's response translator and stored-response paths + so streaming events, terminal responses, GET, background completion, and + idempotent replay return the same bounded + `response.metadata["allagents.workspace"]`. It contains extension version, + effective descriptor digest, logical cwd, source mode, completeness, resolved + commits or OCI manifest/config/layer digests, and workspace-manifest digest. + It never contains origins, physical paths, credentials, or unverified facts. +- **R13.** The materializer resolves `${ENV_VAR}` references from its allowlisted + child environment, uses hermetic Git/registry configuration, removes temporary + auth files before returning, and emits no secret. The fork removes every + configured materializer-only name from `_child_env` and every other agent + subprocess environment independent of name patterns. Prove with a deliberately + innocuous variable name that acquisition secrets and the long-lived + `codex-lb` key are absent from the agent environment, workspace, nested Git + remotes/config, generated CLI configuration, logs, checkpoints, and response + metadata. Provider OAuth remains only in `codex-lb`. +- **R14.** Build one pinned custom HarnessRouter image. Record the upstream + commit, patch-series digest, AllAgents package/version, materializer-contract + version, Codex version, optional verified Pi version/format, image digest, and + supported architecture. Requests without the configured metadata key remain + stock-compatible. CI rebases selected upgrades and runs upstream plus + AllAgents integration tests. +- **R15.** AI Evals owns its Promptfoo provider. It sends the UHP request directly + to HarnessRouter, maps Promptfoo variables to the closed extension, and maps + terminal output, usage, artifacts, provenance, and failures to + `ProviderResponse`. Multi-turn cases retain the prior response ID and send it + as `previous_response_id`. AllAgents documents the contract and examples but + does not depend on Promptfoo at runtime. ### Key Flows -- F1. **Start and advertise** - 1. Resolve cwd or `--workspace`, user workspace, project-specific state root, - disjoint immutable-base cache and invocation roots, cache/task retention, - workspace materializer, listener, advertised URL, digest-pinned acquisition - image, Docker endpoint, source credentials, provider auth locations, and - configured provider executable overrides. - 2. Validate the SQLite state root, cache/invocation roots, workspace identity, - materializer policy, and static acquisition-image reference; compile - repository, snapshot, and target catalogs; verify Codex SDK and Pi RPC/ - package compatibility; and check any global binary override exactly. Do not - contact Docker or the acquisition registry at startup. - 3. Reconcile interrupted turns by removing any recorded acquisition container - and orphan staging, terminating any recorded Linux provider process group, - and reconciling recorded Task/session-owned runtime, view, non-reusable - base, and provider checkpoint state. When an App mint intent is unresolved - or token revocation was pending, confirm the container and local secret are - absent and atomically persist the conservative- or exact-expiry tombstone. - Terminalize the Task failed and release the durable lease only after those - conditions hold; restore only a provably unchanged committed session - checkpoint, otherwise poison the session. Keep readiness false while - uncertainty remains. - 4. Bind loopback HTTP or one specific private address with native TLS; reject - wildcard/public binds, invalid TLS files, and public advertised resolution - before listening. Serve metadata-only probes and publish one Agent Card - whose absolute interface URL, Profile URN, and target/mode allowlist match - the validated direct-listener or loopback-plus-private-proxy topology. - -- F2. **Acquire or reuse repositories and execute one turn** - 1. Negotiate A2A version and the required extension, then validate the strict - request, one text Part, target, session mode/context/head reference, - repository-name/revision map, logical working-directory selector, workspace - access, result schema, deadline, and deployment-wide idempotency claim. - 2. In one SQLite transaction, create or replay the claim/Task, create or lock - the session/head when selected, reserve bytes, and acquire the execution - lease before starting work. Capacity failure settles the Task; `start` - closes its new session, while `resume` advances the lineage head to this - failed Task but preserves the prior committed checkpoint/workspace pair. - The matching terminal projection is reported; no Docker/provider launches. - 3. For a session resume, use its pinned base/runtime, committed workspace - generation when writable, and provider checkpoint. Otherwise, when every - effective revision is a full commit, derive the immutable-base key and pin - a matching validated cache entry. On a miss or for mutable branch/tag - revisions, select source credentials, probe Docker and the exact - digest-pinned acquisition image, and run acquisition with only private - staging, compiled request/policy, and selected credential. - 4. On acquisition, revoke any App token, destroy source credentials, validate - the manifest/staging on the host, and atomically promote it to a reusable - cache entry or non-reusable Task/session-owned base. A cache hit performs no - acquisition, Docker, or credential operation. - 5. For read-write sessions, reserve/materialize a per-turn candidate from the - base or committed generation. Resolve logical cwd in the read-only base, - one-shot view, or candidate; create Task/session-private runtime; run typed - preparation; and start/resume the adapter as a direct host process group - with an explicit environment and supported separated auth/state paths. - Validate structured results while capturing bounded live events. - 6. After direct-process settlement and cancellation escalation, collect - bounded truthful evidence. For a continuing session, verify/checkpoint the - provider handle and retain measured private state; otherwise remove private - runtime/view/non-reusable base and dispose the handle. Atomically settle the - Task, Artifacts, observed termination, cleanup/checkpoint, session head/ - close state, charges, and lease. A nonsettling process or uncertain - checkpoint/cleanup retains owned state and the lease, poisons the session, - makes readiness false, and stops admission pending verified reconciliation. - -- F3. **Acquire or reuse an OCI snapshot and execute one turn** - 1. Perform the same version/extension/session validation and atomic - claim+Task+session-head+lease transaction as F2. - 2. For a resume, use the pinned session base/runtime/view. Otherwise pin a - cache entry matching the named snapshot, manifest digest, workspace-manifest - digest, catalog/layout digest, and acquisition-contract version. On a miss, - probe Docker and the exact digest-pinned acquisition image, then start it - with the digest-pinned reference, staging mount, exact-host registry - credentials/CA material, and frozen network/archive policy. Pull and verify - the direct image manifest, workspace-manifest config, and distributable - layers; apply changesets in order; enforce all limits; and emit the typed - manifest. - 3. On a miss, remove the container and registry material, validate and publish - the immutable base on the host, or remove staging on every non-publication - path. Then select the one-shot or session-owned read-only/private writable - state and settle/checkpoint through the same bare-metal path as F2. Provider - execution never occurs in the acquisition container. - -- F4. **Cancel** - 1. Atomically persist cancellation intent if the Task remains cancelable. - 2. For acquisition, stop/remove the Docker container, source material, and - unpublished staging. For provider work, request graceful adapter abort, - then escalate to process-group `SIGTERM`/`SIGKILL` within bounded periods. - Preserve only observed bounded evidence; settle the Task according to the - winning intent; retain the session only if the adapter proves a consistent - checkpoint, otherwise poison it pending verified cleanup. - 3. Repeated cancellation while intent is pending does not re-signal work. - Cancellation after any terminal state returns A2A - `TaskNotCancelableError`. - -- F5. **Shut down** - 1. Stop new admission before signaling active work. - 2. Persist shutdown intent, remove active acquisition Docker work or escalate - the direct provider process group, collect evidence only after the direct - provider settles, and settle the accepted Task once. - 3. Exit after the bounded settlement and cleanup path. Document that CI runner - teardown is the final orphan boundary and that gateway shutdown does not - prove every model-tool descendant is gone. - -- F6. **Invoke from Promptfoo** - 1. Promptfoo constructs the AI Evals-owned TypeScript provider with - `ProviderOptions`; the provider retains the ID and validates - `options.config` containing the private endpoint, target, optional default - logical working directory/access, and one closed source-mode object. - 2. `callApi(prompt, context?, options?)` applies only valid source/cwd/access - replacements and either strict `allagentsSession` or - `allagentsConversation`. The latter resolves start/continue/close through - the provider's conversation-ID map. The call creates and retains one - high-entropy invocation key per turn and sends one A2A Message with - `configuration.returnImmediately: true`; resume carries the stored context - ID and exact prior head Task reference. - 3. After receiving the Task/context IDs, subscribe to terminal updates. - Resolve a terminal-before-subscribe or disconnected-stream race through - `GetTask` and bounded resubscription. Deadline/abort sends `CancelTask` once - with a fresh cleanup signal. - 4. Validate the terminal Sessions projection and update/remove the selected - conversation-map entry before returning. Put terminal text or structured - output in `ProviderResponse.output`; map `inputTokens -> prompt`, - `outputTokens -> completion`, `cachedInputTokens -> cached`, and - `totalTokens -> total`; and put other usage plus Task/session, Artifact, - logical source, termination, cleanup, and stable failure facts in - `metadata`. Admission/terminal failure returns a safe `error`. - 5. A two-turn fixture uses one explicit conversation ID with `start`, then - `close`; the provider turns the second call into resume with the first - terminal projection's exact context/head. Interleaved fixtures use distinct - IDs. Explicit session IDs support recovery across provider-process loss. - Cached-token usage is observation, never a guaranteed hit. +#### F1. Start the deployment + +1. Start `codex-lb` with provider OAuth, proxy API-key authentication enabled, + and an API key held only by the HarnessRouter gateway. Validate that the + key/model combination authorizes each configured Codex model. +2. Run the AllAgents materializer's bounded `preflight` mode. It validates the + project catalog, snapshot and credential-reference schemas, referenced secret + presence, required Git/OCI tools, hook contract version, and staging/workspace + filesystem relationship without contacting sources. +3. Start the pinned custom HarnessRouter image with durable `/data`, private + listener, HarnessRouter client key, broker mode, matching private + `HARNESS_PUBLIC_BASE_URL` and runner-reachable `HARNESS_GATEWAY_URL`, Codex + `openai`/OpenAI-auth provider fields, materializer command, project + configuration, and allowlisted source secret environment. +4. From the exact container network, verify configured harness IDs/model + allowlists, broker credential minting, authenticated `codex-lb` `/responses` + and `/responses/compact`, a resumed Codex turn, safe roots, and materializer + version. Advertise Pi only if its separate live format/endpoint probe passed. + Any required preflight failure prevents readiness. + +#### F2. Execute the first repository-backed turn + +1. Promptfoo sends one authenticated UHP request with `model`, stock + `metadata.harness_id`, idempotency input, and the AllAgents workspace object. +2. HarnessRouter validates UHP plus generic metadata bounds, creates the + response/session, CASes materialization from `unbound` to `materializing`, and + hydrates a fresh session workspace. +3. Before provider selection, the gateway calls runner `/materialize`. The + AllAgents child validates the descriptor and catalog, resolves exact commits + and source credentials, writes and validates staging, removes credential + state, and returns provenance without publishing. +4. The runner independently validates the result/tree, publishes staging, + writes its marker, initializes the root checkpoint plus each declared + repository's collection cursor, and returns `published`. The gateway stores a + durable checkpoint and CASes the session to `ready`. +5. HarnessRouter applies ordinary input files, resolves the safe nested cwd, and + enters its provider loop. Codex receives only a turn broker credential and + reaches `codex-lb` through HarnessRouter; materialization cannot rerun during + provider fallback. +6. Normal UHP events and every stored/retrieved terminal response include the + same namespaced provenance. Produced-file collection walks the HarnessRouter + root plus each declared nested repository without reporting initial source + files as agent output. + +#### F3. Continue the session + +1. The caller sends `previous_response_id` and omits + `metadata["allagents.workspace"]`. +2. HarnessRouter resolves its current session state and writable workspace, + requires materialization `ready`, and does not invoke the hook again. An + extension-bearing continuation, concurrent active turn, or non-resumable + session fails before runner work. +3. HarnessRouter follows its stock predecessor/session semantics, resumes the + native conversation, and returns pinned provenance plus new output, usage, + and artifacts. AllAgents does not add a stricter head CAS. + +#### F4. Execute an OCI-backed first turn + +1. The caller selects one configured snapshot and immutable manifest/workspace + digests; it never sends the registry origin or credential. +2. The materializer fetches and verifies the direct manifest, config, and layers, + applies changesets under fixed limits, validates the declared repository + layout in staging, and returns exact provenance. +3. The runner publishes and checkpoints through the same state machine as F2. + Any registry, digest, media, path, limit, or layout failure removes staging, + terminalizes the response, and enters neither Git nor provider fallback. + +#### F5. Cancel, fail, or restart + +1. Materializer cancellation or deadline terminates the child and removes + staging. The session becomes `failed` and non-resumable; no provider starts. +2. Agent cancellation and deadline use HarnessRouter's normal UHP lifecycle. +3. Startup reconciles a `materializing` session to `ready` only when the bound + descriptor, published workspace marker, and durable checkpoint all match. + Otherwise it marks the session failed/non-resumable and removes or quarantines + the workspace. It never replays acquisition. A completed `ready` session + rehydrates from its durable checkpoint. +4. Whole-container termination does not preserve the in-flight agent process. + Interrupted agent turns fail according to HarnessRouter behavior. ### Acceptance Examples -- AE1. A caller on permitted Tailscale/private routing discovers either a native - specific-private-address TLS listener or a private HTTPS terminator whose - backend is loopback-only, selects `codex-review`, and receives one durable Task - without an application credential. Startup fixtures reject wildcard/public - binds, a private direct bind without TLS key/cert, a public URL literal, a DNS - name with any public address, and an `http:` remote advertised URL before - listening. Contract URNs are never fetched as endpoints. -- AE2. Any reachable caller can list, retrieve, and cancel a Task created by - another reachable caller and inspect its embedded Artifacts through `GetTask` - or `ListTasks(includeArtifacts: true)`; documentation states this shared trust - model without implying tenant privacy. -- AE3. A launcher-bearing Codex profile without `gateway.enabled: true` is - absent from discovery and rejected when selected. An enabled but drifted - profile fails readiness/new admission. -- AE4. A multi-client profile gateway-enables `codex-review` and `pi-review` as - distinct targets. Both resolve through adapters; neither generated wrapper is - executed. -- AE5. Repository mode accepts declared names and revision overrides, rejects an - undeclared name or URL override, and records the resolved full commits. -- AE6. On a base-acquisition miss, including a mutable branch/tag request, an - applicable GitHub App bypasses its token cache, persists a conservative-expiry - pre-mint intent, mints a repository-scoped read-only token with adequate - lifetime, replaces the intent with the exact expiry, validates the token, and - revokes it after acquisition. A cache hit resolves no source credential. - A corroborated existing repository with no applicable installation uses the - configured `gh` account. An uncorroborated 404, unknown applicability, auth, - mint, validation, or revocation failure does not fall through to `gh` or start - the provider. Process-kill barriers before the mint request, after an - ambiguous/successful mint response, and before revocation confirmation remove - the acquisition container and local secret on restart, fail the Task with - `gateway_restarted`, persist the conservative- or exact-expiry tombstone, and - never report confirmed revocation. -- AE7. Snapshot mode accepts a direct image manifest with matching manifest, - config/workspace, and layer digests; applies gzip/zstd layers and whiteouts in - order; and enforces every fixed limit. Same-origin metadata redirects work; - only layer requests may cross origin to an exact declared host, with - credentials stripped and every resolved address checked. Mutable tags, - indexes, unknown/non-distributable media, descriptor URLs/data, traversal, - foreign layers, digest/size mismatch, malformed whiteouts, undeclared - repositories, redirect loops/rebinding, non-global destinations not declared - for that source, and unapproved origins fail. -- AE8. Repository and snapshot modes produce the same path-free wire-visible - workspace-manifest shape and logical repository set. The gateway separately - validates the acquired base against the exact compiled private destinations. - OCI-contained commit identities are snapshot-attested unless independently - verified; source identity includes completeness and ordered layer digests - without origins. One hundred Tasks using the same immutable identity perform - one full acquisition while the entry remains cached and pinned correctly. - A branch or tag request acquires a non-reusable Task/session-owned base, never - enters the reusable cache, and removes that base during one-shot/closing - settlement or reconciliation. -- AE9. Identical invocation-key replay, including after a lost response, returns - the original Task. Reusing the key with changed target, source, prompt, logical - working directory, workspace access, result schema, session mode, context ID, - prior Task, or close-after-turn value conflicts. Separate one-shot read-only - Tasks may share one physical base while keeping private runtime. Separate - one-shot read-write Tasks receive independent views. A resumed read-write - session pins the same base/prior committed generation and exact head while - creating a new Task-specific candidate. -- AE10. Cancellation during Git/OCI acquisition stops and removes the container - and unpublished staging. Cancellation during Codex/Pi requests graceful abort, - then process-group `SIGTERM` and `SIGKILL` on schedule. The Task records - observed termination/cleanup without claiming descendant quiescence. Any - observed escaped/outliving descendant retains the lease until verified gone - or runner teardown. Cancellation leaves a non-closing session open only when - the adapter proves a consistent provider checkpoint paired with the unchanged - committed workspace generation; a closing session is removed after verified - cleanup. Otherwise it poisons the session. Uncertain acquisition cleanup, - checkpointing, candidate cleanup, or provider settlement retains the active - reservation, reconciliation record, affected Task/session state, and lease; - stops admission; stays unready/non-expiring across configured TTLs; and - reconciles before releasing the lease or restoring/closing the session. -- AE11. Kill fixtures before/after Task/session-head/lease creation, mutable - workspace candidate creation, durable pre-mint App intent, token response, - exact-expiry replacement, revocation confirmation, acquisition-container - start, provider process-group recording, provider checkpoint, prepared - settlement, workspace-generation pointer commit, old-generation cleanup, - terminal transaction, and response acknowledgment leave one recoverable - SQLite truth. Restart removes recorded containers/staging, terminates the - recorded process group, waits on any observed outliving descendant, persists - required revocation tombstones, and terminalizes the interrupted Task without - replay. It discards an uncommitted candidate or retains the database-selected - committed generation and restores only its provably unchanged provider - checkpoint; otherwise it poisons/retains the session. Lease/readiness remain - held/false until provider/observed-descendant absence, credential destruction, - tombstone durability, and owned-state reconciliation are confirmed. - Terminal Tasks/Artifacts remain until Task expiry independently of a live - session. -- AE12. A valid structured result survives later check or evidence failure as a - valid result with an overall failed Task; invalid or absent results are never - published as valid. -- AE13. A gateway-enabled launcher named `codex`, `pi`, or a portable case- - equivalent fails configuration compilation instead of shadowing a built-in - target. -- AE14. Two gateways for different workspaces use distinct private state roots; - a second process for the same root fails the exclusive lock. Wrong-owner, - permissive, linked, or overlapping roots fail startup. Ordinary Bun SQLite - transactions with foreign keys and `synchronous=FULL` recover a committed - Task/claim/lease generation after process-kill fixtures and never acknowledge - an uncommitted Task or publish false success; no custom VFS is required. -- AE15. Barrier-controlled provider-terminal, caller-cancel, deadline, and - shutdown races durably select one internal intent and one abort path during - Docker acquisition, preparation, Codex, Pi, or evidence. Normal settlement - writes the Task, three reserved Artifacts, produced Artifacts, bounded - evidence, result/failure, observed termination, cleanup, exact footprint, - terminal Artifact events, terminal status event, and lease release in one - transaction; replay after a crash includes that terminal event sequence. - Nonsettling-provider and uncertain cleanup/checkpoint cases retain the - reservation, reconciliation record, Task/session-owned state, and poisoned - lease; crossing configured Task/session TTL cannot delete either before - reconciliation. Verified repair atomically replaces the reservation with the - exact footprint, releases the lease, and restores a provable checkpoint or - closes the session. -- AE16. Repeated cancel while cancellation is pending is idempotent; cancel - after canceled, completed, failed, timed out, or rejected returns - `TaskNotCancelableError` without changing the Core `alreadyTerminal` outcome. -- AE17. A workspace containing `setup` shell entries never executes them during - acquisition or startup. The acquisition image receives only staging, - source-only credentials, exact source network policy, and archive limits; it - receives no host home, Docker socket, gateway state, provider auth, Codex, Pi, - or coding harness. Codex and Pi run afterward as direct host processes with - explicit environments that preserve required host identity/auth paths and - omit unrelated ambient values. -- AE18. Evidence collection starts only after the direct provider process has - settled and process-group escalation has completed. Git inspection disables - repository-controlled execution, and the workspace-integrity Artifact - distinguishes observed direct-process termination and cleanup from full - quiescence. Documentation explicitly states that AllAgents provides no - hostile-code or model-tool secret-isolation guarantee. -- AE19. The 1001st unexpired retained Task and any admission for which - `settledFootprints + activeReservations + maxTaskBytes` exceeds the aggregate - limit are rejected with `retention_capacity_exhausted`; no retained Task is - evicted before TTL. Boundary fixtures grow an accepted Task until its projected - event-plus-trajectory bytes reach the terminal-tail reserve, prove the next - event is not committed or published, then settle a maximal 1 MiB valid result - followed by workspace-cleanup failure without dropping that result or - exceeding the reservation. The transaction replaces the reservation with a - measured terminal footprint containing all mandatory Artifacts/events. - `ListTasks` stops before its exact serialized response budget, returns a cursor - to the first omitted Task, and never splits a Task. Invalid budget - relationships fail startup. While one Task holds the execution lease, a - barrier-controlled second request settles `execution_capacity_unavailable` - and launches no acquisition or provider child; races and restart never produce two lease holders. -- AE20. Official HTTP+JSON client fixtures send `A2A-Version: 1.0`, exercise - required-extension activation and both `SendMessage` modes, preserve unrelated - metadata, verify standard `google.rpc.Status` errors, and cover every - `ListTasks` filter, cursor, order, byte-budget, response-field, and artifact- - inclusion rule. They exercise oneShot/start/resume projection, server-generated - context IDs, exact prior-Task references, and immutable Task-per-turn behavior. - A terminal Task contains one Core outcome, one ordered Core execution - trajectory, one AllAgents workspace-integrity Artifact, and referenced - produced Artifacts using unified Parts. -- AE21. The AI Evals Promptfoo fixture has a top-level prompt and disables - sharing, Promptfoo result caching, result writes, and concurrency above one. - It loads both source modes, sends only closed logical inputs, retains one - invocation key across ambiguous retries, and cancels an accepted Task on - abort. One-shot read-only trials share a base; read-write trials use disposable - views. Two interleaved explicit conversation IDs each complete start then - close: the provider stores each terminal projection, sends that ID's exact - returned context/head on turn two, observes turn-one conversation/workspace - changes, and removes the map entry/session after close. A provider restart - resumes once through explicit `allagentsSession` IDs stored by the evaluator. - Native cached-input tokens are reported when present; no cache hit is required. - Safe failure metadata includes code, retryability, accepted Task ID, and - session state. Calls with - omitted context work; unknown variables, stale heads, changed pinned inputs, - invalid or escaping relative directories, physical paths, mutable revisions, - origins, destinations, materializer choices, or undeclared names fail before - provider execution. - -### Success Criteria - -- `allagents-gateway serve` starts from a real workspace with no deployment YAML. -- Loopback HTTP, loopback behind private HTTPS ingress, and native - specific-private-address TLS listeners work with the matching advertised URL; - wildcard/public binds and public URL resolution fail startup; probes reflect - admission. -- The official A2A client exercises version and required-profile negotiation, - both send modes, stream, get, complete list/pagination semantics, subscribe, - replay, cancel, terminal cancel errors, Task-embedded Artifacts, standard - HTTP+JSON errors, and expiry. Hand-authored HEC Core vectors independently - exercise the complete invocation DTO, neutral states and cancel dispositions, - the closed state/failure/result/retryability outcome matrix and negative - combinations, including retryable and non-retryable extension failures, - ordered progress/tool events, canonical structured payloads, deterministic - prefix truncation, exact outcome/trajectory descriptor linkage, - result/usage/outcome settlement, idempotency, deadlines, - cancellation/provider/deadline races, and portable failure retryability - through a transport-neutral test adapter and the A2A binding. Sessions vectors - independently exercise oneShot/start/resume/close, exact predecessor, - linearization, pinning, expiry, capacity, restart/poison handling, immutable - committed workspace generations, per-turn candidates, crash before/after - provider/workspace pointer commit, and truthful cache usage. Binding vectors - exercise session/context ID, prior-Task and terminal-session projection, Core- - to-A2A state/cancel/event/Artifact/error mapping, list/response budgets, replay, - and unknown-field rejection. Workspace/composed-profile vectors exercise - module composition, produced-Artifact bytes, integrity, and cleanup. -- An AI Evals-style Promptfoo custom-provider fixture consumes secure-default - YAML for both source modes, applies logical cwd/access per trial, proves - one-shot base/view behavior, isolates two interleaved explicit conversation - IDs, uses each stored terminal projection for exact resume, closes both, - recovers once through explicit evaluator-stored session IDs after provider - restart, propagates post-acceptance cancellation, reports provider-native - cached-input usage without guaranteeing a hit, and maps each terminal Task to - `ProviderResponse` without adding Promptfoo to the AllAgents runtime. -- Built-in Codex/Pi and gateway-enabled profile targets pass one backend - conformance suite, including reserved-ID collisions, host-auth/private-state - separation, explicit environments, per-mode advertisement, one-shot and - start/resume/checkpoint/dispose, cancellation escalation, native cache-usage - truthfulness, and Codex native-schema gating. -- Direct Git and OCI snapshot acquisition in the exact digest-pinned image - produces equivalent typed manifests and truthful provenance; repeated - immutable requests reuse one validated base and the image is removed before - provider execution. GitHub App eligibility, 404 ambiguity, unknown failure, - no-installation `gh` fallback, base-acquisition token validation/revocation, OCI - authentication/challenge handling, staging validation, and pre-provider - credential teardown are proven end to end. -- No request can supply a command, executable, URL, physical cwd, configured - destination, credential, mutable OCI tag, backend or materializer override, - arbitrary environment value, Docker option, image reference, or mount. -- SQLite crash/race/restart, acquisition-container cleanup, host provider - process-group cancellation, and truthful post-settlement evidence scenarios - pass without claiming complete descendant containment. -- The independently packaged Bun gateway passes a trusted-network smoke against - project and user workspaces under `/tmp/`; a CLI-only install fetches neither - the gateway package nor the acquisition image. +- **AE1.** A stock UHP request without the configured metadata key produces the + same response and conformance result on upstream HarnessRouter and the fork. +- **AE2.** An unauthenticated request is rejected. An authenticated Codex turn + reaches `codex-lb` while the literal HarnessRouter client key and long-lived + `codex-lb` key are absent from the agent environment and filesystem. +- **AE3.** Repository mode resolves configured branch/default/HEAD refs to full + commits, prepares every execution-eligible repository, preserves nested Git + history, starts in a validated nested cwd, and returns path-free provenance. +- **AE4.** HarnessRouter rejects non-object/oversized metadata and an extension + on continuation before the hook. The hook rejects unknown logical names, + caller-provided URLs, absolute/traversal paths, commands, environment fields, + credentials, originless/local repositories, and duplicate names/destinations + before source network access or agent launch. +- **AE5.** Two turns linked by `previous_response_id` preserve a file and native + conversation context. The hook runs once; produced-file collection reports + modifications inside every nested repository but not the initial source tree. +- **AE6.** A continuation omitting the extension succeeds. Any continuation + containing the configured workspace key is rejected without changing the + workspace. A new revision uses a new session. +- **AE7.** Explicit UHP input files overlay materialized paths after the + pre-agent checkpoint and before agent launch. +- **AE8.** A materializer crash, timeout, cancellation, malformed result, + publication crash, checkpoint failure, or Git failure starts no provider, + leaks no credential, and leaves the session failed/non-resumable rather than + partially ready. Provider fallback never reruns materialization. +- **AE9.** OCI mode accepts a valid digest-pinned fixture with gzip/zstd layers + and whiteouts and rejects mutable tags, indexes, mismatched digests/sizes, + traversal, escaping links, devices, sparse files, unknown media types, and + declared-limit overflow. +- **AE10.** Codex uses HarnessRouter's custom Responses integration and broker. + Pi is advertised only if its separate live custom-format probe passes. Failure + disables Pi without owner trust, direct-provider fallback, or another proxy. +- **AE11.** Restart after a completed first turn preserves the session and allows + continuation. Restart during materialization recovers only from a matching + published marker and durable checkpoint; otherwise it fails without automatic + replay. Restart during an agent turn follows HarnessRouter's interrupted-turn + failure behavior. +- **AE12.** The custom image records exact upstream, patch, materializer, Codex, + optional Pi, and image versions; rebuilding locked inputs produces equivalent + contract and conformance results. +- **AE13.** Promptfoo maps one stable materializer failure and one HarnessRouter + execution failure to failed `ProviderResponse` results with code, safe message, + and metadata; neither becomes a successful empty response. ### Scope Boundaries **In scope** -- The transport-neutral Harness Execution Contract Core and Sessions, their A2A - 1.0 HTTP+JSON binding, the AllAgents coding-workspace extension, and their one - required composed coding-execution Profile Extension. -- One gateway process and one active gateway-controlled turn at a time initially. -- Built-in and gateway-enabled profile-backed Codex/Pi host execution. -- Docker-only acquisition of direct declared Git repositories and named OCI - workspace snapshots when no reusable validated base exists. -- Reusable immutable bases with Task/session-private runtime state for read-only - execution; non-reusable Task/session-owned bases for mutable revisions; - disposable one-shot writable views and retained session-private writable - views for read-write execution. -- Logical workspace-root or declared-repository-relative provider cwd plus - explicit `readOnly | readWrite` access selected at runtime. -- GitHub App and configured GitHub CLI acquisition credentials. -- Local durable Task/session/evidence storage, bounded base caching, - process-group cancellation, checkpoint/workspace cleanup, and provenance. -- Loopback and trusted-private-network access through either loopback/private-TLS - ingress or a native specific-private-address TLS listener. +- HarnessRouter workspace-integration patch and generic materializer boundary. +- Versioned AllAgents workspace descriptor, hook request/result, and provenance. +- Project workspace schema additions and catalog projection. +- Deterministic Git and immutable OCI acquisition. +- Root/nested-repository checkpoint and produced-file integration. +- Source and provider credential containment with leak verification. +- Custom image build and pinned release metadata. +- HarnessRouter broker plus `codex-lb` Codex Responses configuration. +- Promptfoo contract examples and one-shot/two-turn success/failure E2E. +- Upstream-ready generic hook patch and maintenance procedure. **Out of scope** -- Application authentication, tenant isolation, caller-private Tasks, and all - public-Internet exposure. -- `gateway.yaml`, `worker.yaml`, remote workers, mTLS worker links, Kubernetes - routing, autoscaling, and multiple gateway replicas. -- Caller-provided physical workspaces/cwds, repository or registry origins, - mutable OCI tags, custom materializers, Dockerfiles, Compose files, or - acquisition commands. -- GitHub Enterprise Server and multiple ordered Apps/accounts in the initial - delivery. -- OpenCode, Claude, Copilot, OMP, arbitrary CLI, and TUI adapters. -- A Responses/UHP binding, session branching, and concurrent turns; version one - implements only linear resumable Sessions through A2A, while any second - transport or fork semantics remain future independently versioned work. -- Evaluation orchestration and automatic retries. -- Per-provider containers; cgroups, pidfds, namespaces, nftables, `openat2`, a - native platform layer, non-bypassable spawn mediation, hostile-code - containment, and secret isolation from model-invoked tools. -- Non-Linux gateway execution in v1; ordinary `allagents` CLI behavior remains - cross-platform. +- A2A, HEC, Agent Cards, or a second execution protocol. +- An `allagents-gateway` server, task database, session engine, process + supervisor, Codex SDK adapter, Pi RPC adapter, or artifact service. +- Promptfoo runtime code inside AllAgents. +- Provider OAuth handling outside `codex-lb`. +- Caller-provided origins, credentials, commands, host paths, materializers, or + Docker options. +- Public multi-tenancy, per-caller authorization, Kubernetes workers, session + branching, concurrent turns in one session, or guaranteed prompt-cache hits. +- Exact rollback of workspace mutations between successful session turns. ### Sources -- [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) -- [AHP decision inputs](../research/agent-host-protocol-decision-inputs.md) +- [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) +- [HarnessRouter repository](https://github.com/HarnessRouter/harnessrouter) +- [HarnessRouter self-hosting guide](https://github.com/HarnessRouter/harnessrouter/blob/main/docs/self-hosting-guide.md) +- [UHP 2026-09-12 architecture](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/architecture.md) +- [UHP sessions](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/sessions.md) +- [UHP lifecycle](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/lifecycle.md) +- [UHP files](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/files.md) +- [`codex-lb` repository](https://github.com/Soju06/codex-lb) +- [`codex-lb` client setup](https://github.com/Soju06/codex-lb/blob/main/docs/client-setup.md) +- [`codex-lb` API keys](https://github.com/Soju06/codex-lb/blob/main/docs/api-keys.md) +- [`codex-lb` routing](https://github.com/Soju06/codex-lb/blob/main/docs/routing.md) - [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) - [Source credential broker precedents](../research/source-credential-broker-precedents.md) -- [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) -- [A2A life of a Task and multi-turn contexts](https://a2a-protocol.org/v1.0.0/topics/life-of-a-task/) -- [A2A extension guide](https://a2a-protocol.org/latest/topics/extensions/) -- [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) -- [Unified Harness Protocol](https://unifiedharnessprotocol.org/) -- [UHP Sessions](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/sessions.md) -- [UHP governance](https://github.com/HarnessRouter/harnessrouter/blob/76c0d0a55682f953ca10c44bdd0645a4475a4ebd/protocol/GOVERNANCE.md) -- [UHP implementations](https://github.com/HarnessRouter/harnessrouter/blob/76c0d0a55682f953ca10c44bdd0645a4475a4ebd/protocol/IMPLEMENTATIONS.md) -- [Bun workspaces](https://bun.sh/docs/install/workspaces) -- [Bun SQLite](https://bun.sh/docs/api/sqlite) -- [Promptfoo custom providers](https://www.promptfoo.dev/docs/providers/custom-api/) -- [Promptfoo configuration reference](https://github.com/promptfoo/promptfoo/blob/main/site/docs/configuration/reference.md) -- [OpenAI Codex SDK thread continuation](https://developers.openai.com/codex/codex-sdk) -- [OpenAI Codex app-server thread/turn model](https://developers.openai.com/codex/app-server) -- [OpenAI conversation state](https://developers.openai.com/api/docs/guides/conversation-state) -- [OpenAI prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching) -- [OpenAI structured outputs](https://developers.openai.com/api/docs/guides/structured-outputs/) -- [GitHub App installation tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) -- [Git credential helpers](https://git-scm.com/docs/gitcredentials) -- [Docker credential stores](https://docs.docker.com/reference/cli/docker/login/#credential-stores) -- [OCI Image Specification](https://github.com/opencontainers/image-spec) -- [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec) -- [GitHub Container registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry) -- [JFrog Artifactory Docker repositories](https://jfrog.com/help/r/jfrog-artifactory-documentation/docker-repositories) -- [JFrog Container Registry image](https://hub.docker.com/r/jfrog/artifactory-jcr) -- [Docker OverlayFS storage driver](https://docs.docker.com/engine/storage/drivers/overlayfs-driver/) -- [Docker VFS copy fallback](https://docs.docker.com/engine/storage/drivers/vfs-driver/) -- [Windows ReFS block cloning](https://learn.microsoft.com/en-us/windows-server/storage/refs/block-cloning) -- [GitHub-hosted runners](https://docs.github.com/en/actions/reference/runners/github-hosted-runners) --- ## Planning Contract -### Key Technical Decisions - -- KTD1. **Gate the official A2A JavaScript SDK and the first Harness Execution - Contract binding in the shipped Bun server direction.** Pin the exact SDK - version and prove Agent Card discovery, both send modes, streaming, Task - get/list/cancel, resubscription, required-profile negotiation, metadata - preservation, and HTTP error envelopes by driving the production gateway - server with an independent official client fixture. Run the transport-neutral - Core semantic vectors against the A2A binding, then run binding-specific - one-Message/one-Task, state/event, schema, context-ID, Artifact, and standard - error conformance. Implement the SDK's public request-handler seam while - AllAgents owns UUIDv7 creation, atomic `createOrReplay`, monotonic settlement, - listing, retention, expiry, and HTTP+JSON error details. Do not use an SDK - default store as the transaction boundary, fork A2A core types, or expose a - second version-one wire protocol. -- KTD2. **Keep the harness contract transport-neutral inside one narrow - TypeScript package.** `packages/workspace-config` owns project/user parsing and - compiled catalogs. `packages/execution-contracts` owns the transport-neutral - Harness Execution Contract Core and Sessions types/semantic vectors, the A2A - binding, the AllAgents coding-workspace extension, composed request/Task/ - Artifact schemas, adapter contracts, and evidence schemas. Core types import - no Sessions, A2A, UHP, workspace, provider, or evaluator types. Sessions imports - Core identity/outcome types but no A2A types. The binding, Sessions, and - workspace groups compose one required v1 A2A Profile URN because the gateway - supports one-shot and resumable execution. A future binding must pass the same - Core and Sessions semantic vectors plus its own wire suite. - `packages/acquisition-contracts` owns acquisition requests, typed manifests, - OCI snapshot rules, and fixed limits. Check the normative Core, Sessions, - A2A-binding, and workspace-extension text, generated JSON Schemas, - hand-authored conformance vectors, and golden accepted/rejected examples into - `contracts/` for the host gateway, acquirer image, public docs, and consumer - fixtures. Do not create a generic `core`/`common` package, a second v1 wire - endpoint, or a speculative shared package. -- KTD3. **Use one gateway supervisor, not a remote worker protocol.** The Bun - gateway owns Task state, immutable-base caching, Task runtime/view - materialization, provider child processes, evidence, termination, and cleanup. - It creates an ephemeral Docker container only when no reusable validated base - exists, removes it before provider execution, and launches Codex/Pi directly - on the trusted host in Linux process groups. -- KTD4. **Make application authentication intentionally absent.** All Tasks, - sessions, and Artifacts share one deployment namespace. Remote service uses - loopback behind private TLS ingress or native TLS on one specific private - address; wildcard/public listeners and public exposure are prohibited. Bind - and advertised interface URL remain distinct. (session-settled: user-directed.) -- KTD5. **Compile gateway configuration from existing workspace files.** Add - `workspaceSnapshots` to the project schema and `gateway.enabled` to strict - profile-client schemas. A gateway-only compiler normalizes the project - repository catalog and resolves each public launcher ID to one profile/client. - Add no deployment YAML. (session-settled: user-directed.) -- KTD6. **Keep source, cwd, and access input logical and closed.** Repository - requests carry only declared-name revisions; snapshot requests carry only a - declared snapshot name and immutable digests. Working-directory requests - select only the effective workspace root or a declared repository plus a - bounded relative directory. `workspaceAccess` is exactly `readOnly` or - `readWrite`. One-shot read-only Tasks may share the immutable base and one-shot - read-write Tasks receive Task-ID-derived views; sessions pin the base and - retain session-private runtime/view state between turns. The gateway never - accepts or returns a caller path or materializer choice. Include canonical - source identity, logical cwd, access, and session projection in idempotency and - provenance. -- KTD7. **Freeze Docker-only base acquisition.** Build `apps/acquirer` once as a - multi-architecture GHCR image and select it by manifest digest. Each request - without a reusable validated base starts a fresh container with one staging - bind mount, a typed acquisition request, source-only credentials, strict - source network policy, fixed size/archive limits, no host home, and no Docker - socket. - The image owns hermetic Git plus the minimal OCI Distribution client - and implements only the v1 direct-image manifest/config/layer profile, - RFC 8785 workspace-manifest config, explicit authentication/redirect rules, - streaming digest checks, and changeset application. It emits a typed manifest, - exits, and is removed before host validation/publication. It contains and - downloads no Codex, Pi, or other coding harness. A deterministic producer - fixture freezes the format. -- KTD8. **Select GitHub credentials by provable three-way eligibility.** On a - base-acquisition miss, App lookup 200 is eligible; 404 is ineligible only with - independent repository-existence proof; all ambiguous outcomes are unknown. - Fresh App tokens bypass credential cache, are validated and revoked, and only - positive ineligibility permits the configured `gh` account. The selected token - enters only the acquisition container; base-cache hits resolve no credential. - (session-settled: user-directed.) -- KTD9. **Keep one behavior-focused `codex | pi` adapter registry.** Direct - targets and gateway-enabled profile targets resolve to the same narrow - AllAgents-owned TypeScript adapter and one-shot/Session conformance suite; - profile context modifies server-owned configuration, never public argv. Codex - uses pinned `@openai/codex-sdk` `startThread`/`resumeThread` first; app-server - is allowed only for a proven required SDK gap. Pi uses a pinned supported - package/RPC surface and advertises Session capability only after exact - create/resume/checkpoint/dispose probes. Neither adapter downloads runtimes per - request, replays transcript text as fake resumption, or adopts AI SDK - Harnesses. A global binary override requires an exact compatibility probe. -- KTD10. **Keep durable Task and Session truth inside ordinary Bun SQLite - ownership.** The gateway holds the process-lifetime `bun:sqlite` connection, - private state root, and exclusive lock. SQLite uses foreign keys, - transactional `createOrReplay`/session-head/lease/settlement/checkpoint/expiry - operations, WAL where supported, and `synchronous=FULL`; acknowledge only - committed state. Claims, Tasks, Sessions, turns, events, opaque provider - checkpoint handles, pinned workspace/config digests, session charges, bounded - Artifact bytes, execution lease, acquisition-container/staging/transient-base - identity, provider process-group identity, internal outcome intent, and expiry - live in tables. Startup integrity or durability failure stops admission and - prevents false success. Do not build a custom VFS or native file layer. -- KTD11. **Treat the trusted Linux CI job as the provider isolation boundary.** - The gateway uses Docker only when no reusable validated base exists. Codex and - Pi run bare metal with the same CI-job authority as the gateway and existing - host - auth. Read-only is a consumer-selected cooperative contract with private - runtime state, optional-lock suppression, and native provider policy where - available; it is not hostile-code containment. - Construct provider environments explicitly to preserve required identity/auth - paths while omitting unrelated ambient values, but do not claim this protects - secrets from model-invoked tools. Linux cancellation is adapter abort, then - process-group `SIGTERM`, then `SIGKILL`; runner teardown is the final orphan - boundary. Do not add cgroups, pidfds, `openat2`, namespaces, nftables, native - containment packages, per-provider Docker, or spawn mediation. -- KTD12. **Capture live events, then collect bounded evidence after the direct - provider settles.** Evidence retains bounded source, Git, provider, result, - Artifact, observed termination, and cleanup facts. Git inspection disables - repository-controlled execution. Evidence and docs must not turn process- - group termination into a claim that all descendants are quiescent or that - model-tool output is redacted. -- KTD13. **Use one private Bun workspace without coupling releases.** The root - package is private orchestration. `apps/cli` publishes `allagents`; - `apps/gateway` publishes `allagents-gateway`; `apps/acquirer` is never - published to npm and ships only as a digest-pinned multi-architecture GHCR - image. Shared packages are limited to `packages/workspace-config`, - `packages/execution-contracts`, and `packages/acquisition-contracts`; - generated portable fixtures live under `contracts/`. - - CLI and gateway have independent versions, tags, changelogs, triggers, npm - tarballs, and release jobs. A CLI-only install resolves neither the gateway nor - the acquisition image. A gateway release first builds the acquisition image - once for the exact commit, resolves and records its multi-architecture - manifest plus supported platform digests, runs package and registry checks - against those exact immutable artifacts, and only then publishes the exact - `allagents-gateway` npm tarball. A gateway-only release never publishes - `allagents`; no Rust, Cargo, native binary, or platform npm package exists. -- KTD14. **Use tiered OCI registry conformance bound to exact release - artifacts.** Every pull request runs a local Distribution fixture and a live - public digest-pinned GHCR snapshot pull through the exact acquirer image. A - reusable release workflow adds authenticated least-privilege GHCR and pinned - private-CA JFrog Artifactory/JCR coverage. - - The callable workflow receives the exact gateway npm tarball, acquisition - multi-architecture manifest digest, per-platform image digests where the - registry supports them, build commit, and expected compatibility output; it - never rebuilds either artifact. Reports record the tested commit, npm tarball - digest, acquisition manifest/platform digests, architecture, image/registry - identity, auth mode, snapshot descriptor digests, and compatibility output, - including partial evidence on red paths. They cover valid anonymous and - authenticated pulls plus wrong credentials, insufficient permissions, digest - mismatch, missing/wrong CA, invalid media, and repository-path failures. The - gateway release must verify GHCR and JFrog against those exact artifacts - before npm publication; the JFrog target need not run on every pull request. - -### Package compatibility contract - -`allagents-gateway compatibility --format json` emits one strict, versioned -object containing `product: "allagents-gateway"`, `gatewayVersion`, -`buildCommit`, `runtime: "bun"`, the pinned acquisition image repository and -multi-architecture manifest digest, supported acquisition platforms/digests, -and supported Harness Execution Contract Core, Sessions, A2A binding/profile, -coding-workspace extension, workspace, execution-contract, -acquisition-contract, and snapshot versions. The packed npm tarball, -clean-install smoke, registry workflow, and release workflow consume this same -object. - -An optional `allagents gateway ...` dispatcher locates but never installs the -separate gateway. It accepts independent CLI and gateway versions only when the -product identity and required contract-version ranges intersect; otherwise it -prints a clear install/upgrade error and does not start the service. The gateway -rejects an acquisition image whose manifest digest, platform digest, build -identity, or acquisition-contract version differs from its release metadata. -Golden fixtures cover exact matches, supported CLI/gateway version skew, -unsupported contract versions, wrong image manifests/platforms, divergent npm -tarball or image build identities, and newest/oldest supported pairs. There are -no platform npm packages or native-binary compatibility checks. - ### High-Level Technical Design ```mermaid flowchart TB - ER[Evaluation runner] --> AB[A2A binding] - CP[Chat platform] --> AB - AG[Another agent] --> AB - AB --> HC[Harness Execution Contract Core] - AB --> HS[Harness Execution Contract Sessions] - AB --> CW[AllAgents coding-workspace extension] - HC --> G[Bun gateway host process] - HS --> G - CW --> G - G --> S[Bun SQLite Task and session store] - G --> W[workspace-config compiler] - W --> PW[Project workspace.yaml] - W --> UW[User workspace.yaml] - G --> BL[Reusable immutable-base lookup] - BL -->|hit| RB[Validated reusable base and pin] - BL -->|miss or mutable revision| D[Docker acquisition coordinator] - D --> A[Digest-pinned acquirer container] - A --> Git[Declared Git repositories] - A --> OCI[Named OCI snapshot] - A --> ST[Staging plus typed manifest] - ST --> V[Host validation and atomic base promotion] - V -->|exact identity| RB - V -->|mutable revision| TB[Task or session-owned transient base] - RB --> RO[Read-only base plus private runtime] - TB --> RO - RB --> M[Block clone or rootless OverlayFS or copy] - TB --> M - M --> RW[Task or session-owned writable view] - RO --> WD[Logical cwd resolver] - RW --> WD - WD --> R[Closed host adapter registry] - R --> Codex[Pinned Codex SDK thread] - R --> Pi[Pinned Pi RPC/package session] - Codex --> PG[Linux provider process group per turn] - Pi --> PG - PG --> E[Direct-process settlement then bounded evidence] - E --> C[Checkpoint session or remove one-shot/session state] - C --> S + PF[Promptfoo provider] -->|UHP + HR API key| GW[HarnessRouter gateway] + GW -->|first-turn materialize| RUN[HarnessRouter runner] + RUN -->|opaque JSON in, typed envelope out| MAT[AllAgents materializer] + MAT --> CFG[project workspace.yaml] + MAT --> GIT[Git sources] + MAT --> OCI[OCI registry] + RUN -->|turn broker credential| GW + GW -->|codex-lb API key| LB[codex-lb Responses API] + LB -->|OAuth + routing| MODEL[Model provider] + GW --> DATA[(durable session checkpoints)] ``` -### Configuration Contract - -No `gateway.yaml` or `worker.yaml` is introduced. - -**CLI flags and environment** - -| Concern | CLI | Environment | Default | -|---|---|---|---| -| Listener | `--listen` | `ALLAGENTS_GATEWAY_LISTEN` | `127.0.0.1:4732`; IP literal only; no wildcard | -| Advertised interface URL | `--advertise-url` | `ALLAGENTS_GATEWAY_ADVERTISE_URL` | `http://127.0.0.1:4732` only with the default listener; otherwise required | -| Native TLS certificate | `--tls-cert-file` | `ALLAGENTS_GATEWAY_TLS_CERT_FILE` | unset; required with key for a specific non-loopback listener | -| Native TLS private key | `--tls-key-file` | `ALLAGENTS_GATEWAY_TLS_KEY_FILE` | unset; required with certificate for a specific non-loopback listener | -| Project workspace | `--workspace` | `ALLAGENTS_GATEWAY_WORKSPACE` | cwd | -| State directory | `--state-dir` | `ALLAGENTS_GATEWAY_STATE_DIR` | `~/.allagents/gateway/` | -| Invocation workspace root | `--invocation-root` | `ALLAGENTS_GATEWAY_INVOCATION_ROOT` | `~/.allagents/gateway-workspaces/` | -| Immutable-base cache root | `--base-cache-dir` | `ALLAGENTS_GATEWAY_BASE_CACHE_DIR` | `~/.allagents/gateway-cache/` | -| Immutable-base cache budget | `--base-cache-max-bytes` | `ALLAGENTS_GATEWAY_BASE_CACHE_MAX_BYTES` | `64GiB` | -| Workspace materializer | `--workspace-materializer` | `ALLAGENTS_GATEWAY_WORKSPACE_MATERIALIZER` | `auto` (`auto | cow | copy`) | -| Automatic copy ceiling | `--max-auto-copy-bytes` | `ALLAGENTS_GATEWAY_MAX_AUTO_COPY_BYTES` | `1GiB` | -| Terminal Task TTL | `--task-ttl` | `ALLAGENTS_GATEWAY_TASK_TTL` | `24h` | -| Retained Task limit | `--max-retained-tasks` | `ALLAGENTS_GATEWAY_MAX_RETAINED_TASKS` | `1000` | -| Idle session TTL | `--session-ttl` | `ALLAGENTS_GATEWAY_SESSION_TTL` | `24h`, refreshed after each committed turn | -| Retained session limit | `--max-retained-sessions` | `ALLAGENTS_GATEWAY_MAX_RETAINED_SESSIONS` | `100` | -| Per-session live bytes | `--max-session-bytes` | `ALLAGENTS_GATEWAY_MAX_SESSION_BYTES` | `10GiB` | -| Aggregate retained session bytes | `--max-retained-session-bytes` | `ALLAGENTS_GATEWAY_MAX_RETAINED_SESSION_BYTES` | `100GiB` | -| Per-Task retained bytes | `--max-task-bytes` | `ALLAGENTS_GATEWAY_MAX_TASK_BYTES` | `64MiB` | -| Aggregate retained bytes | `--max-retained-bytes` | `ALLAGENTS_GATEWAY_MAX_RETAINED_BYTES` | `1GiB` | -| Serialized response bytes | `--max-response-bytes` | `ALLAGENTS_GATEWAY_MAX_RESPONSE_BYTES` | `96MiB` | -| Acquisition image | `--acquisition-image` | `ALLAGENTS_GATEWAY_ACQUISITION_IMAGE` | release-embedded `ghcr.io/.../allagents-acquirer@sha256:` | -| Docker endpoint | `--docker-host` | `ALLAGENTS_GATEWAY_DOCKER_HOST` | existing local Docker context/socket | -| Docker acquisition network | `--acquisition-network` | `ALLAGENTS_GATEWAY_ACQUISITION_NETWORK` | release-documented acquisition-only network | -| Acquisition timeout | `--acquisition-timeout` | `ALLAGENTS_GATEWAY_ACQUISITION_TIMEOUT` | `900s`, capped by remaining Task deadline | -| GitHub App ID | `--github-app-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_ID` | unset | -| App private key file | `--github-app-private-key-file` | `ALLAGENTS_GATEWAY_GITHUB_APP_PRIVATE_KEY_FILE` | unset | -| App installation ID | `--github-app-installation-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_INSTALLATION_ID` | discovered/unset | -| GitHub App mint request timeout | `--github-app-mint-timeout` | `ALLAGENTS_GATEWAY_GITHUB_APP_MINT_TIMEOUT` | `30s` | -| GitHub CLI account | `--github-cli-account` | `ALLAGENTS_GATEWAY_GITHUB_CLI_ACCOUNT` | unset | -| OCI auth file | `--oci-auth-file` | `ALLAGENTS_GATEWAY_OCI_AUTH_FILE` | unset | -| OCI credential helper | `--oci-credential-helper` | `ALLAGENTS_GATEWAY_OCI_CREDENTIAL_HELPER` | unset | -| OCI CA bundle map | `--oci-ca-bundle-map` | `ALLAGENTS_GATEWAY_OCI_CA_BUNDLE_MAP` | system roots only | -| Codex host auth location | `--codex-auth-home` | `ALLAGENTS_GATEWAY_CODEX_AUTH_HOME`, then `CODEX_HOME` | existing host location; usable only through a proven separate auth input | -| Codex binary override | `--codex-bin` | `ALLAGENTS_GATEWAY_CODEX_BIN` | pinned SDK-managed surface; unset | -| Pi host auth location | `--pi-auth-home` | `ALLAGENTS_GATEWAY_PI_AUTH_HOME` | unset; usable only through a proven separate auth input | -| Pi binary override | `--pi-bin` | `ALLAGENTS_GATEWAY_PI_BIN` | pinned package/RPC surface; unset | -| Graceful abort period | `--abort-grace` | `ALLAGENTS_GATEWAY_ABORT_GRACE` | `10s` | -| SIGTERM period | `--term-grace` | `ALLAGENTS_GATEWAY_TERM_GRACE` | `10s` | -| Final cleanup period | `--cleanup-timeout` | `ALLAGENTS_GATEWAY_CLEANUP_TIMEOUT` | `30s` | - -`--max-task-bytes` is valid from `8MiB` through `512MiB` and must exceed the -generated `terminalTailReserveBytes`. `--max-retained-bytes` must be at least -`--max-task-bytes`, and `--max-response-bytes` must be at least -`--max-task-bytes`; startup rejects any other relationship. The per-Task -footprint measures the exact serialized one-Task response including its -envelope, base64, and JSON escaping. A page adds Tasks only while exact -production serialization remains within `--max-response-bytes`. -`--max-session-bytes` measures provider state, private runtime, pinned -non-reusable base, committed workspace generation, and any reserved candidate/ -superseded generation. Admission reserves the projected candidate before -materialization; an over-limit turn creates no provider process. -`--max-retained-session-bytes` is at least `--max-session-bytes`. Session expiry -uses the same verified cleanup/poison rule as `closeAfterTurn`; it never silently -abandons retained state. -The GitHub App mint request timeout is 1-120 seconds and is part of the pre-mint -conservative-expiry calculation. - -Precedence is CLI over gateway-specific environment over provider-standard -environment over default. `--codex-auth-home`, -`ALLAGENTS_GATEWAY_CODEX_AUTH_HOME`, then ordinary `CODEX_HOME` select only the -host auth source; the child process receives a Task/session-private mutable state -home, never that auth path as its writable provider home. The advertised value -is the absolute URL placed in `AgentCard.supportedInterfaces`. `--listen` accepts -one IP literal plus port and rejects `0.0.0.0`, `::`, and public addresses. -Specific non-loopback listeners require both readable TLS files and private -HTTPS advertised resolution; the gateway itself serves TLS. A loopback listener -may advertise matching loopback HTTP or private HTTPS through an operator TLS -terminator whose only backend is that loopback socket. Startup rejects public -URL literals, any public DNS answer, unresolved hosts, mismatched TLS options, -and remote `http:` URLs. TLS inputs must be readable regular PEM files; the -private key is current-user owned and not group/world accessible, the certificate -and key must match, and both parse before binding. The acquisition image must be -a full -`repository@sha256:` reference; tags are rejected. -The gateway verifies that the local platform resolves to the release-recorded -platform digest before starting acquisition. - -Docker is a base-acquisition dependency only when no reusable validated base -exists. The configured endpoint must support creating, waiting for, stopping, -and removing a container plus bind-mounting gateway-created staging. A validated -cache hit does not contact Docker or resolve a source credential. The gateway -never passes the -Docker socket into the container. The acquisition network is preconfigured by -the operator to reach only declared Git/OCI source hosts and required auth/ -redirect hosts; the gateway supplies the stricter per-request host policy to the -acquirer. No Docker flag, mount, network, image, or environment override is -accepted from A2A. - -Credential and CA paths are resolved on the trusted host, must be current-user -owned regular files with private permissions, and are read only for acquisition. -Setting both OCI credential options is a startup error. `--oci-auth-file` -accepts at most 1 MiB of strict UTF-8 Docker-config JSON containing only -`auths`; each exact registry key contains one bounded `auth` or -`identitytoken`. `credsStore`, `credHelpers`, proxy/plugin fields, commands, -duplicate keys, and unknown members are rejected. - -The fixed OCI helper receives argv `[helperPath, "get"]` without a shell and the -raw exact Docker lookup key on stdin. Exit-zero stdout is one bounded strict JSON -object with nonempty `Username` and `Secret` plus optional matching `ServerURL`. -Timeout, nonzero exit, signal, malformed output, mismatch, or empty credentials -fails with `source_auth_oci_failed`; stderr is secret-bearing and never logged. - -`--oci-ca-bundle-map` names a bounded strict JSON file mapping exact normalized -`host[:port]` keys to private PEM CA files. Only the bundle for the exact -registry, token service, or declared layer-redirect host augments system roots; -there is no insecure-TLS switch. Registry access begins anonymously and accepts -only bounded same-origin Basic or Distribution Bearer behavior plus the -documented Docker Hub token service. Cross-origin redirects remain limited to -layer `GET`/`HEAD` requests for exact declared hosts, with credentials stripped -and every hop checked. The gateway passes only the selected source credential -and exact CA material into the acquisition container and destroys both before -provider execution. - -Provider auth locations are never copied, mounted into Docker, parsed by -AllAgents, imported into another store, or used as the mutable provider-state -home. The direct Codex/Pi process receives a private state root and only the -pinned adapter's supported separate auth reference. Binary overrides are -absolute host paths and must pass the pinned adapter's -exact version/protocol probe at readiness; they are not request-selectable. -The explicit provider environment starts from an allowlist rather than the -gateway's complete environment, but this is leakage reduction, not isolation. - -The immutable-base cache and invocation roots are current-user owned, private, -and disjoint from state, project, profile, provider-auth, and each other. -Acquisition writes a unique directory under `/.staging`; host -validation completes before an atomic same-filesystem rename to the final -cache-key directory, `/transient/tasks/`, or -`/transient/sessions/`. Active Task/session -references pin reusable entries. Least-recently-used eviction enforces the byte -budget and removes only unpinned reusable bases. Every non-publication path -removes its staging directory, and startup reconciles orphan staging and -recorded transient bases before readiness. - -One-shot state lives only at -`/tasks//{runtime,workspace?}`. Retained session state -lives only at -`/sessions//{runtime,workspaces,candidates}`. -Read-only cwd resolves in the pinned -reusable or non-reusable base. `auto` probes same-filesystem block clone first, -then rootless OverlayFS on Linux, then ordinary copy only when the base does not -exceed `--max-auto-copy-bytes`. `cow` requires block clone or rootless OverlayFS -and fails readiness when neither is available. `copy` is the explicit portable, -higher-I/O backend and may exceed the automatic copy ceiling. It does not by -itself make the v1 gateway available on Windows; process lifecycle -and cancellation remain Linux-only in this plan. Startup logs the selected -capabilities without paths. No mode uses writable hard links. Startup rejects -overlapping roots and stale mounts it cannot safely reconcile. - -The derived workspace ID is a stable digest of the canonical project-workspace -path and is verified against SQLite metadata. Retention includes Task records, -Artifact bytes, events, and invocation-key claims; expiry is transactional. Task -expiry does not evict a pinned base, and base eviction does not remove retained -Task metadata. When the unexpired Task-count limit is reached, new admission -fails rather than evicting retained Tasks. - -**Project workspace additions** - -```yaml -repositories: - - name: allagents - source: https://github.com/EntityProcess/allagents.git - path: allagents - branch: main - -workspaceSnapshots: - evaluation: - repository: ghcr.io/entityprocess/allagents-workspaces - layerRedirectHosts: - - pkg-containers.githubusercontent.com - enterprise: - repository: company.jfrog.io/docker-local/allagents-workspaces +The gateway owns generic metadata bounds, session materialization state, +provider-loop ordering, checkpoint persistence, and response metadata. The +runner owns hook invocation, staged publication, checkpoint/collection setup, +safe nested cwd, and agent launch. The materializer owns only AllAgents schema, +catalog, acquisition, credential selection, staging validation, and provenance; +it never speaks UHP or publishes the live workspace. + +### Extension Contract + +Initial UHP request fragment: + +```json +{ + "model": "gpt-5.6-sol", + "input": "Review the service", + "metadata": { + "harness_id": "codex-review", + "allagents.workspace": { + "version": "1", + "source": { + "kind": "repositories", + "revisions": { + "api": "refs/pull/123/head" + } + }, + "workingDirectory": { + "kind": "repository", + "repository": "api", + "path": "packages/service" + } + } + } +} ``` -Snapshot names use the portable profile-name vocabulary. Repositories must have -unique stable names for remote acquisition. Non-Docker-Hub repository values -contain only an exact registry `host[:port]/repository-path` identity and an -optional exact `layerRedirectHosts` allowlist; never tags, digests, credentials, -or extraction paths. - -Docker Hub uses only the canonical declaration -`docker.io//` with an explicit namespace. The gateway -maps that declaration to API origin `https://registry-1.docker.io`, Docker -credential lookup key `https://index.docker.io/v1/`, Bearer service -`registry.docker.io`, and token realm `https://auth.docker.io/token`; -`index.docker.io` and `registry-1.docker.io` declarations are rejected as -aliases. GHCR, JFrog Artifactory/JCR, and compatible private OCI registries keep -their declared exact host. An absent allowlist rejects cross-origin layer -redirects. - -**User workspace additions** - -```yaml -profiles: - review: - clients: - - name: codex - launcher: codex-review - gateway: - enabled: true +Continuation fragment: + +```json +{ + "model": "gpt-5.6-sol", + "previous_response_id": "resp_previous", + "input": "Now fix the highest-severity finding" +} ``` -The nested object is strict and initially contains only `enabled: true`. -Absence or `false` keeps the client unavailable through the gateway. Enablement -requires a launcher, an initial supported backend, and a healthy installed -profile with matching declaration digest. - -**Promptfoo custom-provider consumption** - -AI Evals implements Promptfoo's -[`ApiProvider`](https://www.promptfoo.dev/docs/providers/custom-api/) in -TypeScript. Its `constructor(options: ProviderOptions)` requires and stores a -nonempty `options.id`, validates `options.config`, and `id()` returns that -stored value. -`callApi(prompt, context?, options?)` reads -`context?.vars?.allagentsSource`, -`context?.vars?.allagentsWorkingDirectory`, -`context?.vars?.allagentsWorkspaceAccess`, -`context?.vars?.allagentsSession`, and -`context?.vars?.allagentsConversation` when present, plus -`options?.abortSignal` for cancellation. - -Static YAML defines the source mode, logical names, and optional default logical -working directory and workspace access: - -```yaml -prompts: - - file://./prompts/coding-task.txt - -sharing: false -evaluateOptions: - maxConcurrency: 1 - cache: false -commandLineOptions: - write: false - share: false - -providers: - - id: file://./providers/allagents-a2a.ts - label: codex-direct - config: - endpoint: https://allagents-gateway.example.internal - target: codex - workingDirectory: - kind: repository - repository: allagents - workspaceAccess: readOnly - source: - kind: repositories - revisions: - allagents: 0123456789abcdef0123456789abcdef01234567 - - - id: file://./providers/allagents-a2a.ts - label: codex-evaluation-snapshot - config: - endpoint: https://allagents-gateway.example.internal - target: codex - workingDirectory: - kind: repository - repository: allagents - workspaceAccess: readWrite - source: - kind: workspaceSnapshot - snapshot: evaluation - digest: sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef - workspaceManifestDigest: sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789 - -tests: - - description: direct repositories at an exact commit - providers: [codex-direct] - vars: - allagentsSource: - revisions: - allagents: fedcba9876543210fedcba9876543210fedcba98 - allagentsWorkingDirectory: - kind: repository - repository: allagents - path: apps/gateway - allagentsWorkspaceAccess: readOnly - - - description: immutable prebuilt workspace - providers: [codex-evaluation-snapshot] - vars: - allagentsSource: - digest: sha256:fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210 - workspaceManifestDigest: sha256:6789abcdef0123456789abcdef0123456789abcdef0123456789abcdef012345 - allagentsWorkingDirectory: - kind: repository - repository: allagents - path: apps/gateway - allagentsWorkspaceAccess: readWrite +Successful response metadata fragment: + +```json +{ + "allagents.workspace": { + "version": "1", + "descriptorDigest": "sha256:...", + "workingDirectory": { + "kind": "repository", + "repository": "api", + "path": "packages/service" + }, + "sourceIdentity": { + "kind": "repositories", + "complete": true, + "repositories": [ + { + "name": "api", + "requestedRevision": "refs/pull/123/head", + "resolvedCommit": "0123456789abcdef0123456789abcdef01234567" + } + ] + }, + "workspaceManifestDigest": "sha256:..." + } +} ``` -The gateway admits one active gateway-controlled invocation transactionally. -Promptfoo keeps `maxConcurrency: 1` to avoid predictably creating failed -capacity Tasks; other trusted callers need no external queue for admission -correctness. Disabling cache, local -result writes, and sharing is the safe baseline for confidential prompts and -opaque provider output. Consumers may enable persistence or sharing only after -defining their own access, retention, destination, and redaction policy. - -`allagents` is a declared repository name used only as a revision-override key; -repository mode still materializes the complete configured set. `evaluation` is -the logical snapshot handle. The provider sends the source mode, optional named -revisions, and immutable digests, not -`https://github.com/EntityProcess/allagents.git` or -`ghcr.io/entityprocess/allagents-workspaces`. The gateway resolves origins and -credentials server-side and omits them from A2A source-identity responses. - -`context?.vars?.allagentsSource` remains limited to source leaves. In repository -mode it may contain exactly `revisions`, whose keys must already exist in static -`config.source.revisions` and whose values are full lowercase 40-hex commits. In -snapshot mode it may contain exactly `digest` and/or -`workspaceManifestDigest`, both full lowercase `sha256:` digests. Present leaves -replace static leaves; absent leaves retain static values. Source kind, -repository-name allowlist, and snapshot name remain static. - -`context?.vars?.allagentsWorkingDirectory` replaces the complete static -selector for that trial. It is exactly `workspaceRoot` or a declared repository -name plus an optional `RelativeDirectory`; the gateway performs catalog and -post-acquisition directory validation. `allagentsWorkspaceAccess` replaces the -static access value with exactly `readOnly` or `readWrite`. Missing access -defaults to `readWrite`. Neither variable accepts an absolute path, configured -destination, materializer, cache key, `.` or `..` segment, backslash, symlink -escape, or non-directory. Unknown members, mutable revisions, origins, -destinations, credentials, and commands fail before provider execution. - -Each `callApi` creates one high-entropy invocation key and sends `SendMessage` -with `returnImmediately: true`, then follows the accepted Task through -`SubscribeToTask`, `GetTask`, and bounded resubscription. `oneShot` receives -Task-private runtime and a disposable writable view when needed. `start` stores -the returned context/head; `resume` sends that exact context/head and receives a -new Task while retaining the session's provider/workspace state. The final -configured turn sets `closeAfterTurn`. The caller chooses neither physical path -nor materializer. An abort/deadline sends one `CancelTask` with a fresh cleanup -signal. Ambiguous submission retry reuses the same key, canonical request, Task, -base/view, cwd, access, and session projection. -The provider returns terminal text or validated structured result as -`ProviderResponse.output`. It maps gateway usage exactly as -`inputTokens -> tokenUsage.prompt`, `outputTokens -> tokenUsage.completion`, -`cachedInputTokens -> tokenUsage.cached`, and -`totalTokens -> tokenUsage.total`; provider-specific counters remain in -`metadata`. Task ID, session context/head, Artifact references, logical source -identity, logical working directory, workspace access, termination, cleanup, -and stable failure `code`/`retryable`/accepted `taskId` also remain in metadata, -without origins, configured destinations, or physical paths. Admission and -terminal failures use a safe `ProviderResponse.error`. This provider is AI Evals -code; AllAgents has no Promptfoo runtime dependency. - -### Error and Status Mapping - -Every unsuccessful HTTP response has `Content-Type: application/json` and the -A2A 1.0 `google.rpc.Status` JSON shape under `error`. Standard A2A errors include -`google.rpc.ErrorInfo` with domain `a2a-protocol.org` and the specified uppercase -reason. Custom admission errors include `google.rpc.ErrorInfo` with domain -`allagents.dev`, uppercase stable-code reason, and string metadata `code`, -`retryable`, and optional `taskId`; field validation also includes -`google.rpc.BadRequest`. No HTTP+JSON response uses JSON-RPC `.data`. - -| Condition | Stable code and A2A/HTTP+JSON outcome | Fresh-invocation retryable | -|---|---|---| -| Unsupported A2A version | HTTP 400 A2A `VersionNotSupportedError`; no Task | No | -| Missing required extension | HTTP 400 A2A `ExtensionSupportRequiredError`; no Task | No | -| Malformed request, context ID, source, working-directory selector, workspace access, digest, schema, prompt, or unknown target/source/repository | HTTP 400 `INVALID_ARGUMENT`; `invalid_execution_request`; no Task | No | -| Invocation-key conflict | HTTP 409 `ALREADY_EXISTS`; `invocation_key_conflict`; no new Task | No | -| Identical retained invocation replay | Existing Task with embedded Artifacts | N/A | -| Follow-up Message to an active or terminal Task | HTTP 400 A2A `UnsupportedOperationError`; existing Task unchanged | No | -| Resume names an unknown or expired session | HTTP 404 `NOT_FOUND`; `session_expired`; no Task | No; start a new session | -| Resume names a stale/non-head prior Task or changes pinned target/source/cwd/access/provider configuration | HTTP 409 `FAILED_PRECONDITION`; `session_head_mismatch` or `session_configuration_mismatch`; no Task | No; refresh head or start a new session | -| Resume targets an active session turn | HTTP 409 `FAILED_PRECONDITION`; `session_busy`; no Task | Yes, after the active turn settles | -| Session checkpoint is poisoned or cleanup is unresolved | HTTP 409 `FAILED_PRECONDITION`; `session_not_resumable`; no Task | No, until verified operator reconciliation | -| Cancel after terminal state | HTTP 400 A2A `TaskNotCancelableError` | No | -| Retained Task/session count or aggregate byte capacity exhausted | HTTP 429 `RESOURCE_EXHAUSTED`; `retention_capacity_exhausted`; `Retry-After`; no Task | Yes, after Task/session expiry | -| Runtime capacity unavailable after acceptance | `execution_capacity_unavailable`; failed Task | Yes | -| A composed extension reports failure | Core `execution_extension_failed`; failed Task; exact extension failure remains in its Artifact | Per paired extension row | -| Valid logical cwd resolves to a missing, non-directory, or escaping path after acquisition | `execution_working_directory_invalid`; failed Task; no provider start; no physical path returned | No | -| Required copy-on-write materializer unavailable, or `auto` would copy above its ceiling | `workspace_materialization_unavailable`; failed Task; no provider start | No | -| Task/session-private runtime, non-reusable base, provider checkpoint, or writable-view creation/removal fails | Workspace/session `workspace_cleanup_failed`; Core `execution_extension_failed`; failed non-expiring Task; poison the session and retain reservation, cleanup record, owned state, and lease until verified repair | Yes only as a fresh invocation after operator repair | -| App absent/ineligible and configured `gh` succeeds | Continue with recorded provider class | N/A | -| App applicability unknown | `source_auth_applicability_unknown`; failed Task; no fallback | Yes for rate-limit/service causes only | -| Selected App config/auth/mint/validation/revocation failure | `source_auth_failed`; failed Task; no fallback | No | -| Selected App permission/repository denial | `source_auth_denied`; failed Task; no fallback | No | -| Selected App rate limit | `source_auth_rate_limited`; failed Task; no fallback | Yes | -| Selected App service failure | `source_auth_unavailable`; failed Task; no fallback | Yes | -| `gh` account missing or token resolution fails | `source_auth_unavailable`; failed Task | No | -| Git revision/identity failure | `source_git_identity_invalid`; failed Task | No | -| Git transport failure | `source_git_unavailable`; failed Task | Yes | -| OCI helper timeout, process, protocol, or credential failure | `source_auth_oci_failed`; failed Task; no fallback | No | -| OCI auth/challenge/digest/manifest/extraction validation failure | `source_snapshot_invalid`; failed Task; no Git fallback | No | -| OCI registry service failure | `source_snapshot_unavailable`; failed Task; no Git fallback | Yes | -| Deadline expires | Core `timedOut`; `execution_deadline_exceeded`; abort/terminate; A2A failed Task | Yes | -| Known provider permission denial | `execution_permission_denied`; rejected Task | No | -| Unknown provider protocol or result shape | `provider_protocol_invalid`; failed Task | No | -| Valid structured result exceeds retained footprint after optional evidence truncation | `execution_result_too_large`; failed Task; `result.reason: "retentionLimitExceeded"` | No | -| Cancellation after acquisition removal or direct provider settlement, with successful termination and cleanup | Core `canceled`; `execution_canceled`; A2A canceled Task | No | -| Acquisition container or unpublished staging cannot be removed | Workspace `source_cleanup_failed`; Core `execution_extension_failed`; failed non-expiring Task; retain reservation/reconciliation record/lease; stop admission until verified repair | No | -| Direct provider does not settle after abort/`SIGTERM`/`SIGKILL` | `execution_termination_failed`; failed non-expiring Task; no filesystem/Git evidence; retain reservation, Task/session-owned state, reconciliation record, and lease until post-teardown verification | No | -| State store durability/integrity failure | `state_store_failed`; stop admission; request active-work abort; no success | No | -| Restart finds interrupted Task | `gateway_restarted`; failed Task; persist any required App-token revocation tombstone before releasing the lease; never replay the turn; restore only a provably unchanged prior session checkpoint, otherwise poison the session | Yes as a new turn only when the session remains resumable | -| Task retention expiry | HTTP 404 A2A `TaskNotFoundError`; does not delete a live session checkpoint | Yes as a new invocation | -| Session idle expiry | Verified provider/workspace cleanup, then `session_expired` on resume; cleanup uncertainty poisons and retains the session | No; start a new session | - -Accepted-Task Core-origin failures use the Core outcome Artifact's strict -`failure` and HEC-only cause union. Accepted workspace/source failures instead -pair Core `execution_extension_failed`/`extension` with the workspace-integrity -Artifact's exact code, safe message, retryability, and one closed cause from -`sourceAuth | sourceGit | sourceSnapshot | sourceCleanup | workingDirectory | -workspaceMaterialization | workspaceCleanup`. The composed schema and -conformance matrix own this pairing; neither module imports the other's codes. -Retryability says whether a caller may create a fresh invocation; it never -enables automatic Task retry or provider/source fallback. Promptfoo copies the -workspace failure when present, otherwise the Core failure, plus retryability -and accepted Task ID. Provider identifiers, credentials, paths, and raw upstream -messages enter neither Artifact nor Promptfoo metadata. +### Materializer Hook Contract + +HarnessRouter configuration names one metadata key, absolute executable path, +maximum runtime, request/result byte limits, and allowlisted environment names. +The runner launches the executable directly without a shell. Standard error is +diagnostic-only, bounded, secret-checked, and never copied verbatim to callers. + +The hook supports two operations: + +- `preflight`: validate contract version, project catalog, credential-reference + syntax and presence, required binaries, and filesystem assumptions without + source network access; and +- `materialize`: validate the opaque descriptor, write only to the supplied + sibling staging root, and return without publishing. + +The materialize request contains the generic contract version, opaque metadata +value, session workspace and fixed staging roots, project configuration root, +and deadline. Secret values are injected only through the configured allowlisted +child environment; credential identifiers and values are absent from JSON. + +The generic result is either: + +- `completed`, effective relative cwd, effective descriptor digest, + workspace-manifest digest, complete path-free public metadata, and declared + nested-repository roots; or +- `failed`, stable code, safe message, retryability, and any verified incomplete + public metadata. + +The runner validates the result and staged tree independently. It rejects an +unknown envelope field/version, digest mismatch, physical path in public +metadata, incomplete success, undeclared repository root, escaping cwd, or tree +that does not match the manifest. The runner then owns publication, marker and +checkpoint setup; a valid result never means the live workspace is already +published. + +### Fork Maintenance Contract + +- Keep the fork in a dedicated repository/branch with the upstream remote intact. +- Pin production images to an upstream commit, never a moving branch. +- Keep the materializer changes as a small ordered patch series with focused + commits and no formatting churn. +- For every selected upstream upgrade: rebase the patch series, inspect upstream + changes in touched gateway/runner/session code, run upstream tests and UHP + conformance, run AllAgents hook/session E2E, rebuild the image, and record the + new inputs and digest. +- Prepare the upstream proposal as a generic command/plugin seam. Do not require + upstream to understand AllAgents metadata, Git catalogs, OCI manifests, or + Promptfoo. +- If upstream accepts an equivalent seam, delete the patch rather than retaining + a compatibility layer. -### Phased Delivery +### Risks and Mitigations -1. In a clean `/tmp/` npm prefix, install the current `allagents` package and - record the red E2E showing that `allagents-gateway serve` is unavailable and - that no acquisition image is fetched. -2. Execute U0 as a bounded feasibility gate: establish the private Bun - workspace layout; prove Harness Execution Contract Core and Sessions - semantics, the shipped A2A binding and required coding-workspace extension, - and anonymous trusted-network conformance direction with the official - JavaScript client; prove a minimal Promptfoo custom provider can activate the - profile, consume a terminal Task, and resume a second turn; pin and probe - Codex SDK and Pi RPC/package one-shot/session surfaces; characterize explicit - provider environments and Linux process groups; build/run the digest-pinned - acquisition image for both supported architectures; and prove independent - CLI/gateway packaging plus exact release binding. -3. Freeze the normative Harness Execution Contract Core and Sessions, A2A - binding, AllAgents coding-workspace extension, their composed v1 Profile URN - and wire schema, independent module/binding/extension conformance vectors, - workspace additions, snapshot format, execution/acquisition contracts, error - vocabulary, SQLite schema/transactions, compatibility output, and release - manifest. -4. Build the Bun SQLite Task/session store, AllAgents A2A request handler, - HTTP+JSON/SSE server, minimal backend interface/registry, and fake adapter. -5. Add direct host-process supervision, explicit environment construction, - read-only runtime separation, read-write materialization, process-group - cancellation, typed preparation, bounded evidence, terminal arbitration, - restart handling, and cleanup around the fake adapter. -6. Add Docker-only Git and OCI acquisition for requests without reusable bases, - with source-only credentials, strict mount/network/archive limits, typed - manifest emission, host validation, reusable-cache or non-reusable-base - publication, pinning, cleanup, reuse, and eviction. -7. Add Codex through the pinned SDK, then Pi through the pinned supported - package/RPC surface, against the same conformance suite. -8. Run final implementation review and fix important correctness, security, - contract, reliability, DRY, and coverage findings. -9. Run green packed CLI/gateway `/tmp/` smokes, exact multi-architecture - acquirer-image tests, public GHCR conformance, exact-release authenticated - GHCR/JFrog conformance, repository quality gates, user documentation, and - release evidence. - -### System-Wide Impact - -- **Package surface:** Convert the root to private Bun workspace orchestration. - `apps/cli` publishes `allagents`; `apps/gateway` publishes - `allagents-gateway`; `apps/acquirer` publishes no npm package and builds only - the digest-pinned GHCR image. The ordinary CLI has no gateway dependency. - GitHub Actions has independent CLI and gateway release triggers. Gateway - release builds and verifies the acquisition image first, then publishes the - exact npm tarball; a gateway-only run never publishes the CLI. -- **Runtime surface:** `apps/gateway` owns gateway behavior end to end. Docker - exists only at the base-acquisition boundary when no reusable validated base - exists. Codex and Pi execute directly on the trusted Linux runner with - existing host - authentication. Packages share contracts and configuration, not generic - implementation helpers; do not add `core`, `common`, native IPC, or dual - implementations. -- **Schema surface:** `packages/workspace-config` extends project schemas with - named snapshots/exact redirect hosts and user profile-client schemas with - gateway enablement. `packages/execution-contracts` and - `packages/acquisition-contracts` generate versioned JSON Schemas and fixtures - under `contracts/`; update extension, snapshot-format, and configuration docs. -- **Dependency surface:** Pin Bun, the official A2A JavaScript SDK, - `@openai/codex-sdk`, the supported Pi package/RPC dependency, and the minimal - Git/OCI/archive dependencies used by `apps/acquirer` in the Bun lockfile. - Minimize dependencies per workspace and scan both the npm tarball and image. -- **State surface:** Add one bounded private Bun SQLite state root, one bounded - immutable-base cache, and Task/session-private runtime plus optional writable - view roots. Do not alter provider profile or authentication state. -- **Security surface:** Network reachability authorizes callers. The acquisition - container has staging, source-only credentials, and strict source policy but - no host home or Docker socket. Provider execution has trusted CI-job - authority; explicit environments reduce accidental leakage but do not isolate - secrets or hostile code from model tools. -- **Compatibility:** Existing workspace files remain valid because new fields - are optional; request access defaults to `readWrite`. Gateway startup applies - stricter catalog, cache-root, materializer, and provider-readiness rules. - CLI/gateway version skew is governed by contract ranges; gateway/image - compatibility is exact by manifest digest and acquisition-contract version. +- **Fork drift:** Keep the patch ordered and narrow, pin commits, rebase only + selected releases, and run both upstream and integration suites. +- **Gateway/runner durability split:** Use the explicit materialization CAS plus + workspace marker and pre-agent checkpoint. Fault every boundary and fail + incomplete sessions closed rather than attempting replay. +- **Nested Git versus HarnessRouter root Git:** Keep repository `.git` state, + ignore declared roots in HarnessRouter's root index, and extend produced/list/ + file/ack/checkpoint/hydrate behavior to validate and walk every declared root. +- **Provider fallback:** Run materialization before the provider candidate loop + and surface a typed non-provider failure; a ready marker prevents reruns. +- **Credential leakage:** Broker the `codex-lb` key and use subprocess-only source + credentials, hermetic configuration, leak scans, and hostile fixtures. +- **Partial publication:** The materializer writes staging only. The runner + validates and publishes with recovery markers; no agent runs until the gateway + durably stores the resulting checkpoint and marks the session ready. +- **Descriptor/session drift:** Accept the key only on the initial request and + persist the hook's effective digest/provenance for every later response. +- **OCI attack surface:** Use a closed media profile, streaming digest checks, + fixed limits, strict path/link/type validation, and exact-host redirect policy. +- **Pi incompatibility:** Keep Pi capability-gated; a failed format/endpoint probe + removes it from advertised harnesses without fallback. +- **HarnessRouter restart semantics:** Claim persistence only for completed state + on durable storage; interrupted work fails and is not replayed. +- **Provider cache assumptions:** Report native cached-input usage when available; + never promise a cache hit. +- **Upstream rejection:** The pinned fork remains supported; upstream delivery is + maintenance reduction, not a launch dependency. -### Risks and Mitigations +### Phased Delivery -- **A2A, binding, or provider-surface immaturity:** Pin exact JavaScript package - versions and run U0 wire/provider probes before production units. The gate - must prove that applications, agents, and evaluation runners can use the same - Harness Execution Contract semantics through the A2A binding, and that - trusted-network anonymous operation can make an accurate A2A 1.0 conformance - claim. If authentication semantics make that impossible, amend the ADR before - production work; do not silently add identity or weaken the claim. If the - Codex SDK lacks a required capability, document proof before selecting pinned - app-server; if neither works, the target is unavailable rather than silently - scraped. -- **Harness-contract fragmentation:** Publish one normative Core with schemas, - examples, and executable semantic vectors; keep the A2A binding and AllAgents - coding-workspace extension separate, then expose only their one composed v1 - URI. Add no UHP/Responses or bespoke fallback wire in v1. Future bindings must - pass the same Core vectors rather than redefine lifecycle semantics. -- **Premature standard claim:** Describe HEC as a contract and standard - candidate until multiple independent implementations, multiple bindings, - neutral governance, and cross-binding conformance exist. -- **Contract drift:** Generate binding, extension, and composed schemas plus - accepted/rejected fixtures from the three narrow packages and run drift - checks in the gateway, acquirer, docs, and consumer fixtures. Keep Core - semantic vectors hand-authored and independent from generated types. -- **Install-size regression:** Keep CLI and gateway workspace dependency graphs - separate, report packed/installed sizes, enforce budgets, and fail CLI-only - smoke if it resolves the gateway or acquisition image. -- **Accidental network exposure:** Reject wildcard/public binds. Require either a - loopback-only backend behind private HTTPS ingress or a native TLS listener on - one specific private address, plus private advertised resolution and external - ACLs. Startup/docs state that every reachable peer has full authority. Public - exposure requires application authentication and an amended ADR first. -- **Profile identity drift:** Derive targets only from current validated user - declarations and matching installed state; never resurrect declaration-missing - launchers from retained profile state. -- **Working-directory escape or mutable cross-trial reuse:** Accept only the - closed logical selector and `RelativeDirectory` grammar, resolve through the - compiled catalog, and require an existing directory beneath the selected - repository. Read-only Tasks share only the gateway-managed immutable base and - keep private runtime state; read-write views derive from Task IDs. Never expose - or accept a resolved host path. -- **Read-only contract violated by the prompt or provider:** Do not inspect - prompts or claim a sandbox. Disable optional Git locks, request native provider - read-only policy when available, isolate runtime writes, and document that - consumers must choose `readWrite` when project mutation is required. Do not - add a per-Task mount, chmod traversal, or full-tree verification in v1. A - violating provider can contaminate the base and later Tasks; the operator must - evict that entry before reuse. -- **Large workspace duplication or unsupported copy-on-write:** Acquire each - immutable identity once, pin shared bases, prefer block clone, fall back to - rootless OverlayFS, and retain explicit `copy` for portability. `auto` refuses - a full copy above its byte ceiling; startup reports capabilities, and CI users - provision enough disk or choose a larger/self-hosted runner. -- **Base-cache corruption or unbounded growth:** Bind keys to immutable source, - catalog/layout, and acquisition-contract identity; publish atomically; keep - roots private; pin active entries; evict only unpinned least-recently-used - entries under a byte budget; and stop admission on detected metadata or - filesystem inconsistency. This is trusted-runner state, not a hostile-process - integrity boundary. -- **Acquisition credential leakage:** Mount only staging, inject only the - selected source credential and exact-host CA material, never mount host home - or Docker socket, remove the container before provider execution, and scan - the manifest/staging/logs for gateway-managed credential values. -- **Identity-changing fallback:** Classify App applicability as eligible, - ineligible, or unknown; require repository-existence proof for 404 - ineligibility; only positive ineligibility permits `gh`. -- **OCI registry/archive abuse:** Require immutable digests, a closed - manifest/config/layer profile, exact host/redirect policy, changeset - semantics, fixed extraction limits, safe paths/types/links, and exact catalog - validation inside the image and again at the host publication boundary. -- **Untrusted provider execution:** The acquired workspace and model tools run - with the same authority as the trusted CI job. Mitigate by using ephemeral - runners or an operator-managed VM/container boundary, least-privilege CI - credentials, explicit provider environments, no automatic setup commands, - and clear documentation. Do not describe AllAgents as a sandbox. -- **Evidence overclaim:** Collect only after the direct provider process settles, - keep evidence bounded, record process-group signals and observed cleanup, and - explicitly avoid claiming full descendant quiescence or output redaction. -- **Provider/API churn:** Pin SDK/package/protocol/model compatibility, require - exact probes for binary overrides, retain native fixtures, and share one - adapter conformance suite. Never download a provider runtime per request. -- **Orphaned processes:** Use a new Linux process group per provider, persist its - leader identity, escalate abort to `SIGTERM`/`SIGKILL`, and retain lease/unready - state for any observed escaped/outliving descendant until verified gone or - runner teardown. An unobserved descendant may overlap a later admitted turn; - operators requiring OS-wide exclusivity must use an ephemeral runner boundary. -- **Store corruption or disclosure:** Use a current-user private state root, - exclusive gateway lock, ordinary Bun SQLite transactions, foreign keys, - `synchronous=FULL`, integrity checks, and bounded data. Integrity/durability - failure stops admission and prevents false success. -- **Artifact mismatch:** Bind every release report to the exact gateway npm - tarball digest and acquisition manifest/platform digests. Reject rebuilt, - mutable-tagged, wrong-commit, or contract-incompatible substitutes. - -### Assumptions - -- The initial deployment is one gateway process and one transactionally enforced - active invocation on a trusted Linux CI runner. -- Every external network peer able to connect is trusted with all available - targets and retained Tasks. -- The CI job, VM, or deployment container is the isolation boundary. AllAgents - does not isolate hostile repository code, provider credentials, MCP secrets, - or host network access from model-invoked tools. -- The selected project workspace is operator-controlled and compiles to 1-64 - uniquely named GitHub repositories with collision-free destinations. -- GitHub.com is the only authenticated Git host in the initial delivery. -- OCI snapshots use HTTPS Docker Hub, GHCR, JFrog Artifactory/JCR, or compatible - private OCI registries and the frozen v1 direct-image format. -- Docker is available solely for base-acquisition containers when no reusable - validated base exists, and the operator-provided acquisition network enforces - the deployment's source egress boundary. Immutable repository cache reuse - requires full commit IDs. -- Codex and Pi are installed or provided by pinned workspace dependencies before - gateway start. Existing host authentication is reusable only when the pinned - adapter proves a public auth reference separate from private mutable state. -- The implementation units after U0 assume the Bun/A2A/provider/process/acquirer - feasibility gates passed. A failed provider probe disables that target; a - failed architecture or release-binding gate stops the affected release rather - than introducing Rust, native platform packages, or a split runtime. +1. Red E2E against stock HarnessRouter: prove arbitrary metadata is neither + forwarded to Codex/Pi nor returned as workspace provenance. +2. Fork spike: prove a fake hook runs through a dedicated pre-provider operation, + publishes/checkpoints once, survives two-turn reuse, supports a safe nested + cwd, reports nested-repository files, and cannot rerun under provider fallback. +3. Prove brokered Codex through `codex-lb`; probe Pi's separate supported format + and mark it available or unavailable without changing the architecture. +4. Freeze generic hook envelope/state fixtures and AllAgents descriptor, + configuration, provenance, and failure fixtures. +5. Implement project schema projection, preflight, Git materialization, + credential containment, and checkpoint/collection integration. +6. Implement OCI materialization and its archive/registry security profile. +7. Run Promptfoo one-shot, continuation, cancellation, restart, and failure + mappings; review both repositories; build the exact image; run green E2E and + conformance; document operations; prepare the generic upstream patch. --- ## Implementation Units -### U0. Harness-contract, Bun, provider, process, and acquirer feasibility - -- **Goal:** Prove the settled Bun architecture can preserve the - transport-neutral Harness Execution Contract Core and Sessions semantics - through their first A2A binding, independent distribution, supported provider - create/resume/checkpoint control, Linux cancellation, read-only shared-base - execution, retained session workspace, read-write materialization, and exact - acquisition-image release binding before production implementation. -- **Requirements:** R1-R3, R8, R13-R16, R18; AE8-AE11, AE17-AE18, AE20-AE21; - KTD1-KTD3, KTD6-KTD7, KTD9, KTD11-KTD14. -- **Files:** private root `package.json`/`bun.lock`, `apps/cli`, - `apps/gateway`, `apps/acquirer`, the three named `packages/` workspaces, - representative generated fixtures under `contracts/`, acquisition Dockerfile/ - image metadata, provider and workspace-materializer feasibility probes, - process-group probe, independent CLI/gateway pack scripts, and gateway release - workflow skeleton. -- **Approach:** Move the existing CLI into `apps/cli` without changing its - public package or behavior. Establish `apps/gateway` as the separately packed - Bun executable package and `apps/acquirer` as image-only code. Pin the - official A2A JavaScript SDK and drive a minimal production-direction server - through every required operation. Add a small transport-neutral in-memory - adapter and representative Core, Sessions, A2A-binding, workspace-extension, - and composed-profile schemas. Run the same hand-authored Core vectors through - the in-memory adapter and A2A binding to prove per-turn lifecycle, ordered - progress/tool trajectory, result/usage/failure settlement, cancellation, and - artifacts are binding-independent. Prove Sessions start/resume/close, exact - predecessor chaining, linearization, pinned configuration, restart handling, - and workspace continuity independently, then prove the A2A context/task- - reference mapping. Separately prove required profile activation, - one-Message/one-Task-per-turn behavior, A2A state/event/Artifact mapping, and - standard errors. Drive two turns through the same server via a throwaway - Promptfoo custom provider to prove an application can resume context and - observe provider-reported cache usage without a second server protocol. - Resolve whether anonymous trusted-private-network operation can claim A2A 1.0 - conformance before dependent work. - Probe `@openai/codex-sdk` for start/continue/resume, opaque thread identity, - events, native abort, usage/cache reporting, structured-output support, and - private state-home/host-auth separation; consider app-server only when a named - required capability is proven absent. Probe the supported Pi package/RPC - surface for equivalent create/resume/checkpoint/dispose, events, abort, usage, - and auth/state separation. Prove exact compatibility rejection for global - binary overrides. - - Run a real Linux child in a new process group and demonstrate graceful abort, - `SIGTERM`, and `SIGKILL`; an escaped/outliving fixture must retain the lease - and readiness=false until verified disappearance. Also prove the explicit - limit that unobserved descendants are not contained. Prove one immutable base - can serve repeated read-only Tasks with private runtime state; probe block - cloning and rootless OverlayFS; verify independent writable changes/removal; - and prove explicit copy behavior plus the automatic copy ceiling. Build the - acquirer image for - every supported architecture, run it with only a staging mount and synthetic - source secret, verify typed manifest output and container removal, and prove - the image has no provider runtime or Docker socket. Pack CLI and gateway - separately, prove a CLI-only install fetches neither gateway nor image, and - define the immutable gateway-tarball/acquisition-manifest release record. -- **Execution note:** U0 is a feasibility gate, not partial production - scaffolding. Do not paper over missing SDK/RPC behavior with TUI scraping, - AI SDK Harnesses, per-request downloads, per-provider Docker, or native - containment machinery. A missing provider capability disables that provider; - failed A2A, process, acquisition, or release-binding feasibility returns the - affected design for revision before dependent units. -- **Verification:** Hand-authored Core and Sessions semantic vectors pass - unchanged through the in-memory adapter and A2A binding; official JavaScript - client and representative binding/workspace/composed-profile conformance - fixtures pass against the Bun server. The Promptfoo probe completes two - terminal Tasks under one conversation ID: turn two sends turn one's returned - context ID and exact head reference and proves conversation/workspace - continuity before close. The A2A authentication conformance result is - recorded; provider probes record exact pinned versions and auth/state-path - behavior; shared read-only base, private runtime, reflink, rootless-overlay, - explicit copy, cleanup, environment, and process-group probes pass on Linux; - multi-architecture image manifests/digests are recorded and the image boundary - rejects extra mounts/credentials/network; independent packed CLI/gateway - installs and compatibility fixtures pass; CLI-only installation fetches - neither gateway nor acquisition image. - -### U1. Workspace packages, contracts, SQLite, and release foundation - -- **Goal:** Freeze the monorepo ownership, workspace configuration, Harness - Execution Contract Core, Sessions, and first binding, acquisition contracts, - ordinary SQLite transactions, and exact release artifact binding before - runtime implementation. -- **Requirements:** R1-R3, R5-R9, R11-R12, R18; AE3-AE9, AE13-AE14, AE16, - AE20-AE21; KTD1-KTD2, KTD5-KTD8, KTD10, KTD13-KTD14. -- **Files:** `packages/workspace-config`, `packages/execution-contracts`, - `packages/acquisition-contracts`, normative Core/binding/extension text, - generated `contracts/` schemas/conformance vectors/golden examples, gateway - SQLite schema/migrations, release scripts/workflows, deterministic snapshot - producer/conformance fixture, snapshot-format assets, and configuration docs. -- **Approach:** Move authoritative project/user parsing and gateway catalog - compilation into `workspace-config`; add strict named `workspaceSnapshots`, - exact redirect hosts, and nested `gateway.enabled` without changing ordinary - CLI behavior. Define four separately versioned contract modules in - `execution-contracts`: the transport-neutral Harness Execution Contract Core, - its Sessions extension, its A2A binding, and the AllAgents coding-workspace - extension; compose binding, Sessions, and workspace extension into one - required v1 A2A Profile URN and request. Core owns one turn's invocation, - target, deadline, idempotency/replay, ordered progress and tool-call/result - trajectory, cancellation, result, usage, portable failure codes, and artifacts - without importing Sessions, A2A, UHP, workspace, provider, or evaluator types. - Sessions owns oneShot/start/resume, durable identity, ordered linear turns, - exact predecessor, checkpoint/expiry/close states, and portable session errors. - The A2A binding owns Agent Card parameters and skill semantics, version/header - activation, Message/Task/state/event/Artifact mapping, context-ID and prior- - Task mapping, and standard error mapping. The coding-workspace extension owns - source, logical working directory and relative-path grammar, workspace access/ - default, workspace failure codes, provenance, produced files, integrity, and - cleanup evidence. Each module exposes an independently validatable schema and - conformance group; only the composed v1 profile owns cross-module pinning, the - merged failure union, and simultaneous Core outcome/trajectory plus AllAgents - workspace-integrity Artifact requirements. - - Generate strict binding, extension, and composed wire schemas from canonical - types. Keep Core semantic vectors and normative prose hand-authored as an - independent oracle; run the same vectors through an abstract in-memory adapter - and the A2A binding. Check the Core, binding, extension, schemas, vectors, and - accepted/rejected fixtures beneath `contracts/`, and include a deliberate - contract-perturbation fixture that proves drift between generated types and - the normative contract turns CI red. - - Define acquisition contracts for the closed request, path-free typed manifest, - immutable-base cache key, private compiled-layout checks, OCI media/change-set - profile, fixed limits, and canonical digests. - - Add private `bun:sqlite` ownership with foreign keys, WAL where supported, - `synchronous=FULL`, migrations, one execution lease, `createOrReplay`, - immutable-base metadata and active pins, internal outcome intent, atomic - settlement, transactional expiry, and recorded acquisition-container/provider- - process identities. Establish independent CLI/gateway versions and release - triggers. The gateway release record binds the exact npm tarball digest to the - acquirer multi-architecture manifest and supported platform digests; the image - is verified before npm publication. -- **Execution note:** Do not add catch-all `core` or `common` packages, a custom - VFS, native file primitives, native/platform npm packages, or runtime - compatibility shims. Start with external wire/manifest fixtures and stable - rejection codes. Fault - SQLite transactions and process exit around commit/acknowledgment boundaries, - not filesystem attacks the ordinary SQLite contract does not claim to defeat. -- **Verification:** Workspace parsing/catalog fixtures; Core and Sessions - semantic vectors through both the in-memory adapter and A2A binding; binding- - specific context/session ID generation, exact prior-Task projection, - state/cancel/event/Artifact mapping, and byte budgets; binding/extension/ - composed-schema drift; hand-authored lifecycle/composition conformance; full - Core outcome matrix, extension retryability pairings, and rejected cross- - products; Sessions start/resume/close, head linearization, pinning, expiry, - capacity, and poison/reconciliation; canonical ordered execution trajectories - plus negative descriptor-linkage cases; ambiguous/identical/divergent replay; - deadline/cancellation/race/expiry behavior; produced-Artifact bytes and - descriptor linkage; wire/manifest accepted/rejected examples; deliberate - contract perturbation; canonicalization; Artifact cardinality; SQLite Task/ - session commit/replay/head/lease/settlement/expiry/crash and revocation- - tombstone fixtures; independent package versions; CLI-only and gateway clean- - registry installs; compatibility skew/image-mismatch matrix; exact tarball/ - image release record; and idempotent absent/identical/divergent publication - fixtures pass. - -### U2. Deployment-wide Task/session store and A2A server - -- **Goal:** Serve the A2A lifecycle without application authentication and keep - durable deployment-wide Task, idempotency, and linear session truth behind a - fake backend. -- **Requirements:** R1-R5, R8, R13, R16-R18; AE1-AE2, AE9, AE11-AE16, - AE19-AE21; KTD1-KTD4, KTD9-KTD10. -- **Files:** `apps/gateway` Task/session-store module, Agent Card, A2A request - handler, HTTP+JSON/SSE server, pagination/retention, backend registry/fake - adapter, health/readiness, `allagents-gateway` command, and focused integration - tests. -- **Approach:** Implement flags/environment precedence, private bind/advertised- - URL validation, private state/lock, SQLite transactions, startup integrity and - interrupted-turn reconciliation, A2A version/profile negotiation, exact - `SendMessage` modes and `ListTasks` semantics, one immutable Task per turn, - Sessions start/resume/close and exact-head linearization, lifecycle/state/event - constraints, standard/custom `google.rpc.Status` errors, durable - `createOrReplay`, one execution lease, internal outcome intent plus atomic - terminal/checkpoint settlement, bounded events/Artifact/session-workspace - bytes, no early eviction, transactional Task/session expiry, deployment-wide - listing/cancellation, deadline handling, and graceful shutdown against a fake - adapter. -- **Execution note:** Use an independent official JavaScript A2A client to prove - one external caller can read and cancel another caller's Task; that is expected - trusted-network behavior. Kill gateway subprocesses around SQLite transaction, - commit, acknowledgment, cancellation-intent, Artifact, and settlement - boundaries. Do not add caller ownership or an application credential. -- **Verification:** Discovery, both send modes, stream/get/full list/subscribe/ - cancel/replay/Task expiry, start/resume/close/session expiry, stale-head and - concurrent-turn rejection, HTTP errors, loopback/direct-private-TLS/ - loopback-proxy advertised URL combinations, wildcard/public listener and URL - rejection, TLS-file validation, probes, retained/active capacity, competing - lock, SQLite crash/fault, deadline, shutdown, restart, and fake-backend tests - pass. - -### U3. Host process supervisor and backend contract - -- **Goal:** Run fake-backed direct host turns through shared read-only, - one-shot/private read-write, and session candidate/committed workspace - selection, logical cwd resolution, typed preparation, explicit environment - construction, provider checkpointing, process-group cancellation, evidence, - arbitration, and cleanup with truthful limits before real adapters. -- **Requirements:** R3, R5, R8, R13-R16, R18; AE8-AE12, AE14-AE21; - KTD3, KTD6, KTD9-KTD12. -- **Files:** `apps/gateway` backend types/registry, immutable-base manager, - workspace/session materializer, provider environment builder, Linux process- - group supervisor, invocation/session state machines, typed preparation, - evidence collector, result validator, cleanup/restart reconciliation, fake - process fixtures, and lifecycle tests. -- **Approach:** Define the minimal adapter contract for availability, - one-shot/session capabilities, start/resume/checkpoint/dispose, access-aware - invoke/events, graceful abort, direct-process settlement, result/usage/cache - evidence, and disposal. Resolve fake targets without executing generated - launchers or setup commands. For read-only, resolve cwd in the immutable base - and allocate Task- or session-private runtime state. For read-write, - materialize a Task- or session-ID-derived view via block clone, rootless - OverlayFS, or explicit copy. Resolve workspace-root and repository-relative - selectors, reject missing/non-directory/escaping paths, and pass only the - effective cwd, runtime paths, and access mode to the adapter. Start each turn's - direct provider in a new process group, persist its leader PID/process-start - marker before marking execution started, and build its environment from a - reviewed allowlist preserving required host identity/auth paths. Commit one - internal intent across provider terminal, cancel, deadline, and shutdown. - Escalate adapter abort to process-group `SIGTERM`/`SIGKILL`; capture bounded - live events; collect filesystem/Git evidence only after direct-process - settlement; then checkpoint a retained session or remove one-shot/closing - state using the R16 prepared-settlement/cleanup protocol. A non-settling, - observed-outliving, or uncertain checkpoint/candidate path emits truthful - failure/evidence, retains the lease, and blocks admission until reconciliation. -- **Execution note:** Fixtures distinguish observation from guarantee. Exercise - child/grandchild processes, including one that escapes/outlives the direct - process; it must retain lease/readiness state until the fixture exits. The - gateway never labels process-group cleanup as complete descendant quiescence; - an unobserved escape remains outside the v1 guarantee. CI runner teardown is - the final orphan boundary. No cgroups, pidfds, namespaces, nftables, `openat2`, - spawn broker, provider container, or isolation claim. -- **Verification:** Deterministic per-turn/session lifecycle; shared-base reuse - without shared runtime; one-shot views and read-write session prior/candidate/ - committed generations across reflink/rootless-overlay/copy; candidate byte - reservations; crash before/after workspace-pointer commit; adapter-native - read-only policy where available; cwd/escape rejection; exact session head and - provider/workspace checkpoint pair; one gateway-controlled lease/turn; - auth/private-state environment separation; binary-override compatibility; - abort/TERM/KILL timing; cancellation/deadline/shutdown races; observed - escaped-descendant lease retention; poisoned Task/session behavior; - post-teardown reconciliation; evidence ordering; result states; cleanup - outcomes; and truthful unobserved-orphan/cache-limit fixtures pass on Linux. - -### U4. Docker-only Git and OCI immutable-base acquisition - -- **Goal:** Materialize declared repository sets and named OCI snapshots when no - reusable validated base exists, validate and promote bases on the host, and - prove reuse and non-reusable cleanup without placing providers in Docker. -- **Requirements:** R6, R8-R12, R15-R16, R18; AE5-AE11, AE15, AE17-AE19; - KTD3, KTD6-KTD8, KTD10-KTD14. -- **Files:** `apps/acquirer` Git/OCI implementations and entrypoint, - `packages/acquisition-contracts`, `apps/gateway` Docker coordinator, - immutable-base cache/pin/eviction manager and host staging/manifest validator, - deterministic producer fixture, local/GHCR/JFrog fixtures, reusable registry- - conformance workflow, and focused tests. -- **Approach:** Derive cacheability and keys from the compiled catalog, - acquisition-contract version, layout digest, and immutable source identity. - A valid hit pins the base and starts no container or credential flow. On a - miss, the host creates private staging, resolves only the selected source - credential, and starts the exact digest-pinned image with staging as its sole - writable bind, no host home, no Docker socket, and per-request source policy. - In repository mode the host coordinator implements the App eligibility table, - mints and injects only the fresh repository-scoped token, validates/revokes it, - selects `gh` only for positive ineligibility, and never falls back after - selected-provider failure. Branch/tag requests bypass reusable bases. In - snapshot mode implement anonymous-first bounded Basic/Bearer authentication, - canonical Docker Hub normalization, exact-host CA/realm/redirect rules, - direct-image media profile, streaming digest verification, gzip/zstd - changesets/whiteouts, fixed limits, and path-free manifest/private layout - checks. - - The acquirer emits only typed manifest and staging content, then exits. The - gateway removes it, destroys source material, validates manifest, limits, and - exact catalog again on the host, and atomically publishes the base. Private- - root, pinning, budgeted unpinned-LRU eviction, and - restart fixtures cover cache lifecycle. Image probes prove no Codex/Pi/harness, - provider auth, host home, gateway state, or Docker control reaches acquisition. - Snapshot failure never invokes Git fallback. -- **Execution note:** Every PR runs local Git, local Distribution, and live - public digest-pinned GHCR against the exact built image. Release conformance - reuses the exact gateway npm tarball plus multi-architecture acquisition - manifest/platform digests without rebuilding. Authenticated GHCR uses least- - privilege pull credentials; pinned JFrog JCR uses HTTPS/private CA/private - repository/pull-only identity. Run platform-specific cases only where the - registry/runner supports that architecture and record coverage explicitly. -- **Verification:** App three-way selection, base-acquisition token lifetime/ - validation/revocation, cache-hit no-credential/no-container behavior, `gh` - fallback, Git revisions, immutable key invalidation, pin/eviction/restart, - non-reusable branch/tag base cleanup, Docker mount/env/network/credential/limit - enforcement, container and orphan-staging removal, - host revalidation/atomic publication, OCI auth/realm/redirect/CA/media/digest/ - size/whiteout/path/catalog cases, clean leak scans, equivalent typed manifests, - and exact local/public GHCR/authenticated GHCR/private-CA JFrog reports pass. - -### U5. Codex SDK adapter - -- **Goal:** Run built-in and profile-backed Codex targets on the trusted host - through the pinned SDK while preserving one-shot and resumable thread - semantics, progress, result, usage/cache reporting, cancellation, existing - authentication, and truthful evidence. -- **Requirements:** R7-R8, R13-R16, R18; AE1, AE3-AE4, AE10-AE12, - AE15, AE17-AE21; KTD9, KTD11-KTD12. -- **Files:** `apps/gateway` Codex adapter, typed profile projection, environment - policy, SDK thread/checkpoint fixtures, shared conformance tests, and optional - credentialed smoke tests. -- **Approach:** Use pinned `@openai/codex-sdk` first. Call `startThread()` for a - session start, `resumeThread(threadId)` for each later turn, and a fresh thread - for one-shot execution; pass resolved cwd, access mode, Task/session-private - runtime/state home, and typed profile settings. Persist only the opaque thread - ID after successful turn settlement. For `readOnly`, request native read-only - policy when supported and keep preparation outside the base. Pass API - credentials through the explicit allowlist, or reference an existing ChatGPT - login only through the U0-proven separate auth mechanism; never point mutable - SDK thread state at the host auth home. Stream/normalize events and native - usage including cached input; connect abort to U3; bound evidence; and dispose - private state on close. Use app-server only if U0 recorded a specific required - SDK gap and pin/probe its protocol. Pass native `outputSchema` only for the - supported Structured Outputs subset; otherwise add JSON guidance and use the - common terminal validator. -- **Execution note:** Characterize pinned SDK/model auth, abort, event, tool, and - schema behavior before normalization. Provider and model tools retain trusted - CI-job authority; tests inspect the explicit environment but make no hostile- - code, network, or secret-isolation claim. Do not import Promptfoo or AI SDK - Harnesses and do not download Codex per request. -- **Verification:** Shared adapter conformance; built-in/profile targets; API - credentials and any advertised ChatGPT-login path each prove host-auth/private- - state separation; start/two-turn resume/close, exact opaque checkpoint, - conversation/workspace continuity, close cleanup, native cached-input - reporting without a required hit, environment allowlist, exact override probe, - event/usage/result normalization, graceful/TERM/KILL cancellation, native- - schema and validated-fallback paths, deadline, malformed provider payload, and - opt-in credentialed smoke pass outside Docker. - -### U6. Pi RPC adapter - -- **Goal:** Run built-in and profile-backed Pi targets on the trusted host through - the pinned supported package/RPC surface with the same one-shot/session - lifecycle and honest capability/cache reporting. -- **Requirements:** R7-R8, R13-R16, R18; AE3-AE4, AE10-AE12, AE15, - AE17-AE21; KTD9, KTD11-KTD12. -- **Files:** `apps/gateway` Pi adapter/RPC parser, restricted policy extension, - typed profile projection, environment policy, session checkpoint fixtures, - shared conformance tests, and optional credentialed smoke tests. -- **Approach:** Launch Pi directly in resolved cwd with access mode, - Task/session-private runtime/state paths, typed invocation configuration, - strict RPC, explicit supported tools/extensions, deterministic permissions, - validated events, bounded evidence, and U3 cancellation escalation. Reference - a host Pi auth location only through the U0-proven separate mechanism. Use the - pinned native create/resume/checkpoint/dispose surface for Sessions; advertise - only modes whose auth/state/checkpoint combination passes. Request native - read-only policy when supported. Never copy, mount, parse, or import Pi auth. - Repository extensions and unrestricted built-ins remain disabled. A global Pi - binary override must pass the exact pinned version/protocol probe. -- **Execution note:** Characterize and pin Pi's RPC/auth/session/abort/event - contract. Pi-specific facts remain bounded native evidence rather than public - schema branches. Model tools retain trusted CI-job authority; do not claim the - explicit environment isolates provider/MCP/operator secrets. -- **Verification:** Shared adapter conformance; built-in/profile targets; - host-auth/private-state separation; advertised modes match one-shot/create/ - resume/checkpoint/dispose probes; two-turn continuity and close when available; - native cache usage only; environment allowlist; exact override probe; strict - malformed/unknown RPC rejection; event/usage/result normalization; graceful/ - TERM/KILL cancellation; deadline; and opt-in credentialed smoke outside - Docker. Malformed RPC can never produce success. - -### U7. End-to-end delivery and documentation - -- **Goal:** Prove independently released Bun CLI/gateway packages and the exact - acquisition image, then document the trusted-network and trusted-runner model, - workspace/source configuration, host auth, registry coverage, Promptfoo - consumption, installation, release ordering, and limits. -- **Requirements:** R1-R19; F1-F6; AE1-AE21; KTD1-KTD14. -- **Files:** published profile/snapshot-format pages, gateway guide/reference, - configuration reference, README, CHANGELOGs, real project/user workspaces, - AI Evals-style Promptfoo YAML/provider contract fixture, E2E fixtures, - CLI-only and gateway packed-install smokes, acquisition-image release record, - GHCR/JFrog reports, size/SBOM evidence, and independent release evidence. -- **Approach:** After final review, pack `apps/cli` and `apps/gateway` - independently without publishing. Prove CLI-only installation resolves - neither gateway nor image; install the gateway tarball in a clean trusted - Linux environment with Docker and pre-existing supported Codex/Pi host auth. - Create project/user workspaces under `/tmp/`; serve on loopback and a specific - private TLS address, plus a loopback-only fake private-TLS proxy; reject - wildcard/public listeners; test probes and the complete A2A lifecycle; acquire - local Git/OCI plus live registry fixtures through the exact image; and run - Promptfoo consumer fixture in both source modes with per-trial logical cwd and - workspace access. Prove read-only Tasks reuse one immutable base without - shared runtime state, read-write Tasks receive independent disposable views, - and requests carry only logical source, cwd, and access data. - - Run registry workflows with the exact gateway tarball, acquisition manifest, - supported platform digests, and build commit. Gateway publication is blocked - until the image has passed required GHCR/JFrog conformance. Publish the - transport-neutral HEC Core and Sessions contracts, A2A binding, - coding-workspace extension, composed Profile URN, and conformance entry points - without claiming a neutral standard or Responses/UHP binding. Document that - network peers have full Task/session authority, providers/model tools have - CI-job authority, explicit environments are not isolation, evidence follows - only direct-process settlement, Docker is acquisition-only, and ephemeral - runner teardown is the final orphan boundary. -- **Execution note:** Green smoke uses the release-candidate npm tarball and - exact acquisition image artifacts, never a checkout rebuild. The consumer - fixture is AI Evals-owned test/documentation code; AllAgents runtime does not - import Promptfoo. -- **Verification:** CLI-only/gateway clean installs and sizes, independent - release dry runs, compatibility/image mismatch fixtures, local Distribution - and public digest-pinned GHCR on every PR, authenticated GHCR and private-CA - JFrog release conformance against exact artifacts, complete A2A/Task/provider/ - acquisition E2E, Promptfoo contract fixture, Bun typecheck/lint/test/build, - dependency/image scans, generated contract/docs drift, docs build, and exact - red/green commands/results in the PR description. +### U0. HarnessRouter fork and hook feasibility + +- **Goal:** Prove the smallest production-direction fork can materialize and + durably checkpoint one workspace before provider dispatch while preserving + stock UHP requests. +- **Repositories/files:** HarnessRouter fork `gateway/app.py`, + `runner/server.py`, response/session persistence, checkpoint/produced-file + helpers, runner/gateway tests, image entrypoint/Dockerfile, and fake hook. +- **Approach:** Pin upstream. Add configured opaque metadata extraction and + bounds, generic result envelope, materialization CAS, dedicated runner + operation before the provider loop, response-translator persistence, safe + nested cwd, staged publication, pre-agent checkpoint, and nested-repository + collection. Add startup preflight. Configure broker mode and probe Codex plus + optional Pi route separately. +- **Verification:** Upstream UHP conformance stays green. Stock requests are + unchanged. Faults at every state/publication/checkpoint boundary fail closed. + Provider fallback cannot rerun the hook. A second turn reuses workspace and + provenance without the extension. Root and nested repository files collect + correctly, escaping cwd fails, broker secrets stay outside the agent, and Pi + advertisement exactly matches its probe. + +### U1. AllAgents workspace contracts and Git materializer + +- **Goal:** Implement the versioned schemas, authoritative catalog projection, + deterministic Git acquisition, logical cwd resolution, and provenance. +- **Files:** `src/models/workspace-config.ts`, + `src/models/execution-workspace.ts`, `src/core/execution-workspace.ts`, + `src/core/workspace-repo.ts`, `src/cli/commands/workspace.ts` or one narrowly + registered integration command, generated workspace schemas, build packaging, + configuration documentation, unit fixtures, and Git E2E fixtures. +- **Approach:** Reuse authoritative workspace parsing and source normalization. + Extend the project schema with strict snapshot and environment credential + references; add descriptor/hook/result schemas, defaults, canonicalization, + and a `preflight` mode. Expose a direct no-shell materializer entrypoint. + Resolve the complete execution-eligible catalog to exact commits in staging, + preserve nested `.git`, enforce the closed Git policy, validate destinations + and cwd, compute the workspace manifest, and return without publishing. +- **Verification:** Local HTTPS fixtures cover branches, tags, commits, PR refs, + configured defaults and symbolic HEAD, multiple repositories, optional-name + fallback, conflicting/originless/local sources, root/duplicate destinations, + unknown names, missing directories, traversal, leading-dash/control/refspec + revisions, ambiguous shorthand, hooks/helpers/filters, submodules, LFS, + file/ext protocols, redirects, cancellation, timeout, partial cleanup, + canonical defaults, preflight failures, and exact provenance. + +### U2. Session binding, failures, and credential containment + +- **Goal:** Make the fork/materializer boundary durable, fail-closed, and safe for + continued sessions. +- **Repositories/files:** HarnessRouter session/response persistence and tests; + AllAgents credential-selection/environment code and hostile fixtures. +- **Approach:** Persist `unbound/materializing/ready/failed`, opaque request + digest, effective descriptor digest, public provenance, workspace marker, and + checkpoint digest through compare-and-set transitions. Extend every response + construction/retrieval/replay path with identical public metadata. Resolve + source `${ENV_VAR}` references only in the child, remove the complete + materializer-only name set from agent children, broker the provider key, and + scan workspace, nested Git, CLI state, checkpoint, logs, and responses. +- **Verification:** Initial idempotent replay preserves one result; continuation + omits the extension and reuses ready state; extension-bearing continuation + fails. Crashes around hook/publication/checkpoint/CAS reconcile to ready only + when the bound descriptor, published marker, and durable checkpoint all match; + missing/corrupt markers, descriptor mismatch, and checkpoint mismatch become + failed/non-resumable. Acquisition secrets, including a non-secret-looking + configured variable name, and the long-lived `codex-lb` key are absent + everywhere the shell-enabled agent can read. + +### U3. Immutable OCI workspace materialization + +- **Goal:** Add the second closed source mode without weakening Git behavior or + allowing fallback. +- **Files:** AllAgents OCI client, manifest/archive validator, workspace-manifest + types, deterministic snapshot producer fixture, local registry E2E, and + security fixtures. +- **Approach:** Resolve only configured registries, implement bounded + Basic/Bearer authentication and exact-host redirect policy, verify direct + manifest/config/layers while streaming, apply changesets in staging, validate + paths/types/limits/catalog, and return through the same hook envelope as Git. + The HarnessRouter runner remains the sole publisher. +- **Verification:** Local Distribution fixtures cover anonymous and authenticated + pulls, private CA, gzip/zstd, whiteouts, digest/size mismatch, redirects, + rebinding policy, indexes, unknown/foreign media, traversal, escaping links, + devices, sparse files, limit overflow, cancellation, cleanup, and no Git + fallback. + +### U4. Codex, optional Pi, `codex-lb`, and Promptfoo E2E + +- **Goal:** Prove the real execution path, broker boundary, session continuity, + and consumer success/failure mapping. +- **Repositories/files:** custom image/configuration, HarnessRouter integration + fixtures, AI Evals Promptfoo provider/configuration in its owning repository, + and deployment examples in AllAgents docs. +- **Approach:** Configure Codex with HarnessRouter's custom Responses integration, + the `codex-lb` base, provider `name = "openai"`, and + `requires_openai_auth = true`. Enable private broker mode and `codex-lb` proxy + API-key authentication so only a turn credential enters the runner. Probe Pi + separately with a supported custom format and documented `codex-lb` endpoint; + advertise it only on success. Run Promptfoo Git/OCI one-shot and two-turn cases + plus materializer and agent failures. +- **Verification:** Through the exact container network, `/responses`, + `/responses/compact`, and a resumed turn authenticate and use an allowed model. + Turn two sees turn one's conversation and file mutation; `codex-lb` + continuation affinity holds; cancellation terminates the real turn; hop keys + and OAuth stay in their owning services; Pi capability is truthful; Promptfoo + returns successful output/usage/artifacts/provenance and maps both failure + classes to failed, coded responses rather than empty success. + +### U5. Release, operations, review, and upstream preparation + +- **Goal:** Produce a reproducible supported image and an upstream-ready generic + hook proposal. +- **Repositories/files:** image build/release workflow, dependency lock and + provenance record, fork-maintenance guide, deployment/reference docs, + changelog, PR descriptions, and upstream patch series. +- **Approach:** Build from exact upstream/fork/AllAgents/agent inputs, emit image + digest and SBOM, run all conformance and E2E gates against that image, document + durable volumes, keys, private networking, upgrades, rollback, backup, failure + recovery, and the CE isolation boundary. Review both codebases before the final + green E2E. Split and explain the generic HarnessRouter patch for upstream. +- **Verification:** A clean host can deploy the recorded image and reproduce Git, + OCI, one-shot, continuation, cancellation, restart, and failure scenarios from + documented commands. Rebase rehearsal against the selected next upstream + commit either passes or reports an explicit incompatibility before release. --- ## Verification Contract -| Gate | Applies to | Required evidence | -|---|---|---| -| Bun architecture feasibility | U0 | Exact A2A SDK pin and official-client operations; HEC Core/Sessions vectors through an in-memory adapter/A2A binding; two-turn Promptfoo context/head handoff; pinned Codex/Pi mode, checkpoint, auth/private-state, and cache probes; shared read-only base/private runtime; read-write prior/candidate/commit/crash probes across materializers; abort/TERM/KILL plus observed-outliving lease retention and truthful unobserved-descendant limit; multi-architecture acquirer image; independent packed installs; immutable tarball/image binding | -| Bun repository quality | U0-U7 | One lockfile; private root orchestration; workspace-scoped typecheck/lint/test/build; dependency and image scans; generated-contract drift; minimized runtime dependencies; packed and installed size budgets | -| Package and release separation | U0-U1, U7 | Independent `allagents` and `allagents-gateway` versions/tags/triggers/tarballs; image-first gateway release; exact npm tarball plus acquisition manifest/platform digests; idempotent publication; CLI-only install fetches neither gateway nor image; gateway-only release never publishes the CLI | -| Workspace and contract packages | U0-U1 | Only `workspace-config`, `execution-contracts`, and `acquisition-contracts` shared packages; separately versioned HEC Core, Sessions, A2A binding, and coding-workspace extension inside `execution-contracts`; generated portable `contracts/` fixtures; normalized catalogs/defaults/order/collision keys/stable errors; project/user parsing; schema/spec drift; no catch-all `core`/`common` package | -| Wire contract | U0-U2 | Official JavaScript client against the Bun gateway; normative transport-neutral HEC Core per-turn Invocation, Sessions start/resume/close/checkpoint semantics, neutral state/cancel outcomes, ordered execution-event stream, Core outcome/trajectory, A2A binding, coding-workspace extension/integrity, and one composed Profile URN; shared Core/Sessions vectors plus binding/extension/composed conformance; A2A card/version/profile, context/session and prior-Task mapping, one-Message/one-Task-per-turn, unified Parts, three reserved Artifacts, both send modes, exact byte-bounded listing, state/cancel/event/Artifact/error projection, metadata, and request/result fixtures | -| Trusted-network and runner model | U2-U7 | Loopback HTTP, loopback behind private HTTPS ingress, and native TLS on a specific private address; wildcard/public listener and public advertised-address rejection; TLS file and topology validation; external ACL enforcement; shared external Task/session visibility and cancellation; trusted Linux CI job/VM/deployment container as provider isolation boundary; one gateway-controlled turn with truthful escaped-descendant limit; read-only cooperative best-effort; explicit no-hostile-code/no-secret-isolation wording; metadata-only probes | -| Durable Task and session lifecycle | U1-U3 | Private gateway-owned `bun:sqlite`; foreign keys and `synchronous=FULL`; transactions for create/replay, exact head compare-and-set, Task/session and prior/candidate byte reservations, base pins, lease, intent, prepared settlement, terminal Task/Artifacts/events, provider-checkpoint plus workspace-generation advancement, superseded-generation cleanup, close, and expiry; crash before/after pointer commit; poisoned state retains resources until reconciliation; no early eviction; process/store faults and restart/tombstone reconciliation | -| Acquisition-container boundary | U0, U4, U7 | One fresh container when no reusable validated base exists and none on cache hit; exact digest-pinned image; staging-only writable bind; selected repository/registry credentials and CA material only; no App private key, host home, Docker socket, gateway state, provider auth, Codex, Pi, or harness downloads; strict source network/size/archive policy; typed manifest; exit/removal before host validation and provider execution; orphan-staging cleanup | -| Immutable-base cache | U0-U4, U7 | Key binds acquisition contract, compiled catalog/layout, and immutable source; exact commit/digest reuse; branch/tag bypass into non-reusable Task/session-owned bases; atomic promotion; active Task/session pins; unpinned LRU byte-budget eviction; one acquisition across repeated immutable read-only trials/turns; no cached credentials; transient-base cleanup | -| Repository acquisition | U0, U4 | Compiled-name resolution; hermetic Git/full commits; App 200/404/ambiguous eligibility; durable conservative-expiry pre-mint intent; exact-expiry token validation/revocation; crash or ambiguous response at every mint/revocation boundary; cache-hit no credential; `gh` only after positive ineligibility; acquisition sub-budget; no provider start on failure | -| OCI acquisition | U4 | Canonical Docker Hub plus GHCR/JFrog/private-registry matrix; exact-key auth/helper/CA; bounded Basic/Bearer; redirect/rebinding policy; direct-image/config/layer media; descriptor verification; path-free manifest/private layout; changesets/whiteouts; fixed limits; no fallback | -| Registry and exact-artifact conformance | U4, U7 | Local Distribution and public digest-pinned GHCR on every PR; authenticated GHCR and private-CA JFrog release targets; exact gateway npm tarball plus acquisition multi-architecture manifest/platform digests without rebuild; positive/negative auth/permission/CA/media/path cases; explicit architecture coverage | -| Host supervisor lifecycle | U3 | One active gateway-controlled lease/turn; reusable-base read-only/private-runtime and non-reusable-base paths; one-shot views versus immutable committed session generations plus per-turn candidates; provider PID/start identity before started state; explicit environment and separated host-auth/private-state paths; cancel/deadline/shutdown races; abort then TERM/KILL; prepared settlement and checkpoint/workspace-pointer commit; cleanup before lease release; non-settling or observed-outliving failure retains Task/session state, lease, and unready status; truthful unobserved-descendant limit | -| Truthful bounded evidence | U3-U6 | Durable live ordered Core progress/tool events; canonical JSON/text payloads; deterministic prefix truncation; one result per call and interrupted-call rules; truthful complete/truncated markers; collection only after direct provider settlement/escalation; hermetic Git inspection; observed termination/cleanup; no claim of full descendant quiescence, hostile-code containment, secret isolation, opaque-output redaction, unobserved events, or inferred cache hits | -| Backend conformance | U0, U2-U3, U5-U6 | Narrow access-aware adapter contract; HEC lifecycle/event/trajectory and Sessions create/resume/checkpoint/dispose suites for fake/Codex/Pi; per-target mode advertisement; private mutable state with proven separate host-auth reference; read-only bases/private runtime, one-shot views, and session candidate/committed generations; resolved cwd/access; native cached-token reporting only; exact binary probes; host execution outside acquisition Docker; no transcript replay, AI SDK Harnesses, or per-request download | -| Structured result | U1, U3, U5-U6 | Public grammar; Codex native-subset gate and validated fallback; valid/invalid/not-produced/retention-limit states; three reserved Artifact cardinalities; oversized or malformed provider/RPC result cannot publish success | -| Repository quality | All | Bun install/typecheck/lint/test/build; focused and full suites; clean-registry packed installs; dependency/image audit; generated schema/spec checks; docs build | -| Packaged gateway E2E | U7 | Recorded red/green `/tmp/` commands; exact gateway/image artifacts; Git/local OCI/GHCR/JFrog; one-shot reuse/cleanup; read-write candidate commit/crash recovery; two-turn session continuity/close/expiry; direct-private TLS and loopback/private-proxy topology plus wildcard/public rejection; capacity/replay/cancel/deadline/shutdown/restart; separated Codex/Pi auth/private state; truthful trust/cache docs | -| Promptfoo consumption | U7 | Secure-default AI Evals YAML for both source modes; optional context; strict logical override/session inputs; provider-managed `allagentsConversation` map keyed by explicit ID with isolated interleaving and start/continue/close; manual `allagentsSession` crash recovery; two terminal Tasks with exact returned context/head; one-shot base/view behavior; nonblocking subscribe/cancel; provenance and output/usage/native-cache/error metadata; no AllAgents Promptfoo dependency | +| Gate | Required evidence | +|---|---| +| Stock compatibility | Upstream HarnessRouter tests and UHP conformance pass; requests without the configured metadata key are unchanged. | +| Hook ordering | Dedicated materialization finishes, publishes, and checkpoints before provider selection; fallback never reruns it. | +| Durable state | Fault injection proves only sessions with matching bound descriptor, published marker, and durable checkpoint become ready; missing/corrupt/mismatched evidence fails non-resumable without replay. | +| Session continuity | Two real turns share native conversation and writable workspace; continuation omits the extension and invocation count is one. | +| Workspace integration | Safe nested cwd, repository-mode root/nested Git checkpoints, snapshot private tree baselines, produced list/file/ack, hydrate, and initial-source suppression pass. | +| Git acquisition | Closed transport/config policy, constrained revisions, exact commits, non-root destinations, catalog validation, and partial cleanup pass against local HTTPS remotes. | +| OCI acquisition | Digest/media/path/link/type/limit matrix passes against a real local registry. | +| Credential boundary | Every configured materializer-only environment name, all source secrets, and the long-lived `codex-lb` key are absent from every agent-readable environment/file/checkpoint and public output. | +| Provider boundary | Private broker minting works with the configured private public-base/gateway URL; Codex uses authenticated brokered `/responses` and `/responses/compact` with required provider fields; Pi is advertised only after its separate route probe; OAuth remains solely in `codex-lb`. | +| Lifecycle | Streaming, cancellation, idempotency, artifacts, usage, response metadata, completed restart, and interrupted-work failure match the contract. | +| Packaging | Exact image digest records upstream commit, patch digest, materializer contract, agent versions, and SBOM. | +| Consumer | Promptfoo one-shot/two-turn Git/OCI success and materializer/agent failure mappings pass. | +| Review | Final review findings in both repositories are resolved before the final built-image E2E. | ## Definition of Done -### Global - -- Every R1-R19 requirement is implemented or explicitly demonstrated by a - passing acceptance scenario; F1-F6 and AE1-AE21 agree with the implementation - and error table. -- U0's HEC Core/Sessions/A2A-binding, Bun, provider, process, acquirer, and - package gates - pass before dependent units. The private root, three apps, three named - packages, and generated `contracts/` fixtures are the complete shared layout; - no speculative shared package, native sidecar, or split runtime remains. -- `allagents` and `allagents-gateway` remain independently versioned and - released. A CLI-only install fetches neither gateway nor acquisition image. - Gateway release builds/verifies the exact acquisition image and registry - reports before publishing the bound npm tarball. -- The gateway starts without `gateway.yaml` or `worker.yaml` and defaults to - loopback HTTP. Remote service is loopback-only behind private HTTPS ingress or - native TLS on one specific private address. Startup rejects wildcard/public - binds, missing/mismatched TLS inputs, public advertised resolution, and remote - HTTP; health/readiness remain metadata-only. -- Network reachability within that private boundary authorizes callers; Task and - session visibility/idempotency are deployment-wide. Provider execution uses - the trusted Linux CI job/VM/deployment-container boundary. Host authentication - is referenced only through an adapter-proven path separate from private - mutable provider state. Documentation says AllAgents does not contain hostile - repository code or isolate provider/MCP/operator secrets from model tools. -- Project workspace declarations compile to the exact repository/snapshot - catalog; user declarations own profile launcher gateway enablement; built-in - IDs cannot be shadowed. Requests may select only the effective workspace root - or a declared repository plus a bounded relative directory and may select only - `readOnly | readWrite` access. They cannot supply physical/configured - destination paths, origins, credentials, commands, provider environments, - materializers, cache keys, Docker images/options/mounts, or provider permission - policy. -- The published transport-neutral HEC Core Invocation, neutral state/cancel - outcomes, ordered canonical progress/tool events, deterministic trajectory, - outcome schema, Sessions extension, A2A binding, AllAgents coding-workspace - extension, and composed v1 Profile URN pass common Core/Session suites plus - official-client, binding-specific, extension, and composed-profile suites. - Binding tests own Agent Card/version/activation, context-ID/session and prior- - Task mapping, one-Message/one-Task-per-turn, Core-to-A2A state/cancel/event/ - Artifact/error mapping, unified Parts, three reserved Artifacts, both send - modes, exact byte-bounded `ListTasks`, metadata, canonicalization, retention, - and cancellation. Sessions tests own start/resume/close, exact head, - linearization, pinned configuration, expiry, restart/poison handling, and - cache-usage truthfulness. Workspace tests own logical cwd/access, source/ - provenance, candidate/committed session generations, produced bytes, - integrity, and cleanup. - The release calls HEC a contract/standard candidate, not a neutral standard, - and exposes no Responses/UHP binding in v1. -- Secure-default AI Evals Promptfoo YAML selects repository mode with optional - named revisions or snapshot mode with immutable digests, can replace logical - cwd/access per trial, and can start/resume/close a linear session. Each - `callApi` maps to one nonblocking Task; resumed Tasks retain provider context - and the session workspace. Ambiguous retries retain the same invocation key, - context/head, base/view, cwd, and access. The provider propagates cancellation, - normalizes usage including native cached-input tokens, and returns safe - failure/logical provenance without origins, configured destinations, physical - paths, or an AllAgents Promptfoo dependency. -- Every request without a reusable validated base starts the exact digest-pinned - image with staging as its only writable bind plus source-only credentials and - strict policy. A valid cache hit starts no container and resolves no - credential. The image has - no host home, Docker socket, gateway state, provider auth, Codex, Pi, or - harness download path. It emits a typed manifest, exits, and is removed before - host validation, immutable-base publication, typed preparation, or provider - execution. Git and OCI modes produce one path-free manifest contract while the - host validates exact private destinations. OCI v1 remains registry-neutral - across Docker Hub, GHCR, JFrog, and compatible private registries with - immutable digests, descriptor verification, exact redirect/auth/CA rules, - changesets, fixed limits, no cross-mode fallback, and truthful source - verification. -- App eligibility/ambiguous 404 handling, durable pre-mint intent, - exact-expiry token validation/revocation, ambiguous-response and crash - tombstones through every mint/revocation boundary, cache-hit credential - avoidance, positive-ineligibility selection, OCI auth/challenges, exact-host - CA, and acquisition credential teardown pass. Local Distribution and public - digest-pinned GHCR run on every PR; authenticated GHCR and private-CA JFrog - release reports match the exact gateway tarball, acquisition manifest, - supported platform digests, commit, and compatibility output. -- Codex uses pinned `@openai/codex-sdk` first; app-server is used only for a - recorded SDK capability gap. Codex/Pi advertise only auth/mode combinations - that prove host-auth references are separate from Task/session-private mutable - state. Auth files are never copied, mounted, parsed, or imported, and provider - runtimes are never downloaded per request. -- Direct providers start in Linux process groups with explicit environments and - separated auth/private-state paths. One-shot read-only Tasks and read-only - sessions use shared immutable bases with private runtime; one-shot read-write - Tasks use disposable views; read-write sessions use immutable committed - generations plus per-turn candidates. Cancellation escalates adapter abort to - `SIGTERM`/`SIGKILL`. Evidence starts only after the direct provider settles. A - non-settling provider or observed outliving descendant publishes no - filesystem/Git evidence, retains Task/session state plus the lease, and blocks - readiness until verified disappearance/reconciliation or runner teardown. - Evidence reports observed cleanup, not descendant containment. -- Private Bun SQLite ownership, foreign keys, and full synchronization cover - create/replay, base pins, one lease, outcome races, prepared Task/Artifact/ - event settlement, provider-checkpoint/workspace-generation advancement, - old-generation cleanup, terminal-tail/aggregate reservations, immutable Tasks, - poison-exempt expiry, crash around every pointer/cleanup boundary, restart, - mint/revocation tombstones, retained-count/per-Task/aggregate/response/session - byte limits, cache eviction, and cleanup fault tests without a custom VFS or - native file layer. -- Evaluation orchestration, public-Internet authentication/exposure, remote - workers, Responses/UHP binding, session branching/concurrent turns, - caller-selected custom materializers, per-provider Docker, native containment - primitives, non-Linux gateway execution, and multi-tenant policy remain absent. - -### Per unit - -- U0: HEC Core/Sessions vectors pass through an in-memory adapter/A2A binding; - official-client, composed-profile, and two-turn Promptfoo context/head probes - pass; anonymous-conformance is recorded; pinned Codex/Pi mode/checkpoint, - auth/private-state, and native-cache probes pass; read-only/private-runtime and - read-write prior/candidate/commit/crash materializer probes pass; process-group - escalation plus observed-outliving lease retention and truthful unobserved- - descendant limits pass; multi-architecture image, independent packed installs, - and exact tarball/image release binding pass. -- U1: Three narrow packages and generated fixtures, workspace additions, - normative HEC Core/Sessions/A2A-binding/coding-workspace-extension text and - separate conformance groups, composed v1 wire contract, Core Invocation/state/ - cancel/event/outcome/trajectory semantics, Sessions identity/head/checkpoint/ - expiry/close semantics, binding-owned context IDs and prior-Task references, - three reserved Artifacts, execution/acquisition contracts including logical - cwd, workspace access, relative-path grammar, base-cache identity, and - materialization errors, Bun SQLite Task/session transactions/reservations and - byte budgets, published contract/snapshot format, independent versions, - compatibility matrix, and image-first release fixtures agree. -- U2: Official-client operations, version/profile/error/list and response-budget - semantics, Core/Sessions-to-A2A state/cancel/event/context/head/Artifact - mapping, logical cwd/access defaults/canonicalization/workspace integrity, - deployment-wide replay/visibility, linear start/resume/close/expiry and - stale-head/busy rejection, SQLite lock/crash/lease/reservation behavior, - listeners/advertised URL/probes, deadline/shutdown/restart/retention, and fake - backend pass. -- U3: The fake lifecycle proves shared immutable-base read-only/private-runtime - execution, one-shot views, and read-write session prior/candidate/committed - generations across every materializer; candidate reservations; cwd/escape - rejection; separated auth/private-state environments; durable events/ - trajectory; process-group escalation and observed-outliving lease retention; - outcome races; prepared Task/Artifact/event settlement plus atomic provider- - checkpoint/workspace-pointer advancement; crashes before/after pointer commit; - cleanup/poison/restart behavior; byte bounds; and truthful orphan limits. -- U4: Git/OCI fixtures, App/`gh` selection, pre-mint intent and mint/revocation - tombstone reconciliation, cache hit/miss/key/pin/eviction, exact acquisition - image boundary, staging-only mount, source credentials/network/limits, typed - manifest, host revalidation/publication, local/public/authenticated GHCR, - private-CA JFrog, exact manifest/platform digests, no fallback, and leak scans - pass. -- U5: Codex passes shared one-shot/session, access, event, cancellation, - checkpoint, cache-reporting, and private-state/host-auth separation conformance - through the pinned SDK or documented app-server fallback, receives resolved - cwd/runtime/access, runs outside Docker, and records optional credentialed - smoke evidence. -- U6: Pi passes the same host-process/auth-state conformance through the pinned - RPC/package surface and cannot publish success from malformed RPC. Each mode is - advertised only when its one-shot/create/resume/checkpoint/dispose probes pass. -- U7: Final review is resolved; packed smokes and `/tmp/` E2E, independent - release/size/SBOM evidence, public GHCR, authenticated GHCR/private-CA JFrog - exact-artifact reports, Promptfoo one-shot and isolated/interleaved two-turn - conversation-map plus explicit-session recovery fixtures, logical cwd/access, - truthful cached-input usage, repository gates, published schemas/specs, - threat-model docs, and reproducible PR instructions are complete. +- ADR 0002, this plan, implementation, deployment topology, and request examples + agree on UHP, the fork, the behind-router materializer, and `codex-lb`. +- No A2A, HEC, `allagents-gateway`, custom task/session store, direct provider + adapter, or client-side workspace expansion remains in implementation scope. +- R1-R15 and AE1-AE13 are implemented and verified against the exact released + image. +- Stock UHP requests and upstream conformance remain green. +- Git and OCI materialization publish and checkpoint before provider dispatch and + happen exactly once per extension-bearing session. +- Continuation preserves native conversation/workspace state, omits the + extension, and follows HarnessRouter's predecessor semantics. +- Repository-mode roots preserve usable Git state. Every source mode preserves + truthful root/nested produced files across checkpoint/hydrate. +- Acquisition credentials and provider OAuth respect their separate boundaries. +- The image, patch series, materializer contract, operational docs, and rollback + procedure are reproducible from pinned inputs. +- The generic HarnessRouter hook patch is ready to propose upstream, but the + shipped system remains operable from the maintained fork if it is not accepted. diff --git a/docs/research/agent-host-protocol-decision-inputs.md b/docs/research/agent-host-protocol-decision-inputs.md index e49d39f8..d49cc824 100644 --- a/docs/research/agent-host-protocol-decision-inputs.md +++ b/docs/research/agent-host-protocol-decision-inputs.md @@ -1,5 +1,9 @@ # Agent Host Protocol decision inputs for the execution gateway +> Historical decision input. [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) +> superseded this A2A recommendation after selecting UHP through HarnessRouter. +> Retained as research provenance, not current implementation direction. + ## Decision Keep A2A 1.0 plus the versioned AllAgents extension as the execution gateway's @@ -102,7 +106,7 @@ The conclusion is based on: - direct inspection of `microsoft/vscode` commit [`046944034292b5479b4e9a50ad1a508033ffb64f`](https://github.com/microsoft/vscode/tree/046944034292b5479b4e9a50ad1a508033ffb64f), whose generated registry identifies AHP `0.9.0`; and -- [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md). +- [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md). The inspected implementation demonstrates host-owned state/sequencing, provider-neutral adapters for Copilot, Claude, and Codex, layered persistence, From ae26f69470c091db42d1e163b08de0fe35baaaad Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Tue, 22 Sep 2026 12:48:04 +1000 Subject: [PATCH 15/44] docs(architecture): present HarnessRouter design standalone --- .../0002-adopt-uhp-through-harnessrouter.md | 47 +++--- ...0837-feat-coding-execution-gateway-plan.md | 34 +++-- .../agent-host-protocol-decision-inputs.md | 116 -------------- .../harbor-repository-materialization.md | 104 ++++++------- .../source-credential-broker-precedents.md | 141 ++++++++---------- 5 files changed, 155 insertions(+), 287 deletions(-) delete mode 100644 docs/research/agent-host-protocol-decision-inputs.md diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 04f1c921..ee3bd9e6 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -1,9 +1,7 @@ # ADR 0002: Adopt UHP through HarnessRouter with an AllAgents workspace materializer - Status: Accepted; implementation pending -- Date: 2026-09-17 -- Updated: 2026-09-21 -- Supersedes: the original A2A/HEC execution-gateway design recorded by this ADR +- Date: 2026-09-21 ## Decision @@ -12,11 +10,11 @@ pinned HarnessRouter Community Edition deployment for remote Codex execution. Pi remains capability-gated until a real probe proves a HarnessRouter-supported custom-provider format against `codex-lb`. -AllAgents will not implement an A2A gateway, a separate Harness Execution -Contract, provider adapters, or its own task/session engine. HarnessRouter owns -authentication, UHP request and response semantics, streaming, cancellation, -idempotency, session continuity, per-session workspaces, agent execution, usage, -and artifacts. +HarnessRouter owns authentication, UHP request and response semantics, streaming, +cancellation, idempotency, session continuity, per-session workspaces, agent +execution, usage, and artifacts. AllAgents owns the workspace descriptor, +deterministic source materialization, and provenance returned through the +HarnessRouter response. The initial deployment will use a narrow AllAgents-maintained HarnessRouter fork. The fork adds a generic pre-turn workspace-materializer hook. An AllAgents @@ -120,9 +118,9 @@ acquisition failure starts no agent process and never falls through to another source mode or credential identity. Workspaces are writable and private to the HarnessRouter session. Version one -does not add the former `readOnly` optimization or copy-on-write generations. -The fork extends HarnessRouter's existing workspace checkpoint with a durable -pre-agent materialization state and nested-repository collection metadata. +does not provide read-only workspaces or copy-on-write generations. The fork +extends HarnessRouter's existing workspace checkpoint with a durable pre-agent +materialization state and nested-repository collection metadata. Completed session state survives a HarnessRouter restart when its documented durable data volume is preserved. An in-flight agent process does not survive @@ -250,28 +248,21 @@ complete workspace identity. ## Consequences -This decision deletes substantial custom scope: +HarnessRouter is the execution control plane. AllAgents does not add a parallel +task/session store, streaming lifecycle, process supervisor, artifact service, +provider adapter, or Promptfoo-specific runtime. -- no A2A server or Agent Card; -- no HEC schemas, bindings, or conformance suite; -- no AllAgents Task/session SQLite store; -- no custom SSE lifecycle or cancellation protocol; -- no direct Codex SDK or Pi RPC adapters; -- no custom process supervisor or artifact store; -- no separate `allagents-gateway` npm product; and -- no Promptfoo-specific runtime in AllAgents. +AllAgents owns workspace selection, deterministic Git/OCI materialization, +source credentials, provenance, the HarnessRouter integration patch, and +deployment documentation. -AllAgents instead owns the smaller differentiated surface: workspace selection, -deterministic Git/OCI materialization, source credentials, provenance, the -HarnessRouter integration patch, and deployment documentation. - -The cost is a temporary fork and custom image. The fork must be rebased and -tested against upstream releases until the generic hook is accepted or an -equivalent supported extension exists. +The integration requires a maintained fork and custom image. The fork must be +rebased and tested against upstream releases until the generic hook is accepted +or an equivalent supported extension exists. ## Alternatives rejected -- **Custom A2A/HEC gateway:** duplicates mature UHP/HarnessRouter session, +- **Custom execution gateway:** duplicates mature UHP/HarnessRouter session, streaming, cancellation, authentication, artifact, and provider behavior. - **Thin adapter in front of stock HarnessRouter:** avoids a fork but introduces another network service and makes source acquisition a client-side concern. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 600d5e67..6faae67d 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -45,7 +45,8 @@ execution: code if either source credentials or the long-lived `codex-lb` key reach the agent, or if the fork cannot preserve stock UHP behavior and conformance. Do not fall back to prompt instructions, an MCP acquisition tool, client-side repository - upload, A2A/HEC, owner-trust provider keys, or a new task/session engine. + upload, owner-trust provider keys, a second execution protocol, or a parallel + task/session engine. - **Tail ownership:** Implementation owns focused tests in both repositories, upstream UHP conformance, built-image smoke tests, exact Git/OCI E2E, two-turn Promptfoo success and failure verification, credential leak checks, @@ -57,7 +58,7 @@ execution: code ### Summary -AllAgents uses HarnessRouter as the execution gateway instead of building one. +AllAgents uses HarnessRouter as the execution gateway. HarnessRouter exposes UHP, authenticates callers, creates and persists sessions, streams events, runs Codex and capability-gated Pi, handles cancellation and idempotency, and returns output, usage, and artifacts. @@ -78,10 +79,10 @@ beyond HarnessRouter's UHP behavior. ### Problem Frame -Stock HarnessRouter already implements the expensive generic execution concerns. -Rebuilding those concerns behind A2A would add a second protocol, lifecycle, -session store, process supervisor, artifact model, provider integration, and -conformance burden without differentiating AllAgents. +HarnessRouter already implements the generic execution concerns. Reimplementing +them in AllAgents would add a second protocol, lifecycle, session store, process +supervisor, artifact model, provider integration, and conformance burden without +differentiating the product. Stock HarnessRouter does not expose a documented generic pre-turn seam for a server-side Git/OCI descriptor. It does not copy arbitrary request metadata into @@ -115,8 +116,8 @@ caller responsible for acquisition. The temporary fork closes those seams. ### Key Decisions -- **Use UHP, not A2A or HEC.** UHP `2026-09-12` is the only northbound execution - contract. HarnessRouter conformance is authoritative. +- **Use UHP as the northbound contract.** UHP `2026-09-12` is the only + northbound execution contract. HarnessRouter conformance is authoritative. - **Fork narrowly and upstream later.** Delivery uses an AllAgents-maintained fork. The upstreamable layer is a configured opaque-metadata key, immutable first-turn binding, typed command envelope, durable pre-provider lifecycle, @@ -135,9 +136,9 @@ caller responsible for acquisition. The temporary fork closes those seams. the agent. Only `codex-lb` handles provider OAuth. - **Preserve stock UHP requests.** Requests without the configured metadata key behave exactly as upstream. -- **Use a custom image, not an `allagents-gateway` package.** The image combines a - pinned HarnessRouter revision, reviewed patch series, pinned agent runtimes, - and the AllAgents materializer executable. +- **Use a custom HarnessRouter image.** The image combines a pinned HarnessRouter + revision, reviewed patch series, pinned agent runtimes, and the AllAgents + materializer executable. ### Requirements @@ -444,9 +445,9 @@ caller responsible for acquisition. The temporary fork closes those seams. **Out of scope** -- A2A, HEC, Agent Cards, or a second execution protocol. -- An `allagents-gateway` server, task database, session engine, process - supervisor, Codex SDK adapter, Pi RPC adapter, or artifact service. +- A second northbound execution protocol or parallel task/session control plane. +- A separate AllAgents network gateway, process supervisor, provider adapter, or + artifact service. - Promptfoo runtime code inside AllAgents. - Provider OAuth handling outside `codex-lb`. - Caller-provided origins, credentials, commands, host paths, materializers, or @@ -815,8 +816,9 @@ published. - ADR 0002, this plan, implementation, deployment topology, and request examples agree on UHP, the fork, the behind-router materializer, and `codex-lb`. -- No A2A, HEC, `allagents-gateway`, custom task/session store, direct provider - adapter, or client-side workspace expansion remains in implementation scope. +- No second execution protocol, parallel task/session control plane, separate + AllAgents gateway, direct provider adapter, or client-side workspace expansion + remains in implementation scope. - R1-R15 and AE1-AE13 are implemented and verified against the exact released image. - Stock UHP requests and upstream conformance remain green. diff --git a/docs/research/agent-host-protocol-decision-inputs.md b/docs/research/agent-host-protocol-decision-inputs.md deleted file mode 100644 index d49cc824..00000000 --- a/docs/research/agent-host-protocol-decision-inputs.md +++ /dev/null @@ -1,116 +0,0 @@ -# Agent Host Protocol decision inputs for the execution gateway - -> Historical decision input. [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) -> superseded this A2A recommendation after selecting UHP through HarnessRouter. -> Retained as research provenance, not current implementation direction. - -## Decision - -Keep A2A 1.0 plus the versioned AllAgents extension as the execution gateway's -northbound contract. Treat the Agent Host Protocol (AHP) as an optional future -protocol behind the gateway for a compatible backend or beside it for a -collaborative session client. - -AHP does not replace ADR 0002's deployment-wide Task identity and idempotency, -network trust boundary, immutable source handling, cleanup, terminal evidence, -or bounded result retention. - -The initial backend set is Codex and Pi; OpenCode is deferred. They are peer -execution adapters behind one conformance contract; provider-specific process, -session, permission, cancellation, and evidence behavior stays below that seam. - -This note records the AllAgents-specific consequences. The reusable research, -source inspection, and full protocol comparison live in the AI Research Wiki: - -- [Agent Host Protocol](https://github.com/tsoyang-org/ai-research-wiki/blob/main/entities/agent-host-protocol.md) -- [Agent Host Architecture](https://github.com/tsoyang-org/ai-research-wiki/blob/main/concepts/agent-host-architecture.md) -- [Agent Host Protocol vs Agent2Agent](https://github.com/tsoyang-org/ai-research-wiki/blob/main/comparisons/agent-host-protocol-vs-agent2agent.md) -- [VS Code Agent Host source note](https://github.com/tsoyang-org/ai-research-wiki/blob/main/raw/articles/vscode-agent-host-architecture.md) - -## Boundary - -| Concern | AllAgents A2A gateway | AHP host/session layer | -|---|---|---| -| Northbound consumer | AI Evals and future trusted-network execution clients | IDE, browser, CLI, or collaborative operator client | -| Primary lifecycle | One addressable Task per accepted execution | Long-running session/chat with shared clients | -| Public identity | Agent Card, Message, Task, Artifact, invocation key | Host, client, channel, session, chat, turn, tool call | -| State | Deployment-wide Task status, messages, artifacts, retention | Snapshots, ordered actions, reducers, reconnect | -| Authorization | Network reachability; no application caller identity | Endpoint/resource auth and tool confirmation | -| Cancellation | Cancel Task, abort backend, terminate, clean up, report terminal outcome | Cancel interactive turn and call provider-native abort | -| Evidence | Source, output, usage/cost, traces, file changes, artifacts, failures, cleanup, completeness, provenance | Live changesets and provider/session state | -| Isolation | Single-process supervisor with invocation-owned child containment | Not supplied by the shared host process | - -The identities must be correlated rather than reused. At minimum retain the A2A -Task ID, AllAgents invocation key, backend execution/session ID, -provider-native thread/chat ID, and trace ID. - -## Adopt now - -1. Define one narrow backend adapter contract for create/invoke, progress, - permission decisions, cancellation, terminalization, evidence collection, - shutdown, native evidence passthrough, and explicit capabilities. -2. Keep A2A and backend responsibilities separate inside one gateway service. - The A2A layer owns deployment-wide Task/idempotency identity, backend - selection, normalized results, cancellation propagation, and retention. The - invocation supervisor owns source acquisition, provider child processes, - mutable workspaces, evidence capture, process termination, and cleanup. -3. Propagate `CancelTask` and deadlines through the adapter to the - provider-native abort primitive, then persist terminal status and cleanup - outcome. Transport closure is not cancellation. -4. Separate network authorization, execution permission policy, and - provider/resource credentials. The initial gateway has no application caller - identity. -5. Combine normalized file operations with bounded provider-native - diffs/checkpoints/trajectories. Declare attribution limits and - incompleteness rather than treating the final working-tree diff as exact - agent causality. -6. Persist terminal facts independently of progress streams and telemetry. -7. Capability-gate backend behavior instead of inferring it from provider names - or software versions. -8. Keep Task lists and metadata bounded; store large logs, diffs, traces, and - produced artifacts behind references with size, redaction, and truncation - metadata. - -## Defer - -- An AHP backend adapter until a selected backend actually exposes AHP. -- An AHP server or multi-client reducer/reconciliation engine until a - collaborative session client is a product requirement. -- Client-contributed tools and customizations for unattended evaluation - profiles. -- Active-session reconnection beyond A2A Task lookup, subscription, and - terminal result retrieval. -- AHP local endpoint discovery, SSH host selection, and tunnel multiplexing. -- Remote worker ownership, routing, and session transport until the initial - single-process gateway needs a separate execution host. -- Generic changeset review/operation state. -- Long-lived session/chat catalogs and provider-native session adoption. - -## Reject - -- Replacing A2A with AHP for the execution gateway. -- Running evaluated agents or writable repositories in the gateway process. -- Treating AHP changesets as the complete AllAgents evidence envelope. -- Treating AHP action replay or session restoration as execution idempotency. -- Copying VS Code's local connection-token model as gateway authentication. -- Branching on provider names above the adapter boundary. -- Making a connected interactive client a hidden prerequisite for unattended - execution. - -## Evidence - -The conclusion is based on: - -- Microsoft's [Agent Host architecture article](https://code.visualstudio.com/blogs/2026/08/26/agent-host-architecture); -- the official [Agent Host Protocol documentation](https://microsoft.github.io/agent-host-protocol/); -- direct inspection of `microsoft/vscode` commit - [`046944034292b5479b4e9a50ad1a508033ffb64f`](https://github.com/microsoft/vscode/tree/046944034292b5479b4e9a50ad1a508033ffb64f), - whose generated registry identifies AHP `0.9.0`; and -- [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md). - -The inspected implementation demonstrates host-owned state/sequencing, -provider-neutral adapters for Copilot, Claude, and Codex, layered persistence, -bounded reconnect replay, provider-native cancellation, client-owned tools, -permission translation, Git checkpoint plus SDK edit evidence, and local/remote -host placement. These observations support the architecture seams above; they -do not supply the public execution guarantees retained by ADR 0002. diff --git a/docs/research/harbor-repository-materialization.md b/docs/research/harbor-repository-materialization.md index 38b6be3f..f6caba95 100644 --- a/docs/research/harbor-repository-materialization.md +++ b/docs/research/harbor-repository-materialization.md @@ -2,21 +2,21 @@ ## Decision -Borrow Harbor's content-addressed package cache, sparse Git reads, staged publication, -and prebuilt-environment option. Do not copy its task model as the execution gateway's -workspace contract. - -Harbor does not expose a first-class, general-purpose "repositories in a workspace" -layer. It first downloads a Harbor **task package**. The task then defines an execution -environment with a Dockerfile, Compose file, or prebuilt image. Acquisition of the -repository the agent edits is therefore benchmark- and task-owned: it may be baked into -an image, cloned by a Dockerfile, copied as task content, or otherwise prepared by the -task author. - -For AllAgents, repository and workspace provenance must remain explicit in the -public execution request and terminal evidence. The initial gateway supports -only declared Git repositories and named digest-pinned OCI workspace snapshots; -custom materializers remain deferred. +Use Harbor's content-addressed package cache, sparse Git reads, staged +publication, and prebuilt-environment model as inputs to the AllAgents workspace +materializer. Keep source selection and provenance in the AllAgents contract +rather than adopting Harbor's task-owned workspace model. + +Harbor does not expose a first-class, general-purpose “repositories in a +workspace” layer. It first downloads a Harbor **task package**. The task then +defines an execution environment with a Dockerfile, Compose file, or prebuilt +image. Acquisition of the repository the agent edits may be baked into an image, +cloned by a Dockerfile, copied as task content, or prepared by the task author. + +AllAgents keeps repository and workspace provenance explicit in the initial +workspace descriptor and response metadata. The materializer accepts only +declared Git repositories and named digest-pinned OCI workspace snapshots; +custom materializers are outside the version-one contract. ## What Harbor fetches @@ -37,8 +37,8 @@ workspace. Harbor also accepts an omitted commit or a mutable ref and resolves it to a commit. AllAgents permits a caller to override a declared repository with a branch, tag, or commit for developer convenience, but resolves and records the -full commit before provider execution. Reproducibility-sensitive callers use a -full commit; OCI snapshots remain digest-pinned at admission. +full commit before agent execution. Reproducibility-sensitive callers use a full +commit; OCI snapshots remain digest-pinned at admission. ### Task packages from the package registry @@ -73,7 +73,7 @@ an upstream `mswebench/...:pr-...` base image that already contains the reposito `/home/{repo_name}`. Its Dockerfile creates `/workspace/{repo_name}` as a symlink and sets that as `WORKDIR`; Harbor itself never clones that application repository. -## Lessons for the AllAgents execution gateway +## Lessons for the AllAgents workspace materializer ### Adopt @@ -110,7 +110,7 @@ Use exactly two initial source modes: 1. **Direct declared Git repositories** for the normal case. A request selects configured repository names and may override only their revisions. The - gateway resolves and records full commits and enforces collision-free + materializer resolves and records full commits and enforces collision-free destinations. 2. **Named OCI workspace snapshots** for large, preassembled workspaces. The project workspace declares the repository; the request supplies immutable @@ -122,48 +122,50 @@ through to the other after admission. ### Do not copy - Unresolved mutable Git refs as terminal execution identities. Branch and tag - overrides are valid only when the gateway resolves and records a full commit - before provider execution. + overrides are valid only when the materializer resolves and records a full + commit before agent execution. - Mutable OCI tags or package `latest` as accepted snapshot identities. - Harbor's broad Git transport set (`http`, `ssh`, and `git` as well as HTTPS) at - a service boundary. The gateway keeps canonical credential-free HTTPS, - destination-policy revalidation, disabled redirects/helpers/filters/hooks/ + the materializer boundary. AllAgents permits only canonical credential-free + HTTPS with configured hosts, disabled redirects/helpers/filters/hooks/ submodules, and full-commit verification. - A non-fatal Git LFS miss. If declared workspace content cannot be materialized, - preparation must fail before provider execution. -- Hashing a prebuilt image reference string as environment identity. Resolve and pin - the OCI manifest digest. -- Arbitrary task-authored Dockerfiles, Compose files, or public-network setup as caller - input. Harbor runs benchmark definitions trusted by the evaluator; the gateway - accepts remote service requests and has a different threat boundary. + preparation fails before agent execution. +- Hashing a prebuilt image reference string as environment identity. Resolve and + pin the OCI manifest digest. +- Arbitrary task-authored Dockerfiles, Compose files, or public-network setup as + caller input. Harbor runs benchmark definitions trusted by the evaluator; + AllAgents accepts authenticated service requests with a different trust + boundary. - Treating a container image alone as sufficient provenance. An image can carry the correct files while obscuring which repositories, commits, generator, and setup produced them. ## Recommended boundary -The gateway supervisor executes a dedicated acquisition phase before any -provider starts: - -1. Validate the normalized source request and its declared repository or - snapshot identities before any network access. -2. Resolve phase-scoped source credentials without exposing them to typed - provider preparation, the model, or later evidence collection. -3. Populate a gateway-owned staging directory on the final publication - filesystem, or pull and unpack a digest-pinned workspace snapshot there. -4. Verify repository commits, paths, limits, content, the expected manifest - digest, and the standard workspace manifest; distinguish gateway-verified - identities from snapshot-attested claims. -5. Stop acquisition processes, revoke credentials, remove helpers and mounts, - and retain only the validated credential-free staging tree. -6. Atomically rename that tree into the final workspace, record provenance, run - adapter-owned typed preparation, record the baseline, and only then launch - the provider runtime. Project or user `setup` shell commands are not run. - -Operator-registered materializers, custom builders, and third source variants -are deferred until direct Git and OCI snapshots cannot satisfy a demonstrated -deployment need. Adding one requires a new decision for trust, configuration, -credential, provenance, and isolation boundaries. +The HarnessRouter runner invokes the AllAgents materializer before provider +selection: + +1. HarnessRouter validates generic metadata bounds, creates the session, and + enters the durable materialization state. +2. The materializer validates the workspace descriptor and configured repository + or snapshot identities before source network access. +3. The materializer resolves phase-scoped source credentials without exposing + them to the coding agent or later evidence collection. +4. The materializer populates a fixed staging directory on the publication + filesystem, or pulls and unpacks a digest-pinned workspace snapshot there. +5. The materializer verifies commits, paths, limits, content, the expected + manifest digest, and the standard workspace manifest; snapshot-attested claims + remain distinct from independently verified identities. +6. The materializer stops acquisition processes, removes credentials, helpers, + and mounts, and returns only the validated credential-free staging tree plus + bounded provenance. +7. The runner independently validates staging, publishes it, creates checkpoint + and collection baselines, applies UHP input files, and only then launches the + coding agent. Project or user `setup` shell commands are not run. + +Operator-selected builders and additional source variants require a new decision +for trust, configuration, credential, provenance, and isolation boundaries. The practical conclusion is narrow: Harbor is strong evidence for content- addressed input bundles and staged publication. It is not evidence for making diff --git a/docs/research/source-credential-broker-precedents.md b/docs/research/source-credential-broker-precedents.md index f2c69f5b..08fbe1ce 100644 --- a/docs/research/source-credential-broker-precedents.md +++ b/docs/research/source-credential-broker-precedents.md @@ -2,29 +2,24 @@ ## Decision -The execution gateway does **not** need a standalone Git credential broker for -the initial trusted-network deployment. It supports two in-process trusted -providers for `github.com`: a configured GitHub App and a configured, -account-pinned `gh auth token --hostname github.com --user ` fallback. - -The App is preferred whenever an App-authenticated repository-coverage check -proves an installation eligible. `gh` is considered only when the App is absent -or coverage is positively ineligible; unknown discovery, authentication, -permission, rate-limit, or service failures fail closed. Ambient `GH_TOKEN`, -`GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` are removed -from the CLI helper environment. - -Either token is exposed only to the one-shot acquisition process through an -invocation-scoped Git credential helper. The helper, token, and acquisition -process are gone before adapter preparation or provider execution. Git -credential helpers and Git Credential Manager establish the process-boundary -precedent, but arbitrary configured helpers are not part of the selected -implementation. A local helper is a broker in the security sense; it is not a -separately deployed network service. - -Central token minters, authenticated delivery leases, remote workers, and -multi-tenant credential policy are deferred until ADR 0002's deployment -boundary is reconsidered. +The initial trusted-network deployment uses deployment-supplied source +credentials referenced from the project `workspace.yaml` as `${ENV_VAR}` values. +The AllAgents materializer receives only the configured credential variables in +its allowlisted child environment. HarnessRouter removes every +materializer-only variable from agent child environments regardless of its name. + +Git credentials are exposed only to the acquisition process through a +short-lived, materializer-owned credential helper or registry-auth channel. +The materializer uses hermetic Git and registry configuration, removes temporary +auth state before returning, and emits no secret in logs, provenance, checkpoints, +or response metadata. It never consults arbitrary ambient credential helpers and +never falls through to a different credential identity after a failure. + +Git credential helpers, GitHub App installation tokens, and BuildKit secret +mounts establish the process- and phase-boundary precedents. The deployment does +not require a standalone network credential broker. Central token minting, +delivery leases, remote workers, and multi-tenant credential policy require a +separate decision if the deployment boundary changes. ## Precedents @@ -62,16 +57,13 @@ by default or sooner if the daemon dies [options](https://git-scm.com/docs/git-credential-cache#_options)). This is a local process/socket boundary, not a remotely reachable credential service. -**Relevance.** Git helpers and GCM prove that a local credential provider can -be an on-demand process rather than a network service. AllAgents does not, -however, inherit or invoke an arbitrary configured helper chain. Its closed -provider registry permits only the selected GitHub App token or an explicit -GitHub CLI provider pinned to a configured non-secret account when App -eligibility is positively absent. The CLI invokes -`gh auth token --hostname github.com --user ` without ambient GitHub -token variables. Its output reaches only the one-shot acquisition child; -adapter preparation and the coding runtime inherit neither helper configuration -nor token. +**Relevance.** Git helpers and GCM prove that a local credential provider can be +an on-demand process rather than a network service. The AllAgents materializer +does not inherit or invoke the host's configured helper chain. It creates a +closed helper for the selected deployment credential, invokes Git with an +isolated home and system/global configuration disabled, and removes the helper +before returning. The coding-agent runtime inherits neither the helper +configuration nor its credential. ### SSH agent forwarding @@ -136,12 +128,12 @@ Checkout's credential file is a convenience capability inside that job, not a long-term credential store, and its post-job deletion is defense in depth rather than the token's revocation mechanism. -**Relevance.** This is the closest production precedent for AllAgents: keep the -App private key in the trusted gateway process, issue one fresh least-privilege -token for a particular repository acquisition, expose it only during that -phase, and remove its local material afterward. GitHub enforces repository, -read-only contents permission, and expiry; the gateway separately binds the -acquisition to the retained Task and effective configuration digest. +**Relevance.** GitHub App installation tokens are useful deployment inputs +because repository scope, read-only contents permission, and expiry are enforced +by GitHub. Token minting remains outside the materializer contract. If an +operator supplies such a token through the configured environment reference, +the materializer still treats it as a phase-scoped acquisition secret and binds +the resulting source identity to the session's effective descriptor digest. ### BuildKit secret and SSH mounts @@ -172,47 +164,44 @@ When an agent socket is supplied, SSH access is available for the mounted instruction without adding the private key to the image ([Dockerfile SSH mount](https://docs.docker.com/reference/dockerfile/#run---mounttypessh)). -**Relevance.** AllAgents should copy the phase-scoping pattern, not necessarily -BuildKit itself: inject a token or agent capability only into the trusted source -acquisition operation, then tear down the mount/socket/environment before setup -or agent execution. Like BuildKit, this delivery mechanism does not mint -credentials and does not eliminate the need for a central issuer in production. +**Relevance.** AllAgents uses the same phase-scoping pattern: inject a token only +into the trusted source-acquisition operation, then remove the +mount/socket/environment before agent execution. Like BuildKit, this delivery +mechanism does not mint credentials and does not make code with access to the +secret trustworthy. ## Recommendation for AllAgents -### Initial trusted-network gateway - -1. Resolve only canonical `github.com` HTTPS origins in the initial delivery. -2. Determine App applicability through an App-authenticated GitHub API client, - or verify an explicitly configured installation ID against the repository. - Model the result as `eligible`, `ineligible`, or `unknown`. -3. For `eligible`, use focused - [`@octokit/auth-app`](https://github.com/octokit/auth-app.js) authentication - and mint a fresh token narrowed to the repository and read-only contents. - Require remaining lifetime greater than the gateway's at-most-900-second - acquisition sub-budget plus a 60-second clock-skew margin. -4. For a missing App or proven `ineligible`, a trusted local deployment may use - the configured `gh auth token --hostname github.com --user ` - provider. Include the account in the acquisition-policy digest and strip - ambient token variables. An `unknown` App result never falls through. -5. Treat provider order as eligibility, not retry. Once App is selected, - configuration, authentication, minting, authorization, repository coverage, - rate-limit, or service failure terminates acquisition. -6. Give the resolved token only to the dedicated acquisition subprocess through - a temporary helper channel. Remove the channel and terminate the process - before atomically publishing the credential-free verified workspace. -7. Do not require or auto-start a network credential service. Keep App issuer - material and GitHub/OCI auth stores inaccessible to the adapter process and - model-invoked tools. - -### Deferred remote or multi-tenant deployment +### Initial trusted-network deployment + +1. Store only `${ENV_VAR}` references in the project workspace configuration; + reject literal credentials and caller-supplied credential identifiers. +2. Supply secret values through the deployment environment and validate required + names during materializer preflight without contacting sources. +3. Pass only the referenced, allowlisted names to the materializer child. Remove + the complete allowlist from every coding-agent child independent of + secret-looking name patterns. +4. Select one configured credential identity before acquisition. Authentication, + authorization, rate-limit, or service failure terminates acquisition and + never falls through to another identity or source mode. +5. Give the credential only to the dedicated acquisition subprocess through a + temporary helper or registry-auth channel. Invoke helpers directly without a + shell and bound their input, output, stderr, and lifetime. +6. Use isolated Git/registry configuration. Prevent credentials from entering + remote URLs, Git config, generated CLI config, workspace files, nested + repositories, checkpoints, logs, provenance, or response metadata. +7. Remove helper files, auth configuration, and the credential-bearing process + before returning the validated staging tree to HarnessRouter. +8. Verify containment with a deliberately non-secret-looking environment name, + because name-based secret filters are not the security boundary. + +### Remote or multi-tenant deployment A future deployment may require a central token minter, authenticated single-use delivery leases, entitlement generations, revocation reconciliation, worker -identity, fencing, and a snapshot-delivery protocol. Those mechanisms are not -part of the selected single-process architecture. They require a separate -decision when remote workers or tenant isolation become product requirements. +identity, fencing, and a snapshot-delivery protocol. Those mechanisms require a +separate decision when remote workers or tenant isolation become product +requirements. -The resulting initial rule is: **credential reuse is acquisition-subprocess- -mediated and ends before provider execution.** Remote or multi-tenant issuance -policy remains deferred; a standalone credential service is not required now. +The resulting rule is: **source credentials exist only during the materializer's +acquisition phase and never enter the coding-agent environment.** From 5bd1fff9ce64447b61e0cbbcacd98a68834acf14 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Tue, 22 Sep 2026 14:55:54 +1000 Subject: [PATCH 16/44] docs(architecture): prefer harness-native OAuth --- .../0002-adopt-uhp-through-harnessrouter.md | 219 ++++-- ...0837-feat-coding-execution-gateway-plan.md | 672 +++++++++++------- 2 files changed, 581 insertions(+), 310 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index ee3bd9e6..6d76408b 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -6,15 +6,20 @@ ## Decision AllAgents will use the Unified Harness Protocol (UHP) `2026-09-12` through a -pinned HarnessRouter Community Edition deployment for remote Codex execution. -Pi remains capability-gated until a real probe proves a HarnessRouter-supported -custom-provider format against `codex-lb`. +pinned HarnessRouter Community Edition deployment for remote Codex and Pi +execution. -HarnessRouter owns authentication, UHP request and response semantics, streaming, -cancellation, idempotency, session continuity, per-session workspaces, agent -execution, usage, and artifacts. AllAgents owns the workspace descriptor, -deterministic source materialization, and provenance returned through the -HarnessRouter response. +The selected harness owns provider authentication. Codex signs in through +`codex login`; Pi signs in through its `/login` flow for the configured provider. +Those native OAuth sessions are the default and require no provider-route API +key. An explicitly configured API-key-authenticated proxy is a last-resort +route, never an automatic fallback from failed OAuth. + +HarnessRouter owns caller authentication, UHP request and response semantics, +streaming, cancellation, idempotency, session continuity, per-session +workspaces, agent execution, usage, and artifacts. AllAgents owns the workspace +descriptor, deterministic source materialization, and provenance returned +through the HarnessRouter response. The initial deployment will use a narrow AllAgents-maintained HarnessRouter fork. The fork adds a generic pre-turn workspace-materializer hook. An AllAgents @@ -38,25 +43,36 @@ flowchart TB GATEWAY[Forked HarnessRouter gateway] RUNNER[HarnessRouter runner] MATERIALIZER[AllAgents workspace materializer] - LB[codex-lb] + HARNESS[Selected Codex or Pi harness] + AUTH[Durable harness-native OAuth profile] + PROXY[Optional authenticated proxy] MODEL[Model provider] CLIENT -->|UHP + allagents.workspace| GATEWAY GATEWAY -->|runner /materialize| RUNNER RUNNER -->|generic pre-turn hook| MATERIALIZER MATERIALIZER -->|prepared staging + provenance| RUNNER - RUNNER -->|invocation-scoped broker credential| GATEWAY - GATEWAY -->|long-lived codex-lb API key| LB - LB -->|provider OAuth and routing| MODEL + RUNNER --> HARNESS + AUTH -.->|native mode: login and refresh| HARNESS + HARNESS -->|native mode| MODEL + HARNESS -.->|proxy mode: scoped turn credential| GATEWAY + GATEWAY -.->|long-lived proxy client key| PROXY + PROXY -.-> MODEL ``` -Promptfoo authenticates to HarnessRouter with a HarnessRouter API key. -HarnessRouter runs in brokered sandbox-auth mode: the gateway holds a separate -`codex-lb` API key and gives the agent only an invocation-scoped broker -credential. `codex-lb` owns provider OAuth, account selection, continuation -affinity, and provider routing. Neither the `codex-lb` key nor provider OAuth -credentials enter Promptfoo, request metadata, the materializer, or the agent -workspace. +Promptfoo authenticates to HarnessRouter with a HarnessRouter API key. That +control-plane credential is separate from provider authentication. In the +default route, the selected Codex or Pi process uses its own durable OAuth +profile and refreshes it through the harness's native mechanism. HarnessRouter +does not translate that OAuth session into an API key. + +Native OAuth is an owner-trust mode: the selected harness and tool subprocesses +running under the same operating-system identity may access and emit its +credential. The gateway/runner does not automatically serialize the auth file +into source trees, checkpoints, produced-file records, passive logs, request +metadata, or response metadata, and it mounts no other profile. Deployments that +cannot accept active exfiltration risk must explicitly configure the brokered +proxy route. ## Protocol boundary @@ -103,14 +119,17 @@ overlay source files deterministically. ## Workspace and session semantics -The workspace descriptor is accepted only when creating the first response in a -session. HarnessRouter binds its canonical digest and resolved provenance to the -session before agent execution. +The workspace descriptor and selected authentication binding are accepted only +when creating the first response in a session. HarnessRouter binds the canonical +workspace digest, resolved provenance, harness target, auth mode, and native +profile or proxy-connection identity/digest before agent execution. A continuation uses `previous_response_id` and the same HarnessRouter session -workspace. It must omit the workspace descriptor; the session's pinned -descriptor and provenance remain authoritative. A new source revision or -working directory requires a new session. +workspace. It must omit the workspace descriptor and reuse the persisted auth +binding. A deployment configuration change never silently switches mode, +profile, or connection; if the exact binding is unavailable, continuation fails +closed until it is restored. A new source revision, working directory, or auth +binding requires a new session. The materializer runs before the first agent turn and never depends on the model reading a prompt, calling an MCP tool, or extracting an archive. Source @@ -131,7 +150,7 @@ the session resumable. ## Fork boundary The HarnessRouter fork is limited to the workspace-integration seam and the -custom Codex-provider rendering needed by `codex-lb`. The workspace seam: +harness-native authentication-state seam. The workspace seam: 1. recognizes one configured, bounded metadata key on the first UHP response; 2. treats its JSON value as opaque, canonicalizes it, and binds it to the session; @@ -148,18 +167,36 @@ custom Codex-provider rendering needed by `codex-lb`. The workspace seam: 10. applies ordinary input files only after successful materialization; and 11. strips every configured materializer-only environment name from agent children. +The authentication-state seam separates session conversation state from durable +per-harness OAuth state. In native mode it projects only the selected profile +into the Codex or Pi home and permits the harness to persist token refreshes. It +prevents the gateway/runner lifecycle from copying auth files into checkpoints, +produced-file records, passive logs, or public metadata. Other profile roots are +not mounted. +This does not prevent the selected harness or same-identity tools from reading +or emitting the credential inside the accepted owner-trust boundary. + +The projection mechanism must preserve each harness's credential-file write and +atomic-replacement behavior. A locally committed refresh uses a same-filesystem +temporary file, file and parent-directory `fsync`, atomic rename, and validation. +A crash after the provider rotates credentials but before local commit may leave +the profile stale; restart marks it `repair-required` when validation fails and +requires native login again. It never switches profiles or activates the proxy. +Version one holds a per-profile lock for every refresh-capable turn and every +login, logout, or repair operation. A second turn for that profile waits or +fails before launch. + The generic fork layer does not understand the AllAgents descriptor. It enforces only the configured key, JSON/size bounds, immutable first-turn binding, hook envelope, lifecycle, and response namespace. The external AllAgents executable owns schema/default validation, workspace configuration, Git/OCI acquisition, -credential selection, filesystem policy, and provenance. +source-credential selection, filesystem policy, and provenance. The fork must preserve stock behavior for requests without the configured key -and must continue to pass upstream UHP conformance. The separate provider seam -only renders the documented `codex-lb` Codex identity and OpenAI-auth capability -fields. The maintained patch series is pinned to an upstream commit, covered by -focused integration tests, and kept free of unrelated changes. The intended -upstream contributions are these generic integration fixes, not the +and must continue to pass upstream UHP conformance. The maintained patch series +is pinned to an upstream commit, covered by focused integration tests, and kept +free of unrelated changes. The intended upstream contributions are the generic +materializer boundary and secure harness-auth state separation, not the AllAgents-specific descriptor schema. ## Source authority and credentials @@ -185,46 +222,76 @@ and workspace-manifest digests. It verifies manifest, config, layer sizes and digests, applies OCI whiteouts, validates the resulting declared workspace layout, and records the ordered layer digests. -Source credentials are selected server-side and exist only for the -materialization subprocess. The materializer must use hermetic Git/registry -configuration, prevent credentials from being persisted in Git configuration -or remote URLs, remove temporary credential state before returning, and emit no -secret value. The fork removes every configured materializer-only environment -name from every agent child independent of the variable's spelling. The agent -process receives neither the acquisition credential nor the acquisition -environment. +Source credentials are selected server-side from an owner-only secret mount or +credential-store handle available to the runner, not from the long-lived service +environment. The runner resolves exactly the selected value when it constructs +the materializer child environment; the gateway/runner base environment and +every agent child remain credential-free. The materializer must use hermetic +Git/registry configuration, prevent credentials from being persisted in Git +configuration or remote URLs, remove temporary credential state before +returning, and emit no secret value. If a configured source secret appears in +the service or agent environment, the runner refuses to launch the agent. ## Trust and deployment -HarnessRouter API authentication is mandatory even on a private network. -Operators should still bind it to loopback or a private network and enforce -Tailscale ACLs, firewall policy, or equivalent controls. Version one is not a -public multi-tenant service. +HarnessRouter API authentication is mandatory on every externally reachable UHP, +response/session retrieval, stream, cancellation, file, and artifact endpoint, +even on a private network. Gateway-to-runner operations are not externally +routable and are mutually authenticated. Operators should still bind the +deployment to loopback or a private network and enforce Tailscale ACLs, firewall +policy, or equivalent controls. Version one is not a public multi-tenant service. HarnessRouter CE provides per-session operating-system identities and workspace -directories, not a hostile-code sandbox. Agent tools may access capabilities -available to their runner environment. Operators requiring stronger isolation -must place the complete HarnessRouter deployment inside an ephemeral VM or -equivalent boundary. +directories, not a hostile-code sandbox. Native harness OAuth therefore requires +an operator-owned, private deployment: agent tools sharing the harness identity +may access that harness's OAuth profile. Operators requiring stronger provider +credential isolation must use the explicit brokered proxy route or place the +complete deployment inside a stronger isolation boundary. The deployment uses a pinned custom HarnessRouter image containing: +- an OCI base image pinned by digest; - the pinned HarnessRouter CE revision plus the reviewed patch series; -- the AllAgents materializer executable and its pinned runtime; -- the source-acquisition tools required by the accepted Git/OCI contract; and -- pinned HarnessRouter-supported Codex and optional Pi versions. - -`codex-lb` remains a separate service with proxy API-key authentication enabled. -The private deployment sets both `HARNESS_PUBLIC_BASE_URL` and the -runner-reachable `HARNESS_GATEWAY_URL` to the same gateway address; “public” here -means the base advertised across the private deployment, not Internet exposure. -Codex uses the `codex-lb` Responses endpoint with the provider identity and -OpenAI-auth capability fields required for `/responses` and -`/responses/compact`; readiness proves both plus a resumed turn through the -exact container network. Pi is advertised only if a release-gating probe proves -a separate HarnessRouter-supported custom format and `codex-lb` endpoint; -failure disables Pi rather than exposing a direct provider credential or adding -another proxy. +- the AllAgents materializer executable and its locked runtime dependencies; +- version-locked OS packages and Git/OCI source-acquisition tools; and +- pinned HarnessRouter-supported Codex and Pi versions. + +AllAgents publishes the `linux/amd64` release image as the public package +`ghcr.io/allagentsdev/harnessrouter`. Version and commit tags are mutable +discovery labels; deployment configuration pins the published manifest digest. +A protected release workflow publishes from an approved ref, uses commit-pinned +actions, and separates unprivileged build/test jobs from the environment-approved +publish job. GitHub's package permission replaces third-party registry +credentials. The final manifest digest receives GitHub/Sigstore build-provenance +and SBOM attestations. Both must verify the expected repository, workflow, ref, +subject digest, and predicate before deployment. + +Each configured harness target binds exactly one authentication union: +`nativeOAuth` plus a profile, or `proxyApiKey` plus a proxy connection. +`nativeOAuth` is the default. The operator runs `codex login` against a dedicated +Codex auth root or Pi `/login` against a dedicated Pi auth root during controlled +setup. Codex uses file credential storage under `CODEX_HOME`; Pi uses +`~/.pi/agent/auth.json`. Both harnesses own token refresh. Conversation and +rollout state remain session-scoped, while refreshed OAuth state persists in the +selected auth root outside the workspace checkpoint. + +The runner verifies the selected binding and a live turn before advertising the +target: login status and refresh for native OAuth, or proxy configuration, +broker, and endpoint compatibility for `proxyApiKey`. A missing, expired, +revoked, or unrefreshable OAuth profile disables that target; it does not select +another profile or fall through to an API key. + +`proxyApiKey` is an optional, explicit last-resort mode. HarnessRouter keeps the +long-lived proxy client key in the gateway. It gives the harness a +non-refreshable broker credential bound to one proxy audience, harness target, +model allowlist, response/turn ID, and the UHP deadline plus minimal clock skew. +The token may authorize the bounded provider calls, compaction, and retries +needed during that active turn. Cancellation or terminal completion revokes it; +logs, checkpoints, artifacts, and stored responses do not passively persist it. +A configured proxy such as `codex-lb` owns its upstream provider authentication. +The HarnessRouter broker must reject wrong-audience, wrong-model, wrong-turn, +expired, or revoked credentials. Native OAuth failure never activates this route +automatically. ## Failure behavior @@ -240,8 +307,10 @@ another proxy. cancellation behavior. - **HarnessRouter restart:** preserve completed state from the durable volume; fail interrupted turns without automatic replay. -- **Provider failure:** return HarnessRouter's normalized UHP failure without - source fallback or provider-credential leakage. +- **Provider authentication failure:** fail the selected harness target without + switching OAuth profiles or activating the proxy/API-key route. +- **Provider execution failure:** return HarnessRouter's normalized UHP failure + without source fallback or credential material in public output. Failures report only verified provenance. Partial acquisition never appears as a complete workspace identity. @@ -254,16 +323,22 @@ provider adapter, or Promptfoo-specific runtime. AllAgents owns workspace selection, deterministic Git/OCI materialization, source credentials, provenance, the HarnessRouter integration patch, and -deployment documentation. +deployment documentation. The selected harness owns provider OAuth login and +refresh. Native OAuth deliberately places that profile inside the +operator-controlled harness trust boundary; source-acquisition credentials +remain isolated from the harness. The integration requires a maintained fork and custom image. The fork must be -rebased and tested against upstream releases until the generic hook is accepted -or an equivalent supported extension exists. +rebased and tested against upstream releases until the generic seams are +accepted or equivalent supported extensions exist. ## Alternatives rejected - **Custom execution gateway:** duplicates mature UHP/HarnessRouter session, streaming, cancellation, authentication, artifact, and provider behavior. +- **Proxy-first provider authentication:** adds a mandatory API key and network + hop even when Codex or Pi can use the operator's subscription directly. The + proxy remains an explicit compatibility and isolation fallback. - **Thin adapter in front of stock HarnessRouter:** avoids a fork but introduces another network service and makes source acquisition a client-side concern. - **Put the descriptor in the prompt:** lets the model control acquisition and @@ -279,10 +354,10 @@ or an equivalent supported extension exists. ## Deliberate limits Version one does not add evaluation datasets, scoring, assertions, automatic -retries, session branching, concurrent turns within one session, caller-supplied -origins, public multi-tenancy, arbitrary materializer commands, mutable OCI -tags, transparent source-mode fallback, or guaranteed provider prompt-cache -hits. +retries, session branching, concurrent turns within one session, concurrent +refresh-capable turns sharing an auth profile, caller-supplied origins, public +multi-tenancy, arbitrary materializer commands, mutable OCI tags, transparent +source-mode fallback, or guaranteed provider prompt-cache hits. ## Reconsider when diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 6faae67d..25f7a343 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -13,44 +13,51 @@ execution: code ## Goal Capsule -- **Objective:** Let Promptfoo and other authenticated UHP clients run Codex - against a configured AllAgents Git workspace or immutable OCI workspace - snapshot, continue the same conversation and writable workspace with - `previous_response_id`, and receive source provenance, output, usage, and - artifacts. Pi is optional and advertised only after its `codex-lb` route - passes a release-gating probe. +- **Objective:** Let Promptfoo and other authenticated UHP clients run a + configured Codex or Pi harness against an AllAgents Git workspace or immutable + OCI workspace snapshot, continue the same conversation and writable workspace + with `previous_response_id`, and receive source provenance, output, usage, and + artifacts. - **Means:** Deploy a pinned HarnessRouter CE fork. Preserve HarnessRouter's UHP, - authentication, session, streaming, cancellation, artifact, and agent-runner - behavior. Add a generic first-turn materializer boundary, nested working - directory support, durable materialization state, and nested-repository - checkpoint/collection support. Implement Git/OCI semantics in a separate - AllAgents executable. Route Codex through HarnessRouter's credential broker to - `codex-lb`; keep the long-lived `codex-lb` key in the gateway and provider OAuth - in `codex-lb`. + caller authentication, session, streaming, cancellation, artifact, and + agent-runner behavior. Add a generic first-turn materializer boundary, nested + working-directory support, durable materialization state, nested-repository + checkpoint/collection support, and a separation between session state and + durable harness-native OAuth state. Implement Git/OCI semantics in a separate + AllAgents executable. Codex authenticates through `codex login`; Pi + authenticates through its `/login` flow for the selected provider. An + API-key-authenticated proxy is an explicit last-resort target mode. - **Authority:** [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) - owns the protocol, fork, trust, workspace, and provider-routing decisions. - UHP `2026-09-12` and HarnessRouter's conformance suite own execution-wire - behavior. The project `workspace.yaml` owns logical Git/OCI sources and - environment-variable credential references; deployment secrets provide the - values; HarnessRouter configuration owns harness/model/provider targets. The - namespaced AllAgents extension owns acquisition and provenance semantics. -- **Execution order:** Prove the fork seam, brokered Codex route, materialization - state machine, checkpoint integration, and optional Pi route against a real - HarnessRouter runner; freeze the generic hook and AllAgents contracts; - implement Git then OCI acquisition; run Promptfoo one-shot and continuation - E2E; complete release, fork-maintenance, and upstream-ready documentation. + owns the protocol, fork, trust, workspace, harness-authentication, and + provider-routing decisions. UHP `2026-09-12` and HarnessRouter's conformance + suite own execution-wire behavior. The project `workspace.yaml` owns logical + Git/OCI sources and environment-variable credential references; deployment + secrets provide the values. HarnessRouter configuration owns harness IDs, + model allowlists, and authentication bindings. The namespaced + AllAgents extension owns acquisition and provenance semantics. +- **Execution order:** Prove the fork seam, Codex and Pi native-OAuth routes, + auth-state isolation, materialization state machine, and checkpoint integration + against a real HarnessRouter runner; freeze the generic hook and AllAgents + contracts; implement Git then OCI acquisition; prove an explicit proxy mode + separately; run Promptfoo one-shot and continuation E2E; complete release, + fork-maintenance, and upstream-ready documentation. - **Stop conditions:** Stop before production implementation if materialization cannot complete and durably checkpoint before provider dispatch, if nested Git workspaces cannot be collected without corrupting HarnessRouter checkpoints, - if either source credentials or the long-lived `codex-lb` key reach the agent, - or if the fork cannot preserve stock UHP behavior and conformance. Do not fall - back to prompt instructions, an MCP acquisition tool, client-side repository - upload, owner-trust provider keys, a second execution protocol, or a parallel - task/session engine. + if source credentials enter the harness, if the gateway/runner automatically + copies harness OAuth files into a materialized source tree, checkpoint, + produced-file record, passive log, or public metadata, if local OAuth state + cannot be persisted atomically and validated fail-closed while conversation + state remains session-scoped, or if the fork cannot preserve stock UHP + behavior and conformance. Do not fall back to prompt + instructions, an MCP acquisition tool, client-side repository upload, another + OAuth profile, an implicit API-key route, a second execution protocol, or a + parallel task/session engine. - **Tail ownership:** Implementation owns focused tests in both repositories, - upstream UHP conformance, built-image smoke tests, exact Git/OCI E2E, - two-turn Promptfoo success and failure verification, credential leak checks, - documentation, and a clean upstreamable HarnessRouter patch series. + upstream UHP conformance, built-image smoke tests, exact Git/OCI E2E, native + Codex/Pi OAuth and explicit proxy-mode E2E, two-turn Promptfoo success and + failure verification, credential boundary checks, documentation, and a clean + upstreamable HarnessRouter patch series. --- @@ -60,7 +67,7 @@ execution: code AllAgents uses HarnessRouter as the execution gateway. HarnessRouter exposes UHP, authenticates callers, creates and persists sessions, -streams events, runs Codex and capability-gated Pi, handles cancellation and +streams events, runs configured Codex and Pi harnesses, handles cancellation and idempotency, and returns output, usage, and artifacts. The missing product-specific capability is deterministic workspace acquisition @@ -106,13 +113,20 @@ caller responsible for acquisition. The temporary fork closes those seams. the tree, and returns provenance. - **A4. HarnessRouter runner:** Owns the per-session operating-system identity, publication, checkpoints, nested-repository collection, input files, safe - nested cwd, and Codex/Pi process. -- **A5. `codex-lb`:** Exposes the Codex Responses endpoint, accepts the - gateway-held API key through HarnessRouter's broker, owns provider OAuth and - account routing, and preserves continuation affinity. -- **A6. Operator:** Pins and deploys the custom image, mounts durable data and - project workspace configuration, supplies deployment-only source credential - values through the allowlisted environment, and controls private-network access. + nested cwd, selected Codex/Pi process, session conversation state, and + projection of the selected durable auth profile. +- **A5. Harness-native auth profile:** One dedicated durable credential root for + one Codex or Pi harness target. The harness owns login and token refresh. The + gateway lifecycle never copies its files into workspace checkpoints or public + metadata; active harness access is part of the owner-trust boundary. +- **A6. Optional authenticated proxy:** A last-resort, explicitly configured + target mode. The gateway keeps the long-lived proxy client key, the proxy owns + upstream provider authentication, and the harness receives only a + non-refreshable, scoped turn credential that the HarnessRouter broker validates. +- **A7. Operator:** Pins and deploys the custom image, mounts durable data and + project workspace configuration, completes each native harness login, supplies + deployment-only source credential values, selects any explicit proxy targets, + and controls private-network access. ### Key Decisions @@ -121,8 +135,9 @@ caller responsible for acquisition. The temporary fork closes those seams. - **Fork narrowly and upstream later.** Delivery uses an AllAgents-maintained fork. The upstreamable layer is a configured opaque-metadata key, immutable first-turn binding, typed command envelope, durable pre-provider lifecycle, - safe nested cwd, and checkpoint/collection integration. It contains no - AllAgents Git/OCI schema logic. Upstream acceptance is not critical-path. + safe nested cwd, checkpoint/collection integration, and separation of durable + harness-auth state from session state. It contains no AllAgents Git/OCI schema + logic. Upstream acceptance is not critical-path. - **Run the AllAgents component behind HarnessRouter.** The materializer is a subprocess hook, not another HTTP gateway and not a custom agent backend. - **Materialize once per extension-bearing session.** An extension-bearing @@ -131,14 +146,20 @@ caller responsible for acquisition. The temporary fork closes those seams. - **Keep source authority server-side.** Callers select logical source names and revisions but cannot send origins, credentials, host paths, commands, or Docker options. -- **Broker provider access.** Promptfoo uses a HarnessRouter API key. The gateway - keeps the `codex-lb` key and mints an invocation-scoped broker credential for - the agent. Only `codex-lb` handles provider OAuth. +- **Prefer harness-native OAuth.** Promptfoo's HarnessRouter API key authenticates + the UHP caller only. Codex and Pi use their own login, token storage, refresh, + and provider request path; native mode has no provider-route API key. +- **Make proxy auth explicit.** `proxyApiKey` is a last-resort target mode for a + compatibility or stronger-isolation requirement. OAuth failure never activates + it, and a session never changes its persisted authentication binding. - **Preserve stock UHP requests.** Requests without the configured metadata key behave exactly as upstream. - **Use a custom HarnessRouter image.** The image combines a pinned HarnessRouter revision, reviewed patch series, pinned agent runtimes, and the AllAgents materializer executable. +- **Publish from an AllAgents-owned registry.** Release the public image as + `ghcr.io/allagentsdev/harnessrouter`; tags identify releases, but deployment + and E2E pin the published manifest digest. ### Requirements @@ -147,27 +168,50 @@ caller responsible for acquisition. The temporary fork closes those seams. - **R1.** Pin HarnessRouter CE to a reviewed upstream commit and UHP version `2026-09-12`. The deployment must pass the applicable upstream conformance suite without weakening, replacing, or reinterpreting stock UHP behavior. -- **R2.** Require a HarnessRouter API key for every execution endpoint. Bind the - service to loopback or a private interface and document the remaining need for - Tailscale ACLs, firewall policy, or equivalent network controls. -- **R3.** HarnessRouter deployment configuration owns stable harness IDs and - their backend, model allowlist, and provider integration. Requests select a - harness with stock `metadata.harness_id` and a model with `model`; AllAgents - profiles are not projected into this catalog. Codex uses a custom - `api_format: "responses"` integration pointed at the `codex-lb` - `/backend-api/codex` base. Its generated provider block must use - `name = "openai"` and `requires_openai_auth = true`, including remote - compaction. Pi is advertised only when a separate real probe proves a - HarnessRouter-supported Pi custom format against a documented `codex-lb` - endpoint; otherwise Pi is unavailable without fallback. -- **R4.** Set `HR_SANDBOX_TRUST` to a non-owner broker mode and configure both - `HARNESS_PUBLIC_BASE_URL` and runner-reachable `HARNESS_GATEWAY_URL` to the same - private gateway address so stock broker eligibility and runner routing agree. - The gateway holds the long-lived `codex-lb` key; the runner receives only a - turn credential and broker URL. Preserve HarnessRouter streaming, - cancellation, idempotency, files, artifacts, usage, errors, completed-session - persistence, and per-session workspace/UID isolation. Do not duplicate those - capabilities in AllAgents. +- **R2.** Require a HarnessRouter API key for every externally reachable UHP, + response/session retrieval, stream, cancellation, file, and artifact endpoint. + Reject unauthenticated requests before disclosing resource existence or + metadata. Gateway-to-runner operations are not externally routable and are + mutually authenticated. Bind the service to loopback or a private interface + and document the remaining need for Tailscale ACLs, firewall policy, or + equivalent network controls. +- **R3.** HarnessRouter deployment configuration owns stable harness IDs, + backend, model allowlist, and exactly one auth binding: + `{ mode: "nativeOAuth", profile: ConfigName }` or + `{ mode: "proxyApiKey", connection: ConfigName }`. A proxy connection is a + closed server-side record containing private HTTPS base URL, expected TLS + identity/CA, HarnessRouter-supported API format and endpoint set, gateway-only + proxy-client-key secret handle, broker audience, and requested-to-proxy model + map. Callers cannot override any field. Requests select a harness with stock + `metadata.harness_id` and a model with `model`; AllAgents profiles are not + projected into this catalog. Codex and Pi targets default to `nativeOAuth`. + A target is advertised only after its selected auth binding passes: profile + login/refresh/live-turn checks for native OAuth, or schema, TLS, broker, + model-map, endpoint, and live compatibility checks for `proxyApiKey`. On the + first turn, the gateway persists the harness target, auth mode, binding + identity, and canonical binding-config digest in the session. Continuations + require that exact binding; deployment config changes never switch it. +- **R4.** Add a durable auth root outside session workspaces with one + least-access profile directory per harness target. Controlled setup runs + `CODEX_HOME= codex login` with file credential storage for Codex or + runs Pi `/login` in an isolated Pi home for the configured provider. The + runner projects only the selected profile's exact auth files into the + session-specific CLI home and keeps conversation/rollout state session-scoped. + The projection must preserve the harness's credential-file write and + atomic-replacement behavior. A locally committed refresh uses a same-filesystem + temporary file, file `fsync`, atomic rename, parent-directory `fsync`, and + validation. A crash after the provider rotates credentials but before local + commit can leave the profile stale; restart then marks it `repair-required` + and requires native login instead of changing profile or auth mode. The + gateway/runner must not automatically serialize auth files into root or nested + checkpoints, produced-file records, passive logs/traces, materializer input, + or response metadata. Native mode sets explicit owner trust because the + harness and tool subprocesses sharing its operating-system identity may read + or emit that credential. Version one holds a per-profile lock for every + refresh-capable turn and every login, logout, or repair operation; a second + turn for that profile waits or fails before launch. + Preserve HarnessRouter streaming, cancellation, idempotency, files, artifacts, + completed-session persistence, and per-session workspace/UID isolation. #### Workspace extension and hook @@ -195,28 +239,33 @@ caller responsible for acquisition. The temporary fork closes those seams. accept at most 1 MiB on stdout and 64 KiB on stderr, and use the smaller of 900 seconds or the remaining UHP deadline. The generic request contains the opaque metadata value, session workspace and fixed sibling staging roots, - project configuration root, and deadline. Source secret values arrive only in - a configured allowlisted child environment; the runner subtracts every name - in that allowlist from all agent child environments regardless of its spelling. - The typed result is `completed` with effective relative cwd and bounded public - metadata or `failed` with stable code, safe message, and retryability. A - materializer failure terminalizes the UHP response and never enters provider - fallback. + project configuration root, and deadline. Source values come from an + owner-only runner secret mount or credential-store handle, never the + gateway/runner base environment. The runner resolves only the selected handle + and constructs the allowlisted materializer child environment. It refuses + agent launch if a configured secret name or value appears in the service or + agent environment. The typed result is `completed` with effective relative cwd + and bounded public metadata or `failed` with stable code, safe message, and + retryability. A materializer failure terminalizes the UHP response and never + enters provider fallback. - **R8.** Persist a CAS-protected session materialization state: - `unbound -> materializing -> ready` or `failed`. The runner validates successful - staging, publishes it with a recoverable same-filesystem rename protocol, - writes a descriptor/provenance marker, initializes the HarnessRouter root - checkpoint and nested-repository collection baselines, and returns - `published` with the checkpoint digest. Before provider dispatch, the gateway - stores that digest and public metadata and CASes the session to `ready`. On - restart in `materializing`, reconcile to `ready` only when the workspace - marker and durable checkpoint match the bound descriptor; otherwise mark the - session non-resumable, remove or quarantine the workspace, and never replay - acquisition. Provider fallback sees `ready` state only and cannot invoke the - hook. Ordinary input files are applied only afterward. - A validated logical cwd may be the root or a symlink-safe descendant; the - runner derives UID isolation from the session root and rejects cross-session - or escaping paths. + `unbound -> materializing -> ready` or `failed`, plus the resolved harness + target, auth mode, native-profile or proxy-connection identity, and canonical + binding-config digest. The runner validates successful staging, publishes it + with a recoverable same-filesystem rename protocol, writes a + descriptor/provenance marker, initializes the HarnessRouter root checkpoint + and nested-repository collection baselines, and returns `published` with the + checkpoint digest. Before provider dispatch, the gateway stores that digest + and public metadata and CASes the session to `ready`. On restart in + `materializing`, reconcile to `ready` only when the workspace marker and + durable checkpoint match the bound descriptor; otherwise mark the session + non-resumable, remove or quarantine the workspace, and never replay acquisition. + Every continuation resolves the persisted auth binding by identity and digest; + if unavailable or changed, it fails before runner work instead of selecting a + replacement. Provider fallback sees `ready` state only and cannot invoke the + hook. Ordinary input files are applied only afterward. A validated logical cwd + may be the root or a symlink-safe descendant; the runner derives UID isolation + from the session root and rejects cross-session or escaping paths. #### Source acquisition and provenance @@ -280,19 +329,44 @@ caller responsible for acquisition. The temporary fork closes those seams. It never contains origins, physical paths, credentials, or unverified facts. - **R13.** The materializer resolves `${ENV_VAR}` references from its allowlisted child environment, uses hermetic Git/registry configuration, removes temporary - auth files before returning, and emits no secret. The fork removes every - configured materializer-only name from `_child_env` and every other agent - subprocess environment independent of name patterns. Prove with a deliberately - innocuous variable name that acquisition secrets and the long-lived - `codex-lb` key are absent from the agent environment, workspace, nested Git - remotes/config, generated CLI configuration, logs, checkpoints, and response - metadata. Provider OAuth remains only in `codex-lb`. -- **R14.** Build one pinned custom HarnessRouter image. Record the upstream - commit, patch-series digest, AllAgents package/version, materializer-contract - version, Codex version, optional verified Pi version/format, image digest, and - supported architecture. Requests without the configured metadata key remain - stock-compatible. CI rebases selected upgrades and runs upstream plus - AllAgents integration tests. + auth files before returning, and emits no secret. Prove with a deliberately + innocuous variable name and value that source credentials and the HarnessRouter + caller API key are absent from the gateway/runner base environment, every agent + environment, workspace, nested Git remotes/config, generated CLI configuration, + logs, checkpoints, and response metadata. In native mode there is no + provider-route API key. The selected OAuth profile is intentionally readable by + the harness trust boundary; the gateway/runner never automatically copies it + into materialized source trees, checkpoints, produced-file records, passive + logs, materializer input, or public metadata. An active same-identity harness + or tool can exfiltrate it; that risk is explicit in owner-trust mode. + + In proxy mode, the long-lived proxy client key and upstream provider + credentials stay in their owning services. The non-refreshable broker token is + bound to one proxy audience, harness target, model allowlist, response/turn ID, + and the UHP deadline plus minimal clock skew. It may authorize the bounded + provider requests, compaction, and retries required during that active turn; + cancellation or terminal completion revokes it. Passive persistence never + stores it. The HarnessRouter broker rejects wrong-audience, wrong-model, + wrong-turn, expired, or revoked tokens. +- **R14.** Build and publish a pinned `linux/amd64` custom HarnessRouter image as + the public package `ghcr.io/allagentsdev/harnessrouter`. Replace or disable the + inherited Docker Hub release path. The Dockerfile pins every base image by + digest; runtime lockfiles and version-locked OS packages, Git/OCI tools, Codex, + and Pi define the remaining build inputs. The build fails on any unpinned + input. A no-write job builds, tests, and exports the identified image artifact + using commit-pinned third-party actions. A separate protected, + environment-approved publish job accepts only an approved release/tag ref and + uses `GITHUB_TOKEN` with `contents: read`, `packages: write`, + `attestations: write`, and `id-token: write`. It publishes unique version and + commit tags as mutable discovery labels, reads back the registry manifest, and + creates GitHub/Sigstore build-provenance and SBOM attestations whose subject is + the final manifest digest. Package visibility is public and verified with an + anonymous digest pull. Deployment fails unless both attestations verify the + expected owner, repository, workflow, approved ref, subject digest, predicates, + base-image digest, runtime lockfiles, OS package set, Git/OCI tool versions, and + Codex/Pi versions. Release E2E uses that digest, never `latest`. Requests + without the configured metadata key remain stock-compatible. CI rebases + selected upgrades and runs upstream plus AllAgents integration tests. - **R15.** AI Evals owns its Promptfoo provider. It sends the UHP request directly to HarnessRouter, maps Promptfoo variables to the closed extension, and maps terminal output, usage, artifacts, provenance, and failures to @@ -304,31 +378,42 @@ caller responsible for acquisition. The temporary fork closes those seams. #### F1. Start the deployment -1. Start `codex-lb` with provider OAuth, proxy API-key authentication enabled, - and an API key held only by the HarnessRouter gateway. Validate that the - key/model combination authorizes each configured Codex model. -2. Run the AllAgents materializer's bounded `preflight` mode. It validates the +1. Run the AllAgents materializer's bounded `preflight` mode. It validates the project catalog, snapshot and credential-reference schemas, referenced secret presence, required Git/OCI tools, hook contract version, and staging/workspace filesystem relationship without contacting sources. -3. Start the pinned custom HarnessRouter image with durable `/data`, private - listener, HarnessRouter client key, broker mode, matching private - `HARNESS_PUBLIC_BASE_URL` and runner-reachable `HARNESS_GATEWAY_URL`, Codex - `openai`/OpenAI-auth provider fields, materializer command, project - configuration, and allowlisted source secret environment. -4. From the exact container network, verify configured harness IDs/model - allowlists, broker credential minting, authenticated `codex-lb` `/responses` - and `/responses/compact`, a resumed Codex turn, safe roots, and materializer - version. Advertise Pi only if its separate live format/endpoint probe passed. - Any required preflight failure prevents readiness. +2. In a controlled operator context, initialize each dedicated auth profile: + run Codex login with that target's `CODEX_HOME`, or run Pi `/login` with that + target's isolated Pi home and configured provider. Persist only the selected + harness profile; do not copy a general developer home into the service. +3. If native OAuth cannot satisfy the deployment's trust or compatibility + requirement, deliberately select a separately configured `proxyApiKey` + deployment profile. Validate its closed proxy connection, start the tested + proxy, keep its client key and upstream credentials outside the runner, and + enable HarnessRouter broker mode. Never configure it as automatic failover + for a native target. +4. Start the attestation-verified, digest-pinned custom HarnessRouter image with + durable session data, durable harness auth roots, a private listener, + HarnessRouter caller key, materializer command, project configuration, + owner-only source-secret mount/credential-store handle, and the configured + native or proxy trust mode. +5. From the exact container network, verify each advertised harness ID and model + allowlist, safe roots, materializer version, selected auth binding, and a live + turn. Native targets exercise login status, atomic local refresh persistence, + stale-profile repair, same-binding continuation, and serialized overlapping + turns. An explicit proxy deployment exercises schema/TLS/model-map/endpoint + compatibility plus bounded in-turn use and wrong-scope/expired/revoked + rejection. Any required + preflight or auth failure prevents readiness. #### F2. Execute the first repository-backed turn 1. Promptfoo sends one authenticated UHP request with `model`, stock `metadata.harness_id`, idempotency input, and the AllAgents workspace object. -2. HarnessRouter validates UHP plus generic metadata bounds, creates the - response/session, CASes materialization from `unbound` to `materializing`, and - hydrates a fresh session workspace. +2. HarnessRouter validates UHP plus generic metadata bounds, resolves and + persists the selected harness target and canonical auth-binding identity/ + config digest, creates the response/session, CASes materialization from + `unbound` to `materializing`, and hydrates a fresh session workspace. 3. Before provider selection, the gateway calls runner `/materialize`. The AllAgents child validates the descriptor and catalog, resolves exact commits and source credentials, writes and validates staging, removes credential @@ -337,10 +422,12 @@ caller responsible for acquisition. The temporary fork closes those seams. writes its marker, initializes the root checkpoint plus each declared repository's collection cursor, and returns `published`. The gateway stores a durable checkpoint and CASes the session to `ready`. -5. HarnessRouter applies ordinary input files, resolves the safe nested cwd, and - enters its provider loop. Codex receives only a turn broker credential and - reaches `codex-lb` through HarnessRouter; materialization cannot rerun during - provider fallback. +5. HarnessRouter applies ordinary input files and resolves the safe nested cwd. + In `nativeOAuth`, it projects only the persisted Codex or Pi profile and the + harness calls its provider directly. In `proxyApiKey`, it projects no native + profile and supplies only the scoped turn credential and broker base URL for + the persisted proxy connection. Materialization cannot rerun during provider + retry/fallback, and failure never changes auth mode. 6. Normal UHP events and every stored/retrieved terminal response include the same namespaced provenance. Produced-file collection walks the HarnessRouter root plus each declared nested repository without reporting initial source @@ -351,12 +438,15 @@ caller responsible for acquisition. The temporary fork closes those seams. 1. The caller sends `previous_response_id` and omits `metadata["allagents.workspace"]`. 2. HarnessRouter resolves its current session state and writable workspace, - requires materialization `ready`, and does not invoke the hook again. An + requires materialization `ready`, and resolves the persisted auth-binding + identity and config digest. An unavailable or changed binding, extension-bearing continuation, concurrent active turn, or non-resumable session fails before runner work. -3. HarnessRouter follows its stock predecessor/session semantics, resumes the - native conversation, and returns pinned provenance plus new output, usage, - and artifacts. AllAgents does not add a stricter head CAS. +3. HarnessRouter follows its stock predecessor/session semantics. Native mode + projects the persisted profile; proxy mode mints a new scoped turn token for + the persisted connection. It resumes the native conversation and returns + pinned provenance plus new output, usage, and artifacts without changing auth + mode or binding. #### F4. Execute an OCI-backed first turn @@ -386,9 +476,11 @@ caller responsible for acquisition. The temporary fork closes those seams. - **AE1.** A stock UHP request without the configured metadata key produces the same response and conformance result on upstream HarnessRouter and the fork. -- **AE2.** An unauthenticated request is rejected. An authenticated Codex turn - reaches `codex-lb` while the literal HarnessRouter client key and long-lived - `codex-lb` key are absent from the agent environment and filesystem. +- **AE2.** Every unauthenticated external create, continuation, GET, stream, + cancel, file, and artifact request fails before resource existence or metadata + is disclosed. An authenticated native Codex or Pi turn uses the selected OAuth + profile; the HarnessRouter caller key is absent from the agent environment and + filesystem, and no provider-route API key exists in that mode. - **AE3.** Repository mode resolves configured branch/default/HEAD refs to full commits, prepares every execution-eligible repository, preserves nested Git history, starts in a validated nested cwd, and returns path-free provenance. @@ -413,17 +505,27 @@ caller responsible for acquisition. The temporary fork closes those seams. and whiteouts and rejects mutable tags, indexes, mismatched digests/sizes, traversal, escaping links, devices, sparse files, unknown media types, and declared-limit overflow. -- **AE10.** Codex uses HarnessRouter's custom Responses integration and broker. - Pi is advertised only if its separate live custom-format probe passes. Failure - disables Pi without owner trust, direct-provider fallback, or another proxy. -- **AE11.** Restart after a completed first turn preserves the session and allows - continuation. Restart during materialization recovers only from a matching - published marker and durable checkpoint; otherwise it fails without automatic - replay. Restart during an agent turn follows HarnessRouter's interrupted-turn - failure behavior. -- **AE12.** The custom image records exact upstream, patch, materializer, Codex, - optional Pi, and image versions; rebuilding locked inputs produces equivalent - contract and conformance results. +- **AE10.** Codex signs in and refreshes through Codex CLI; Pi signs in and + refreshes through Pi for its configured provider. Missing, revoked, expired, + unrefreshable, or locally stale-after-crash OAuth marks only that profile + unavailable or `repair-required`; it never selects another profile or proxy. + A second turn sharing a profile waits or fails before launch. A separately + configured `proxyApiKey` deployment's broker permits the bounded multi-request + provider flow during the active turn and rejects wrong-audience, wrong-model, + wrong-turn, expired, or revoked credentials. +- **AE11.** Restart after a completed first turn preserves the session and exact + persisted auth binding. Removing or changing that binding makes continuation + fail before runner work; restoring the matching identity and config digest + restores eligibility. Restart during materialization recovers only from a + matching published marker and durable checkpoint; otherwise it fails without + automatic replay. Restart during an agent turn follows HarnessRouter's + interrupted-turn failure behavior. +- **AE12.** The protected publish job releases the public `linux/amd64` GHCR + package without Docker Hub credentials. An anonymous client reads the manifest, + verifies the GitHub/Sigstore build-provenance and SBOM attestations' expected + owner, repository, workflow, approved ref, subject digest, and predicate, and + pulls that digest rather than `latest`. The evidence records upstream, patch, + materializer, Codex, and Pi inputs; mismatches fail closed. - **AE13.** Promptfoo maps one stable materializer failure and one HarnessRouter execution failure to failed `ProviderResponse` results with code, safe message, and metadata; neither becomes a successful empty response. @@ -432,16 +534,18 @@ caller responsible for acquisition. The temporary fork closes those seams. **In scope** -- HarnessRouter workspace-integration patch and generic materializer boundary. +- HarnessRouter workspace-integration and harness-auth-state patches. - Versioned AllAgents workspace descriptor, hook request/result, and provenance. - Project workspace schema additions and catalog projection. - Deterministic Git and immutable OCI acquisition. - Root/nested-repository checkpoint and produced-file integration. -- Source and provider credential containment with leak verification. -- Custom image build and pinned release metadata. -- HarnessRouter broker plus `codex-lb` Codex Responses configuration. +- Source credential isolation and native-OAuth trust-boundary verification. +- Native Codex and Pi auth-profile bootstrap, refresh, readiness, and continuity. +- Explicit, separately validated API-key-authenticated proxy deployment mode. +- Public GHCR image publishing, digest-pinned release metadata, SBOM, and + provenance. - Promptfoo contract examples and one-shot/two-turn success/failure E2E. -- Upstream-ready generic hook patch and maintenance procedure. +- Upstream-ready generic hook and auth-state patches plus maintenance procedure. **Out of scope** @@ -449,7 +553,8 @@ caller responsible for acquisition. The temporary fork closes those seams. - A separate AllAgents network gateway, process supervisor, provider adapter, or artifact service. - Promptfoo runtime code inside AllAgents. -- Provider OAuth handling outside `codex-lb`. +- A custom OAuth broker, token translation layer, or automatic native-to-proxy + credential fallback. - Caller-provided origins, credentials, commands, host paths, materializers, or Docker options. - Public multi-tenancy, per-caller authorization, Kubernetes workers, session @@ -465,10 +570,10 @@ caller responsible for acquisition. The temporary fork closes those seams. - [UHP sessions](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/sessions.md) - [UHP lifecycle](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/lifecycle.md) - [UHP files](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/files.md) -- [`codex-lb` repository](https://github.com/Soju06/codex-lb) -- [`codex-lb` client setup](https://github.com/Soju06/codex-lb/blob/main/docs/client-setup.md) -- [`codex-lb` API keys](https://github.com/Soju06/codex-lb/blob/main/docs/api-keys.md) -- [`codex-lb` routing](https://github.com/Soju06/codex-lb/blob/main/docs/routing.md) +- [Codex authentication](https://developers.openai.com/codex/auth) +- [Pi model providers and OAuth](https://pi.dev/docs/latest/providers) +- [GitHub Container Registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry) +- [`codex-lb` optional proxy](https://github.com/Soju06/codex-lb) - [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) - [Source credential broker precedents](../research/source-credential-broker-precedents.md) @@ -486,18 +591,26 @@ flowchart TB MAT --> CFG[project workspace.yaml] MAT --> GIT[Git sources] MAT --> OCI[OCI registry] - RUN -->|turn broker credential| GW - GW -->|codex-lb API key| LB[codex-lb Responses API] - LB -->|OAuth + routing| MODEL[Model provider] + RUN --> HARNESS[Selected Codex or Pi harness] + AUTH[(dedicated durable OAuth profile)] -.->|native mode only| HARNESS + HARNESS -->|native mode| MODEL[Model provider] + HARNESS -.->|proxy mode: scoped turn credential| GW + GW -.->|long-lived proxy client key| PROXY[optional provider proxy] + PROXY -.-> MODEL GW --> DATA[(durable session checkpoints)] ``` The gateway owns generic metadata bounds, session materialization state, -provider-loop ordering, checkpoint persistence, and response metadata. The -runner owns hook invocation, staged publication, checkpoint/collection setup, -safe nested cwd, and agent launch. The materializer owns only AllAgents schema, -catalog, acquisition, credential selection, staging validation, and provenance; -it never speaks UHP or publishes the live workspace. +persisted auth-binding identity, provider-loop ordering, checkpoint persistence, +response metadata, and optional proxy brokering. The runner owns hook invocation, +staged publication, checkpoint/collection setup, safe nested cwd, and agent +launch. In native mode it projects the bound OAuth profile; in proxy mode it +projects no native profile and supplies only the turn broker capability. The +selected harness owns native provider OAuth login and refresh. Its durable auth +root is outside session checkpoints; it is available to the harness trust +boundary but never to the materializer. The materializer owns only AllAgents +schema, catalog, acquisition, source-credential selection, staging validation, +and provenance; it never speaks UHP or publishes the live workspace. ### Extension Contract @@ -610,9 +723,12 @@ published. changes in touched gateway/runner/session code, run upstream tests and UHP conformance, run AllAgents hook/session E2E, rebuild the image, and record the new inputs and digest. -- Prepare the upstream proposal as a generic command/plugin seam. Do not require - upstream to understand AllAgents metadata, Git catalogs, OCI manifests, or - Promptfoo. +- Publish releases to `ghcr.io/allagentsdev/harnessrouter`, record the manifest + digest, and rehearse deployment from that digest rather than a local build or + mutable tag. +- Prepare upstream proposals as generic command/plugin and harness-auth-state + seams. Do not require upstream to understand AllAgents metadata, Git catalogs, + OCI manifests, Promptfoo, or a specific OAuth provider. - If upstream accepts an equivalent seam, delete the patch rather than retaining a compatibility layer. @@ -626,10 +742,21 @@ published. - **Nested Git versus HarnessRouter root Git:** Keep repository `.git` state, ignore declared roots in HarnessRouter's root index, and extend produced/list/ file/ack/checkpoint/hydrate behavior to validate and walk every declared root. -- **Provider fallback:** Run materialization before the provider candidate loop - and surface a typed non-provider failure; a ready marker prevents reruns. -- **Credential leakage:** Broker the `codex-lb` key and use subprocess-only source - credentials, hermetic configuration, leak scans, and hostile fixtures. +- **Provider retry/fallback:** Run materialization before the provider candidate + loop and surface a typed non-provider failure; a ready marker prevents reruns. +- **Source credential leakage:** Use subprocess-only source credentials, + hermetic configuration, leak scans, and hostile fixtures. +- **Native OAuth exposure:** Treat the selected profile as available to the + harness and same-identity tools. Mount no other profile and prevent passive + gateway/runner persistence from serializing the auth file. A malicious harness + or tool can still emit its contents; use explicit proxy mode when this + owner-trust boundary is unacceptable. +- **OAuth refresh loss or races:** Hold one per-profile lock for every + refresh-capable turn and administrative mutation. Persist local writes through + same-filesystem temp-write, file `fsync`, atomic rename, parent `fsync`, and + validation. Fault process death around local persistence; if remote rotation + leaves the committed profile invalid, mark it `repair-required`. Never switch + profiles or auth mode, and never run shared-profile turns concurrently. - **Partial publication:** The materializer writes staging only. The runner validates and publishes with recovery markers; no agent runs until the gateway durably stores the resulting checkpoint and marks the session ready. @@ -637,32 +764,41 @@ published. persist the hook's effective digest/provenance for every later response. - **OCI attack surface:** Use a closed media profile, streaming digest checks, fixed limits, strict path/link/type validation, and exact-host redirect policy. -- **Pi incompatibility:** Keep Pi capability-gated; a failed format/endpoint probe - removes it from advertised harnesses without fallback. +- **Harness auth drift:** Pin Codex and Pi versions and require non-secret login, + live-turn, refresh, and continuation probes before advertising each target. - **HarnessRouter restart semantics:** Claim persistence only for completed state on durable storage; interrupted work fails and is not replayed. - **Provider cache assumptions:** Report native cached-input usage when available; never promise a cache hit. +- **Registry/tag or publisher compromise:** Use a protected, environment-approved + publish job with pinned actions and no write authority in build/test. Verify + the attested owner/repository/workflow/ref/subject digest before deployment. - **Upstream rejection:** The pinned fork remains supported; upstream delivery is maintenance reduction, not a launch dependency. ### Phased Delivery 1. Red E2E against stock HarnessRouter: prove arbitrary metadata is neither - forwarded to Codex/Pi nor returned as workspace provenance. + forwarded to Codex/Pi nor returned as workspace provenance, and document the + stock separation between session and excluded auth files. 2. Fork spike: prove a fake hook runs through a dedicated pre-provider operation, publishes/checkpoints once, survives two-turn reuse, supports a safe nested cwd, reports nested-repository files, and cannot rerun under provider fallback. -3. Prove brokered Codex through `codex-lb`; probe Pi's separate supported format - and mark it available or unavailable without changing the architecture. -4. Freeze generic hook envelope/state fixtures and AllAgents descriptor, + Prove one selected harness auth profile persists refresh without entering the + checkpoint or exposing another profile, and prove a second turn sharing that + profile waits or fails before launch. +3. Prove native Codex and Pi login, refresh, live turn, continuation, and + failure behavior. Prove an explicit authenticated-proxy deployment separately; + no native auth failure may route to it. +4. Freeze generic hook envelope/state/auth fixtures and AllAgents descriptor, configuration, provenance, and failure fixtures. 5. Implement project schema projection, preflight, Git materialization, credential containment, and checkpoint/collection integration. 6. Implement OCI materialization and its archive/registry security profile. 7. Run Promptfoo one-shot, continuation, cancellation, restart, and failure - mappings; review both repositories; build the exact image; run green E2E and - conformance; document operations; prepare the generic upstream patch. + mappings; review both repositories; publish the exact GHCR image; deploy its + digest; run green E2E and conformance; document operations; prepare the + generic upstream patches. --- @@ -679,15 +815,22 @@ published. - **Approach:** Pin upstream. Add configured opaque metadata extraction and bounds, generic result envelope, materialization CAS, dedicated runner operation before the provider loop, response-translator persistence, safe - nested cwd, staged publication, pre-agent checkpoint, and nested-repository - collection. Add startup preflight. Configure broker mode and probe Codex plus - optional Pi route separately. + nested cwd, staged publication, pre-agent checkpoint, nested-repository + collection, and durable auth-profile projection outside session state. Add + startup preflight. Probe Codex and Pi native auth separately. - **Verification:** Upstream UHP conformance stays green. Stock requests are unchanged. Faults at every state/publication/checkpoint boundary fail closed. - Provider fallback cannot rerun the hook. A second turn reuses workspace and - provenance without the extension. Root and nested repository files collect - correctly, escaping cwd fails, broker secrets stay outside the agent, and Pi - advertisement exactly matches its probe. + Provider fallback cannot rerun the hook. A continuation reuses workspace, + provenance, and the persisted auth binding without the extension. A changed or + unavailable binding fails before runner work; an overlapping turn on the same + native profile waits or fails before launch. Root and nested repository files + collect correctly, escaping cwd fails, and only the selected auth profile is + visible to the harness identity. Fault immediately before, during, and after + local refresh persistence; restart sees a complete locally committed file and + either validates it or marks the profile `repair-required`, never silently + changing profile or auth mode. An inert-agent probe confirms the gateway/runner + never automatically serializes the auth file into a checkpoint, produced-file + record, passive log, or response. ### U1. AllAgents workspace contracts and Git materializer @@ -720,20 +863,31 @@ published. - **Repositories/files:** HarnessRouter session/response persistence and tests; AllAgents credential-selection/environment code and hostile fixtures. - **Approach:** Persist `unbound/materializing/ready/failed`, opaque request - digest, effective descriptor digest, public provenance, workspace marker, and - checkpoint digest through compare-and-set transitions. Extend every response - construction/retrieval/replay path with identical public metadata. Resolve - source `${ENV_VAR}` references only in the child, remove the complete - materializer-only name set from agent children, broker the provider key, and - scan workspace, nested Git, CLI state, checkpoint, logs, and responses. + digest, effective descriptor digest, public provenance, workspace marker, + checkpoint digest, and canonical auth-binding identity/digest through + compare-and-set transitions. Extend every response construction/retrieval/ + replay path with identical public metadata. Resolve source `${ENV_VAR}` + references only when constructing the materializer child from an owner-only + runner secret source; reject configured names or values in the base service or + agent environment. In native mode, project only the selected harness OAuth + profile and exclude it from passive session persistence. In explicit proxy + mode, broker the proxy client key with the required audience, target, model, + turn, expiry, and revocation constraints. Scan workspace, nested Git, CLI + session state, checkpoints, logs, and responses. - **Verification:** Initial idempotent replay preserves one result; continuation - omits the extension and reuses ready state; extension-bearing continuation - fails. Crashes around hook/publication/checkpoint/CAS reconcile to ready only - when the bound descriptor, published marker, and durable checkpoint all match; - missing/corrupt markers, descriptor mismatch, and checkpoint mismatch become - failed/non-resumable. Acquisition secrets, including a non-secret-looking - configured variable name, and the long-lived `codex-lb` key are absent - everywhere the shell-enabled agent can read. + omits the extension and reuses ready state plus the exact auth binding; + extension-bearing continuation or changed/unavailable binding fails. Crashes + around hook/publication/checkpoint/CAS reconcile to ready only when the bound + descriptor, published marker, and durable checkpoint all match; missing or + mismatched evidence becomes failed/non-resumable. Source secrets and the + HarnessRouter caller key are absent from the base service and every + shell-enabled agent path. The selected OAuth profile is available only through + its harness home; other profiles are inaccessible. An inert-agent probe + confirms no gateway/runner path automatically serializes it into checkpoints, + produced-file records, passive logs, or public metadata; an active tool can + still exfiltrate it in owner-trust mode. Native mode has no provider-route API + key. Proxy tests allow bounded in-turn provider calls and reject every + out-of-scope, expired, or revoked broker token. ### U3. Immutable OCI workspace materialization @@ -753,44 +907,68 @@ published. devices, sparse files, limit overflow, cancellation, cleanup, and no Git fallback. -### U4. Codex, optional Pi, `codex-lb`, and Promptfoo E2E - -- **Goal:** Prove the real execution path, broker boundary, session continuity, - and consumer success/failure mapping. -- **Repositories/files:** custom image/configuration, HarnessRouter integration - fixtures, AI Evals Promptfoo provider/configuration in its owning repository, - and deployment examples in AllAgents docs. -- **Approach:** Configure Codex with HarnessRouter's custom Responses integration, - the `codex-lb` base, provider `name = "openai"`, and - `requires_openai_auth = true`. Enable private broker mode and `codex-lb` proxy - API-key authentication so only a turn credential enters the runner. Probe Pi - separately with a supported custom format and documented `codex-lb` endpoint; - advertise it only on success. Run Promptfoo Git/OCI one-shot and two-turn cases - plus materializer and agent failures. -- **Verification:** Through the exact container network, `/responses`, - `/responses/compact`, and a resumed turn authenticate and use an allowed model. - Turn two sees turn one's conversation and file mutation; `codex-lb` - continuation affinity holds; cancellation terminates the real turn; hop keys - and OAuth stay in their owning services; Pi capability is truthful; Promptfoo - returns successful output/usage/artifacts/provenance and maps both failure - classes to failed, coded responses rather than empty success. +### U4. Harness-native OAuth, optional proxy, and Promptfoo E2E + +- **Goal:** Prove both real harness-native auth paths, their explicit trust + boundary, session continuity, optional proxy isolation, and consumer + success/failure mapping. +- **Repositories/files:** custom image/configuration, auth-profile setup and + projection, HarnessRouter integration fixtures, AI Evals Promptfoo + provider/configuration in its owning repository, and deployment examples in + AllAgents docs. +- **Approach:** Configure dedicated Codex and Pi auth roots. Bootstrap them only + through each harness's login flow; do not inject a provider-route API key. + Exercise login status, live turns, atomic local refresh persistence, + stale-after-remote-rotation repair, same-binding continuation, serialized + overlapping turns, profile isolation, and missing/revoked credential failure. + In a separate explicit deployment profile, validate the closed proxy connection + and broker audience/target/model/turn/expiry/revocation contract. Run Promptfoo + Git/OCI one-shot and two-turn cases plus materializer, authentication, and + agent failures. +- **Verification:** Through the exact container network, Codex and Pi authenticate + through their own OAuth sessions and use allowed models. Turn two sees turn + one's conversation and file mutation with the same persisted auth binding; an + overlapping turn sharing that profile waits or fails before launch. A config + change or unavailable binding fails before runner work. Cancellation terminates + the real turn. OAuth failure disables the target without selecting another + profile or proxy. Faults around local refresh persistence leave a complete file + that either validates or marks the profile `repair-required`; unselected + profiles remain inaccessible. The separately configured proxy permits bounded + multi-request use inside the active turn and rejects wrong-audience, + wrong-target, wrong-model, wrong-turn, expired, or revoked credentials. + Promptfoo returns successful output/usage/artifacts/provenance and maps all + failure classes to failed, coded responses rather than empty success. ### U5. Release, operations, review, and upstream preparation -- **Goal:** Produce a reproducible supported image and an upstream-ready generic - hook proposal. -- **Repositories/files:** image build/release workflow, dependency lock and +- **Goal:** Produce a reproducible, registry-published supported image and + upstream-ready generic hook and auth-state proposals. +- **Repositories/files:** GHCR image build/release workflow, dependency lock and provenance record, fork-maintenance guide, deployment/reference docs, changelog, PR descriptions, and upstream patch series. -- **Approach:** Build from exact upstream/fork/AllAgents/agent inputs, emit image - digest and SBOM, run all conformance and E2E gates against that image, document - durable volumes, keys, private networking, upgrades, rollback, backup, failure - recovery, and the CE isolation boundary. Review both codebases before the final - green E2E. Split and explain the generic HarnessRouter patch for upstream. -- **Verification:** A clean host can deploy the recorded image and reproduce Git, - OCI, one-shot, continuation, cancellation, restart, and failure scenarios from - documented commands. Rebase rehearsal against the selected next upstream - commit either passes or reports an explicit incompatibility before release. +- **Approach:** Build from exact upstream/fork/AllAgents/agent inputs. Every base + image is digest-pinned; runtime lockfiles and version-locked OS packages, + Git/OCI tools, Codex, and Pi close the input set. The build fails on unpinned + input. Replace the inherited Docker Hub path with a no-write build/test job and + a separate protected, environment-approved GHCR publish job. Pin every + third-party action by commit. Publish the `linux/amd64` image to + `ghcr.io/allagentsdev/harnessrouter`, read back its manifest, create + build-provenance and SBOM attestations for the final digest, and run conformance + plus E2E only after verifying the expected repository, workflow, approved ref, + subject digest, predicates, complete build inputs, and anonymous pull. Document + durable session/auth volumes, native login/repair, optional proxy mode, private + networking, upgrades, rollback, secret-safe backup, and the CE owner-trust + boundary. Review both codebases before final green E2E. Split and explain the + generic HarnessRouter patches for upstream. +- **Verification:** A clean `linux/amd64` host verifies both attestations and the + pinned base/runtime/OS/Git/OCI/harness input set, then anonymously pulls the + public package by subject digest and reproduces Git, OCI, native Codex/Pi, + optional proxy, one-shot, continuation, cancellation, restart, and failure + scenarios from documented commands. No Docker Hub credential is required. An + unapproved ref or mismatched owner, repository, workflow, digest, attestation, + predicate, or build input fails closed. Rebase rehearsal against the selected + next upstream commit either passes or reports an explicit incompatibility + before release. --- @@ -799,37 +977,55 @@ published. | Gate | Required evidence | |---|---| | Stock compatibility | Upstream HarnessRouter tests and UHP conformance pass; requests without the configured metadata key are unchanged. | +| Caller authentication | Every unauthenticated external create, continuation, retrieval, stream, cancellation, file, and artifact request fails before resource existence or metadata disclosure; runner operations are private and mutually authenticated. | | Hook ordering | Dedicated materialization finishes, publishes, and checkpoints before provider selection; fallback never reruns it. | | Durable state | Fault injection proves only sessions with matching bound descriptor, published marker, and durable checkpoint become ready; missing/corrupt/mismatched evidence fails non-resumable without replay. | -| Session continuity | Two real turns share native conversation and writable workspace; continuation omits the extension and invocation count is one. | +| Session continuity | Two real turns share native conversation, writable workspace, and the persisted auth-binding identity/digest; continuation omits the extension. A changed or unavailable binding fails before runner work. | | Workspace integration | Safe nested cwd, repository-mode root/nested Git checkpoints, snapshot private tree baselines, produced list/file/ack, hydrate, and initial-source suppression pass. | | Git acquisition | Closed transport/config policy, constrained revisions, exact commits, non-root destinations, catalog validation, and partial cleanup pass against local HTTPS remotes. | | OCI acquisition | Digest/media/path/link/type/limit matrix passes against a real local registry. | -| Credential boundary | Every configured materializer-only environment name, all source secrets, and the long-lived `codex-lb` key are absent from every agent-readable environment/file/checkpoint and public output. | -| Provider boundary | Private broker minting works with the configured private public-base/gateway URL; Codex uses authenticated brokered `/responses` and `/responses/compact` with required provider fields; Pi is advertised only after its separate route probe; OAuth remains solely in `codex-lb`. | +| Credential boundary | Every configured source secret and the HarnessRouter caller key are absent from the base service and every agent-readable source/session path, checkpoint, log, and public output. The selected OAuth profile is available inside the documented harness owner-trust boundary; other profiles are inaccessible, and an inert-agent probe proves the gateway/runner never serializes the auth file automatically. | +| Provider boundary | Codex and Pi login, live-turn, atomic local refresh, crash/stale-profile repair, bound continuation, serialized overlapping turns, and failure probes use harness-native OAuth without a provider-route API key. OAuth failure never changes profile or auth mode. A separate proxy broker permits bounded in-turn calls and rejects wrong-scope, expired, or revoked tokens. | | Lifecycle | Streaming, cancellation, idempotency, artifacts, usage, response metadata, completed restart, and interrupted-work failure match the contract. | -| Packaging | Exact image digest records upstream commit, patch digest, materializer contract, agent versions, and SBOM. | +| Packaging | The public `linux/amd64` GHCR manifest and GitHub/Sigstore build-provenance and SBOM attestations are verified for expected owner, repository, workflow, approved ref, subject digest, predicates, base-image digest, runtime lockfiles, OS packages, Git/OCI tools, and harness versions. Deployment uses that digest and publishing needs no Docker Hub credential. | | Consumer | Promptfoo one-shot/two-turn Git/OCI success and materializer/agent failure mappings pass. | | Review | Final review findings in both repositories are resolved before the final built-image E2E. | ## Definition of Done - ADR 0002, this plan, implementation, deployment topology, and request examples - agree on UHP, the fork, the behind-router materializer, and `codex-lb`. + agree on UHP, the fork, the behind-router materializer, harness-native OAuth, + explicit proxy fallback, and GHCR digest-pinned distribution. - No second execution protocol, parallel task/session control plane, separate - AllAgents gateway, direct provider adapter, or client-side workspace expansion - remains in implementation scope. + AllAgents gateway, direct provider adapter, custom OAuth broker, automatic + auth-mode fallback, or client-side workspace expansion remains in + implementation scope. - R1-R15 and AE1-AE13 are implemented and verified against the exact released image. -- Stock UHP requests and upstream conformance remain green. +- Stock UHP requests and upstream conformance remain green. Every external + create/read/control/file/artifact path requires the HarnessRouter caller key; + runner operations remain private and mutually authenticated. - Git and OCI materialization publish and checkpoint before provider dispatch and happen exactly once per extension-bearing session. -- Continuation preserves native conversation/workspace state, omits the - extension, and follows HarnessRouter's predecessor semantics. +- Continuation preserves native conversation/workspace state and the persisted + auth-binding identity/digest, omits the extension, and follows HarnessRouter's + predecessor semantics. A changed or unavailable binding fails before runner + work. Every native auth profile serializes refresh-capable turns. Local refresh + writes are atomic and validated; a stale credential after remote rotation + becomes `repair-required` rather than changing profile or auth mode. - Repository-mode roots preserve usable Git state. Every source mode preserves truthful root/nested produced files across checkpoint/hydrate. -- Acquisition credentials and provider OAuth respect their separate boundaries. -- The image, patch series, materializer contract, operational docs, and rollback - procedure are reproducible from pinned inputs. +- Source credential values are read from an owner-only runner secret source and + injected only into the selected materializer child; they never enter the + gateway/runner base environment or harness. Provider OAuth is available inside + the explicitly accepted owner-trust boundary; the gateway/runner never + automatically persists the auth file in session state or public metadata. + Long-lived proxy client keys remain brokered, and turn tokens enforce audience, + target, model, turn, expiry, and revocation while permitting bounded in-turn + provider calls. +- The public `linux/amd64` GHCR manifest, verified build-provenance and SBOM + attestations, pinned base/runtime/OS/Git/OCI/harness inputs, patch series, + materializer contract, operational docs, and rollback procedure are + reproducible from pinned inputs. - The generic HarnessRouter hook patch is ready to propose upstream, but the shipped system remains operable from the maintained fork if it is not accepted. From ea1f7e1550705c74bcdd90a123ec92ab4ed8e3d9 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Tue, 22 Sep 2026 15:51:39 +1000 Subject: [PATCH 17/44] docs(architecture): gate workspace work on native auth --- .../0002-adopt-uhp-through-harnessrouter.md | 38 +++- ...0837-feat-coding-execution-gateway-plan.md | 191 +++++++++++------- 2 files changed, 153 insertions(+), 76 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 6d76408b..56c695a7 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -1,6 +1,6 @@ # ADR 0002: Adopt UHP through HarnessRouter with an AllAgents workspace materializer -- Status: Accepted; implementation pending +- Status: Accepted; implementation gated on native-auth feasibility - Date: 2026-09-21 ## Decision @@ -199,6 +199,41 @@ free of unrelated changes. The intended upstream contributions are the generic materializer boundary and secure harness-auth state separation, not the AllAgents-specific descriptor schema. +## Phase-zero feasibility gate + +The harness-native auth adapter is a blocking phase-zero spike. Production +workspace-materializer implementation must not begin until a minimal pinned image +using the release's HarnessRouter, base-image, and Codex/Pi inputs proves the +adapter with real provider traffic. The spike does not need the AllAgents +materializer, Git acquisition, or OCI acquisition. + +The gate evidence records the HarnessRouter commit, base-image digest, Codex +version, Pi version, and auth-adapter patch digest. Those inputs are frozen for +dependent work. Changing any of them invalidates the gate: dependent work must +stop until both native targets pass again on the new input set. + +Each required native target must prove: + +1. operator-controlled native login in its dedicated profile root; +2. a real first turn and continuation without a provider-route API key; +3. persisted auth-binding identity across restart and fail-closed behavior when + that binding is changed or unavailable; +4. session-specific conversation state with only the selected auth profile + visible to the harness identity; +5. serialized overlapping turns for one profile; +6. complete local credential files after termination before, during, and after + refresh persistence, with invalid post-rotation state becoming + `repair-required`; +7. no automatic gateway/runner serialization of the auth file into checkpoints, + produced-file records, passive logs, or response metadata; and +8. explicit acknowledgement that same-identity harness tools can read or emit + the selected credential. + +The proxy route cannot satisfy this gate on behalf of a native target. If either +required native target fails, dependent implementation stops. Continuing with a +proxy-only target or narrower harness scope requires an explicit decision change; +the implementation must not introduce an implicit fallback or credential shim. + ## Source authority and credentials The project `workspace.yaml` remains the source of truth for logical repository @@ -363,6 +398,7 @@ source-mode fallback, or guaranteed provider prompt-cache hits. Revisit this decision when: +- either required harness-native OAuth target cannot pass the phase-zero gate; - upstream HarnessRouter accepts the generic materializer hook or exposes an equivalent supported extension; - the maintained patch grows beyond the narrow integration boundary; diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 25f7a343..552f7fa4 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -35,24 +35,29 @@ execution: code secrets provide the values. HarnessRouter configuration owns harness IDs, model allowlists, and authentication bindings. The namespaced AllAgents extension owns acquisition and provenance semantics. -- **Execution order:** Prove the fork seam, Codex and Pi native-OAuth routes, - auth-state isolation, materialization state machine, and checkpoint integration - against a real HarnessRouter runner; freeze the generic hook and AllAgents - contracts; implement Git then OCI acquisition; prove an explicit proxy mode - separately; run Promptfoo one-shot and continuation E2E; complete release, - fork-maintenance, and upstream-ready documentation. -- **Stop conditions:** Stop before production implementation if materialization - cannot complete and durably checkpoint before provider dispatch, if nested Git - workspaces cannot be collected without corrupting HarnessRouter checkpoints, - if source credentials enter the harness, if the gateway/runner automatically - copies harness OAuth files into a materialized source tree, checkpoint, - produced-file record, passive log, or public metadata, if local OAuth state - cannot be persisted atomically and validated fail-closed while conversation - state remains session-scoped, or if the fork cannot preserve stock UHP - behavior and conformance. Do not fall back to prompt - instructions, an MCP acquisition tool, client-side repository upload, another - OAuth profile, an implicit API-key route, a second execution protocol, or a - parallel task/session engine. +- **Execution order:** First build the minimal custom image and pass the blocking + native-auth adapter gate for both Codex and Pi without implementing the + AllAgents materializer. Only then prove the workspace fork seam, + materialization state machine, and checkpoint integration against the real + runner; freeze the generic hook and AllAgents contracts; implement Git then + OCI acquisition; prove an explicit proxy mode separately; run Promptfoo + one-shot and continuation E2E; complete release, fork-maintenance, and + upstream-ready documentation. +- **Stop conditions:** Stop before production workspace implementation if either + required native target cannot pass the phase-zero gate: real login, first turn, + continuation, binding persistence, profile isolation, serialized overlapping + turns, refresh fault behavior, and passive-persistence checks. Also stop if + materialization cannot complete and durably checkpoint before provider + dispatch, nested Git workspaces cannot be collected without corrupting + HarnessRouter checkpoints, source credentials enter the harness, the + gateway/runner automatically copies harness OAuth files into a materialized + source tree, checkpoint, produced-file record, passive log, or public metadata, + local OAuth state cannot be persisted atomically and validated fail-closed + while conversation state remains session-scoped, or the fork cannot preserve + stock UHP behavior and conformance. Do not fall back to prompt instructions, + an MCP acquisition tool, client-side repository upload, another OAuth profile, + an implicit API-key route, a second execution protocol, or a parallel + task/session engine. - **Tail ownership:** Implementation owns focused tests in both repositories, upstream UHP conformance, built-image smoke tests, exact Git/OCI E2E, native Codex/Pi OAuth and explicit proxy-mode E2E, two-turn Promptfoo success and @@ -778,61 +783,88 @@ published. ### Phased Delivery -1. Red E2E against stock HarnessRouter: prove arbitrary metadata is neither - forwarded to Codex/Pi nor returned as workspace provenance, and document the - stock separation between session and excluded auth files. -2. Fork spike: prove a fake hook runs through a dedicated pre-provider operation, - publishes/checkpoints once, survives two-turn reuse, supports a safe nested - cwd, reports nested-repository files, and cannot rerun under provider fallback. - Prove one selected harness auth profile persists refresh without entering the - checkpoint or exposing another profile, and prove a second turn sharing that - profile waits or fails before launch. -3. Prove native Codex and Pi login, refresh, live turn, continuation, and - failure behavior. Prove an explicit authenticated-proxy deployment separately; - no native auth failure may route to it. -4. Freeze generic hook envelope/state/auth fixtures and AllAgents descriptor, +1. Phase-zero native-auth adapter spike: build the pinned minimal image without + the AllAgents materializer and implement only auth-profile projection, + persisted binding identity, per-profile serialization, and passive exclusion. +2. Run the blocking Codex and Pi native-auth gate with real provider traffic: + login, first turn, continuation, restart, changed-binding failure, overlapping + turns, refresh faults, profile isolation, and checkpoint/log/output scans. If + either required target fails, stop and revisit ADR 0002 before workspace work. +3. Red E2E against stock HarnessRouter: prove arbitrary metadata is neither + forwarded to Codex/Pi nor returned as workspace provenance. +4. Workspace fork spike: prove a fake hook runs through a dedicated pre-provider + operation, publishes/checkpoints once, survives two-turn reuse, supports a + safe nested cwd, reports nested-repository files, and cannot rerun under + provider fallback. +5. Freeze generic hook envelope/state/auth fixtures and AllAgents descriptor, configuration, provenance, and failure fixtures. -5. Implement project schema projection, preflight, Git materialization, +6. Implement project schema projection, preflight, Git materialization, credential containment, and checkpoint/collection integration. -6. Implement OCI materialization and its archive/registry security profile. -7. Run Promptfoo one-shot, continuation, cancellation, restart, and failure - mappings; review both repositories; publish the exact GHCR image; deploy its - digest; run green E2E and conformance; document operations; prepare the - generic upstream patches. +7. Implement OCI materialization and its archive/registry security profile. +8. Prove the separately configured authenticated-proxy mode, then run Promptfoo + one-shot, continuation, cancellation, restart, and failure mappings. +9. Review both repositories; publish the exact GHCR image; verify and deploy its + attested digest; run green E2E and conformance; document operations; prepare + the generic upstream patches. --- ## Implementation Units -### U0. HarnessRouter fork and hook feasibility - -- **Goal:** Prove the smallest production-direction fork can materialize and - durably checkpoint one workspace before provider dispatch while preserving - stock UHP requests. +### U0. Harness-native auth adapter feasibility gate + +- **Goal:** Prove the native Codex and Pi authentication architecture before any + production workspace-materializer implementation. +- **Repositories/files:** Minimal pinned HarnessRouter fork image, + `runner/server.py`, Codex/Pi launch and home setup, auth-profile projection, + session auth-binding persistence, checkpoint exclusions, fault fixtures, and + focused runner/gateway tests. Do not add the AllAgents materializer or Git/OCI + acquisition in this unit. +- **Approach:** Initialize dedicated profiles only through `codex login` and Pi + `/login`. Project the selected auth files into session-specific homes while + keeping conversation state session-scoped. Persist the binding identity/digest, + serialize every refresh-capable turn per profile, preserve atomic local writes, + mark invalid post-rotation state `repair-required`, mount no other profile, and + prevent passive checkpoint/log/output serialization. Use the actual pinned + harness versions and real provider traffic. + Freeze and record the HarnessRouter commit, base-image digest, Codex version, + Pi version, and auth-adapter patch digest used by the gate. +- **Verification:** For both Codex and Pi, complete login, a real first turn, + continuation, and restart without a provider-route API key. Change or remove + the binding and prove continuation fails before runner work. Force overlapping + turns and prove the second waits or fails before launch. Terminate immediately + before, during, and after local refresh persistence; restart must see a + complete file that validates or becomes `repair-required`. Prove unselected + profiles and other sessions' conversation state are inaccessible. With an + inert agent, scan checkpoints, produced-file records, passive logs, and response + metadata for automatic credential serialization. Record that an active + same-identity tool can still read or emit the selected credential. + Preserve those exact input identities with the evidence. +- **Gate:** U1-U6 must not begin until both required native targets pass. Failure + stops dependent work and reopens ADR 0002; proxy-only scope requires an + explicit decision change and cannot count as a passing native gate. Any change + to a frozen input invalidates the gate and stops dependent work until both + native targets pass again on the new input set. + +### U1. HarnessRouter fork and hook feasibility + +- **Goal:** Prove the smallest production-direction workspace fork can + materialize and durably checkpoint one workspace before provider dispatch + while preserving stock UHP requests. - **Repositories/files:** HarnessRouter fork `gateway/app.py`, `runner/server.py`, response/session persistence, checkpoint/produced-file - helpers, runner/gateway tests, image entrypoint/Dockerfile, and fake hook. -- **Approach:** Pin upstream. Add configured opaque metadata extraction and - bounds, generic result envelope, materialization CAS, dedicated runner - operation before the provider loop, response-translator persistence, safe - nested cwd, staged publication, pre-agent checkpoint, nested-repository - collection, and durable auth-profile projection outside session state. Add - startup preflight. Probe Codex and Pi native auth separately. -- **Verification:** Upstream UHP conformance stays green. Stock requests are + helpers, runner/gateway tests, and a fake materializer hook. +- **Approach:** Add configured opaque metadata extraction and bounds, generic + result envelope, materialization CAS, a dedicated runner operation before the + provider loop, response-translator persistence, safe nested cwd, staged + publication, pre-agent checkpoint, and nested-repository collection. +- **Verification:** Upstream UHP conformance stays green and stock requests are unchanged. Faults at every state/publication/checkpoint boundary fail closed. Provider fallback cannot rerun the hook. A continuation reuses workspace, - provenance, and the persisted auth binding without the extension. A changed or - unavailable binding fails before runner work; an overlapping turn on the same - native profile waits or fails before launch. Root and nested repository files - collect correctly, escaping cwd fails, and only the selected auth profile is - visible to the harness identity. Fault immediately before, during, and after - local refresh persistence; restart sees a complete locally committed file and - either validates it or marks the profile `repair-required`, never silently - changing profile or auth mode. An inert-agent probe confirms the gateway/runner - never automatically serializes the auth file into a checkpoint, produced-file - record, passive log, or response. - -### U1. AllAgents workspace contracts and Git materializer + provenance, and the persisted auth binding without the extension. Root and + nested repository files collect correctly, and an escaping cwd fails. + +### U2. AllAgents workspace contracts and Git materializer - **Goal:** Implement the versioned schemas, authoritative catalog projection, deterministic Git acquisition, logical cwd resolution, and provenance. @@ -856,7 +888,7 @@ published. file/ext protocols, redirects, cancellation, timeout, partial cleanup, canonical defaults, preflight failures, and exact provenance. -### U2. Session binding, failures, and credential containment +### U3. Session binding, failures, and credential containment - **Goal:** Make the fork/materializer boundary durable, fail-closed, and safe for continued sessions. @@ -889,7 +921,7 @@ published. key. Proxy tests allow bounded in-turn provider calls and reject every out-of-scope, expired, or revoked broker token. -### U3. Immutable OCI workspace materialization +### U4. Immutable OCI workspace materialization - **Goal:** Add the second closed source mode without weakening Git behavior or allowing fallback. @@ -907,20 +939,21 @@ published. devices, sparse files, limit overflow, cancellation, cleanup, and no Git fallback. -### U4. Harness-native OAuth, optional proxy, and Promptfoo E2E +### U5. Harness-native OAuth, optional proxy, and Promptfoo E2E -- **Goal:** Prove both real harness-native auth paths, their explicit trust - boundary, session continuity, optional proxy isolation, and consumer - success/failure mapping. +- **Goal:** Carry the phase-zero auth invariants unchanged into the complete + workspace image and prove session continuity, optional proxy isolation, and + consumer success/failure mapping. - **Repositories/files:** custom image/configuration, auth-profile setup and projection, HarnessRouter integration fixtures, AI Evals Promptfoo provider/configuration in its owning repository, and deployment examples in AllAgents docs. -- **Approach:** Configure dedicated Codex and Pi auth roots. Bootstrap them only - through each harness's login flow; do not inject a provider-route API key. - Exercise login status, live turns, atomic local refresh persistence, - stale-after-remote-rotation repair, same-binding continuation, serialized - overlapping turns, profile isolation, and missing/revoked credential failure. +- **Approach:** Reuse the accepted U0 adapter and fixtures; do not redesign the + native credential boundary here. Configure dedicated Codex and Pi auth roots + and bootstrap them only through each harness's login flow. Exercise login + status, live turns, atomic local refresh persistence, stale-credential repair + after remote rotation, same-binding continuation, serialized overlapping + turns, profile isolation, and missing/revoked credential failure. In a separate explicit deployment profile, validate the closed proxy connection and broker audience/target/model/turn/expiry/revocation contract. Run Promptfoo Git/OCI one-shot and two-turn cases plus materializer, authentication, and @@ -939,7 +972,7 @@ published. Promptfoo returns successful output/usage/artifacts/provenance and maps all failure classes to failed, coded responses rather than empty success. -### U5. Release, operations, review, and upstream preparation +### U6. Release, operations, review, and upstream preparation - **Goal:** Produce a reproducible, registry-published supported image and upstream-ready generic hook and auth-state proposals. @@ -948,7 +981,9 @@ published. changelog, PR descriptions, and upstream patch series. - **Approach:** Build from exact upstream/fork/AllAgents/agent inputs. Every base image is digest-pinned; runtime lockfiles and version-locked OS packages, - Git/OCI tools, Codex, and Pi close the input set. The build fails on unpinned + Git/OCI tools, Codex, and Pi close the input set. U6 must use the input + identities frozen by the current U0 evidence. If any covered input must change, + stop release work and rerun U0 before resuming. The build fails on unpinned input. Replace the inherited Docker Hub path with a no-write build/test job and a separate protected, environment-approved GHCR publish job. Pin every third-party action by commit. Publish the `linux/amd64` image to @@ -976,6 +1011,7 @@ published. | Gate | Required evidence | |---|---| +| Native-auth feasibility | Before workspace-materializer production work, the minimal image proves real Codex and Pi login, first turn, continuation, binding persistence, profile isolation, serialized overlap, refresh-fault repair, and passive exclusion without a provider-route API key. Evidence records the HarnessRouter commit, base-image digest, Codex and Pi versions, and auth-adapter patch digest. Both required targets pass; proxy mode is not substitute evidence, and changing a recorded input invalidates the gate until both pass again. | | Stock compatibility | Upstream HarnessRouter tests and UHP conformance pass; requests without the configured metadata key are unchanged. | | Caller authentication | Every unauthenticated external create, continuation, retrieval, stream, cancellation, file, and artifact request fails before resource existence or metadata disclosure; runner operations are private and mutually authenticated. | | Hook ordering | Dedicated materialization finishes, publishes, and checkpoints before provider selection; fallback never reruns it. | @@ -996,6 +1032,11 @@ published. - ADR 0002, this plan, implementation, deployment topology, and request examples agree on UHP, the fork, the behind-router materializer, harness-native OAuth, explicit proxy fallback, and GHCR digest-pinned distribution. +- The recorded U0 gate evidence predates U1-U6 implementation and shows both + required native targets passed on the recorded input set. Every later change + to a covered input has replacement passing evidence before dependent work + resumes. A failed or narrowed target has a superseding explicit ADR rather than + an implicit proxy or credential workaround. - No second execution protocol, parallel task/session control plane, separate AllAgents gateway, direct provider adapter, custom OAuth broker, automatic auth-mode fallback, or client-side workspace expansion remains in From 3ec035ab1540e256346934faf7be16cb8c36427c Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Tue, 22 Sep 2026 19:11:40 +1000 Subject: [PATCH 18/44] docs(architecture): compare E2B with execution gateway Document E2B's sandbox and self-hosting boundaries, explain why it is not a HarnessRouter replacement, and keep E2B-derived runtime work out of version one. --- .../e2b-execution-gateway-patterns.md | 186 ++++++++++++++++++ 1 file changed, 186 insertions(+) create mode 100644 docs/research/e2b-execution-gateway-patterns.md diff --git a/docs/research/e2b-execution-gateway-patterns.md b/docs/research/e2b-execution-gateway-patterns.md new file mode 100644 index 00000000..22df48cb --- /dev/null +++ b/docs/research/e2b-execution-gateway-patterns.md @@ -0,0 +1,186 @@ +# E2B execution gateway patterns + +## Scope and evidence date + +This note describes E2B's public product and source as of **2026-09-22**. Source links to code pin runtime commit [`9dd5b72`](https://github.com/e2b-dev/runtime/tree/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8) (2026-09-22) and SDK commit [`ccaf9fc`](https://github.com/e2b-dev/E2B/tree/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b) (2026-09-18). Links to unversioned official product documentation were accessed 2026-09-22. + +## Product boundary + +E2B exposes **sandbox infrastructure**, not an agent-neutral execution protocol. Its public contract creates and controls Linux sandboxes, runs commands and PTYs, reads and writes files, exposes guest ports, and manages templates, snapshots, and volumes. The official [coding-agent guide](https://docs.e2b.dev/use-cases/coding-agents.md) requires the application to install its chosen agent in a template and extract the resulting diff or files. The [Codex integration](https://docs.e2b.dev/agents/codex.md) likewise creates a sandbox, passes a Codex credential, clones a repository, invokes `codex exec`, interprets Codex-specific output, retrieves a diff, and kills the sandbox in caller code. The [public OpenAPI document](https://docs.e2b.dev/openapi-public.yaml) describes sandbox infrastructure resources rather than a common agent run/session/event/result model. + +That makes E2B usable as a sandbox provider beneath an execution gateway. Repository acquisition, harness selection, credential policy, prompt construction, reconnect/retry behavior, normalized events, terminal outcomes, and result provenance remain responsibilities above E2B. + +## Runtime architecture + +The open [`e2b-dev/runtime`](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/README.md) repository describes itself as the complete backend used by E2B Cloud, Enterprise, and Embed. Its [architecture specification](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/ARCHITECTURE.md) defines these boundaries: + +- **API/control plane:** authenticates callers, enforces quotas, places sandboxes, and records durable and ephemeral state. +- **Orchestrator:** runs on each KVM host and owns Firecracker, cgroups, network namespaces, veth/tap devices, NBD-backed root filesystems, snapshots, caches, and template builds. +- **Client proxy/data plane:** routes sandbox traffic directly to the selected orchestrator; guest traffic does not pass through the control-plane API. +- **`envd`:** runs inside each microVM and exposes authenticated process, PTY, filesystem, watcher, upload/download, signal, and port APIs. Public proxying rejects internal init, upgrade, freeze, and thaw routes. +- **State stores:** PostgreSQL holds durable metadata; Redis holds running-sandbox and routing state; ClickHouse holds events, metrics, and optionally logs; object storage holds template and paused-sandbox artifacts. + +Each sandbox is one Firecracker microVM with its own guest kernel. The Firecracker process gets its own cgroup and network namespace; the host supplies a COW root filesystem, memory restore, and network policy. This is a stronger guest boundary than a shared-kernel container, while still trusting a privileged, root-running host orchestrator and the host kernel/KVM boundary ([architecture](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/ARCHITECTURE.md), [security](https://e2b.dev/security)). + +## Lifecycle, templates, and persistence + +A template is a pre-booted snapshot of memory, filesystem, and machine state. Template builds execute layered phases, hash step inputs, cache reusable layers, and produce immutable artifacts. Sandbox creation restores that snapshot, lazily faults memory through `userfaultfd`, and overlays root filesystem writes. The result is fast creation without making the container image or install recipe the live runtime boundary ([architecture](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/ARCHITECTURE.md), [template mechanics](https://docs.e2b.dev/template/how-it-works.md), [cache semantics](https://docs.e2b.dev/template/caching.md)). + +E2B distinguishes three related artifacts: + +- **Declarative template:** reproducible start state built from a recipe; the preferred durable baseline. +- **Paused sandbox:** preserves filesystem, memory, and processes until explicitly killed in managed E2B ([persistence](https://docs.e2b.dev/sandbox/persistence.md)). +- **Live snapshot/fork:** captures a running sandbox as a reusable checkpoint; active PTY, WebSocket, and command streams disconnect and must be re-established ([snapshots](https://docs.e2b.dev/sandbox/snapshots.md)). + +Template and paused-sandbox storage use the same broad artifact shape: memory, rootfs, VM state, metadata, and memory/rootfs index files. Persistent volumes are a separate, private-beta resource, and their content path is served by a separate `belt` API rather than the main control-plane API ([volumes](https://docs.e2b.dev/volumes.md), [architecture](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/ARCHITECTURE.md)). E2B's file API transfers and manipulates guest files, but it does not define an agent-run artifact manifest or source/workspace provenance model ([filesystem upload](https://docs.e2b.dev/filesystem/upload.md)). + +## SDK and API shape + +The JavaScript and Python SDKs expose sandbox lifecycle, commands/process streaming, PTYs, filesystem operations, networking, templates, snapshots, volumes, and secrets. Callers can target another deployment through `E2B_API_URL`, `E2B_SANDBOX_URL`, an API key, or an explicit client/domain ([connection configuration](https://github.com/e2b-dev/E2B/blob/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b/packages/js-sdk/src/connectionConfig.ts), [custom client](https://docs.e2b.dev/client.md)). This is a clean provider seam, but it is a sandbox API seam rather than an agent/harness abstraction. + +Version compatibility needs active management. SDK release [`e2b@2.51.0`](https://github.com/e2b-dev/E2B/releases/tag/e2b%402.51.0) (2026-09-18) moved create/connect behavior onto v2 endpoints. The SDK [changelog](https://github.com/e2b-dev/E2B/blob/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b/packages/js-sdk/CHANGELOG.md) also warns that an older self-hosted/BYOC control plane can silently ignore a newer resume option. A provider integration therefore needs an explicit tested SDK/control-plane compatibility range. + +## Deployment and self-hosting boundary + +“Self-hosted E2B” currently describes more than one materially different operating model: + +| Mode | Where it runs | Who operates it | Current boundary | +| --- | --- | --- | --- | +| E2B Cloud | E2B account | E2B | Managed control and data planes. | +| BYOC | Customer AWS or GCP account/VPC | E2B | E2B provisions, monitors, and upgrades it; E2B Cloud remains the management/control plane. Official docs explicitly say this is managed deployment, not self-hosting ([BYOC](https://docs.e2b.dev/byoc.md), [security](https://e2b.dev/security)). | +| E2B Embed | One operator-owned KVM machine | Operator | The whole functional stack and sandboxes run locally; available as Compose, one-GCE-instance Terraform, or a single-node Kubernetes StatefulSet ([Embed](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/README.md)). | +| Private Cloud | Customer environment | Customer/E2B contract not yet public | Listed as “in development”; intended to keep both planes within the customer boundary ([enterprise](https://e2b.dev/enterprise)). | + +E2B is therefore self-runnable today as a complete **single-node functional stack**, but the public package is not a complete, supported **production multi-node self-operated distribution**. The Embed repository calls Compose, GCP Terraform, and Kubernetes “single-machine evaluation packages, not deployment patterns.” The current enterprise page calls Embed available and suitable for self-hosting/embedding while separately listing Private Cloud as in development. Both statements matter: availability does not establish HA, production operations, or air-gap support. + +### E2B Embed operational facts + +The [Compose guide](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/compose/README.md) requires Linux, KVM, `/dev/net/tun`, cgroup v2, NBD, 4 KiB pages, hugepages, Docker Engine 27+, Compose 2.24+, about 12 GiB RAM, and 20 GiB free disk. It supports x86-64 and arm64, with newer arm64 kernel requirements. It is not a rootless or container-only Firecracker deployment. + +The [reference](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/docs/REFERENCE.md) shows that Embed includes local PostgreSQL, Redis, ClickHouse, Vector, dashboard/API, client proxy, template builder, orchestrator, and local template/build storage. Logs remain in local ClickHouse for seven days. Running sandboxes end when the orchestrator stops; the launcher now sweeps sandbox cgroups on termination and after a crash. + +Durability depends on packaging: + +- Compose named volumes and local storage survive ordinary stack shutdown, but running VMs do not. +- The [Kubernetes package](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/kubernetes/README.md) is a privileged, host-network/host-PID StatefulSet pinned to one labelled KVM node. It uses node-local `hostPath`; sandboxes end on pod deletion/restart, and the guide calls for a dedicated node. +- The [GCP Terraform package](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/terraform/gcp/README.md) creates a managed instance group of one with no persistent data disk. Instance replacement loses databases and built templates. + +The default installation is not turnkey air-gapped. Initial Compose setup fetches images from Docker Hub and Google Artifact Registry and binaries from Google Storage/GitHub; template builds normally pull a base image. Published binaries are pinned and accompanied by SHA-256 files, but the public runtime repository is a read-only mirror of E2B's internal source-of-truth monorepo ([release process](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/RELEASING.md), [Embed reference](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/docs/REFERENCE.md)). No official offline-mirroring deployment procedure was found in the current public documentation. + +### Managed-feature parity + +The open runtime is substantial, but standard Embed is not configuration-equivalent to E2B Cloud/BYOC: + +- Embed's [Compose definition](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/compose/compose.yaml) does not configure the separate secret-store backend and explicitly disables volume-content token support. The API returns an error when the secret backend/feature is unavailable ([secret handler](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/packages/api/internal/handlers/secrets.go)). +- The SDK documents SOCKS5 egress proxying as Cloud/BYOC functionality that an open-source runtime deployment rejects ([SDK source](https://github.com/e2b-dev/E2B/blob/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b/packages/js-sdk/src/sandbox/sandboxApi.ts)). +- Workload-identity definitions can cross the open API/orchestrator contract, but the open [orchestrator protocol](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/packages/orchestrator/orchestrator.proto) explicitly says it does not mint, sign, or deliver the credential. + +These are feature-boundary facts, not evidence that the single-node runtime is a stub: it can build templates and run real Firecracker sandboxes through the same SDK surface. + +## Security and networking boundary + +The Firecracker/KVM boundary is complemented by per-sandbox cgroups, namespaces, NBD devices, NAT, nftables, and token-authenticated `envd`. Public sandbox ingress can require an access token. Those controls do not remove operator obligations around the privileged host and management network. + +Embed deliberately exposes a low-level local stack. Its [README](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/README.md) says 13 ports bind all interfaces. Only API, dashboard, and client proxy ports 3000–3002 are for trusted clients; the other ten must be firewalled. In particular, orchestrator gRPC on port 5008 is unauthenticated and grants full orchestrator control. Embed does not supply wildcard DNS or TLS. Its plain-HTTP dashboard configuration cannot mark the team-key cookie `Secure` ([reference](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/docs/REFERENCE.md)). + +Outbound internet access defaults on. E2B supports IP/CIDR allow and deny lists plus HTTP Host/TLS SNI domain allowlists, but its [network documentation](https://docs.e2b.dev/network/internet-access.md) records important limits: domain filtering sees Host only on HTTP/80 and SNI only on TLS/443; it does not cover UDP/QUIC or arbitrary ports; allow wins over deny; shared CDN/IP use weakens domain isolation; and a blocked TCP connection can appear established until application data is attempted. E2B itself says a domain allowlist is a routing control, not a strict security boundary on shared infrastructure. + +## Operations and observability + +Runtime services emit OpenTelemetry. Orchestrators publish lifecycle events and host statistics; ClickHouse stores metrics/events and optionally logs. Production architecture supports centralized observability, while Embed intentionally uses local Vector → ClickHouse with a fixed seven-day retention and no Loki or LaunchDarkly dependency ([architecture](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/ARCHITECTURE.md), [Embed reference](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/docs/REFERENCE.md)). + +A 2025 [self-hosting report](https://github.com/e2b-dev/runtime/issues/1421) documented orphaned Firecracker processes/veth state after an orchestrator crash, cache-locality concerns, and difficult Kubernetes resource accounting. At the time, E2B recommended replacing the node. Current runtime code adds startup reclaim and single-instance locking, and Embed's launcher sweeps cgroups before restart. The history is still useful: privileged node reconciliation, cache placement, and capacity accounting are production concerns, not incidental packaging details. + +## Patterns supported by the evidence + +Patterns that can be evaluated independently of an E2B adoption decision: + +1. **Separate lifecycle control from sandbox traffic.** Keep placement, quotas, and durable state in the control plane; route high-volume process/file/port traffic directly through a data-plane proxy. +2. **Hide host mechanics behind a node-local orchestrator.** The gateway should not understand NBD, Firecracker, cgroups, namespaces, or snapshot files. +3. **Use an explicit lifecycle state machine.** Running, pausing, paused, resuming, snapshotting, forking, and killed states need durable identities and well-defined terminal behavior. +4. **Separate reproducible templates from live checkpoints.** A build recipe and a memory-preserving snapshot answer different provenance and recovery questions. +5. **Make cold-start optimizations content-addressed.** Hash steps and inputs, share immutable layers, prefetch likely pages, and use COW overlays rather than copying a workspace/rootfs on every start. +6. **Keep the guest API narrow.** Authenticated process/PTY and filesystem operations are a useful runtime primitive; private init/upgrade/checkpoint routes should not share the public proxy path. +7. **Treat networking as a first-class per-run contract.** Record ingress authentication and egress policy with the run. E2B's documented domain-filter limits show why policy claims must match enforcement layers. +8. **Design crash reconciliation with the runtime.** Startup reclaim, idempotent teardown, single-instance locks, and explicit cache/storage recovery belong in the node contract before multi-node production use. +9. **Expose runtime telemetry without making it the agent protocol.** Lifecycle events, resource metrics, logs, and trace context should correlate with a gateway run ID, while normalized agent events remain above the sandbox provider. +10. **Pin a provider compatibility matrix.** SDK, API, guest daemon, kernel, Firecracker, and template versions change on different cadences; deployment provenance needs immutable versions and checksums. + +Patterns that should remain above any E2B provider adapter are harness/provider routing, credentials and source authorization, workspace provenance, event normalization, completion taxonomy, result/artifact manifests, retries/idempotency, and cross-provider conformance. + +## Licensing and current activity + +The runtime and dashboard repositories use Apache-2.0 ([runtime license](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/LICENSE), [dashboard license](https://github.com/e2b-dev/dashboard/blob/main/LICENSE)). The JavaScript and Python SDK package licenses are MIT ([JS SDK](https://github.com/e2b-dev/E2B/blob/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b/packages/js-sdk/LICENSE)). Runtime `main` had changes on 2026-09-22, runtime release [`2026.30`](https://github.com/e2b-dev/runtime/releases/tag/2026.30) was published 2026-09-10, and SDK release [`e2b@2.51.0`](https://github.com/e2b-dev/E2B/releases/tag/e2b%402.51.0) was published 2026-09-18. Embed itself is recent and still explicitly framed as evaluation packaging in source. + +## Primary sources + +- [Runtime repository and architecture](https://github.com/e2b-dev/runtime/tree/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8) — source dated 2026-09-22. +- [SDK repository](https://github.com/e2b-dev/E2B/tree/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b) — source dated 2026-09-18. +- [E2B Embed](https://github.com/e2b-dev/runtime/tree/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed) — source dated 2026-09-22. +- [Official E2B documentation index](https://docs.e2b.dev/llms.txt) — accessed 2026-09-22. +- [Enterprise deployment options](https://e2b.dev/enterprise), [BYOC](https://docs.e2b.dev/byoc.md), and [security](https://e2b.dev/security) — accessed 2026-09-22. +- [Runtime release 2026.30](https://github.com/e2b-dev/runtime/releases/tag/2026.30) — published 2026-09-10. +- [SDK release 2.51.0](https://github.com/e2b-dev/E2B/releases/tag/e2b%402.51.0) — published 2026-09-18. + +## AllAgents comparison and decision + +### Verdict + +**Reject E2B as a replacement for the planned UHP/HarnessRouter gateway. Trial it later only as a stronger sandbox runtime beneath the runner if hostile-code isolation becomes a product requirement.** + +The current AllAgents decision is accepted but not implemented: [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) selects UHP `2026-09-12` through a pinned HarnessRouter CE fork, and the [implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md) assigns the wire protocol, caller authentication, normalized streaming, cancellation, idempotency, conversation continuity, harness execution, usage, and artifacts to HarnessRouter. AllAgents owns deterministic Git/OCI materialization and source provenance. Repository inspection found no gateway, fork, materializer, or deployment implementation yet. + +E2B does not implement that contract. It creates an isolated machine and exposes low-level process, filesystem, network, and lifecycle APIs. Its official Codex and Pi integrations leave command construction, harness credentials, event parsing, continuation, and result extraction in caller code. Replacing HarnessRouter with E2B would therefore recreate the custom gateway, session, event-normalization, harness-adapter, and artifact layers that ADR 0002 rejected. + +### Does E2B do the same thing? + +No. The systems overlap at the execution-workspace layer but own different abstractions. + +| Concern | Planned AllAgents gateway | E2B | +| --- | --- | --- | +| Northbound contract | UHP request, ordered events, cancellation, idempotency, continuation, files, usage, artifacts, and normalized errors | Sandbox REST API plus process/filesystem APIs; no agent-neutral request/event/result protocol | +| Harness execution | HarnessRouter selects and runs configured Codex or Pi targets | Caller starts an agent-specific command inside a sandbox and interprets its output | +| Session identity | `previous_response_id` binds conversation, writable workspace, harness target, auth binding, and provenance | Sandbox ID binds machine state; agent thread/session identity remains harness- and caller-specific | +| Workspace acquisition | Server-authoritative logical source catalog; exact Git commits or immutable OCI digests; pre-agent checkpoint; returned provenance | Caller uploads files, clones Git, or starts from a template/snapshot; no agent-run source-provenance contract | +| Isolation | Private per-session UID/workspace; explicitly not a hostile-code sandbox | One Firecracker microVM and guest kernel per sandbox | +| Credentials | Separate caller, source, and provider trust domains; native OAuth owner-trust mode or explicit brokered proxy | Core sandbox auth plus caller-supplied agent/Git credentials; managed egress secrets and workload identity are not fully present in standard Embed | +| Results | UHP output, usage, artifacts, produced-file collection, and identical provenance across stream/retrieval/replay paths | Guest files, command streams, VM snapshots, and templates; the caller defines an agent result or artifact manifest | +| Deployment | Planned pinned private HarnessRouter image with durable sessions and a narrow AllAgents hook | Multi-service KVM stack with API, proxies, orchestrator, guest daemon, three datastores, and template/snapshot storage | + +E2B could occupy the runner's future sandbox-runtime slot. HarnessRouter and the AllAgents materializer would still remain above it. + +### Is it completely self-hosted? + +The answer depends on the operating standard: + +- **Yes for a functional single-node deployment.** Apache-2.0 E2B Embed runs the API, dashboard, proxies, PostgreSQL, Redis, ClickHouse, Vector, template builder, Firecracker orchestrator, storage, and sandboxes on an operator-owned KVM host. It generates its own team API key and stores its runtime data locally. +- **No for a turnkey production-equivalent distribution.** The source guides call every Embed shape a “single-machine evaluation package, not a deployment pattern.” Compose, the one-node Kubernetes StatefulSet, and the one-instance GCP module do not establish HA, multi-node recovery, or production scaling. +- **No for managed-feature parity.** Standard Embed does not configure every managed backend. Current gaps include the separate secret store, volume-content service/token path, managed SOCKS5 egress, and credential delivery for workload identity. +- **No for turnkey air-gapped installation.** Installation fetches pinned public images and binaries from Docker Hub, Google Artifact Registry/Storage, and GitHub; no current public offline-mirroring guide was found. +- **BYOC is not self-hosting.** E2B provisions, monitors, and operates BYOC in the customer's AWS or GCP account, while E2B Cloud remains part of its management plane. E2B's fully private production option is listed as in development. + +The [enterprise page](https://e2b.dev/enterprise) markets Embed as an available self-hosting pattern. The [Embed source guide](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/README.md) sets the narrower operational boundary. For architecture decisions, use the source guide's single-node/evaluation limit. + +### No E2B-derived changes for version one + +Version one should adopt none of E2B's runtime patterns. The accepted gateway already has a coherent private owner-trust scope, and E2B addresses requirements that version one explicitly excludes: hostile-code isolation, public multi-tenancy, VM suspension and forking, multi-node placement, and microVM startup optimization. + +Do not add an E2B dependency, a sandbox-provider interface, a guest daemon, egress-policy machinery, runtime telemetry infrastructure, or any other future-runtime seam to version one. The runner boundary is already a sufficient future integration point. Building an abstraction before a second runtime and a measured requirement exist would increase the initial implementation and verification burden without satisfying an acceptance criterion. + +Existing requirements for private runner routes, credential containment, crash-consistent materialization, immutable source identity, release pinning, and provenance remain necessary on their own merits. They are not E2B adoption. + +If a reconsideration trigger is reached later, E2B provides a useful checklist for that separate design: place stronger isolation beneath the runner; separate public and private control routes; bind egress and ingress policy to the execution identity; keep environment identity distinct from source provenance; keep long-lived credentials outside the guest; reclaim orphaned runtime resources after crashes; correlate runtime telemetry with UHP identifiers without creating another agent protocol; and pin the sandbox SDK, API, guest daemon, kernel, and runtime as one tested compatibility set. + +### Patterns to defer or reject + +- **Defer Firecracker, memory snapshots, forking, lazy restore, COW root filesystems, placement, and multi-node scheduling.** They solve hostile multi-tenancy, recovery, or startup-cost problems that v1 does not claim. Adopt them only after a requirement or measurement justifies their operational weight. +- **Reject E2B's API as the AllAgents northbound contract.** It would discard UHP conformance and make callers own harness-specific behavior. +- **Reject caller-side repository acquisition and inline long-lived credentials.** E2B's convenience examples conflict with the server-authoritative source catalog, hermetic materializer, and credential-containment requirements. +- **Reject open internet by default for a future sandboxed mode.** Outbound access should be an explicit target policy. +- **Reject live VM snapshots as source provenance.** They are useful recovery artifacts, not reproducible evidence of which repositories and commits an agent received. + +### Reconsideration conditions + +Evaluate E2B as a runner backend when AllAgents must execute mutually untrusted tenant code, support public multi-tenancy, preserve in-flight processes across suspension, fork live workspaces, or meet measured sandbox-start targets that process/UID isolation cannot satisfy. The trial must keep UHP and AllAgents provenance above E2B, pin an SDK/runtime compatibility pair, prove private-route containment and default-deny egress, and exercise crash recovery on the exact self-hosted deployment shape. + +E2B is not the ADR's “second independent UHP implementation” reconsideration trigger because it does not implement UHP. It becomes relevant when the isolation requirement changes, not because it duplicates the current gateway. From 1c8e9c787787a2a881ebd344c2a36b33694f27e7 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Tue, 22 Sep 2026 21:13:38 +1000 Subject: [PATCH 19/44] docs(architecture): close execution handoff gaps --- .../0002-adopt-uhp-through-harnessrouter.md | 77 +- ...0837-feat-coding-execution-gateway-plan.md | 703 ++++++++++++++---- 2 files changed, 610 insertions(+), 170 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 56c695a7..a0127ad4 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -13,7 +13,10 @@ The selected harness owns provider authentication. Codex signs in through `codex login`; Pi signs in through its `/login` flow for the configured provider. Those native OAuth sessions are the default and require no provider-route API key. An explicitly configured API-key-authenticated proxy is a last-resort -route, never an automatic fallback from failed OAuth. +route, never an automatic fallback from failed OAuth. Optional describes +deployment configuration, not release scope: version one implements and verifies +the route so operators that reject the native owner-trust boundary have a +supported alternative. HarnessRouter owns caller authentication, UHP request and response semantics, streaming, cancellation, idempotency, session continuity, per-session @@ -182,9 +185,20 @@ temporary file, file and parent-directory `fsync`, atomic rename, and validation A crash after the provider rotates credentials but before local commit may leave the profile stale; restart marks it `repair-required` when validation fails and requires native login again. It never switches profiles or activates the proxy. -Version one holds a per-profile lock for every refresh-capable turn and every -login, logout, or repair operation. A second turn for that profile waits or -fails before launch. +Version one supports exactly one active refresh-capable turn per native profile +and holds that profile lock for every turn and every login, logout, or repair +operation. Admission first atomically claims the UHP `Idempotency-Key`; concurrent +same-key requests share one admission/result, and same-session overlap returns +stock `session_busy`. Only a genuinely new cross-session turn tries the +zero-waiter profile lock. Collision returns HTTP 503 `harness_unavailable` with +`detail.reason: "allagents_auth_profile_busy"` before response allocation, +runner work, or materialization. + +The runner turn supervisor persists the admission record and owns the profile +lock through descendant termination, terminal-state acknowledgement, and refresh +commit. Gateway-only failure cannot release it. Runner failure leaves a durable +fence; startup blocks readiness and admission until descendant and profile +reconciliation. Operators provision distinct profiles for parallel capacity. The generic fork layer does not understand the AllAgents descriptor. It enforces only the configured key, JSON/size bounds, immutable first-turn binding, hook @@ -252,10 +266,20 @@ rejects option-like or refspec-shaped values, resolves advertised refs to full commits before agent execution, fetches by verified object ID, and records those commits in provenance. -Snapshot mode accepts only a configured OCI repository plus immutable manifest -and workspace-manifest digests. It verifies manifest, config, layer sizes and -digests, applies OCI whiteouts, validates the resulting declared workspace -layout, and records the ordered layer digests. +Snapshot mode accepts only a configured OCI repository plus immutable image- +manifest and workspace-manifest digests. It verifies the image manifest, +canonical workspace-manifest bytes, layer sizes and digests, applies OCI +whiteouts, validates the resulting declared workspace layout against the +manifest, and records the ordered layer digests. + +Both source modes produce the same versioned canonical workspace manifest. Its +RFC 8785 bytes enumerate every directory, regular file, and symbolic link in +logical path order with normalized mode, size, content digest, or link target as +applicable. Git mode computes it from completed staging. OCI mode carries the +same bytes in the configured workspace-manifest blob and must reproduce them +after applying the layers. The runner receives the manifest through a private +bounded result root, verifies its digest and the staged tree independently, and +never publishes the manifest as source content. Source credentials are selected server-side from an owner-only secret mount or credential-store handle available to the runner, not from the long-lived service @@ -291,6 +315,11 @@ The deployment uses a pinned custom HarnessRouter image containing: - version-locked OS packages and Git/OCI source-acquisition tools; and - pinned HarnessRouter-supported Codex and Pi versions. +The runtime grants only the runner a delegated cgroup v2 subtree and applies an +`on-failure` restart policy. Readiness stays false unless that delegation is +usable and startup has removed or quarantined every orphaned materializer +cgroup. + AllAgents publishes the `linux/amd64` release image as the public package `ghcr.io/allagentsdev/harnessrouter`. Version and commit tags are mutable discovery labels; deployment configuration pins the published manifest digest. @@ -316,8 +345,10 @@ broker, and endpoint compatibility for `proxyApiKey`. A missing, expired, revoked, or unrefreshable OAuth profile disables that target; it does not select another profile or fall through to an API key. -`proxyApiKey` is an optional, explicit last-resort mode. HarnessRouter keeps the -long-lived proxy client key in the gateway. It gives the harness a +`proxyApiKey` is optional to configure but its implementation and verification +remain required version-one scope. It is an explicit last-resort mode. +HarnessRouter keeps the long-lived proxy client key in the gateway. It gives the +harness a non-refreshable broker credential bound to one proxy audience, harness target, model allowlist, response/turn ID, and the UHP deadline plus minimal clock skew. The token may authorize the bounded provider calls, compaction, and retries @@ -333,11 +364,27 @@ automatically. - **Invalid extension:** the AllAgents hook rejects it before acquisition. - **Extension on a continuation:** reject without changing session state. - **Unknown logical source or working directory:** fail before network access. +- **Busy auth profile:** after atomic idempotency replay and stock `session_busy` + precedence, a genuinely new cross-session turn fails immediately with HTTP 503 + `harness_unavailable` and + `detail.reason: "allagents_auth_profile_busy"` before response allocation; + never queue it on the profile lock. - **Source authentication or acquisition failure:** remove partial workspace - state, return a stable materializer failure, and start no agent or provider + state, return the plan's cataloged UHP failure, and start no agent or provider fallback. -- **Materializer timeout or crash:** terminate the hook, remove partial source - state, return failure, and start no agent. +- **Materializer timeout, cancellation, malformed result, crash, or live + descendant after parent exit:** create a runner-owned cgroup v2 leaf and start + the child inside it atomically with `clone3(CLONE_INTO_CGROUP)` or a stopped, + secret-free pre-exec move-and-verify handshake. The child cannot escape or + administer the subtree; process groups and post-exec migration are + insufficient. On every outcome, use `cgroup.kill` when membership remains and + wait for `cgroup.events` to report `populated 0` before any terminal response + or event, private-manifest read, publication, secret release, or cleanup. A + completed parent with a live descendant returns the containment failure even + when forced kill succeeds. An unquiescent leaf enters internal non-terminal + `containment_pending`; the runner exits for required restart, and GET/stream + stay non-terminal until startup proves the old boundary empty. Only then expose + failed containment or the UHP-mandated `cancelled`/`incomplete` status. - **Agent cancellation or timeout:** use HarnessRouter's UHP lifecycle and cancellation behavior. - **HarnessRouter restart:** preserve completed state from the durable volume; @@ -346,6 +393,10 @@ automatically. switching OAuth profiles or activating the proxy/API-key route. - **Provider execution failure:** return HarnessRouter's normalized UHP failure without source fallback or credential material in public output. +- **Public error mapping:** use UHP request errors before response allocation and + terminal failed responses afterward. New codes carry the `allagents_` vendor + prefix. Promptfoo maps every non-success to a coded error, never successful + empty output or an automatic retry. Failures report only verified provenance. Partial acquisition never appears as a complete workspace identity. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 552f7fa4..e4cb20e5 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,7 +1,7 @@ --- title: "UHP Coding-Agent Execution through HarnessRouter - Plan" date: 2026-09-18 -updated: 2026-09-21 +updated: 2026-09-22 type: feat artifact_contract: ce-unified-plan/v1 artifact_readiness: implementation-ready @@ -26,7 +26,9 @@ execution: code durable harness-native OAuth state. Implement Git/OCI semantics in a separate AllAgents executable. Codex authenticates through `codex login`; Pi authenticates through its `/login` flow for the selected provider. An - API-key-authenticated proxy is an explicit last-resort target mode. + API-key-authenticated proxy is an explicit last-resort target mode. Optional + describes deployment configuration, not release scope: version one implements + and verifies it for operators that reject the native owner-trust boundary. - **Authority:** [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) owns the protocol, fork, trust, workspace, harness-authentication, and provider-routing decisions. UHP `2026-09-12` and HarnessRouter's conformance @@ -112,10 +114,10 @@ caller responsible for acquisition. The temporary fork closes those seams. and session identity, treats the configured workspace metadata value as bounded opaque JSON, drives materialization before provider fallback, and returns hook metadata on every response path. -- **A3. AllAgents materializer:** A non-network executable invoked before the - first turn. It validates the AllAgents descriptor, reads the mounted project - workspace configuration, acquires Git or OCI sources into staging, validates - the tree, and returns provenance. +- **A3. AllAgents materializer:** A subprocess executable that exposes no + listening service, invoked before the first turn. It validates the AllAgents + descriptor, reads the mounted project workspace configuration, acquires Git or + OCI sources into staging, validates the tree, and returns provenance. - **A4. HarnessRouter runner:** Owns the per-session operating-system identity, publication, checkpoints, nested-repository collection, input files, safe nested cwd, selected Codex/Pi process, session conversation state, and @@ -155,8 +157,10 @@ caller responsible for acquisition. The temporary fork closes those seams. the UHP caller only. Codex and Pi use their own login, token storage, refresh, and provider request path; native mode has no provider-route API key. - **Make proxy auth explicit.** `proxyApiKey` is a last-resort target mode for a - compatibility or stronger-isolation requirement. OAuth failure never activates - it, and a session never changes its persisted authentication binding. + compatibility or stronger-isolation requirement. It is optional to configure + but remains required version-one implementation and verification scope. OAuth + failure never activates it, and a session never changes its persisted + authentication binding. - **Preserve stock UHP requests.** Requests without the configured metadata key behave exactly as upstream. - **Use a custom HarnessRouter image.** The image combines a pinned HarnessRouter @@ -212,11 +216,38 @@ caller responsible for acquisition. The temporary fork closes those seams. checkpoints, produced-file records, passive logs/traces, materializer input, or response metadata. Native mode sets explicit owner trust because the harness and tool subprocesses sharing its operating-system identity may read - or emit that credential. Version one holds a per-profile lock for every - refresh-capable turn and every login, logout, or repair operation; a second - turn for that profile waits or fails before launch. - Preserve HarnessRouter streaming, cancellation, idempotency, files, artifacts, - completed-session persistence, and per-session workspace/UID isolation. + or emit that credential. In `nativeOAuth`, version one supports exactly one + active refresh-capable turn per profile and holds that profile lock for every + turn and every login, logout, or repair operation. Admission preserves UHP + precedence with an atomic `Idempotency-Key` claim around lookup and admission. + The single claim owner proceeds; simultaneous same-key arrivals wait on that + claim and receive the owner's result without a second profile-lock attempt. If + the owner fails before response allocation, the gateway publishes that same + request error to current waiters and removes the claim so a later retry can try + again. A new turn in an already-active session returns `session_busy`; only + then does a genuinely new executable turn try the native profile lock. + Cross-session collision returns HTTP 503 `harness_unavailable` with + `detail.reason: "allagents_auth_profile_busy"` before response allocation, + runner work, or materialization. No per-profile waiter queue exists; the UHP + idempotency claim wait is part of one logical request, not such a queue. + + Admission is a private runner operation: the runner turn supervisor persists an + active-profile admission record, takes the operating-system advisory lock, and + returns an opaque admission token before the gateway allocates a response. The + runner—not the gateway—owns that lock through descendant termination and + refresh commit. A gateway-only crash therefore leaves the lock held; restart + reconciles the token and active runner before admitting another turn. The + runner releases only after the gateway acknowledges durable terminal + response/state and the runner has either durably committed native refresh state + or marked the profile `repair-required`. If the runner process dies, the OS + releases the lock, but its durable admission record keeps readiness/admission + closed until startup proves all descendant boundaries empty and validates or + repairs the profile. Ordinary and administrative paths use one `finally` + release/ack protocol. `proxyApiKey` uses no native profile lock. + Operators provision distinct native profiles when they require parallel turn + capacity. Preserve HarnessRouter streaming, + cancellation, idempotency, files, artifacts, completed-session persistence, + and per-session workspace/UID isolation. #### Workspace extension and hook @@ -243,28 +274,66 @@ caller responsible for acquisition. The temporary fork closes those seams. after fresh-session hydrate and before `/turn`. Pass at most 128 KiB on stdin, accept at most 1 MiB on stdout and 64 KiB on stderr, and use the smaller of 900 seconds or the remaining UHP deadline. The generic request contains the - opaque metadata value, session workspace and fixed sibling staging roots, - project configuration root, and deadline. Source values come from an - owner-only runner secret mount or credential-store handle, never the + opaque metadata value, session workspace, fixed sibling staging and private + result roots, project configuration root, and deadline. Source values come + from an owner-only runner secret mount or credential-store handle, never the gateway/runner base environment. The runner resolves only the selected handle - and constructs the allowlisted materializer child environment. It refuses - agent launch if a configured secret name or value appears in the service or - agent environment. The typed result is `completed` with effective relative cwd - and bounded public metadata or `failed` with stable code, safe message, and - retryability. A materializer failure terminalizes the UHP response and never - enters provider fallback. + and constructs the allowlisted materializer child environment. Startup + preflight blocks readiness if a configured secret name or value appears in the + service or agent environment. The per-request recheck, after allocation but + before child or agent launch, returns failed + `allagents_secret_boundary_violation` on the same condition. Before injecting + secrets, the runner creates a + per-materialization cgroup v2 leaf under a runner-owned delegated subtree and + starts the child inside it atomically with `clone3(CLONE_INTO_CGROUP)`. Where + that primitive is unavailable, it forks a child with no secret material and + holds it at a pre-exec handshake; the parent moves it to the leaf, verifies the + exact membership through `/proc//cgroup`, then delivers the one-shot secret + bundle and releases the child to construct its environment and `exec`. The + stopped child cannot execute user code or fork before verified membership. The + child cannot write the parent `cgroup.procs` or administer the subtree; a + process group or post-exec PID migration is insufficient. + On every outcome, including a parent that returns `completed`, the runner + closes the hook streams, checks + `cgroup.events`, uses `cgroup.kill` when membership remains, and waits a bounded + interval for `populated 0`. It does not accept success, read the private + manifest, validate or publish staging, release the secret environment, or + remove roots before the cgroup is empty. Membership remaining after a parent + reports `completed` is itself failed + `allagents_workspace_containment_breach`, even when `cgroup.kill` subsequently + reaches `populated 0`; that recovered violation does not require runner exit. + If the cgroup cannot become empty, the runner reports an internal + `containment_pending` reason. The gateway CASes only the internal session state + to `containment_pending`/non-resumable and acknowledges that receipt; it emits + no terminal stream event and GET continues to show non-terminal `in_progress`. + The runner exits nonzero after that acknowledgement or a bounded + acknowledgement deadline. The required `on-failure` restart policy destroys + the old container boundary; startup keeps readiness false, removes or + quarantines orphaned cgroups, and reports `containment_reconciled` only after + the old boundary is proven empty. Only then may the gateway terminalize and + expose the response: client cancellation becomes `cancelled`, declared-budget + exhaustion becomes `incomplete`, and every other case becomes failed + `allagents_workspace_containment_breach`. No terminal response/state, terminal + event, result read, publication, secret release, or cleanup becomes observable + before that proof. The typed hook result is `completed` with effective relative + cwd, a private workspace-manifest reference and digest, and bounded public + metadata, or `failed` with a cataloged code, safe message, and retryability. + Materializer failure never enters provider fallback. - **R8.** Persist a CAS-protected session materialization state: - `unbound -> materializing -> ready` or `failed`, plus the resolved harness - target, auth mode, native-profile or proxy-connection identity, and canonical - binding-config digest. The runner validates successful staging, publishes it - with a recoverable same-filesystem rename protocol, writes a + `unbound -> materializing -> ready`, `failed`, or internal + `containment_pending -> failed`, plus the resolved harness target, auth mode, + native-profile or proxy-connection identity, and canonical + binding-config digest. The runner validates the private workspace manifest and + successful staging independently, publishes staging with a recoverable + same-filesystem rename protocol, removes the private result root, writes a descriptor/provenance marker, initializes the HarnessRouter root checkpoint and nested-repository collection baselines, and returns `published` with the checkpoint digest. Before provider dispatch, the gateway stores that digest - and public metadata and CASes the session to `ready`. On restart in - `materializing`, reconcile to `ready` only when the workspace marker and - durable checkpoint match the bound descriptor; otherwise mark the session - non-resumable, remove or quarantine the workspace, and never replay acquisition. + and public metadata and CASes the session to `ready`. + On restart in `materializing`, reconcile to `ready` only when the workspace + marker and durable checkpoint match the bound descriptor; otherwise mark the + session non-resumable, remove or quarantine the workspace, and never replay + acquisition. Every continuation resolves the persisted auth binding by identity and digest; if unavailable or changed, it fails before runner work instead of selecting a replacement. Provider fallback sees `ready` state only and cannot invoke the @@ -310,19 +379,30 @@ caller responsible for acquisition. The temporary fork closes those seams. submodule recursion, Git LFS hydration, or configured clean/smudge filters. Preserve each repository's `.git` directory for the coding agent. Failure never falls through to snapshot mode or another credential identity. + Limit one materialization to 128 repositories, 500,000 filesystem entries, and + 32 GiB across the staged workspace. A count or byte violation returns failed + `allagents_workspace_limit_exceeded`. Exhausting the declared UHP time budget + returns `incomplete` with `error: null`; only an unexpected execution timeout + before that budget returns failed `timeout`. Every outcome proves the cgroup + empty before removing staging and starts no provider. - **R11.** Snapshot mode constructs a server-side immutable OCI reference from the selected snapshot's configured repository and caller-provided digest. Accept only a direct OCI image manifest with at most 64 distributable - tar/gzip/zstd layers and the entry's configured workspace-manifest media type; - redirects may not change registry authority. Verify manifest, config, layer - size and digest before use; apply OCI whiteouts; limit manifest and config to - 4 MiB each, total compressed layers to 8 GiB, expanded bytes to 32 GiB, - entries to 500,000, one regular file to 4 GiB, paths to 4096 UTF-8 bytes and - 128 components, and one PAX/extended header to 1 MiB. Reject devices, sockets, - traversal, escaping links, sparse files, unknown or foreign layers, mutable - tags, and undeclared output. Validate the final logical repository catalog and - workspace-manifest digest in staging. Snapshot repository roots need not - contain `.git`; after publication the runner creates private collection + tar/gzip/zstd layers. Its config descriptor must use the entry's configured + workspace-manifest media type and address canonical workspace-manifest bytes; + redirects may not change registry authority. Verify the image manifest, + workspace manifest, layer size, and digest before use; apply OCI whiteouts; + limit the image manifest to 4 MiB, the workspace-manifest blob to 128 MiB, + its `repositories` array to 128 items, total compressed layers to 8 GiB, + expanded bytes to 32 GiB, entries to 500,000, one regular file to 4 GiB, paths + to 4096 UTF-8 bytes and 128 components, and one PAX/extended header to 1 MiB. + The runner independently rejects a 129th repository root even when the archive + and fetched manifest otherwise agree. Reject devices, + sockets, traversal, escaping links, sparse files, unknown or foreign layers, + mutable tags, and undeclared output. Recompute the canonical workspace + manifest from staging and require it to match both the fetched manifest bytes + and caller-provided digest. Snapshot repository roots need not contain `.git`; + after publication the runner creates private collection baselines from the verified trees so later produced-file reporting remains truthful. - **R12.** Extend HarnessRouter's response translator and stored-response paths @@ -330,8 +410,9 @@ caller responsible for acquisition. The temporary fork closes those seams. idempotent replay return the same bounded `response.metadata["allagents.workspace"]`. It contains extension version, effective descriptor digest, logical cwd, source mode, completeness, resolved - commits or OCI manifest/config/layer digests, and workspace-manifest digest. - It never contains origins, physical paths, credentials, or unverified facts. + commits or OCI image/workspace-manifest/layer digests, and the canonical + workspace-manifest digest. It never contains origins, physical paths, + credentials, or unverified facts. - **R13.** The materializer resolves `${ENV_VAR}` references from its allowlisted child environment, uses hermetic Git/registry configuration, removes temporary auth files before returning, and emits no secret. Prove with a deliberately @@ -375,9 +456,11 @@ caller responsible for acquisition. The temporary fork closes those seams. - **R15.** AI Evals owns its Promptfoo provider. It sends the UHP request directly to HarnessRouter, maps Promptfoo variables to the closed extension, and maps terminal output, usage, artifacts, provenance, and failures to - `ProviderResponse`. Multi-turn cases retain the prior response ID and send it - as `previous_response_id`. AllAgents documents the contract and examples but - does not depend on Promptfoo at runtime. + `ProviderResponse`. Every non-success follows the Failure Contract's exact + status/error/retryability/metadata mapping; none becomes empty success or an + automatic retry. Multi-turn cases retain the prior response ID and send it as + `previous_response_id`. AllAgents documents the contract and examples but does + not depend on Promptfoo at runtime. ### Key Flows @@ -400,25 +483,38 @@ caller responsible for acquisition. The temporary fork closes those seams. 4. Start the attestation-verified, digest-pinned custom HarnessRouter image with durable session data, durable harness auth roots, a private listener, HarnessRouter caller key, materializer command, project configuration, - owner-only source-secret mount/credential-store handle, and the configured - native or proxy trust mode. + owner-only source-secret mount/credential-store handle, the configured native + or proxy trust mode, a runner-owned delegated cgroup v2 subtree, and required + `on-failure` restart policy. Startup sweeps or quarantines orphaned + materializer cgroups and withholds readiness unless delegation is usable and + every prior boundary is empty. 5. From the exact container network, verify each advertised harness ID and model allowlist, safe roots, materializer version, selected auth binding, and a live turn. Native targets exercise login status, atomic local refresh persistence, - stale-profile repair, same-binding continuation, and serialized overlapping - turns. An explicit proxy deployment exercises schema/TLS/model-map/endpoint - compatibility plus bounded in-turn use and wrong-scope/expired/revoked - rejection. Any required - preflight or auth failure prevents readiness. + stale-profile repair, same-binding continuation, and fail-fast overlapping + turns. An explicit proxy deployment exercises + schema/TLS/model-map/endpoint compatibility plus bounded in-turn use and + wrong-scope/expired/revoked rejection. Any required preflight, containment, + or auth failure prevents readiness. #### F2. Execute the first repository-backed turn 1. Promptfoo sends one authenticated UHP request with `model`, stock `metadata.harness_id`, idempotency input, and the AllAgents workspace object. -2. HarnessRouter validates UHP plus generic metadata bounds, resolves and - persists the selected harness target and canonical auth-binding identity/ - config digest, creates the response/session, CASes materialization from - `unbound` to `materializing`, and hydrates a fresh session workspace. +2. HarnessRouter validates UHP plus generic metadata bounds and atomically claims + the `Idempotency-Key` around lookup and admission. One owner proceeds; + simultaneous same-key arrivals wait on the claim and receive the owner's + result without another admission attempt. For a genuinely new turn, the owner + resolves the selected target and canonical auth-binding identity/config + digest. In `nativeOAuth` it asks the runner supervisor to persist an admission + record and acquire the profile's zero-waiter try-lock; a cross-session + collision publishes the cataloged HTTP 503 to current same-key waiters, then + removes the pre-allocation claim so a later retry can try again. The runner + returns an opaque admission token and retains the lock. `proxyApiKey` acquires + no native profile lock. Once admitted, HarnessRouter binds the idempotency + claim to the response, persists the binding, creates the response/session, + CASes materialization from `unbound` to `materializing`, and hydrates a fresh + session workspace. 3. Before provider selection, the gateway calls runner `/materialize`. The AllAgents child validates the descriptor and catalog, resolves exact commits and source credentials, writes and validates staging, removes credential @@ -436,22 +532,34 @@ caller responsible for acquisition. The temporary fork closes those seams. 6. Normal UHP events and every stored/retrieved terminal response include the same namespaced provenance. Produced-file collection walks the HarnessRouter root plus each declared nested repository without reporting initial source - files as agent output. + files as agent output. A single native-mode `finally` path covers completion, + cancellation, and every post-admission failure: it persists terminal state, + durably commits refresh state or marks the profile `repair-required`, and + acknowledges the admission token. Only then may the runner release the + profile lock. #### F3. Continue the session 1. The caller sends `previous_response_id` and omits `metadata["allagents.workspace"]`. -2. HarnessRouter resolves its current session state and writable workspace, - requires materialization `ready`, and resolves the persisted auth-binding - identity and config digest. An unavailable or changed binding, - extension-bearing continuation, concurrent active turn, or non-resumable - session fails before runner work. +2. HarnessRouter atomically claims the `Idempotency-Key` around lookup and + admission. A simultaneous duplicate waits for or returns the owner's result + without another admission attempt. For a new request it resolves the current + session state and writable workspace, requires materialization `ready`, and + resolves the persisted auth-binding identity and config digest. An + extension-bearing continuation, changed or unavailable binding, or + non-resumable session fails before runner work. Same-session concurrency + returns stock `session_busy` before profile admission. Only then does + `nativeOAuth` acquire the runner-owned zero-waiter profile lock; cross-session + saturation returns cataloged `harness_unavailable`. A pre-allocation error is + delivered to claim waiters before claim removal. Proxy mode acquires no native + profile lock. 3. HarnessRouter follows its stock predecessor/session semantics. Native mode projects the persisted profile; proxy mode mints a new scoped turn token for the persisted connection. It resumes the native conversation and returns pinned provenance plus new output, usage, and artifacts without changing auth - mode or binding. + mode or binding. The native `finally` boundary releases the profile lock on + every terminal outcome. #### F4. Execute an OCI-backed first turn @@ -466,8 +574,22 @@ caller responsible for acquisition. The temporary fork closes those seams. #### F5. Cancel, fail, or restart -1. Materializer cancellation or deadline terminates the child and removes - staging. The session becomes `failed` and non-resumable; no provider starts. +1. On every materializer outcome, including a parent that returns `completed`, + the runner proves the cgroup empty before exposing any terminal result, + reading the manifest, publishing, releasing secrets, or cleanup. Membership + after a parent + reports `completed` produces failed + `allagents_workspace_containment_breach` even if `cgroup.kill` empties the + leaf; the runner may remain ready after cleanup in that recovered case. If + `populated 0` cannot be proved, the runner reports internal + `containment_pending`; the gateway persists and acknowledges only that + non-terminal state, withholding terminal GET/stream visibility. The runner + exits nonzero after acknowledgement or its bounded deadline. Restart destroys + the old boundary and keeps readiness false until it proves the old cgroup + empty, then reports reconciliation. Only after that proof does the gateway + expose `cancelled` for client cancellation, `incomplete` for declared-budget + stop, or failed `allagents_workspace_containment_breach` otherwise. No provider + starts. 2. Agent cancellation and deadline use HarnessRouter's normal UHP lifecycle. 3. Startup reconciles a `materializing` session to `ready` only when the bound descriptor, published workspace marker, and durable checkpoint all match. @@ -476,6 +598,12 @@ caller responsible for acquisition. The temporary fork closes those seams. rehydrates from its durable checkpoint. 4. Whole-container termination does not preserve the in-flight agent process. Interrupted agent turns fail according to HarnessRouter behavior. +5. Every failure after native admission persists terminal state and valid or + `repair-required` refresh state before acknowledging the runner's admission + token. A gateway-only crash leaves the runner-held lock intact until restart + reconciliation; a runner crash leaves a durable admission record that blocks + readiness until descendant/profile reconciliation. Only then can a new turn + acquire the profile; a failed request is never retained as a waiter. ### Acceptance Examples @@ -489,10 +617,14 @@ caller responsible for acquisition. The temporary fork closes those seams. - **AE3.** Repository mode resolves configured branch/default/HEAD refs to full commits, prepares every execution-eligible repository, preserves nested Git history, starts in a validated nested cwd, and returns path-free provenance. -- **AE4.** HarnessRouter rejects non-object/oversized metadata and an extension - on continuation before the hook. The hook rejects unknown logical names, - caller-provided URLs, absolute/traversal paths, commands, environment fields, - credentials, originless/local repositories, and duplicate names/destinations +- **AE4.** HarnessRouter maps a non-object extension to HTTP 400 `invalid_input`, + an oversized extension to HTTP 413 `allagents_workspace_too_large`, and an + extension on continuation to HTTP 409 `allagents_workspace_immutable`; each is + an `invalid_request_error` with `param: "metadata.allagents.workspace"` and + `detail.retryable: false`, before the hook or response allocation. The hook + rejects unknown logical names, caller-provided URLs, absolute/traversal paths, + commands, environment fields, credentials, originless/local repositories, and + duplicate names/destinations before source network access or agent launch. - **AE5.** Two turns linked by `previous_response_id` preserve a file and native conversation context. The hook runs once; produced-file collection reports @@ -502,10 +634,21 @@ caller responsible for acquisition. The temporary fork closes those seams. workspace. A new revision uses a new session. - **AE7.** Explicit UHP input files overlay materialized paths after the pre-agent checkpoint and before agent launch. -- **AE8.** A materializer crash, timeout, cancellation, malformed result, - publication crash, checkpoint failure, or Git failure starts no provider, - leaks no credential, and leaves the session failed/non-resumable rather than - partially ready. Provider fallback never reruns materialization. +- **AE8.** A materializer spawn/nonzero/crash or malformed/oversized result maps + to `allagents_materializer_failed`; publication/marker, checkpoint/baseline, and + materialization-state/CAS failures map respectively to + `allagents_workspace_publication_failed`, + `allagents_workspace_checkpoint_failed`, and + `allagents_workspace_state_failed`. A post-allocation secret-boundary recheck + maps to `allagents_secret_boundary_violation`. Each starts no provider, leaks + no credential, and leaves the session failed/non-resumable rather than + partially ready. Atomic-placement, `setsid()`, and double-fork fixtures include + a parent that returns `completed` while a descendant attempts a delayed write; + that case returns `allagents_workspace_containment_breach` even when forced + kill empties the cgroup. An unquiescent leaf preserves public cancellation/ + budget status when mandated, records the containment reason, and exits the + runner nonzero. Provider fallback never reruns materialization. After every + fault, a new native turn proves profile admission was safely reconciled. - **AE9.** OCI mode accepts a valid digest-pinned fixture with gzip/zstd layers and whiteouts and rejects mutable tags, indexes, mismatched digests/sizes, traversal, escaping links, devices, sparse files, unknown media types, and @@ -514,10 +657,16 @@ caller responsible for acquisition. The temporary fork closes those seams. refreshes through Pi for its configured provider. Missing, revoked, expired, unrefreshable, or locally stale-after-crash OAuth marks only that profile unavailable or `repair-required`; it never selects another profile or proxy. - A second turn sharing a profile waits or fails before launch. A separately - configured `proxyApiKey` deployment's broker permits the bounded multi-request - provider flow during the active turn and rejects wrong-audience, wrong-model, - wrong-turn, expired, or revoked credentials. + A barriered pair of simultaneous first arrivals with one `Idempotency-Key` + produces one admission and one result; a new continuation in that active + session returns `session_busy`; and genuinely new turns in other sessions + sharing the profile fail immediately with cataloged `harness_unavailable` + before allocation. Same-key waiters receive a pre-allocation error before its + claim is removed; they never fall through to a second execution. Cross-session + callers are never queued and cannot acquire later. A separately configured + `proxyApiKey` deployment's broker permits bounded multi-request provider flow + and rejects wrong-audience, wrong-model, wrong-turn, expired, or revoked + credentials. - **AE11.** Restart after a completed first turn preserves the session and exact persisted auth binding. Removing or changing that binding makes continuation fail before runner work; restoring the matching identity and config digest @@ -531,9 +680,12 @@ caller responsible for acquisition. The temporary fork closes those seams. owner, repository, workflow, approved ref, subject digest, and predicate, and pulls that digest rather than `latest`. The evidence records upstream, patch, materializer, Codex, and Pi inputs; mismatches fail closed. -- **AE13.** Promptfoo maps one stable materializer failure and one HarnessRouter - execution failure to failed `ProviderResponse` results with code, safe message, - and metadata; neither becomes a successful empty response. +- **AE13.** Promptfoo maps request-time auth errors, every cataloged materializer + failure, UHP `failed`/`incomplete`/`cancelled` results, and HarnessRouter + execution failures to `ProviderResponse.error`. It preserves the wire error + code when one exists and otherwise uses the exact terminal status, plus + retryability, safe message, and verified metadata. None becomes successful + empty output or an automatic retry. ### Scope Boundaries @@ -695,21 +847,23 @@ The hook supports two operations: - `preflight`: validate contract version, project catalog, credential-reference syntax and presence, required binaries, and filesystem assumptions without source network access; and -- `materialize`: validate the opaque descriptor, write only to the supplied - sibling staging root, and return without publishing. +- `materialize`: validate the opaque descriptor, write source content only to the + supplied sibling staging root, write the canonical manifest only to the + supplied private result root, and return without publishing. The materialize request contains the generic contract version, opaque metadata -value, session workspace and fixed staging roots, project configuration root, -and deadline. Secret values are injected only through the configured allowlisted -child environment; credential identifiers and values are absent from JSON. +value, session workspace, fixed staging and private result roots, project +configuration root, and deadline. Secret values are injected only through the +configured allowlisted child environment; credential identifiers and values are +absent from JSON. The generic result is either: - `completed`, effective relative cwd, effective descriptor digest, - workspace-manifest digest, complete path-free public metadata, and declared - nested-repository roots; or -- `failed`, stable code, safe message, retryability, and any verified incomplete - public metadata. + `workspaceManifest: { path: "workspace-manifest.json", digest }`, complete + path-free public metadata, and declared nested-repository roots; or +- `failed`, cataloged code, safe message, retryability, and any verified + incomplete public metadata. The runner validates the result and staged tree independently. It rejects an unknown envelope field/version, digest mismatch, physical path in public @@ -718,6 +872,131 @@ that does not match the manifest. The runner then owns publication, marker and checkpoint setup; a valid result never means the live workspace is already published. +### Workspace Manifest Contract + +The source tree includes one generated normative +`workspace-manifest.schema.json`, imported unchanged by the Git materializer, +OCI producer/materializer, runner validator, and their contract fixtures. The +document is at most 128 MiB and is an object with `additionalProperties: false`, +required string `version` fixed to `"1"`, required `repositories`, and required +`entries`. + +`repositories` is an array with at most 128 items. Every item is an object with +`additionalProperties: false` and exactly the required string fields `name` and +`destination`, validated as `ConfigName` and non-root `RelativeDirectory`. +Names and destinations are each unique; items are sorted by the UTF-8 bytes of +the NFC-normalized `name`. Every destination must exactly equal the `path` of a +directory entry in the same manifest. Duplicate destinations, missing +destination entries, and destinations naming files or symbolic links are +invalid even when the manifest digest is correct. + +`entries` is an array with at most 500,000 items. Every item has +`additionalProperties: false` and is exactly one of: + +- directory: required string fields `path`, `type: "directory"`, and + `mode: "040755"`; +- regular file: required string fields `path`, `type: "file"`, + `mode: "100644" | "100755"`, and + `sha256: "sha256:<64 lowercase hex>"`, plus integer `size` from zero through + 4 GiB; or +- symbolic link: required string fields `path`, `type: "symlink"`, + `mode: "120000"`, and `target`. + +Paths are unique, non-empty, relative POSIX paths sorted by their NFC-normalized +UTF-8 bytes. Every path component and symlink target must already be valid UTF-8 +and NFC; implementations reject rather than normalize non-UTF-8 or non-NFC +values. Paths and targets containing NUL, absolute paths, missing parents, or +links escaping the workspace are invalid. The root is implicit and has no entry. +Hard links are expanded to regular-file entries. + +The digest is `sha256:` plus the lowercase SHA-256 of the RFC 8785 bytes. Git +mode computes those bytes after completing staging. OCI mode requires its +configured workspace-manifest blob to contain the same canonical bytes and +copies them to the private result root. The runner resolves only the fixed +`workspace-manifest.json` relative path, validates it against the shared schema, +verifies its size and digest, walks staging without following links, reconstructs +the same catalog and entries, and requires byte-for-byte canonical equality +before publication. The private result root is never published or exposed +through UHP. + +The frozen cross-repository fixture is: + +```json +{"entries":[{"mode":"040755","path":"services","type":"directory"},{"mode":"040755","path":"services/api","type":"directory"},{"mode":"100644","path":"services/api/README.md","sha256":"sha256:98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4","size":3,"type":"file"},{"mode":"120000","path":"services/api/current","target":"README.md","type":"symlink"}],"repositories":[{"destination":"services/api","name":"api"}],"version":"1"} +``` + +Those exact bytes digest to +`sha256:667fef29fd8d241818263c5697075ba99eb86331c41eda5a27a812dc8771e4f8`. +The file bytes are `hi\n`. A change to the schema, fixture bytes, or digest is a +versioned contract change, not an implementation detail. + +### Failure Contract + +Failures before response allocation use the UHP non-2xx error envelope. Failures +after allocation return HTTP 200 with terminal `status: "failed"` and the same +error object in `response.error`; workspace failures use +`type: "harness_error"`, a safe message, `param: null`, and +`detail: { "retryable": }`. Stock cancellation remains terminal +`status: "cancelled"`. The hook's `retryable` field must equal this catalog: + +| Code | UHP placement | Retryable | +|---|---|---| +| `invalid_input` | HTTP 400 `invalid_request_error` before response allocation for a non-object workspace extension; `param` is `metadata.allagents.workspace` | no | +| `allagents_workspace_too_large` | HTTP 413 `invalid_request_error` before response allocation for the 64-KiB/depth bound; `param` is `metadata.allagents.workspace`; `detail.max_bytes` is 65536 for the byte bound | no | +| `allagents_workspace_immutable` | HTTP 409 `invalid_request_error` before response allocation when a continuation contains the workspace extension; `param` is `metadata.allagents.workspace` | no | +| `harness_unavailable` / `detail.reason: "allagents_auth_profile_busy"` | HTTP 503 `server_error` before response allocation for a saturated auth profile; `param` is null | yes | +| `harness_unavailable` / `detail.reason: "allagents_auth_profile_unavailable"` | HTTP 503 `server_error` before response allocation for an unavailable or repair-required auth binding; `param` is null | yes | +| `allagents_workspace_invalid` | failed response for post-allocation descriptor, catalog, path, layout, OCI image-manifest shape, index, or unsupported/foreign media rejection that is not a numeric limit | no | +| `allagents_workspace_source_auth_failed` | failed response for Git or registry credential rejection | no | +| `allagents_workspace_acquisition_failed` | failed response when Git, registry, HTTP, or transport I/O prevents complete byte acquisition; excludes digest, schema, and limit failures | no | +| `allagents_workspace_limit_exceeded` | failed response for source/workspace/archive/manifest repository, entry, byte, layer, file, path, or header limits; excludes hook request/stdout/stderr envelope limits | no | +| `timeout` | failed response only for an unexpected execution timeout before the declared task budget; a declared time/step budget remains UHP `incomplete` with `error: null` | no | +| `allagents_materializer_failed` | failed response for materializer spawn/nonzero/crash or malformed/oversized hook output | no | +| `allagents_secret_boundary_violation` | failed response when a post-allocation recheck finds a configured source-secret name or value in the service or agent environment | no | +| `allagents_workspace_manifest_invalid` | failed response for the workspace-manifest media type, schema, RFC 8785 bytes, or declared workspace-manifest digest | no | +| `allagents_workspace_integrity_mismatch` | failed response for OCI image/config/layer descriptor digest or size mismatch, or when staging differs from the verified workspace manifest | no | +| `allagents_workspace_publication_failed` | failed response for recoverable staging publication or marker failure | no | +| `allagents_workspace_checkpoint_failed` | failed response for root/nested checkpoint or collection-baseline failure | no | +| `allagents_workspace_state_failed` | failed response for a materialization-state persistence/CAS failure | no | +| `allagents_workspace_containment_breach` | failed response for completed-parent/live-descendant even if forced kill succeeds; an unquiescent leaf first enters internal non-terminal `containment_pending`, exits/restarts, and becomes public only after the old boundary is proven empty, preserving client-cancellation or declared-budget status | no | + +Classification order is normative. A completed-parent/live-descendant violation +is the failed containment code. Any other containment breach supersedes the +hook's earlier result for internal session state but never overwrites +UHP-mandated public `cancelled` or `incomplete`; absent either, it becomes the +failed containment code. Otherwise a declared UHP budget produces `incomplete`; +an unexpected execution timeout produces `timeout`; the post-allocation secret +boundary recheck precedes numeric limits; numeric limits are classified before +generic schema validation; and remaining failures use source authentication, +semantic workspace validation, acquisition I/O, materializer process/envelope +validation, workspace-manifest validation, cryptographic/tree integrity, +publication, checkpoint, then durable-state failure in that order. +One terminal outcome carries exactly one code. Thus a 129-item repository array +is `allagents_workspace_limit_exceeded`, malformed canonical workspace-manifest +bytes are `allagents_workspace_manifest_invalid`, registry transport failure is +`allagents_workspace_acquisition_failed`, and an OCI layer digest mismatch is +`allagents_workspace_integrity_mismatch`. + +Deployment `preflight` runs before readiness and allocates no UHP response. Its +failure stays operational: readiness is false and the safe operator diagnostic +uses the same classification vocabulary without pretending a task failed. + +Vendor codes and vendor detail reasons use the required `allagents_` prefix. A +request failure includes `detail.retryable`; the busy-profile reason omits +`retry_after_ms` rather than guessing. Promptfoo maps a non-2xx or `failed` +response to `ProviderResponse.error = ": "`. It maps +`incomplete` to `ProviderResponse.error = "incomplete: task stopped at a budget"` +and `cancelled` to +`ProviderResponse.error = "cancelled: task cancelled by client"`; both have +`code: null` and `retryable: false`. Every failure includes +`metadata.uhp = { httpStatus, responseStatus, code, reason, retryable }`. +`responseStatus` is null for a pre-allocation non-2xx request error and is the +actual terminal status for an HTTP-200 response. Verified +`metadata.allagentsWorkspace` is included when available; `reason` is the error +detail reason or the incomplete detail reason when present, otherwise null. +Promptfoo never converts a non-2xx, failed, incomplete, or cancelled UHP result +into successful empty output and performs no automatic retry. + ### Fork Maintenance Contract - Keep the fork in a dedicated repository/branch with the upstream remote intact. @@ -750,25 +1029,41 @@ published. - **Provider retry/fallback:** Run materialization before the provider candidate loop and surface a typed non-provider failure; a ready marker prevents reruns. - **Source credential leakage:** Use subprocess-only source credentials, - hermetic configuration, leak scans, and hostile fixtures. + hermetic configuration, leak scans, hostile fixtures, and a non-escapable + cgroup v2 boundary. Prove `populated 0` before interpreting success, reading + result files, publishing, releasing secrets, or cleanup. - **Native OAuth exposure:** Treat the selected profile as available to the harness and same-identity tools. Mount no other profile and prevent passive gateway/runner persistence from serializing the auth file. A malicious harness or tool can still emit its contents; use explicit proxy mode when this owner-trust boundary is unacceptable. -- **OAuth refresh loss or races:** Hold one per-profile lock for every - refresh-capable turn and administrative mutation. Persist local writes through - same-filesystem temp-write, file `fsync`, atomic rename, parent `fsync`, and - validation. Fault process death around local persistence; if remote rotation - leaves the committed profile invalid, mark it `repair-required`. Never switch - profiles or auth mode, and never run shared-profile turns concurrently. -- **Partial publication:** The materializer writes staging only. The runner - validates and publishes with recovery markers; no agent runs until the gateway - durably stores the resulting checkpoint and marks the session ready. +- **OAuth refresh loss, races, or queue collapse:** Have the runner supervisor + persist the admission record and hold the zero-waiter per-profile advisory lock + through descendant termination, terminal-state acknowledgement, and refresh + commit. Gateway death cannot release it; runner death leaves a durable fence + that blocks readiness until startup reconciliation. Preserve idempotency and + same-session precedence before cross-session admission. Cross-session overlap + returns `harness_unavailable` with + `detail.reason: "allagents_auth_profile_busy"` before allocation and can never + acquire later. Provision distinct profiles for parallelism. Persist local + writes through same-filesystem temp-write, file `fsync`, atomic rename, parent + `fsync`, and validation. Fault process death around local persistence; if + remote rotation leaves the committed profile invalid, mark it + `repair-required`. Never switch profiles or auth mode, and never run + shared-profile turns concurrently. +- **Partial publication or forged staging:** The materializer writes source + content only to staging and the canonical manifest only to the private result + root. The runner reconstructs and compares the tree, then publishes with + recovery markers; no agent runs until the gateway durably stores the resulting + checkpoint and marks the session ready. - **Descriptor/session drift:** Accept the key only on the initial request and persist the hook's effective digest/provenance for every later response. - **OCI attack surface:** Use a closed media profile, streaming digest checks, fixed limits, strict path/link/type validation, and exact-host redirect policy. +- **Catalog scale:** Reject more than 128 repositories, 500,000 entries, 32 GiB + staged content, or work exceeding the request deadline. Exercise the supported + boundary with representative multi-repository fixtures and publish those + limits for Promptfoo operators. - **Harness auth drift:** Pin Codex and Pi versions and require non-secret login, live-turn, refresh, and continuation probes before advertising each target. - **HarnessRouter restart semantics:** Claim persistence only for completed state @@ -796,10 +1091,12 @@ published. operation, publishes/checkpoints once, survives two-turn reuse, supports a safe nested cwd, reports nested-repository files, and cannot rerun under provider fallback. -5. Freeze generic hook envelope/state/auth fixtures and AllAgents descriptor, - configuration, provenance, and failure fixtures. +5. Freeze generic hook envelope/state/auth fixtures, canonical workspace-manifest + bytes, and AllAgents descriptor, configuration, provenance, and failure + fixtures. 6. Implement project schema projection, preflight, Git materialization, - credential containment, and checkpoint/collection integration. + credential containment, bounded catalog acquisition, and + checkpoint/collection integration. 7. Implement OCI materialization and its archive/registry security profile. 8. Prove the separately configured authenticated-proxy mode, then run Promptfoo one-shot, continuation, cancellation, restart, and failure mappings. @@ -831,10 +1128,20 @@ published. Pi version, and auth-adapter patch digest used by the gate. - **Verification:** For both Codex and Pi, complete login, a real first turn, continuation, and restart without a provider-route API key. Change or remove - the binding and prove continuation fails before runner work. Force overlapping - turns and prove the second waits or fails before launch. Terminate immediately - before, during, and after local refresh persistence; restart must see a - complete file that validates or becomes `repair-required`. Prove unselected + the binding and prove continuation fails before runner work. Use a barrier to + make two first arrivals atomically contend for the same new `Idempotency-Key` + and prove one admission/one result; prove a new same-session continuation + returns `session_busy`; and prove genuinely new cross-session turns fail + immediately with cataloged `harness_unavailable` before allocation. Same-key + waiters receive any pre-allocation owner error before claim removal; no rejected + cross-session request can acquire later. Record representative turn duration + and the one-active-turn-per-profile, zero-waiter operator capacity rule. Inject + materializer, publication, checkpoint, provider, cancellation, gateway-only + crash, runner crash, and whole-process-death faults; after each, prove + reconciliation completes before the next new turn acquires the runner-owned + fenced profile lock. + Terminate before, during, and after local refresh persistence; restart must see + a complete file that validates or becomes `repair-required`. Prove unselected profiles and other sessions' conversation state are inaccessible. With an inert agent, scan checkpoints, produced-file records, passive logs, and response metadata for automatic credential serialization. Record that an active @@ -855,14 +1162,33 @@ published. `runner/server.py`, response/session persistence, checkpoint/produced-file helpers, runner/gateway tests, and a fake materializer hook. - **Approach:** Add configured opaque metadata extraction and bounds, generic - result envelope, materialization CAS, a dedicated runner operation before the - provider loop, response-translator persistence, safe nested cwd, staged - publication, pre-agent checkpoint, and nested-repository collection. + result envelope, generated workspace-manifest schema and frozen canonical + fixture, materialization CAS, a dedicated runner operation before the provider + loop, response-translator persistence, safe nested cwd, staged publication, + pre-agent checkpoint, runner-owned cgroup v2 containment, and + nested-repository collection. - **Verification:** Upstream UHP conformance stays green and stock requests are - unchanged. Faults at every state/publication/checkpoint boundary fail closed. - Provider fallback cannot rerun the hook. A continuation reuses workspace, - provenance, and the persisted auth binding without the extension. Root and - nested repository files collect correctly, and an escaping cwd fails. + unchanged. Faults at every state/publication/checkpoint boundary fail closed + with the exact catalog code: materializer process/envelope, publication/marker, + checkpoint/baseline, state/CAS, and secret-boundary fixtures cover their rows. + The runner independently rejects a forged manifest, changed staging entry, + escaping link, undeclared or 129th repository, duplicate/missing repository + destination, repository destination naming a file or symlink, or invalid + private manifest path. `clone3(CLONE_INTO_CGROUP)` and stopped pre-exec + fallback fixtures immediately fork, call `setsid()`, and double-fork; cases + cover cancellation, deadline, and a `completed` parent whose descendant + attempts a delayed write. All prove `populated 0` before any terminal result, + manifest read, publication, secret release, or cleanup. The completed-parent + violation + returns the containment code even after successful kill. An unquiescent fixture + proves internal `containment_pending`, bounded acknowledgement, nonzero runner + exit, `on-failure` restart, old-boundary emptiness, and readiness held false + through orphan sweep. GET and stream remain non-terminal until reconciliation, + after which mandated cancellation/budget or failed-containment status appears. + Provider fallback cannot rerun the + hook. A continuation reuses workspace, provenance, and the persisted auth + binding without the extension. Root and nested repository files collect + correctly, and an escaping cwd fails. ### U2. AllAgents workspace contracts and Git materializer @@ -875,18 +1201,21 @@ published. configuration documentation, unit fixtures, and Git E2E fixtures. - **Approach:** Reuse authoritative workspace parsing and source normalization. Extend the project schema with strict snapshot and environment credential - references; add descriptor/hook/result schemas, defaults, canonicalization, - and a `preflight` mode. Expose a direct no-shell materializer entrypoint. - Resolve the complete execution-eligible catalog to exact commits in staging, - preserve nested `.git`, enforce the closed Git policy, validate destinations - and cwd, compute the workspace manifest, and return without publishing. + references; generate the normative workspace-manifest schema; add + descriptor/hook/result schemas, defaults, canonicalization, and a `preflight` + mode. Expose a direct no-shell materializer entrypoint. Resolve the complete + execution-eligible catalog to exact commits in staging, preserve nested + `.git`, enforce the closed Git policy, validate destinations and cwd, compute + the workspace manifest, and return without publishing. - **Verification:** Local HTTPS fixtures cover branches, tags, commits, PR refs, configured defaults and symbolic HEAD, multiple repositories, optional-name fallback, conflicting/originless/local sources, root/duplicate destinations, unknown names, missing directories, traversal, leading-dash/control/refspec revisions, ambiguous shorthand, hooks/helpers/filters, submodules, LFS, file/ext protocols, redirects, cancellation, timeout, partial cleanup, - canonical defaults, preflight failures, and exact provenance. + canonical defaults, preflight failures, exact provenance, generated-schema + validation, frozen canonical fixture bytes/digest, manifest reconstruction, + and the 128-repository, 500,000-entry, 32-GiB, and deadline boundaries. ### U3. Session binding, failures, and credential containment @@ -894,32 +1223,47 @@ published. continued sessions. - **Repositories/files:** HarnessRouter session/response persistence and tests; AllAgents credential-selection/environment code and hostile fixtures. -- **Approach:** Persist `unbound/materializing/ready/failed`, opaque request +- **Approach:** Persist `unbound/materializing/containment_pending/ready/failed`, + opaque request digest, effective descriptor digest, public provenance, workspace marker, checkpoint digest, and canonical auth-binding identity/digest through compare-and-set transitions. Extend every response construction/retrieval/ - replay path with identical public metadata. Resolve source `${ENV_VAR}` - references only when constructing the materializer child from an owner-only - runner secret source; reject configured names or values in the base service or - agent environment. In native mode, project only the selected harness OAuth - profile and exclude it from passive session persistence. In explicit proxy - mode, broker the proxy client key with the required audience, target, model, - turn, expiry, and revocation constraints. Scan workspace, nested Git, CLI - session state, checkpoints, logs, and responses. + replay path with identical public metadata and the exact failure catalog. + Resolve source `${ENV_VAR}` references only when constructing the materializer + child from an owner-only runner secret source; reject configured names or + values in the base service or agent environment. Use a delegated, + child-inaccessible cgroup v2 leaf and prove it empty before accepting any hook + outcome, reading result files, publishing, releasing secrets, or cleaning + staging. In native mode, project only the selected harness OAuth profile and + exclude it from passive session persistence. In explicit proxy mode, broker + the proxy client key with the required audience, target, model, turn, expiry, + and revocation constraints. Scan workspace, nested Git, CLI session state, + checkpoints, logs, and responses. - **Verification:** Initial idempotent replay preserves one result; continuation omits the extension and reuses ready state plus the exact auth binding; extension-bearing continuation or changed/unavailable binding fails. Crashes around hook/publication/checkpoint/CAS reconcile to ready only when the bound descriptor, published marker, and durable checkpoint all match; missing or - mismatched evidence becomes failed/non-resumable. Source secrets and the - HarnessRouter caller key are absent from the base service and every - shell-enabled agent path. The selected OAuth profile is available only through - its harness home; other profiles are inaccessible. An inert-agent probe - confirms no gateway/runner path automatically serializes it into checkpoints, - produced-file records, passive logs, or public metadata; an active tool can - still exfiltrate it in owner-trust mode. Native mode has no provider-route API - key. Proxy tests allow bounded in-turn provider calls and reject every - out-of-scope, expired, or revoked broker token. + mismatched evidence becomes failed/non-resumable with the exact materializer, + publication, checkpoint, or state code. Cancellation, deadline, malformed + output, completed-parent/live-descendant, and runner-shutdown fixtures leave + the cgroup empty before cleanup. A recovered completed-parent violation returns + `allagents_workspace_containment_breach`; failure to prove emptiness records + internal `containment_pending`, completes the internal gateway handshake or + bounded timeout, exits/restarts the runner, and withholds readiness plus every + terminal GET/stream result until the old boundary is proven empty. Reconciled + cancellation/budget retains its mandated public status; other cases become + failed containment. Startup and post-allocation secret-boundary fixtures + respectively block readiness with no UHP response and return + `allagents_secret_boundary_violation`. Source secrets and the HarnessRouter + caller key are absent from the base service and every shell-enabled agent path. + The selected OAuth profile is available only through its harness home; other + profiles are inaccessible. An inert-agent probe confirms no gateway/runner + path automatically serializes it into checkpoints, produced-file records, + passive logs, or public metadata; an active tool can still exfiltrate it in + owner-trust mode. Native mode has no provider-route API key. Proxy tests allow + bounded in-turn provider calls and reject every out-of-scope, expired, or + revoked broker token. ### U4. Immutable OCI workspace materialization @@ -929,15 +1273,24 @@ published. types, deterministic snapshot producer fixture, local registry E2E, and security fixtures. - **Approach:** Resolve only configured registries, implement bounded - Basic/Bearer authentication and exact-host redirect policy, verify direct - manifest/config/layers while streaming, apply changesets in staging, validate - paths/types/limits/catalog, and return through the same hook envelope as Git. - The HarnessRouter runner remains the sole publisher. + Basic/Bearer authentication and exact-host redirect policy, verify the direct + manifest, canonical workspace-manifest blob, and layers while streaming, apply + changesets in staging, validate paths/types/limits/catalog, reconstruct the + canonical manifest, and return through the same hook envelope as Git. The + HarnessRouter runner remains the sole publisher. - **Verification:** Local Distribution fixtures cover anonymous and authenticated - pulls, private CA, gzip/zstd, whiteouts, digest/size mismatch, redirects, - rebinding policy, indexes, unknown/foreign media, traversal, escaping links, - devices, sparse files, limit overflow, cancellation, cleanup, and no Git - fallback. + pulls, private CA, gzip/zstd, whiteouts, redirects, rebinding policy, indexes, + unknown/foreign media, traversal, escaping links, devices, sparse files, + cancellation, cleanup, and no Git fallback. They assert exact precedence: + transport failure is `allagents_workspace_acquisition_failed`; an OCI index, + malformed image-manifest shape, unknown/foreign media, or + duplicate/missing/non-directory repository destination is + `allagents_workspace_invalid`; 129 repositories or any other numeric overflow + is `allagents_workspace_limit_exceeded`; workspace-manifest media/schema/ + canonical-byte/declared-digest failure is + `allagents_workspace_manifest_invalid`; and OCI image/config/layer digest/size + or final staged-tree mismatch is + `allagents_workspace_integrity_mismatch`. ### U5. Harness-native OAuth, optional proxy, and Promptfoo E2E @@ -960,17 +1313,29 @@ published. agent failures. - **Verification:** Through the exact container network, Codex and Pi authenticate through their own OAuth sessions and use allowed models. Turn two sees turn - one's conversation and file mutation with the same persisted auth binding; an - overlapping turn sharing that profile waits or fails before launch. A config + one's conversation and file mutation with the same persisted auth binding. A + barriered pair of simultaneous first arrivals with one `Idempotency-Key` + produces one admission and one result; a new same-session continuation returns + `session_busy`; and genuinely new cross-session turns sharing the profile fail + immediately with cataloged `harness_unavailable` before allocation. Same-key + waiters receive any pre-allocation owner error before claim removal; none starts + a second execution. Cross-session callers are not queued. After each + materializer, publication, checkpoint, provider, cancellation, gateway-only + crash, runner crash, and whole-process-death fault, prove reconciliation + completes before the next new turn acquires the runner-owned fenced profile + lock. A config change or unavailable binding fails before runner work. Cancellation terminates the real turn. OAuth failure disables the target without selecting another profile or proxy. Faults around local refresh persistence leave a complete file that either validates or marks the profile `repair-required`; unselected - profiles remain inaccessible. The separately configured proxy permits bounded - multi-request use inside the active turn and rejects wrong-audience, - wrong-target, wrong-model, wrong-turn, expired, or revoked credentials. - Promptfoo returns successful output/usage/artifacts/provenance and maps all - failure classes to failed, coded responses rather than empty success. + profiles remain + inaccessible. The separately configured proxy permits bounded multi-request + use inside the active turn and rejects wrong-audience, wrong-target, + wrong-model, wrong-turn, expired, or revoked credentials. Promptfoo exercises + every catalog row plus UHP incomplete/cancelled outcomes and returns either + successful output/usage/artifacts/provenance or the exact + `ProviderResponse.error`/metadata mapping, preserving the error code when one + exists and terminal status otherwise, never empty success or automatic retry. ### U6. Release, operations, review, and upstream preparation @@ -1015,16 +1380,19 @@ published. | Stock compatibility | Upstream HarnessRouter tests and UHP conformance pass; requests without the configured metadata key are unchanged. | | Caller authentication | Every unauthenticated external create, continuation, retrieval, stream, cancellation, file, and artifact request fails before resource existence or metadata disclosure; runner operations are private and mutually authenticated. | | Hook ordering | Dedicated materialization finishes, publishes, and checkpoints before provider selection; fallback never reruns it. | +| Manifest integrity | Git and OCI share the generated normative schema and frozen canonical fixture. The runner rejects forged manifests, changed staging, unsupported modes/types, escaping links, undeclared or 129th repositories, duplicate/missing/non-directory repository destinations, invalid private paths, and digest or byte mismatches before publication. | +| Materializer containment | Atomic-placement, immediate-fork, `setsid()`, and double-fork fixtures prove the delegated cgroup reaches `populated 0` before any terminal result, manifest read, publication, secret release, or cleanup. A completed-parent/live-descendant violation returns the containment code after successful kill. An unquiescent boundary enters internal non-terminal `containment_pending`, exits/restarts the runner, withholds terminal GET/stream results and readiness until old-boundary emptiness, then exposes the mandated cancellation/budget status or failed containment. | +| Capacity envelope | Native profiles admit one active refresh-capable turn and zero cross-session waiters. UHP precedence is atomic: simultaneous duplicate idempotency keys share one claim/admission/result, new same-session overlap returns `session_busy`, and only genuinely new cross-session overlap fails with cataloged `harness_unavailable` before allocation. Pre-allocation claim errors reach current waiters before claim removal. After every post-admission fault, reconciliation precedes reacquisition of the runner-owned fenced lock. Git and OCI reject more than 128 repositories, 500,000 entries, 32 GiB staged content, or work beyond the request deadline. | | Durable state | Fault injection proves only sessions with matching bound descriptor, published marker, and durable checkpoint become ready; missing/corrupt/mismatched evidence fails non-resumable without replay. | | Session continuity | Two real turns share native conversation, writable workspace, and the persisted auth-binding identity/digest; continuation omits the extension. A changed or unavailable binding fails before runner work. | | Workspace integration | Safe nested cwd, repository-mode root/nested Git checkpoints, snapshot private tree baselines, produced list/file/ack, hydrate, and initial-source suppression pass. | | Git acquisition | Closed transport/config policy, constrained revisions, exact commits, non-root destinations, catalog validation, and partial cleanup pass against local HTTPS remotes. | | OCI acquisition | Digest/media/path/link/type/limit matrix passes against a real local registry. | | Credential boundary | Every configured source secret and the HarnessRouter caller key are absent from the base service and every agent-readable source/session path, checkpoint, log, and public output. The selected OAuth profile is available inside the documented harness owner-trust boundary; other profiles are inaccessible, and an inert-agent probe proves the gateway/runner never serializes the auth file automatically. | -| Provider boundary | Codex and Pi login, live-turn, atomic local refresh, crash/stale-profile repair, bound continuation, serialized overlapping turns, and failure probes use harness-native OAuth without a provider-route API key. OAuth failure never changes profile or auth mode. A separate proxy broker permits bounded in-turn calls and rejects wrong-scope, expired, or revoked tokens. | +| Provider boundary | Codex and Pi login, live-turn, atomic local refresh, crash/stale-profile repair, bound continuation, atomic idempotency/session/profile admission, and failure probes use harness-native OAuth without a provider-route API key. OAuth failure never changes profile or auth mode. Simultaneous duplicate keys share one admission/result, same-session overlap returns `session_busy`, and new cross-session profile overlap returns cataloged `harness_unavailable` before allocation. A separate proxy broker permits bounded in-turn calls and rejects wrong-scope, expired, or revoked tokens. | | Lifecycle | Streaming, cancellation, idempotency, artifacts, usage, response metadata, completed restart, and interrupted-work failure match the contract. | | Packaging | The public `linux/amd64` GHCR manifest and GitHub/Sigstore build-provenance and SBOM attestations are verified for expected owner, repository, workflow, approved ref, subject digest, predicates, base-image digest, runtime lockfiles, OS packages, Git/OCI tools, and harness versions. Deployment uses that digest and publishing needs no Docker Hub credential. | -| Consumer | Promptfoo one-shot/two-turn Git/OCI success and materializer/agent failure mappings pass. | +| Consumer | Promptfoo one-shot/two-turn Git/OCI success passes. Every failure-catalog row and UHP failed/incomplete/cancelled result maps to the exact `ProviderResponse.error` and metadata, preserving a wire code when present and terminal status otherwise, never empty success or automatic retry. | | Review | Final review findings in both repositories are resolved before the final built-image E2E. | ## Definition of Done @@ -1048,12 +1416,33 @@ published. runner operations remain private and mutually authenticated. - Git and OCI materialization publish and checkpoint before provider dispatch and happen exactly once per extension-bearing session. +- The generated normative workspace-manifest schema, frozen canonical fixture, + private result-root transport, Git generation, OCI blob, RFC 8785 digest, and + independent runner reconstruction agree byte-for-byte before publication and + enforce the same 128-repository/500,000-entry limits. +- Every materializer outcome proves the delegated cgroup reaches `populated 0` + before any terminal result, result read, publication, secret release, or + cleanup. A completed-parent/live-descendant violation returns the cataloged + containment failure after successful kill. An unquiescent boundary stays + internal/non-terminal through runner exit/restart; deployment + restart-on-failure destroys the old container, and readiness plus terminal + GET/stream visibility remain withheld until startup proves the old boundary + empty. +- Published operator limits state one active turn and zero cross-session profile + waiters per native auth profile, while UHP idempotency duplicates share one + atomic claim/result, plus the 128-repository, 500,000-entry, 32-GiB, and + request-deadline acquisition caps. - Continuation preserves native conversation/workspace state and the persisted auth-binding identity/digest, omits the extension, and follows HarnessRouter's predecessor semantics. A changed or unavailable binding fails before runner - work. Every native auth profile serializes refresh-capable turns. Local refresh - writes are atomic and validated; a stale credential after remote rotation - becomes `repair-required` rather than changing profile or auth mode. + work. Simultaneous duplicate idempotency keys share one atomic claim, + admission, and result; new same-session overlap returns `session_busy`; and + genuinely new cross-session profile overlap fails with cataloged + `harness_unavailable` before response, runner, or materializer allocation. + Runner-owned durable fencing survives gateway failure, and reconciliation + precedes lock reacquisition after runner failure. Local refresh writes are + validated; a stale credential after remote rotation becomes `repair-required` + rather than changing profile or auth mode. - Repository-mode roots preserve usable Git state. Every source mode preserves truthful root/nested produced files across checkpoint/hydrate. - Source credential values are read from an owner-only runner secret source and From 27359e9a81aee02733d94a0c2b0330015815b6c9 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Wed, 23 Sep 2026 14:18:54 +1000 Subject: [PATCH 20/44] docs(architecture): define shared workspace lifecycle --- .../0002-adopt-uhp-through-harnessrouter.md | 687 +++-- ...0837-feat-coding-execution-gateway-plan.md | 2261 ++++++++++------- 2 files changed, 1888 insertions(+), 1060 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index a0127ad4..145514ff 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -2,6 +2,7 @@ - Status: Accepted; implementation gated on native-auth feasibility - Date: 2026-09-21 +- Updated: 2026-09-23 ## Decision @@ -19,17 +20,20 @@ the route so operators that reject the native owner-trust boundary have a supported alternative. HarnessRouter owns caller authentication, UHP request and response semantics, -streaming, cancellation, idempotency, session continuity, per-session -workspaces, agent execution, usage, and artifacts. AllAgents owns the workspace -descriptor, deterministic source materialization, and provenance returned -through the HarnessRouter response. +streaming, cancellation, idempotency, session continuity, workspace-generation +publication, session attachments, retention, quotas, garbage collection, agent +execution, usage, and artifacts. AllAgents owns the +`metadata["allagents.workspace"]` JSON descriptor, its resolution against the +project `workspace.yaml`, deterministic Git/OCI generation construction, and +provenance returned through the HarnessRouter response. The initial deployment will use a narrow AllAgents-maintained HarnessRouter fork. -The fork adds a generic pre-turn workspace-materializer hook. An AllAgents -materializer behind that hook interprets a namespaced workspace descriptor, -acquires the configured Git repository set or an immutable OCI workspace -snapshot, and returns normalized provenance before HarnessRouter launches the -agent. +Its generic pre-turn workspace hook resolves a verified immutable source +generation. HarnessRouter reuses an existing generation when possible; otherwise +an AllAgents materializer builds private staging and HarnessRouter atomically +publishes it. Each session then receives either a shared read-only attachment or +a private editable workspace derived from that generation before the harness +starts. The fork is a delivery mechanism, not a new protocol. All fork changes must be structured for a later upstream contribution. Delivery does not depend on @@ -45,17 +49,28 @@ flowchart TB CLIENT[Promptfoo or another UHP client] GATEWAY[Forked HarnessRouter gateway] RUNNER[HarnessRouter runner] - MATERIALIZER[AllAgents workspace materializer] + MATERIALIZER[AllAgents generation builder] + GENERATIONS[Immutable workspace generations] + READONLY[Read-only session attachment] + EDITABLE[Private editable session workspace] + LIFECYCLE[Leases, retention, quotas, and GC] HARNESS[Selected Codex or Pi harness] - AUTH[Durable harness-native OAuth profile] + AUTH[Bound harness-native OAuth profile] PROXY[Optional authenticated proxy] MODEL[Model provider] - CLIENT -->|UHP + allagents.workspace| GATEWAY - GATEWAY -->|runner /materialize| RUNNER - RUNNER -->|generic pre-turn hook| MATERIALIZER - MATERIALIZER -->|prepared staging + provenance| RUNNER - RUNNER --> HARNESS + CLIENT -->|UHP + metadata.allagents.workspace| GATEWAY + GATEWAY -->|session CAS + prepare/ack| RUNNER + RUNNER -->|validate, resolve, or cache-miss build| MATERIALIZER + MATERIALIZER -->|verified staging + provenance| RUNNER + RUNNER -->|atomic publish or reuse| GENERATIONS + GENERATIONS -->|read-only mount + lease| READONLY + GENERATIONS -->|private copy| EDITABLE + READONLY --> HARNESS + EDITABLE --> HARNESS + LIFECYCLE -->|expire, delete, or evict| GENERATIONS + LIFECYCLE --> READONLY + LIFECYCLE --> EDITABLE AUTH -.->|native mode: login and refresh| HARNESS HARNESS -->|native mode| MODEL HARNESS -.->|proxy mode: scoped turn credential| GATEWAY @@ -71,11 +86,15 @@ does not translate that OAuth session into an API key. Native OAuth is an owner-trust mode: the selected harness and tool subprocesses running under the same operating-system identity may access and emit its -credential. The gateway/runner does not automatically serialize the auth file -into source trees, checkpoints, produced-file records, passive logs, request -metadata, or response metadata, and it mounts no other profile. Deployments that -cannot accept active exfiltration risk must explicitly configure the brokered -proxy route. +credential during an active turn. The runner projects only the selected profile +through a turn-scoped mount namespace or equivalent same-filesystem view; it +never copies the credential into durable session state. Before terminal +acknowledgement and profile-lock release it commits or rejects refresh, removes +the projection, and verifies retained homes clean. Restart removes or quarantines +stale projections before readiness or profile reacquisition. Checkpoints, +produced-file records, backups, passive logs, request metadata, and response +metadata never serialize the auth file. Deployments that cannot accept active +exfiltration risk must explicitly configure the brokered proxy route. ## Protocol boundary @@ -84,13 +103,15 @@ request, ordered streaming events, `previous_response_id` continuation, cancellation, files, artifacts, usage, lifecycle, and error semantics. HarnessRouter's UHP conformance suite is the protocol oracle. -AllAgents adds one namespaced request extension: +AllAgents adds one namespaced JSON request extension: ```json { "metadata": { "allagents.workspace": { "version": "1", + "access": "readOnly", + "retention": "session", "source": { "kind": "repositories", "revisions": { @@ -107,104 +128,253 @@ AllAgents adds one namespaced request extension: } ``` -The extension identifies configured sources by logical name. Callers cannot -supply repository or registry origins, host paths, credentials, commands, -environment variables, materializer executables, or Docker options. - -The materializer returns a strict result containing the resolved source -identity, logical working directory, workspace-manifest digest, and bounded -failure information. HarnessRouter returns that result in namespaced response -metadata. It does not expose configured origins, physical paths, or credentials. - -Ordinary UHP input files remain supported. HarnessRouter applies the materialized -workspace first and request input files second, so explicit attachments may -overlay source files deterministically. - -## Workspace and session semantics - -The workspace descriptor and selected authentication binding are accepted only -when creating the first response in a session. HarnessRouter binds the canonical -workspace digest, resolved provenance, harness target, auth mode, and native -profile or proxy-connection identity/digest before agent execution. - -A continuation uses `previous_response_id` and the same HarnessRouter session -workspace. It must omit the workspace descriptor and reuse the persisted auth -binding. A deployment configuration change never silently switches mode, -profile, or connection; if the exact binding is unavailable, continuation fails -closed until it is restored. A new source revision, working directory, or auth -binding requires a new session. - -The materializer runs before the first agent turn and never depends on the model -reading a prompt, calling an MCP tool, or extracting an archive. Source -acquisition failure starts no agent process and never falls through to another -source mode or credential identity. - -Workspaces are writable and private to the HarnessRouter session. Version one -does not provide read-only workspaces or copy-on-write generations. The fork -extends HarnessRouter's existing workspace checkpoint with a durable pre-agent -materialization state and nested-repository collection metadata. - -Completed session state survives a HarnessRouter restart when its documented -durable data volume is preserved. An in-flight agent process does not survive -whole-container termination. Interrupted turns fail and are not replayed -automatically; a later continuation is allowed only when HarnessRouter reports -the session resumable. +`metadata["allagents.workspace"]` is the first-response session descriptor, not +the project configuration file. It selects sources by logical name from the +server-owned project `workspace.yaml` and requests an attachment policy. It +cannot add or override repository or registry origins, destinations, +credentials, host paths, commands, environment variables, materializer +executables, or Docker options. + +`access` is required and is exactly `readOnly` or `editable`. `retention` is +optional and defaults to `session`; `persistent` is accepted only when enabled +by deployment policy and within persistent-workspace quotas. Access and +retention are independent: either access mode may use either retention class. + +Namespaced response metadata contains the effective descriptor digest, resolved +source and generation identity, logical working directory, workspace-manifest +digest, access mode, retention class, and effective expiry. Active turns and +`persistent` sessions report `expiresAt: null`; a `session` terminal +acknowledgement sets the timestamp returned by terminal, retrieval, and replay +paths. Failures before attachment `ready` omit workspace metadata entirely; +terminal failures after `ready` include the same complete public object. +Metadata never exposes the raw request digest, configured origins, physical +paths, credentials, internal generation-epoch/lease/attachment identifiers, or +other sessions' quota state. + +Ordinary UHP input files remain supported for `editable` sessions and are +applied to the private workspace after generation attachment. A `readOnly` +request containing workspace input files is rejected after response allocation +but before source byte acquisition; the system never shadows a read-only +generation with an implicit writable layer. + +## Workspace generations and session semantics + +On the first response, HarnessRouter atomically binds the session to one +canonical workspace descriptor, one verified generation key and publication +epoch, one access mode, one retention class, one harness target, and exactly one +native profile or proxy connection. A new source revision, logical working +directory, access mode, retention class, harness, or authentication binding +requires a new session. + +A workspace generation is immutable source-visible content identified by a +canonical resolved-source key and a verified workspace-manifest digest. Each +publication also has a unique internal epoch ID. At most one live epoch exists +for a key; a later epoch may begin only after durable logical and physical +eviction of the prior one completes, and existing sessions never substitute it. +In repository mode publication also binds a separately validated semantic Git- +state record to the exact resolved commits; volatile `.git` pack/index bytes do +not fragment identity. Sharing authorization and selected credential-reference +identities do fragment the key; credential values do not. Access, retention, +logical working directory, harness target, profile, and session identity do not. + +Concurrent requests for the same absent epoch share one runner-owned build claim +and observe one atomically published result. Each request retains its own +cancellation and deadline: cancellation detaches only that waiter, and the build +continues while another live waiter exists. Failed or partial staging never +becomes attachable. A ready hit or completed build acquires a durable provisional +attachment pin under the same generation lock before mount or copy; garbage +collection cannot race that pin. + +A `readOnly` session mounts the published generation read-only. Multiple sessions +using different harnesses or authentication profiles may execute concurrently +against the same generation while keeping their operating-system identity, +conversation state, harness home, temporary files, logs, outputs, and response +state separate. The filesystem, not caller intent, enforces generation +immutability. + +An `editable` session receives a unique private writable copy derived from the +generation. After a shared publication, each editable waiter independently proves +the generation's full physical byte/inode usage fits its admitted hard allowance; +a non-fitting waiter fails alone without invalidating the ready epoch or another +waiter. No writable inode or checkpoint is shared with another session, and +mutations never flow back into the generation. The allowance covers the private +tree, UHP input overlays, root/nested checkpoints, and produced-file state +throughout every turn and continuation. Exceeding it fails the turn without +changing access or retention. Read-only sessions have no workspace mutation +checkpoint or produced-file delta; editable sessions preserve private mutations, +checkpoints, and produced files. + +The gateway is the sole writer of session attachment, expiry, and tombstone +state. The runner owns generation/reference/mount/private-resource state. It +durably prepares resources and returns opaque attachment evidence; the gateway +commits `ready` and acknowledges it. That acknowledgement causes the runner to +release the provisional pin exactly once. Restart either preserves a committed +attachment or rolls an uncommitted prepare back; neither component independently +binds the other's state. + +One generation use updates `lastUsedAt` only when the gateway commits an +attachment `ready`. Before that it remains null. After acknowledgement, the +runner records `max(existing, readyCommitTimestamp)` under the generation lock; +startup can replay a missed update idempotently from committed gateway evidence. +Publication or a failed prepare does not count as use. Eviction orders null +`lastUsedAt` first by `publishedAt`, then non-null `lastUsedAt`, then +`publishedAt`, ascending generation-key bytes, and epoch-ID bytes. + +A continuation uses `previous_response_id`, omits +`metadata["allagents.workspace"]`, and reuses the original session binding only +when its exact key/epoch evidence remains valid and retention is persistent or +its session idle deadline is unexpired. It sees the same immutable epoch in +`readOnly` mode or the same private writable workspace in `editable` mode. It +cannot change the generation, access, retention, logical working directory, +harness, profile, or proxy connection. A changed or unavailable authentication +binding fails closed until restored. Known missing or corrupt attachment evidence +returns `allagents_workspace_non_resumable`; it never resolves source, +rematerializes, or substitutes a rebuilt epoch. + +Generation resolution and attachment finish before the first agent turn. On a +cache miss, materialization, independent verification, and atomic publication +also finish first; on a hit, byte acquisition is skipped. Source failure starts +no agent process and never falls through to another source mode, access mode, +credential identity, or provider route. + +Every active operation holds a durable session lease and has no idle expiry. One +gateway session CAS checks `session_busy`, exact attachment/binding, and expiry/ +deletion together. A busy or invalid attempt changes no deadline. A valid +`session` continuation stores and clears its unexpired deadline in a provisional +turn-admission fence before the zero-waiter profile attempt. Profile success +commits active; pre-allocation profile failure restores the exact original +deadline when still future or tombstones the session if it elapsed. A read-only +session keeps an exact generation-epoch reference until expiry or deletion. An +editable session holds a provisional generation pin through successful private- +copy attachment, then retains only its private workspace and provenance. Idle +time starts only after durable terminal acknowledgement; GET, stream polling, and +idempotent replay do not renew it. + +Deployment policy supplies finite, nonzero limits for session idle TTL, staging +bytes and concurrent builds, published-generation bytes/count, each editable +session's hard bytes/inodes, total reserved private bytes/inodes, total sessions, +persistent sessions, and tombstone bytes/count/TTL. Before exposing a workspace +response/session, one idempotent admission token durably reserves its generic +session slot and fixed-size tombstone slot. Invalid descriptors remain charged +through finite failed-response retention, tombstoning, and purge. After +validation, `persistent` and the stable editable private-reservation ID are +authorized/reserved before source resolution. Active leases, durable references, +and provisional pins are never evicted. Ready zero-reference/zero-pin epochs are +the only generation GC candidates; failed deletion remains quarantined and +counted, prevents same-key republication, and never advertises freed capacity. If +protected state consumes available quota, new admission fails instead of +deleting protected state or changing policy. + +Expiry or authenticated deletion atomically tombstones the session before +cleanup, rejects new continuations, waits for active work, credential projections, +and mounts to quiesce, removes private state, and releases every reference and +reservation exactly once. Retained tombstones live at least as long as response/ +idempotency records and produce `allagents_workspace_expired`; bounded compaction +then purges both lifecycle identity and its reserved slot, after which the stock +non-disclosing unknown-ID error applies. Neither outcome silently rematerializes. + +Completed, unexpired session state and generation records survive a +HarnessRouter restart when the documented durable volume is preserved. Startup +reconciles build waiters, provisional pins, publications, leases, mounts, private +quota usage, credential projections, tombstones/compaction, and deletion before +readiness or garbage collection. An internal `containment_pending` session stays +non-terminal until its recorded cgroup is empty. In-flight agent processes do not +survive whole-container termination; interrupted turns fail and are not replayed +automatically. ## Fork boundary The HarnessRouter fork is limited to the workspace-integration seam and the harness-native authentication-state seam. The workspace seam: -1. recognizes one configured, bounded metadata key on the first UHP response; -2. treats its JSON value as opaque, canonicalizes it, and binds it to the session; -3. calls a dedicated runner materialization endpoint once, before provider - selection or fallback; -4. invokes the configured materializer executable and validates its typed envelope; -5. publishes and checkpoints the prepared workspace before any agent starts; -6. allows a symlink-safe logical working directory beneath the session root while - preserving session-UID isolation; -7. preserves nested repository Git state and collects their produced files; -8. persists and returns bounded hook metadata through streaming, terminal, - retrieval, and idempotent-replay response paths; -9. rejects the workspace key on continuations; -10. applies ordinary input files only after successful materialization; and -11. strips every configured materializer-only environment name from agent children. +1. recognizes one configured, bounded JSON metadata key and, before response + allocation, uses the idempotent admission transaction to reserve its generic + session and tombstone slots together with applicable profile admission; +2. canonicalizes the initial JSON, records its raw digest, binds it to the + allocated session, and invokes source-free generic `validate` to return + effective access, requested retention, effective descriptor digest/cwd, a + bounded selected credential-reference set, and a private normalized- + descriptor reference; the runner verifies that set against secret-free + preflight declarations and credential-store handles; +3. makes the runner the sole persistence authority and, before source + resolution, reserves any persistence slot and one stable editable hard-private + byte/inode reservation ID; +4. invokes `resolve` with the exact validated descriptor and selected reference + set to produce an exact private source-plan reference/digest, generation key, + and bounded provenance; +5. atomically joins or creates a runner-owned generation epoch build, preserves + each waiter's cancellation/deadline, takes ownership of the exact resolved + plan and selected reference identities, reserves full staging/prospective- + generation capacity before byte acquisition, and acquires a provisional pin + before handing a ready epoch to attachment; +6. invokes `materialize` with that resolved plan and selected set only on a miss, + independently validates staging, semantic Git state, and its manifest, proves + the materializer boundary empty, computes physical retained byte/inode usage, + and under the generation lock atomically converts prospective capacity to + actual usage, releases excess plus staging reservation, persists accounting, + and publishes the immutable epoch before waiter pins; +7. independently checks each editable waiter's initial copy fit, then has the + runner prepare a read-only epoch reference/mount or a private editable copy + carrying the admitted private-reservation ID, has the gateway alone commit the + attachment `ready`, and on acknowledgement transfers the reservation without + a second debit and releases the provisional pin; +8. allows a symlink-safe logical working directory while preserving per-session + identity and writable-state isolation; +9. checkpoints and collects produced files only from private editable state and + enforces its byte/inode quota across every turn; +10. persists bounded generation, attachment, retention, expiry, and provenance + metadata through streaming, terminal, retrieval, and idempotent replay paths; +11. rejects the workspace key on continuations and never renews retention for + polling or replay; +12. applies ordinary input files only to editable private state; +13. strips every configured materializer-only environment name from agent + children; and +14. owns crash-safe build/pin/reference reconciliation, bounded tombstones, + expiry, deletion, quota admission, and deterministic eviction of only ready + unreferenced and unpinned generation epochs. The authentication-state seam separates session conversation state from durable per-harness OAuth state. In native mode it projects only the selected profile -into the Codex or Pi home and permits the harness to persist token refreshes. It -prevents the gateway/runner lifecycle from copying auth files into checkpoints, -produced-file records, passive logs, or public metadata. Other profile roots are -not mounted. -This does not prevent the selected harness or same-identity tools from reading -or emitting the credential inside the accepted owner-trust boundary. - -The projection mechanism must preserve each harness's credential-file write and -atomic-replacement behavior. A locally committed refresh uses a same-filesystem -temporary file, file and parent-directory `fsync`, atomic rename, and validation. -A crash after the provider rotates credentials but before local commit may leave -the profile stale; restart marks it `repair-required` when validation fails and -requires native login again. It never switches profiles or activates the proxy. -Version one supports exactly one active refresh-capable turn per native profile -and holds that profile lock for every turn and every login, logout, or repair -operation. Admission first atomically claims the UHP `Idempotency-Key`; concurrent -same-key requests share one admission/result, and same-session overlap returns -stock `session_busy`. Only a genuinely new cross-session turn tries the +into the Codex or Pi home for the active turn and permits the harness to persist +token refreshes. The projection preserves the harness's credential-file write +and atomic-replacement behavior without copying the credential into the retained +session home. It is removed after descendant termination and refresh disposition, +before terminal acknowledgement or lock release. Checkpoints, produced-file +records, backups, passive logs, and public metadata exclude it. Other profile +roots are never mounted. This does not prevent the selected harness or +same-identity tools from reading or emitting the credential inside the accepted +owner-trust boundary. + +A locally committed refresh uses a same-filesystem temporary file, file and +parent-directory `fsync`, atomic rename, and validation. A crash after the +provider rotates credentials but before local commit may leave the profile stale; +restart marks it `repair-required` when validation fails, removes or quarantines +stale projections, and requires native login again before readiness. It never +switches profiles or activates the proxy. Version one supports exactly one active +refresh-capable turn per native profile and holds that profile lock for every turn +and every login, logout, or repair operation. Admission first atomically claims +the UHP `Idempotency-Key`; concurrent same-key requests share one +admission/result. The session CAS returns stock `session_busy` before changing +the deadline, then provisionally fences a genuinely new turn before it tries the zero-waiter profile lock. Collision returns HTTP 503 `harness_unavailable` with `detail.reason: "allagents_auth_profile_busy"` before response allocation, -runner work, or materialization. +runner work, or materialization, and rolls the session fence back to the exact +future deadline or an elapsed-deadline tombstone. The runner turn supervisor persists the admission record and owns the profile -lock through descendant termination, terminal-state acknowledgement, and refresh -commit. Gateway-only failure cannot release it. Runner failure leaves a durable -fence; startup blocks readiness and admission until descendant and profile -reconciliation. Operators provision distinct profiles for parallel capacity. +lock through descendant termination, refresh disposition, projection teardown, +and terminal-state acknowledgement. Gateway-only failure cannot release it. +Runner failure leaves a durable fence; startup blocks readiness and admission +until descendant, projection, and profile reconciliation. Operators provision +distinct profiles for parallel capacity. The generic fork layer does not understand the AllAgents descriptor. It enforces -only the configured key, JSON/size bounds, immutable first-turn binding, hook -envelope, lifecycle, and response namespace. The external AllAgents executable -owns schema/default validation, workspace configuration, Git/OCI acquisition, -source-credential selection, filesystem policy, and provenance. +only the configured key, JSON/size bounds, immutable first-turn binding, typed +hook envelope, runner-owned persistence authorization, lifecycle, and response +namespace. The external AllAgents executable owns schema/default validation, +workspace configuration, Git/OCI acquisition, source-credential selection, +source-tree construction policy, and provenance; it never authorizes retention +or writes live session state. The fork must preserve stock behavior for requests without the configured key and must continue to pass upstream UHP conformance. The maintained patch series @@ -238,8 +408,9 @@ Each required native target must prove: 6. complete local credential files after termination before, during, and after refresh persistence, with invalid post-rotation state becoming `repair-required`; -7. no automatic gateway/runner serialization of the auth file into checkpoints, - produced-file records, passive logs, or response metadata; and +7. active-turn-only projection teardown on success, failure, cancellation, and + crash recovery, with no auth path retained in homes, mounts, checkpoints, + produced-file records, backups, passive logs, or response metadata; and 8. explicit acknowledgement that same-identity harness tools can read or emit the selected credential. @@ -251,61 +422,114 @@ the implementation must not introduce an implicit fallback or credential shim. ## Source authority and credentials The project `workspace.yaml` remains the source of truth for logical repository -names, origins, non-root destinations, and default revisions. Repository -execution names are explicit `name` values or the portable basename of `path`; -duplicate names or destinations, root destinations, and local/originless entries -make execution preflight fail. Its schema gains a strict `workspaceSnapshots` -catalog and environment-variable credential references; secret values remain -deployment-only. HarnessRouter owns harness/model/provider configuration. The -user workspace does not become a second source catalog, and no `gateway.yaml` or -caller-controlled registry is introduced. +names, origins, non-root destinations, default revisions, snapshot catalog, and +environment-variable credential references. Repository execution names are +explicit `name` values or the portable basename of `path`; duplicate names, +duplicate or ancestor/descendant destinations, root destinations, and +local/originless entries make execution preflight fail. Secret values remain +deployment-only. + +The UHP `metadata["allagents.workspace"]` JSON object only selects from that +catalog and requests session access and retention. It is not parsed as, merged +with, or persisted as a replacement for `workspace.yaml`. HarnessRouter +deployment configuration owns harness/model/provider targets, persistence +authorization, idle TTLs, quotas, and garbage-collection policy. No +`gateway.yaml` or caller-controlled source registry is introduced. + +The materializer `preflight` receives no secret values. It validates hook/catalog +versions, reference syntax, and required source tools and returns a bounded set of +configured credential-reference names or opaque IDs. The runner, not the hook, +checks those store handles and all actual staging/result/generation filesystem +relationships. Source-free `validate` returns the bounded selected subset; the +runner injects values for only that set into `resolve` or `materialize`. Repository mode acquires the complete declared repository set, with optional revision overrides by logical name. It accepts only a bounded ref-name grammar, rejects option-like or refspec-shaped values, resolves advertised refs to full commits before agent execution, fetches by verified object ID, and records those -commits in provenance. +commits in provenance. It preserves `.git` for coding tools but hermetically +normalizes the allowed detached-HEAD configuration/ref set and removes reflogs, +`FETCH_HEAD`, locks, hooks, worktree links, alternates, shallow/replace/graft +state, extra refs, extra objects, and credential-bearing configuration. The +runner independently verifies HEAD, an index exactly matching the resolved commit +tree, its canonical object-set digest, and exactly the transitive required object +closure with no extras. It then proves the source-visible manifest equals exactly +the union of each resolved commit tree prefixed by its pairwise non-overlapping +destination plus only necessary destination ancestor directories. Undeclared +paths outside that union fail integrity validation. Snapshot mode accepts only a configured OCI repository plus immutable image- manifest and workspace-manifest digests. It verifies the image manifest, canonical workspace-manifest bytes, layer sizes and digests, applies OCI whiteouts, validates the resulting declared workspace layout against the -manifest, and records the ordered layer digests. +manifest, rejects `.git` administrative subtrees, and records the ordered layer +digests. Snapshots requiring Git history use repository mode. Both source modes produce the same versioned canonical workspace manifest. Its -RFC 8785 bytes enumerate every directory, regular file, and symbolic link in -logical path order with normalized mode, size, content digest, or link target as -applicable. Git mode computes it from completed staging. OCI mode carries the -same bytes in the configured workspace-manifest blob and must reproduce them -after applying the layers. The runner receives the manifest through a private -bounded result root, verifies its digest and the staged tree independently, and -never publishes the manifest as source content. +RFC 8785 bytes enumerate every source-visible directory, regular file, and +symbolic link in logical path order with normalized mode, size, content digest, +or link target as applicable. Repository mode omits only separately validated +`.git` administrative subtrees so volatile pack/index/stat representation does +not fragment identity; no source-visible path may be omitted. Git mode computes +the manifest from completed staging and binds it to the semantic Git-state record +for the resolved commits. OCI mode carries the same bytes in the configured +workspace-manifest blob and must reproduce them after applying the layers. The +runner receives the manifest through a private bounded result root, verifies its +digest, source-visible tree, and any omitted Git state independently, and never +places the manifest in source content. + +Before materialization, resolution computes a canonical generation key from every +input that can affect source-visible bytes, declared agent-visible filesystem +semantics, or sharing authorization: descriptor and hook contract versions, +deployment authorization scope, bounded selected credential-reference identities, +resolved commits or immutable OCI digests, normalized destinations, applicable +catalog identity, and acquisition policy. Physical paths, access, retention, +working directory, harness/profile identity, credential values, and volatile Git +administrative representation are excluded. Publication binds that key and one +internal epoch to one verified workspace-manifest digest and, in repository mode, +one semantic Git-state record. Materialize receives the exact private source-only +resolved-plan bytes/digest and selected reference set returned by resolve; it +never re-resolves source. + +The generation backing store is owner-writable and never exposed writable to a +session. Publication is a recoverable same-filesystem atomic transition. +Editable copies may use a safe copy or snapshot mechanism but may not share +mutable inodes with the generation. Corrupt, partial, quarantined, or deleting +generations are not attachable. Source credentials are selected server-side from an owner-only secret mount or credential-store handle available to the runner, not from the long-lived service -environment. The runner resolves exactly the selected value when it constructs -the materializer child environment; the gateway/runner base environment and -every agent child remain credential-free. The materializer must use hermetic -Git/registry configuration, prevent credentials from being persisted in Git -configuration or remote URLs, remove temporary credential state before -returning, and emit no secret value. If a configured source secret appears in -the service or agent environment, the runner refuses to launch the agent. +environment. After preflight declaration and validate selection, the runner +resolves only the exact selected values when it constructs a source-access hook +child environment; the gateway/runner base environment and every agent child +remain credential-free. The materializer must use hermetic Git/registry +configuration, prevent credentials from being persisted in Git configuration or +remote URLs, remove temporary credential state before returning, and emit no +secret value. If a configured source secret appears in the service or agent +environment, the runner refuses to launch the agent. ## Trust and deployment HarnessRouter API authentication is mandatory on every externally reachable UHP, -response/session retrieval, stream, cancellation, file, and artifact endpoint, -even on a private network. Gateway-to-runner operations are not externally +response/session retrieval, stream, cancellation, file, artifact, persistence, +deletion, and lifecycle-administration endpoint, even on a private network. +Unauthenticated requests disclose neither existence nor retention state. +Gateway-to-runner operations are not externally routable and are mutually authenticated. Operators should still bind the deployment to loopback or a private network and enforce Tailscale ACLs, firewall policy, or equivalent controls. Version one is not a public multi-tenant service. -HarnessRouter CE provides per-session operating-system identities and workspace -directories, not a hostile-code sandbox. Native harness OAuth therefore requires -an operator-owned, private deployment: agent tools sharing the harness identity -may access that harness's OAuth profile. Operators requiring stronger provider -credential isolation must use the explicit brokered proxy route or place the -complete deployment inside a stronger isolation boundary. +HarnessRouter CE provides per-session operating-system identities and private +runtime state, not a hostile-code sandbox. Immutable generations may be mounted +read-only into multiple session identities, but no session receives write access +to their backing store. Editable source state, harness homes, temporary files, +outputs, and checkpoints remain private to one session identity. + +Native harness OAuth therefore requires an operator-owned, private deployment: +agent tools sharing the harness identity may access that harness's OAuth profile. +Operators requiring stronger provider credential isolation must use the explicit +brokered proxy route or place the complete deployment inside a stronger +isolation boundary. The deployment uses a pinned custom HarnessRouter image containing: @@ -316,9 +540,23 @@ The deployment uses a pinned custom HarnessRouter image containing: - pinned HarnessRouter-supported Codex and Pi versions. The runtime grants only the runner a delegated cgroup v2 subtree and applies an -`on-failure` restart policy. Readiness stays false unless that delegation is -usable and startup has removed or quarantined every orphaned materializer -cgroup. +`on-failure` restart policy. The attested image, durable volumes, project +configuration, secret handle, and cgroup delegation are mounted in non-serving +initialization mode before preflight or reconciliation. Readiness stays false +unless that delegation is usable and startup has removed or quarantined every +orphaned materializer cgroup and credential projection. + +Readiness also requires finite session idle TTL; build/staging, generation, +per-editable-session hard byte/inode, total private reservation, session, +persistence, and tombstone quotas; a writable private staging and editable- +workspace volume; and a protected immutable generation store. The runner +validates their actual mount, same-filesystem publication, quota, and isolation +relationships; the hook does not. Readiness also requires successful startup +reconciliation of build waiters, provisional pins, references, quota usage, +mounts, credential projections, tombstone compaction, and interrupted deletion. +Operators must be able to observe aggregate generation, private-workspace, +persistent-session, tombstone, quarantine, and failed-deletion capacity without +receiving source paths or credentials. AllAgents publishes the `linux/amd64` release image as the public package `ghcr.io/allagentsdev/harnessrouter`. Version and commit tags are mutable @@ -337,7 +575,8 @@ Codex auth root or Pi `/login` against a dedicated Pi auth root during controlle setup. Codex uses file credential storage under `CODEX_HOME`; Pi uses `~/.pi/agent/auth.json`. Both harnesses own token refresh. Conversation and rollout state remain session-scoped, while refreshed OAuth state persists in the -selected auth root outside the workspace checkpoint. +selected auth root outside immutable generations and private workspace +checkpoints. The runner verifies the selected binding and a live turn before advertising the target: login status and refresh for native OAuth, or proxy configuration, @@ -361,58 +600,122 @@ automatically. ## Failure behavior -- **Invalid extension:** the AllAgents hook rejects it before acquisition. -- **Extension on a continuation:** reject without changing session state. -- **Unknown logical source or working directory:** fail before network access. +- **Invalid extension:** the source-free hook validation rejects unknown or + malformed source, access, retention, or logical-working-directory fields + before source resolution. +- **Unauthorized persistence:** after response allocation but before source + resolution or byte acquisition, the runner rejects + `retention: "persistent"` when deployment policy does not authorize it; the + hook never authorizes and the runner never silently downgrades it to `session`. +- **Read-only input overlay:** after response allocation but before source + resolution, reject workspace input files on a `readOnly` request. A runtime + write receives the filesystem's read-only failure and never causes copy-up or + mode conversion. +- **Extension on a continuation:** reject without changing session state, + acquiring a lease, or clearing its idle deadline. +- **Expired, deleted, or purged session:** while its bounded tombstone remains, + return `allagents_workspace_expired` before runner or profile work. After + tombstone and matching response/idempotency retention are purged, return the + stock non-disclosing unknown-predecessor error. Neither rematerializes source. +- **Non-resumable session:** a known attached session with missing or corrupt + bound generation key/epoch, reference, publication, private workspace, or + checkpoint evidence returns HTTP 409 `allagents_workspace_non_resumable` + before profile admission. It never substitutes a rebuilt epoch. Because the + attachment previously reached `ready`, the error includes its committed + complete public workspace metadata. - **Busy auth profile:** after atomic idempotency replay and stock `session_busy` precedence, a genuinely new cross-session turn fails immediately with HTTP 503 `harness_unavailable` and - `detail.reason: "allagents_auth_profile_busy"` before response allocation; - never queue it on the profile lock. -- **Source authentication or acquisition failure:** remove partial workspace - state, return the plan's cataloged UHP failure, and start no agent or provider - fallback. + `detail.reason: "allagents_auth_profile_busy"` before response allocation, + runner work, or materialization; sessions using other profiles may continue + concurrently. +- **Capacity exhaustion:** before response allocation, atomically reserve generic + session/tombstone capacity or return HTTP 503 + `allagents_workspace_capacity_exceeded` without exposing state. After + validation but before source resolution, reserve persistence and the one + editable hard byte/inode allowance. After resolve but before byte acquisition, + a miss claim reserves full staging and prospective generation byte/count + capacity. Independent post-build full-tree accounting atomically converts the + generation reservation to physical retained usage and releases excess and + staging before ready. Each editable waiter then independently proves the + initial copy fits its hard allowance; a non-fitting waiter fails + `allagents_workspace_private_quota_exceeded` without invalidating the epoch or + siblings. Evict only ready epochs with zero references and zero provisional + pins. Never evict protected state or alter access or retention. +- **Editable quota exhaustion:** fail an editable waiter whose initial copy does + not fit, or let the filesystem deny a later private-workspace write beyond its + reserved byte or inode allowance; the runner returns + `allagents_workspace_private_quota_exceeded`. Persistent sessions cannot exceed + the same fixed envelope; retries never change mode or retention. +- **Source authentication or acquisition failure:** remove only unpublished + staging, publish no generation, and start no agent or provider fallback. An + existing verified generation is not poisoned by a failed competing build. + Cancelling one build waiter detaches only it; other live waiters keep the + runner-owned build alive. +- **Generation, attachment, or private-copy failure:** quarantine corrupt or + incomplete state, release reservations and provisional pins exactly once, + acquire no live attachment, and fail closed. A session never substitutes + another generation after binding. Prepare/ack reconciliation preserves only a + gateway-committed ready attachment. - **Materializer timeout, cancellation, malformed result, crash, or live descendant after parent exit:** create a runner-owned cgroup v2 leaf and start the child inside it atomically with `clone3(CLONE_INTO_CGROUP)` or a stopped, secret-free pre-exec move-and-verify handshake. The child cannot escape or - administer the subtree; process groups and post-exec migration are - insufficient. On every outcome, use `cgroup.kill` when membership remains and - wait for `cgroup.events` to report `populated 0` before any terminal response - or event, private-manifest read, publication, secret release, or cleanup. A - completed parent with a live descendant returns the containment failure even - when forced kill succeeds. An unquiescent leaf enters internal non-terminal - `containment_pending`; the runner exits for required restart, and GET/stream - stay non-terminal until startup proves the old boundary empty. Only then expose - failed containment or the UHP-mandated `cancelled`/`incomplete` status. -- **Agent cancellation or timeout:** use HarnessRouter's UHP lifecycle and - cancellation behavior. -- **HarnessRouter restart:** preserve completed state from the durable volume; - fail interrupted turns without automatic replay. + administer the subtree. On every outcome, kill remaining members and wait for + `cgroup.events` to report `populated 0` before any terminal result, private + manifest read, generation publication, secret release, or cleanup. A completed + parent with a live descendant returns the containment failure. An unquiescent + leaf remains internal `containment_pending`; the runner exits, blocks readiness + and terminal visibility, and startup transitions once to the preserved failed, + cancelled, or incomplete outcome only after proving it empty. +- **Expiry or deletion race:** atomically fence new turns before unmounting or + deleting. Repeated deletion is idempotent; failed physical deletion remains + quarantined and counted against quota for retry. Tombstone compaction is + bounded, ordered, durable, and never precedes response/idempotency retention. +- **HarnessRouter restart:** preserve completed, unexpired or persistent state; + reconcile generic and provisional turn admission, staging/prospective- + generation reservations, publication/accounting, builds/waiters, provisional + pins, references, private reservation transfer and actual usage, attachment + prepare/ack, credential projections, tombstones/compaction, and cleanup before + readiness. Interrupted turns fail without automatic replay. - **Provider authentication failure:** fail the selected harness target without - switching OAuth profiles or activating the proxy/API-key route. + switching OAuth profiles or activating the proxy/API-key route. Refresh + disposition and projection teardown still complete before the profile lock is + released. - **Provider execution failure:** return HarnessRouter's normalized UHP failure without source fallback or credential material in public output. - **Public error mapping:** use UHP request errors before response allocation and terminal failed responses afterward. New codes carry the `allagents_` vendor prefix. Promptfoo maps every non-success to a coded error, never successful - empty output or an automatic retry. - -Failures report only verified provenance. Partial acquisition never appears as a -complete workspace identity. + empty output or an automatic retry. Failures before attachment `ready` omit + workspace metadata. Failures after `ready` include the same complete verified + public workspace object as success; internal epoch/reservation identifiers stay + private. Partial acquisition never appears as a complete workspace identity. ## Consequences -HarnessRouter is the execution control plane. AllAgents does not add a parallel -task/session store, streaming lifecycle, process supervisor, artifact service, -provider adapter, or Promptfoo-specific runtime. - -AllAgents owns workspace selection, deterministic Git/OCI materialization, -source credentials, provenance, the HarnessRouter integration patch, and -deployment documentation. The selected harness owns provider OAuth login and -refresh. Native OAuth deliberately places that profile inside the -operator-controlled harness trust boundary; source-acquisition credentials -remain isolated from the harness. +HarnessRouter remains the sole execution control plane. Its generation index, +session references, retention timestamps, tombstones, and deletion state are +subordinate workspace lifecycle state, not a parallel task/session API. +AllAgents does not add another streaming lifecycle, process supervisor, artifact +service, provider adapter, or Promptfoo-specific runtime. + +AllAgents owns workspace selection, deterministic Git/OCI generation +construction, source credentials, and provenance. HarnessRouter owns atomic +generation publication, read-only attachment, private editable copies, quotas, +expiry, deletion, and garbage collection. The selected harness owns provider +OAuth login and refresh. + +Read-only sessions avoid repeated byte acquisition and may run different +harness/profile bindings concurrently against one immutable generation. Editable +sessions consume a reserved private byte/inode envelope and never share +mutations. Persistent sessions, active read-only references, provisional pins, +and retained tombstones reduce available capacity, making crash-consistent +accounting, deterministic LRU tie-breaking, bounded compaction, and admission +operational requirements. Native OAuth deliberately exposes only the selected +turn-scoped profile projection inside the operator-controlled harness trust +boundary; source-acquisition credentials remain isolated from the harness and +every published generation. The integration requires a maintained fork and custom image. The fork must be rebased and tested against upstream releases until the generic seams are @@ -434,24 +737,42 @@ accepted or equivalent supported extensions exist. - **Upload every source file as UHP input files:** works for small regular-file snapshots, but loses exact symlink, mode, and OCI layer semantics and moves repository acquisition to every caller. +- **Rematerialize a private source tree for every trial:** is simple but repeats + network, CPU, and storage work for identical read-only jobs and prevents safe + concurrent reuse of verified immutable content. - **Wait for upstream before delivery:** makes the product schedule depend on a project we do not maintain. ## Deliberate limits Version one does not add evaluation datasets, scoring, assertions, automatic -retries, session branching, concurrent turns within one session, concurrent -refresh-capable turns sharing an auth profile, caller-supplied origins, public +retries, session branching, concurrent turns within one session, simultaneous +refresh-capable turns sharing one auth profile, caller-supplied origins, public multi-tenancy, arbitrary materializer commands, mutable OCI tags, transparent source-mode fallback, or guaranteed provider prompt-cache hits. +Read-only attachments never copy up or become editable. Editable workspaces never +share mutations across sessions. Callers cannot choose arbitrary TTLs, bypass +persistence quotas, or convert retention on continuation. Leased or pinned state +is never an eviction candidate, and retention is never unbounded by default. + ## Reconsider when Revisit this decision when: - either required harness-native OAuth target cannot pass the phase-zero gate; -- upstream HarnessRouter accepts the generic materializer hook or exposes an - equivalent supported extension; +- the host cannot enforce immutable read-only generation mounts across session + identities; +- continuation, expiry, deletion, and lease acquisition cannot be made + linearizable and crash-safe; +- generation churn or authorized persistent demand cannot fit practical bounded + quotas; +- editable derivation requires stronger filesystem semantics than a private copy + can provide; +- upstream HarnessRouter accepts the generic workspace lifecycle seam or exposes + an equivalent supported extension; +- UHP adopts a standard workspace attachment or retention contract that + supersedes the namespaced extension; - the maintained patch grows beyond the narrow integration boundary; - HarnessRouter changes or removes required UHP/session/provider behavior; - exact per-turn workspace rollback becomes a product requirement; diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index e4cb20e5..4fe3e8ba 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -14,52 +14,61 @@ execution: code ## Goal Capsule - **Objective:** Let Promptfoo and other authenticated UHP clients run a - configured Codex or Pi harness against an AllAgents Git workspace or immutable - OCI workspace snapshot, continue the same conversation and writable workspace - with `previous_response_id`, and receive source provenance, output, usage, and - artifacts. + configured Codex or Pi harness against a verified AllAgents Git or OCI + workspace generation. Concurrent read-only trials may share that immutable + generation across harnesses and profiles; each editable trial receives a + private writable workspace. A continuation reuses the same session attachment + through `previous_response_id`. Default sessions expire under bounded policy; + explicitly authorized persistent sessions remain pinned until deletion. - **Means:** Deploy a pinned HarnessRouter CE fork. Preserve HarnessRouter's UHP, caller authentication, session, streaming, cancellation, artifact, and - agent-runner behavior. Add a generic first-turn materializer boundary, nested - working-directory support, durable materialization state, nested-repository - checkpoint/collection support, and a separation between session state and - durable harness-native OAuth state. Implement Git/OCI semantics in a separate - AllAgents executable. Codex authenticates through `codex login`; Pi + agent-runner behavior. Add a generic generation resolve/build/attach boundary, + immutable generation store, read-only mounts, private editable copies, durable + leases, retention and quota state, garbage collection, nested logical working + directories, mode-specific checkpoint/collection behavior, and separation + between session state and durable harness-native OAuth state. Implement + Git/OCI semantics in a separate AllAgents executable. Codex authenticates + through `codex login`; Pi authenticates through its `/login` flow for the selected provider. An API-key-authenticated proxy is an explicit last-resort target mode. Optional describes deployment configuration, not release scope: version one implements and verifies it for operators that reject the native owner-trust boundary. - **Authority:** [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) - owns the protocol, fork, trust, workspace, harness-authentication, and - provider-routing decisions. UHP `2026-09-12` and HarnessRouter's conformance - suite own execution-wire behavior. The project `workspace.yaml` owns logical - Git/OCI sources and environment-variable credential references; deployment - secrets provide the values. HarnessRouter configuration owns harness IDs, - model allowlists, and authentication bindings. The namespaced - AllAgents extension owns acquisition and provenance semantics. + owns the protocol, fork, trust, generation, attachment, retention, + harness-authentication, and provider-routing decisions. UHP `2026-09-12` and + HarnessRouter's conformance suite own execution-wire behavior. The project + `workspace.yaml` owns only the logical Git/OCI source catalog and + environment-variable credential references; deployment secrets provide the + values. HarnessRouter configuration owns harness IDs, model allowlists, + authentication bindings, persistence authorization, finite TTLs, quotas, and + garbage-collection policy. The namespaced UHP JSON extension owns per-session + source selection, access, retention request, logical cwd, and provenance + semantics; it is not `workspace.yaml`. - **Execution order:** First build the minimal custom image and pass the blocking native-auth adapter gate for both Codex and Pi without implementing the - AllAgents materializer. Only then prove the workspace fork seam, - materialization state machine, and checkpoint integration against the real - runner; freeze the generic hook and AllAgents contracts; implement Git then - OCI acquisition; prove an explicit proxy mode separately; run Promptfoo - one-shot and continuation E2E; complete release, fork-maintenance, and - upstream-ready documentation. + AllAgents materializer. Then prove the generation claim/publication seam, + shared read-only attachment, private editable derivation, lease fencing, and + retention state against the real runner. Freeze the generic hook and + AllAgents contracts; implement Git then OCI generation construction; implement + TTL/pinning, quotas, deletion, and garbage collection; prove explicit proxy + mode separately; run concurrent read-only, editable, continuation, expiry, and + Promptfoo E2E; complete release, fork-maintenance, and upstream documentation. - **Stop conditions:** Stop before production workspace implementation if either required native target cannot pass the phase-zero gate: real login, first turn, - continuation, binding persistence, profile isolation, serialized overlapping - turns, refresh fault behavior, and passive-persistence checks. Also stop if - materialization cannot complete and durably checkpoint before provider - dispatch, nested Git workspaces cannot be collected without corrupting - HarnessRouter checkpoints, source credentials enter the harness, the - gateway/runner automatically copies harness OAuth files into a materialized - source tree, checkpoint, produced-file record, passive log, or public metadata, - local OAuth state cannot be persisted atomically and validated fail-closed - while conversation state remains session-scoped, or the fork cannot preserve - stock UHP behavior and conformance. Do not fall back to prompt instructions, - an MCP acquisition tool, client-side repository upload, another OAuth profile, - an implicit API-key route, a second execution protocol, or a parallel - task/session engine. + continuation, binding persistence, profile isolation, mutually exclusive + overlapping turns, refresh fault behavior, and passive-persistence checks. + Also stop if the host cannot enforce immutable multi-session read-only mounts; + concurrent identical requests can publish more than one generation; editable + sessions can alias writable state; continuation, lease, expiry, deletion, and + GC cannot be fenced crash-safely; protected state can be evicted; retained + storage cannot be bounded; nested Git workspaces cannot be collected without + corrupting HarnessRouter checkpoints; source credentials enter a generation or + harness; the gateway/runner serializes harness OAuth files into source, + checkpoints, produced records, passive logs, or public metadata; or the fork + cannot preserve stock UHP behavior. Do not fall back to prompt instructions, + an MCP acquisition tool, client-side repository upload, another generation, + access or retention mode, OAuth profile, implicit API-key route, second + execution protocol, or parallel task/session engine. - **Tail ownership:** Implementation owns focused tests in both repositories, upstream UHP conformance, built-image smoke tests, exact Git/OCI E2E, native Codex/Pi OAuth and explicit proxy-mode E2E, two-turn Promptfoo success and @@ -77,19 +86,21 @@ HarnessRouter exposes UHP, authenticates callers, creates and persists sessions, streams events, runs configured Codex and Pi harnesses, handles cancellation and idempotency, and returns output, usage, and artifacts. -The missing product-specific capability is deterministic workspace acquisition -before the first agent turn. A focused HarnessRouter fork calls a generic -materializer boundary after allocating the session workspace but before provider -selection. The AllAgents executable validates the product-specific descriptor, -writes and validates staging, and returns path-free provenance. The runner -publishes staging, initializes HarnessRouter and nested-repository checkpoints, -persists a pre-agent checkpoint, and only then permits provider dispatch. +The missing product-specific capability is deterministic source-generation +resolution before the first agent turn. A focused HarnessRouter fork calls a +generic hook after allocating the UHP session but before provider selection. The +AllAgents executable validates the JSON descriptor against the project +`workspace.yaml`, resolves an immutable source plan, and builds verified staging +only on a generation cache miss. The runner atomically publishes or reuses the +generation, records the session attachment and retention state, then mounts it +read-only or creates a private editable copy before provider dispatch. A continuation supplies `previous_response_id`, omits the workspace extension, -and uses HarnessRouter's current native conversation and writable session -workspace. A different revision, snapshot, or working directory requires a new -session. AllAgents does not add stricter predecessor-head or branching semantics -beyond HarnessRouter's UHP behavior. +and uses HarnessRouter's current native conversation plus the bound attachment. +Read-only sessions see the same immutable generation; editable sessions see the +same private mutations. A different source revision, working directory, access +mode, retention class, harness, or authentication binding requires a new +session. An expired or deleted session is never silently rematerialized. ### Problem Frame @@ -110,30 +121,38 @@ caller responsible for acquisition. The temporary fork closes those seams. - **A1. UHP caller:** Promptfoo or another application holding a HarnessRouter API key. It chooses a configured HarnessRouter harness ID and model, prompt, initial workspace descriptor, and optional continuation predecessor. -- **A2. HarnessRouter gateway:** Authenticates and validates UHP, owns response - and session identity, treats the configured workspace metadata value as - bounded opaque JSON, drives materialization before provider fallback, and - returns hook metadata on every response path. +- **A2. HarnessRouter gateway:** Authenticates and validates UHP; owns response + and session identity; is the sole writer of session attachment, expiry, and + tombstone state; treats the configured workspace metadata value as bounded + opaque JSON; drives prepare/ack before provider fallback; and returns hook + metadata on every response path. - **A3. AllAgents materializer:** A subprocess executable that exposes no - listening service, invoked before the first turn. It validates the AllAgents - descriptor, reads the mounted project workspace configuration, acquires Git or - OCI sources into staging, validates the tree, and returns provenance. -- **A4. HarnessRouter runner:** Owns the per-session operating-system identity, - publication, checkpoints, nested-repository collection, input files, safe - nested cwd, selected Codex/Pi process, session conversation state, and - projection of the selected durable auth profile. + listening service. It validates the AllAgents descriptor, reads the project + `workspace.yaml`, resolves immutable Git/OCI source plans, builds private + staging on cache misses, validates the tree, and returns generation identity + and provenance. It never authorizes persistence or publishes live state. +- **A4. HarnessRouter runner:** Owns generation claims/publication and the + resource journal: provisional pins, durable references, read-only mounts, + private editable copies and quotas, per-session operating-system identity and + runtime state, mode-specific checkpoints and produced files, safe nested cwd, + selected Codex/Pi process, conversation state, active-turn auth projection, + cleanup, and garbage collection. It prepares attachment evidence but never + writes gateway session attachment/expiry/tombstone transitions. - **A5. Harness-native auth profile:** One dedicated durable credential root for one Codex or Pi harness target. The harness owns login and token refresh. The - gateway lifecycle never copies its files into workspace checkpoints or public - metadata; active harness access is part of the owner-trust boundary. + runner projects it only for an active turn and verifies teardown before + acknowledgement; checkpoints, backups, and public metadata never copy it. + Active harness access is part of the owner-trust boundary. - **A6. Optional authenticated proxy:** A last-resort, explicitly configured target mode. The gateway keeps the long-lived proxy client key, the proxy owns upstream provider authentication, and the harness receives only a non-refreshable, scoped turn credential that the HarnessRouter broker validates. -- **A7. Operator:** Pins and deploys the custom image, mounts durable data and - project workspace configuration, completes each native harness login, supplies - deployment-only source credential values, selects any explicit proxy targets, - and controls private-network access. +- **A7. Operator:** Pins and deploys the custom image, mounts durable generation, + session, editable-workspace, and auth storage, completes each native harness + login, supplies deployment-only source credentials, authorizes persistent + sessions, configures finite TTL/byte/inode/count/tombstone quotas, operates + deletion and GC, selects explicit proxy targets, and controls private-network + access. ### Key Decisions @@ -141,15 +160,25 @@ caller responsible for acquisition. The temporary fork closes those seams. northbound execution contract. HarnessRouter conformance is authoritative. - **Fork narrowly and upstream later.** Delivery uses an AllAgents-maintained fork. The upstreamable layer is a configured opaque-metadata key, immutable - first-turn binding, typed command envelope, durable pre-provider lifecycle, - safe nested cwd, checkpoint/collection integration, and separation of durable - harness-auth state from session state. It contains no AllAgents Git/OCI schema - logic. Upstream acceptance is not critical-path. + first-turn binding, typed validate/resolve/materialize envelopes, generation + claim/publication, attachment prepare/ack and lease lifecycle, safe nested cwd, + mode-specific checkpoint/collection integration, bounded retention/GC, and + separation of durable harness-auth state from session state. It contains no + AllAgents Git/OCI schema logic. Upstream acceptance is not critical-path. - **Run the AllAgents component behind HarnessRouter.** The materializer is a subprocess hook, not another HTTP gateway and not a custom agent backend. -- **Materialize once per extension-bearing session.** An extension-bearing - initial request creates and checkpoints the workspace. Continuations must omit - the extension and reuse the session through `previous_response_id`. +- **Publish once per generation; attach once per session.** Concurrent requests + for one immutable source plan share one claim and verified publication. + Read-only sessions share that generation; editable sessions receive private + writable copies. Continuations omit the extension and reuse the original + attachment through `previous_response_id`. +- **Separate access from retention.** `readOnly` versus `editable` controls + mutability. Default `session` versus authorized `persistent` controls + lifetime. Neither axis changes the other, and continuation can change neither. +- **Bound retained state.** Active leases, durable references, provisional pins, + and persistent sessions are protected. Expired state and bounded tombstones are + purged before deterministic eviction of unreferenced/unpinned generations. + Admission fails when protected state consumes finite quota. - **Keep source authority server-side.** Callers select logical source names and revisions but cannot send origins, credentials, host paths, commands, or Docker options. @@ -178,9 +207,10 @@ caller responsible for acquisition. The temporary fork closes those seams. `2026-09-12`. The deployment must pass the applicable upstream conformance suite without weakening, replacing, or reinterpreting stock UHP behavior. - **R2.** Require a HarnessRouter API key for every externally reachable UHP, - response/session retrieval, stream, cancellation, file, and artifact endpoint. - Reject unauthenticated requests before disclosing resource existence or - metadata. Gateway-to-runner operations are not externally routable and are + response/session retrieval, stream, cancellation, file, artifact, persistence, + deletion, and lifecycle-administration endpoint. Reject unauthenticated + requests before disclosing resource existence, expiry, retention, or metadata. + Gateway-to-runner operations are not externally routable and are mutually authenticated. Bind the service to loopback or a private interface and document the remaining need for Tailscale ACLs, firewall policy, or equivalent network controls. @@ -206,140 +236,275 @@ caller responsible for acquisition. The temporary fork closes those seams. runs Pi `/login` in an isolated Pi home for the configured provider. The runner projects only the selected profile's exact auth files into the session-specific CLI home and keeps conversation/rollout state session-scoped. - The projection must preserve the harness's credential-file write and - atomic-replacement behavior. A locally committed refresh uses a same-filesystem - temporary file, file `fsync`, atomic rename, parent-directory `fsync`, and - validation. A crash after the provider rotates credentials but before local - commit can leave the profile stale; restart then marks it `repair-required` - and requires native login instead of changing profile or auth mode. The - gateway/runner must not automatically serialize auth files into root or nested - checkpoints, produced-file records, passive logs/traces, materializer input, - or response metadata. Native mode sets explicit owner trust because the - harness and tool subprocesses sharing its operating-system identity may read - or emit that credential. In `nativeOAuth`, version one supports exactly one - active refresh-capable turn per profile and holds that profile lock for every - turn and every login, logout, or repair operation. Admission preserves UHP - precedence with an atomic `Idempotency-Key` claim around lookup and admission. - The single claim owner proceeds; simultaneous same-key arrivals wait on that - claim and receive the owner's result without a second profile-lock attempt. If - the owner fails before response allocation, the gateway publishes that same - request error to current waiters and removes the claim so a later retry can try - again. A new turn in an already-active session returns `session_busy`; only - then does a genuinely new executable turn try the native profile lock. - Cross-session collision returns HTTP 503 `harness_unavailable` with + The projection exists only for the active turn. It uses a directory-level + mount namespace or an equivalently isolated same-filesystem view that preserves + the harness's credential-file write and atomic-replacement behavior; it never + copies credentials into durable session state. + + A locally committed refresh uses a same-filesystem temporary file, file + `fsync`, atomic rename, parent-directory `fsync`, and validation. A crash after + the provider rotates credentials but before local commit can leave the profile + stale; restart then marks it `repair-required` and requires native login + instead of changing profile or auth mode. After the harness and descendants + stop, but before terminal acknowledgement or profile-lock release, the runner + commits or rejects refresh state, unmounts and removes the projection, and + verifies that credential paths are absent from the retained CLI home. Startup + removes or quarantines stale projections before readiness or profile + reacquisition. Success, failure, cancellation, crash repair, expiry, deletion, + checkpoint, backup, and produced-file paths all preserve this boundary. + + The gateway/runner must not automatically serialize auth files into root or + nested checkpoints, produced-file records, passive logs/traces, materializer + input, backup, or response metadata. Native mode sets explicit owner trust + because the harness and tool subprocesses sharing its operating-system + identity may read or emit that credential. In `nativeOAuth`, version one + supports exactly one active refresh-capable turn per profile and holds that + profile lock for every turn and every login, logout, or repair operation. + Admission preserves UHP precedence with an atomic `Idempotency-Key` claim + around lookup and admission. The single claim owner proceeds; simultaneous + same-key arrivals wait on that claim and receive the owner's result without a + second profile-lock attempt. If the owner fails before response allocation, + the gateway publishes that same request error to current waiters and removes + the claim so a later retry can try again. A new turn in an already-active + session returns `session_busy`; only then does a genuinely new executable turn + try the native profile lock. Cross-session collision returns HTTP 503 + `harness_unavailable` with `detail.reason: "allagents_auth_profile_busy"` before response allocation, runner work, or materialization. No per-profile waiter queue exists; the UHP idempotency claim wait is part of one logical request, not such a queue. - Admission is a private runner operation: the runner turn supervisor persists an - active-profile admission record, takes the operating-system advisory lock, and - returns an opaque admission token before the gateway allocates a response. The - runner—not the gateway—owns that lock through descendant termination and - refresh commit. A gateway-only crash therefore leaves the lock held; restart - reconciles the token and active runner before admitting another turn. The - runner releases only after the gateway acknowledges durable terminal - response/state and the runner has either durably committed native refresh state - or marked the profile `repair-required`. If the runner process dies, the OS - releases the lock, but its durable admission record keeps readiness/admission - closed until startup proves all descendant boundaries empty and validates or - repairs the profile. Ordinary and administrative paths use one `finally` - release/ack protocol. `proxyApiKey` uses no native profile lock. - Operators provision distinct native profiles when they require parallel turn - capacity. Preserve HarnessRouter streaming, - cancellation, idempotency, files, artifacts, completed-session persistence, - and per-session workspace/UID isolation. + For a continuation, the gateway's one session CAS checks `session_busy`, + attachment/binding evidence, and expiry/deletion together. On success it + records the original idle deadline in a provisional turn-admission fence, + clears that deadline, and marks the session admission-pending before the runner + tries the zero-waiter profile lock. A same-session collision changes nothing. + If profile admission fails before response allocation, the gateway rolls the + fence back: it restores the exact original deadline when still future, or + tombstones the session when that deadline has elapsed. Only a returned profile + admission token commits the fence to active. + + Admission is a private runner operation: the runner turn supervisor persists + an active-profile admission record, takes the operating-system advisory lock, + and returns an opaque admission token before the gateway allocates a response. + The runner—not the gateway—owns that lock through descendant termination, + refresh commit, and projection teardown. A gateway-only crash therefore leaves + the lock held; restart reconciles the token and active runner before admitting + another turn. The runner releases only after the gateway acknowledges durable + terminal response/state, and after the runner has committed native refresh + state or marked the profile `repair-required` and proved the projection absent. + If the runner process dies, the OS releases the lock, but its durable admission + record keeps readiness/admission closed until startup proves all descendant + boundaries empty, removes stale projections, and validates or repairs the + profile. Ordinary and administrative paths use one `finally` release/ack + protocol. `proxyApiKey` uses no native profile lock. Operators provision + distinct native profiles when they require parallel turn capacity. Preserve + HarnessRouter streaming, cancellation, idempotency, files, artifacts, + retention-bounded completed session persistence, per-session UID/runtime + isolation, and immutable generation sharing. #### Workspace extension and hook - **R5.** On an initial response request, accept one optional JSON object of at most 64 KiB and 32 levels at `metadata["allagents.workspace"]`. - HarnessRouter checks only those generic bounds, canonicalizes the opaque value - with RFC 8785, and binds its digest to the new session. A continuation must - omit this key. The AllAgents hook validates the exact v1 object - `{ version: "1", source, workingDirectory? }`; `source` is exactly + HarnessRouter checks only generic bounds, canonicalizes the opaque value with + RFC 8785, records the raw request descriptor digest, and binds it to the new + session. A continuation must omit this key; generic gateway validation rejects + an extension-bearing continuation before session lookup/CAS and changes no + session state or deadline. The AllAgents hook's source-free `validate` + operation validates and defaults the exact v1 object + `{ version: "1", access, retention?, source, workingDirectory? }`. `access` is + exactly `readOnly | editable`; omitted `retention` means `session`, otherwise + it is exactly `session | persistent`. `source` is exactly `{ kind: "repositories", revisions?: Record }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, workspaceManifestDigest: Digest }`; `workingDirectory` is exactly `{ kind: "workspaceRoot" }` or `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }`. -- **R6.** The AllAgents hook expands omitted `revisions` to `{}` and omitted - `workingDirectory` to `{ kind: "workspaceRoot" }`; an omitted repository - `path` remains absent and an empty path is invalid. It NFC-normalizes strings, - sorts maps, rejects unknown fields, and hashes RFC 8785 bytes as the effective - descriptor digest. Project-config default refs affect resolved provenance, not - this request digest. Accepted/rejected fixtures prove omitted and - explicit-default forms canonicalize identically. -- **R7.** Add one runner-side `/materialize` operation outside the provider - candidate loop. It invokes a configured executable directly without a shell - after fresh-session hydrate and before `/turn`. Pass at most 128 KiB on stdin, - accept at most 1 MiB on stdout and 64 KiB on stderr, and use the smaller of - 900 seconds or the remaining UHP deadline. The generic request contains the - opaque metadata value, session workspace, fixed sibling staging and private - result roots, project configuration root, and deadline. Source values come - from an owner-only runner secret mount or credential-store handle, never the - gateway/runner base environment. The runner resolves only the selected handle - and constructs the allowlisted materializer child environment. Startup - preflight blocks readiness if a configured secret name or value appears in the - service or agent environment. The per-request recheck, after allocation but - before child or agent launch, returns failed - `allagents_secret_boundary_violation` on the same condition. Before injecting - secrets, the runner creates a - per-materialization cgroup v2 leaf under a runner-owned delegated subtree and - starts the child inside it atomically with `clone3(CLONE_INTO_CGROUP)`. Where - that primitive is unavailable, it forks a child with no secret material and - holds it at a pre-exec handshake; the parent moves it to the leaf, verifies the - exact membership through `/proc//cgroup`, then delivers the one-shot secret - bundle and releases the child to construct its environment and `exec`. The - stopped child cannot execute user code or fork before verified membership. The - child cannot write the parent `cgroup.procs` or administer the subtree; a - process group or post-exec PID migration is insufficient. - On every outcome, including a parent that returns `completed`, the runner - closes the hook streams, checks - `cgroup.events`, uses `cgroup.kill` when membership remains, and waits a bounded - interval for `populated 0`. It does not accept success, read the private - manifest, validate or publish staging, release the secret environment, or - remove roots before the cgroup is empty. Membership remaining after a parent - reports `completed` is itself failed - `allagents_workspace_containment_breach`, even when `cgroup.kill` subsequently - reaches `populated 0`; that recovered violation does not require runner exit. - If the cgroup cannot become empty, the runner reports an internal - `containment_pending` reason. The gateway CASes only the internal session state - to `containment_pending`/non-resumable and acknowledges that receipt; it emits - no terminal stream event and GET continues to show non-terminal `in_progress`. - The runner exits nonzero after that acknowledgement or a bounded - acknowledgement deadline. The required `on-failure` restart policy destroys - the old container boundary; startup keeps readiness false, removes or - quarantines orphaned cgroups, and reports `containment_reconciled` only after - the old boundary is proven empty. Only then may the gateway terminalize and - expose the response: client cancellation becomes `cancelled`, declared-budget - exhaustion becomes `incomplete`, and every other case becomes failed - `allagents_workspace_containment_breach`. No terminal response/state, terminal - event, result read, publication, secret release, or cleanup becomes observable - before that proof. The typed hook result is `completed` with effective relative - cwd, a private workspace-manifest reference and digest, and bounded public - metadata, or `failed` with a cataloged code, safe message, and retryability. - Materializer failure never enters provider fallback. -- **R8.** Persist a CAS-protected session materialization state: - `unbound -> materializing -> ready`, `failed`, or internal - `containment_pending -> failed`, plus the resolved harness target, auth mode, - native-profile or proxy-connection identity, and canonical - binding-config digest. The runner validates the private workspace manifest and - successful staging independently, publishes staging with a recoverable - same-filesystem rename protocol, removes the private result root, writes a - descriptor/provenance marker, initializes the HarnessRouter root checkpoint - and nested-repository collection baselines, and returns `published` with the - checkpoint digest. Before provider dispatch, the gateway stores that digest - and public metadata and CASes the session to `ready`. - On restart in `materializing`, reconcile to `ready` only when the workspace - marker and durable checkpoint match the bound descriptor; otherwise mark the - session non-resumable, remove or quarantine the workspace, and never replay + The hook validates and reports the requested retention but never authorizes it. + The runner is the sole persistence authority: before source resolution or byte + acquisition it authorizes `persistent`, reserves the session and persistence + slots, or fails `allagents_workspace_persistence_forbidden`. +- **R6.** The AllAgents hook expands omitted `retention` to `session`, omitted + `revisions` to `{}`, and omitted `workingDirectory` to + `{ kind: "workspaceRoot" }`; an omitted repository `path` remains absent and an + empty path is invalid. It NFC-normalizes strings, sorts maps, rejects unknown + fields, and hashes RFC 8785 bytes as the effective descriptor digest. A + separate canonical generation key covers only inputs that can affect + source-visible bytes, declared agent-visible filesystem semantics, or sharing + authorization: hook/schema versions, deployment authorization scope, bounded + selected credential-reference identities, resolved commits or OCI digests, + normalized destinations, catalog identity, and acquisition policy. Access, + retention, logical cwd, harness/profile, session identity, physical paths, + credential values, and volatile Git administrative representation do not + fragment that key. Publication binds it to the independently verified + workspace-manifest digest and semantic Git record when applicable. + Omitted and explicit default values have the same effective descriptor digest. + The raw request descriptor digest records the exact initial JSON only in + private session state; public response metadata names and returns only + `effectiveDescriptorDigest`. +- **R7.** Replace the one-shot session materialization call with one private + runner `/workspace/prepare` operation outside the provider candidate loop. It + invokes a configured executable directly without a shell using typed + `validate`, `resolve`, and `materialize` commands. `validate` performs only + source-free schema/default/catalog checks and returns a private normalized + descriptor reference/digest plus effective access, requested retention, + logical cwd, effective descriptor digest, and a bounded sorted list of selected + credential-reference names/opaque IDs—not values. The runner verifies those + references were declared by preflight and that their handles exist, then maps + only that selected set into source-access child environments. After runner + authorization and admission, `resolve` consumes that exact validated + descriptor and selected credential-reference set, resolves exact Git commits + or OCI identity, and returns a private canonical resolved-plan path/digest, + generation key, effective cwd, and bounded public provenance. `materialize` + receives that exact resolved-plan path/digest and selected set and never + re-resolves source. + + For each ready lookup or completed build, the runner validates ready evidence + and acquires a durable provisional attachment pin under the same generation + lock before returning it to one session. A miss creates one runner-owned keyed + build claim with a configured maximum lifetime of at most 900 seconds, + independent of any one UHP request deadline. The claim atomically takes + ownership of the validated source-only resolved-plan bytes and selected + credential-reference identities in its private durable root and binds their + digest to the key before acquisition. + Concurrent sessions attach as waiters to that claim. Each waiter applies its own + cancellation and deadline; cancellation detaches only that waiter, the build + continues while any live waiter remains, and the runner cancels and cleans the + build when none remain. All live waiters receive the one publication or build + failure without making one request's shorter deadline authoritative for the + others. + + The validate request carries the bounded ordinary workspace-input-file count + so `readOnly` fails before source access. Before response allocation, the + idempotent admission transaction reserves one generic workspace-session slot + and one fixed-size tombstone slot; it publishes neither token nor response + unless both are durable. These reservations remain through post-allocation + validation failure, failed-response retention, tombstoning, and purge. After + validate but before resolve, the runner authorizes and reserves any + persistence slot and, for `editable`, creates one stable private-reservation ID + for the full configured per-session byte/inode allowance. + + Before a miss claim acquires source bytes, its owner reserves the full + configured staging and prospective-generation byte/count allowance. After + materialization, the runner's independent no-follow full-tree walk, including + separately validated Git administrative state, computes physical retained + byte/inode usage. Under the generation lock, successful publication atomically + converts the prospective generation reservation to actual usage, releases its + excess and the staging reservation, and persists that accounting transition + before ready state or waiter pins become visible. Private copy-fit is not a + shared-build condition: after publication, each editable waiter compares total + physical generation bytes/inodes with its own hard allowance. A waiter that + cannot fit fails `allagents_workspace_private_quota_exceeded` and releases only + its access-specific reservations/pin; read-only and fitting editable waiters + continue. If none remain, the valid ready generation has zero pins and is GC + eligible. Build failure occurs before publication or attachment and before + agent launch; unpublished staging and access/build-specific reservations/pins + release exactly once, while generic session/tombstone reservations follow + failed-response lifecycle. + + Hook stdin is at most 128 KiB, stdout 1 MiB, and stderr 64 KiB. Source values + come from an owner-only runner secret mount or credential-store handle, never + the gateway/runner base environment. Preflight returns bounded configured + credential-reference identities but receives no values; the runner verifies + handle presence itself. For each source-access operation it resolves only the + selected validated references and constructs the allowlisted child + environment. Startup and per-request rechecks reject configured secret names + or values in the service or agent environment. + + Before injecting secrets, the runner creates a per-hook cgroup v2 leaf under a + delegated subtree and starts the child inside it atomically with + `clone3(CLONE_INTO_CGROUP)` or a stopped, secret-free pre-exec + move-and-verify handshake. On every outcome it closes streams, uses + `cgroup.kill` when needed, and proves `cgroup.events` reports `populated 0` + before accepting success, reading private results, publishing a generation, + releasing secrets, or cleaning roots. A completed parent with a live + descendant fails `allagents_workspace_containment_breach`. An unquiescent leaf + enters internal `containment_pending`; the runner exits, restart keeps + readiness false, and no terminal result becomes public until the old boundary + is proven empty. + + A completed materialize result contains only generation-scoped data: the + generation key, private `workspace-manifest.json` reference/digest, declared + repository roots, semantic Git validation records when applicable, and bounded + verified source identity. Each waiter retains its own effective descriptor + digest, logical cwd, access/retention, and requested-source provenance from + validate/resolve; a shared build result never overwrites them. The runner + independently validates the result, staged tree, manifest, and key, then makes + the generation backing tree owner-writable only and atomically publishes it. A + hook failure never enters provider fallback. +- **R8.** Persist separate CAS-protected state machines. A generation epoch is + `absent -> building -> ready`, `quarantined`, or `deleting`; it stores its key, + unique internal epoch ID, `publishedAt`, nullable `lastUsedAt`, manifest digest, + publication/accounting marker, physical byte/inode counts, durable reference + count, provisional attachment pins, and build waiters. There is at most one + live publication per key/epoch and one result per concurrent claim. A new epoch + for the same key may begin only after the prior epoch's durable logical and + physical eviction completes; it serves new sessions only. Attachments bind key + plus epoch, so an existing session never substitutes a rebuilt epoch. + + A gateway-owned session attachment is + `unbound -> validating -> resolving -> attaching -> ready`, + `containment_pending`, `expired`, `deleting`, `deleted`, `purged`, or `failed`; + it stores raw/effective descriptor digests, generation key/epoch, access, + retention, logical cwd, expiry, harness/auth binding, runner-supplied + attachment evidence, and any provisional turn-admission fence with its original + deadline. The gateway is the sole writer of session attachment, expiry, turn- + admission-fence, and tombstone transitions. The runner alone writes its + generation and resource journal. + + Before provider dispatch, the runner prepares resources and durably returns an + opaque attachment token and evidence; it does not bind the gateway session. + The gateway CASes `attaching -> ready`, persists that evidence, and + acknowledges the token. For `readOnly`, the runner holds the provisional pin + while it acquires a session-long generation-epoch reference and verifies a + read-only mount with no writable alias; after the ready acknowledgement it + releases the provisional pin exactly once. For `editable`, it requires the + stable private-reservation ID admitted before resolve and compares the + generation's independently measured total physical bytes/inodes with that + allowance. Failure detaches only that waiter. A fitting waiter creates and + validates a unique writable copy with no mutable inode shared with the + generation and initializes root/nested checkpoints and collection baselines. + Its attachment evidence contains the epoch and opaque reservation ID. Ready + acknowledgement transfers the reservation from admission to the private + workspace without a second debit, then releases the provisional pin exactly + once. Failure before acknowledgement releases the reservation and prepared + resources once unless reconciliation proves that the gateway committed ready. + Expiry or deletion releases the ready workspace's reservation once. Startup + reconciles both halves of this prepare/ack and quota-transfer protocol. + + The editable hard quota covers the private tree, UHP input overlays, + root/nested checkpoints, and produced-file state for every turn and + continuation. The filesystem quota backend must deny writes beyond either + byte or inode allowance and surface exhaustion to the runner; the runner + terminates that turn as failed `allagents_workspace_private_quota_exceeded` + without changing access or retention. Actual usage and reserved allowance are + persisted and reconciled before readiness. Only editable state receives + ordinary UHP input files, mutation checkpoints, and produced-file collection. + A `readOnly` request containing workspace input files fails before source acquisition. - Every continuation resolves the persisted auth binding by identity and digest; - if unavailable or changed, it fails before runner work instead of selecting a - replacement. Provider fallback sees `ready` state only and cannot invoke the - hook. Ordinary input files are applied only afterward. A validated logical cwd - may be the root or a symlink-safe descendant; the runner derives UID isolation - from the session root and rejects cross-session or escaping paths. + + `lastUsedAt` remains null until the gateway commits an attachment `ready`. + After acknowledgement, the runner updates it under the generation lock to + `max(existing, readyCommitTimestamp)`; startup can replay a missed update + idempotently from committed gateway evidence. GC orders null `lastUsedAt` + epochs first by `publishedAt`, generation-key bytes, and epoch-ID bytes; then + non-null epochs by `lastUsedAt`, `publishedAt`, key bytes, and epoch-ID bytes. + Publication or failed prepare does not count as use. + + A continuation requires attachment `ready`, the exact generation key/epoch, + access, retention, harness, and auth binding, and either persistent retention + or a `session` idle deadline strictly later than the admission instant. One CAS + also rejects `session_busy`, saves and clears that deadline in a provisional + admission fence, and marks admission pending. Profile success commits active; + pre-allocation profile failure restores the saved future deadline or tombstones + the session if it elapsed. Only durable terminal acknowledgement starts a new + idle deadline. GET, stream polling, and idempotent replay do not renew it. + Missing or corrupt attachment evidence fails non-resumable without acquisition + replay or replacement. Provider fallback sees only `ready` state and cannot + resolve, build, attach, or change policy. + The runner resolves a symlink-safe logical cwd + beneath the mounted generation or private workspace and rejects cross-session + or escaping paths. #### Source acquisition and provenance @@ -354,14 +519,18 @@ caller responsible for acquisition. The temporary fork closes those seams. execution-eligible repositories. Secret values remain deployment-only. A repository's logical name is explicit `name` or the portable basename of normalized `path`. Execution-eligible Git destinations must be unique, - non-empty, non-root relative child paths so their `.git` directories cannot - collide with HarnessRouter's root checkpoint repository. Reuse one shared - source resolver: `source` as a supported HTTPS URL is complete when `repo` is - absent; otherwise `source` names the supported host/provider and `repo` names - its repository. Conflicting forms, local/originless entries, duplicate - repository names, escaping destinations, and unsupported schemes make - materializer preflight fail. HarnessRouter owns harness/model/provider targets, - and the user workspace is not a materialization catalog. + pairwise non-overlapping, non-empty, non-root relative child paths so their + source trees and `.git` directories cannot collide with one another or with + HarnessRouter's root checkpoint repository. Reuse one shared source resolver: + `source` as a supported HTTPS URL is complete when `repo` is absent; otherwise + `source` names the supported host/provider and `repo` names its repository. + Conflicting forms, local/originless entries, duplicate repository names, + duplicate or ancestor/descendant destinations, escaping destinations, and + unsupported schemes make materializer preflight fail. + HarnessRouter owns harness/model/provider targets, persistence authorization, + TTLs, quotas, and GC. The project `workspace.yaml` + remains only the materialization catalog; it never contains session access, + retention, lease, or eviction state. - **R10.** Repository mode materializes every execution-eligible declared repository. Optional revisions override only matching logical names; otherwise use configured `branch`, then the remote symbolic HEAD. `RevisionText` is at @@ -377,8 +546,29 @@ caller responsible for acquisition. The temporary fork closes those seams. `GIT_CONFIG_NOSYSTEM=1`, no global config, empty credential helper, disabled hooks, `protocol.file.allow=never`, `protocol.ext.allow=never`, and no submodule recursion, Git LFS hydration, or configured clean/smudge filters. - Preserve each repository's `.git` directory for the coding agent. Failure never - falls through to snapshot mode or another credential identity. + Preserve each repository's `.git` directory for the coding agent, but do not + treat volatile Git administrative bytes as generation identity. The + materializer constructs a hermetic detached-HEAD repository at the resolved + commit, removes reflogs, `FETCH_HEAD`, lock/shallow/replace/graft state, hooks, + worktree links, alternates, extra refs, unreachable objects, and + credential-bearing configuration, and normalizes the allowed config/ref set + and index. The workspace manifest excludes declared repository `.git` + administrative subtrees. The runner separately proves each allowed `.git` + path belongs to its declared root; HEAD resolves to the recorded commit; the + index equals that commit tree with no staged delta; configuration and refs are + closed; and the object database equals the complete transitive object closure + of the commit with no missing, corrupt, or extra objects. It records a canonical + sorted object-ID/type/size-set digest as part of semantic Git state. + + The runner independently reads each resolved commit tree, prefixes it with that + repository's destination, and requires the complete source-visible manifest to + equal exactly the union of those trees plus only the destination ancestor + directories needed to connect them. Undeclared files, links, or directories + outside that union fail integrity validation. Publication records the semantic + Git validation alongside the manifest digest. Pack compression/layout and + index stat-cache data may vary physically but cannot change the semantic + Git-state record, generation key, or source-visible manifest digest. Failure + never falls through to snapshot mode or another credential identity. Limit one materialization to 128 repositories, 500,000 filesystem entries, and 32 GiB across the staged workspace. A count or byte violation returns failed `allagents_workspace_limit_exceeded`. Exhausting the declared UHP time budget @@ -401,30 +591,48 @@ caller responsible for acquisition. The temporary fork closes those seams. sockets, traversal, escaping links, sparse files, unknown or foreign layers, mutable tags, and undeclared output. Recompute the canonical workspace manifest from staging and require it to match both the fetched manifest bytes - and caller-provided digest. Snapshot repository roots need not contain `.git`; - after publication the runner creates private collection - baselines from the verified trees so later produced-file reporting remains - truthful. + and caller-provided digest. Snapshot mode rejects `.git` administrative + subtrees; snapshots that require Git history use repository mode. After + publication the runner creates private collection baselines from the verified + trees so later produced-file reporting remains truthful. + + Both source modes produce the same reusable immutable-generation abstraction. + Repository `.git` state is readable but immutable in `readOnly` attachments and + independently writable only in private `editable` copies. Generation + acquisition limits apply per build; retained-generation and private-workspace + quotas apply independently. - **R12.** Extend HarnessRouter's response translator and stored-response paths - so streaming events, terminal responses, GET, background completion, and - idempotent replay return the same bounded - `response.metadata["allagents.workspace"]`. It contains extension version, - effective descriptor digest, logical cwd, source mode, completeness, resolved - commits or OCI image/workspace-manifest/layer digests, and the canonical - workspace-manifest digest. It never contains origins, physical paths, - credentials, or unverified facts. + with a stage-dependent contract. Before attachment `ready`, non-2xx request + errors and allocated terminal failures omit + `response.metadata["allagents.workspace"]`; error codes identify the failed + stage, and no placeholder or partial/unverified generation identity is emitted. + Once attachment commits `ready`, streaming events, provider terminal responses, + GET, background completion, and idempotent replay return the same immutable + bounded fields: extension version, `effectiveDescriptorDigest`, generation + key, canonical workspace-manifest digest, logical cwd, access, resolved + retention, source completeness, and resolved Git/OCI provenance. Active + streaming metadata has `expiresAt: null`. For `session` retention, durable + terminal acknowledgement atomically sets `expiresAt`; the terminal event, + stored response, GET, background completion, and idempotent replay then return + that same timestamp. `persistent` always returns `expiresAt: null`. Metadata + never contains raw request digest, generation epoch, origins, physical paths, + credentials or references, lease/attachment/reservation tokens, counts, + authorization rules, or other sessions' quota state. - **R13.** The materializer resolves `${ENV_VAR}` references from its allowlisted child environment, uses hermetic Git/registry configuration, removes temporary auth files before returning, and emits no secret. Prove with a deliberately innocuous variable name and value that source credentials and the HarnessRouter - caller API key are absent from the gateway/runner base environment, every agent - environment, workspace, nested Git remotes/config, generated CLI configuration, - logs, checkpoints, and response metadata. In native mode there is no - provider-route API key. The selected OAuth profile is intentionally readable by - the harness trust boundary; the gateway/runner never automatically copies it - into materialized source trees, checkpoints, produced-file records, passive - logs, materializer input, or public metadata. An active same-identity harness - or tool can exfiltrate it; that risk is explicit in owner-trust mode. + caller API key are absent from the gateway/runner base environment, every + staging tree, published generation, private editable workspace, agent + environment, nested Git remote/config, generated CLI configuration, log, + checkpoint, and response. In native mode there is no provider-route API key. + The selected OAuth profile is intentionally readable by the harness trust + boundary only through its active-turn projection; the gateway/runner never + copies it into staging, generations, durable session homes, private workspace + checkpoints, produced-file records, backups, passive logs, materializer input, + or public metadata. Finalization and restart reconciliation verify projection + absence. An active same-identity harness or tool can exfiltrate it; that risk + is explicit in owner-trust mode. In proxy mode, the long-lived proxy client key and upstream provider credentials stay in their owning services. The non-refreshable broker token is @@ -434,7 +642,54 @@ caller responsible for acquisition. The temporary fork closes those seams. cancellation or terminal completion revokes it. Passive persistence never stores it. The HarnessRouter broker rejects wrong-audience, wrong-model, wrong-turn, expired, or revoked tokens. -- **R14.** Build and publish a pinned `linux/amd64` custom HarnessRouter image as +- **R14.** Configure finite, nonzero limits for session idle TTL, staging bytes + and concurrent builds, published-generation bytes/count, per-editable-session + hard bytes/inodes, total private reserved bytes/inodes, total sessions, + authorized persistent sessions, and tombstone bytes/count/TTL. Workspace + response admission reserves one generic session slot and one fixed-size + tombstone slot before making the response/session visible, so semantic + validation failure and later expiry/deletion cannot escape capacity accounting. + Those slots remain through failed-response retention and eventual + tombstone/purge. Readiness is false when required policy is absent or lifecycle + reconciliation is incomplete. Active work holds a durable lease and has no idle + deadline. For `session` retention, one provisional continuation-admission CAS + saves and clears a valid prior deadline; profile success commits active, while + pre-allocation profile failure restores that deadline if future or tombstones + if elapsed. Durable terminal acknowledgement starts a new deadline. + `persistent` bypasses idle expiry only after the runner authorizes it and + reserves its slot before source resolution. A post-allocation failure before + valid retention exists uses the deployment's finite failed-response retention, + then consumes its reserved tombstone slot and purges through the same bounded + lifecycle. + + Collection atomically tombstones an expired or operator-deleted session before + cleanup, fences new turns, waits for active processes and mounts to quiesce, + deletes private state, releases each generation reference and quota reservation + exactly once, and transitions `deleted -> purged` only after durable physical + cleanup and tombstone retention. Tombstones contain only bounded identifiers + and terminal lifecycle facts. Their TTL is at least the maximum response and + idempotency retention; compaction has deterministic age/key order and durable + accounting. While retained, continuation returns HTTP 410 + `allagents_workspace_expired`. After both tombstone and matching response/ + idempotency retention expire, the predecessor is indistinguishable from an + unknown ID and receives the stock non-disclosing error; neither path resolves + or materializes source. + + Generation GC evicts only ready epochs with zero durable references and zero + provisional pins. Null `lastUsedAt` epochs sort first by `publishedAt`, + generation-key bytes, and epoch-ID bytes; non-null epochs then sort by + `lastUsedAt`, `publishedAt`, key bytes, and epoch-ID bytes. It rechecks both + protections under the generation lock, durably records logical eviction, and + completes physical deletion before permitting a new epoch for that key. Failed + deletion stays quarantined and counted against quota. Startup reconciles + generic admission slots, build/staging/prospective-generation reservations, + publication/accounting markers, build waiters, provisional pins, references, + mounts, private reservation transfers and actual usage, copies, tombstones, + compaction, and deletion before readiness or GC. If only protected + state remains, new admission fails + `allagents_workspace_capacity_exceeded`; no protected state is deleted and no + access, retention, source, profile, or provider route changes. +- **R15.** Build and publish a pinned `linux/amd64` custom HarnessRouter image as the public package `ghcr.io/allagentsdev/harnessrouter`. Replace or disable the inherited Docker Hub release path. The Dockerfile pins every base image by digest; runtime lockfiles and version-locked OS packages, Git/OCI tools, Codex, @@ -453,7 +708,7 @@ caller responsible for acquisition. The temporary fork closes those seams. Codex/Pi versions. Release E2E uses that digest, never `latest`. Requests without the configured metadata key remain stock-compatible. CI rebases selected upgrades and runs upstream plus AllAgents integration tests. -- **R15.** AI Evals owns its Promptfoo provider. It sends the UHP request directly +- **R16.** AI Evals owns its Promptfoo provider. It sends the UHP request directly to HarnessRouter, maps Promptfoo variables to the closed extension, and maps terminal output, usage, artifacts, provenance, and failures to `ProviderResponse`. Every non-success follows the Failure Contract's exact @@ -466,243 +721,373 @@ caller responsible for acquisition. The temporary fork closes those seams. #### F1. Start the deployment -1. Run the AllAgents materializer's bounded `preflight` mode. It validates the - project catalog, snapshot and credential-reference schemas, referenced secret - presence, required Git/OCI tools, hook contract version, and staging/workspace - filesystem relationship without contacting sources. -2. In a controlled operator context, initialize each dedicated auth profile: +1. Launch the attestation-verified image in non-serving initialization mode with + durable session, generation, editable-workspace, and auth volumes; finite + idle TTL and staging/generation/private/session/persistence/tombstone quotas; + caller key; materializer command; project configuration; owner-only + source-secret handle; native or proxy trust mode; delegated cgroup v2 subtree; + and `on-failure` restart policy. Verify the image and mounted inputs before + running checks that depend on them. +2. The runner validates read-only mount enforcement, private-copy isolation, + finite lifecycle policy, storage relationships, and cgroup delegation. It + reconciles incomplete generation claims/publications, build waiters, + provisional pins, durable session references, mounts, private editable + workspaces and quota usage, auth projections, tombstones/compaction, and + interrupted deletions. Sweep orphaned cgroups and credential projections only + after proving each old process boundary empty. Do not start GC or serving. +3. Run the mounted AllAgents hook's bounded `preflight` mode. It validates hook + version, project catalog, snapshot and credential-reference syntax, and + required Git/OCI tools without source network access or secret values. It + returns the bounded configured credential-reference identities; the runner, + not the hook, verifies their credential-store handles are present. +4. In a controlled operator context, initialize each dedicated auth profile: run Codex login with that target's `CODEX_HOME`, or run Pi `/login` with that target's isolated Pi home and configured provider. Persist only the selected - harness profile; do not copy a general developer home into the service. -3. If native OAuth cannot satisfy the deployment's trust or compatibility - requirement, deliberately select a separately configured `proxyApiKey` - deployment profile. Validate its closed proxy connection, start the tested - proxy, keep its client key and upstream credentials outside the runner, and - enable HarnessRouter broker mode. Never configure it as automatic failover - for a native target. -4. Start the attestation-verified, digest-pinned custom HarnessRouter image with - durable session data, durable harness auth roots, a private listener, - HarnessRouter caller key, materializer command, project configuration, - owner-only source-secret mount/credential-store handle, the configured native - or proxy trust mode, a runner-owned delegated cgroup v2 subtree, and required - `on-failure` restart policy. Startup sweeps or quarantines orphaned - materializer cgroups and withholds readiness unless delegation is usable and - every prior boundary is empty. -5. From the exact container network, verify each advertised harness ID and model - allowlist, safe roots, materializer version, selected auth binding, and a live - turn. Native targets exercise login status, atomic local refresh persistence, - stale-profile repair, same-binding continuation, and fail-fast overlapping - turns. An explicit proxy deployment exercises - schema/TLS/model-map/endpoint compatibility plus bounded in-turn use and - wrong-scope/expired/revoked rejection. Any required preflight, containment, - or auth failure prevents readiness. + harness profile. If native OAuth cannot satisfy the deployment's trust or + compatibility requirement, deliberately select and validate a separately + configured `proxyApiKey` deployment profile; never make it automatic failover. +5. Start the private listener in probe-only, not-ready mode after reconciliation, + preflight, auth validation, and lifecycle-policy validation succeed. External + traffic and GC remain disabled. +6. From the exact container network, use the deployment probe identity to verify + each advertised harness/model, selected auth binding, safe roots, materializer + version, generation store, mount enforcement, lifecycle policy, and a live + turn. Native targets exercise login, active-turn-only projection, refresh, + teardown, same-binding continuation, and fail-fast overlapping turns. Proxy + targets exercise schema/TLS/model-map/endpoint compatibility and scoped broker + use. Only after every probe succeeds does the deployment atomically enable + external serving, GC, and readiness. Any preflight, containment, auth, mount, + probe, or lifecycle failure keeps readiness false. #### F2. Execute the first repository-backed turn 1. Promptfoo sends one authenticated UHP request with `model`, stock - `metadata.harness_id`, idempotency input, and the AllAgents workspace object. -2. HarnessRouter validates UHP plus generic metadata bounds and atomically claims - the `Idempotency-Key` around lookup and admission. One owner proceeds; - simultaneous same-key arrivals wait on the claim and receive the owner's - result without another admission attempt. For a genuinely new turn, the owner - resolves the selected target and canonical auth-binding identity/config - digest. In `nativeOAuth` it asks the runner supervisor to persist an admission - record and acquire the profile's zero-waiter try-lock; a cross-session - collision publishes the cataloged HTTP 503 to current same-key waiters, then - removes the pre-allocation claim so a later retry can try again. The runner - returns an opaque admission token and retains the lock. `proxyApiKey` acquires - no native profile lock. Once admitted, HarnessRouter binds the idempotency - claim to the response, persists the binding, creates the response/session, - CASes materialization from `unbound` to `materializing`, and hydrates a fresh - session workspace. -3. Before provider selection, the gateway calls runner `/materialize`. The - AllAgents child validates the descriptor and catalog, resolves exact commits - and source credentials, writes and validates staging, removes credential - state, and returns provenance without publishing. -4. The runner independently validates the result/tree, publishes staging, - writes its marker, initializes the root checkpoint plus each declared - repository's collection cursor, and returns `published`. The gateway stores a - durable checkpoint and CASes the session to `ready`. -5. HarnessRouter applies ordinary input files and resolves the safe nested cwd. - In `nativeOAuth`, it projects only the persisted Codex or Pi profile and the - harness calls its provider directly. In `proxyApiKey`, it projects no native - profile and supplies only the scoped turn credential and broker base URL for - the persisted proxy connection. Materialization cannot rerun during provider - retry/fallback, and failure never changes auth mode. -6. Normal UHP events and every stored/retrieved terminal response include the - same namespaced provenance. Produced-file collection walks the HarnessRouter - root plus each declared nested repository without reporting initial source - files as agent output. A single native-mode `finally` path covers completion, - cancellation, and every post-admission failure: it persists terminal state, - durably commits refresh state or marks the profile `repair-required`, and - acknowledges the admission token. Only then may the runner release the - profile lock. + `metadata.harness_id`, idempotency input, and the + `metadata["allagents.workspace"]` JSON descriptor. The descriptor explicitly + selects `readOnly` or `editable`; omitted retention means `session`. +2. HarnessRouter validates UHP and generic metadata bounds and atomically claims + the `Idempotency-Key`. The private runner admission transaction resolves the + selected target/auth-binding digest, applies existing profile admission, and + durably reserves one generic workspace-session slot plus one fixed-size + tombstone slot before response allocation. The admission token owns all three; + any pre-allocation failure rolls them back exactly once. Duplicate same-key + arrivals share one admission/result; new same-session overlap returns + `session_busy`; a genuinely new cross-session turn colliding on one native + profile fails cataloged `harness_unavailable`; unavailable generic lifecycle + capacity fails `allagents_workspace_capacity_exceeded`. Both occur before + response allocation. Distinct profiles may proceed concurrently. +3. The gateway consumes that token, creates the response/session in `validating`, + and persists the opaque descriptor, raw request digest, generic reservation + IDs, and harness/auth binding before making it visible. It then invokes the + hook's source-free `validate` operation. The runner consumes typed access, + requested retention, effective descriptor digest, logical cwd, normalized + descriptor reference, and bounded selected credential-reference identities. + It proves the selected set is a subset of preflight declarations, verifies + only those handles, rejects read-only input files, rechecks the secret + boundary, authorizes persistence, and reserves any persistence slot plus one + stable full-hard-private-allowance ID for editable access. Failure releases + access-specific reservations once, persists the terminal failed response under + finite failed-response retention, and retains its generic session/tombstone + slots through tombstoning and purge; no source is resolved or acquired. +4. The gateway CASes `validating -> resolving`. `resolve` consumes the exact + validated descriptor and selected credential set, resolves configured source + to immutable identity, and returns the generation key, private source-only + resolved-plan path/digest, effective cwd, and request provenance. Under the + generation lock, a valid ready epoch hit acquires a durable provisional pin + and skips acquisition. A miss joins the current build epoch or, only after an + evicted prior epoch is durably gone, creates a new runner-owned epoch/claim. + Before first byte acquisition, its owner stores the exact resolved plan and + selected credential-reference identities in claim-owned durable private state + and reserves the full staging and prospective-generation byte/count + allowance. Each request waits only to its own deadline; cancellation detaches + only that waiter. +5. Only a live miss claim invokes `materialize` with the exact resolved-plan + path/digest and fixed private staging/result roots. The child writes and + validates staging, removes credential state, and returns the manifest. The + runner proves the cgroup empty and independently validates the tree, semantic + Git state, result, and key. The runner's independent full-tree accounting, + including validated Git administrative state, supplies physical retained + byte/inode usage. Under the generation lock, one atomic publication/accounting + transition converts prospective generation capacity to actual usage, releases + excess and the staging reservation, records epoch/publishedAt, and persists + ready state before giving each live waiter a provisional pin. Build failure + removes unpublished staging and releases access/build-specific reservations; + generic session/tombstone reservations remain with their failed responses. +6. For each pinned waiter independently, the runner rejects an editable + attachment whose full physical generation bytes/inodes exceed its hard + allowance, releasing only that waiter's pin and access-specific reservations; + other waiters continue against the valid ready epoch. Otherwise the gateway + CASes that session `resolving -> attaching`, and the runner prepares either a + durable read-only epoch reference plus verified mount or a unique private copy + plus checkpoints and collection baselines. It returns an opaque token/evidence. + The gateway alone CASes `attaching -> ready`, stores key/epoch evidence, and + acknowledges the token. Under the generation lock, the runner releases that + provisional pin exactly once and advances `lastUsedAt` to at least the + ready-commit timestamp. Prepare/ack recovery preserves the committed + attachment or rolls that waiter's resources/reservations back once. +7. Only after attachment `ready` do response events include the complete + workspace metadata; active streaming uses `expiresAt: null`. Only editable + sessions accept ordinary UHP input-file overlays. HarnessRouter resolves the + safe logical cwd, creates the active-turn-only native credential projection or + scoped proxy credential, and dispatches the harness. Provider retry/fallback + cannot validate, resolve, build, attach, or change any binding. +8. Read-only collection reports no workspace mutation; editable collection walks + the private root and declared repositories without reporting initial source + files. After descendants stop, native finalization commits refresh state or + marks `repair-required`, removes the credential projection, and proves retained + homes/checkpoints clean. The gateway then durably stores the terminal response + and, for `session`, sets one expiry timestamp used by the terminal event, GET, + background completion, and replay. Only after that acknowledgement may the + runner release the profile lock. #### F3. Continue the session 1. The caller sends `previous_response_id` and omits `metadata["allagents.workspace"]`. -2. HarnessRouter atomically claims the `Idempotency-Key` around lookup and - admission. A simultaneous duplicate waits for or returns the owner's result - without another admission attempt. For a new request it resolves the current - session state and writable workspace, requires materialization `ready`, and - resolves the persisted auth-binding identity and config digest. An - extension-bearing continuation, changed or unavailable binding, or - non-resumable session fails before runner work. Same-session concurrency - returns stock `session_busy` before profile admission. Only then does - `nativeOAuth` acquire the runner-owned zero-waiter profile lock; cross-session - saturation returns cataloged `harness_unavailable`. A pre-allocation error is - delivered to claim waiters before claim removal. Proxy mode acquires no native - profile lock. -3. HarnessRouter follows its stock predecessor/session semantics. Native mode - projects the persisted profile; proxy mode mints a new scoped turn token for - the persisted connection. It resumes the native conversation and returns - pinned provenance plus new output, usage, and artifacts without changing auth - mode or binding. The native `finally` boundary releases the profile lock on - every terminal outcome. +2. HarnessRouter atomically claims the `Idempotency-Key`. For a new request, + generic continuation validation first rejects an extension-bearing request + without session lookup/CAS or deadline change. It then resolves the gateway- + owned attachment and enters one session CAS whose predicates include + `session_busy`, exact attachment/binding, and expiry/deletion. Busy or binding- + mismatch rejection changes no deadline. A retained expired or deleted session + returns HTTP 410 `allagents_workspace_expired` before profile or runner work. + After tombstone plus response/idempotency retention has been purged, the + predecessor receives the stock non-disclosing unknown-ID error. None resolves + source. +3. Only when every predicate succeeds does that CAS require attachment `ready` + plus persistent retention or an idle deadline later than admission, then save + and clear that deadline in a provisional turn-admission fence. The runner + verifies the exact epoch reference/mount or editable private checkpoint before + profile admission. Missing/corrupt evidence returns HTTP 409 + `allagents_workspace_non_resumable` and rolls the fence back by restoring the + original future deadline or tombstoning if elapsed, without source resolution, + acquisition, or rematerialization. +4. A native turn then tries the existing zero-waiter profile lock. Same-profile + cross-session saturation returns cataloged `harness_unavailable`; proxy mode + has no native lock. On any pre-allocation profile failure, the gateway rolls + back the provisional fence, restoring the exact original deadline if it + remains future or tombstoning the session if it elapsed. A returned admission + token commits the fence to active and exposes `expiresAt: null`. Sessions using + different profiles may execute concurrently against the same generation + epoch; polling and replay change no deadline or admission state. +5. HarnessRouter resumes the native conversation and original attachment. + Read-only source remains immutable; editable prior mutations remain visible. + Output, usage, artifacts, and pinned provenance return without changing any + binding. The same refresh/projection teardown and terminal-ack protocol as the + first turn sets the next expiry and releases the profile lock on every + terminal outcome. #### F4. Execute an OCI-backed first turn 1. The caller selects one configured snapshot and immutable manifest/workspace digests; it never sends the registry origin or credential. -2. The materializer fetches and verifies the direct manifest, config, and layers, - applies changesets under fixed limits, validates the declared repository - layout in staging, and returns exact provenance. -3. The runner publishes and checkpoints through the same state machine as F2. - Any registry, digest, media, path, limit, or layout failure removes staging, - terminalizes the response, and enters neither Git nor provider fallback. +2. Resolve computes the OCI generation key. A ready epoch is reused. Otherwise a + current claim is joined or, after completed eviction, a new epoch owner + fetches and verifies the direct manifest, config, workspace manifest, and + layers; applies changesets under fixed limits; and returns verified staging + and provenance. +3. The runner publishes the same immutable-generation-epoch abstraction as Git, + then follows the same per-waiter read-only or editable attachment path. + Registry, digest, media, path, limit, or layout failure removes only + unpublished staging and enters neither Git nor provider fallback. #### F5. Cancel, fail, or restart -1. On every materializer outcome, including a parent that returns `completed`, - the runner proves the cgroup empty before exposing any terminal result, - reading the manifest, publishing, releasing secrets, or cleanup. Membership - after a parent - reports `completed` produces failed - `allagents_workspace_containment_breach` even if `cgroup.kill` empties the - leaf; the runner may remain ready after cleanup in that recovered case. If - `populated 0` cannot be proved, the runner reports internal - `containment_pending`; the gateway persists and acknowledges only that - non-terminal state, withholding terminal GET/stream visibility. The runner - exits nonzero after acknowledgement or its bounded deadline. Restart destroys - the old boundary and keeps readiness false until it proves the old cgroup - empty, then reports reconciliation. Only after that proof does the gateway - expose `cancelled` for client cancellation, `incomplete` for declared-budget - stop, or failed `allagents_workspace_containment_breach` otherwise. No provider - starts. -2. Agent cancellation and deadline use HarnessRouter's normal UHP lifecycle. -3. Startup reconciles a `materializing` session to `ready` only when the bound - descriptor, published workspace marker, and durable checkpoint all match. - Otherwise it marks the session failed/non-resumable and removes or quarantines - the workspace. It never replays acquisition. A completed `ready` session - rehydrates from its durable checkpoint. -4. Whole-container termination does not preserve the in-flight agent process. - Interrupted agent turns fail according to HarnessRouter behavior. -5. Every failure after native admission persists terminal state and valid or - `repair-required` refresh state before acknowledging the runner's admission - token. A gateway-only crash leaves the runner-held lock intact until restart - reconciliation; a runner crash leaves a durable admission record that blocks - readiness until descendant/profile reconciliation. Only then can a new turn - acquire the profile; a failed request is never retained as a waiter. +1. On every hook outcome, the runner proves the cgroup empty before exposing a + terminal result, reading a manifest, publishing a generation, releasing + secrets, or cleanup. Completed-parent/live-descendant returns + `allagents_workspace_containment_breach`; an unquiescent leaf remains internal + `containment_pending`, exits/restarts the runner, withholds readiness and + terminal visibility, and reconciles only after proving the old boundary empty. +2. Cancellation before attachment detaches only that request from a shared build; + the build continues for other live waiters and stops only when none remain or + its runner-owned deadline expires. Agent cancellation and deadline after + dispatch use HarnessRouter's normal UHP lifecycle. Whole-container termination + does not preserve an in-flight agent; interrupted turns fail without replay. +3. Startup reconciles generation-epoch + `building/ready/quarantined/deleting` evidence separately from session + `validating/resolving/attaching/ready/containment_pending/expired/deleting/ + deleted/purged/failed` evidence. `containment_pending` stays non-terminal and + blocks readiness until the recorded cgroup is empty; it then transitions once + to the preserved `failed`, `cancelled`, or `incomplete` outcome. A ready + read-only session requires its exact generation key/epoch marker, reference, + and mount. A ready editable session requires its private publication marker, + reserved quota, actual-usage accounting, and matching checkpoint. Missing or + mismatched state becomes non-resumable; a later epoch is never substituted. +4. Every terminal outcome after native admission persists refresh disposition, + removes the active credential projection, verifies retained homes clean, and + stores terminal response/expiry before acknowledging the admission token. + Gateway or runner crashes retain the durable admission fence until descendant, + projection, and profile reconciliation; only then can another turn acquire the + profile. +5. Restart reconciles all lifecycle state before GC or readiness. It completes or + rolls back interrupted provisional turn admission against any runner profile + token, then reconciles generic admission, staging/prospective-generation + reservation, publication/accounting conversion, provisional-pin/reference + acquisition, attachment prepare/ack and private-reservation transfer, + mount/copy creation, private usage accounting, tombstoning, unmount, release, + compaction, and physical deletion without duplicating a debit/reference, + leaking a pin, extending an original deadline, or exposing a partially deleted + resource. +6. A completed session resumes only with attachment `ready` and valid evidence, + and when retention is persistent or its session idle deadline is unexpired. + Known invalid evidence returns `allagents_workspace_non_resumable`; it never + rematerializes or substitutes a later generation epoch. + +#### F6. Expire, delete, and collect workspace state + +1. An accepted turn holds an active lease and no idle expiry. Idle expiry or + authenticated operator deletion CASes the session to a bounded tombstone + first; a racing continuation either wins admission and clears the old deadline + or observes the tombstone. +2. The collector fences new work, waits for process, credential-projection, and + mount quiescence, removes editable private state, releases each generation + reference and private quota reservation exactly once, and durably records + deletion. Persistent sessions skip idle expiry but use the same explicit-delete + path. +3. Under storage pressure, GC selects only ready generation epochs with zero + references and zero provisional pins. Null `lastUsedAt` epochs order first by + `publishedAt`, generation-key bytes, and epoch-ID bytes; non-null epochs then + order by `lastUsedAt`, `publishedAt`, key bytes, and epoch-ID bytes. It + rechecks protection under lock, records durable logical eviction, removes + physical state, and only after both complete permits a new epoch for that key. +4. Tombstone compaction runs in age/key order only after the maximum response and + idempotency retention has elapsed, durably transitions `deleted -> purged`, + and releases the reserved tombstone slot. A later predecessor lookup uses the + stock non-disclosing unknown-ID error and never rematerializes. +5. Failed deletion remains quarantined and counted. If high-water quota cannot be + reduced because all state is active or pinned, new admission fails + `allagents_workspace_capacity_exceeded`; no protected workspace is removed. ### Acceptance Examples - **AE1.** A stock UHP request without the configured metadata key produces the same response and conformance result on upstream HarnessRouter and the fork. - **AE2.** Every unauthenticated external create, continuation, GET, stream, - cancel, file, and artifact request fails before resource existence or metadata - is disclosed. An authenticated native Codex or Pi turn uses the selected OAuth - profile; the HarnessRouter caller key is absent from the agent environment and - filesystem, and no provider-route API key exists in that mode. -- **AE3.** Repository mode resolves configured branch/default/HEAD refs to full - commits, prepares every execution-eligible repository, preserves nested Git - history, starts in a validated nested cwd, and returns path-free provenance. + cancel, file, artifact, and lifecycle request fails before resource existence + or metadata is disclosed. An authenticated native Codex or Pi turn sees only + the selected active-turn OAuth projection; the HarnessRouter caller key is + absent from the agent environment and filesystem, no provider-route API key + exists in that mode, and the projection is absent from retained homes, + checkpoints, backups, and mounts after every terminal or recovered outcome. +- **AE3.** Repository mode resolves configured refs to exact commits and + publishes one verified immutable generation. Two simultaneous `readOnly` + sessions using different harness/profile bindings share one generation build, + see identical bytes and nested Git history, start in their own validated + logical cwd, and cannot write the generation or observe each other's home, + conversation, temporary files, logs, or outputs. - **AE4.** HarnessRouter maps a non-object extension to HTTP 400 `invalid_input`, - an oversized extension to HTTP 413 `allagents_workspace_too_large`, and an - extension on continuation to HTTP 409 `allagents_workspace_immutable`; each is - an `invalid_request_error` with `param: "metadata.allagents.workspace"` and - `detail.retryable: false`, before the hook or response allocation. The hook - rejects unknown logical names, caller-provided URLs, absolute/traversal paths, - commands, environment fields, credentials, originless/local repositories, and - duplicate names/destinations - before source network access or agent launch. -- **AE5.** Two turns linked by `previous_response_id` preserve a file and native - conversation context. The hook runs once; produced-file collection reports - modifications inside every nested repository but not the initial source tree. -- **AE6.** A continuation omitting the extension succeeds. Any continuation - containing the configured workspace key is rejected without changing the - workspace. A new revision uses a new session. -- **AE7.** Explicit UHP input files overlay materialized paths after the - pre-agent checkpoint and before agent launch. -- **AE8.** A materializer spawn/nonzero/crash or malformed/oversized result maps - to `allagents_materializer_failed`; publication/marker, checkpoint/baseline, and - materialization-state/CAS failures map respectively to - `allagents_workspace_publication_failed`, - `allagents_workspace_checkpoint_failed`, and - `allagents_workspace_state_failed`. A post-allocation secret-boundary recheck - maps to `allagents_secret_boundary_violation`. Each starts no provider, leaks - no credential, and leaves the session failed/non-resumable rather than - partially ready. Atomic-placement, `setsid()`, and double-fork fixtures include - a parent that returns `completed` while a descendant attempts a delayed write; - that case returns `allagents_workspace_containment_breach` even when forced - kill empties the cgroup. An unquiescent leaf preserves public cancellation/ - budget status when mandated, records the containment reason, and exits the - runner nonzero. Provider fallback never reruns materialization. After every - fault, a new native turn proves profile admission was safely reconciled. + an oversized extension to HTTP 413 `allagents_workspace_too_large`, an + extension on continuation to HTTP 409 `allagents_workspace_immutable`, and a + retained expired/tombstoned continuation to HTTP 410 + `allagents_workspace_expired`. The hook validates unknown access/retention, + read-only input conflicts, logical names, caller URLs, paths, commands, + credential-reference selection, and duplicate names/destinations before source + access. Preflight receives no secret values; the runner alone verifies declared + credential handles, storage relationships, persistence authorization, and + session/persistence/build reservations. Exact source byte admission may fail + only after bounded staging reveals size, but before publication, attachment, or + agent launch. +- **AE5.** Two `editable` turns linked by `previous_response_id` preserve native + conversation and a private file mutation. A separate editable trial from the + same generation receives a unique clean copy and cannot observe or mutate the + first. An editable waiter whose initial copy cannot fit fails its own + `allagents_workspace_private_quota_exceeded` response without invalidating the + ready epoch or a concurrent read-only/fitting waiter. Produced-file collection + reports only private changes. Growth across turns cannot exceed the session's + reserved hard quota. Two read-only turns preserve conversation but have no + workspace mutation checkpoint or produced-file delta. +- **AE6.** Continuation omits the extension and preserves the exact generation + epoch, access, retention, cwd, harness, and profile. Any attempted rebinding is + rejected. One CAS rejects same-session overlap without changing expiry and + provisionally saves/clears an unexpired idle deadline. Profile success commits + active; pre-allocation profile failure restores that deadline when future or + tombstones if elapsed. Only terminal acknowledgement sets the next expiry. + Polling and replay do neither. A retained expired/deleted predecessor returns + HTTP 410; a known corrupt attachment returns HTTP 409 + `allagents_workspace_non_resumable`; a fully purged predecessor returns the + stock unknown-ID error. None rematerializes or substitutes an epoch. +- **AE7.** Explicit UHP input files overlay only an editable private workspace + after its initial checkpoint and before agent launch. A read-only request with + workspace input files fails `allagents_workspace_read_only`; a runtime write + receives a filesystem read-only error with no copy-up or mode change. +- **AE8.** Faults at pre-allocation generic session/tombstone admission, + validate, selected-credential verification, resolve, keyed epoch claim/waiter + cancellation, staging/generation reservation and publication-accounting + conversion, containment, provisional pin, attachment prepare/ack, read-only + epoch reference/mount, editable reservation-transfer/copy/checkpoint, expiry, + unmount, release, tombstone compaction, and deletion either reconcile to one + complete protected resource or fail closed. No agent sees staging, duplicate + live epoch publication, partial private state, or a generation without required + protection. Every reservation, pin, and reference debits and releases exactly + once. Provider fallback never reruns the hook. - **AE9.** OCI mode accepts a valid digest-pinned fixture with gzip/zstd layers and whiteouts and rejects mutable tags, indexes, mismatched digests/sizes, traversal, escaping links, devices, sparse files, unknown media types, and - declared-limit overflow. -- **AE10.** Codex signs in and refreshes through Codex CLI; Pi signs in and - refreshes through Pi for its configured provider. Missing, revoked, expired, - unrefreshable, or locally stale-after-crash OAuth marks only that profile - unavailable or `repair-required`; it never selects another profile or proxy. - A barriered pair of simultaneous first arrivals with one `Idempotency-Key` - produces one admission and one result; a new continuation in that active - session returns `session_busy`; and genuinely new turns in other sessions - sharing the profile fail immediately with cataloged `harness_unavailable` - before allocation. Same-key waiters receive a pre-allocation error before its - claim is removed; they never fall through to a second execution. Cross-session - callers are never queued and cannot acquire later. A separately configured - `proxyApiKey` deployment's broker permits bounded multi-request provider flow - and rejects wrong-audience, wrong-model, wrong-turn, expired, or revoked - credentials. -- **AE11.** Restart after a completed first turn preserves the session and exact - persisted auth binding. Removing or changing that binding makes continuation - fail before runner work; restoring the matching identity and config digest - restores eligibility. Restart during materialization recovers only from a - matching published marker and durable checkpoint; otherwise it fails without - automatic replay. Restart during an agent turn follows HarnessRouter's - interrupted-turn failure behavior. + declared-limit overflow. Repeated Git and OCI requests with identical resolved + plan, sharing authorization, and selected credential-reference identities reuse + their matching epoch regardless of access, retention, cwd, harness, profile, or + session. +- **AE10.** Codex and Pi own login and refresh. Missing, revoked, expired, + unrefreshable, or stale-after-crash OAuth affects only that profile and never + selects another profile or proxy. Same-key arrivals share one admission and + result; same-session overlap returns `session_busy`; new cross-session turns on + the same profile fail immediately with cataloged `harness_unavailable`. + Sessions using different profiles can run concurrently on one read-only + generation. Success, failure, cancellation, and crash recovery all commit or + reject refresh and remove the active credential projection before the next + profile admission. Proxy mode enforces audience, target, model, turn, expiry, + and revocation. +- **AE11.** Restart preserves an unexpired or persistent session, exact + attachment, and auth binding after lifecycle reconciliation. Missing or + changed auth fails before runner work. Missing generation/reference or private + checkpoint returns the cataloged non-resumable error without replay. A + `containment_pending` session remains non-terminal until its cgroup is empty; + interrupted builds, pins, copies, quota records, auth projections, tombstones, + compaction, and deletions reconcile without resurrection or double release. - **AE12.** The protected publish job releases the public `linux/amd64` GHCR - package without Docker Hub credentials. An anonymous client reads the manifest, - verifies the GitHub/Sigstore build-provenance and SBOM attestations' expected - owner, repository, workflow, approved ref, subject digest, and predicate, and - pulls that digest rather than `latest`. The evidence records upstream, patch, - materializer, Codex, and Pi inputs; mismatches fail closed. -- **AE13.** Promptfoo maps request-time auth errors, every cataloged materializer - failure, UHP `failed`/`incomplete`/`cancelled` results, and HarnessRouter - execution failures to `ProviderResponse.error`. It preserves the wire error - code when one exists and otherwise uses the exact terminal status, plus - retryability, safe message, and verified metadata. None becomes successful - empty output or an automatic retry. + package without Docker Hub credentials. Anonymous verification covers the + expected build-provenance/SBOM identity, subject digest, and pinned inputs. +- **AE13.** Promptfoo maps every cataloged request, generation, attachment, + persistence, read-only, capacity, expiry, materializer, auth, provider, and UHP + terminal failure to the exact `ProviderResponse.error` and metadata. Failures + before attachment `ready` omit workspace metadata; later terminal failures + include its complete public metadata. None becomes empty success or an + automatic retry. +- **AE14.** With tiny deterministic quotas and a fake clock, invalid descriptors + cannot create unaccounted response/session records; active and persistent + sessions survive collection; terminal acknowledgement starts idle expiry; + expired editable state and its one private reservation are deleted; repeated + successful unique publications return staging capacity to baseline; read-only + references and provisional pins release exactly once; never-attached epochs + with null `lastUsedAt` evict first by `publishedAt`, then used epochs by + `lastUsedAt` and `publishedAt`, with generation-key/epoch ties; a new epoch + cannot publish until prior logical and physical eviction completes; tombstones + remain byte/count + bounded and purge only after response/idempotency retention; failed deletion + stays quarantined/accounted; and all-protected capacity returns + `allagents_workspace_capacity_exceeded`. ### Scope Boundaries **In scope** -- HarnessRouter workspace-integration and harness-auth-state patches. -- Versioned AllAgents workspace descriptor, hook request/result, and provenance. -- Project workspace schema additions and catalog projection. -- Deterministic Git and immutable OCI acquisition. -- Root/nested-repository checkpoint and produced-file integration. +- HarnessRouter generation, attachment, lease, retention, quota, GC, + workspace-integration, and harness-auth-state patches. +- Versioned AllAgents JSON workspace descriptor, validate/resolve/materialize + hook contracts, generation identity, and provenance. +- Project `workspace.yaml` source-catalog additions and projection. +- Deterministic Git and immutable OCI generation construction. +- Shared read-only mounts, private editable copies, mode-specific + root/nested-repository checkpoint and produced-file integration. +- Session idle expiry, persistent authorization, operator deletion, bounded + storage admission, restart reconciliation, and generation eviction. - Source credential isolation and native-OAuth trust-boundary verification. - Native Codex and Pi auth-profile bootstrap, refresh, readiness, and continuity. -- Explicit, separately validated API-key-authenticated proxy deployment mode. -- Public GHCR image publishing, digest-pinned release metadata, SBOM, and - provenance. -- Promptfoo contract examples and one-shot/two-turn success/failure E2E. -- Upstream-ready generic hook and auth-state patches plus maintenance procedure. +- Explicit authenticated-proxy deployment mode. +- Public GHCR publishing, digest-pinned release metadata, SBOM, and provenance. +- Promptfoo concurrent/one-shot/two-turn/lifecycle success and failure E2E. +- Upstream-ready generic hook, lifecycle, and auth-state patches. **Out of scope** @@ -742,32 +1127,42 @@ caller responsible for acquisition. The temporary fork closes those seams. ```mermaid flowchart TB - PF[Promptfoo provider] -->|UHP + HR API key| GW[HarnessRouter gateway] - GW -->|first-turn materialize| RUN[HarnessRouter runner] - RUN -->|opaque JSON in, typed envelope out| MAT[AllAgents materializer] - MAT --> CFG[project workspace.yaml] + PF[Promptfoo provider] -->|UHP + HR API key + workspace JSON| GW[HarnessRouter gateway] + GW -->|session CAS + attachment prepare/ack| RUN[HarnessRouter runner] + RUN -->|typed validate, resolve, or cache-miss materialize| MAT[AllAgents materializer] + MAT --> CFG[project workspace.yaml source catalog] MAT --> GIT[Git sources] MAT --> OCI[OCI registry] - RUN --> HARNESS[Selected Codex or Pi harness] + RUN -->|atomic publish or reuse| GEN[(immutable generation store)] + GEN -->|read-only mount + reference| RO[read-only session] + GEN -->|private copy| EDIT[editable session] + RO --> HARNESS[Selected Codex or Pi harness] + EDIT --> HARNESS + LIFE[(leases, retention, quotas, GC)] --> RO + LIFE --> EDIT + LIFE --> GEN AUTH[(dedicated durable OAuth profile)] -.->|native mode only| HARNESS HARNESS -->|native mode| MODEL[Model provider] HARNESS -.->|proxy mode: scoped turn credential| GW GW -.->|long-lived proxy client key| PROXY[optional provider proxy] PROXY -.-> MODEL - GW --> DATA[(durable session checkpoints)] + GW --> DATA[(durable session and attachment state)] ``` -The gateway owns generic metadata bounds, session materialization state, -persisted auth-binding identity, provider-loop ordering, checkpoint persistence, -response metadata, and optional proxy brokering. The runner owns hook invocation, -staged publication, checkpoint/collection setup, safe nested cwd, and agent -launch. In native mode it projects the bound OAuth profile; in proxy mode it -projects no native profile and supplies only the turn broker capability. The -selected harness owns native provider OAuth login and refresh. Its durable auth -root is outside session checkpoints; it is available to the harness trust -boundary but never to the materializer. The materializer owns only AllAgents -schema, catalog, acquisition, source-credential selection, staging validation, -and provenance; it never speaks UHP or publishes the live workspace. +The gateway is the sole writer of session attachment, expiry, and tombstone +state; it also owns generic metadata bounds, provider-loop ordering, response +metadata, and optional proxy brokering. The runner owns keyed generation claims, +the resource journal, hook invocation, independent verification, atomic +publication, provisional pins, durable references, read-only mounts, private +editable copies, mode-specific checkpoints, quota admission, deletion, GC, safe +cwd, and agent launch. Attachment uses a durable prepare/evidence/ack protocol: +the runner prepares resources, the gateway alone commits `ready`, and the runner +finalizes or rolls back from that acknowledgement. The selected harness owns +native OAuth login and refresh; its auth root is outside every generation and +session checkpoint. The materializer owns only the AllAgents JSON schema, +`workspace.yaml` source catalog, source resolution, acquisition, staging +validation, and provenance. It never speaks UHP, authorizes persistence, owns +leases, publishes live state, or writes the gateway session state machine. ### Extension Contract @@ -781,6 +1176,8 @@ Initial UHP request fragment: "harness_id": "codex-review", "allagents.workspace": { "version": "1", + "access": "readOnly", + "retention": "session", "source": { "kind": "repositories", "revisions": { @@ -807,13 +1204,17 @@ Continuation fragment: } ``` -Successful response metadata fragment: +Successful terminal response metadata fragment: ```json { "allagents.workspace": { "version": "1", - "descriptorDigest": "sha256:...", + "effectiveDescriptorDigest": "sha256:...", + "generationKey": "sha256:...", + "access": "readOnly", + "retention": "session", + "expiresAt": "2026-09-24T12:00:00Z", "workingDirectory": { "kind": "repository", "repository": "api", @@ -842,35 +1243,66 @@ maximum runtime, request/result byte limits, and allowlisted environment names. The runner launches the executable directly without a shell. Standard error is diagnostic-only, bounded, secret-checked, and never copied verbatim to callers. -The hook supports two operations: - -- `preflight`: validate contract version, project catalog, credential-reference - syntax and presence, required binaries, and filesystem assumptions without - source network access; and -- `materialize`: validate the opaque descriptor, write source content only to the - supplied sibling staging root, write the canonical manifest only to the - supplied private result root, and return without publishing. - -The materialize request contains the generic contract version, opaque metadata -value, session workspace, fixed staging and private result roots, project -configuration root, and deadline. Secret values are injected only through the -configured allowlisted child environment; credential identifiers and values are -absent from JSON. - -The generic result is either: - -- `completed`, effective relative cwd, effective descriptor digest, - `workspaceManifest: { path: "workspace-manifest.json", digest }`, complete - path-free public metadata, and declared nested-repository roots; or -- `failed`, cataloged code, safe message, retryability, and any verified - incomplete public metadata. - -The runner validates the result and staged tree independently. It rejects an -unknown envelope field/version, digest mismatch, physical path in public -metadata, incomplete success, undeclared repository root, escaping cwd, or tree -that does not match the manifest. The runner then owns publication, marker and -checkpoint setup; a valid result never means the live workspace is already -published. +The hook supports four operations: + +- `preflight`: validate contract version, project source catalog, + credential-reference syntax, and required binaries without source network + access or secret values, then return the bounded configured credential- + reference names/opaque IDs. The runner verifies the corresponding store + handles and all staging/result filesystem relationships itself; +- `validate`: validate and default the opaque JSON descriptor and catalog names + without source access, then return a private normalized-descriptor path/digest, + effective descriptor digest, effective access, requested retention, logical + cwd, and a bounded sorted selected credential-reference subset; +- `resolve`: consume that exact normalized descriptor and selected reference set, + resolve immutable source identity, and return a private canonical source-only + resolved-plan path/digest, generation key, effective cwd, and bounded request + provenance without writing source bytes. The plan contains only generation-key + inputs—resolved commits or OCI digests/layers, normalized destinations, + catalog/policy and sharing-authorization identity, and selected credential- + reference identities—and omits credential values, access, retention, cwd, + requested-ref spelling, harness/profile, and session; equal generation keys + therefore require identical plan bytes; and +- `materialize`: consume those exact resolved-plan bytes and selected reference + set at the supplied private path, verify their supplied digest, write source + content only to supplied generation staging, write the canonical manifest only + to the private result root, and return without publishing or re-resolving + source. + +Preflight runs once per deployment and receives no credential values. Validate +and resolve run once for a new session. Materialize runs only for a runner-owned +cache-miss generation claim; concurrent waiters consume the same ready +publication or failure. Validate receives opaque metadata and bounded +workspace-input-file count. Resolve receives the validated-descriptor path and +digest plus the exact selected credential-reference identities. Materialize +receives the resolved-plan path and digest, that same set, and fixed generation +staging/result roots. All operations receive the generic contract version and +project configuration root. Validate/resolve use the request's bounded remaining +deadline; shared materialize uses the runner-owned build deadline and is +cancelled only when no live waiter remains. The runner resolves values for only +the validated selected set and injects them only into source-access operations +through the allowlisted child environment; values never appear in JSON, +generation keys, or persisted plans. + +Validate returns effective access, requested retention, effective descriptor +digest/cwd, selected credential-reference identities, and its private normalized- +descriptor path/digest. Resolve repeats those request-bound values and adds the +generation key, private source-only resolved-plan path/digest, and bounded +request provenance. Materialize returns that same generation key plus +`workspaceManifest: { path: "workspace-manifest.json", digest }`, declared +repository roots, semantic Git validation records when applicable, and bounded +generation-scoped verified source identity. It does not return or choose a +waiter's descriptor digest, cwd, access, retention, or requested-source +provenance. Any operation may return the cataloged `failed` envelope. + +The runner checks that every private path is relative to its operation-specific +result root, opens it without symlink traversal, validates size and digest, and +passes the exact bytes/reference to the next operation. It rejects unknown +fields/versions, plan or generation-key drift, physical paths in public +metadata, incomplete success, forged manifests, undeclared roots, escaping cwd, +or staging that differs from the manifest. It then owns immutable publication +and resource preparation; the gateway remains the only session-attachment +writer. Valid hook output never means live state is published or attached. ### Workspace Manifest Contract @@ -907,17 +1339,33 @@ UTF-8 bytes. Every path component and symlink target must already be valid UTF-8 and NFC; implementations reject rather than normalize non-UTF-8 or non-NFC values. Paths and targets containing NUL, absolute paths, missing parents, or links escaping the workspace are invalid. The root is implicit and has no entry. -Hard links are expanded to regular-file entries. +Hard links are expanded to regular-file entries. Entries enumerate every +source-visible path. In repository mode only, each declared repository's +separately validated `.git` directory and descendants are omitted because +volatile pack/index layout is not source identity; no other path may be omitted. The digest is `sha256:` plus the lowercase SHA-256 of the RFC 8785 bytes. Git -mode computes those bytes after completing staging. OCI mode requires its -configured workspace-manifest blob to contain the same canonical bytes and -copies them to the private result root. The runner resolves only the fixed -`workspace-manifest.json` relative path, validates it against the shared schema, -verifies its size and digest, walks staging without following links, reconstructs -the same catalog and entries, and requires byte-for-byte canonical equality -before publication. The private result root is never published or exposed -through UHP. +mode computes those bytes after completing staging and performs the separate +semantic `.git` validation required by R10. OCI mode rejects `.git` +administrative subtrees, requires its configured workspace-manifest blob to +contain the same canonical bytes, and copies them to the private result root. +The runner resolves only the fixed `workspace-manifest.json` relative path, +validates it against the shared schema, verifies its size and digest, walks +staging without following links, reconstructs the same catalog and entries while +skipping only approved Git administrative roots, and requires byte-for-byte +canonical equality before publication. It separately revalidates every skipped +Git root against the resolved commit and safe-state rules. The private result +root is never published or exposed through UHP. + +The immutable generation key is not the workspace-manifest digest: it is the +pre-build digest of the resolved source plan used for keyed reuse. Atomic +publication binds that key to exactly one verified source-visible manifest +digest and, in repository mode, one semantic Git-state record for the resolved +commits. Access, retention, cwd, harness/profile, and session identity are not +manifest fields and cannot fragment or mutate generation content. The backing +tree, including validated `.git` state, becomes owner-writable only and is +exposed to sessions solely through verified read-only mounts or independent +private editable copies. The frozen cross-repository fixture is: @@ -944,38 +1392,55 @@ error object in `response.error`; workspace failures use | `invalid_input` | HTTP 400 `invalid_request_error` before response allocation for a non-object workspace extension; `param` is `metadata.allagents.workspace` | no | | `allagents_workspace_too_large` | HTTP 413 `invalid_request_error` before response allocation for the 64-KiB/depth bound; `param` is `metadata.allagents.workspace`; `detail.max_bytes` is 65536 for the byte bound | no | | `allagents_workspace_immutable` | HTTP 409 `invalid_request_error` before response allocation when a continuation contains the workspace extension; `param` is `metadata.allagents.workspace` | no | +| `allagents_workspace_expired` | HTTP 410 `invalid_request_error` before runner/profile work while a continuation's expired or deleted session tombstone remains retained; `param` is `previous_response_id` | no | +| `allagents_workspace_non_resumable` | HTTP 409 `invalid_request_error` before profile admission when a known attached session's bound generation key/epoch/reference/publication/private/checkpoint evidence is missing or corrupt; `param` is `previous_response_id`; because the attachment previously reached `ready`, include its committed complete public workspace metadata | no | | `harness_unavailable` / `detail.reason: "allagents_auth_profile_busy"` | HTTP 503 `server_error` before response allocation for a saturated auth profile; `param` is null | yes | | `harness_unavailable` / `detail.reason: "allagents_auth_profile_unavailable"` | HTTP 503 `server_error` before response allocation for an unavailable or repair-required auth binding; `param` is null | yes | -| `allagents_workspace_invalid` | failed response for post-allocation descriptor, catalog, path, layout, OCI image-manifest shape, index, or unsupported/foreign media rejection that is not a numeric limit | no | +| `allagents_workspace_invalid` | failed response for post-allocation descriptor, catalog, path, layout, access/retention value, OCI shape/index, or unsupported media rejection that is not a numeric limit | no | +| `allagents_workspace_persistence_forbidden` | failed response when `persistent` retention is not authorized for the selected deployment target | no | +| `allagents_workspace_read_only` | failed response when a read-only initial request contains workspace input files or attachment policy would create writable shadow state | no | +| `allagents_workspace_capacity_exceeded` | HTTP 503 `server_error` before response allocation when generic session/tombstone admission cannot reserve capacity; otherwise a failed response when finite staging, generation, private, session, or persistence capacity cannot be reserved after safe eviction | yes | +| `allagents_workspace_private_quota_exceeded` | failed response when an editable waiter's initial generation copy cannot fit or an attached editable turn exhausts its fixed per-session byte or inode allowance; access and retention remain unchanged | no | | `allagents_workspace_source_auth_failed` | failed response for Git or registry credential rejection | no | | `allagents_workspace_acquisition_failed` | failed response when Git, registry, HTTP, or transport I/O prevents complete byte acquisition; excludes digest, schema, and limit failures | no | -| `allagents_workspace_limit_exceeded` | failed response for source/workspace/archive/manifest repository, entry, byte, layer, file, path, or header limits; excludes hook request/stdout/stderr envelope limits | no | -| `timeout` | failed response only for an unexpected execution timeout before the declared task budget; a declared time/step budget remains UHP `incomplete` with `error: null` | no | -| `allagents_materializer_failed` | failed response for materializer spawn/nonzero/crash or malformed/oversized hook output | no | -| `allagents_secret_boundary_violation` | failed response when a post-allocation recheck finds a configured source-secret name or value in the service or agent environment | no | -| `allagents_workspace_manifest_invalid` | failed response for the workspace-manifest media type, schema, RFC 8785 bytes, or declared workspace-manifest digest | no | -| `allagents_workspace_integrity_mismatch` | failed response for OCI image/config/layer descriptor digest or size mismatch, or when staging differs from the verified workspace manifest | no | -| `allagents_workspace_publication_failed` | failed response for recoverable staging publication or marker failure | no | -| `allagents_workspace_checkpoint_failed` | failed response for root/nested checkpoint or collection-baseline failure | no | -| `allagents_workspace_state_failed` | failed response for a materialization-state persistence/CAS failure | no | -| `allagents_workspace_containment_breach` | failed response for completed-parent/live-descendant even if forced kill succeeds; an unquiescent leaf first enters internal non-terminal `containment_pending`, exits/restarts, and becomes public only after the old boundary is proven empty, preserving client-cancellation or declared-budget status | no | - -Classification order is normative. A completed-parent/live-descendant violation -is the failed containment code. Any other containment breach supersedes the -hook's earlier result for internal session state but never overwrites -UHP-mandated public `cancelled` or `incomplete`; absent either, it becomes the -failed containment code. Otherwise a declared UHP budget produces `incomplete`; -an unexpected execution timeout produces `timeout`; the post-allocation secret -boundary recheck precedes numeric limits; numeric limits are classified before -generic schema validation; and remaining failures use source authentication, -semantic workspace validation, acquisition I/O, materializer process/envelope -validation, workspace-manifest validation, cryptographic/tree integrity, -publication, checkpoint, then durable-state failure in that order. -One terminal outcome carries exactly one code. Thus a 129-item repository array -is `allagents_workspace_limit_exceeded`, malformed canonical workspace-manifest -bytes are `allagents_workspace_manifest_invalid`, registry transport failure is -`allagents_workspace_acquisition_failed`, and an OCI layer digest mismatch is -`allagents_workspace_integrity_mismatch`. +| `allagents_workspace_limit_exceeded` | failed response for source/archive/manifest repository, entry, byte, layer, file, path, or header limits | no | +| `timeout` | failed response only for an unexpected execution timeout before the declared task budget; declared budget exhaustion remains UHP `incomplete` | no | +| `allagents_materializer_failed` | failed response for validate/resolve/materialize spawn, nonzero, crash, or malformed/oversized hook output | no | +| `allagents_secret_boundary_violation` | failed response when a post-allocation recheck finds a configured source-secret name or value in service or agent state | no | +| `allagents_workspace_manifest_invalid` | failed response for workspace-manifest media type, schema, canonical bytes, or declared digest | no | +| `allagents_workspace_integrity_mismatch` | failed response for source descriptor digest/size mismatch, validated/resolved-plan or generation-key drift, staging/manifest mismatch, semantic Git-state failure, or corrupt ready generation | no | +| `allagents_workspace_publication_failed` | failed response for generation claim/publication/marker failure | no | +| `allagents_workspace_attachment_failed` | failed response for read-only mount/reference or private editable copy/publication failure | no | +| `allagents_workspace_checkpoint_failed` | failed response for editable root/nested checkpoint or collection-baseline failure | no | +| `allagents_workspace_state_failed` | failed response for generation/session/pin/reference/quota/expiry/tombstone/purge persistence or CAS failure | no | +| `allagents_workspace_containment_breach` | failed response for completed-parent/live-descendant even if forced kill succeeds; an unquiescent leaf remains internal until restart proves it empty | no | + +Classification order is normative. For continuation, generic validation rejects +an extension-bearing request before session lookup/CAS. The session CAS then +linearizes busy, exact binding, and expiry/deletion predicates without mutating +state on rejection; a known attached but corrupt physical attachment returns +non-resumable and rolls back its provisional admission before profile admission. +For an initial request, generic metadata errors precede the idempotency claim; +stock `session_busy`, then native-profile availability, then generic session/ +tombstone capacity determine +pre-allocation admission. After allocation, source-free descriptor validation +and selected-reference verification precede secret-boundary recheck, persistence +authorization, read-only conflict, access-specific capacity reservations, source +resolution, build/staging/prospective-generation capacity, acquisition I/O, +post-build full-tree accounting, numeric limits, source semantic validation, +hook envelope, manifest, cryptographic/tree/Git/generation integrity, +publication, per-waiter private fit, attachment, editable checkpoint, private +runtime quota, then durable state failure. A completed-parent/live-descendant +violation is failed +containment; other containment failure supersedes hook state but never overwrites +UHP-mandated `cancelled` or `incomplete`. Declared budget exhaustion produces +`incomplete`; an unexpected earlier timeout produces `timeout`. + +One terminal outcome carries exactly one code. Capacity is retryable because +expiry or operator deletion may free protected space, but no retry delay is +guessed and Promptfoo does not retry automatically. A failed physical GC deletion +is quarantined/accounted operational state; it becomes a task error only when +admission cannot reserve capacity. Deployment `preflight` runs before readiness and allocates no UHP response. Its failure stays operational: readiness is false and the safe operator diagnostic @@ -991,8 +1456,11 @@ and `cancelled` to `code: null` and `retryable: false`. Every failure includes `metadata.uhp = { httpStatus, responseStatus, code, reason, retryable }`. `responseStatus` is null for a pre-allocation non-2xx request error and is the -actual terminal status for an HTTP-200 response. Verified -`metadata.allagentsWorkspace` is included when available; `reason` is the error +actual terminal status for an HTTP-200 response. Before attachment reaches +`ready`, failures omit `metadata.allagentsWorkspace` entirely. After `ready`, +terminal failures include the same complete verified workspace metadata as +success, with the actual terminal expiry value. Internal epoch, reservation, +claim, pin, and physical-path identifiers are never public. `reason` is the error detail reason or the incomplete detail reason when present, otherwise null. Promptfoo never converts a non-2xx, failed, incomplete, or cancelled UHP result into successful empty output and performs no automatic retry. @@ -1019,90 +1487,109 @@ into successful empty output and performs no automatic retry. ### Risks and Mitigations - **Fork drift:** Keep the patch ordered and narrow, pin commits, rebase only - selected releases, and run both upstream and integration suites. -- **Gateway/runner durability split:** Use the explicit materialization CAS plus - workspace marker and pre-agent checkpoint. Fault every boundary and fail - incomplete sessions closed rather than attempting replay. -- **Nested Git versus HarnessRouter root Git:** Keep repository `.git` state, - ignore declared roots in HarnessRouter's root index, and extend produced/list/ - file/ack/checkpoint/hydrate behavior to validate and walk every declared root. -- **Provider retry/fallback:** Run materialization before the provider candidate - loop and surface a typed non-provider failure; a ready marker prevents reruns. + selected releases, and run upstream conformance plus lifecycle E2E. +- **Gateway/runner durability split:** Make the gateway the sole session-state + writer and the runner the sole resource-journal writer. Fault every + claim/publication/pin/prepare/evidence/ready-ack/reference/expiry/deletion CAS; + reconciliation preserves a gateway-committed attachment or rolls resources + back once without replaying acquisition. +- **Generation-key collision or incomplete identity:** Generate the key from a + versioned canonical resolved plan containing every source-visible + byte/layout-affecting input, bind it to one manifest digest and semantic Git + record, and reject drift before reuse. Exclude volatile `.git` representation + only after closed semantic validation. +- **Concurrent build and publication race:** Use one runner-owned keyed claim, + private staging, independent reconstruction, atomic publication, and one + shared result. Each request detaches on its own cancellation/deadline; one + waiter cannot cancel another, and the build stops when no live waiter remains. + Never attach `building`, quarantined, or deleting state. +- **Read-only escape or writable alias:** Keep the generation backing store + owner-writable only, verify mount flags and mount topology, forbid writable + bind aliases and hard-linked private copies, and probe writes through root, + nested repositories, symlinks, and alternate paths. +- **Editable cross-session leakage or growth:** Create a unique private tree and + checkpoint namespace per fitting session, verify inode separation, reject only + a waiter whose initial copy cannot fit, reserve its full byte/inode allowance + from global capacity, enforce that hard quota through every continuation, and + scan produced files against only its private baseline. +- **Nested Git versus mode-specific checkpoints:** Preserve repository `.git` + state inside the generation. Read-only sessions do not mutate or checkpoint + it; editable copies ignore declared roots in the HarnessRouter root index and + extend list/file/ack/checkpoint/hydrate across private nested repositories. +- **Lease, expiry, and deletion races:** Linearize unexpired-idle or persistent + turn admission against tombstoning; hold a provisional pin through attachment + prepare/ack; persist exact epoch references before mount exposure; recheck + reference/pin protection under lock; release exactly once; and reconcile leaked + or under-counted state before readiness or GC. +- **Pinned capacity starvation and tombstone growth:** Configure finite staging, + generation, per-private-session and total private byte/inode, session, + persistent, and tombstone limits plus high/low watermarks. Reserve a tombstone + slot at session admission, compact only after response/idempotency retention, + expire ordinary state, and evict only zero-reference/zero-pin epochs with null + `lastUsedAt` first by `publishedAt`, then used epochs by `lastUsedAt`, + `publishedAt`, key, and epoch. Reject admission when protected state consumes + capacity. +- **Failed deletion or corrupt generation:** Quarantine and continue accounting + for it. Never advertise freed bytes, resurrect physical state, or substitute a + rebuilt generation inside an existing session. +- **Provider retry/fallback:** Complete one attachment before the provider loop + and persist a ready marker; retry cannot resolve, build, attach, or change + source/access/retention/auth mode. - **Source credential leakage:** Use subprocess-only source credentials, - hermetic configuration, leak scans, hostile fixtures, and a non-escapable - cgroup v2 boundary. Prove `populated 0` before interpreting success, reading - result files, publishing, releasing secrets, or cleanup. -- **Native OAuth exposure:** Treat the selected profile as available to the - harness and same-identity tools. Mount no other profile and prevent passive - gateway/runner persistence from serializing the auth file. A malicious harness - or tool can still emit its contents; use explicit proxy mode when this - owner-trust boundary is unacceptable. -- **OAuth refresh loss, races, or queue collapse:** Have the runner supervisor - persist the admission record and hold the zero-waiter per-profile advisory lock - through descendant termination, terminal-state acknowledgement, and refresh - commit. Gateway death cannot release it; runner death leaves a durable fence - that blocks readiness until startup reconciliation. Preserve idempotency and - same-session precedence before cross-session admission. Cross-session overlap - returns `harness_unavailable` with - `detail.reason: "allagents_auth_profile_busy"` before allocation and can never - acquire later. Provision distinct profiles for parallelism. Persist local - writes through same-filesystem temp-write, file `fsync`, atomic rename, parent - `fsync`, and validation. Fault process death around local persistence; if - remote rotation leaves the committed profile invalid, mark it - `repair-required`. Never switch profiles or auth mode, and never run - shared-profile turns concurrently. -- **Partial publication or forged staging:** The materializer writes source - content only to staging and the canonical manifest only to the private result - root. The runner reconstructs and compares the tree, then publishes with - recovery markers; no agent runs until the gateway durably stores the resulting - checkpoint and marks the session ready. -- **Descriptor/session drift:** Accept the key only on the initial request and - persist the hook's effective digest/provenance for every later response. + hermetic configuration, leak scans across staging/generations/private copies, + and a non-escapable cgroup boundary proven empty before result handling. +- **Native OAuth exposure:** Treat the selected profile as available to its + harness and same-identity tools only during an active turn. Use a + same-filesystem namespace projection, mount no other profile, never copy it to + durable session state, and verify teardown before terminal acknowledgement. +- **OAuth refresh loss or concurrency:** Retain the runner-owned zero-waiter + per-profile lock through descendant termination, refresh disposition, + credential-projection teardown, and terminal acknowledgement. Distinct + profiles provide parallelism on one generation; same-profile cross-session + overlap remains fail-fast. Reconcile stale projections and profile state before + reacquisition and mark stale remote rotation `repair-required`. +- **Descriptor/session drift:** Accept the JSON key only initially and persist + descriptor, generation, access, retention, cwd, harness, and auth identity for + every later response. Continuation never re-resolves. - **OCI attack surface:** Use a closed media profile, streaming digest checks, - fixed limits, strict path/link/type validation, and exact-host redirect policy. -- **Catalog scale:** Reject more than 128 repositories, 500,000 entries, 32 GiB - staged content, or work exceeding the request deadline. Exercise the supported - boundary with representative multi-repository fixtures and publish those - limits for Promptfoo operators. -- **Harness auth drift:** Pin Codex and Pi versions and require non-secret login, - live-turn, refresh, and continuation probes before advertising each target. -- **HarnessRouter restart semantics:** Claim persistence only for completed state - on durable storage; interrupted work fails and is not replayed. -- **Provider cache assumptions:** Report native cached-input usage when available; - never promise a cache hit. -- **Registry/tag or publisher compromise:** Use a protected, environment-approved - publish job with pinned actions and no write authority in build/test. Verify - the attested owner/repository/workflow/ref/subject digest before deployment. -- **Upstream rejection:** The pinned fork remains supported; upstream delivery is - maintenance reduction, not a launch dependency. + fixed limits, strict path/link/type validation, and exact-host redirects. +- **HarnessRouter restart semantics:** Promise continuation only for completed, + unexpired or persistent state with valid attachment evidence. Interrupted + turns fail and are not replayed. +- **Registry/tag or publisher compromise:** Use protected publication with pinned + actions and verify attested owner/repository/workflow/ref/subject digest. +- **Upstream rejection:** The pinned fork remains supported; upstream delivery + reduces maintenance but is not a launch dependency. ### Phased Delivery -1. Phase-zero native-auth adapter spike: build the pinned minimal image without - the AllAgents materializer and implement only auth-profile projection, - persisted binding identity, per-profile serialization, and passive exclusion. -2. Run the blocking Codex and Pi native-auth gate with real provider traffic: - login, first turn, continuation, restart, changed-binding failure, overlapping - turns, refresh faults, profile isolation, and checkpoint/log/output scans. If - either required target fails, stop and revisit ADR 0002 before workspace work. -3. Red E2E against stock HarnessRouter: prove arbitrary metadata is neither - forwarded to Codex/Pi nor returned as workspace provenance. -4. Workspace fork spike: prove a fake hook runs through a dedicated pre-provider - operation, publishes/checkpoints once, survives two-turn reuse, supports a - safe nested cwd, reports nested-repository files, and cannot rerun under - provider fallback. -5. Freeze generic hook envelope/state/auth fixtures, canonical workspace-manifest - bytes, and AllAgents descriptor, configuration, provenance, and failure - fixtures. -6. Implement project schema projection, preflight, Git materialization, - credential containment, bounded catalog acquisition, and - checkpoint/collection integration. -7. Implement OCI materialization and its archive/registry security profile. -8. Prove the separately configured authenticated-proxy mode, then run Promptfoo - one-shot, continuation, cancellation, restart, and failure mappings. -9. Review both repositories; publish the exact GHCR image; verify and deploy its - attested digest; run green E2E and conformance; document operations; prepare - the generic upstream patches. +1. Build the pinned minimal image and pass the blocking Codex/Pi native-auth gate + without workspace code. +2. Red E2E against stock HarnessRouter: arbitrary metadata is neither forwarded + to Codex/Pi nor returned as workspace provenance. +3. Workspace lifecycle fork spike: a fake `preflight/validate/resolve/materialize` + hook, concurrent identical epoch claims, independently cancelled waiters, one + immutable publication, provisional pins, two read-only mounts, one quota- + bounded private editable copy, per-waiter copy-fit failure, attachment + prepare/ack, epoch eviction/republication, and restart reconciliation. +4. Prove read-only enforcement, writable-copy isolation and growth limits, + mode-specific checkpoint/collection, nested cwd, provider-fallback non-reentry, + and unexpired/persistent continuation reuse. Stop if any invariant needs prompt + or client cooperation. +5. Freeze hook/state, generation-key, manifest, semantic Git, descriptor, + response/expiry, retention, failure, and lifecycle fixtures. +6. Implement `workspace.yaml` source projection, Git validate/resolve/materialize, + credential containment, bounded acquisition, and generation publication. +7. Implement OCI generation construction through the same publication and + attachment path. +8. Implement terminal-time TTL, persistent authorization, operator deletion, + hard private quotas, bounded tombstones, provisional-pin-aware deterministic + LRU eviction, and crash recovery. +9. Prove explicit proxy mode and run Promptfoo concurrent read-only, isolated + editable, continuation, expiry/deletion, capacity, cancellation, restart, and + failure mappings. +10. Review both repositories; publish and attest the GHCR digest; run green E2E + and conformance; document lifecycle operations; prepare upstream patches. --- @@ -1118,14 +1605,16 @@ into successful empty output and performs no automatic retry. focused runner/gateway tests. Do not add the AllAgents materializer or Git/OCI acquisition in this unit. - **Approach:** Initialize dedicated profiles only through `codex login` and Pi - `/login`. Project the selected auth files into session-specific homes while - keeping conversation state session-scoped. Persist the binding identity/digest, - serialize every refresh-capable turn per profile, preserve atomic local writes, - mark invalid post-rotation state `repair-required`, mount no other profile, and - prevent passive checkpoint/log/output serialization. Use the actual pinned - harness versions and real provider traffic. - Freeze and record the HarnessRouter commit, base-image digest, Codex version, - Pi version, and auth-adapter patch digest used by the gate. + `/login`. Use an active-turn-only directory-level mount namespace or equivalent + same-filesystem credential view while keeping conversation state + session-scoped. Persist binding identity/digest, serialize every + refresh-capable turn per profile, preserve atomic local writes, mark invalid + post-rotation state `repair-required`, tear down and verify the projection + before terminal acknowledgement, mount no other profile, and prevent passive + checkpoint/backup/log/output serialization. Use the actual pinned harness + versions and real provider traffic. Freeze and record the HarnessRouter commit, + base-image digest, Codex version, Pi version, and auth-adapter patch digest used + by the gate. - **Verification:** For both Codex and Pi, complete login, a real first turn, continuation, and restart without a provider-route API key. Change or remove the binding and prove continuation fails before runner work. Use a barrier to @@ -1136,17 +1625,18 @@ into successful empty output and performs no automatic retry. waiters receive any pre-allocation owner error before claim removal; no rejected cross-session request can acquire later. Record representative turn duration and the one-active-turn-per-profile, zero-waiter operator capacity rule. Inject - materializer, publication, checkpoint, provider, cancellation, gateway-only - crash, runner crash, and whole-process-death faults; after each, prove - reconciliation completes before the next new turn acquires the runner-owned - fenced profile lock. - Terminate before, during, and after local refresh persistence; restart must see - a complete file that validates or becomes `repair-required`. Prove unselected - profiles and other sessions' conversation state are inaccessible. With an - inert agent, scan checkpoints, produced-file records, passive logs, and response + provider, cancellation, gateway-only crash, runner crash, and + whole-process-death faults; after each, prove descendants are empty, stale + projections are removed or quarantined, retained CLI homes contain no + credential path, and reconciliation completes before the next new turn + acquires the runner-owned fenced profile lock. Terminate before, during, and + after local refresh persistence; restart must see a complete file that + validates or becomes `repair-required`. Prove unselected profiles and other + sessions' conversation state are inaccessible. With an inert agent, scan + checkpoints, produced-file records, backups, passive logs, and response metadata for automatic credential serialization. Record that an active - same-identity tool can still read or emit the selected credential. - Preserve those exact input identities with the evidence. + same-identity tool can still read or emit the selected credential. Preserve + those exact input identities with the evidence. - **Gate:** U1-U6 must not begin until both required native targets pass. Failure stops dependent work and reopens ADR 0002; proxy-only scope requires an explicit decision change and cannot count as a passing native gate. Any change @@ -1155,187 +1645,190 @@ into successful empty output and performs no automatic retry. ### U1. HarnessRouter fork and hook feasibility -- **Goal:** Prove the smallest production-direction workspace fork can - materialize and durably checkpoint one workspace before provider dispatch - while preserving stock UHP requests. +- **Goal:** Prove the smallest production-direction fork can atomically publish + one immutable generation, attach it in both access modes, and reconcile its + lifecycle before provider dispatch while preserving stock UHP. - **Repositories/files:** HarnessRouter fork `gateway/app.py`, - `runner/server.py`, response/session persistence, checkpoint/produced-file - helpers, runner/gateway tests, and a fake materializer hook. -- **Approach:** Add configured opaque metadata extraction and bounds, generic - result envelope, generated workspace-manifest schema and frozen canonical - fixture, materialization CAS, a dedicated runner operation before the provider - loop, response-translator persistence, safe nested cwd, staged publication, - pre-agent checkpoint, runner-owned cgroup v2 containment, and - nested-repository collection. -- **Verification:** Upstream UHP conformance stays green and stock requests are - unchanged. Faults at every state/publication/checkpoint boundary fail closed - with the exact catalog code: materializer process/envelope, publication/marker, - checkpoint/baseline, state/CAS, and secret-boundary fixtures cover their rows. - The runner independently rejects a forged manifest, changed staging entry, - escaping link, undeclared or 129th repository, duplicate/missing repository - destination, repository destination naming a file or symlink, or invalid - private manifest path. `clone3(CLONE_INTO_CGROUP)` and stopped pre-exec - fallback fixtures immediately fork, call `setsid()`, and double-fork; cases - cover cancellation, deadline, and a `completed` parent whose descendant - attempts a delayed write. All prove `populated 0` before any terminal result, - manifest read, publication, secret release, or cleanup. The completed-parent - violation - returns the containment code even after successful kill. An unquiescent fixture - proves internal `containment_pending`, bounded acknowledgement, nonzero runner - exit, `on-failure` restart, old-boundary emptiness, and readiness held false - through orphan sweep. GET and stream remain non-terminal until reconciliation, - after which mandated cancellation/budget or failed-containment status appears. - Provider fallback cannot rerun the - hook. A continuation reuses workspace, provenance, and the persisted auth - binding without the extension. Root and nested repository files collect - correctly, and an escaping cwd fails. + `runner/server.py`, generation/session persistence, mount/copy and + checkpoint/produced-file helpers, runner/gateway tests, and a fake + preflight/validate/resolve/materialize hook. +- **Approach:** Add opaque metadata bounds, typed operation envelopes, + runner-owned authorization/admission, generation-key/epoch claims with + independent waiter cancellation, separate generation/resource and gateway + session CAS state, canonical manifest fixtures, atomic publication, + provisional pins, attachment prepare/ack, durable epoch references, verified + read-only mounts, per-waiter copy-fit and unique hard-quota-bounded editable + copies, mode-specific checkpoint/collection, safe nested cwd, stage-dependent + response metadata, and cgroup containment. Add fake finite TTL, persistence, + tombstone, quota, deletion, and epoch-republication state sufficient to prove + restart ordering; U3 completes production policy and GC. +- **Verification:** Upstream UHP conformance stays green. Two concurrent + identical read-only initial requests execute fake materialize once, attach the + same generation under separate UIDs and harness/profile bindings, deny writes + through root/nested/symlink/alternate paths, and isolate runtime state. Two + editable sessions receive inode-independent private trees; one mutation and + checkpoint never appears in the other or generation. Continuation reuses its + original mode and state without the extension. + + Fault every validate/resolve/claim/waiter/containment/publication/pin/ + prepare/ready-ack/reference/mount/copy/quota/checkpoint/CAS boundary. The runner + rejects forged manifests, changed staging, escaping links, invalid repository + destinations, writable aliases, and generation-key drift. Restart exposes only + a complete publication plus valid attachment evidence; provider fallback never + invokes the hook again. ### U2. AllAgents workspace contracts and Git materializer -- **Goal:** Implement the versioned schemas, authoritative catalog projection, - deterministic Git acquisition, logical cwd resolution, and provenance. +- **Goal:** Implement the JSON descriptor, source-only `workspace.yaml` catalog, + canonical generation identity, deterministic Git construction, logical cwd, + and provenance. - **Files:** `src/models/workspace-config.ts`, `src/models/execution-workspace.ts`, `src/core/execution-workspace.ts`, - `src/core/workspace-repo.ts`, `src/cli/commands/workspace.ts` or one narrowly - registered integration command, generated workspace schemas, build packaging, - configuration documentation, unit fixtures, and Git E2E fixtures. + `src/core/workspace-repo.ts`, one narrow CLI integration entrypoint, generated + schemas, build packaging, configuration docs, and Git E2E fixtures. - **Approach:** Reuse authoritative workspace parsing and source normalization. - Extend the project schema with strict snapshot and environment credential - references; generate the normative workspace-manifest schema; add - descriptor/hook/result schemas, defaults, canonicalization, and a `preflight` - mode. Expose a direct no-shell materializer entrypoint. Resolve the complete - execution-eligible catalog to exact commits in staging, preserve nested - `.git`, enforce the closed Git policy, validate destinations and cwd, compute - the workspace manifest, and return without publishing. -- **Verification:** Local HTTPS fixtures cover branches, tags, commits, PR refs, - configured defaults and symbolic HEAD, multiple repositories, optional-name - fallback, conflicting/originless/local sources, root/duplicate destinations, - unknown names, missing directories, traversal, leading-dash/control/refspec - revisions, ambiguous shorthand, hooks/helpers/filters, submodules, LFS, - file/ext protocols, redirects, cancellation, timeout, partial cleanup, - canonical defaults, preflight failures, exact provenance, generated-schema - validation, frozen canonical fixture bytes/digest, manifest reconstruction, - and the 128-repository, 500,000-entry, 32-GiB, and deadline boundaries. + Keep access/retention out of `workspace.yaml`; add them to the JSON execution + descriptor. Generate the manifest schema and add descriptor/preflight/ + validate/resolve/materialize/result schemas, defaults, canonicalization, + generation-key construction, and credential-reference selection. Preflight + returns configured reference identities without values; validate selects a + bounded subset without source access; the runner verifies their handles and + injects only that selected set into source-access children. Resolve the complete + catalog to exact commits without writing source bytes, and materialize only the + exact cache-miss resolved plan into staging. Preserve nested `.git` while + excluding volatile administrative bytes from the source-visible manifest, + enforce closed semantic Git validation, and prove the manifest equals the union + of resolved commit trees at pairwise non-overlapping destinations plus necessary + ancestor directories. Validate destinations/cwd, compute the manifest, and + return without publishing. +- **Verification:** Local HTTPS fixtures cover refs/defaults/HEAD, multiple + repositories, catalog errors, duplicate and ancestor/descendant destinations, + undeclared root/side files, revision grammar, helpers, submodules/LFS/file + protocols, redirects, cancellation, partial cleanup, descriptor defaults, + access/retention validation, configured/selected credential-reference identity + and secret-free validation, generation-key inclusion and exclusion rules, + exact provenance, schema fixtures, commit-tree/manifest reconstruction, + concurrent identical resolve identity, and repository/entry/byte/deadline + boundaries. Different cwd/access/retention/harness/profile/session inputs + produce the same generation key only when resolved source, sharing scope, and + selected credential-reference identities match. ### U3. Session binding, failures, and credential containment -- **Goal:** Make the fork/materializer boundary durable, fail-closed, and safe for - continued sessions. -- **Repositories/files:** HarnessRouter session/response persistence and tests; - AllAgents credential-selection/environment code and hostile fixtures. -- **Approach:** Persist `unbound/materializing/containment_pending/ready/failed`, - opaque request - digest, effective descriptor digest, public provenance, workspace marker, - checkpoint digest, and canonical auth-binding identity/digest through - compare-and-set transitions. Extend every response construction/retrieval/ - replay path with identical public metadata and the exact failure catalog. - Resolve source `${ENV_VAR}` references only when constructing the materializer - child from an owner-only runner secret source; reject configured names or - values in the base service or agent environment. Use a delegated, - child-inaccessible cgroup v2 leaf and prove it empty before accepting any hook - outcome, reading result files, publishing, releasing secrets, or cleaning - staging. In native mode, project only the selected harness OAuth profile and - exclude it from passive session persistence. In explicit proxy mode, broker - the proxy client key with the required audience, target, model, turn, expiry, - and revocation constraints. Scan workspace, nested Git, CLI session state, - checkpoints, logs, and responses. -- **Verification:** Initial idempotent replay preserves one result; continuation - omits the extension and reuses ready state plus the exact auth binding; - extension-bearing continuation or changed/unavailable binding fails. Crashes - around hook/publication/checkpoint/CAS reconcile to ready only when the bound - descriptor, published marker, and durable checkpoint all match; missing or - mismatched evidence becomes failed/non-resumable with the exact materializer, - publication, checkpoint, or state code. Cancellation, deadline, malformed - output, completed-parent/live-descendant, and runner-shutdown fixtures leave - the cgroup empty before cleanup. A recovered completed-parent violation returns - `allagents_workspace_containment_breach`; failure to prove emptiness records - internal `containment_pending`, completes the internal gateway handshake or - bounded timeout, exits/restarts the runner, and withholds readiness plus every - terminal GET/stream result until the old boundary is proven empty. Reconciled - cancellation/budget retains its mandated public status; other cases become - failed containment. Startup and post-allocation secret-boundary fixtures - respectively block readiness with no UHP response and return - `allagents_secret_boundary_violation`. Source secrets and the HarnessRouter - caller key are absent from the base service and every shell-enabled agent path. - The selected OAuth profile is available only through its harness home; other - profiles are inaccessible. An inert-agent probe confirms no gateway/runner - path automatically serializes it into checkpoints, produced-file records, - passive logs, or public metadata; an active tool can still exfiltrate it in - owner-trust mode. Native mode has no provider-route API key. Proxy tests allow - bounded in-turn provider calls and reject every out-of-scope, expired, or - revoked broker token. +- **Goal:** Complete crash-safe attachment retention, bounded disposal, exact + failures, and credential containment for continued sessions. +- **Repositories/files:** HarnessRouter generation/session/reference persistence, + lifecycle scheduler and operator deletion path, quota/GC configuration and + tests; AllAgents credential environment and hostile fixtures. +- **Approach:** Persist runner generation/resource state separately from the + gateway-owned session state; use attachment prepare/evidence/ready-ack and + reconcile both halves. Persist raw/effective descriptor digests, generation + key/epoch/manifest/semantic-Git/provenance, published/last-used timestamps, + access, retention, expiry, attachment evidence, and auth binding. Implement + active leases, terminal-time idle expiry, bounded tombstones and purge, + authorized persistent pins, provisional attachment pins, fixed private + byte/inode reservations and runtime enforcement, per-waiter initial copy fit, + editable cleanup, read-only epoch-reference release, deterministic eviction of + ready zero-reference/zero-pin epochs, completed-eviction fencing before + republication, deletion quarantine, and startup reconciliation. Extend every + response/replay path and Promptfoo mapping with the exact failure catalog and + stage-dependent public metadata. + + Resolve source secrets only in the selected hook child, prove its cgroup empty + before results/publication/cleanup, project only the selected native OAuth + profile during the active turn, tear down and verify it before terminal + acknowledgement, and broker proxy mode separately. Scan staging, generations, + private workspaces, retained homes, mounts, checkpoints, backups, logs, and + responses. +- **Verification:** Fake-clock and tiny-quota fixtures prove generic + session/tombstone admission precedes visible response creation, invalid + descriptors remain accounted through failed-response purge, and one CAS + rejects busy/expired continuation admission while saving and clearing an + unexpired idle deadline. Profile admission either commits active or restores + the future deadline/tombstones an elapsed one; only terminal acknowledgement + sets the next deadline. Polling/replay do not; expiry races linearize; + persistent requests authorize before source access; explicit deletion is + idempotent; active, referenced, and pinned state is never evicted; one + editable session produces exactly one private reservation debit across success + and every crash point; + one non-fitting editable waiter fails without affecting a read-only or fitting + sibling; byte/inode growth fails at that allowance across continuations; + expired private workspaces release it; references and provisional pins release + exactly once; repeated successful publications return staging reservation to + baseline; null-last-used epochs sort first by publication time, then used + epochs by last-used time, with key/epoch tie-breaks; a new epoch waits for + complete prior eviction; bounded tombstones purge only after response/ + idempotency retention; failed deletion stays quarantined; and all-protected + capacity returns the cataloged retryable failure. + + Crash every generation-epoch/session/build-waiter/pin/prepare/ready-ack/ + reference/mount/copy/quota/tombstone/unmount/purge/delete transition and require + reconciliation before readiness or GC. Continuation succeeds only for valid + exact-epoch evidence whose retention is persistent or session idle deadline is + unexpired; retained expiry returns 410, corrupt evidence returns 409 + non-resumable, and purged identity returns stock unknown, all without + rematerialization or epoch substitution. Credential fixtures prove source + secrets and caller keys absent everywhere agent-readable; the selected OAuth + credential is visible only through the active-turn owner-trust projection and + is absent after teardown. Proxy tokens reject every invalid scope or lifetime. ### U4. Immutable OCI workspace materialization -- **Goal:** Add the second closed source mode without weakening Git behavior or - allowing fallback. -- **Files:** AllAgents OCI client, manifest/archive validator, workspace-manifest - types, deterministic snapshot producer fixture, local registry E2E, and - security fixtures. -- **Approach:** Resolve only configured registries, implement bounded - Basic/Bearer authentication and exact-host redirect policy, verify the direct - manifest, canonical workspace-manifest blob, and layers while streaming, apply - changesets in staging, validate paths/types/limits/catalog, reconstruct the - canonical manifest, and return through the same hook envelope as Git. The - HarnessRouter runner remains the sole publisher. -- **Verification:** Local Distribution fixtures cover anonymous and authenticated - pulls, private CA, gzip/zstd, whiteouts, redirects, rebinding policy, indexes, - unknown/foreign media, traversal, escaping links, devices, sparse files, - cancellation, cleanup, and no Git fallback. They assert exact precedence: - transport failure is `allagents_workspace_acquisition_failed`; an OCI index, - malformed image-manifest shape, unknown/foreign media, or - duplicate/missing/non-directory repository destination is - `allagents_workspace_invalid`; 129 repositories or any other numeric overflow - is `allagents_workspace_limit_exceeded`; workspace-manifest media/schema/ - canonical-byte/declared-digest failure is - `allagents_workspace_manifest_invalid`; and OCI image/config/layer digest/size - or final staged-tree mismatch is - `allagents_workspace_integrity_mismatch`. +- **Goal:** Add OCI as the second immutable generation source without weakening + Git reuse, attachment, retention, or failure behavior. +- **Files:** AllAgents OCI client, manifest/archive validator, + workspace-manifest types, deterministic producer fixture, local registry E2E, + generation fixtures, and security fixtures. +- **Approach:** Resolve only configured registries; implement bounded + Basic/Bearer auth and exact-host redirects; compute the immutable resolved plan; + on a generation miss verify manifest/config/workspace-manifest/layers while + streaming; reject `.git` administrative subtrees; apply staging changesets; + validate paths/types/limits/catalog; and return through the same envelope as + Git. The runner remains the sole publisher/resource preparer and the gateway + the sole session-attachment writer. +- **Verification:** Distribution fixtures cover auth, private CA, compression, + whiteouts, redirects, rebinding, indexes, foreign media, traversal, links, + devices, sparse files, cancellation, cleanup, no Git fallback, and exact error + precedence. Concurrent identical OCI requests produce one publication; + read-only sessions share it; editable sessions get private copies; access, + retention, cwd, harness/profile, and session do not fragment its generation + key. ### U5. Harness-native OAuth, optional proxy, and Promptfoo E2E -- **Goal:** Carry the phase-zero auth invariants unchanged into the complete - workspace image and prove session continuity, optional proxy isolation, and - consumer success/failure mapping. -- **Repositories/files:** custom image/configuration, auth-profile setup and - projection, HarnessRouter integration fixtures, AI Evals Promptfoo - provider/configuration in its owning repository, and deployment examples in - AllAgents docs. -- **Approach:** Reuse the accepted U0 adapter and fixtures; do not redesign the - native credential boundary here. Configure dedicated Codex and Pi auth roots - and bootstrap them only through each harness's login flow. Exercise login - status, live turns, atomic local refresh persistence, stale-credential repair - after remote rotation, same-binding continuation, serialized overlapping - turns, profile isolation, and missing/revoked credential failure. - In a separate explicit deployment profile, validate the closed proxy connection - and broker audience/target/model/turn/expiry/revocation contract. Run Promptfoo - Git/OCI one-shot and two-turn cases plus materializer, authentication, and - agent failures. -- **Verification:** Through the exact container network, Codex and Pi authenticate - through their own OAuth sessions and use allowed models. Turn two sees turn - one's conversation and file mutation with the same persisted auth binding. A - barriered pair of simultaneous first arrivals with one `Idempotency-Key` - produces one admission and one result; a new same-session continuation returns - `session_busy`; and genuinely new cross-session turns sharing the profile fail - immediately with cataloged `harness_unavailable` before allocation. Same-key - waiters receive any pre-allocation owner error before claim removal; none starts - a second execution. Cross-session callers are not queued. After each - materializer, publication, checkpoint, provider, cancellation, gateway-only - crash, runner crash, and whole-process-death fault, prove reconciliation - completes before the next new turn acquires the runner-owned fenced profile - lock. A config - change or unavailable binding fails before runner work. Cancellation terminates - the real turn. OAuth failure disables the target without selecting another - profile or proxy. Faults around local refresh persistence leave a complete file - that either validates or marks the profile `repair-required`; unselected - profiles remain - inaccessible. The separately configured proxy permits bounded multi-request - use inside the active turn and rejects wrong-audience, wrong-target, - wrong-model, wrong-turn, expired, or revoked credentials. Promptfoo exercises - every catalog row plus UHP incomplete/cancelled outcomes and returns either - successful output/usage/artifacts/provenance or the exact - `ProviderResponse.error`/metadata mapping, preserving the error code when one - exists and terminal status otherwise, never empty success or automatic retry. +- **Goal:** Carry auth invariants into the complete image and prove concurrent + generation sharing, editable isolation, lifecycle policy, proxy isolation, and + consumer mappings. +- **Repositories/files:** custom image/configuration, auth-profile setup, + HarnessRouter lifecycle/integration fixtures, AI Evals Promptfoo provider + configuration, and deployment examples. +- **Approach:** Reuse the accepted U0 auth adapter. Configure dedicated Codex and + Pi profiles; exercise login, live turns, atomic refresh, active-turn projection + teardown, stale repair, same-binding continuation, same-profile fail-fast + exclusion, different-profile concurrency, and profile isolation. Separately + validate proxy broker scope. Run Promptfoo Git/OCI read-only concurrency, + independently cancelled shared-build waiters, editable two-turn growth and + cross-trial isolation, persistence, expiry/deletion/purge, capacity, restart, + cancellation, and every failure mapping. +- **Verification:** Codex and Pi use native OAuth without a provider-route key. + Concurrent sessions with different profiles and harnesses share one read-only + generation while conversation/home/log/output state remains isolated. + Same-profile cross-session overlap retains the cataloged fail-fast result; + same-session overlap returns `session_busy`; idempotent duplicates share one + admission/result. Editable turn two sees turn one's mutation; a different trial + sees a clean private copy. + + Real-image lifecycle probes cover terminal-time TTL, retained-expiry HTTP 410, + purged-predecessor stock failure, HTTP 409 non-resumable, authorized + persistence before acquisition, operator deletion, provisional-pin protection, + hard private byte/inode enforcement across continuations, deterministic + generation eviction, bounded tombstones, deletion quarantine, and restart + reconciliation. Read-only write/input probes cannot copy up. Success, failure, + cancellation, and crash probes find no credential projection after terminal + acknowledgement. All generation, attachment, lifecycle, materializer, provider, + cancellation, and crash failures reconcile before the next profile admission. + Promptfoo returns exact coded errors and metadata, never empty success or + automatic retry. ### U6. Release, operations, review, and upstream preparation @@ -1344,31 +1837,30 @@ into successful empty output and performs no automatic retry. - **Repositories/files:** GHCR image build/release workflow, dependency lock and provenance record, fork-maintenance guide, deployment/reference docs, changelog, PR descriptions, and upstream patch series. -- **Approach:** Build from exact upstream/fork/AllAgents/agent inputs. Every base - image is digest-pinned; runtime lockfiles and version-locked OS packages, - Git/OCI tools, Codex, and Pi close the input set. U6 must use the input - identities frozen by the current U0 evidence. If any covered input must change, - stop release work and rerun U0 before resuming. The build fails on unpinned - input. Replace the inherited Docker Hub path with a no-write build/test job and - a separate protected, environment-approved GHCR publish job. Pin every - third-party action by commit. Publish the `linux/amd64` image to - `ghcr.io/allagentsdev/harnessrouter`, read back its manifest, create - build-provenance and SBOM attestations for the final digest, and run conformance - plus E2E only after verifying the expected repository, workflow, approved ref, - subject digest, predicates, complete build inputs, and anonymous pull. Document - durable session/auth volumes, native login/repair, optional proxy mode, private - networking, upgrades, rollback, secret-safe backup, and the CE owner-trust - boundary. Review both codebases before final green E2E. Split and explain the - generic HarnessRouter patches for upstream. -- **Verification:** A clean `linux/amd64` host verifies both attestations and the - pinned base/runtime/OS/Git/OCI/harness input set, then anonymously pulls the - public package by subject digest and reproduces Git, OCI, native Codex/Pi, - optional proxy, one-shot, continuation, cancellation, restart, and failure - scenarios from documented commands. No Docker Hub credential is required. An - unapproved ref or mismatched owner, repository, workflow, digest, attestation, - predicate, or build input fails closed. Rebase rehearsal against the selected - next upstream commit either passes or reports an explicit incompatibility - before release. +- **Approach:** Build from exact upstream/fork/AllAgents/agent inputs. Pin base + images, lockfiles, OS packages, Git/OCI tools, Codex, and Pi. U6 uses the + identities frozen by current U0 evidence; changing one stops release and + reruns U0. Replace inherited Docker Hub publication with separated no-write + build/test and protected GHCR publish jobs using commit-pinned actions. Publish + `linux/amd64`, read back the manifest, and attach verified build-provenance and + SBOM attestations before E2E. + + Document durable generation/session/private/auth volumes; JSON descriptor + versus project `workspace.yaml`; native login/repair and active-turn projection + teardown; proxy mode; terminal-time TTL, hard private quotas, provisional pins, + watermark, persistence authorization, deletion, bounded tombstone compaction, + quarantine, deterministic GC, capacity, metrics, backup, upgrade, and rollback + procedures; and the owner-trust boundary. Review both repositories before + final green E2E and prepare generic generation/attachment/lifecycle and + auth-state patches for upstream. +- **Verification:** A clean `linux/amd64` host verifies attestations and pinned + inputs, anonymously pulls by digest, configures finite lifecycle policy, and + reproduces Git/OCI generation reuse, cross-harness/profile read-only + concurrency, editable isolation, continuation, persistence, expiry/deletion, + capacity pressure, GC, cancellation, restart, native auth, proxy, and every + documented failure. No Docker Hub credential is required. Wrong provenance, + build input, lifecycle configuration, or unverified generation store prevents + readiness or release. Rebase rehearsal reports incompatibility before release. --- @@ -1376,86 +1868,101 @@ into successful empty output and performs no automatic retry. | Gate | Required evidence | |---|---| -| Native-auth feasibility | Before workspace-materializer production work, the minimal image proves real Codex and Pi login, first turn, continuation, binding persistence, profile isolation, serialized overlap, refresh-fault repair, and passive exclusion without a provider-route API key. Evidence records the HarnessRouter commit, base-image digest, Codex and Pi versions, and auth-adapter patch digest. Both required targets pass; proxy mode is not substitute evidence, and changing a recorded input invalidates the gate until both pass again. | -| Stock compatibility | Upstream HarnessRouter tests and UHP conformance pass; requests without the configured metadata key are unchanged. | -| Caller authentication | Every unauthenticated external create, continuation, retrieval, stream, cancellation, file, and artifact request fails before resource existence or metadata disclosure; runner operations are private and mutually authenticated. | -| Hook ordering | Dedicated materialization finishes, publishes, and checkpoints before provider selection; fallback never reruns it. | -| Manifest integrity | Git and OCI share the generated normative schema and frozen canonical fixture. The runner rejects forged manifests, changed staging, unsupported modes/types, escaping links, undeclared or 129th repositories, duplicate/missing/non-directory repository destinations, invalid private paths, and digest or byte mismatches before publication. | -| Materializer containment | Atomic-placement, immediate-fork, `setsid()`, and double-fork fixtures prove the delegated cgroup reaches `populated 0` before any terminal result, manifest read, publication, secret release, or cleanup. A completed-parent/live-descendant violation returns the containment code after successful kill. An unquiescent boundary enters internal non-terminal `containment_pending`, exits/restarts the runner, withholds terminal GET/stream results and readiness until old-boundary emptiness, then exposes the mandated cancellation/budget status or failed containment. | -| Capacity envelope | Native profiles admit one active refresh-capable turn and zero cross-session waiters. UHP precedence is atomic: simultaneous duplicate idempotency keys share one claim/admission/result, new same-session overlap returns `session_busy`, and only genuinely new cross-session overlap fails with cataloged `harness_unavailable` before allocation. Pre-allocation claim errors reach current waiters before claim removal. After every post-admission fault, reconciliation precedes reacquisition of the runner-owned fenced lock. Git and OCI reject more than 128 repositories, 500,000 entries, 32 GiB staged content, or work beyond the request deadline. | -| Durable state | Fault injection proves only sessions with matching bound descriptor, published marker, and durable checkpoint become ready; missing/corrupt/mismatched evidence fails non-resumable without replay. | -| Session continuity | Two real turns share native conversation, writable workspace, and the persisted auth-binding identity/digest; continuation omits the extension. A changed or unavailable binding fails before runner work. | -| Workspace integration | Safe nested cwd, repository-mode root/nested Git checkpoints, snapshot private tree baselines, produced list/file/ack, hydrate, and initial-source suppression pass. | -| Git acquisition | Closed transport/config policy, constrained revisions, exact commits, non-root destinations, catalog validation, and partial cleanup pass against local HTTPS remotes. | -| OCI acquisition | Digest/media/path/link/type/limit matrix passes against a real local registry. | -| Credential boundary | Every configured source secret and the HarnessRouter caller key are absent from the base service and every agent-readable source/session path, checkpoint, log, and public output. The selected OAuth profile is available inside the documented harness owner-trust boundary; other profiles are inaccessible, and an inert-agent probe proves the gateway/runner never serializes the auth file automatically. | -| Provider boundary | Codex and Pi login, live-turn, atomic local refresh, crash/stale-profile repair, bound continuation, atomic idempotency/session/profile admission, and failure probes use harness-native OAuth without a provider-route API key. OAuth failure never changes profile or auth mode. Simultaneous duplicate keys share one admission/result, same-session overlap returns `session_busy`, and new cross-session profile overlap returns cataloged `harness_unavailable` before allocation. A separate proxy broker permits bounded in-turn calls and rejects wrong-scope, expired, or revoked tokens. | -| Lifecycle | Streaming, cancellation, idempotency, artifacts, usage, response metadata, completed restart, and interrupted-work failure match the contract. | -| Packaging | The public `linux/amd64` GHCR manifest and GitHub/Sigstore build-provenance and SBOM attestations are verified for expected owner, repository, workflow, approved ref, subject digest, predicates, base-image digest, runtime lockfiles, OS packages, Git/OCI tools, and harness versions. Deployment uses that digest and publishing needs no Docker Hub credential. | -| Consumer | Promptfoo one-shot/two-turn Git/OCI success passes. Every failure-catalog row and UHP failed/incomplete/cancelled result maps to the exact `ProviderResponse.error` and metadata, preserving a wire code when present and terminal status otherwise, never empty success or automatic retry. | -| Review | Final review findings in both repositories are resolved before the final built-image E2E. | +| Native-auth feasibility | Before workspace work, the minimal image proves real Codex and Pi login, continuation, binding persistence, profile isolation, mutually exclusive same-profile turns, refresh repair, active-turn projection teardown on success/cancel/crash, and passive exclusion from checkpoints/backups/logs. Recorded pinned inputs invalidate the gate when changed. | +| Stock compatibility | Upstream HarnessRouter tests and UHP conformance pass; requests without the metadata key are unchanged. | +| Caller authentication | Every external create, continuation, retrieval, stream, cancellation, file, artifact, and lifecycle administration path authenticates before existence or metadata disclosure. | +| Generation ordering | Generic session/tombstone admission precedes response visibility; secret-free preflight/validate and selected-reference verification plus access-specific authorization/reservation precede resolve. A miss reserves staging/prospective generation before acquisition; containment, full-tree accounting, commit-tree/Git/manifest verification, and atomic accounting conversion precede ready state/pins. Runner prepare plus gateway ready-ack precede provider dispatch; fallback never reenters. | +| Shared-build cancellation | One request cancellation/deadline detaches only that waiter. A build continues for remaining live waiters, stops when none remain or its runner-owned deadline expires, and produces at most one publication/failure for its epoch. | +| Manifest integrity | Git and OCI share the normative source-visible schema and fixture. Git administrative bytes are omitted only after exact semantic validation, and repository-mode content must equal the union of resolved commit trees at non-overlapping destinations plus necessary ancestors; OCI rejects `.git`. Plan/key drift, undeclared paths, forged manifests, changed staging, invalid paths/types/links/destinations, and digest mismatches fail before publication. | +| Shared read-only generation | Concurrent sessions using different harnesses/profiles share one exact generation epoch. Root, nested, symlink, and alternate-path writes fail; runtime/session/auth/output state remains isolated. | +| Editable isolation | Every fitting editable trial receives an inode-independent private tree and reserved hard byte/inode allowance covering overlays/checkpoints/produced state. A non-fitting waiter fails alone; continuation preserves a fitting trial's mutations but cannot grow past its envelope; siblings and the generation remain unchanged. | +| Materializer containment | Fork/double-fork/cancellation/deadline fixtures prove `populated 0` before result read, publication, secret release, or cleanup. `containment_pending` blocks terminal visibility/readiness through restart and resolves once after quiescence. | +| Capacity envelope | Native profiles retain one active turn and zero waiters. Source build limits and finite staging/generation/private-byte/private-inode/session/persistence/tombstone quotas reject overflow. Invalid descriptors cannot bypass generic admission; one editable session creates one private debit; successful publication releases staging capacity. References and provisional pins prevent eviction; all-protected capacity returns the cataloged retryable failure. | +| Durable lifecycle | Fault injection covers generic and provisional turn admission, generation epochs, build/staging/generation reservations, publication/accounting conversion, build waiters, provisional pins, attachment prepare/ready-ack and private-reservation transfer, references, mounts, private usage, expiry, tombstones/purge, unmount, deletion, quarantine, and GC. Startup reconciles before readiness; no deadline extends, no debit duplicates/leaks, no second epoch appears before prior eviction completes, and no session silently rematerializes. | +| Retention and disposal | Fake-clock evidence proves one session CAS rejects busy/expired continuation admission, provisionally saves/clears a valid deadline, and either commits active after profile admission or restores the exact future deadline/tombstones an elapsed one after pre-allocation profile failure. Terminal acknowledgement alone sets the next `expiresAt`; polls/replays do not renew. Invalid failed responses stay accounted through purge; retained expiry returns HTTP 410; purge returns stock unknown; persistence authorizes before source access; operator deletion is idempotent. Null `lastUsedAt` epochs evict first by `publishedAt`; used epochs order by `lastUsedAt`, then `publishedAt`, generation key, and epoch. | +| Session continuity | Both modes preserve conversation and fixed generation key/epoch/access/retention/cwd/harness/auth binding while persistent or unexpired; editable preserves private files; read-only remains immutable. Corrupt known evidence returns HTTP 409 non-resumable with no source access or later-epoch substitution. | +| Git acquisition | Closed transport/config, constrained revisions, exact commits, exact object closure/index semantics, catalog validation, generation reuse, and partial cleanup pass against local HTTPS remotes. | +| OCI acquisition | Digest/media/path/link/type/limit, `.git` rejection, generation reuse, and attachment matrix pass against a local registry. | +| Credential boundary | Preflight sees no secret values and returns bounded configured reference identities; validate selects a bounded subset; the runner verifies handles and injects only that selected set into source-access children. Source secrets and caller keys are absent from staging, generations, private trees, base environments, checkpoints, backups, logs, and output. The selected OAuth profile is visible only through its active-turn projection, which is absent before acknowledgement and after restart reconciliation. | +| Provider boundary | Codex/Pi native OAuth, refresh repair, projection teardown, idempotency/session/profile admission, different-profile concurrency, same-profile fail-fast exclusion, and explicit proxy scope all pass without implicit switching. | +| Packaging | The public GHCR digest and provenance/SBOM attestations verify exact inputs; deployment uses that digest and finite lifecycle configuration. | +| Consumer | Promptfoo concurrent/one-shot/two-turn/lifecycle success and every cataloged or UHP terminal failure map exactly. Active streams expose null expiry; terminal/GET/replay expose one stable expiry. Failures before attachment ready omit workspace metadata; later terminal failures include the complete public object. None becomes empty success or automatic retry. | +| Review | Final review findings in both repositories are resolved before final built-image E2E. | ## Definition of Done -- ADR 0002, this plan, implementation, deployment topology, and request examples - agree on UHP, the fork, the behind-router materializer, harness-native OAuth, - explicit proxy fallback, and GHCR digest-pinned distribution. -- The recorded U0 gate evidence predates U1-U6 implementation and shows both - required native targets passed on the recorded input set. Every later change - to a covered input has replacement passing evidence before dependent work - resumes. A failed or narrowed target has a superseding explicit ADR rather than - an implicit proxy or credential workaround. -- No second execution protocol, parallel task/session control plane, separate - AllAgents gateway, direct provider adapter, custom OAuth broker, automatic - auth-mode fallback, or client-side workspace expansion remains in - implementation scope. -- R1-R15 and AE1-AE13 are implemented and verified against the exact released - image. -- Stock UHP requests and upstream conformance remain green. Every external - create/read/control/file/artifact path requires the HarnessRouter caller key; - runner operations remain private and mutually authenticated. -- Git and OCI materialization publish and checkpoint before provider dispatch and - happen exactly once per extension-bearing session. -- The generated normative workspace-manifest schema, frozen canonical fixture, - private result-root transport, Git generation, OCI blob, RFC 8785 digest, and - independent runner reconstruction agree byte-for-byte before publication and - enforce the same 128-repository/500,000-entry limits. -- Every materializer outcome proves the delegated cgroup reaches `populated 0` - before any terminal result, result read, publication, secret release, or - cleanup. A completed-parent/live-descendant violation returns the cataloged - containment failure after successful kill. An unquiescent boundary stays - internal/non-terminal through runner exit/restart; deployment - restart-on-failure destroys the old container, and readiness plus terminal - GET/stream visibility remain withheld until startup proves the old boundary - empty. -- Published operator limits state one active turn and zero cross-session profile - waiters per native auth profile, while UHP idempotency duplicates share one - atomic claim/result, plus the 128-repository, 500,000-entry, 32-GiB, and - request-deadline acquisition caps. -- Continuation preserves native conversation/workspace state and the persisted - auth-binding identity/digest, omits the extension, and follows HarnessRouter's - predecessor semantics. A changed or unavailable binding fails before runner - work. Simultaneous duplicate idempotency keys share one atomic claim, - admission, and result; new same-session overlap returns `session_busy`; and - genuinely new cross-session profile overlap fails with cataloged - `harness_unavailable` before response, runner, or materializer allocation. - Runner-owned durable fencing survives gateway failure, and reconciliation - precedes lock reacquisition after runner failure. Local refresh writes are - validated; a stale credential after remote rotation becomes `repair-required` - rather than changing profile or auth mode. -- Repository-mode roots preserve usable Git state. Every source mode preserves - truthful root/nested produced files across checkpoint/hydrate. -- Source credential values are read from an owner-only runner secret source and - injected only into the selected materializer child; they never enter the - gateway/runner base environment or harness. Provider OAuth is available inside - the explicitly accepted owner-trust boundary; the gateway/runner never - automatically persists the auth file in session state or public metadata. - Long-lived proxy client keys remain brokered, and turn tokens enforce audience, - target, model, turn, expiry, and revocation while permitting bounded in-turn - provider calls. -- The public `linux/amd64` GHCR manifest, verified build-provenance and SBOM - attestations, pinned base/runtime/OS/Git/OCI/harness inputs, patch series, - materializer contract, operational docs, and rollback procedure are - reproducible from pinned inputs. -- The generic HarnessRouter hook patch is ready to propose upstream, but the - shipped system remains operable from the maintained fork if it is not accepted. +- ADR 0002, this plan, implementation, topology, and request examples agree on + the UHP JSON descriptor, project `workspace.yaml` source catalog, immutable + generations, read-only and editable attachments, bounded retention, native + OAuth, explicit proxy mode, and GHCR digest-pinned distribution. +- The U0 evidence predates U1-U6 and both native targets pass on the recorded + inputs; changed inputs have replacement evidence before dependent work resumes. +- No second execution protocol/control plane, separate AllAgents gateway, direct + provider adapter, custom OAuth broker, automatic fallback, or client-side + source expansion remains. +- R1-R16 and AE1-AE14 pass against the exact released image; stock UHP requests + and conformance remain green. +- One canonical resolved source plan produces at most one live verified immutable + publication per generation epoch and one result per concurrent claim. A later + epoch begins only after prior logical and physical eviction completes. Build + waiters retain independent deadlines and cancellation. Access, retention, cwd, + harness/profile, and session identity do not fragment the key. +- Concurrent read-only sessions with different harness/profile bindings share + generation bytes but no mutable runtime, auth, conversation, output, or + lifecycle state. Filesystem probes prove no writable path or copy-up. +- Each fitting editable trial has a private writable tree with no mutable inode + shared with the generation or another session. Its reserved hard byte/inode + allowance covers every turn, overlay, checkpoint, and produced-file record. + Non-fitting waiters fail independently. Continuation preserves only its own + mutations and produced-file history. +- Generation publication and every validate/resolve/materialize result remain + behind cgroup quiescence, exact commit-tree/source-visible manifest and semantic + Git validation, full physical accounting, atomic reservation conversion, + provisional pins, and attachment prepare/ready-ack evidence before provider + dispatch. Successful publication releases staging reservation before ready. +- Finite TTL and quotas cover builds/staging, generations, per-session hard + private bytes/inodes, total private reservations, sessions, persistence, and + tombstones. Generic session/tombstone admission precedes response visibility + and covers invalid failed responses through purge. One stable editable + reservation transfers to ready state without a second debit. One session CAS + rejects busy/expired continuation admission and provisionally saves/clears its + valid deadline; profile success commits active, while pre-allocation failure + restores the exact future deadline or tombstones an elapsed one. Terminal + acknowledgement sets the next expiry; polls/replays do not. Persistent + retention requires authorization and reservation before source access. +- Expiry and deletion tombstone first, fence work, quiesce projections/mounts, + delete private state, and release reservations/references exactly once. + Tombstones compact only after response/idempotency retention. GC evicts only + ready zero-reference/zero-pin epochs: null `lastUsedAt` first by `publishedAt`, + then non-null `lastUsedAt`, `publishedAt`, generation key, and epoch ID. + Failures stay quarantined/accounted, block same-key republication, and + protected-capacity exhaustion rejects admission. +- Restart reconciles generic admission, build waiters, staging/prospective- + generation reservations, publication/accounting, provisional pins, + prepare/ready-ack and private-reservation transfer, references, mounts, private + usage/copies, `containment_pending`, credential projections, tombstones/purge, + deletion, and profile fences before readiness or GC. Existing sessions never + silently reacquire source or change binding. +- Continuation omits the extension and preserves exact generation key/epoch, + access, retention, cwd, harness, auth, and conversation while persistent or + unexpired. Read-only remains immutable; editable retains private files; + retained expiry/deletion returns HTTP 410, corrupt known evidence returns HTTP + 409 non-resumable, and purged identity returns stock unknown, all without source + access, rematerialization, or later-epoch substitution. +- Native auth retains atomic idempotency and same-session precedence, one active + refresh-capable turn and zero waiters per profile, different-profile + concurrency on one generation, fail-closed refresh repair, active-turn-only + credential projection with verified teardown before terminal acknowledgement, + and no implicit profile or proxy switching. +- Preflight receives no secret values; validate returns a bounded selected + credential-reference set, and the runner injects only that exact set into + source-access hook children. Source credentials never enter staging, + generations, private session state, or harnesses. OAuth is visible only within + the accepted active-turn selected-profile owner-trust boundary and is absent + from retained homes, checkpoints, backups, logs, and mounts after teardown; + proxy credentials remain scoped and brokered. +- The public `linux/amd64` image, attestations, pinned inputs, lifecycle + configuration, patch series, materializer contract, operational deletion/GC + docs, and rollback procedure reproduce from pinned inputs. +- Generic generation/attachment/lifecycle and auth-state patches are ready for + upstream proposal; the maintained fork remains operable if not accepted. From 9b4e94a0e5ad029ab968c23168f5375352fd3588 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Wed, 23 Sep 2026 15:23:07 +1000 Subject: [PATCH 21/44] docs(architecture): accept caller repository URLs --- .../0002-adopt-uhp-through-harnessrouter.md | 220 +++++--- ...0837-feat-coding-execution-gateway-plan.md | 530 ++++++++++-------- 2 files changed, 447 insertions(+), 303 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 145514ff..1c0e2bd6 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -23,9 +23,10 @@ HarnessRouter owns caller authentication, UHP request and response semantics, streaming, cancellation, idempotency, session continuity, workspace-generation publication, session attachments, retention, quotas, garbage collection, agent execution, usage, and artifacts. AllAgents owns the -`metadata["allagents.workspace"]` JSON descriptor, its resolution against the -project `workspace.yaml`, deterministic Git/OCI generation construction, and -provenance returned through the HarnessRouter response. +`metadata["allagents.workspace"]` JSON descriptor, including caller-supplied +HTTPS Git repository URLs, deterministic Git/OCI generation construction, and +provenance returned through the HarnessRouter response. Project +`workspace.yaml` is not the authority for caller-requested Git origins. The initial deployment will use a narrow AllAgents-maintained HarnessRouter fork. Its generic pre-turn workspace hook resolves a verified immutable source @@ -114,9 +115,14 @@ AllAgents adds one namespaced JSON request extension: "retention": "session", "source": { "kind": "repositories", - "revisions": { - "api": "main" - } + "repositories": [ + { + "name": "api", + "url": "https://github.com/acme/api.git", + "revision": "main", + "destination": "api" + } + ] }, "workingDirectory": { "kind": "repository", @@ -129,11 +135,12 @@ AllAgents adds one namespaced JSON request extension: ``` `metadata["allagents.workspace"]` is the first-response session descriptor, not -the project configuration file. It selects sources by logical name from the -server-owned project `workspace.yaml` and requests an attachment policy. It -cannot add or override repository or registry origins, destinations, -credentials, host paths, commands, environment variables, materializer -executables, or Docker options. +the project configuration file. Promptfoo supplies each repository's logical +name, HTTPS Git URL, optional revision, and destination. The pre-turn +materializer resolves and loads those repositories before the harness starts, so +the harness receives the populated workspace; this is not a model-directed +in-agent `git clone`. The descriptor cannot supply credentials, host paths, +commands, environment variables, materializer executables, or Docker options. `access` is required and is exactly `readOnly` or `editable`. `retention` is optional and defaults to `session`; `persistent` is accepted only when enabled @@ -141,15 +148,21 @@ by deployment policy and within persistent-workspace quotas. Access and retention are independent: either access mode may use either retention class. Namespaced response metadata contains the effective descriptor digest, resolved -source and generation identity, logical working directory, workspace-manifest -digest, access mode, retention class, and effective expiry. Active turns and -`persistent` sessions report `expiresAt: null`; a `session` terminal -acknowledgement sets the timestamp returned by terminal, retrieval, and replay -paths. Failures before attachment `ready` omit workspace metadata entirely; -terminal failures after `ready` include the same complete public object. -Metadata never exposes the raw request digest, configured origins, physical -paths, credentials, internal generation-epoch/lease/attachment identifiers, or -other sessions' quota state. +source provenance, public `generationId`, logical working directory, workspace- +manifest digest, access mode, retention class, and effective expiry. +`generationId` is the SHA-256 digest of versioned RFC 8785 bytes containing only +the returned normalized source provenance, normalized destinations, and +workspace-manifest digest; it is never an internal cache, authorization, +attachment, or lookup key. Active turns and `persistent` sessions report +`expiresAt: null`; a `session` terminal acknowledgement sets the timestamp +returned by terminal, retrieval, and replay paths. Failures before attachment +`ready` omit workspace metadata entirely; terminal failures after `ready` include +the same complete public object. Metadata never exposes the private generation +key, raw request digest, URL credentials (which requests cannot contain), +credential-scope mappings or selected references/values, redirect-chain URLs, +resolved network addresses, physical paths, internal generation-epoch/lease/ +attachment identifiers, or other sessions' quota state. Normalized caller- +supplied repository URLs and resolved commits are returned as source provenance. Ordinary UHP input files remain supported for `editable` sessions and are applied to the private workspace after generation attachment. A `readOnly` @@ -421,49 +434,75 @@ the implementation must not introduce an implicit fallback or credential shim. ## Source authority and credentials -The project `workspace.yaml` remains the source of truth for logical repository -names, origins, non-root destinations, default revisions, snapshot catalog, and -environment-variable credential references. Repository execution names are -explicit `name` values or the portable basename of `path`; duplicate names, -duplicate or ancestor/descendant destinations, root destinations, and -local/originless entries make execution preflight fail. Secret values remain -deployment-only. - -The UHP `metadata["allagents.workspace"]` JSON object only selects from that -catalog and requests session access and retention. It is not parsed as, merged -with, or persisted as a replacement for `workspace.yaml`. HarnessRouter -deployment configuration owns harness/model/provider targets, persistence -authorization, idle TTLs, quotas, and garbage-collection policy. No -`gateway.yaml` or caller-controlled source registry is introduced. - -The materializer `preflight` receives no secret values. It validates hook/catalog -versions, reference syntax, and required source tools and returns a bounded set of -configured credential-reference names or opaque IDs. The runner, not the hook, -checks those store handles and all actual staging/result/generation filesystem -relationships. Source-free `validate` returns the bounded selected subset; the -runner injects values for only that set into `resolve` or `materialize`. - -Repository mode acquires the complete declared repository set, with optional -revision overrides by logical name. It accepts only a bounded ref-name grammar, -rejects option-like or refspec-shaped values, resolves advertised refs to full -commits before agent execution, fetches by verified object ID, and records those -commits in provenance. It preserves `.git` for coding tools but hermetically -normalizes the allowed detached-HEAD configuration/ref set and removes reflogs, -`FETCH_HEAD`, locks, hooks, worktree links, alternates, shallow/replace/graft -state, extra refs, extra objects, and credential-bearing configuration. The -runner independently verifies HEAD, an index exactly matching the resolved commit -tree, its canonical object-set digest, and exactly the transitive required object -closure with no extras. It then proves the source-visible manifest equals exactly -the union of each resolved commit tree prefixed by its pairwise non-overlapping -destination plus only necessary destination ancestor directories. Undeclared -paths outside that union fail integrity validation. +Project `workspace.yaml` remains the source of truth for the ordinary local +AllAgents workspace and the optional operator-owned OCI snapshot catalog. It is +not a Git origin allowlist and is not consulted to translate repository names in +a UHP request. HarnessRouter deployment configuration owns harness/model/provider +targets, persistence authorization, idle TTLs, quotas, garbage-collection policy, +outbound network policy, and optional source-credential scope mappings. + +For repository mode, the UHP JSON descriptor supplies one through 128 repository +objects containing required `name`, `url`, and `destination` fields plus an +optional `revision`. `workingDirectory.repository` references `name`; +`destination` is a non-empty, non-root relative path. Names and destinations are +unique, and destinations are pairwise non-overlapping. The descriptor is session +input, not parsed as, merged with, or persisted as a replacement for +`workspace.yaml`. + +An authenticated caller may request any repository reachable through the +deployment's HTTPS egress boundary. Before parsing, validation rejects ASCII +controls, whitespace, and backslashes. It parses once with the WHATWG URL +Standard and requires the input bytes to equal the serialized URL exactly. That +serialization must use `https`, an ASCII lowercase IDNA A-label DNS hostname +without a trailing dot, no userinfo/query/fragment or IP literal, no explicit +default port, a non-empty repository path, and no percent-encoded control, slash, +backslash, or dot segment. The same serialization and structured `(scheme, host, +effectivePort)` origin drive policy, credentials, redirects, DNS, provenance, +generation identity, and the exact Git/libcurl request. Local paths and `file`, +`ssh`, `git`, and extension transports are rejected. + +The acquisition child cannot bypass the deployment connector through direct +network access or inherited proxy configuration. Resolution and each connection +or redirect reject the entire DNS answer set if any address is loopback, link- +local, private, reserved, metadata, or otherwise non-public; the connector pins +one approved address for each connection. At most five redirects are accepted. +Each is parsed and serialized by the same rules, re-resolved, and rechecked. +Deployment policy may further restrict egress but does not require every +repository to be predeclared. + +The materializer `preflight` receives no repository URL or secret value. It +validates hook/policy versions, credential-scope mapping syntax, and source tools, +then returns bounded configured credential-reference names or opaque IDs. A +credential scope is either an exact structured origin or that origin plus a +canonical repository-path segment prefix; a prefix matches complete segments, +never raw strings. Source-free `validate` selects the matching rule with the most +path segments for each normalized URL. The runner, not the hook, verifies those +store handles and actual filesystem relationships and injects values only into +`resolve` or `materialize`. + +Repository mode acquires exactly the caller-declared repository set. It accepts +only a bounded ref-name grammar, rejects option-like or refspec-shaped values, +resolves the requested revision—or the remote symbolic HEAD when omitted—to a +full commit before agent execution, fetches by verified object ID, and records +the normalized URL, requested revision, and commit in provenance. It preserves +`.git` for coding tools but hermetically normalizes the allowed detached-HEAD +configuration/ref set and removes reflogs, `FETCH_HEAD`, locks, hooks, worktree +links, alternates, shallow/replace/graft state, extra refs, extra objects, and +credential-bearing configuration. The runner independently verifies HEAD, an +index exactly matching the resolved commit tree, its canonical object-set digest, +and exactly the transitive required object closure with no extras. It then proves +the source-visible manifest equals exactly the union of each resolved commit tree +prefixed by its pairwise non-overlapping destination plus only necessary +destination ancestor directories. Undeclared paths outside that union fail +integrity validation. Snapshot mode accepts only a configured OCI repository plus immutable image- manifest and workspace-manifest digests. It verifies the image manifest, canonical workspace-manifest bytes, layer sizes and digests, applies OCI whiteouts, validates the resulting declared workspace layout against the manifest, rejects `.git` administrative subtrees, and records the ordered layer -digests. Snapshots requiring Git history use repository mode. +digests. Version-one snapshot requests use `workspaceRoot`; snapshots requiring +Git history or repository-relative working directories use repository mode. Both source modes produce the same versioned canonical workspace manifest. Its RFC 8785 bytes enumerate every source-visible directory, regular file, and @@ -481,15 +520,17 @@ places the manifest in source content. Before materialization, resolution computes a canonical generation key from every input that can affect source-visible bytes, declared agent-visible filesystem semantics, or sharing authorization: descriptor and hook contract versions, -deployment authorization scope, bounded selected credential-reference identities, -resolved commits or immutable OCI digests, normalized destinations, applicable -catalog identity, and acquisition policy. Physical paths, access, retention, -working directory, harness/profile identity, credential values, and volatile Git -administrative representation are excluded. Publication binds that key and one +deployment authorization scope, normalized caller Git URLs, bounded selected +credential-reference identities, resolved commits or immutable OCI digests, +normalized destinations, snapshot identity when applicable, and acquisition/ +egress policy version. Logical repository names, physical paths, access, +retention, working directory, harness/profile identity, credential values, and +volatile Git administrative representation are excluded. Publication binds that +key and one internal epoch to one verified workspace-manifest digest and, in repository mode, one semantic Git-state record. Materialize receives the exact private source-only -resolved-plan bytes/digest and selected reference set returned by resolve; it -never re-resolves source. +plan bytes/digest and selected reference set returned by resolve; it never +re-resolves source. The generation backing store is owner-writable and never exposed writable to a session. Publication is a recoverable same-filesystem atomic transition. @@ -498,15 +539,21 @@ mutable inodes with the generation. Corrupt, partial, quarantined, or deleting generations are not attachable. Source credentials are selected server-side from an owner-only secret mount or -credential-store handle available to the runner, not from the long-lived service -environment. After preflight declaration and validate selection, the runner -resolves only the exact selected values when it constructs a source-access hook -child environment; the gateway/runner base environment and every agent child -remain credential-free. The materializer must use hermetic Git/registry -configuration, prevent credentials from being persisted in Git configuration or -remote URLs, remove temporary credential state before returning, and emit no -secret value. If a configured source secret appears in the service or agent -environment, the runner refuses to launch the agent. +credential-store handle available to the runner, not from request JSON or the +long-lived service environment. Anonymous access is used when no configured +credential scope matches. Otherwise the runner resolves only the selected value +when constructing a source-access child environment. That child has an isolated +HOME, no inherited proxy variables, no direct network path, hermetic Git/ +registry configuration, `credential.useHttpPath=true`, and an ephemeral helper +that independently rejects any protocol, host, effective port, or canonical +repository path outside the selected structured scope. Each redirect is checked +against the originally selected scope; credentials are stripped whenever it +leaves that scope, including a same-origin path-prefix escape, and a redirect +never selects a new credential. Credentials are never encoded in the URL, +persisted in Git configuration or remote URLs, or emitted. Temporary credential +state is removed before return. The gateway/runner base environment and every +agent child remain credential-free; if a configured source secret appears there, +the runner refuses to launch the agent. ## Trust and deployment @@ -601,8 +648,8 @@ automatically. ## Failure behavior - **Invalid extension:** the source-free hook validation rejects unknown or - malformed source, access, retention, or logical-working-directory fields - before source resolution. + malformed repository names, URLs, revisions, destinations, source, access, + retention, or logical-working-directory fields before source resolution. - **Unauthorized persistence:** after response allocation but before source resolution or byte acquisition, the runner rejects `retention: "persistent"` when deployment policy does not authorize it; the @@ -647,11 +694,15 @@ automatically. reserved byte or inode allowance; the runner returns `allagents_workspace_private_quota_exceeded`. Persistent sessions cannot exceed the same fixed envelope; retries never change mode or retention. -- **Source authentication or acquisition failure:** remove only unpublished - staging, publish no generation, and start no agent or provider fallback. An - existing verified generation is not poisoned by a failed competing build. - Cancelling one build waiter detaches only it; other live waiters keep the - runner-owned build alive. +- **Source policy, authentication, or acquisition failure:** reject any + destination that resolves or redirects outside the permitted public HTTPS + egress boundary before source bytes reach staging. Strip credentials whenever + a redirect leaves the originally selected structured scope, including a same- + origin path-prefix escape. For authentication or transport failure, remove only + unpublished staging, publish no generation, and start no agent or provider + fallback. An existing verified generation is not poisoned by a failed + competing build. Cancelling one build waiter detaches only it; other live + waiters keep the runner-owned build alive. - **Generation, attachment, or private-copy failure:** quarantine corrupt or incomplete state, release reservations and provisional pins exactly once, acquire no live attachment, and fail closed. A session never substitutes @@ -747,9 +798,10 @@ accepted or equivalent supported extensions exist. Version one does not add evaluation datasets, scoring, assertions, automatic retries, session branching, concurrent turns within one session, simultaneous -refresh-capable turns sharing one auth profile, caller-supplied origins, public -multi-tenancy, arbitrary materializer commands, mutable OCI tags, transparent -source-mode fallback, or guaranteed provider prompt-cache hits. +refresh-capable turns sharing one auth profile, caller-supplied credentials, +non-HTTPS or private-network Git origins, public multi-tenancy, arbitrary +materializer commands, mutable OCI tags, transparent source-mode fallback, or +guaranteed provider prompt-cache hits. Read-only attachments never copy up or become editable. Editable workspaces never share mutations across sessions. Callers cannot choose arbitrary TTLs, bypass @@ -776,6 +828,8 @@ Revisit this decision when: - the maintained patch grows beyond the narrow integration boundary; - HarnessRouter changes or removes required UHP/session/provider behavior; - exact per-turn workspace rollback becomes a product requirement; +- the host cannot enforce public-address egress validation, connection address + pinning, bounded redirect revalidation, and out-of-scope credential stripping; - source acquisition must run in a stronger isolation boundary; - callers require a public multi-tenant authorization model; or - a second independent UHP implementation offers a materially smaller and more diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 4fe3e8ba..fcb68190 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -36,14 +36,15 @@ execution: code - **Authority:** [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) owns the protocol, fork, trust, generation, attachment, retention, harness-authentication, and provider-routing decisions. UHP `2026-09-12` and - HarnessRouter's conformance suite own execution-wire behavior. The project - `workspace.yaml` owns only the logical Git/OCI source catalog and - environment-variable credential references; deployment secrets provide the - values. HarnessRouter configuration owns harness IDs, model allowlists, - authentication bindings, persistence authorization, finite TTLs, quotas, and - garbage-collection policy. The namespaced UHP JSON extension owns per-session - source selection, access, retention request, logical cwd, and provenance - semantics; it is not `workspace.yaml`. + HarnessRouter's conformance suite own execution-wire behavior. The namespaced + UHP JSON extension owns caller-supplied HTTPS Git URLs, revisions, + destinations, per-session source selection, access, retention request, logical + cwd, and provenance semantics. Project `workspace.yaml` remains ordinary local + workspace configuration plus the optional operator-owned OCI snapshot catalog; + it is not a Git origin catalog for UHP. HarnessRouter deployment configuration + owns egress policy, source-credential scope mappings, harness IDs, model + allowlists, authentication bindings, persistence authorization, finite TTLs, + quotas, and garbage-collection policy. - **Execution order:** First build the minimal custom image and pass the blocking native-auth adapter gate for both Codex and Pi without implementing the AllAgents materializer. Then prove the generation claim/publication seam, @@ -89,11 +90,12 @@ idempotency, and returns output, usage, and artifacts. The missing product-specific capability is deterministic source-generation resolution before the first agent turn. A focused HarnessRouter fork calls a generic hook after allocating the UHP session but before provider selection. The -AllAgents executable validates the JSON descriptor against the project -`workspace.yaml`, resolves an immutable source plan, and builds verified staging -only on a generation cache miss. The runner atomically publishes or reuses the -generation, records the session attachment and retention state, then mounts it -read-only or creates a private editable copy before provider dispatch. +Promptfoo request names the HTTPS Git repositories to load; the AllAgents +executable validates those URLs against deployment egress policy, resolves an +immutable source plan, and builds verified staging only on a generation cache +miss. The runner atomically publishes or reuses the generation, records the +session attachment and retention state, then mounts it read-only or creates a +private editable copy before provider dispatch. A continuation supplies `previous_response_id`, omits the workspace extension, and uses HarnessRouter's current native conversation plus the bound attachment. @@ -120,17 +122,19 @@ caller responsible for acquisition. The temporary fork closes those seams. - **A1. UHP caller:** Promptfoo or another application holding a HarnessRouter API key. It chooses a configured HarnessRouter harness ID and model, prompt, - initial workspace descriptor, and optional continuation predecessor. + caller-supplied HTTPS Git repositories or configured OCI snapshot, initial + workspace policy, and optional continuation predecessor. - **A2. HarnessRouter gateway:** Authenticates and validates UHP; owns response and session identity; is the sole writer of session attachment, expiry, and tombstone state; treats the configured workspace metadata value as bounded opaque JSON; drives prepare/ack before provider fallback; and returns hook metadata on every response path. - **A3. AllAgents materializer:** A subprocess executable that exposes no - listening service. It validates the AllAgents descriptor, reads the project - `workspace.yaml`, resolves immutable Git/OCI source plans, builds private - staging on cache misses, validates the tree, and returns generation identity - and provenance. It never authorizes persistence or publishes live state. + listening service. It validates the AllAgents descriptor and caller Git URLs, + reads deployment acquisition policy and the optional project OCI snapshot + catalog, resolves immutable Git/OCI source plans, builds private staging on + cache misses, validates the tree, and returns generation identity and + provenance. It never authorizes persistence or publishes live state. - **A4. HarnessRouter runner:** Owns generation claims/publication and the resource journal: provisional pins, durable references, read-only mounts, private editable copies and quotas, per-session operating-system identity and @@ -149,10 +153,10 @@ caller responsible for acquisition. The temporary fork closes those seams. non-refreshable, scoped turn credential that the HarnessRouter broker validates. - **A7. Operator:** Pins and deploys the custom image, mounts durable generation, session, editable-workspace, and auth storage, completes each native harness - login, supplies deployment-only source credentials, authorizes persistent - sessions, configures finite TTL/byte/inode/count/tombstone quotas, operates - deletion and GC, selects explicit proxy targets, and controls private-network - access. + login, configures HTTPS egress and optional source-credential scopes, + authorizes persistent sessions, configures finite TTL/byte/inode/count/ + tombstone quotas, operates deletion and GC, selects explicit proxy targets, + and controls private-network access. ### Key Decisions @@ -179,9 +183,10 @@ caller responsible for acquisition. The temporary fork closes those seams. and persistent sessions are protected. Expired state and bounded tombstones are purged before deterministic eviction of unreferenced/unpinned generations. Admission fails when protected state consumes finite quota. -- **Keep source authority server-side.** Callers select logical source names and - revisions but cannot send origins, credentials, host paths, commands, or - Docker options. +- **Let authenticated callers select Git origins.** Promptfoo supplies canonical + HTTPS Git URLs, revisions, logical names, and destinations. The service accepts + any repository reachable through safe public egress; callers cannot supply + credentials, non-HTTPS transports, host paths, commands, or Docker options. - **Prefer harness-native OAuth.** Promptfoo's HarnessRouter API key authenticates the UHP caller only. Codex and Pi use their own login, token storage, refresh, and provider request path; native mode has no provider-route API key. @@ -315,29 +320,34 @@ caller responsible for acquisition. The temporary fork closes those seams. `{ version: "1", access, retention?, source, workingDirectory? }`. `access` is exactly `readOnly | editable`; omitted `retention` means `session`, otherwise it is exactly `session | persistent`. `source` is exactly - `{ kind: "repositories", revisions?: Record }` or - `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, - workspaceManifestDigest: Digest }`; `workingDirectory` is exactly + `{ kind: "repositories", repositories: NonEmptyArray<{ name: ConfigName, + url: HttpsGitUrl, revision?: RevisionText, destination: RelativeDirectory }> }` + or `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, + workspaceManifestDigest: Digest }`. `workingDirectory` is exactly `{ kind: "workspaceRoot" }` or `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }`. + The repository form is valid only when `source.kind` is `repositories` and its + `repository` names one request entry; v1 snapshot requests use `workspaceRoot`. The hook validates and reports the requested retention but never authorizes it. The runner is the sole persistence authority: before source resolution or byte acquisition it authorizes `persistent`, reserves the session and persistence slots, or fails `allagents_workspace_persistence_forbidden`. -- **R6.** The AllAgents hook expands omitted `retention` to `session`, omitted - `revisions` to `{}`, and omitted `workingDirectory` to - `{ kind: "workspaceRoot" }`; an omitted repository `path` remains absent and an - empty path is invalid. It NFC-normalizes strings, sorts maps, rejects unknown - fields, and hashes RFC 8785 bytes as the effective descriptor digest. A - separate canonical generation key covers only inputs that can affect - source-visible bytes, declared agent-visible filesystem semantics, or sharing - authorization: hook/schema versions, deployment authorization scope, bounded - selected credential-reference identities, resolved commits or OCI digests, - normalized destinations, catalog identity, and acquisition policy. Access, +- **R6.** The AllAgents hook expands omitted `retention` to `session` and omitted + `workingDirectory` to `{ kind: "workspaceRoot" }`; an omitted repository + `revision` remains absent and means remote symbolic HEAD. It canonicalizes each + HTTPS URL, NFC-normalizes strings, sorts repository entries by name, rejects + unknown fields, and hashes RFC 8785 bytes as the effective descriptor digest. + A separate canonical generation key covers only inputs that can affect source- + visible bytes, declared agent-visible filesystem semantics, or sharing + authorization: hook/schema versions, deployment authorization scope, normalized + caller Git URLs, bounded selected credential-reference identities, resolved + commits or OCI digests, normalized destinations, snapshot identity when + applicable, and acquisition/egress policy version. Repository names, access, retention, logical cwd, harness/profile, session identity, physical paths, credential values, and volatile Git administrative representation do not - fragment that key. Publication binds it to the independently verified - workspace-manifest digest and semantic Git record when applicable. + fragment that key. + Publication binds it to the independently verified workspace-manifest digest + and semantic Git record when applicable. Omitted and explicit default values have the same effective descriptor digest. The raw request descriptor digest records the exact initial JSON only in private session state; public response metadata names and returns only @@ -346,18 +356,18 @@ caller responsible for acquisition. The temporary fork closes those seams. runner `/workspace/prepare` operation outside the provider candidate loop. It invokes a configured executable directly without a shell using typed `validate`, `resolve`, and `materialize` commands. `validate` performs only - source-free schema/default/catalog checks and returns a private normalized - descriptor reference/digest plus effective access, requested retention, - logical cwd, effective descriptor digest, and a bounded sorted list of selected - credential-reference names/opaque IDs—not values. The runner verifies those - references were declared by preflight and that their handles exist, then maps - only that selected set into source-access child environments. After runner - authorization and admission, `resolve` consumes that exact validated - descriptor and selected credential-reference set, resolves exact Git commits - or OCI identity, and returns a private canonical resolved-plan path/digest, - generation key, effective cwd, and bounded public provenance. `materialize` - receives that exact resolved-plan path/digest and selected set and never - re-resolves source. + source-free schema/default/URL/destination/policy checks and returns a private + normalized descriptor reference/digest plus effective access, requested + retention, logical cwd, effective descriptor digest, and the bounded sorted + credential-reference names/opaque IDs selected by deployment policy for the + requested origins—not values. The runner verifies those references were + declared by preflight and that their handles exist, then maps only that selected + set into source-access child environments. After runner authorization and + admission, `resolve` consumes that exact validated descriptor and selected set, + resolves exact Git commits or OCI identity, and returns a private canonical + resolved-plan path/digest, generation key, effective cwd, and bounded public + provenance. `materialize` receives that exact resolved-plan path/digest and + selected set and never re-resolves source. For each ready lookup or completed build, the runner validates ready evidence and acquires a durable provisional attachment pin under the same generation @@ -508,44 +518,70 @@ caller responsible for acquisition. The temporary fork closes those seams. #### Source acquisition and provenance -- **R9.** Parse the project `workspace.yaml` through its authoritative schema. - Add strict project-only `workspaceSnapshots` entries: +- **R9.** Parse the project `workspace.yaml` through its authoritative schema + only for the optional strict project-owned `workspaceSnapshots` entries: `{ name: ConfigName, repository: OciRepository, workspaceManifestMediaType: MediaType, executionCredential?: "${ENV_VAR}" }`. - `OciRepository` is a normalized `registry-host/repository-path` with no scheme, - tag, digest, userinfo, query, or fragment. Reject unknown fields, literal - secrets, and duplicate snapshot names; snapshot entries do not merge with user - configuration. Add the same optional environment-reference field to - execution-eligible repositories. Secret values remain deployment-only. A - repository's logical name is explicit `name` or the portable basename of - normalized `path`. Execution-eligible Git destinations must be unique, - pairwise non-overlapping, non-empty, non-root relative child paths so their - source trees and `.git` directories cannot collide with one another or with - HarnessRouter's root checkpoint repository. Reuse one shared source resolver: - `source` as a supported HTTPS URL is complete when `repo` is absent; otherwise - `source` names the supported host/provider and `repo` names its repository. - Conflicting forms, local/originless entries, duplicate repository names, - duplicate or ancestor/descendant destinations, escaping destinations, and - unsupported schemes make materializer preflight fail. - HarnessRouter owns harness/model/provider targets, persistence authorization, - TTLs, quotas, and GC. The project `workspace.yaml` - remains only the materialization catalog; it never contains session access, - retention, lease, or eviction state. -- **R10.** Repository mode materializes every execution-eligible declared - repository. Optional revisions override only matching logical names; otherwise - use configured `branch`, then the remote symbolic HEAD. `RevisionText` is at - most 255 ASCII bytes and is either a full 40-hex object ID or a - `git-check-ref-format`-equivalent ref name. Reject leading dashes, whitespace - and controls, refspec colons, glob metacharacters, traversal-like components, - `@{`, and `.lock` components. Resolve a validated full ref, or an unambiguous - shorthand under `refs/heads/` or `refs/tags/`, with `ls-remote`; accept object - IDs only when advertised. Subsequent fetch/checkout commands receive only the - verified object ID with explicit end-of-options handling, never caller text. - Allow only argument-vector HTTPS Git operations to exact configured hosts, with - no URL credentials, query, fragment, or redirects. Use an isolated HOME plus - `GIT_CONFIG_NOSYSTEM=1`, no global config, empty credential helper, disabled - hooks, `protocol.file.allow=never`, `protocol.ext.allow=never`, and no - submodule recursion, Git LFS hydration, or configured clean/smudge filters. + Reject unknown fields, literal secrets, and duplicate snapshot names; snapshot + entries do not merge with user configuration. Git repository URLs do not come + from `workspace.yaml`. + + Repository mode takes one through 128 request entries. Each has a unique + `name`, canonical absolute `https` `url`, optional `revision`, and unique, + pairwise non-overlapping `destination`. Before parsing, reject ASCII controls, + whitespace, and backslashes. Parse once with the WHATWG URL Standard and + require the input bytes to equal its serialized URL exactly. The serialization + must have an ASCII lowercase IDNA A-label DNS hostname without a trailing dot, + no userinfo/query/fragment or IP literal, no explicit default port, a non-empty + repository path, and no percent-encoded control, slash, backslash, or dot + segment. The same serialization and structured `(scheme, host, effectivePort)` + origin are used for policy, credentials, redirects, DNS, provenance, + generation identity, and the exact Git/libcurl request. Local paths and non- + HTTPS schemes fail source-free validation. Destinations are non-empty, + non-root relative child paths and cannot collide with HarnessRouter's root + checkpoint repository. Duplicate names, duplicate or ancestor/descendant + destinations, escaping destinations, and unsupported URL forms also fail. + + HarnessRouter deployment configuration owns harness/model/provider targets, + persistence authorization, TTLs, quotas, GC, outbound egress policy, and + optional source-credential scope mappings. A scope is either an exact + structured origin or an origin plus canonical repository-path segment prefix; + path prefixes match only complete segments, never raw strings. The matching + rule with the most path segments selects one secret reference; callers never + select the reference or supply its value. No match means anonymous acquisition. + Deployment policy may narrow public egress but does not require every + repository URL to be predeclared. Project `workspace.yaml` never contains + session access, retention, lease, or eviction state. +- **R10.** Repository mode materializes exactly the caller-declared repository + set. Use the requested `revision`, or the remote symbolic HEAD when omitted. + `RevisionText` is at most 255 ASCII bytes and is either a full 40-hex object ID + or a `git-check-ref-format`-equivalent ref name. Reject leading dashes, + whitespace and controls, refspec colons, glob metacharacters, traversal-like + components, `@{`, and `.lock` components. Resolve a validated full ref, or an + unambiguous shorthand under `refs/heads/` or `refs/tags/`, with `ls-remote`; + accept object IDs only when advertised. Subsequent fetch/checkout commands + receive only the verified object ID with explicit end-of-options handling, + never caller revision text. + + Allow only argument-vector HTTPS Git operations through the deployment's + acquisition egress connector. The child cannot bypass it: clear every proxy/ + `NO_PROXY` environment variable, disable Git `http.proxy` and remote proxy + configuration, and permit no direct network path. Before every connection and + each of at most five HTTPS redirects, resolve the canonical hostname and reject + the entire answer set if any address is loopback, link-local, private, reserved, + metadata, or otherwise non-public; pin one approved address for that connection + so DNS rebinding cannot escape the check. Parse and serialize every redirect by + the same URL rules and compare structured origins. Re-evaluate the originally + selected credential scope at every hop, strip its credential whenever the + target leaves that scope—including a same-origin path-prefix escape—and never + select a new credential because of a redirect. + + Use an isolated HOME plus `GIT_CONFIG_NOSYSTEM=1`, no global config, + `credential.useHttpPath=true`, and an explicit ephemeral credential helper + bound to the selected structured origin/path scope. The helper independently + rejects any protocol, host, effective port, or canonical repository path + outside that rule. Disable hooks, `protocol.file`, `protocol.ext`, submodule + recursion, Git LFS hydration, and configured clean/smudge filters. Preserve each repository's `.git` directory for the coding agent, but do not treat volatile Git administrative bytes as generation identity. The materializer constructs a hermetic detached-HEAD repository at the resolved @@ -608,16 +644,22 @@ caller responsible for acquisition. The temporary fork closes those seams. stage, and no placeholder or partial/unverified generation identity is emitted. Once attachment commits `ready`, streaming events, provider terminal responses, GET, background completion, and idempotent replay return the same immutable - bounded fields: extension version, `effectiveDescriptorDigest`, generation - key, canonical workspace-manifest digest, logical cwd, access, resolved - retention, source completeness, and resolved Git/OCI provenance. Active - streaming metadata has `expiresAt: null`. For `session` retention, durable - terminal acknowledgement atomically sets `expiresAt`; the terminal event, - stored response, GET, background completion, and idempotent replay then return - that same timestamp. `persistent` always returns `expiresAt: null`. Metadata - never contains raw request digest, generation epoch, origins, physical paths, - credentials or references, lease/attachment/reservation tokens, counts, - authorization rules, or other sessions' quota state. + bounded fields: extension version, `effectiveDescriptorDigest`, public + `generationId`, canonical workspace-manifest digest, logical cwd, access, + resolved retention, source completeness, and resolved Git/OCI provenance. + `generationId` is the SHA-256 digest of versioned RFC 8785 bytes containing + only the returned normalized source provenance, normalized destinations, and + workspace-manifest digest. It is metadata-only and is never a cache, + authorization, attachment, or lookup key. Active streaming metadata has + `expiresAt: null`. For `session` retention, durable terminal acknowledgement + atomically sets `expiresAt`; the terminal event, stored response, GET, + background completion, and idempotent replay then return that same timestamp. + `persistent` always returns `expiresAt: null`. Metadata may return normalized + caller-supplied repository URLs as provenance but never contains the private + generation key, raw request digest, generation epoch, redirect-chain URLs, + resolved network addresses, deployment credential-scope mappings or selected + references, physical paths, credential values, lease/attachment/reservation + tokens, counts, authorization rules, or other sessions' quota state. - **R13.** The materializer resolves `${ENV_VAR}` references from its allowlisted child environment, uses hermetic Git/registry configuration, removes temporary auth files before returning, and emits no secret. Prove with a deliberately @@ -724,7 +766,8 @@ caller responsible for acquisition. The temporary fork closes those seams. 1. Launch the attestation-verified image in non-serving initialization mode with durable session, generation, editable-workspace, and auth volumes; finite idle TTL and staging/generation/private/session/persistence/tombstone quotas; - caller key; materializer command; project configuration; owner-only + caller key; materializer command; project snapshot configuration; public- + egress enforcement; optional origin-to-secret-reference mappings; owner-only source-secret handle; native or proxy trust mode; delegated cgroup v2 subtree; and `on-failure` restart policy. Verify the image and mounted inputs before running checks that depend on them. @@ -735,11 +778,11 @@ caller responsible for acquisition. The temporary fork closes those seams. workspaces and quota usage, auth projections, tombstones/compaction, and interrupted deletions. Sweep orphaned cgroups and credential projections only after proving each old process boundary empty. Do not start GC or serving. -3. Run the mounted AllAgents hook's bounded `preflight` mode. It validates hook - version, project catalog, snapshot and credential-reference syntax, and - required Git/OCI tools without source network access or secret values. It - returns the bounded configured credential-reference identities; the runner, - not the hook, verifies their credential-store handles are present. +3. Run the mounted AllAgents hook's bounded `preflight` mode. It validates hook, + egress-policy, optional snapshot-catalog, origin-mapping, credential-reference, + and required Git/OCI tool syntax without repository URLs, source network + access, or secret values. It returns the bounded configured credential- + reference identities; the runner verifies their credential-store handles. 4. In a controlled operator context, initialize each dedicated auth profile: run Codex login with that target's `CODEX_HOME`, or run Pi `/login` with that target's isolated Pi home and configured provider. Persist only the selected @@ -763,8 +806,9 @@ caller responsible for acquisition. The temporary fork closes those seams. 1. Promptfoo sends one authenticated UHP request with `model`, stock `metadata.harness_id`, idempotency input, and the - `metadata["allagents.workspace"]` JSON descriptor. The descriptor explicitly - selects `readOnly` or `editable`; omitted retention means `session`. + `metadata["allagents.workspace"]` JSON descriptor containing the HTTPS Git + repositories to load. The descriptor explicitly selects `readOnly` or + `editable`; omitted retention means `session`. 2. HarnessRouter validates UHP and generic metadata bounds and atomically claims the `Idempotency-Key`. The private runner admission transaction resolves the selected target/auth-binding digest, applies existing profile admission, and @@ -781,17 +825,18 @@ caller responsible for acquisition. The temporary fork closes those seams. IDs, and harness/auth binding before making it visible. It then invokes the hook's source-free `validate` operation. The runner consumes typed access, requested retention, effective descriptor digest, logical cwd, normalized - descriptor reference, and bounded selected credential-reference identities. - It proves the selected set is a subset of preflight declarations, verifies - only those handles, rejects read-only input files, rechecks the secret + caller repository entries, descriptor reference, and bounded credential- + reference identities selected by credential-scope policy. It proves the selected + set is a subset of preflight declarations, verifies only those handles, rejects + unsafe URLs/destinations and read-only input files, rechecks the secret boundary, authorizes persistence, and reserves any persistence slot plus one stable full-hard-private-allowance ID for editable access. Failure releases access-specific reservations once, persists the terminal failed response under finite failed-response retention, and retains its generic session/tombstone slots through tombstoning and purge; no source is resolved or acquired. 4. The gateway CASes `validating -> resolving`. `resolve` consumes the exact - validated descriptor and selected credential set, resolves configured source - to immutable identity, and returns the generation key, private source-only + validated caller repositories and selected credential set, safely resolves + them to immutable commits, and returns the generation key, private source-only resolved-plan path/digest, effective cwd, and request provenance. Under the generation lock, a valid ready epoch hit acquires a durable provisional pin and skips acquisition. A miss joins the current build epoch or, only after an @@ -970,24 +1015,27 @@ caller responsible for acquisition. The temporary fork closes those seams. absent from the agent environment and filesystem, no provider-route API key exists in that mode, and the projection is absent from retained homes, checkpoints, backups, and mounts after every terminal or recovered outcome. -- **AE3.** Repository mode resolves configured refs to exact commits and - publishes one verified immutable generation. Two simultaneous `readOnly` - sessions using different harness/profile bindings share one generation build, - see identical bytes and nested Git history, start in their own validated - logical cwd, and cannot write the generation or observe each other's home, - conversation, temporary files, logs, or outputs. +- **AE3.** Repository mode resolves Promptfoo-supplied HTTPS URLs and revisions + to exact commits and publishes one verified immutable generation. Two + simultaneous `readOnly` sessions using different harness/profile bindings and + the same normalized request share one generation build, see identical bytes + and nested Git history, start in their own validated logical cwd, and cannot + write the generation or observe each other's home, conversation, temporary + files, logs, or outputs. - **AE4.** HarnessRouter maps a non-object extension to HTTP 400 `invalid_input`, an oversized extension to HTTP 413 `allagents_workspace_too_large`, an extension on continuation to HTTP 409 `allagents_workspace_immutable`, and a retained expired/tombstoned continuation to HTTP 410 - `allagents_workspace_expired`. The hook validates unknown access/retention, - read-only input conflicts, logical names, caller URLs, paths, commands, - credential-reference selection, and duplicate names/destinations before source - access. Preflight receives no secret values; the runner alone verifies declared - credential handles, storage relationships, persistence authorization, and - session/persistence/build reservations. Exact source byte admission may fail - only after bounded staging reveals size, but before publication, attachment, or - agent launch. + `allagents_workspace_expired`. Source-free validation rejects unknown access/ + retention, malformed names/revisions/destinations, userinfo or secrets in URLs, + non-HTTPS transports, IP literals, and duplicate/overlapping destinations. + Acquisition rejects loopback/link-local/private/reserved/metadata destinations, + DNS rebinding, unsafe redirects, and out-of-scope credential forwarding before + source bytes reach staging. Preflight receives no request URL or secret value; + the runner alone verifies selected credential handles, storage relationships, + persistence authorization, and session/persistence/build reservations. Exact + source byte admission may fail only after bounded staging reveals size, but + before publication, attachment, or agent launch. - **AE5.** Two `editable` turns linked by `previous_response_id` preserve native conversation and a private file mutation. A separate editable trial from the same generation receives a unique clean copy and cannot observe or mutate the @@ -1074,9 +1122,11 @@ caller responsible for acquisition. The temporary fork closes those seams. - HarnessRouter generation, attachment, lease, retention, quota, GC, workspace-integration, and harness-auth-state patches. -- Versioned AllAgents JSON workspace descriptor, validate/resolve/materialize - hook contracts, generation identity, and provenance. -- Project `workspace.yaml` source-catalog additions and projection. +- Versioned AllAgents JSON workspace descriptor with caller-supplied HTTPS Git + repositories, validate/resolve/materialize hook contracts, generation identity, + and provenance. +- Safe public egress enforcement, source-credential scope mapping, and optional + project `workspace.yaml` snapshot-catalog additions. - Deterministic Git and immutable OCI generation construction. - Shared read-only mounts, private editable copies, mode-specific root/nested-repository checkpoint and produced-file integration. @@ -1097,8 +1147,8 @@ caller responsible for acquisition. The temporary fork closes those seams. - Promptfoo runtime code inside AllAgents. - A custom OAuth broker, token translation layer, or automatic native-to-proxy credential fallback. -- Caller-provided origins, credentials, commands, host paths, materializers, or - Docker options. +- Caller-provided credentials, non-HTTPS/private-network origins, commands, host + paths, materializers, or Docker options. - Public multi-tenancy, per-caller authorization, Kubernetes workers, session branching, concurrent turns in one session, or guaranteed prompt-cache hits. - Exact rollback of workspace mutations between successful session turns. @@ -1130,9 +1180,10 @@ flowchart TB PF[Promptfoo provider] -->|UHP + HR API key + workspace JSON| GW[HarnessRouter gateway] GW -->|session CAS + attachment prepare/ack| RUN[HarnessRouter runner] RUN -->|typed validate, resolve, or cache-miss materialize| MAT[AllAgents materializer] - MAT --> CFG[project workspace.yaml source catalog] - MAT --> GIT[Git sources] - MAT --> OCI[OCI registry] + MAT --> POLICY[egress and source-credential scope policy] + MAT --> CFG[optional workspace.yaml snapshot catalog] + MAT --> GIT[caller-requested HTTPS Git sources] + MAT --> OCI[configured OCI registry] RUN -->|atomic publish or reuse| GEN[(immutable generation store)] GEN -->|read-only mount + reference| RO[read-only session] GEN -->|private copy| EDIT[editable session] @@ -1159,10 +1210,11 @@ cwd, and agent launch. Attachment uses a durable prepare/evidence/ack protocol: the runner prepares resources, the gateway alone commits `ready`, and the runner finalizes or rolls back from that acknowledgement. The selected harness owns native OAuth login and refresh; its auth root is outside every generation and -session checkpoint. The materializer owns only the AllAgents JSON schema, -`workspace.yaml` source catalog, source resolution, acquisition, staging -validation, and provenance. It never speaks UHP, authorizes persistence, owns -leases, publishes live state, or writes the gateway session state machine. +session checkpoint. The materializer owns only the AllAgents JSON schema, caller +Git URL validation, optional `workspace.yaml` snapshot catalog, source resolution, +acquisition, staging validation, and provenance. It never speaks UHP, authorizes +persistence, owns leases, publishes live state, or writes the gateway session +state machine. ### Extension Contract @@ -1180,9 +1232,14 @@ Initial UHP request fragment: "retention": "session", "source": { "kind": "repositories", - "revisions": { - "api": "refs/pull/123/head" - } + "repositories": [ + { + "name": "api", + "url": "https://github.com/acme/api.git", + "revision": "refs/pull/123/head", + "destination": "api" + } + ] }, "workingDirectory": { "kind": "repository", @@ -1211,7 +1268,7 @@ Successful terminal response metadata fragment: "allagents.workspace": { "version": "1", "effectiveDescriptorDigest": "sha256:...", - "generationKey": "sha256:...", + "generationId": "sha256:...", "access": "readOnly", "retention": "session", "expiresAt": "2026-09-24T12:00:00Z", @@ -1226,6 +1283,8 @@ Successful terminal response metadata fragment: "repositories": [ { "name": "api", + "url": "https://github.com/acme/api.git", + "destination": "api", "requestedRevision": "refs/pull/123/head", "resolvedCommit": "0123456789abcdef0123456789abcdef01234567" } @@ -1245,22 +1304,27 @@ diagnostic-only, bounded, secret-checked, and never copied verbatim to callers. The hook supports four operations: -- `preflight`: validate contract version, project source catalog, - credential-reference syntax, and required binaries without source network - access or secret values, then return the bounded configured credential- - reference names/opaque IDs. The runner verifies the corresponding store - handles and all staging/result filesystem relationships itself; -- `validate`: validate and default the opaque JSON descriptor and catalog names - without source access, then return a private normalized-descriptor path/digest, - effective descriptor digest, effective access, requested retention, logical - cwd, and a bounded sorted selected credential-reference subset; +- `preflight`: validate contract, acquisition/egress policy, optional snapshot + catalog, credential-scope mapping, credential-reference syntax, and required + binaries without request repository URLs, source network access, or secret + values, then return the bounded configured credential-reference names/opaque + IDs. The runner verifies the corresponding store handles and all staging/result + filesystem relationships itself; +- `validate`: validate and default the opaque JSON descriptor, caller repository + URLs/names/revisions/destinations, snapshot name when applicable, and logical + cwd without source access; select the bounded credential-reference subset from + deployment credential-scope mappings; then return a private normalized- + descriptor path/digest, effective descriptor digest, effective access, + requested retention, + logical cwd, and that selected set; - `resolve`: consume that exact normalized descriptor and selected reference set, - resolve immutable source identity, and return a private canonical source-only - resolved-plan path/digest, generation key, effective cwd, and bounded request - provenance without writing source bytes. The plan contains only generation-key - inputs—resolved commits or OCI digests/layers, normalized destinations, - catalog/policy and sharing-authorization identity, and selected credential- - reference identities—and omits credential values, access, retention, cwd, + safely resolve immutable source identity, and return a private canonical + source-only resolved-plan path/digest, generation key, effective cwd, and + bounded request provenance without writing source bytes. The plan contains + only generation-key inputs—normalized caller URLs, resolved commits or OCI + digests/layers, normalized destinations, snapshot/acquisition/egress and + sharing-authorization identity, and selected credential-reference identities— + and omits repository names, credential values, access, retention, cwd, requested-ref spelling, harness/profile, and session; equal generation keys therefore require identical plan bytes; and - `materialize`: consume those exact resolved-plan bytes and selected reference @@ -1276,13 +1340,13 @@ publication or failure. Validate receives opaque metadata and bounded workspace-input-file count. Resolve receives the validated-descriptor path and digest plus the exact selected credential-reference identities. Materialize receives the resolved-plan path and digest, that same set, and fixed generation -staging/result roots. All operations receive the generic contract version and -project configuration root. Validate/resolve use the request's bounded remaining -deadline; shared materialize uses the runner-owned build deadline and is -cancelled only when no live waiter remains. The runner resolves values for only -the validated selected set and injects them only into source-access operations -through the allowlisted child environment; values never appear in JSON, -generation keys, or persisted plans. +staging/result roots. All operations receive the generic contract version, +project snapshot-configuration root, and deployment acquisition-policy version. +Validate/resolve use the request's bounded remaining deadline; shared materialize +uses the runner-owned build deadline and is cancelled only when no live waiter +remains. The runner resolves values for only the validated selected set and +injects them only into source-access operations through the allowlisted child +environment; values never appear in JSON, generation keys, or persisted plans. Validate returns effective access, requested retention, effective descriptor digest/cwd, selected credential-reference identities, and its private normalized- @@ -1314,13 +1378,15 @@ required string `version` fixed to `"1"`, required `repositories`, and required `entries`. `repositories` is an array with at most 128 items. Every item is an object with -`additionalProperties: false` and exactly the required string fields `name` and -`destination`, validated as `ConfigName` and non-root `RelativeDirectory`. -Names and destinations are each unique; items are sorted by the UTF-8 bytes of -the NFC-normalized `name`. Every destination must exactly equal the `path` of a -directory entry in the same manifest. Duplicate destinations, missing -destination entries, and destinations naming files or symbolic links are -invalid even when the manifest digest is correct. +`additionalProperties: false` and exactly one required string field, +`destination`, validated as a non-root `RelativeDirectory`. Destinations are +unique and items are sorted by the UTF-8 bytes of the NFC-normalized destination. +Every destination must exactly equal the `path` of a directory entry in the same +manifest. Duplicate destinations, missing destination entries, and destinations +naming files or symbolic links are invalid even when the manifest digest is +correct. Logical repository names remain per-request descriptor/provenance data +and do not enter the generation-scoped manifest or semantic Git-state record; +those records identify repository roots by destination. `entries` is an array with at most 500,000 items. Every item has `additionalProperties: false` and is exactly one of: @@ -1370,11 +1436,11 @@ private editable copies. The frozen cross-repository fixture is: ```json -{"entries":[{"mode":"040755","path":"services","type":"directory"},{"mode":"040755","path":"services/api","type":"directory"},{"mode":"100644","path":"services/api/README.md","sha256":"sha256:98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4","size":3,"type":"file"},{"mode":"120000","path":"services/api/current","target":"README.md","type":"symlink"}],"repositories":[{"destination":"services/api","name":"api"}],"version":"1"} +{"entries":[{"mode":"040755","path":"services","type":"directory"},{"mode":"040755","path":"services/api","type":"directory"},{"mode":"100644","path":"services/api/README.md","sha256":"sha256:98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4","size":3,"type":"file"},{"mode":"120000","path":"services/api/current","target":"README.md","type":"symlink"}],"repositories":[{"destination":"services/api"}],"version":"1"} ``` Those exact bytes digest to -`sha256:667fef29fd8d241818263c5697075ba99eb86331c41eda5a27a812dc8771e4f8`. +`sha256:658d89a3127eb79b1479960d3f264456c42c170920cac79e0a6e1837db60d543`. The file bytes are `hi\n`. A change to the schema, fixture bytes, or digest is a versioned contract change, not an implementation detail. @@ -1396,7 +1462,7 @@ error object in `response.error`; workspace failures use | `allagents_workspace_non_resumable` | HTTP 409 `invalid_request_error` before profile admission when a known attached session's bound generation key/epoch/reference/publication/private/checkpoint evidence is missing or corrupt; `param` is `previous_response_id`; because the attachment previously reached `ready`, include its committed complete public workspace metadata | no | | `harness_unavailable` / `detail.reason: "allagents_auth_profile_busy"` | HTTP 503 `server_error` before response allocation for a saturated auth profile; `param` is null | yes | | `harness_unavailable` / `detail.reason: "allagents_auth_profile_unavailable"` | HTTP 503 `server_error` before response allocation for an unavailable or repair-required auth binding; `param` is null | yes | -| `allagents_workspace_invalid` | failed response for post-allocation descriptor, catalog, path, layout, access/retention value, OCI shape/index, or unsupported media rejection that is not a numeric limit | no | +| `allagents_workspace_invalid` | failed response for post-allocation caller repository descriptor, URL/egress-policy, snapshot catalog, path, layout, access/retention value, OCI shape/index, or unsupported media rejection that is not a numeric limit | no | | `allagents_workspace_persistence_forbidden` | failed response when `persistent` retention is not authorized for the selected deployment target | no | | `allagents_workspace_read_only` | failed response when a read-only initial request contains workspace input files or attachment policy would create writable shadow state | no | | `allagents_workspace_capacity_exceeded` | HTTP 503 `server_error` before response allocation when generic session/tombstone admission cannot reserve capacity; otherwise a failed response when finite staging, generation, private, session, or persistence capacity cannot be reserved after safe eviction | yes | @@ -1479,8 +1545,8 @@ into successful empty output and performs no automatic retry. digest, and rehearse deployment from that digest rather than a local build or mutable tag. - Prepare upstream proposals as generic command/plugin and harness-auth-state - seams. Do not require upstream to understand AllAgents metadata, Git catalogs, - OCI manifests, Promptfoo, or a specific OAuth provider. + seams. Do not require upstream to understand AllAgents metadata, Git URL + semantics, OCI manifests, Promptfoo, or a specific OAuth provider. - If upstream accepts an equivalent seam, delete the patch rather than retaining a compatibility layer. @@ -1498,6 +1564,13 @@ into successful empty output and performs no automatic retry. byte/layout-affecting input, bind it to one manifest digest and semantic Git record, and reject drift before reuse. Exclude volatile `.git` representation only after closed semantic validation. +- **Caller-controlled Git URL SSRF or credential forwarding:** Use one strict + canonical URL serialization across validation, policy, credentials, DNS, and + Git. Force every connection and bounded redirect through the public-address + acquisition connector, reject mixed answer sets, and pin the approved address + against DNS rebinding. Bind the helper to a structured credential scope and + strip the credential on any scope escape, including same-origin redirects. + Clear inherited proxies and deny a direct network path. - **Concurrent build and publication race:** Use one runner-owned keyed claim, private staging, independent reconstruction, atomic publication, and one shared result. Each request detaches on its own cancellation/deadline; one @@ -1578,10 +1651,11 @@ into successful empty output and performs no automatic retry. or client cooperation. 5. Freeze hook/state, generation-key, manifest, semantic Git, descriptor, response/expiry, retention, failure, and lifecycle fixtures. -6. Implement `workspace.yaml` source projection, Git validate/resolve/materialize, - credential containment, bounded acquisition, and generation publication. -7. Implement OCI generation construction through the same publication and - attachment path. +6. Implement caller-repository JSON validation, acquisition egress enforcement, + credential-scope mapping, Git validate/resolve/materialize, credential + containment, bounded acquisition, and generation publication. +7. Implement configured `workspace.yaml` OCI snapshot construction through the + same publication and attachment path. 8. Implement terminal-time TTL, persistent authorization, operator deletion, hard private quotas, bounded tombstones, provisional-pin-aware deterministic LRU eviction, and crash recovery. @@ -1679,38 +1753,50 @@ into successful empty output and performs no automatic retry. ### U2. AllAgents workspace contracts and Git materializer -- **Goal:** Implement the JSON descriptor, source-only `workspace.yaml` catalog, - canonical generation identity, deterministic Git construction, logical cwd, - and provenance. +- **Goal:** Implement the caller-repository JSON descriptor, optional + `workspace.yaml` snapshot catalog, canonical generation identity, deterministic + Git construction, logical cwd, and provenance. - **Files:** `src/models/workspace-config.ts`, `src/models/execution-workspace.ts`, `src/core/execution-workspace.ts`, - `src/core/workspace-repo.ts`, one narrow CLI integration entrypoint, generated - schemas, build packaging, configuration docs, and Git E2E fixtures. -- **Approach:** Reuse authoritative workspace parsing and source normalization. - Keep access/retention out of `workspace.yaml`; add them to the JSON execution - descriptor. Generate the manifest schema and add descriptor/preflight/ + `src/core/workspace-repo.ts`, acquisition egress integration, one narrow CLI + entrypoint, generated schemas, build packaging, configuration docs, and Git E2E + fixtures. +- **Approach:** Reuse authoritative URL/path normalization and workspace snapshot + parsing. Keep caller Git URLs plus access/retention in the JSON execution + descriptor; keep only operator-owned snapshot catalog entries in + `workspace.yaml`. Generate the manifest schema and add descriptor/preflight/ validate/resolve/materialize/result schemas, defaults, canonicalization, - generation-key construction, and credential-reference selection. Preflight - returns configured reference identities without values; validate selects a - bounded subset without source access; the runner verifies their handles and - injects only that selected set into source-access children. Resolve the complete - catalog to exact commits without writing source bytes, and materialize only the - exact cache-miss resolved plan into staging. Preserve nested `.git` while - excluding volatile administrative bytes from the source-visible manifest, - enforce closed semantic Git validation, and prove the manifest equals the union - of resolved commit trees at pairwise non-overlapping destinations plus necessary - ancestor directories. Validate destinations/cwd, compute the manifest, and - return without publishing. -- **Verification:** Local HTTPS fixtures cover refs/defaults/HEAD, multiple - repositories, catalog errors, duplicate and ancestor/descendant destinations, - undeclared root/side files, revision grammar, helpers, submodules/LFS/file - protocols, redirects, cancellation, partial cleanup, descriptor defaults, - access/retention validation, configured/selected credential-reference identity - and secret-free validation, generation-key inclusion and exclusion rules, - exact provenance, schema fixtures, commit-tree/manifest reconstruction, - concurrent identical resolve identity, and repository/entry/byte/deadline - boundaries. Different cwd/access/retention/harness/profile/session inputs - produce the same generation key only when resolved source, sharing scope, and + generation-key construction, origin-policy credential selection, and public- + egress enforcement. Preflight returns configured reference identities without + request URLs or values; validate checks URLs/names/revisions/destinations and + selects a bounded mapped subset without source access; the runner verifies + their handles and injects only that selected set into source-access children. + Resolve exactly the caller-declared repositories to commits without writing + source bytes, and materialize only the exact cache-miss resolved plan into + staging. Preserve nested `.git` while excluding volatile administrative bytes + from the source-visible manifest, enforce closed semantic Git validation, and + prove the manifest equals the union of resolved commit trees at pairwise non- + overlapping destinations plus necessary ancestor directories. Validate + destinations/cwd, compute the manifest, and return without publishing. +- **Verification:** Local public-address HTTPS fixtures cover caller URLs, + refs/defaults/HEAD, multiple repositories, duplicate and ancestor/descendant + destinations, undeclared root/side files, revision grammar, helpers, + submodules/LFS/file/ext/ssh protocols, bounded safe redirects, cancellation, + partial cleanup, descriptor defaults, access/retention validation, anonymous + and scope-mapped credential identity, private generation-key/public + generation-ID separation, exact URL provenance, schema fixtures, commit-tree/ + manifest reconstruction, concurrent identical resolve identity, and + repository/entry/byte/deadline boundaries. Network fixtures reject userinfo, + IP literals, controls, whitespace, backslashes, noncanonical IDNA/default-port/ + trailing-dot forms, encoded separators/dot segments, loopback, link-local, + private, reserved, metadata, mixed public/private DNS answers, DNS rebinding, + unsafe redirects, raw-prefix lexical siblings, same-origin scope escapes, + redirect-selected credentials, inherited proxy bypass, and other out-of-scope + credential forwarding. Repository requests that differ only in logical names + share one private generation key and manifest but retain their own names in + descriptor/provenance and cwd resolution. Different cwd/access/retention/ + harness/profile/session inputs also preserve the key when normalized URLs, + resolved source, destinations, sharing scope, egress-policy version, and selected credential-reference identities match. ### U3. Session binding, failures, and credential containment @@ -1806,10 +1892,11 @@ into successful empty output and performs no automatic retry. Pi profiles; exercise login, live turns, atomic refresh, active-turn projection teardown, stale repair, same-binding continuation, same-profile fail-fast exclusion, different-profile concurrency, and profile isolation. Separately - validate proxy broker scope. Run Promptfoo Git/OCI read-only concurrency, + validate proxy broker scope. Run Promptfoo requests that select anonymous public + and origin-mapped private HTTPS Git repositories, Git/OCI read-only concurrency, independently cancelled shared-build waiters, editable two-turn growth and cross-trial isolation, persistence, expiry/deletion/purge, capacity, restart, - cancellation, and every failure mapping. + cancellation, unsafe-URL/egress rejection, and every failure mapping. - **Verification:** Codex and Pi use native OAuth without a provider-route key. Concurrent sessions with different profiles and harnesses share one read-only generation while conversation/home/log/output state remains isolated. @@ -1845,12 +1932,14 @@ into successful empty output and performs no automatic retry. `linux/amd64`, read back the manifest, and attach verified build-provenance and SBOM attestations before E2E. - Document durable generation/session/private/auth volumes; JSON descriptor - versus project `workspace.yaml`; native login/repair and active-turn projection - teardown; proxy mode; terminal-time TTL, hard private quotas, provisional pins, - watermark, persistence authorization, deletion, bounded tombstone compaction, - quarantine, deterministic GC, capacity, metrics, backup, upgrade, and rollback - procedures; and the owner-trust boundary. Review both repositories before + Document durable generation/session/private/auth volumes; caller-supplied Git + URLs in the JSON descriptor versus the optional operator-owned + `workspace.yaml` snapshot catalog; public-egress and source-credential scope + policy; native login/repair and active-turn projection teardown; proxy mode; + terminal-time TTL, hard private quotas, provisional pins, watermark, + persistence authorization, deletion, bounded tombstone compaction, quarantine, + deterministic GC, capacity, metrics, backup, upgrade, and rollback procedures; + and the owner-trust boundary. Review both repositories before final green E2E and prepare generic generation/attachment/lifecycle and auth-state patches for upstream. - **Verification:** A clean `linux/amd64` host verifies attestations and pinned @@ -1881,7 +1970,7 @@ into successful empty output and performs no automatic retry. | Durable lifecycle | Fault injection covers generic and provisional turn admission, generation epochs, build/staging/generation reservations, publication/accounting conversion, build waiters, provisional pins, attachment prepare/ready-ack and private-reservation transfer, references, mounts, private usage, expiry, tombstones/purge, unmount, deletion, quarantine, and GC. Startup reconciles before readiness; no deadline extends, no debit duplicates/leaks, no second epoch appears before prior eviction completes, and no session silently rematerializes. | | Retention and disposal | Fake-clock evidence proves one session CAS rejects busy/expired continuation admission, provisionally saves/clears a valid deadline, and either commits active after profile admission or restores the exact future deadline/tombstones an elapsed one after pre-allocation profile failure. Terminal acknowledgement alone sets the next `expiresAt`; polls/replays do not renew. Invalid failed responses stay accounted through purge; retained expiry returns HTTP 410; purge returns stock unknown; persistence authorizes before source access; operator deletion is idempotent. Null `lastUsedAt` epochs evict first by `publishedAt`; used epochs order by `lastUsedAt`, then `publishedAt`, generation key, and epoch. | | Session continuity | Both modes preserve conversation and fixed generation key/epoch/access/retention/cwd/harness/auth binding while persistent or unexpired; editable preserves private files; read-only remains immutable. Corrupt known evidence returns HTTP 409 non-resumable with no source access or later-epoch substitution. | -| Git acquisition | Closed transport/config, constrained revisions, exact commits, exact object closure/index semantics, catalog validation, generation reuse, and partial cleanup pass against local HTTPS remotes. | +| Git acquisition | Caller-supplied canonical HTTPS URLs, public-address egress enforcement, DNS-rebinding and redirect defense, structured-scope credential isolation, constrained revisions, exact commits, closed transport/config, exact object closure/index semantics, generation reuse, and partial cleanup pass against local network fixtures. | | OCI acquisition | Digest/media/path/link/type/limit, `.git` rejection, generation reuse, and attachment matrix pass against a local registry. | | Credential boundary | Preflight sees no secret values and returns bounded configured reference identities; validate selects a bounded subset; the runner verifies handles and injects only that selected set into source-access children. Source secrets and caller keys are absent from staging, generations, private trees, base environments, checkpoints, backups, logs, and output. The selected OAuth profile is visible only through its active-turn projection, which is absent before acknowledgement and after restart reconciliation. | | Provider boundary | Codex/Pi native OAuth, refresh repair, projection teardown, idempotency/session/profile admission, different-profile concurrency, same-profile fail-fast exclusion, and explicit proxy scope all pass without implicit switching. | @@ -1892,9 +1981,10 @@ into successful empty output and performs no automatic retry. ## Definition of Done - ADR 0002, this plan, implementation, topology, and request examples agree on - the UHP JSON descriptor, project `workspace.yaml` source catalog, immutable - generations, read-only and editable attachments, bounded retention, native - OAuth, explicit proxy mode, and GHCR digest-pinned distribution. + caller-supplied HTTPS Git repositories in the UHP JSON descriptor, the optional + project `workspace.yaml` OCI snapshot catalog, immutable generations, read-only + and editable attachments, bounded retention, native OAuth, explicit proxy mode, + and GHCR digest-pinned distribution. - The U0 evidence predates U1-U6 and both native targets pass on the recorded inputs; changed inputs have replacement evidence before dependent work resumes. - No second execution protocol/control plane, separate AllAgents gateway, direct From 5a48415a475ceae252c65c89dbdb5ce88e4621f7 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Wed, 23 Sep 2026 22:53:13 +1000 Subject: [PATCH 22/44] docs(architecture): simplify repository workspace contract --- .../0002-adopt-uhp-through-harnessrouter.md | 1460 ++++++++--------- ...0837-feat-coding-execution-gateway-plan.md | 270 +-- 2 files changed, 818 insertions(+), 912 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 1c0e2bd6..f23a9f08 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -6,831 +6,721 @@ ## Decision -AllAgents will use the Unified Harness Protocol (UHP) `2026-09-12` through a -pinned HarnessRouter Community Edition deployment for remote Codex and Pi -execution. - -The selected harness owns provider authentication. Codex signs in through -`codex login`; Pi signs in through its `/login` flow for the configured provider. -Those native OAuth sessions are the default and require no provider-route API -key. An explicitly configured API-key-authenticated proxy is a last-resort -route, never an automatic fallback from failed OAuth. Optional describes -deployment configuration, not release scope: version one implements and verifies -the route so operators that reject the native owner-trust boundary have a -supported alternative. - -HarnessRouter owns caller authentication, UHP request and response semantics, -streaming, cancellation, idempotency, session continuity, workspace-generation -publication, session attachments, retention, quotas, garbage collection, agent -execution, usage, and artifacts. AllAgents owns the -`metadata["allagents.workspace"]` JSON descriptor, including caller-supplied -HTTPS Git repository URLs, deterministic Git/OCI generation construction, and -provenance returned through the HarnessRouter response. Project -`workspace.yaml` is not the authority for caller-requested Git origins. - -The initial deployment will use a narrow AllAgents-maintained HarnessRouter fork. -Its generic pre-turn workspace hook resolves a verified immutable source -generation. HarnessRouter reuses an existing generation when possible; otherwise -an AllAgents materializer builds private staging and HarnessRouter atomically -publishes it. Each session then receives either a shared read-only attachment or -a private editable workspace derived from that generation before the harness -starts. - -The fork is a delivery mechanism, not a new protocol. All fork changes must be -structured for a later upstream contribution. Delivery does not depend on -upstream acceptance or timing. - -Implementation details live in the -[coding-agent execution gateway plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). - -## Topology +AllAgents will use the Unified Harness Protocol (UHP) `2026-09-12` through a pinned HarnessRouter Community Edition deployment for remote Codex and Pi execution. UHP remains the only execution protocol. + +HarnessRouter will keep responsibility for caller authentication, UHP behavior, sessions, streaming, cancellation, idempotency, harness execution, usage, artifacts, and lifecycle state. + +A narrow AllAgents-maintained fork will add a generic pre-turn workspace hook. The hook will validate and resolve the caller's source, build a verified immutable workspace when needed, and attach it before the coding harness starts. + +Codex and Pi will use their native login and refresh behavior by default. An +authenticated provider proxy remains an explicitly configured last resort. +Native-auth failure must never activate the proxy automatically. Version one +still implements and verifies proxy mode even when a deployment does not use it. + +Project `workspace.yaml` will remain ordinary local AllAgents configuration plus an optional operator-owned OCI snapshot catalog. It will not control which Git repositories a UHP caller may request. + +The fork is delivery machinery, not a second protocol. We will keep the changes narrow and suitable for upstreaming, but delivery will not depend on upstream acceptance. + +## Main flow + +HarnessRouter has two relevant components. The gateway owns public response and +session transitions. The runner owns processes, filesystems, and resource state. +The external AllAgents materializer validates and builds source. + +An initial workspace-backed turn follows this order: + +1. The gateway authenticates the caller and claims the idempotency key. The + runner selects the harness and authentication binding, admits the native + profile when applicable, and reserves session and tombstone capacity. +2. The gateway allocates the response and session, stores the opaque workspace + descriptor, and asks the materializer to validate it without accessing source. +3. The runner authorizes persistent retention when requested and reserves the + editable workspace allowance when needed. +4. The materializer resolves every Git ref or OCI digest to immutable source + identity and returns provenance. It does not write source bytes during this + step. +5. The runner reuses a valid ready generation, joins an existing build for the + same generation key, or claims a new build and reserves staging and generation + capacity. +6. On a cache miss, the materializer builds private staging. The runner + independently verifies the source tree, Git or OCI state, manifest, limits, + and generation key before publishing it atomically. +7. The runner pins the generation, then mounts it read-only or creates an inode- + independent private editable copy. +8. The gateway commits the attachment as `ready`. The runner acknowledges that + commit, transfers or releases reservations, and releases the provisional pin + exactly once. +9. The runner projects the selected native OAuth profile or issues the configured + scoped proxy credential. HarnessRouter starts the coding harness in the + requested working directory. +10. After the harness and descendants stop, the runner finalizes authentication + state and removes the credential projection. The gateway stores the terminal + response and starts the idle deadline for `session` retention. + +After the first turn: + +1. A continuation reuses only the exact bound attachment. +2. Expiry or deletion tombstones the session before cleanup, waits for active + resources to quiesce, and releases references and reservations exactly once. + +Source resolution, workspace attachment, and authentication selection happen before provider execution. Provider retry or fallback cannot repeat or change them. + +## Phase-zero feasibility gate + +Production workspace implementation must not begin until a minimal pinned image proves native authentication against real provider traffic for both Codex and Pi. The spike excludes the workspace materializer and Git or OCI acquisition. + +Freeze this evidence set: + +| Input | Required identity | +|---|---| +| HarnessRouter | Exact upstream commit | +| Base image | Manifest digest | +| Codex | Exact version | +| Pi | Exact version | +| Auth adapter | Patch digest | + +Changing any input invalidates the evidence. Dependent work remains blocked until both native targets pass again. + +Each target must prove all eight behaviors: + +1. The operator can complete native login in a controlled environment. +2. A real first turn and continuation succeed without a provider-route API key. +3. The selected authentication binding survives restart and fails closed when unavailable. +4. Session conversation state remains separate while only the selected profile is visible. +5. Overlapping refresh-capable turns for one profile are serialized. +6. Credential files remain complete before, during, and after refresh; invalid post-rotation state becomes `repair-required`. +7. Success, failure, cancellation, and crash recovery remove the active projection. Credentials remain absent from retained homes, checkpoints, produced-file records, backups, passive logs, and response metadata. +8. The evidence explicitly records that the selected harness and same-identity tools can read or emit the credential during an active turn. + +The proxy route cannot satisfy this gate. Failure of either native target stops dependent implementation. A proxy-only release or narrower harness scope requires a new decision. + +## System map and ownership ```mermaid flowchart TB - CLIENT[Promptfoo or another UHP client] - GATEWAY[Forked HarnessRouter gateway] - RUNNER[HarnessRouter runner] - MATERIALIZER[AllAgents generation builder] - GENERATIONS[Immutable workspace generations] - READONLY[Read-only session attachment] - EDITABLE[Private editable session workspace] - LIFECYCLE[Leases, retention, quotas, and GC] - HARNESS[Selected Codex or Pi harness] - AUTH[Bound harness-native OAuth profile] - PROXY[Optional authenticated proxy] - MODEL[Model provider] - - CLIENT -->|UHP + metadata.allagents.workspace| GATEWAY - GATEWAY -->|session CAS + prepare/ack| RUNNER - RUNNER -->|validate, resolve, or cache-miss build| MATERIALIZER - MATERIALIZER -->|verified staging + provenance| RUNNER - RUNNER -->|atomic publish or reuse| GENERATIONS - GENERATIONS -->|read-only mount + lease| READONLY - GENERATIONS -->|private copy| EDITABLE - READONLY --> HARNESS + CLIENT[Promptfoo or another UHP client] -->|UHP plus optional workspace metadata| GATEWAY[HarnessRouter gateway] + GATEWAY --> RUNNER[HarnessRouter runner] + RUNNER -->|validate, resolve, materialize| MATERIALIZER[AllAgents materializer] + MATERIALIZER --> SOURCE[HTTPS Git or configured OCI registry] + RUNNER --> GENERATION[(immutable generations)] + GENERATION --> READONLY[shared read-only attachment] + GENERATION --> EDITABLE[private editable copy] + READONLY --> HARNESS[Codex or Pi harness] EDITABLE --> HARNESS - LIFECYCLE -->|expire, delete, or evict| GENERATIONS - LIFECYCLE --> READONLY - LIFECYCLE --> EDITABLE - AUTH -.->|native mode: login and refresh| HARNESS - HARNESS -->|native mode| MODEL - HARNESS -.->|proxy mode: scoped turn credential| GATEWAY - GATEWAY -.->|long-lived proxy client key| PROXY - PROXY -.-> MODEL + RUNNER --> AUTH[(native auth profiles)] + AUTH -->|active-turn projection| HARNESS + HARNESS -->|native OAuth| PROVIDER[model provider] + HARNESS -.->|explicit proxy mode| GATEWAY + GATEWAY -.-> PROXY[authenticated provider proxy] + PROXY -.-> PROVIDER + GATEWAY --> SESSION[(session and attachment state)] + RUNNER --> RESOURCE[(generation and resource journal)] ``` -Promptfoo authenticates to HarnessRouter with a HarnessRouter API key. That -control-plane credential is separate from provider authentication. In the -default route, the selected Codex or Pi process uses its own durable OAuth -profile and refreshes it through the harness's native mechanism. HarnessRouter -does not translate that OAuth session into an API key. - -Native OAuth is an owner-trust mode: the selected harness and tool subprocesses -running under the same operating-system identity may access and emit its -credential during an active turn. The runner projects only the selected profile -through a turn-scoped mount namespace or equivalent same-filesystem view; it -never copies the credential into durable session state. Before terminal -acknowledgement and profile-lock release it commits or rejects refresh, removes -the projection, and verifies retained homes clean. Restart removes or quarantines -stale projections before readiness or profile reacquisition. Checkpoints, -produced-file records, backups, passive logs, request metadata, and response -metadata never serialize the auth file. Deployments that cannot accept active -exfiltration risk must explicitly configure the brokered proxy route. - -## Protocol boundary - -UHP is the sole execution wire contract. Version one uses its Responses-shaped -request, ordered streaming events, `previous_response_id` continuation, -cancellation, files, artifacts, usage, lifecycle, and error semantics. -HarnessRouter's UHP conformance suite is the protocol oracle. - -AllAgents adds one namespaced JSON request extension: +| Actor | Owns | +|---|---| +| UHP client | Prompt, model, harness ID, workspace descriptor, continuation ID | +| HarnessRouter gateway | Caller authentication, UHP validation, public response and session state, attachment `ready`, expiry, tombstones | +| HarnessRouter runner | Generation claims and publication, source child processes, mounts, private copies, quotas, references, pins, profile admission | +| AllAgents materializer | Descriptor defaults, URL and source validation, Git or OCI resolution, staging construction, canonical manifest, provenance | +| Coding harness | Provider login, native token refresh, conversation execution | +| Operator | Deployment policy, egress, credential scopes, authentication profiles, persistence authorization, quotas, deletion, garbage collection | + +The gateway is the only writer of public session attachment, expiry, and tombstone state. The runner is the only writer of generation and resource state. The materializer cannot authorize persistence or publish live state. + +## Protocol and workspace descriptor + +UHP is the sole wire contract and its conformance suite is the protocol oracle. The fork must preserve its Responses-shaped requests, ordered streaming events, `previous_response_id`, cancellation, files, artifacts, usage, lifecycle, and error behavior. + +Version one adds one namespaced first-turn extension: `metadata["allagents.workspace"]`. + +Before response allocation, the gateway requires this extension to be a JSON +object no larger than 64 KiB and no deeper than 32 levels. A non-object receives +HTTP 400 `invalid_input`; a byte or depth overflow receives HTTP 413 +`allagents_workspace_too_large`. + +### Request fields + +| Field | Required | Contract | +|---|---:|---| +| `version` | yes | Exactly `"1"` | +| `access` | yes | `readOnly` or `editable` | +| `retention` | no | `session` by default, or authorized `persistent` | +| `source` | yes | Repository list or configured workspace snapshot | +| `workingDirectory` | no | `{ "kind": "workspaceRoot" }` by default, or `{ "kind": "workspacePath", "path": "…" }` | + +Repository mode accepts one through 128 entries. + +Each repository entry has this shape: + +| Field | Required | Contract | +|---|---:|---| +| `url` | yes | Canonical public HTTPS Git URL; the same URL may appear more than once | +| `ref` | no | Full ref name, unambiguous branch or tag shorthand, or full 40-hex commit ID; omission means remote symbolic HEAD | +| `destination` | yes | Unique, non-root relative directory; destinations must not overlap or collide with a runner-owned control namespace | + +A workspace snapshot source instead names one configured snapshot plus immutable image and workspace-manifest digests. + +Reserved control namespaces include HarnessRouter's root checkpoint repository. +Source-free validation rejects a destination that equals, contains, or is +contained by a reserved namespace. + +`workspacePath` is relative to the mounted workspace and must name a directory in the resolved source manifest. The same working-directory contract applies to repository and snapshot sources. + +Example: ```json { - "metadata": { - "allagents.workspace": { - "version": "1", - "access": "readOnly", - "retention": "session", - "source": { - "kind": "repositories", - "repositories": [ - { - "name": "api", - "url": "https://github.com/acme/api.git", - "revision": "main", - "destination": "api" - } - ] - }, - "workingDirectory": { - "kind": "repository", - "repository": "api", - "path": "packages/service" + "version": "1", + "access": "readOnly", + "retention": "session", + "source": { + "kind": "repositories", + "repositories": [ + { + "url": "https://github.com/acme/api.git", + "ref": "refs/pull/123/head", + "destination": "api" } - } + ] + }, + "workingDirectory": { + "kind": "workspacePath", + "path": "api/packages/service" } } ``` -`metadata["allagents.workspace"]` is the first-response session descriptor, not -the project configuration file. Promptfoo supplies each repository's logical -name, HTTPS Git URL, optional revision, and destination. The pre-turn -materializer resolves and loads those repositories before the harness starts, so -the harness receives the populated workspace; this is not a model-directed -in-agent `git clone`. The descriptor cannot supply credentials, host paths, -commands, environment variables, materializer executables, or Docker options. - -`access` is required and is exactly `readOnly` or `editable`. `retention` is -optional and defaults to `session`; `persistent` is accepted only when enabled -by deployment policy and within persistent-workspace quotas. Access and -retention are independent: either access mode may use either retention class. - -Namespaced response metadata contains the effective descriptor digest, resolved -source provenance, public `generationId`, logical working directory, workspace- -manifest digest, access mode, retention class, and effective expiry. -`generationId` is the SHA-256 digest of versioned RFC 8785 bytes containing only -the returned normalized source provenance, normalized destinations, and -workspace-manifest digest; it is never an internal cache, authorization, -attachment, or lookup key. Active turns and `persistent` sessions report -`expiresAt: null`; a `session` terminal acknowledgement sets the timestamp -returned by terminal, retrieval, and replay paths. Failures before attachment -`ready` omit workspace metadata entirely; terminal failures after `ready` include -the same complete public object. Metadata never exposes the private generation -key, raw request digest, URL credentials (which requests cannot contain), -credential-scope mappings or selected references/values, redirect-chain URLs, -resolved network addresses, physical paths, internal generation-epoch/lease/ -attachment identifiers, or other sessions' quota state. Normalized caller- -supplied repository URLs and resolved commits are returned as source provenance. - -Ordinary UHP input files remain supported for `editable` sessions and are -applied to the private workspace after generation attachment. A `readOnly` -request containing workspace input files is rejected after response allocation -but before source byte acquisition; the system never shadows a read-only -generation with an implicit writable layer. - -## Workspace generations and session semantics - -On the first response, HarnessRouter atomically binds the session to one -canonical workspace descriptor, one verified generation key and publication -epoch, one access mode, one retention class, one harness target, and exactly one -native profile or proxy connection. A new source revision, logical working -directory, access mode, retention class, harness, or authentication binding -requires a new session. - -A workspace generation is immutable source-visible content identified by a -canonical resolved-source key and a verified workspace-manifest digest. Each -publication also has a unique internal epoch ID. At most one live epoch exists -for a key; a later epoch may begin only after durable logical and physical -eviction of the prior one completes, and existing sessions never substitute it. -In repository mode publication also binds a separately validated semantic Git- -state record to the exact resolved commits; volatile `.git` pack/index bytes do -not fragment identity. Sharing authorization and selected credential-reference -identities do fragment the key; credential values do not. Access, retention, -logical working directory, harness target, profile, and session identity do not. - -Concurrent requests for the same absent epoch share one runner-owned build claim -and observe one atomically published result. Each request retains its own -cancellation and deadline: cancellation detaches only that waiter, and the build -continues while another live waiter exists. Failed or partial staging never -becomes attachable. A ready hit or completed build acquires a durable provisional -attachment pin under the same generation lock before mount or copy; garbage -collection cannot race that pin. - -A `readOnly` session mounts the published generation read-only. Multiple sessions -using different harnesses or authentication profiles may execute concurrently -against the same generation while keeping their operating-system identity, -conversation state, harness home, temporary files, logs, outputs, and response -state separate. The filesystem, not caller intent, enforces generation -immutability. - -An `editable` session receives a unique private writable copy derived from the -generation. After a shared publication, each editable waiter independently proves -the generation's full physical byte/inode usage fits its admitted hard allowance; -a non-fitting waiter fails alone without invalidating the ready epoch or another -waiter. No writable inode or checkpoint is shared with another session, and -mutations never flow back into the generation. The allowance covers the private -tree, UHP input overlays, root/nested checkpoints, and produced-file state -throughout every turn and continuation. Exceeding it fails the turn without -changing access or retention. Read-only sessions have no workspace mutation -checkpoint or produced-file delta; editable sessions preserve private mutations, -checkpoints, and produced files. - -The gateway is the sole writer of session attachment, expiry, and tombstone -state. The runner owns generation/reference/mount/private-resource state. It -durably prepares resources and returns opaque attachment evidence; the gateway -commits `ready` and acknowledges it. That acknowledgement causes the runner to -release the provisional pin exactly once. Restart either preserves a committed -attachment or rolls an uncommitted prepare back; neither component independently -binds the other's state. - -One generation use updates `lastUsedAt` only when the gateway commits an -attachment `ready`. Before that it remains null. After acknowledgement, the -runner records `max(existing, readyCommitTimestamp)` under the generation lock; -startup can replay a missed update idempotently from committed gateway evidence. -Publication or a failed prepare does not count as use. Eviction orders null -`lastUsedAt` first by `publishedAt`, then non-null `lastUsedAt`, then -`publishedAt`, ascending generation-key bytes, and epoch-ID bytes. - -A continuation uses `previous_response_id`, omits -`metadata["allagents.workspace"]`, and reuses the original session binding only -when its exact key/epoch evidence remains valid and retention is persistent or -its session idle deadline is unexpired. It sees the same immutable epoch in -`readOnly` mode or the same private writable workspace in `editable` mode. It -cannot change the generation, access, retention, logical working directory, -harness, profile, or proxy connection. A changed or unavailable authentication -binding fails closed until restored. Known missing or corrupt attachment evidence -returns `allagents_workspace_non_resumable`; it never resolves source, -rematerializes, or substitutes a rebuilt epoch. - -Generation resolution and attachment finish before the first agent turn. On a -cache miss, materialization, independent verification, and atomic publication -also finish first; on a hit, byte acquisition is skipped. Source failure starts -no agent process and never falls through to another source mode, access mode, -credential identity, or provider route. - -Every active operation holds a durable session lease and has no idle expiry. One -gateway session CAS checks `session_busy`, exact attachment/binding, and expiry/ -deletion together. A busy or invalid attempt changes no deadline. A valid -`session` continuation stores and clears its unexpired deadline in a provisional -turn-admission fence before the zero-waiter profile attempt. Profile success -commits active; pre-allocation profile failure restores the exact original -deadline when still future or tombstones the session if it elapsed. A read-only -session keeps an exact generation-epoch reference until expiry or deletion. An -editable session holds a provisional generation pin through successful private- -copy attachment, then retains only its private workspace and provenance. Idle -time starts only after durable terminal acknowledgement; GET, stream polling, and -idempotent replay do not renew it. - -Deployment policy supplies finite, nonzero limits for session idle TTL, staging -bytes and concurrent builds, published-generation bytes/count, each editable -session's hard bytes/inodes, total reserved private bytes/inodes, total sessions, -persistent sessions, and tombstone bytes/count/TTL. Before exposing a workspace -response/session, one idempotent admission token durably reserves its generic -session slot and fixed-size tombstone slot. Invalid descriptors remain charged -through finite failed-response retention, tombstoning, and purge. After -validation, `persistent` and the stable editable private-reservation ID are -authorized/reserved before source resolution. Active leases, durable references, -and provisional pins are never evicted. Ready zero-reference/zero-pin epochs are -the only generation GC candidates; failed deletion remains quarantined and -counted, prevents same-key republication, and never advertises freed capacity. If -protected state consumes available quota, new admission fails instead of -deleting protected state or changing policy. - -Expiry or authenticated deletion atomically tombstones the session before -cleanup, rejects new continuations, waits for active work, credential projections, -and mounts to quiesce, removes private state, and releases every reference and -reservation exactly once. Retained tombstones live at least as long as response/ -idempotency records and produce `allagents_workspace_expired`; bounded compaction -then purges both lifecycle identity and its reserved slot, after which the stock -non-disclosing unknown-ID error applies. Neither outcome silently rematerializes. - -Completed, unexpired session state and generation records survive a -HarnessRouter restart when the documented durable volume is preserved. Startup -reconciles build waiters, provisional pins, publications, leases, mounts, private -quota usage, credential projections, tombstones/compaction, and deletion before -readiness or garbage collection. An internal `containment_pending` session stays -non-terminal until its recorded cgroup is empty. In-flight agent processes do not -survive whole-container termination; interrupted turns fail and are not replayed -automatically. +The descriptor is session input, not project configuration. Promptfoo supplies the repositories to load. The materializer loads them before the harness starts; the model never performs the initial clone. + +The descriptor cannot supply credentials, host paths, commands, environment variables, materializer executables, or Docker options. + +A continuation supplies `previous_response_id` and must omit the extension. It reuses the original descriptor, attachment, access, retention, working directory, harness, and authentication binding. + +### Access and retention + +Access and retention are independent: + +| | `readOnly` | `editable` | +|---|---|---| +| Workspace | Shared immutable generation | Private inode-independent copy | +| Initial UHP files | Rejected before source acquisition | Applied after attachment | +| Writes | Filesystem rejects them; no copy-up | Allowed within the private quota | +| Checkpoints | No source mutation checkpoint | Root and nested repositories use private checkpoints | +| Cross-session mutation | Impossible | Impossible | + +| Retention | Behavior | +|---|---| +| `session` | Default. Idle expiry starts only after durable terminal acknowledgement. | +| `persistent` | No idle expiry. Requires deployment authorization and reserved capacity before source resolution. | + +Neither retries nor continuations can change access or retention. + +### Public workspace metadata + +Once attachment reaches `ready`, terminal events, retrieval, background completion, replay, and later terminal failures return the same verified workspace object. + +| Public field | Meaning | +|---|---| +| `effectiveDescriptorDigest` | Digest of the normalized descriptor and defaults | +| `generationId` | Public content identifier | +| `sourceIdentity` | Normalized URL, destination, `requestedRef` when supplied, and `resolvedCommit`, or verified OCI identity | +| `workingDirectory` | Effective `workspaceRoot` or `workspacePath` | +| `workspaceManifestDigest` | Verified source-visible manifest digest | +| `access`, `retention`, `expiresAt` | Effective workspace policy and expiry | + +`generationId` is the SHA-256 digest of versioned RFC 8785 bytes containing only returned source provenance, normalized destinations, and the workspace-manifest digest. It is metadata only. It is never a cache, authorization, attachment, or lookup key. + +Active turns and persistent sessions report `expiresAt: null`. For `session` retention, durable terminal acknowledgement sets the timestamp returned by terminal, retrieval, and replay paths. Polling and replay do not renew it. + +Failures before attachment reaches `ready` omit workspace metadata. Failures after `ready` include the complete committed object. + +Public metadata never exposes the private generation key, raw request digest, URL credentials, credential-scope mappings, selected credential references or values, redirect-chain URLs, resolved network addresses, physical paths, internal epoch, lease, reservation, claim, pin, or attachment identifiers, or other sessions' quota state. + +## Source authority and acquisition + +### Configuration boundary + +| Source | Authority | +|---|---| +| Caller-requested Git | The UHP JSON descriptor supplies URL, optional ref, and destination | +| OCI snapshot | Operator-owned `workspace.yaml` snapshot catalog plus caller-supplied immutable digests | +| Harness, model, persistence, quota, and egress policy | HarnessRouter deployment configuration | +| Source credentials | Operator-owned secret store and credential-scope mappings | + +`workspace.yaml` is not a Git-origin allowlist. The service may accept any repository reachable through its safe public HTTPS egress boundary. + +### URL and network rules + +Before parsing a Git URL, validation rejects ASCII controls, whitespace, and backslashes. It then parses the URL once with the WHATWG URL Standard and requires the input bytes to equal the serialized URL exactly. + +The serialized URL must meet all of these rules: + +- scheme is `https`; +- hostname is an ASCII lowercase IDNA A-label DNS name without a trailing dot; +- no userinfo, query, fragment, IP literal, or explicit default port; +- path is non-empty; and +- no percent-encoded control, slash, backslash, or dot segment. + +The same serialization and structured `(scheme, host, effectivePort)` origin drive policy, credentials, redirects, DNS, provenance, generation identity, and the URL passed to Git and libcurl. Local paths and `file`, `ssh`, `git`, and extension transports are rejected. + +Every connection follows this sequence: + +1. Route the acquisition child through the deployment connector. The child has no direct network path and no inherited proxy configuration. +2. Resolve the canonical hostname. Reject the whole answer set if any address is loopback, link-local, private, reserved, metadata, or otherwise non-public. +3. Pin one approved address for that connection so DNS rebinding cannot change the destination. +4. Accept at most five HTTPS redirects. Parse, serialize, resolve, and validate every hop again. + +Deployment policy may further restrict public egress, but it does not need to list every allowed repository. + +### Source credentials + +Callers cannot provide credentials or credential-reference names. + +A credential scope is either an exact structured origin or that origin plus a canonical repository-path segment prefix. Prefixes match complete path segments, never raw strings. The matching rule with the most path segments selects one server-owned secret reference. No match means anonymous acquisition. + +Preflight receives no request URL or secret value. It validates policy and configured reference syntax, then returns bounded reference names or opaque IDs. The runner verifies the selected handles before source access. + +A source-access child receives only the selected value. It has an isolated home, +`GIT_CONFIG_NOSYSTEM=1`, no global Git configuration, no inherited proxy +variables, no Git or remote proxy configuration, and no direct network path. Its +ephemeral credential helper uses `credential.useHttpPath=true` and independently +enforces the selected protocol, host, port, and path scope. + +Every redirect is checked against the original scope. The connector strips the credential when a redirect leaves that scope, including a same-origin path escape. A redirect never selects a new credential. + +Credentials are never encoded in URLs, persisted in Git configuration or remote +URLs, or returned in hook output. Temporary credential state is removed before +return. The gateway, runner base environment, published generation, editable +copy, and every agent child remain credential-free. If a configured source secret +appears in the service or agent environment, the runner refuses to launch the +agent. + +### Git resolution and verification + +`ref` is at most 255 ASCII bytes. It may be a full 40-hex object ID or a ref name accepted by rules equivalent to `git check-ref-format`. + +Validation rejects leading dashes, whitespace, controls, refspec colons, glob metacharacters, traversal-like components, `@{`, and `.lock` components. It resolves a full ref or unambiguous branch or tag shorthand with `ls-remote`. A full object ID is accepted only when advertised. + +The materializer records the normalized URL, requested ref when present, and resolved commit in provenance. Fetch and checkout commands receive only the verified object ID, never caller ref text. + +Every Git and libcurl operation uses an argument vector without a shell and +explicit end-of-options handling. Hooks, `file` and `ext` protocols, submodule +recursion, Git LFS hydration, and configured clean and smudge filters are disabled +before the materializer touches caller-selected source. + +The published repository keeps `.git` for coding tools, but the materializer normalizes it to a closed detached-HEAD state. It removes reflogs, `FETCH_HEAD`, locks, hooks, worktree links, alternates, shallow, replace, and graft state, extra refs and objects, and credential-bearing configuration. + +The runner independently verifies: + +- `HEAD` resolves to the recorded commit; +- the index exactly matches that commit tree; +- the object database contains the complete required transitive closure, with no missing, corrupt, or extra objects; +- the canonical object-ID, type, and size set matches its recorded digest; and +- source-visible content equals the union of the resolved commit trees at their declared destinations, plus only the ancestor directories needed to connect them. + +Any undeclared path fails integrity validation. + +### OCI snapshots + +Snapshot mode accepts only a direct OCI image manifest from the configured +repository, selected by immutable image-manifest and workspace-manifest digests. +Redirects may not change registry authority. The config descriptor must use the +snapshot's configured workspace-manifest media type and address the canonical +workspace-manifest bytes. + +| Limit | Maximum | +|---|---:| +| Distributable tar, gzip, or zstd layers | 64 | +| Image manifest | 4 MiB | +| Workspace-manifest blob | 128 MiB | +| Repository roots | 128 | +| Compressed layers | 8 GiB | +| Expanded tree | 32 GiB | +| Filesystem entries | 500,000 | +| One regular file | 4 GiB | +| One UTF-8 path | 4096 bytes and 128 components | +| One PAX or extended header | 1 MiB | + +Before writing an entry, the materializer checks its type, path, link target, and +declared size. It rejects devices, sockets, traversal, escaping links, sparse +files, unknown or foreign layers, mutable tags, and undeclared output. The runner +independently rejects a 129th repository root. + +The materializer verifies the image manifest, workspace manifest, and every layer +size and digest before use, then applies OCI whiteouts. It recomputes the canonical +manifest from staging and requires it to match both the fetched manifest bytes and +the caller-provided digest. Snapshot mode rejects `.git` administrative subtrees. +Snapshots that require Git history use repository mode. + +### Canonical workspace manifest + +Both source modes produce the same versioned canonical manifest. Its RFC 8785 bytes enumerate every source-visible directory, regular file, and symbolic link in logical path order, including normalized mode, size, content digest, or link target. + +Repository roots are identified by destination. Git mode may omit only separately verified `.git` administrative subtrees. OCI mode rejects them. No source-visible path may be omitted. + +The runner reads the manifest through a private bounded result root, verifies its digest, walks staging without following links, reconstructs the same entries, and requires byte-for-byte canonical equality. The manifest never appears inside the published source tree. + +### Generation identity + +The private generation key is computed before materialization. It includes every input that can change source-visible bytes, filesystem semantics, or sharing authorization. + +| Included | Excluded | +|---|---| +| Descriptor and hook contract versions | Access and retention | +| Deployment authorization scope | Working directory | +| Normalized caller Git URLs | Harness, profile, and session identity | +| Resolved commits or immutable OCI digests | Physical paths | +| Normalized destinations | Credential values | +| Selected credential-reference identities | Caller ref spelling after it resolves to the same commit | +| Snapshot identity when applicable | Volatile Git pack, index, and stat representation | +| Acquisition and egress policy version | | + +Publication binds one private key and one internal epoch to one verified workspace-manifest digest and, for repositories, one semantic Git-state record. Materialization receives the exact private resolved plan and never resolves source again. + +## Generation publication and attachments + +A generation is immutable source content identified by its private generation key, manifest digest, and unique internal epoch. At most one live epoch may exist for a key. A replacement epoch cannot begin until durable logical and physical eviction of the prior epoch completes. + +Concurrent cache misses for the same key join one runner-owned build claim. Each +waiter keeps its own deadline and cancellation. Cancelling one waiter does not +cancel the build while another waiter remains; the runner cancels it when no +waiter remains. Failed or partial staging is never attachable, and a failed +competing build does not poison an existing verified generation. + +Before a mount or copy, the runner acquires a provisional pin under the generation lock. Garbage collection cannot race that pin. + +The generation backing store remains owner-writable and is never exposed writable to a session. Publication is a recoverable same-filesystem atomic transition. Editable copies may not share mutable inodes with the generation or another session. + +Read-only sessions share source bytes but keep their operating-system identity, +conversation, home, temporary files, logs, outputs, and response state separate. + + +The gateway and runner commit an attachment in three steps: + +1. The runner prepares the mount or private copy and returns opaque evidence. +2. The gateway commits attachment state as `ready`. +3. The runner acknowledges that commit, creates the durable reference or transfers the private reservation, and releases the provisional pin exactly once. + +Restart preserves a gateway-committed attachment. It rolls back an uncommitted prepare. + +`lastUsedAt` changes only when the gateway commits `ready`. Its initial value is null. A ready commit sets it to the later of the existing value and commit timestamp under the generation lock. Replay is idempotent. Publication and failed prepare do not count as use. + +Garbage collection orders candidates as follows: + +1. Generations with null `lastUsedAt`, ordered by `publishedAt`. +2. Other generations, ordered by `lastUsedAt`, then `publishedAt`. +3. Ties use ascending generation-key bytes, then epoch-ID bytes. + +Only ready generations with zero references and zero provisional pins are candidates. + +## Session lifecycle, retention, and recovery + +### Session binding and continuation + +Deployment configuration gives each harness exactly one authentication binding. +A proxy binding is a closed server-side record containing its private HTTPS base +URL, expected TLS identity or CA, supported API format and endpoint set, +gateway-only client-key handle, broker audience, and requested-to-proxy model +map. Callers cannot override these fields. + +A target is advertised only after its binding passes readiness checks. Native +OAuth checks login, refresh, and a live turn. Proxy mode checks schema, TLS, +broker, model mapping, endpoints, and live compatibility. + +The first response binds one normalized descriptor, generation key and epoch, +access mode, retention class, working directory, harness target, authentication +mode, binding identity, and canonical binding-configuration digest. + +A continuation uses `previous_response_id`, omits workspace metadata input, +requires the exact bound attachment and binding-configuration digest, and +succeeds only when retention is persistent or the session idle deadline is still +in the future. + +A changed ref, resolved source, working directory, access, retention, harness, +authentication mode, binding identity, or binding-configuration digest requires +a new session. A continuation never re-resolves source, follows a changed +same-named connection, or substitutes a later generation epoch. + +Missing or corrupt generation, reference, publication, private workspace, or checkpoint evidence returns `allagents_workspace_non_resumable`. The system does not rebuild the missing state for that session. + +### Turn admission and idle expiry + +Every active operation holds a durable lease and has no idle expiry. + +One gateway compare-and-swap checks `session_busy`, exact binding, and expiry or deletion together. A busy or invalid turn changes no deadline. + +For an eligible continuation, the gateway saves and clears the current idle deadline in a provisional admission fence before the runner attempts to acquire the selected native profile. Profile success commits the session as active. A pre-allocation profile failure restores the exact saved deadline when it is still future, or tombstones the session if that deadline elapsed. + +After terminal acknowledgement, a `session` workspace receives one idle deadline. GET, polling, background completion, and replay never extend it. Persistent sessions keep `expiresAt: null`. + +### Capacity + +Every deployment limit must be finite and nonzero: + +| Capacity | Required limit | +|---|---| +| Sessions | Total active and retained sessions | +| Failed identity | Tombstone count, bytes, and TTL | +| Builds | Concurrent builds and staging bytes | +| Generations | Published count and bytes | +| Editable workspaces | Per-session hard bytes and inodes | +| Private storage | Total reserved bytes and inodes | +| Persistence | Persistent session count | +| Idle retention | Session idle TTL | + +Before a response is visible, one idempotent admission token reserves a generic session slot and a fixed-size tombstone slot. Invalid descriptors remain charged through failed-response retention, tombstoning, and purge. + +After validation and before source resolution, the runner authorizes persistence +and reserves its slot. Editable access also receives one stable private- +reservation ID. Its hard byte and inode allowance covers the private tree, UHP +overlays, root and nested-repository checkpoints, and produced-file state across +every turn and continuation. + +A cache miss reserves staging and prospective generation capacity before byte acquisition. Independent full-tree accounting converts that reservation to actual retained usage before publication. + +A waiter receives an editable copy only when the complete generation fits its private allowance. A non-fitting waiter fails alone and does not invalidate the shared generation or another waiter. + +Protected state is never evicted. If leases, references, pins, or other protected resources consume capacity, admission fails instead. + +### Expiry and deletion + +Expiry and authenticated deletion follow this order: + +1. Atomically tombstone the session and reject new continuations. +2. Wait for active work, credential projections, and mounts to quiesce. +3. Remove private state. +4. Release every generation reference and reservation exactly once. +5. Record successful deletion, or keep failed physical deletion quarantined and counted for retry. + +A retained tombstone lives at least as long as matching response and idempotency records and returns `allagents_workspace_expired`. Bounded compaction removes the identity and reserved tombstone slot only after those records expire. Later requests receive the stock non-disclosing unknown-predecessor error. + +Neither path rematerializes source. + +### Restart recovery + +Completed unexpired or persistent sessions and ready generations survive restart. + +Before readiness or garbage collection, the gateway and runner reconcile generic +admission tokens and reservations, active leases, turn-admission fences, build +claims and waiters, staging and generation reservations, publications, pins, +references, mounts, editable usage, attachment prepare and acknowledgement, +credential projections, tombstones, compaction, quarantine, and interrupted +deletion. + +An internal `containment_pending` session remains non-terminal until its recorded cgroup is empty. Whole-container termination does not preserve agent processes; interrupted turns fail and are not replayed. + +## Provider authentication + +| | Native OAuth | Authenticated proxy | +|---|---|---| +| Default | Yes | No; explicit configuration only | +| Provider credential owner | Codex or Pi harness profile | Proxy service | +| Harness receives | Selected turn-scoped profile projection | Non-refreshable scoped turn credential | +| Refresh | Harness-native | Not allowed for the turn credential | +| Automatic fallback | Never | Never | + +Promptfoo's HarnessRouter API key authenticates the UHP caller only. HarnessRouter never translates it into provider credentials. + +Each native harness target has one dedicated durable authentication root outside generations, editable workspaces, session checkpoints, and conversation state. Codex uses file credential storage under `CODEX_HOME`. Pi uses `~/.pi/agent/auth.json` after controlled `/login`. + +Missing, expired, revoked, or unrefreshable native OAuth disables that harness +target. HarnessRouter does not switch to another profile or provider route. + + +Native OAuth uses an owner-trust boundary. During an active turn, the selected harness and same-operating-system-identity tools may read or emit that profile's credential. Operators that require stronger isolation must use the explicit proxy route or isolate the whole deployment more strongly. + +Login, logout, and repair acquire the same runner-owned zero-waiter profile lock +as an active turn. They use the same durable fence and `finally` release and +acknowledgement protocol. + +A native turn follows this order: + +1. Acquire the runner-owned profile lock. Version one allows exactly one active refresh-capable turn per profile and no waiters. +2. Project only the selected profile through a turn-scoped mount namespace or equivalent same-filesystem view that preserves native atomic file replacement. +3. Run the harness and descendants. +4. Commit or reject refresh state after descendants stop. +5. Remove the projection and verify the retained session home is clean. +6. Persist terminal acknowledgement, then release the profile lock. + +A local refresh commit uses a same-filesystem temporary file, file `fsync`, atomic rename, parent-directory `fsync`, and validation. If a crash after provider rotation leaves invalid local state, restart marks the profile `repair-required` and requires native login again. It never switches profiles or activates the proxy. + +HarnessRouter claims each `Idempotency-Key` atomically. Requests with the same +key share one result. + +Different profiles may run concurrently on one generation. A new cross-session turn that collides on a busy profile fails immediately before response allocation with HTTP 503 `harness_unavailable` and reason `allagents_auth_profile_busy`. Stock idempotent replay and same-session `session_busy` take precedence. + +The runner supervisor holds turn admission and the profile lock through +descendant termination, refresh disposition, projection teardown, and terminal +acknowledgement. Gateway failure cannot release them. Runner failure leaves a +durable fence. Startup blocks readiness and profile admission until it reconciles +that fence and every stale projection. + +In proxy mode, the gateway issues a non-refreshable credential bound to one proxy +audience, harness target, model allowlist, response and turn ID, and the UHP +deadline plus minimal clock skew. It may authorize only the bounded provider +calls, compaction, and retries needed by that turn. Cancellation or terminal +completion revokes it. The broker rejects wrong audience, model, turn, expiry, or +revocation. Checkpoints, logs, artifacts, stored responses, and retained session +state never persist the proxy turn credential. ## Fork boundary -The HarnessRouter fork is limited to the workspace-integration seam and the -harness-native authentication-state seam. The workspace seam: - -1. recognizes one configured, bounded JSON metadata key and, before response - allocation, uses the idempotent admission transaction to reserve its generic - session and tombstone slots together with applicable profile admission; -2. canonicalizes the initial JSON, records its raw digest, binds it to the - allocated session, and invokes source-free generic `validate` to return - effective access, requested retention, effective descriptor digest/cwd, a - bounded selected credential-reference set, and a private normalized- - descriptor reference; the runner verifies that set against secret-free - preflight declarations and credential-store handles; -3. makes the runner the sole persistence authority and, before source - resolution, reserves any persistence slot and one stable editable hard-private - byte/inode reservation ID; -4. invokes `resolve` with the exact validated descriptor and selected reference - set to produce an exact private source-plan reference/digest, generation key, - and bounded provenance; -5. atomically joins or creates a runner-owned generation epoch build, preserves - each waiter's cancellation/deadline, takes ownership of the exact resolved - plan and selected reference identities, reserves full staging/prospective- - generation capacity before byte acquisition, and acquires a provisional pin - before handing a ready epoch to attachment; -6. invokes `materialize` with that resolved plan and selected set only on a miss, - independently validates staging, semantic Git state, and its manifest, proves - the materializer boundary empty, computes physical retained byte/inode usage, - and under the generation lock atomically converts prospective capacity to - actual usage, releases excess plus staging reservation, persists accounting, - and publishes the immutable epoch before waiter pins; -7. independently checks each editable waiter's initial copy fit, then has the - runner prepare a read-only epoch reference/mount or a private editable copy - carrying the admitted private-reservation ID, has the gateway alone commit the - attachment `ready`, and on acknowledgement transfers the reservation without - a second debit and releases the provisional pin; -8. allows a symlink-safe logical working directory while preserving per-session - identity and writable-state isolation; -9. checkpoints and collects produced files only from private editable state and - enforces its byte/inode quota across every turn; -10. persists bounded generation, attachment, retention, expiry, and provenance - metadata through streaming, terminal, retrieval, and idempotent replay paths; -11. rejects the workspace key on continuations and never renews retention for - polling or replay; -12. applies ordinary input files only to editable private state; -13. strips every configured materializer-only environment name from agent - children; and -14. owns crash-safe build/pin/reference reconciliation, bounded tombstones, - expiry, deletion, quota admission, and deterministic eviction of only ready - unreferenced and unpinned generation epochs. - -The authentication-state seam separates session conversation state from durable -per-harness OAuth state. In native mode it projects only the selected profile -into the Codex or Pi home for the active turn and permits the harness to persist -token refreshes. The projection preserves the harness's credential-file write -and atomic-replacement behavior without copying the credential into the retained -session home. It is removed after descendant termination and refresh disposition, -before terminal acknowledgement or lock release. Checkpoints, produced-file -records, backups, passive logs, and public metadata exclude it. Other profile -roots are never mounted. This does not prevent the selected harness or -same-identity tools from reading or emitting the credential inside the accepted -owner-trust boundary. - -A locally committed refresh uses a same-filesystem temporary file, file and -parent-directory `fsync`, atomic rename, and validation. A crash after the -provider rotates credentials but before local commit may leave the profile stale; -restart marks it `repair-required` when validation fails, removes or quarantines -stale projections, and requires native login again before readiness. It never -switches profiles or activates the proxy. Version one supports exactly one active -refresh-capable turn per native profile and holds that profile lock for every turn -and every login, logout, or repair operation. Admission first atomically claims -the UHP `Idempotency-Key`; concurrent same-key requests share one -admission/result. The session CAS returns stock `session_busy` before changing -the deadline, then provisionally fences a genuinely new turn before it tries the -zero-waiter profile lock. Collision returns HTTP 503 `harness_unavailable` with -`detail.reason: "allagents_auth_profile_busy"` before response allocation, -runner work, or materialization, and rolls the session fence back to the exact -future deadline or an elapsed-deadline tombstone. - -The runner turn supervisor persists the admission record and owns the profile -lock through descendant termination, refresh disposition, projection teardown, -and terminal-state acknowledgement. Gateway-only failure cannot release it. -Runner failure leaves a durable fence; startup blocks readiness and admission -until descendant, projection, and profile reconciliation. Operators provision -distinct profiles for parallel capacity. - -The generic fork layer does not understand the AllAgents descriptor. It enforces -only the configured key, JSON/size bounds, immutable first-turn binding, typed -hook envelope, runner-owned persistence authorization, lifecycle, and response -namespace. The external AllAgents executable owns schema/default validation, -workspace configuration, Git/OCI acquisition, source-credential selection, -source-tree construction policy, and provenance; it never authorizes retention -or writes live session state. - -The fork must preserve stock behavior for requests without the configured key -and must continue to pass upstream UHP conformance. The maintained patch series -is pinned to an upstream commit, covered by focused integration tests, and kept -free of unrelated changes. The intended upstream contributions are the generic -materializer boundary and secure harness-auth state separation, not the -AllAgents-specific descriptor schema. +The HarnessRouter fork is limited to two generic seams: -## Phase-zero feasibility gate +1. A pre-turn workspace hook with typed `preflight`, `validate`, `resolve`, and `materialize` operations. +2. A harness-authentication-state seam that keeps provider profiles separate from conversation and workspace state. + +| Hook operation | Responsibility | +|---|---| +| `preflight` | Validate contract and policy versions, configured credential references, and required tools without request URLs, network access, or secret values | +| `validate` | Apply descriptor defaults, validate URL, ref, destination, access, retention, and working-directory syntax, and select bounded credential references without source access | +| `resolve` | Resolve immutable Git commits or OCI identity and return the private resolved plan, generation key, effective working directory, and public provenance without writing source bytes | +| `materialize` | Consume the exact resolved plan on a cache miss, write only private staging and result roots, and return the manifest without publishing or re-resolving source | + +The generic fork understands only the configured metadata key, generic JSON and byte limits, immutable first-turn binding, the typed hook envelope, runner resource ownership, lifecycle state, and the response namespace. It does not understand the AllAgents schema, Git, OCI, or credential-selection policy. -The harness-native auth adapter is a blocking phase-zero spike. Production -workspace-materializer implementation must not begin until a minimal pinned image -using the release's HarnessRouter, base-image, and Codex/Pi inputs proves the -adapter with real provider traffic. The spike does not need the AllAgents -materializer, Git acquisition, or OCI acquisition. - -The gate evidence records the HarnessRouter commit, base-image digest, Codex -version, Pi version, and auth-adapter patch digest. Those inputs are frozen for -dependent work. Changing any of them invalidates the gate: dependent work must -stop until both native targets pass again on the new input set. - -Each required native target must prove: - -1. operator-controlled native login in its dedicated profile root; -2. a real first turn and continuation without a provider-route API key; -3. persisted auth-binding identity across restart and fail-closed behavior when - that binding is changed or unavailable; -4. session-specific conversation state with only the selected auth profile - visible to the harness identity; -5. serialized overlapping turns for one profile; -6. complete local credential files after termination before, during, and after - refresh persistence, with invalid post-rotation state becoming - `repair-required`; -7. active-turn-only projection teardown on success, failure, cancellation, and - crash recovery, with no auth path retained in homes, mounts, checkpoints, - produced-file records, backups, passive logs, or response metadata; and -8. explicit acknowledgement that same-identity harness tools can read or emit - the selected credential. - -The proxy route cannot satisfy this gate on behalf of a native target. If either -required native target fails, dependent implementation stops. Continuing with a -proxy-only target or narrower harness scope requires an explicit decision change; -the implementation must not introduce an implicit fallback or credential shim. - -## Source authority and credentials - -Project `workspace.yaml` remains the source of truth for the ordinary local -AllAgents workspace and the optional operator-owned OCI snapshot catalog. It is -not a Git origin allowlist and is not consulted to translate repository names in -a UHP request. HarnessRouter deployment configuration owns harness/model/provider -targets, persistence authorization, idle TTLs, quotas, garbage-collection policy, -outbound network policy, and optional source-credential scope mappings. - -For repository mode, the UHP JSON descriptor supplies one through 128 repository -objects containing required `name`, `url`, and `destination` fields plus an -optional `revision`. `workingDirectory.repository` references `name`; -`destination` is a non-empty, non-root relative path. Names and destinations are -unique, and destinations are pairwise non-overlapping. The descriptor is session -input, not parsed as, merged with, or persisted as a replacement for -`workspace.yaml`. - -An authenticated caller may request any repository reachable through the -deployment's HTTPS egress boundary. Before parsing, validation rejects ASCII -controls, whitespace, and backslashes. It parses once with the WHATWG URL -Standard and requires the input bytes to equal the serialized URL exactly. That -serialization must use `https`, an ASCII lowercase IDNA A-label DNS hostname -without a trailing dot, no userinfo/query/fragment or IP literal, no explicit -default port, a non-empty repository path, and no percent-encoded control, slash, -backslash, or dot segment. The same serialization and structured `(scheme, host, -effectivePort)` origin drive policy, credentials, redirects, DNS, provenance, -generation identity, and the exact Git/libcurl request. Local paths and `file`, -`ssh`, `git`, and extension transports are rejected. - -The acquisition child cannot bypass the deployment connector through direct -network access or inherited proxy configuration. Resolution and each connection -or redirect reject the entire DNS answer set if any address is loopback, link- -local, private, reserved, metadata, or otherwise non-public; the connector pins -one approved address for each connection. At most five redirects are accepted. -Each is parsed and serialized by the same rules, re-resolved, and rechecked. -Deployment policy may further restrict egress but does not require every -repository to be predeclared. - -The materializer `preflight` receives no repository URL or secret value. It -validates hook/policy versions, credential-scope mapping syntax, and source tools, -then returns bounded configured credential-reference names or opaque IDs. A -credential scope is either an exact structured origin or that origin plus a -canonical repository-path segment prefix; a prefix matches complete segments, -never raw strings. Source-free `validate` selects the matching rule with the most -path segments for each normalized URL. The runner, not the hook, verifies those -store handles and actual filesystem relationships and injects values only into -`resolve` or `materialize`. - -Repository mode acquires exactly the caller-declared repository set. It accepts -only a bounded ref-name grammar, rejects option-like or refspec-shaped values, -resolves the requested revision—or the remote symbolic HEAD when omitted—to a -full commit before agent execution, fetches by verified object ID, and records -the normalized URL, requested revision, and commit in provenance. It preserves -`.git` for coding tools but hermetically normalizes the allowed detached-HEAD -configuration/ref set and removes reflogs, `FETCH_HEAD`, locks, hooks, worktree -links, alternates, shallow/replace/graft state, extra refs, extra objects, and -credential-bearing configuration. The runner independently verifies HEAD, an -index exactly matching the resolved commit tree, its canonical object-set digest, -and exactly the transitive required object closure with no extras. It then proves -the source-visible manifest equals exactly the union of each resolved commit tree -prefixed by its pairwise non-overlapping destination plus only necessary -destination ancestor directories. Undeclared paths outside that union fail -integrity validation. - -Snapshot mode accepts only a configured OCI repository plus immutable image- -manifest and workspace-manifest digests. It verifies the image manifest, -canonical workspace-manifest bytes, layer sizes and digests, applies OCI -whiteouts, validates the resulting declared workspace layout against the -manifest, rejects `.git` administrative subtrees, and records the ordered layer -digests. Version-one snapshot requests use `workspaceRoot`; snapshots requiring -Git history or repository-relative working directories use repository mode. - -Both source modes produce the same versioned canonical workspace manifest. Its -RFC 8785 bytes enumerate every source-visible directory, regular file, and -symbolic link in logical path order with normalized mode, size, content digest, -or link target as applicable. Repository mode omits only separately validated -`.git` administrative subtrees so volatile pack/index/stat representation does -not fragment identity; no source-visible path may be omitted. Git mode computes -the manifest from completed staging and binds it to the semantic Git-state record -for the resolved commits. OCI mode carries the same bytes in the configured -workspace-manifest blob and must reproduce them after applying the layers. The -runner receives the manifest through a private bounded result root, verifies its -digest, source-visible tree, and any omitted Git state independently, and never -places the manifest in source content. - -Before materialization, resolution computes a canonical generation key from every -input that can affect source-visible bytes, declared agent-visible filesystem -semantics, or sharing authorization: descriptor and hook contract versions, -deployment authorization scope, normalized caller Git URLs, bounded selected -credential-reference identities, resolved commits or immutable OCI digests, -normalized destinations, snapshot identity when applicable, and acquisition/ -egress policy version. Logical repository names, physical paths, access, -retention, working directory, harness/profile identity, credential values, and -volatile Git administrative representation are excluded. Publication binds that -key and one -internal epoch to one verified workspace-manifest digest and, in repository mode, -one semantic Git-state record. Materialize receives the exact private source-only -plan bytes/digest and selected reference set returned by resolve; it never -re-resolves source. - -The generation backing store is owner-writable and never exposed writable to a -session. Publication is a recoverable same-filesystem atomic transition. -Editable copies may use a safe copy or snapshot mechanism but may not share -mutable inodes with the generation. Corrupt, partial, quarantined, or deleting -generations are not attachable. - -Source credentials are selected server-side from an owner-only secret mount or -credential-store handle available to the runner, not from request JSON or the -long-lived service environment. Anonymous access is used when no configured -credential scope matches. Otherwise the runner resolves only the selected value -when constructing a source-access child environment. That child has an isolated -HOME, no inherited proxy variables, no direct network path, hermetic Git/ -registry configuration, `credential.useHttpPath=true`, and an ephemeral helper -that independently rejects any protocol, host, effective port, or canonical -repository path outside the selected structured scope. Each redirect is checked -against the originally selected scope; credentials are stripped whenever it -leaves that scope, including a same-origin path-prefix escape, and a redirect -never selects a new credential. Credentials are never encoded in the URL, -persisted in Git configuration or remote URLs, or emitted. Temporary credential -state is removed before return. The gateway/runner base environment and every -agent child remain credential-free; if a configured source secret appears there, -the runner refuses to launch the agent. - -## Trust and deployment - -HarnessRouter API authentication is mandatory on every externally reachable UHP, -response/session retrieval, stream, cancellation, file, artifact, persistence, -deletion, and lifecycle-administration endpoint, even on a private network. -Unauthenticated requests disclose neither existence nor retention state. -Gateway-to-runner operations are not externally -routable and are mutually authenticated. Operators should still bind the -deployment to loopback or a private network and enforce Tailscale ACLs, firewall -policy, or equivalent controls. Version one is not a public multi-tenant service. - -HarnessRouter CE provides per-session operating-system identities and private -runtime state, not a hostile-code sandbox. Immutable generations may be mounted -read-only into multiple session identities, but no session receives write access -to their backing store. Editable source state, harness homes, temporary files, -outputs, and checkpoints remain private to one session identity. - -Native harness OAuth therefore requires an operator-owned, private deployment: -agent tools sharing the harness identity may access that harness's OAuth profile. -Operators requiring stronger provider credential isolation must use the explicit -brokered proxy route or place the complete deployment inside a stronger -isolation boundary. - -The deployment uses a pinned custom HarnessRouter image containing: +Requests without the extension keep stock behavior. Upstream UHP conformance must stay green. Production pins an upstream commit and carries a focused patch series with no unrelated changes. + +The upstream proposal should contain only the generic workspace and authentication-state seams. If upstream accepts an equivalent interface, remove the corresponding fork patch rather than keeping a compatibility layer. + +### Materializer containment + +Each hook invocation receives one runner-owned cgroup-v2 leaf under the delegated +subtree. + +| Property | Requirement | +|---|---| +| Placement | Put the child in the leaf atomically with `clone3(CLONE_INTO_CGROUP)`, or use a stopped, secret-free pre-exec move-and-verify handshake | +| Authority | The child and its descendants cannot administer or escape the leaf | +| Termination | Cancellation, deadline, or parent exit with live descendants fails the invocation; use `cgroup.kill` when descendants remain | +| Proof | Require `cgroup.events` to report `populated 0` before reading a result, publishing, releasing a secret, cleaning roots, or exposing terminal state | + +If the leaf cannot be emptied, internal state becomes `containment_pending`. +Readiness and terminal visibility remain blocked until restart reconciliation +proves it empty and records the preserved outcome once. + +## Trust, deployment, and release + +HarnessRouter API authentication is mandatory on every externally reachable +create, continuation, retrieval, stream, cancellation, file, artifact, +persistence, deletion, and lifecycle-administration endpoint. This remains true +on a private network. Authentication fails before resource lookup, disclosure, or +mutation, so an unauthenticated request reveals neither session existence nor +retention state. + +The gateway-to-runner channel is mutually authenticated and not externally routable. The service binds to loopback or a private network with equivalent ACL or firewall controls. Version one is not a public multi-tenant service. + +HarnessRouter supplies per-session operating-system identities. It is not a hostile-code sandbox. No session identity may write the generation backing store. Editable workspaces, homes, temporary files, output, and checkpoints remain session-private. + +The pinned production image contains: - an OCI base image pinned by digest; -- the pinned HarnessRouter CE revision plus the reviewed patch series; -- the AllAgents materializer executable and its locked runtime dependencies; -- version-locked OS packages and Git/OCI source-acquisition tools; and +- the pinned HarnessRouter CE commit and reviewed patch series; +- the AllAgents materializer and locked runtime dependencies; +- version-locked OS packages and Git or OCI tools; and - pinned HarnessRouter-supported Codex and Pi versions. -The runtime grants only the runner a delegated cgroup v2 subtree and applies an -`on-failure` restart policy. The attested image, durable volumes, project -configuration, secret handle, and cgroup delegation are mounted in non-serving -initialization mode before preflight or reconciliation. Readiness stays false -unless that delegation is usable and startup has removed or quarantined every -orphaned materializer cgroup and credential projection. - -Readiness also requires finite session idle TTL; build/staging, generation, -per-editable-session hard byte/inode, total private reservation, session, -persistence, and tombstone quotas; a writable private staging and editable- -workspace volume; and a protected immutable generation store. The runner -validates their actual mount, same-filesystem publication, quota, and isolation -relationships; the hook does not. Readiness also requires successful startup -reconciliation of build waiters, provisional pins, references, quota usage, -mounts, credential projections, tombstone compaction, and interrupted deletion. -Operators must be able to observe aggregate generation, private-workspace, -persistent-session, tombstone, quarantine, and failed-deletion capacity without -receiving source paths or credentials. - -AllAgents publishes the `linux/amd64` release image as the public package -`ghcr.io/allagentsdev/harnessrouter`. Version and commit tags are mutable -discovery labels; deployment configuration pins the published manifest digest. -A protected release workflow publishes from an approved ref, uses commit-pinned -actions, and separates unprivileged build/test jobs from the environment-approved -publish job. GitHub's package permission replaces third-party registry -credentials. The final manifest digest receives GitHub/Sigstore build-provenance -and SBOM attestations. Both must verify the expected repository, workflow, ref, -subject digest, and predicate before deployment. - -Each configured harness target binds exactly one authentication union: -`nativeOAuth` plus a profile, or `proxyApiKey` plus a proxy connection. -`nativeOAuth` is the default. The operator runs `codex login` against a dedicated -Codex auth root or Pi `/login` against a dedicated Pi auth root during controlled -setup. Codex uses file credential storage under `CODEX_HOME`; Pi uses -`~/.pi/agent/auth.json`. Both harnesses own token refresh. Conversation and -rollout state remain session-scoped, while refreshed OAuth state persists in the -selected auth root outside immutable generations and private workspace -checkpoints. - -The runner verifies the selected binding and a live turn before advertising the -target: login status and refresh for native OAuth, or proxy configuration, -broker, and endpoint compatibility for `proxyApiKey`. A missing, expired, -revoked, or unrefreshable OAuth profile disables that target; it does not select -another profile or fall through to an API key. - -`proxyApiKey` is optional to configure but its implementation and verification -remain required version-one scope. It is an explicit last-resort mode. -HarnessRouter keeps the long-lived proxy client key in the gateway. It gives the -harness a -non-refreshable broker credential bound to one proxy audience, harness target, -model allowlist, response/turn ID, and the UHP deadline plus minimal clock skew. -The token may authorize the bounded provider calls, compaction, and retries -needed during that active turn. Cancellation or terminal completion revokes it; -logs, checkpoints, artifacts, and stored responses do not passively persist it. -A configured proxy such as `codex-lb` owns its upstream provider authentication. -The HarnessRouter broker must reject wrong-audience, wrong-model, wrong-turn, -expired, or revoked credentials. Native OAuth failure never activates this route -automatically. +Only the runner receives the delegated cgroup v2 subtree. The container uses `on-failure` restart policy. + +Startup begins in non-serving mode. Readiness requires all of the following: + +- verified image attestations and mounted inputs; +- usable cgroup delegation and cleanup of orphaned materializer cgroups; +- removal or quarantine of stale credential projections; +- finite lifecycle and capacity limits; +- writable staging and editable volumes; +- a protected generation store; +- verified mount, same-filesystem, quota, and isolation relationships; +- successful materializer preflight and native or proxy authentication checks; and +- complete startup reconciliation before serving or garbage collection. + +Operators must be able to observe aggregate generation, editable-workspace, persistent-session, tombstone, quarantine, and failed-deletion capacity without exposing source paths or credentials. + +AllAgents publishes the public `linux/amd64` image as `ghcr.io/allagentsdev/harnessrouter`. Version and commit tags are discovery labels, not immutable deployment identities. Deployments pin the manifest digest. + +The release workflow uses an approved ref, commit-pinned actions, an unprivileged build and test job, and a separate environment-approved publish job. GitHub package permission replaces third-party registry credentials. + +The final digest receives GitHub/Sigstore build-provenance and SBOM attestations. Deployment verifies the expected repository, workflow, ref, subject digest, predicate, base-image digest, lockfiles, OS packages, source tools, and Codex and Pi versions. ## Failure behavior -- **Invalid extension:** the source-free hook validation rejects unknown or - malformed repository names, URLs, revisions, destinations, source, access, - retention, or logical-working-directory fields before source resolution. -- **Unauthorized persistence:** after response allocation but before source - resolution or byte acquisition, the runner rejects - `retention: "persistent"` when deployment policy does not authorize it; the - hook never authorizes and the runner never silently downgrades it to `session`. -- **Read-only input overlay:** after response allocation but before source - resolution, reject workspace input files on a `readOnly` request. A runtime - write receives the filesystem's read-only failure and never causes copy-up or - mode conversion. -- **Extension on a continuation:** reject without changing session state, - acquiring a lease, or clearing its idle deadline. -- **Expired, deleted, or purged session:** while its bounded tombstone remains, - return `allagents_workspace_expired` before runner or profile work. After - tombstone and matching response/idempotency retention are purged, return the - stock non-disclosing unknown-predecessor error. Neither rematerializes source. -- **Non-resumable session:** a known attached session with missing or corrupt - bound generation key/epoch, reference, publication, private workspace, or - checkpoint evidence returns HTTP 409 `allagents_workspace_non_resumable` - before profile admission. It never substitutes a rebuilt epoch. Because the - attachment previously reached `ready`, the error includes its committed - complete public workspace metadata. -- **Busy auth profile:** after atomic idempotency replay and stock `session_busy` - precedence, a genuinely new cross-session turn fails immediately with HTTP 503 - `harness_unavailable` and - `detail.reason: "allagents_auth_profile_busy"` before response allocation, - runner work, or materialization; sessions using other profiles may continue - concurrently. -- **Capacity exhaustion:** before response allocation, atomically reserve generic - session/tombstone capacity or return HTTP 503 - `allagents_workspace_capacity_exceeded` without exposing state. After - validation but before source resolution, reserve persistence and the one - editable hard byte/inode allowance. After resolve but before byte acquisition, - a miss claim reserves full staging and prospective generation byte/count - capacity. Independent post-build full-tree accounting atomically converts the - generation reservation to physical retained usage and releases excess and - staging before ready. Each editable waiter then independently proves the - initial copy fits its hard allowance; a non-fitting waiter fails - `allagents_workspace_private_quota_exceeded` without invalidating the epoch or - siblings. Evict only ready epochs with zero references and zero provisional - pins. Never evict protected state or alter access or retention. -- **Editable quota exhaustion:** fail an editable waiter whose initial copy does - not fit, or let the filesystem deny a later private-workspace write beyond its - reserved byte or inode allowance; the runner returns - `allagents_workspace_private_quota_exceeded`. Persistent sessions cannot exceed - the same fixed envelope; retries never change mode or retention. -- **Source policy, authentication, or acquisition failure:** reject any - destination that resolves or redirects outside the permitted public HTTPS - egress boundary before source bytes reach staging. Strip credentials whenever - a redirect leaves the originally selected structured scope, including a same- - origin path-prefix escape. For authentication or transport failure, remove only - unpublished staging, publish no generation, and start no agent or provider - fallback. An existing verified generation is not poisoned by a failed - competing build. Cancelling one build waiter detaches only it; other live - waiters keep the runner-owned build alive. -- **Generation, attachment, or private-copy failure:** quarantine corrupt or - incomplete state, release reservations and provisional pins exactly once, - acquire no live attachment, and fail closed. A session never substitutes - another generation after binding. Prepare/ack reconciliation preserves only a - gateway-committed ready attachment. -- **Materializer timeout, cancellation, malformed result, crash, or live - descendant after parent exit:** create a runner-owned cgroup v2 leaf and start - the child inside it atomically with `clone3(CLONE_INTO_CGROUP)` or a stopped, - secret-free pre-exec move-and-verify handshake. The child cannot escape or - administer the subtree. On every outcome, kill remaining members and wait for - `cgroup.events` to report `populated 0` before any terminal result, private - manifest read, generation publication, secret release, or cleanup. A completed - parent with a live descendant returns the containment failure. An unquiescent - leaf remains internal `containment_pending`; the runner exits, blocks readiness - and terminal visibility, and startup transitions once to the preserved failed, - cancelled, or incomplete outcome only after proving it empty. -- **Expiry or deletion race:** atomically fence new turns before unmounting or - deleting. Repeated deletion is idempotent; failed physical deletion remains - quarantined and counted against quota for retry. Tombstone compaction is - bounded, ordered, durable, and never precedes response/idempotency retention. -- **HarnessRouter restart:** preserve completed, unexpired or persistent state; - reconcile generic and provisional turn admission, staging/prospective- - generation reservations, publication/accounting, builds/waiters, provisional - pins, references, private reservation transfer and actual usage, attachment - prepare/ack, credential projections, tombstones/compaction, and cleanup before - readiness. Interrupted turns fail without automatic replay. -- **Provider authentication failure:** fail the selected harness target without - switching OAuth profiles or activating the proxy/API-key route. Refresh - disposition and projection teardown still complete before the profile lock is - released. -- **Provider execution failure:** return HarnessRouter's normalized UHP failure - without source fallback or credential material in public output. -- **Public error mapping:** use UHP request errors before response allocation and - terminal failed responses afterward. New codes carry the `allagents_` vendor - prefix. Promptfoo maps every non-success to a coded error, never successful - empty output or an automatic retry. Failures before attachment `ready` omit - workspace metadata. Failures after `ready` include the same complete verified - public workspace object as success; internal epoch/reservation identifiers stay - private. Partial acquisition never appears as a complete workspace identity. +The system fails closed. Source, access mode, retention, credentials, and provider route never change as a recovery shortcut. + +| Failure | Public behavior | Required effect | +|---|---|---| +| Non-object workspace extension | HTTP 400 `invalid_input` before allocation | Reject before session lookup or durable admission | +| Extension exceeds 64 KiB or 32 levels | HTTP 413 `allagents_workspace_too_large` before allocation | Reject before canonicalization, session lookup, or durable admission | +| Invalid bounded descriptor shape, URL syntax, ref syntax, destination, or working-directory syntax | `allagents_workspace_invalid` after allocation | Reject without source access | +| Unknown or unadvertised ref | Failed response during resolution | Allow only bounded remote ref resolution; acquire no source bytes and create no attachment or agent | +| `workspacePath` is missing or not a directory | `allagents_workspace_invalid` after checking the verified manifest | Reject before attachment or agent launch | +| Unauthorized `persistent` retention | `allagents_workspace_persistence_forbidden` | No source resolution or byte acquisition; no downgrade | +| Initial files with `readOnly` | `allagents_workspace_read_only` | No source acquisition and no writable shadow layer | +| Workspace extension on a continuation | Invalid request | No session mutation, lease, or deadline change | +| Expired or deleted retained session | HTTP 410 `allagents_workspace_expired` | No runner or profile work; no rematerialization | +| Purged predecessor | Stock non-disclosing unknown-predecessor error | No rematerialization | +| Missing or corrupt bound attachment evidence | HTTP 409 `allagents_workspace_non_resumable` | No profile admission or epoch substitution; return committed workspace metadata | +| Busy native profile after replay and `session_busy` checks | HTTP 503 `harness_unavailable`, reason `allagents_auth_profile_busy` | Fail before allocation, runner work, or materialization | +| Generic capacity unavailable | HTTP 503 `allagents_workspace_capacity_exceeded` | Admit no response or source work | +| Editable copy or later growth exceeds its allowance | `allagents_workspace_private_quota_exceeded` | Fail only that waiter or turn; preserve mode and retention | +| Source policy, authentication, or transport failure | Coded failed response | Remove unpublished staging; publish nothing; start no agent or provider fallback | +| Generation, attachment, or copy failure | Coded failed response | Quarantine incomplete state and release reservations and pins exactly once | +| Materializer timeout, crash, malformed output, or live descendant | Coded materializer or containment failure | Wait for cgroup quiescence before result handling, secret release, or cleanup | +| Provider authentication failure | Normalized UHP failure | Do not switch profile or activate proxy; finish teardown before lock release | +| Provider execution failure | Normalized UHP failure | No source fallback and no credential material in output | + +Capacity is reserved in order: generic session and tombstone before response visibility; persistence and editable allowance after validation and before source resolution; staging and prospective generation after resolution and before byte acquisition; actual retained usage before publication. Each reservation is released or transferred exactly once. + +A failed physical deletion remains quarantined and counted. The system never advertises that capacity as free, resurrects the resource, or permits a same-key replacement before physical deletion completes. + +New vendor codes use the `allagents_` prefix. Promptfoo maps every non-success to a coded error and never converts failure into empty success or automatic retry. ## Consequences -HarnessRouter remains the sole execution control plane. Its generation index, -session references, retention timestamps, tombstones, and deletion state are -subordinate workspace lifecycle state, not a parallel task/session API. -AllAgents does not add another streaming lifecycle, process supervisor, artifact -service, provider adapter, or Promptfoo-specific runtime. - -AllAgents owns workspace selection, deterministic Git/OCI generation -construction, source credentials, and provenance. HarnessRouter owns atomic -generation publication, read-only attachment, private editable copies, quotas, -expiry, deletion, and garbage collection. The selected harness owns provider -OAuth login and refresh. - -Read-only sessions avoid repeated byte acquisition and may run different -harness/profile bindings concurrently against one immutable generation. Editable -sessions consume a reserved private byte/inode envelope and never share -mutations. Persistent sessions, active read-only references, provisional pins, -and retained tombstones reduce available capacity, making crash-consistent -accounting, deterministic LRU tie-breaking, bounded compaction, and admission -operational requirements. Native OAuth deliberately exposes only the selected -turn-scoped profile projection inside the operator-controlled harness trust -boundary; source-acquisition credentials remain isolated from the harness and -every published generation. - -The integration requires a maintained fork and custom image. The fork must be -rebased and tested against upstream releases until the generic seams are -accepted or equivalent supported extensions exist. +HarnessRouter remains the sole execution and session control plane. AllAgents adds workspace preparation and source policy without adding another streaming API, process supervisor, artifact service, provider adapter, or task engine. + +Shared read-only generations avoid repeated acquisition and may serve different harnesses and profiles concurrently. Editable sessions trade that reuse for a reserved private byte and inode envelope. + +Persistent sessions, active references, provisional pins, retained tombstones, and quarantined deletion failures consume finite capacity. Crash-consistent accounting and admission rejection are operational requirements, not optional optimizations. + +Native OAuth deliberately trusts the selected harness and same-identity tools during an active turn. Source-acquisition credentials remain outside that boundary and never enter a generation or agent environment. + +The maintained fork must be rebased and tested against selected upstream releases until equivalent supported seams exist. ## Alternatives rejected -- **Custom execution gateway:** duplicates mature UHP/HarnessRouter session, - streaming, cancellation, authentication, artifact, and provider behavior. -- **Proxy-first provider authentication:** adds a mandatory API key and network - hop even when Codex or Pi can use the operator's subscription directly. The - proxy remains an explicit compatibility and isolation fallback. -- **Thin adapter in front of stock HarnessRouter:** avoids a fork but introduces - another network service and makes source acquisition a client-side concern. -- **Put the descriptor in the prompt:** lets the model control acquisition and - is not deterministic or safe. -- **Expose acquisition as an MCP tool:** depends on the model choosing to call it - and runs too late to define the initial working directory. -- **Upload every source file as UHP input files:** works for small regular-file - snapshots, but loses exact symlink, mode, and OCI layer semantics and moves - repository acquisition to every caller. -- **Rematerialize a private source tree for every trial:** is simple but repeats - network, CPU, and storage work for identical read-only jobs and prevents safe - concurrent reuse of verified immutable content. -- **Wait for upstream before delivery:** makes the product schedule depend on a - project we do not maintain. +| Alternative | Why rejected | +|---|---| +| Custom execution gateway | Duplicates mature UHP session, streaming, cancellation, authentication, artifact, and provider behavior | +| Proxy-first provider authentication | Adds a mandatory API key and network hop when native Codex or Pi authentication works | +| Thin adapter in front of stock HarnessRouter | Adds another network service and pushes source lifecycle outside the session control plane | +| Descriptor in the prompt | Lets the model control acquisition and is neither deterministic nor safe | +| Acquisition through an MCP tool | Runs only if the model chooses it and cannot define the initial working directory | +| Upload every source file as UHP input | Loses exact Git history, symlink, mode, and OCI layer semantics and moves acquisition to every caller | +| Rematerialize a private tree for every trial | Repeats network, CPU, and storage work and prevents safe immutable sharing | +| Wait for upstream | Makes delivery depend on a project we do not maintain | ## Deliberate limits -Version one does not add evaluation datasets, scoring, assertions, automatic -retries, session branching, concurrent turns within one session, simultaneous -refresh-capable turns sharing one auth profile, caller-supplied credentials, -non-HTTPS or private-network Git origins, public multi-tenancy, arbitrary -materializer commands, mutable OCI tags, transparent source-mode fallback, or -guaranteed provider prompt-cache hits. +Version one does not add evaluation datasets, scoring, assertions, automatic retries, session branching, concurrent turns within one session, simultaneous refresh-capable turns for one native profile, caller-supplied credentials, non-HTTPS or private-network Git origins, public multi-tenancy, arbitrary materializer commands, mutable OCI tags, transparent source-mode fallback, or guaranteed provider prompt-cache hits. -Read-only attachments never copy up or become editable. Editable workspaces never -share mutations across sessions. Callers cannot choose arbitrary TTLs, bypass -persistence quotas, or convert retention on continuation. Leased or pinned state -is never an eviction candidate, and retention is never unbounded by default. +Read-only attachments never copy up or become editable. Editable sessions never share mutations. Callers cannot choose arbitrary TTLs, bypass persistence quotas, or change retention on continuation. Leased, referenced, or pinned state is never evicted. Default retention is always bounded. ## Reconsider when Revisit this decision when: -- either required harness-native OAuth target cannot pass the phase-zero gate; -- the host cannot enforce immutable read-only generation mounts across session - identities; -- continuation, expiry, deletion, and lease acquisition cannot be made - linearizable and crash-safe; -- generation churn or authorized persistent demand cannot fit practical bounded - quotas; -- editable derivation requires stronger filesystem semantics than a private copy - can provide; -- upstream HarnessRouter accepts the generic workspace lifecycle seam or exposes - an equivalent supported extension; -- UHP adopts a standard workspace attachment or retention contract that - supersedes the namespaced extension; +- either required native harness fails the phase-zero gate; +- the host cannot enforce immutable read-only generation mounts across sessions; +- continuation, expiry, deletion, and lease acquisition cannot be linearized and recovered safely; +- generation churn or authorized persistent demand cannot fit practical finite quotas; +- editable derivation requires stronger filesystem semantics than an independent private copy; +- upstream HarnessRouter accepts the generic workspace or authentication-state seam; +- UHP adopts a standard workspace attachment or retention contract that replaces this extension; - the maintained patch grows beyond the narrow integration boundary; -- HarnessRouter changes or removes required UHP/session/provider behavior; -- exact per-turn workspace rollback becomes a product requirement; -- the host cannot enforce public-address egress validation, connection address - pinning, bounded redirect revalidation, and out-of-scope credential stripping; -- source acquisition must run in a stronger isolation boundary; +- HarnessRouter removes required UHP, session, or provider behavior; +- exact per-turn workspace rollback becomes a requirement; +- the host cannot enforce public-address egress validation, address pinning, redirect revalidation, and out-of-scope credential stripping; +- source acquisition needs a stronger isolation boundary; - callers require a public multi-tenant authorization model; or -- a second independent UHP implementation offers a materially smaller and more - stable integration surface. +- another UHP implementation offers a materially smaller and more stable integration surface. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index fcb68190..ea0f1fdf 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -37,10 +37,11 @@ execution: code owns the protocol, fork, trust, generation, attachment, retention, harness-authentication, and provider-routing decisions. UHP `2026-09-12` and HarnessRouter's conformance suite own execution-wire behavior. The namespaced - UHP JSON extension owns caller-supplied HTTPS Git URLs, revisions, - destinations, per-session source selection, access, retention request, logical - cwd, and provenance semantics. Project `workspace.yaml` remains ordinary local - workspace configuration plus the optional operator-owned OCI snapshot catalog; + UHP JSON extension owns caller-supplied HTTPS Git URLs, refs, destinations, + per-session source selection, access, retention request, workspace-relative + working directory, and provenance semantics. Project `workspace.yaml` remains + ordinary local workspace configuration plus the optional operator-owned OCI + snapshot catalog; it is not a Git origin catalog for UHP. HarnessRouter deployment configuration owns egress policy, source-credential scope mappings, harness IDs, model allowlists, authentication bindings, persistence authorization, finite TTLs, @@ -100,9 +101,9 @@ private editable copy before provider dispatch. A continuation supplies `previous_response_id`, omits the workspace extension, and uses HarnessRouter's current native conversation plus the bound attachment. Read-only sessions see the same immutable generation; editable sessions see the -same private mutations. A different source revision, working directory, access -mode, retention class, harness, or authentication binding requires a new -session. An expired or deleted session is never silently rematerialized. +same private mutations. A different source ref, working directory, access mode, +retention class, harness, or authentication binding requires a new session. +An expired or deleted session is never silently rematerialized. ### Problem Frame @@ -184,8 +185,9 @@ caller responsible for acquisition. The temporary fork closes those seams. purged before deterministic eviction of unreferenced/unpinned generations. Admission fails when protected state consumes finite quota. - **Let authenticated callers select Git origins.** Promptfoo supplies canonical - HTTPS Git URLs, revisions, logical names, and destinations. The service accepts - any repository reachable through safe public egress; callers cannot supply + HTTPS Git URLs, optional refs, and unique destinations. The same URL may appear + more than once at different refs or destinations. The service accepts any + repository reachable through safe public egress; callers cannot supply credentials, non-HTTPS transports, host paths, commands, or Docker options. - **Prefer harness-native OAuth.** Promptfoo's HarnessRouter API key authenticates the UHP caller only. Codex and Pi use their own login, token storage, refresh, @@ -317,35 +319,37 @@ caller responsible for acquisition. The temporary fork closes those seams. an extension-bearing continuation before session lookup/CAS and changes no session state or deadline. The AllAgents hook's source-free `validate` operation validates and defaults the exact v1 object - `{ version: "1", access, retention?, source, workingDirectory? }`. `access` is - exactly `readOnly | editable`; omitted `retention` means `session`, otherwise - it is exactly `session | persistent`. `source` is exactly - `{ kind: "repositories", repositories: NonEmptyArray<{ name: ConfigName, - url: HttpsGitUrl, revision?: RevisionText, destination: RelativeDirectory }> }` - or `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, - workspaceManifestDigest: Digest }`. `workingDirectory` is exactly - `{ kind: "workspaceRoot" }` or - `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }`. - The repository form is valid only when `source.kind` is `repositories` and its - `repository` names one request entry; v1 snapshot requests use `workspaceRoot`. - The hook validates and reports the requested retention but never authorizes it. + `{ version: "1", access, retention?, source, workingDirectory? }`. + `access` is exactly `readOnly | editable`; omitted `retention` means `session`, + otherwise it is exactly `session | persistent`. + + `source` is exactly one of: + - `{ kind: "repositories", repositories: NonEmptyArray<{ + url: HttpsGitUrl, ref?: RefText, destination: RelativeDirectory }> }`; or + - `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, + workspaceManifestDigest: Digest }`. + + `workingDirectory` is exactly `{ kind: "workspaceRoot" }` or + `{ kind: "workspacePath", path: RelativeDirectory }`. The workspace path is + relative to the mounted workspace and must name a directory in the resolved + source manifest. It works for both repository and snapshot sources. The hook + validates and reports requested retention but never authorizes it. The runner is the sole persistence authority: before source resolution or byte acquisition it authorizes `persistent`, reserves the session and persistence slots, or fails `allagents_workspace_persistence_forbidden`. - **R6.** The AllAgents hook expands omitted `retention` to `session` and omitted - `workingDirectory` to `{ kind: "workspaceRoot" }`; an omitted repository - `revision` remains absent and means remote symbolic HEAD. It canonicalizes each - HTTPS URL, NFC-normalizes strings, sorts repository entries by name, rejects + `workingDirectory` to `{ kind: "workspaceRoot" }`; an omitted repository `ref` + remains absent and means the remote symbolic HEAD. It canonicalizes each HTTPS + URL, NFC-normalizes strings, sorts repository entries by destination, rejects unknown fields, and hashes RFC 8785 bytes as the effective descriptor digest. A separate canonical generation key covers only inputs that can affect source- visible bytes, declared agent-visible filesystem semantics, or sharing authorization: hook/schema versions, deployment authorization scope, normalized caller Git URLs, bounded selected credential-reference identities, resolved commits or OCI digests, normalized destinations, snapshot identity when - applicable, and acquisition/egress policy version. Repository names, access, - retention, logical cwd, harness/profile, session identity, physical paths, - credential values, and volatile Git administrative representation do not - fragment that key. + applicable, and acquisition/egress policy version. Access, retention, logical + cwd, harness/profile, session identity, physical paths, credential values, and + volatile Git administrative representation do not fragment that key. Publication binds it to the independently verified workspace-manifest digest and semantic Git record when applicable. Omitted and explicit default values have the same effective descriptor digest. @@ -360,7 +364,7 @@ caller responsible for acquisition. The temporary fork closes those seams. normalized descriptor reference/digest plus effective access, requested retention, logical cwd, effective descriptor digest, and the bounded sorted credential-reference names/opaque IDs selected by deployment policy for the - requested origins—not values. The runner verifies those references were + requested URLs—not values. The runner verifies those references were declared by preflight and that their handles exist, then maps only that selected set into source-access child environments. After runner authorization and admission, `resolve` consumes that exact validated descriptor and selected set, @@ -526,9 +530,9 @@ caller responsible for acquisition. The temporary fork closes those seams. entries do not merge with user configuration. Git repository URLs do not come from `workspace.yaml`. - Repository mode takes one through 128 request entries. Each has a unique - `name`, canonical absolute `https` `url`, optional `revision`, and unique, - pairwise non-overlapping `destination`. Before parsing, reject ASCII controls, + Repository mode takes one through 128 request entries. Each has a canonical + absolute `https` `url`, optional `ref`, and unique, pairwise non-overlapping + `destination`. URLs need not be unique. Before parsing, reject ASCII controls, whitespace, and backslashes. Parse once with the WHATWG URL Standard and require the input bytes to equal its serialized URL exactly. The serialization must have an ASCII lowercase IDNA A-label DNS hostname without a trailing dot, @@ -539,8 +543,8 @@ caller responsible for acquisition. The temporary fork closes those seams. generation identity, and the exact Git/libcurl request. Local paths and non- HTTPS schemes fail source-free validation. Destinations are non-empty, non-root relative child paths and cannot collide with HarnessRouter's root - checkpoint repository. Duplicate names, duplicate or ancestor/descendant - destinations, escaping destinations, and unsupported URL forms also fail. + checkpoint repository. Duplicate or ancestor/descendant destinations, escaping + destinations, and unsupported URL forms also fail. HarnessRouter deployment configuration owns harness/model/provider targets, persistence authorization, TTLs, quotas, GC, outbound egress policy, and @@ -553,15 +557,14 @@ caller responsible for acquisition. The temporary fork closes those seams. repository URL to be predeclared. Project `workspace.yaml` never contains session access, retention, lease, or eviction state. - **R10.** Repository mode materializes exactly the caller-declared repository - set. Use the requested `revision`, or the remote symbolic HEAD when omitted. - `RevisionText` is at most 255 ASCII bytes and is either a full 40-hex object ID - or a `git-check-ref-format`-equivalent ref name. Reject leading dashes, - whitespace and controls, refspec colons, glob metacharacters, traversal-like - components, `@{`, and `.lock` components. Resolve a validated full ref, or an - unambiguous shorthand under `refs/heads/` or `refs/tags/`, with `ls-remote`; - accept object IDs only when advertised. Subsequent fetch/checkout commands - receive only the verified object ID with explicit end-of-options handling, - never caller revision text. + set. Use the requested `ref`, or the remote symbolic HEAD when omitted. + `RefText` is at most 255 ASCII bytes and is either a full 40-hex object ID or a + `git-check-ref-format`-equivalent ref name. Reject leading dashes, whitespace + and controls, refspec colons, glob metacharacters, traversal-like components, + `@{`, and `.lock` components. Resolve a validated full ref, or an unambiguous + shorthand under `refs/heads/` or `refs/tags/`, with `ls-remote`; accept object + IDs only when advertised. Subsequent fetch/checkout commands receive only the + verified object ID with explicit end-of-options handling, never caller ref text. Allow only argument-vector HTTPS Git operations through the deployment's acquisition egress connector. The child cannot bypass it: clear every proxy/ @@ -724,10 +727,10 @@ caller responsible for acquisition. The temporary fork closes those seams. protections under the generation lock, durably records logical eviction, and completes physical deletion before permitting a new epoch for that key. Failed deletion stays quarantined and counted against quota. Startup reconciles - generic admission slots, build/staging/prospective-generation reservations, - publication/accounting markers, build waiters, provisional pins, references, - mounts, private reservation transfers and actual usage, copies, tombstones, - compaction, and deletion before readiness or GC. If only protected + generic admission slots, active leases, build/staging/prospective-generation + reservations, publication/accounting markers, build waiters, provisional pins, + references, mounts, private reservation transfers and actual usage, copies, + tombstones, compaction, and deletion before readiness or GC. If only protected state remains, new admission fails `allagents_workspace_capacity_exceeded`; no protected state is deleted and no access, retention, source, profile, or provider route changes. @@ -858,23 +861,28 @@ caller responsible for acquisition. The temporary fork closes those seams. ready state before giving each live waiter a provisional pin. Build failure removes unpublished staging and releases access/build-specific reservations; generic session/tombstone reservations remain with their failed responses. -6. For each pinned waiter independently, the runner rejects an editable - attachment whose full physical generation bytes/inodes exceed its hard - allowance, releasing only that waiter's pin and access-specific reservations; - other waiters continue against the valid ready epoch. Otherwise the gateway - CASes that session `resolving -> attaching`, and the runner prepares either a - durable read-only epoch reference plus verified mount or a unique private copy - plus checkpoints and collection baselines. It returns an opaque token/evidence. - The gateway alone CASes `attaching -> ready`, stores key/epoch evidence, and - acknowledges the token. Under the generation lock, the runner releases that - provisional pin exactly once and advances `lastUsedAt` to at least the - ready-commit timestamp. Prepare/ack recovery preserves the committed - attachment or rolls that waiter's resources/reservations back once. +6. For each pinned waiter independently, the runner first validates its logical + cwd against the independently verified manifest. A missing or non-directory + `workspacePath` releases only that waiter's pin and access-specific + reservations and persists its terminal failed response; no attachment is + prepared. The runner then rejects an editable attachment whose full physical + generation bytes/inodes exceed its hard allowance, again releasing only that + waiter's pin and access-specific reservations. Other waiters continue against + the valid ready epoch. Otherwise the gateway CASes that session + `resolving -> attaching`, and the runner prepares either a durable read-only + epoch reference plus verified mount or a unique private copy plus checkpoints + and collection baselines. It returns an opaque token/evidence. The gateway + alone CASes `attaching -> ready`, stores key/epoch evidence, and acknowledges + the token. Under the generation lock, the runner releases that provisional pin + exactly once and advances `lastUsedAt` to at least the ready-commit timestamp. + Prepare/ack recovery preserves the committed attachment or rolls that waiter's + resources/reservations back once. 7. Only after attachment `ready` do response events include the complete workspace metadata; active streaming uses `expiresAt: null`. Only editable - sessions accept ordinary UHP input-file overlays. HarnessRouter resolves the - safe logical cwd, creates the active-turn-only native credential projection or - scoped proxy credential, and dispatches the harness. Provider retry/fallback + sessions accept ordinary UHP input-file overlays. HarnessRouter uses the + already validated logical cwd, creates the active-turn-only native credential + projection or scoped proxy credential, and dispatches the harness. Provider + retry/fallback cannot validate, resolve, build, attach, or change any binding. 8. Read-only collection reports no workspace mutation; editable collection walks the private root and declared repositories without reporting initial source @@ -967,9 +975,9 @@ caller responsible for acquisition. The temporary fork closes those seams. profile. 5. Restart reconciles all lifecycle state before GC or readiness. It completes or rolls back interrupted provisional turn admission against any runner profile - token, then reconciles generic admission, staging/prospective-generation - reservation, publication/accounting conversion, provisional-pin/reference - acquisition, attachment prepare/ack and private-reservation transfer, + token, then reconciles generic admission, active leases, staging/prospective- + generation reservation, publication/accounting conversion, provisional-pin/ + reference acquisition, attachment prepare/ack and private-reservation transfer, mount/copy creation, private usage accounting, tombstoning, unmount, release, compaction, and physical deletion without duplicating a debit/reference, leaking a pin, extending an original deadline, or exposing a partially deleted @@ -1015,8 +1023,8 @@ caller responsible for acquisition. The temporary fork closes those seams. absent from the agent environment and filesystem, no provider-route API key exists in that mode, and the projection is absent from retained homes, checkpoints, backups, and mounts after every terminal or recovered outcome. -- **AE3.** Repository mode resolves Promptfoo-supplied HTTPS URLs and revisions - to exact commits and publishes one verified immutable generation. Two +- **AE3.** Repository mode resolves Promptfoo-supplied HTTPS URLs and refs to + exact commits and publishes one verified immutable generation. Two simultaneous `readOnly` sessions using different harness/profile bindings and the same normalized request share one generation build, see identical bytes and nested Git history, start in their own validated logical cwd, and cannot @@ -1027,11 +1035,16 @@ caller responsible for acquisition. The temporary fork closes those seams. extension on continuation to HTTP 409 `allagents_workspace_immutable`, and a retained expired/tombstoned continuation to HTTP 410 `allagents_workspace_expired`. Source-free validation rejects unknown access/ - retention, malformed names/revisions/destinations, userinfo or secrets in URLs, - non-HTTPS transports, IP literals, and duplicate/overlapping destinations. - Acquisition rejects loopback/link-local/private/reserved/metadata destinations, - DNS rebinding, unsafe redirects, and out-of-scope credential forwarding before - source bytes reach staging. Preflight receives no request URL or secret value; + retention, malformed refs/destinations/working-directory shapes, escaping + workspace paths, userinfo or secrets in URLs, non-HTTPS transports, IP + literals, and duplicate/overlapping destinations. An unknown or unadvertised + ref fails during bounded resolution without source-byte acquisition. After a + cache hit or verified build, the runner rejects a `workspacePath` that is + missing or not a directory before attachment or agent launch. Acquisition + rejects loopback/link-local/private/reserved/metadata + destinations, DNS rebinding, unsafe redirects, and out-of-scope credential + forwarding before source bytes reach staging. Preflight receives no request URL + or secret value; the runner alone verifies selected credential handles, storage relationships, persistence authorization, and session/persistence/build reservations. Exact source byte admission may fail only after bounded staging reveals size, but @@ -1234,17 +1247,15 @@ Initial UHP request fragment: "kind": "repositories", "repositories": [ { - "name": "api", "url": "https://github.com/acme/api.git", - "revision": "refs/pull/123/head", + "ref": "refs/pull/123/head", "destination": "api" } ] }, "workingDirectory": { - "kind": "repository", - "repository": "api", - "path": "packages/service" + "kind": "workspacePath", + "path": "api/packages/service" } } } @@ -1273,19 +1284,17 @@ Successful terminal response metadata fragment: "retention": "session", "expiresAt": "2026-09-24T12:00:00Z", "workingDirectory": { - "kind": "repository", - "repository": "api", - "path": "packages/service" + "kind": "workspacePath", + "path": "api/packages/service" }, "sourceIdentity": { "kind": "repositories", "complete": true, "repositories": [ { - "name": "api", "url": "https://github.com/acme/api.git", "destination": "api", - "requestedRevision": "refs/pull/123/head", + "requestedRef": "refs/pull/123/head", "resolvedCommit": "0123456789abcdef0123456789abcdef01234567" } ] @@ -1311,12 +1320,12 @@ The hook supports four operations: IDs. The runner verifies the corresponding store handles and all staging/result filesystem relationships itself; - `validate`: validate and default the opaque JSON descriptor, caller repository - URLs/names/revisions/destinations, snapshot name when applicable, and logical - cwd without source access; select the bounded credential-reference subset from - deployment credential-scope mappings; then return a private normalized- - descriptor path/digest, effective descriptor digest, effective access, - requested retention, - logical cwd, and that selected set; + URLs/refs/destinations, snapshot name when applicable, and the syntax and + lexical safety of the workspace-relative working directory without source + access; select the bounded credential-reference subset from deployment + credential-scope mappings; then return a + private normalized-descriptor path/digest, effective descriptor digest, + effective access, requested retention, logical cwd, and that selected set; - `resolve`: consume that exact normalized descriptor and selected reference set, safely resolve immutable source identity, and return a private canonical source-only resolved-plan path/digest, generation key, effective cwd, and @@ -1324,8 +1333,8 @@ The hook supports four operations: only generation-key inputs—normalized caller URLs, resolved commits or OCI digests/layers, normalized destinations, snapshot/acquisition/egress and sharing-authorization identity, and selected credential-reference identities— - and omits repository names, credential values, access, retention, cwd, - requested-ref spelling, harness/profile, and session; equal generation keys + and omits credential values, access, retention, cwd, requested-ref spelling, + harness/profile, and session; equal generation keys therefore require identical plan bytes; and - `materialize`: consume those exact resolved-plan bytes and selected reference set at the supplied private path, verify their supplied digest, write source @@ -1333,6 +1342,10 @@ The hook supports four operations: to the private result root, and return without publishing or re-resolving source. +After a ready-generation cache hit or a successful materialization, the runner +checks the logical cwd against the independently verified manifest before it +creates an attachment or launches an agent. + Preflight runs once per deployment and receives no credential values. Validate and resolve run once for a new session. Materialize runs only for a runner-owned cache-miss generation claim; concurrent waiters consume the same ready @@ -1384,9 +1397,8 @@ unique and items are sorted by the UTF-8 bytes of the NFC-normalized destination Every destination must exactly equal the `path` of a directory entry in the same manifest. Duplicate destinations, missing destination entries, and destinations naming files or symbolic links are invalid even when the manifest digest is -correct. Logical repository names remain per-request descriptor/provenance data -and do not enter the generation-scoped manifest or semantic Git-state record; -those records identify repository roots by destination. +correct. Repository roots are identified by destination in the generation-scoped +manifest and semantic Git-state record. `entries` is an array with at most 500,000 items. Every item has `additionalProperties: false` and is exactly one of: @@ -1495,8 +1507,9 @@ authorization, read-only conflict, access-specific capacity reservations, source resolution, build/staging/prospective-generation capacity, acquisition I/O, post-build full-tree accounting, numeric limits, source semantic validation, hook envelope, manifest, cryptographic/tree/Git/generation integrity, -publication, per-waiter private fit, attachment, editable checkpoint, private -runtime quota, then durable state failure. A completed-parent/live-descendant +publication, per-waiter manifest-cwd validation, private fit, attachment, +editable checkpoint, private runtime quota, then durable state failure. A +completed-parent/live-descendant violation is failed containment; other containment failure supersedes hook state but never overwrites UHP-mandated `cancelled` or `incomplete`. Declared budget exhaustion produces @@ -1766,38 +1779,41 @@ into successful empty output and performs no automatic retry. descriptor; keep only operator-owned snapshot catalog entries in `workspace.yaml`. Generate the manifest schema and add descriptor/preflight/ validate/resolve/materialize/result schemas, defaults, canonicalization, - generation-key construction, origin-policy credential selection, and public- - egress enforcement. Preflight returns configured reference identities without - request URLs or values; validate checks URLs/names/revisions/destinations and - selects a bounded mapped subset without source access; the runner verifies - their handles and injects only that selected set into source-access children. - Resolve exactly the caller-declared repositories to commits without writing - source bytes, and materialize only the exact cache-miss resolved plan into - staging. Preserve nested `.git` while excluding volatile administrative bytes - from the source-visible manifest, enforce closed semantic Git validation, and - prove the manifest equals the union of resolved commit trees at pairwise non- - overlapping destinations plus necessary ancestor directories. Validate - destinations/cwd, compute the manifest, and return without publishing. + generation-key construction, credential-scope selection, and public-egress + enforcement. Preflight returns configured reference identities without request + URLs or values; validate checks URLs/refs/destinations and the syntax and + lexical safety of the workspace path, then selects a bounded mapped subset + without source access. The runner verifies their handles and injects only that + selected set into source-access children. Resolve exactly the caller-declared + repositories to commits without writing source bytes, and materialize only the + exact cache-miss resolved plan into staging. Preserve nested `.git` while + excluding volatile administrative bytes from the source-visible manifest, + enforce closed semantic Git validation, and prove the manifest equals the + union of resolved commit trees at pairwise non-overlapping destinations plus + necessary ancestor directories. Validate destinations, compute the manifest, + and return without publishing. The runner validates each waiter's logical cwd + against that verified manifest before attachment. - **Verification:** Local public-address HTTPS fixtures cover caller URLs, - refs/defaults/HEAD, multiple repositories, duplicate and ancestor/descendant - destinations, undeclared root/side files, revision grammar, helpers, - submodules/LFS/file/ext/ssh protocols, bounded safe redirects, cancellation, - partial cleanup, descriptor defaults, access/retention validation, anonymous - and scope-mapped credential identity, private generation-key/public - generation-ID separation, exact URL provenance, schema fixtures, commit-tree/ - manifest reconstruction, concurrent identical resolve identity, and - repository/entry/byte/deadline boundaries. Network fixtures reject userinfo, - IP literals, controls, whitespace, backslashes, noncanonical IDNA/default-port/ - trailing-dot forms, encoded separators/dot segments, loopback, link-local, + refs/defaults/HEAD, the same URL at different refs/destinations, multiple + repositories, duplicate and ancestor/descendant destinations, undeclared root/ + side files, ref grammar, helpers, submodules/LFS/file/ext/ssh protocols, bounded + safe redirects, cancellation, partial cleanup, descriptor defaults, access/ + retention validation, `workspaceRoot` and valid/invalid `workspacePath` for Git + and snapshots, anonymous and scope-mapped credential identity, private + generation-key/public generation-ID separation, exact URL provenance, schema + fixtures, commit-tree/manifest reconstruction, concurrent identical resolve + identity, and repository/entry/byte/deadline boundaries. Network fixtures + reject userinfo, IP literals, controls, whitespace, backslashes, noncanonical + IDNA, explicit-default-port, and trailing-dot forms, encoded separators/dot + segments, loopback, link-local, private, reserved, metadata, mixed public/private DNS answers, DNS rebinding, unsafe redirects, raw-prefix lexical siblings, same-origin scope escapes, redirect-selected credentials, inherited proxy bypass, and other out-of-scope - credential forwarding. Repository requests that differ only in logical names - share one private generation key and manifest but retain their own names in - descriptor/provenance and cwd resolution. Different cwd/access/retention/ - harness/profile/session inputs also preserve the key when normalized URLs, - resolved source, destinations, sharing scope, egress-policy version, and - selected credential-reference identities match. + credential forwarding. Requests whose different ref spellings resolve to the + same URL, commit, destination, sharing scope, egress-policy version, and + selected credential-reference identities share one private generation key. + Different working directories, access, retention, harness/profile, and session + inputs also preserve that key. ### U3. Session binding, failures, and credential containment @@ -1967,10 +1983,10 @@ into successful empty output and performs no automatic retry. | Editable isolation | Every fitting editable trial receives an inode-independent private tree and reserved hard byte/inode allowance covering overlays/checkpoints/produced state. A non-fitting waiter fails alone; continuation preserves a fitting trial's mutations but cannot grow past its envelope; siblings and the generation remain unchanged. | | Materializer containment | Fork/double-fork/cancellation/deadline fixtures prove `populated 0` before result read, publication, secret release, or cleanup. `containment_pending` blocks terminal visibility/readiness through restart and resolves once after quiescence. | | Capacity envelope | Native profiles retain one active turn and zero waiters. Source build limits and finite staging/generation/private-byte/private-inode/session/persistence/tombstone quotas reject overflow. Invalid descriptors cannot bypass generic admission; one editable session creates one private debit; successful publication releases staging capacity. References and provisional pins prevent eviction; all-protected capacity returns the cataloged retryable failure. | -| Durable lifecycle | Fault injection covers generic and provisional turn admission, generation epochs, build/staging/generation reservations, publication/accounting conversion, build waiters, provisional pins, attachment prepare/ready-ack and private-reservation transfer, references, mounts, private usage, expiry, tombstones/purge, unmount, deletion, quarantine, and GC. Startup reconciles before readiness; no deadline extends, no debit duplicates/leaks, no second epoch appears before prior eviction completes, and no session silently rematerializes. | +| Durable lifecycle | Fault injection covers generic and provisional turn admission, active leases, generation epochs, build/staging/generation reservations, publication/accounting conversion, build waiters, provisional pins, attachment prepare/ready-ack and private-reservation transfer, references, mounts, private usage, expiry, tombstones/purge, unmount, deletion, quarantine, and GC. Startup reconciles before readiness; no deadline extends, no debit duplicates/leaks, no second epoch appears before prior eviction completes, and no session silently rematerializes. | | Retention and disposal | Fake-clock evidence proves one session CAS rejects busy/expired continuation admission, provisionally saves/clears a valid deadline, and either commits active after profile admission or restores the exact future deadline/tombstones an elapsed one after pre-allocation profile failure. Terminal acknowledgement alone sets the next `expiresAt`; polls/replays do not renew. Invalid failed responses stay accounted through purge; retained expiry returns HTTP 410; purge returns stock unknown; persistence authorizes before source access; operator deletion is idempotent. Null `lastUsedAt` epochs evict first by `publishedAt`; used epochs order by `lastUsedAt`, then `publishedAt`, generation key, and epoch. | | Session continuity | Both modes preserve conversation and fixed generation key/epoch/access/retention/cwd/harness/auth binding while persistent or unexpired; editable preserves private files; read-only remains immutable. Corrupt known evidence returns HTTP 409 non-resumable with no source access or later-epoch substitution. | -| Git acquisition | Caller-supplied canonical HTTPS URLs, public-address egress enforcement, DNS-rebinding and redirect defense, structured-scope credential isolation, constrained revisions, exact commits, closed transport/config, exact object closure/index semantics, generation reuse, and partial cleanup pass against local network fixtures. | +| Git acquisition | Caller-supplied canonical HTTPS URLs, public-address egress enforcement, DNS-rebinding and redirect defense, structured-scope credential isolation, constrained refs, exact commits, closed transport/config, exact object closure/index semantics, generation reuse, and partial cleanup pass against local network fixtures. | | OCI acquisition | Digest/media/path/link/type/limit, `.git` rejection, generation reuse, and attachment matrix pass against a local registry. | | Credential boundary | Preflight sees no secret values and returns bounded configured reference identities; validate selects a bounded subset; the runner verifies handles and injects only that selected set into source-access children. Source secrets and caller keys are absent from staging, generations, private trees, base environments, checkpoints, backups, logs, and output. The selected OAuth profile is visible only through its active-turn projection, which is absent before acknowledgement and after restart reconciliation. | | Provider boundary | Codex/Pi native OAuth, refresh repair, projection teardown, idempotency/session/profile admission, different-profile concurrency, same-profile fail-fast exclusion, and explicit proxy scope all pass without implicit switching. | @@ -2027,8 +2043,8 @@ into successful empty output and performs no automatic retry. then non-null `lastUsedAt`, `publishedAt`, generation key, and epoch ID. Failures stay quarantined/accounted, block same-key republication, and protected-capacity exhaustion rejects admission. -- Restart reconciles generic admission, build waiters, staging/prospective- - generation reservations, publication/accounting, provisional pins, +- Restart reconciles generic admission, active leases, build waiters, staging/ + prospective-generation reservations, publication/accounting, provisional pins, prepare/ready-ack and private-reservation transfer, references, mounts, private usage/copies, `containment_pending`, credential projections, tombstones/purge, deletion, and profile fences before readiness or GC. Existing sessions never From a9d29fe41b017ab08210768ce5d270895f5dad44 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Thu, 24 Sep 2026 09:41:43 +1000 Subject: [PATCH 23/44] docs(architecture): align workspace contracts --- .../0002-adopt-uhp-through-harnessrouter.md | 51 +++-- ...0837-feat-coding-execution-gateway-plan.md | 200 ++++++++++++------ .../research/workspace-contract-incumbents.md | 160 ++++++++++++++ 3 files changed, 328 insertions(+), 83 deletions(-) create mode 100644 docs/research/workspace-contract-incumbents.md diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index f23a9f08..a5d55ff9 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -2,7 +2,7 @@ - Status: Accepted; implementation gated on native-auth feasibility - Date: 2026-09-21 -- Updated: 2026-09-23 +- Updated: 2026-09-24 ## Decision @@ -17,7 +17,7 @@ authenticated provider proxy remains an explicitly configured last resort. Native-auth failure must never activate the proxy automatically. Version one still implements and verifies proxy mode even when a deployment does not use it. -Project `workspace.yaml` will remain ordinary local AllAgents configuration plus an optional operator-owned OCI snapshot catalog. It will not control which Git repositories a UHP caller may request. +Project `workspace.yaml` will remain ordinary local AllAgents configuration plus an optional operator-owned OCI snapshot catalog. Its local repository entries use `path` for the checkout location and one canonical `url` for remote identity; the provider-specific `source` plus `repo` pair is removed as a clean schema cutover. It will not control which Git repositories a UHP caller may request. The fork is delivery machinery, not a second protocol. We will keep the changes narrow and suitable for upstreaming, but delivery will not depend on upstream acceptance. @@ -159,7 +159,7 @@ Each repository entry has this shape: | `ref` | no | Full ref name, unambiguous branch or tag shorthand, or full 40-hex commit ID; omission means remote symbolic HEAD | | `destination` | yes | Unique, non-root relative directory; destinations must not overlap or collide with a runner-owned control namespace | -A workspace snapshot source instead names one configured snapshot plus immutable image and workspace-manifest digests. +A workspace snapshot source instead has the exact shape `{ "kind": "workspaceSnapshot", "snapshotName": ConfigName, "imageManifestDigest": Digest, "workspaceManifestDigest": Digest }`. `snapshotName` selects an operator-owned catalog entry; `imageManifestDigest` identifies the accepted direct OCI image manifest; `workspaceManifestDigest` identifies the canonical source-visible manifest. Reserved control namespaces include HarnessRouter's root checkpoint repository. Source-free validation rejects a destination that equals, contains, or is @@ -197,6 +197,26 @@ The descriptor cannot supply credentials, host paths, commands, environment vari A continuation supplies `previous_response_id` and must omit the extension. It reuses the original descriptor, attachment, access, retention, working directory, harness, and authentication binding. +### Contract vocabulary and benchmark compatibility + +The JSON descriptor uses one canonical vocabulary rather than aliases for benchmark-specific names. `url`, optional `ref`, and `destination` describe requested Git materialization; `workingDirectory` describes the logical workspace-relative command directory. Public provenance preserves `requestedRef` separately from `resolvedCommit`. The contract does not also accept Harbor `git_url` or `workdir`, SWE-bench `repo` or `base_commit`, or Devfile `revision` or `clonePath`. + +The local `workspace.yaml` contract represents a different boundary: + +```yaml +repositories: + - path: ../api + url: https://github.com/acme/api.git + managed: sync + branch: main +``` + +`path` remains the existing or managed local checkout location. `url` replaces the lossy `source` plus `repo` pair. `branch` remains branch-specific because managed synchronization performs branch checkout and pull; it does not claim arbitrary detached-ref semantics. Path-only unmanaged entries may omit `url`; a managed entry requires it. The schema, CLI, generated schemas, examples, and tests cut over together without accepting both shapes indefinitely. + +Harbor and SWE-bench/Hugging Face are benchmark-ingestion precedents, not alternate workspace field vocabularies. Harbor clones a task repository and materializes its Dockerfile, Compose definition, or prebuilt `environment.docker_image`; SWE-bench records `repo` and `base_commit` and builds or pulls layered instance images. A future adapter may compile those records into the canonical AllAgents workspace and deployment inputs while preserving their upstream identity. + +A runnable Harbor or SWE-bench image is not automatically an AllAgents `workspaceSnapshot`. The former may combine source, tools, services, verifier assumptions, and runtime configuration; the latter is a source-only OCI artifact with a separately verified workspace manifest. Caller-selected task packages, runtime images, and verifiers require a separate versioned task/environment boundary rather than overloading `source`. + ### Access and retention Access and retention are independent: @@ -243,8 +263,8 @@ Public metadata never exposes the private generation key, raw request digest, UR | Source | Authority | |---|---| -| Caller-requested Git | The UHP JSON descriptor supplies URL, optional ref, and destination | -| OCI snapshot | Operator-owned `workspace.yaml` snapshot catalog plus caller-supplied immutable digests | +| Caller-requested Git | The UHP JSON descriptor supplies `url`, optional `ref`, and `destination` | +| OCI snapshot | Operator-owned `workspace.yaml` snapshot catalog plus caller-supplied `snapshotName`, `imageManifestDigest`, and `workspaceManifestDigest` | | Harness, model, persistence, quota, and egress policy | HarnessRouter deployment configuration | | Source credentials | Operator-owned secret store and credential-scope mappings | @@ -323,11 +343,11 @@ Any undeclared path fails integrity validation. ### OCI snapshots -Snapshot mode accepts only a direct OCI image manifest from the configured -repository, selected by immutable image-manifest and workspace-manifest digests. -Redirects may not change registry authority. The config descriptor must use the -snapshot's configured workspace-manifest media type and address the canonical -workspace-manifest bytes. +Snapshot mode accepts only the direct OCI image manifest selected by +`imageManifestDigest` from the repository owned by the `snapshotName` catalog +entry. Redirects may not change registry authority. The config descriptor must +use that entry's configured workspace-manifest media type and address the +canonical bytes selected by `workspaceManifestDigest`. | Limit | Maximum | |---|---:| @@ -347,10 +367,11 @@ declared size. It rejects devices, sockets, traversal, escaping links, sparse files, unknown or foreign layers, mutable tags, and undeclared output. The runner independently rejects a 129th repository root. -The materializer verifies the image manifest, workspace manifest, and every layer -size and digest before use, then applies OCI whiteouts. It recomputes the canonical -manifest from staging and requires it to match both the fetched manifest bytes and -the caller-provided digest. Snapshot mode rejects `.git` administrative subtrees. +The materializer verifies `imageManifestDigest`, `workspaceManifestDigest`, and +every layer size and digest before use, then applies OCI whiteouts. It recomputes +the canonical manifest from staging and requires it to match both the fetched +manifest bytes and the caller-provided `workspaceManifestDigest`. Snapshot mode +rejects `.git` administrative subtrees. Snapshots that require Git history use repository mode. ### Canonical workspace manifest @@ -702,7 +723,7 @@ The maintained fork must be rebased and tested against selected upstream release ## Deliberate limits -Version one does not add evaluation datasets, scoring, assertions, automatic retries, session branching, concurrent turns within one session, simultaneous refresh-capable turns for one native profile, caller-supplied credentials, non-HTTPS or private-network Git origins, public multi-tenancy, arbitrary materializer commands, mutable OCI tags, transparent source-mode fallback, or guaranteed provider prompt-cache hits. +Version one does not add evaluation datasets, Harbor task ingestion, SWE-bench/Hugging Face ingestion, caller-selected runtime images or verifiers, scoring, assertions, automatic retries, session branching, concurrent turns within one session, simultaneous refresh-capable turns for one native profile, caller-supplied credentials, non-HTTPS or private-network Git origins, public multi-tenancy, arbitrary materializer commands, mutable OCI tags, transparent source-mode fallback, or guaranteed provider prompt-cache hits. Read-only attachments never copy up or become editable. Editable sessions never share mutations. Callers cannot choose arbitrary TTLs, bypass persistence quotas, or change retention on continuation. Leased, referenced, or pinned state is never evicted. Default retention is always bounded. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index ea0f1fdf..78c5f272 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,7 +1,7 @@ --- title: "UHP Coding-Agent Execution through HarnessRouter - Plan" date: 2026-09-18 -updated: 2026-09-22 +updated: 2026-09-24 type: feat artifact_contract: ce-unified-plan/v1 artifact_readiness: implementation-ready @@ -189,6 +189,18 @@ caller responsible for acquisition. The temporary fork closes those seams. more than once at different refs or destinations. The service accepts any repository reachable through safe public egress; callers cannot supply credentials, non-HTTPS transports, host paths, commands, or Docker options. +- **Use one canonical workspace vocabulary.** The execution descriptor keeps + `url`, optional `ref`, `destination`, and logical `workingDirectory`; response + provenance keeps `requestedRef` separate from `resolvedCommit`. Local + `workspace.yaml` repository entries keep `path` and replace the provider- + specific `source` plus `repo` pair with one canonical `url`. Benchmark-specific + aliases are accepted only by future adapters, never by the canonical schema. +- **Keep benchmark task/environment identity separate from workspace source.** + Harbor task repositories and `environment.docker_image`, and SWE-bench/Hugging + Face `repo`, `base_commit`, and instance images, are adapter inputs. A runnable + benchmark image is not an AllAgents source-only `workspaceSnapshot`; direct + task packages, environment images, and verifiers need a separate versioned + boundary if added later. - **Prefer harness-native OAuth.** Promptfoo's HarnessRouter API key authenticates the UHP caller only. Codex and Pi use their own login, token storage, refresh, and provider request path; native mode has no provider-route API key. @@ -326,8 +338,8 @@ caller responsible for acquisition. The temporary fork closes those seams. `source` is exactly one of: - `{ kind: "repositories", repositories: NonEmptyArray<{ url: HttpsGitUrl, ref?: RefText, destination: RelativeDirectory }> }`; or - - `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, - workspaceManifestDigest: Digest }`. + - `{ kind: "workspaceSnapshot", snapshotName: ConfigName, + imageManifestDigest: Digest, workspaceManifestDigest: Digest }`. `workingDirectory` is exactly `{ kind: "workspaceRoot" }` or `{ kind: "workspacePath", path: RelativeDirectory }`. The workspace path is @@ -346,7 +358,8 @@ caller responsible for acquisition. The temporary fork closes those seams. visible bytes, declared agent-visible filesystem semantics, or sharing authorization: hook/schema versions, deployment authorization scope, normalized caller Git URLs, bounded selected credential-reference identities, resolved - commits or OCI digests, normalized destinations, snapshot identity when + commits or the exact OCI `imageManifestDigest` and + `workspaceManifestDigest`, normalized destinations, `snapshotName` when applicable, and acquisition/egress policy version. Access, retention, logical cwd, harness/profile, session identity, physical paths, credential values, and volatile Git administrative representation do not fragment that key. @@ -522,13 +535,28 @@ caller responsible for acquisition. The temporary fork closes those seams. #### Source acquisition and provenance -- **R9.** Parse the project `workspace.yaml` through its authoritative schema - only for the optional strict project-owned `workspaceSnapshots` entries: +- **R9.** Make one clean public `workspace.yaml` repository-schema cutover: + retain `repositories[].path` as the existing or managed local checkout + location; replace the provider-specific `source` plus `repo` pair with one + optional canonical credential-free HTTPS `url`; retain `branch` because + managed synchronization implements branch checkout and pull rather than + arbitrary detached refs. Path-only unmanaged entries may omit `url`; any + truthy `managed` entry requires it. `workspace repo add` records a normalized + URL, converting recognized SSH provider remotes to canonical HTTPS without + persisting userinfo. An explicit one-time migration rewrites unambiguous + legacy provider/identifier pairs and rejects unknown or credential-bearing + forms with repair guidance. The normal parser and generated v2 schema accept + only the new shape; there is no dual-field compatibility path. + + The execution materializer parses the project `workspace.yaml` through that + authoritative schema only for optional strict project-owned + `workspaceSnapshots` entries: `{ name: ConfigName, repository: OciRepository, workspaceManifestMediaType: MediaType, executionCredential?: "${ENV_VAR}" }`. Reject unknown fields, literal secrets, and duplicate snapshot names; snapshot - entries do not merge with user configuration. Git repository URLs do not come - from `workspace.yaml`. + entries do not merge with user configuration. Local `repositories[].url` + entries never form an execution allowlist and are not copied into a UHP + request. Repository mode takes one through 128 request entries. Each has a canonical absolute `https` `url`, optional `ref`, and unique, pairwise non-overlapping @@ -615,12 +643,13 @@ caller responsible for acquisition. The temporary fork closes those seams. before that budget returns failed `timeout`. Every outcome proves the cgroup empty before removing staging and starts no provider. - **R11.** Snapshot mode constructs a server-side immutable OCI reference from - the selected snapshot's configured repository and caller-provided digest. - Accept only a direct OCI image manifest with at most 64 distributable - tar/gzip/zstd layers. Its config descriptor must use the entry's configured - workspace-manifest media type and address canonical workspace-manifest bytes; - redirects may not change registry authority. Verify the image manifest, - workspace manifest, layer size, and digest before use; apply OCI whiteouts; + the repository configured by `snapshotName` and the caller-provided + `imageManifestDigest`. Accept only a direct OCI image manifest with at most 64 + distributable tar/gzip/zstd layers. Its config descriptor must use the catalog + entry's configured workspace-manifest media type and address the canonical + bytes selected by `workspaceManifestDigest`; redirects may not change registry + authority. Verify both named manifests plus every layer size and digest before + use; apply OCI whiteouts; limit the image manifest to 4 MiB, the workspace-manifest blob to 128 MiB, its `repositories` array to 128 items, total compressed layers to 8 GiB, expanded bytes to 32 GiB, entries to 500,000, one regular file to 4 GiB, paths @@ -630,7 +659,7 @@ caller responsible for acquisition. The temporary fork closes those seams. sockets, traversal, escaping links, sparse files, unknown or foreign layers, mutable tags, and undeclared output. Recompute the canonical workspace manifest from staging and require it to match both the fetched manifest bytes - and caller-provided digest. Snapshot mode rejects `.git` administrative + and `workspaceManifestDigest`. Snapshot mode rejects `.git` administrative subtrees; snapshots that require Git history use repository mode. After publication the runner creates private collection baselines from the verified trees so later produced-file reporting remains truthful. @@ -1162,6 +1191,9 @@ caller responsible for acquisition. The temporary fork closes those seams. credential fallback. - Caller-provided credentials, non-HTTPS/private-network origins, commands, host paths, materializers, or Docker options. +- Direct Harbor task-package ingestion, SWE-bench/Hugging Face dataset ingestion, + caller-selected runtime images, benchmark verifiers, or compatibility aliases + inside `allagents.workspace`. - Public multi-tenancy, per-caller authorization, Kubernetes workers, session branching, concurrent turns in one session, or guaranteed prompt-cache hits. - Exact rollback of workspace mutations between successful session turns. @@ -1180,6 +1212,7 @@ caller responsible for acquisition. The temporary fork closes those seams. - [GitHub Container Registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry) - [`codex-lb` optional proxy](https://github.com/Soju06/codex-lb) - [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) +- [Workspace contract incumbent comparison](../research/workspace-contract-incumbents.md) - [Source credential broker precedents](../research/source-credential-broker-precedents.md) --- @@ -1262,6 +1295,17 @@ Initial UHP request fragment: } ``` +Workspace-snapshot source fragment: + +```json +{ + "kind": "workspaceSnapshot", + "snapshotName": "benchmark-fixture", + "imageManifestDigest": "sha256:...", + "workspaceManifestDigest": "sha256:..." +} +``` + Continuation fragment: ```json @@ -1320,22 +1364,23 @@ The hook supports four operations: IDs. The runner verifies the corresponding store handles and all staging/result filesystem relationships itself; - `validate`: validate and default the opaque JSON descriptor, caller repository - URLs/refs/destinations, snapshot name when applicable, and the syntax and - lexical safety of the workspace-relative working directory without source - access; select the bounded credential-reference subset from deployment - credential-scope mappings; then return a - private normalized-descriptor path/digest, effective descriptor digest, - effective access, requested retention, logical cwd, and that selected set; + URLs/refs/destinations, `snapshotName`, `imageManifestDigest`, and + `workspaceManifestDigest` when applicable, and the syntax and lexical safety + of the workspace-relative working directory without source access; select the + bounded credential-reference subset from deployment credential-scope mappings; + then return a private normalized-descriptor path/digest, effective descriptor + digest, effective access, requested retention, logical cwd, and that selected + set; - `resolve`: consume that exact normalized descriptor and selected reference set, safely resolve immutable source identity, and return a private canonical source-only resolved-plan path/digest, generation key, effective cwd, and bounded request provenance without writing source bytes. The plan contains - only generation-key inputs—normalized caller URLs, resolved commits or OCI - digests/layers, normalized destinations, snapshot/acquisition/egress and - sharing-authorization identity, and selected credential-reference identities— - and omits credential values, access, retention, cwd, requested-ref spelling, - harness/profile, and session; equal generation keys - therefore require identical plan bytes; and + only generation-key inputs—normalized caller URLs, resolved commits or exact + OCI image/workspace-manifest digests and layers, normalized destinations, + snapshot/acquisition/egress and sharing-authorization identity, and selected + credential-reference identities—and omits credential values, access, + retention, cwd, requested-ref spelling, harness/profile, and session; equal + generation keys therefore require identical plan bytes; and - `materialize`: consume those exact resolved-plan bytes and selected reference set at the supplied private path, verify their supplied digest, write source content only to supplied generation staging, write the canonical manifest only @@ -1766,23 +1811,28 @@ into successful empty output and performs no automatic retry. ### U2. AllAgents workspace contracts and Git materializer -- **Goal:** Implement the caller-repository JSON descriptor, optional - `workspace.yaml` snapshot catalog, canonical generation identity, deterministic - Git construction, logical cwd, and provenance. +- **Goal:** Implement the caller-repository JSON descriptor, the clean + `workspace.yaml` repository-URL migration and optional snapshot catalog, + canonical generation identity, deterministic Git construction, logical cwd, + and provenance. - **Files:** `src/models/workspace-config.ts`, `src/models/execution-workspace.ts`, `src/core/execution-workspace.ts`, - `src/core/workspace-repo.ts`, acquisition egress integration, one narrow CLI - entrypoint, generated schemas, build packaging, configuration docs, and Git E2E - fixtures. + `src/core/workspace-repo.ts`, `src/core/managed-repos.ts`, workspace CLI and + migration metadata, acquisition egress integration, generated v2 schemas, + build packaging, configuration docs, and Git E2E fixtures. - **Approach:** Reuse authoritative URL/path normalization and workspace snapshot - parsing. Keep caller Git URLs plus access/retention in the JSON execution - descriptor; keep only operator-owned snapshot catalog entries in - `workspace.yaml`. Generate the manifest schema and add descriptor/preflight/ - validate/resolve/materialize/result schemas, defaults, canonicalization, - generation-key construction, credential-scope selection, and public-egress - enforcement. Preflight returns configured reference identities without request - URLs or values; validate checks URLs/refs/destinations and the syntax and - lexical safety of the workspace path, then selects a bounded mapped subset + parsing. Cut local repository configuration from `source` plus `repo` to + `url`, preserving `path` and branch-specific managed semantics; provide the + explicit one-time migration and remove legacy fields from ordinary parsing, + output, docs, and schemas. Keep caller Git URLs plus access/retention in the + JSON execution descriptor; keep only operator-owned snapshot catalog entries + relevant to execution in `workspace.yaml`. Generate the manifest schema and + add descriptor/preflight/validate/resolve/materialize/result schemas, defaults, + canonicalization, generation-key construction, credential-scope selection, + and public-egress enforcement. Preflight returns configured reference + identities without request URLs or values; validate checks URLs, refs, + destinations, and the syntax and lexical safety of the workspace path, then + selects a bounded mapped subset without source access. The runner verifies their handles and injects only that selected set into source-access children. Resolve exactly the caller-declared repositories to commits without writing source bytes, and materialize only the @@ -1793,16 +1843,21 @@ into successful empty output and performs no automatic retry. necessary ancestor directories. Validate destinations, compute the manifest, and return without publishing. The runner validates each waiter's logical cwd against that verified manifest before attachment. -- **Verification:** Local public-address HTTPS fixtures cover caller URLs, - refs/defaults/HEAD, the same URL at different refs/destinations, multiple - repositories, duplicate and ancestor/descendant destinations, undeclared root/ - side files, ref grammar, helpers, submodules/LFS/file/ext/ssh protocols, bounded - safe redirects, cancellation, partial cleanup, descriptor defaults, access/ - retention validation, `workspaceRoot` and valid/invalid `workspacePath` for Git - and snapshots, anonymous and scope-mapped credential identity, private - generation-key/public generation-ID separation, exact URL provenance, schema - fixtures, commit-tree/manifest reconstruction, concurrent identical resolve - identity, and repository/entry/byte/deadline boundaries. Network fixtures +- **Verification:** Schema/CLI fixtures migrate every supported legacy provider + pair to a credential-free canonical URL, preserve `path`, `branch`, skills, + descriptions, and managed mode, require `url` for managed entries, retain + path-only unmanaged entries, and reject ambiguous, credential-bearing, mixed + old/new, or legacy shapes in the normal parser and v2 schema. Local public- + address HTTPS fixtures cover caller URLs, refs/defaults/HEAD, the same URL at + different refs/destinations, multiple repositories, duplicate and ancestor/ + descendant destinations, undeclared root/side files, ref grammar, helpers, + submodules/LFS/file/ext/ssh protocols, bounded safe redirects, cancellation, + partial cleanup, descriptor defaults, access/retention validation, + `workspaceRoot` and valid/invalid `workspacePath` for Git and snapshots, + anonymous and scope-mapped credential identity, private generation-key/public + generation-ID separation, exact URL provenance, schema fixtures, commit-tree/ + manifest reconstruction, concurrent identical resolve identity, and + repository/entry/byte/deadline boundaries. Network fixtures reject userinfo, IP literals, controls, whitespace, backslashes, noncanonical IDNA, explicit-default-port, and trailing-dot forms, encoded separators/dot segments, loopback, link-local, @@ -1881,20 +1936,25 @@ into successful empty output and performs no automatic retry. - **Files:** AllAgents OCI client, manifest/archive validator, workspace-manifest types, deterministic producer fixture, local registry E2E, generation fixtures, and security fixtures. -- **Approach:** Resolve only configured registries; implement bounded - Basic/Bearer auth and exact-host redirects; compute the immutable resolved plan; - on a generation miss verify manifest/config/workspace-manifest/layers while +- **Approach:** Resolve only configured registries; treat `snapshotName` as the + operator catalog selector and `imageManifestDigest` as the required direct OCI + image-manifest identity; implement bounded Basic/Bearer auth and exact-host + redirects; compute the immutable resolved plan; on a generation miss verify + the image manifest, config, `workspaceManifestDigest`, and layers while streaming; reject `.git` administrative subtrees; apply staging changesets; validate paths/types/limits/catalog; and return through the same envelope as - Git. The runner remains the sole publisher/resource preparer and the gateway - the sole session-attachment writer. -- **Verification:** Distribution fixtures cover auth, private CA, compression, - whiteouts, redirects, rebinding, indexes, foreign media, traversal, links, - devices, sparse files, cancellation, cleanup, no Git fallback, and exact error - precedence. Concurrent identical OCI requests produce one publication; - read-only sessions share it; editable sessions get private copies; access, - retention, cwd, harness/profile, and session do not fragment its generation - key. + Git. A runnable Harbor or SWE-bench instance image requires an explicit + adapter/transform and is never relabeled as a source-only snapshot. The runner + remains the sole publisher/resource preparer and the gateway the sole session- + attachment writer. +- **Verification:** Distribution fixtures cover exact request-field naming, + `snapshotName` lookup, direct `imageManifestDigest` enforcement, workspace- + manifest equality, auth, private CA, compression, whiteouts, redirects, + rebinding, indexes, foreign media, traversal, links, devices, sparse files, + cancellation, cleanup, no Git fallback, and exact error precedence. + Concurrent identical OCI requests produce one publication; read-only sessions + share it; editable sessions get private copies; access, retention, cwd, + harness/profile, and session do not fragment its generation key. ### U5. Harness-native OAuth, optional proxy, and Promptfoo E2E @@ -1996,11 +2056,15 @@ into successful empty output and performs no automatic retry. ## Definition of Done -- ADR 0002, this plan, implementation, topology, and request examples agree on - caller-supplied HTTPS Git repositories in the UHP JSON descriptor, the optional - project `workspace.yaml` OCI snapshot catalog, immutable generations, read-only - and editable attachments, bounded retention, native OAuth, explicit proxy mode, - and GHCR digest-pinned distribution. +- ADR 0002, this plan, implementation, generated schemas, configuration docs, + topology, and request examples agree on canonical `url` vocabulary; local + `workspace.yaml` uses `path` plus optional `url` with no ordinary + `source`/`repo` compatibility fields; the UHP descriptor uses `url`, optional + `ref`, and `destination`; snapshot requests use `snapshotName`, + `imageManifestDigest`, and `workspaceManifestDigest`; runtime environment and + benchmark task identity remain separate; and immutable generations, read-only + and editable attachments, bounded retention, native OAuth, explicit proxy + mode, and GHCR digest-pinned distribution remain consistent. - The U0 evidence predates U1-U6 and both native targets pass on the recorded inputs; changed inputs have replacement evidence before dependent work resumes. - No second execution protocol/control plane, separate AllAgents gateway, direct diff --git a/docs/research/workspace-contract-incumbents.md b/docs/research/workspace-contract-incumbents.md new file mode 100644 index 00000000..7ec751b6 --- /dev/null +++ b/docs/research/workspace-contract-incumbents.md @@ -0,0 +1,160 @@ +# Workspace contract incumbents + +## Decision + +AllAgents should **not** replace its execution-gateway workspace descriptor with Harbor, Devfile, Dev Containers, E2B, Daytona, GitHub Codespaces, or Gitpod. No examined contract standardizes the same boundary: caller-selected Git or OCI source, deterministic materialization, attachment before the harness starts, a logical working directory, and immutable resolved provenance returned only after attachment is committed. + +The closest portable **source-layout precedent** is Devfile 2.3's `projects` model. The closest field-level operational API is Daytona's Git clone operation. GitHub Codespaces and Gitpod Classic are stronger examples of products that bind source acquisition to workspace lifecycle, but both are provider-specific. None is a compatible normative replacement. + +The recommended contract stack is therefore: + +1. **Northbound execution protocol:** UHP remains the sole request/response protocol. +2. **Workspace/source descriptor:** retain `metadata["allagents.workspace"]` version 1 as an AllAgents-owned extension. +3. **Runtime sandbox:** keep provider APIs behind the gateway; Harbor ASP is a promising future runtime seam, not a workspace descriptor. +4. **Immutable artifacts:** use Git commit identity and OCI Image Specification descriptors/manifests as the normative identities, while retaining the AllAgents workspace manifest and attachment result as the binding provenance record. + +Devfile should be cited as design precedent for repository URL/revision/destination concepts, not claimed as an implemented profile or conformance target. + +## The four contracts are different + +| Layer | AllAgents boundary | Best established precedent | Assessment | +| --- | --- | --- | --- | +| Northbound execution | UHP requests, events, results, cancellation, and continuation | UHP | Already selected. A workspace standard should not displace the execution protocol. | +| Workspace/source descriptor | `{url, ref?, destination}` or an OCI snapshot, plus `workingDirectory` | Devfile `projects` is the closest portable schema; Codespaces/Gitpod are product precedents | No incumbent covers AllAgents' complete semantics. Keep the extension. | +| Runtime sandbox | Process, filesystem, network, and lifecycle implementation behind the gateway | Harbor ASP, E2B, Daytona, Dev Containers | These contracts start at or after sandbox provisioning. They can inform or implement the southbound seam without becoming the northbound source contract. | +| Immutable artifacts | Resolved Git commit or OCI manifest digest, canonical workspace manifest, committed attachment metadata | Git object identity and OCI Image Specification 1.1.1 | Adopt the artifact standards directly. No workspace incumbent supplies the complete result record. | + +This separation matters. Choosing one product contract across all four layers would either expose provider operations northbound or weaken the source and provenance guarantees already specified in [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) and the [execution-gateway implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). + +## Current AllAgents contract to preserve + +The current design has a small request surface and a comparatively strong result contract: + +- A repository source has a canonical public HTTPS `url`, optional `ref`, and required unique non-root relative `destination`. A request may contain multiple repositories. +- Omitted `ref` means the remote symbolic HEAD; a supplied ref is resolved fail-closed to a full commit. The result preserves both requested and resolved identities. +- An OCI workspace source is selected by immutable image/workspace-manifest digests from an operator-owned snapshot catalog. +- `workingDirectory` is a logical union: `workspaceRoot`, or `workspacePath` with a relative path that must be a directory in the verified workspace manifest. It is not a host path or provider mount path. +- Source credentials and policy are server-owned and limited to acquisition. They are not request fields or agent environment variables. +- Materialization and attachment complete before initial UHP files or the provider process are admitted. Public provenance is emitted only after the gateway has committed a `ready` attachment and the runner has acknowledged or reserved it. +- Continuations reuse the exact descriptor, attachment, access mode, retention, working directory, harness, and authorization context rather than accepting a new source request. + +The comparison below treats those properties as requirements rather than matching field names alone. + +## Incumbent comparison + +### Harbor: benchmark materialization incumbent, not a direct invocation schema + +Harbor absolutely materializes runnable workspaces. Its `--repo` input clones a benchmark repository from GitHub, GitLab, or Hugging Face, optionally pinned to a branch, tag, or commit. Each selected task then supplies an instruction, verifier, and environment. The environment can be built from a Dockerfile or Compose file or pulled as a prebuilt image through `environment.docker_image`; `environment.workdir` selects the command working directory. A `BaseEnvironment` provider starts that filesystem and exposes execution and file-transfer operations to the agent and verifier. + +The important distinction is between two repositories that coding benchmarks often collapse: + +1. the **benchmark/task repository**, selected by Harbor `--repo`, which contains `task.toml`, instructions, environment definitions, and tests; and +2. the **target application repository**, which is normally baked into the task image or acquired by task-authored environment setup. + +Harbor has a strong, reusable contract for the first item and for the resulting runnable environment. Its Git dataset identifier does not independently describe an arbitrary set of target repositories, their checkout destinations, source credentials, or the resolved provenance returned to a caller. The prebuilt image field likewise selects the whole task environment rather than identifying a source-only OCI workspace with a separately verified source manifest. + +AllAgents should therefore follow Harbor's **architecture** for benchmark interoperability: immutable task packages, prebuilt OCI environments, explicit workdir/resources/network policy, isolated trials, and verifier separation. A Harbor adapter can compile a selected task and environment into the AllAgents execution inputs. The Harbor task schema should not replace the smaller direct-execution descriptor used when a caller supplies target repositories or a workspace snapshot without a benchmark package. + +Harbor's newer [Agent Sandbox Protocol (ASP)](https://docs.harborframework.com/core-concepts/sandboxes/asp) is a separate layer. Its draft v0 `.asp.json` describes an already provisioned sandbox reachable over SSH and supplies an absolute sandbox workspace path. It standardizes harness-to-sandbox execute/read/write behavior, explicitly leaving provisioning to the orchestrator. ASP may become a useful southbound runtime adapter, but it does not specify how Git or OCI content becomes that workspace. + +Primary evidence: [Git datasets](https://docs.harborframework.com/core-concepts/datasets/git-repos), [task packages](https://docs.harborframework.com/core-concepts/tasks/overview), [environment materialization](https://docs.harborframework.com/core-concepts/tasks/environment), [task configuration](https://docs.harborframework.com/core-concepts/tasks/configuration), pinned [task config](https://github.com/harbor-framework/harbor/blob/cdb76bae6dc88d5bca1c8f0754bbba300d6574b4/src/harbor/models/task/config.py), [job config](https://github.com/harbor-framework/harbor/blob/cdb76bae6dc88d5bca1c8f0754bbba300d6574b4/src/harbor/models/job/config.py), and [Git acquisition](https://github.com/harbor-framework/harbor/blob/cdb76bae6dc88d5bca1c8f0754bbba300d6574b4/src/harbor/registry/client/git_repo.py). + +### Hugging Face and SWE-bench: registry plus specialized materializer + +Hugging Face also participates in real workspace materialization, but the responsibility is split. The Hub stores every dataset as a Git repository. A SWE-bench dataset row then identifies the target GitHub repository with `repo`, pins its state with `base_commit`, and can provide `environment_setup_commit`, patches, tests, and issue text. The SWE-bench harness converts that benchmark record into layered Docker artifacts—base, repository environment, and per-instance images—then starts the instance image, applies the model patch, runs tests, and grades the result. + +That is a concrete and widely used Git-to-OCI workspace pipeline. Hugging Face itself supplies registry and dataset-repository contracts; SWE-bench supplies the coding-task schema and execution harness. Neither exposes one general multi-repository invocation schema. SWE-bench is intentionally specialized to one repository/base commit and its test-transition metadata. + +For compatibility, an AllAgents benchmark adapter should map a SWE-bench `repo` plus `base_commit` to repository mode. A prepared instance image must instead pass through a future task/environment boundary, or an importer must extract its checkout and produce a source-only `workspaceSnapshot` with a separately verified workspace manifest. The adapter must never register the runnable image itself as a workspace snapshot. This is an ingestion mapping, not a reason to replace the direct descriptor: AllAgents still needs multiple destinations, Git-or-OCI selection, strict credential and egress policy, continuation binding, and resolved attachment provenance. + +Primary evidence: the [SWE-bench dataset schema on Hugging Face](https://huggingface.co/datasets/princeton-nlp/SWE-bench), [Hugging Face dataset repository model](https://huggingface.co/docs/hub/datasets-overview), and [SWE-bench evaluation harness](https://www.swebench.com/SWE-bench/reference/harness). + +### Devfile 2.3: closest portable source-layout precedent + +Devfile is the strongest portable comparison. It is a CNCF Sandbox project with an open governance process and documented implementations including Eclipse Che and `odo`. Its normative schema defines `projects[]` with: + +- a required project `name`; +- `git.remotes`, mapping remote names to URLs; +- optional `checkoutFrom.remote` and `checkoutFrom.revision`; +- optional relative `clonePath`, defaulting to the project name; and +- ZIP sources as an alternative to Git. + +This is a clear semantic match for source URL, revision, and destination. Devfile also defines a projects root and projects are mapped into runtime components through `sourceMapping`. + +It is not a safe wholesale replacement: + +- The schema permits multiple named remotes where AllAgents deliberately accepts one canonical source URL per repository. +- Devfile's revision description permits the default branch when the requested revision is absent or not found; AllAgents requires a supplied ref to fail closed. +- `clonePath` is optional and defaults from the project name; AllAgents requires an explicit collision-checked destination. +- ZIP sources have no required content digest. There is no OCI workspace-source variant. +- Devfile has no standard resolved-commit result, canonical source-visible manifest, generation identity, or attachment-commit acknowledgement. +- Runtime implementations own credential behavior. The DevWorkspace Operator, for example, may expose configured Git credentials to workspace containers; that is weaker than acquisition-only credentials. +- Devfile lifecycle events and component `sourceMapping` configure a development environment. They do not define the UHP timing rule that source is attached before the harness starts and metadata appears only after attachment commit. + +AllAgents should cite and follow Devfile's vocabulary where it fits, while preserving stricter semantics: + +| AllAgents | Devfile precedent | Decision | +| --- | --- | --- | +| `url` | `projects[].git.remotes.` | Keep one canonical public HTTPS URL; do not import named-remote ambiguity. | +| `ref` | `checkoutFrom.revision` | Keep `ref`, strict resolution, and requested/resolved identity. Do not adopt fallback-on-miss behavior. | +| `destination` | `clonePath` | Treat this as the closest direct precedent, but keep it required and collision checked. | +| `workspaceRoot` | projects root / `$PROJECTS_ROOT` | Same conceptual root; keep the typed logical value rather than exposing a container path. | +| `workspacePath` | component `sourceMapping` is adjacent, not equivalent | Keep manifest-verified relative path semantics. | +| resolved Git/OCI provenance | no equivalent | Retain the AllAgents result model. | + +Primary evidence: the pinned [Devfile 2.3 JSON Schema](https://github.com/devfile/api/blob/v2.3.0/schemas/latest/devfile.json), [schema reference](https://devfile.io/docs/2.3.0/devfile-schema), [project authoring guide](https://devfile.io/docs/2.3.0/adding-projects), [governance](https://github.com/devfile/api/blob/v2.3.0/GOVERNANCE.md), [CNCF project record](https://www.cncf.io/projects/devfile/), [documented users](https://devfile.io/docs/2.3.0/users-of-devfile), and the DevWorkspace Operator's pinned [Git-credential behavior](https://github.com/devfile/devworkspace-operator/blob/9df10c1baba8e7d88948a21077e3a06fd2cca639/docs/additional-configuration.adoc). + +### Development Containers: environment standard, not source standard + +The Development Container Specification assumes a project workspace/source folder already exists. It standardizes how that folder is mounted or opened in an image-, Dockerfile-, or Compose-based development container. Relevant fields include `workspaceMount`, `workspaceFolder`, image/build/Compose selection, Features, and ordered lifecycle commands. + +`workspaceFolder` is a container/editor path, not a source descriptor. `workspaceMount` is a runtime mount expression, not a portable source identity. Lifecycle hooks run after implementations have made source available, and the specification does not define repository URL/ref/destination, Git resolution, OCI workspace snapshots, or a resolved provenance response. Feature lockfiles add integrity for Dev Container Features, not for the application workspace. + +This is the most credible portable standard for a possible future **development-environment layer**, with official support listed for VS Code, Visual Studio, IntelliJ, the reference CLI, GitHub Codespaces, CodeSandbox, DevPod, and Ona. It should not be stretched into the acquisition layer. + +Primary evidence: pinned [normative specification](https://github.com/devcontainers/spec/blob/c95ffeed1d059abfe9ffbe79762dc2fa4e7c2421/docs/specs/devcontainer-reference.md), [JSON Schema](https://github.com/devcontainers/spec/blob/c95ffeed1d059abfe9ffbe79762dc2fa4e7c2421/schemas/devContainer.base.schema.json), [field and lifecycle reference](https://github.com/devcontainers/spec/blob/c95ffeed1d059abfe9ffbe79762dc2fa4e7c2421/docs/specs/devcontainerjson-reference.md), [supporting tools](https://github.com/devcontainers/spec/blob/c95ffeed1d059abfe9ffbe79762dc2fa4e7c2421/docs/specs/supporting-tools.md), and [contribution process](https://github.com/devcontainers/spec/blob/c95ffeed1d059abfe9ffbe79762dc2fa4e7c2421/CONTRIBUTING.md). + +### E2B and Daytona: runtime providers with clone operations + +E2B creates a sandbox from a template and exposes filesystem, process, pause/resume, snapshot, and Git operations. Its sandbox-creation schema has template, timeout, network, metadata, environment, MCP, IAM, and volume fields, but no repository source. Git clone is a runtime SDK operation with URL/path/branch/depth and inline credentials. E2B warns that credentials stored in the sandbox are readable by the agent. E2B is consequently a possible runtime backend, not an agent-neutral execution or workspace contract. + +Daytona is the closest field-level operational match: its Git clone operation accepts `url`, `path`, optional branch or commit, credentials, depth, and an insecure-TLS option. But this is an imperative operation against an already-created Daytona sandbox. It does not standardize multi-source declaration, strict canonicalization, immutable result provenance, or committed attachment timing. Its per-operation credentials and optional TLS bypass also conflict with the AllAgents trust boundary. Older Daytona workspace models coupled repository metadata, devcontainer/build configuration, and provider workspace state, illustrating the portability cost of adopting a vendor workspace object. + +Primary evidence: pinned E2B [OpenAPI schema](https://github.com/e2b-dev/E2B/blob/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b/spec/openapi.yml), [sandbox SDK](https://docs.e2b.dev/sdk-reference/js-sdk/v2.51.0/sandbox), [template definition](https://docs.e2b.dev/template/defining-template), [Git integration](https://docs.e2b.dev/sandbox/git-integration), Daytona [Git operations](https://www.daytona.io/docs/en/git-operations), and pinned Daytona [workspace](https://github.com/daytonaio/daytona/blob/dfb50e8a31e9a93b31181113d7b44b657cf27168/pkg/models/workspace.go), [repository](https://github.com/daytonaio/daytona/blob/dfb50e8a31e9a93b31181113d7b44b657cf27168/pkg/apiclient/model_git_repository.go), and [workspace-creation](https://github.com/daytonaio/daytona/blob/dfb50e8a31e9a93b31181113d7b44b657cf27168/pkg/apiclient/model_create_workspace_dto.go) models. + +### GitHub Codespaces and Gitpod Classic: lifecycle precedents, not portable standards + +GitHub Codespaces creates a managed environment in the context of a GitHub repository. Its repository-scoped REST endpoint accepts `ref`, machine/location choices, `devcontainer_path`, `working_directory`, idle timeout, and retention. The repository is implied by the endpoint and the service delegates environment setup to Dev Containers. This is strong evidence for resolving source before environment startup and for treating working directory separately from source identity. It is not suitable as the AllAgents descriptor because it is GitHub-specific, single-repository, and does not expose the same resolved Git/OCI provenance or attachment transaction. + +Gitpod Classic/Ona similarly combines a context URL with workspace initialization and supports `additionalRepositories` plus checkout locations in `.gitpod.yml`. It is a useful product precedent for multiple checkouts, but its API and configuration are service-specific and mix source, image/build, and task lifecycle concerns. + +Primary evidence: GitHub's [create-codespace REST operation](https://docs.github.com/en/rest/codespaces/codespaces?apiVersion=2022-11-28#create-a-codespace-in-a-repository), [Codespaces CLI](https://cli.github.com/manual/gh_codespace_create), [Dev Container introduction](https://docs.github.com/en/codespaces/setting-up-your-project-for-codespaces/adding-a-dev-container-configuration/introduction-to-dev-containers), Gitpod Classic's [public API](https://ona.com/docs/classic/user/references/gitpod-public-api), and [`.gitpod.yml` reference](https://ona.com/docs/classic/user/references/gitpod-yml). + +## Normative artifact standards + +The source descriptor should remain AllAgents-owned, but its immutable artifact identities should not be invented locally. + +For OCI snapshots, the [OCI Image Specification 1.1.1 descriptor](https://github.com/opencontainers/image-spec/blob/v1.1.1/descriptor.md) defines content identity with media type, digest, and size, including verification against the digest. The [image manifest](https://github.com/opencontainers/image-spec/blob/v1.1.1/manifest.md) defines the config descriptor and ordered layer descriptors. Those are the correct normative identities for the snapshot artifact. The AllAgents workspace manifest remains necessary because OCI does not define the source-visible workspace tree, destination layout, logical working directory, or gateway attachment result. + +For Git, a full commit object ID is the resolved source identity. The request still needs the original ref because a branch/tag name and its resolved commit answer different audit questions. Neither Git nor OCI defines when a runner has successfully attached that content, so `effectiveDescriptorDigest`, `generationId`, `sourceIdentity`, `workingDirectory`, and `workspaceManifestDigest` must remain AllAgents result fields. + +## Recommendation and adoption rule + +Adopt the following rule for future changes: + +- **Normative:** UHP northbound; AllAgents workspace extension for acquisition and attachment; Git commit identity and OCI Image Specification 1.1.1 for immutable artifacts. +- **Benchmark compatibility:** ingest Harbor task packages and SWE-bench/Hugging Face records through adapters. Preserve Harbor's task/environment/verifier split and its preference for prebuilt OCI environments; map benchmark source identities into the canonical AllAgents descriptor. +- **Source-layout precedent:** use Devfile 2.3 `projects` semantics when adding or naming direct repository fields. Document intentional divergence, especially for URL shape, revision behavior, and destination paths. +- **Runtime precedent:** evaluate Harbor ASP as a southbound execute/filesystem adapter when its draft stabilizes. E2B and Daytona remain provider adapters. Dev Containers may define an optional environment-building layer after source acquisition. +- **Do not conflate:** Harbor's benchmark repository with the target application source; an environment image with source-only provenance; or a vendor sandbox/codespace object with the northbound contract. Do not adopt Devfile's default-on-missing revision behavior, ZIP-without-digest source, or runtime credential exposure. + +This is deliberately a layered answer rather than a claim that AllAgents has invented a universal workspace standard. The narrow extension exists because the examined standards stop either before source acquisition or before committed, immutable provenance. + +## Existing research status + +- [Harbor repository materialization](./harbor-repository-materialization.md) correctly identifies Harbor's task-repository cloning, package cache, staged publication, and prebuilt-environment model. Its statement that Harbor lacks a first-class arbitrary target-repository layer remains accurate, but should not be read as saying Harbor lacks workspace materialization. Its descriptions of AllAgents selecting configured repository names or overriding a declared repository ref are stale: current Git mode accepts caller-supplied canonical public HTTPS URLs; only OCI snapshots use the operator-owned catalog. +- [E2B execution-gateway patterns](./e2b-execution-gateway-patterns.md) remains correct that E2B is a runtime provider rather than a replacement northbound protocol. Its description of a server-authoritative logical source catalog is stale for Git sources and remains applicable only to the OCI snapshot catalog. +- The general AI research wiki has relevant Harbor and benchmark-provenance coverage but no dedicated Devfile, Dev Containers, E2B, Daytona, or Codespaces contract comparison. Its primary-source links informed source discovery; its prose is not a normative input here. +- The private AllAgents research wiki contains one execution-gateway comparison based on the older A2A-era decision baseline. That coverage is now historical because ADR 0002 selects UHP. No private synthesis or conclusion is reproduced in this public note. + +ADR 0002 and the implementation plan now codify this layered result: the canonical JSON request keeps `url`, `ref`, `destination`, and logical `workingDirectory`; local `workspace.yaml` replaces `source` plus `repo` with `url` while retaining `path`; OCI requests use explicit `snapshotName`, `imageManifestDigest`, and `workspaceManifestDigest`; and benchmark task/environment ingestion remains a separate future adapter boundary. The two older public research notes still need their stale pre-cutover descriptions corrected when they are next maintained; any private-wiki refresh remains a separate private edit. From 158542fe4484c8c71a1216cfef2813f5148d8459 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Thu, 24 Sep 2026 10:33:39 +1000 Subject: [PATCH 24/44] docs(architecture): allow Git history in OCI snapshots --- .../0002-adopt-uhp-through-harnessrouter.md | 56 +++- ...0837-feat-coding-execution-gateway-plan.md | 307 ++++++++++++------ .../research/workspace-contract-incumbents.md | 12 +- 3 files changed, 259 insertions(+), 116 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index a5d55ff9..10837616 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -215,7 +215,7 @@ repositories: Harbor and SWE-bench/Hugging Face are benchmark-ingestion precedents, not alternate workspace field vocabularies. Harbor clones a task repository and materializes its Dockerfile, Compose definition, or prebuilt `environment.docker_image`; SWE-bench records `repo` and `base_commit` and builds or pulls layered instance images. A future adapter may compile those records into the canonical AllAgents workspace and deployment inputs while preserving their upstream identity. -A runnable Harbor or SWE-bench image is not automatically an AllAgents `workspaceSnapshot`. The former may combine source, tools, services, verifier assumptions, and runtime configuration; the latter is a source-only OCI artifact with a separately verified workspace manifest. Caller-selected task packages, runtime images, and verifiers require a separate versioned task/environment boundary rather than overloading `source`. +A runnable Harbor or SWE-bench image is not automatically an AllAgents `workspaceSnapshot`. The former may combine source, tools, services, verifier assumptions, and runtime configuration. An AllAgents snapshot is a workspace source artifact: it may carry normalized offline Git history, but it does not select the runtime environment or verifier. Caller-selected task packages, runtime images, and verifiers require a separate versioned task/environment boundary rather than overloading `source`. ### Access and retention @@ -244,7 +244,7 @@ Once attachment reaches `ready`, terminal events, retrieval, background completi |---|---| | `effectiveDescriptorDigest` | Digest of the normalized descriptor and defaults | | `generationId` | Public content identifier | -| `sourceIdentity` | Normalized URL, destination, `requestedRef` when supplied, and `resolvedCommit`, or verified OCI identity | +| `sourceIdentity` | Normalized URL, destination, `requestedRef` when supplied, and `resolvedCommit`; or verified `snapshotName` and `imageManifestDigest` plus each root's destination and optional `resolvedCommit` and `objectSetDigest`, never a Git remote URL | | `workingDirectory` | Effective `workspaceRoot` or `workspacePath` | | `workspaceManifestDigest` | Verified source-visible manifest digest | | `access`, `retention`, `expiresAt` | Effective workspace policy and expiry | @@ -362,25 +362,48 @@ canonical bytes selected by `workspaceManifestDigest`. | One UTF-8 path | 4096 bytes and 128 components | | One PAX or extended header | 1 MiB | +Workspace-manifest version 2 lets each repository item describe either a +tree-only root or a history-bearing root. A history-bearing item adds `git` with +`resolvedCommit` and `objectSetDigest`. Its destination must contain exactly one +`.git` directory; a tree-only root must contain none. The snapshot's immutable +digests bind the commit and object-set identity. The artifact contains no +configured Git remote, and no Git remote URL is required or returned. Private +evaluations can still use `git log`, `git blame`, and historical diffs offline. + Before writing an entry, the materializer checks its type, path, link target, and declared size. It rejects devices, sockets, traversal, escaping links, sparse files, unknown or foreign layers, mutable tags, and undeclared output. The runner independently rejects a 129th repository root. -The materializer verifies `imageManifestDigest`, `workspaceManifestDigest`, and -every layer size and digest before use, then applies OCI whiteouts. It recomputes -the canonical manifest from staging and requires it to match both the fetched -manifest bytes and the caller-provided `workspaceManifestDigest`. Snapshot mode -rejects `.git` administrative subtrees. -Snapshots that require Git history use repository mode. +After applying OCI whiteouts, the materializer verifies +`imageManifestDigest`, `workspaceManifestDigest`, every layer size and digest, +and the recomputed source-visible manifest. For every history-bearing root it +then applies semantic Git verification: detached `HEAD` at `resolvedCommit`, an +index equal to that commit tree, an object database equal to the complete +transitive closure whose canonical digest is `objectSetDigest`, and +source-visible descendants equal to the same commit tree. Dirty, staged, +untracked, missing, or modified source fails validation. + +Snapshot Git state is offline. It must contain no remotes, branch-upstream +configuration, credential helpers, config includes, hooks, worktree links, +alternates, shallow, replace, or graft state, reflogs, `FETCH_HEAD`, extra refs, +unreachable objects, or credential-bearing configuration. Physical `.git` +entries and bytes count toward acquisition and retained-generation limits even +though their volatile representation is excluded from the source-visible +manifest. ### Canonical workspace manifest Both source modes produce the same versioned canonical manifest. Its RFC 8785 bytes enumerate every source-visible directory, regular file, and symbolic link in logical path order, including normalized mode, size, content digest, or link target. -Repository roots are identified by destination. Git mode may omit only separately verified `.git` administrative subtrees. OCI mode rejects them. No source-visible path may be omitted. - -The runner reads the manifest through a private bounded result root, verifies its digest, walks staging without following links, reconstructs the same entries, and requires byte-for-byte canonical equality. The manifest never appears inside the published source tree. +Repository roots are identified by unique, pairwise non-overlapping +destinations. A declared, separately verified `.git` subtree is omitted from +source-visible entries in either source mode; any undeclared `.git` path is +invalid. The runner verifies the manifest digest, walks staging without following +links, reconstructs the same source-visible entries, and requires byte-for-byte +canonical equality. It separately verifies every omitted Git root against its +declared commit and object-set digest. The manifest never appears inside the +published source tree. ### Generation identity @@ -391,13 +414,18 @@ The private generation key is computed before materialization. It includes every | Descriptor and hook contract versions | Access and retention | | Deployment authorization scope | Working directory | | Normalized caller Git URLs | Harness, profile, and session identity | -| Resolved commits or immutable OCI digests | Physical paths | +| Resolved commits or exact OCI image and workspace-manifest digests | Physical paths | | Normalized destinations | Credential values | | Selected credential-reference identities | Caller ref spelling after it resolves to the same commit | -| Snapshot identity when applicable | Volatile Git pack, index, and stat representation | +| Snapshot identity when applicable | Repository-mode volatile Git pack, index, and stat representation | | Acquisition and egress policy version | | -Publication binds one private key and one internal epoch to one verified workspace-manifest digest and, for repositories, one semantic Git-state record. Materialization receives the exact private resolved plan and never resolves source again. +OCI generation reuse is artifact-exact. Repacking snapshot `.git` data changes +the image digest, generation key, and public OCI identity even when the semantic +Git state is unchanged. Semantic Git verification proves what one artifact +contains; it does not deduplicate distinct OCI artifacts. + +Publication binds one private key and one internal epoch to one verified workspace-manifest digest and every declared semantic Git-state record, whether Git was acquired from a remote or carried offline in an OCI snapshot. Materialization receives the exact private resolved plan and never resolves source again. ## Generation publication and attachments diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 78c5f272..fe1aa703 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -198,9 +198,9 @@ caller responsible for acquisition. The temporary fork closes those seams. - **Keep benchmark task/environment identity separate from workspace source.** Harbor task repositories and `environment.docker_image`, and SWE-bench/Hugging Face `repo`, `base_commit`, and instance images, are adapter inputs. A runnable - benchmark image is not an AllAgents source-only `workspaceSnapshot`; direct - task packages, environment images, and verifiers need a separate versioned - boundary if added later. + benchmark image is not an AllAgents workspace source artifact. A snapshot may + contain normalized offline Git history, but direct task packages, environment + images, and verifiers need a separate versioned boundary if added later. - **Prefer harness-native OAuth.** Promptfoo's HarnessRouter API key authenticates the UHP caller only. Codex and Pi use their own login, token storage, refresh, and provider request path; native mode has no provider-route API key. @@ -361,10 +361,12 @@ caller responsible for acquisition. The temporary fork closes those seams. commits or the exact OCI `imageManifestDigest` and `workspaceManifestDigest`, normalized destinations, `snapshotName` when applicable, and acquisition/egress policy version. Access, retention, logical - cwd, harness/profile, session identity, physical paths, credential values, and - volatile Git administrative representation do not fragment that key. - Publication binds it to the independently verified workspace-manifest digest - and semantic Git record when applicable. + cwd, harness/profile, session identity, physical paths, and credential values + do not fragment that key. In repository mode, volatile Git pack, index, and + stat representation also does not fragment it. OCI reuse is artifact-exact: + repacking a snapshot changes its image digest and therefore its generation key + even when its semantic Git state is unchanged. Publication binds the key to the + independently verified workspace-manifest digest and every semantic Git record. Omitted and explicit default values have the same effective descriptor digest. The raw request descriptor digest records the exact initial JSON only in private session state; public response metadata names and returns only @@ -649,23 +651,44 @@ caller responsible for acquisition. The temporary fork closes those seams. entry's configured workspace-manifest media type and address the canonical bytes selected by `workspaceManifestDigest`; redirects may not change registry authority. Verify both named manifests plus every layer size and digest before - use; apply OCI whiteouts; - limit the image manifest to 4 MiB, the workspace-manifest blob to 128 MiB, - its `repositories` array to 128 items, total compressed layers to 8 GiB, - expanded bytes to 32 GiB, entries to 500,000, one regular file to 4 GiB, paths - to 4096 UTF-8 bytes and 128 components, and one PAX/extended header to 1 MiB. - The runner independently rejects a 129th repository root even when the archive - and fetched manifest otherwise agree. Reject devices, - sockets, traversal, escaping links, sparse files, unknown or foreign layers, - mutable tags, and undeclared output. Recompute the canonical workspace - manifest from staging and require it to match both the fetched manifest bytes - and `workspaceManifestDigest`. Snapshot mode rejects `.git` administrative - subtrees; snapshots that require Git history use repository mode. After + use; apply OCI whiteouts; limit the image manifest to 4 MiB, the workspace- + manifest blob to 128 MiB, its `repositories` array to 128 items, total + compressed layers to 8 GiB, expanded bytes to 32 GiB, entries to 500,000, one + regular file to 4 GiB, paths to 4096 UTF-8 bytes and 128 components, and one + PAX/extended header to 1 MiB. The runner independently rejects a 129th + repository root even when the archive and fetched manifest otherwise agree. + Reject devices, sockets, traversal, escaping links, sparse files, unknown or + foreign layers, mutable tags, and undeclared output. + + Each snapshot repository root is either tree-only or declares + `git: { resolvedCommit, objectSetDigest }` in the workspace manifest. Snapshot + destinations are pairwise non-overlapping. A tree-only root rejects `.git`. A + history-bearing root must contain one `.git` directory at its destination. + The producer must normalize it before publication; the materializer + independently verifies the detached `HEAD`, exact index/tree, complete + transitive object closure, and canonical object set required by repository + mode. It reads the declared commit tree and requires every source-visible + descendant of that destination to equal it, with no staged, dirty, missing, or + untracked path. It rejects rather than repairs nonconforming state and requires + the computed digest to equal `objectSetDigest`. + + Snapshot Git state is offline: reject every remote, branch-upstream setting, + credential helper, config include, hook, worktree link, alternate, shallow, + replace, graft, reflog, `FETCH_HEAD`, extra ref, unreachable object, and + credential-bearing configuration. The Git administrative state contains no + configured remote, and no Git remote URL appears in the workspace manifest or + returned provenance. Exclude only declared and + verified `.git` subtrees from source-visible entries; any other `.git` path + fails integrity validation. + Physical Git entries and bytes still count toward acquisition and + retained-generation limits. + + Recompute the canonical workspace manifest from staging and require it to + match both the fetched manifest bytes and `workspaceManifestDigest`. After publication the runner creates private collection baselines from the verified - trees so later produced-file reporting remains truthful. - - Both source modes produce the same reusable immutable-generation abstraction. - Repository `.git` state is readable but immutable in `readOnly` attachments and + trees so later produced-file reporting remains truthful. Both source modes + produce the same reusable immutable-generation abstraction. Validated `.git` + state from either mode is readable but immutable in `readOnly` attachments and independently writable only in private `editable` copies. Generation acquisition limits apply per build; retained-generation and private-workspace quotas apply independently. @@ -679,6 +702,10 @@ caller responsible for acquisition. The temporary fork closes those seams. bounded fields: extension version, `effectiveDescriptorDigest`, public `generationId`, canonical workspace-manifest digest, logical cwd, access, resolved retention, source completeness, and resolved Git/OCI provenance. + Snapshot `sourceIdentity` retains `snapshotName` and `imageManifestDigest` and + mirrors each verified root's `destination` and optional `git` declaration in + the exact shape below. History-bearing roots expose `resolvedCommit` and + `objectSetDigest`, but no Git remote URL. `generationId` is the SHA-256 digest of versioned RFC 8785 bytes containing only the returned normalized source provenance, normalized destinations, and workspace-manifest digest. It is metadata-only and is never a cache, @@ -961,17 +988,17 @@ caller responsible for acquisition. The temporary fork closes those seams. #### F4. Execute an OCI-backed first turn -1. The caller selects one configured snapshot and immutable manifest/workspace - digests; it never sends the registry origin or credential. +1. The caller selects one configured snapshot and immutable image/workspace- + manifest digests; it never sends the registry origin or credential. 2. Resolve computes the OCI generation key. A ready epoch is reused. Otherwise a - current claim is joined or, after completed eviction, a new epoch owner - fetches and verifies the direct manifest, config, workspace manifest, and - layers; applies changesets under fixed limits; and returns verified staging - and provenance. + current claim is joined or, after completed eviction, a new epoch owner fetches + and verifies the direct manifest, config, workspace manifest, and layers; + applies changesets under fixed limits; validates every declared offline Git + root; and returns verified staging, semantic Git records, and provenance. 3. The runner publishes the same immutable-generation-epoch abstraction as Git, then follows the same per-waiter read-only or editable attachment path. - Registry, digest, media, path, limit, or layout failure removes only - unpublished staging and enters neither Git nor provider fallback. + Registry, digest, media, path, limit, layout, or semantic Git failure removes + only unpublished staging and enters neither Git nor provider fallback. #### F5. Cancel, fail, or restart @@ -1111,13 +1138,18 @@ caller responsible for acquisition. The temporary fork closes those seams. live epoch publication, partial private state, or a generation without required protection. Every reservation, pin, and reference debits and releases exactly once. Provider fallback never reruns the hook. -- **AE9.** OCI mode accepts a valid digest-pinned fixture with gzip/zstd layers - and whiteouts and rejects mutable tags, indexes, mismatched digests/sizes, - traversal, escaping links, devices, sparse files, unknown media types, and - declared-limit overflow. Repeated Git and OCI requests with identical resolved - plan, sharing authorization, and selected credential-reference identities reuse - their matching epoch regardless of access, retention, cwd, harness, profile, or - session. +- **AE9.** OCI mode accepts valid digest-pinned tree-only and history-bearing + fixtures with gzip/zstd layers and whiteouts. The history fixture has no + remotes or credentials and supports offline `git log`, `git blame`, and + historical diff from its declared detached commit. OCI rejects undeclared + `.git`, overlapping repository destinations, remote or credential + configuration, unsafe Git administrative state, dirty or untracked worktree + content, commit-tree or object-set mismatch, mutable tags, indexes, mismatched + digests/sizes, traversal, escaping links, devices, sparse files, unknown media + types, and declared-limit overflow. Repeated Git and OCI requests with + identical resolved plan, sharing authorization, and selected credential- + reference identities reuse their matching epoch regardless of access, + retention, cwd, harness, profile, or session. - **AE10.** Codex and Pi own login and refresh. Missing, revoked, expired, unrefreshable, or stale-after-crash OAuth affects only that profile and never selects another profile or proxy. Same-key arrivals share one admission and @@ -1348,6 +1380,34 @@ Successful terminal response metadata fragment: } ``` +For snapshot source, `sourceIdentity` has this exact shape: + +```json +{ + "kind": "workspaceSnapshot", + "complete": true, + "snapshotName": "benchmark-fixture", + "imageManifestDigest": "sha256:...", + "repositories": [ + { + "destination": "api", + "git": { + "resolvedCommit": "0123456789abcdef0123456789abcdef01234567", + "objectSetDigest": "sha256:..." + } + }, + { + "destination": "docs" + } + ] +} +``` + +The sorted `repositories` array mirrors the verified workspace-manifest root +declarations. The top-level response field carries `workspaceManifestDigest`. +Snapshot identity retains `snapshotName` and `imageManifestDigest`; repository +subrecords contain no `url` or `requestedRef`. + ### Materializer Hook Contract HarnessRouter configuration names one metadata key, absolute executable path, @@ -1432,18 +1492,34 @@ The source tree includes one generated normative `workspace-manifest.schema.json`, imported unchanged by the Git materializer, OCI producer/materializer, runner validator, and their contract fixtures. The document is at most 128 MiB and is an object with `additionalProperties: false`, -required string `version` fixed to `"1"`, required `repositories`, and required +required string `version` fixed to `"2"`, required `repositories`, and required `entries`. `repositories` is an array with at most 128 items. Every item is an object with -`additionalProperties: false` and exactly one required string field, -`destination`, validated as a non-root `RelativeDirectory`. Destinations are -unique and items are sorted by the UTF-8 bytes of the NFC-normalized destination. -Every destination must exactly equal the `path` of a directory entry in the same -manifest. Duplicate destinations, missing destination entries, and destinations -naming files or symbolic links are invalid even when the manifest digest is -correct. Repository roots are identified by destination in the generation-scoped -manifest and semantic Git-state record. +`additionalProperties: false`, required `destination`, and optional `git`. +`destination` is a non-root `RelativeDirectory`. When present, `git` is an +object with `additionalProperties: false` and exactly two required string +fields: `resolvedCommit`, a full lowercase 40-hex commit ID, and +`objectSetDigest`, a `sha256:` digest of the canonical object-set bytes defined +below. Absence of `git` declares a tree-only root. +Destinations are unique, pairwise non-overlapping, and sorted by the UTF-8 bytes +of the NFC-normalized destination. Every destination must exactly equal the +`path` of a directory entry in the same manifest. Duplicate, ancestor/descendant, +or missing destinations, and destinations naming files or symbolic links, are +invalid even when the manifest digest is correct. Repository roots are +identified by destination in the generation-scoped manifest and any semantic +Git-state record. + +Canonical object-set bytes use ASCII and Git SHA-1 object IDs. Starting at +`resolvedCommit`, enumerate each unique reachable commit, tree, and blob, +including all commit parents and their trees. Sort records by the ASCII bytes of +the 40-character lowercase object ID. Emit exactly +` SP SP LF` for each object, where `type` is `commit`, +`tree`, or `blob`, and `size` is the unpadded base-10 byte length of the +uncompressed object content. There is one ASCII space at each `SP`, every record +ends in LF including the last, and no other bytes are present. `objectSetDigest` +is the SHA-256 of that concatenation. Pack layout, compression, offsets, and +filenames do not participate. `entries` is an array with at most 500,000 items. Every item has `additionalProperties: false` and is exactly one of: @@ -1463,43 +1539,67 @@ and NFC; implementations reject rather than normalize non-UTF-8 or non-NFC values. Paths and targets containing NUL, absolute paths, missing parents, or links escaping the workspace are invalid. The root is implicit and has no entry. Hard links are expanded to regular-file entries. Entries enumerate every -source-visible path. In repository mode only, each declared repository's -separately validated `.git` directory and descendants are omitted because -volatile pack/index layout is not source identity; no other path may be omitted. - -The digest is `sha256:` plus the lowercase SHA-256 of the RFC 8785 bytes. Git -mode computes those bytes after completing staging and performs the separate -semantic `.git` validation required by R10. OCI mode rejects `.git` -administrative subtrees, requires its configured workspace-manifest blob to -contain the same canonical bytes, and copies them to the private result root. +source-visible path. A repository item with `git` may omit only its separately +validated `.git` directory and descendants; an item without `git` may omit +nothing and must not contain `.git`. Any `.git` outside a declared history- +bearing repository root is invalid. + +The digest is `sha256:` plus the lowercase SHA-256 of the RFC 8785 bytes. +Repository mode computes those bytes after completing staging and adds `git` +metadata from its resolved commits. OCI mode requires its configured workspace- +manifest blob to contain the canonical bytes, including any declared offline Git +metadata, and copies them to the private result root. + The runner resolves only the fixed `workspace-manifest.json` relative path, validates it against the shared schema, verifies its size and digest, walks -staging without following links, reconstructs the same catalog and entries while -skipping only approved Git administrative roots, and requires byte-for-byte -canonical equality before publication. It separately revalidates every skipped -Git root against the resolved commit and safe-state rules. The private result -root is never published or exposed through UHP. +staging without following links, reconstructs the same catalog and source-visible +entries while skipping only declared Git administrative roots, and requires +byte-for-byte canonical equality before publication. It separately revalidates +every skipped Git root against its declared commit, exact object set, closed +configuration and refs, and safe-state rules. For each history-bearing root, the +manifest descendants must equal the prefixed commit tree exactly; dirty, +untracked, missing, or modified paths fail. The private result root is never +published or exposed through UHP. The immutable generation key is not the workspace-manifest digest: it is the pre-build digest of the resolved source plan used for keyed reuse. Atomic publication binds that key to exactly one verified source-visible manifest -digest and, in repository mode, one semantic Git-state record for the resolved -commits. Access, retention, cwd, harness/profile, and session identity are not -manifest fields and cannot fragment or mutate generation content. The backing -tree, including validated `.git` state, becomes owner-writable only and is -exposed to sessions solely through verified read-only mounts or independent -private editable copies. +digest and every declared semantic Git-state record. Access, retention, cwd, +harness/profile, and session identity are not manifest fields and cannot +fragment or mutate generation content. The backing tree, including validated +`.git` state, becomes owner-writable only and is exposed to sessions solely +through verified read-only mounts or independent private editable copies. -The frozen cross-repository fixture is: +The frozen history-bearing fixture is: ```json -{"entries":[{"mode":"040755","path":"services","type":"directory"},{"mode":"040755","path":"services/api","type":"directory"},{"mode":"100644","path":"services/api/README.md","sha256":"sha256:98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4","size":3,"type":"file"},{"mode":"120000","path":"services/api/current","target":"README.md","type":"symlink"}],"repositories":[{"destination":"services/api"}],"version":"1"} +{"entries":[{"mode":"040755","path":"services","type":"directory"},{"mode":"040755","path":"services/api","type":"directory"},{"mode":"100644","path":"services/api/README.md","sha256":"sha256:98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4","size":3,"type":"file"},{"mode":"120000","path":"services/api/current","target":"README.md","type":"symlink"}],"repositories":[{"destination":"services/api","git":{"objectSetDigest":"sha256:b3b49d7b3ff3fd8c4fb3acbf7de36d92f6a4029f3244064f79e37408463288b5","resolvedCommit":"cd95f8951573f4e021ddb57594dd3376a2b2f644"}}],"version":"2"} +``` + +Those exact manifest bytes digest to +`sha256:e870d43fa34f5f863ef7382e70844c4a2ebe86d79dac14505f005f4dda448e70`. +The fixture is a two-commit SHA-1 repository. Parent +`8884f45f3cf3e306cf0eeaa39febd41808151db5` contains `README.md` with bytes +`old\n`. The declared head contains `README.md` with bytes `hi\n` and symlink +`current` with target bytes `README.md`. Both commits use +`Eval Fixture ` as author and committer, timestamps +`0 +0000` and `1 +0000`, and messages `initial\n` and `current\n`, +respectively. Its canonical object-set bytes are: + +```text +3367afdbbf91e638efe983616377c60477cc6612 blob 4 +42061c01a1c70097d1e4579f29a5adf40abdec95 blob 9 +45b983be36b73c0788dc9cbcb76cbb80fc7bb057 blob 3 +4f5089b76757df68fb1b6b02be2c8da302d03550 tree 37 +785c0b097dc21d45f9726d412592b7609526c4c6 tree 72 +8884f45f3cf3e306cf0eeaa39febd41808151db5 commit 166 +cd95f8951573f4e021ddb57594dd3376a2b2f644 commit 214 ``` -Those exact bytes digest to -`sha256:658d89a3127eb79b1479960d3f264456c42c170920cac79e0a6e1837db60d543`. -The file bytes are `hi\n`. A change to the schema, fixture bytes, or digest is a -versioned contract change, not an implementation detail. +Those bytes, including the final LF, digest to +`sha256:b3b49d7b3ff3fd8c4fb3acbf7de36d92f6a4029f3244064f79e37408463288b5`. +A change to the schema, fixture bytes, or either digest is a versioned contract +change, not an implementation detail. ### Failure Contract @@ -1931,30 +2031,42 @@ into successful empty output and performs no automatic retry. ### U4. Immutable OCI workspace materialization -- **Goal:** Add OCI as the second immutable generation source without weakening - Git reuse, attachment, retention, or failure behavior. -- **Files:** AllAgents OCI client, manifest/archive validator, - workspace-manifest types, deterministic producer fixture, local registry E2E, - generation fixtures, and security fixtures. +- **Goal:** Add OCI as the second immutable generation source, including + tree-only and normalized offline-history snapshots, without weakening Git + reuse, attachment, retention, or failure behavior. +- **Files:** AllAgents OCI client, manifest/archive and semantic Git validator, + workspace-manifest types, tree-only and history-bearing producer fixtures, + local registry E2E, generation fixtures, and security fixtures. - **Approach:** Resolve only configured registries; treat `snapshotName` as the operator catalog selector and `imageManifestDigest` as the required direct OCI image-manifest identity; implement bounded Basic/Bearer auth and exact-host redirects; compute the immutable resolved plan; on a generation miss verify the image manifest, config, `workspaceManifestDigest`, and layers while - streaming; reject `.git` administrative subtrees; apply staging changesets; - validate paths/types/limits/catalog; and return through the same envelope as - Git. A runnable Harbor or SWE-bench instance image requires an explicit - adapter/transform and is never relabeled as a source-only snapshot. The runner - remains the sole publisher/resource preparer and the gateway the sole session- - attachment writer. + streaming; apply staging changesets; validate paths, types, limits, and the + catalog; and return through the same envelope as Git. Reject overlapping + repository destinations. For each manifest root that declares `git`, validate + detached `HEAD`, index/tree equality, exact object closure and digest, exact + worktree/commit equality, closed refs/config, and the absence of remotes, + credentials, and unsafe administrative state. Reject `.git` in tree-only or + undeclared locations. A runnable Harbor or SWE-bench instance image still + requires an explicit adapter/transform and is never relabeled as a workspace + source snapshot. The runner remains the sole publisher/resource preparer and + the gateway the sole session-attachment writer. - **Verification:** Distribution fixtures cover exact request-field naming, `snapshotName` lookup, direct `imageManifestDigest` enforcement, workspace- - manifest equality, auth, private CA, compression, whiteouts, redirects, - rebinding, indexes, foreign media, traversal, links, devices, sparse files, - cancellation, cleanup, no Git fallback, and exact error precedence. - Concurrent identical OCI requests produce one publication; read-only sessions - share it; editable sessions get private copies; access, retention, cwd, - harness/profile, and session do not fragment its generation key. + manifest equality, tree-only snapshots, offline history with no configured + remote, `git log`/`git blame`/historical diff in read-only and editable + attachments, private editable Git-state isolation, exact commit/object-set/ + worktree mismatch, overlapping destinations, producer removal and materializer + rejection of remote or credential configuration, forbidden config/includes/ + hooks/alternates/worktrees/shallow/replace/graft/ref/reflog state, undeclared + `.git`, physical accounting, + auth, private CA, compression, whiteouts, redirects, rebinding, indexes, + foreign media, traversal, links, devices, sparse files, cancellation, cleanup, + no Git fallback, and exact error precedence. Concurrent identical OCI requests + produce one publication; read-only sessions share it; editable sessions get + private copies; access, retention, cwd, harness/profile, and session do not + fragment its generation key. ### U5. Harness-native OAuth, optional proxy, and Promptfoo E2E @@ -2038,7 +2150,7 @@ into successful empty output and performs no automatic retry. | Caller authentication | Every external create, continuation, retrieval, stream, cancellation, file, artifact, and lifecycle administration path authenticates before existence or metadata disclosure. | | Generation ordering | Generic session/tombstone admission precedes response visibility; secret-free preflight/validate and selected-reference verification plus access-specific authorization/reservation precede resolve. A miss reserves staging/prospective generation before acquisition; containment, full-tree accounting, commit-tree/Git/manifest verification, and atomic accounting conversion precede ready state/pins. Runner prepare plus gateway ready-ack precede provider dispatch; fallback never reenters. | | Shared-build cancellation | One request cancellation/deadline detaches only that waiter. A build continues for remaining live waiters, stops when none remain or its runner-owned deadline expires, and produces at most one publication/failure for its epoch. | -| Manifest integrity | Git and OCI share the normative source-visible schema and fixture. Git administrative bytes are omitted only after exact semantic validation, and repository-mode content must equal the union of resolved commit trees at non-overlapping destinations plus necessary ancestors; OCI rejects `.git`. Plan/key drift, undeclared paths, forged manifests, changed staging, invalid paths/types/links/destinations, and digest mismatches fail before publication. | +| Manifest integrity | Git and OCI share one source-visible schema with pairwise non-overlapping repository destinations. A root may omit `.git` only when its manifest item declares history and the runner validates the detached commit, exact index/tree and object set, exact source-visible worktree, closed configuration and refs, and safe administrative state. Tree-only and undeclared `.git` fail. Git-acquired content equals the union of resolved commit trees at their destinations plus necessary ancestors. Plan/key drift, undeclared paths, forged manifests, changed staging, invalid paths/types/links/destinations, semantic Git mismatch, and digest mismatch fail before publication. | | Shared read-only generation | Concurrent sessions using different harnesses/profiles share one exact generation epoch. Root, nested, symlink, and alternate-path writes fail; runtime/session/auth/output state remains isolated. | | Editable isolation | Every fitting editable trial receives an inode-independent private tree and reserved hard byte/inode allowance covering overlays/checkpoints/produced state. A non-fitting waiter fails alone; continuation preserves a fitting trial's mutations but cannot grow past its envelope; siblings and the generation remain unchanged. | | Materializer containment | Fork/double-fork/cancellation/deadline fixtures prove `populated 0` before result read, publication, secret release, or cleanup. `containment_pending` blocks terminal visibility/readiness through restart and resolves once after quiescence. | @@ -2047,7 +2159,7 @@ into successful empty output and performs no automatic retry. | Retention and disposal | Fake-clock evidence proves one session CAS rejects busy/expired continuation admission, provisionally saves/clears a valid deadline, and either commits active after profile admission or restores the exact future deadline/tombstones an elapsed one after pre-allocation profile failure. Terminal acknowledgement alone sets the next `expiresAt`; polls/replays do not renew. Invalid failed responses stay accounted through purge; retained expiry returns HTTP 410; purge returns stock unknown; persistence authorizes before source access; operator deletion is idempotent. Null `lastUsedAt` epochs evict first by `publishedAt`; used epochs order by `lastUsedAt`, then `publishedAt`, generation key, and epoch. | | Session continuity | Both modes preserve conversation and fixed generation key/epoch/access/retention/cwd/harness/auth binding while persistent or unexpired; editable preserves private files; read-only remains immutable. Corrupt known evidence returns HTTP 409 non-resumable with no source access or later-epoch substitution. | | Git acquisition | Caller-supplied canonical HTTPS URLs, public-address egress enforcement, DNS-rebinding and redirect defense, structured-scope credential isolation, constrained refs, exact commits, closed transport/config, exact object closure/index semantics, generation reuse, and partial cleanup pass against local network fixtures. | -| OCI acquisition | Digest/media/path/link/type/limit, `.git` rejection, generation reuse, and attachment matrix pass against a local registry. | +| OCI acquisition | Digest/media/path/link/type/limit checks, tree-only and normalized offline-history fixtures, producer removal and materializer rejection of remotes and credentials, semantic Git verification, generation reuse, and the attachment matrix pass against a local registry. | | Credential boundary | Preflight sees no secret values and returns bounded configured reference identities; validate selects a bounded subset; the runner verifies handles and injects only that selected set into source-access children. Source secrets and caller keys are absent from staging, generations, private trees, base environments, checkpoints, backups, logs, and output. The selected OAuth profile is visible only through its active-turn projection, which is absent before acknowledgement and after restart reconciliation. | | Provider boundary | Codex/Pi native OAuth, refresh repair, projection teardown, idempotency/session/profile admission, different-profile concurrency, same-profile fail-fast exclusion, and explicit proxy scope all pass without implicit switching. | | Packaging | The public GHCR digest and provenance/SBOM attestations verify exact inputs; deployment uses that digest and finite lifecycle configuration. | @@ -2061,10 +2173,13 @@ into successful empty output and performs no automatic retry. `workspace.yaml` uses `path` plus optional `url` with no ordinary `source`/`repo` compatibility fields; the UHP descriptor uses `url`, optional `ref`, and `destination`; snapshot requests use `snapshotName`, - `imageManifestDigest`, and `workspaceManifestDigest`; runtime environment and - benchmark task identity remain separate; and immutable generations, read-only - and editable attachments, bounded retention, native OAuth, explicit proxy - mode, and GHCR digest-pinned distribution remain consistent. + `imageManifestDigest`, and `workspaceManifestDigest`; snapshot repository roots + may be tree-only or carry declared, normalized offline Git history without a + configured Git remote; runtime environment and benchmark task identity remain + separate; + and immutable generations, read-only and editable attachments, bounded + retention, native OAuth, explicit proxy mode, and GHCR digest-pinned + distribution remain consistent. - The U0 evidence predates U1-U6 and both native targets pass on the recorded inputs; changed inputs have replacement evidence before dependent work resumes. - No second execution protocol/control plane, separate AllAgents gateway, direct diff --git a/docs/research/workspace-contract-incumbents.md b/docs/research/workspace-contract-incumbents.md index 7ec751b6..0ed7587a 100644 --- a/docs/research/workspace-contract-incumbents.md +++ b/docs/research/workspace-contract-incumbents.md @@ -51,7 +51,7 @@ The important distinction is between two repositories that coding benchmarks oft 1. the **benchmark/task repository**, selected by Harbor `--repo`, which contains `task.toml`, instructions, environment definitions, and tests; and 2. the **target application repository**, which is normally baked into the task image or acquired by task-authored environment setup. -Harbor has a strong, reusable contract for the first item and for the resulting runnable environment. Its Git dataset identifier does not independently describe an arbitrary set of target repositories, their checkout destinations, source credentials, or the resolved provenance returned to a caller. The prebuilt image field likewise selects the whole task environment rather than identifying a source-only OCI workspace with a separately verified source manifest. +Harbor has a strong, reusable contract for the first item and for the resulting runnable environment. Its Git dataset identifier does not independently describe an arbitrary set of target repositories, their checkout destinations, source credentials, or the resolved provenance returned to a caller. The prebuilt image field likewise selects the whole task environment rather than a workspace source artifact with a separately verified manifest. An AllAgents source artifact may preserve normalized offline Git history, but it still excludes tools, services, verifier assumptions, and runtime configuration. AllAgents should therefore follow Harbor's **architecture** for benchmark interoperability: immutable task packages, prebuilt OCI environments, explicit workdir/resources/network policy, isolated trials, and verifier separation. A Harbor adapter can compile a selected task and environment into the AllAgents execution inputs. The Harbor task schema should not replace the smaller direct-execution descriptor used when a caller supplies target repositories or a workspace snapshot without a benchmark package. @@ -65,7 +65,7 @@ Hugging Face also participates in real workspace materialization, but the respon That is a concrete and widely used Git-to-OCI workspace pipeline. Hugging Face itself supplies registry and dataset-repository contracts; SWE-bench supplies the coding-task schema and execution harness. Neither exposes one general multi-repository invocation schema. SWE-bench is intentionally specialized to one repository/base commit and its test-transition metadata. -For compatibility, an AllAgents benchmark adapter should map a SWE-bench `repo` plus `base_commit` to repository mode. A prepared instance image must instead pass through a future task/environment boundary, or an importer must extract its checkout and produce a source-only `workspaceSnapshot` with a separately verified workspace manifest. The adapter must never register the runnable image itself as a workspace snapshot. This is an ingestion mapping, not a reason to replace the direct descriptor: AllAgents still needs multiple destinations, Git-or-OCI selection, strict credential and egress policy, continuation binding, and resolved attachment provenance. +For compatibility, an AllAgents benchmark adapter should map a SWE-bench `repo` plus `base_commit` to repository mode. A prepared instance image must instead pass through a future task/environment boundary, or an importer must extract its checkout into a `workspaceSnapshot` with a separately verified workspace manifest. The importer may preserve normalized Git history for offline evaluation, but it must remove remotes, credentials, and unsafe administrative state. The adapter must never register the runnable image itself as a workspace snapshot. This is an ingestion mapping, not a reason to replace the direct descriptor: AllAgents still needs multiple destinations, Git-or-OCI selection, strict credential and egress policy, continuation binding, and resolved attachment provenance. Primary evidence: the [SWE-bench dataset schema on Hugging Face](https://huggingface.co/datasets/princeton-nlp/SWE-bench), [Hugging Face dataset repository model](https://huggingface.co/docs/hub/datasets-overview), and [SWE-bench evaluation harness](https://www.swebench.com/SWE-bench/reference/harness). @@ -134,9 +134,9 @@ Primary evidence: GitHub's [create-codespace REST operation](https://docs.github The source descriptor should remain AllAgents-owned, but its immutable artifact identities should not be invented locally. -For OCI snapshots, the [OCI Image Specification 1.1.1 descriptor](https://github.com/opencontainers/image-spec/blob/v1.1.1/descriptor.md) defines content identity with media type, digest, and size, including verification against the digest. The [image manifest](https://github.com/opencontainers/image-spec/blob/v1.1.1/manifest.md) defines the config descriptor and ordered layer descriptors. Those are the correct normative identities for the snapshot artifact. The AllAgents workspace manifest remains necessary because OCI does not define the source-visible workspace tree, destination layout, logical working directory, or gateway attachment result. +For OCI snapshots, the [OCI Image Specification 1.1.1 descriptor](https://github.com/opencontainers/image-spec/blob/v1.1.1/descriptor.md) defines content identity with media type, digest, and size, including verification against the digest. The [image manifest](https://github.com/opencontainers/image-spec/blob/v1.1.1/manifest.md) defines the config descriptor and ordered layer descriptors. Those are the correct normative identities for the snapshot artifact. OCI does not define the source-visible workspace tree, destination layout, logical working directory, or the semantic state of an embedded Git repository. The AllAgents workspace manifest therefore declares each repository root as tree-only or history-bearing. A history-bearing root records its resolved commit and canonical object-set digest, while the runner verifies the offline `.git` state and absence of remotes. -For Git, a full commit object ID is the resolved source identity. The request still needs the original ref because a branch/tag name and its resolved commit answer different audit questions. Neither Git nor OCI defines when a runner has successfully attached that content, so `effectiveDescriptorDigest`, `generationId`, `sourceIdentity`, `workingDirectory`, and `workspaceManifestDigest` must remain AllAgents result fields. +For Git, a full commit object ID is the resolved source identity. The request still needs the original ref because a branch/tag name and its resolved commit answer different audit questions. OCI snapshot provenance instead retains the verified `snapshotName` and `imageManifestDigest`; each history-bearing root adds only its destination, resolved commit, and object-set digest, with no repository URL or requested ref. Neither Git nor OCI defines when a runner has successfully attached that content, so `effectiveDescriptorDigest`, `generationId`, `sourceIdentity`, `workingDirectory`, and `workspaceManifestDigest` must remain AllAgents result fields. ## Recommendation and adoption rule @@ -146,7 +146,7 @@ Adopt the following rule for future changes: - **Benchmark compatibility:** ingest Harbor task packages and SWE-bench/Hugging Face records through adapters. Preserve Harbor's task/environment/verifier split and its preference for prebuilt OCI environments; map benchmark source identities into the canonical AllAgents descriptor. - **Source-layout precedent:** use Devfile 2.3 `projects` semantics when adding or naming direct repository fields. Document intentional divergence, especially for URL shape, revision behavior, and destination paths. - **Runtime precedent:** evaluate Harbor ASP as a southbound execute/filesystem adapter when its draft stabilizes. E2B and Daytona remain provider adapters. Dev Containers may define an optional environment-building layer after source acquisition. -- **Do not conflate:** Harbor's benchmark repository with the target application source; an environment image with source-only provenance; or a vendor sandbox/codespace object with the northbound contract. Do not adopt Devfile's default-on-missing revision behavior, ZIP-without-digest source, or runtime credential exposure. +- **Do not conflate:** Harbor's benchmark repository with the target application source; a runnable environment image with a workspace source snapshot, even when the snapshot preserves Git history; or a vendor sandbox/codespace object with the northbound contract. Do not adopt Devfile's default-on-missing revision behavior, ZIP-without-digest source, or runtime credential exposure. This is deliberately a layered answer rather than a claim that AllAgents has invented a universal workspace standard. The narrow extension exists because the examined standards stop either before source acquisition or before committed, immutable provenance. @@ -157,4 +157,4 @@ This is deliberately a layered answer rather than a claim that AllAgents has inv - The general AI research wiki has relevant Harbor and benchmark-provenance coverage but no dedicated Devfile, Dev Containers, E2B, Daytona, or Codespaces contract comparison. Its primary-source links informed source discovery; its prose is not a normative input here. - The private AllAgents research wiki contains one execution-gateway comparison based on the older A2A-era decision baseline. That coverage is now historical because ADR 0002 selects UHP. No private synthesis or conclusion is reproduced in this public note. -ADR 0002 and the implementation plan now codify this layered result: the canonical JSON request keeps `url`, `ref`, `destination`, and logical `workingDirectory`; local `workspace.yaml` replaces `source` plus `repo` with `url` while retaining `path`; OCI requests use explicit `snapshotName`, `imageManifestDigest`, and `workspaceManifestDigest`; and benchmark task/environment ingestion remains a separate future adapter boundary. The two older public research notes still need their stale pre-cutover descriptions corrected when they are next maintained; any private-wiki refresh remains a separate private edit. +ADR 0002 and the implementation plan now codify this layered result: the canonical JSON request keeps `url`, `ref`, `destination`, and logical `workingDirectory`; local `workspace.yaml` replaces `source` plus `repo` with `url` while retaining `path`; OCI requests use explicit `snapshotName`, `imageManifestDigest`, and `workspaceManifestDigest`; workspace-manifest version 2 lets each snapshot root be tree-only or carry normalized offline Git history without a configured Git remote; and benchmark task/environment ingestion remains a separate future adapter boundary. The two older public research notes still need their stale pre-cutover descriptions corrected when they are next maintained; any private-wiki refresh remains a separate private edit. From e83834a34fe02c0a5acd93bb7f1d2528599f720b Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Thu, 24 Sep 2026 12:50:01 +1000 Subject: [PATCH 25/44] docs(architecture): protect editable change collection --- .../0002-adopt-uhp-through-harnessrouter.md | 87 ++++- ...0837-feat-coding-execution-gateway-plan.md | 308 +++++++++++------- 2 files changed, 262 insertions(+), 133 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 10837616..6ba6efcf 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -45,8 +45,8 @@ An initial workspace-backed turn follows this order: 6. On a cache miss, the materializer builds private staging. The runner independently verifies the source tree, Git or OCI state, manifest, limits, and generation key before publishing it atomically. -7. The runner pins the generation, then mounts it read-only or creates an inode- - independent private editable copy. +7. The runner pins the generation, then mounts it read-only or creates a private + writable view with no mutable state shared with another session. 8. The gateway commits the attachment as `ready`. The runner acknowledges that commit, transfers or releases reservations, and releases the provisional pin exactly once. @@ -104,7 +104,7 @@ flowchart TB MATERIALIZER --> SOURCE[HTTPS Git or configured OCI registry] RUNNER --> GENERATION[(immutable generations)] GENERATION --> READONLY[shared read-only attachment] - GENERATION --> EDITABLE[private editable copy] + GENERATION --> EDITABLE[private editable view] READONLY --> HARNESS[Codex or Pi harness] EDITABLE --> HARNESS RUNNER --> AUTH[(native auth profiles)] @@ -121,7 +121,7 @@ flowchart TB |---|---| | UHP client | Prompt, model, harness ID, workspace descriptor, continuation ID | | HarnessRouter gateway | Caller authentication, UHP validation, public response and session state, attachment `ready`, expiry, tombstones | -| HarnessRouter runner | Generation claims and publication, source child processes, mounts, private copies, quotas, references, pins, profile admission | +| HarnessRouter runner | Generation claims and publication, source child processes, mounts, private writable views, quotas, references, pins, profile admission | | AllAgents materializer | Descriptor defaults, URL and source validation, Git or OCI resolution, staging construction, canonical manifest, provenance | | Coding harness | Provider login, native token refresh, conversation execution | | Operator | Deployment policy, egress, credential scopes, authentication profiles, persistence authorization, quotas, deletion, garbage collection | @@ -213,9 +213,18 @@ repositories: `path` remains the existing or managed local checkout location. `url` replaces the lossy `source` plus `repo` pair. `branch` remains branch-specific because managed synchronization performs branch checkout and pull; it does not claim arbitrary detached-ref semantics. Path-only unmanaged entries may omit `url`; a managed entry requires it. The schema, CLI, generated schemas, examples, and tests cut over together without accepting both shapes indefinitely. -Harbor and SWE-bench/Hugging Face are benchmark-ingestion precedents, not alternate workspace field vocabularies. Harbor clones a task repository and materializes its Dockerfile, Compose definition, or prebuilt `environment.docker_image`; SWE-bench records `repo` and `base_commit` and builds or pulls layered instance images. A future adapter may compile those records into the canonical AllAgents workspace and deployment inputs while preserving their upstream identity. +Harbor sits beside the AllAgents gateway at Promptfoo's provider boundary. +Promptfoo calls the gateway over UHP for AllAgents-backed rows; a Harbor provider +calls Harbor for container-native rows, where Harbor owns task setup, execution, +verification, artifacts, and teardown. Harbor is not an +`allagents.workspace` backend, and its task schema is not compiled into the +workspace descriptor. -A runnable Harbor or SWE-bench image is not automatically an AllAgents `workspaceSnapshot`. The former may combine source, tools, services, verifier assumptions, and runtime configuration. An AllAgents snapshot is a workspace source artifact: it may carry normalized offline Git history, but it does not select the runtime environment or verifier. Caller-selected task packages, runtime images, and verifiers require a separate versioned task/environment boundary rather than overloading `source`. +Harbor and SWE-bench/Hugging Face remain useful precedents for source and +benchmark packaging. Their runnable images may combine source, tools, services, +verifier assumptions, and runtime configuration, so they are not AllAgents +`workspaceSnapshot` artifacts. An AllAgents snapshot contains source and may +carry normalized offline Git history; it does not select a runtime or verifier. ### Access and retention @@ -223,7 +232,7 @@ Access and retention are independent: | | `readOnly` | `editable` | |---|---|---| -| Workspace | Shared immutable generation | Private inode-independent copy | +| Workspace | Shared immutable generation | Private writable view; no mutable state shared across sessions | | Initial UHP files | Rejected before source acquisition | Applied after attachment | | Writes | Filesystem rejects them; no copy-up | Allowed within the private quota | | Checkpoints | No source mutation checkpoint | Root and nested repositories use private checkpoints | @@ -236,6 +245,50 @@ Access and retention are independent: Neither retries nor continuations can change access or retention. +### Editable change collection + +The verified generation is the first-turn baseline. The runner does not copy or +inventory the complete private workspace again before the harness starts. + +1. For a history-bearing root, the protected baseline is its recorded commit and + generation-owned object store. For a tree-only root, it is the canonical + workspace manifest. The runner keeps this baseline outside the editable + workspace. +2. For each turn, the runner prepares and verifies the private view, then arms + candidate tracking before it applies UHP input overlays or gives any + non-runner process writable access. It durably binds that coverage marker to + the generation and previous turn state and keeps tracking active through + descendant quiescence. +3. After the harness and its descendants stop, the runner obtains changed-path + candidates from the runner-owned tracker or storage state. It compares their + final type, mode, and content with the protected baseline through + root-confined, no-follow reads. +4. If uninterrupted coverage cannot be proven, or candidate state is missing, + incomplete, overflowed, or uncertain after recovery, the runner walks the + complete private view without following links and performs the same bounded + comparison. + +The produced-file domain is every source-visible path under the declared +workspace roots. The only exclusions are the original administrative `.git` +subtrees identified by the protected generation record. Their mutations persist +for continuation but are not produced files. An agent-created `.git` elsewhere +is ordinary source-visible content. Candidate and full-scan paths use this same +protected classification; final Git discovery or ignore rules cannot change it. + +For a continuation, the runner starts from the previous protected cumulative +path state and applies verified candidates, or rebuilds that state with the +fallback scan. It stores entries only for content that differs from the +generation; unchanged paths inherit their generation state. Comparing the new +state with the previous checkpoint yields the turn's produced-file delta without +retaining or comparing a second full workspace. + +Editable `.git` state remains session-private and survives continuation for +coding tools. After the harness starts, it is not authoritative for provenance +or change collection. The runner never trusts its refs, configuration, index, +hooks, alternates, or ignore rules, and an agent-edited ignore file cannot hide +a produced path. Candidate tracking is an optimization; the bounded full-tree +comparison remains the correctness fallback. + ### Public workspace metadata Once attachment reaches `ready`, terminal events, retrieval, background completion, replay, and later terminal failures return the same verified workspace object. @@ -312,7 +365,7 @@ Every redirect is checked against the original scope. The connector strips the c Credentials are never encoded in URLs, persisted in Git configuration or remote URLs, or returned in hook output. Temporary credential state is removed before return. The gateway, runner base environment, published generation, editable -copy, and every agent child remain credential-free. If a configured source secret +view, and every agent child remain credential-free. If a configured source secret appears in the service or agent environment, the runner refuses to launch the agent. @@ -437,9 +490,9 @@ cancel the build while another waiter remains; the runner cancels it when no waiter remains. Failed or partial staging is never attachable, and a failed competing build does not poison an existing verified generation. -Before a mount or copy, the runner acquires a provisional pin under the generation lock. Garbage collection cannot race that pin. +Before attaching a view, the runner acquires a provisional pin under the generation lock. Garbage collection cannot race that pin. -The generation backing store remains owner-writable and is never exposed writable to a session. Publication is a recoverable same-filesystem atomic transition. Editable copies may not share mutable inodes with the generation or another session. +The generation backing store remains owner-writable and is never exposed writable to a session. Publication is a recoverable same-filesystem atomic transition. Editable views may not share mutable state with the generation or another session. Read-only sessions share source bytes but keep their operating-system identity, conversation, home, temporary files, logs, outputs, and response state separate. @@ -447,7 +500,7 @@ conversation, home, temporary files, logs, outputs, and response state separate. The gateway and runner commit an attachment in three steps: -1. The runner prepares the mount or private copy and returns opaque evidence. +1. The runner prepares the read-only mount or private writable view and returns opaque evidence. 2. The gateway commits attachment state as `ready`. 3. The runner acknowledges that commit, creates the durable reference or transfers the private reservation, and releases the provisional pin exactly once. @@ -521,14 +574,14 @@ Every deployment limit must be finite and nonzero: Before a response is visible, one idempotent admission token reserves a generic session slot and a fixed-size tombstone slot. Invalid descriptors remain charged through failed-response retention, tombstoning, and purge. After validation and before source resolution, the runner authorizes persistence -and reserves its slot. Editable access also receives one stable private- -reservation ID. Its hard byte and inode allowance covers the private tree, UHP +and reserves its slot. Editable access also receives one stable private-view +reservation ID. Its hard byte and inode allowance covers the writable view, UHP overlays, root and nested-repository checkpoints, and produced-file state across every turn and continuation. A cache miss reserves staging and prospective generation capacity before byte acquisition. Independent full-tree accounting converts that reservation to actual retained usage before publication. -A waiter receives an editable copy only when the complete generation fits its private allowance. A non-fitting waiter fails alone and does not invalidate the shared generation or another waiter. +A waiter receives an editable view only when the complete generation fits its private allowance. A non-fitting waiter fails alone and does not invalidate the shared generation or another waiter. Protected state is never evicted. If leases, references, pins, or other protected resources consume capacity, admission fails instead. @@ -711,9 +764,9 @@ The system fails closed. Source, access mode, retention, credentials, and provid | Missing or corrupt bound attachment evidence | HTTP 409 `allagents_workspace_non_resumable` | No profile admission or epoch substitution; return committed workspace metadata | | Busy native profile after replay and `session_busy` checks | HTTP 503 `harness_unavailable`, reason `allagents_auth_profile_busy` | Fail before allocation, runner work, or materialization | | Generic capacity unavailable | HTTP 503 `allagents_workspace_capacity_exceeded` | Admit no response or source work | -| Editable copy or later growth exceeds its allowance | `allagents_workspace_private_quota_exceeded` | Fail only that waiter or turn; preserve mode and retention | +| Editable view or later growth exceeds its allowance | `allagents_workspace_private_quota_exceeded` | Fail only that waiter or turn; preserve mode and retention | | Source policy, authentication, or transport failure | Coded failed response | Remove unpublished staging; publish nothing; start no agent or provider fallback | -| Generation, attachment, or copy failure | Coded failed response | Quarantine incomplete state and release reservations and pins exactly once | +| Generation, attachment, or private-view failure | Coded failed response | Quarantine incomplete state and release reservations and pins exactly once | | Materializer timeout, crash, malformed output, or live descendant | Coded materializer or containment failure | Wait for cgroup quiescence before result handling, secret release, or cleanup | | Provider authentication failure | Normalized UHP failure | Do not switch profile or activate proxy; finish teardown before lock release | | Provider execution failure | Normalized UHP failure | No source fallback and no credential material in output | @@ -763,7 +816,7 @@ Revisit this decision when: - the host cannot enforce immutable read-only generation mounts across sessions; - continuation, expiry, deletion, and lease acquisition cannot be linearized and recovered safely; - generation churn or authorized persistent demand cannot fit practical finite quotas; -- editable derivation requires stronger filesystem semantics than an independent private copy; +- editable derivation requires stronger filesystem semantics than a private writable view with no shared mutable state; - upstream HarnessRouter accepts the generic workspace or authentication-state seam; - UHP adopts a standard workspace attachment or retention contract that replaces this extension; - the maintained patch grows beyond the narrow integration boundary; diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index fe1aa703..d75da5a1 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -23,8 +23,8 @@ execution: code - **Means:** Deploy a pinned HarnessRouter CE fork. Preserve HarnessRouter's UHP, caller authentication, session, streaming, cancellation, artifact, and agent-runner behavior. Add a generic generation resolve/build/attach boundary, - immutable generation store, read-only mounts, private editable copies, durable - leases, retention and quota state, garbage collection, nested logical working + immutable generation store, shared read-only mounts, private writable views, + durable leases, retention and quota state, garbage collection, nested logical directories, mode-specific checkpoint/collection behavior, and separation between session state and durable harness-native OAuth state. Implement Git/OCI semantics in a separate AllAgents executable. Codex authenticates @@ -95,8 +95,8 @@ Promptfoo request names the HTTPS Git repositories to load; the AllAgents executable validates those URLs against deployment egress policy, resolves an immutable source plan, and builds verified staging only on a generation cache miss. The runner atomically publishes or reuses the generation, records the -session attachment and retention state, then mounts it read-only or creates a -private editable copy before provider dispatch. +session attachment and retention state, then attaches either a shared read-only +mount or a private writable view before provider dispatch. A continuation supplies `previous_response_id`, omits the workspace extension, and uses HarnessRouter's current native conversation plus the bound attachment. @@ -138,7 +138,7 @@ caller responsible for acquisition. The temporary fork closes those seams. provenance. It never authorizes persistence or publishes live state. - **A4. HarnessRouter runner:** Owns generation claims/publication and the resource journal: provisional pins, durable references, read-only mounts, - private editable copies and quotas, per-session operating-system identity and + private writable views and quotas, per-session operating-system identity and runtime state, mode-specific checkpoints and produced files, safe nested cwd, selected Codex/Pi process, conversation state, active-turn auth projection, cleanup, and garbage collection. It prepares attachment evidence but never @@ -175,7 +175,7 @@ caller responsible for acquisition. The temporary fork closes those seams. - **Publish once per generation; attach once per session.** Concurrent requests for one immutable source plan share one claim and verified publication. Read-only sessions share that generation; editable sessions receive private - writable copies. Continuations omit the extension and reuse the original + writable views. Continuations omit the extension and reuse the original attachment through `previous_response_id`. - **Separate access from retention.** `readOnly` versus `editable` controls mutability. Default `session` versus authorized `persistent` controls @@ -194,13 +194,14 @@ caller responsible for acquisition. The temporary fork closes those seams. provenance keeps `requestedRef` separate from `resolvedCommit`. Local `workspace.yaml` repository entries keep `path` and replace the provider- specific `source` plus `repo` pair with one canonical `url`. Benchmark-specific - aliases are accepted only by future adapters, never by the canonical schema. -- **Keep benchmark task/environment identity separate from workspace source.** - Harbor task repositories and `environment.docker_image`, and SWE-bench/Hugging - Face `repo`, `base_commit`, and instance images, are adapter inputs. A runnable - benchmark image is not an AllAgents workspace source artifact. A snapshot may - contain normalized offline Git history, but direct task packages, environment - images, and verifiers need a separate versioned boundary if added later. + aliases are not accepted by the canonical schema. +- **Keep provider lanes separate.** Promptfoo calls the AllAgents gateway over + UHP for AllAgents-backed rows. A Harbor provider calls Harbor for + container-native rows, where Harbor owns setup, execution, verification, + artifacts, and teardown. Harbor is not an `allagents.workspace` backend and + its task schema is not compiled into the workspace descriptor. Harbor and + SWE-bench/Hugging Face remain packaging precedents, but their runnable images + are not AllAgents source snapshots. - **Prefer harness-native OAuth.** Promptfoo's HarnessRouter API key authenticates the UHP caller only. Codex and Pi use their own login, token storage, refresh, and provider request path; native mode has no provider-route API key. @@ -420,7 +421,7 @@ caller responsible for acquisition. The temporary fork closes those seams. byte/inode usage. Under the generation lock, successful publication atomically converts the prospective generation reservation to actual usage, releases its excess and the staging reservation, and persists that accounting transition - before ready state or waiter pins become visible. Private copy-fit is not a + before ready state or waiter pins become visible. Private-view fit is not a shared-build condition: after publication, each editable waiter compares total physical generation bytes/inodes with its own hard allowance. A waiter that cannot fit fails `allagents_workspace_private_quota_exceeded` and releases only @@ -491,26 +492,72 @@ caller responsible for acquisition. The temporary fork closes those seams. stable private-reservation ID admitted before resolve and compares the generation's independently measured total physical bytes/inodes with that allowance. Failure detaches only that waiter. A fitting waiter creates and - validates a unique writable copy with no mutable inode shared with the - generation and initializes root/nested checkpoints and collection baselines. - Its attachment evidence contains the epoch and opaque reservation ID. Ready - acknowledgement transfers the reservation from admission to the private - workspace without a second debit, then releases the provisional pin exactly - once. Failure before acknowledgement releases the reservation and prepared - resources once unless reconciliation proves that the gateway committed ready. - Expiry or deletion releases the ready workspace's reservation once. Startup - reconciles both halves of this prepare/ack and quota-transfer protocol. - - The editable hard quota covers the private tree, UHP input overlays, + validates a private writable view with no mutable state shared with the + generation or another session. A backend may use a full copy, reflink, + copy-on-write view, or storage clone only after proving the same isolation, + quota, accounting, and cleanup behavior. The runner initializes root/nested + checkpoints and stores protected collection state outside the editable + workspace. Its attachment evidence contains the epoch and opaque reservation + ID. Ready acknowledgement transfers the reservation from admission to the + private workspace without a second debit, then releases the provisional pin + exactly once. Failure before acknowledgement releases the reservation and + prepared resources once unless reconciliation proves that the gateway + committed ready. Expiry or deletion releases the ready workspace's reservation + once. Startup reconciles both halves of this prepare/ack and quota-transfer + protocol. + + The editable hard quota covers the private view, UHP input overlays, root/nested checkpoints, and produced-file state for every turn and - continuation. The filesystem quota backend must deny writes beyond either - byte or inode allowance and surface exhaustion to the runner; the runner - terminates that turn as failed `allagents_workspace_private_quota_exceeded` - without changing access or retention. Actual usage and reserved allowance are - persisted and reconciled before readiness. Only editable state receives - ordinary UHP input files, mutation checkpoints, and produced-file collection. - A `readOnly` request containing workspace input files fails before source - acquisition. + continuation. The filesystem quota backend denies writes beyond either byte or + inode allowance and surfaces exhaustion to the runner; the runner terminates + that turn as failed `allagents_workspace_private_quota_exceeded` without + changing access or retention. Actual usage and reserved allowance are persisted + and reconciled before readiness. Only editable state receives ordinary UHP + input files, mutation checkpoints, and produced-file collection. A `readOnly` + request containing workspace input files fails before source acquisition. + + The verified generation is the first-turn collection baseline; attachment + does not walk, hash, or copy the complete private view again. For a + history-bearing root, the protected descriptor names its recorded commit and + generation-owned object store. For a tree-only root, it names the canonical + workspace manifest. The runner stores these descriptors outside the editable + workspace. + + For every turn, the runner prepares and verifies the private view, then arms + candidate tracking before it applies a UHP input overlay or gives any + non-runner process writable access. It durably binds that coverage marker to + the generation and prior protected turn state. Tracking remains active through + runner-applied overlays and harness-cgroup quiescence. + + After the harness cgroup is empty, the runner obtains additions, deletions, + type and mode changes, and content-change candidates from a runner-owned + change tracker or storage state. It verifies every candidate against the + protected descriptor and final workspace with root-confined, no-follow reads. + Candidate tracking is an optimization. If uninterrupted coverage cannot be + proven, or its state is missing, incomplete, overflowed, or uncertain after + recovery, the runner walks the complete private view without following links + and reconstructs the bounded cumulative difference from the generation. + + Before terminal acknowledgement, the runner durably stores protected path state + only for content that differs from the generation. Unchanged paths inherit + generation state. On continuation, it applies verified candidates to the prior + cumulative state, or rebuilds that state with the fallback scan, then compares + the result with the prior state to produce the turn delta. It never retains or + compares a second full workspace. A path restored to its prior-turn state + produces no turn delta; transient-write auditing is outside this contract. + + The produced-file domain is every source-visible path under the declared + workspace roots. The only exclusions are the original administrative `.git` + subtrees identified by the protected generation record. Their mutations + persist for continuation but are not produced files. An agent-created `.git` + elsewhere is ordinary source-visible content. Candidate and full-scan paths + use this same protected classification; final Git discovery or ignore rules + cannot change it. + + Editable `.git` state remains part of the private session for coding tools and + continuation, but collection never trusts its repository identity, refs, + configuration, index, hooks, alternates, or ignore rules. An agent-edited + ignore file cannot hide a produced path. `lastUsedAt` remains null until the gateway commits an attachment `ready`. After acknowledgement, the runner updates it under the generation lock to @@ -684,14 +731,14 @@ caller responsible for acquisition. The temporary fork closes those seams. retained-generation limits. Recompute the canonical workspace manifest from staging and require it to - match both the fetched manifest bytes and `workspaceManifestDigest`. After - publication the runner creates private collection baselines from the verified - trees so later produced-file reporting remains truthful. Both source modes - produce the same reusable immutable-generation abstraction. Validated `.git` - state from either mode is readable but immutable in `readOnly` attachments and - independently writable only in private `editable` copies. Generation - acquisition limits apply per build; retained-generation and private-workspace - quotas apply independently. + match both the fetched manifest bytes and `workspaceManifestDigest`. Publication + preserves that manifest and every semantic Git record as the protected + collection baseline; it does not build a second per-session inventory. Both + source modes produce the same reusable immutable-generation abstraction. + Validated `.git` state from either mode is readable but immutable in `readOnly` + attachments and independently writable only in private `editable` views. + Generation acquisition limits apply per build; retained-generation and + private-workspace quotas apply independently. - **R12.** Extend HarnessRouter's response translator and stored-response paths with a stage-dependent contract. Before attachment `ready`, non-2xx request errors and allocated terminal failures omit @@ -785,7 +832,7 @@ caller responsible for acquisition. The temporary fork closes those seams. deletion stays quarantined and counted against quota. Startup reconciles generic admission slots, active leases, build/staging/prospective-generation reservations, publication/accounting markers, build waiters, provisional pins, - references, mounts, private reservation transfers and actual usage, copies, + references, mounts, private reservation transfers, view state and actual usage, tombstones, compaction, and deletion before readiness or GC. If only protected state remains, new admission fails `allagents_workspace_capacity_exceeded`; no protected state is deleted and no @@ -830,11 +877,11 @@ caller responsible for acquisition. The temporary fork closes those seams. source-secret handle; native or proxy trust mode; delegated cgroup v2 subtree; and `on-failure` restart policy. Verify the image and mounted inputs before running checks that depend on them. -2. The runner validates read-only mount enforcement, private-copy isolation, +2. The runner validates read-only mount enforcement, private-view isolation, finite lifecycle policy, storage relationships, and cgroup delegation. It reconciles incomplete generation claims/publications, build waiters, - provisional pins, durable session references, mounts, private editable - workspaces and quota usage, auth projections, tombstones/compaction, and + provisional pins, durable session references, mounts, private writable + views and quota usage, auth projections, tombstones/compaction, and interrupted deletions. Sweep orphaned cgroups and credential projections only after proving each old process boundary empty. Do not start GC or serving. 3. Run the mounted AllAgents hook's bounded `preflight` mode. It validates hook, @@ -926,11 +973,12 @@ caller responsible for acquisition. The temporary fork closes those seams. waiter's pin and access-specific reservations. Other waiters continue against the valid ready epoch. Otherwise the gateway CASes that session `resolving -> attaching`, and the runner prepares either a durable read-only - epoch reference plus verified mount or a unique private copy plus checkpoints - and collection baselines. It returns an opaque token/evidence. The gateway - alone CASes `attaching -> ready`, stores key/epoch evidence, and acknowledges - the token. Under the generation lock, the runner releases that provisional pin - exactly once and advances `lastUsedAt` to at least the ready-commit timestamp. + epoch reference plus verified mount or a private writable view plus + checkpoints and protected collection state. It returns opaque token/evidence. + The gateway alone CASes `attaching -> ready`, stores key/epoch evidence, and + acknowledges the token. Under the generation lock, the runner releases that + provisional pin exactly once and advances `lastUsedAt` to at least the + ready-commit timestamp. Prepare/ack recovery preserves the committed attachment or rolls that waiter's resources/reservations back once. 7. Only after attachment `ready` do response events include the complete @@ -940,12 +988,15 @@ caller responsible for acquisition. The temporary fork closes those seams. projection or scoped proxy credential, and dispatches the harness. Provider retry/fallback cannot validate, resolve, build, attach, or change any binding. -8. Read-only collection reports no workspace mutation; editable collection walks - the private root and declared repositories without reporting initial source - files. After descendants stop, native finalization commits refresh state or - marks `repair-required`, removes the credential projection, and proves retained - homes/checkpoints clean. The gateway then durably stores the terminal response - and, for `session`, sets one expiry timestamp used by the terminal event, GET, +8. Read-only collection reports no workspace mutation. After descendants stop, + editable collection verifies trusted changed-path candidates against the + protected generation and prior turn state, or performs a bounded full-tree + scan when candidate state is not trustworthy. It never reports initial source + files or trusts editable `.git` metadata. Native finalization then commits + refresh state or marks `repair-required`, removes the credential projection, + and proves retained homes/checkpoints clean. The gateway then durably stores + the terminal response and, for `session`, sets one expiry timestamp used by + the terminal event, GET, background completion, and replay. Only after that acknowledgement may the runner release the profile lock. @@ -1034,7 +1085,7 @@ caller responsible for acquisition. The temporary fork closes those seams. token, then reconciles generic admission, active leases, staging/prospective- generation reservation, publication/accounting conversion, provisional-pin/ reference acquisition, attachment prepare/ack and private-reservation transfer, - mount/copy creation, private usage accounting, tombstoning, unmount, release, + mount/view creation, private usage accounting, tombstoning, unmount, release, compaction, and physical deletion without duplicating a debit/reference, leaking a pin, extending an original deadline, or exposing a partially deleted resource. @@ -1107,13 +1158,20 @@ caller responsible for acquisition. The temporary fork closes those seams. before publication, attachment, or agent launch. - **AE5.** Two `editable` turns linked by `previous_response_id` preserve native conversation and a private file mutation. A separate editable trial from the - same generation receives a unique clean copy and cannot observe or mutate the - first. An editable waiter whose initial copy cannot fit fails its own - `allagents_workspace_private_quota_exceeded` response without invalidating the - ready epoch or a concurrent read-only/fitting waiter. Produced-file collection - reports only private changes. Growth across turns cannot exceed the session's - reserved hard quota. Two read-only turns preserve conversation but have no - workspace mutation checkpoint or produced-file delta. + same generation receives a unique clean writable view and cannot observe or + mutate the first. An editable waiter whose initial view cannot fit fails its + own `allagents_workspace_private_quota_exceeded` response without invalidating + the ready epoch or a concurrent read-only/fitting waiter. First-turn collection + uses the protected generation without a redundant full pre-agent inventory. + Candidate tracking begins before input overlays or writable process exposure; + a missing or discontinuous coverage marker forces the full scan. Candidate and + full-scan paths return the same source-visible delta, exclude the protected + declared Git administrative subtrees, report an agent-created `.git` elsewhere + as ordinary content, and cannot be hidden by editable Git metadata or ignore + rules. + Continuation reports the delta from the prior protected turn state and cannot + grow past the session's reserved hard quota. Two read-only turns preserve + conversation but have no workspace mutation checkpoint or produced-file delta. - **AE6.** Continuation omits the extension and preserves the exact generation epoch, access, retention, cwd, harness, and profile. Any attempted rebinding is rejected. One CAS rejects same-session overlap without changing expiry and @@ -1125,14 +1183,15 @@ caller responsible for acquisition. The temporary fork closes those seams. `allagents_workspace_non_resumable`; a fully purged predecessor returns the stock unknown-ID error. None rematerializes or substitutes an epoch. - **AE7.** Explicit UHP input files overlay only an editable private workspace - after its initial checkpoint and before agent launch. A read-only request with - workspace input files fails `allagents_workspace_read_only`; a runtime write - receives a filesystem read-only error with no copy-up or mode change. + after its initial checkpoint and durable candidate coverage begins, but before + agent launch. A read-only request with workspace input files fails + `allagents_workspace_read_only`; a runtime write receives a filesystem + read-only error with no copy-up or mode change. - **AE8.** Faults at pre-allocation generic session/tombstone admission, validate, selected-credential verification, resolve, keyed epoch claim/waiter cancellation, staging/generation reservation and publication-accounting conversion, containment, provisional pin, attachment prepare/ack, read-only - epoch reference/mount, editable reservation-transfer/copy/checkpoint, expiry, + epoch reference/mount, editable reservation-transfer/view/checkpoint, expiry, unmount, release, tombstone compaction, and deletion either reconcile to one complete protected resource or fail closed. No agent sees staging, duplicate live epoch publication, partial private state, or a generation without required @@ -1165,7 +1224,7 @@ caller responsible for acquisition. The temporary fork closes those seams. changed auth fails before runner work. Missing generation/reference or private checkpoint returns the cataloged non-resumable error without replay. A `containment_pending` session remains non-terminal until its cgroup is empty; - interrupted builds, pins, copies, quota records, auth projections, tombstones, + interrupted builds, pins, views, quota records, auth projections, tombstones, compaction, and deletions reconcile without resurrection or double release. - **AE12.** The protected publish job releases the public `linux/amd64` GHCR package without Docker Hub credentials. Anonymous verification covers the @@ -1202,8 +1261,8 @@ caller responsible for acquisition. The temporary fork closes those seams. - Safe public egress enforcement, source-credential scope mapping, and optional project `workspace.yaml` snapshot-catalog additions. - Deterministic Git and immutable OCI generation construction. -- Shared read-only mounts, private editable copies, mode-specific - root/nested-repository checkpoint and produced-file integration. +- Shared read-only mounts, private writable views, mode-specific root/nested- + repository checkpoint and produced-file integration. - Session idle expiry, persistent authorization, operator deletion, bounded storage admission, restart reconciliation, and generation eviction. - Source credential isolation and native-OAuth trust-boundary verification. @@ -1223,9 +1282,10 @@ caller responsible for acquisition. The temporary fork closes those seams. credential fallback. - Caller-provided credentials, non-HTTPS/private-network origins, commands, host paths, materializers, or Docker options. -- Direct Harbor task-package ingestion, SWE-bench/Hugging Face dataset ingestion, - caller-selected runtime images, benchmark verifiers, or compatibility aliases - inside `allagents.workspace`. +- Harbor provider execution remains a separate Promptfoo lane outside this plan. + Direct Harbor task-package ingestion, SWE-bench/Hugging Face dataset ingestion, + caller-selected runtime images, benchmark verifiers, and compatibility aliases + remain invalid inside `allagents.workspace`. - Public multi-tenancy, per-caller authorization, Kubernetes workers, session branching, concurrent turns in one session, or guaranteed prompt-cache hits. - Exact rollback of workspace mutations between successful session turns. @@ -1264,7 +1324,7 @@ flowchart TB MAT --> OCI[configured OCI registry] RUN -->|atomic publish or reuse| GEN[(immutable generation store)] GEN -->|read-only mount + reference| RO[read-only session] - GEN -->|private copy| EDIT[editable session] + GEN -->|private writable view| EDIT[editable session] RO --> HARNESS[Selected Codex or Pi harness] EDIT --> HARNESS LIFE[(leases, retention, quotas, GC)] --> RO @@ -1283,7 +1343,7 @@ state; it also owns generic metadata bounds, provider-loop ordering, response metadata, and optional proxy brokering. The runner owns keyed generation claims, the resource journal, hook invocation, independent verification, atomic publication, provisional pins, durable references, read-only mounts, private -editable copies, mode-specific checkpoints, quota admission, deletion, GC, safe +writable views, mode-specific checkpoints, quota admission, deletion, GC, safe cwd, and agent launch. Attachment uses a durable prepare/evidence/ack protocol: the runner prepares resources, the gateway alone commits `ready`, and the runner finalizes or rolls back from that acknowledgement. The selected harness owns @@ -1568,7 +1628,7 @@ digest and every declared semantic Git-state record. Access, retention, cwd, harness/profile, and session identity are not manifest fields and cannot fragment or mutate generation content. The backing tree, including validated `.git` state, becomes owner-writable only and is exposed to sessions solely -through verified read-only mounts or independent private editable copies. +through verified read-only mounts or private writable views. The frozen history-bearing fixture is: @@ -1623,7 +1683,7 @@ error object in `response.error`; workspace failures use | `allagents_workspace_persistence_forbidden` | failed response when `persistent` retention is not authorized for the selected deployment target | no | | `allagents_workspace_read_only` | failed response when a read-only initial request contains workspace input files or attachment policy would create writable shadow state | no | | `allagents_workspace_capacity_exceeded` | HTTP 503 `server_error` before response allocation when generic session/tombstone admission cannot reserve capacity; otherwise a failed response when finite staging, generation, private, session, or persistence capacity cannot be reserved after safe eviction | yes | -| `allagents_workspace_private_quota_exceeded` | failed response when an editable waiter's initial generation copy cannot fit or an attached editable turn exhausts its fixed per-session byte or inode allowance; access and retention remain unchanged | no | +| `allagents_workspace_private_quota_exceeded` | failed response when an editable waiter's private view cannot fit the complete generation or an attached editable turn exhausts its fixed per-session byte or inode allowance; access and retention remain unchanged | no | | `allagents_workspace_source_auth_failed` | failed response for Git or registry credential rejection | no | | `allagents_workspace_acquisition_failed` | failed response when Git, registry, HTTP, or transport I/O prevents complete byte acquisition; excludes digest, schema, and limit failures | no | | `allagents_workspace_limit_exceeded` | failed response for source/archive/manifest repository, entry, byte, layer, file, path, or header limits | no | @@ -1633,8 +1693,8 @@ error object in `response.error`; workspace failures use | `allagents_workspace_manifest_invalid` | failed response for workspace-manifest media type, schema, canonical bytes, or declared digest | no | | `allagents_workspace_integrity_mismatch` | failed response for source descriptor digest/size mismatch, validated/resolved-plan or generation-key drift, staging/manifest mismatch, semantic Git-state failure, or corrupt ready generation | no | | `allagents_workspace_publication_failed` | failed response for generation claim/publication/marker failure | no | -| `allagents_workspace_attachment_failed` | failed response for read-only mount/reference or private editable copy/publication failure | no | -| `allagents_workspace_checkpoint_failed` | failed response for editable root/nested checkpoint or collection-baseline failure | no | +| `allagents_workspace_attachment_failed` | failed response for read-only mount/reference or private writable-view publication failure | no | +| `allagents_workspace_checkpoint_failed` | failed response for editable root/nested checkpoint or protected collection-state failure | no | | `allagents_workspace_state_failed` | failed response for generation/session/pin/reference/quota/expiry/tombstone/purge persistence or CAS failure | no | | `allagents_workspace_containment_breach` | failed response for completed-parent/live-descendant even if forced kill succeeds; an unquiescent leaf remains internal until restart proves it empty | no | @@ -1736,17 +1796,23 @@ into successful empty output and performs no automatic retry. Never attach `building`, quarantined, or deleting state. - **Read-only escape or writable alias:** Keep the generation backing store owner-writable only, verify mount flags and mount topology, forbid writable - bind aliases and hard-linked private copies, and probe writes through root, - nested repositories, symlinks, and alternate paths. -- **Editable cross-session leakage or growth:** Create a unique private tree and - checkpoint namespace per fitting session, verify inode separation, reject only - a waiter whose initial copy cannot fit, reserve its full byte/inode allowance - from global capacity, enforce that hard quota through every continuation, and - scan produced files against only its private baseline. + bind aliases and hard-link aliases, and probe writes through root, nested + repositories, symlinks, and alternate paths. +- **Editable cross-session leakage or growth:** Create a unique private writable + view and checkpoint namespace per fitting session, prove that no mutable state + or writable alias is shared, reject only a waiter whose initial view cannot + fit, reserve its full byte/inode allowance from global capacity, and enforce + that hard quota through every continuation. +- **Produced-file drift or expensive full inventories:** Use the verified + generation as the first-turn baseline and retain only protected cumulative + path state that differs from it. Verify trusted candidates after quiescence; + if candidate state is incomplete, overflowed, or uncertain after recovery, + perform a bounded no-follow full-tree scan. Never trust editable `.git`, + agent-supplied paths, or final ignore rules. - **Nested Git versus mode-specific checkpoints:** Preserve repository `.git` state inside the generation. Read-only sessions do not mutate or checkpoint - it; editable copies ignore declared roots in the HarnessRouter root index and - extend list/file/ack/checkpoint/hydrate across private nested repositories. + it; editable views preserve private Git state for continuation while + collection uses only protected generation and turn state. - **Lease, expiry, and deletion races:** Linearize unexpired-idle or persistent turn admission against tombstoning; hold a provisional pin through attachment prepare/ack; persist exact epoch references before mount exposure; recheck @@ -1767,8 +1833,8 @@ into successful empty output and performs no automatic retry. and persist a ready marker; retry cannot resolve, build, attach, or change source/access/retention/auth mode. - **Source credential leakage:** Use subprocess-only source credentials, - hermetic configuration, leak scans across staging/generations/private copies, - and a non-escapable cgroup boundary proven empty before result handling. + hermetic configuration, leak scans across staging, generations, and private + views, and a non-escapable cgroup boundary proven empty before result handling. - **Native OAuth exposure:** Treat the selected profile as available to its harness and same-identity tools only during an active turn. Use a same-filesystem namespace projection, mount no other profile, never copy it to @@ -1801,9 +1867,9 @@ into successful empty output and performs no automatic retry. 3. Workspace lifecycle fork spike: a fake `preflight/validate/resolve/materialize` hook, concurrent identical epoch claims, independently cancelled waiters, one immutable publication, provisional pins, two read-only mounts, one quota- - bounded private editable copy, per-waiter copy-fit failure, attachment + bounded private writable view, per-waiter view-fit failure, attachment prepare/ack, epoch eviction/republication, and restart reconciliation. -4. Prove read-only enforcement, writable-copy isolation and growth limits, +4. Prove read-only enforcement, writable-view isolation and growth limits, mode-specific checkpoint/collection, nested cwd, provider-fallback non-reentry, and unexpired/persistent continuation reuse. Stop if any invariant needs prompt or client cooperation. @@ -1881,7 +1947,7 @@ into successful empty output and performs no automatic retry. one immutable generation, attach it in both access modes, and reconcile its lifecycle before provider dispatch while preserving stock UHP. - **Repositories/files:** HarnessRouter fork `gateway/app.py`, - `runner/server.py`, generation/session persistence, mount/copy and + `runner/server.py`, generation/session persistence, mount/view and checkpoint/produced-file helpers, runner/gateway tests, and a fake preflight/validate/resolve/materialize hook. - **Approach:** Add opaque metadata bounds, typed operation envelopes, @@ -1889,21 +1955,27 @@ into successful empty output and performs no automatic retry. independent waiter cancellation, separate generation/resource and gateway session CAS state, canonical manifest fixtures, atomic publication, provisional pins, attachment prepare/ack, durable epoch references, verified - read-only mounts, per-waiter copy-fit and unique hard-quota-bounded editable - copies, mode-specific checkpoint/collection, safe nested cwd, stage-dependent - response metadata, and cgroup containment. Add fake finite TTL, persistence, - tombstone, quota, deletion, and epoch-republication state sufficient to prove - restart ordering; U3 completes production policy and GC. + read-only mounts, per-waiter fit checks and unique hard-quota-bounded private + writable views, mode-specific checkpoints, protected sparse collection state, + candidate verification with full-scan fallback, safe nested cwd, + stage-dependent response metadata, and cgroup containment. Add fake finite TTL, + persistence, tombstone, quota, deletion, and epoch-republication state + sufficient to prove restart ordering; U3 completes production policy and GC. - **Verification:** Upstream UHP conformance stays green. Two concurrent identical read-only initial requests execute fake materialize once, attach the same generation under separate UIDs and harness/profile bindings, deny writes through root/nested/symlink/alternate paths, and isolate runtime state. Two - editable sessions receive inode-independent private trees; one mutation and - checkpoint never appears in the other or generation. Continuation reuses its - original mode and state without the extension. + editable sessions receive private writable views; one mutation and checkpoint + never appears in the other or generation. Continuation reuses its original + mode and state without the extension. A large clean fixture creates no + redundant pre-agent inventory. Candidate collection and forced full-scan + fallback produce the same per-turn source-visible delta. Both exclude changes + to the protected declared `.git` subtrees while counting source-visible ignore + files and an agent-created `.git` elsewhere; a coverage gap before a UHP input + overlay forces the full scan. Fault every validate/resolve/claim/waiter/containment/publication/pin/ - prepare/ready-ack/reference/mount/copy/quota/checkpoint/CAS boundary. The runner + prepare/ready-ack/reference/mount/view/quota/checkpoint/CAS boundary. The runner rejects forged manifests, changed staging, escaping links, invalid repository destinations, writable aliases, and generation-key drift. Restart exposes only a complete publication plus valid attachment evidence; provider fallback never @@ -1984,7 +2056,7 @@ into successful empty output and performs no automatic retry. access, retention, expiry, attachment evidence, and auth binding. Implement active leases, terminal-time idle expiry, bounded tombstones and purge, authorized persistent pins, provisional attachment pins, fixed private - byte/inode reservations and runtime enforcement, per-waiter initial copy fit, + byte/inode reservations and runtime enforcement, per-waiter initial view fit, editable cleanup, read-only epoch-reference release, deterministic eviction of ready zero-reference/zero-pin epochs, completed-eviction fencing before republication, deletion quarantine, and startup reconciliation. Extend every @@ -2019,7 +2091,7 @@ into successful empty output and performs no automatic retry. capacity returns the cataloged retryable failure. Crash every generation-epoch/session/build-waiter/pin/prepare/ready-ack/ - reference/mount/copy/quota/tombstone/unmount/purge/delete transition and require + reference/mount/view/quota/tombstone/unmount/purge/delete transition and require reconciliation before readiness or GC. Continuation succeeds only for valid exact-epoch evidence whose retention is persistent or session idle deadline is unexpired; retained expiry returns 410, corrupt evidence returns 409 @@ -2048,9 +2120,10 @@ into successful empty output and performs no automatic retry. detached `HEAD`, index/tree equality, exact object closure and digest, exact worktree/commit equality, closed refs/config, and the absence of remotes, credentials, and unsafe administrative state. Reject `.git` in tree-only or - undeclared locations. A runnable Harbor or SWE-bench instance image still - requires an explicit adapter/transform and is never relabeled as a workspace - source snapshot. The runner remains the sole publisher/resource preparer and + undeclared locations. Runnable Harbor and SWE-bench environments remain + outside `allagents.workspace`; Promptfoo invokes Harbor through its separate + provider lane. They are never relabeled as workspace source snapshots. The + runner remains the sole publisher/resource preparer and the gateway the sole session-attachment writer. - **Verification:** Distribution fixtures cover exact request-field naming, `snapshotName` lookup, direct `imageManifestDigest` enforcement, workspace- @@ -2065,8 +2138,8 @@ into successful empty output and performs no automatic retry. foreign media, traversal, links, devices, sparse files, cancellation, cleanup, no Git fallback, and exact error precedence. Concurrent identical OCI requests produce one publication; read-only sessions share it; editable sessions get - private copies; access, retention, cwd, harness/profile, and session do not - fragment its generation key. + private writable views; access, retention, cwd, harness/profile, and session + do not fragment its generation key. ### U5. Harness-native OAuth, optional proxy, and Promptfoo E2E @@ -2091,7 +2164,7 @@ into successful empty output and performs no automatic retry. Same-profile cross-session overlap retains the cataloged fail-fast result; same-session overlap returns `session_busy`; idempotent duplicates share one admission/result. Editable turn two sees turn one's mutation; a different trial - sees a clean private copy. + sees a clean private writable view. Real-image lifecycle probes cover terminal-time TTL, retained-expiry HTTP 410, purged-predecessor stock failure, HTTP 409 non-resumable, authorized @@ -2152,7 +2225,8 @@ into successful empty output and performs no automatic retry. | Shared-build cancellation | One request cancellation/deadline detaches only that waiter. A build continues for remaining live waiters, stops when none remain or its runner-owned deadline expires, and produces at most one publication/failure for its epoch. | | Manifest integrity | Git and OCI share one source-visible schema with pairwise non-overlapping repository destinations. A root may omit `.git` only when its manifest item declares history and the runner validates the detached commit, exact index/tree and object set, exact source-visible worktree, closed configuration and refs, and safe administrative state. Tree-only and undeclared `.git` fail. Git-acquired content equals the union of resolved commit trees at their destinations plus necessary ancestors. Plan/key drift, undeclared paths, forged manifests, changed staging, invalid paths/types/links/destinations, semantic Git mismatch, and digest mismatch fail before publication. | | Shared read-only generation | Concurrent sessions using different harnesses/profiles share one exact generation epoch. Root, nested, symlink, and alternate-path writes fail; runtime/session/auth/output state remains isolated. | -| Editable isolation | Every fitting editable trial receives an inode-independent private tree and reserved hard byte/inode allowance covering overlays/checkpoints/produced state. A non-fitting waiter fails alone; continuation preserves a fitting trial's mutations but cannot grow past its envelope; siblings and the generation remain unchanged. | +| Editable isolation | Every fitting editable trial receives a private writable view with no mutable state shared with the generation or another session, plus a reserved hard byte/inode allowance covering overlays, checkpoints, and produced state. A non-fitting waiter fails alone; continuation preserves a fitting trial's mutations but cannot grow past its envelope. | +| Produced-file integrity | A clean first turn creates no redundant full-workspace inventory. Candidate tracking is durably active before input overlays or writable process exposure and remains active through quiescence; a missing or discontinuous coverage marker forces the bounded no-follow full scan. Candidate and full-scan paths produce the same source-visible additions, deletions, type/mode changes, and content changes. Both exclude only the declared Git administrative subtrees recorded by the protected generation, report an agent-created `.git` elsewhere as ordinary content, and ignore final Git discovery, ignore rules, and agent-supplied path lists. Continuation derives its delta from protected prior turn state. | | Materializer containment | Fork/double-fork/cancellation/deadline fixtures prove `populated 0` before result read, publication, secret release, or cleanup. `containment_pending` blocks terminal visibility/readiness through restart and resolves once after quiescence. | | Capacity envelope | Native profiles retain one active turn and zero waiters. Source build limits and finite staging/generation/private-byte/private-inode/session/persistence/tombstone quotas reject overflow. Invalid descriptors cannot bypass generic admission; one editable session creates one private debit; successful publication releases staging capacity. References and provisional pins prevent eviction; all-protected capacity returns the cataloged retryable failure. | | Durable lifecycle | Fault injection covers generic and provisional turn admission, active leases, generation epochs, build/staging/generation reservations, publication/accounting conversion, build waiters, provisional pins, attachment prepare/ready-ack and private-reservation transfer, references, mounts, private usage, expiry, tombstones/purge, unmount, deletion, quarantine, and GC. Startup reconciles before readiness; no deadline extends, no debit duplicates/leaks, no second epoch appears before prior eviction completes, and no session silently rematerializes. | @@ -2160,7 +2234,7 @@ into successful empty output and performs no automatic retry. | Session continuity | Both modes preserve conversation and fixed generation key/epoch/access/retention/cwd/harness/auth binding while persistent or unexpired; editable preserves private files; read-only remains immutable. Corrupt known evidence returns HTTP 409 non-resumable with no source access or later-epoch substitution. | | Git acquisition | Caller-supplied canonical HTTPS URLs, public-address egress enforcement, DNS-rebinding and redirect defense, structured-scope credential isolation, constrained refs, exact commits, closed transport/config, exact object closure/index semantics, generation reuse, and partial cleanup pass against local network fixtures. | | OCI acquisition | Digest/media/path/link/type/limit checks, tree-only and normalized offline-history fixtures, producer removal and materializer rejection of remotes and credentials, semantic Git verification, generation reuse, and the attachment matrix pass against a local registry. | -| Credential boundary | Preflight sees no secret values and returns bounded configured reference identities; validate selects a bounded subset; the runner verifies handles and injects only that selected set into source-access children. Source secrets and caller keys are absent from staging, generations, private trees, base environments, checkpoints, backups, logs, and output. The selected OAuth profile is visible only through its active-turn projection, which is absent before acknowledgement and after restart reconciliation. | +| Credential boundary | Preflight sees no secret values and returns bounded configured reference identities; validate selects a bounded subset; the runner verifies handles and injects only that selected set into source-access children. Source secrets and caller keys are absent from staging, generations, private views, base environments, checkpoints, backups, logs, and output. The selected OAuth profile is visible only through its active-turn projection, which is absent before acknowledgement and after refresh finalization, including failure, cancellation, restart, and continuation. Same-profile overlap fails before allocation; the active harness and same-identity tools remain an explicit owner-trust boundary. | | Provider boundary | Codex/Pi native OAuth, refresh repair, projection teardown, idempotency/session/profile admission, different-profile concurrency, same-profile fail-fast exclusion, and explicit proxy scope all pass without implicit switching. | | Packaging | The public GHCR digest and provenance/SBOM attestations verify exact inputs; deployment uses that digest and finite lifecycle configuration. | | Consumer | Promptfoo concurrent/one-shot/two-turn/lifecycle success and every cataloged or UHP terminal failure map exactly. Active streams expose null expiry; terminal/GET/replay expose one stable expiry. Failures before attachment ready omit workspace metadata; later terminal failures include the complete public object. None becomes empty success or automatic retry. | @@ -2195,11 +2269,13 @@ into successful empty output and performs no automatic retry. - Concurrent read-only sessions with different harness/profile bindings share generation bytes but no mutable runtime, auth, conversation, output, or lifecycle state. Filesystem probes prove no writable path or copy-up. -- Each fitting editable trial has a private writable tree with no mutable inode +- Each fitting editable trial has a private writable view with no mutable state shared with the generation or another session. Its reserved hard byte/inode allowance covers every turn, overlay, checkpoint, and produced-file record. - Non-fitting waiters fail independently. Continuation preserves only its own - mutations and produced-file history. + Non-fitting waiters fail independently. Collection uses the verified generation + plus protected sparse turn state, produces the same delta through candidate and + full-scan paths, and never trusts editable Git metadata. Continuation preserves + only its own mutations and produced-file history. - Generation publication and every validate/resolve/materialize result remain behind cgroup quiescence, exact commit-tree/source-visible manifest and semantic Git validation, full physical accounting, atomic reservation conversion, From 5c3fb790c1985c7dd1486205d2859b9a1b1be6ae Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Thu, 24 Sep 2026 16:21:51 +1000 Subject: [PATCH 26/44] docs(architecture): explain OAuth renewal locking --- .../0002-adopt-uhp-through-harnessrouter.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 6ba6efcf..a90e7804 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -87,8 +87,8 @@ Each target must prove all eight behaviors: 2. A real first turn and continuation succeed without a provider-route API key. 3. The selected authentication binding survives restart and fails closed when unavailable. 4. Session conversation state remains separate while only the selected profile is visible. -5. Overlapping refresh-capable turns for one profile are serialized. -6. Credential files remain complete before, during, and after refresh; invalid post-rotation state becomes `repair-required`. +5. Two turns cannot use the same profile at once, because either may renew and replace its OAuth credentials. +6. Credential files remain complete before, during, and after renewal; an invalid saved update makes the profile `repair-required`. 7. Success, failure, cancellation, and crash recovery remove the active projection. Credentials remain absent from retained homes, checkpoints, produced-file records, backups, passive logs, and response metadata. 8. The evidence explicitly records that the selected harness and same-identity tools can read or emit the credential during an active turn. @@ -632,20 +632,24 @@ target. HarnessRouter does not switch to another profile or provider route. Native OAuth uses an owner-trust boundary. During an active turn, the selected harness and same-operating-system-identity tools may read or emit that profile's credential. Operators that require stronger isolation must use the explicit proxy route or isolate the whole deployment more strongly. +During a turn, Codex or Pi may renew an expired OAuth token and replace the +profile's stored credentials. If two turns did that at once, one could overwrite +the other's update. + Login, logout, and repair acquire the same runner-owned zero-waiter profile lock as an active turn. They use the same durable fence and `finally` release and acknowledgement protocol. A native turn follows this order: -1. Acquire the runner-owned profile lock. Version one allows exactly one active refresh-capable turn per profile and no waiters. +1. Acquire the runner-owned profile lock. Version one allows one active turn per profile. A second turn fails immediately instead of waiting. 2. Project only the selected profile through a turn-scoped mount namespace or equivalent same-filesystem view that preserves native atomic file replacement. 3. Run the harness and descendants. -4. Commit or reject refresh state after descendants stop. +4. After descendants stop, validate any renewed credentials and either save or reject the update. 5. Remove the projection and verify the retained session home is clean. 6. Persist terminal acknowledgement, then release the profile lock. -A local refresh commit uses a same-filesystem temporary file, file `fsync`, atomic rename, parent-directory `fsync`, and validation. If a crash after provider rotation leaves invalid local state, restart marks the profile `repair-required` and requires native login again. It never switches profiles or activates the proxy. +When the harness renews credentials, the runner writes them through a same-filesystem temporary file, file `fsync`, atomic rename, parent-directory `fsync`, and validation. If the provider issued a replacement token but a crash leaves invalid local state, restart marks the profile `repair-required` and requires native login again. It never switches profiles or activates the proxy. HarnessRouter claims each `Idempotency-Key` atomically. Requests with the same key share one result. @@ -804,7 +808,7 @@ The maintained fork must be rebased and tested against selected upstream release ## Deliberate limits -Version one does not add evaluation datasets, Harbor task ingestion, SWE-bench/Hugging Face ingestion, caller-selected runtime images or verifiers, scoring, assertions, automatic retries, session branching, concurrent turns within one session, simultaneous refresh-capable turns for one native profile, caller-supplied credentials, non-HTTPS or private-network Git origins, public multi-tenancy, arbitrary materializer commands, mutable OCI tags, transparent source-mode fallback, or guaranteed provider prompt-cache hits. +Version one does not add evaluation datasets, Harbor task ingestion, SWE-bench/Hugging Face ingestion, caller-selected runtime images or verifiers, scoring, assertions, automatic retries, session branching, concurrent turns within one session, two turns using the same native authentication profile at once, caller-supplied credentials, non-HTTPS or private-network Git origins, public multi-tenancy, arbitrary materializer commands, mutable OCI tags, transparent source-mode fallback, or guaranteed provider prompt-cache hits. Read-only attachments never copy up or become editable. Editable sessions never share mutations. Callers cannot choose arbitrary TTLs, bypass persistence quotas, or change retention on continuation. Leased, referenced, or pinned state is never evicted. Default retention is always bounded. From cc21dca4a86b18dc4e7ccb2cd48623898fbeb8e1 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sun, 27 Sep 2026 16:37:41 +1000 Subject: [PATCH 27/44] docs(architecture): allow concurrent profile turns --- .../0002-adopt-uhp-through-harnessrouter.md | 134 +++-- ...0837-feat-coding-execution-gateway-plan.md | 509 ++++++++++-------- 2 files changed, 391 insertions(+), 252 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index a90e7804..734f8c17 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -81,16 +81,17 @@ Freeze this evidence set: Changing any input invalidates the evidence. Dependent work remains blocked until both native targets pass again. -Each target must prove all eight behaviors: +Each target must prove all nine behaviors: 1. The operator can complete native login in a controlled environment. 2. A real first turn and continuation succeed without a provider-route API key. 3. The selected authentication binding survives restart and fails closed when unavailable. 4. Session conversation state remains separate while only the selected profile is visible. -5. Two turns cannot use the same profile at once, because either may renew and replace its OAuth credentials. -6. Credential files remain complete before, during, and after renewal; an invalid saved update makes the profile `repair-required`. -7. Success, failure, cancellation, and crash recovery remove the active projection. Credentials remain absent from retained homes, checkpoints, produced-file records, backups, passive logs, and response metadata. -8. The evidence explicitly records that the selected harness and same-identity tools can read or emit the credential during an active turn. +5. Two real turns using the same profile and a valid token succeed concurrently. +6. When two turns start with an expired token, renewal is coordinated before either calls the provider: one renews, the other rereads the saved update, both succeed, and the credential file remains valid. +7. Credential files remain complete through renewal and injected crashes; an invalid saved update makes the profile `repair-required`. +8. Success, failure, cancellation, and crash recovery remove every active projection. Credentials remain absent from retained homes, checkpoints, produced-file records, backups, passive logs, and response metadata. +9. The evidence explicitly records that the selected harness and same-identity tools can read or emit the credential during an active turn. The proxy route cannot satisfy this gate. Failure of either native target stops dependent implementation. A proxy-only release or narrower harness scope requires a new decision. @@ -550,9 +551,15 @@ Missing or corrupt generation, reference, publication, private workspace, or che Every active operation holds a durable lease and has no idle expiry. -One gateway compare-and-swap checks `session_busy`, exact binding, and expiry or deletion together. A busy or invalid turn changes no deadline. +One gateway compare-and-swap checks `session_busy`, exact binding, and expiry or +deletion together. A busy or invalid turn changes no deadline. -For an eligible continuation, the gateway saves and clears the current idle deadline in a provisional admission fence before the runner attempts to acquire the selected native profile. Profile success commits the session as active. A pre-allocation profile failure restores the exact saved deadline when it is still future, or tombstones the session if that deadline elapsed. +For an eligible continuation, the gateway saves and clears the current idle +deadline in a provisional admission fence before the runner checks profile +readiness and reserves one configured concurrent-turn slot for that profile. +Success commits the session as active. A pre-allocation profile failure restores +the exact saved deadline when it is still future, or tombstones the session if +that deadline elapsed. After terminal acknowledgement, a `session` workspace receives one idle deadline. GET, polling, background completion, and replay never extend it. Persistent sessions keep `expiresAt: null`. @@ -565,6 +572,7 @@ Every deployment limit must be finite and nonzero: | Sessions | Total active and retained sessions | | Failed identity | Tombstone count, bytes, and TTL | | Builds | Concurrent builds and staging bytes | +| Native authentication | At least two concurrent turns per profile; finite renewal timeout | | Generations | Published count and bytes | | Editable workspaces | Per-session hard bytes and inodes | | Private storage | Total reserved bytes and inodes | @@ -618,49 +626,98 @@ An internal `containment_pending` session remains non-terminal until its recorde |---|---|---| | Default | Yes | No; explicit configuration only | | Provider credential owner | Codex or Pi harness profile | Proxy service | -| Harness receives | Selected turn-scoped profile projection | Non-refreshable scoped turn credential | -| Refresh | Harness-native | Not allowed for the turn credential | +| Harness receives | Selected turn-scoped view of the shared profile | Non-refreshable scoped turn credential | +| Refresh | Harness-native through the per-profile renewal coordinator | Not allowed for the turn credential | | Automatic fallback | Never | Never | Promptfoo's HarnessRouter API key authenticates the UHP caller only. HarnessRouter never translates it into provider credentials. -Each native harness target has one dedicated durable authentication root outside generations, editable workspaces, session checkpoints, and conversation state. Codex uses file credential storage under `CODEX_HOME`. Pi uses `~/.pi/agent/auth.json` after controlled `/login`. - -Missing, expired, revoked, or unrefreshable native OAuth disables that harness -target. HarnessRouter does not switch to another profile or provider route. +Each configured native profile has one dedicated durable authentication root +outside generations, editable workspaces, session checkpoints, and conversation +state. Compatible harness targets may reference the same profile. Codex uses +file credential storage under `CODEX_HOME`. Pi uses +`~/.pi/agent/auth.json` after controlled `/login`. +Missing, revoked, or unrefreshable native OAuth disables that harness target. An +expired but refreshable token is renewed through the coordinator. HarnessRouter +does not switch to another profile or provider route. Native OAuth uses an owner-trust boundary. During an active turn, the selected harness and same-operating-system-identity tools may read or emit that profile's credential. Operators that require stronger isolation must use the explicit proxy route or isolate the whole deployment more strongly. -During a turn, Codex or Pi may renew an expired OAuth token and replace the -profile's stored credentials. If two turns did that at once, one could overwrite -the other's update. - -Login, logout, and repair acquire the same runner-owned zero-waiter profile lock -as an active turn. They use the same durable fence and `finally` release and -acknowledgement protocol. +Multiple turns may use one native profile at the same time. Reading a valid token +does not require a lock. Renewal does, because two provider refresh calls using +the same old token can invalidate or overwrite each other. + +The pinned auth adapter remembers which credential contents a turn used and must +enter the runner-owned renewal coordinator before calling the provider's refresh +endpoint. Waiting for the renewal lock stops at the turn deadline. Once the +coordinator records a pending renewal, it—not the turn—owns the transaction: + +1. Acquire the renewal lock and reread the current credential file. +2. If its contents changed since the turn last read them, reload the saved + credential, release the lock, and skip the provider refresh call. +3. Otherwise, record the renewal attempt before the provider call. +4. Renew once and replace the credential file with a same-filesystem temporary + file, file `fsync`, atomic rename, parent-directory `fsync`, and validation. +5. Mark the renewal complete and release the lock as soon as the valid credential + is visible. + +Cancellation or the turn deadline cannot release a recorded renewal. The +coordinator uses its own finite renewal timeout. On timeout or process failure, +it stops the refresh process, waits until that process tree is gone, marks the +profile `repair-required`, and only then releases the lock. The turn cannot reach +terminal acknowledgement before that outcome is durable. + +A turn must never call the provider's refresh endpoint outside that coordinator. +If the pinned Codex or Pi version cannot acquire the renewal lock before calling +the provider, that target fails the phase-zero gate. + +A `repair-required` profile accepts no new turns or renewal attempts. A turn +already running may finish if its current access token still works; otherwise it +returns the normal provider-authentication failure. It never switches profiles or +activates the proxy. + +Login, logout, and repair use a separate durable maintenance fence. Its +`pending` state stops new admissions and waits for active turns to finish. While +the fence is pending or active, a new turn receives the retryable +`allagents_auth_profile_unavailable` error. If the operator deadline expires +before maintenance starts, the runner removes the pending fence and normal +admission resumes. + +Once maintenance becomes `active`, timeout or cancellation requests process +termination but never releases the fence. The runner waits for the process tree +to stop, validates the profile or marks it `repair-required`, records the +outcome, and only then resumes admission. Startup reconciles every pending or +active maintenance record before that profile becomes ready. A native turn follows this order: -1. Acquire the runner-owned profile lock. Version one allows one active turn per profile. A second turn fails immediately instead of waiting. -2. Project only the selected profile through a turn-scoped mount namespace or equivalent same-filesystem view that preserves native atomic file replacement. -3. Run the harness and descendants. -4. After descendants stop, validate any renewed credentials and either save or reject the update. +1. Reserve one configured concurrent-turn slot for the profile. +2. Project only that profile through a turn-scoped mount namespace or equivalent + same-filesystem view. Concurrent turns see the same atomically replaced + credential file. +3. Run the harness and descendants. The auth adapter coordinates renewal only + when needed. +4. After descendants stop, verify that turn has no incomplete renewal record. + Reconcile uncertain state or mark the profile `repair-required`. 5. Remove the projection and verify the retained session home is clean. -6. Persist terminal acknowledgement, then release the profile lock. - -When the harness renews credentials, the runner writes them through a same-filesystem temporary file, file `fsync`, atomic rename, parent-directory `fsync`, and validation. If the provider issued a replacement token but a crash leaves invalid local state, restart marks the profile `repair-required` and requires native login again. It never switches profiles or activates the proxy. +6. Persist terminal acknowledgement, then release the profile turn slot. HarnessRouter claims each `Idempotency-Key` atomically. Requests with the same -key share one result. +key share one result. Same-session overlap still returns `session_busy`. -Different profiles may run concurrently on one generation. A new cross-session turn that collides on a busy profile fails immediately before response allocation with HTTP 503 `harness_unavailable` and reason `allagents_auth_profile_busy`. Stock idempotent replay and same-session `session_busy` take precedence. +Different sessions may run concurrently with the same or different profiles. +Each profile has a finite concurrent-turn limit of at least two. Saturation fails +before response allocation with HTTP 503 `harness_unavailable` and reason +`allagents_auth_profile_capacity_exceeded`. -The runner supervisor holds turn admission and the profile lock through -descendant termination, refresh disposition, projection teardown, and terminal -acknowledgement. Gateway failure cannot release them. Runner failure leaves a -durable fence. Startup blocks readiness and profile admission until it reconciles -that fence and every stale projection. +The runner supervisor holds each turn's admission token and credential projection +through descendant termination, credential validation, projection teardown, and +terminal acknowledgement. It holds renewal ownership from the durable pending +record through commit or a durable `repair-required` fence. Gateway failure +cannot release the turn token. Runner failure leaves durable state. Startup +blocks that profile's admission until it reconciles every turn token, renewal +record, maintenance record, and stale projection. In proxy mode, the gateway issues a non-refreshable credential bound to one proxy audience, harness target, model allowlist, response and turn ID, and the UHP @@ -766,13 +823,14 @@ The system fails closed. Source, access mode, retention, credentials, and provid | Expired or deleted retained session | HTTP 410 `allagents_workspace_expired` | No runner or profile work; no rematerialization | | Purged predecessor | Stock non-disclosing unknown-predecessor error | No rematerialization | | Missing or corrupt bound attachment evidence | HTTP 409 `allagents_workspace_non_resumable` | No profile admission or epoch substitution; return committed workspace metadata | -| Busy native profile after replay and `session_busy` checks | HTTP 503 `harness_unavailable`, reason `allagents_auth_profile_busy` | Fail before allocation, runner work, or materialization | +| Native profile reaches its configured concurrent-turn limit after replay and `session_busy` checks | HTTP 503 `harness_unavailable`, reason `allagents_auth_profile_capacity_exceeded` | Fail before allocation, runner work, or materialization | +| Native profile is unavailable, `repair-required`, or under maintenance | HTTP 503 `harness_unavailable`, reason `allagents_auth_profile_unavailable` | Fail before allocation without switching profile or activating the proxy | | Generic capacity unavailable | HTTP 503 `allagents_workspace_capacity_exceeded` | Admit no response or source work | | Editable view or later growth exceeds its allowance | `allagents_workspace_private_quota_exceeded` | Fail only that waiter or turn; preserve mode and retention | | Source policy, authentication, or transport failure | Coded failed response | Remove unpublished staging; publish nothing; start no agent or provider fallback | | Generation, attachment, or private-view failure | Coded failed response | Quarantine incomplete state and release reservations and pins exactly once | | Materializer timeout, crash, malformed output, or live descendant | Coded materializer or containment failure | Wait for cgroup quiescence before result handling, secret release, or cleanup | -| Provider authentication failure | Normalized UHP failure | Do not switch profile or activate proxy; finish teardown before lock release | +| Provider authentication failure | Normalized UHP failure | Do not switch profile or activate proxy; validate credentials, remove the projection, and release turn capacity | | Provider execution failure | Normalized UHP failure | No source fallback and no credential material in output | Capacity is reserved in order: generic session and tombstone before response visibility; persistence and editable allowance after validation and before source resolution; staging and prospective generation after resolution and before byte acquisition; actual retained usage before publication. Each reservation is released or transferred exactly once. @@ -785,7 +843,7 @@ New vendor codes use the `allagents_` prefix. Promptfoo maps every non-success t HarnessRouter remains the sole execution and session control plane. AllAgents adds workspace preparation and source policy without adding another streaming API, process supervisor, artifact service, provider adapter, or task engine. -Shared read-only generations avoid repeated acquisition and may serve different harnesses and profiles concurrently. Editable sessions trade that reuse for a reserved private byte and inode envelope. +Shared read-only generations avoid repeated acquisition and may serve concurrent sessions using the same or different harness profiles. Editable sessions trade that reuse for a reserved private byte and inode envelope. Persistent sessions, active references, provisional pins, retained tombstones, and quarantined deletion failures consume finite capacity. Crash-consistent accounting and admission rejection are operational requirements, not optional optimizations. @@ -808,7 +866,7 @@ The maintained fork must be rebased and tested against selected upstream release ## Deliberate limits -Version one does not add evaluation datasets, Harbor task ingestion, SWE-bench/Hugging Face ingestion, caller-selected runtime images or verifiers, scoring, assertions, automatic retries, session branching, concurrent turns within one session, two turns using the same native authentication profile at once, caller-supplied credentials, non-HTTPS or private-network Git origins, public multi-tenancy, arbitrary materializer commands, mutable OCI tags, transparent source-mode fallback, or guaranteed provider prompt-cache hits. +Version one does not add evaluation datasets, Harbor task ingestion, SWE-bench/Hugging Face ingestion, caller-selected runtime images or verifiers, scoring, assertions, automatic retries, session branching, concurrent turns within one session, caller-supplied credentials, non-HTTPS or private-network Git origins, public multi-tenancy, arbitrary materializer commands, mutable OCI tags, transparent source-mode fallback, or guaranteed provider prompt-cache hits. Read-only attachments never copy up or become editable. Editable sessions never share mutations. Callers cannot choose arbitrary TTLs, bypass persistence quotas, or change retention on continuation. Leased, referenced, or pinned state is never evicted. Default retention is always bounded. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index d75da5a1..f2305ea5 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -57,8 +57,9 @@ execution: code Promptfoo E2E; complete release, fork-maintenance, and upstream documentation. - **Stop conditions:** Stop before production workspace implementation if either required native target cannot pass the phase-zero gate: real login, first turn, - continuation, binding persistence, profile isolation, mutually exclusive - overlapping turns, refresh fault behavior, and passive-persistence checks. + continuation, binding persistence, profile isolation, concurrent same-profile + turns with valid and expired tokens, coordinated renewal, refresh fault + behavior, and passive-persistence checks. Also stop if the host cannot enforce immutable multi-session read-only mounts; concurrent identical requests can publish more than one generation; editable sessions can alias writable state; continuation, lease, expiry, deletion, and @@ -144,10 +145,11 @@ caller responsible for acquisition. The temporary fork closes those seams. cleanup, and garbage collection. It prepares attachment evidence but never writes gateway session attachment/expiry/tombstone transitions. - **A5. Harness-native auth profile:** One dedicated durable credential root for - one Codex or Pi harness target. The harness owns login and token refresh. The - runner projects it only for an active turn and verifies teardown before - acknowledgement; checkpoints, backups, and public metadata never copy it. - Active harness access is part of the owner-trust boundary. + one Codex or Pi harness target. Multiple turns may use it concurrently. The + harness owns login and token refresh; the pinned auth adapter coordinates only + renewal. The runner projects the profile only for an active turn and verifies + teardown before acknowledgement. Checkpoints, backups, and public metadata + never copy it. Active harness access is part of the owner-trust boundary. - **A6. Optional authenticated proxy:** A last-resort, explicitly configured target mode. The gateway keeps the long-lived proxy client key, the proxy owns upstream provider authentication, and the harness receives only a @@ -245,82 +247,108 @@ caller responsible for acquisition. The temporary fork closes those seams. `metadata.harness_id` and a model with `model`; AllAgents profiles are not projected into this catalog. Codex and Pi targets default to `nativeOAuth`. A target is advertised only after its selected auth binding passes: profile - login/refresh/live-turn checks for native OAuth, or schema, TLS, broker, - model-map, endpoint, and live compatibility checks for `proxyApiKey`. On the - first turn, the gateway persists the harness target, auth mode, binding + login, concurrent renewal, and live-turn checks for native OAuth, or schema, + TLS, broker, model-map, endpoint, and live compatibility checks for + `proxyApiKey`. On the first turn, the gateway persists the harness target, + auth mode, binding identity, and canonical binding-config digest in the session. Continuations require that exact binding; deployment config changes never switch it. - **R4.** Add a durable auth root outside session workspaces with one - least-access profile directory per harness target. Controlled setup runs - `CODEX_HOME= codex login` with file credential storage for Codex or - runs Pi `/login` in an isolated Pi home for the configured provider. The - runner projects only the selected profile's exact auth files into the - session-specific CLI home and keeps conversation/rollout state session-scoped. - The projection exists only for the active turn. It uses a directory-level - mount namespace or an equivalently isolated same-filesystem view that preserves - the harness's credential-file write and atomic-replacement behavior; it never - copies credentials into durable session state. - - A locally committed refresh uses a same-filesystem temporary file, file - `fsync`, atomic rename, parent-directory `fsync`, and validation. A crash after - the provider rotates credentials but before local commit can leave the profile - stale; restart then marks it `repair-required` and requires native login - instead of changing profile or auth mode. After the harness and descendants - stop, but before terminal acknowledgement or profile-lock release, the runner - commits or rejects refresh state, unmounts and removes the projection, and - verifies that credential paths are absent from the retained CLI home. Startup - removes or quarantines stale projections before readiness or profile - reacquisition. Success, failure, cancellation, crash repair, expiry, deletion, - checkpoint, backup, and produced-file paths all preserve this boundary. - - The gateway/runner must not automatically serialize auth files into root or - nested checkpoints, produced-file records, passive logs/traces, materializer - input, backup, or response metadata. Native mode sets explicit owner trust - because the harness and tool subprocesses sharing its operating-system - identity may read or emit that credential. In `nativeOAuth`, version one - supports exactly one active refresh-capable turn per profile and holds that - profile lock for every turn and every login, logout, or repair operation. - Admission preserves UHP precedence with an atomic `Idempotency-Key` claim - around lookup and admission. The single claim owner proceeds; simultaneous - same-key arrivals wait on that claim and receive the owner's result without a - second profile-lock attempt. If the owner fails before response allocation, - the gateway publishes that same request error to current waiters and removes - the claim so a later retry can try again. A new turn in an already-active - session returns `session_busy`; only then does a genuinely new executable turn - try the native profile lock. Cross-session collision returns HTTP 503 - `harness_unavailable` with - `detail.reason: "allagents_auth_profile_busy"` before response allocation, - runner work, or materialization. No per-profile waiter queue exists; the UHP - idempotency claim wait is part of one logical request, not such a queue. - - For a continuation, the gateway's one session CAS checks `session_busy`, - attachment/binding evidence, and expiry/deletion together. On success it - records the original idle deadline in a provisional turn-admission fence, - clears that deadline, and marks the session admission-pending before the runner - tries the zero-waiter profile lock. A same-session collision changes nothing. - If profile admission fails before response allocation, the gateway rolls the - fence back: it restores the exact original deadline when still future, or - tombstones the session when that deadline has elapsed. Only a returned profile - admission token commits the fence to active. - - Admission is a private runner operation: the runner turn supervisor persists - an active-profile admission record, takes the operating-system advisory lock, - and returns an opaque admission token before the gateway allocates a response. - The runner—not the gateway—owns that lock through descendant termination, - refresh commit, and projection teardown. A gateway-only crash therefore leaves - the lock held; restart reconciles the token and active runner before admitting - another turn. The runner releases only after the gateway acknowledges durable - terminal response/state, and after the runner has committed native refresh - state or marked the profile `repair-required` and proved the projection absent. - If the runner process dies, the OS releases the lock, but its durable admission - record keeps readiness/admission closed until startup proves all descendant - boundaries empty, removes stale projections, and validates or repairs the - profile. Ordinary and administrative paths use one `finally` release/ack - protocol. `proxyApiKey` uses no native profile lock. Operators provision - distinct native profiles when they require parallel turn capacity. Preserve - HarnessRouter streaming, cancellation, idempotency, files, artifacts, - retention-bounded completed session persistence, per-session UID/runtime - isolation, and immutable generation sharing. + least-access directory per configured native profile. A profile belongs to one + harness type and provider, but compatible targets may reference it. Controlled + setup runs `CODEX_HOME= codex login` with file credential storage for + Codex or runs Pi `/login` in an isolated Pi home for the configured provider. + + The runner projects only the selected profile's exact auth files into each + session-specific CLI home. Conversation and rollout state remain + session-scoped. The projection exists only for the active turn and uses a + directory-level mount namespace or equivalent same-filesystem view. Concurrent + turns using one profile see the same atomically replaced credential file; no + credential is copied into durable session state. + + Each native profile has a finite concurrent-turn limit of at least two. A value + below two is invalid configuration. Admission + preserves UHP precedence: atomically claim the `Idempotency-Key`, return + `session_busy` for a second turn in one session, then reserve a profile turn + slot. Same-key arrivals share the owner's result without a second reservation. + Different sessions may reserve slots on the same profile. Saturation fails + before response allocation with HTTP 503 `harness_unavailable` and + `detail.reason: "allagents_auth_profile_capacity_exceeded"`. + + For a continuation, one session CAS checks `session_busy`, attachment and + binding evidence, and expiry or deletion. It then saves and clears the original + idle deadline in a provisional fence before profile readiness and capacity + admission. Admission success commits the session as active. Pre-allocation + failure restores the exact future deadline or tombstones the session if it + elapsed. + + Reading a valid credential needs no profile-wide lock. When the pinned auth + adapter loads a credential, it records a private version equal to the SHA-256 + digest of the exact credential-file bytes. The digest remains inside the auth + coordinator and is never logged or returned. + + Waiting to acquire the runner-owned renewal mutex is bounded by the requesting + turn's deadline. After acquisition, the coordinator rereads the profile: + + 1. If the current version differs from the adapter's base version, reload the + saved credential, release the mutex, and skip the provider refresh call. + 2. Otherwise, durably record the turn ID, base version, and pending state before + the provider call. + 3. Call the provider once and replace the credential file with a + same-filesystem temporary file, file `fsync`, atomic rename, + parent-directory `fsync`, and validation. + 4. Record the committed version and release the mutex as soon as the valid + credential is visible. + + Once the pending record is durable, the coordinator owns the transaction + independently of turn cancellation or deadline. It uses a separate finite + renewal-transaction timeout. On timeout or caller-process failure, it + terminates the refresh process, proves the process boundary empty, durably + marks the profile `repair-required`, and only then releases the mutex. The + requesting turn cannot reach terminal acknowledgement before that outcome is + durable. + + No turn may call the provider's refresh endpoint outside this coordinator. If + the pinned Codex or Pi version cannot acquire the mutex before the provider + call, that target fails the phase-zero gate. The runner never changes profile + or auth mode. + + A `repair-required` profile rejects new admissions and renewal attempts. A turn + already running may finish with its current access token; provider rejection + becomes its normal authentication failure. The runner never switches profile + or auth mode. + + The runner owns a durable admission token for each active turn, not one + exclusive lock for the whole profile. It holds that token and the turn's + credential projection through descendant termination, projection teardown, + and gateway acknowledgement of durable terminal state. Renewal ownership lasts + from the durable pending record through a committed credential or a durable + `repair-required` fence. Runner failure leaves durable admission and renewal + records; startup blocks only that profile's readiness until it proves + descendant boundaries empty, removes stale projections, validates credentials, + and reconciles every turn, renewal, and maintenance record. + + Login, logout, and repair use a durable maintenance journal with + `pending | active | completed | failed` states. `pending` stops new profile + admissions and waits for active turns to finish. While maintenance is pending + or active, new turns fail before response allocation with + `allagents_auth_profile_unavailable`. If the operator deadline expires before + `active`, the runner records failure, removes the fence, and resumes admission. + Once `active`, timeout or cancellation requests process termination but never + releases the fence. The runner proves the administrative process boundary + empty, validates the profile or marks it `repair-required`, records the + outcome, and only then resumes admission. Startup reconciles every maintenance + record before that profile becomes ready. `proxyApiKey` uses neither native + profile turn slots nor the renewal coordinator. + + The gateway and runner never serialize auth files into root or nested + checkpoints, produced-file records, passive logs or traces, materializer + input, backups, or response metadata. Native mode is an explicit owner-trust + boundary: the harness and same-operating-system-identity tools may read or emit + the selected credential during an active turn. Preserve HarnessRouter + streaming, cancellation, idempotency, files, artifacts, retention-bounded + completed session persistence, per-session UID and runtime isolation, and + immutable generation sharing. #### Workspace extension and hook @@ -790,18 +818,20 @@ caller responsible for acquisition. The temporary fork closes those seams. cancellation or terminal completion revokes it. Passive persistence never stores it. The HarnessRouter broker rejects wrong-audience, wrong-model, wrong-turn, expired, or revoked tokens. -- **R14.** Configure finite, nonzero limits for session idle TTL, staging bytes - and concurrent builds, published-generation bytes/count, per-editable-session - hard bytes/inodes, total private reserved bytes/inodes, total sessions, - authorized persistent sessions, and tombstone bytes/count/TTL. Workspace - response admission reserves one generic session slot and one fixed-size - tombstone slot before making the response/session visible, so semantic - validation failure and later expiry/deletion cannot escape capacity accounting. - Those slots remain through failed-response retention and eventual - tombstone/purge. Readiness is false when required policy is absent or lifecycle - reconciliation is incomplete. Active work holds a durable lease and has no idle - deadline. For `session` retention, one provisional continuation-admission CAS - saves and clears a valid prior deadline; profile success commits active, while +- **R14.** Configure finite, nonzero limits for session idle TTL, renewal + transaction timeout, staging bytes and concurrent builds, published-generation + bytes/count, per-editable-session hard bytes/inodes, total private reserved + bytes/inodes, total sessions, authorized persistent sessions, and tombstone + bytes/count/TTL. Every native profile also has a finite concurrent-turn limit + of at least two. Workspace response admission reserves one generic + session slot and one fixed-size tombstone slot before making the + response/session visible, so semantic validation failure and later + expiry/deletion cannot escape capacity accounting. Those slots remain through + failed-response retention and eventual tombstone/purge. Readiness is false when + required policy is absent or lifecycle reconciliation is incomplete. Active + work holds a durable lease and has no idle deadline. For `session` retention, + one provisional continuation-admission CAS saves and clears a valid prior + deadline; profile readiness and capacity admission commit active, while pre-allocation profile failure restores that deadline if future or tombstones if elapsed. Durable terminal acknowledgement starts a new deadline. `persistent` bypasses idle expiry only after the runner authorizes it and @@ -895,18 +925,24 @@ caller responsible for acquisition. The temporary fork closes those seams. harness profile. If native OAuth cannot satisfy the deployment's trust or compatibility requirement, deliberately select and validate a separately configured `proxyApiKey` deployment profile; never make it automatic failover. -5. Start the private listener in probe-only, not-ready mode after reconciliation, - preflight, auth validation, and lifecycle-policy validation succeed. External - traffic and GC remain disabled. +5. Start the private listener in probe-only, not-ready mode after shared + coordinator, generation, session, mount, and lifecycle reconciliation plus + preflight succeed. Reconcile each auth profile independently. An unavailable + or `repair-required` profile disables only bindings that reference it. + External traffic and GC remain disabled. 6. From the exact container network, use the deployment probe identity to verify - each advertised harness/model, selected auth binding, safe roots, materializer + each configured harness/model, selected auth binding, safe roots, materializer version, generation store, mount enforcement, lifecycle policy, and a live - turn. Native targets exercise login, active-turn-only projection, refresh, - teardown, same-binding continuation, and fail-fast overlapping turns. Proxy - targets exercise schema/TLS/model-map/endpoint compatibility and scoped broker - use. Only after every probe succeeds does the deployment atomically enable - external serving, GC, and readiness. Any preflight, containment, auth, mount, - probe, or lifecycle failure keeps readiness false. + turn. Native targets exercise login, active-turn-only projection, + same-binding continuation, two concurrent same-profile turns with a valid + token, two with a forced-expired token, one coordinated provider refresh, + teardown, and repair after injected refresh faults. Proxy targets exercise + schema/TLS/model-map/endpoint compatibility and scoped broker use. Publish the + advertised target catalog from successful binding probes, then atomically + enable external serving, GC, and global readiness. A shared coordinator, + containment, mount, preflight, or lifecycle failure keeps global readiness + false. A profile-specific authentication failure marks only that binding + unavailable; healthy bindings remain advertised and serve requests. #### F2. Execute the first repository-backed turn @@ -917,15 +953,17 @@ caller responsible for acquisition. The temporary fork closes those seams. `editable`; omitted retention means `session`. 2. HarnessRouter validates UHP and generic metadata bounds and atomically claims the `Idempotency-Key`. The private runner admission transaction resolves the - selected target/auth-binding digest, applies existing profile admission, and - durably reserves one generic workspace-session slot plus one fixed-size - tombstone slot before response allocation. The admission token owns all three; - any pre-allocation failure rolls them back exactly once. Duplicate same-key - arrivals share one admission/result; new same-session overlap returns - `session_busy`; a genuinely new cross-session turn colliding on one native - profile fails cataloged `harness_unavailable`; unavailable generic lifecycle - capacity fails `allagents_workspace_capacity_exceeded`. Both occur before - response allocation. Distinct profiles may proceed concurrently. + selected target and auth-binding digest, checks profile readiness, and durably + reserves one configured turn slot for that profile, one generic + workspace-session slot, and one fixed-size tombstone slot before response + allocation. The admission token owns all three; any pre-allocation failure + rolls them back exactly once. Duplicate same-key arrivals share one + admission/result. New same-session overlap returns `session_busy`. A native + profile at its configured turn limit fails with cataloged + `harness_unavailable`; unavailable generic lifecycle capacity fails + `allagents_workspace_capacity_exceeded`. Both failures occur before response + allocation. Different sessions may proceed concurrently with the same or + different profiles. 3. The gateway consumes that token, creates the response/session in `validating`, and persists the opaque descriptor, raw request digest, generic reservation IDs, and harness/auth binding before making it visible. It then invokes the @@ -985,20 +1023,21 @@ caller responsible for acquisition. The temporary fork closes those seams. workspace metadata; active streaming uses `expiresAt: null`. Only editable sessions accept ordinary UHP input-file overlays. HarnessRouter uses the already validated logical cwd, creates the active-turn-only native credential - projection or scoped proxy credential, and dispatches the harness. Provider - retry/fallback - cannot validate, resolve, build, attach, or change any binding. + projection or scoped proxy credential, and dispatches the harness. When a + native token needs renewal, the auth adapter enters the profile's renewal + coordinator before the provider call. Provider retry or fallback cannot + validate, resolve, build, attach, or change any binding. 8. Read-only collection reports no workspace mutation. After descendants stop, editable collection verifies trusted changed-path candidates against the protected generation and prior turn state, or performs a bounded full-tree scan when candidate state is not trustworthy. It never reports initial source - files or trusts editable `.git` metadata. Native finalization then commits - refresh state or marks `repair-required`, removes the credential projection, - and proves retained homes/checkpoints clean. The gateway then durably stores - the terminal response and, for `session`, sets one expiry timestamp used by - the terminal event, GET, - background completion, and replay. Only after that acknowledgement may the - runner release the profile lock. + files or trusts editable `.git` metadata. Native finalization verifies that + turn has no incomplete renewal record, removes that turn's credential + projection, and proves retained homes and checkpoints clean. The gateway then + durably stores the terminal response and, for `session`, sets one expiry + timestamp used by the terminal event, GET, background completion, and replay. + Only after that acknowledgement may the runner release the turn's profile + capacity slot. #### F3. Continue the session @@ -1022,20 +1061,20 @@ caller responsible for acquisition. The temporary fork closes those seams. `allagents_workspace_non_resumable` and rolls the fence back by restoring the original future deadline or tombstoning if elapsed, without source resolution, acquisition, or rematerialization. -4. A native turn then tries the existing zero-waiter profile lock. Same-profile - cross-session saturation returns cataloged `harness_unavailable`; proxy mode - has no native lock. On any pre-allocation profile failure, the gateway rolls - back the provisional fence, restoring the exact original deadline if it - remains future or tombstoning the session if it elapsed. A returned admission - token commits the fence to active and exposes `expiresAt: null`. Sessions using - different profiles may execute concurrently against the same generation - epoch; polling and replay change no deadline or admission state. +4. A native turn checks profile readiness and reserves one configured + concurrent-turn slot; proxy mode has no native slot. Profile saturation or + any other pre-allocation profile failure rolls the provisional fence back, + restoring the exact original deadline if it remains future or tombstoning the + session if it elapsed. A returned admission token commits the fence to active + and exposes `expiresAt: null`. Sessions using the same or different profiles + may execute concurrently against the same generation epoch; polling and + replay change no deadline or admission state. 5. HarnessRouter resumes the native conversation and original attachment. Read-only source remains immutable; editable prior mutations remain visible. Output, usage, artifacts, and pinned provenance return without changing any - binding. The same refresh/projection teardown and terminal-ack protocol as the - first turn sets the next expiry and releases the profile lock on every - terminal outcome. + binding. The same renewal coordination, projection teardown, and terminal + acknowledgement as the first turn sets the next expiry and releases that + turn's profile slot on every terminal outcome. #### F4. Execute an OCI-backed first turn @@ -1074,12 +1113,12 @@ caller responsible for acquisition. The temporary fork closes those seams. and mount. A ready editable session requires its private publication marker, reserved quota, actual-usage accounting, and matching checkpoint. Missing or mismatched state becomes non-resumable; a later epoch is never substituted. -4. Every terminal outcome after native admission persists refresh disposition, - removes the active credential projection, verifies retained homes clean, and - stores terminal response/expiry before acknowledging the admission token. - Gateway or runner crashes retain the durable admission fence until descendant, - projection, and profile reconciliation; only then can another turn acquire the - profile. +4. Every terminal outcome after native admission records any incomplete renewal, + removes that turn's credential projection, verifies its retained home clean, + and stores terminal response and expiry before acknowledging the admission + token and releasing its profile slot. Gateway or runner crashes retain the + durable turn fence until descendants, projection, and profile state reconcile. + New profile admissions remain blocked only when readiness cannot be proven. 5. Restart reconciles all lifecycle state before GC or readiness. It completes or rolls back interrupted provisional turn admission against any runner profile token, then reconciles generic admission, active leases, staging/prospective- @@ -1212,13 +1251,18 @@ caller responsible for acquisition. The temporary fork closes those seams. - **AE10.** Codex and Pi own login and refresh. Missing, revoked, expired, unrefreshable, or stale-after-crash OAuth affects only that profile and never selects another profile or proxy. Same-key arrivals share one admission and - result; same-session overlap returns `session_busy`; new cross-session turns on - the same profile fail immediately with cataloged `harness_unavailable`. - Sessions using different profiles can run concurrently on one read-only - generation. Success, failure, cancellation, and crash recovery all commit or - reject refresh and remove the active credential projection before the next - profile admission. Proxy mode enforces audience, target, model, turn, expiry, - and revocation. + result; same-session overlap returns `session_busy`. Different sessions using + the same profile run concurrently up to its configured limit. With a valid + token, neither waits for a profile-wide lock. With a forced-expired token, + both succeed while the renewal coordinator permits one provider refresh and + makes the second turn reread the saved update. No provider refresh occurs + outside the coordinator. Success, failure, cancellation, and crash recovery + remove each turn's credential projection and release its capacity slot. + Incomplete renewal either reconciles to a valid credential or marks only that + profile `repair-required`. That state blocks new turns and renewal attempts; + an already-running turn either finishes with its current token or returns the + normal provider-authentication failure. Proxy mode enforces audience, target, + model, turn, expiry, and revocation. - **AE11.** Restart preserves an unexpired or persistent session, exact attachment, and auth binding after lifecycle reconciliation. Missing or changed auth fails before runner work. Missing generation/reference or private @@ -1677,8 +1721,8 @@ error object in `response.error`; workspace failures use | `allagents_workspace_immutable` | HTTP 409 `invalid_request_error` before response allocation when a continuation contains the workspace extension; `param` is `metadata.allagents.workspace` | no | | `allagents_workspace_expired` | HTTP 410 `invalid_request_error` before runner/profile work while a continuation's expired or deleted session tombstone remains retained; `param` is `previous_response_id` | no | | `allagents_workspace_non_resumable` | HTTP 409 `invalid_request_error` before profile admission when a known attached session's bound generation key/epoch/reference/publication/private/checkpoint evidence is missing or corrupt; `param` is `previous_response_id`; because the attachment previously reached `ready`, include its committed complete public workspace metadata | no | -| `harness_unavailable` / `detail.reason: "allagents_auth_profile_busy"` | HTTP 503 `server_error` before response allocation for a saturated auth profile; `param` is null | yes | -| `harness_unavailable` / `detail.reason: "allagents_auth_profile_unavailable"` | HTTP 503 `server_error` before response allocation for an unavailable or repair-required auth binding; `param` is null | yes | +| `harness_unavailable` / `detail.reason: "allagents_auth_profile_capacity_exceeded"` | HTTP 503 `server_error` before response allocation when a native profile has reached its configured concurrent-turn limit; `param` is null | yes | +| `harness_unavailable` / `detail.reason: "allagents_auth_profile_unavailable"` | HTTP 503 `server_error` before response allocation for an unavailable or repair-required auth binding, or while profile maintenance is pending or active; `param` is null | yes | | `allagents_workspace_invalid` | failed response for post-allocation caller repository descriptor, URL/egress-policy, snapshot catalog, path, layout, access/retention value, OCI shape/index, or unsupported media rejection that is not a numeric limit | no | | `allagents_workspace_persistence_forbidden` | failed response when `persistent` retention is not authorized for the selected deployment target | no | | `allagents_workspace_read_only` | failed response when a read-only initial request contains workspace input files or attachment policy would create writable shadow state | no | @@ -1704,9 +1748,9 @@ linearizes busy, exact binding, and expiry/deletion predicates without mutating state on rejection; a known attached but corrupt physical attachment returns non-resumable and rolls back its provisional admission before profile admission. For an initial request, generic metadata errors precede the idempotency claim; -stock `session_busy`, then native-profile availability, then generic session/ -tombstone capacity determine -pre-allocation admission. After allocation, source-free descriptor validation +stock `session_busy`, then native-profile readiness and concurrent-turn capacity, +then generic session/tombstone capacity determine pre-allocation admission. After +allocation, source-free descriptor validation and selected-reference verification precede secret-boundary recheck, persistence authorization, read-only conflict, access-specific capacity reservations, source resolution, build/staging/prospective-generation capacity, acquisition I/O, @@ -1839,12 +1883,13 @@ into successful empty output and performs no automatic retry. harness and same-identity tools only during an active turn. Use a same-filesystem namespace projection, mount no other profile, never copy it to durable session state, and verify teardown before terminal acknowledgement. -- **OAuth refresh loss or concurrency:** Retain the runner-owned zero-waiter - per-profile lock through descendant termination, refresh disposition, - credential-projection teardown, and terminal acknowledgement. Distinct - profiles provide parallelism on one generation; same-profile cross-session - overlap remains fail-fast. Reconcile stale projections and profile state before - reacquisition and mark stale remote rotation `repair-required`. +- **OAuth renewal loss or concurrency:** Allow concurrent turns to read one + profile. Require the pinned auth adapter to acquire the runner-owned renewal + mutex before any provider refresh call, reread the credential after acquiring + it, skip refresh when another turn already renewed, and atomically save one + valid update. Persist an incomplete-renewal record across crashes and mark the + profile `repair-required` when validity cannot be proven. Never hold the + renewal mutex for the whole turn. - **Descriptor/session drift:** Accept the JSON key only initially and persist descriptor, generation, access, retention, cwd, harness, and auth identity for every later response. Continuation never re-resolves. @@ -1899,42 +1944,70 @@ into successful empty output and performs no automatic retry. production workspace-materializer implementation. - **Repositories/files:** Minimal pinned HarnessRouter fork image, `runner/server.py`, Codex/Pi launch and home setup, auth-profile projection, - session auth-binding persistence, checkpoint exclusions, fault fixtures, and - focused runner/gateway tests. Do not add the AllAgents materializer or Git/OCI - acquisition in this unit. + renewal coordinator and journal, session auth-binding persistence, checkpoint + exclusions, fault fixtures, and focused runner/gateway tests. Do not add the + AllAgents materializer or Git/OCI acquisition in this unit. - **Approach:** Initialize dedicated profiles only through `codex login` and Pi `/login`. Use an active-turn-only directory-level mount namespace or equivalent same-filesystem credential view while keeping conversation state - session-scoped. Persist binding identity/digest, serialize every - refresh-capable turn per profile, preserve atomic local writes, mark invalid - post-rotation state `repair-required`, tear down and verify the projection - before terminal acknowledgement, mount no other profile, and prevent passive - checkpoint/backup/log/output serialization. Use the actual pinned harness - versions and real provider traffic. Freeze and record the HarnessRouter commit, - base-image digest, Codex version, Pi version, and auth-adapter patch digest used - by the gate. + session-scoped. Configure a finite concurrent-turn limit per profile; do not + lock the profile for a whole turn. Patch the auth adapter to acquire the + runner-owned renewal mutex before the provider refresh call, reread the current + credential and version, skip a redundant refresh, or record and atomically + save one renewal. Preserve incomplete renewal records across crashes. Mark a + profile `repair-required` only when startup cannot prove its credential valid. + Tear down each turn's projection before terminal acknowledgement, mount no + other profile, and prevent passive checkpoint, backup, log, output, or response + serialization. Login, logout, and repair use the durable maintenance journal. + Use the actual pinned harness versions and real provider traffic. Freeze and + record the HarnessRouter commit, base-image digest, Codex version, Pi version, + and auth-adapter patch digest used by the gate. - **Verification:** For both Codex and Pi, complete login, a real first turn, continuation, and restart without a provider-route API key. Change or remove - the binding and prove continuation fails before runner work. Use a barrier to - make two first arrivals atomically contend for the same new `Idempotency-Key` - and prove one admission/one result; prove a new same-session continuation - returns `session_busy`; and prove genuinely new cross-session turns fail - immediately with cataloged `harness_unavailable` before allocation. Same-key - waiters receive any pre-allocation owner error before claim removal; no rejected - cross-session request can acquire later. Record representative turn duration - and the one-active-turn-per-profile, zero-waiter operator capacity rule. Inject - provider, cancellation, gateway-only crash, runner crash, and - whole-process-death faults; after each, prove descendants are empty, stale - projections are removed or quarantined, retained CLI homes contain no - credential path, and reconciliation completes before the next new turn - acquires the runner-owned fenced profile lock. Terminate before, during, and - after local refresh persistence; restart must see a complete file that - validates or becomes `repair-required`. Prove unselected profiles and other - sessions' conversation state are inaccessible. With an inert agent, scan - checkpoints, produced-file records, backups, passive logs, and response - metadata for automatic credential serialization. Record that an active - same-identity tool can still read or emit the selected credential. Preserve - those exact input identities with the evidence. + the binding and prove continuation fails before runner work. + + Use barriers for four concurrency cases. Two arrivals with the same new + `Idempotency-Key` produce one admission and one result. A second turn in one + session returns `session_busy`. Two different sessions using one profile and a + valid token execute concurrently without a renewal mutex on the normal request + path. Two different sessions starting with a forced-expired token both + succeed: exactly one provider refresh occurs, the other turn rereads the saved + credential, and no provider refresh happens outside the coordinator. Reject a + configured profile limit of one. Set the limit to two and prove a third turn + fails before response allocation with + `allagents_auth_profile_capacity_exceeded`. + + Inject cancellation, the requesting turn's deadline, gateway-only crash, + runner crash, and whole-process death before the pending renewal record, after + that record but before the provider call, after provider rotation, during + atomic local save, and after commit. Before the pending record, the turn may + release the mutex. After it, the coordinator retains ownership independent of + the turn, no sibling can issue a second refresh, and terminal acknowledgement + waits for a committed credential or durable `repair-required` fence. On + coordinator timeout, prove the refresh process boundary empty before releasing + the mutex. + + Restart must reconcile every turn slot and renewal record, see a complete + credential that validates or mark only that profile `repair-required`, remove + or quarantine stale projections, find no credential path in retained CLI + homes, and prove old descendant boundaries empty before readiness. Break one + profile and prove its bindings return `allagents_auth_profile_unavailable` + while healthy bindings remain advertised and global readiness succeeds. + + For login, logout, and repair, inject timeout and crash while `pending`, during + the active command, after credential mutation, and before final validation. A + pending operation may fail and resume admission only before it becomes active. + An active fence survives cancellation, deadline, and restart until the + administrative process is gone and the profile validates or becomes + `repair-required`. New turns receive + `allagents_auth_profile_unavailable` throughout maintenance. + + Prove unselected profiles and other sessions' conversation state are + inaccessible. With an inert agent, scan checkpoints, produced-file records, + backups, passive logs, and response metadata for automatic credential + serialization. Record that an active same-identity tool can still read or emit + the selected credential. Preserve those exact input identities with the + evidence. - **Gate:** U1-U6 must not begin until both required native targets pass. Failure stops dependent work and reopens ADR 0002; proxy-only scope requires an explicit decision change and cannot count as a passing native gate. Any change @@ -2150,21 +2223,27 @@ into successful empty output and performs no automatic retry. HarnessRouter lifecycle/integration fixtures, AI Evals Promptfoo provider configuration, and deployment examples. - **Approach:** Reuse the accepted U0 auth adapter. Configure dedicated Codex and - Pi profiles; exercise login, live turns, atomic refresh, active-turn projection - teardown, stale repair, same-binding continuation, same-profile fail-fast - exclusion, different-profile concurrency, and profile isolation. Separately - validate proxy broker scope. Run Promptfoo requests that select anonymous public - and origin-mapped private HTTPS Git repositories, Git/OCI read-only concurrency, - independently cancelled shared-build waiters, editable two-turn growth and - cross-trial isolation, persistence, expiry/deletion/purge, capacity, restart, - cancellation, unsafe-URL/egress rejection, and every failure mapping. + Pi profiles; exercise login, live turns, coordinated atomic renewal, + active-turn projection teardown, stale repair, same-binding continuation, + same-profile and different-profile concurrency, configured profile capacity, + and profile isolation. Separately validate proxy broker scope. Run Promptfoo + requests that select anonymous public and origin-mapped private HTTPS Git + repositories, Git/OCI read-only concurrency, independently cancelled + shared-build waiters, editable two-turn growth and cross-trial isolation, + persistence, expiry/deletion/purge, capacity, restart, cancellation, + unsafe-URL/egress rejection, and every failure mapping. - **Verification:** Codex and Pi use native OAuth without a provider-route key. - Concurrent sessions with different profiles and harnesses share one read-only - generation while conversation/home/log/output state remains isolated. - Same-profile cross-session overlap retains the cataloged fail-fast result; - same-session overlap returns `session_busy`; idempotent duplicates share one - admission/result. Editable turn two sees turn one's mutation; a different trial - sees a clean private writable view. + Concurrent sessions using the same profile share one read-only generation + while conversation, home, log, and output state remain isolated. A + forced-expired token produces one coordinated provider refresh and two + successful turns. Cancellation after provider rotation does not release + renewal ownership or permit a second refresh; terminal acknowledgement waits + for commit or a durable repair fence. Same-session overlap returns + `session_busy`; idempotent duplicates share one admission and result; + configured profile saturation returns the cataloged capacity error. A damaged + profile becomes unavailable while healthy bindings remain advertised and + global readiness stays true. Editable turn two sees turn one's mutation; a + different trial sees a clean private writable view. Real-image lifecycle probes cover terminal-time TTL, retained-expiry HTTP 410, purged-predecessor stock failure, HTTP 409 non-resumable, authorized @@ -2218,24 +2297,24 @@ into successful empty output and performs no automatic retry. | Gate | Required evidence | |---|---| -| Native-auth feasibility | Before workspace work, the minimal image proves real Codex and Pi login, continuation, binding persistence, profile isolation, mutually exclusive same-profile turns, refresh repair, active-turn projection teardown on success/cancel/crash, and passive exclusion from checkpoints/backups/logs. Recorded pinned inputs invalidate the gate when changed. | +| Native-auth feasibility | Before workspace work, the minimal image proves real Codex and Pi login, continuation, binding persistence, profile isolation, concurrent same-profile turns with valid and forced-expired tokens, exactly one coordinated provider refresh, renewal ownership that survives turn cancellation/deadline, durable maintenance recovery, per-profile failure isolation, active-turn projection teardown on success/cancel/crash, and passive exclusion from checkpoints/backups/logs. Recorded pinned inputs invalidate the gate when changed. | | Stock compatibility | Upstream HarnessRouter tests and UHP conformance pass; requests without the metadata key are unchanged. | | Caller authentication | Every external create, continuation, retrieval, stream, cancellation, file, artifact, and lifecycle administration path authenticates before existence or metadata disclosure. | | Generation ordering | Generic session/tombstone admission precedes response visibility; secret-free preflight/validate and selected-reference verification plus access-specific authorization/reservation precede resolve. A miss reserves staging/prospective generation before acquisition; containment, full-tree accounting, commit-tree/Git/manifest verification, and atomic accounting conversion precede ready state/pins. Runner prepare plus gateway ready-ack precede provider dispatch; fallback never reenters. | | Shared-build cancellation | One request cancellation/deadline detaches only that waiter. A build continues for remaining live waiters, stops when none remain or its runner-owned deadline expires, and produces at most one publication/failure for its epoch. | | Manifest integrity | Git and OCI share one source-visible schema with pairwise non-overlapping repository destinations. A root may omit `.git` only when its manifest item declares history and the runner validates the detached commit, exact index/tree and object set, exact source-visible worktree, closed configuration and refs, and safe administrative state. Tree-only and undeclared `.git` fail. Git-acquired content equals the union of resolved commit trees at their destinations plus necessary ancestors. Plan/key drift, undeclared paths, forged manifests, changed staging, invalid paths/types/links/destinations, semantic Git mismatch, and digest mismatch fail before publication. | -| Shared read-only generation | Concurrent sessions using different harnesses/profiles share one exact generation epoch. Root, nested, symlink, and alternate-path writes fail; runtime/session/auth/output state remains isolated. | +| Shared read-only generation | Concurrent sessions using the same or different harness profiles share one exact generation epoch. Root, nested, symlink, and alternate-path writes fail; runtime/session/auth/output state remains isolated. | | Editable isolation | Every fitting editable trial receives a private writable view with no mutable state shared with the generation or another session, plus a reserved hard byte/inode allowance covering overlays, checkpoints, and produced state. A non-fitting waiter fails alone; continuation preserves a fitting trial's mutations but cannot grow past its envelope. | | Produced-file integrity | A clean first turn creates no redundant full-workspace inventory. Candidate tracking is durably active before input overlays or writable process exposure and remains active through quiescence; a missing or discontinuous coverage marker forces the bounded no-follow full scan. Candidate and full-scan paths produce the same source-visible additions, deletions, type/mode changes, and content changes. Both exclude only the declared Git administrative subtrees recorded by the protected generation, report an agent-created `.git` elsewhere as ordinary content, and ignore final Git discovery, ignore rules, and agent-supplied path lists. Continuation derives its delta from protected prior turn state. | | Materializer containment | Fork/double-fork/cancellation/deadline fixtures prove `populated 0` before result read, publication, secret release, or cleanup. `containment_pending` blocks terminal visibility/readiness through restart and resolves once after quiescence. | -| Capacity envelope | Native profiles retain one active turn and zero waiters. Source build limits and finite staging/generation/private-byte/private-inode/session/persistence/tombstone quotas reject overflow. Invalid descriptors cannot bypass generic admission; one editable session creates one private debit; successful publication releases staging capacity. References and provisional pins prevent eviction; all-protected capacity returns the cataloged retryable failure. | -| Durable lifecycle | Fault injection covers generic and provisional turn admission, active leases, generation epochs, build/staging/generation reservations, publication/accounting conversion, build waiters, provisional pins, attachment prepare/ready-ack and private-reservation transfer, references, mounts, private usage, expiry, tombstones/purge, unmount, deletion, quarantine, and GC. Startup reconciles before readiness; no deadline extends, no debit duplicates/leaks, no second epoch appears before prior eviction completes, and no session silently rematerializes. | +| Capacity envelope | Every native profile has a finite concurrent-turn limit of at least two; one is invalid, saturation fails before response allocation, and admitted same-profile turns proceed concurrently. The renewal transaction has a separate finite timeout. Source build limits and finite staging/generation/private-byte/private-inode/session/persistence/tombstone quotas reject overflow. Invalid descriptors cannot bypass generic admission; one editable session creates one private debit; successful publication releases staging capacity. References and provisional pins prevent eviction; all-protected capacity returns the cataloged retryable failure. | +| Durable lifecycle | Fault injection covers generic and provisional turn admission, native profile turn slots, renewal and maintenance records, active leases, generation epochs, build/staging/generation reservations, publication/accounting conversion, build waiters, provisional pins, attachment prepare/ready-ack and private-reservation transfer, references, mounts, private usage, expiry, tombstones/purge, unmount, deletion, quarantine, and GC. Startup reconciles before readiness; no deadline releases an active renewal or maintenance fence, no debit duplicates or leaks, no second epoch appears before prior eviction completes, and no session silently rematerializes. | | Retention and disposal | Fake-clock evidence proves one session CAS rejects busy/expired continuation admission, provisionally saves/clears a valid deadline, and either commits active after profile admission or restores the exact future deadline/tombstones an elapsed one after pre-allocation profile failure. Terminal acknowledgement alone sets the next `expiresAt`; polls/replays do not renew. Invalid failed responses stay accounted through purge; retained expiry returns HTTP 410; purge returns stock unknown; persistence authorizes before source access; operator deletion is idempotent. Null `lastUsedAt` epochs evict first by `publishedAt`; used epochs order by `lastUsedAt`, then `publishedAt`, generation key, and epoch. | | Session continuity | Both modes preserve conversation and fixed generation key/epoch/access/retention/cwd/harness/auth binding while persistent or unexpired; editable preserves private files; read-only remains immutable. Corrupt known evidence returns HTTP 409 non-resumable with no source access or later-epoch substitution. | | Git acquisition | Caller-supplied canonical HTTPS URLs, public-address egress enforcement, DNS-rebinding and redirect defense, structured-scope credential isolation, constrained refs, exact commits, closed transport/config, exact object closure/index semantics, generation reuse, and partial cleanup pass against local network fixtures. | | OCI acquisition | Digest/media/path/link/type/limit checks, tree-only and normalized offline-history fixtures, producer removal and materializer rejection of remotes and credentials, semantic Git verification, generation reuse, and the attachment matrix pass against a local registry. | -| Credential boundary | Preflight sees no secret values and returns bounded configured reference identities; validate selects a bounded subset; the runner verifies handles and injects only that selected set into source-access children. Source secrets and caller keys are absent from staging, generations, private views, base environments, checkpoints, backups, logs, and output. The selected OAuth profile is visible only through its active-turn projection, which is absent before acknowledgement and after refresh finalization, including failure, cancellation, restart, and continuation. Same-profile overlap fails before allocation; the active harness and same-identity tools remain an explicit owner-trust boundary. | -| Provider boundary | Codex/Pi native OAuth, refresh repair, projection teardown, idempotency/session/profile admission, different-profile concurrency, same-profile fail-fast exclusion, and explicit proxy scope all pass without implicit switching. | +| Credential boundary | Preflight sees no secret values and returns bounded configured reference identities; validate selects a bounded subset; the runner verifies handles and injects only that selected set into source-access children. Source secrets and caller keys are absent from staging, generations, private views, base environments, checkpoints, backups, logs, and output. The selected OAuth profile is visible only through each active turn's projection, which is absent before acknowledgement and after that turn's teardown. Concurrent turns may share the profile; the active harness and same-identity tools remain an explicit owner-trust boundary. | +| Provider boundary | Codex/Pi native OAuth, coordinated same-profile renewal, renewal ownership across turn cancellation/deadline, durable maintenance, profile-local repair/readiness, projection teardown, idempotency and same-session precedence, finite profile capacity of at least two, same-profile concurrency, and explicit proxy scope all pass without implicit switching. | | Packaging | The public GHCR digest and provenance/SBOM attestations verify exact inputs; deployment uses that digest and finite lifecycle configuration. | | Consumer | Promptfoo concurrent/one-shot/two-turn/lifecycle success and every cataloged or UHP terminal failure map exactly. Active streams expose null expiry; terminal/GET/replay expose one stable expiry. Failures before attachment ready omit workspace metadata; later terminal failures include the complete public object. None becomes empty success or automatic retry. | | Review | Final review findings in both repositories are resolved before final built-image E2E. | @@ -2301,20 +2380,22 @@ into successful empty output and performs no automatic retry. - Restart reconciles generic admission, active leases, build waiters, staging/ prospective-generation reservations, publication/accounting, provisional pins, prepare/ready-ack and private-reservation transfer, references, mounts, private - usage/copies, `containment_pending`, credential projections, tombstones/purge, - deletion, and profile fences before readiness or GC. Existing sessions never - silently reacquire source or change binding. + usage/views, `containment_pending`, credential projections, native profile turn + slots, renewal and maintenance records, tombstones/purge, and deletion before + readiness or GC. Existing sessions never silently reacquire source or change + binding. - Continuation omits the extension and preserves exact generation key/epoch, access, retention, cwd, harness, auth, and conversation while persistent or unexpired. Read-only remains immutable; editable retains private files; retained expiry/deletion returns HTTP 410, corrupt known evidence returns HTTP 409 non-resumable, and purged identity returns stock unknown, all without source access, rematerialization, or later-epoch substitution. -- Native auth retains atomic idempotency and same-session precedence, one active - refresh-capable turn and zero waiters per profile, different-profile - concurrency on one generation, fail-closed refresh repair, active-turn-only - credential projection with verified teardown before terminal acknowledgement, - and no implicit profile or proxy switching. +- Native auth retains atomic idempotency and same-session precedence, a finite + concurrent-turn limit of at least two per profile, concurrent same-profile + execution, one renewal transaction at a time whose ownership survives turn + cancellation and deadline, durable maintenance fencing, profile-local repair + and readiness, active-turn-only credential projection with verified teardown + before terminal acknowledgement, and no implicit profile or proxy switching. - Preflight receives no secret values; validate returns a bounded selected credential-reference set, and the runner injects only that exact set into source-access hook children. Source credentials never enter staging, From a40a4086bb1748f17a3f7ca67fe7a1377d08d11d Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sun, 27 Sep 2026 18:12:06 +1000 Subject: [PATCH 28/44] docs(architecture): clarify gateway release verification --- .../0002-adopt-uhp-through-harnessrouter.md | 67 +++++++- ...0837-feat-coding-execution-gateway-plan.md | 158 +++++++++++++----- 2 files changed, 183 insertions(+), 42 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 734f8c17..e8c9a1fe 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -747,6 +747,34 @@ Requests without the extension keep stock behavior. Upstream UHP conformance mus The upstream proposal should contain only the generic workspace and authentication-state seams. If upstream accepts an equivalent interface, remove the corresponding fork patch rather than keeping a compatibility layer. +### Upstream status + +We rechecked upstream on 2026-09-27 at HarnessRouter +[`5f82db1d`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), +also released as +[`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4). +Upstream now has a session lease before workspace hydration, per-session +workspaces, durable checkpoints, safe input and plugin-package writes, and +per-turn API-key brokering. We will reuse those pieces rather than replace them. + +The two required seams are still missing at that pin. Ordinary request metadata +does not reach the runner; the source says that only its System One probe is +forwarded +([gateway source](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/gateway/app.py#L7289-L7293)). +The runner's authentication shape contains API keys, endpoints, and cloud +credentials, but no native profile identity or OAuth state +([authentication source](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L1386-L1405)). +Codex and Pi credential files are deliberately excluded from session checkpoints +([checkpoint source](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L476-L503)), +with no separately durable profile store or projection contract to replace them. +Upstream therefore does not provide the native-profile renewal coordination, +repair fencing, or profile-local readiness required here. + +Therefore stock HarnessRouter still cannot implement this decision. The maintained +fork remains necessary, but only for the two seams above. Every upstream pin +change must repeat this inspection. If upstream supplies either equivalent seam, +we delete that downstream patch. + ### Materializer containment Each hook invocation receives one runner-owned cgroup-v2 leaf under the delegated @@ -802,9 +830,42 @@ Operators must be able to observe aggregate generation, editable-workspace, pers AllAgents publishes the public `linux/amd64` image as `ghcr.io/allagentsdev/harnessrouter`. Version and commit tags are discovery labels, not immutable deployment identities. Deployments pin the manifest digest. -The release workflow uses an approved ref, commit-pinned actions, an unprivileged build and test job, and a separate environment-approved publish job. GitHub package permission replaces third-party registry credentials. - -The final digest receives GitHub/Sigstore build-provenance and SBOM attestations. Deployment verifies the expected repository, workflow, ref, subject digest, predicate, base-image digest, lockfiles, OS packages, source tools, and Codex and Pi versions. +The release workflow uses commit-pinned actions and separates untrusted, +credential-free pull-request testing from protected release testing. A +credential-bearing job never runs for a pull-request event or PR-controlled ref. +It accepts only an image artifact whose recorded digest, source commit, and +workflow identity match the approved release ref. + +Green release verification runs the service, not just its build. A GitHub Actions +job starts the exact image as a local container on loopback or a private Docker +network, waits for readiness, and runs the AI Evals repository's locked Promptfoo +CLI and provider against it. This is an ephemeral test deployment, not a public +HarnessRouter service. A hand-written HTTP call may provide an additional smoke +test, but it cannot replace the real Promptfoo path. + +Pull-request jobs use a deterministic local provider fixture and receive no real +provider credentials. An environment-approved release job mounts the real Codex +and Pi profiles and proves native login, continuation, concurrency, renewal, and +cleanup. If a standard GitHub-hosted runner cannot provide the required mount or +cgroup behavior, the job receives a dedicated one-job self-hosted runner. That +runner is destroyed after credential teardown and is never reused for an +untrusted job; the workflow may not weaken or skip checks to fit a runner. + +The protected workflow pushes an untagged candidate by digest, creates +GitHub/Sigstore build-provenance and SBOM attestations, and anonymously pulls that +digest into a fresh protected job. That job repeats the complete Promptfoo path, +including native Codex and Pi cases, then creates a signed green-E2E attestation +for the same digest. Only after all three attestations verify does the workflow +apply version and commit discovery tags or mark the release complete. Deployment +requires the expected repository, workflow, approved ref, subject digest, +predicate, base-image digest, lockfiles, OS packages, source tools, Codex and Pi +versions, pinned AI Evals commit, Promptfoo lockfile identity, and successful +green-E2E attestation. A candidate without that final attestation is not +deployable. + +Promptfoo and AI Evals are pinned by the AI Evals lockfile and commit; the +workflow never downloads a floating `promptfoo@latest`. GitHub package +permission replaces third-party registry credentials. ## Failure behavior diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index f2305ea5..c91f18cb 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,7 +1,7 @@ --- title: "UHP Coding-Agent Execution through HarnessRouter - Plan" date: 2026-09-18 -updated: 2026-09-24 +updated: 2026-09-27 type: feat artifact_contract: ce-unified-plan/v1 artifact_readiness: implementation-ready @@ -872,28 +872,60 @@ caller responsible for acquisition. The temporary fork closes those seams. inherited Docker Hub release path. The Dockerfile pins every base image by digest; runtime lockfiles and version-locked OS packages, Git/OCI tools, Codex, and Pi define the remaining build inputs. The build fails on any unpinned - input. A no-write job builds, tests, and exports the identified image artifact - using commit-pinned third-party actions. A separate protected, - environment-approved publish job accepts only an approved release/tag ref and - uses `GITHUB_TOKEN` with `contents: read`, `packages: write`, - `attestations: write`, and `id-token: write`. It publishes unique version and - commit tags as mutable discovery labels, reads back the registry manifest, and - creates GitHub/Sigstore build-provenance and SBOM attestations whose subject is - the final manifest digest. Package visibility is public and verified with an - anonymous digest pull. Deployment fails unless both attestations verify the - expected owner, repository, workflow, approved ref, subject digest, predicates, - base-image digest, runtime lockfiles, OS package set, Git/OCI tool versions, and - Codex/Pi versions. Release E2E uses that digest, never `latest`. Requests - without the configured metadata key remain stock-compatible. CI rebases - selected upgrades and runs upstream plus AllAgents integration tests. -- **R16.** AI Evals owns its Promptfoo provider. It sends the UHP request directly - to HarnessRouter, maps Promptfoo variables to the closed extension, and maps - terminal output, usage, artifacts, provenance, and failures to - `ProviderResponse`. Every non-success follows the Failure Contract's exact - status/error/retryability/metadata mapping; none becomes empty success or an - automatic retry. Multi-turn cases retain the prior response ID and send it as - `previous_response_id`. AllAgents documents the contract and examples but does - not depend on Promptfoo at runtime. + input. + + A no-write GitHub Actions job builds, tests, and exports the identified image + artifact using commit-pinned third-party actions. It starts that exact artifact + as a local container on loopback or a private Docker network, waits for + readiness, checks out AI Evals at a pinned commit, installs its locked + dependencies, and runs its actual Promptfoo CLI and provider against + HarnessRouter. A direct UHP smoke script may supplement this test but cannot + replace Promptfoo. The job never downloads a floating `promptfoo@latest`. + + The pull-request path uses a deterministic local provider fixture and receives + no real provider credentials. The native-auth path never runs for a + `pull_request` event or PR-controlled ref. It requires environment approval, + an approved release/tag ref, and an image artifact whose recorded digest, + source commit, and producing workflow identity match that ref. It mounts the + real Codex and Pi profiles and runs the native-auth cases against that exact + artifact. If a standard hosted runner cannot prove the required mount and + cgroup behavior, use a dedicated ephemeral self-hosted Actions runner for one + job. Destroy it after credential teardown and never schedule an untrusted job + on it; do not weaken or skip checks to fit a runner. + + A separate protected publish job accepts only that approved artifact. It uses + `GITHUB_TOKEN` with `contents: read`, `packages: write`, + `attestations: write`, and `id-token: write`. It first pushes an untagged + candidate by digest, reads back the registry manifest, and creates + GitHub/Sigstore build-provenance and SBOM attestations whose subject is that + digest. Package visibility is public and verified with an anonymous digest + pull into a fresh protected job. That job repeats both the PR-safe and real + native-profile Promptfoo suites against the pulled bytes and creates a signed + green-E2E attestation recording the subject digest, approved source ref, AI + Evals commit, Promptfoo lockfile identity, scenario identity, and successful + workflow run. Only after build provenance, SBOM, and green-E2E attestations all + verify does the workflow apply unique version and commit discovery tags or + mark the release complete. + + Deployment fails unless those three attestations verify the expected owner, + repository, workflow, approved ref, subject digest, predicates, base-image + digest, runtime lockfiles, OS package set, Git/OCI tool versions, Codex/Pi + versions, AI Evals commit, Promptfoo lockfile, and successful scenario run. An + untagged candidate without the green-E2E attestation is not deployable. + Requests without the configured metadata key remain stock-compatible. CI + rebases selected upgrades and runs upstream plus AllAgents integration tests. +- **R16.** AI Evals owns its Promptfoo provider and the executable green-E2E + configuration. The workflow invokes the Promptfoo version from AI Evals' + lockfile through that repository's package script. The provider sends the UHP + request directly to the locally running HarnessRouter, maps Promptfoo variables + to the closed extension, and maps terminal output, usage, artifacts, + provenance, and failures to `ProviderResponse`. Every non-success follows the + Failure Contract's exact status/error/retryability/metadata mapping; none + becomes empty success or an automatic retry. Multi-turn cases retain the prior + response ID and send it as `previous_response_id`. Green requires the real + Promptfoo process to exit successfully and the scenario assertions to pass; a + hand-written client is not consumer proof. AllAgents documents the contract + and examples but does not depend on Promptfoo at runtime. ### Key Flows @@ -1775,7 +1807,7 @@ failure stays operational: readiness is false and the safe operator diagnostic uses the same classification vocabulary without pretending a task failed. Vendor codes and vendor detail reasons use the required `allagents_` prefix. A -request failure includes `detail.retryable`; the busy-profile reason omits +request failure includes `detail.retryable`; the profile-capacity reason omits `retry_after_ms` rather than guessing. Promptfoo maps a non-2xx or `failed` response to `ProviderResponse.error = ": "`. It maps `incomplete` to `ProviderResponse.error = "incomplete: task stopped at a budget"` @@ -1795,6 +1827,18 @@ into successful empty output and performs no automatic retry. ### Fork Maintenance Contract +The 2026-09-27 upstream recheck inspected HarnessRouter +[`5f82db1d`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), +released as +[`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4). +Stock now provides a session lease before hydration, per-session workspaces, +durable checkpoints, safe file/package materialization, and per-turn API-key +brokering. The fork must reuse those lifecycle and security primitives. Stock +still forwards no ordinary request metadata to the runner and provides neither +the pre-turn Git/OCI materializer contract nor durable named native OAuth +profiles. Both planned seams therefore remain necessary at this pin. +ADR 0002's upstream-status section records the source evidence. + - Keep the fork in a dedicated repository/branch with the upstream remote intact. - Pin production images to an upstream commit, never a moving branch. - Keep the materializer changes as a small ordered patch series with focused @@ -1803,6 +1847,9 @@ into successful empty output and performs no automatic retry. changes in touched gateway/runner/session code, run upstream tests and UHP conformance, run AllAgents hook/session E2E, rebuild the image, and record the new inputs and digest. +- Before accepting every new upstream pin, repeat the documented two-seam + capability check. Record which upstream source replaced any patch, and delete + that patch in the same upgrade; absence of a recheck blocks the upgrade. - Publish releases to `ghcr.io/allagentsdev/harnessrouter`, record the manifest digest, and rehearse deployment from that digest rather than a local build or mutable tag. @@ -2232,6 +2279,13 @@ into successful empty output and performs no automatic retry. shared-build waiters, editable two-turn growth and cross-trial isolation, persistence, expiry/deletion/purge, capacity, restart, cancellation, unsafe-URL/egress rejection, and every failure mapping. + + GitHub Actions is the executable test host. Its driver starts the built image + locally, waits for HarnessRouter readiness, checks out the pinned AI Evals + commit, installs the lockfile, and invokes AI Evals' Promptfoo package script + against the local UHP endpoint. The pull-request lane uses a deterministic + local provider fixture; the protected lane uses the real Codex and Pi profiles. + Both lanes exercise the same Promptfoo provider and scenario definitions. - **Verification:** Codex and Pi use native OAuth without a provider-route key. Concurrent sessions using the same profile share one read-only generation while conversation, home, log, and output state remain isolated. A @@ -2257,6 +2311,12 @@ into successful empty output and performs no automatic retry. Promptfoo returns exact coded errors and metadata, never empty success or automatic retry. + The Promptfoo process must exit successfully and its assertions must identify + the expected output, continuation, provenance, artifacts, and coded failures. + The job also checks HarnessRouter logs and the temporary roots for leaked + processes, mounts, credentials, and retained test state. A direct UHP script is + useful for diagnosis but does not satisfy this consumer E2E. + ### U6. Release, operations, review, and upstream preparation - **Goal:** Produce a reproducible, registry-published supported image and @@ -2267,10 +2327,18 @@ into successful empty output and performs no automatic retry. - **Approach:** Build from exact upstream/fork/AllAgents/agent inputs. Pin base images, lockfiles, OS packages, Git/OCI tools, Codex, and Pi. U6 uses the identities frozen by current U0 evidence; changing one stops release and - reruns U0. Replace inherited Docker Hub publication with separated no-write - build/test and protected GHCR publish jobs using commit-pinned actions. Publish - `linux/amd64`, read back the manifest, and attach verified build-provenance and - SBOM attestations before E2E. + reruns U0. Replace inherited Docker Hub publication with a no-write build/test + job, a protected native-auth test job, a protected candidate-publish job, and a + final promotion job, all using commit-pinned actions. The no-write job runs the + PR-safe Promptfoo suite against the exported image artifact. The native-auth + job runs only from an approved release/tag ref and accepts only the artifact + provenance bound to that exact ref; it never runs pull-request code. Push the + `linux/amd64` candidate without discovery tags, read back the manifest, and + attach verified build-provenance and SBOM attestations. A fresh protected job + anonymously pulls that digest, runs the PR-safe and real native-profile + Promptfoo suites against the registry bytes, and creates the green-E2E + attestation. Only then may promotion apply version/commit tags or complete the + release. Document durable generation/session/private/auth volumes; caller-supplied Git URLs in the JSON descriptor versus the optional operator-owned @@ -2282,14 +2350,26 @@ into successful empty output and performs no automatic retry. and the owner-trust boundary. Review both repositories before final green E2E and prepare generic generation/attachment/lifecycle and auth-state patches for upstream. -- **Verification:** A clean `linux/amd64` host verifies attestations and pinned - inputs, anonymously pulls by digest, configures finite lifecycle policy, and - reproduces Git/OCI generation reuse, cross-harness/profile read-only - concurrency, editable isolation, continuation, persistence, expiry/deletion, - capacity pressure, GC, cancellation, restart, native auth, proxy, and every - documented failure. No Docker Hub credential is required. Wrong provenance, - build input, lifecycle configuration, or unverified generation store prevents - readiness or release. Rebase rehearsal reports incompatibility before release. +- **Verification:** A clean `linux/amd64` GitHub Actions job verifies the + candidate's build-provenance and SBOM attestations and pinned inputs, + anonymously pulls it by digest, starts HarnessRouter locally, waits for + readiness, and runs the pinned AI Evals Promptfoo package script. It configures + finite lifecycle policy and reproduces Git/OCI generation reuse, + cross-harness/profile read-only concurrency, editable isolation, continuation, + persistence, expiry/deletion, capacity pressure, GC, cancellation, restart, + native auth, proxy, and every documented failure. Credential-bearing jobs run + only for the approved ref. If standard hosted runners cannot provide the + required mount or cgroup behavior, each such job receives a dedicated + ephemeral one-job self-hosted runner that is destroyed after credential + teardown and never executes untrusted work. + + The successful job creates a signed green-E2E attestation for the candidate + digest before promotion. No version/commit discovery tag or completed release + exists before that attestation verifies. No Docker Hub credential is required. + Wrong provenance, build input, lifecycle configuration, unverified generation + store, Promptfoo assertion, leak check, process exit, or test-attestation + identity prevents promotion and deployment. Rebase rehearsal reports + incompatibility before release. --- @@ -2298,7 +2378,7 @@ into successful empty output and performs no automatic retry. | Gate | Required evidence | |---|---| | Native-auth feasibility | Before workspace work, the minimal image proves real Codex and Pi login, continuation, binding persistence, profile isolation, concurrent same-profile turns with valid and forced-expired tokens, exactly one coordinated provider refresh, renewal ownership that survives turn cancellation/deadline, durable maintenance recovery, per-profile failure isolation, active-turn projection teardown on success/cancel/crash, and passive exclusion from checkpoints/backups/logs. Recorded pinned inputs invalidate the gate when changed. | -| Stock compatibility | Upstream HarnessRouter tests and UHP conformance pass; requests without the metadata key are unchanged. | +| Stock compatibility | Every upstream pin records a source-backed check for the workspace-hook and native-auth-profile seams. Any equivalent upstream seam replaces its downstream patch in the same upgrade. Upstream HarnessRouter tests and UHP conformance pass; requests without the metadata key are unchanged. | | Caller authentication | Every external create, continuation, retrieval, stream, cancellation, file, artifact, and lifecycle administration path authenticates before existence or metadata disclosure. | | Generation ordering | Generic session/tombstone admission precedes response visibility; secret-free preflight/validate and selected-reference verification plus access-specific authorization/reservation precede resolve. A miss reserves staging/prospective generation before acquisition; containment, full-tree accounting, commit-tree/Git/manifest verification, and atomic accounting conversion precede ready state/pins. Runner prepare plus gateway ready-ack precede provider dispatch; fallback never reenters. | | Shared-build cancellation | One request cancellation/deadline detaches only that waiter. A build continues for remaining live waiters, stops when none remain or its runner-owned deadline expires, and produces at most one publication/failure for its epoch. | @@ -2315,8 +2395,8 @@ into successful empty output and performs no automatic retry. | OCI acquisition | Digest/media/path/link/type/limit checks, tree-only and normalized offline-history fixtures, producer removal and materializer rejection of remotes and credentials, semantic Git verification, generation reuse, and the attachment matrix pass against a local registry. | | Credential boundary | Preflight sees no secret values and returns bounded configured reference identities; validate selects a bounded subset; the runner verifies handles and injects only that selected set into source-access children. Source secrets and caller keys are absent from staging, generations, private views, base environments, checkpoints, backups, logs, and output. The selected OAuth profile is visible only through each active turn's projection, which is absent before acknowledgement and after that turn's teardown. Concurrent turns may share the profile; the active harness and same-identity tools remain an explicit owner-trust boundary. | | Provider boundary | Codex/Pi native OAuth, coordinated same-profile renewal, renewal ownership across turn cancellation/deadline, durable maintenance, profile-local repair/readiness, projection teardown, idempotency and same-session precedence, finite profile capacity of at least two, same-profile concurrency, and explicit proxy scope all pass without implicit switching. | -| Packaging | The public GHCR digest and provenance/SBOM attestations verify exact inputs; deployment uses that digest and finite lifecycle configuration. | -| Consumer | Promptfoo concurrent/one-shot/two-turn/lifecycle success and every cataloged or UHP terminal failure map exactly. Active streams expose null expiry; terminal/GET/replay expose one stable expiry. Failures before attachment ready omit workspace metadata; later terminal failures include the complete public object. None becomes empty success or automatic retry. | +| Packaging | The no-write GitHub Actions job starts the exported image and passes the credential-free Promptfoo suite. Credential-bearing jobs accept only an artifact bound to the approved release ref and never run pull-request code. The protected workflow pushes an untagged GHCR candidate, verifies build-provenance/SBOM attestations, anonymously pulls the digest into a fresh job, and passes both Promptfoo suites. That job creates the signed green-E2E attestation before version/commit tags or release completion. Deployment requires all three attestations. A hosted-runner limitation selects a dedicated ephemeral one-job self-hosted runner that is destroyed after credential teardown, never a weaker check or reused host. | +| Consumer | The Promptfoo version from AI Evals' lockfile runs through that repository's package script against the local HarnessRouter endpoint. Its process exits successfully for concurrent, one-shot, two-turn, and lifecycle success scenarios, and its assertions prove every cataloged or UHP terminal failure maps exactly. Active streams expose null expiry; terminal/GET/replay expose one stable expiry. Failures before attachment ready omit workspace metadata; later terminal failures include the complete public object. None becomes empty success or automatic retry. A direct HTTP smoke test cannot substitute for this gate. | | Review | Final review findings in both repositories are resolved before final built-image E2E. | ## Definition of Done From ac54bc0081a4dba889970b0d949e63377da7db03 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sun, 27 Sep 2026 19:49:13 +1000 Subject: [PATCH 29/44] docs(gateway): narrow v1 architecture --- .../0002-adopt-uhp-through-harnessrouter.md | 1018 +---- ...0837-feat-coding-execution-gateway-plan.md | 3299 +++++------------ 2 files changed, 996 insertions(+), 3321 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index e8c9a1fe..c2772495 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -1,951 +1,229 @@ -# ADR 0002: Adopt UHP through HarnessRouter with an AllAgents workspace materializer +# ADR 0002: Build AllAgents Gateway as a narrow HarnessRouter downstream -- Status: Accepted; implementation gated on native-auth feasibility +- Status: Accepted - Date: 2026-09-21 -- Updated: 2026-09-24 +- Updated: 2026-09-27 -## Decision - -AllAgents will use the Unified Harness Protocol (UHP) `2026-09-12` through a pinned HarnessRouter Community Edition deployment for remote Codex and Pi execution. UHP remains the only execution protocol. - -HarnessRouter will keep responsibility for caller authentication, UHP behavior, sessions, streaming, cancellation, idempotency, harness execution, usage, artifacts, and lifecycle state. - -A narrow AllAgents-maintained fork will add a generic pre-turn workspace hook. The hook will validate and resolve the caller's source, build a verified immutable workspace when needed, and attach it before the coding harness starts. - -Codex and Pi will use their native login and refresh behavior by default. An -authenticated provider proxy remains an explicitly configured last resort. -Native-auth failure must never activate the proxy automatically. Version one -still implements and verifies proxy mode even when a deployment does not use it. - -Project `workspace.yaml` will remain ordinary local AllAgents configuration plus an optional operator-owned OCI snapshot catalog. Its local repository entries use `path` for the checkout location and one canonical `url` for remote identity; the provider-specific `source` plus `repo` pair is removed as a clean schema cutover. It will not control which Git repositories a UHP caller may request. - -The fork is delivery machinery, not a second protocol. We will keep the changes narrow and suitable for upstreaming, but delivery will not depend on upstream acceptance. - -## Main flow - -HarnessRouter has two relevant components. The gateway owns public response and -session transitions. The runner owns processes, filesystems, and resource state. -The external AllAgents materializer validates and builds source. - -An initial workspace-backed turn follows this order: - -1. The gateway authenticates the caller and claims the idempotency key. The - runner selects the harness and authentication binding, admits the native - profile when applicable, and reserves session and tombstone capacity. -2. The gateway allocates the response and session, stores the opaque workspace - descriptor, and asks the materializer to validate it without accessing source. -3. The runner authorizes persistent retention when requested and reserves the - editable workspace allowance when needed. -4. The materializer resolves every Git ref or OCI digest to immutable source - identity and returns provenance. It does not write source bytes during this - step. -5. The runner reuses a valid ready generation, joins an existing build for the - same generation key, or claims a new build and reserves staging and generation - capacity. -6. On a cache miss, the materializer builds private staging. The runner - independently verifies the source tree, Git or OCI state, manifest, limits, - and generation key before publishing it atomically. -7. The runner pins the generation, then mounts it read-only or creates a private - writable view with no mutable state shared with another session. -8. The gateway commits the attachment as `ready`. The runner acknowledges that - commit, transfers or releases reservations, and releases the provisional pin - exactly once. -9. The runner projects the selected native OAuth profile or issues the configured - scoped proxy credential. HarnessRouter starts the coding harness in the - requested working directory. -10. After the harness and descendants stop, the runner finalizes authentication - state and removes the credential projection. The gateway stores the terminal - response and starts the idle deadline for `session` retention. - -After the first turn: - -1. A continuation reuses only the exact bound attachment. -2. Expiry or deletion tombstones the session before cleanup, waits for active - resources to quiesce, and releases references and reservations exactly once. +## Context -Source resolution, workspace attachment, and authentication selection happen before provider execution. Provider retry or fallback cannot repeat or change them. +Promptfoo needs a remote coding-harness endpoint that can prepare a repository before a turn, preserve that checkout across continuations, and expose the result through the Unified Harness Protocol (UHP). HarnessRouter already owns the difficult execution-plane behavior: UHP requests and streams, sessions, cancellation, files, artifacts, caller authentication, harness processes, and custom harness configuration. Replacing that control plane would create a second implementation of behavior we already need. -## Phase-zero feasibility gate +The examined baseline is UHP [`2026-09-12`](https://github.com/HarnessRouter/harnessrouter/tree/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/versions/2026-09-12) at HarnessRouter commit [`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), released as [`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4). -Production workspace implementation must not begin until a minimal pinned image proves native authentication against real provider traffic for both Codex and Pi. The spike excludes the workspace materializer and Git or OCI acquisition. +UHP already reserves `metadata` for additive extensions. At the pinned commit, the request schema accepts an open metadata object ([UHP schema](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/schema/uhp-2026-09-12.openapi.yaml#L1051-L1057)). That is an existing protocol extension point, not an extension framework: stock HarnessRouter gives arbitrary metadata no runner semantics. It extracts only nested `metadata.systemone` ([gateway extraction](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/gateway/app.py#L7571-L7572)) and forwards only that probe to the runner ([runner handoff](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/gateway/app.py#L7008-L7011)). Workspace semantics therefore require a downstream change. +The AllAgents key follows the same nested namespace convention as the existing `metadata.systemone.script`: `metadata.allagents.workspace`. The shared convention is the metadata shape, not System One's special forwarding behavior. -Freeze this evidence set: - -| Input | Required identity | -|---|---| -| HarnessRouter | Exact upstream commit | -| Base image | Manifest digest | -| Codex | Exact version | -| Pi | Exact version | -| Auth adapter | Patch digest | +## Decision -Changing any input invalidates the evidence. Dependent work remains blocked until both native targets pass again. +We will ship **AllAgents Gateway** as a rebase-friendly downstream of HarnessRouter. The existing GitHub fork `allagentsdev/harnessrouter` will be renamed to [`allagentsdev/allagents-gateway`](https://github.com/allagentsdev/allagents-gateway), preserving its fork relationship and history. We will not create a third repository that consumes the fork. -Each target must prove all nine behaviors: +The downstream adds one product feature: on a first turn, recognize the exact `metadata.allagents.workspace` object, validate and materialize one repository into a private session checkout, bind it immutably to the session, and start the selected harness there. A continuation reuses that exact checkout. -1. The operator can complete native login in a controlled environment. -2. A real first turn and continuation succeed without a provider-route API key. -3. The selected authentication binding survives restart and fails closed when unavailable. -4. Session conversation state remains separate while only the selected profile is visible. -5. Two real turns using the same profile and a valid token succeed concurrently. -6. When two turns start with an expired token, renewal is coordinated before either calls the provider: one renews, the other rereads the saved update, both succeed, and the credential file remains valid. -7. Credential files remain complete through renewal and injected crashes; an invalid saved update makes the profile `repair-required`. -8. Success, failure, cancellation, and crash recovery remove every active projection. Credentials remain absent from retained homes, checkpoints, produced-file records, backups, passive logs, and response metadata. -9. The evidence explicitly records that the selected harness and same-identity tools can read or emit the credential during an active turn. +The supported v1 harnesses are **Codex** and **OMP**. Provider traffic goes only through an existing, separately operated OAuth-to-OpenAI-compatible gateway. Promptfoo calls AllAgents Gateway directly over UHP. -The proxy route cannot satisfy this gate. Failure of either native target stops dependent implementation. A proxy-only release or narrower harness scope requires a new decision. +The downstream is not a general workspace platform. It adds no extension registry, dynamically selected hook, arbitrary materializer command, or second protocol. It invokes exactly one operator-configured materializer that is included in the AllAgents Gateway image. -## System map and ownership +## System flow and ownership ```mermaid -flowchart TB - CLIENT[Promptfoo or another UHP client] -->|UHP plus optional workspace metadata| GATEWAY[HarnessRouter gateway] - GATEWAY --> RUNNER[HarnessRouter runner] - RUNNER -->|validate, resolve, materialize| MATERIALIZER[AllAgents materializer] - MATERIALIZER --> SOURCE[HTTPS Git or configured OCI registry] - RUNNER --> GENERATION[(immutable generations)] - GENERATION --> READONLY[shared read-only attachment] - GENERATION --> EDITABLE[private editable view] - READONLY --> HARNESS[Codex or Pi harness] - EDITABLE --> HARNESS - RUNNER --> AUTH[(native auth profiles)] - AUTH -->|active-turn projection| HARNESS - HARNESS -->|native OAuth| PROVIDER[model provider] - HARNESS -.->|explicit proxy mode| GATEWAY - GATEWAY -.-> PROXY[authenticated provider proxy] - PROXY -.-> PROVIDER - GATEWAY --> SESSION[(session and attachment state)] - RUNNER --> RESOURCE[(generation and resource journal)] +flowchart LR + P[Promptfoo] -->|UHP + caller API key| A[AllAgents Gateway] + A -->|fixed workspace seam| M[In-image Git materializer] + M -->|anonymous HTTPS| G[Public Git repository] + A --> C[(Private session checkout)] + C --> H[Codex or OMP] + H -->|short-lived turn credential| B[HarnessRouter loopback broker] + B -->|Responses or Chat Completions| O[OAuth-to-OpenAI-compatible gateway] + O --> V[Model provider] + A --> D[(/data sessions and state)] ``` -| Actor | Owns | +| Component | Owns | |---|---| -| UHP client | Prompt, model, harness ID, workspace descriptor, continuation ID | -| HarnessRouter gateway | Caller authentication, UHP validation, public response and session state, attachment `ready`, expiry, tombstones | -| HarnessRouter runner | Generation claims and publication, source child processes, mounts, private writable views, quotas, references, pins, profile admission | -| AllAgents materializer | Descriptor defaults, URL and source validation, Git or OCI resolution, staging construction, canonical manifest, provenance | -| Coding harness | Provider login, native token refresh, conversation execution | -| Operator | Deployment policy, egress, credential scopes, authentication profiles, persistence authorization, quotas, deletion, garbage collection | - -The gateway is the only writer of public session attachment, expiry, and tombstone state. The runner is the only writer of generation and resource state. The materializer cannot authorize persistence or publish live state. - -## Protocol and workspace descriptor - -UHP is the sole wire contract and its conformance suite is the protocol oracle. The fork must preserve its Responses-shaped requests, ordered streaming events, `previous_response_id`, cancellation, files, artifacts, usage, lifecycle, and error behavior. - -Version one adds one namespaced first-turn extension: `metadata["allagents.workspace"]`. - -Before response allocation, the gateway requires this extension to be a JSON -object no larger than 64 KiB and no deeper than 32 levels. A non-object receives -HTTP 400 `invalid_input`; a byte or depth overflow receives HTTP 413 -`allagents_workspace_too_large`. - -### Request fields - -| Field | Required | Contract | -|---|---:|---| -| `version` | yes | Exactly `"1"` | -| `access` | yes | `readOnly` or `editable` | -| `retention` | no | `session` by default, or authorized `persistent` | -| `source` | yes | Repository list or configured workspace snapshot | -| `workingDirectory` | no | `{ "kind": "workspaceRoot" }` by default, or `{ "kind": "workspacePath", "path": "…" }` | +| Promptfoo | Prompt, model, `metadata.harness_id`, first-turn workspace request, continuation ID, and evaluation assertions | +| HarnessRouter gateway and runner | Caller authentication, UHP validation and conformance, idempotency, session hydration, streaming, cancellation, harness execution, files, artifacts, and lifecycle state | +| AllAgents workspace seam | Exact metadata recognition, first-turn binding, continuation lookup, ordering before harness execution, and normalized workspace failures | +| Fixed materializer | URL and ref validation, safe Git resolution and acquisition, exact commit provenance, checkout validation, resource enforcement, cancellation, and cleanup | +| HarnessRouter custom harness definition | Reusable remote harness configuration: Codex or OMP base harness, model defaults, instructions, tools, skills, and server-owned provider route | +| HarnessRouter loopback broker | Per-turn scoped credential minting and exchange; the harness process never receives the long-lived external-gateway API key | +| External OAuth gateway | Provider login, OAuth token storage, refresh, repair, provider API compatibility, and provider authorization | +| Operator | Caller credentials, custom harnesses, provider endpoint and API key, egress policy, limits, TTL, deployment, upgrades, and deletion policy | -Repository mode accepts one through 128 entries. +The materializer never owns UHP sessions or provider credentials. The external OAuth gateway never owns source acquisition or UHP session state. Promptfoo never receives source or provider credentials. -Each repository entry has this shape: +## Request contract -| Field | Required | Contract | -|---|---:|---| -| `url` | yes | Canonical public HTTPS Git URL; the same URL may appear more than once | -| `ref` | no | Full ref name, unambiguous branch or tag shorthand, or full 40-hex commit ID; omission means remote symbolic HEAD | -| `destination` | yes | Unique, non-root relative directory; destinations must not overlap or collide with a runner-owned control namespace | - -A workspace snapshot source instead has the exact shape `{ "kind": "workspaceSnapshot", "snapshotName": ConfigName, "imageManifestDigest": Digest, "workspaceManifestDigest": Digest }`. `snapshotName` selects an operator-owned catalog entry; `imageManifestDigest` identifies the accepted direct OCI image manifest; `workspaceManifestDigest` identifies the canonical source-visible manifest. - -Reserved control namespaces include HarnessRouter's root checkpoint repository. -Source-free validation rejects a destination that equals, contains, or is -contained by a reserved namespace. - -`workspacePath` is relative to the mounted workspace and must name a directory in the resolved source manifest. The same working-directory contract applies to repository and snapshot sources. - -Example: +A workspace-backed first turn uses the normal UHP `POST /v1/responses` request. The exact v1 extension shape is: ```json { - "version": "1", - "access": "readOnly", - "retention": "session", - "source": { - "kind": "repositories", - "repositories": [ - { - "url": "https://github.com/acme/api.git", - "ref": "refs/pull/123/head", - "destination": "api" + "model": "gpt-5.4", + "input": "Implement the requested change.", + "metadata": { + "harness_id": "chrn_…", + "allagents": { + "workspace": { + "version": "1", + "repository": { + "url": "https://github.com/example/project.git", + "ref": "refs/heads/main" + }, + "workingDirectory": "packages/service" } - ] - }, - "workingDirectory": { - "kind": "workspacePath", - "path": "api/packages/service" + } } } ``` -The descriptor is session input, not project configuration. Promptfoo supplies the repositories to load. The materializer loads them before the harness starts; the model never performs the initial clone. - -The descriptor cannot supply credentials, host paths, commands, environment variables, materializer executables, or Docker options. - -A continuation supplies `previous_response_id` and must omit the extension. It reuses the original descriptor, attachment, access, retention, working directory, harness, and authentication binding. - -### Contract vocabulary and benchmark compatibility - -The JSON descriptor uses one canonical vocabulary rather than aliases for benchmark-specific names. `url`, optional `ref`, and `destination` describe requested Git materialization; `workingDirectory` describes the logical workspace-relative command directory. Public provenance preserves `requestedRef` separately from `resolvedCommit`. The contract does not also accept Harbor `git_url` or `workdir`, SWE-bench `repo` or `base_commit`, or Devfile `revision` or `clonePath`. - -The local `workspace.yaml` contract represents a different boundary: - -```yaml -repositories: - - path: ../api - url: https://github.com/acme/api.git - managed: sync - branch: main -``` - -`path` remains the existing or managed local checkout location. `url` replaces the lossy `source` plus `repo` pair. `branch` remains branch-specific because managed synchronization performs branch checkout and pull; it does not claim arbitrary detached-ref semantics. Path-only unmanaged entries may omit `url`; a managed entry requires it. The schema, CLI, generated schemas, examples, and tests cut over together without accepting both shapes indefinitely. - -Harbor sits beside the AllAgents gateway at Promptfoo's provider boundary. -Promptfoo calls the gateway over UHP for AllAgents-backed rows; a Harbor provider -calls Harbor for container-native rows, where Harbor owns task setup, execution, -verification, artifacts, and teardown. Harbor is not an -`allagents.workspace` backend, and its task schema is not compiled into the -workspace descriptor. - -Harbor and SWE-bench/Hugging Face remain useful precedents for source and -benchmark packaging. Their runnable images may combine source, tools, services, -verifier assumptions, and runtime configuration, so they are not AllAgents -`workspaceSnapshot` artifacts. An AllAgents snapshot contains source and may -carry normalized offline Git history; it does not select a runtime or verifier. - -### Access and retention - -Access and retention are independent: - -| | `readOnly` | `editable` | -|---|---|---| -| Workspace | Shared immutable generation | Private writable view; no mutable state shared across sessions | -| Initial UHP files | Rejected before source acquisition | Applied after attachment | -| Writes | Filesystem rejects them; no copy-up | Allowed within the private quota | -| Checkpoints | No source mutation checkpoint | Root and nested repositories use private checkpoints | -| Cross-session mutation | Impossible | Impossible | - -| Retention | Behavior | -|---|---| -| `session` | Default. Idle expiry starts only after durable terminal acknowledgement. | -| `persistent` | No idle expiry. Requires deployment authorization and reserved capacity before source resolution. | - -Neither retries nor continuations can change access or retention. - -### Editable change collection - -The verified generation is the first-turn baseline. The runner does not copy or -inventory the complete private workspace again before the harness starts. - -1. For a history-bearing root, the protected baseline is its recorded commit and - generation-owned object store. For a tree-only root, it is the canonical - workspace manifest. The runner keeps this baseline outside the editable - workspace. -2. For each turn, the runner prepares and verifies the private view, then arms - candidate tracking before it applies UHP input overlays or gives any - non-runner process writable access. It durably binds that coverage marker to - the generation and previous turn state and keeps tracking active through - descendant quiescence. -3. After the harness and its descendants stop, the runner obtains changed-path - candidates from the runner-owned tracker or storage state. It compares their - final type, mode, and content with the protected baseline through - root-confined, no-follow reads. -4. If uninterrupted coverage cannot be proven, or candidate state is missing, - incomplete, overflowed, or uncertain after recovery, the runner walks the - complete private view without following links and performs the same bounded - comparison. - -The produced-file domain is every source-visible path under the declared -workspace roots. The only exclusions are the original administrative `.git` -subtrees identified by the protected generation record. Their mutations persist -for continuation but are not produced files. An agent-created `.git` elsewhere -is ordinary source-visible content. Candidate and full-scan paths use this same -protected classification; final Git discovery or ignore rules cannot change it. - -For a continuation, the runner starts from the previous protected cumulative -path state and applies verified candidates, or rebuilds that state with the -fallback scan. It stores entries only for content that differs from the -generation; unchanged paths inherit their generation state. Comparing the new -state with the previous checkpoint yields the turn's produced-file delta without -retaining or comparing a second full workspace. - -Editable `.git` state remains session-private and survives continuation for -coding tools. After the harness starts, it is not authoritative for provenance -or change collection. The runner never trusts its refs, configuration, index, -hooks, alternates, or ignore rules, and an agent-edited ignore file cannot hide -a produced path. Candidate tracking is an optimization; the bounded full-tree -comparison remains the correctness fallback. - -### Public workspace metadata - -Once attachment reaches `ready`, terminal events, retrieval, background completion, replay, and later terminal failures return the same verified workspace object. - -| Public field | Meaning | -|---|---| -| `effectiveDescriptorDigest` | Digest of the normalized descriptor and defaults | -| `generationId` | Public content identifier | -| `sourceIdentity` | Normalized URL, destination, `requestedRef` when supplied, and `resolvedCommit`; or verified `snapshotName` and `imageManifestDigest` plus each root's destination and optional `resolvedCommit` and `objectSetDigest`, never a Git remote URL | -| `workingDirectory` | Effective `workspaceRoot` or `workspacePath` | -| `workspaceManifestDigest` | Verified source-visible manifest digest | -| `access`, `retention`, `expiresAt` | Effective workspace policy and expiry | - -`generationId` is the SHA-256 digest of versioned RFC 8785 bytes containing only returned source provenance, normalized destinations, and the workspace-manifest digest. It is metadata only. It is never a cache, authorization, attachment, or lookup key. - -Active turns and persistent sessions report `expiresAt: null`. For `session` retention, durable terminal acknowledgement sets the timestamp returned by terminal, retrieval, and replay paths. Polling and replay do not renew it. - -Failures before attachment reaches `ready` omit workspace metadata. Failures after `ready` include the complete committed object. +`metadata.harness_id` is HarnessRouter's existing harness selector. HarnessRouter documents custom harnesses as reusable configurations with a fixed base harness and selects them through `metadata.harness_id` ([custom harness behavior](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/README.md#L155-L163), [UHP selection](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/README.md#L186-L203)). For this product, a custom harness is the remote equivalent of a reusable profile. It is not an AllAgents CLI profile and is not projected from a developer machine. -Public metadata never exposes the private generation key, raw request digest, URL credentials, credential-scope mappings, selected credential references or values, redirect-chain URLs, resolved network addresses, physical paths, internal epoch, lease, reservation, claim, pin, or attachment identifiers, or other sessions' quota state. +`metadata.allagents.workspace` has exactly these fields: -## Source authority and acquisition - -### Configuration boundary - -| Source | Authority | -|---|---| -| Caller-requested Git | The UHP JSON descriptor supplies `url`, optional `ref`, and `destination` | -| OCI snapshot | Operator-owned `workspace.yaml` snapshot catalog plus caller-supplied `snapshotName`, `imageManifestDigest`, and `workspaceManifestDigest` | -| Harness, model, persistence, quota, and egress policy | HarnessRouter deployment configuration | -| Source credentials | Operator-owned secret store and credential-scope mappings | - -`workspace.yaml` is not a Git-origin allowlist. The service may accept any repository reachable through its safe public HTTPS egress boundary. - -### URL and network rules - -Before parsing a Git URL, validation rejects ASCII controls, whitespace, and backslashes. It then parses the URL once with the WHATWG URL Standard and requires the input bytes to equal the serialized URL exactly. - -The serialized URL must meet all of these rules: - -- scheme is `https`; -- hostname is an ASCII lowercase IDNA A-label DNS name without a trailing dot; -- no userinfo, query, fragment, IP literal, or explicit default port; -- path is non-empty; and -- no percent-encoded control, slash, backslash, or dot segment. - -The same serialization and structured `(scheme, host, effectivePort)` origin drive policy, credentials, redirects, DNS, provenance, generation identity, and the URL passed to Git and libcurl. Local paths and `file`, `ssh`, `git`, and extension transports are rejected. +| Field | Required | Contract | +|---|---:|---| +| `version` | yes | Exactly `"1"`. Other versions fail before source access. | +| `repository.url` | yes | Anonymous public HTTPS Git URL. No embedded credentials, userinfo, query, fragment, alternate protocol, or local path. | +| `repository.ref` | no | Advertised `refs/heads/*` or `refs/tags/*`, or unambiguous branch/tag shorthand. Omission means the remote default branch. Raw object IDs, other namespaces, ambiguous names, and unfetchable values fail. V1 accepts SHA-1 repositories only and records the resolved 40-hex commit. | +| `workingDirectory` | no | Relative POSIX directory beneath the checkout. Omission means the repository root. Absolute paths, empty components, `.`/`..` traversal, platform-specific separators, and any symlink escape fail. | -Every connection follows this sequence: +No other keys are accepted at any level of this object. The object cannot carry credentials, headers, environment variables, commands, destination paths, Docker settings, materializer selection, resource limits, retention, or provider configuration. Request size, string length, nesting depth, and parsing work are bounded before source access. -1. Route the acquisition child through the deployment connector. The child has no direct network path and no inherited proxy configuration. -2. Resolve the canonical hostname. Reject the whole answer set if any address is loopback, link-local, private, reserved, metadata, or otherwise non-public. -3. Pin one approved address for that connection so DNS rebinding cannot change the destination. -4. Accept at most five HTTPS redirects. Parse, serialize, resolve, and validate every hop again. +A first turn may omit `metadata.allagents.workspace`; stock HarnessRouter behavior then remains available. A session that starts without it cannot add it on a continuation. -Deployment policy may further restrict public egress, but it does not need to list every allowed repository. +After successful binding, public response metadata records the normalized requested URL, optional requested ref, exact resolved commit, and effective working directory under `metadata.allagents.workspace`. Internal checkout IDs and host paths remain private. The resolved commit, not a mutable branch or tag, is the provenance authority for the session. -### Source credentials +## First turn and continuation semantics -Callers cannot provide credentials or credential-reference names. +For a workspace-backed first turn: -A credential scope is either an exact structured origin or that origin plus a canonical repository-path segment prefix. Prefixes match complete path segments, never raw strings. The matching rule with the most path segments selects one server-owned secret reference. No match means anonymous acquisition. +1. HarnessRouter authenticates the caller, validates the UHP request, establishes idempotency, and resolves whether the request creates or reuses a session. +2. Any reused session, whether selected by `previous_response_id` or `metadata.session_id`, rejects workspace metadata. If it already has an AllAgents workspace binding, its stored harness and binding win and a caller-supplied harness mismatch fails. An unbound session with no workspace metadata retains upstream routing. +3. For a new workspace-backed session, a workspace-aware hydrate creates an isolated empty session allocation but skips the stock empty Git initialization. It reserves bounded materialization capacity before provider or harness execution. +4. The materializer resolves the optional advertised ref to one exact commit, builds and validates a private checkout in staging, then atomically publishes it into the empty runner-designated checkout root. +5. The runner keeps its control root as a sibling of the checkout, never inside repository content. It derives the execution working directory as a validated descendant of the checkout. +6. HarnessRouter atomically binds the descriptor, resolved commit, checkout root, control root, execution working directory, cleanup deadline, and selected harness to the session. +7. The selected custom harness starts in the execution working directory, while its home, credentials, scratch, skills, and checkpoint control state remain anchored under the runner-owned control root. -Preflight receives no request URL or secret value. It validates policy and configured reference syntax, then returns bounded reference names or opaque IDs. The runner verifies the selected handles before source access. +A continuation selects an existing session with `previous_response_id` or HarnessRouter's existing `metadata.session_id` recovery path and omits `metadata.allagents.workspace`. For a workspace-bound session, the gateway derives the harness from stored state; if the caller supplies a different harness, the request fails before hydration. The continuation reuses the exact private checkout, including edits from earlier turns, and the original resolved-commit provenance. Supplying workspace metadata on any reused session is invalid, even if byte-for-byte identical. The gateway never resolves the ref again, clones a replacement, changes the working directory, or silently starts a fresh session. -A source-access child receives only the selected value. It has an isolated home, -`GIT_CONFIG_NOSYSTEM=1`, no global Git configuration, no inherited proxy -variables, no Git or remote proxy configuration, and no direct network path. Its -ephemeral credential helper uses `credential.useHttpPath=true` and independently -enforces the selected protocol, host, port, and path scope. +If the bound checkout is expired, missing, corrupt, or cannot be proven to belong to the predecessor, continuation fails closed. There is no rematerialization, source fallback, or checkout substitution. The bounded ephemeral TTL is operator-configured; ordinary completion does not immediately remove a checkout that remains eligible for continuation. Expiry and explicit deletion use the same minimal idempotent cleanup path. -Every redirect is checked against the original scope. The connector strips the credential when a redirect leaves that scope, including a same-origin path escape. A redirect never selects a new credential. +## Materializer and downstream boundary -Credentials are never encoded in URLs, persisted in Git configuration or remote -URLs, or returned in hook output. Temporary credential state is removed before -return. The gateway, runner base environment, published generation, editable -view, and every agent child remain credential-free. If a configured source secret -appears in the service or agent environment, the runner refuses to launch the -agent. +The only maintained HarnessRouter seam is a workspace validate/materialize call **after session resolution and the workspace-aware hydration step, but before provider or harness execution**. It has two paths: -### Git resolution and verification +- first turn: allocate an empty session root without stock Git initialization, validate and materialize into its checkout child, create the separate control child, validate the execution working directory, and commit the binding; +- continuation: hydrate the bound session root, then load and verify the existing checkout, control root, and execution working directory without invoking source acquisition. -`ref` is at most 255 ASCII bytes. It may be a full 40-hex object ID or a ref name accepted by rules equivalent to `git check-ref-format`. +The downstream code recognizes only `metadata.allagents.workspace` and calls one configured in-image materializer. The caller cannot name an implementation. There is no registry, plugin lifecycle, generic hook graph, network materializer service, or reusable extension SDK. -Validation rejects leading dashes, whitespace, controls, refspec colons, glob metacharacters, traversal-like components, `@{`, and `.lock` components. It resolves a full ref or unambiguous branch or tag shorthand with `ls-remote`. A full object ID is accepted only when advertised. +The materializer contract is intentionally small: normalized descriptor in; an empty runner-assigned checkout target, cancellation, and fixed resource limits supplied by the runner; either a verified private checkout plus provenance, or a coded failure out. HarnessRouter remains responsible for session, checkpoint, file/artifact, and process lifecycle. Runner control state never lives inside the checkout, and the materializer cannot write it. -The materializer records the normalized URL, requested ref when present, and resolved commit in provenance. Fetch and checkout commands receive only the verified object ID, never caller ref text. +Requests without the AllAgents object retain upstream behavior, including routing for reused unbound sessions, and the pinned UHP conformance suite remains the protocol oracle. Each upstream rebase must review the patch against the metadata, session-resolution, hydration, checkpoint, runner-root, and custom-provider paths. If upstream gains an equivalent narrow lifecycle seam, remove the downstream patch rather than retain a compatibility layer. -Every Git and libcurl operation uses an argument vector without a shell and -explicit end-of-options handling. Hooks, `file` and `ext` protocols, submodule -recursion, Git LFS hydration, and configured clean and smudge filters are disabled -before the materializer touches caller-selected source. +## Provider authentication and harness configuration -The published repository keeps `.git` for coding tools, but the materializer normalizes it to a closed detached-HEAD state. It removes reflogs, `FETCH_HEAD`, locks, hooks, worktree links, alternates, shallow, replace, and graft state, extra refs and objects, and credential-bearing configuration. +Provider authentication is proxy-only. Each deployment configures one external OAuth-to-OpenAI-compatible gateway base URL and API key server-side. HarnessRouter represents that endpoint with two protocol-specific logical connections using the same secret: Responses for Codex and OpenAI Chat Completions for OMP. Each harness policy contains exactly its matching connection, with no fallback. The UHP caller cannot supply or override the endpoint, key, transport, or route. -The runner independently verifies: +The external OAuth gateway owns login, token persistence, refresh, and repair. AllAgents Gateway does not implement provider login, import local credentials, mount developer credential files, or coordinate token refresh. HarnessRouter's caller API key authenticates the UHP caller only and is never reused as a provider credential. -- `HEAD` resolves to the recorded commit; -- the index exactly matches that commit tree; -- the object database contains the complete required transitive closure, with no missing, corrupt, or extra objects; -- the canonical object-ID, type, and size set matches its recorded digest; and -- source-visible content equals the union of the resolved commit trees at their declared destinations, plus only the ancestor directories needed to connect them. +The deployment must use HarnessRouter's brokered sandbox mode, not the self-host image's `HR_SANDBOX_TRUST=owner` pass-through default. The gateway exchanges the long-lived external-gateway key server-side and gives each harness only a short-lived, session-scoped credential plus the loopback broker URL. Readiness fails if the local broker cannot mint and exchange that credential. The long-lived key never enters the harness process environment or session files. Scoped turn credentials may exist only in the active process environment or a per-turn ephemeral config root; the runner deletes them before checkpointing or exposing any file, artifact, log, or response. -Any undeclared path fails integrity validation. +Codex requires an OpenAI Responses-compatible endpoint. This matches the pinned runner, which states that current Codex supports Responses rather than Chat Completions ([Codex endpoint behavior](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L1907-L1915)); HarnessRouter supports Codex against custom endpoints that provide the Responses format ([provider compatibility](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/docs/self-hosting-guide.md#L331-L350)). OMP uses the same external gateway's OpenAI Chat Completions surface in v1, which its pinned builder supports ([OMP endpoint behavior](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L2609-L2644)). An incompatible route fails readiness or the turn; it never changes protocol or provider automatically. -### OCI snapshots +V1 custom harnesses use Codex or ordinary session-local OMP. OMP starts from the container's/session's own configuration. It does not import AllAgents profiles, host profiles, or developer state. There is no profile synchronization, profile projection, or AllAgents CLI integration. -Snapshot mode accepts only the direct OCI image manifest selected by -`imageManifestDigest` from the repository owned by the `snapshotName` catalog -entry. Redirects may not change registry authority. The config descriptor must -use that entry's configured workspace-manifest media type and address the -canonical bytes selected by `workspaceManifestDigest`. +## Source and workspace security invariants -| Limit | Maximum | -|---|---:| -| Distributable tar, gzip, or zstd layers | 64 | -| Image manifest | 4 MiB | -| Workspace-manifest blob | 128 MiB | -| Repository roots | 128 | -| Compressed layers | 8 GiB | -| Expanded tree | 32 GiB | -| Filesystem entries | 500,000 | -| One regular file | 4 GiB | -| One UTF-8 path | 4096 bytes and 128 components | -| One PAX or extended header | 1 MiB | +Repository content is untrusted. The implementation must preserve all of these invariants: -Workspace-manifest version 2 lets each repository item describe either a -tree-only root or a history-bearing root. A history-bearing item adds `git` with -`resolvedCommit` and `objectSetDigest`. Its destination must contain exactly one -`.git` directory; a tree-only root must contain none. The snapshot's immutable -digests bind the commit and object-set identity. The artifact contains no -configured Git remote, and no Git remote URL is required or returned. Private -evaluations can still use `git log`, `git blame`, and historical diffs offline. +- Caller authentication is required before session lookup, source access, continuation, retrieval, streaming, cancellation, file/artifact access, or deletion. Authentication failure does not disclose whether a session exists. +- Source is exactly one anonymous public HTTPS Git repository. URL parsing happens before DNS or process launch. Credentials and caller-controlled proxy settings are rejected and never forwarded. +- Every initial host and redirect target is re-parsed and re-authorized. DNS answers are checked against loopback, link-local, private, reserved, multicast, metadata-service, and otherwise non-public ranges; the approved address is pinned for the connection so DNS rebinding cannot change it. Redirect count, response size, and time are bounded. +- Git runs with a sanitized environment and isolated configuration. Interactive credentials, repository hooks, checkout filters, Git LFS, submodules, alternates, and non-HTTPS helpers or protocols, including `file`, `ssh`, and `ext`, are disabled or rejected. Repository configuration cannot weaken those rules. +- Ref discovery and fetch are bounded. V1 accepts only advertised branch/tag refs in SHA-1 repositories, ties the checkout to the exact resolved 40-hex commit, and records requested URL/ref plus resolved commit as provenance. +- The checkout is private and editable by one session only. No mutable state is shared across sessions. The runner-owned session root has separate checkout and control children; repository content can never overlap the control root. +- All path operations are rooted, no-follow where appropriate, and checked for traversal and symlink escape. The execution working directory must resolve to a real directory inside the checkout. Harness home, credentials, scratch, skills, and control state remain anchored under the sibling control root regardless of that working directory. +- Materialization and cleanup have hard process, descendant, wall-clock, byte, inode, file-count, and concurrency bounds. Cancellation terminates the complete acquisition process tree before cleanup and terminal acknowledgement. +- Partial staging is never attached. Cleanup is deterministic and idempotent after success, failure, cancellation, restart, expiry, and deletion. A path whose deletion failed is not reused or reported as free. +- Long-lived provider and caller credentials never enter Git arguments, the harness process environment, checkout or session files, response metadata, artifacts, logs, or provenance. A short-lived broker token may enter only the active harness environment or per-turn ephemeral config and is removed before checkpointing or public file collection. Source-controlled configuration cannot select the provider endpoint. -Before writing an entry, the materializer checks its type, path, link target, and -declared size. It rejects devices, sockets, traversal, escaping links, sparse -files, unknown or foreign layers, mutable tags, and undeclared output. The runner -independently rejects a 129th repository root. +HarnessRouter's per-session process isolation remains useful, but this deployment is not represented as a hostile-code sandbox. The service binds to loopback by default and requires an explicit operator decision and network controls before broader exposure. -After applying OCI whiteouts, the materializer verifies -`imageManifestDigest`, `workspaceManifestDigest`, every layer size and digest, -and the recomputed source-visible manifest. For every history-bearing root it -then applies semantic Git verification: detached `HEAD` at `resolvedCommit`, an -index equal to that commit tree, an object database equal to the complete -transitive closure whose canonical digest is `objectSetDigest`, and -source-visible descendants equal to the same commit tree. Dirty, staged, -untracked, missing, or modified source fails validation. +## Failure behavior -Snapshot Git state is offline. It must contain no remotes, branch-upstream -configuration, credential helpers, config includes, hooks, worktree links, -alternates, shallow, replace, or graft state, reflogs, `FETCH_HEAD`, extra refs, -unreachable objects, or credential-bearing configuration. Physical `.git` -entries and bytes count toward acquisition and retained-generation limits even -though their volatile representation is excluded from the source-visible -manifest. +The gateway fails closed without changing source, checkout, harness, model route, or provider protocol as a recovery shortcut. -### Canonical workspace manifest - -Both source modes produce the same versioned canonical manifest. Its RFC 8785 bytes enumerate every source-visible directory, regular file, and symbolic link in logical path order, including normalized mode, size, content digest, or link target. - -Repository roots are identified by unique, pairwise non-overlapping -destinations. A declared, separately verified `.git` subtree is omitted from -source-visible entries in either source mode; any undeclared `.git` path is -invalid. The runner verifies the manifest digest, walks staging without following -links, reconstructs the same source-visible entries, and requires byte-for-byte -canonical equality. It separately verifies every omitted Git root against its -declared commit and object-set digest. The manifest never appears inside the -published source tree. - -### Generation identity - -The private generation key is computed before materialization. It includes every input that can change source-visible bytes, filesystem semantics, or sharing authorization. - -| Included | Excluded | +| Failure | Behavior | |---|---| -| Descriptor and hook contract versions | Access and retention | -| Deployment authorization scope | Working directory | -| Normalized caller Git URLs | Harness, profile, and session identity | -| Resolved commits or exact OCI image and workspace-manifest digests | Physical paths | -| Normalized destinations | Credential values | -| Selected credential-reference identities | Caller ref spelling after it resolves to the same commit | -| Snapshot identity when applicable | Repository-mode volatile Git pack, index, and stat representation | -| Acquisition and egress policy version | | - -OCI generation reuse is artifact-exact. Repacking snapshot `.git` data changes -the image digest, generation key, and public OCI identity even when the semantic -Git state is unchanged. Semantic Git verification proves what one artifact -contains; it does not deduplicate distinct OCI artifacts. - -Publication binds one private key and one internal epoch to one verified workspace-manifest digest and every declared semantic Git-state record, whether Git was acquired from a remote or carried offline in an OCI snapshot. Materialization receives the exact private resolved plan and never resolves source again. - -## Generation publication and attachments - -A generation is immutable source content identified by its private generation key, manifest digest, and unique internal epoch. At most one live epoch may exist for a key. A replacement epoch cannot begin until durable logical and physical eviction of the prior epoch completes. - -Concurrent cache misses for the same key join one runner-owned build claim. Each -waiter keeps its own deadline and cancellation. Cancelling one waiter does not -cancel the build while another waiter remains; the runner cancels it when no -waiter remains. Failed or partial staging is never attachable, and a failed -competing build does not poison an existing verified generation. - -Before attaching a view, the runner acquires a provisional pin under the generation lock. Garbage collection cannot race that pin. - -The generation backing store remains owner-writable and is never exposed writable to a session. Publication is a recoverable same-filesystem atomic transition. Editable views may not share mutable state with the generation or another session. - -Read-only sessions share source bytes but keep their operating-system identity, -conversation, home, temporary files, logs, outputs, and response state separate. - - -The gateway and runner commit an attachment in three steps: - -1. The runner prepares the read-only mount or private writable view and returns opaque evidence. -2. The gateway commits attachment state as `ready`. -3. The runner acknowledges that commit, creates the durable reference or transfers the private reservation, and releases the provisional pin exactly once. - -Restart preserves a gateway-committed attachment. It rolls back an uncommitted prepare. - -`lastUsedAt` changes only when the gateway commits `ready`. Its initial value is null. A ready commit sets it to the later of the existing value and commit timestamp under the generation lock. Replay is idempotent. Publication and failed prepare do not count as use. +| Malformed, oversized, nested too deeply, or unknown workspace field | Reject as invalid UHP input before source access. | +| Disallowed URL, DNS answer, redirect, protocol, ref, or Git feature | Fail the response before attachment; remove bounded staging; do not start a harness or provider call. | +| Ref does not resolve to one permitted commit | Fail with source-resolution error; do not guess a default or fetch arbitrary objects. | +| Resource or concurrency limit unavailable | Reject or fail with a retryable capacity error before starting unbounded work. | +| Materializer timeout, crash, cancellation, or live descendant | Terminate and reap the process tree, clean staging idempotently, and return a coded failure. | +| Working directory missing, not a directory, or escaping through traversal/symlink | Fail before harness execution. | +| Workspace metadata present on any reused session | Reject the request without changing the existing session or extending its TTL. | +| Bound checkout expired, missing, corrupt, or mismatched | Fail continuation; do not clone, substitute, or resurrect it. | +| External OAuth gateway authentication or provider failure | Return the normalized UHP failure; do not switch endpoint, protocol, credential, or harness. | +| Cleanup failure | Keep the allocation unavailable, report operational failure, and retry the same idempotent cleanup path. | -Garbage collection orders candidates as follows: +Promptfoo treats non-success as an evaluation error. It does not turn gateway failures into empty successes or implicit retries. -1. Generations with null `lastUsedAt`, ordered by `publishedAt`. -2. Other generations, ordered by `lastUsedAt`, then `publishedAt`. -3. Ties use ascending generation-key bytes, then epoch-ID bytes. +## Repository, deployment, and release boundary -Only ready generations with zero references and zero provisional pins are candidates. +`allagentsdev/allagents-gateway` is the implementation product. The repository contains the preserved HarnessRouter downstream, narrow workspace patch, materializer, Docker Compose deployment, pinned Promptfoo dependency and scenarios, and image release workflow. `allagentsdev/allagents` remains the local Bun CLI repository and contains only this integration decision and planning material; v1 adds no `allagents gateway start` command or other CLI coupling. -## Session lifecycle, retention, and recovery +Once the existing fork has been renamed and `allagentsdev/allagents-gateway` exists, that repository's code, lockfiles, Compose file, limits, runbooks, and implementation documentation are authoritative for implementation detail. This ADR remains authoritative for the integration and product boundary. An implementation need that contradicts this boundary requires reconsidering the decision, not silently expanding the downstream. -### Session binding and continuation +The supported deployment is one container started by Docker Compose, bound to loopback by default, with durable `/data` and `HR_BACKENDS=codex,omp`. The materializer ships in that image; it is not another service. -Deployment configuration gives each harness exactly one authentication binding. -A proxy binding is a closed server-side record containing its private HTTPS base -URL, expected TLS identity or CA, supported API format and endpoint set, -gateway-only client-key handle, broker audience, and requested-to-proxy model -map. Callers cannot override these fields. +The public image is `ghcr.io/allagentsdev/allagents-gateway`. It has its own versions and release cadence, independent of the `allagents` npm CLI. Deployments pin image digests. Releases produce standard SBOM and build-provenance attestations and run upstream UHP conformance against the built image. -A target is advertised only after its binding passes readiness checks. Native -OAuth checks login, refresh, and a live turn. Proxy mode checks schema, TLS, -broker, model mapping, endpoints, and live compatibility. +Promptfoo is lockfile-pinned in the gateway repository and calls the built image directly over UHP. Release verification exercises both Codex and OMP through the configured external provider gateway. No intermediate evaluation repository or custom green-E2E attestation format is part of the product. -The first response binds one normalized descriptor, generation key and epoch, -access mode, retention class, working directory, harness target, authentication -mode, binding identity, and canonical binding-configuration digest. - -A continuation uses `previous_response_id`, omits workspace metadata input, -requires the exact bound attachment and binding-configuration digest, and -succeeds only when retention is persistent or the session idle deadline is still -in the future. - -A changed ref, resolved source, working directory, access, retention, harness, -authentication mode, binding identity, or binding-configuration digest requires -a new session. A continuation never re-resolves source, follows a changed -same-named connection, or substitutes a later generation epoch. - -Missing or corrupt generation, reference, publication, private workspace, or checkpoint evidence returns `allagents_workspace_non_resumable`. The system does not rebuild the missing state for that session. - -### Turn admission and idle expiry - -Every active operation holds a durable lease and has no idle expiry. - -One gateway compare-and-swap checks `session_busy`, exact binding, and expiry or -deletion together. A busy or invalid turn changes no deadline. - -For an eligible continuation, the gateway saves and clears the current idle -deadline in a provisional admission fence before the runner checks profile -readiness and reserves one configured concurrent-turn slot for that profile. -Success commits the session as active. A pre-allocation profile failure restores -the exact saved deadline when it is still future, or tombstones the session if -that deadline elapsed. - -After terminal acknowledgement, a `session` workspace receives one idle deadline. GET, polling, background completion, and replay never extend it. Persistent sessions keep `expiresAt: null`. - -### Capacity - -Every deployment limit must be finite and nonzero: - -| Capacity | Required limit | -|---|---| -| Sessions | Total active and retained sessions | -| Failed identity | Tombstone count, bytes, and TTL | -| Builds | Concurrent builds and staging bytes | -| Native authentication | At least two concurrent turns per profile; finite renewal timeout | -| Generations | Published count and bytes | -| Editable workspaces | Per-session hard bytes and inodes | -| Private storage | Total reserved bytes and inodes | -| Persistence | Persistent session count | -| Idle retention | Session idle TTL | +## Alternatives rejected -Before a response is visible, one idempotent admission token reserves a generic session slot and a fixed-size tombstone slot. Invalid descriptors remain charged through failed-response retention, tombstoning, and purge. - -After validation and before source resolution, the runner authorizes persistence -and reserves its slot. Editable access also receives one stable private-view -reservation ID. Its hard byte and inode allowance covers the writable view, UHP -overlays, root and nested-repository checkpoints, and produced-file state across -every turn and continuation. - -A cache miss reserves staging and prospective generation capacity before byte acquisition. Independent full-tree accounting converts that reservation to actual retained usage before publication. - -A waiter receives an editable view only when the complete generation fits its private allowance. A non-fitting waiter fails alone and does not invalidate the shared generation or another waiter. - -Protected state is never evicted. If leases, references, pins, or other protected resources consume capacity, admission fails instead. - -### Expiry and deletion - -Expiry and authenticated deletion follow this order: - -1. Atomically tombstone the session and reject new continuations. -2. Wait for active work, credential projections, and mounts to quiesce. -3. Remove private state. -4. Release every generation reference and reservation exactly once. -5. Record successful deletion, or keep failed physical deletion quarantined and counted for retry. - -A retained tombstone lives at least as long as matching response and idempotency records and returns `allagents_workspace_expired`. Bounded compaction removes the identity and reserved tombstone slot only after those records expire. Later requests receive the stock non-disclosing unknown-predecessor error. - -Neither path rematerializes source. - -### Restart recovery - -Completed unexpired or persistent sessions and ready generations survive restart. - -Before readiness or garbage collection, the gateway and runner reconcile generic -admission tokens and reservations, active leases, turn-admission fences, build -claims and waiters, staging and generation reservations, publications, pins, -references, mounts, editable usage, attachment prepare and acknowledgement, -credential projections, tombstones, compaction, quarantine, and interrupted -deletion. - -An internal `containment_pending` session remains non-terminal until its recorded cgroup is empty. Whole-container termination does not preserve agent processes; interrupted turns fail and are not replayed. - -## Provider authentication - -| | Native OAuth | Authenticated proxy | -|---|---|---| -| Default | Yes | No; explicit configuration only | -| Provider credential owner | Codex or Pi harness profile | Proxy service | -| Harness receives | Selected turn-scoped view of the shared profile | Non-refreshable scoped turn credential | -| Refresh | Harness-native through the per-profile renewal coordinator | Not allowed for the turn credential | -| Automatic fallback | Never | Never | - -Promptfoo's HarnessRouter API key authenticates the UHP caller only. HarnessRouter never translates it into provider credentials. - -Each configured native profile has one dedicated durable authentication root -outside generations, editable workspaces, session checkpoints, and conversation -state. Compatible harness targets may reference the same profile. Codex uses -file credential storage under `CODEX_HOME`. Pi uses -`~/.pi/agent/auth.json` after controlled `/login`. - -Missing, revoked, or unrefreshable native OAuth disables that harness target. An -expired but refreshable token is renewed through the coordinator. HarnessRouter -does not switch to another profile or provider route. - -Native OAuth uses an owner-trust boundary. During an active turn, the selected harness and same-operating-system-identity tools may read or emit that profile's credential. Operators that require stronger isolation must use the explicit proxy route or isolate the whole deployment more strongly. - -Multiple turns may use one native profile at the same time. Reading a valid token -does not require a lock. Renewal does, because two provider refresh calls using -the same old token can invalidate or overwrite each other. - -The pinned auth adapter remembers which credential contents a turn used and must -enter the runner-owned renewal coordinator before calling the provider's refresh -endpoint. Waiting for the renewal lock stops at the turn deadline. Once the -coordinator records a pending renewal, it—not the turn—owns the transaction: - -1. Acquire the renewal lock and reread the current credential file. -2. If its contents changed since the turn last read them, reload the saved - credential, release the lock, and skip the provider refresh call. -3. Otherwise, record the renewal attempt before the provider call. -4. Renew once and replace the credential file with a same-filesystem temporary - file, file `fsync`, atomic rename, parent-directory `fsync`, and validation. -5. Mark the renewal complete and release the lock as soon as the valid credential - is visible. - -Cancellation or the turn deadline cannot release a recorded renewal. The -coordinator uses its own finite renewal timeout. On timeout or process failure, -it stops the refresh process, waits until that process tree is gone, marks the -profile `repair-required`, and only then releases the lock. The turn cannot reach -terminal acknowledgement before that outcome is durable. - -A turn must never call the provider's refresh endpoint outside that coordinator. -If the pinned Codex or Pi version cannot acquire the renewal lock before calling -the provider, that target fails the phase-zero gate. - -A `repair-required` profile accepts no new turns or renewal attempts. A turn -already running may finish if its current access token still works; otherwise it -returns the normal provider-authentication failure. It never switches profiles or -activates the proxy. - -Login, logout, and repair use a separate durable maintenance fence. Its -`pending` state stops new admissions and waits for active turns to finish. While -the fence is pending or active, a new turn receives the retryable -`allagents_auth_profile_unavailable` error. If the operator deadline expires -before maintenance starts, the runner removes the pending fence and normal -admission resumes. - -Once maintenance becomes `active`, timeout or cancellation requests process -termination but never releases the fence. The runner waits for the process tree -to stop, validates the profile or marks it `repair-required`, records the -outcome, and only then resumes admission. Startup reconciles every pending or -active maintenance record before that profile becomes ready. - -A native turn follows this order: - -1. Reserve one configured concurrent-turn slot for the profile. -2. Project only that profile through a turn-scoped mount namespace or equivalent - same-filesystem view. Concurrent turns see the same atomically replaced - credential file. -3. Run the harness and descendants. The auth adapter coordinates renewal only - when needed. -4. After descendants stop, verify that turn has no incomplete renewal record. - Reconcile uncertain state or mark the profile `repair-required`. -5. Remove the projection and verify the retained session home is clean. -6. Persist terminal acknowledgement, then release the profile turn slot. - -HarnessRouter claims each `Idempotency-Key` atomically. Requests with the same -key share one result. Same-session overlap still returns `session_busy`. - -Different sessions may run concurrently with the same or different profiles. -Each profile has a finite concurrent-turn limit of at least two. Saturation fails -before response allocation with HTTP 503 `harness_unavailable` and reason -`allagents_auth_profile_capacity_exceeded`. - -The runner supervisor holds each turn's admission token and credential projection -through descendant termination, credential validation, projection teardown, and -terminal acknowledgement. It holds renewal ownership from the durable pending -record through commit or a durable `repair-required` fence. Gateway failure -cannot release the turn token. Runner failure leaves durable state. Startup -blocks that profile's admission until it reconciles every turn token, renewal -record, maintenance record, and stale projection. - -In proxy mode, the gateway issues a non-refreshable credential bound to one proxy -audience, harness target, model allowlist, response and turn ID, and the UHP -deadline plus minimal clock skew. It may authorize only the bounded provider -calls, compaction, and retries needed by that turn. Cancellation or terminal -completion revokes it. The broker rejects wrong audience, model, turn, expiry, or -revocation. Checkpoints, logs, artifacts, stored responses, and retained session -state never persist the proxy turn credential. - -## Fork boundary - -The HarnessRouter fork is limited to two generic seams: - -1. A pre-turn workspace hook with typed `preflight`, `validate`, `resolve`, and `materialize` operations. -2. A harness-authentication-state seam that keeps provider profiles separate from conversation and workspace state. - -| Hook operation | Responsibility | -|---|---| -| `preflight` | Validate contract and policy versions, configured credential references, and required tools without request URLs, network access, or secret values | -| `validate` | Apply descriptor defaults, validate URL, ref, destination, access, retention, and working-directory syntax, and select bounded credential references without source access | -| `resolve` | Resolve immutable Git commits or OCI identity and return the private resolved plan, generation key, effective working directory, and public provenance without writing source bytes | -| `materialize` | Consume the exact resolved plan on a cache miss, write only private staging and result roots, and return the manifest without publishing or re-resolving source | - -The generic fork understands only the configured metadata key, generic JSON and byte limits, immutable first-turn binding, the typed hook envelope, runner resource ownership, lifecycle state, and the response namespace. It does not understand the AllAgents schema, Git, OCI, or credential-selection policy. - -Requests without the extension keep stock behavior. Upstream UHP conformance must stay green. Production pins an upstream commit and carries a focused patch series with no unrelated changes. - -The upstream proposal should contain only the generic workspace and authentication-state seams. If upstream accepts an equivalent interface, remove the corresponding fork patch rather than keeping a compatibility layer. - -### Upstream status - -We rechecked upstream on 2026-09-27 at HarnessRouter -[`5f82db1d`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), -also released as -[`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4). -Upstream now has a session lease before workspace hydration, per-session -workspaces, durable checkpoints, safe input and plugin-package writes, and -per-turn API-key brokering. We will reuse those pieces rather than replace them. - -The two required seams are still missing at that pin. Ordinary request metadata -does not reach the runner; the source says that only its System One probe is -forwarded -([gateway source](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/gateway/app.py#L7289-L7293)). -The runner's authentication shape contains API keys, endpoints, and cloud -credentials, but no native profile identity or OAuth state -([authentication source](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L1386-L1405)). -Codex and Pi credential files are deliberately excluded from session checkpoints -([checkpoint source](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L476-L503)), -with no separately durable profile store or projection contract to replace them. -Upstream therefore does not provide the native-profile renewal coordination, -repair fencing, or profile-local readiness required here. - -Therefore stock HarnessRouter still cannot implement this decision. The maintained -fork remains necessary, but only for the two seams above. Every upstream pin -change must repeat this inspection. If upstream supplies either equivalent seam, -we delete that downstream patch. - -### Materializer containment - -Each hook invocation receives one runner-owned cgroup-v2 leaf under the delegated -subtree. - -| Property | Requirement | +| Alternative | Why rejected | |---|---| -| Placement | Put the child in the leaf atomically with `clone3(CLONE_INTO_CGROUP)`, or use a stopped, secret-free pre-exec move-and-verify handshake | -| Authority | The child and its descendants cannot administer or escape the leaf | -| Termination | Cancellation, deadline, or parent exit with live descendants fails the invocation; use `cgroup.kill` when descendants remain | -| Proof | Require `cgroup.events` to report `populated 0` before reading a result, publishing, releasing a secret, cleaning roots, or exposing terminal state | - -If the leaf cannot be emptied, internal state becomes `containment_pending`. -Readiness and terminal visibility remain blocked until restart reconciliation -proves it empty and records the preserved outcome once. - -## Trust, deployment, and release - -HarnessRouter API authentication is mandatory on every externally reachable -create, continuation, retrieval, stream, cancellation, file, artifact, -persistence, deletion, and lifecycle-administration endpoint. This remains true -on a private network. Authentication fails before resource lookup, disclosure, or -mutation, so an unauthenticated request reveals neither session existence nor -retention state. - -The gateway-to-runner channel is mutually authenticated and not externally routable. The service binds to loopback or a private network with equivalent ACL or firewall controls. Version one is not a public multi-tenant service. - -HarnessRouter supplies per-session operating-system identities. It is not a hostile-code sandbox. No session identity may write the generation backing store. Editable workspaces, homes, temporary files, output, and checkpoints remain session-private. - -The pinned production image contains: - -- an OCI base image pinned by digest; -- the pinned HarnessRouter CE commit and reviewed patch series; -- the AllAgents materializer and locked runtime dependencies; -- version-locked OS packages and Git or OCI tools; and -- pinned HarnessRouter-supported Codex and Pi versions. - -Only the runner receives the delegated cgroup v2 subtree. The container uses `on-failure` restart policy. - -Startup begins in non-serving mode. Readiness requires all of the following: - -- verified image attestations and mounted inputs; -- usable cgroup delegation and cleanup of orphaned materializer cgroups; -- removal or quarantine of stale credential projections; -- finite lifecycle and capacity limits; -- writable staging and editable volumes; -- a protected generation store; -- verified mount, same-filesystem, quota, and isolation relationships; -- successful materializer preflight and native or proxy authentication checks; and -- complete startup reconciliation before serving or garbage collection. - -Operators must be able to observe aggregate generation, editable-workspace, persistent-session, tombstone, quarantine, and failed-deletion capacity without exposing source paths or credentials. - -AllAgents publishes the public `linux/amd64` image as `ghcr.io/allagentsdev/harnessrouter`. Version and commit tags are discovery labels, not immutable deployment identities. Deployments pin the manifest digest. - -The release workflow uses commit-pinned actions and separates untrusted, -credential-free pull-request testing from protected release testing. A -credential-bearing job never runs for a pull-request event or PR-controlled ref. -It accepts only an image artifact whose recorded digest, source commit, and -workflow identity match the approved release ref. - -Green release verification runs the service, not just its build. A GitHub Actions -job starts the exact image as a local container on loopback or a private Docker -network, waits for readiness, and runs the AI Evals repository's locked Promptfoo -CLI and provider against it. This is an ephemeral test deployment, not a public -HarnessRouter service. A hand-written HTTP call may provide an additional smoke -test, but it cannot replace the real Promptfoo path. - -Pull-request jobs use a deterministic local provider fixture and receive no real -provider credentials. An environment-approved release job mounts the real Codex -and Pi profiles and proves native login, continuation, concurrency, renewal, and -cleanup. If a standard GitHub-hosted runner cannot provide the required mount or -cgroup behavior, the job receives a dedicated one-job self-hosted runner. That -runner is destroyed after credential teardown and is never reused for an -untrusted job; the workflow may not weaken or skip checks to fit a runner. - -The protected workflow pushes an untagged candidate by digest, creates -GitHub/Sigstore build-provenance and SBOM attestations, and anonymously pulls that -digest into a fresh protected job. That job repeats the complete Promptfoo path, -including native Codex and Pi cases, then creates a signed green-E2E attestation -for the same digest. Only after all three attestations verify does the workflow -apply version and commit discovery tags or mark the release complete. Deployment -requires the expected repository, workflow, approved ref, subject digest, -predicate, base-image digest, lockfiles, OS packages, source tools, Codex and Pi -versions, pinned AI Evals commit, Promptfoo lockfile identity, and successful -green-E2E attestation. A candidate without that final attestation is not -deployable. - -Promptfoo and AI Evals are pinned by the AI Evals lockfile and commit; the -workflow never downloads a floating `promptfoo@latest`. GitHub package -permission replaces third-party registry credentials. - -## Failure behavior - -The system fails closed. Source, access mode, retention, credentials, and provider route never change as a recovery shortcut. - -| Failure | Public behavior | Required effect | -|---|---|---| -| Non-object workspace extension | HTTP 400 `invalid_input` before allocation | Reject before session lookup or durable admission | -| Extension exceeds 64 KiB or 32 levels | HTTP 413 `allagents_workspace_too_large` before allocation | Reject before canonicalization, session lookup, or durable admission | -| Invalid bounded descriptor shape, URL syntax, ref syntax, destination, or working-directory syntax | `allagents_workspace_invalid` after allocation | Reject without source access | -| Unknown or unadvertised ref | Failed response during resolution | Allow only bounded remote ref resolution; acquire no source bytes and create no attachment or agent | -| `workspacePath` is missing or not a directory | `allagents_workspace_invalid` after checking the verified manifest | Reject before attachment or agent launch | -| Unauthorized `persistent` retention | `allagents_workspace_persistence_forbidden` | No source resolution or byte acquisition; no downgrade | -| Initial files with `readOnly` | `allagents_workspace_read_only` | No source acquisition and no writable shadow layer | -| Workspace extension on a continuation | Invalid request | No session mutation, lease, or deadline change | -| Expired or deleted retained session | HTTP 410 `allagents_workspace_expired` | No runner or profile work; no rematerialization | -| Purged predecessor | Stock non-disclosing unknown-predecessor error | No rematerialization | -| Missing or corrupt bound attachment evidence | HTTP 409 `allagents_workspace_non_resumable` | No profile admission or epoch substitution; return committed workspace metadata | -| Native profile reaches its configured concurrent-turn limit after replay and `session_busy` checks | HTTP 503 `harness_unavailable`, reason `allagents_auth_profile_capacity_exceeded` | Fail before allocation, runner work, or materialization | -| Native profile is unavailable, `repair-required`, or under maintenance | HTTP 503 `harness_unavailable`, reason `allagents_auth_profile_unavailable` | Fail before allocation without switching profile or activating the proxy | -| Generic capacity unavailable | HTTP 503 `allagents_workspace_capacity_exceeded` | Admit no response or source work | -| Editable view or later growth exceeds its allowance | `allagents_workspace_private_quota_exceeded` | Fail only that waiter or turn; preserve mode and retention | -| Source policy, authentication, or transport failure | Coded failed response | Remove unpublished staging; publish nothing; start no agent or provider fallback | -| Generation, attachment, or private-view failure | Coded failed response | Quarantine incomplete state and release reservations and pins exactly once | -| Materializer timeout, crash, malformed output, or live descendant | Coded materializer or containment failure | Wait for cgroup quiescence before result handling, secret release, or cleanup | -| Provider authentication failure | Normalized UHP failure | Do not switch profile or activate proxy; validate credentials, remove the projection, and release turn capacity | -| Provider execution failure | Normalized UHP failure | No source fallback and no credential material in output | - -Capacity is reserved in order: generic session and tombstone before response visibility; persistence and editable allowance after validation and before source resolution; staging and prospective generation after resolution and before byte acquisition; actual retained usage before publication. Each reservation is released or transferred exactly once. - -A failed physical deletion remains quarantined and counted. The system never advertises that capacity as free, resurrects the resource, or permits a same-key replacement before physical deletion completes. - -New vendor codes use the `allagents_` prefix. Promptfoo maps every non-success to a coded error and never converts failure into empty success or automatic retry. - -## Consequences - -HarnessRouter remains the sole execution and session control plane. AllAgents adds workspace preparation and source policy without adding another streaming API, process supervisor, artifact service, provider adapter, or task engine. +| Build a new execution gateway | Duplicates HarnessRouter's UHP, sessions, streaming, cancellation, files, artifacts, and harness supervision. | +| Put a thin service in front of stock HarnessRouter | Splits checkout and session ownership across services and still cannot place the workspace at the correct runner lifecycle point. | +| Create a third repository that consumes the HarnessRouter fork | Loses the clear downstream history and adds a release/rebase boundary without adding product isolation. | +| Wait for stock HarnessRouter | The pinned version accepts arbitrary metadata but forwards only the System One probe; it has no workspace lifecycle semantics. | +| Add a general metadata extension or materializer framework | V1 has one object and one implementation. A framework would enlarge the fork before a second use case exists. | +| Put repository instructions in the prompt or a model tool | Makes acquisition model-dependent, non-deterministic, too late to set the initial working directory, and unsafe for credentials and provenance. | +| Make the local AllAgents CLI or its profiles the remote control plane | Couples a local developer tool to an independently deployed service and duplicates HarnessRouter custom harnesses. | +| Manage provider login inside AllAgents Gateway | Duplicates the external OAuth gateway's ownership of login, refresh, and repair and expands the credential attack surface. | +| Upload every source file through UHP | Pushes acquisition to every caller and loses authoritative Git ref-to-commit provenance and repository behavior. | -Shared read-only generations avoid repeated acquisition and may serve concurrent sessions using the same or different harness profiles. Editable sessions trade that reuse for a reserved private byte and inode envelope. +## Deliberate v1 limits -Persistent sessions, active references, provisional pins, retained tombstones, and quarantined deletion failures consume finite capacity. Crash-consistent accounting and admission rejection are operational requirements, not optional optimizations. +V1 supports one anonymous public HTTPS Git repository using SHA-1 object IDs, one private editable checkout per session, an optional advertised branch/tag ref, an optional safe working directory, exact 40-hex commit provenance, and bounded ephemeral retention. -Native OAuth deliberately trusts the selected harness and same-identity tools during an active turn. Source-acquisition credentials remain outside that boundary and never enter a generation or agent environment. +V1 does **not** include raw commit-ID requests, SHA-256 repositories, multiple repositories, private-source credentials, OCI sources, caller-selected runtime images, shared or read-only generations, cross-session caching, persistent workspaces, user-selected TTLs, session branching, checkout migration, or elaborate tombstone and garbage-collection machinery beyond minimal idempotent cleanup. It uses HarnessRouter's existing file and artifact behavior rather than inventing produced-file tracking. -The maintained fork must be rebased and tested against selected upstream releases until equivalent supported seams exist. +Only Codex and OMP are required and release-validated. Other upstream backends and a future Copilot harness are outside this decision. There is no local-profile import, host-profile projection, provider-route override, automatic provider fallback, public multi-tenant authorization model, scoring service, dataset service, or evaluation task engine. -## Alternatives rejected +## Consequences -| Alternative | Why rejected | -|---|---| -| Custom execution gateway | Duplicates mature UHP session, streaming, cancellation, authentication, artifact, and provider behavior | -| Proxy-first provider authentication | Adds a mandatory API key and network hop when native Codex or Pi authentication works | -| Thin adapter in front of stock HarnessRouter | Adds another network service and pushes source lifecycle outside the session control plane | -| Descriptor in the prompt | Lets the model control acquisition and is neither deterministic nor safe | -| Acquisition through an MCP tool | Runs only if the model chooses it and cannot define the initial working directory | -| Upload every source file as UHP input | Loses exact Git history, symlink, mode, and OCI layer semantics and moves acquisition to every caller | -| Rematerialize a private tree for every trial | Repeats network, CPU, and storage work and prevents safe immutable sharing | -| Wait for upstream | Makes delivery depend on a project we do not maintain | +AllAgents Gateway inherits a mature UHP execution plane and keeps the maintained patch reviewable. The cost is an ongoing rebase obligation against pinned HarnessRouter releases and ownership of a security-sensitive Git materializer. -## Deliberate limits +Each session pays for a private checkout and cannot reuse a shared generation. That is intentionally less efficient than a source platform, but it makes mutability, provenance, continuation, quota, and cleanup ownership understandable for v1. -Version one does not add evaluation datasets, Harbor task ingestion, SWE-bench/Hugging Face ingestion, caller-selected runtime images or verifiers, scoring, assertions, automatic retries, session branching, concurrent turns within one session, caller-supplied credentials, non-HTTPS or private-network Git origins, public multi-tenancy, arbitrary materializer commands, mutable OCI tags, transparent source-mode fallback, or guaranteed provider prompt-cache hits. +Provider credential lifecycle stays outside the gateway. This reduces credential code and operational states in the downstream, at the cost of requiring a compatible external OAuth gateway and making its availability part of the service's readiness. -Read-only attachments never copy up or become editable. Editable sessions never share mutations. Callers cannot choose arbitrary TTLs, bypass persistence quotas, or change retention on continuation. Leased, referenced, or pinned state is never evicted. Default retention is always bounded. +The gateway and CLI can release independently. Promptfoo tests the same image and UHP surface that operators deploy. ## Reconsider when -Revisit this decision when: - -- either required native harness fails the phase-zero gate; -- the host cannot enforce immutable read-only generation mounts across sessions; -- continuation, expiry, deletion, and lease acquisition cannot be linearized and recovered safely; -- generation churn or authorized persistent demand cannot fit practical finite quotas; -- editable derivation requires stronger filesystem semantics than a private writable view with no shared mutable state; -- upstream HarnessRouter accepts the generic workspace or authentication-state seam; -- UHP adopts a standard workspace attachment or retention contract that replaces this extension; -- the maintained patch grows beyond the narrow integration boundary; -- HarnessRouter removes required UHP, session, or provider behavior; -- exact per-turn workspace rollback becomes a requirement; -- the host cannot enforce public-address egress validation, address pinning, redirect revalidation, and out-of-scope credential stripping; -- source acquisition needs a stronger isolation boundary; -- callers require a public multi-tenant authorization model; or +Revisit this decision if: + +- UHP standardizes a workspace attachment with equivalent first-turn and continuation semantics; +- upstream HarnessRouter adds an equivalent narrow post-hydration, pre-execution workspace seam; +- the downstream patch grows beyond metadata recognition, binding, and one fixed materializer invocation; +- a second materializer is approved and proves that a registry is simpler than explicit code; +- private repositories, multiple repositories, OCI sources, persistent workspaces, or shared immutable caching become validated product requirements; +- public multi-tenancy or stronger hostile-code isolation becomes a requirement; +- Codex can no longer use the external gateway's Responses surface, or OMP cannot use its configured compatible surface; +- the external gateway can no longer own provider login, refresh, and repair; +- private editable checkouts cannot meet practical storage and cleanup bounds; or - another UHP implementation offers a materially smaller and more stable integration surface. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index c91f18cb..daa47c3e 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,2490 +1,887 @@ --- -title: "UHP Coding-Agent Execution through HarnessRouter - Plan" +title: "AllAgents Gateway v1 - Implementation Plan" date: 2026-09-18 updated: 2026-09-27 type: feat artifact_contract: ce-unified-plan/v1 artifact_readiness: implementation-ready -product_contract_source: ce-plan-bootstrap execution: code --- -# UHP Coding-Agent Execution through HarnessRouter - Plan - -## Goal Capsule - -- **Objective:** Let Promptfoo and other authenticated UHP clients run a - configured Codex or Pi harness against a verified AllAgents Git or OCI - workspace generation. Concurrent read-only trials may share that immutable - generation across harnesses and profiles; each editable trial receives a - private writable workspace. A continuation reuses the same session attachment - through `previous_response_id`. Default sessions expire under bounded policy; - explicitly authorized persistent sessions remain pinned until deletion. -- **Means:** Deploy a pinned HarnessRouter CE fork. Preserve HarnessRouter's UHP, - caller authentication, session, streaming, cancellation, artifact, and - agent-runner behavior. Add a generic generation resolve/build/attach boundary, - immutable generation store, shared read-only mounts, private writable views, - durable leases, retention and quota state, garbage collection, nested logical - directories, mode-specific checkpoint/collection behavior, and separation - between session state and durable harness-native OAuth state. Implement - Git/OCI semantics in a separate AllAgents executable. Codex authenticates - through `codex login`; Pi - authenticates through its `/login` flow for the selected provider. An - API-key-authenticated proxy is an explicit last-resort target mode. Optional - describes deployment configuration, not release scope: version one implements - and verifies it for operators that reject the native owner-trust boundary. -- **Authority:** [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) - owns the protocol, fork, trust, generation, attachment, retention, - harness-authentication, and provider-routing decisions. UHP `2026-09-12` and - HarnessRouter's conformance suite own execution-wire behavior. The namespaced - UHP JSON extension owns caller-supplied HTTPS Git URLs, refs, destinations, - per-session source selection, access, retention request, workspace-relative - working directory, and provenance semantics. Project `workspace.yaml` remains - ordinary local workspace configuration plus the optional operator-owned OCI - snapshot catalog; - it is not a Git origin catalog for UHP. HarnessRouter deployment configuration - owns egress policy, source-credential scope mappings, harness IDs, model - allowlists, authentication bindings, persistence authorization, finite TTLs, - quotas, and garbage-collection policy. -- **Execution order:** First build the minimal custom image and pass the blocking - native-auth adapter gate for both Codex and Pi without implementing the - AllAgents materializer. Then prove the generation claim/publication seam, - shared read-only attachment, private editable derivation, lease fencing, and - retention state against the real runner. Freeze the generic hook and - AllAgents contracts; implement Git then OCI generation construction; implement - TTL/pinning, quotas, deletion, and garbage collection; prove explicit proxy - mode separately; run concurrent read-only, editable, continuation, expiry, and - Promptfoo E2E; complete release, fork-maintenance, and upstream documentation. -- **Stop conditions:** Stop before production workspace implementation if either - required native target cannot pass the phase-zero gate: real login, first turn, - continuation, binding persistence, profile isolation, concurrent same-profile - turns with valid and expired tokens, coordinated renewal, refresh fault - behavior, and passive-persistence checks. - Also stop if the host cannot enforce immutable multi-session read-only mounts; - concurrent identical requests can publish more than one generation; editable - sessions can alias writable state; continuation, lease, expiry, deletion, and - GC cannot be fenced crash-safely; protected state can be evicted; retained - storage cannot be bounded; nested Git workspaces cannot be collected without - corrupting HarnessRouter checkpoints; source credentials enter a generation or - harness; the gateway/runner serializes harness OAuth files into source, - checkpoints, produced records, passive logs, or public metadata; or the fork - cannot preserve stock UHP behavior. Do not fall back to prompt instructions, - an MCP acquisition tool, client-side repository upload, another generation, - access or retention mode, OAuth profile, implicit API-key route, second - execution protocol, or parallel task/session engine. -- **Tail ownership:** Implementation owns focused tests in both repositories, - upstream UHP conformance, built-image smoke tests, exact Git/OCI E2E, native - Codex/Pi OAuth and explicit proxy-mode E2E, two-turn Promptfoo success and - failure verification, credential boundary checks, documentation, and a clean - upstreamable HarnessRouter patch series. - ---- - -## Product Contract - -### Summary - -AllAgents uses HarnessRouter as the execution gateway. -HarnessRouter exposes UHP, authenticates callers, creates and persists sessions, -streams events, runs configured Codex and Pi harnesses, handles cancellation and -idempotency, and returns output, usage, and artifacts. - -The missing product-specific capability is deterministic source-generation -resolution before the first agent turn. A focused HarnessRouter fork calls a -generic hook after allocating the UHP session but before provider selection. The -Promptfoo request names the HTTPS Git repositories to load; the AllAgents -executable validates those URLs against deployment egress policy, resolves an -immutable source plan, and builds verified staging only on a generation cache -miss. The runner atomically publishes or reuses the generation, records the -session attachment and retention state, then attaches either a shared read-only -mount or a private writable view before provider dispatch. - -A continuation supplies `previous_response_id`, omits the workspace extension, -and uses HarnessRouter's current native conversation plus the bound attachment. -Read-only sessions see the same immutable generation; editable sessions see the -same private mutations. A different source ref, working directory, access mode, -retention class, harness, or authentication binding requires a new session. -An expired or deleted session is never silently rematerialized. - -### Problem Frame - -HarnessRouter already implements the generic execution concerns. Reimplementing -them in AllAgents would add a second protocol, lifecycle, session store, process -supervisor, artifact model, provider integration, and conformance burden without -differentiating the product. - -Stock HarnessRouter does not expose a documented generic pre-turn seam for a -server-side Git/OCI descriptor. It does not copy arbitrary request metadata into -response metadata or forward it to ordinary Codex/Pi runners. Input files are -written before the agent and support nested paths, but client-side expansion -loses exact symlink, mode, Git-history, and OCI layer semantics and makes every -caller responsible for acquisition. The temporary fork closes those seams. - -### Actors - -- **A1. UHP caller:** Promptfoo or another application holding a HarnessRouter - API key. It chooses a configured HarnessRouter harness ID and model, prompt, - caller-supplied HTTPS Git repositories or configured OCI snapshot, initial - workspace policy, and optional continuation predecessor. -- **A2. HarnessRouter gateway:** Authenticates and validates UHP; owns response - and session identity; is the sole writer of session attachment, expiry, and - tombstone state; treats the configured workspace metadata value as bounded - opaque JSON; drives prepare/ack before provider fallback; and returns hook - metadata on every response path. -- **A3. AllAgents materializer:** A subprocess executable that exposes no - listening service. It validates the AllAgents descriptor and caller Git URLs, - reads deployment acquisition policy and the optional project OCI snapshot - catalog, resolves immutable Git/OCI source plans, builds private staging on - cache misses, validates the tree, and returns generation identity and - provenance. It never authorizes persistence or publishes live state. -- **A4. HarnessRouter runner:** Owns generation claims/publication and the - resource journal: provisional pins, durable references, read-only mounts, - private writable views and quotas, per-session operating-system identity and - runtime state, mode-specific checkpoints and produced files, safe nested cwd, - selected Codex/Pi process, conversation state, active-turn auth projection, - cleanup, and garbage collection. It prepares attachment evidence but never - writes gateway session attachment/expiry/tombstone transitions. -- **A5. Harness-native auth profile:** One dedicated durable credential root for - one Codex or Pi harness target. Multiple turns may use it concurrently. The - harness owns login and token refresh; the pinned auth adapter coordinates only - renewal. The runner projects the profile only for an active turn and verifies - teardown before acknowledgement. Checkpoints, backups, and public metadata - never copy it. Active harness access is part of the owner-trust boundary. -- **A6. Optional authenticated proxy:** A last-resort, explicitly configured - target mode. The gateway keeps the long-lived proxy client key, the proxy owns - upstream provider authentication, and the harness receives only a - non-refreshable, scoped turn credential that the HarnessRouter broker validates. -- **A7. Operator:** Pins and deploys the custom image, mounts durable generation, - session, editable-workspace, and auth storage, completes each native harness - login, configures HTTPS egress and optional source-credential scopes, - authorizes persistent sessions, configures finite TTL/byte/inode/count/ - tombstone quotas, operates deletion and GC, selects explicit proxy targets, - and controls private-network access. - -### Key Decisions - -- **Use UHP as the northbound contract.** UHP `2026-09-12` is the only - northbound execution contract. HarnessRouter conformance is authoritative. -- **Fork narrowly and upstream later.** Delivery uses an AllAgents-maintained - fork. The upstreamable layer is a configured opaque-metadata key, immutable - first-turn binding, typed validate/resolve/materialize envelopes, generation - claim/publication, attachment prepare/ack and lease lifecycle, safe nested cwd, - mode-specific checkpoint/collection integration, bounded retention/GC, and - separation of durable harness-auth state from session state. It contains no - AllAgents Git/OCI schema logic. Upstream acceptance is not critical-path. -- **Run the AllAgents component behind HarnessRouter.** The materializer is a - subprocess hook, not another HTTP gateway and not a custom agent backend. -- **Publish once per generation; attach once per session.** Concurrent requests - for one immutable source plan share one claim and verified publication. - Read-only sessions share that generation; editable sessions receive private - writable views. Continuations omit the extension and reuse the original - attachment through `previous_response_id`. -- **Separate access from retention.** `readOnly` versus `editable` controls - mutability. Default `session` versus authorized `persistent` controls - lifetime. Neither axis changes the other, and continuation can change neither. -- **Bound retained state.** Active leases, durable references, provisional pins, - and persistent sessions are protected. Expired state and bounded tombstones are - purged before deterministic eviction of unreferenced/unpinned generations. - Admission fails when protected state consumes finite quota. -- **Let authenticated callers select Git origins.** Promptfoo supplies canonical - HTTPS Git URLs, optional refs, and unique destinations. The same URL may appear - more than once at different refs or destinations. The service accepts any - repository reachable through safe public egress; callers cannot supply - credentials, non-HTTPS transports, host paths, commands, or Docker options. -- **Use one canonical workspace vocabulary.** The execution descriptor keeps - `url`, optional `ref`, `destination`, and logical `workingDirectory`; response - provenance keeps `requestedRef` separate from `resolvedCommit`. Local - `workspace.yaml` repository entries keep `path` and replace the provider- - specific `source` plus `repo` pair with one canonical `url`. Benchmark-specific - aliases are not accepted by the canonical schema. -- **Keep provider lanes separate.** Promptfoo calls the AllAgents gateway over - UHP for AllAgents-backed rows. A Harbor provider calls Harbor for - container-native rows, where Harbor owns setup, execution, verification, - artifacts, and teardown. Harbor is not an `allagents.workspace` backend and - its task schema is not compiled into the workspace descriptor. Harbor and - SWE-bench/Hugging Face remain packaging precedents, but their runnable images - are not AllAgents source snapshots. -- **Prefer harness-native OAuth.** Promptfoo's HarnessRouter API key authenticates - the UHP caller only. Codex and Pi use their own login, token storage, refresh, - and provider request path; native mode has no provider-route API key. -- **Make proxy auth explicit.** `proxyApiKey` is a last-resort target mode for a - compatibility or stronger-isolation requirement. It is optional to configure - but remains required version-one implementation and verification scope. OAuth - failure never activates it, and a session never changes its persisted - authentication binding. -- **Preserve stock UHP requests.** Requests without the configured metadata key - behave exactly as upstream. -- **Use a custom HarnessRouter image.** The image combines a pinned HarnessRouter - revision, reviewed patch series, pinned agent runtimes, and the AllAgents - materializer executable. -- **Publish from an AllAgents-owned registry.** Release the public image as - `ghcr.io/allagentsdev/harnessrouter`; tags identify releases, but deployment - and E2E pin the published manifest digest. - -### Requirements - -#### UHP, authentication, and routing - -- **R1.** Pin HarnessRouter CE to a reviewed upstream commit and UHP version - `2026-09-12`. The deployment must pass the applicable upstream conformance - suite without weakening, replacing, or reinterpreting stock UHP behavior. -- **R2.** Require a HarnessRouter API key for every externally reachable UHP, - response/session retrieval, stream, cancellation, file, artifact, persistence, - deletion, and lifecycle-administration endpoint. Reject unauthenticated - requests before disclosing resource existence, expiry, retention, or metadata. - Gateway-to-runner operations are not externally routable and are - mutually authenticated. Bind the service to loopback or a private interface - and document the remaining need for Tailscale ACLs, firewall policy, or - equivalent network controls. -- **R3.** HarnessRouter deployment configuration owns stable harness IDs, - backend, model allowlist, and exactly one auth binding: - `{ mode: "nativeOAuth", profile: ConfigName }` or - `{ mode: "proxyApiKey", connection: ConfigName }`. A proxy connection is a - closed server-side record containing private HTTPS base URL, expected TLS - identity/CA, HarnessRouter-supported API format and endpoint set, gateway-only - proxy-client-key secret handle, broker audience, and requested-to-proxy model - map. Callers cannot override any field. Requests select a harness with stock - `metadata.harness_id` and a model with `model`; AllAgents profiles are not - projected into this catalog. Codex and Pi targets default to `nativeOAuth`. - A target is advertised only after its selected auth binding passes: profile - login, concurrent renewal, and live-turn checks for native OAuth, or schema, - TLS, broker, model-map, endpoint, and live compatibility checks for - `proxyApiKey`. On the first turn, the gateway persists the harness target, - auth mode, binding - identity, and canonical binding-config digest in the session. Continuations - require that exact binding; deployment config changes never switch it. -- **R4.** Add a durable auth root outside session workspaces with one - least-access directory per configured native profile. A profile belongs to one - harness type and provider, but compatible targets may reference it. Controlled - setup runs `CODEX_HOME= codex login` with file credential storage for - Codex or runs Pi `/login` in an isolated Pi home for the configured provider. - - The runner projects only the selected profile's exact auth files into each - session-specific CLI home. Conversation and rollout state remain - session-scoped. The projection exists only for the active turn and uses a - directory-level mount namespace or equivalent same-filesystem view. Concurrent - turns using one profile see the same atomically replaced credential file; no - credential is copied into durable session state. - - Each native profile has a finite concurrent-turn limit of at least two. A value - below two is invalid configuration. Admission - preserves UHP precedence: atomically claim the `Idempotency-Key`, return - `session_busy` for a second turn in one session, then reserve a profile turn - slot. Same-key arrivals share the owner's result without a second reservation. - Different sessions may reserve slots on the same profile. Saturation fails - before response allocation with HTTP 503 `harness_unavailable` and - `detail.reason: "allagents_auth_profile_capacity_exceeded"`. - - For a continuation, one session CAS checks `session_busy`, attachment and - binding evidence, and expiry or deletion. It then saves and clears the original - idle deadline in a provisional fence before profile readiness and capacity - admission. Admission success commits the session as active. Pre-allocation - failure restores the exact future deadline or tombstones the session if it - elapsed. - - Reading a valid credential needs no profile-wide lock. When the pinned auth - adapter loads a credential, it records a private version equal to the SHA-256 - digest of the exact credential-file bytes. The digest remains inside the auth - coordinator and is never logged or returned. - - Waiting to acquire the runner-owned renewal mutex is bounded by the requesting - turn's deadline. After acquisition, the coordinator rereads the profile: - - 1. If the current version differs from the adapter's base version, reload the - saved credential, release the mutex, and skip the provider refresh call. - 2. Otherwise, durably record the turn ID, base version, and pending state before - the provider call. - 3. Call the provider once and replace the credential file with a - same-filesystem temporary file, file `fsync`, atomic rename, - parent-directory `fsync`, and validation. - 4. Record the committed version and release the mutex as soon as the valid - credential is visible. - - Once the pending record is durable, the coordinator owns the transaction - independently of turn cancellation or deadline. It uses a separate finite - renewal-transaction timeout. On timeout or caller-process failure, it - terminates the refresh process, proves the process boundary empty, durably - marks the profile `repair-required`, and only then releases the mutex. The - requesting turn cannot reach terminal acknowledgement before that outcome is - durable. - - No turn may call the provider's refresh endpoint outside this coordinator. If - the pinned Codex or Pi version cannot acquire the mutex before the provider - call, that target fails the phase-zero gate. The runner never changes profile - or auth mode. - - A `repair-required` profile rejects new admissions and renewal attempts. A turn - already running may finish with its current access token; provider rejection - becomes its normal authentication failure. The runner never switches profile - or auth mode. - - The runner owns a durable admission token for each active turn, not one - exclusive lock for the whole profile. It holds that token and the turn's - credential projection through descendant termination, projection teardown, - and gateway acknowledgement of durable terminal state. Renewal ownership lasts - from the durable pending record through a committed credential or a durable - `repair-required` fence. Runner failure leaves durable admission and renewal - records; startup blocks only that profile's readiness until it proves - descendant boundaries empty, removes stale projections, validates credentials, - and reconciles every turn, renewal, and maintenance record. - - Login, logout, and repair use a durable maintenance journal with - `pending | active | completed | failed` states. `pending` stops new profile - admissions and waits for active turns to finish. While maintenance is pending - or active, new turns fail before response allocation with - `allagents_auth_profile_unavailable`. If the operator deadline expires before - `active`, the runner records failure, removes the fence, and resumes admission. - Once `active`, timeout or cancellation requests process termination but never - releases the fence. The runner proves the administrative process boundary - empty, validates the profile or marks it `repair-required`, records the - outcome, and only then resumes admission. Startup reconciles every maintenance - record before that profile becomes ready. `proxyApiKey` uses neither native - profile turn slots nor the renewal coordinator. - - The gateway and runner never serialize auth files into root or nested - checkpoints, produced-file records, passive logs or traces, materializer - input, backups, or response metadata. Native mode is an explicit owner-trust - boundary: the harness and same-operating-system-identity tools may read or emit - the selected credential during an active turn. Preserve HarnessRouter - streaming, cancellation, idempotency, files, artifacts, retention-bounded - completed session persistence, per-session UID and runtime isolation, and - immutable generation sharing. - -#### Workspace extension and hook - -- **R5.** On an initial response request, accept one optional JSON object of at - most 64 KiB and 32 levels at `metadata["allagents.workspace"]`. - HarnessRouter checks only generic bounds, canonicalizes the opaque value with - RFC 8785, records the raw request descriptor digest, and binds it to the new - session. A continuation must omit this key; generic gateway validation rejects - an extension-bearing continuation before session lookup/CAS and changes no - session state or deadline. The AllAgents hook's source-free `validate` - operation validates and defaults the exact v1 object - `{ version: "1", access, retention?, source, workingDirectory? }`. - `access` is exactly `readOnly | editable`; omitted `retention` means `session`, - otherwise it is exactly `session | persistent`. - - `source` is exactly one of: - - `{ kind: "repositories", repositories: NonEmptyArray<{ - url: HttpsGitUrl, ref?: RefText, destination: RelativeDirectory }> }`; or - - `{ kind: "workspaceSnapshot", snapshotName: ConfigName, - imageManifestDigest: Digest, workspaceManifestDigest: Digest }`. - - `workingDirectory` is exactly `{ kind: "workspaceRoot" }` or - `{ kind: "workspacePath", path: RelativeDirectory }`. The workspace path is - relative to the mounted workspace and must name a directory in the resolved - source manifest. It works for both repository and snapshot sources. The hook - validates and reports requested retention but never authorizes it. - The runner is the sole persistence authority: before source resolution or byte - acquisition it authorizes `persistent`, reserves the session and persistence - slots, or fails `allagents_workspace_persistence_forbidden`. -- **R6.** The AllAgents hook expands omitted `retention` to `session` and omitted - `workingDirectory` to `{ kind: "workspaceRoot" }`; an omitted repository `ref` - remains absent and means the remote symbolic HEAD. It canonicalizes each HTTPS - URL, NFC-normalizes strings, sorts repository entries by destination, rejects - unknown fields, and hashes RFC 8785 bytes as the effective descriptor digest. - A separate canonical generation key covers only inputs that can affect source- - visible bytes, declared agent-visible filesystem semantics, or sharing - authorization: hook/schema versions, deployment authorization scope, normalized - caller Git URLs, bounded selected credential-reference identities, resolved - commits or the exact OCI `imageManifestDigest` and - `workspaceManifestDigest`, normalized destinations, `snapshotName` when - applicable, and acquisition/egress policy version. Access, retention, logical - cwd, harness/profile, session identity, physical paths, and credential values - do not fragment that key. In repository mode, volatile Git pack, index, and - stat representation also does not fragment it. OCI reuse is artifact-exact: - repacking a snapshot changes its image digest and therefore its generation key - even when its semantic Git state is unchanged. Publication binds the key to the - independently verified workspace-manifest digest and every semantic Git record. - Omitted and explicit default values have the same effective descriptor digest. - The raw request descriptor digest records the exact initial JSON only in - private session state; public response metadata names and returns only - `effectiveDescriptorDigest`. -- **R7.** Replace the one-shot session materialization call with one private - runner `/workspace/prepare` operation outside the provider candidate loop. It - invokes a configured executable directly without a shell using typed - `validate`, `resolve`, and `materialize` commands. `validate` performs only - source-free schema/default/URL/destination/policy checks and returns a private - normalized descriptor reference/digest plus effective access, requested - retention, logical cwd, effective descriptor digest, and the bounded sorted - credential-reference names/opaque IDs selected by deployment policy for the - requested URLs—not values. The runner verifies those references were - declared by preflight and that their handles exist, then maps only that selected - set into source-access child environments. After runner authorization and - admission, `resolve` consumes that exact validated descriptor and selected set, - resolves exact Git commits or OCI identity, and returns a private canonical - resolved-plan path/digest, generation key, effective cwd, and bounded public - provenance. `materialize` receives that exact resolved-plan path/digest and - selected set and never re-resolves source. - - For each ready lookup or completed build, the runner validates ready evidence - and acquires a durable provisional attachment pin under the same generation - lock before returning it to one session. A miss creates one runner-owned keyed - build claim with a configured maximum lifetime of at most 900 seconds, - independent of any one UHP request deadline. The claim atomically takes - ownership of the validated source-only resolved-plan bytes and selected - credential-reference identities in its private durable root and binds their - digest to the key before acquisition. - Concurrent sessions attach as waiters to that claim. Each waiter applies its own - cancellation and deadline; cancellation detaches only that waiter, the build - continues while any live waiter remains, and the runner cancels and cleans the - build when none remain. All live waiters receive the one publication or build - failure without making one request's shorter deadline authoritative for the - others. - - The validate request carries the bounded ordinary workspace-input-file count - so `readOnly` fails before source access. Before response allocation, the - idempotent admission transaction reserves one generic workspace-session slot - and one fixed-size tombstone slot; it publishes neither token nor response - unless both are durable. These reservations remain through post-allocation - validation failure, failed-response retention, tombstoning, and purge. After - validate but before resolve, the runner authorizes and reserves any - persistence slot and, for `editable`, creates one stable private-reservation ID - for the full configured per-session byte/inode allowance. - - Before a miss claim acquires source bytes, its owner reserves the full - configured staging and prospective-generation byte/count allowance. After - materialization, the runner's independent no-follow full-tree walk, including - separately validated Git administrative state, computes physical retained - byte/inode usage. Under the generation lock, successful publication atomically - converts the prospective generation reservation to actual usage, releases its - excess and the staging reservation, and persists that accounting transition - before ready state or waiter pins become visible. Private-view fit is not a - shared-build condition: after publication, each editable waiter compares total - physical generation bytes/inodes with its own hard allowance. A waiter that - cannot fit fails `allagents_workspace_private_quota_exceeded` and releases only - its access-specific reservations/pin; read-only and fitting editable waiters - continue. If none remain, the valid ready generation has zero pins and is GC - eligible. Build failure occurs before publication or attachment and before - agent launch; unpublished staging and access/build-specific reservations/pins - release exactly once, while generic session/tombstone reservations follow - failed-response lifecycle. - - Hook stdin is at most 128 KiB, stdout 1 MiB, and stderr 64 KiB. Source values - come from an owner-only runner secret mount or credential-store handle, never - the gateway/runner base environment. Preflight returns bounded configured - credential-reference identities but receives no values; the runner verifies - handle presence itself. For each source-access operation it resolves only the - selected validated references and constructs the allowlisted child - environment. Startup and per-request rechecks reject configured secret names - or values in the service or agent environment. - - Before injecting secrets, the runner creates a per-hook cgroup v2 leaf under a - delegated subtree and starts the child inside it atomically with - `clone3(CLONE_INTO_CGROUP)` or a stopped, secret-free pre-exec - move-and-verify handshake. On every outcome it closes streams, uses - `cgroup.kill` when needed, and proves `cgroup.events` reports `populated 0` - before accepting success, reading private results, publishing a generation, - releasing secrets, or cleaning roots. A completed parent with a live - descendant fails `allagents_workspace_containment_breach`. An unquiescent leaf - enters internal `containment_pending`; the runner exits, restart keeps - readiness false, and no terminal result becomes public until the old boundary - is proven empty. - - A completed materialize result contains only generation-scoped data: the - generation key, private `workspace-manifest.json` reference/digest, declared - repository roots, semantic Git validation records when applicable, and bounded - verified source identity. Each waiter retains its own effective descriptor - digest, logical cwd, access/retention, and requested-source provenance from - validate/resolve; a shared build result never overwrites them. The runner - independently validates the result, staged tree, manifest, and key, then makes - the generation backing tree owner-writable only and atomically publishes it. A - hook failure never enters provider fallback. -- **R8.** Persist separate CAS-protected state machines. A generation epoch is - `absent -> building -> ready`, `quarantined`, or `deleting`; it stores its key, - unique internal epoch ID, `publishedAt`, nullable `lastUsedAt`, manifest digest, - publication/accounting marker, physical byte/inode counts, durable reference - count, provisional attachment pins, and build waiters. There is at most one - live publication per key/epoch and one result per concurrent claim. A new epoch - for the same key may begin only after the prior epoch's durable logical and - physical eviction completes; it serves new sessions only. Attachments bind key - plus epoch, so an existing session never substitutes a rebuilt epoch. - - A gateway-owned session attachment is - `unbound -> validating -> resolving -> attaching -> ready`, - `containment_pending`, `expired`, `deleting`, `deleted`, `purged`, or `failed`; - it stores raw/effective descriptor digests, generation key/epoch, access, - retention, logical cwd, expiry, harness/auth binding, runner-supplied - attachment evidence, and any provisional turn-admission fence with its original - deadline. The gateway is the sole writer of session attachment, expiry, turn- - admission-fence, and tombstone transitions. The runner alone writes its - generation and resource journal. - - Before provider dispatch, the runner prepares resources and durably returns an - opaque attachment token and evidence; it does not bind the gateway session. - The gateway CASes `attaching -> ready`, persists that evidence, and - acknowledges the token. For `readOnly`, the runner holds the provisional pin - while it acquires a session-long generation-epoch reference and verifies a - read-only mount with no writable alias; after the ready acknowledgement it - releases the provisional pin exactly once. For `editable`, it requires the - stable private-reservation ID admitted before resolve and compares the - generation's independently measured total physical bytes/inodes with that - allowance. Failure detaches only that waiter. A fitting waiter creates and - validates a private writable view with no mutable state shared with the - generation or another session. A backend may use a full copy, reflink, - copy-on-write view, or storage clone only after proving the same isolation, - quota, accounting, and cleanup behavior. The runner initializes root/nested - checkpoints and stores protected collection state outside the editable - workspace. Its attachment evidence contains the epoch and opaque reservation - ID. Ready acknowledgement transfers the reservation from admission to the - private workspace without a second debit, then releases the provisional pin - exactly once. Failure before acknowledgement releases the reservation and - prepared resources once unless reconciliation proves that the gateway - committed ready. Expiry or deletion releases the ready workspace's reservation - once. Startup reconciles both halves of this prepare/ack and quota-transfer - protocol. - - The editable hard quota covers the private view, UHP input overlays, - root/nested checkpoints, and produced-file state for every turn and - continuation. The filesystem quota backend denies writes beyond either byte or - inode allowance and surfaces exhaustion to the runner; the runner terminates - that turn as failed `allagents_workspace_private_quota_exceeded` without - changing access or retention. Actual usage and reserved allowance are persisted - and reconciled before readiness. Only editable state receives ordinary UHP - input files, mutation checkpoints, and produced-file collection. A `readOnly` - request containing workspace input files fails before source acquisition. - - The verified generation is the first-turn collection baseline; attachment - does not walk, hash, or copy the complete private view again. For a - history-bearing root, the protected descriptor names its recorded commit and - generation-owned object store. For a tree-only root, it names the canonical - workspace manifest. The runner stores these descriptors outside the editable - workspace. - - For every turn, the runner prepares and verifies the private view, then arms - candidate tracking before it applies a UHP input overlay or gives any - non-runner process writable access. It durably binds that coverage marker to - the generation and prior protected turn state. Tracking remains active through - runner-applied overlays and harness-cgroup quiescence. - - After the harness cgroup is empty, the runner obtains additions, deletions, - type and mode changes, and content-change candidates from a runner-owned - change tracker or storage state. It verifies every candidate against the - protected descriptor and final workspace with root-confined, no-follow reads. - Candidate tracking is an optimization. If uninterrupted coverage cannot be - proven, or its state is missing, incomplete, overflowed, or uncertain after - recovery, the runner walks the complete private view without following links - and reconstructs the bounded cumulative difference from the generation. - - Before terminal acknowledgement, the runner durably stores protected path state - only for content that differs from the generation. Unchanged paths inherit - generation state. On continuation, it applies verified candidates to the prior - cumulative state, or rebuilds that state with the fallback scan, then compares - the result with the prior state to produce the turn delta. It never retains or - compares a second full workspace. A path restored to its prior-turn state - produces no turn delta; transient-write auditing is outside this contract. - - The produced-file domain is every source-visible path under the declared - workspace roots. The only exclusions are the original administrative `.git` - subtrees identified by the protected generation record. Their mutations - persist for continuation but are not produced files. An agent-created `.git` - elsewhere is ordinary source-visible content. Candidate and full-scan paths - use this same protected classification; final Git discovery or ignore rules - cannot change it. - - Editable `.git` state remains part of the private session for coding tools and - continuation, but collection never trusts its repository identity, refs, - configuration, index, hooks, alternates, or ignore rules. An agent-edited - ignore file cannot hide a produced path. - - `lastUsedAt` remains null until the gateway commits an attachment `ready`. - After acknowledgement, the runner updates it under the generation lock to - `max(existing, readyCommitTimestamp)`; startup can replay a missed update - idempotently from committed gateway evidence. GC orders null `lastUsedAt` - epochs first by `publishedAt`, generation-key bytes, and epoch-ID bytes; then - non-null epochs by `lastUsedAt`, `publishedAt`, key bytes, and epoch-ID bytes. - Publication or failed prepare does not count as use. - - A continuation requires attachment `ready`, the exact generation key/epoch, - access, retention, harness, and auth binding, and either persistent retention - or a `session` idle deadline strictly later than the admission instant. One CAS - also rejects `session_busy`, saves and clears that deadline in a provisional - admission fence, and marks admission pending. Profile success commits active; - pre-allocation profile failure restores the saved future deadline or tombstones - the session if it elapsed. Only durable terminal acknowledgement starts a new - idle deadline. GET, stream polling, and idempotent replay do not renew it. - Missing or corrupt attachment evidence fails non-resumable without acquisition - replay or replacement. Provider fallback sees only `ready` state and cannot - resolve, build, attach, or change policy. - The runner resolves a symlink-safe logical cwd - beneath the mounted generation or private workspace and rejects cross-session - or escaping paths. - -#### Source acquisition and provenance - -- **R9.** Make one clean public `workspace.yaml` repository-schema cutover: - retain `repositories[].path` as the existing or managed local checkout - location; replace the provider-specific `source` plus `repo` pair with one - optional canonical credential-free HTTPS `url`; retain `branch` because - managed synchronization implements branch checkout and pull rather than - arbitrary detached refs. Path-only unmanaged entries may omit `url`; any - truthy `managed` entry requires it. `workspace repo add` records a normalized - URL, converting recognized SSH provider remotes to canonical HTTPS without - persisting userinfo. An explicit one-time migration rewrites unambiguous - legacy provider/identifier pairs and rejects unknown or credential-bearing - forms with repair guidance. The normal parser and generated v2 schema accept - only the new shape; there is no dual-field compatibility path. - - The execution materializer parses the project `workspace.yaml` through that - authoritative schema only for optional strict project-owned - `workspaceSnapshots` entries: - `{ name: ConfigName, repository: OciRepository, - workspaceManifestMediaType: MediaType, executionCredential?: "${ENV_VAR}" }`. - Reject unknown fields, literal secrets, and duplicate snapshot names; snapshot - entries do not merge with user configuration. Local `repositories[].url` - entries never form an execution allowlist and are not copied into a UHP - request. - - Repository mode takes one through 128 request entries. Each has a canonical - absolute `https` `url`, optional `ref`, and unique, pairwise non-overlapping - `destination`. URLs need not be unique. Before parsing, reject ASCII controls, - whitespace, and backslashes. Parse once with the WHATWG URL Standard and - require the input bytes to equal its serialized URL exactly. The serialization - must have an ASCII lowercase IDNA A-label DNS hostname without a trailing dot, - no userinfo/query/fragment or IP literal, no explicit default port, a non-empty - repository path, and no percent-encoded control, slash, backslash, or dot - segment. The same serialization and structured `(scheme, host, effectivePort)` - origin are used for policy, credentials, redirects, DNS, provenance, - generation identity, and the exact Git/libcurl request. Local paths and non- - HTTPS schemes fail source-free validation. Destinations are non-empty, - non-root relative child paths and cannot collide with HarnessRouter's root - checkpoint repository. Duplicate or ancestor/descendant destinations, escaping - destinations, and unsupported URL forms also fail. - - HarnessRouter deployment configuration owns harness/model/provider targets, - persistence authorization, TTLs, quotas, GC, outbound egress policy, and - optional source-credential scope mappings. A scope is either an exact - structured origin or an origin plus canonical repository-path segment prefix; - path prefixes match only complete segments, never raw strings. The matching - rule with the most path segments selects one secret reference; callers never - select the reference or supply its value. No match means anonymous acquisition. - Deployment policy may narrow public egress but does not require every - repository URL to be predeclared. Project `workspace.yaml` never contains - session access, retention, lease, or eviction state. -- **R10.** Repository mode materializes exactly the caller-declared repository - set. Use the requested `ref`, or the remote symbolic HEAD when omitted. - `RefText` is at most 255 ASCII bytes and is either a full 40-hex object ID or a - `git-check-ref-format`-equivalent ref name. Reject leading dashes, whitespace - and controls, refspec colons, glob metacharacters, traversal-like components, - `@{`, and `.lock` components. Resolve a validated full ref, or an unambiguous - shorthand under `refs/heads/` or `refs/tags/`, with `ls-remote`; accept object - IDs only when advertised. Subsequent fetch/checkout commands receive only the - verified object ID with explicit end-of-options handling, never caller ref text. - - Allow only argument-vector HTTPS Git operations through the deployment's - acquisition egress connector. The child cannot bypass it: clear every proxy/ - `NO_PROXY` environment variable, disable Git `http.proxy` and remote proxy - configuration, and permit no direct network path. Before every connection and - each of at most five HTTPS redirects, resolve the canonical hostname and reject - the entire answer set if any address is loopback, link-local, private, reserved, - metadata, or otherwise non-public; pin one approved address for that connection - so DNS rebinding cannot escape the check. Parse and serialize every redirect by - the same URL rules and compare structured origins. Re-evaluate the originally - selected credential scope at every hop, strip its credential whenever the - target leaves that scope—including a same-origin path-prefix escape—and never - select a new credential because of a redirect. - - Use an isolated HOME plus `GIT_CONFIG_NOSYSTEM=1`, no global config, - `credential.useHttpPath=true`, and an explicit ephemeral credential helper - bound to the selected structured origin/path scope. The helper independently - rejects any protocol, host, effective port, or canonical repository path - outside that rule. Disable hooks, `protocol.file`, `protocol.ext`, submodule - recursion, Git LFS hydration, and configured clean/smudge filters. - Preserve each repository's `.git` directory for the coding agent, but do not - treat volatile Git administrative bytes as generation identity. The - materializer constructs a hermetic detached-HEAD repository at the resolved - commit, removes reflogs, `FETCH_HEAD`, lock/shallow/replace/graft state, hooks, - worktree links, alternates, extra refs, unreachable objects, and - credential-bearing configuration, and normalizes the allowed config/ref set - and index. The workspace manifest excludes declared repository `.git` - administrative subtrees. The runner separately proves each allowed `.git` - path belongs to its declared root; HEAD resolves to the recorded commit; the - index equals that commit tree with no staged delta; configuration and refs are - closed; and the object database equals the complete transitive object closure - of the commit with no missing, corrupt, or extra objects. It records a canonical - sorted object-ID/type/size-set digest as part of semantic Git state. - - The runner independently reads each resolved commit tree, prefixes it with that - repository's destination, and requires the complete source-visible manifest to - equal exactly the union of those trees plus only the destination ancestor - directories needed to connect them. Undeclared files, links, or directories - outside that union fail integrity validation. Publication records the semantic - Git validation alongside the manifest digest. Pack compression/layout and - index stat-cache data may vary physically but cannot change the semantic - Git-state record, generation key, or source-visible manifest digest. Failure - never falls through to snapshot mode or another credential identity. - Limit one materialization to 128 repositories, 500,000 filesystem entries, and - 32 GiB across the staged workspace. A count or byte violation returns failed - `allagents_workspace_limit_exceeded`. Exhausting the declared UHP time budget - returns `incomplete` with `error: null`; only an unexpected execution timeout - before that budget returns failed `timeout`. Every outcome proves the cgroup - empty before removing staging and starts no provider. -- **R11.** Snapshot mode constructs a server-side immutable OCI reference from - the repository configured by `snapshotName` and the caller-provided - `imageManifestDigest`. Accept only a direct OCI image manifest with at most 64 - distributable tar/gzip/zstd layers. Its config descriptor must use the catalog - entry's configured workspace-manifest media type and address the canonical - bytes selected by `workspaceManifestDigest`; redirects may not change registry - authority. Verify both named manifests plus every layer size and digest before - use; apply OCI whiteouts; limit the image manifest to 4 MiB, the workspace- - manifest blob to 128 MiB, its `repositories` array to 128 items, total - compressed layers to 8 GiB, expanded bytes to 32 GiB, entries to 500,000, one - regular file to 4 GiB, paths to 4096 UTF-8 bytes and 128 components, and one - PAX/extended header to 1 MiB. The runner independently rejects a 129th - repository root even when the archive and fetched manifest otherwise agree. - Reject devices, sockets, traversal, escaping links, sparse files, unknown or - foreign layers, mutable tags, and undeclared output. - - Each snapshot repository root is either tree-only or declares - `git: { resolvedCommit, objectSetDigest }` in the workspace manifest. Snapshot - destinations are pairwise non-overlapping. A tree-only root rejects `.git`. A - history-bearing root must contain one `.git` directory at its destination. - The producer must normalize it before publication; the materializer - independently verifies the detached `HEAD`, exact index/tree, complete - transitive object closure, and canonical object set required by repository - mode. It reads the declared commit tree and requires every source-visible - descendant of that destination to equal it, with no staged, dirty, missing, or - untracked path. It rejects rather than repairs nonconforming state and requires - the computed digest to equal `objectSetDigest`. - - Snapshot Git state is offline: reject every remote, branch-upstream setting, - credential helper, config include, hook, worktree link, alternate, shallow, - replace, graft, reflog, `FETCH_HEAD`, extra ref, unreachable object, and - credential-bearing configuration. The Git administrative state contains no - configured remote, and no Git remote URL appears in the workspace manifest or - returned provenance. Exclude only declared and - verified `.git` subtrees from source-visible entries; any other `.git` path - fails integrity validation. - Physical Git entries and bytes still count toward acquisition and - retained-generation limits. - - Recompute the canonical workspace manifest from staging and require it to - match both the fetched manifest bytes and `workspaceManifestDigest`. Publication - preserves that manifest and every semantic Git record as the protected - collection baseline; it does not build a second per-session inventory. Both - source modes produce the same reusable immutable-generation abstraction. - Validated `.git` state from either mode is readable but immutable in `readOnly` - attachments and independently writable only in private `editable` views. - Generation acquisition limits apply per build; retained-generation and - private-workspace quotas apply independently. -- **R12.** Extend HarnessRouter's response translator and stored-response paths - with a stage-dependent contract. Before attachment `ready`, non-2xx request - errors and allocated terminal failures omit - `response.metadata["allagents.workspace"]`; error codes identify the failed - stage, and no placeholder or partial/unverified generation identity is emitted. - Once attachment commits `ready`, streaming events, provider terminal responses, - GET, background completion, and idempotent replay return the same immutable - bounded fields: extension version, `effectiveDescriptorDigest`, public - `generationId`, canonical workspace-manifest digest, logical cwd, access, - resolved retention, source completeness, and resolved Git/OCI provenance. - Snapshot `sourceIdentity` retains `snapshotName` and `imageManifestDigest` and - mirrors each verified root's `destination` and optional `git` declaration in - the exact shape below. History-bearing roots expose `resolvedCommit` and - `objectSetDigest`, but no Git remote URL. - `generationId` is the SHA-256 digest of versioned RFC 8785 bytes containing - only the returned normalized source provenance, normalized destinations, and - workspace-manifest digest. It is metadata-only and is never a cache, - authorization, attachment, or lookup key. Active streaming metadata has - `expiresAt: null`. For `session` retention, durable terminal acknowledgement - atomically sets `expiresAt`; the terminal event, stored response, GET, - background completion, and idempotent replay then return that same timestamp. - `persistent` always returns `expiresAt: null`. Metadata may return normalized - caller-supplied repository URLs as provenance but never contains the private - generation key, raw request digest, generation epoch, redirect-chain URLs, - resolved network addresses, deployment credential-scope mappings or selected - references, physical paths, credential values, lease/attachment/reservation - tokens, counts, authorization rules, or other sessions' quota state. -- **R13.** The materializer resolves `${ENV_VAR}` references from its allowlisted - child environment, uses hermetic Git/registry configuration, removes temporary - auth files before returning, and emits no secret. Prove with a deliberately - innocuous variable name and value that source credentials and the HarnessRouter - caller API key are absent from the gateway/runner base environment, every - staging tree, published generation, private editable workspace, agent - environment, nested Git remote/config, generated CLI configuration, log, - checkpoint, and response. In native mode there is no provider-route API key. - The selected OAuth profile is intentionally readable by the harness trust - boundary only through its active-turn projection; the gateway/runner never - copies it into staging, generations, durable session homes, private workspace - checkpoints, produced-file records, backups, passive logs, materializer input, - or public metadata. Finalization and restart reconciliation verify projection - absence. An active same-identity harness or tool can exfiltrate it; that risk - is explicit in owner-trust mode. - - In proxy mode, the long-lived proxy client key and upstream provider - credentials stay in their owning services. The non-refreshable broker token is - bound to one proxy audience, harness target, model allowlist, response/turn ID, - and the UHP deadline plus minimal clock skew. It may authorize the bounded - provider requests, compaction, and retries required during that active turn; - cancellation or terminal completion revokes it. Passive persistence never - stores it. The HarnessRouter broker rejects wrong-audience, wrong-model, - wrong-turn, expired, or revoked tokens. -- **R14.** Configure finite, nonzero limits for session idle TTL, renewal - transaction timeout, staging bytes and concurrent builds, published-generation - bytes/count, per-editable-session hard bytes/inodes, total private reserved - bytes/inodes, total sessions, authorized persistent sessions, and tombstone - bytes/count/TTL. Every native profile also has a finite concurrent-turn limit - of at least two. Workspace response admission reserves one generic - session slot and one fixed-size tombstone slot before making the - response/session visible, so semantic validation failure and later - expiry/deletion cannot escape capacity accounting. Those slots remain through - failed-response retention and eventual tombstone/purge. Readiness is false when - required policy is absent or lifecycle reconciliation is incomplete. Active - work holds a durable lease and has no idle deadline. For `session` retention, - one provisional continuation-admission CAS saves and clears a valid prior - deadline; profile readiness and capacity admission commit active, while - pre-allocation profile failure restores that deadline if future or tombstones - if elapsed. Durable terminal acknowledgement starts a new deadline. - `persistent` bypasses idle expiry only after the runner authorizes it and - reserves its slot before source resolution. A post-allocation failure before - valid retention exists uses the deployment's finite failed-response retention, - then consumes its reserved tombstone slot and purges through the same bounded - lifecycle. - - Collection atomically tombstones an expired or operator-deleted session before - cleanup, fences new turns, waits for active processes and mounts to quiesce, - deletes private state, releases each generation reference and quota reservation - exactly once, and transitions `deleted -> purged` only after durable physical - cleanup and tombstone retention. Tombstones contain only bounded identifiers - and terminal lifecycle facts. Their TTL is at least the maximum response and - idempotency retention; compaction has deterministic age/key order and durable - accounting. While retained, continuation returns HTTP 410 - `allagents_workspace_expired`. After both tombstone and matching response/ - idempotency retention expire, the predecessor is indistinguishable from an - unknown ID and receives the stock non-disclosing error; neither path resolves - or materializes source. - - Generation GC evicts only ready epochs with zero durable references and zero - provisional pins. Null `lastUsedAt` epochs sort first by `publishedAt`, - generation-key bytes, and epoch-ID bytes; non-null epochs then sort by - `lastUsedAt`, `publishedAt`, key bytes, and epoch-ID bytes. It rechecks both - protections under the generation lock, durably records logical eviction, and - completes physical deletion before permitting a new epoch for that key. Failed - deletion stays quarantined and counted against quota. Startup reconciles - generic admission slots, active leases, build/staging/prospective-generation - reservations, publication/accounting markers, build waiters, provisional pins, - references, mounts, private reservation transfers, view state and actual usage, - tombstones, compaction, and deletion before readiness or GC. If only protected - state remains, new admission fails - `allagents_workspace_capacity_exceeded`; no protected state is deleted and no - access, retention, source, profile, or provider route changes. -- **R15.** Build and publish a pinned `linux/amd64` custom HarnessRouter image as - the public package `ghcr.io/allagentsdev/harnessrouter`. Replace or disable the - inherited Docker Hub release path. The Dockerfile pins every base image by - digest; runtime lockfiles and version-locked OS packages, Git/OCI tools, Codex, - and Pi define the remaining build inputs. The build fails on any unpinned - input. - - A no-write GitHub Actions job builds, tests, and exports the identified image - artifact using commit-pinned third-party actions. It starts that exact artifact - as a local container on loopback or a private Docker network, waits for - readiness, checks out AI Evals at a pinned commit, installs its locked - dependencies, and runs its actual Promptfoo CLI and provider against - HarnessRouter. A direct UHP smoke script may supplement this test but cannot - replace Promptfoo. The job never downloads a floating `promptfoo@latest`. - - The pull-request path uses a deterministic local provider fixture and receives - no real provider credentials. The native-auth path never runs for a - `pull_request` event or PR-controlled ref. It requires environment approval, - an approved release/tag ref, and an image artifact whose recorded digest, - source commit, and producing workflow identity match that ref. It mounts the - real Codex and Pi profiles and runs the native-auth cases against that exact - artifact. If a standard hosted runner cannot prove the required mount and - cgroup behavior, use a dedicated ephemeral self-hosted Actions runner for one - job. Destroy it after credential teardown and never schedule an untrusted job - on it; do not weaken or skip checks to fit a runner. - - A separate protected publish job accepts only that approved artifact. It uses - `GITHUB_TOKEN` with `contents: read`, `packages: write`, - `attestations: write`, and `id-token: write`. It first pushes an untagged - candidate by digest, reads back the registry manifest, and creates - GitHub/Sigstore build-provenance and SBOM attestations whose subject is that - digest. Package visibility is public and verified with an anonymous digest - pull into a fresh protected job. That job repeats both the PR-safe and real - native-profile Promptfoo suites against the pulled bytes and creates a signed - green-E2E attestation recording the subject digest, approved source ref, AI - Evals commit, Promptfoo lockfile identity, scenario identity, and successful - workflow run. Only after build provenance, SBOM, and green-E2E attestations all - verify does the workflow apply unique version and commit discovery tags or - mark the release complete. - - Deployment fails unless those three attestations verify the expected owner, - repository, workflow, approved ref, subject digest, predicates, base-image - digest, runtime lockfiles, OS package set, Git/OCI tool versions, Codex/Pi - versions, AI Evals commit, Promptfoo lockfile, and successful scenario run. An - untagged candidate without the green-E2E attestation is not deployable. - Requests without the configured metadata key remain stock-compatible. CI - rebases selected upgrades and runs upstream plus AllAgents integration tests. -- **R16.** AI Evals owns its Promptfoo provider and the executable green-E2E - configuration. The workflow invokes the Promptfoo version from AI Evals' - lockfile through that repository's package script. The provider sends the UHP - request directly to the locally running HarnessRouter, maps Promptfoo variables - to the closed extension, and maps terminal output, usage, artifacts, - provenance, and failures to `ProviderResponse`. Every non-success follows the - Failure Contract's exact status/error/retryability/metadata mapping; none - becomes empty success or an automatic retry. Multi-turn cases retain the prior - response ID and send it as `previous_response_id`. Green requires the real - Promptfoo process to exit successfully and the scenario assertions to pass; a - hand-written client is not consumer proof. AllAgents documents the contract - and examples but does not depend on Promptfoo at runtime. - -### Key Flows - -#### F1. Start the deployment - -1. Launch the attestation-verified image in non-serving initialization mode with - durable session, generation, editable-workspace, and auth volumes; finite - idle TTL and staging/generation/private/session/persistence/tombstone quotas; - caller key; materializer command; project snapshot configuration; public- - egress enforcement; optional origin-to-secret-reference mappings; owner-only - source-secret handle; native or proxy trust mode; delegated cgroup v2 subtree; - and `on-failure` restart policy. Verify the image and mounted inputs before - running checks that depend on them. -2. The runner validates read-only mount enforcement, private-view isolation, - finite lifecycle policy, storage relationships, and cgroup delegation. It - reconciles incomplete generation claims/publications, build waiters, - provisional pins, durable session references, mounts, private writable - views and quota usage, auth projections, tombstones/compaction, and - interrupted deletions. Sweep orphaned cgroups and credential projections only - after proving each old process boundary empty. Do not start GC or serving. -3. Run the mounted AllAgents hook's bounded `preflight` mode. It validates hook, - egress-policy, optional snapshot-catalog, origin-mapping, credential-reference, - and required Git/OCI tool syntax without repository URLs, source network - access, or secret values. It returns the bounded configured credential- - reference identities; the runner verifies their credential-store handles. -4. In a controlled operator context, initialize each dedicated auth profile: - run Codex login with that target's `CODEX_HOME`, or run Pi `/login` with that - target's isolated Pi home and configured provider. Persist only the selected - harness profile. If native OAuth cannot satisfy the deployment's trust or - compatibility requirement, deliberately select and validate a separately - configured `proxyApiKey` deployment profile; never make it automatic failover. -5. Start the private listener in probe-only, not-ready mode after shared - coordinator, generation, session, mount, and lifecycle reconciliation plus - preflight succeed. Reconcile each auth profile independently. An unavailable - or `repair-required` profile disables only bindings that reference it. - External traffic and GC remain disabled. -6. From the exact container network, use the deployment probe identity to verify - each configured harness/model, selected auth binding, safe roots, materializer - version, generation store, mount enforcement, lifecycle policy, and a live - turn. Native targets exercise login, active-turn-only projection, - same-binding continuation, two concurrent same-profile turns with a valid - token, two with a forced-expired token, one coordinated provider refresh, - teardown, and repair after injected refresh faults. Proxy targets exercise - schema/TLS/model-map/endpoint compatibility and scoped broker use. Publish the - advertised target catalog from successful binding probes, then atomically - enable external serving, GC, and global readiness. A shared coordinator, - containment, mount, preflight, or lifecycle failure keeps global readiness - false. A profile-specific authentication failure marks only that binding - unavailable; healthy bindings remain advertised and serve requests. - -#### F2. Execute the first repository-backed turn - -1. Promptfoo sends one authenticated UHP request with `model`, stock - `metadata.harness_id`, idempotency input, and the - `metadata["allagents.workspace"]` JSON descriptor containing the HTTPS Git - repositories to load. The descriptor explicitly selects `readOnly` or - `editable`; omitted retention means `session`. -2. HarnessRouter validates UHP and generic metadata bounds and atomically claims - the `Idempotency-Key`. The private runner admission transaction resolves the - selected target and auth-binding digest, checks profile readiness, and durably - reserves one configured turn slot for that profile, one generic - workspace-session slot, and one fixed-size tombstone slot before response - allocation. The admission token owns all three; any pre-allocation failure - rolls them back exactly once. Duplicate same-key arrivals share one - admission/result. New same-session overlap returns `session_busy`. A native - profile at its configured turn limit fails with cataloged - `harness_unavailable`; unavailable generic lifecycle capacity fails - `allagents_workspace_capacity_exceeded`. Both failures occur before response - allocation. Different sessions may proceed concurrently with the same or - different profiles. -3. The gateway consumes that token, creates the response/session in `validating`, - and persists the opaque descriptor, raw request digest, generic reservation - IDs, and harness/auth binding before making it visible. It then invokes the - hook's source-free `validate` operation. The runner consumes typed access, - requested retention, effective descriptor digest, logical cwd, normalized - caller repository entries, descriptor reference, and bounded credential- - reference identities selected by credential-scope policy. It proves the selected - set is a subset of preflight declarations, verifies only those handles, rejects - unsafe URLs/destinations and read-only input files, rechecks the secret - boundary, authorizes persistence, and reserves any persistence slot plus one - stable full-hard-private-allowance ID for editable access. Failure releases - access-specific reservations once, persists the terminal failed response under - finite failed-response retention, and retains its generic session/tombstone - slots through tombstoning and purge; no source is resolved or acquired. -4. The gateway CASes `validating -> resolving`. `resolve` consumes the exact - validated caller repositories and selected credential set, safely resolves - them to immutable commits, and returns the generation key, private source-only - resolved-plan path/digest, effective cwd, and request provenance. Under the - generation lock, a valid ready epoch hit acquires a durable provisional pin - and skips acquisition. A miss joins the current build epoch or, only after an - evicted prior epoch is durably gone, creates a new runner-owned epoch/claim. - Before first byte acquisition, its owner stores the exact resolved plan and - selected credential-reference identities in claim-owned durable private state - and reserves the full staging and prospective-generation byte/count - allowance. Each request waits only to its own deadline; cancellation detaches - only that waiter. -5. Only a live miss claim invokes `materialize` with the exact resolved-plan - path/digest and fixed private staging/result roots. The child writes and - validates staging, removes credential state, and returns the manifest. The - runner proves the cgroup empty and independently validates the tree, semantic - Git state, result, and key. The runner's independent full-tree accounting, - including validated Git administrative state, supplies physical retained - byte/inode usage. Under the generation lock, one atomic publication/accounting - transition converts prospective generation capacity to actual usage, releases - excess and the staging reservation, records epoch/publishedAt, and persists - ready state before giving each live waiter a provisional pin. Build failure - removes unpublished staging and releases access/build-specific reservations; - generic session/tombstone reservations remain with their failed responses. -6. For each pinned waiter independently, the runner first validates its logical - cwd against the independently verified manifest. A missing or non-directory - `workspacePath` releases only that waiter's pin and access-specific - reservations and persists its terminal failed response; no attachment is - prepared. The runner then rejects an editable attachment whose full physical - generation bytes/inodes exceed its hard allowance, again releasing only that - waiter's pin and access-specific reservations. Other waiters continue against - the valid ready epoch. Otherwise the gateway CASes that session - `resolving -> attaching`, and the runner prepares either a durable read-only - epoch reference plus verified mount or a private writable view plus - checkpoints and protected collection state. It returns opaque token/evidence. - The gateway alone CASes `attaching -> ready`, stores key/epoch evidence, and - acknowledges the token. Under the generation lock, the runner releases that - provisional pin exactly once and advances `lastUsedAt` to at least the - ready-commit timestamp. - Prepare/ack recovery preserves the committed attachment or rolls that waiter's - resources/reservations back once. -7. Only after attachment `ready` do response events include the complete - workspace metadata; active streaming uses `expiresAt: null`. Only editable - sessions accept ordinary UHP input-file overlays. HarnessRouter uses the - already validated logical cwd, creates the active-turn-only native credential - projection or scoped proxy credential, and dispatches the harness. When a - native token needs renewal, the auth adapter enters the profile's renewal - coordinator before the provider call. Provider retry or fallback cannot - validate, resolve, build, attach, or change any binding. -8. Read-only collection reports no workspace mutation. After descendants stop, - editable collection verifies trusted changed-path candidates against the - protected generation and prior turn state, or performs a bounded full-tree - scan when candidate state is not trustworthy. It never reports initial source - files or trusts editable `.git` metadata. Native finalization verifies that - turn has no incomplete renewal record, removes that turn's credential - projection, and proves retained homes and checkpoints clean. The gateway then - durably stores the terminal response and, for `session`, sets one expiry - timestamp used by the terminal event, GET, background completion, and replay. - Only after that acknowledgement may the runner release the turn's profile - capacity slot. - -#### F3. Continue the session - -1. The caller sends `previous_response_id` and omits - `metadata["allagents.workspace"]`. -2. HarnessRouter atomically claims the `Idempotency-Key`. For a new request, - generic continuation validation first rejects an extension-bearing request - without session lookup/CAS or deadline change. It then resolves the gateway- - owned attachment and enters one session CAS whose predicates include - `session_busy`, exact attachment/binding, and expiry/deletion. Busy or binding- - mismatch rejection changes no deadline. A retained expired or deleted session - returns HTTP 410 `allagents_workspace_expired` before profile or runner work. - After tombstone plus response/idempotency retention has been purged, the - predecessor receives the stock non-disclosing unknown-ID error. None resolves - source. -3. Only when every predicate succeeds does that CAS require attachment `ready` - plus persistent retention or an idle deadline later than admission, then save - and clear that deadline in a provisional turn-admission fence. The runner - verifies the exact epoch reference/mount or editable private checkpoint before - profile admission. Missing/corrupt evidence returns HTTP 409 - `allagents_workspace_non_resumable` and rolls the fence back by restoring the - original future deadline or tombstoning if elapsed, without source resolution, - acquisition, or rematerialization. -4. A native turn checks profile readiness and reserves one configured - concurrent-turn slot; proxy mode has no native slot. Profile saturation or - any other pre-allocation profile failure rolls the provisional fence back, - restoring the exact original deadline if it remains future or tombstoning the - session if it elapsed. A returned admission token commits the fence to active - and exposes `expiresAt: null`. Sessions using the same or different profiles - may execute concurrently against the same generation epoch; polling and - replay change no deadline or admission state. -5. HarnessRouter resumes the native conversation and original attachment. - Read-only source remains immutable; editable prior mutations remain visible. - Output, usage, artifacts, and pinned provenance return without changing any - binding. The same renewal coordination, projection teardown, and terminal - acknowledgement as the first turn sets the next expiry and releases that - turn's profile slot on every terminal outcome. - -#### F4. Execute an OCI-backed first turn - -1. The caller selects one configured snapshot and immutable image/workspace- - manifest digests; it never sends the registry origin or credential. -2. Resolve computes the OCI generation key. A ready epoch is reused. Otherwise a - current claim is joined or, after completed eviction, a new epoch owner fetches - and verifies the direct manifest, config, workspace manifest, and layers; - applies changesets under fixed limits; validates every declared offline Git - root; and returns verified staging, semantic Git records, and provenance. -3. The runner publishes the same immutable-generation-epoch abstraction as Git, - then follows the same per-waiter read-only or editable attachment path. - Registry, digest, media, path, limit, layout, or semantic Git failure removes - only unpublished staging and enters neither Git nor provider fallback. - -#### F5. Cancel, fail, or restart - -1. On every hook outcome, the runner proves the cgroup empty before exposing a - terminal result, reading a manifest, publishing a generation, releasing - secrets, or cleanup. Completed-parent/live-descendant returns - `allagents_workspace_containment_breach`; an unquiescent leaf remains internal - `containment_pending`, exits/restarts the runner, withholds readiness and - terminal visibility, and reconciles only after proving the old boundary empty. -2. Cancellation before attachment detaches only that request from a shared build; - the build continues for other live waiters and stops only when none remain or - its runner-owned deadline expires. Agent cancellation and deadline after - dispatch use HarnessRouter's normal UHP lifecycle. Whole-container termination - does not preserve an in-flight agent; interrupted turns fail without replay. -3. Startup reconciles generation-epoch - `building/ready/quarantined/deleting` evidence separately from session - `validating/resolving/attaching/ready/containment_pending/expired/deleting/ - deleted/purged/failed` evidence. `containment_pending` stays non-terminal and - blocks readiness until the recorded cgroup is empty; it then transitions once - to the preserved `failed`, `cancelled`, or `incomplete` outcome. A ready - read-only session requires its exact generation key/epoch marker, reference, - and mount. A ready editable session requires its private publication marker, - reserved quota, actual-usage accounting, and matching checkpoint. Missing or - mismatched state becomes non-resumable; a later epoch is never substituted. -4. Every terminal outcome after native admission records any incomplete renewal, - removes that turn's credential projection, verifies its retained home clean, - and stores terminal response and expiry before acknowledging the admission - token and releasing its profile slot. Gateway or runner crashes retain the - durable turn fence until descendants, projection, and profile state reconcile. - New profile admissions remain blocked only when readiness cannot be proven. -5. Restart reconciles all lifecycle state before GC or readiness. It completes or - rolls back interrupted provisional turn admission against any runner profile - token, then reconciles generic admission, active leases, staging/prospective- - generation reservation, publication/accounting conversion, provisional-pin/ - reference acquisition, attachment prepare/ack and private-reservation transfer, - mount/view creation, private usage accounting, tombstoning, unmount, release, - compaction, and physical deletion without duplicating a debit/reference, - leaking a pin, extending an original deadline, or exposing a partially deleted - resource. -6. A completed session resumes only with attachment `ready` and valid evidence, - and when retention is persistent or its session idle deadline is unexpired. - Known invalid evidence returns `allagents_workspace_non_resumable`; it never - rematerializes or substitutes a later generation epoch. - -#### F6. Expire, delete, and collect workspace state - -1. An accepted turn holds an active lease and no idle expiry. Idle expiry or - authenticated operator deletion CASes the session to a bounded tombstone - first; a racing continuation either wins admission and clears the old deadline - or observes the tombstone. -2. The collector fences new work, waits for process, credential-projection, and - mount quiescence, removes editable private state, releases each generation - reference and private quota reservation exactly once, and durably records - deletion. Persistent sessions skip idle expiry but use the same explicit-delete - path. -3. Under storage pressure, GC selects only ready generation epochs with zero - references and zero provisional pins. Null `lastUsedAt` epochs order first by - `publishedAt`, generation-key bytes, and epoch-ID bytes; non-null epochs then - order by `lastUsedAt`, `publishedAt`, key bytes, and epoch-ID bytes. It - rechecks protection under lock, records durable logical eviction, removes - physical state, and only after both complete permits a new epoch for that key. -4. Tombstone compaction runs in age/key order only after the maximum response and - idempotency retention has elapsed, durably transitions `deleted -> purged`, - and releases the reserved tombstone slot. A later predecessor lookup uses the - stock non-disclosing unknown-ID error and never rematerializes. -5. Failed deletion remains quarantined and counted. If high-water quota cannot be - reduced because all state is active or pinned, new admission fails - `allagents_workspace_capacity_exceeded`; no protected workspace is removed. - -### Acceptance Examples - -- **AE1.** A stock UHP request without the configured metadata key produces the - same response and conformance result on upstream HarnessRouter and the fork. -- **AE2.** Every unauthenticated external create, continuation, GET, stream, - cancel, file, artifact, and lifecycle request fails before resource existence - or metadata is disclosed. An authenticated native Codex or Pi turn sees only - the selected active-turn OAuth projection; the HarnessRouter caller key is - absent from the agent environment and filesystem, no provider-route API key - exists in that mode, and the projection is absent from retained homes, - checkpoints, backups, and mounts after every terminal or recovered outcome. -- **AE3.** Repository mode resolves Promptfoo-supplied HTTPS URLs and refs to - exact commits and publishes one verified immutable generation. Two - simultaneous `readOnly` sessions using different harness/profile bindings and - the same normalized request share one generation build, see identical bytes - and nested Git history, start in their own validated logical cwd, and cannot - write the generation or observe each other's home, conversation, temporary - files, logs, or outputs. -- **AE4.** HarnessRouter maps a non-object extension to HTTP 400 `invalid_input`, - an oversized extension to HTTP 413 `allagents_workspace_too_large`, an - extension on continuation to HTTP 409 `allagents_workspace_immutable`, and a - retained expired/tombstoned continuation to HTTP 410 - `allagents_workspace_expired`. Source-free validation rejects unknown access/ - retention, malformed refs/destinations/working-directory shapes, escaping - workspace paths, userinfo or secrets in URLs, non-HTTPS transports, IP - literals, and duplicate/overlapping destinations. An unknown or unadvertised - ref fails during bounded resolution without source-byte acquisition. After a - cache hit or verified build, the runner rejects a `workspacePath` that is - missing or not a directory before attachment or agent launch. Acquisition - rejects loopback/link-local/private/reserved/metadata - destinations, DNS rebinding, unsafe redirects, and out-of-scope credential - forwarding before source bytes reach staging. Preflight receives no request URL - or secret value; - the runner alone verifies selected credential handles, storage relationships, - persistence authorization, and session/persistence/build reservations. Exact - source byte admission may fail only after bounded staging reveals size, but - before publication, attachment, or agent launch. -- **AE5.** Two `editable` turns linked by `previous_response_id` preserve native - conversation and a private file mutation. A separate editable trial from the - same generation receives a unique clean writable view and cannot observe or - mutate the first. An editable waiter whose initial view cannot fit fails its - own `allagents_workspace_private_quota_exceeded` response without invalidating - the ready epoch or a concurrent read-only/fitting waiter. First-turn collection - uses the protected generation without a redundant full pre-agent inventory. - Candidate tracking begins before input overlays or writable process exposure; - a missing or discontinuous coverage marker forces the full scan. Candidate and - full-scan paths return the same source-visible delta, exclude the protected - declared Git administrative subtrees, report an agent-created `.git` elsewhere - as ordinary content, and cannot be hidden by editable Git metadata or ignore - rules. - Continuation reports the delta from the prior protected turn state and cannot - grow past the session's reserved hard quota. Two read-only turns preserve - conversation but have no workspace mutation checkpoint or produced-file delta. -- **AE6.** Continuation omits the extension and preserves the exact generation - epoch, access, retention, cwd, harness, and profile. Any attempted rebinding is - rejected. One CAS rejects same-session overlap without changing expiry and - provisionally saves/clears an unexpired idle deadline. Profile success commits - active; pre-allocation profile failure restores that deadline when future or - tombstones if elapsed. Only terminal acknowledgement sets the next expiry. - Polling and replay do neither. A retained expired/deleted predecessor returns - HTTP 410; a known corrupt attachment returns HTTP 409 - `allagents_workspace_non_resumable`; a fully purged predecessor returns the - stock unknown-ID error. None rematerializes or substitutes an epoch. -- **AE7.** Explicit UHP input files overlay only an editable private workspace - after its initial checkpoint and durable candidate coverage begins, but before - agent launch. A read-only request with workspace input files fails - `allagents_workspace_read_only`; a runtime write receives a filesystem - read-only error with no copy-up or mode change. -- **AE8.** Faults at pre-allocation generic session/tombstone admission, - validate, selected-credential verification, resolve, keyed epoch claim/waiter - cancellation, staging/generation reservation and publication-accounting - conversion, containment, provisional pin, attachment prepare/ack, read-only - epoch reference/mount, editable reservation-transfer/view/checkpoint, expiry, - unmount, release, tombstone compaction, and deletion either reconcile to one - complete protected resource or fail closed. No agent sees staging, duplicate - live epoch publication, partial private state, or a generation without required - protection. Every reservation, pin, and reference debits and releases exactly - once. Provider fallback never reruns the hook. -- **AE9.** OCI mode accepts valid digest-pinned tree-only and history-bearing - fixtures with gzip/zstd layers and whiteouts. The history fixture has no - remotes or credentials and supports offline `git log`, `git blame`, and - historical diff from its declared detached commit. OCI rejects undeclared - `.git`, overlapping repository destinations, remote or credential - configuration, unsafe Git administrative state, dirty or untracked worktree - content, commit-tree or object-set mismatch, mutable tags, indexes, mismatched - digests/sizes, traversal, escaping links, devices, sparse files, unknown media - types, and declared-limit overflow. Repeated Git and OCI requests with - identical resolved plan, sharing authorization, and selected credential- - reference identities reuse their matching epoch regardless of access, - retention, cwd, harness, profile, or session. -- **AE10.** Codex and Pi own login and refresh. Missing, revoked, expired, - unrefreshable, or stale-after-crash OAuth affects only that profile and never - selects another profile or proxy. Same-key arrivals share one admission and - result; same-session overlap returns `session_busy`. Different sessions using - the same profile run concurrently up to its configured limit. With a valid - token, neither waits for a profile-wide lock. With a forced-expired token, - both succeed while the renewal coordinator permits one provider refresh and - makes the second turn reread the saved update. No provider refresh occurs - outside the coordinator. Success, failure, cancellation, and crash recovery - remove each turn's credential projection and release its capacity slot. - Incomplete renewal either reconciles to a valid credential or marks only that - profile `repair-required`. That state blocks new turns and renewal attempts; - an already-running turn either finishes with its current token or returns the - normal provider-authentication failure. Proxy mode enforces audience, target, - model, turn, expiry, and revocation. -- **AE11.** Restart preserves an unexpired or persistent session, exact - attachment, and auth binding after lifecycle reconciliation. Missing or - changed auth fails before runner work. Missing generation/reference or private - checkpoint returns the cataloged non-resumable error without replay. A - `containment_pending` session remains non-terminal until its cgroup is empty; - interrupted builds, pins, views, quota records, auth projections, tombstones, - compaction, and deletions reconcile without resurrection or double release. -- **AE12.** The protected publish job releases the public `linux/amd64` GHCR - package without Docker Hub credentials. Anonymous verification covers the - expected build-provenance/SBOM identity, subject digest, and pinned inputs. -- **AE13.** Promptfoo maps every cataloged request, generation, attachment, - persistence, read-only, capacity, expiry, materializer, auth, provider, and UHP - terminal failure to the exact `ProviderResponse.error` and metadata. Failures - before attachment `ready` omit workspace metadata; later terminal failures - include its complete public metadata. None becomes empty success or an - automatic retry. -- **AE14.** With tiny deterministic quotas and a fake clock, invalid descriptors - cannot create unaccounted response/session records; active and persistent - sessions survive collection; terminal acknowledgement starts idle expiry; - expired editable state and its one private reservation are deleted; repeated - successful unique publications return staging capacity to baseline; read-only - references and provisional pins release exactly once; never-attached epochs - with null `lastUsedAt` evict first by `publishedAt`, then used epochs by - `lastUsedAt` and `publishedAt`, with generation-key/epoch ties; a new epoch - cannot publish until prior logical and physical eviction completes; tombstones - remain byte/count - bounded and purge only after response/idempotency retention; failed deletion - stays quarantined/accounted; and all-protected capacity returns - `allagents_workspace_capacity_exceeded`. - -### Scope Boundaries - -**In scope** - -- HarnessRouter generation, attachment, lease, retention, quota, GC, - workspace-integration, and harness-auth-state patches. -- Versioned AllAgents JSON workspace descriptor with caller-supplied HTTPS Git - repositories, validate/resolve/materialize hook contracts, generation identity, - and provenance. -- Safe public egress enforcement, source-credential scope mapping, and optional - project `workspace.yaml` snapshot-catalog additions. -- Deterministic Git and immutable OCI generation construction. -- Shared read-only mounts, private writable views, mode-specific root/nested- - repository checkpoint and produced-file integration. -- Session idle expiry, persistent authorization, operator deletion, bounded - storage admission, restart reconciliation, and generation eviction. -- Source credential isolation and native-OAuth trust-boundary verification. -- Native Codex and Pi auth-profile bootstrap, refresh, readiness, and continuity. -- Explicit authenticated-proxy deployment mode. -- Public GHCR publishing, digest-pinned release metadata, SBOM, and provenance. -- Promptfoo concurrent/one-shot/two-turn/lifecycle success and failure E2E. -- Upstream-ready generic hook, lifecycle, and auth-state patches. - -**Out of scope** - -- A second northbound execution protocol or parallel task/session control plane. -- A separate AllAgents network gateway, process supervisor, provider adapter, or - artifact service. -- Promptfoo runtime code inside AllAgents. -- A custom OAuth broker, token translation layer, or automatic native-to-proxy - credential fallback. -- Caller-provided credentials, non-HTTPS/private-network origins, commands, host - paths, materializers, or Docker options. -- Harbor provider execution remains a separate Promptfoo lane outside this plan. - Direct Harbor task-package ingestion, SWE-bench/Hugging Face dataset ingestion, - caller-selected runtime images, benchmark verifiers, and compatibility aliases - remain invalid inside `allagents.workspace`. -- Public multi-tenancy, per-caller authorization, Kubernetes workers, session - branching, concurrent turns in one session, or guaranteed prompt-cache hits. -- Exact rollback of workspace mutations between successful session turns. - -### Sources - -- [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) -- [HarnessRouter repository](https://github.com/HarnessRouter/harnessrouter) -- [HarnessRouter self-hosting guide](https://github.com/HarnessRouter/harnessrouter/blob/main/docs/self-hosting-guide.md) -- [UHP 2026-09-12 architecture](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/architecture.md) -- [UHP sessions](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/sessions.md) -- [UHP lifecycle](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/lifecycle.md) -- [UHP files](https://github.com/HarnessRouter/harnessrouter/blob/main/protocol/versions/2026-09-12/files.md) -- [Codex authentication](https://developers.openai.com/codex/auth) -- [Pi model providers and OAuth](https://pi.dev/docs/latest/providers) -- [GitHub Container Registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry) -- [`codex-lb` optional proxy](https://github.com/Soju06/codex-lb) -- [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) -- [Workspace contract incumbent comparison](../research/workspace-contract-incumbents.md) -- [Source credential broker precedents](../research/source-credential-broker-precedents.md) - ---- - -## Planning Contract - -### High-Level Technical Design - -```mermaid -flowchart TB - PF[Promptfoo provider] -->|UHP + HR API key + workspace JSON| GW[HarnessRouter gateway] - GW -->|session CAS + attachment prepare/ack| RUN[HarnessRouter runner] - RUN -->|typed validate, resolve, or cache-miss materialize| MAT[AllAgents materializer] - MAT --> POLICY[egress and source-credential scope policy] - MAT --> CFG[optional workspace.yaml snapshot catalog] - MAT --> GIT[caller-requested HTTPS Git sources] - MAT --> OCI[configured OCI registry] - RUN -->|atomic publish or reuse| GEN[(immutable generation store)] - GEN -->|read-only mount + reference| RO[read-only session] - GEN -->|private writable view| EDIT[editable session] - RO --> HARNESS[Selected Codex or Pi harness] - EDIT --> HARNESS - LIFE[(leases, retention, quotas, GC)] --> RO - LIFE --> EDIT - LIFE --> GEN - AUTH[(dedicated durable OAuth profile)] -.->|native mode only| HARNESS - HARNESS -->|native mode| MODEL[Model provider] - HARNESS -.->|proxy mode: scoped turn credential| GW - GW -.->|long-lived proxy client key| PROXY[optional provider proxy] - PROXY -.-> MODEL - GW --> DATA[(durable session and attachment state)] -``` - -The gateway is the sole writer of session attachment, expiry, and tombstone -state; it also owns generic metadata bounds, provider-loop ordering, response -metadata, and optional proxy brokering. The runner owns keyed generation claims, -the resource journal, hook invocation, independent verification, atomic -publication, provisional pins, durable references, read-only mounts, private -writable views, mode-specific checkpoints, quota admission, deletion, GC, safe -cwd, and agent launch. Attachment uses a durable prepare/evidence/ack protocol: -the runner prepares resources, the gateway alone commits `ready`, and the runner -finalizes or rolls back from that acknowledgement. The selected harness owns -native OAuth login and refresh; its auth root is outside every generation and -session checkpoint. The materializer owns only the AllAgents JSON schema, caller -Git URL validation, optional `workspace.yaml` snapshot catalog, source resolution, -acquisition, staging validation, and provenance. It never speaks UHP, authorizes -persistence, owns leases, publishes live state, or writes the gateway session -state machine. - -### Extension Contract - -Initial UHP request fragment: +# AllAgents Gateway v1 - Implementation Plan + +## Goal + +Ship **AllAgents Gateway** as a small, maintainable downstream of HarnessRouter +that lets Promptfoo invoke Codex or OMP over UHP against a caller-selected public +Git repository. + +The implementation repository is +[`allagentsdev/allagents-gateway`](https://github.com/allagentsdev/allagents-gateway). +Establish it by renaming the existing `allagentsdev/harnessrouter` GitHub fork, +not by creating a third repository or wrapping one repository with another. The +rename must preserve the fork relationship, commit history, issues, settings, +and a usable upstream remote. + +The implementation starts from HarnessRouter v0.25.4 at commit +`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` and keeps UHP +`2026-09-12` as the northbound protocol. The public image is +`ghcr.io/allagentsdev/allagents-gateway`, with versions and release notes that +are independent of the AllAgents npm CLI. + +Version one adds one product behavior to stock HarnessRouter: on the first turn +of a session, recognize `metadata.allagents.workspace`, securely materialize one +public HTTPS Git repository into one private editable checkout, bind that exact +checkout to the session, and start the selected harness in the requested safe +working directory. A continuation reuses that checkout without resolving or +cloning again. + +Everything else stays with HarnessRouter: caller authentication, UHP request and +response behavior, session hydration, streaming, idempotency, cancellation, +provider and harness execution, artifacts, and ordinary lifecycle state. + +## Repository boundary + +All production code, image construction, Compose configuration, Promptfoo +configuration, tests, and release automation land in +`allagentsdev/allagents-gateway`. + +This `allagentsdev/allagents` repository remains the local Bun CLI. For v1 it +contains only the accepted ADR and this implementation plan. Specifically: + +- no gateway server, materializer, image build, or provider adapter is added here; +- no `allagents gateway start` command is added; +- local profiles are not uploaded, synchronized, or translated into remote + configuration; +- local `workspace.yaml`, its schema, and its behavior remain unchanged; and +- the gateway does not import an AllAgents host profile or invoke the AllAgents + CLI. + +HarnessRouter custom harness definitions are the complete remote configuration +surface for Codex and OMP. OMP runs as an ordinary session-local harness inside +HarnessRouter. + +## Product boundary + +### In scope + +- UHP `2026-09-12`, exposed by the pinned HarnessRouter downstream. +- Caller authentication using HarnessRouter's existing API-key behavior. +- Exactly one workspace extension: `metadata.allagents.workspace`. +- Exactly one anonymous, public, HTTPS Git repository per new session. +- An optional advertised `refs/heads/*` or `refs/tags/*`, or unambiguous + branch/tag shorthand. Omission means the remote default branch; raw object IDs + and other ref namespaces are not accepted. +- SHA-1 repositories only, with resolution to and recording of one exact + 40-hex commit object ID. +- One private editable checkout per session. +- Immutable first-turn binding and exact-checkout continuation reuse. +- A finite server-owned idle TTL and deterministic, idempotent cleanup. +- Codex and OMP harnesses using one server-configured external + OAuth-to-OpenAI-compatible gateway. +- One container, one `/data` volume, and Docker Compose startup bound to + loopback by default. +- Direct Promptfoo evaluation of the built image. +- Upstream UHP conformance, image SBOM/provenance, and digest-pinned releases. + +### Non-goals + +- More than one repository, destination mapping, or repository composition. +- Non-Git workspace sources, private source credentials, SSH Git transports, + or caller-provided source headers. +- Raw commit-ID requests and SHA-256 Git repositories. +- Read-only workspaces, cross-session workspace reuse, prewarming, or source + object stores. +- Caller-selected lifetime, indefinite sessions, recovery after the configured + expiry, or a new lifetime subsystem. +- A generic extension registry, plugin framework, or multiple materializers. +- Provider login, token refresh, credential repair, or credential projection in + HarnessRouter. Those belong to the external provider gateway. +- A caller-selected provider base URL, provider API key, transport, or fallback + chain. +- Importing local profiles, synchronizing profile state, changing + `workspace.yaml`, or adding an AllAgents CLI command. +- Replacing HarnessRouter sessions, task execution, artifacts, streaming, + cancellation, or idempotency. +- New validation commitments for other HarnessRouter backends. +- Kubernetes, multi-container worker orchestration, or a separately deployed + materializer service. +- A custom release-attestation or green-build framework. + +## External contracts + +### UHP request + +The workspace object is nested metadata, following HarnessRouter's existing +`metadata.systemone.script` precedent. It is not a flat metadata key. ```json { - "model": "gpt-5.6-sol", - "input": "Review the service", + "model": "gpt-5.4", + "input": "Inspect the project and fix the failing command.", "metadata": { - "harness_id": "codex-review", - "allagents.workspace": { - "version": "1", - "access": "readOnly", - "retention": "session", - "source": { - "kind": "repositories", - "repositories": [ - { - "url": "https://github.com/acme/api.git", - "ref": "refs/pull/123/head", - "destination": "api" - } - ] - }, - "workingDirectory": { - "kind": "workspacePath", - "path": "api/packages/service" + "harness_id": "allagents-codex", + "allagents": { + "workspace": { + "version": "1", + "repository": { + "url": "https://github.com/example/project.git", + "ref": "refs/heads/main" + }, + "workingDirectory": "packages/service" } } } } ``` -Workspace-snapshot source fragment: - -```json -{ - "kind": "workspaceSnapshot", - "snapshotName": "benchmark-fixture", - "imageManifestDigest": "sha256:...", - "workspaceManifestDigest": "sha256:..." -} -``` - -Continuation fragment: +The exact v1 JSON shape is: -```json -{ - "model": "gpt-5.6-sol", - "previous_response_id": "resp_previous", - "input": "Now fix the highest-severity finding" +```text +metadata.allagents.workspace = { + version: "1", + repository: { + url: string, + ref?: string + }, + workingDirectory?: string } ``` -Successful terminal response metadata fragment: +Rules: + +1. `workspace` and `repository` must be JSON objects, not arrays or strings. +2. `version` is required and must equal `"1"`. +3. `repository.url` is required. It must be an anonymous public `https://` Git + URL with no user info, query, fragment, alternate transport, or embedded + credential. +4. `repository.ref` is optional and non-empty when present. It must be an + advertised `refs/heads/*` or `refs/tags/*`, or unambiguous branch/tag + shorthand. Raw object IDs and other ref namespaces are rejected. V1 accepts + SHA-1 repositories only and records the exact resolved 40-hex commit as + provenance. +5. `workingDirectory` is optional. Omission means the checkout root. When + present it is a normalized, relative POSIX path to a directory within the + checkout. +6. Unknown fields at every level are rejected. There are no aliases, commands, + environment variables, access modes, lifetime fields, destination paths, + materializer selectors, or provider settings. +7. The gateway applies a small fixed metadata byte/depth bound before session + allocation. The materializer applies field-specific length bounds before + network or filesystem work. +8. A request without `metadata.allagents.workspace` follows unmodified + HarnessRouter behavior. + +### Session binding and public provenance + +After materialization, the session owns one immutable binding containing: + +- canonical requested repository URL; +- requested ref, or an explicit record that it was omitted; +- exact resolved 40-hex commit; +- runner-owned session root under `/data`; +- private checkout root beneath that session root; +- separate runner control root as a sibling of the checkout; +- execution working directory beneath the checkout; +- materializer contract version; +- creation time and server-owned expiry; and +- cleanup state sufficient to make removal idempotent. + +The checkout root and cleanup token are internal and must never appear in UHP +responses, streams, logs, or Promptfoo output. Successful responses expose the +stable portion as nested provenance: ```json { - "allagents.workspace": { - "version": "1", - "effectiveDescriptorDigest": "sha256:...", - "generationId": "sha256:...", - "access": "readOnly", - "retention": "session", - "expiresAt": "2026-09-24T12:00:00Z", - "workingDirectory": { - "kind": "workspacePath", - "path": "api/packages/service" - }, - "sourceIdentity": { - "kind": "repositories", - "complete": true, - "repositories": [ - { - "url": "https://github.com/acme/api.git", - "destination": "api", - "requestedRef": "refs/pull/123/head", + "metadata": { + "allagents": { + "workspace": { + "version": "1", + "repository": { + "url": "https://github.com/example/project.git", + "requestedRef": "refs/heads/main", "resolvedCommit": "0123456789abcdef0123456789abcdef01234567" - } - ] - }, - "workspaceManifestDigest": "sha256:..." + }, + "workingDirectory": "packages/service" + } + } } } ``` -For snapshot source, `sourceIdentity` has this exact shape: +If the ref was omitted, `requestedRef` is omitted rather than synthesized. The +same public provenance is returned on successful continuations and idempotent +response retrieval. -```json -{ - "kind": "workspaceSnapshot", - "complete": true, - "snapshotName": "benchmark-fixture", - "imageManifestDigest": "sha256:...", - "repositories": [ - { - "destination": "api", - "git": { - "resolvedCommit": "0123456789abcdef0123456789abcdef01234567", - "objectSetDigest": "sha256:..." - } - }, - { - "destination": "docs" - } - ] -} -``` +### Continuation -The sorted `repositories` array mirrors the verified workspace-manifest root -declarations. The top-level response field carries `workspaceManifestDigest`. -Snapshot identity retains `snapshotName` and `imageManifestDigest`; repository -subrecords contain no `url` or `requestedRef`. - -### Materializer Hook Contract - -HarnessRouter configuration names one metadata key, absolute executable path, -maximum runtime, request/result byte limits, and allowlisted environment names. -The runner launches the executable directly without a shell. Standard error is -diagnostic-only, bounded, secret-checked, and never copied verbatim to callers. - -The hook supports four operations: - -- `preflight`: validate contract, acquisition/egress policy, optional snapshot - catalog, credential-scope mapping, credential-reference syntax, and required - binaries without request repository URLs, source network access, or secret - values, then return the bounded configured credential-reference names/opaque - IDs. The runner verifies the corresponding store handles and all staging/result - filesystem relationships itself; -- `validate`: validate and default the opaque JSON descriptor, caller repository - URLs/refs/destinations, `snapshotName`, `imageManifestDigest`, and - `workspaceManifestDigest` when applicable, and the syntax and lexical safety - of the workspace-relative working directory without source access; select the - bounded credential-reference subset from deployment credential-scope mappings; - then return a private normalized-descriptor path/digest, effective descriptor - digest, effective access, requested retention, logical cwd, and that selected - set; -- `resolve`: consume that exact normalized descriptor and selected reference set, - safely resolve immutable source identity, and return a private canonical - source-only resolved-plan path/digest, generation key, effective cwd, and - bounded request provenance without writing source bytes. The plan contains - only generation-key inputs—normalized caller URLs, resolved commits or exact - OCI image/workspace-manifest digests and layers, normalized destinations, - snapshot/acquisition/egress and sharing-authorization identity, and selected - credential-reference identities—and omits credential values, access, - retention, cwd, requested-ref spelling, harness/profile, and session; equal - generation keys therefore require identical plan bytes; and -- `materialize`: consume those exact resolved-plan bytes and selected reference - set at the supplied private path, verify their supplied digest, write source - content only to supplied generation staging, write the canonical manifest only - to the private result root, and return without publishing or re-resolving - source. - -After a ready-generation cache hit or a successful materialization, the runner -checks the logical cwd against the independently verified manifest before it -creates an attachment or launches an agent. - -Preflight runs once per deployment and receives no credential values. Validate -and resolve run once for a new session. Materialize runs only for a runner-owned -cache-miss generation claim; concurrent waiters consume the same ready -publication or failure. Validate receives opaque metadata and bounded -workspace-input-file count. Resolve receives the validated-descriptor path and -digest plus the exact selected credential-reference identities. Materialize -receives the resolved-plan path and digest, that same set, and fixed generation -staging/result roots. All operations receive the generic contract version, -project snapshot-configuration root, and deployment acquisition-policy version. -Validate/resolve use the request's bounded remaining deadline; shared materialize -uses the runner-owned build deadline and is cancelled only when no live waiter -remains. The runner resolves values for only the validated selected set and -injects them only into source-access operations through the allowlisted child -environment; values never appear in JSON, generation keys, or persisted plans. - -Validate returns effective access, requested retention, effective descriptor -digest/cwd, selected credential-reference identities, and its private normalized- -descriptor path/digest. Resolve repeats those request-bound values and adds the -generation key, private source-only resolved-plan path/digest, and bounded -request provenance. Materialize returns that same generation key plus -`workspaceManifest: { path: "workspace-manifest.json", digest }`, declared -repository roots, semantic Git validation records when applicable, and bounded -generation-scoped verified source identity. It does not return or choose a -waiter's descriptor digest, cwd, access, retention, or requested-source -provenance. Any operation may return the cataloged `failed` envelope. - -The runner checks that every private path is relative to its operation-specific -result root, opens it without symlink traversal, validates size and digest, and -passes the exact bytes/reference to the next operation. It rejects unknown -fields/versions, plan or generation-key drift, physical paths in public -metadata, incomplete success, forged manifests, undeclared roots, escaping cwd, -or staging that differs from the manifest. It then owns immutable publication -and resource preparation; the gateway remains the only session-attachment -writer. Valid hook output never means live state is published or attached. - -### Workspace Manifest Contract - -The source tree includes one generated normative -`workspace-manifest.schema.json`, imported unchanged by the Git materializer, -OCI producer/materializer, runner validator, and their contract fixtures. The -document is at most 128 MiB and is an object with `additionalProperties: false`, -required string `version` fixed to `"2"`, required `repositories`, and required -`entries`. - -`repositories` is an array with at most 128 items. Every item is an object with -`additionalProperties: false`, required `destination`, and optional `git`. -`destination` is a non-root `RelativeDirectory`. When present, `git` is an -object with `additionalProperties: false` and exactly two required string -fields: `resolvedCommit`, a full lowercase 40-hex commit ID, and -`objectSetDigest`, a `sha256:` digest of the canonical object-set bytes defined -below. Absence of `git` declares a tree-only root. -Destinations are unique, pairwise non-overlapping, and sorted by the UTF-8 bytes -of the NFC-normalized destination. Every destination must exactly equal the -`path` of a directory entry in the same manifest. Duplicate, ancestor/descendant, -or missing destinations, and destinations naming files or symbolic links, are -invalid even when the manifest digest is correct. Repository roots are -identified by destination in the generation-scoped manifest and any semantic -Git-state record. - -Canonical object-set bytes use ASCII and Git SHA-1 object IDs. Starting at -`resolvedCommit`, enumerate each unique reachable commit, tree, and blob, -including all commit parents and their trees. Sort records by the ASCII bytes of -the 40-character lowercase object ID. Emit exactly -` SP SP LF` for each object, where `type` is `commit`, -`tree`, or `blob`, and `size` is the unpadded base-10 byte length of the -uncompressed object content. There is one ASCII space at each `SP`, every record -ends in LF including the last, and no other bytes are present. `objectSetDigest` -is the SHA-256 of that concatenation. Pack layout, compression, offsets, and -filenames do not participate. - -`entries` is an array with at most 500,000 items. Every item has -`additionalProperties: false` and is exactly one of: - -- directory: required string fields `path`, `type: "directory"`, and - `mode: "040755"`; -- regular file: required string fields `path`, `type: "file"`, - `mode: "100644" | "100755"`, and - `sha256: "sha256:<64 lowercase hex>"`, plus integer `size` from zero through - 4 GiB; or -- symbolic link: required string fields `path`, `type: "symlink"`, - `mode: "120000"`, and `target`. - -Paths are unique, non-empty, relative POSIX paths sorted by their NFC-normalized -UTF-8 bytes. Every path component and symlink target must already be valid UTF-8 -and NFC; implementations reject rather than normalize non-UTF-8 or non-NFC -values. Paths and targets containing NUL, absolute paths, missing parents, or -links escaping the workspace are invalid. The root is implicit and has no entry. -Hard links are expanded to regular-file entries. Entries enumerate every -source-visible path. A repository item with `git` may omit only its separately -validated `.git` directory and descendants; an item without `git` may omit -nothing and must not contain `.git`. Any `.git` outside a declared history- -bearing repository root is invalid. - -The digest is `sha256:` plus the lowercase SHA-256 of the RFC 8785 bytes. -Repository mode computes those bytes after completing staging and adds `git` -metadata from its resolved commits. OCI mode requires its configured workspace- -manifest blob to contain the canonical bytes, including any declared offline Git -metadata, and copies them to the private result root. - -The runner resolves only the fixed `workspace-manifest.json` relative path, -validates it against the shared schema, verifies its size and digest, walks -staging without following links, reconstructs the same catalog and source-visible -entries while skipping only declared Git administrative roots, and requires -byte-for-byte canonical equality before publication. It separately revalidates -every skipped Git root against its declared commit, exact object set, closed -configuration and refs, and safe-state rules. For each history-bearing root, the -manifest descendants must equal the prefixed commit tree exactly; dirty, -untracked, missing, or modified paths fail. The private result root is never -published or exposed through UHP. - -The immutable generation key is not the workspace-manifest digest: it is the -pre-build digest of the resolved source plan used for keyed reuse. Atomic -publication binds that key to exactly one verified source-visible manifest -digest and every declared semantic Git-state record. Access, retention, cwd, -harness/profile, and session identity are not manifest fields and cannot -fragment or mutate generation content. The backing tree, including validated -`.git` state, becomes owner-writable only and is exposed to sessions solely -through verified read-only mounts or private writable views. - -The frozen history-bearing fixture is: +A continuation normally supplies only `previous_response_id`: ```json -{"entries":[{"mode":"040755","path":"services","type":"directory"},{"mode":"040755","path":"services/api","type":"directory"},{"mode":"100644","path":"services/api/README.md","sha256":"sha256:98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4","size":3,"type":"file"},{"mode":"120000","path":"services/api/current","target":"README.md","type":"symlink"}],"repositories":[{"destination":"services/api","git":{"objectSetDigest":"sha256:b3b49d7b3ff3fd8c4fb3acbf7de36d92f6a4029f3244064f79e37408463288b5","resolvedCommit":"cd95f8951573f4e021ddb57594dd3376a2b2f644"}}],"version":"2"} +{ + "previous_response_id": "resp_...", + "input": "Now run the focused check and summarize the result." +} ``` -Those exact manifest bytes digest to -`sha256:e870d43fa34f5f863ef7382e70844c4a2ebe86d79dac14505f005f4dda448e70`. -The fixture is a two-commit SHA-1 repository. Parent -`8884f45f3cf3e306cf0eeaa39febd41808151db5` contains `README.md` with bytes -`old\n`. The declared head contains `README.md` with bytes `hi\n` and symlink -`current` with target bytes `README.md`. Both commits use -`Eval Fixture ` as author and committer, timestamps -`0 +0000` and `1 +0000`, and messages `initial\n` and `current\n`, -respectively. Its canonical object-set bytes are: - -```text -3367afdbbf91e638efe983616377c60477cc6612 blob 4 -42061c01a1c70097d1e4579f29a5adf40abdec95 blob 9 -45b983be36b73c0788dc9cbcb76cbb80fc7bb057 blob 3 -4f5089b76757df68fb1b6b02be2c8da302d03550 tree 37 -785c0b097dc21d45f9726d412592b7609526c4c6 tree 72 -8884f45f3cf3e306cf0eeaa39febd41808151db5 commit 166 -cd95f8951573f4e021ddb57594dd3376a2b2f644 commit 214 +HarnessRouter may also reuse a session through its existing +`metadata.session_id` recovery path. After session resolution, every reused +session rejects `metadata.allagents.workspace` regardless of which identifier +selected it. For a workspace-bound session, the stored binding and harness +select execution, and any caller-supplied harness must match exactly; the gateway +rejects a mismatch before Git, hydration, or provider work. A reused session +without an AllAgents binding and without workspace metadata retains pinned +upstream routing behavior. + +A valid workspace continuation reuses the exact checkout, control root, and +execution working directory. A missing, expired, cleaned, or mismatched checkout +fails closed; it is never silently cloned again. + +### Provider and harness contract + +The deployment defines two protocol-specific HarnessRouter connections that +point to the same existing OAuth gateway and use the same server-side base URL +and API-key secret: + +- a Responses-format connection used only by Codex; and +- an OpenAI Chat Completions connection used only by OMP. + +This is one external provider route with two HarnessRouter protocol adapters, +not two credential authorities. The OAuth gateway owns user login, upstream +token storage, refresh, and repair. + +- A new session selects only an allowed `metadata.harness_id` and model. +- A reused workspace-bound session derives its harness from stored session + state; a supplied mismatch fails before execution. Unbound sessions retain + upstream routing. +- Each custom harness has an explicit model allowlist and a one-entry provider + policy pointing to its protocol-specific connection. +- There is no fallback connection or automatic transport switching. +- Provider base URL, API key, transport, headers, and model mapping cannot be + supplied in UHP input or workspace metadata. +- Provider failure is returned as ordinary HarnessRouter/UHP failure. It never + changes workspace or routing state. +- HarnessRouter runs in brokered sandbox mode. The harness receives a + short-lived session-scoped credential and loopback broker URL, never the + long-lived external-gateway key. The scoped credential exists only for the + active turn and is removed before checkpointing or public file collection. + +## Security and resource invariants + +These are release requirements, not later hardening: + +1. **URL and DNS:** accept only public HTTPS destinations. Reject loopback, + link-local, private, carrier-grade NAT, documentation, multicast, reserved, + and otherwise non-public IPv4/IPv6 results. Validate every DNS answer before + connection, pin the validated address for that hop, revalidate every redirect, + and cap redirects. A public name that resolves to any forbidden address + fails closed. +2. **Git protocols:** disable `file`, `ssh`, `git`, `ext`, and helper-driven + alternate protocols. Clear inherited Git configuration and credential + helpers. Set terminal prompting off. Requests never provide credentials. +3. **Repository execution:** disable repository hooks and clean/smudge/process + filters. Do not initialize submodules. Detect and reject gitlinks and Git LFS + pointer-backed content rather than executing helpers or returning a partial + workspace as complete. +4. **Filesystem confinement:** each runner-owned session root has separate + checkout and control children. Build the checkout in sibling staging, then + publish it into an absent checkout target. Keep harness home, credentials, + scratch, skills, and runner state in the control root. Reject absolute paths, + `..`, empty segments, NUL, platform separator ambiguity, and symlinks that + escape the checkout. The execution working directory must exist beneath the + checkout and never changes the control root. +5. **Exact provenance:** resolve the requested ref, fetch the corresponding + commit, detach checkout at that commit, and verify `HEAD` equals the recorded + object ID before publication. Ref movement after resolution cannot change the + bound checkout. +6. **Bounds:** enforce server-owned limits for request bytes, ref and path + lengths, clone/fetch duration, materialized bytes, inodes, process output, + concurrent materializations, active harness time, and idle checkout TTL. + Limits apply during work, not only after completion. +7. **Process control:** run Git and materializer children in a cancellable process + group with a minimal environment, bounded stdout/stderr capture, and a hard + termination deadline. Cancellation must stop descendants. +8. **Publication:** a workspace-aware first hydrate creates an isolated empty + session root without stock Git initialization or other files in the checkout + target. Materialize and validate in sibling staging, persist a pending + reservation, atomically publish staging as the checkout child, create the + separate control child, then commit the usable binding. Startup cleanup + removes abandoned staging, pending reservations, and published-but-unbound + roots. A harness cannot observe staging or a partially validated tree. +9. **Isolation:** never bind one session to another session's checkout. Each + first turn creates a new private checkout even when URL, ref, and resolved + commit are identical. +10. **Cleanup:** removal is safe to repeat, never follows links, is confined to + the recorded session root, and cannot remove another session's data. +11. **Provider secrets:** `HR_SANDBOX_TRUST=owner` is forbidden. The local + HarnessRouter broker must exchange a scoped turn credential for the real + external-gateway key. The long-lived key never enters the harness + environment or filesystem. Generate any CLI credential config in a + per-turn ephemeral control subtree, explicitly exclude it from checkpoints + and public file/artifact APIs, and delete it before terminal persistence. + Neither long-lived keys nor scoped broker tokens may survive in a + checkpoint, session file, artifact, stream, response, or log. + +## Workstreams and implementation phases + +The phases are ordered by dependency. Each phase ends with observable behavior; +implementation does not advance on the strength of source inspection alone. + +### Phase 1: Establish the downstream repository and immutable pins + +**Outcome:** `allagentsdev/allagents-gateway` is the sole implementation +repository and can reproduce the examined HarnessRouter baseline. + +Work: + +1. Rename the existing GitHub fork `allagentsdev/harnessrouter` to + `allagentsdev/allagents-gateway`. Preserve its upstream fork relationship, + branches, history, issues, rules, secrets, and package/container permissions. +2. Set `HarnessRouter/harnessrouter` as the documented upstream remote and record + the baseline commit + `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` / v0.25.4 in release automation. +3. Keep the downstream patch series reviewable: baseline sync commits are + separate from AllAgents behavior commits, and upstream merges never combine + with feature changes. +4. Pin UHP `2026-09-12`; the base image by manifest digest; Codex and OMP runtime + versions; system packages that affect Git/materialization; and every CI action. + No `latest`, floating branch, or unbounded package range may enter a release. +5. Add an exact Promptfoo development dependency and committed lockfile in this + repository. Promptfoo is an image consumer, not a service dependency. +6. Rename image/repository references and release coordinates to + `ghcr.io/allagentsdev/allagents-gateway`. Do not reuse the AllAgents npm + version or publish under the old image name. +7. Capture an upstream-diff check in CI so every release identifies the baseline + and downstream commits included in the image. + +Behavior-focused proof: + +- a clean checkout builds the same downstream commit from recorded pins; +- repository links, image labels, and release metadata identify + `allagentsdev/allagents-gateway` and the pinned upstream commit; and +- changing a pin or lockfile is visible as a reviewed source diff. + +Exit gate: the renamed repository builds the unchanged pinned HarnessRouter +image before workspace behavior is introduced. + +### Phase 2: Define and enforce the exact workspace schema + +**Outcome:** the gateway recognizes only the bounded v1 object and ordinary UHP +requests remain stock behavior. + +Primary area: `gateway/app.py` and focused gateway tests. + +Work: + +1. Parse nested `metadata.allagents.workspace` on initial Responses requests. + Leave handling of unrelated metadata unchanged. +2. Apply generic byte/depth bounds before response/session allocation. +3. Strictly validate the exact request schema defined above, including unknown + field rejection and normalized relative working-directory rules. +4. Extend session resolution to return whether it created or reused a session. + Store the canonical descriptor only when the resolver proves the session is + new. Use one canonical serialization and digest so idempotent replay cannot + create a second binding. +5. Reject workspace metadata for every reused session, whether selected through + `previous_response_id` or `metadata.session_id`, before Git, hydration, + runner, or provider work. +6. For a reused workspace-bound session, derive the harness from stored state + and reject a caller-supplied mismatch before hydration. Leave reused unbound + sessions on the pinned upstream routing path. +7. Map schema and reuse failures into stable UHP errors with the offending + parameter named. Do not expose Python exceptions or internal paths. +8. Keep requests without the object on the untouched upstream path. + +Acceptance examples: + +| Case | Observable result | +|---|---| +| Valid URL only | Accepted; checkout root becomes effective working directory | +| Valid URL, ref, and nested directory | Accepted; values reach the single workspace hook | +| Flat `metadata["allagents.workspace"]` | Rejected as not the v1 extension | +| Missing URL, unknown field, array, or non-string field | 400 before materialization | +| Absolute or parent-traversing working directory | 400 before materialization | +| Continuation containing a workspace object | 409 before materialization | +| Existing `metadata.session_id` plus workspace object | 409 before materialization | +| Workspace-bound session with a different `harness_id` | 409 before hydration | +| No workspace object | Same status, response, and runner path as pinned upstream | + +Tests must assert the HTTP/UHP contract and absence of materializer/provider +activity, not internal helper calls or field-copy plumbing. + +### Phase 3: Add the workspace-aware hydrate and runner seam + +**Outcome:** one workspace-specific path prepares the runner allocation after +session resolution and before provider or harness execution, while ordinary +sessions keep the pinned path. + +Primary areas: `gateway/app.py`, `runner/server.py`, their existing transport, +and focused integration tests. + +Work: + +1. Extend the existing gateway-to-runner turn envelope with an optional canonical + workspace descriptor for a first turn and an optional hydrated workspace + binding for a continuation. Do not add a public endpoint. +2. Add a workspace-aware first-hydrate mode that performs the existing + isolation/wipe and ownership setup but leaves an empty session root with an + absent checkout child. It must not run stock `_git_ensure`, write + `.gitignore`, apply input files, or create harness state before publication. +3. Define one runner-owned materializer interface with two operations: + `materialize(firstTurnDescriptor, checkoutTarget, limits, cancellation)` and + `cleanup(binding)`. There is one configured implementation in v1 and no + registration mechanism. +4. Keep four distinct values in the binding: session root, checkout root, + control root, and execution working directory. The execution directory must + be a no-follow validated descendant of the checkout; the control root must be + a sibling outside repository content. +5. After materialization succeeds, complete the + pending-reservation/publication transition, create the control root, then + atomically commit the usable binding and public provenance. +6. On continuation, hydrate the bound session root and verify all stored roots + before use. Do not resolve, fetch, checkout, or validate caller workspace + metadata again. +7. Adapt the existing runner plumbing to explicit roots: spawn the harness in + the execution working directory; anchor durable HOME, scratch, skills, and + CLI state in the control root; and scope input files, produced-file Git + diffing, file APIs, and artifacts to the checkout. Generate credential-bearing + CLI config only in a per-turn ephemeral control subtree, delete it before + checkpointing, and exclude it defensively from checkpoint and public-file + walkers. Checkpoint the remaining session root. Do not initialize an outer + Git repository or overwrite the source repository's `.gitignore`. +8. Make the first-turn transition idempotent. Same-key replay returns the owning + response/binding. A competing request cannot materialize or bind a second + checkout for the same session. +9. Preserve pinned behavior for non-workspace requests and unbound + continuations. Preserve existing streaming, cancellation, terminal-state, + and provider-error ordering around the workspace path. +10. Ensure workspace failures terminate before the provider receives a request. + +The seam is intentionally workspace-specific. It does not generalize arbitrary +metadata into runner callbacks and does not introduce extension discovery, +capabilities negotiation, plugin loading, or hook chaining. + +Acceptance examples: + +- a probe harness sees a committed file from the repository on its first + instruction; +- a nested `workingDirectory` becomes process cwd while HOME and `.harness` + state remain outside the checkout; +- source-controlled `.harness` paths and `.gitignore` cannot collide with or + rewrite runner state; +- materializer failure produces no provider request, agent process, or published + checkout; +- restart and continuation restore both checkout mutations and control state; +- same-key replay owns one checkout; and +- an ordinary non-workspace request exercises the unchanged upstream sequence. + +### Phase 4: Implement the Git materializer + +**Outcome:** the in-image materializer produces one verified private checkout and +returns a binding only after all safety and resource checks pass. + +Suggested home: a small `runner/allagents_workspace/` package plus a single +in-image executable entry point. Reuse HarnessRouter's process, cancellation, +logging, and session identity primitives instead of creating a daemon. + +Work: + +1. Define typed internal request/result/error records matching the runner seam. + The result contains the internal checkout root, validated execution directory, + and safe public provenance; errors never contain secrets or uncontrolled Git + output. +2. Canonicalize and authorize the HTTPS URL before running Git. Implement the + DNS, redirect, IP-range, and protocol rules as one acquisition path used by + ref resolution and fetch. +3. Resolve omitted ref through the remote symbolic default and requested + `refs/heads/*`, `refs/tags/*`, or unambiguous branch/tag shorthand through + advertised SHA-1 refs. Reject raw object IDs, other ref namespaces, SHA-256 + repositories, missing refs, and ambiguous shorthand. Peel an annotated tag + and verify the final object is one commit. +4. Create staging as a sibling of the absent runner-assigned checkout target, + with a random unguessable component and restrictive ownership/mode. The + caller never influences a host path. +5. Run Git with isolated config and environment. Fetch only the advertised ref + needed for the resolved commit, disable helper execution, and check out + detached. +6. Reject submodule entries and LFS-managed content. Validate tree confinement, + materialized byte/inode limits, checkout `HEAD`, and optional execution + working directory. +7. Atomically rename validated staging to the checkout target and return the + result. Never write the sibling control root or derive a filesystem path + directly from a URL, ref, working directory, response ID, or caller string. +8. Remove staging on every error, timeout, cancellation, or crash-recovery sweep. + A failed attempt cannot become a resumable checkout. +9. Bound concurrent materializations with one server-owned semaphore. Saturation + fails or waits only within the request deadline; it never creates unbounded + processes. +10. Emit structured stage/error codes for URL policy, ref resolution, fetch, + source feature rejection, limits, validation, publication, and cancellation. + Keep stderr capped and private. + +Behavior-focused tests: + +- exact commit checkout when a branch advances between later requests; +- omitted-ref resolution to the advertised default branch; +- safe nested working directory and rejection of file/nonexistent/escaping paths; +- rejection of raw object IDs and SHA-256 repositories; +- a nested working directory that cannot move control state into repository + content; +- redirect and DNS rebinding attempts into forbidden address ranges; +- forbidden Git protocols, embedded credentials, hooks, filters, submodules, and + LFS content; +- byte, inode, time, output, and concurrency limits during materialization; +- cancellation kills Git descendants and removes staging; and +- two sessions requesting the same commit receive different writable roots and + cannot observe each other's mutations. + +Use controlled test origins/resolvers for adverse network cases and one stable +public fixture repository for built-image proof. Tests must observe files, +commit identity, isolation, errors, and process termination rather than mock +argument forwarding. + +### Phase 5: Wire Codex and OMP to the external provider + +**Outcome:** both required harnesses execute through the operator's single +OAuth-to-OpenAI-compatible gateway without exposing or changing its +credentials. + +Primary areas: existing HarnessRouter provider connections/policies, custom +harness definitions, image runtime installation, and focused runner tests. + +Work: + +1. Install exact pinned Codex and OMP versions in the image and enable only the + required release backends with `HR_BACKENDS=codex,omp`. +2. Define stable custom harness IDs, for example `allagents-codex` and + `allagents-omp`, using HarnessRouter custom harness definitions. Store their + instructions, tools, allowed models, and base harness in gateway-owned + configuration. +3. Configure two logical HarnessRouter connections from the same server-side + external-gateway base URL and API-key secret: `responses` for Codex and + `openai` Chat Completions for OMP. Give each harness policy a one-entry chain + containing only its matching connection. +4. Force Codex onto the Responses connection and OMP onto the Chat Completions + connection. Fail startup/readiness if either selected model and endpoint + combination is unsupported. Never switch transport or connection after an + error. +5. Override the self-host image's owner-trust default with + `HR_SANDBOX_TRUST=broker`. Make the existing local + `HARNESS_GATEWAY_URL` satisfy broker availability in self-host mode without + requiring a publicly reachable gateway URL. +6. Add a readiness probe that mints a scoped turn credential, reaches the + loopback broker, and proves the real external-gateway key remains gateway-side. +7. Reject caller-selected models outside the harness allowlist and any request + field that attempts to replace provider routing. +8. Confirm that OMP receives an ordinary session-local home/config and cwd. It + must not read AllAgents local profiles or host configuration. Generate its + credential-bearing `models.json` and `models.yml` under a per-turn ephemeral + control subtree using only the scoped broker token, then delete both before + terminal checkpointing. Add explicit checkpoint and public-file exclusions + for the OMP paths as defense in depth. +9. Redact both the external-gateway key and scoped broker tokens from logs, + traces, stored responses, session files, checkpoints, artifacts, and test + snapshots. + +Acceptance examples: + +- Codex completes a real Responses turn through the external gateway; +- OMP completes a real Chat Completions turn through the external gateway; +- each harness starts inside the materialized working directory; +- an unsupported model fails before provider traffic; +- an invalid provider credential returns the upstream provider failure without + selecting another connection; and +- callers cannot observe or override base URL, API key, or transport. + +### Phase 6: Complete continuation, cancellation, and cleanup + +**Outcome:** the checkout lifecycle follows the existing session lifecycle with +bounded ephemeral storage and no separate lifetime platform. + +Work: + +1. Add a server-owned `ALLAGENTS_WORKSPACE_TTL_SECONDS` with a finite safe + default. Callers cannot set or extend it directly. +2. Start/reset the idle expiry only after a terminal turn is durably recorded. + An active materialization or harness turn is not removed by the idle sweeper. +3. On a valid continuation, verify the stored checkout identity/root and use the + exact prior working directory and mutations. Reset the idle deadline only + after that turn reaches a terminal state. +4. On first-turn cancellation during materialization, terminate the process + group, remove staging, and leave no resumable binding. +5. On cancellation after publication, stop the harness through stock + HarnessRouter behavior and retain the bound checkout only until the ordinary + finite idle deadline, so a permitted continuation sees prior mutations. +6. On expiry or explicit existing-session deletion, mark the binding unavailable + in the session transaction and invoke idempotent confined cleanup. A late + continuation fails closed and cannot recreate the checkout. +7. On startup, remove abandoned staging, pending reservations, + published-but-unbound roots, and cleanup-marked session roots. Do not add a + second database, durable queue, elaborate deletion ledger, or general storage + collector. +8. If cleanup encounters a transient host error, keep the binding unavailable, + report an operator-visible error, and retry the same idempotent removal on the + next bounded sweep/startup. Never make the checkout executable again. + +Acceptance examples: + +| Scenario | Observable result | +|---|---| +| Turn 1 edits a file; turn 2 reads it | Turn 2 sees the edit in the same checkout | +| Turn 2 omits workspace metadata | Stored binding selects checkout and cwd | +| Turn 2 includes workspace metadata | Rejected before Git/provider activity | +| Checkout missing or identity mismatched | Continuation fails; no rematerialization | +| Git cancellation | Child processes exit and staging disappears | +| Harness cancellation | Response is cancelled; no process remains; checkout follows finite idle expiry | +| Expiry races with continuation | Exactly one wins through existing session serialization; checkout is never used after cleanup begins | +| Cleanup called twice or after restart | Same final absent state; no neighboring path changes | + +### Phase 7: Build the one-container operator surface + +**Outcome:** an operator can start the built AllAgents Gateway image with Docker +Compose, one data volume, and no helper service. + +The checked-in Compose contract is equivalent to: + +```yaml +services: + allagents-gateway: + image: ghcr.io/allagentsdev/allagents-gateway:${ALLAGENTS_GATEWAY_VERSION}@${ALLAGENTS_GATEWAY_DIGEST} + ports: + - "127.0.0.1:3000:3000" + env_file: + - .env + environment: + HR_BACKENDS: codex,omp + HR_SANDBOX_TRUST: broker + ALLAGENTS_WORKSPACE_TTL_SECONDS: ${ALLAGENTS_WORKSPACE_TTL_SECONDS:-3600} + volumes: + - allagents-gateway-data:/data + restart: on-failure + +volumes: + allagents-gateway-data: ``` -Those bytes, including the final LF, digest to -`sha256:b3b49d7b3ff3fd8c4fb3acbf7de36d92f6a4029f3244064f79e37408463288b5`. -A change to the schema, fixture bytes, or either digest is a versioned contract -change, not an implementation detail. - -### Failure Contract - -Failures before response allocation use the UHP non-2xx error envelope. Failures -after allocation return HTTP 200 with terminal `status: "failed"` and the same -error object in `response.error`; workspace failures use -`type: "harness_error"`, a safe message, `param: null`, and -`detail: { "retryable": }`. Stock cancellation remains terminal -`status: "cancelled"`. The hook's `retryable` field must equal this catalog: - -| Code | UHP placement | Retryable | -|---|---|---| -| `invalid_input` | HTTP 400 `invalid_request_error` before response allocation for a non-object workspace extension; `param` is `metadata.allagents.workspace` | no | -| `allagents_workspace_too_large` | HTTP 413 `invalid_request_error` before response allocation for the 64-KiB/depth bound; `param` is `metadata.allagents.workspace`; `detail.max_bytes` is 65536 for the byte bound | no | -| `allagents_workspace_immutable` | HTTP 409 `invalid_request_error` before response allocation when a continuation contains the workspace extension; `param` is `metadata.allagents.workspace` | no | -| `allagents_workspace_expired` | HTTP 410 `invalid_request_error` before runner/profile work while a continuation's expired or deleted session tombstone remains retained; `param` is `previous_response_id` | no | -| `allagents_workspace_non_resumable` | HTTP 409 `invalid_request_error` before profile admission when a known attached session's bound generation key/epoch/reference/publication/private/checkpoint evidence is missing or corrupt; `param` is `previous_response_id`; because the attachment previously reached `ready`, include its committed complete public workspace metadata | no | -| `harness_unavailable` / `detail.reason: "allagents_auth_profile_capacity_exceeded"` | HTTP 503 `server_error` before response allocation when a native profile has reached its configured concurrent-turn limit; `param` is null | yes | -| `harness_unavailable` / `detail.reason: "allagents_auth_profile_unavailable"` | HTTP 503 `server_error` before response allocation for an unavailable or repair-required auth binding, or while profile maintenance is pending or active; `param` is null | yes | -| `allagents_workspace_invalid` | failed response for post-allocation caller repository descriptor, URL/egress-policy, snapshot catalog, path, layout, access/retention value, OCI shape/index, or unsupported media rejection that is not a numeric limit | no | -| `allagents_workspace_persistence_forbidden` | failed response when `persistent` retention is not authorized for the selected deployment target | no | -| `allagents_workspace_read_only` | failed response when a read-only initial request contains workspace input files or attachment policy would create writable shadow state | no | -| `allagents_workspace_capacity_exceeded` | HTTP 503 `server_error` before response allocation when generic session/tombstone admission cannot reserve capacity; otherwise a failed response when finite staging, generation, private, session, or persistence capacity cannot be reserved after safe eviction | yes | -| `allagents_workspace_private_quota_exceeded` | failed response when an editable waiter's private view cannot fit the complete generation or an attached editable turn exhausts its fixed per-session byte or inode allowance; access and retention remain unchanged | no | -| `allagents_workspace_source_auth_failed` | failed response for Git or registry credential rejection | no | -| `allagents_workspace_acquisition_failed` | failed response when Git, registry, HTTP, or transport I/O prevents complete byte acquisition; excludes digest, schema, and limit failures | no | -| `allagents_workspace_limit_exceeded` | failed response for source/archive/manifest repository, entry, byte, layer, file, path, or header limits | no | -| `timeout` | failed response only for an unexpected execution timeout before the declared task budget; declared budget exhaustion remains UHP `incomplete` | no | -| `allagents_materializer_failed` | failed response for validate/resolve/materialize spawn, nonzero, crash, or malformed/oversized hook output | no | -| `allagents_secret_boundary_violation` | failed response when a post-allocation recheck finds a configured source-secret name or value in service or agent state | no | -| `allagents_workspace_manifest_invalid` | failed response for workspace-manifest media type, schema, canonical bytes, or declared digest | no | -| `allagents_workspace_integrity_mismatch` | failed response for source descriptor digest/size mismatch, validated/resolved-plan or generation-key drift, staging/manifest mismatch, semantic Git-state failure, or corrupt ready generation | no | -| `allagents_workspace_publication_failed` | failed response for generation claim/publication/marker failure | no | -| `allagents_workspace_attachment_failed` | failed response for read-only mount/reference or private writable-view publication failure | no | -| `allagents_workspace_checkpoint_failed` | failed response for editable root/nested checkpoint or protected collection-state failure | no | -| `allagents_workspace_state_failed` | failed response for generation/session/pin/reference/quota/expiry/tombstone/purge persistence or CAS failure | no | -| `allagents_workspace_containment_breach` | failed response for completed-parent/live-descendant even if forced kill succeeds; an unquiescent leaf remains internal until restart proves it empty | no | - -Classification order is normative. For continuation, generic validation rejects -an extension-bearing request before session lookup/CAS. The session CAS then -linearizes busy, exact binding, and expiry/deletion predicates without mutating -state on rejection; a known attached but corrupt physical attachment returns -non-resumable and rolls back its provisional admission before profile admission. -For an initial request, generic metadata errors precede the idempotency claim; -stock `session_busy`, then native-profile readiness and concurrent-turn capacity, -then generic session/tombstone capacity determine pre-allocation admission. After -allocation, source-free descriptor validation -and selected-reference verification precede secret-boundary recheck, persistence -authorization, read-only conflict, access-specific capacity reservations, source -resolution, build/staging/prospective-generation capacity, acquisition I/O, -post-build full-tree accounting, numeric limits, source semantic validation, -hook envelope, manifest, cryptographic/tree/Git/generation integrity, -publication, per-waiter manifest-cwd validation, private fit, attachment, -editable checkpoint, private runtime quota, then durable state failure. A -completed-parent/live-descendant -violation is failed -containment; other containment failure supersedes hook state but never overwrites -UHP-mandated `cancelled` or `incomplete`. Declared budget exhaustion produces -`incomplete`; an unexpected earlier timeout produces `timeout`. - -One terminal outcome carries exactly one code. Capacity is retryable because -expiry or operator deletion may free protected space, but no retry delay is -guessed and Promptfoo does not retry automatically. A failed physical GC deletion -is quarantined/accounted operational state; it becomes a task error only when -admission cannot reserve capacity. - -Deployment `preflight` runs before readiness and allocates no UHP response. Its -failure stays operational: readiness is false and the safe operator diagnostic -uses the same classification vocabulary without pretending a task failed. - -Vendor codes and vendor detail reasons use the required `allagents_` prefix. A -request failure includes `detail.retryable`; the profile-capacity reason omits -`retry_after_ms` rather than guessing. Promptfoo maps a non-2xx or `failed` -response to `ProviderResponse.error = ": "`. It maps -`incomplete` to `ProviderResponse.error = "incomplete: task stopped at a budget"` -and `cancelled` to -`ProviderResponse.error = "cancelled: task cancelled by client"`; both have -`code: null` and `retryable: false`. Every failure includes -`metadata.uhp = { httpStatus, responseStatus, code, reason, retryable }`. -`responseStatus` is null for a pre-allocation non-2xx request error and is the -actual terminal status for an HTTP-200 response. Before attachment reaches -`ready`, failures omit `metadata.allagentsWorkspace` entirely. After `ready`, -terminal failures include the same complete verified workspace metadata as -success, with the actual terminal expiry value. Internal epoch, reservation, -claim, pin, and physical-path identifiers are never public. `reason` is the error -detail reason or the incomplete detail reason when present, otherwise null. -Promptfoo never converts a non-2xx, failed, incomplete, or cancelled UHP result -into successful empty output and performs no automatic retry. - -### Fork Maintenance Contract - -The 2026-09-27 upstream recheck inspected HarnessRouter -[`5f82db1d`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), -released as -[`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4). -Stock now provides a session lease before hydration, per-session workspaces, -durable checkpoints, safe file/package materialization, and per-turn API-key -brokering. The fork must reuse those lifecycle and security primitives. Stock -still forwards no ordinary request metadata to the runner and provides neither -the pre-turn Git/OCI materializer contract nor durable named native OAuth -profiles. Both planned seams therefore remain necessary at this pin. -ADR 0002's upstream-status section records the source evidence. - -- Keep the fork in a dedicated repository/branch with the upstream remote intact. -- Pin production images to an upstream commit, never a moving branch. -- Keep the materializer changes as a small ordered patch series with focused - commits and no formatting churn. -- For every selected upstream upgrade: rebase the patch series, inspect upstream - changes in touched gateway/runner/session code, run upstream tests and UHP - conformance, run AllAgents hook/session E2E, rebuild the image, and record the - new inputs and digest. -- Before accepting every new upstream pin, repeat the documented two-seam - capability check. Record which upstream source replaced any patch, and delete - that patch in the same upgrade; absence of a recheck blocks the upgrade. -- Publish releases to `ghcr.io/allagentsdev/harnessrouter`, record the manifest - digest, and rehearse deployment from that digest rather than a local build or - mutable tag. -- Prepare upstream proposals as generic command/plugin and harness-auth-state - seams. Do not require upstream to understand AllAgents metadata, Git URL - semantics, OCI manifests, Promptfoo, or a specific OAuth provider. -- If upstream accepts an equivalent seam, delete the patch rather than retaining - a compatibility layer. - -### Risks and Mitigations - -- **Fork drift:** Keep the patch ordered and narrow, pin commits, rebase only - selected releases, and run upstream conformance plus lifecycle E2E. -- **Gateway/runner durability split:** Make the gateway the sole session-state - writer and the runner the sole resource-journal writer. Fault every - claim/publication/pin/prepare/evidence/ready-ack/reference/expiry/deletion CAS; - reconciliation preserves a gateway-committed attachment or rolls resources - back once without replaying acquisition. -- **Generation-key collision or incomplete identity:** Generate the key from a - versioned canonical resolved plan containing every source-visible - byte/layout-affecting input, bind it to one manifest digest and semantic Git - record, and reject drift before reuse. Exclude volatile `.git` representation - only after closed semantic validation. -- **Caller-controlled Git URL SSRF or credential forwarding:** Use one strict - canonical URL serialization across validation, policy, credentials, DNS, and - Git. Force every connection and bounded redirect through the public-address - acquisition connector, reject mixed answer sets, and pin the approved address - against DNS rebinding. Bind the helper to a structured credential scope and - strip the credential on any scope escape, including same-origin redirects. - Clear inherited proxies and deny a direct network path. -- **Concurrent build and publication race:** Use one runner-owned keyed claim, - private staging, independent reconstruction, atomic publication, and one - shared result. Each request detaches on its own cancellation/deadline; one - waiter cannot cancel another, and the build stops when no live waiter remains. - Never attach `building`, quarantined, or deleting state. -- **Read-only escape or writable alias:** Keep the generation backing store - owner-writable only, verify mount flags and mount topology, forbid writable - bind aliases and hard-link aliases, and probe writes through root, nested - repositories, symlinks, and alternate paths. -- **Editable cross-session leakage or growth:** Create a unique private writable - view and checkpoint namespace per fitting session, prove that no mutable state - or writable alias is shared, reject only a waiter whose initial view cannot - fit, reserve its full byte/inode allowance from global capacity, and enforce - that hard quota through every continuation. -- **Produced-file drift or expensive full inventories:** Use the verified - generation as the first-turn baseline and retain only protected cumulative - path state that differs from it. Verify trusted candidates after quiescence; - if candidate state is incomplete, overflowed, or uncertain after recovery, - perform a bounded no-follow full-tree scan. Never trust editable `.git`, - agent-supplied paths, or final ignore rules. -- **Nested Git versus mode-specific checkpoints:** Preserve repository `.git` - state inside the generation. Read-only sessions do not mutate or checkpoint - it; editable views preserve private Git state for continuation while - collection uses only protected generation and turn state. -- **Lease, expiry, and deletion races:** Linearize unexpired-idle or persistent - turn admission against tombstoning; hold a provisional pin through attachment - prepare/ack; persist exact epoch references before mount exposure; recheck - reference/pin protection under lock; release exactly once; and reconcile leaked - or under-counted state before readiness or GC. -- **Pinned capacity starvation and tombstone growth:** Configure finite staging, - generation, per-private-session and total private byte/inode, session, - persistent, and tombstone limits plus high/low watermarks. Reserve a tombstone - slot at session admission, compact only after response/idempotency retention, - expire ordinary state, and evict only zero-reference/zero-pin epochs with null - `lastUsedAt` first by `publishedAt`, then used epochs by `lastUsedAt`, - `publishedAt`, key, and epoch. Reject admission when protected state consumes - capacity. -- **Failed deletion or corrupt generation:** Quarantine and continue accounting - for it. Never advertise freed bytes, resurrect physical state, or substitute a - rebuilt generation inside an existing session. -- **Provider retry/fallback:** Complete one attachment before the provider loop - and persist a ready marker; retry cannot resolve, build, attach, or change - source/access/retention/auth mode. -- **Source credential leakage:** Use subprocess-only source credentials, - hermetic configuration, leak scans across staging, generations, and private - views, and a non-escapable cgroup boundary proven empty before result handling. -- **Native OAuth exposure:** Treat the selected profile as available to its - harness and same-identity tools only during an active turn. Use a - same-filesystem namespace projection, mount no other profile, never copy it to - durable session state, and verify teardown before terminal acknowledgement. -- **OAuth renewal loss or concurrency:** Allow concurrent turns to read one - profile. Require the pinned auth adapter to acquire the runner-owned renewal - mutex before any provider refresh call, reread the credential after acquiring - it, skip refresh when another turn already renewed, and atomically save one - valid update. Persist an incomplete-renewal record across crashes and mark the - profile `repair-required` when validity cannot be proven. Never hold the - renewal mutex for the whole turn. -- **Descriptor/session drift:** Accept the JSON key only initially and persist - descriptor, generation, access, retention, cwd, harness, and auth identity for - every later response. Continuation never re-resolves. -- **OCI attack surface:** Use a closed media profile, streaming digest checks, - fixed limits, strict path/link/type validation, and exact-host redirects. -- **HarnessRouter restart semantics:** Promise continuation only for completed, - unexpired or persistent state with valid attachment evidence. Interrupted - turns fail and are not replayed. -- **Registry/tag or publisher compromise:** Use protected publication with pinned - actions and verify attested owner/repository/workflow/ref/subject digest. -- **Upstream rejection:** The pinned fork remains supported; upstream delivery - reduces maintenance but is not a launch dependency. - -### Phased Delivery - -1. Build the pinned minimal image and pass the blocking Codex/Pi native-auth gate - without workspace code. -2. Red E2E against stock HarnessRouter: arbitrary metadata is neither forwarded - to Codex/Pi nor returned as workspace provenance. -3. Workspace lifecycle fork spike: a fake `preflight/validate/resolve/materialize` - hook, concurrent identical epoch claims, independently cancelled waiters, one - immutable publication, provisional pins, two read-only mounts, one quota- - bounded private writable view, per-waiter view-fit failure, attachment - prepare/ack, epoch eviction/republication, and restart reconciliation. -4. Prove read-only enforcement, writable-view isolation and growth limits, - mode-specific checkpoint/collection, nested cwd, provider-fallback non-reentry, - and unexpired/persistent continuation reuse. Stop if any invariant needs prompt - or client cooperation. -5. Freeze hook/state, generation-key, manifest, semantic Git, descriptor, - response/expiry, retention, failure, and lifecycle fixtures. -6. Implement caller-repository JSON validation, acquisition egress enforcement, - credential-scope mapping, Git validate/resolve/materialize, credential - containment, bounded acquisition, and generation publication. -7. Implement configured `workspace.yaml` OCI snapshot construction through the - same publication and attachment path. -8. Implement terminal-time TTL, persistent authorization, operator deletion, - hard private quotas, bounded tombstones, provisional-pin-aware deterministic - LRU eviction, and crash recovery. -9. Prove explicit proxy mode and run Promptfoo concurrent read-only, isolated - editable, continuation, expiry/deletion, capacity, cancellation, restart, and - failure mappings. -10. Review both repositories; publish and attest the GHCR digest; run green E2E - and conformance; document lifecycle operations; prepare upstream patches. - ---- - -## Implementation Units - -### U0. Harness-native auth adapter feasibility gate - -- **Goal:** Prove the native Codex and Pi authentication architecture before any - production workspace-materializer implementation. -- **Repositories/files:** Minimal pinned HarnessRouter fork image, - `runner/server.py`, Codex/Pi launch and home setup, auth-profile projection, - renewal coordinator and journal, session auth-binding persistence, checkpoint - exclusions, fault fixtures, and focused runner/gateway tests. Do not add the - AllAgents materializer or Git/OCI acquisition in this unit. -- **Approach:** Initialize dedicated profiles only through `codex login` and Pi - `/login`. Use an active-turn-only directory-level mount namespace or equivalent - same-filesystem credential view while keeping conversation state - session-scoped. Configure a finite concurrent-turn limit per profile; do not - lock the profile for a whole turn. Patch the auth adapter to acquire the - runner-owned renewal mutex before the provider refresh call, reread the current - credential and version, skip a redundant refresh, or record and atomically - save one renewal. Preserve incomplete renewal records across crashes. Mark a - profile `repair-required` only when startup cannot prove its credential valid. - Tear down each turn's projection before terminal acknowledgement, mount no - other profile, and prevent passive checkpoint, backup, log, output, or response - serialization. Login, logout, and repair use the durable maintenance journal. - Use the actual pinned harness versions and real provider traffic. Freeze and - record the HarnessRouter commit, base-image digest, Codex version, Pi version, - and auth-adapter patch digest used by the gate. -- **Verification:** For both Codex and Pi, complete login, a real first turn, - continuation, and restart without a provider-route API key. Change or remove - the binding and prove continuation fails before runner work. - - Use barriers for four concurrency cases. Two arrivals with the same new - `Idempotency-Key` produce one admission and one result. A second turn in one - session returns `session_busy`. Two different sessions using one profile and a - valid token execute concurrently without a renewal mutex on the normal request - path. Two different sessions starting with a forced-expired token both - succeed: exactly one provider refresh occurs, the other turn rereads the saved - credential, and no provider refresh happens outside the coordinator. Reject a - configured profile limit of one. Set the limit to two and prove a third turn - fails before response allocation with - `allagents_auth_profile_capacity_exceeded`. - - Inject cancellation, the requesting turn's deadline, gateway-only crash, - runner crash, and whole-process death before the pending renewal record, after - that record but before the provider call, after provider rotation, during - atomic local save, and after commit. Before the pending record, the turn may - release the mutex. After it, the coordinator retains ownership independent of - the turn, no sibling can issue a second refresh, and terminal acknowledgement - waits for a committed credential or durable `repair-required` fence. On - coordinator timeout, prove the refresh process boundary empty before releasing - the mutex. - - Restart must reconcile every turn slot and renewal record, see a complete - credential that validates or mark only that profile `repair-required`, remove - or quarantine stale projections, find no credential path in retained CLI - homes, and prove old descendant boundaries empty before readiness. Break one - profile and prove its bindings return `allagents_auth_profile_unavailable` - while healthy bindings remain advertised and global readiness succeeds. - - For login, logout, and repair, inject timeout and crash while `pending`, during - the active command, after credential mutation, and before final validation. A - pending operation may fail and resume admission only before it becomes active. - An active fence survives cancellation, deadline, and restart until the - administrative process is gone and the profile validates or becomes - `repair-required`. New turns receive - `allagents_auth_profile_unavailable` throughout maintenance. - - Prove unselected profiles and other sessions' conversation state are - inaccessible. With an inert agent, scan checkpoints, produced-file records, - backups, passive logs, and response metadata for automatic credential - serialization. Record that an active same-identity tool can still read or emit - the selected credential. Preserve those exact input identities with the - evidence. -- **Gate:** U1-U6 must not begin until both required native targets pass. Failure - stops dependent work and reopens ADR 0002; proxy-only scope requires an - explicit decision change and cannot count as a passing native gate. Any change - to a frozen input invalidates the gate and stops dependent work until both - native targets pass again on the new input set. - -### U1. HarnessRouter fork and hook feasibility - -- **Goal:** Prove the smallest production-direction fork can atomically publish - one immutable generation, attach it in both access modes, and reconcile its - lifecycle before provider dispatch while preserving stock UHP. -- **Repositories/files:** HarnessRouter fork `gateway/app.py`, - `runner/server.py`, generation/session persistence, mount/view and - checkpoint/produced-file helpers, runner/gateway tests, and a fake - preflight/validate/resolve/materialize hook. -- **Approach:** Add opaque metadata bounds, typed operation envelopes, - runner-owned authorization/admission, generation-key/epoch claims with - independent waiter cancellation, separate generation/resource and gateway - session CAS state, canonical manifest fixtures, atomic publication, - provisional pins, attachment prepare/ack, durable epoch references, verified - read-only mounts, per-waiter fit checks and unique hard-quota-bounded private - writable views, mode-specific checkpoints, protected sparse collection state, - candidate verification with full-scan fallback, safe nested cwd, - stage-dependent response metadata, and cgroup containment. Add fake finite TTL, - persistence, tombstone, quota, deletion, and epoch-republication state - sufficient to prove restart ordering; U3 completes production policy and GC. -- **Verification:** Upstream UHP conformance stays green. Two concurrent - identical read-only initial requests execute fake materialize once, attach the - same generation under separate UIDs and harness/profile bindings, deny writes - through root/nested/symlink/alternate paths, and isolate runtime state. Two - editable sessions receive private writable views; one mutation and checkpoint - never appears in the other or generation. Continuation reuses its original - mode and state without the extension. A large clean fixture creates no - redundant pre-agent inventory. Candidate collection and forced full-scan - fallback produce the same per-turn source-visible delta. Both exclude changes - to the protected declared `.git` subtrees while counting source-visible ignore - files and an agent-created `.git` elsewhere; a coverage gap before a UHP input - overlay forces the full scan. - - Fault every validate/resolve/claim/waiter/containment/publication/pin/ - prepare/ready-ack/reference/mount/view/quota/checkpoint/CAS boundary. The runner - rejects forged manifests, changed staging, escaping links, invalid repository - destinations, writable aliases, and generation-key drift. Restart exposes only - a complete publication plus valid attachment evidence; provider fallback never - invokes the hook again. - -### U2. AllAgents workspace contracts and Git materializer - -- **Goal:** Implement the caller-repository JSON descriptor, the clean - `workspace.yaml` repository-URL migration and optional snapshot catalog, - canonical generation identity, deterministic Git construction, logical cwd, - and provenance. -- **Files:** `src/models/workspace-config.ts`, - `src/models/execution-workspace.ts`, `src/core/execution-workspace.ts`, - `src/core/workspace-repo.ts`, `src/core/managed-repos.ts`, workspace CLI and - migration metadata, acquisition egress integration, generated v2 schemas, - build packaging, configuration docs, and Git E2E fixtures. -- **Approach:** Reuse authoritative URL/path normalization and workspace snapshot - parsing. Cut local repository configuration from `source` plus `repo` to - `url`, preserving `path` and branch-specific managed semantics; provide the - explicit one-time migration and remove legacy fields from ordinary parsing, - output, docs, and schemas. Keep caller Git URLs plus access/retention in the - JSON execution descriptor; keep only operator-owned snapshot catalog entries - relevant to execution in `workspace.yaml`. Generate the manifest schema and - add descriptor/preflight/validate/resolve/materialize/result schemas, defaults, - canonicalization, generation-key construction, credential-scope selection, - and public-egress enforcement. Preflight returns configured reference - identities without request URLs or values; validate checks URLs, refs, - destinations, and the syntax and lexical safety of the workspace path, then - selects a bounded mapped subset - without source access. The runner verifies their handles and injects only that - selected set into source-access children. Resolve exactly the caller-declared - repositories to commits without writing source bytes, and materialize only the - exact cache-miss resolved plan into staging. Preserve nested `.git` while - excluding volatile administrative bytes from the source-visible manifest, - enforce closed semantic Git validation, and prove the manifest equals the - union of resolved commit trees at pairwise non-overlapping destinations plus - necessary ancestor directories. Validate destinations, compute the manifest, - and return without publishing. The runner validates each waiter's logical cwd - against that verified manifest before attachment. -- **Verification:** Schema/CLI fixtures migrate every supported legacy provider - pair to a credential-free canonical URL, preserve `path`, `branch`, skills, - descriptions, and managed mode, require `url` for managed entries, retain - path-only unmanaged entries, and reject ambiguous, credential-bearing, mixed - old/new, or legacy shapes in the normal parser and v2 schema. Local public- - address HTTPS fixtures cover caller URLs, refs/defaults/HEAD, the same URL at - different refs/destinations, multiple repositories, duplicate and ancestor/ - descendant destinations, undeclared root/side files, ref grammar, helpers, - submodules/LFS/file/ext/ssh protocols, bounded safe redirects, cancellation, - partial cleanup, descriptor defaults, access/retention validation, - `workspaceRoot` and valid/invalid `workspacePath` for Git and snapshots, - anonymous and scope-mapped credential identity, private generation-key/public - generation-ID separation, exact URL provenance, schema fixtures, commit-tree/ - manifest reconstruction, concurrent identical resolve identity, and - repository/entry/byte/deadline boundaries. Network fixtures - reject userinfo, IP literals, controls, whitespace, backslashes, noncanonical - IDNA, explicit-default-port, and trailing-dot forms, encoded separators/dot - segments, loopback, link-local, - private, reserved, metadata, mixed public/private DNS answers, DNS rebinding, - unsafe redirects, raw-prefix lexical siblings, same-origin scope escapes, - redirect-selected credentials, inherited proxy bypass, and other out-of-scope - credential forwarding. Requests whose different ref spellings resolve to the - same URL, commit, destination, sharing scope, egress-policy version, and - selected credential-reference identities share one private generation key. - Different working directories, access, retention, harness/profile, and session - inputs also preserve that key. - -### U3. Session binding, failures, and credential containment - -- **Goal:** Complete crash-safe attachment retention, bounded disposal, exact - failures, and credential containment for continued sessions. -- **Repositories/files:** HarnessRouter generation/session/reference persistence, - lifecycle scheduler and operator deletion path, quota/GC configuration and - tests; AllAgents credential environment and hostile fixtures. -- **Approach:** Persist runner generation/resource state separately from the - gateway-owned session state; use attachment prepare/evidence/ready-ack and - reconcile both halves. Persist raw/effective descriptor digests, generation - key/epoch/manifest/semantic-Git/provenance, published/last-used timestamps, - access, retention, expiry, attachment evidence, and auth binding. Implement - active leases, terminal-time idle expiry, bounded tombstones and purge, - authorized persistent pins, provisional attachment pins, fixed private - byte/inode reservations and runtime enforcement, per-waiter initial view fit, - editable cleanup, read-only epoch-reference release, deterministic eviction of - ready zero-reference/zero-pin epochs, completed-eviction fencing before - republication, deletion quarantine, and startup reconciliation. Extend every - response/replay path and Promptfoo mapping with the exact failure catalog and - stage-dependent public metadata. - - Resolve source secrets only in the selected hook child, prove its cgroup empty - before results/publication/cleanup, project only the selected native OAuth - profile during the active turn, tear down and verify it before terminal - acknowledgement, and broker proxy mode separately. Scan staging, generations, - private workspaces, retained homes, mounts, checkpoints, backups, logs, and - responses. -- **Verification:** Fake-clock and tiny-quota fixtures prove generic - session/tombstone admission precedes visible response creation, invalid - descriptors remain accounted through failed-response purge, and one CAS - rejects busy/expired continuation admission while saving and clearing an - unexpired idle deadline. Profile admission either commits active or restores - the future deadline/tombstones an elapsed one; only terminal acknowledgement - sets the next deadline. Polling/replay do not; expiry races linearize; - persistent requests authorize before source access; explicit deletion is - idempotent; active, referenced, and pinned state is never evicted; one - editable session produces exactly one private reservation debit across success - and every crash point; - one non-fitting editable waiter fails without affecting a read-only or fitting - sibling; byte/inode growth fails at that allowance across continuations; - expired private workspaces release it; references and provisional pins release - exactly once; repeated successful publications return staging reservation to - baseline; null-last-used epochs sort first by publication time, then used - epochs by last-used time, with key/epoch tie-breaks; a new epoch waits for - complete prior eviction; bounded tombstones purge only after response/ - idempotency retention; failed deletion stays quarantined; and all-protected - capacity returns the cataloged retryable failure. - - Crash every generation-epoch/session/build-waiter/pin/prepare/ready-ack/ - reference/mount/view/quota/tombstone/unmount/purge/delete transition and require - reconciliation before readiness or GC. Continuation succeeds only for valid - exact-epoch evidence whose retention is persistent or session idle deadline is - unexpired; retained expiry returns 410, corrupt evidence returns 409 - non-resumable, and purged identity returns stock unknown, all without - rematerialization or epoch substitution. Credential fixtures prove source - secrets and caller keys absent everywhere agent-readable; the selected OAuth - credential is visible only through the active-turn owner-trust projection and - is absent after teardown. Proxy tokens reject every invalid scope or lifetime. - -### U4. Immutable OCI workspace materialization - -- **Goal:** Add OCI as the second immutable generation source, including - tree-only and normalized offline-history snapshots, without weakening Git - reuse, attachment, retention, or failure behavior. -- **Files:** AllAgents OCI client, manifest/archive and semantic Git validator, - workspace-manifest types, tree-only and history-bearing producer fixtures, - local registry E2E, generation fixtures, and security fixtures. -- **Approach:** Resolve only configured registries; treat `snapshotName` as the - operator catalog selector and `imageManifestDigest` as the required direct OCI - image-manifest identity; implement bounded Basic/Bearer auth and exact-host - redirects; compute the immutable resolved plan; on a generation miss verify - the image manifest, config, `workspaceManifestDigest`, and layers while - streaming; apply staging changesets; validate paths, types, limits, and the - catalog; and return through the same envelope as Git. Reject overlapping - repository destinations. For each manifest root that declares `git`, validate - detached `HEAD`, index/tree equality, exact object closure and digest, exact - worktree/commit equality, closed refs/config, and the absence of remotes, - credentials, and unsafe administrative state. Reject `.git` in tree-only or - undeclared locations. Runnable Harbor and SWE-bench environments remain - outside `allagents.workspace`; Promptfoo invokes Harbor through its separate - provider lane. They are never relabeled as workspace source snapshots. The - runner remains the sole publisher/resource preparer and - the gateway the sole session-attachment writer. -- **Verification:** Distribution fixtures cover exact request-field naming, - `snapshotName` lookup, direct `imageManifestDigest` enforcement, workspace- - manifest equality, tree-only snapshots, offline history with no configured - remote, `git log`/`git blame`/historical diff in read-only and editable - attachments, private editable Git-state isolation, exact commit/object-set/ - worktree mismatch, overlapping destinations, producer removal and materializer - rejection of remote or credential configuration, forbidden config/includes/ - hooks/alternates/worktrees/shallow/replace/graft/ref/reflog state, undeclared - `.git`, physical accounting, - auth, private CA, compression, whiteouts, redirects, rebinding, indexes, - foreign media, traversal, links, devices, sparse files, cancellation, cleanup, - no Git fallback, and exact error precedence. Concurrent identical OCI requests - produce one publication; read-only sessions share it; editable sessions get - private writable views; access, retention, cwd, harness/profile, and session - do not fragment its generation key. - -### U5. Harness-native OAuth, optional proxy, and Promptfoo E2E - -- **Goal:** Carry auth invariants into the complete image and prove concurrent - generation sharing, editable isolation, lifecycle policy, proxy isolation, and - consumer mappings. -- **Repositories/files:** custom image/configuration, auth-profile setup, - HarnessRouter lifecycle/integration fixtures, AI Evals Promptfoo provider - configuration, and deployment examples. -- **Approach:** Reuse the accepted U0 auth adapter. Configure dedicated Codex and - Pi profiles; exercise login, live turns, coordinated atomic renewal, - active-turn projection teardown, stale repair, same-binding continuation, - same-profile and different-profile concurrency, configured profile capacity, - and profile isolation. Separately validate proxy broker scope. Run Promptfoo - requests that select anonymous public and origin-mapped private HTTPS Git - repositories, Git/OCI read-only concurrency, independently cancelled - shared-build waiters, editable two-turn growth and cross-trial isolation, - persistence, expiry/deletion/purge, capacity, restart, cancellation, - unsafe-URL/egress rejection, and every failure mapping. - - GitHub Actions is the executable test host. Its driver starts the built image - locally, waits for HarnessRouter readiness, checks out the pinned AI Evals - commit, installs the lockfile, and invokes AI Evals' Promptfoo package script - against the local UHP endpoint. The pull-request lane uses a deterministic - local provider fixture; the protected lane uses the real Codex and Pi profiles. - Both lanes exercise the same Promptfoo provider and scenario definitions. -- **Verification:** Codex and Pi use native OAuth without a provider-route key. - Concurrent sessions using the same profile share one read-only generation - while conversation, home, log, and output state remain isolated. A - forced-expired token produces one coordinated provider refresh and two - successful turns. Cancellation after provider rotation does not release - renewal ownership or permit a second refresh; terminal acknowledgement waits - for commit or a durable repair fence. Same-session overlap returns - `session_busy`; idempotent duplicates share one admission and result; - configured profile saturation returns the cataloged capacity error. A damaged - profile becomes unavailable while healthy bindings remain advertised and - global readiness stays true. Editable turn two sees turn one's mutation; a - different trial sees a clean private writable view. - - Real-image lifecycle probes cover terminal-time TTL, retained-expiry HTTP 410, - purged-predecessor stock failure, HTTP 409 non-resumable, authorized - persistence before acquisition, operator deletion, provisional-pin protection, - hard private byte/inode enforcement across continuations, deterministic - generation eviction, bounded tombstones, deletion quarantine, and restart - reconciliation. Read-only write/input probes cannot copy up. Success, failure, - cancellation, and crash probes find no credential projection after terminal - acknowledgement. All generation, attachment, lifecycle, materializer, provider, - cancellation, and crash failures reconcile before the next profile admission. - Promptfoo returns exact coded errors and metadata, never empty success or - automatic retry. - - The Promptfoo process must exit successfully and its assertions must identify - the expected output, continuation, provenance, artifacts, and coded failures. - The job also checks HarnessRouter logs and the temporary roots for leaked - processes, mounts, credentials, and retained test state. A direct UHP script is - useful for diagnosis but does not satisfy this consumer E2E. - -### U6. Release, operations, review, and upstream preparation - -- **Goal:** Produce a reproducible, registry-published supported image and - upstream-ready generic hook and auth-state proposals. -- **Repositories/files:** GHCR image build/release workflow, dependency lock and - provenance record, fork-maintenance guide, deployment/reference docs, - changelog, PR descriptions, and upstream patch series. -- **Approach:** Build from exact upstream/fork/AllAgents/agent inputs. Pin base - images, lockfiles, OS packages, Git/OCI tools, Codex, and Pi. U6 uses the - identities frozen by current U0 evidence; changing one stops release and - reruns U0. Replace inherited Docker Hub publication with a no-write build/test - job, a protected native-auth test job, a protected candidate-publish job, and a - final promotion job, all using commit-pinned actions. The no-write job runs the - PR-safe Promptfoo suite against the exported image artifact. The native-auth - job runs only from an approved release/tag ref and accepts only the artifact - provenance bound to that exact ref; it never runs pull-request code. Push the - `linux/amd64` candidate without discovery tags, read back the manifest, and - attach verified build-provenance and SBOM attestations. A fresh protected job - anonymously pulls that digest, runs the PR-safe and real native-profile - Promptfoo suites against the registry bytes, and creates the green-E2E - attestation. Only then may promotion apply version/commit tags or complete the - release. - - Document durable generation/session/private/auth volumes; caller-supplied Git - URLs in the JSON descriptor versus the optional operator-owned - `workspace.yaml` snapshot catalog; public-egress and source-credential scope - policy; native login/repair and active-turn projection teardown; proxy mode; - terminal-time TTL, hard private quotas, provisional pins, watermark, - persistence authorization, deletion, bounded tombstone compaction, quarantine, - deterministic GC, capacity, metrics, backup, upgrade, and rollback procedures; - and the owner-trust boundary. Review both repositories before - final green E2E and prepare generic generation/attachment/lifecycle and - auth-state patches for upstream. -- **Verification:** A clean `linux/amd64` GitHub Actions job verifies the - candidate's build-provenance and SBOM attestations and pinned inputs, - anonymously pulls it by digest, starts HarnessRouter locally, waits for - readiness, and runs the pinned AI Evals Promptfoo package script. It configures - finite lifecycle policy and reproduces Git/OCI generation reuse, - cross-harness/profile read-only concurrency, editable isolation, continuation, - persistence, expiry/deletion, capacity pressure, GC, cancellation, restart, - native auth, proxy, and every documented failure. Credential-bearing jobs run - only for the approved ref. If standard hosted runners cannot provide the - required mount or cgroup behavior, each such job receives a dedicated - ephemeral one-job self-hosted runner that is destroyed after credential - teardown and never executes untrusted work. - - The successful job creates a signed green-E2E attestation for the candidate - digest before promotion. No version/commit discovery tag or completed release - exists before that attestation verifies. No Docker Hub credential is required. - Wrong provenance, build input, lifecycle configuration, unverified generation - store, Promptfoo assertion, leak check, process exit, or test-attestation - identity prevents promotion and deployment. Rebase rehearsal reports - incompatibility before release. - ---- - -## Verification Contract - -| Gate | Required evidence | -|---|---| -| Native-auth feasibility | Before workspace work, the minimal image proves real Codex and Pi login, continuation, binding persistence, profile isolation, concurrent same-profile turns with valid and forced-expired tokens, exactly one coordinated provider refresh, renewal ownership that survives turn cancellation/deadline, durable maintenance recovery, per-profile failure isolation, active-turn projection teardown on success/cancel/crash, and passive exclusion from checkpoints/backups/logs. Recorded pinned inputs invalidate the gate when changed. | -| Stock compatibility | Every upstream pin records a source-backed check for the workspace-hook and native-auth-profile seams. Any equivalent upstream seam replaces its downstream patch in the same upgrade. Upstream HarnessRouter tests and UHP conformance pass; requests without the metadata key are unchanged. | -| Caller authentication | Every external create, continuation, retrieval, stream, cancellation, file, artifact, and lifecycle administration path authenticates before existence or metadata disclosure. | -| Generation ordering | Generic session/tombstone admission precedes response visibility; secret-free preflight/validate and selected-reference verification plus access-specific authorization/reservation precede resolve. A miss reserves staging/prospective generation before acquisition; containment, full-tree accounting, commit-tree/Git/manifest verification, and atomic accounting conversion precede ready state/pins. Runner prepare plus gateway ready-ack precede provider dispatch; fallback never reenters. | -| Shared-build cancellation | One request cancellation/deadline detaches only that waiter. A build continues for remaining live waiters, stops when none remain or its runner-owned deadline expires, and produces at most one publication/failure for its epoch. | -| Manifest integrity | Git and OCI share one source-visible schema with pairwise non-overlapping repository destinations. A root may omit `.git` only when its manifest item declares history and the runner validates the detached commit, exact index/tree and object set, exact source-visible worktree, closed configuration and refs, and safe administrative state. Tree-only and undeclared `.git` fail. Git-acquired content equals the union of resolved commit trees at their destinations plus necessary ancestors. Plan/key drift, undeclared paths, forged manifests, changed staging, invalid paths/types/links/destinations, semantic Git mismatch, and digest mismatch fail before publication. | -| Shared read-only generation | Concurrent sessions using the same or different harness profiles share one exact generation epoch. Root, nested, symlink, and alternate-path writes fail; runtime/session/auth/output state remains isolated. | -| Editable isolation | Every fitting editable trial receives a private writable view with no mutable state shared with the generation or another session, plus a reserved hard byte/inode allowance covering overlays, checkpoints, and produced state. A non-fitting waiter fails alone; continuation preserves a fitting trial's mutations but cannot grow past its envelope. | -| Produced-file integrity | A clean first turn creates no redundant full-workspace inventory. Candidate tracking is durably active before input overlays or writable process exposure and remains active through quiescence; a missing or discontinuous coverage marker forces the bounded no-follow full scan. Candidate and full-scan paths produce the same source-visible additions, deletions, type/mode changes, and content changes. Both exclude only the declared Git administrative subtrees recorded by the protected generation, report an agent-created `.git` elsewhere as ordinary content, and ignore final Git discovery, ignore rules, and agent-supplied path lists. Continuation derives its delta from protected prior turn state. | -| Materializer containment | Fork/double-fork/cancellation/deadline fixtures prove `populated 0` before result read, publication, secret release, or cleanup. `containment_pending` blocks terminal visibility/readiness through restart and resolves once after quiescence. | -| Capacity envelope | Every native profile has a finite concurrent-turn limit of at least two; one is invalid, saturation fails before response allocation, and admitted same-profile turns proceed concurrently. The renewal transaction has a separate finite timeout. Source build limits and finite staging/generation/private-byte/private-inode/session/persistence/tombstone quotas reject overflow. Invalid descriptors cannot bypass generic admission; one editable session creates one private debit; successful publication releases staging capacity. References and provisional pins prevent eviction; all-protected capacity returns the cataloged retryable failure. | -| Durable lifecycle | Fault injection covers generic and provisional turn admission, native profile turn slots, renewal and maintenance records, active leases, generation epochs, build/staging/generation reservations, publication/accounting conversion, build waiters, provisional pins, attachment prepare/ready-ack and private-reservation transfer, references, mounts, private usage, expiry, tombstones/purge, unmount, deletion, quarantine, and GC. Startup reconciles before readiness; no deadline releases an active renewal or maintenance fence, no debit duplicates or leaks, no second epoch appears before prior eviction completes, and no session silently rematerializes. | -| Retention and disposal | Fake-clock evidence proves one session CAS rejects busy/expired continuation admission, provisionally saves/clears a valid deadline, and either commits active after profile admission or restores the exact future deadline/tombstones an elapsed one after pre-allocation profile failure. Terminal acknowledgement alone sets the next `expiresAt`; polls/replays do not renew. Invalid failed responses stay accounted through purge; retained expiry returns HTTP 410; purge returns stock unknown; persistence authorizes before source access; operator deletion is idempotent. Null `lastUsedAt` epochs evict first by `publishedAt`; used epochs order by `lastUsedAt`, then `publishedAt`, generation key, and epoch. | -| Session continuity | Both modes preserve conversation and fixed generation key/epoch/access/retention/cwd/harness/auth binding while persistent or unexpired; editable preserves private files; read-only remains immutable. Corrupt known evidence returns HTTP 409 non-resumable with no source access or later-epoch substitution. | -| Git acquisition | Caller-supplied canonical HTTPS URLs, public-address egress enforcement, DNS-rebinding and redirect defense, structured-scope credential isolation, constrained refs, exact commits, closed transport/config, exact object closure/index semantics, generation reuse, and partial cleanup pass against local network fixtures. | -| OCI acquisition | Digest/media/path/link/type/limit checks, tree-only and normalized offline-history fixtures, producer removal and materializer rejection of remotes and credentials, semantic Git verification, generation reuse, and the attachment matrix pass against a local registry. | -| Credential boundary | Preflight sees no secret values and returns bounded configured reference identities; validate selects a bounded subset; the runner verifies handles and injects only that selected set into source-access children. Source secrets and caller keys are absent from staging, generations, private views, base environments, checkpoints, backups, logs, and output. The selected OAuth profile is visible only through each active turn's projection, which is absent before acknowledgement and after that turn's teardown. Concurrent turns may share the profile; the active harness and same-identity tools remain an explicit owner-trust boundary. | -| Provider boundary | Codex/Pi native OAuth, coordinated same-profile renewal, renewal ownership across turn cancellation/deadline, durable maintenance, profile-local repair/readiness, projection teardown, idempotency and same-session precedence, finite profile capacity of at least two, same-profile concurrency, and explicit proxy scope all pass without implicit switching. | -| Packaging | The no-write GitHub Actions job starts the exported image and passes the credential-free Promptfoo suite. Credential-bearing jobs accept only an artifact bound to the approved release ref and never run pull-request code. The protected workflow pushes an untagged GHCR candidate, verifies build-provenance/SBOM attestations, anonymously pulls the digest into a fresh job, and passes both Promptfoo suites. That job creates the signed green-E2E attestation before version/commit tags or release completion. Deployment requires all three attestations. A hosted-runner limitation selects a dedicated ephemeral one-job self-hosted runner that is destroyed after credential teardown, never a weaker check or reused host. | -| Consumer | The Promptfoo version from AI Evals' lockfile runs through that repository's package script against the local HarnessRouter endpoint. Its process exits successfully for concurrent, one-shot, two-turn, and lifecycle success scenarios, and its assertions prove every cataloged or UHP terminal failure maps exactly. Active streams expose null expiry; terminal/GET/replay expose one stable expiry. Failures before attachment ready omit workspace metadata; later terminal failures include the complete public object. None becomes empty success or automatic retry. A direct HTTP smoke test cannot substitute for this gate. | -| Review | Final review findings in both repositories are resolved before final built-image E2E. | - -## Definition of Done - -- ADR 0002, this plan, implementation, generated schemas, configuration docs, - topology, and request examples agree on canonical `url` vocabulary; local - `workspace.yaml` uses `path` plus optional `url` with no ordinary - `source`/`repo` compatibility fields; the UHP descriptor uses `url`, optional - `ref`, and `destination`; snapshot requests use `snapshotName`, - `imageManifestDigest`, and `workspaceManifestDigest`; snapshot repository roots - may be tree-only or carry declared, normalized offline Git history without a - configured Git remote; runtime environment and benchmark task identity remain - separate; - and immutable generations, read-only and editable attachments, bounded - retention, native OAuth, explicit proxy mode, and GHCR digest-pinned - distribution remain consistent. -- The U0 evidence predates U1-U6 and both native targets pass on the recorded - inputs; changed inputs have replacement evidence before dependent work resumes. -- No second execution protocol/control plane, separate AllAgents gateway, direct - provider adapter, custom OAuth broker, automatic fallback, or client-side - source expansion remains. -- R1-R16 and AE1-AE14 pass against the exact released image; stock UHP requests - and conformance remain green. -- One canonical resolved source plan produces at most one live verified immutable - publication per generation epoch and one result per concurrent claim. A later - epoch begins only after prior logical and physical eviction completes. Build - waiters retain independent deadlines and cancellation. Access, retention, cwd, - harness/profile, and session identity do not fragment the key. -- Concurrent read-only sessions with different harness/profile bindings share - generation bytes but no mutable runtime, auth, conversation, output, or - lifecycle state. Filesystem probes prove no writable path or copy-up. -- Each fitting editable trial has a private writable view with no mutable state - shared with the generation or another session. Its reserved hard byte/inode - allowance covers every turn, overlay, checkpoint, and produced-file record. - Non-fitting waiters fail independently. Collection uses the verified generation - plus protected sparse turn state, produces the same delta through candidate and - full-scan paths, and never trusts editable Git metadata. Continuation preserves - only its own mutations and produced-file history. -- Generation publication and every validate/resolve/materialize result remain - behind cgroup quiescence, exact commit-tree/source-visible manifest and semantic - Git validation, full physical accounting, atomic reservation conversion, - provisional pins, and attachment prepare/ready-ack evidence before provider - dispatch. Successful publication releases staging reservation before ready. -- Finite TTL and quotas cover builds/staging, generations, per-session hard - private bytes/inodes, total private reservations, sessions, persistence, and - tombstones. Generic session/tombstone admission precedes response visibility - and covers invalid failed responses through purge. One stable editable - reservation transfers to ready state without a second debit. One session CAS - rejects busy/expired continuation admission and provisionally saves/clears its - valid deadline; profile success commits active, while pre-allocation failure - restores the exact future deadline or tombstones an elapsed one. Terminal - acknowledgement sets the next expiry; polls/replays do not. Persistent - retention requires authorization and reservation before source access. -- Expiry and deletion tombstone first, fence work, quiesce projections/mounts, - delete private state, and release reservations/references exactly once. - Tombstones compact only after response/idempotency retention. GC evicts only - ready zero-reference/zero-pin epochs: null `lastUsedAt` first by `publishedAt`, - then non-null `lastUsedAt`, `publishedAt`, generation key, and epoch ID. - Failures stay quarantined/accounted, block same-key republication, and - protected-capacity exhaustion rejects admission. -- Restart reconciles generic admission, active leases, build waiters, staging/ - prospective-generation reservations, publication/accounting, provisional pins, - prepare/ready-ack and private-reservation transfer, references, mounts, private - usage/views, `containment_pending`, credential projections, native profile turn - slots, renewal and maintenance records, tombstones/purge, and deletion before - readiness or GC. Existing sessions never silently reacquire source or change - binding. -- Continuation omits the extension and preserves exact generation key/epoch, - access, retention, cwd, harness, auth, and conversation while persistent or - unexpired. Read-only remains immutable; editable retains private files; - retained expiry/deletion returns HTTP 410, corrupt known evidence returns HTTP - 409 non-resumable, and purged identity returns stock unknown, all without source - access, rematerialization, or later-epoch substitution. -- Native auth retains atomic idempotency and same-session precedence, a finite - concurrent-turn limit of at least two per profile, concurrent same-profile - execution, one renewal transaction at a time whose ownership survives turn - cancellation and deadline, durable maintenance fencing, profile-local repair - and readiness, active-turn-only credential projection with verified teardown - before terminal acknowledgement, and no implicit profile or proxy switching. -- Preflight receives no secret values; validate returns a bounded selected - credential-reference set, and the runner injects only that exact set into - source-access hook children. Source credentials never enter staging, - generations, private session state, or harnesses. OAuth is visible only within - the accepted active-turn selected-profile owner-trust boundary and is absent - from retained homes, checkpoints, backups, logs, and mounts after teardown; - proxy credentials remain scoped and brokered. -- The public `linux/amd64` image, attestations, pinned inputs, lifecycle - configuration, patch series, materializer contract, operational deletion/GC - docs, and rollback procedure reproduce from pinned inputs. -- Generic generation/attachment/lifecycle and auth-state patches are ready for - upstream proposal; the maintained fork remains operable if not accepted. +The real file must also carry HarnessRouter's required authentication, secret, +health, and process settings. The operator supplies one external provider base +URL and API key through two protocol-specific HarnessRouter connection records; +the Compose file does not bake either value into the image. + +Startup contract: + +1. Copy the example environment file and set non-default Console/caller + credentials, the external provider route, model allowlists, TTL/resource + limits, and an immutable image version plus manifest digest. +2. Run `docker compose up -d`. +3. Readiness succeeds only after the gateway, runner, enabled Codex/OMP runtimes, + materializer executable, writable `/data`, custom harnesses, both + protocol-specific provider connections, and loopback credential broker pass + startup checks. Owner-trust credential pass-through fails readiness. +4. The public UHP base remains HarnessRouter's existing + `http://127.0.0.1:3000/api/harness`; remote access requires operator-owned TLS + and network controls. +5. Restarting the container with the same volume preserves unexpired + HarnessRouter sessions and their private checkouts. Startup reconciliation + removes only abandoned staging or cleanup-marked roots. + +Container requirements: + +- the materializer is installed in the same image and invoked locally; +- Git and certificate roots are pinned and present; +- the service binds to loopback by default; +- `/data` is the only required durable mount; +- secrets are runtime inputs, not layers, labels, build args, or example values; +- the image has a standard SBOM and build provenance; and +- image labels record gateway version, source revision, upstream commit, and UHP + version. + +### Phase 8: Add direct Promptfoo smoke and E2E coverage + +**Outcome:** the lockfile-pinned Promptfoo installation calls the built image +directly over UHP and proves user-visible workspace behavior. + +Suggested area: `e2e/promptfoo/` in `allagentsdev/allagents-gateway`, containing +only Promptfoo config, small fixtures/assertions, and package metadata. + +Work: + +1. Configure Promptfoo's OpenAI Responses-compatible provider directly against + `/api/harness/v1/responses`, with the HarnessRouter API key in an environment + variable and nested workspace metadata in the request. Do not place another + HTTP adapter or repository between Promptfoo and the gateway. +2. Use a stable public Git fixture with known commits and a task whose answer or + file change can only succeed if the harness started in the materialized + checkout. +3. Run the same first-turn scenario for `allagents-codex` and `allagents-omp`, + with each harness's allowed model and configured provider transport. +4. Include a two-turn scenario that mutates a uniquely named file on turn one + and reads/changes it on turn two via `previous_response_id` without resending + workspace metadata. +5. Include negative cases for malformed metadata, forbidden URL resolution, + missing ref, invalid working directory, materialization limit, provider + failure, cancellation during Git, cancellation during harness execution, and + continuation after cleanup. + Include reused-session workspace injection and cross-harness continuation + mismatch cases. +6. Inspect public response metadata to verify the expected resolved commit and + absence of checkout paths and credentials. + Use canary values for the external-gateway key and scoped broker token and + assert both are absent from session files, checkpoints, artifacts, logs, and + reports after each turn. +7. Exercise the built image, not an in-process gateway. Store only sanitized + reports; do not upload provider traffic, prompts containing secrets, or data + volume contents. + +Direct smoke/E2E is the proof of the feature. Focused permanent tests remain only +where they protect schema boundaries, ordering, security invariants, session +transitions, and cleanup races. Do not add tests that merely assert config keys, +field copies, mocks, or source text. + +Release-blocking E2E matrix: + +| Harness | First turn | Continuation | Cancellation | Provider failure | +|---|---:|---:|---:|---:| +| Codex / Responses | required | required | required | required | +| OMP / Chat Completions | required | required | required | required | + +### Phase 9: Conformance, upstream maintenance, and release + +**Outcome:** the downstream preserves stock HarnessRouter behavior and publishes +a reproducible, independently versioned image. + +Work: + +1. Run the complete UHP `2026-09-12` conformance suite against the built image. + Workspace tests supplement it; they do not replace or relax upstream cases. +2. Run upstream HarnessRouter integration coverage for gateway, runner, Codex, + OMP, sessions, streaming, cancellation, idempotency, files, artifacts, and + non-workspace requests. +3. Run the direct Promptfoo E2E matrix against the exact image candidate. +4. Review the downstream diff against + `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`. Keep the seam patch separable + from the AllAgents schema/materializer implementation so generally useful + changes can be proposed upstream without blocking release. +5. Build and publish the v1 `linux/amd64` image as + `ghcr.io/allagentsdev/allagents-gateway:`, attach standard + SBOM/provenance, and record its manifest digest. Additional architectures are + separate release work after v1. +6. Verify a clean Compose deployment using that digest, a fresh volume, both + harnesses, a first turn, a continuation, cancellation, expiry, restart, and + cleanup. +7. Publish release notes containing gateway version, source commit, upstream + baseline, UHP version, pinned Codex/OMP versions, Promptfoo version, image + digest, known limitations, and upgrade/rollback instructions. +8. For future upstream updates, first merge/rebase the new upstream baseline as + its own change, rerun conformance and built-image E2E, then replay or revise + the narrow downstream patches. Never mix an upstream baseline jump with a + product behavior change. + +Release gate: + +- all required focused tests pass; +- UHP conformance passes without exclusions introduced by this work; +- built-image Codex and OMP first-turn/continuation E2E passes through the + external provider gateway; +- security failure and cancellation cases leave no child process or staging + directory; +- the external-gateway key is absent from harness environments, and both it and + scoped broker tokens are absent from persisted or public session surfaces, + while both protocol-specific broker routes succeed; +- expiry and cleanup make the checkout unavailable and remove it idempotently; +- ordinary non-workspace HarnessRouter requests remain compatible; +- image digest, SBOM, provenance, pins, and downstream diff are available; and +- the Compose smoke succeeds from a fresh checkout and fresh `/data` volume. + +## Error contract + +Use stable, stage-oriented detail codes under HarnessRouter's existing UHP error +shape. Final names should follow upstream conventions, but the observable +categories are fixed: + +| Condition | HTTP class | Retry guidance | +|---|---:|---| +| Invalid workspace JSON or path | 400 | Caller must change request | +| Workspace supplied for any reused session | 409 | Caller must omit workspace | +| URL/ref/source feature rejected | 400 | Caller must change source | +| Forbidden DNS/redirect destination | 400 | Caller or operator must change source/network policy | +| Materialization exceeds a fixed source limit | 413 | Caller must choose a smaller repository | +| Materializer concurrency unavailable | 503 | Retry with backoff within caller deadline | +| Git/network timeout before binding | 504 | Retry creates a new first-turn attempt under normal idempotency rules | +| Materialization cancelled | Existing UHP cancelled outcome | Do not retry under the cancelled response ID | +| Bound checkout missing, expired, or cleanup-started | 410 | Start a new session with a new workspace request | +| Provider unavailable or rejects credentials | Existing upstream provider error | Repair external provider gateway; no route fallback | + +A failure before publication exposes no checkout identity or provenance. A +failure after a binding exists may include the already committed public +provenance, but never internal paths, Git stderr, DNS details that disclose +private topology, or provider secrets. + +## Configuration ownership + +| Value | Owner | Caller-overridable? | +|---|---|---:| +| Repository URL, optional ref, optional working directory | Initial UHP request | yes, within strict schema/policy | +| Harness ID and allowed model | New-session request constrained by server definition; stored workspace-bound session on reuse | only among configured values on creation; mismatch rejected only for workspace-bound reuse | +| External provider base URL/API key | Operator secret configuration | no | +| Codex/OMP provider transport | Operator harness definition | no | +| Materializer executable | Image/operator startup configuration | no | +| DNS/redirect/Git restrictions | Image and operator policy | no weakening by caller | +| Byte/inode/time/concurrency limits | Operator within image-safe bounds | no | +| Idle checkout TTL | Operator within image-safe bounds | no | +| Checkout path/identity | Runner | no | +| Resolved commit | Materializer observation | no | + +## Delivery sequence and ownership + +A practical implementation sequence inside `allagentsdev/allagents-gateway` is: + +1. Repository maintainer performs the fork rename and establishes release/image + permissions and pins. +2. Gateway owner implements the exact nested schema, canonical binding input, + continuation rejection, and public error mapping. +3. Runner owner implements the single seam, first-turn/continuation ordering, and + cancellation propagation. +4. Materializer owner implements secure Git acquisition, validation, + publication, and cleanup under the runner contract. +5. Harness owner pins Codex/OMP, configures the two protocol adapters to the + single external provider route, and defines the custom harnesses. +6. Container owner integrates the materializer, `/data` layout, readiness, + Compose contract, and standard supply-chain outputs. +7. E2E owner adds the lockfile-pinned direct Promptfoo matrix and built-image + smoke. +8. Release owner runs upstream conformance, reviews the downstream diff, and + publishes the independently versioned digest. + +Schema and seam contracts must merge before materializer and E2E work depend on +them. Git security, provider wiring, and container work can proceed in parallel +once those contracts are frozen. No implementation phase requires a change in +the AllAgents CLI repository. + +## Definition of done + +V1 is done when an operator can deploy one digest-pinned +`ghcr.io/allagentsdev/allagents-gateway` container with Docker Compose, configure +one external OAuth-to-OpenAI-compatible provider route through two +protocol-specific HarnessRouter connections and two custom harnesses, and have +lockfile-pinned Promptfoo directly: + +1. start a Codex or OMP UHP session with the exact + `metadata.allagents.workspace` object; +2. observe a securely resolved exact Git commit in public provenance; +3. run the harness inside a private editable checkout at the requested safe + directory; +4. continue through either supported session-reference path with the same + stored harness, mutations, and exact checkout; +5. cancel Git or harness work without leaked descendants or staging; +6. keep the long-lived external-gateway key out of harness environments and + persisted session data while both brokered protocol routes succeed; +7. receive bounded, stable failures without provider calls for invalid or unsafe + workspaces; +8. lose access after the server-owned expiry and see deterministic idempotent + cleanup; and +9. run stock non-workspace UHP requests with upstream-compatible behavior. + +No gateway implementation code, CLI command, profile migration, or +`workspace.yaml` change is required in `allagentsdev/allagents` to satisfy this +definition. \ No newline at end of file From 8bad2b796687107cbef7459863c0eb1c8f7293d8 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sun, 27 Sep 2026 20:22:55 +1000 Subject: [PATCH 30/44] docs(architecture): keep HarnessRouter upstream-first --- .../0002-adopt-uhp-through-harnessrouter.md | 164 ++- ...0837-feat-coding-execution-gateway-plan.md | 1312 +++++++++-------- 2 files changed, 804 insertions(+), 672 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index c2772495..9e586116 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -1,4 +1,4 @@ -# ADR 0002: Build AllAgents Gateway as a narrow HarnessRouter downstream +# ADR 0002: Add workspace materialization to HarnessRouter - Status: Accepted - Date: 2026-09-21 @@ -10,50 +10,57 @@ Promptfoo needs a remote coding-harness endpoint that can prepare a repository b The examined baseline is UHP [`2026-09-12`](https://github.com/HarnessRouter/harnessrouter/tree/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/versions/2026-09-12) at HarnessRouter commit [`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), released as [`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4). -UHP already reserves `metadata` for additive extensions. At the pinned commit, the request schema accepts an open metadata object ([UHP schema](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/schema/uhp-2026-09-12.openapi.yaml#L1051-L1057)). That is an existing protocol extension point, not an extension framework: stock HarnessRouter gives arbitrary metadata no runner semantics. It extracts only nested `metadata.systemone` ([gateway extraction](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/gateway/app.py#L7571-L7572)) and forwards only that probe to the runner ([runner handoff](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/gateway/app.py#L7008-L7011)). Workspace semantics therefore require a downstream change. -The AllAgents key follows the same nested namespace convention as the existing `metadata.systemone.script`: `metadata.allagents.workspace`. The shared convention is the metadata shape, not System One's special forwarding behavior. +UHP already reserves `metadata` for additive extensions. At the pinned commit, the request schema accepts an open metadata object ([UHP schema](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/schema/uhp-2026-09-12.openapi.yaml#L1051-L1057)). That is an existing protocol extension point, not an extension framework: stock HarnessRouter gives arbitrary metadata no runner semantics. It extracts only nested `metadata.systemone` ([gateway extraction](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/gateway/app.py#L7571-L7572)) and forwards only that probe to the runner ([runner handoff](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/gateway/app.py#L7008-L7011)). Workspace semantics therefore require coordinated protocol and implementation changes. + +The open metadata object permits implementation-specific extensions, but it does not make their semantics part of UHP. `metadata.workspace` becomes a standard UHP contract only if accepted through upstream governance. Until then, the fork may expose it only as a documented HarnessRouter extension. The protocol or fork release already owns schema evolution, so the workspace object must not introduce a nested version field. ## Decision -We will ship **AllAgents Gateway** as a rebase-friendly downstream of HarnessRouter. The existing GitHub fork `allagentsdev/harnessrouter` will be renamed to [`allagentsdev/allagents-gateway`](https://github.com/allagentsdev/allagents-gateway), preserving its fork relationship and history. We will not create a third repository that consumes the fork. +We will keep [`allagentsdev/harnessrouter`](https://github.com/allagentsdev/harnessrouter) as the existing GitHub fork of [`HarnessRouter/harnessrouter`](https://github.com/HarnessRouter/harnessrouter), preserving its name, fork relationship, and history. We will not create a replacement repository, rename the fork, or add a new repository or CLI integration. + +Workspace materialization is a generic HarnessRouter capability and will be proposed upstream first. Before substantial implementation, we will open an upstream issue describing the use case, security model, request shape, lifecycle semantics, and conformance expectations. We will then follow HarnessRouter and UHP governance, including a UHP Enhancement Proposal (UEP) when required for protocol semantics. An accepted contribution must update the UHP specification, versioned schema, reference implementation, conformance coverage, changelog, and user/operator documentation together. If the issue receives no maintainer decision within 30 calendar days after opening and a follow-up is posted after day 14, the project records the proposal as deferred and may use the documented downstream-extension path. -The downstream adds one product feature: on a first turn, recognize the exact `metadata.allagents.workspace` object, validate and materialize one repository into a private session checkout, bind it immutably to the session, and start the selected harness there. A continuation reuses that exact checkout. +The proposed first-turn contract is `metadata.workspace`. It validates and materializes one repository into a private session checkout, binds that checkout immutably to the session, and starts the selected harness in the requested directory. A continuation reuses that exact checkout. We will seek upstream ownership first. If upstream rejects or defers the contract, the fork may carry the same bounded behavior as an explicitly documented downstream HarnessRouter extension; it must not describe that extension as standard UHP behavior. If a later UHP release reserves an incompatible `metadata.workspace`, the fork must migrate cleanly rather than preserve conflicting aliases. -The supported v1 harnesses are **Codex** and **OMP**. Provider traffic goes only through an existing, separately operated OAuth-to-OpenAI-compatible gateway. Promptfoo calls AllAgents Gateway directly over UHP. +The fork remains the AllAgents distribution point. Generic protocol, lifecycle, session-binding, and materializer seams are proposed upstream first and live in the fork only when upstream declines or defers them. Deployment defaults, custom harness definitions, provider wiring, Promptfoo scenarios, and publication of the downstream image remain AllAgents-owned in either path. The supported distribution harnesses are **Codex** and **OMP**. Provider traffic goes only through an existing, separately operated OAuth-to-OpenAI-compatible gateway, and Promptfoo calls HarnessRouter directly over UHP. -The downstream is not a general workspace platform. It adds no extension registry, dynamically selected hook, arbitrary materializer command, or second protocol. It invokes exactly one operator-configured materializer that is included in the AllAgents Gateway image. +This is not a general workspace platform. The proposal adds no extension registry, dynamically selected hook, arbitrary materializer command, or second protocol. It invokes exactly one operator-configured Git materializer included in the HarnessRouter image. ## System flow and ownership ```mermaid flowchart LR - P[Promptfoo] -->|UHP + caller API key| A[AllAgents Gateway] - A -->|fixed workspace seam| M[In-image Git materializer] + P[Promptfoo] -->|UHP + caller API key| H[HarnessRouter] + H -->|generic workspace seam| M[In-image Git materializer] M -->|anonymous HTTPS| G[Public Git repository] - A --> C[(Private session checkout)] - C --> H[Codex or OMP] - H -->|short-lived turn credential| B[HarnessRouter loopback broker] + H --> S[(Private session state)] + H --> C[(Private session checkout)] + C --> X[Codex or OMP] + X -->|short-lived turn credential| B[HarnessRouter loopback broker] B -->|Responses or Chat Completions| O[OAuth-to-OpenAI-compatible gateway] O --> V[Model provider] - A --> D[(/data sessions and state)] + H --> D[(/data sessions and state)] ``` | Component | Owns | |---|---| -| Promptfoo | Prompt, model, `metadata.harness_id`, first-turn workspace request, continuation ID, and evaluation assertions | -| HarnessRouter gateway and runner | Caller authentication, UHP validation and conformance, idempotency, session hydration, streaming, cancellation, harness execution, files, artifacts, and lifecycle state | -| AllAgents workspace seam | Exact metadata recognition, first-turn binding, continuation lookup, ordering before harness execution, and normalized workspace failures | -| Fixed materializer | URL and ref validation, safe Git resolution and acquisition, exact commit provenance, checkout validation, resource enforcement, cancellation, and cleanup | +| Promptfoo | Prompt, model, `metadata.harness_id`, first-turn `metadata.workspace` request, continuation ID, and evaluation assertions | +| UHP specification and governance | When accepted upstream: reservation and versioned meaning of `metadata.workspace`, its schema, lifecycle semantics, error contract, and conformance requirements | +| `allagentsdev/harnessrouter` release | When downstream-only: extension shape and revision, implementation lifecycle, errors, extension tests, changelog, and docs; never UHP conformance ownership | +| HarnessRouter gateway and runner | Caller authentication, UHP validation and conformance, idempotency, session hydration, workspace binding, streaming, cancellation, harness execution, files, artifacts, and lifecycle state | +| Generic workspace seam | Exact metadata recognition, first-turn binding, continuation lookup, ordering before harness execution, and normalized workspace failures | +| Fixed Git materializer | URL and ref validation, safe Git resolution and acquisition, exact commit provenance, checkout validation, resource enforcement, cancellation, and cleanup | | HarnessRouter custom harness definition | Reusable remote harness configuration: Codex or OMP base harness, model defaults, instructions, tools, skills, and server-owned provider route | | HarnessRouter loopback broker | Per-turn scoped credential minting and exchange; the harness process never receives the long-lived external-gateway API key | | External OAuth gateway | Provider login, OAuth token storage, refresh, repair, provider API compatibility, and provider authorization | +| AllAgents distribution configuration | Deployment defaults, Codex and OMP custom harness definitions, provider wiring, Promptfoo scenarios, and image publication | | Operator | Caller credentials, custom harnesses, provider endpoint and API key, egress policy, limits, TTL, deployment, upgrades, and deletion policy | -The materializer never owns UHP sessions or provider credentials. The external OAuth gateway never owns source acquisition or UHP session state. Promptfoo never receives source or provider credentials. +The materializer never owns UHP sessions or provider credentials. The external OAuth gateway never owns source acquisition or UHP session state. Promptfoo never receives source or provider credentials. Distribution configuration must not redefine the generic request or lifecycle contract. ## Request contract -A workspace-backed first turn uses the normal UHP `POST /v1/responses` request. The exact v1 extension shape is: +A workspace-backed first turn uses the normal UHP `POST /v1/responses` request. The proposed shape is: ```json { @@ -61,73 +68,78 @@ A workspace-backed first turn uses the normal UHP `POST /v1/responses` request. "input": "Implement the requested change.", "metadata": { "harness_id": "chrn_…", - "allagents": { - "workspace": { - "version": "1", - "repository": { - "url": "https://github.com/example/project.git", - "ref": "refs/heads/main" - }, - "workingDirectory": "packages/service" - } + "workspace": { + "repository": { + "url": "https://github.com/example/project.git", + "ref": "refs/heads/main" + }, + "working_directory": "packages/service" } } } ``` -`metadata.harness_id` is HarnessRouter's existing harness selector. HarnessRouter documents custom harnesses as reusable configurations with a fixed base harness and selects them through `metadata.harness_id` ([custom harness behavior](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/README.md#L155-L163), [UHP selection](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/README.md#L186-L203)). For this product, a custom harness is the remote equivalent of a reusable profile. It is not an AllAgents CLI profile and is not projected from a developer machine. +`metadata.harness_id` is HarnessRouter's existing harness selector. HarnessRouter documents custom harnesses as reusable configurations with a fixed base harness and selects them through `metadata.harness_id` ([custom harness behavior](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/README.md#L155-L163), [UHP selection](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/README.md#L186-L203)). For this distribution, a custom harness is the remote equivalent of a reusable profile. It is not an AllAgents CLI profile and is not projected from a developer machine. -`metadata.allagents.workspace` has exactly these fields: +`metadata.workspace` has exactly these fields: | Field | Required | Contract | |---|---:|---| -| `version` | yes | Exactly `"1"`. Other versions fail before source access. | | `repository.url` | yes | Anonymous public HTTPS Git URL. No embedded credentials, userinfo, query, fragment, alternate protocol, or local path. | | `repository.ref` | no | Advertised `refs/heads/*` or `refs/tags/*`, or unambiguous branch/tag shorthand. Omission means the remote default branch. Raw object IDs, other namespaces, ambiguous names, and unfetchable values fail. V1 accepts SHA-1 repositories only and records the resolved 40-hex commit. | -| `workingDirectory` | no | Relative POSIX directory beneath the checkout. Omission means the repository root. Absolute paths, empty components, `.`/`..` traversal, platform-specific separators, and any symlink escape fail. | +| `working_directory` | no | Relative POSIX directory beneath the checkout. Omission means the repository root. Absolute paths, empty components, `.`/`..` traversal, platform-specific separators, and any symlink escape fail. | -No other keys are accepted at any level of this object. The object cannot carry credentials, headers, environment variables, commands, destination paths, Docker settings, materializer selection, resource limits, retention, or provider configuration. Request size, string length, nesting depth, and parsing work are bounded before source access. +An accepted UHP protocol version governs the standard schema; otherwise the fork release governs the documented extension shape. The workspace object has no nested schema marker. No other keys are accepted at any level of the object. It cannot carry credentials, headers, environment variables, commands, destination paths, Docker settings, materializer selection, resource limits, retention, or provider configuration. Request size, string length, nesting depth, and parsing work are bounded before source access. -A first turn may omit `metadata.allagents.workspace`; stock HarnessRouter behavior then remains available. A session that starts without it cannot add it on a continuation. +A first turn may omit `metadata.workspace`; stock HarnessRouter behavior then remains available. A session that starts without it cannot add it on a continuation. -After successful binding, public response metadata records the normalized requested URL, optional requested ref, exact resolved commit, and effective working directory under `metadata.allagents.workspace`. Internal checkout IDs and host paths remain private. The resolved commit, not a mutable branch or tag, is the provenance authority for the session. +After successful binding, public response metadata records the normalized requested URL, optional requested ref, exact resolved commit, and requested `working_directory` when supplied under `metadata.workspace`. Omission means the checkout root and remains omitted in the response. Internal checkout IDs and host paths remain private. The resolved commit, not a mutable branch or tag, is the provenance authority for the session. ## First turn and continuation semantics For a workspace-backed first turn: 1. HarnessRouter authenticates the caller, validates the UHP request, establishes idempotency, and resolves whether the request creates or reuses a session. -2. Any reused session, whether selected by `previous_response_id` or `metadata.session_id`, rejects workspace metadata. If it already has an AllAgents workspace binding, its stored harness and binding win and a caller-supplied harness mismatch fails. An unbound session with no workspace metadata retains upstream routing. -3. For a new workspace-backed session, a workspace-aware hydrate creates an isolated empty session allocation but skips the stock empty Git initialization. It reserves bounded materialization capacity before provider or harness execution. -4. The materializer resolves the optional advertised ref to one exact commit, builds and validates a private checkout in staging, then atomically publishes it into the empty runner-designated checkout root. -5. The runner keeps its control root as a sibling of the checkout, never inside repository content. It derives the execution working directory as a validated descendant of the checkout. -6. HarnessRouter atomically binds the descriptor, resolved commit, checkout root, control root, execution working directory, cleanup deadline, and selected harness to the session. -7. The selected custom harness starts in the execution working directory, while its home, credentials, scratch, skills, and checkpoint control state remain anchored under the runner-owned control root. +2. Any reused session, whether selected by `previous_response_id` or `metadata.session_id`, rejects workspace metadata. If it already has a workspace binding, its stored harness and binding win and a caller-supplied harness mismatch fails. An unbound session with no workspace metadata retains ordinary routing. +3. For a new workspace-backed session, workspace-aware hydration creates an isolated empty session allocation but skips stock empty Git initialization. It reserves bounded materialization capacity before provider or harness execution. +4. The materializer resolves the optional advertised ref to one exact commit, builds and validates a private checkout in staging, and crash-safely publishes it into the empty runner-designated checkout root. Publication never exposes a partial checkout. +5. The runner keeps four distinct concepts and paths: the session root, the checkout root, the control root, and the execution root. The checkout and control roots are non-overlapping children of the runner-owned session allocation. The execution root is a validated directory within the checkout, never the control root. +6. HarnessRouter crash-safely binds the descriptor, workspace-contract owner and immutable revision, resolved commit, checkout root, control root, execution root, cleanup deadline, and selected harness to the session. The contract revision is the UHP release when standardized or the exact downstream source revision otherwise. A restart observes either the complete binding and published checkout or neither; recovery removes unattached staging. +7. The selected custom harness starts in the execution root, while its home, credentials, scratch, skills, and checkpoint control state remain anchored under the control root. -A continuation selects an existing session with `previous_response_id` or HarnessRouter's existing `metadata.session_id` recovery path and omits `metadata.allagents.workspace`. For a workspace-bound session, the gateway derives the harness from stored state; if the caller supplies a different harness, the request fails before hydration. The continuation reuses the exact private checkout, including edits from earlier turns, and the original resolved-commit provenance. Supplying workspace metadata on any reused session is invalid, even if byte-for-byte identical. The gateway never resolves the ref again, clones a replacement, changes the working directory, or silently starts a fresh session. +A continuation selects an existing session with `previous_response_id` or HarnessRouter's existing `metadata.session_id` recovery path and omits `metadata.workspace`. For a workspace-bound session, the gateway derives the harness from stored state; if the caller supplies a different harness, the request fails before hydration. The continuation reuses the exact private checkout, including edits from earlier turns, and the original resolved-commit provenance. Supplying workspace metadata on any reused session is invalid, even if byte-for-byte identical. The gateway never resolves the ref again, clones a replacement, changes the working directory, or silently starts a fresh session. -If the bound checkout is expired, missing, corrupt, or cannot be proven to belong to the predecessor, continuation fails closed. There is no rematerialization, source fallback, or checkout substitution. The bounded ephemeral TTL is operator-configured; ordinary completion does not immediately remove a checkout that remains eligible for continuation. Expiry and explicit deletion use the same minimal idempotent cleanup path. +If the bound checkout is expired, missing, corrupt, belongs to an unsupported workspace-contract revision, or cannot be proven to belong to the predecessor, continuation fails closed. Before activating an incompatible contract, an upgrade must explicitly migrate compatible bindings or drain and delete them; it must not retain conflicting aliases. There is no rematerialization, source fallback, or checkout substitution. The bounded ephemeral TTL is operator-configured; ordinary completion does not immediately remove a checkout that remains eligible for continuation. Expiry and explicit deletion use the same minimal idempotent cleanup path. -## Materializer and downstream boundary +## Generic upstream seam and distribution boundary -The only maintained HarnessRouter seam is a workspace validate/materialize call **after session resolution and the workspace-aware hydration step, but before provider or harness execution**. It has two paths: +The proposed generic HarnessRouter seam is a workspace validate/materialize call **after session resolution and workspace-aware hydration, but before provider or harness execution**. It has two paths: -- first turn: allocate an empty session root without stock Git initialization, validate and materialize into its checkout child, create the separate control child, validate the execution working directory, and commit the binding; -- continuation: hydrate the bound session root, then load and verify the existing checkout, control root, and execution working directory without invoking source acquisition. +- first turn: allocate an empty session root without stock Git initialization, validate and materialize into its checkout root, create the separate control root, validate the execution root, and commit the binding; +- continuation: hydrate the bound session root, then load and verify the existing checkout, control, and execution roots without invoking source acquisition. -The downstream code recognizes only `metadata.allagents.workspace` and calls one configured in-image materializer. The caller cannot name an implementation. There is no registry, plugin lifecycle, generic hook graph, network materializer service, or reusable extension SDK. +The owning HarnessRouter implementation recognizes only the selected `metadata.workspace` contract and calls one operator-configured in-image materializer. That code lands upstream when accepted and remains an explicit fork patch otherwise. The caller cannot name an implementation. There is no registry, plugin lifecycle, generic hook graph, network materializer service, or reusable extension SDK. The materializer contract is intentionally small: normalized descriptor in; an empty runner-assigned checkout target, cancellation, and fixed resource limits supplied by the runner; either a verified private checkout plus provenance, or a coded failure out. HarnessRouter remains responsible for session, checkpoint, file/artifact, and process lifecycle. Runner control state never lives inside the checkout, and the materializer cannot write it. -Requests without the AllAgents object retain upstream behavior, including routing for reused unbound sessions, and the pinned UHP conformance suite remains the protocol oracle. Each upstream rebase must review the patch against the metadata, session-resolution, hydration, checkpoint, runner-root, and custom-provider paths. If upstream gains an equivalent narrow lifecycle seam, remove the downstream patch rather than retain a compatibility layer. +Upstream-first is a contribution sequence, not a downstream namespace claim: + +1. Open a HarnessRouter issue before substantial implementation and confirm maintainers' preferred protocol process. +2. Submit a UEP when UHP governance requires one for the generic key and semantics. +3. If accepted, change the UHP specification and versioned schema, reference implementation, conformance suite, changelog, and documentation as one coherent contract. +4. If rejected or deferred, record the upstream decision and carry only the same bounded workspace patch in the fork, labeled and tested as a downstream extension rather than a UHP standard. +5. Keep URL policy, Git acquisition, workspace binding, continuation behavior, root separation, failure normalization, and the lifecycle seam generic in either implementation path. +6. Keep deployment defaults, custom harness definitions, external-provider wiring, Promptfoo scenarios, and downstream image publication in `allagentsdev/harnessrouter`. + +Requests without `metadata.workspace` retain upstream behavior, including routing for reused unbound sessions, and the pinned UHP conformance suite remains the base protocol oracle. Each rebase of `allagentsdev/harnessrouter` must review distribution changes against the metadata, session-resolution, hydration, checkpoint, runner-root, and custom-provider paths. When the workspace capability is downstream-only, its tests and release notes identify that status separately from UHP conformance. ## Provider authentication and harness configuration Provider authentication is proxy-only. Each deployment configures one external OAuth-to-OpenAI-compatible gateway base URL and API key server-side. HarnessRouter represents that endpoint with two protocol-specific logical connections using the same secret: Responses for Codex and OpenAI Chat Completions for OMP. Each harness policy contains exactly its matching connection, with no fallback. The UHP caller cannot supply or override the endpoint, key, transport, or route. -The external OAuth gateway owns login, token persistence, refresh, and repair. AllAgents Gateway does not implement provider login, import local credentials, mount developer credential files, or coordinate token refresh. HarnessRouter's caller API key authenticates the UHP caller only and is never reused as a provider credential. +The external OAuth gateway owns login, token persistence, refresh, and repair. HarnessRouter does not implement provider login, import local credentials, mount developer credential files, or coordinate token refresh. HarnessRouter's caller API key authenticates the UHP caller only and is never reused as a provider credential. -The deployment must use HarnessRouter's brokered sandbox mode, not the self-host image's `HR_SANDBOX_TRUST=owner` pass-through default. The gateway exchanges the long-lived external-gateway key server-side and gives each harness only a short-lived, session-scoped credential plus the loopback broker URL. Readiness fails if the local broker cannot mint and exchange that credential. The long-lived key never enters the harness process environment or session files. Scoped turn credentials may exist only in the active process environment or a per-turn ephemeral config root; the runner deletes them before checkpointing or exposing any file, artifact, log, or response. +The deployment must use HarnessRouter's brokered sandbox mode, not the self-host image's `HR_SANDBOX_TRUST=owner` pass-through default. The broker exchanges the long-lived external-gateway key server-side and gives each harness only a short-lived, session-scoped credential plus the loopback broker URL. Readiness fails if the local broker cannot mint and exchange that credential. The long-lived key never enters the harness process environment or session files. Scoped turn credentials may exist only in the active process environment or a per-turn ephemeral config root; the runner deletes them before checkpointing or exposing any file, artifact, log, or response. Codex requires an OpenAI Responses-compatible endpoint. This matches the pinned runner, which states that current Codex supports Responses rather than Chat Completions ([Codex endpoint behavior](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L1907-L1915)); HarnessRouter supports Codex against custom endpoints that provide the Responses format ([provider compatibility](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/docs/self-hosting-guide.md#L331-L350)). OMP uses the same external gateway's OpenAI Chat Completions surface in v1, which its pinned builder supports ([OMP endpoint behavior](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L2609-L2644)). An incompatible route fails readiness or the turn; it never changes protocol or provider automatically. @@ -142,44 +154,46 @@ Repository content is untrusted. The implementation must preserve all of these i - Every initial host and redirect target is re-parsed and re-authorized. DNS answers are checked against loopback, link-local, private, reserved, multicast, metadata-service, and otherwise non-public ranges; the approved address is pinned for the connection so DNS rebinding cannot change it. Redirect count, response size, and time are bounded. - Git runs with a sanitized environment and isolated configuration. Interactive credentials, repository hooks, checkout filters, Git LFS, submodules, alternates, and non-HTTPS helpers or protocols, including `file`, `ssh`, and `ext`, are disabled or rejected. Repository configuration cannot weaken those rules. - Ref discovery and fetch are bounded. V1 accepts only advertised branch/tag refs in SHA-1 repositories, ties the checkout to the exact resolved 40-hex commit, and records requested URL/ref plus resolved commit as provenance. -- The checkout is private and editable by one session only. No mutable state is shared across sessions. The runner-owned session root has separate checkout and control children; repository content can never overlap the control root. -- All path operations are rooted, no-follow where appropriate, and checked for traversal and symlink escape. The execution working directory must resolve to a real directory inside the checkout. Harness home, credentials, scratch, skills, and control state remain anchored under the sibling control root regardless of that working directory. +- The checkout is private and editable by one session only. No mutable state is shared across sessions. The runner-owned allocation maintains distinct session, checkout, control, and execution roots; repository content can never overlap the control root. +- All path operations are rooted, no-follow where appropriate, and checked for traversal and symlink escape. The execution root must resolve to a real directory inside the checkout root. Harness home, credentials, scratch, skills, and control state remain anchored under the control root regardless of the execution root. - Materialization and cleanup have hard process, descendant, wall-clock, byte, inode, file-count, and concurrency bounds. Cancellation terminates the complete acquisition process tree before cleanup and terminal acknowledgement. -- Partial staging is never attached. Cleanup is deterministic and idempotent after success, failure, cancellation, restart, expiry, and deletion. A path whose deletion failed is not reused or reported as free. +- Publication is crash-safe and partial staging is never attached. Cleanup is deterministic and idempotent after success, failure, cancellation, restart, expiry, and deletion. A path whose deletion failed is not reused or reported as free. - Long-lived provider and caller credentials never enter Git arguments, the harness process environment, checkout or session files, response metadata, artifacts, logs, or provenance. A short-lived broker token may enter only the active harness environment or per-turn ephemeral config and is removed before checkpointing or public file collection. Source-controlled configuration cannot select the provider endpoint. HarnessRouter's per-session process isolation remains useful, but this deployment is not represented as a hostile-code sandbox. The service binds to loopback by default and requires an explicit operator decision and network controls before broader exposure. ## Failure behavior -The gateway fails closed without changing source, checkout, harness, model route, or provider protocol as a recovery shortcut. +HarnessRouter fails closed without changing source, checkout, harness, model route, or provider protocol as a recovery shortcut. | Failure | Behavior | |---|---| -| Malformed, oversized, nested too deeply, or unknown workspace field | Reject as invalid UHP input before source access. | +| Malformed, oversized, nested too deeply, or unknown workspace field | Reject as invalid request input before source access. | +| Workspace extension not accepted upstream | Ship it only as a documented downstream HarnessRouter extension; never represent it as UHP-standard behavior. | | Disallowed URL, DNS answer, redirect, protocol, ref, or Git feature | Fail the response before attachment; remove bounded staging; do not start a harness or provider call. | | Ref does not resolve to one permitted commit | Fail with source-resolution error; do not guess a default or fetch arbitrary objects. | | Resource or concurrency limit unavailable | Reject or fail with a retryable capacity error before starting unbounded work. | | Materializer timeout, crash, cancellation, or live descendant | Terminate and reap the process tree, clean staging idempotently, and return a coded failure. | | Working directory missing, not a directory, or escaping through traversal/symlink | Fail before harness execution. | +| Crash during checkout publication or session binding | Recover to either the complete published checkout and binding or no attachment; never expose partial staging or reuse an uncertain path. | | Workspace metadata present on any reused session | Reject the request without changing the existing session or extending its TTL. | -| Bound checkout expired, missing, corrupt, or mismatched | Fail continuation; do not clone, substitute, or resurrect it. | +| Bound checkout expired, missing, corrupt, mismatched, or owned by an unsupported contract revision | Fail continuation; do not clone, substitute, resurrect, or reinterpret it. | | External OAuth gateway authentication or provider failure | Return the normalized UHP failure; do not switch endpoint, protocol, credential, or harness. | | Cleanup failure | Keep the allocation unavailable, report operational failure, and retry the same idempotent cleanup path. | -Promptfoo treats non-success as an evaluation error. It does not turn gateway failures into empty successes or implicit retries. +Promptfoo treats non-success as an evaluation error. It does not turn HarnessRouter failures into empty successes or implicit retries. ## Repository, deployment, and release boundary -`allagentsdev/allagents-gateway` is the implementation product. The repository contains the preserved HarnessRouter downstream, narrow workspace patch, materializer, Docker Compose deployment, pinned Promptfoo dependency and scenarios, and image release workflow. `allagentsdev/allagents` remains the local Bun CLI repository and contains only this integration decision and planning material; v1 adds no `allagents gateway start` command or other CLI coupling. +`allagentsdev/harnessrouter` remains the implementation and distribution repository and remains a GitHub fork of `HarnessRouter/harnessrouter`. It carries the upstream baseline plus the smallest necessary downstream distribution revision. It does not become a separate product repository. `allagentsdev/allagents` remains the local Bun CLI repository and contains this integration decision and planning material; v1 adds no gateway command or other CLI coupling. -Once the existing fork has been renamed and `allagentsdev/allagents-gateway` exists, that repository's code, lockfiles, Compose file, limits, runbooks, and implementation documentation are authoritative for implementation detail. This ADR remains authoritative for the integration and product boundary. An implementation need that contradicts this boundary requires reconsidering the decision, not silently expanding the downstream. +Generic workspace capability is proposed through the upstream contribution process. If accepted, it lands in the UHP specification, schema, reference implementation, conformance suite, changelog, and docs, and the fork consumes that release. If upstream rejects or defers it, the fork may carry the smallest complete workspace patch and identifies that delta in release notes, source metadata, and tests. In either path, deployment defaults, Codex and OMP custom harness definitions, provider wiring, pinned Promptfoo scenarios, and image release automation remain downstream. Generic protocol or lifecycle fixes discovered downstream continue to be proposed upstream rather than hidden behind product-specific seams. The supported deployment is one container started by Docker Compose, bound to loopback by default, with durable `/data` and `HR_BACKENDS=codex,omp`. The materializer ships in that image; it is not another service. -The public image is `ghcr.io/allagentsdev/allagents-gateway`. It has its own versions and release cadence, independent of the `allagents` npm CLI. Deployments pin image digests. Releases produce standard SBOM and build-provenance attestations and run upstream UHP conformance against the built image. +The public image is `ghcr.io/allagentsdev/harnessrouter`. Tags identify the upstream HarnessRouter baseline plus the downstream AllAgents revision; deployments pin the resulting image digest. Releases produce standard SBOM and build-provenance attestations and run upstream UHP conformance against the built image. -Promptfoo is lockfile-pinned in the gateway repository and calls the built image directly over UHP. Release verification exercises both Codex and OMP through the configured external provider gateway. No intermediate evaluation repository or custom green-E2E attestation format is part of the product. +Promptfoo is lockfile-pinned in `allagentsdev/harnessrouter` and calls the built image directly over UHP. Release verification exercises both Codex and OMP through the configured external provider gateway. No intermediate evaluation repository or custom green-E2E attestation format is part of the distribution. ## Alternatives rejected @@ -187,12 +201,13 @@ Promptfoo is lockfile-pinned in the gateway repository and calls the built image |---|---| | Build a new execution gateway | Duplicates HarnessRouter's UHP, sessions, streaming, cancellation, files, artifacts, and harness supervision. | | Put a thin service in front of stock HarnessRouter | Splits checkout and session ownership across services and still cannot place the workspace at the correct runner lifecycle point. | -| Create a third repository that consumes the HarnessRouter fork | Loses the clear downstream history and adds a release/rebase boundary without adding product isolation. | -| Wait for stock HarnessRouter | The pinned version accepts arbitrary metadata but forwards only the System One probe; it has no workspace lifecycle semantics. | -| Add a general metadata extension or materializer framework | V1 has one object and one implementation. A framework would enlarge the fork before a second use case exists. | +| Rename the existing fork or create a third repository around it | Loses the clear upstream relationship or adds a release/rebase boundary without product isolation. | +| Present a downstream workspace extension as standard UHP behavior | The open metadata object permits an implementation extension, but only upstream governance can standardize its meaning and conformance requirements. | +| Wait without engaging upstream | The pinned version accepts arbitrary metadata but forwards only the System One probe; an issue and, if required, a UEP are the path to obtaining workspace lifecycle semantics. | +| Add a general metadata extension or materializer framework | V1 has one object and one implementation. A framework would enlarge the change before a second use case exists. | | Put repository instructions in the prompt or a model tool | Makes acquisition model-dependent, non-deterministic, too late to set the initial working directory, and unsafe for credentials and provenance. | | Make the local AllAgents CLI or its profiles the remote control plane | Couples a local developer tool to an independently deployed service and duplicates HarnessRouter custom harnesses. | -| Manage provider login inside AllAgents Gateway | Duplicates the external OAuth gateway's ownership of login, refresh, and repair and expands the credential attack surface. | +| Manage provider login inside HarnessRouter | Duplicates the external OAuth gateway's ownership of login, refresh, and repair and expands the credential attack surface. | | Upload every source file through UHP | Pushes acquisition to every caller and loses authoritative Git ref-to-commit provenance and repository behavior. | ## Deliberate v1 limits @@ -201,25 +216,28 @@ V1 supports one anonymous public HTTPS Git repository using SHA-1 object IDs, on V1 does **not** include raw commit-ID requests, SHA-256 repositories, multiple repositories, private-source credentials, OCI sources, caller-selected runtime images, shared or read-only generations, cross-session caching, persistent workspaces, user-selected TTLs, session branching, checkout migration, or elaborate tombstone and garbage-collection machinery beyond minimal idempotent cleanup. It uses HarnessRouter's existing file and artifact behavior rather than inventing produced-file tracking. -Only Codex and OMP are required and release-validated. Other upstream backends and a future Copilot harness are outside this decision. There is no local-profile import, host-profile projection, provider-route override, automatic provider fallback, public multi-tenant authorization model, scoring service, dataset service, or evaluation task engine. +Only Codex and OMP are required and release-validated by the AllAgents distribution. Other upstream backends and a future Copilot harness are outside this decision. There is no local-profile import, host-profile projection, provider-route override, automatic provider fallback, public multi-tenant authorization model, scoring service, dataset service, or evaluation task engine. ## Consequences -AllAgents Gateway inherits a mature UHP execution plane and keeps the maintained patch reviewable. The cost is an ongoing rebase obligation against pinned HarnessRouter releases and ownership of a security-sensitive Git materializer. +HarnessRouter gains a bounded workspace-materialization contract without creating a separate gateway product. The preferred outcome is a governed UHP capability; the fallback is a clearly labeled downstream extension in the existing fork. The contribution cost includes upstream design review, UEP work when required, and coordinated specification/schema/implementation/conformance changes when maintainers accept the contract. + +`allagentsdev/harnessrouter` retains a clear fork relationship and a narrow distribution delta. Its image and the `allagents` npm CLI release independently. Promptfoo tests the same digest-pinned image and UHP surface that operators deploy. Each session pays for a private checkout and cannot reuse a shared generation. That is intentionally less efficient than a source platform, but it makes mutability, provenance, continuation, quota, and cleanup ownership understandable for v1. -Provider credential lifecycle stays outside the gateway. This reduces credential code and operational states in the downstream, at the cost of requiring a compatible external OAuth gateway and making its availability part of the service's readiness. +Provider credential lifecycle stays outside HarnessRouter. This reduces credential code and operational states in the distribution, at the cost of requiring a compatible external OAuth gateway and making its availability part of service readiness. -The gateway and CLI can release independently. Promptfoo tests the same image and UHP surface that operators deploy. +Upstream review may accept, reject, defer, or reshape the proposal. Acceptance lets the fork delete generic workspace patches and consume the upstream release. Rejection or deferral leaves those patches visible as a downstream HarnessRouter extension; it does not justify a repository rename or a false claim that UHP conformance covers the extension. ## Reconsider when Revisit this decision if: -- UHP standardizes a workspace attachment with equivalent first-turn and continuation semantics; -- upstream HarnessRouter adds an equivalent narrow post-hydration, pre-execution workspace seam; -- the downstream patch grows beyond metadata recognition, binding, and one fixed materializer invocation; +- upstream HarnessRouter or UHP governance materially changes the generic workspace contract or later reserves an incompatible key; +- maintainers require a different lifecycle point, schema, or extension mechanism; +- an accepted upstream path cannot keep the specification, schema, reference implementation, conformance, changelog, and documentation aligned as one coherent versioned contract; +- the generic seam grows beyond metadata recognition, binding, and one fixed materializer invocation; - a second materializer is approved and proves that a registry is simpler than explicit code; - private repositories, multiple repositories, OCI sources, persistent workspaces, or shared immutable caching become validated product requirements; - public multi-tenancy or stronger hostile-code isolation becomes a requirement; diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index daa47c3e..f2197f07 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,5 +1,5 @@ --- -title: "AllAgents Gateway v1 - Implementation Plan" +title: "HarnessRouter Workspace Execution - Implementation Plan" date: 2026-09-18 updated: 2026-09-27 type: feat @@ -8,72 +8,128 @@ artifact_readiness: implementation-ready execution: code --- -# AllAgents Gateway v1 - Implementation Plan +# HarnessRouter Workspace Execution - Implementation Plan ## Goal -Ship **AllAgents Gateway** as a small, maintainable downstream of HarnessRouter -that lets Promptfoo invoke Codex or OMP over UHP against a caller-selected public -Git repository. - -The implementation repository is -[`allagentsdev/allagents-gateway`](https://github.com/allagentsdev/allagents-gateway). -Establish it by renaming the existing `allagentsdev/harnessrouter` GitHub fork, -not by creating a third repository or wrapping one repository with another. The -rename must preserve the fork relationship, commit history, issues, settings, -and a usable upstream remote. - -The implementation starts from HarnessRouter v0.25.4 at commit -`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` and keeps UHP -`2026-09-12` as the northbound protocol. The public image is -`ghcr.io/allagentsdev/allagents-gateway`, with versions and release notes that -are independent of the AllAgents npm CLI. - -Version one adds one product behavior to stock HarnessRouter: on the first turn -of a session, recognize `metadata.allagents.workspace`, securely materialize one -public HTTPS Git repository into one private editable checkout, bind that exact -checkout to the session, and start the selected harness in the requested safe -working directory. A continuation reuses that checkout without resolving or -cloning again. - -Everything else stays with HarnessRouter: caller authentication, UHP request and -response behavior, session hydration, streaming, idempotency, cancellation, -provider and harness execution, artifacts, and ordinary lifecycle state. - -## Repository boundary - -All production code, image construction, Compose configuration, Promptfoo -configuration, tests, and release automation land in -`allagentsdev/allagents-gateway`. - -This `allagentsdev/allagents` repository remains the local Bun CLI. For v1 it -contains only the accepted ADR and this implementation plan. Specifically: - -- no gateway server, materializer, image build, or provider adapter is added here; -- no `allagents gateway start` command is added; -- local profiles are not uploaded, synchronized, or translated into remote - configuration; -- local `workspace.yaml`, its schema, and its behavior remain unchanged; and -- the gateway does not import an AllAgents host profile or invoke the AllAgents - CLI. - -HarnessRouter custom harness definitions are the complete remote configuration -surface for Codex and OMP. OMP runs as an ordinary session-local harness inside -HarnessRouter. +Add a generic workspace contract and the runner capabilities needed to execute +Codex or OMP in a caller-selected public Git repository. Propose the protocol, +schema, implementation, conformance cases, changelog, and documentation through +HarnessRouter's upstream governance process first. If upstream declines or +defers the contribution, carry the same bounded behavior as a clearly labeled +downstream HarnessRouter extension. Keep the existing GitHub fork at +[`allagentsdev/harnessrouter`](https://github.com/allagentsdev/harnessrouter) and +publish an AllAgents-maintained image at +`ghcr.io/allagentsdev/harnessrouter`. + +A new session may supply `metadata.workspace`. HarnessRouter securely resolves +one anonymous public HTTPS Git repository to an advertised SHA-1 commit, +materializes one private editable checkout, binds that exact checkout to the +session, and starts the selected harness in the requested safe working +directory. A continuation reuses the checkout and its mutations without +resolving or cloning again. + +Everything else remains HarnessRouter behavior: caller authentication, UHP +request and response handling, session hydration, streaming, idempotency, +cancellation, provider and harness execution, artifacts, and ordinary lifecycle +state. + +## Governance and repository boundary + +### Upstream-first gate + +`metadata.workspace` and the associated runner behavior solve a generic +HarnessRouter problem, so upstream gets the first opportunity to own them. The +fork may ship the contract after an upstream decision, but must label it as a +downstream HarnessRouter extension unless UHP governance standardizes it. Before +substantial implementation work: + +1. Open an issue in `HarnessRouter/harnessrouter` that describes the use case, + request and response shapes, security boundary, session semantics, runner + seam, lifecycle, error categories, and intended conformance coverage. +2. Ask maintainers to confirm the required governance path and ownership of the + metadata key, configuration names, and runner interfaces. +3. Write or amend the required UHP Enhancement Proposal (UEP) before changing + protocol semantics. Follow the repository's contribution and UHP governance + rules for discussion, review, compatibility, and approval. +4. Record the issue and UEP links in the implementation pull requests and in the + downstream release notes. + +The gate is passed when maintainers have selected the governance and ownership +path for `metadata.workspace`. Acceptance starts the coordinated upstream +contract; rejection or deferral starts the downstream-extension path. If no +maintainer decision arrives within 30 calendar days after the issue opens and a +follow-up is posted after day 14, record the proposal as deferred. A downstream +release must not claim that its workspace behavior is part of UHP or covered by +UHP conformance. If a later UHP release reserves an incompatible key or shape, +the fork migrates cleanly instead of retaining conflicting aliases. + +### Upstream deliverables when accepted + +An accepted upstream change is complete only when the same reviewed contract +appears in all relevant surfaces: + +- the UHP specification; +- the machine-readable UHP schema; +- the HarnessRouter reference implementation; +- the UHP conformance suite; +- the HarnessRouter changelog; and +- user and operator documentation. + +Generic workspace parsing, session binding, runner root separation, +materialization, lifecycle, errors, and conformance behavior are proposed +upstream together. If maintainers decline or defer that ownership, the fork +carries the smallest complete patch in the corresponding HarnessRouter paths, +with separate extension tests and release notes. The implementation remains +generic and must not couple lifecycle behavior to AllAgents harness IDs. + +### Downstream repository + +Keep `allagentsdev/harnessrouter` as the GitHub fork of +`HarnessRouter/harnessrouter`, preserving its fork relationship, history, +issues, settings, protections, and upstream remote. Do not create another +repository and do not add a wrapper repository. + +The preferred fork delta contains only the downstream pieces specific to the +AllAgents deployment: + +- deployment defaults; +- custom Codex and OMP harness definitions; +- provider connection and policy wiring; +- direct Promptfoo scenarios and sanitized reports; and +- image build and publication for `ghcr.io/allagentsdev/harnessrouter`. + +If upstream declines or defers workspace ownership, the fork also carries the +smallest complete workspace contract and runner/materializer patch. Source +metadata, tests, and release notes distinguish that extension from standard UHP. + +The initial examined baseline is HarnessRouter v0.25.4 at commit +`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`, with UHP `2026-09-12`. +Development records that baseline exactly. The release baseline must be an exact +upstream tag and commit. Workspace behavior then comes either from an accepted +upstream release or from explicitly identified downstream commits applied to +that baseline. + +The `allagentsdev/allagents` repository remains the local Bun CLI. This plan +adds no server, materializer, provider adapter, CLI command, local-profile +synchronization, or `workspace.yaml` behavior there. HarnessRouter custom +harness definitions are the complete remote configuration surface for Codex and +OMP. ## Product boundary ### In scope -- UHP `2026-09-12`, exposed by the pinned HarnessRouter downstream. -- Caller authentication using HarnessRouter's existing API-key behavior. -- Exactly one workspace extension: `metadata.allagents.workspace`. +- The UHP baseline selected through the upstream issue and UEP process, plus the + accepted upstream workspace contract or documented downstream extension. +- Caller authentication through HarnessRouter's existing API-key behavior. +- One generic workspace object at `metadata.workspace`. - Exactly one anonymous, public, HTTPS Git repository per new session. - An optional advertised `refs/heads/*` or `refs/tags/*`, or unambiguous branch/tag shorthand. Omission means the remote default branch; raw object IDs - and other ref namespaces are not accepted. -- SHA-1 repositories only, with resolution to and recording of one exact - 40-hex commit object ID. + and other ref namespaces are rejected. +- SHA-1 repositories only, with resolution to and recording of one exact 40-hex + commit object ID. - One private editable checkout per session. - Immutable first-turn binding and exact-checkout continuation reuse. - A finite server-owned idle TTL and deterministic, idempotent cleanup. @@ -82,18 +138,19 @@ HarnessRouter. - One container, one `/data` volume, and Docker Compose startup bound to loopback by default. - Direct Promptfoo evaluation of the built image. -- Upstream UHP conformance, image SBOM/provenance, and digest-pinned releases. +- Upstream UHP conformance, separate downstream-extension coverage when needed, + image SBOM/provenance, and digest-pinned releases. ### Non-goals - More than one repository, destination mapping, or repository composition. -- Non-Git workspace sources, private source credentials, SSH Git transports, - or caller-provided source headers. +- Non-Git workspace sources, private source credentials, SSH Git transports, or + caller-provided source headers. - Raw commit-ID requests and SHA-256 Git repositories. -- Read-only workspaces, cross-session workspace reuse, prewarming, or source - object stores. -- Caller-selected lifetime, indefinite sessions, recovery after the configured - expiry, or a new lifetime subsystem. +- Read-only workspaces, cross-session checkout reuse, prewarming, or shared + source object stores. +- Caller-selected lifetime, indefinite sessions, recovery after configured + expiry, or a separate lifetime subsystem. - A generic extension registry, plugin framework, or multiple materializers. - Provider login, token refresh, credential repair, or credential projection in HarnessRouter. Those belong to the external provider gateway. @@ -103,17 +160,16 @@ HarnessRouter. `workspace.yaml`, or adding an AllAgents CLI command. - Replacing HarnessRouter sessions, task execution, artifacts, streaming, cancellation, or idempotency. -- New validation commitments for other HarnessRouter backends. +- New validation commitments for unrelated HarnessRouter backends. - Kubernetes, multi-container worker orchestration, or a separately deployed materializer service. -- A custom release-attestation or green-build framework. +- A custom attestation or green-build framework. ## External contracts ### UHP request -The workspace object is nested metadata, following HarnessRouter's existing -`metadata.systemone.script` precedent. It is not a flat metadata key. +The proposed generic request shape is: ```json { @@ -121,56 +177,61 @@ The workspace object is nested metadata, following HarnessRouter's existing "input": "Inspect the project and fix the failing command.", "metadata": { "harness_id": "allagents-codex", - "allagents": { - "workspace": { - "version": "1", - "repository": { - "url": "https://github.com/example/project.git", - "ref": "refs/heads/main" - }, - "workingDirectory": "packages/service" - } + "workspace": { + "repository": { + "url": "https://github.com/example/project.git", + "ref": "refs/heads/main" + }, + "working_directory": "packages/service" } } } ``` -The exact v1 JSON shape is: +The exact request object proposed to UHP is: ```text -metadata.allagents.workspace = { - version: "1", +metadata.workspace = { repository: { url: string, ref?: string }, - workingDirectory?: string + working_directory?: string } ``` +The workspace object has no independent schema marker. An accepted UHP release +owns its standard schema; while it is downstream-only, the fork release owns the +extension shape. + Rules: 1. `workspace` and `repository` must be JSON objects, not arrays or strings. -2. `version` is required and must equal `"1"`. -3. `repository.url` is required. It must be an anonymous public `https://` Git +2. `repository.url` is required. It must be an anonymous public `https://` Git URL with no user info, query, fragment, alternate transport, or embedded credential. -4. `repository.ref` is optional and non-empty when present. It must be an +3. `repository.ref` is optional and non-empty when present. It must identify an advertised `refs/heads/*` or `refs/tags/*`, or unambiguous branch/tag - shorthand. Raw object IDs and other ref namespaces are rejected. V1 accepts - SHA-1 repositories only and records the exact resolved 40-hex commit as - provenance. -5. `workingDirectory` is optional. Omission means the checkout root. When - present it is a normalized, relative POSIX path to a directory within the + shorthand. Raw object IDs and other ref namespaces are rejected. Only SHA-1 + repositories are accepted, and the exact resolved 40-hex commit is recorded + as provenance. +4. `working_directory` is optional. Omission means the checkout root. When + present it is a normalized relative POSIX path to a directory within the checkout. -6. Unknown fields at every level are rejected. There are no aliases, commands, +5. Unknown fields at every level are rejected. There are no aliases, commands, environment variables, access modes, lifetime fields, destination paths, materializer selectors, or provider settings. -7. The gateway applies a small fixed metadata byte/depth bound before session +6. HarnessRouter applies a small fixed metadata byte/depth bound before session allocation. The materializer applies field-specific length bounds before network or filesystem work. -8. A request without `metadata.allagents.workspace` follows unmodified - HarnessRouter behavior. +7. A request without `metadata.workspace` follows ordinary upstream behavior. + +The upstream issue and UEP own final standard names when accepted. If review +changes the proposed shape, update the specification, schema, reference +implementation, conformance suite, examples, and this plan together. A +downstream-only implementation keeps the proposed names, documents their +extension status, and tracks any later UHP collision as a required clean +migration. ### Session binding and public provenance @@ -183,35 +244,37 @@ After materialization, the session owns one immutable binding containing: - private checkout root beneath that session root; - separate runner control root as a sibling of the checkout; - execution working directory beneath the checkout; -- materializer contract version; +- the governing UHP release; +- the workspace-contract owner and immutable revision, expressed internally as + the accepted UHP release or exact downstream source commit; - creation time and server-owned expiry; and - cleanup state sufficient to make removal idempotent. -The checkout root and cleanup token are internal and must never appear in UHP -responses, streams, logs, or Promptfoo output. Successful responses expose the -stable portion as nested provenance: +The checkout root, session root, control root, and cleanup token are internal and +must never appear in UHP responses, streams, logs, artifacts, or Promptfoo +reports. Successful responses expose only stable public provenance: ```json { "metadata": { - "allagents": { - "workspace": { - "version": "1", - "repository": { - "url": "https://github.com/example/project.git", - "requestedRef": "refs/heads/main", - "resolvedCommit": "0123456789abcdef0123456789abcdef01234567" - }, - "workingDirectory": "packages/service" - } + "workspace": { + "repository": { + "url": "https://github.com/example/project.git", + "requested_ref": "refs/heads/main", + "resolved_commit": "0123456789abcdef0123456789abcdef01234567" + }, + "working_directory": "packages/service" } } } ``` -If the ref was omitted, `requestedRef` is omitted rather than synthesized. The -same public provenance is returned on successful continuations and idempotent -response retrieval. +If the ref was omitted, `requested_ref` is omitted rather than synthesized. If +the working directory was omitted, `working_directory` is omitted. The same +public provenance is returned on successful continuations and idempotent +response retrieval. The selected contract and schema distinguish accepted +request fields from read-only response provenance fields; the UEP owns that +distinction when upstream accepts the contract. ### Continuation @@ -226,402 +289,362 @@ A continuation normally supplies only `previous_response_id`: HarnessRouter may also reuse a session through its existing `metadata.session_id` recovery path. After session resolution, every reused -session rejects `metadata.allagents.workspace` regardless of which identifier -selected it. For a workspace-bound session, the stored binding and harness -select execution, and any caller-supplied harness must match exactly; the gateway -rejects a mismatch before Git, hydration, or provider work. A reused session -without an AllAgents binding and without workspace metadata retains pinned -upstream routing behavior. - -A valid workspace continuation reuses the exact checkout, control root, and -execution working directory. A missing, expired, cleaned, or mismatched checkout -fails closed; it is never silently cloned again. +session rejects `metadata.workspace`, regardless of which identifier selected +it. For a workspace-bound session, stored binding and harness state select +execution, and any caller-supplied harness must match exactly. A mismatch fails +before Git, hydration, or provider work. A reused unbound session without +workspace metadata retains ordinary upstream routing behavior. + +A valid workspace continuation reuses the exact session root, checkout root, +control root, execution working directory, and checkout mutations. It verifies +that the runtime supports the binding's recorded workspace-contract owner and +revision. A missing, expired, cleaned, identity-mismatched, or unsupported +binding fails closed; HarnessRouter never silently clones or reinterprets it. +Before activating an incompatible contract, an upgrade must explicitly migrate +compatible bindings or drain and delete them. ### Provider and harness contract -The deployment defines two protocol-specific HarnessRouter connections that -point to the same existing OAuth gateway and use the same server-side base URL -and API-key secret: +The downstream deployment defines two protocol-specific HarnessRouter +connections that point to one external OAuth gateway and use the same +server-side base URL and API-key secret: - a Responses-format connection used only by Codex; and - an OpenAI Chat Completions connection used only by OMP. -This is one external provider route with two HarnessRouter protocol adapters, -not two credential authorities. The OAuth gateway owns user login, upstream -token storage, refresh, and repair. +This is one external provider route with two logical protocol adapters, not two +credential authorities. The external gateway owns user login, upstream token +storage, refresh, and repair. - A new session selects only an allowed `metadata.harness_id` and model. -- A reused workspace-bound session derives its harness from stored session - state; a supplied mismatch fails before execution. Unbound sessions retain - upstream routing. +- A reused workspace-bound session derives its harness from stored state; a + supplied mismatch fails before execution. Unbound sessions retain upstream + routing. - Each custom harness has an explicit model allowlist and a one-entry provider - policy pointing to its protocol-specific connection. + policy pointing to its matching protocol-specific connection. - There is no fallback connection or automatic transport switching. - Provider base URL, API key, transport, headers, and model mapping cannot be supplied in UHP input or workspace metadata. -- Provider failure is returned as ordinary HarnessRouter/UHP failure. It never - changes workspace or routing state. -- HarnessRouter runs in brokered sandbox mode. The harness receives a - short-lived session-scoped credential and loopback broker URL, never the - long-lived external-gateway key. The scoped credential exists only for the - active turn and is removed before checkpointing or public file collection. +- Provider failure is returned as an ordinary HarnessRouter/UHP failure. It does + not change workspace or routing state. +- HarnessRouter runs in brokered sandbox mode. A harness receives a short-lived, + session-scoped credential and loopback broker URL, never the long-lived + external-gateway key. The scoped credential exists only for the active turn + and is removed before checkpointing or public file collection. ## Security and resource invariants These are release requirements, not later hardening: -1. **URL and DNS:** accept only public HTTPS destinations. Reject loopback, +1. **URL and DNS:** Accept only public HTTPS destinations. Reject loopback, link-local, private, carrier-grade NAT, documentation, multicast, reserved, and otherwise non-public IPv4/IPv6 results. Validate every DNS answer before connection, pin the validated address for that hop, revalidate every redirect, - and cap redirects. A public name that resolves to any forbidden address - fails closed. -2. **Git protocols:** disable `file`, `ssh`, `git`, `ext`, and helper-driven + and cap redirects. A public name that resolves to any forbidden address fails + closed. +2. **Git protocols:** Disable `file`, `ssh`, `git`, `ext`, and helper-driven alternate protocols. Clear inherited Git configuration and credential - helpers. Set terminal prompting off. Requests never provide credentials. -3. **Repository execution:** disable repository hooks and clean/smudge/process + helpers. Disable terminal prompting. Requests never provide credentials. +3. **Repository execution:** Disable repository hooks and clean/smudge/process filters. Do not initialize submodules. Detect and reject gitlinks and Git LFS - pointer-backed content rather than executing helpers or returning a partial + pointer-backed content instead of executing helpers or returning a partial workspace as complete. -4. **Filesystem confinement:** each runner-owned session root has separate - checkout and control children. Build the checkout in sibling staging, then - publish it into an absent checkout target. Keep harness home, credentials, - scratch, skills, and runner state in the control root. Reject absolute paths, - `..`, empty segments, NUL, platform separator ambiguity, and symlinks that - escape the checkout. The execution working directory must exist beneath the - checkout and never changes the control root. -5. **Exact provenance:** resolve the requested ref, fetch the corresponding - commit, detach checkout at that commit, and verify `HEAD` equals the recorded - object ID before publication. Ref movement after resolution cannot change the - bound checkout. -6. **Bounds:** enforce server-owned limits for request bytes, ref and path - lengths, clone/fetch duration, materialized bytes, inodes, process output, - concurrent materializations, active harness time, and idle checkout TTL. - Limits apply during work, not only after completion. -7. **Process control:** run Git and materializer children in a cancellable process +4. **Root separation:** Keep four explicit values: session root, checkout root, + control root, and execution working directory. The checkout and control roots + are separate children of the session root. Harness HOME, credentials, + scratch, skills, and runner state live only under control. The execution + directory is a no-follow-validated descendant of checkout. Repository content + cannot become control state. +5. **Filesystem confinement:** Build the checkout in sibling staging and publish + it only into an absent checkout target. Reject absolute paths, `..`, empty + segments, NUL, platform-separator ambiguity, and symlinks that escape the + checkout. +6. **Exact provenance:** Resolve the requested ref, fetch its commit, check out + detached, and verify `HEAD` equals the recorded object ID before publication. + Later ref movement cannot change the bound checkout. +7. **Bounds:** Enforce server-owned limits for request bytes, URL/ref/path + lengths, redirects, clone/fetch duration, materialized bytes, inodes, process + output, concurrent materializations, active harness time, and idle checkout + TTL. Apply limits during work, not only after completion. +8. **Process control:** Run Git and materializer children in a cancellable process group with a minimal environment, bounded stdout/stderr capture, and a hard - termination deadline. Cancellation must stop descendants. -8. **Publication:** a workspace-aware first hydrate creates an isolated empty - session root without stock Git initialization or other files in the checkout - target. Materialize and validate in sibling staging, persist a pending - reservation, atomically publish staging as the checkout child, create the - separate control child, then commit the usable binding. Startup cleanup - removes abandoned staging, pending reservations, and published-but-unbound - roots. A harness cannot observe staging or a partially validated tree. -9. **Isolation:** never bind one session to another session's checkout. Each - first turn creates a new private checkout even when URL, ref, and resolved - commit are identical. -10. **Cleanup:** removal is safe to repeat, never follows links, is confined to + termination deadline. Cancellation stops descendants. +9. **Crash-safe publication:** A workspace-aware first hydrate creates an + isolated empty session root with checkout absent. Persist a pending + reservation, materialize and validate in sibling staging, atomically publish + staging as checkout, create the separate control root, then atomically commit + the usable binding. Startup reconciliation removes abandoned staging, pending + reservations, published-but-unbound roots, and cleanup-marked roots. A + harness can never observe staging or a partially validated tree. +10. **Isolation:** Never bind one session to another session's checkout. Each + first turn creates a private checkout even when URL, ref, and resolved commit + are identical. +11. **Cleanup:** Removal is safe to repeat, never follows links, is confined to the recorded session root, and cannot remove another session's data. -11. **Provider secrets:** `HR_SANDBOX_TRUST=owner` is forbidden. The local - HarnessRouter broker must exchange a scoped turn credential for the real - external-gateway key. The long-lived key never enters the harness - environment or filesystem. Generate any CLI credential config in a - per-turn ephemeral control subtree, explicitly exclude it from checkpoints - and public file/artifact APIs, and delete it before terminal persistence. - Neither long-lived keys nor scoped broker tokens may survive in a - checkpoint, session file, artifact, stream, response, or log. +12. **Provider secrets:** `HR_SANDBOX_TRUST=owner` is forbidden. The local broker + exchanges a scoped turn credential for the real external-gateway key. The + long-lived key never enters the harness environment or filesystem. Generate + CLI credential config only in a per-turn ephemeral control subtree, exclude + it from checkpoints and public file/artifact APIs, and delete it before + terminal persistence. Neither long-lived keys nor scoped broker tokens may + survive in a checkpoint, session file, artifact, stream, response, or log. ## Workstreams and implementation phases -The phases are ordered by dependency. Each phase ends with observable behavior; -implementation does not advance on the strength of source inspection alone. +The phases are ordered by dependency. Each ends with observable behavior; source +inspection alone is not an exit criterion. + +### Phase 1: Establish the upstream issue and ownership decision -### Phase 1: Establish the downstream repository and immutable pins +**Outcome:** HarnessRouter maintainers have selected the governance path and +either accepted upstream ownership or recorded that the fork must own the +extension. -**Outcome:** `allagentsdev/allagents-gateway` is the sole implementation -repository and can reproduce the examined HarnessRouter baseline. +Work: + +1. Open the upstream issue with the exact request shape, response provenance, + first-turn-only rule, continuation behavior, failure categories, source + restrictions, root model, lifecycle, and non-goals. +2. Link prior security analysis without importing AllAgents-specific naming into + the generic contract. +3. Draft the required UEP and identify compatibility effects on UHP clients, + schemas, conformance runners, and existing metadata behavior. +4. Obtain maintainer direction on allocation of `metadata.workspace`, generic + configuration names, materializer placement, runner interface, and release + target. +5. If no decision arrives within 30 calendar days after the issue opens, post a + follow-up after day 14 and record non-response at day 30 as deferral. +6. Split reviewable upstream changes according to maintainer preference while + keeping one coherent protocol contract. +7. Record explicit decisions and update every example when review changes a + name or behavior. + +Exit gate: + +- the upstream issue exists and links the UEP when required; +- maintainers have made an explicit decision, or the documented non-response + window has elapsed and is recorded as deferral; +- accepted semantics are reserved through governance, or downstream semantics + are explicitly labeled as a HarnessRouter fork extension; and +- no unresolved decision blocks implementation in the selected ownership path. + +### Phase 2: Pin the fork baseline and supply chain + +**Outcome:** `allagentsdev/harnessrouter` can reproducibly build the examined +upstream baseline and identify both that baseline and every downstream workspace +commit when the extension is not accepted upstream. Work: -1. Rename the existing GitHub fork `allagentsdev/harnessrouter` to - `allagentsdev/allagents-gateway`. Preserve its upstream fork relationship, - branches, history, issues, rules, secrets, and package/container permissions. -2. Set `HarnessRouter/harnessrouter` as the documented upstream remote and record - the baseline commit - `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` / v0.25.4 in release automation. -3. Keep the downstream patch series reviewable: baseline sync commits are - separate from AllAgents behavior commits, and upstream merges never combine - with feature changes. -4. Pin UHP `2026-09-12`; the base image by manifest digest; Codex and OMP runtime - versions; system packages that affect Git/materialization; and every CI action. - No `latest`, floating branch, or unbounded package range may enter a release. -5. Add an exact Promptfoo development dependency and committed lockfile in this - repository. Promptfoo is an image consumer, not a service dependency. -6. Rename image/repository references and release coordinates to - `ghcr.io/allagentsdev/allagents-gateway`. Do not reuse the AllAgents npm - version or publish under the old image name. -7. Capture an upstream-diff check in CI so every release identifies the baseline - and downstream commits included in the image. +1. Preserve `HarnessRouter/harnessrouter` as the documented upstream remote and + record v0.25.4 commit + `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` as the initial examined baseline. +2. Keep upstream synchronization commits separate from downstream deployment + commits. Never combine a baseline jump with a product behavior change. +3. Pin the selected UHP release, base image by manifest digest, Codex and OMP + runtime releases, Git/materialization system packages, Promptfoo dependency + and lockfile, and every CI action. No `latest`, floating branch, or unbounded + package range enters a release. +4. Configure image publication for `ghcr.io/allagentsdev/harnessrouter`. +5. Add an upstream-diff check so each candidate records its exact upstream tag + and commit plus every downstream commit in the image. +6. Use the upstream-baseline-plus-downstream-revision tag convention + `-allagents.`, for example + `v0.25.4-allagents.1`. Increment the downstream revision for any image change + on the same upstream baseline; reset it to `.1` when the upstream baseline + tag changes. +7. Use the same tag for the downstream source release and OCI image, then deploy + the image only by manifest digest, for example + `ghcr.io/allagentsdev/harnessrouter:v0.25.4-allagents.1@sha256:`. Behavior-focused proof: -- a clean checkout builds the same downstream commit from recorded pins; -- repository links, image labels, and release metadata identify - `allagentsdev/allagents-gateway` and the pinned upstream commit; and -- changing a pin or lockfile is visible as a reviewed source diff. +- a clean fork checkout builds the recorded baseline from pinned inputs; +- source links, OCI labels, release metadata, and SBOM identify + `allagentsdev/harnessrouter`, the exact upstream commit, and the downstream + revision; and +- changing a pin, baseline, or lockfile appears as a reviewed source diff. -Exit gate: the renamed repository builds the unchanged pinned HarnessRouter -image before workspace behavior is introduced. +Exit gate: the unchanged pinned baseline image builds before workspace behavior +or downstream deployment configuration is added. -### Phase 2: Define and enforce the exact workspace schema +### Phase 3: Land the workspace contract in the selected ownership path -**Outcome:** the gateway recognizes only the bounded v1 object and ordinary UHP -requests remain stock behavior. +**Outcome:** the contract, parser, and behavior tests agree on one bounded +first-turn workspace object; requests without it retain ordinary behavior. When +accepted upstream, specification, machine schema, conformance, changelog, and +user documentation define the same standard contract. -Primary area: `gateway/app.py` and focused gateway tests. +Primary areas: UHP specification/schema and conformance when accepted upstream; +`gateway/app.py`, focused gateway tests, changelog, and user documentation in +either ownership path. Work: -1. Parse nested `metadata.allagents.workspace` on initial Responses requests. - Leave handling of unrelated metadata unchanged. -2. Apply generic byte/depth bounds before response/session allocation. -3. Strictly validate the exact request schema defined above, including unknown - field rejection and normalized relative working-directory rules. -4. Extend session resolution to return whether it created or reused a session. - Store the canonical descriptor only when the resolver proves the session is - new. Use one canonical serialization and digest so idempotent replay cannot - create a second binding. -5. Reject workspace metadata for every reused session, whether selected through +1. Define the request and public provenance shapes through the upstream review. + An accepted UHP release owns standard compatibility and schema evolution; + otherwise the downstream fork release owns the extension shape. +2. Parse `metadata.workspace` only on an initial request. Leave unrelated + metadata unchanged. +3. Apply generic metadata byte/depth bounds before response or session + allocation. +4. Strictly validate required and optional fields, unknown-field rejection, + anonymous public HTTPS policy, and normalized relative + `working_directory` rules. +5. Extend session resolution to report whether it created or reused a session. + Store the canonical descriptor only after proving the session is new. Use one + canonical serialization and digest so idempotent replay cannot create a + second binding. +6. Reject workspace metadata for every reused session selected by either `previous_response_id` or `metadata.session_id`, before Git, hydration, runner, or provider work. -6. For a reused workspace-bound session, derive the harness from stored state - and reject a caller-supplied mismatch before hydration. Leave reused unbound - sessions on the pinned upstream routing path. -7. Map schema and reuse failures into stable UHP errors with the offending - parameter named. Do not expose Python exceptions or internal paths. -8. Keep requests without the object on the untouched upstream path. +7. For a reused workspace-bound session, derive the harness from stored state and + reject a caller-supplied mismatch before hydration. Leave reused unbound + sessions on the ordinary route. +8. Map validation and reuse failures to stable UHP errors with the offending + parameter named. Never expose exceptions or internal paths. +9. When accepted upstream, update the specification, machine schema, reference + implementation, conformance suite, changelog, and docs in the same change + set. Otherwise update the fork implementation, extension tests, changelog, + and docs together without altering UHP conformance definitions. Acceptance examples: | Case | Observable result | |---|---| -| Valid URL only | Accepted; checkout root becomes effective working directory | +| Valid URL only | Accepted; checkout root becomes the effective working directory | | Valid URL, ref, and nested directory | Accepted; values reach the single workspace hook | -| Flat `metadata["allagents.workspace"]` | Rejected as not the v1 extension | | Missing URL, unknown field, array, or non-string field | 400 before materialization | -| Absolute or parent-traversing working directory | 400 before materialization | -| Continuation containing a workspace object | 409 before materialization | +| Absolute or parent-traversing `working_directory` | 400 before materialization | +| Continuation containing `metadata.workspace` | 409 before materialization | | Existing `metadata.session_id` plus workspace object | 409 before materialization | | Workspace-bound session with a different `harness_id` | 409 before hydration | -| No workspace object | Same status, response, and runner path as pinned upstream | +| No workspace object | Same status, response, and runner path as upstream baseline | -Tests must assert the HTTP/UHP contract and absence of materializer/provider -activity, not internal helper calls or field-copy plumbing. +Tests assert the HTTP/UHP contract and absence of materializer/provider activity, +not helper calls or field-copy plumbing. -### Phase 3: Add the workspace-aware hydrate and runner seam +### Phase 4: Land runner and materializer seams in the selected ownership path -**Outcome:** one workspace-specific path prepares the runner allocation after -session resolution and before provider or harness execution, while ordinary -sessions keep the pinned path. +**Outcome:** HarnessRouter can prepare a workspace after session resolution and +before provider execution while preserving its ordinary path. Primary areas: `gateway/app.py`, `runner/server.py`, their existing transport, -and focused integration tests. +the materializer package, focused integration tests, and operator docs. These +land upstream when accepted and otherwise remain an explicit fork patch. Work: 1. Extend the existing gateway-to-runner turn envelope with an optional canonical - workspace descriptor for a first turn and an optional hydrated workspace - binding for a continuation. Do not add a public endpoint. -2. Add a workspace-aware first-hydrate mode that performs the existing - isolation/wipe and ownership setup but leaves an empty session root with an - absent checkout child. It must not run stock `_git_ensure`, write - `.gitignore`, apply input files, or create harness state before publication. + first-turn workspace descriptor and an optional hydrated binding for a + continuation. Do not add a public endpoint. +2. Add a first-hydrate mode that performs existing isolation, wipe, and ownership + setup but leaves an empty session root with checkout absent. It must not run + stock Git initialization, write `.gitignore`, apply input files, or create + harness state before publication. 3. Define one runner-owned materializer interface with two operations: - `materialize(firstTurnDescriptor, checkoutTarget, limits, cancellation)` and - `cleanup(binding)`. There is one configured implementation in v1 and no - registration mechanism. -4. Keep four distinct values in the binding: session root, checkout root, - control root, and execution working directory. The execution directory must - be a no-follow validated descendant of the checkout; the control root must be - a sibling outside repository content. -5. After materialization succeeds, complete the - pending-reservation/publication transition, create the control root, then - atomically commit the usable binding and public provenance. -6. On continuation, hydrate the bound session root and verify all stored roots - before use. Do not resolve, fetch, checkout, or validate caller workspace - metadata again. -7. Adapt the existing runner plumbing to explicit roots: spawn the harness in - the execution working directory; anchor durable HOME, scratch, skills, and - CLI state in the control root; and scope input files, produced-file Git - diffing, file APIs, and artifacts to the checkout. Generate credential-bearing - CLI config only in a per-turn ephemeral control subtree, delete it before - checkpointing, and exclude it defensively from checkpoint and public-file - walkers. Checkpoint the remaining session root. Do not initialize an outer - Git repository or overwrite the source repository's `.gitignore`. -8. Make the first-turn transition idempotent. Same-key replay returns the owning - response/binding. A competing request cannot materialize or bind a second + `materialize(first_turn_descriptor, checkout_target, limits, cancellation)` + and `cleanup(binding)`. Use one configured implementation; do not add plugin + discovery, registration, or hook chaining. +4. Preserve the four explicit root values in the binding. Validate the execution + working directory without following links; keep control as a sibling of + checkout. +5. After materialization, finish the pending-reservation/publication transition, + create control, then atomically commit the binding and public provenance. +6. On continuation, hydrate and verify the stored roots and identity. Do not + resolve, fetch, checkout, or validate caller workspace metadata again. +7. Spawn the harness in the execution working directory. Anchor durable HOME, + scratch, skills, and CLI state in control. Scope input files, produced-file + Git diffing, file APIs, and artifacts to checkout. Checkpoint the session root + only after removing ephemeral credential state. Do not initialize an outer + repository or overwrite the source repository's `.gitignore`. +8. Make first-turn transition idempotent. Same-key replay returns the owning + response and binding. A competing request cannot materialize or bind a second checkout for the same session. -9. Preserve pinned behavior for non-workspace requests and unbound - continuations. Preserve existing streaming, cancellation, terminal-state, - and provider-error ordering around the workspace path. -10. Ensure workspace failures terminate before the provider receives a request. +9. Preserve streaming, cancellation, terminal-state, idempotency, artifact, and + provider-error ordering. Workspace failure terminates before provider traffic. +10. Implement typed materializer request, result, and error records. Results + include internal roots plus safe public provenance; errors contain neither + secrets nor uncontrolled Git output. +11. Canonicalize and authorize the HTTPS URL before Git. Use one acquisition path + for DNS, redirects, IP policy, ref advertisement, and fetch. +12. Resolve omitted ref through the advertised symbolic default. Resolve full + branch/tag refs or unambiguous shorthand from advertised SHA-1 refs. Reject + raw object IDs, other namespaces, SHA-256 repositories, missing refs, and + ambiguous shorthand. Peel annotated tags and require a commit. +13. Create restrictive random staging as a sibling of the absent checkout target. + Fetch only the advertised ref needed for the resolved commit, disable helper + execution, and check out detached with isolated Git configuration. +14. Reject submodule entries and LFS-managed content. Validate confinement, + bytes, inodes, `HEAD`, and optional execution directory. +15. Atomically publish validated staging to checkout. Never write control or + derive a host path from URL, ref, working directory, response ID, or other + caller text. +16. Remove staging on every error, timeout, cancellation, and recovery sweep. + Bound concurrency with one server-owned semaphore and emit stable staged + errors with capped private stderr. +17. Document the runner/materializer contract and configuration in the owning + HarnessRouter changelog and operator docs; include it in UHP documentation + only when accepted as standard behavior. -The seam is intentionally workspace-specific. It does not generalize arbitrary -metadata into runner callbacks and does not introduce extension discovery, -capabilities negotiation, plugin loading, or hook chaining. - -Acceptance examples: - -- a probe harness sees a committed file from the repository on its first - instruction; -- a nested `workingDirectory` becomes process cwd while HOME and `.harness` - state remain outside the checkout; -- source-controlled `.harness` paths and `.gitignore` cannot collide with or - rewrite runner state; -- materializer failure produces no provider request, agent process, or published - checkout; -- restart and continuation restore both checkout mutations and control state; -- same-key replay owns one checkout; and -- an ordinary non-workspace request exercises the unchanged upstream sequence. - -### Phase 4: Implement the Git materializer - -**Outcome:** the in-image materializer produces one verified private checkout and -returns a binding only after all safety and resource checks pass. - -Suggested home: a small `runner/allagents_workspace/` package plus a single -in-image executable entry point. Reuse HarnessRouter's process, cancellation, -logging, and session identity primitives instead of creating a daemon. - -Work: - -1. Define typed internal request/result/error records matching the runner seam. - The result contains the internal checkout root, validated execution directory, - and safe public provenance; errors never contain secrets or uncontrolled Git - output. -2. Canonicalize and authorize the HTTPS URL before running Git. Implement the - DNS, redirect, IP-range, and protocol rules as one acquisition path used by - ref resolution and fetch. -3. Resolve omitted ref through the remote symbolic default and requested - `refs/heads/*`, `refs/tags/*`, or unambiguous branch/tag shorthand through - advertised SHA-1 refs. Reject raw object IDs, other ref namespaces, SHA-256 - repositories, missing refs, and ambiguous shorthand. Peel an annotated tag - and verify the final object is one commit. -4. Create staging as a sibling of the absent runner-assigned checkout target, - with a random unguessable component and restrictive ownership/mode. The - caller never influences a host path. -5. Run Git with isolated config and environment. Fetch only the advertised ref - needed for the resolved commit, disable helper execution, and check out - detached. -6. Reject submodule entries and LFS-managed content. Validate tree confinement, - materialized byte/inode limits, checkout `HEAD`, and optional execution - working directory. -7. Atomically rename validated staging to the checkout target and return the - result. Never write the sibling control root or derive a filesystem path - directly from a URL, ref, working directory, response ID, or caller string. -8. Remove staging on every error, timeout, cancellation, or crash-recovery sweep. - A failed attempt cannot become a resumable checkout. -9. Bound concurrent materializations with one server-owned semaphore. Saturation - fails or waits only within the request deadline; it never creates unbounded - processes. -10. Emit structured stage/error codes for URL policy, ref resolution, fetch, - source feature rejection, limits, validation, publication, and cancellation. - Keep stderr capped and private. - -Behavior-focused tests: - -- exact commit checkout when a branch advances between later requests; -- omitted-ref resolution to the advertised default branch; -- safe nested working directory and rejection of file/nonexistent/escaping paths; -- rejection of raw object IDs and SHA-256 repositories; -- a nested working directory that cannot move control state into repository - content; -- redirect and DNS rebinding attempts into forbidden address ranges; -- forbidden Git protocols, embedded credentials, hooks, filters, submodules, and - LFS content; -- byte, inode, time, output, and concurrency limits during materialization; -- cancellation kills Git descendants and removes staging; and -- two sessions requesting the same commit receive different writable roots and - cannot observe each other's mutations. - -Use controlled test origins/resolvers for adverse network cases and one stable -public fixture repository for built-image proof. Tests must observe files, -commit identity, isolation, errors, and process termination rather than mock -argument forwarding. - -### Phase 5: Wire Codex and OMP to the external provider - -**Outcome:** both required harnesses execute through the operator's single -OAuth-to-OpenAI-compatible gateway without exposing or changing its -credentials. - -Primary areas: existing HarnessRouter provider connections/policies, custom -harness definitions, image runtime installation, and focused runner tests. - -Work: - -1. Install exact pinned Codex and OMP versions in the image and enable only the - required release backends with `HR_BACKENDS=codex,omp`. -2. Define stable custom harness IDs, for example `allagents-codex` and - `allagents-omp`, using HarnessRouter custom harness definitions. Store their - instructions, tools, allowed models, and base harness in gateway-owned - configuration. -3. Configure two logical HarnessRouter connections from the same server-side - external-gateway base URL and API-key secret: `responses` for Codex and - `openai` Chat Completions for OMP. Give each harness policy a one-entry chain - containing only its matching connection. -4. Force Codex onto the Responses connection and OMP onto the Chat Completions - connection. Fail startup/readiness if either selected model and endpoint - combination is unsupported. Never switch transport or connection after an - error. -5. Override the self-host image's owner-trust default with - `HR_SANDBOX_TRUST=broker`. Make the existing local - `HARNESS_GATEWAY_URL` satisfy broker availability in self-host mode without - requiring a publicly reachable gateway URL. -6. Add a readiness probe that mints a scoped turn credential, reaches the - loopback broker, and proves the real external-gateway key remains gateway-side. -7. Reject caller-selected models outside the harness allowlist and any request - field that attempts to replace provider routing. -8. Confirm that OMP receives an ordinary session-local home/config and cwd. It - must not read AllAgents local profiles or host configuration. Generate its - credential-bearing `models.json` and `models.yml` under a per-turn ephemeral - control subtree using only the scoped broker token, then delete both before - terminal checkpointing. Add explicit checkpoint and public-file exclusions - for the OMP paths as defense in depth. -9. Redact both the external-gateway key and scoped broker tokens from logs, - traces, stored responses, session files, checkpoints, artifacts, and test - snapshots. - -Acceptance examples: - -- Codex completes a real Responses turn through the external gateway; -- OMP completes a real Chat Completions turn through the external gateway; -- each harness starts inside the materialized working directory; -- an unsupported model fails before provider traffic; -- an invalid provider credential returns the upstream provider failure without - selecting another connection; and -- callers cannot observe or override base URL, API key, or transport. - -### Phase 6: Complete continuation, cancellation, and cleanup +Behavior-focused proof: -**Outcome:** the checkout lifecycle follows the existing session lifecycle with -bounded ephemeral storage and no separate lifetime platform. +- a probe harness sees a committed repository file on its first instruction; +- `working_directory` becomes process cwd while HOME and runner state remain + outside checkout; +- repository `.harness` paths and `.gitignore` cannot collide with control state; +- exact commit remains fixed if the branch later advances; +- omitted ref selects the advertised default branch; +- raw object IDs, SHA-256 repositories, escaping paths, forbidden destinations, + protocols, credentials, hooks, filters, submodules, and LFS content fail; +- byte, inode, time, output, redirect, and concurrency limits apply during work; +- materializer failure produces no provider request, agent process, binding, or + published checkout; +- cancellation kills Git descendants and removes staging; +- restart and continuation restore checkout mutations and control state; +- two sessions at the same commit receive distinct writable roots; and +- ordinary requests exercise the unchanged upstream sequence. + +Use controlled origins and resolvers for hostile network cases and a stable +public fixture repository for built-image proof. Tests observe files, commit +identity, isolation, errors, and process termination rather than mock argument +forwarding. + +### Phase 5: Complete continuation and cleanup lifecycle + +**Outcome:** checkout lifecycle follows the existing session lifecycle with +bounded ephemeral storage and crash-safe recovery in the selected ownership +path. Work: -1. Add a server-owned `ALLAGENTS_WORKSPACE_TTL_SECONDS` with a finite safe - default. Callers cannot set or extend it directly. -2. Start/reset the idle expiry only after a terminal turn is durably recorded. - An active materialization or harness turn is not removed by the idle sweeper. -3. On a valid continuation, verify the stored checkout identity/root and use the - exact prior working directory and mutations. Reset the idle deadline only - after that turn reaches a terminal state. +1. Add the server-owned idle TTL configuration selected through review, with a + finite safe default. Callers cannot set or extend it. +2. Start or reset idle expiry only after a terminal turn is durably recorded. An + active materialization or harness turn is never removed by the sweeper. +3. On continuation, verify stored identity, contract owner and revision, and all + roots, then use the same working directory and mutations. Reset expiry only + after the turn becomes terminal. 4. On first-turn cancellation during materialization, terminate the process group, remove staging, and leave no resumable binding. -5. On cancellation after publication, stop the harness through stock - HarnessRouter behavior and retain the bound checkout only until the ordinary - finite idle deadline, so a permitted continuation sees prior mutations. -6. On expiry or explicit existing-session deletion, mark the binding unavailable - in the session transaction and invoke idempotent confined cleanup. A late - continuation fails closed and cannot recreate the checkout. -7. On startup, remove abandoned staging, pending reservations, +5. On cancellation after publication, stop the harness through ordinary + HarnessRouter behavior and retain the checkout only until finite idle expiry, + allowing a permitted continuation to see prior mutations. +6. On expiry or existing-session deletion, atomically mark the binding + unavailable before confined idempotent cleanup. A late continuation fails + closed and cannot recreate checkout. +7. On startup, reconcile abandoned staging, pending reservations, published-but-unbound roots, and cleanup-marked session roots. Do not add a - second database, durable queue, elaborate deletion ledger, or general storage - collector. -8. If cleanup encounters a transient host error, keep the binding unavailable, - report an operator-visible error, and retry the same idempotent removal on the - next bounded sweep/startup. Never make the checkout executable again. + second database, durable queue, deletion ledger, or general storage collector. +8. If cleanup encounters a transient host error, keep binding unavailable, emit + an operator-visible error, and retry the same idempotent removal on the next + bounded sweep or startup. Never make checkout executable again. +9. Before activating an incompatible workspace contract, explicitly migrate + compatible bindings or drain and delete them. Unsupported revisions fail + closed; do not add aliases or reinterpret stored descriptors. +10. Add lifecycle behavior to UHP conformance only where accepted and + protocol-visible. Always cover implementation behavior with focused + HarnessRouter integration tests in the owning repository. Acceptance examples: @@ -629,24 +652,65 @@ Acceptance examples: |---|---| | Turn 1 edits a file; turn 2 reads it | Turn 2 sees the edit in the same checkout | | Turn 2 omits workspace metadata | Stored binding selects checkout and cwd | -| Turn 2 includes workspace metadata | Rejected before Git/provider activity | +| Turn 2 includes workspace metadata | Rejected before Git or provider activity | | Checkout missing or identity mismatched | Continuation fails; no rematerialization | | Git cancellation | Child processes exit and staging disappears | -| Harness cancellation | Response is cancelled; no process remains; checkout follows finite idle expiry | -| Expiry races with continuation | Exactly one wins through existing session serialization; checkout is never used after cleanup begins | -| Cleanup called twice or after restart | Same final absent state; no neighboring path changes | +| Harness cancellation | Response is cancelled; checkout follows finite idle expiry | +| Expiry races with continuation | Existing session serialization selects one winner; checkout is not used after cleanup begins | +| Cleanup runs twice or after restart | Same absent final state; no neighboring path changes | + +### Phase 6: Add downstream Codex, OMP, and provider wiring + +**Outcome:** the fork supplies AllAgents deployment policy while consuming the +accepted upstream capability or its explicitly documented downstream workspace +patch unchanged. + +Work in `allagentsdev/harnessrouter`: + +1. Install exact pinned Codex and OMP releases and enable only required release + backends with `HR_BACKENDS=codex,omp`. +2. Define stable custom harness IDs such as `allagents-codex` and + `allagents-omp`. Store instructions, tools, allowed models, and base harness + in downstream deployment configuration. +3. Configure two logical connections using the same external-gateway base URL + and API-key secret: Responses for Codex and OpenAI Chat Completions for OMP. + Give each harness policy one matching connection and no fallback. +4. Fail startup/readiness if either selected model and endpoint combination is + unsupported. Never switch transport or connection after an error. +5. Force `HR_SANDBOX_TRUST=broker`. Configure the local loopback broker so + self-host mode does not require a public broker URL. +6. Add readiness proof that mints a scoped turn credential, reaches the loopback + broker, and keeps the real external-gateway key gateway-side. +7. Reject caller-selected models outside the harness allowlist and any request + field that attempts to replace provider routing. +8. Give OMP ordinary session-local home/config and cwd. Generate its + credential-bearing `models.json` and `models.yml` only under a per-turn + ephemeral control subtree using the scoped token. Delete both before terminal + checkpointing and exclude their paths from checkpoints and public walkers. +9. Redact the external key and scoped tokens from logs, traces, stored responses, + session files, checkpoints, artifacts, and snapshots. + +Acceptance examples: + +- Codex completes a real Responses turn through the external gateway; +- OMP completes a real Chat Completions turn through the same external gateway; +- each harness starts inside the materialized execution working directory; +- unsupported model fails before provider traffic; +- invalid provider credentials return the ordinary provider failure without + selecting another connection; and +- callers cannot observe or override provider URL, key, or transport. ### Phase 7: Build the one-container operator surface -**Outcome:** an operator can start the built AllAgents Gateway image with Docker -Compose, one data volume, and no helper service. +**Outcome:** an operator can start the AllAgents-maintained HarnessRouter image +with Docker Compose, one data volume, and no helper service. The checked-in Compose contract is equivalent to: ```yaml services: - allagents-gateway: - image: ghcr.io/allagentsdev/allagents-gateway:${ALLAGENTS_GATEWAY_VERSION}@${ALLAGENTS_GATEWAY_DIGEST} + harnessrouter: + image: ghcr.io/allagentsdev/harnessrouter:${HARNESSROUTER_IMAGE_TAG}@${HARNESSROUTER_IMAGE_DIGEST} ports: - "127.0.0.1:3000:3000" env_file: @@ -654,89 +718,87 @@ services: environment: HR_BACKENDS: codex,omp HR_SANDBOX_TRUST: broker - ALLAGENTS_WORKSPACE_TTL_SECONDS: ${ALLAGENTS_WORKSPACE_TTL_SECONDS:-3600} + HR_WORKSPACE_TTL_SECONDS: ${HR_WORKSPACE_TTL_SECONDS:-3600} volumes: - - allagents-gateway-data:/data + - harnessrouter-data:/data restart: on-failure volumes: - allagents-gateway-data: + harnessrouter-data: ``` -The real file must also carry HarnessRouter's required authentication, secret, +The final environment key uses the upstream-approved name when accepted and the +documented fork name otherwise. The actual Compose file also carries +HarnessRouter's required caller authentication, secret, health, and process settings. The operator supplies one external provider base -URL and API key through two protocol-specific HarnessRouter connection records; -the Compose file does not bake either value into the image. +URL and key through two protocol-specific connection records; neither value is +baked into the image. Startup contract: 1. Copy the example environment file and set non-default Console/caller - credentials, the external provider route, model allowlists, TTL/resource - limits, and an immutable image version plus manifest digest. + credentials, provider route, model allowlists, TTL/resource limits, immutable + image tag, and manifest digest. 2. Run `docker compose up -d`. -3. Readiness succeeds only after the gateway, runner, enabled Codex/OMP runtimes, - materializer executable, writable `/data`, custom harnesses, both - protocol-specific provider connections, and loopback credential broker pass - startup checks. Owner-trust credential pass-through fails readiness. -4. The public UHP base remains HarnessRouter's existing +3. Readiness succeeds only after HarnessRouter, runner, Codex/OMP runtimes, + materializer, writable `/data`, custom harnesses, both provider connections, + and loopback credential broker pass startup checks. Owner-trust pass-through + fails readiness. +4. The UHP base remains HarnessRouter's existing `http://127.0.0.1:3000/api/harness`; remote access requires operator-owned TLS and network controls. -5. Restarting the container with the same volume preserves unexpired - HarnessRouter sessions and their private checkouts. Startup reconciliation - removes only abandoned staging or cleanup-marked roots. +5. Restart with the same volume preserves unexpired sessions and private + checkouts. Reconciliation removes only abandoned or cleanup-marked data. Container requirements: - the materializer is installed in the same image and invoked locally; - Git and certificate roots are pinned and present; -- the service binds to loopback by default; +- service binds to loopback by default; - `/data` is the only required durable mount; -- secrets are runtime inputs, not layers, labels, build args, or example values; -- the image has a standard SBOM and build provenance; and -- image labels record gateway version, source revision, upstream commit, and UHP - version. +- secrets are runtime inputs, not layers, labels, build arguments, or examples; +- image has standard SBOM and build provenance; and +- OCI labels record source revision, upstream tag and commit, downstream + revision, UHP release, and runtime releases. ### Phase 8: Add direct Promptfoo smoke and E2E coverage -**Outcome:** the lockfile-pinned Promptfoo installation calls the built image -directly over UHP and proves user-visible workspace behavior. +**Outcome:** lockfile-pinned Promptfoo calls the built image directly over UHP +and proves user-visible workspace behavior. -Suggested area: `e2e/promptfoo/` in `allagentsdev/allagents-gateway`, containing -only Promptfoo config, small fixtures/assertions, and package metadata. +Suggested downstream area: `e2e/promptfoo/` in +`allagentsdev/harnessrouter`, containing only Promptfoo configuration, small +fixtures/assertions, and package metadata. Work: 1. Configure Promptfoo's OpenAI Responses-compatible provider directly against `/api/harness/v1/responses`, with the HarnessRouter API key in an environment - variable and nested workspace metadata in the request. Do not place another - HTTP adapter or repository between Promptfoo and the gateway. -2. Use a stable public Git fixture with known commits and a task whose answer or - file change can only succeed if the harness started in the materialized - checkout. + variable and `metadata.workspace` in the request. Do not place an HTTP adapter + or another repository between Promptfoo and HarnessRouter. +2. Use a stable public Git fixture with known commits and a task that succeeds + only when the harness starts in the materialized checkout. 3. Run the same first-turn scenario for `allagents-codex` and `allagents-omp`, - with each harness's allowed model and configured provider transport. -4. Include a two-turn scenario that mutates a uniquely named file on turn one - and reads/changes it on turn two via `previous_response_id` without resending - workspace metadata. -5. Include negative cases for malformed metadata, forbidden URL resolution, - missing ref, invalid working directory, materialization limit, provider - failure, cancellation during Git, cancellation during harness execution, and - continuation after cleanup. - Include reused-session workspace injection and cross-harness continuation - mismatch cases. -6. Inspect public response metadata to verify the expected resolved commit and - absence of checkout paths and credentials. - Use canary values for the external-gateway key and scoped broker token and - assert both are absent from session files, checkpoints, artifacts, logs, and - reports after each turn. -7. Exercise the built image, not an in-process gateway. Store only sanitized - reports; do not upload provider traffic, prompts containing secrets, or data - volume contents. - -Direct smoke/E2E is the proof of the feature. Focused permanent tests remain only -where they protect schema boundaries, ordering, security invariants, session -transitions, and cleanup races. Do not add tests that merely assert config keys, -field copies, mocks, or source text. + using each harness's allowed model and provider transport. +4. Add a two-turn scenario that mutates a uniquely named file on turn one and + reads or changes it on turn two through `previous_response_id`, without + resending workspace metadata. +5. Cover malformed metadata, forbidden URL resolution, missing ref, invalid + working directory, source limit, provider failure, cancellation during Git, + cancellation during harness execution, continuation after cleanup, + reused-session workspace injection, and cross-harness mismatch. +6. Assert the expected resolved commit in public provenance and absence of + internal paths and credentials. +7. Use canary external keys and scoped tokens; assert both are absent from + session files, checkpoints, artifacts, logs, reports, and data retained after + each turn. +8. Exercise the built image, not an in-process server. Store only sanitized + reports; never upload provider traffic, secret-bearing prompts, or volume + contents. + +Focused permanent tests protect schema boundaries, security invariants, +ordering, session transitions, and cleanup races. Do not add tests that merely +assert config keys, field copies, mocks, or source text. Release-blocking E2E matrix: @@ -745,143 +807,195 @@ Release-blocking E2E matrix: | Codex / Responses | required | required | required | required | | OMP / Chat Completions | required | required | required | required | -### Phase 9: Conformance, upstream maintenance, and release +### Phase 9: Integrate the upstream baseline and release the downstream image -**Outcome:** the downstream preserves stock HarnessRouter behavior and publishes -a reproducible, independently versioned image. +**Outcome:** the fork consumes an exact upstream baseline, carries only the +workspace delta required by the recorded ownership decision, preserves ordinary +HarnessRouter behavior, and publishes a reproducible image. Work: -1. Run the complete UHP `2026-09-12` conformance suite against the built image. - Workspace tests supplement it; they do not replace or relax upstream cases. -2. Run upstream HarnessRouter integration coverage for gateway, runner, Codex, +1. Confirm the upstream decision. If accepted, verify that the + UEP/specification, schema, reference implementation, conformance, changelog, + and docs landed together. If declined or deferred, record that decision and + the exact downstream workspace commits. +2. Advance the fork in a standalone synchronization change to the exact upstream + release tag and commit selected as the baseline. Rebase or replay the + downstream workspace and deployment commits separately. +3. Run the complete selected UHP conformance suite against the built image. + Downstream scenarios supplement it; they do not replace or exclude upstream + cases. +4. Run upstream HarnessRouter integration coverage for gateway, runner, Codex, OMP, sessions, streaming, cancellation, idempotency, files, artifacts, and - non-workspace requests. -3. Run the direct Promptfoo E2E matrix against the exact image candidate. -4. Review the downstream diff against - `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`. Keep the seam patch separable - from the AllAgents schema/materializer implementation so generally useful - changes can be proposed upstream without blocking release. -5. Build and publish the v1 `linux/amd64` image as - `ghcr.io/allagentsdev/allagents-gateway:`, attach standard - SBOM/provenance, and record its manifest digest. Additional architectures are - separate release work after v1. -6. Verify a clean Compose deployment using that digest, a fresh volume, both - harnesses, a first turn, a continuation, cancellation, expiry, restart, and + ordinary requests. +5. Run the direct Promptfoo matrix against the exact image candidate. +6. Review the fork diff against its recorded upstream baseline. It must contain + only the declared workspace extension when needed, deployment defaults, + custom harness definitions, provider wiring, Promptfoo scenarios, and image + publication changes. Reusable protocol or runner improvements continue to be + proposed upstream. +7. Tag source and image using `-allagents.`, publish + `ghcr.io/allagentsdev/harnessrouter:`, attach SBOM/provenance, and record + the manifest digest. The first release targets `linux/amd64`; additional + architectures are separate work. +8. Verify a clean Compose deployment using the digest, a fresh volume, both + harnesses, first turn, continuation, cancellation, expiry, restart, and cleanup. -7. Publish release notes containing gateway version, source commit, upstream - baseline, UHP version, pinned Codex/OMP versions, Promptfoo version, image - digest, known limitations, and upgrade/rollback instructions. -8. For future upstream updates, first merge/rebase the new upstream baseline as - its own change, rerun conformance and built-image E2E, then replay or revise - the narrow downstream patches. Never mix an upstream baseline jump with a - product behavior change. +9. Publish release notes containing upstream issue/UEP links, UHP release, + upstream tag and commit, downstream revision and source commit, pinned + Codex/OMP and Promptfoo releases, image digest, known limitations, and + upgrade/rollback instructions. +10. For future updates, synchronize the new upstream baseline alone, rerun + conformance and built-image E2E, then replay or revise the declared + downstream workspace and deployment patches. Never mix baseline movement + with product behavior. Release gate: -- all required focused tests pass; -- UHP conformance passes without exclusions introduced by this work; +- the upstream ownership decision is recorded and all deliverables for the + selected path are complete; +- all focused tests pass; +- UHP conformance passes without downstream exclusions; - built-image Codex and OMP first-turn/continuation E2E passes through the external provider gateway; -- security failure and cancellation cases leave no child process or staging - directory; -- the external-gateway key is absent from harness environments, and both it and - scoped broker tokens are absent from persisted or public session surfaces, - while both protocol-specific broker routes succeed; -- expiry and cleanup make the checkout unavailable and remove it idempotently; -- ordinary non-workspace HarnessRouter requests remain compatible; -- image digest, SBOM, provenance, pins, and downstream diff are available; and -- the Compose smoke succeeds from a fresh checkout and fresh `/data` volume. +- security failures and cancellation leave no descendant or staging directory; +- the external key is absent from harness environments, and it plus scoped + tokens are absent from persistent and public surfaces while both broker routes + succeed; +- expiry and cleanup make checkout unavailable and remove it idempotently; +- ordinary requests remain upstream-compatible; +- the fork diff contains only the declared downstream surface, including the + workspace extension when upstream did not accept it; +- digest, SBOM, provenance, pins, baseline, and downstream revision are + available; and +- Compose smoke succeeds from a fresh checkout and `/data` volume. ## Error contract -Use stable, stage-oriented detail codes under HarnessRouter's existing UHP error -shape. Final names should follow upstream conventions, but the observable -categories are fixed: +The selected workspace contract defines stable, stage-oriented detail codes +under HarnessRouter's UHP error shape. When accepted upstream, the UEP, +specification, schema where applicable, reference implementation, conformance +expectations, and docs agree on these observable categories. Otherwise the fork +implementation, extension tests, changelog, and docs agree without claiming UHP +standardization: | Condition | HTTP class | Retry guidance | |---|---:|---| | Invalid workspace JSON or path | 400 | Caller must change request | -| Workspace supplied for any reused session | 409 | Caller must omit workspace | -| URL/ref/source feature rejected | 400 | Caller must change source | -| Forbidden DNS/redirect destination | 400 | Caller or operator must change source/network policy | -| Materialization exceeds a fixed source limit | 413 | Caller must choose a smaller repository | -| Materializer concurrency unavailable | 503 | Retry with backoff within caller deadline | -| Git/network timeout before binding | 504 | Retry creates a new first-turn attempt under normal idempotency rules | -| Materialization cancelled | Existing UHP cancelled outcome | Do not retry under the cancelled response ID | -| Bound checkout missing, expired, or cleanup-started | 410 | Start a new session with a new workspace request | -| Provider unavailable or rejects credentials | Existing upstream provider error | Repair external provider gateway; no route fallback | +| Workspace supplied for a reused session | 409 | Caller must omit workspace | +| URL, ref, or source feature rejected | 400 | Caller must change source | +| Forbidden DNS or redirect destination | 400 | Caller or operator must change source/network policy | +| Materialization exceeds fixed source limit | 413 | Caller must choose a smaller repository | +| Materializer concurrency unavailable | 503 | Retry with backoff inside caller deadline | +| Git/network timeout before binding | 504 | Retry through ordinary idempotency rules | +| Materialization cancelled | Existing UHP cancelled outcome | Do not retry under cancelled response ID | +| Bound checkout missing, expired, cleanup-started, or on an unsupported contract revision | 410 | Start a new session with a new workspace request | +| Provider unavailable or rejects credentials | Existing provider error | Repair external provider gateway; no route fallback | A failure before publication exposes no checkout identity or provenance. A -failure after a binding exists may include the already committed public -provenance, but never internal paths, Git stderr, DNS details that disclose -private topology, or provider secrets. +failure after binding may include already committed public provenance, but never +internal paths, uncontrolled Git stderr, private network topology, or provider +secrets. ## Configuration ownership | Value | Owner | Caller-overridable? | |---|---|---:| -| Repository URL, optional ref, optional working directory | Initial UHP request | yes, within strict schema/policy | -| Harness ID and allowed model | New-session request constrained by server definition; stored workspace-bound session on reuse | only among configured values on creation; mismatch rejected only for workspace-bound reuse | -| External provider base URL/API key | Operator secret configuration | no | -| Codex/OMP provider transport | Operator harness definition | no | -| Materializer executable | Image/operator startup configuration | no | -| DNS/redirect/Git restrictions | Image and operator policy | no weakening by caller | -| Byte/inode/time/concurrency limits | Operator within image-safe bounds | no | +| Repository URL, optional ref, optional working directory | Initial UHP request | yes, within schema and policy | +| Harness ID and allowed model | New-session request constrained by server definition; stored state on reuse | only among configured values on creation | +| External provider base URL and API key | Operator secret configuration | no | +| Codex/OMP provider transport | Downstream harness definition | no | +| Materializer implementation | HarnessRouter image/runtime configuration in the owning upstream or fork implementation | no | +| DNS, redirect, and Git restrictions | HarnessRouter safe defaults plus operator policy | no weakening by caller | +| Byte, inode, time, and concurrency limits | Operator within image-safe bounds | no | | Idle checkout TTL | Operator within image-safe bounds | no | -| Checkout path/identity | Runner | no | +| Session, checkout, control, and execution paths | Runner | no | | Resolved commit | Materializer observation | no | +| Workspace contract owner and revision | HarnessRouter release and session binding | no | ## Delivery sequence and ownership -A practical implementation sequence inside `allagentsdev/allagents-gateway` is: - -1. Repository maintainer performs the fork rename and establishes release/image - permissions and pins. -2. Gateway owner implements the exact nested schema, canonical binding input, - continuation rejection, and public error mapping. -3. Runner owner implements the single seam, first-turn/continuation ordering, and - cancellation propagation. -4. Materializer owner implements secure Git acquisition, validation, - publication, and cleanup under the runner contract. -5. Harness owner pins Codex/OMP, configures the two protocol adapters to the - single external provider route, and defines the custom harnesses. -6. Container owner integrates the materializer, `/data` layout, readiness, - Compose contract, and standard supply-chain outputs. -7. E2E owner adds the lockfile-pinned direct Promptfoo matrix and built-image - smoke. -8. Release owner runs upstream conformance, reviews the downstream diff, and - publishes the independently versioned digest. - -Schema and seam contracts must merge before materializer and E2E work depend on -them. Git security, provider wiring, and container work can proceed in parallel -once those contracts are frozen. No implementation phase requires a change in -the AllAgents CLI repository. +A practical sequence is: + +1. Protocol owner opens the upstream issue, drives the UEP when required, and + records the governance decision or documented timed deferral before + substantial implementation. +2. Fork maintainer pins the examined baseline, supply chain, image publication, + and baseline-plus-revision release convention in + `allagentsdev/harnessrouter`. +3. If upstream accepts ownership, upstream protocol and runner owners land the + coordinated specification, schema, parsing, session rules, root separation, + materialization, lifecycle, errors, conformance, changelog, and docs. +4. If upstream declines or defers ownership, fork owners land the same bounded + implementation with separate extension tests, changelog, and docs, and record + every downstream workspace commit against the upstream baseline. +5. Downstream harness owner adds Codex/OMP definitions and two logical provider + connections to the one external route. +6. Downstream container owner adds deployment defaults, readiness, `/data`, + Compose, OCI labels, SBOM, and provenance. +7. Downstream E2E owner adds lockfile-pinned direct Promptfoo scenarios and + built-image smoke. +8. Release owner proves the recorded upstream baseline and declared downstream + diff, runs UHP conformance plus extension E2E, then publishes the tagged + digest. + +Upstream runner code depends only on an accepted upstream contract. The fork may +implement after the upstream ownership decision and must keep downstream +extension coverage separate from UHP conformance. Materializer security and +runner lifecycle may proceed in parallel after the selected contract freezes. +Provider wiring, downstream container work, and Promptfoo scenario authoring may +then proceed against that contract. No phase changes the AllAgents CLI +repository. ## Definition of done -V1 is done when an operator can deploy one digest-pinned -`ghcr.io/allagentsdev/allagents-gateway` container with Docker Compose, configure -one external OAuth-to-OpenAI-compatible provider route through two -protocol-specific HarnessRouter connections and two custom harnesses, and have -lockfile-pinned Promptfoo directly: - -1. start a Codex or OMP UHP session with the exact - `metadata.allagents.workspace` object; -2. observe a securely resolved exact Git commit in public provenance; -3. run the harness inside a private editable checkout at the requested safe - directory; -4. continue through either supported session-reference path with the same - stored harness, mutations, and exact checkout; -5. cancel Git or harness work without leaked descendants or staging; -6. keep the long-lived external-gateway key out of harness environments and - persisted session data while both brokered protocol routes succeed; -7. receive bounded, stable failures without provider calls for invalid or unsafe - workspaces; -8. lose access after the server-owned expiry and see deterministic idempotent - cleanup; and -9. run stock non-workspace UHP requests with upstream-compatible behavior. - -No gateway implementation code, CLI command, profile migration, or -`workspace.yaml` change is required in `allagentsdev/allagents` to satisfy this -definition. \ No newline at end of file +This work is done when: + +1. The upstream issue and required UEP have a recorded decision. On acceptance, + the UHP specification, schema, HarnessRouter reference implementation, + conformance suite, changelog, and docs define the same generic + `metadata.workspace` contract. On rejection or deferral, the fork + implementation, extension tests, changelog, and docs define it consistently + without claiming UHP standardization. +2. `allagentsdev/harnessrouter` remains the GitHub fork and its release diff from + the recorded upstream baseline contains only the declared workspace extension + when needed, AllAgents deployment defaults, custom harness definitions, + provider wiring, Promptfoo scenarios, and image publication. +3. An operator can deploy one digest-pinned + `ghcr.io/allagentsdev/harnessrouter:-allagents.` + container with Docker Compose and one `/data` volume. +4. Promptfoo can directly start either custom harness with a public HTTPS Git + repository, optional advertised ref, and optional safe `working_directory`. +5. The public response reports the securely resolved exact commit without + internal paths, while the harness runs inside a private editable checkout. +6. Continuation through either supported session-reference path verifies the + recorded workspace-contract owner and revision, then reuses the same stored + harness, mutations, checkout, control state, and execution directory without + new Git work. +7. Session, checkout, control, and execution roots remain separated; staging is + invisible; binding publication and startup recovery are crash-safe. +8. Git and harness cancellation leak no descendants or staging, and expiry makes + the binding unavailable before deterministic idempotent cleanup. +9. Codex uses Responses and OMP uses Chat Completions through two logical + connections to one external OAuth gateway, with no fallback. +10. The long-lived external key never enters the harness, and neither it nor + scoped broker credentials survive in persisted or public surfaces. +11. Unsafe or invalid sources, limits, reused-session injection, missing + checkout, and provider failures return the agreed bounded errors without + violating ordering or making unintended provider calls. +12. Direct built-image Promptfoo coverage passes for both harnesses, first turn, + continuation, cancellation, provider failure, security negatives, restart, + expiry, and cleanup. +13. Ordinary requests remain upstream-compatible and the complete selected UHP + conformance suite passes without downstream exclusions. +14. Source tag, OCI tag, digest, SBOM, provenance, exact upstream tag/commit, + downstream revision/source commit, runtime pins, and release notes are + published and mutually consistent. +15. No implementation code, CLI command, profile migration, or `workspace.yaml` + change is required in `allagentsdev/allagents`. + +Upstream rejection or deferral does not rename the product or block the +distribution. It changes ownership: the workspace patch remains visible in +`allagentsdev/harnessrouter`, its release notes identify it as a downstream +HarnessRouter extension, and any later conflicting UHP standard triggers a +clean migration. From 58ed1e38bc960833758ad2af5f7900e7ec07c301 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sun, 27 Sep 2026 22:26:00 +1000 Subject: [PATCH 31/44] docs(architecture): restore mandatory OCI workspaces --- .../0002-adopt-uhp-through-harnessrouter.md | 492 +++-- ...0837-feat-coding-execution-gateway-plan.md | 1907 +++++++++-------- 2 files changed, 1329 insertions(+), 1070 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 9e586116..25d799d4 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -1,4 +1,4 @@ -# ADR 0002: Add workspace materialization to HarnessRouter +# ADR 0002: Initialize HarnessRouter session workspaces from Git or OCI - Status: Accepted - Date: 2026-09-21 @@ -6,61 +6,78 @@ ## Context -Promptfoo needs a remote coding-harness endpoint that can prepare a repository before a turn, preserve that checkout across continuations, and expose the result through the Unified Harness Protocol (UHP). HarnessRouter already owns the difficult execution-plane behavior: UHP requests and streams, sessions, cancellation, files, artifacts, caller authentication, harness processes, and custom harness configuration. Replacing that control plane would create a second implementation of behavior we already need. +Promptfoo needs a remote coding-harness endpoint that can prepare large repositories before the first turn, preserve their state across continuations, and expose exact source provenance through the Unified Harness Protocol (UHP). -The examined baseline is UHP [`2026-09-12`](https://github.com/HarnessRouter/harnessrouter/tree/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/versions/2026-09-12) at HarnessRouter commit [`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), released as [`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4). +HarnessRouter already owns the execution-plane lifecycle we need. Its runner materializes an existing per-session workspace either as a fresh workspace or by hydrating a checkpoint, and HarnessRouter already owns session identity, user and sandbox isolation, checkpoint transport, TTL, cancellation, files, artifacts, harness processes, and cleanup. The missing capability is narrower: on a first turn, initialize that runner-owned workspace from declared Git repositories or a verified OCI workspace snapshot before the harness starts. -UHP already reserves `metadata` for additive extensions. At the pinned commit, the request schema accepts an open metadata object ([UHP schema](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/schema/uhp-2026-09-12.openapi.yaml#L1051-L1057)). That is an existing protocol extension point, not an extension framework: stock HarnessRouter gives arbitrary metadata no runner semantics. It extracts only nested `metadata.systemone` ([gateway extraction](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/gateway/app.py#L7571-L7572)) and forwards only that probe to the runner ([runner handoff](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/gateway/app.py#L7008-L7011)). Workspace semantics therefore require coordinated protocol and implementation changes. +The examined baseline is UHP [`2026-09-12`](https://github.com/HarnessRouter/harnessrouter/tree/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/versions/2026-09-12) at HarnessRouter commit [`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), released as [`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4). UHP reserves `metadata` for additive extensions, but stock HarnessRouter gives arbitrary metadata no workspace-initialization semantics. The extension is therefore downstream HarnessRouter behavior until UHP governance standardizes an equivalent contract. -The open metadata object permits implementation-specific extensions, but it does not make their semantics part of UHP. `metadata.workspace` becomes a standard UHP contract only if accepted through upstream governance. Until then, the fork may expose it only as a documented HarnessRouter extension. The protocol or fork release already owns schema evolution, so the workspace object must not introduce a nested version field. +Large repositories are part of the minimum useful product, not a later optimization. Repository mode uses bounded shallow history to reduce Git history transfer, but that does not reduce a large working tree's transfer or extraction cost. OCI workspace snapshots and verified immutable-generation reuse solve that case and are therefore mandatory, release-blocking v1 capabilities alongside Git acquisition. ## Decision -We will keep [`allagentsdev/harnessrouter`](https://github.com/allagentsdev/harnessrouter) as the existing GitHub fork of [`HarnessRouter/harnessrouter`](https://github.com/HarnessRouter/harnessrouter), preserving its name, fork relationship, and history. We will not create a replacement repository, rename the fork, or add a new repository or CLI integration. +We will keep [`allagentsdev/harnessrouter`](https://github.com/allagentsdev/harnessrouter) as the existing fork of [`HarnessRouter/harnessrouter`](https://github.com/HarnessRouter/harnessrouter), preserving its name, fork relationship, and history. We will extend HarnessRouter's existing session workspace initialization path in that fork. We will not create a parallel checkout service, a second workspace abstraction, a replacement repository, or a new protocol. -Workspace materialization is a generic HarnessRouter capability and will be proposed upstream first. Before substantial implementation, we will open an upstream issue describing the use case, security model, request shape, lifecycle semantics, and conformance expectations. We will then follow HarnessRouter and UHP governance, including a UHP Enhancement Proposal (UEP) when required for protocol semantics. An accepted contribution must update the UHP specification, versioned schema, reference implementation, conformance coverage, changelog, and user/operator documentation together. If the issue receives no maintainer decision within 30 calendar days after opening and a follow-up is posted after day 14, the project records the proposal as deferred and may use the documented downstream-extension path. +UHP remains the only northbound protocol. A first turn may add `metadata.workspace`; requests that omit it retain stock HarnessRouter behavior unchanged. The extension initializes the same workspace that HarnessRouter would otherwise create fresh. Continuations omit the extension and use HarnessRouter's existing checkpoint hydration and session lifecycle to recover the exact bound attachment. -The proposed first-turn contract is `metadata.workspace`. It validates and materializes one repository into a private session checkout, binds that checkout immutably to the session, and starts the selected harness in the requested directory. A continuation reuses that exact checkout. We will seek upstream ownership first. If upstream rejects or defers the contract, the fork may carry the same bounded behavior as an explicitly documented downstream HarnessRouter extension; it must not describe that extension as standard UHP behavior. If a later UHP release reserves an incompatible `metadata.workspace`, the fork must migrate cleanly rather than preserve conflicting aliases. +V1 supports both of these source modes: -The fork remains the AllAgents distribution point. Generic protocol, lifecycle, session-binding, and materializer seams are proposed upstream first and live in the fork only when upstream declines or defers them. Deployment defaults, custom harness definitions, provider wiring, Promptfoo scenarios, and publication of the downstream image remain AllAgents-owned in either path. The supported distribution harnesses are **Codex** and **OMP**. Provider traffic goes only through an existing, separately operated OAuth-to-OpenAI-compatible gateway, and Promptfoo calls HarnessRouter directly over UHP. +- one or more Git repositories placed at declared, pairwise non-overlapping destinations; and +- an operator-catalogued OCI workspace snapshot selected by direct immutable digests. -This is not a general workspace platform. The proposal adds no extension registry, dynamically selected hook, arbitrary materializer command, or second protocol. It invokes exactly one operator-configured Git materializer included in the HarnessRouter image. +OCI is a source artifact for the workspace. It is not the HarnessRouter runtime image, a benchmark environment, a verifier, or a caller-selected container. There is no Git fallback for an OCI failure and no OCI fallback for a Git failure. -## System flow and ownership +Implementation in the fork starts immediately. Opening an upstream issue, writing a UHP proposal, or waiting for an upstream decision is not an implementation or release gate. After the downstream implementation and release evidence prove the capability, we may propose the generic contract upstream. Until accepted upstream, releases must label `metadata.workspace` as a documented `allagentsdev/harnessrouter` extension rather than standard UHP behavior. -```mermaid -flowchart LR - P[Promptfoo] -->|UHP + caller API key| H[HarnessRouter] - H -->|generic workspace seam| M[In-image Git materializer] - M -->|anonymous HTTPS| G[Public Git repository] - H --> S[(Private session state)] - H --> C[(Private session checkout)] - C --> X[Codex or OMP] - X -->|short-lived turn credential| B[HarnessRouter loopback broker] - B -->|Responses or Chat Completions| O[OAuth-to-OpenAI-compatible gateway] - O --> V[Model provider] - H --> D[(/data sessions and state)] -``` +The supported harnesses remain **Codex** and **OMP**. Provider traffic continues through the separately operated OAuth-to-OpenAI-compatible gateway using HarnessRouter's brokered credentials. This decision adds no AllAgents CLI integration and changes no local project workspace configuration. + +## Existing lifecycle and extension point + +The fork must preserve the stock ownership boundary: -| Component | Owns | +| Existing HarnessRouter responsibility | Extension responsibility | |---|---| -| Promptfoo | Prompt, model, `metadata.harness_id`, first-turn `metadata.workspace` request, continuation ID, and evaluation assertions | -| UHP specification and governance | When accepted upstream: reservation and versioned meaning of `metadata.workspace`, its schema, lifecycle semantics, error contract, and conformance requirements | -| `allagentsdev/harnessrouter` release | When downstream-only: extension shape and revision, implementation lifecycle, errors, extension tests, changelog, and docs; never UHP conformance ownership | -| HarnessRouter gateway and runner | Caller authentication, UHP validation and conformance, idempotency, session hydration, workspace binding, streaming, cancellation, harness execution, files, artifacts, and lifecycle state | -| Generic workspace seam | Exact metadata recognition, first-turn binding, continuation lookup, ordering before harness execution, and normalized workspace failures | -| Fixed Git materializer | URL and ref validation, safe Git resolution and acquisition, exact commit provenance, checkout validation, resource enforcement, cancellation, and cleanup | -| HarnessRouter custom harness definition | Reusable remote harness configuration: Codex or OMP base harness, model defaults, instructions, tools, skills, and server-owned provider route | -| HarnessRouter loopback broker | Per-turn scoped credential minting and exchange; the harness process never receives the long-lived external-gateway API key | -| External OAuth gateway | Provider login, OAuth token storage, refresh, repair, provider API compatibility, and provider authorization | -| AllAgents distribution configuration | Deployment defaults, Codex and OMP custom harness definitions, provider wiring, Promptfoo scenarios, and image publication | -| Operator | Caller credentials, custom harnesses, provider endpoint and API key, egress policy, limits, TTL, deployment, upgrades, and deletion policy | - -The materializer never owns UHP sessions or provider credentials. The external OAuth gateway never owns source acquisition or UHP session state. Promptfoo never receives source or provider credentials. Distribution configuration must not redefine the generic request or lifecycle contract. +| Allocate the session workspace and user/sandbox identity | Validate the first-turn workspace descriptor | +| Choose fresh materialization or checkpoint hydration | Resolve Git commits or exact OCI artifact identity | +| Transport and restore checkpoints | Build or reuse one verified immutable generation | +| Start the harness in the session workspace | Attach that generation read-only or as a private editable copy | +| Track sessions, TTL, cancellation, files, and cleanup | Persist source provenance and attachment identity with the session | +| Resume an existing session workspace | Verify and reuse the exact prior attachment without resolving source again | + +For a new workspace-backed session, source initialization runs after +authentication, request validation, idempotency, session resolution, and +allocation of HarnessRouter's fresh session workspace, but before provider work +or harness execution. It attaches source content at the runner-designated +workspace root; it does not allocate another workspace root or move lifecycle +ownership out of HarnessRouter. + +Stock HarnessRouter writes `.harness` state, generated instruction files, +plugins, skills, MCP configuration, HOME, and conversation state under the +workspace. That is incompatible with a shared read-only source mount. For +workspace-backed sessions, the fork creates a per-session writable control root +inside the existing session allocation but outside source content and redirects +all mutable harness/runtime state there. Generated instructions use a +harness-supported external instruction channel or non-shadowing session mount; +they never overwrite, overlay, or copy up a source path. If Codex or OMP cannot +honor that separation, the read-only release gate fails. + +For continuation or recovery, workspace-aware checkpoint hydration first +restores and validates the minimal control metadata needed for attachment, +acquires and mounts the exact protected generation, then restores the remaining +session-local mutable state around it. The source initializer is not invoked. +Git refs are not resolved again, OCI is not fetched again, and a newer +generation is not substituted. + +Workspace initialization reuses HarnessRouter's cancellation, process +containment, user isolation, quota, TTL, checkpoint transport, deletion, and +crash-recovery machinery. Source staging and immutable generations are internal +runner resources subordinate to that lifecycle, not caller-visible workspaces. ## Request contract -A workspace-backed first turn uses the normal UHP `POST /v1/responses` request. The proposed shape is: +A workspace-backed first turn uses the normal UHP `POST /v1/responses` endpoint. `metadata.workspace` is first-turn-only and has no nested schema version. + +Repository mode: ```json { @@ -69,179 +86,368 @@ A workspace-backed first turn uses the normal UHP `POST /v1/responses` request. "metadata": { "harness_id": "chrn_…", "workspace": { - "repository": { - "url": "https://github.com/example/project.git", - "ref": "refs/heads/main" + "access": "editable", + "retention": "session", + "source": { + "kind": "repositories", + "repositories": [ + { + "url": "https://github.com/acme/api.git", + "ref": "refs/heads/main", + "destination": "services/api" + }, + { + "url": "https://github.com/acme/web.git", + "destination": "services/web" + } + ] }, - "working_directory": "packages/service" + "working_directory": "services/api/packages/server" } } } ``` -`metadata.harness_id` is HarnessRouter's existing harness selector. HarnessRouter documents custom harnesses as reusable configurations with a fixed base harness and selects them through `metadata.harness_id` ([custom harness behavior](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/README.md#L155-L163), [UHP selection](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/README.md#L186-L203)). For this distribution, a custom harness is the remote equivalent of a reusable profile. It is not an AllAgents CLI profile and is not projected from a developer machine. +OCI snapshot mode: + +```json +{ + "model": "gpt-5.4", + "input": "Implement the requested change.", + "metadata": { + "harness_id": "chrn_…", + "workspace": { + "access": "read_only", + "source": { + "kind": "workspace_snapshot", + "snapshot_name": "large-monorepo", + "image_manifest_digest": "sha256:…", + "workspace_manifest_digest": "sha256:…" + }, + "working_directory": "packages/compiler" + } + } +} +``` `metadata.workspace` has exactly these fields: | Field | Required | Contract | |---|---:|---| -| `repository.url` | yes | Anonymous public HTTPS Git URL. No embedded credentials, userinfo, query, fragment, alternate protocol, or local path. | -| `repository.ref` | no | Advertised `refs/heads/*` or `refs/tags/*`, or unambiguous branch/tag shorthand. Omission means the remote default branch. Raw object IDs, other namespaces, ambiguous names, and unfetchable values fail. V1 accepts SHA-1 repositories only and records the resolved 40-hex commit. | -| `working_directory` | no | Relative POSIX directory beneath the checkout. Omission means the repository root. Absolute paths, empty components, `.`/`..` traversal, platform-specific separators, and any symlink escape fail. | +| `access` | yes | `read_only` or `editable`. | +| `retention` | no | `session` by default, or `persistent` when deployment policy authorizes it. | +| `source` | yes | Exactly one `repositories` or `workspace_snapshot` object as defined below. | +| `working_directory` | no | Workspace-relative POSIX directory. Omission means the workspace root. | + +A repository source has exactly `kind: "repositories"` and `repositories`. The array contains 1 to 128 entries. Each entry has exactly: + +| Field | Required | Contract | +|---|---:|---| +| `url` | yes | Canonical public HTTPS Git URL. No userinfo, query, fragment, local path, or alternate transport. | +| `ref` | no | Advertised full ref or unambiguous branch/tag shorthand. Omission uses the advertised remote default. The resolved commit, not the ref spelling, is authoritative. | +| `destination` | yes | Non-root, workspace-relative POSIX directory. Destinations must be unique, pairwise non-overlapping, and disjoint from runner-owned control paths. | + +Depth is not caller-selectable. The repository acquisition policy defaults every entry to Git depth `2`; that effective depth is returned as provenance and participates in generation identity. + +A snapshot source has exactly: + +| Field | Required | Contract | +|---|---:|---| +| `kind` | yes | `workspace_snapshot`. | +| `snapshot_name` | yes | Selects an operator-owned catalog entry. It is not a registry repository or URL. | +| `image_manifest_digest` | yes | Direct `sha256:` digest of the accepted OCI image manifest. Mutable tags and indexes are not accepted as source identity. | +| `workspace_manifest_digest` | yes | `sha256:` digest of the canonical workspace manifest expected from that artifact. | + +The OCI catalog is HarnessRouter deployment configuration owned by the operator. It maps `snapshot_name` to a fixed registry repository, allowed media types, trust policy, and server-side registry credential reference. A caller never supplies a registry origin, repository, tag, header, or credential. + +`working_directory` is interpreted only after the verified tree exists. It must resolve, without symlink escape, to a real directory inside the workspace. Absolute paths, empty components, `.` or `..` components, platform-specific separators, and reserved control paths are invalid. + +Unknown keys are rejected at every level. The descriptor cannot contain credentials, headers, host paths, environment variables, commands, runtime images, Docker settings, materializer selection, resource limits, provider routes, or caller-selected TTLs. Request size, string length, array length, nesting, and validation work are bounded before source access. +UHP input files are workspace mutations. They are accepted only for `editable` +workspaces, after the private copy exists and before the initial produced-file +baseline is sealed. A `read_only` first turn containing workspace input files +fails before source resolution. Harness assets, HOME, credentials, scratch, and +checkpoint control remain in runner-owned session paths outside immutable source +content in both modes. + +A first turn may omit `metadata.workspace`; stock behavior then remains unchanged. A session created without workspace metadata cannot add it later. Any reused session selected through `previous_response_id` or HarnessRouter's existing session-recovery metadata must omit `metadata.workspace`, even if the repeated object is byte-for-byte identical. + +## Response and provenance contract + +After attachment reaches `ready`, terminal events, response retrieval, replay, and later terminal failures expose the same committed `metadata.workspace` object. All public fields use snake case. + +Repository response: + +```json +{ + "metadata": { + "workspace": { + "access": "editable", + "retention": "session", + "working_directory": "services/api/packages/server", + "effective_descriptor_digest": "sha256:…", + "generation_id": "sha256:…", + "workspace_manifest_digest": "sha256:…", + "provenance": { + "kind": "repositories", + "repositories": [ + { + "url": "https://github.com/acme/api.git", + "requested_ref": "refs/heads/main", + "resolved_commit": "0123456789abcdef0123456789abcdef01234567", + "destination": "services/api", + "depth": 2 + }, + { + "url": "https://github.com/acme/web.git", + "resolved_commit": "89abcdef0123456789abcdef0123456789abcdef", + "destination": "services/web", + "depth": 2 + } + ] + }, + "expires_at": "2026-09-28T00:00:00Z" + } + } +} +``` + +Snapshot response: + +```json +{ + "metadata": { + "workspace": { + "access": "read_only", + "retention": "session", + "working_directory": "packages/compiler", + "effective_descriptor_digest": "sha256:…", + "generation_id": "sha256:…", + "workspace_manifest_digest": "sha256:…", + "provenance": { + "kind": "workspace_snapshot", + "snapshot_name": "large-monorepo", + "image_manifest_digest": "sha256:…", + "workspace_manifest_digest": "sha256:…", + "repositories": [ + { + "destination": ".", + "resolved_commit": "0123456789abcdef0123456789abcdef01234567", + "object_set_digest": "sha256:…" + } + ] + }, + "expires_at": "2026-09-28T00:00:00Z" + } + } +} +``` -An accepted UHP protocol version governs the standard schema; otherwise the fork release governs the documented extension shape. The workspace object has no nested schema marker. No other keys are accepted at any level of the object. It cannot carry credentials, headers, environment variables, commands, destination paths, Docker settings, materializer selection, resource limits, retention, or provider configuration. Request size, string length, nesting depth, and parsing work are bounded before source access. +The response fields are exact: -A first turn may omit `metadata.workspace`; stock HarnessRouter behavior then remains available. A session that starts without it cannot add it on a continuation. +| Field | Contract | +|---|---| +| `access` | Effective immutable access mode. | +| `retention` | Effective retention after authorization. It is never silently downgraded. | +| `working_directory` | Effective workspace-relative directory; the root is represented as `.`. | +| `effective_descriptor_digest` | Digest of the normalized first-turn descriptor, including applied defaults. | +| `generation_id` | Public content identifier for the verified immutable generation. It is not an authorization token or cache lookup key. | +| `workspace_manifest_digest` | Digest of the verified canonical source-visible manifest. | +| `provenance` | One of the exact source-mode objects below. | +| `expires_at` | Effective expiry timestamp for `session` retention, or `null` for authorized `persistent` retention. | -After successful binding, public response metadata records the normalized requested URL, optional requested ref, exact resolved commit, and requested `working_directory` when supplied under `metadata.workspace`. Omission means the checkout root and remains omitted in the response. Internal checkout IDs and host paths remain private. The resolved commit, not a mutable branch or tag, is the provenance authority for the session. +Repository provenance contains `kind: "repositories"` and the request-order `repositories` array. Each entry contains normalized `url`, `destination`, exact `resolved_commit`, effective `depth`, and `requested_ref` only when the request supplied `ref`. Branch or tag movement does not change stored provenance for an existing session. -## First turn and continuation semantics +Snapshot provenance contains `kind: "workspace_snapshot"`, `snapshot_name`, exact `image_manifest_digest`, exact `workspace_manifest_digest`, and a manifest-order `repositories` array. Each declared root contains `destination`; a history-bearing root additionally contains `resolved_commit` and `object_set_digest`. Tree-only roots contain neither. Snapshot provenance never exposes a registry origin, repository, credential reference, redirect, or physical path. -For a workspace-backed first turn: +Failures before attachment reaches `ready` omit workspace metadata. Failures after `ready` return the complete committed object. Internal generation keys, policy versions, authorization scope, mount paths, attachment IDs, pins, leases, reservations, and other sessions' state remain private. -1. HarnessRouter authenticates the caller, validates the UHP request, establishes idempotency, and resolves whether the request creates or reuses a session. -2. Any reused session, whether selected by `previous_response_id` or `metadata.session_id`, rejects workspace metadata. If it already has a workspace binding, its stored harness and binding win and a caller-supplied harness mismatch fails. An unbound session with no workspace metadata retains ordinary routing. -3. For a new workspace-backed session, workspace-aware hydration creates an isolated empty session allocation but skips stock empty Git initialization. It reserves bounded materialization capacity before provider or harness execution. -4. The materializer resolves the optional advertised ref to one exact commit, builds and validates a private checkout in staging, and crash-safely publishes it into the empty runner-designated checkout root. Publication never exposes a partial checkout. -5. The runner keeps four distinct concepts and paths: the session root, the checkout root, the control root, and the execution root. The checkout and control roots are non-overlapping children of the runner-owned session allocation. The execution root is a validated directory within the checkout, never the control root. -6. HarnessRouter crash-safely binds the descriptor, workspace-contract owner and immutable revision, resolved commit, checkout root, control root, execution root, cleanup deadline, and selected harness to the session. The contract revision is the UHP release when standardized or the exact downstream source revision otherwise. A restart observes either the complete binding and published checkout or neither; recovery removes unattached staging. -7. The selected custom harness starts in the execution root, while its home, credentials, scratch, skills, and checkpoint control state remain anchored under the control root. +## Immutable generations and attachment behavior -A continuation selects an existing session with `previous_response_id` or HarnessRouter's existing `metadata.session_id` recovery path and omits `metadata.workspace`. For a workspace-bound session, the gateway derives the harness from stored state; if the caller supplies a different harness, the request fails before hydration. The continuation reuses the exact private checkout, including edits from earlier turns, and the original resolved-commit provenance. Supplying workspace metadata on any reused session is invalid, even if byte-for-byte identical. The gateway never resolves the ref again, clones a replacement, changes the working directory, or silently starts a fresh session. +Both source modes produce the same versioned canonical workspace manifest. It enumerates source-visible directories, regular files, and symbolic links in logical path order with normalized mode, size, content digest, or link target. HarnessRouter independently walks staging without following links, recomputes the canonical bytes, and requires the supplied and computed manifest digests to match before publication. -If the bound checkout is expired, missing, corrupt, belongs to an unsupported workspace-contract revision, or cannot be proven to belong to the predecessor, continuation fails closed. Before activating an incompatible contract, an upgrade must explicitly migrate compatible bindings or drain and delete them; it must not retain conflicting aliases. There is no rematerialization, source fallback, or checkout substitution. The bounded ephemeral TTL is operator-configured; ordinary completion does not immediately remove a checkout that remains eligible for continuation. Expiry and explicit deletion use the same minimal idempotent cleanup path. +The runner computes a private generation key from every input that can change source bytes, filesystem semantics, or sharing authorization. For Git this includes the normalized repository URLs, resolved commits, destinations, shallow depth (`2` in v1), acquisition-policy revision, and materializer contract revision. For OCI it includes the catalog identity, exact image-manifest and workspace-manifest digests, trust-policy revision, and materializer contract revision. Access, retention, working directory, harness, session, and physical paths are excluded because they do not change the generation's immutable bytes. A later depth or acquisition-policy change therefore cannot reuse an incompatible Git generation. -## Generic upstream seam and distribution boundary +Identical normalized source identity publishes exactly one live verified generation. The cache has two levels: an operator-only bare Git mirror/object cache per canonical repository URL for bounded acquisition, followed by an immutable multi-repository generation keyed by canonical URLs, exact resolved commits, destinations, depth, acquisition-policy revision, and materializer contract revision. Refreshes of one bare cache are serialized, and in-flight acquisition and generation misses singleflight by generation key. Publication is crash-safe: partial or failed staging is never attachable, and garbage collection cannot remove a generation while a build waiter, provisional pin, durable session reference, or attachment lease protects it. -The proposed generic HarnessRouter seam is a workspace validate/materialize call **after session resolution and workspace-aware hydration, but before provider or harness execution**. It has two paths: +Attachment depends on `access`: -- first turn: allocate an empty session root without stock Git initialization, validate and materialize into its checkout root, create the separate control root, validate the execution root, and commit the binding; -- continuation: hydrate the bound session root, then load and verify the existing checkout, control, and execution roots without invoking source acquisition. +- Every `read_only` Git request for the same normalized source identity leases and attaches the same cached immutable shallow-generation bytes; it never clones or copies that repository again. Every `read_only` OCI request for the same snapshot and generation identity likewise leases and attaches the exact digest-keyed generation. These sessions retain separate user and sandbox identity, checkpoint state, home, scratch space, control state, logs, outputs, and response state. Filesystem enforcement makes each attachment read-only; there is no copy-up path, and neither the generation backing store nor the writable acquisition cache is exposed to a session. +- `editable` sessions reuse the same acquisition cache and pinned verified generation as input, then receive a private, quota-bounded, inode-independent writable copy. No mutable inode may be shared with the generation, Git object cache, or another session. Later turns and checkpoints operate on that private copy. -The owning HarnessRouter implementation recognizes only the selected `metadata.workspace` contract and calls one operator-configured in-image materializer. That code lands upstream when accepted and remains an explicit fork patch otherwise. The caller cannot name an implementation. There is no registry, plugin lifecycle, generic hook graph, network materializer service, or reusable extension SDK. +An attachment binds the normalized descriptor, exact generation key and epoch, access, retention, working directory, selected harness, provenance, and manifest digest to the HarnessRouter session. Continuation reuses that exact attachment. It never re-resolves source, changes access or retention, selects another generation with the same public ID, or rebuilds missing state. -The materializer contract is intentionally small: normalized descriptor in; an empty runner-assigned checkout target, cancellation, and fixed resource limits supplied by the runner; either a verified private checkout plus provenance, or a coded failure out. HarnessRouter remains responsible for session, checkpoint, file/artifact, and process lifecycle. Runner control state never lives inside the checkout, and the materializer cannot write it. +## Git acquisition and integrity -Upstream-first is a contribution sequence, not a downstream namespace claim: +Repository content is untrusted. Repository mode has a fixed v1 acquisition policy: `depth = 2` for every repository. This is Git's depth semantics—the requested tip plus bounded reachable history—not a guarantee of exactly two total commits when the tip is a merge. -1. Open a HarnessRouter issue before substantial implementation and confirm maintainers' preferred protocol process. -2. Submit a UEP when UHP governance requires one for the generic key and semantics. -3. If accepted, change the UHP specification and versioned schema, reference implementation, conformance suite, changelog, and documentation as one coherent contract. -4. If rejected or deferred, record the upstream decision and carry only the same bounded workspace patch in the fork, labeled and tested as a downstream extension rather than a UHP standard. -5. Keep URL policy, Git acquisition, workspace binding, continuation behavior, root separation, failure normalization, and the lifecycle seam generic in either implementation path. -6. Keep deployment defaults, custom harness definitions, external-provider wiring, Promptfoo scenarios, and downstream image publication in `allagentsdev/harnessrouter`. +Git initialization must satisfy all of these requirements: -Requests without `metadata.workspace` retain upstream behavior, including routing for reused unbound sessions, and the pinned UHP conformance suite remains the base protocol oracle. Each rebase of `allagentsdev/harnessrouter` must review distribution changes against the metadata, session-resolution, hydration, checkpoint, runner-root, and custom-provider paths. When the workspace capability is downstream-only, its tests and release notes identify that status separately from UHP conformance. +- Parse and authorize every URL before DNS or process launch. Only canonical public HTTPS origins are accepted. Revalidate every redirect; reject private, loopback, link-local, reserved, metadata-service, and otherwise disallowed addresses; pin approved addresses against DNS rebinding. +- Run Git without a shell, with a sanitized environment and isolated configuration. Disable interactive credentials, inherited proxy configuration, hooks, checkout filters, Git LFS hydration, submodule recursion, alternates, and non-HTTPS helpers and protocols. +- Resolve only an advertised branch, advertised tag, or advertised remote default to one exact commit before acquisition. Fetch that selected ref with `--depth=2`, then verify the fetched tip equals the previously resolved commit. Fetch and checkout commands must not substitute caller text for the resolved selection. If the remote cannot satisfy the bounded shallow fetch, fail; never silently deepen, unshallow, or fall back to a full clone. +- Keep one server-owned bare shallow Git mirror/object cache per canonical repository URL behind the generation builder so repeated acquisition can reuse fetched objects. The cache is mutable operator-only runner infrastructure, never a session attachment. Refreshes are serialized, and it is inaccessible to harness users, credentials, hooks, and workspace writes. Publication selects only the resolved ref's bounded object graph into the immutable generation; unrelated cached refs and objects are never exposed. The generation contains its own normalized shallow repository state, so later cache updates cannot change it. +- Preserve the generation's normalized `.git/shallow` metadata and the acquired recent history so offline commands such as `git log` and recent diffs work within the fetched boundary. Remove credential-bearing remotes, hooks, worktree links, alternates, replace and graft state, locks, reflogs, and transient fetch state. Verify detached `HEAD`, shallow boundary, index-to-tree equality, included object integrity, and source-visible content against the recorded commit and acquisition policy. +- Apply finite time, transferred-byte, inode, file-count, process, descendant, and concurrency limits across all repositories. Shallow depth reduces history transfer; it does not solve large working-tree transfer or materialization, which is why OCI snapshots remain mandatory. +- Stage every repository beneath its declared destination and reject overlaps, undeclared files, cross-root links, traversal, or reserved-path collisions. Publish the complete multi-repository generation atomically or publish nothing. +- Record normalized URL, optional requested ref, exact resolved commit, destination, and effective depth for every repository. A failure in any repository fails the whole source; partial repository sets are never attached. -## Provider authentication and harness configuration +## OCI acquisition and integrity -Provider authentication is proxy-only. Each deployment configures one external OAuth-to-OpenAI-compatible gateway base URL and API key server-side. HarnessRouter represents that endpoint with two protocol-specific logical connections using the same secret: Responses for Codex and OpenAI Chat Completions for OMP. Each harness policy contains exactly its matching connection, with no fallback. The UHP caller cannot supply or override the endpoint, key, transport, or route. +OCI snapshot support is mandatory in v1 and release-blocking. Snapshot acquisition must satisfy all of these requirements: -The external OAuth gateway owns login, token persistence, refresh, and repair. HarnessRouter does not implement provider login, import local credentials, mount developer credential files, or coordinate token refresh. HarnessRouter's caller API key authenticates the UHP caller only and is never reused as a provider credential. +- Resolve `snapshot_name` only through the operator-owned catalog. Fetch only the direct image manifest named by `image_manifest_digest`; do not follow mutable tags, accept an index in its place, change registry authority on redirect, or expose catalog registry details to the caller. +- Verify the image manifest digest, media type, descriptor sizes, every selected layer digest and size, the catalog-defined workspace-manifest media type, the workspace-manifest blob digest, and the recomputed source-visible manifest digest. +- Enforce the v1 envelope before and during extraction: at most 64 distributable tar/gzip/zstd layers; a 4 MiB image manifest; a 128 MiB workspace manifest with at most 128 repository roots; 8 GiB total compressed layer bytes; 32 GiB expanded source bytes; 500,000 entries; 4 GiB per regular file; paths of at most 4096 UTF-8 bytes and 128 components; and 1 MiB per PAX or extended header. For each layer and for the aggregate artifact, expanded bytes divided by `max(compressed_bytes, 1)` must not exceed `100`. Cumulative-size and expansion-ratio checks apply while streaming, not only after extraction. Operators may configure lower limits, never higher ones without a contract revision. +- Apply layers in order with OCI whiteout and opaque-directory semantics. Whiteouts are metadata operations, not source-visible files. Reject malformed, duplicate, conflicting, or out-of-root whiteouts. +- Before writing each entry, validate its normalized relative path, type, declared size, mode, and link target. Reject absolute paths, traversal, NULs, escaping hard links or symbolic links, devices, sockets, FIFOs, sparse-file tricks, unsupported types, and entries that collide with runner-owned paths. Extraction uses rooted, no-follow operations and cannot write through a previously extracted link. +- Require the canonical workspace manifest to declare every source-visible entry and repository root. Undeclared output, missing entries, type changes, digest mismatches, and paths outside declared roots fail closed. -The deployment must use HarnessRouter's brokered sandbox mode, not the self-host image's `HR_SANDBOX_TRUST=owner` pass-through default. The broker exchanges the long-lived external-gateway key server-side and gives each harness only a short-lived, session-scoped credential plus the loopback broker URL. Readiness fails if the local broker cannot mint and exchange that credential. The long-lived key never enters the harness process environment or session files. Scoped turn credentials may exist only in the active process environment or a per-turn ephemeral config root; the runner deletes them before checkpointing or exposing any file, artifact, log, or response. +A workspace snapshot may contain normalized offline Git history for any declared repository root. A history-bearing root records `resolved_commit` and `object_set_digest`; a tree-only root records neither. History-bearing roots must have detached `HEAD` at the recorded commit, an index equal to that tree, the complete required object closure matching `object_set_digest`, and no dirty, staged, untracked, unreachable, or extra source-visible state. They must contain no remote, credential helper, config include, hook, worktree link, alternate, shallow, replace, graft, reflog, transient fetch state, or credential-bearing configuration. OCI restore never contacts Git, and failure of snapshot or embedded Git verification never falls back to cloning. -Codex requires an OpenAI Responses-compatible endpoint. This matches the pinned runner, which states that current Codex supports Responses rather than Chat Completions ([Codex endpoint behavior](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L1907-L1915)); HarnessRouter supports Codex against custom endpoints that provide the Responses format ([provider compatibility](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/docs/self-hosting-guide.md#L331-L350)). OMP uses the same external gateway's OpenAI Chat Completions surface in v1, which its pinned builder supports ([OMP endpoint behavior](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L2609-L2644)). An incompatible route fails readiness or the turn; it never changes protocol or provider automatically. +The direct image digest is part of OCI identity even when two artifacts have the same source-visible tree. Repacking layers or offline Git objects creates a different generation identity. Semantic verification proves an artifact's contents; it does not silently deduplicate distinct artifacts. -V1 custom harnesses use Codex or ordinary session-local OMP. OMP starts from the container's/session's own configuration. It does not import AllAgents profiles, host profiles, or developer state. There is no profile synchronization, profile projection, or AllAgents CLI integration. +## Produced files, checkpoints, and continuation -## Source and workspace security invariants +HarnessRouter's existing Files API and produced-file collection remain the only +public file surface. The fork adapts the existing root-Git-oriented bookkeeping +rather than introducing a second Files API or parallel change tracker. It stores +workspace-backed cursors and indexes under the per-session control root; it does +not initialize or mutate a bookkeeping `.git` directory inside an immutable +generation. -Repository content is untrusted. The implementation must preserve all of these invariants: +Bookkeeping must understand the declared repository roots and source mode: -- Caller authentication is required before session lookup, source access, continuation, retrieval, streaming, cancellation, file/artifact access, or deletion. Authentication failure does not disclose whether a session exists. -- Source is exactly one anonymous public HTTPS Git repository. URL parsing happens before DNS or process launch. Credentials and caller-controlled proxy settings are rejected and never forwarded. -- Every initial host and redirect target is re-parsed and re-authorized. DNS answers are checked against loopback, link-local, private, reserved, multicast, metadata-service, and otherwise non-public ranges; the approved address is pinned for the connection so DNS rebinding cannot change it. Redirect count, response size, and time are bounded. -- Git runs with a sanitized environment and isolated configuration. Interactive credentials, repository hooks, checkout filters, Git LFS, submodules, alternates, and non-HTTPS helpers or protocols, including `file`, `ssh`, and `ext`, are disabled or rejected. Repository configuration cannot weaken those rules. -- Ref discovery and fetch are bounded. V1 accepts only advertised branch/tag refs in SHA-1 repositories, ties the checkout to the exact resolved 40-hex commit, and records requested URL/ref plus resolved commit as provenance. -- The checkout is private and editable by one session only. No mutable state is shared across sessions. The runner-owned allocation maintains distinct session, checkout, control, and execution roots; repository content can never overlap the control root. -- All path operations are rooted, no-follow where appropriate, and checked for traversal and symlink escape. The execution root must resolve to a real directory inside the checkout root. Harness home, credentials, scratch, skills, and control state remain anchored under the control root regardless of the execution root. -- Materialization and cleanup have hard process, descendant, wall-clock, byte, inode, file-count, and concurrency bounds. Cancellation terminates the complete acquisition process tree before cleanup and terminal acknowledgement. -- Publication is crash-safe and partial staging is never attached. Cleanup is deterministic and idempotent after success, failure, cancellation, restart, expiry, and deletion. A path whose deletion failed is not reused or reported as free. -- Long-lived provider and caller credentials never enter Git arguments, the harness process environment, checkout or session files, response metadata, artifacts, logs, or provenance. A short-lived broker token may enter only the active harness environment or per-turn ephemeral config and is removed before checkpointing or public file collection. Source-controlled configuration cannot select the provider endpoint. +- Git repository roots compare editable state with their recorded resolved + commits. +- History-bearing snapshot roots use their verified commit and object-set + records. +- Tree-only snapshot roots compare with the canonical workspace manifest. +- Workspace paths outside declared repository roots use an external + runner-owned baseline in the control root. +- Git control data, generation metadata, credentials, checkpoint control state, + and runner-owned paths are never reported as produced files. -HarnessRouter's per-session process isolation remains useful, but this deployment is not represented as a hostile-code sandbox. The service binds to loopback by default and requires an explicit operator decision and network controls before broader exposure. +The adaptation must represent additions, modifications, deletions, renames, and +mode changes across multiple nested repository roots without assuming or +writing a single root `.git` directory. `read_only` attachments cannot produce +source mutations. Editable produced-file state and nested repository state are +covered by the session's private quota and lifecycle. + +Checkpoint behavior is access-specific. A `read_only` checkpoint excludes the +generation mount and all source bytes; it persists session-local mutable state +separately from the exact generation-key, epoch, manifest, durable reference, +and attachment evidence. Hydration validates that minimal binding/control +metadata, reattaches the same protected generation, then restores the remaining +harness state before the turn. An `editable` checkpoint contains the private +workspace copy and its nested repository state, never the bare acquisition +cache or immutable generation backing store. Checkpoint creation must not run a +root Git commit against a read-only attachment. + +A continuation supplies the existing predecessor/session reference and omits +`metadata.workspace`. HarnessRouter performs the access-specific hydration, +verifies the stored attachment evidence, and starts the stored harness in the +stored working directory. Edits from prior editable turns remain visible. A +changed ref, source digest, working directory, access, retention, or harness +requires a new session. + +If attachment, generation, private-copy, checkpoint, or provenance evidence is expired, missing, corrupt, or inconsistent, continuation fails closed. It does not clone, repull, restore from OCI again, substitute another generation, or silently start a fresh session. + +## Provider authentication and harness configuration + +Provider authentication remains proxy-only. Each deployment configures one external OAuth-to-OpenAI-compatible gateway base URL and API key server-side. HarnessRouter represents that endpoint with two protocol-specific logical connections using the same secret: Responses for Codex and OpenAI Chat Completions for OMP. Each harness policy contains exactly its matching connection, with no fallback. The UHP caller cannot supply or override the endpoint, key, transport, or route. + +The external OAuth gateway owns login, token persistence, refresh, repair, provider API compatibility, and provider authorization. HarnessRouter does not implement provider login, import local credentials, mount developer credential files, or coordinate provider token refresh. HarnessRouter's caller API key authenticates the UHP caller only and is never reused as a provider credential. + +The deployment uses HarnessRouter's brokered sandbox mode, not owner-trust credential pass-through. The broker exchanges the long-lived external-gateway key server-side and gives each harness only a short-lived, session-scoped credential plus the loopback broker URL. The long-lived key never enters the harness process environment, session workspace, checkpoint, file output, artifact, log, response, or source provenance. + +Codex uses the gateway's OpenAI Responses-compatible surface. OMP uses the same gateway's OpenAI Chat Completions-compatible surface. V1 custom harnesses are Codex and ordinary session-local OMP only. OMP starts from container/session configuration; it does not import AllAgents profiles, host profiles, or developer state. ## Failure behavior -HarnessRouter fails closed without changing source, checkout, harness, model route, or provider protocol as a recovery shortcut. +The implementation fails closed without changing source mode, source identity, access, retention, harness, model route, or provider protocol as a recovery shortcut. | Failure | Behavior | |---|---| -| Malformed, oversized, nested too deeply, or unknown workspace field | Reject as invalid request input before source access. | -| Workspace extension not accepted upstream | Ship it only as a documented downstream HarnessRouter extension; never represent it as UHP-standard behavior. | -| Disallowed URL, DNS answer, redirect, protocol, ref, or Git feature | Fail the response before attachment; remove bounded staging; do not start a harness or provider call. | -| Ref does not resolve to one permitted commit | Fail with source-resolution error; do not guess a default or fetch arbitrary objects. | -| Resource or concurrency limit unavailable | Reject or fail with a retryable capacity error before starting unbounded work. | -| Materializer timeout, crash, cancellation, or live descendant | Terminate and reap the process tree, clean staging idempotently, and return a coded failure. | -| Working directory missing, not a directory, or escaping through traversal/symlink | Fail before harness execution. | -| Crash during checkout publication or session binding | Recover to either the complete published checkout and binding or no attachment; never expose partial staging or reuse an uncertain path. | -| Workspace metadata present on any reused session | Reject the request without changing the existing session or extending its TTL. | -| Bound checkout expired, missing, corrupt, mismatched, or owned by an unsupported contract revision | Fail continuation; do not clone, substitute, resurrect, or reinterpret it. | -| External OAuth gateway authentication or provider failure | Return the normalized UHP failure; do not switch endpoint, protocol, credential, or harness. | -| Cleanup failure | Keep the allocation unavailable, report operational failure, and retry the same idempotent cleanup path. | +| Malformed, oversized, too-deep, or unknown workspace field | Reject before source access and without mutating an existing session. | +| Workspace metadata on a continuation or reused session | Reject without changing the attachment, checkpoint, or TTL. | +| Workspace input files with `read_only` access | Reject before source resolution; never overlay or copy up immutable source. | +| Unauthorized `persistent` retention | Fail before source resolution; do not downgrade to `session`. | +| Invalid Git URL, ref, destination, network target, redirect, or feature | Fail the response, cancel bounded source work, and remove staging; do not start a harness or provider call. | +| Any repository in a multi-repository source fails | Fail the complete source; never attach a partial set. | +| Unknown snapshot, digest mismatch, mutable reference, disallowed registry transition, or unsupported media type | Fail OCI acquisition; do not try Git or another snapshot. | +| Layer limit, extraction violation, whiteout error, unsafe path/link/type, or manifest mismatch | Terminate extraction, quarantine or remove staging, and publish nothing. | +| Resource or concurrency capacity unavailable | Return a coded retryable capacity failure before unbounded acquisition. | +| Materializer timeout, crash, cancellation, or live descendant | Terminate and reap the complete process tree before cleanup and terminal acknowledgement. | +| Working directory missing, not a directory, or escaping by traversal/link | Fail before attachment and harness execution. | +| Crash during publication or attachment commit | Recover to either a complete verified attachment or no attachment; never expose partial staging. | +| Missing or corrupt bound state on continuation | Fail as non-resumable; never rematerialize or substitute. | +| External provider authentication or execution failure | Return the normalized UHP failure; do not switch endpoint, protocol, credential, or harness. | +| Cleanup failure | Quarantine and continue accounting for the allocation; retry the same idempotent cleanup path. | + +Promptfoo treats every non-success as an evaluation error. It does not convert a workspace failure to an empty success, source fallback, or implicit retry. -Promptfoo treats non-success as an evaluation error. It does not turn HarnessRouter failures into empty successes or implicit retries. +## Fork, deployment, and release boundary -## Repository, deployment, and release boundary +`allagentsdev/harnessrouter` remains the implementation and distribution repository and remains a GitHub fork of `HarnessRouter/harnessrouter`. The downstream patch extends the fresh-workspace initialization point, source provenance, generation storage, and existing produced-file bookkeeping while leaving UHP requests without `metadata.workspace` on stock paths. -`allagentsdev/harnessrouter` remains the implementation and distribution repository and remains a GitHub fork of `HarnessRouter/harnessrouter`. It carries the upstream baseline plus the smallest necessary downstream distribution revision. It does not become a separate product repository. `allagentsdev/allagents` remains the local Bun CLI repository and contains this integration decision and planning material; v1 adds no gateway command or other CLI coupling. +The source initializers and generation manager ship inside the HarnessRouter image; they are not another network service. The supported deployment remains one HarnessRouter container with durable `/data`, loopback binding by default, and `HR_BACKENDS=codex,omp`. -Generic workspace capability is proposed through the upstream contribution process. If accepted, it lands in the UHP specification, schema, reference implementation, conformance suite, changelog, and docs, and the fork consumes that release. If upstream rejects or defers it, the fork may carry the smallest complete workspace patch and identifies that delta in release notes, source metadata, and tests. In either path, deployment defaults, Codex and OMP custom harness definitions, provider wiring, pinned Promptfoo scenarios, and image release automation remain downstream. Generic protocol or lifecycle fixes discovered downstream continue to be proposed upstream rather than hidden behind product-specific seams. +The public image remains `ghcr.io/allagentsdev/harnessrouter`. Tags identify the upstream HarnessRouter baseline plus the downstream revision; deployments pin the resulting image manifest digest. Releases produce standard SBOM and build-provenance attestations and run the upstream UHP conformance suite against the built image. -The supported deployment is one container started by Docker Compose, bound to loopback by default, with durable `/data` and `HR_BACKENDS=codex,omp`. The materializer ships in that image; it is not another service. +Release verification must exercise both Codex and OMP through the configured external provider gateway. In addition, v1 cannot release without: -The public image is `ghcr.io/allagentsdev/harnessrouter`. Tags identify the upstream HarnessRouter baseline plus the downstream AllAgents revision; deployments pin the resulting image digest. Releases produce standard SBOM and build-provenance attestations and run upstream UHP conformance against the built image. +1. an end-to-end OCI test using at least 2 GiB of expanded source bytes and 100,000 source-visible filesystem entries that fetches by direct manifest digest, applies layers and whiteouts, verifies the workspace manifest and any offline Git history, starts a harness in `working_directory`, and continues the same session successfully; and +2. a cache-reuse proof showing that identical Git and OCI source identities publish once, concurrent cache misses singleflight, every concurrent or later `read_only` task/session leases the same immutable generation bytes without cloning or copying, read-only checkpoints contain no generation bytes or source Git writes, mutable harness state remains per-session outside source, `editable` sessions derive inode-independent copies from the pinned generation, unrelated bare-cache refs are not exposed, and continuation reattaches the exact generation without reacquisition. -Promptfoo is lockfile-pinned in `allagentsdev/harnessrouter` and calls the built image directly over UHP. Release verification exercises both Codex and OMP through the configured external provider gateway. No intermediate evaluation repository or custom green-E2E attestation format is part of the distribution. +These are release gates, not deferred performance tests. Git and OCI failure-path coverage must also prove that no partial generation or source-mode fallback becomes visible. + +Downstream implementation and release do not wait on upstream work. Once downstream evidence exists, maintainers may propose the generic capability upstream. If upstream accepts an equivalent contract, the fork should remove the superseded patch and migrate cleanly; it must not retain conflicting aliases or claim downstream conformance before acceptance. ## Alternatives rejected | Alternative | Why rejected | |---|---| -| Build a new execution gateway | Duplicates HarnessRouter's UHP, sessions, streaming, cancellation, files, artifacts, and harness supervision. | -| Put a thin service in front of stock HarnessRouter | Splits checkout and session ownership across services and still cannot place the workspace at the correct runner lifecycle point. | -| Rename the existing fork or create a third repository around it | Loses the clear upstream relationship or adds a release/rebase boundary without product isolation. | -| Present a downstream workspace extension as standard UHP behavior | The open metadata object permits an implementation extension, but only upstream governance can standardize its meaning and conformance requirements. | -| Wait without engaging upstream | The pinned version accepts arbitrary metadata but forwards only the System One probe; an issue and, if required, a UEP are the path to obtaining workspace lifecycle semantics. | -| Add a general metadata extension or materializer framework | V1 has one object and one implementation. A framework would enlarge the change before a second use case exists. | -| Put repository instructions in the prompt or a model tool | Makes acquisition model-dependent, non-deterministic, too late to set the initial working directory, and unsafe for credentials and provenance. | -| Make the local AllAgents CLI or its profiles the remote control plane | Couples a local developer tool to an independently deployed service and duplicates HarnessRouter custom harnesses. | -| Manage provider login inside HarnessRouter | Duplicates the external OAuth gateway's ownership of login, refresh, and repair and expands the credential attack surface. | -| Upload every source file through UHP | Pushes acquisition to every caller and loses authoritative Git ref-to-commit provenance and repository behavior. | +| Build a new execution gateway | Duplicates HarnessRouter's UHP, sessions, workspace lifecycle, streaming, cancellation, files, artifacts, and harness supervision. | +| Put a workspace service in front of HarnessRouter | Splits source and session ownership and cannot safely participate in checkpoint hydration, continuation, or produced-file bookkeeping. | +| Create a second checkout root inside each session | Competes with the runner-owned workspace, duplicates cleanup and quota state, and makes files and checkpoints ambiguous. | +| Ship Git first and defer OCI | Fails the minimum large-repository use case and makes release viability depend on repeated acquisition. | +| Treat OCI as a runtime or benchmark image | Mixes source provenance with tools, services, verifier assumptions, and execution policy. | +| Wait for upstream before implementation | Makes delivery depend on a project we do not maintain and delays the evidence needed for a useful upstream proposal. | +| Add a general plugin or materializer framework | V1 has two explicit source modes and no demonstrated need for caller-selectable plugins. | +| Put source instructions in the prompt or a model tool | Makes acquisition model-dependent, non-deterministic, too late to set the initial directory, and unsafe for provenance. | +| Upload every source file through UHP | Pushes acquisition to callers and loses authoritative Git and OCI identity, history, links, and modes. | +| Make the AllAgents CLI the remote control plane | Couples local developer configuration to an independently deployed HarnessRouter service. | +| Add a new Files API for initialized workspaces | Duplicates HarnessRouter behavior instead of adapting its existing produced-file bookkeeping. | ## Deliberate v1 limits -V1 supports one anonymous public HTTPS Git repository using SHA-1 object IDs, one private editable checkout per session, an optional advertised branch/tag ref, an optional safe working directory, exact 40-hex commit provenance, and bounded ephemeral retention. +V1 supports public HTTPS Git repositories acquired at fixed depth `2`, multiple pairwise non-overlapping destinations, advertised branch/tag/default refs, exact resolved-commit and effective-depth provenance, server-owned bare acquisition caches, operator-catalogued OCI snapshots selected by direct digests, optional normalized offline Git history, `read_only` and `editable` attachments, bounded `session` retention, authorized `persistent` retention, and a workspace-relative working directory. -V1 does **not** include raw commit-ID requests, SHA-256 repositories, multiple repositories, private-source credentials, OCI sources, caller-selected runtime images, shared or read-only generations, cross-session caching, persistent workspaces, user-selected TTLs, session branching, checkout migration, or elaborate tombstone and garbage-collection machinery beyond minimal idempotent cleanup. It uses HarnessRouter's existing file and artifact behavior rather than inventing produced-file tracking. +V1 does not include caller-supplied registry origins or credentials, mutable OCI tags, OCI indexes as source identity, transparent Git/OCI fallback, caller-selected runtime images or benchmark environments, arbitrary materializer commands, private-network Git origins, caller-selected TTLs, session branching, access or retention changes on continuation, or public multi-tenant authorization. It does not add an AllAgents CLI command or change project workspace configuration. -Only Codex and OMP are required and release-validated by the AllAgents distribution. Other upstream backends and a future Copilot harness are outside this decision. There is no local-profile import, host-profile projection, provider-route override, automatic provider fallback, public multi-tenant authorization model, scoring service, dataset service, or evaluation task engine. +Only Codex and OMP are required and release-validated. Other HarnessRouter backends, local-profile import, host-profile projection, provider-route override, automatic provider fallback, scoring, datasets, assertions, and evaluation-task orchestration are outside this decision. ## Consequences -HarnessRouter gains a bounded workspace-materialization contract without creating a separate gateway product. The preferred outcome is a governed UHP capability; the fallback is a clearly labeled downstream extension in the existing fork. The contribution cost includes upstream design review, UEP work when required, and coordinated specification/schema/implementation/conformance changes when maintainers accept the contract. - -`allagentsdev/harnessrouter` retains a clear fork relationship and a narrow distribution delta. Its image and the `allagents` npm CLI release independently. Promptfoo tests the same digest-pinned image and UHP surface that operators deploy. +HarnessRouter remains the sole execution, workspace, and session control plane. The fork gains deterministic first-turn source initialization without adding a new northbound API, process supervisor, checkpoint system, file service, or workspace lifecycle. -Each session pays for a private checkout and cannot reuse a shared generation. That is intentionally less efficient than a source platform, but it makes mutability, provenance, continuation, quota, and cleanup ownership understandable for v1. +Mandatory OCI support and generation accounting make v1 more substantial than a Git clone hook, but they make the minimum large-repository use case viable. Shared immutable generations avoid repeated acquisition for `read_only` sessions; private inode-independent copies preserve isolation for `editable` sessions. -Provider credential lifecycle stays outside HarnessRouter. This reduces credential code and operational states in the distribution, at the cost of requiring a compatible external OAuth gateway and making its availability part of service readiness. +The operator assumes finite capacity management for staging, generations, editable copies, persistent sessions, tombstones, and quarantined deletion failures. Protected or uncertain state is never advertised as free capacity. -Upstream review may accept, reject, defer, or reshape the proposal. Acceptance lets the fork delete generic workspace patches and consume the upstream release. Rejection or deferral leaves those patches visible as a downstream HarnessRouter extension; it does not justify a repository rename or a false claim that UHP conformance covers the extension. +Provider credential lifecycle remains outside HarnessRouter. The distribution depends on the external OAuth-to-OpenAI-compatible gateway, while the harness sees only brokered short-lived credentials. ## Reconsider when Revisit this decision if: -- upstream HarnessRouter or UHP governance materially changes the generic workspace contract or later reserves an incompatible key; -- maintainers require a different lifecycle point, schema, or extension mechanism; -- an accepted upstream path cannot keep the specification, schema, reference implementation, conformance, changelog, and documentation aligned as one coherent versioned contract; -- the generic seam grows beyond metadata recognition, binding, and one fixed materializer invocation; -- a second materializer is approved and proves that a registry is simpler than explicit code; -- private repositories, multiple repositories, OCI sources, persistent workspaces, or shared immutable caching become validated product requirements; -- public multi-tenancy or stronger hostile-code isolation becomes a requirement; -- Codex can no longer use the external gateway's Responses surface, or OMP cannot use its configured compatible surface; -- the external gateway can no longer own provider login, refresh, and repair; -- private editable checkouts cannot meet practical storage and cleanup bounds; or +- UHP or upstream HarnessRouter adopts an equivalent workspace-source contract; +- HarnessRouter changes its fresh/checkpoint workspace lifecycle so the extension point no longer preserves one authoritative session workspace; +- the host cannot enforce immutable shared generations and inode-independent editable copies; +- large-repository OCI materialization or cache reuse cannot meet finite release limits; +- continuation and attachment recovery cannot fail closed without source reacquisition; +- source acquisition requires a stronger isolation boundary; +- public multi-tenancy or caller-owned private-source credentials become requirements; +- Codex or OMP can no longer use the external provider gateway's required compatible surface; or - another UHP implementation offers a materially smaller and more stable integration surface. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index f2197f07..0b33705b 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,5 +1,5 @@ --- -title: "HarnessRouter Workspace Execution - Implementation Plan" +title: "HarnessRouter Workspace Composition - Implementation Plan" date: 2026-09-18 updated: 2026-09-27 type: feat @@ -8,994 +8,1047 @@ artifact_readiness: implementation-ready execution: code --- -# HarnessRouter Workspace Execution - Implementation Plan +# HarnessRouter Workspace Composition - Implementation Plan + +## Stock behavior inventory + +The pinned HarnessRouter baseline already owns the session workspace lifecycle. +The fork must extend that lifecycle rather than introduce another workspace +abstraction: + +1. Session identity implicitly selects the session workspace; callers do not + currently describe a source workspace. +2. Fresh hydration creates the session workspace as an empty root Git + repository. +3. Continuation restores the session checkpoint selected by the existing + response/session identity. +4. Attached files, `.harness` state, generated root instructions, plugins, + skills, MCP configuration, HOME, and conversation state are materialized + under the hydrated workspace before the harness turn starts. +5. Produced-file collection uses a cursor over the root Git repository. +6. Stock checkpointing mutates that root Git repository and archives the entire + workspace; stock hydration clears the workspace before restoring the archive. +7. No stock UHP request field names a Git source, an OCI source, or a reusable + immutable generation. + +These are the starting facts and the integration constraints. The implementation +keeps the existing session allocation, hydrate/checkpoint cycle, file and +artifact APIs, user/sandbox isolation, cancellation, TTL, cleanup, and harness +supervision. It adds first-turn source composition at the existing hydration +boundary, stores the resulting attachment in the existing session state, and +adapts the existing produced-file cursor for nested repositories. It does not +create a second workspace, second session database, second Files API, external +materializer service, or parallel lifecycle. ## Goal -Add a generic workspace contract and the runner capabilities needed to execute -Codex or OMP in a caller-selected public Git repository. Propose the protocol, -schema, implementation, conformance cases, changelog, and documentation through -HarnessRouter's upstream governance process first. If upstream declines or -defers the contribution, carry the same bounded behavior as a clearly labeled -downstream HarnessRouter extension. Keep the existing GitHub fork at -[`allagentsdev/harnessrouter`](https://github.com/allagentsdev/harnessrouter) and -publish an AllAgents-maintained image at -`ghcr.io/allagentsdev/harnessrouter`. - -A new session may supply `metadata.workspace`. HarnessRouter securely resolves -one anonymous public HTTPS Git repository to an advertised SHA-1 commit, -materializes one private editable checkout, binds that exact checkout to the -session, and starts the selected harness in the requested safe working -directory. A continuation reuses the checkout and its mutations without -resolving or cloning again. - -Everything else remains HarnessRouter behavior: caller authentication, UHP -request and response handling, session hydration, streaming, idempotency, -cancellation, provider and harness execution, artifacts, and ordinary lifecycle -state. - -## Governance and repository boundary - -### Upstream-first gate - -`metadata.workspace` and the associated runner behavior solve a generic -HarnessRouter problem, so upstream gets the first opportunity to own them. The -fork may ship the contract after an upstream decision, but must label it as a -downstream HarnessRouter extension unless UHP governance standardizes it. Before -substantial implementation work: - -1. Open an issue in `HarnessRouter/harnessrouter` that describes the use case, - request and response shapes, security boundary, session semantics, runner - seam, lifecycle, error categories, and intended conformance coverage. -2. Ask maintainers to confirm the required governance path and ownership of the - metadata key, configuration names, and runner interfaces. -3. Write or amend the required UHP Enhancement Proposal (UEP) before changing - protocol semantics. Follow the repository's contribution and UHP governance - rules for discussion, review, compatibility, and approval. -4. Record the issue and UEP links in the implementation pull requests and in the - downstream release notes. - -The gate is passed when maintainers have selected the governance and ownership -path for `metadata.workspace`. Acceptance starts the coordinated upstream -contract; rejection or deferral starts the downstream-extension path. If no -maintainer decision arrives within 30 calendar days after the issue opens and a -follow-up is posted after day 14, record the proposal as deferred. A downstream -release must not claim that its workspace behavior is part of UHP or covered by -UHP conformance. If a later UHP release reserves an incompatible key or shape, -the fork migrates cleanly instead of retaining conflicting aliases. - -### Upstream deliverables when accepted - -An accepted upstream change is complete only when the same reviewed contract -appears in all relevant surfaces: - -- the UHP specification; -- the machine-readable UHP schema; -- the HarnessRouter reference implementation; -- the UHP conformance suite; -- the HarnessRouter changelog; and -- user and operator documentation. - -Generic workspace parsing, session binding, runner root separation, -materialization, lifecycle, errors, and conformance behavior are proposed -upstream together. If maintainers decline or defer that ownership, the fork -carries the smallest complete patch in the corresponding HarnessRouter paths, -with separate extension tests and release notes. The implementation remains -generic and must not couple lifecycle behavior to AllAgents harness IDs. - -### Downstream repository - -Keep `allagentsdev/harnessrouter` as the GitHub fork of -`HarnessRouter/harnessrouter`, preserving its fork relationship, history, -issues, settings, protections, and upstream remote. Do not create another -repository and do not add a wrapper repository. - -The preferred fork delta contains only the downstream pieces specific to the -AllAgents deployment: - -- deployment defaults; -- custom Codex and OMP harness definitions; -- provider connection and policy wiring; -- direct Promptfoo scenarios and sanitized reports; and -- image build and publication for `ghcr.io/allagentsdev/harnessrouter`. - -If upstream declines or defers workspace ownership, the fork also carries the -smallest complete workspace contract and runner/materializer patch. Source -metadata, tests, and release notes distinguish that extension from standard UHP. - -The initial examined baseline is HarnessRouter v0.25.4 at commit -`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`, with UHP `2026-09-12`. -Development records that baseline exactly. The release baseline must be an exact -upstream tag and commit. Workspace behavior then comes either from an accepted -upstream release or from explicitly identified downstream commits applied to -that baseline. - -The `allagentsdev/allagents` repository remains the local Bun CLI. This plan -adds no server, materializer, provider adapter, CLI command, local-profile -synchronization, or `workspace.yaml` behavior there. HarnessRouter custom -harness definitions are the complete remote configuration surface for Codex and -OMP. - -## Product boundary +Extend `allagentsdev/harnessrouter` so a new UHP session can compose its existing +HarnessRouter workspace from either multiple Git repositories or a mandatory OCI +workspace snapshot. Bind the verified immutable generation and its access mode +to the session before the first harness turn. Continuations omit the descriptor +and recover the exact attachment through the access-specific workspace-aware +checkpoint path. + +OCI workspace snapshots are a release-blocking v1 source, not a later +optimization. Large repositories are part of the minimum deliverable. The Git +path limits history transfer with a fixed shallow fetch, but it still transfers +and checks out every working-tree byte; it therefore does not replace OCI for +large workspaces. + +Keep all other boundaries unchanged: + +- UHP is the only northbound execution protocol. +- Requests without `metadata.workspace` take the stock path without new source, + generation, attachment, or response semantics. +- Codex and OMP are the supported harnesses. +- Both use the existing separately operated OAuth-to-OpenAI-compatible provider + gateway through server-owned, brokered credentials. +- `allagentsdev/harnessrouter` remains the existing GitHub fork and + `ghcr.io/allagentsdev/harnessrouter` remains the image name. +- There is no AllAgents CLI implementation, profile import, local gateway + command, or `workspace.yaml` change. + +## Product and ownership boundary ### In scope -- The UHP baseline selected through the upstream issue and UEP process, plus the - accepted upstream workspace contract or documented downstream extension. -- Caller authentication through HarnessRouter's existing API-key behavior. -- One generic workspace object at `metadata.workspace`. -- Exactly one anonymous, public, HTTPS Git repository per new session. -- An optional advertised `refs/heads/*` or `refs/tags/*`, or unambiguous - branch/tag shorthand. Omission means the remote default branch; raw object IDs - and other ref namespaces are rejected. -- SHA-1 repositories only, with resolution to and recording of one exact 40-hex - commit object ID. -- One private editable checkout per session. -- Immutable first-turn binding and exact-checkout continuation reuse. -- A finite server-owned idle TTL and deterministic, idempotent cleanup. -- Codex and OMP harnesses using one server-configured external - OAuth-to-OpenAI-compatible gateway. -- One container, one `/data` volume, and Docker Compose startup bound to - loopback by default. -- Direct Promptfoo evaluation of the built image. -- Upstream UHP conformance, separate downstream-extension coverage when needed, - image SBOM/provenance, and digest-pinned releases. - -### Non-goals - -- More than one repository, destination mapping, or repository composition. -- Non-Git workspace sources, private source credentials, SSH Git transports, or - caller-provided source headers. -- Raw commit-ID requests and SHA-256 Git repositories. -- Read-only workspaces, cross-session checkout reuse, prewarming, or shared - source object stores. -- Caller-selected lifetime, indefinite sessions, recovery after configured - expiry, or a separate lifetime subsystem. -- A generic extension registry, plugin framework, or multiple materializers. -- Provider login, token refresh, credential repair, or credential projection in - HarnessRouter. Those belong to the external provider gateway. -- A caller-selected provider base URL, provider API key, transport, or fallback - chain. -- Importing local profiles, synchronizing profile state, changing - `workspace.yaml`, or adding an AllAgents CLI command. -- Replacing HarnessRouter sessions, task execution, artifacts, streaming, - cancellation, or idempotency. -- New validation commitments for unrelated HarnessRouter backends. -- Kubernetes, multi-container worker orchestration, or a separately deployed - materializer service. -- A custom attestation or green-build framework. - -## External contracts - -### UHP request - -The proposed generic request shape is: +- A strict first-turn-only `metadata.workspace` HarnessRouter extension. +- Repository composition from one or more caller-declared repositories at + pairwise non-overlapping workspace-relative destinations. +- Fixed depth-2 Git acquisition with exact resolved-commit provenance and useful + recent offline history. +- Operator-cataloged OCI workspace snapshots selected by direct image-manifest + and workspace-manifest digests. +- One immutable generation store shared by Git and OCI sources. +- Reuse of a verified immutable generation across sessions. +- Shared immutable bytes for `read_only` sessions and inode-independent private + copies for `editable` sessions. +- Existing session continuation, checkpoint, cancellation, TTL, deletion, + restart reconciliation, files, artifacts, and produced-file behavior adapted + to the attachment. +- Authorized persistent retention through the existing session lifecycle. +- Exact source provenance and bounded, coded failures. +- Direct Promptfoo coverage against the built image, including a large OCI + workspace and second-session cache reuse. +- Digest-pinned publication to GHCR with SBOM and build provenance. + +### Explicit non-goals + +- A separate workspace service, workspace database, scheduler, Files API, or + task protocol. +- Caller-selected runtime/container images or benchmark environments. An OCI + workspace snapshot is source content only. +- Caller-provided registry origins, registry credentials, headers, proxy + settings, or source commands. +- Arbitrary materializer plugins, hook discovery, or a public generation API. +- Silent Git fallback for an OCI failure, silent OCI fallback for a Git failure, + or silent deepening/full-clone fallback for a bounded Git failure. +- Submodule initialization, Git LFS hydration, checkout filters, or repository + hook execution. +- Provider login, refresh, or repair in HarnessRouter; the external provider + gateway retains that responsibility. +- A caller-selected provider route, API key, transport, or fallback chain. +- Upstream acceptance as a release condition. Upstreaming is considered only + after downstream release evidence exists. + +## Request contract + +`metadata.workspace` is accepted only when the request creates a new session. It +uses snake_case and has no nested schema version: ```json { - "model": "gpt-5.4", - "input": "Inspect the project and fix the failing command.", "metadata": { "harness_id": "allagents-codex", "workspace": { - "repository": { - "url": "https://github.com/example/project.git", - "ref": "refs/heads/main" + "access": "editable", + "retention": "session", + "source": { + "kind": "repositories", + "repositories": [ + { + "url": "https://github.com/example/service.git", + "ref": "refs/heads/main", + "destination": "service" + }, + { + "url": "https://github.com/example/shared.git", + "destination": "libraries/shared" + } + ] }, - "working_directory": "packages/service" + "working_directory": "service" } } } ``` -The exact request object proposed to UHP is: +The exact shape is: ```text metadata.workspace = { - repository: { - url: string, - ref?: string - }, + access: "read_only" | "editable", + retention?: "session" | "persistent", + source: + | { + kind: "repositories", + repositories: Array<{ + url: string, + ref?: string, + destination: string + }> + } + | { + kind: "workspace_snapshot", + snapshot_name: string, + image_manifest_digest: string, + workspace_manifest_digest: string + }, working_directory?: string } ``` -The workspace object has no independent schema marker. An accepted UHP release -owns its standard schema; while it is downstream-only, the fork release owns the -extension shape. - Rules: -1. `workspace` and `repository` must be JSON objects, not arrays or strings. -2. `repository.url` is required. It must be an anonymous public `https://` Git - URL with no user info, query, fragment, alternate transport, or embedded - credential. -3. `repository.ref` is optional and non-empty when present. It must identify an - advertised `refs/heads/*` or `refs/tags/*`, or unambiguous branch/tag - shorthand. Raw object IDs and other ref namespaces are rejected. Only SHA-1 - repositories are accepted, and the exact resolved 40-hex commit is recorded - as provenance. -4. `working_directory` is optional. Omission means the checkout root. When - present it is a normalized relative POSIX path to a directory within the - checkout. -5. Unknown fields at every level are rejected. There are no aliases, commands, - environment variables, access modes, lifetime fields, destination paths, - materializer selectors, or provider settings. -6. HarnessRouter applies a small fixed metadata byte/depth bound before session - allocation. The materializer applies field-specific length bounds before - network or filesystem work. -7. A request without `metadata.workspace` follows ordinary upstream behavior. - -The upstream issue and UEP own final standard names when accepted. If review -changes the proposed shape, update the specification, schema, reference -implementation, conformance suite, examples, and this plan together. A -downstream-only implementation keeps the proposed names, documents their -extension status, and tracks any later UHP collision as a required clean -migration. - -### Session binding and public provenance - -After materialization, the session owns one immutable binding containing: - -- canonical requested repository URL; -- requested ref, or an explicit record that it was omitted; -- exact resolved 40-hex commit; -- runner-owned session root under `/data`; -- private checkout root beneath that session root; -- separate runner control root as a sibling of the checkout; -- execution working directory beneath the checkout; -- the governing UHP release; -- the workspace-contract owner and immutable revision, expressed internally as - the accepted UHP release or exact downstream source commit; -- creation time and server-owned expiry; and -- cleanup state sufficient to make removal idempotent. - -The checkout root, session root, control root, and cleanup token are internal and -must never appear in UHP responses, streams, logs, artifacts, or Promptfoo -reports. Successful responses expose only stable public provenance: +1. `access` and `source` are required. `retention` defaults to `session`. +2. `persistent` is accepted only after existing caller/session authorization + succeeds and before source resolution, network traffic, or generation claims. +3. `working_directory` and every repository `destination` are normalized + workspace-relative POSIX paths. The effective working directory must be a real + directory inside the final attached workspace without traversal or link + escape. +4. Repository destinations must be unique and pairwise non-overlapping: no two + destinations may be equal, and neither may be an ancestor of another. +5. The repository array contains 1 to 128 entries. Each URL, optional ref, and + destination is bounded before network or filesystem work. Lower runtime + capacity fails with the coded capacity error; it does not change schema + validity. +6. `source` is a closed discriminated union. Unknown fields and mixed Git/OCI + fields fail validation. +7. `snapshot_name` selects an operator-owned HarnessRouter deployment catalog + entry. The request supplies only the two `sha256:` digests; it never supplies + a registry, repository, credential, certificate, or mirror. +8. A continuation selected through `previous_response_id` or the existing + session recovery mechanism omits `metadata.workspace`. Supplying it on a + reused session fails before hydrate, source access, generation lookup, or + provider traffic, even when it is identical to the stored value. +9. A session created without workspace metadata remains a stock session and + cannot add workspace metadata later. +10. UHP input files are workspace mutations. Reject them on a `read_only` + first turn before source resolution. For `editable`, apply them only after + the private copy exists and before the initial produced-file baseline. + Harness assets and runner control state remain outside source content. +11. Workspace fields cannot contain commands, environment variables, resource + limits, provider settings, or harness settings. + +A workspace snapshot request is therefore: ```json { "metadata": { + "harness_id": "allagents-omp", "workspace": { - "repository": { - "url": "https://github.com/example/project.git", - "requested_ref": "refs/heads/main", - "resolved_commit": "0123456789abcdef0123456789abcdef01234567" + "access": "read_only", + "source": { + "kind": "workspace_snapshot", + "snapshot_name": "monorepo-release", + "image_manifest_digest": "sha256:...", + "workspace_manifest_digest": "sha256:..." }, - "working_directory": "packages/service" + "working_directory": "packages/api" } } } ``` -If the ref was omitted, `requested_ref` is omitted rather than synthesized. If -the working directory was omitted, `working_directory` is omitted. The same -public provenance is returned on successful continuations and idempotent -response retrieval. The selected contract and schema distinguish accepted -request fields from read-only response provenance fields; the UEP owns that -distinction when upstream accepts the contract. - -### Continuation - -A continuation normally supplies only `previous_response_id`: - -```json -{ - "previous_response_id": "resp_...", - "input": "Now run the focused check and summarize the result." -} -``` - -HarnessRouter may also reuse a session through its existing -`metadata.session_id` recovery path. After session resolution, every reused -session rejects `metadata.workspace`, regardless of which identifier selected -it. For a workspace-bound session, stored binding and harness state select -execution, and any caller-supplied harness must match exactly. A mismatch fails -before Git, hydration, or provider work. A reused unbound session without -workspace metadata retains ordinary upstream routing behavior. - -A valid workspace continuation reuses the exact session root, checkout root, -control root, execution working directory, and checkout mutations. It verifies -that the runtime supports the binding's recorded workspace-contract owner and -revision. A missing, expired, cleaned, identity-mismatched, or unsupported -binding fails closed; HarnessRouter never silently clones or reinterprets it. -Before activating an incompatible contract, an upgrade must explicitly migrate -compatible bindings or drain and delete them. - -### Provider and harness contract - -The downstream deployment defines two protocol-specific HarnessRouter -connections that point to one external OAuth gateway and use the same -server-side base URL and API-key secret: - -- a Responses-format connection used only by Codex; and -- an OpenAI Chat Completions connection used only by OMP. - -This is one external provider route with two logical protocol adapters, not two -credential authorities. The external gateway owns user login, upstream token -storage, refresh, and repair. - -- A new session selects only an allowed `metadata.harness_id` and model. -- A reused workspace-bound session derives its harness from stored state; a - supplied mismatch fails before execution. Unbound sessions retain upstream - routing. -- Each custom harness has an explicit model allowlist and a one-entry provider - policy pointing to its matching protocol-specific connection. -- There is no fallback connection or automatic transport switching. -- Provider base URL, API key, transport, headers, and model mapping cannot be - supplied in UHP input or workspace metadata. -- Provider failure is returned as an ordinary HarnessRouter/UHP failure. It does - not change workspace or routing state. -- HarnessRouter runs in brokered sandbox mode. A harness receives a short-lived, - session-scoped credential and loopback broker URL, never the long-lived - external-gateway key. The scoped credential exists only for the active turn - and is removed before checkpointing or public file collection. - -## Security and resource invariants - -These are release requirements, not later hardening: - -1. **URL and DNS:** Accept only public HTTPS destinations. Reject loopback, - link-local, private, carrier-grade NAT, documentation, multicast, reserved, - and otherwise non-public IPv4/IPv6 results. Validate every DNS answer before - connection, pin the validated address for that hop, revalidate every redirect, - and cap redirects. A public name that resolves to any forbidden address fails - closed. -2. **Git protocols:** Disable `file`, `ssh`, `git`, `ext`, and helper-driven - alternate protocols. Clear inherited Git configuration and credential - helpers. Disable terminal prompting. Requests never provide credentials. -3. **Repository execution:** Disable repository hooks and clean/smudge/process - filters. Do not initialize submodules. Detect and reject gitlinks and Git LFS - pointer-backed content instead of executing helpers or returning a partial - workspace as complete. -4. **Root separation:** Keep four explicit values: session root, checkout root, - control root, and execution working directory. The checkout and control roots - are separate children of the session root. Harness HOME, credentials, - scratch, skills, and runner state live only under control. The execution - directory is a no-follow-validated descendant of checkout. Repository content - cannot become control state. -5. **Filesystem confinement:** Build the checkout in sibling staging and publish - it only into an absent checkout target. Reject absolute paths, `..`, empty - segments, NUL, platform-separator ambiguity, and symlinks that escape the - checkout. -6. **Exact provenance:** Resolve the requested ref, fetch its commit, check out - detached, and verify `HEAD` equals the recorded object ID before publication. - Later ref movement cannot change the bound checkout. -7. **Bounds:** Enforce server-owned limits for request bytes, URL/ref/path - lengths, redirects, clone/fetch duration, materialized bytes, inodes, process - output, concurrent materializations, active harness time, and idle checkout - TTL. Apply limits during work, not only after completion. -8. **Process control:** Run Git and materializer children in a cancellable process - group with a minimal environment, bounded stdout/stderr capture, and a hard - termination deadline. Cancellation stops descendants. -9. **Crash-safe publication:** A workspace-aware first hydrate creates an - isolated empty session root with checkout absent. Persist a pending - reservation, materialize and validate in sibling staging, atomically publish - staging as checkout, create the separate control root, then atomically commit - the usable binding. Startup reconciliation removes abandoned staging, pending - reservations, published-but-unbound roots, and cleanup-marked roots. A - harness can never observe staging or a partially validated tree. -10. **Isolation:** Never bind one session to another session's checkout. Each - first turn creates a private checkout even when URL, ref, and resolved commit - are identical. -11. **Cleanup:** Removal is safe to repeat, never follows links, is confined to - the recorded session root, and cannot remove another session's data. -12. **Provider secrets:** `HR_SANDBOX_TRUST=owner` is forbidden. The local broker - exchanges a scoped turn credential for the real external-gateway key. The - long-lived key never enters the harness environment or filesystem. Generate - CLI credential config only in a per-turn ephemeral control subtree, exclude - it from checkpoints and public file/artifact APIs, and delete it before - terminal persistence. Neither long-lived keys nor scoped broker tokens may - survive in a checkpoint, session file, artifact, stream, response, or log. - -## Workstreams and implementation phases - -The phases are ordered by dependency. Each ends with observable behavior; source -inspection alone is not an exit criterion. - -### Phase 1: Establish the upstream issue and ownership decision - -**Outcome:** HarnessRouter maintainers have selected the governance path and -either accepted upstream ownership or recorded that the fork must own the -extension. +## Session binding and public provenance + +The fork stores workspace binding fields in the existing session/checkpoint +record. The binding contains: + +- the canonical effective descriptor and its digest; +- the source kind and exact resolved source identity; +- the generation key, immutable publication/epoch identity, and verified tree + manifest digest; +- `access`, effective `retention`, and normalized `working_directory`; +- the attachment method and evidence needed to prove the restored session still + refers to that exact generation; +- exact public provenance; +- existing session expiry/deletion state; and +- the selected harness/provider binding already owned by the session. + +Public response `metadata.workspace` has exactly `access`, `retention`, +`working_directory`, `effective_descriptor_digest`, `generation_id`, +`workspace_manifest_digest`, `provenance`, and `expires_at`. +`working_directory` is always present and uses `.` for the workspace root. +`expires_at` is the effective timestamp for `session` retention and `null` only +for authorized `persistent` retention. + +Repository `provenance` has `kind: "repositories"` and a request-order +`repositories` array. Each entry has normalized `url`, `destination`, exact +`resolved_commit`, effective `depth`, and `requested_ref` only when the request +supplied a ref. Snapshot `provenance` has `kind: "workspace_snapshot"`, +`snapshot_name`, exact `image_manifest_digest`, exact +`workspace_manifest_digest`, and a manifest-order `repositories` array. Each +snapshot root has `destination`; a history-bearing root also has +`resolved_commit` and `object_set_digest`, while a tree-only root has neither. + +Acquisition-policy revisions, catalog origins, mirrors, credentials, host and +mount paths, attachment IDs, leases, and internal generation keys are not +public. A continuation and response replay return the same committed object; +they never report a newly resolved ref or substituted generation. + +## Existing lifecycle integration + +The first-turn sequence is: + +1. Authenticate and validate the stock UHP envelope and establish existing + idempotency ownership. +2. Resolve whether the request creates or reuses a session. +3. If new and workspace-backed, validate and authorize the closed workspace + descriptor, then store a pending binding in the existing session transition. +4. Run fresh hydration only far enough to allocate the existing session + workspace and isolation identity. Skip stock empty-root Git initialization + for workspace-backed sessions; do not allocate another checkout root. +5. Create a per-session writable control root in that allocation but outside + source content. Redirect `.harness` state, HOME, conversation data, plugins, + skills, MCP/configuration, credentials, scratch, and produced/checkpoint + bookkeeping to it. +6. Resolve the immutable source identity, claim or reuse the generation, and + attach it at the existing workspace path with the requested access mode. +7. Reject workspace input files for `read_only`. For `editable`, apply them to + the private copy before the initial produced-file baseline. Supply generated + instructions through a harness-supported external instruction path or a + non-shadowing session mount; never write, overlay, or copy up a source path. +8. Establish runner-owned outer and nested produced-file cursors, validate the + effective working directory, and prove a `read_only` attachment has no + writable alias. +9. Atomically mark the existing session attachment ready, then continue through + ordinary provider selection and harness execution. +10. Collect files/artifacts and checkpoint through the access-specific path: + `read_only` stores only session-local mutable state plus exact attachment + evidence and excludes generation bytes; `editable` checkpoints its private + workspace copy. Stream events, set terminal state, and schedule cleanup + through existing HarnessRouter paths. + +A continuation does not parse or resolve a source. Workspace-aware hydration +first restores and validates only the control metadata needed for attachment: +generation key/epoch, manifest, durable reference, and attachment evidence. It +then acquires and mounts that exact protected generation, and only afterward +restores the remaining session-local harness state around the source mount. A +missing, expired, corrupt, wrong-generation, or unsupported attachment fails +closed. Hydration must not reacquire Git, contact an OCI registry, use another +cached generation, archive or restore shared generation bytes, or start with an +empty workspace. + +`retention: "session"` follows existing finite session TTL and deletion. +Authorized `persistent` retention pins the existing session and its generation +reference until explicit deletion or operator policy permits removal; it does +not create a second retention scheduler. Polling and response replay do not +extend retention. Cleanup makes the session unavailable before releasing its +attachment, private copy, or generation reference and remains idempotent across +restart. + +## Generation and attachment design + +A generation is a verified immutable source artifact, not a runnable workspace +or session. It is stored outside session allocations under runner-owned `/data` +state and can be attached only through the existing hydrate path. + +### Identity + +The canonical generation key includes only immutable source and layout inputs: + +- source kind; +- for every repository in request order: canonical normalized URL, exact + resolved commit, and normalized destination; or the selected OCI catalog + identity plus both direct digests; +- the normalized source-layout/workspace-manifest schema revision; +- materializer contract revision; +- Git fetch depth (`2`) and Git acquisition-policy revision for repository + sources; +- OCI extraction and validation-policy revision for snapshot sources; and +- any operator acquisition-policy identity that can change resulting bytes. + +The key excludes session ID, response ID, harness, provider, access, retention, +working directory, and caller display data. Those values affect attachment or +execution, not generation bytes. Different request ref spellings that resolve to +the same canonical repositories, commits, destinations, depth, and policy reuse +the same generation. Exact provenance retains the original requested values even +when the immutable generation key is shared. + + +### Git acquisition cache + +Repository sources use two server-owned cache levels inside runner-owned +`/data`; neither is a session workspace: + +1. One operator-only bare shallow acquisition mirror exists per canonical + repository URL. Only the runner's acquisition worker can write it. Every + refresh of that repository is serialized through the same mirror, and + concurrent refreshes of the same normalized ref request use one in-flight + operation. Depth and acquisition-policy revisions belong to immutable commit + snapshots and generation identity, not the mutable mirror key. +2. Verified depth-2 commit snapshots from that mirror feed immutable multi- + repository generations keyed by canonical URLs, exact commits, destinations, + depth, policy, and materializer contract revision. A generation is the only + cache object that can be attached to a session. + +The bare mirror stores acquired objects and shallow-boundary metadata so a +second generation needing the same commit does not clone or transfer its pack +again. Updates import a verified bounded fetch atomically; they never mutate a +published commit snapshot, silently deepen it, or make a partially refreshed +mirror eligible for generation construction. The generation builder exports a +self-contained repository with no alternates and no writable link to the mirror. +Mirror paths, file descriptors, credentials, refs, and writable internals are +never mounted into or disclosed to a session. + +Resolving a mutable advertised ref may contact the origin to determine its +current exact commit. Once resolution yields an identity already present in the +mirror and generation store, there is no source pack acquisition, checkout, +tree materialization, or publication. Separate counters distinguish ref +advertisement from source-byte acquisition so cache-reuse proof cannot count a +remote pack fetch as a hit. + +### Publication and reuse + +- Build into a random private sibling staging directory. +- Persist one singleflight claim per bare-mirror refresh and one per generation + key. Concurrent misses perform at most one remote pack acquisition and one + immutable generation publication. Other requests wait independently, and one + waiter's cancellation does not cancel work still needed by another live + waiter. +- Stream validation and accounting during acquisition. Verify the final manifest + before publication. +- Atomically rename verified staging into an immutable publication and record its + complete metadata in runner state. +- Treat bare commit snapshots and published generations as immutable. Startup + verifies recorded ownership, publication completeness, shallow boundary, + policy revision, and manifest evidence before readiness. +- A repository generation hit may refresh mutable ref advertisement, but performs + no clone, pack fetch, checkout, tree copy, or second publication after the + exact normalized source identity matches. An OCI digest-keyed hit performs no + registry manifest/blob request, extraction, tree copy, or second publication. + Both record only a new session lease/reference. +- Every `read_only` session whose normalized source identity matches a cached + generation attaches that same immutable generation. It cannot choose to + reclone, re-extract, rematerialize, or copy cached source bytes. +- Failed or cancelled builds remove staging after descendants stop. A partially + refreshed mirror, commit snapshot, or generation is quarantined and never + attached. +- Existing leases/session references protect a generation from cleanup. Existing + cleanup scheduling removes only complete, unreferenced generations under + bounded operator policy. Mirror/object-cache cleanup is separately confined + and cannot invalidate a referenced generation. + +### Access-specific attachment + +- `read_only`: take a lease and bind the cached immutable generation directly at + the existing session workspace through a read-only filesystem view. A second + matching session performs zero source-tree copy and shares the same verified + generation inodes/bytes, while the writable control root, HOME, checkpoint, + conversation, produced records, harness runtime, and lifecycle state remain + isolated. Workspace input files are absent. After control-root assets and + external instructions are ready, verify that root, nested paths, symlink + paths, bind aliases, and alternate descriptors cannot write or copy up into + source. +- `editable`: take a lease, then create a private, inode-independent tree from + the verified generation inside the existing session workspace allocation. + Reflink/copy is allowed only when it yields independent inodes and writes + cannot alter the generation or another session. Hard-linked mutable files are + forbidden. +- Both modes preserve nested repository administrative state allowed by the + source manifest. The attachment never exposes the bare Git mirror or moves + harness HOME, credentials, scratch, or checkpoint control into repository + content. +- Continuation reuses the exact lease and attachment. An editable continuation + sees its mutations; a read-only continuation sees the same immutable + generation and its own session-local state. + +## Implementation phases + +Each phase ends with observable proof. Source inspection or mock-forwarding +assertions are not sufficient. + +### Phase 1: Pin the baseline and record stock behavior + +**Outcome:** the unchanged baseline is reproducible and the fork has regression +proof for the lifecycle being extended. Work: -1. Open the upstream issue with the exact request shape, response provenance, - first-turn-only rule, continuation behavior, failure categories, source - restrictions, root model, lifecycle, and non-goals. -2. Link prior security analysis without importing AllAgents-specific naming into - the generic contract. -3. Draft the required UEP and identify compatibility effects on UHP clients, - schemas, conformance runners, and existing metadata behavior. -4. Obtain maintainer direction on allocation of `metadata.workspace`, generic - configuration names, materializer placement, runner interface, and release - target. -5. If no decision arrives within 30 calendar days after the issue opens, post a - follow-up after day 14 and record non-response at day 30 as deferral. -6. Split reviewable upstream changes according to maintainer preference while - keeping one coherent protocol contract. -7. Record explicit decisions and update every example when review changes a - name or behavior. - -Exit gate: - -- the upstream issue exists and links the UEP when required; -- maintainers have made an explicit decision, or the documented non-response - window has elapsed and is recorded as deferral; -- accepted semantics are reserved through governance, or downstream semantics - are explicitly labeled as a HarnessRouter fork extension; and -- no unresolved decision blocks implementation in the selected ownership path. - -### Phase 2: Pin the fork baseline and supply chain - -**Outcome:** `allagentsdev/harnessrouter` can reproducibly build the examined -upstream baseline and identify both that baseline and every downstream workspace -commit when the extension is not accepted upstream. +1. Pin the initial examined baseline to HarnessRouter `v0.25.4`, commit + `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`, and UHP `2026-09-12`. Before + implementation, record the exact release baseline actually selected; move it + only in a standalone synchronization change. +2. Preserve `HarnessRouter/harnessrouter` as the upstream remote and + `allagentsdev/harnessrouter` as the existing fork. +3. Pin base image, OS packages, Git and OCI libraries/tools, Codex, OMP, + Promptfoo, lockfiles, and CI actions. Build the unchanged image first. +4. Add focused characterization scenarios for fresh empty-root Git hydration, + continuation checkpoint restore, attached-file/harness-asset ordering, + root-Git produced-file cursor behavior, cancellation, TTL cleanup, restart, + and requests with arbitrary metadata. +5. Capture stock UHP request/stream/error behavior for requests without + `metadata.workspace`; these traces become compatibility fixtures. +6. Record the precise gateway/session/hydrate/runner/checkpoint/files call path + in code comments or tests where the fork seam lands. Do not add a generic + extension framework. + +Exit proof: + +- a clean checkout builds the unchanged pinned image; +- characterization runs demonstrate all seven inventory facts; and +- a stock request completes through each supported existing route with no + workspace-specific state. + +### Phase 2: Add request validation and immutable session binding + +**Outcome:** the gateway accepts the exact first-turn descriptor and binds it to +the existing session transition without changing stock requests. + +Primary surfaces are the existing UHP request handling/session resolution in +`gateway/app.py`, the existing gateway-to-runner turn envelope, existing session +persistence, focused integration tests, changelog, and extension documentation. Work: -1. Preserve `HarnessRouter/harnessrouter` as the documented upstream remote and - record v0.25.4 commit - `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` as the initial examined baseline. -2. Keep upstream synchronization commits separate from downstream deployment - commits. Never combine a baseline jump with a product behavior change. -3. Pin the selected UHP release, base image by manifest digest, Codex and OMP - runtime releases, Git/materialization system packages, Promptfoo dependency - and lockfile, and every CI action. No `latest`, floating branch, or unbounded - package range enters a release. -4. Configure image publication for `ghcr.io/allagentsdev/harnessrouter`. -5. Add an upstream-diff check so each candidate records its exact upstream tag - and commit plus every downstream commit in the image. -6. Use the upstream-baseline-plus-downstream-revision tag convention - `-allagents.`, for example - `v0.25.4-allagents.1`. Increment the downstream revision for any image change - on the same upstream baseline; reset it to `.1` when the upstream baseline - tag changes. -7. Use the same tag for the downstream source release and OCI image, then deploy - the image only by manifest digest, for example - `ghcr.io/allagentsdev/harnessrouter:v0.25.4-allagents.1@sha256:`. - -Behavior-focused proof: - -- a clean fork checkout builds the recorded baseline from pinned inputs; -- source links, OCI labels, release metadata, and SBOM identify - `allagentsdev/harnessrouter`, the exact upstream commit, and the downstream - revision; and -- changing a pin, baseline, or lockfile appears as a reviewed source diff. - -Exit gate: the unchanged pinned baseline image builds before workspace behavior -or downstream deployment configuration is added. - -### Phase 3: Land the workspace contract in the selected ownership path - -**Outcome:** the contract, parser, and behavior tests agree on one bounded -first-turn workspace object; requests without it retain ordinary behavior. When -accepted upstream, specification, machine schema, conformance, changelog, and -user documentation define the same standard contract. - -Primary areas: UHP specification/schema and conformance when accepted upstream; -`gateway/app.py`, focused gateway tests, changelog, and user documentation in -either ownership path. +1. Parse only `metadata.workspace`; keep unrelated metadata behavior unchanged. +2. Apply metadata byte, nesting, list-count, and string-length bounds before + session allocation or source work. +3. Strictly validate the closed union, snake_case names, access, retention, + destinations, working directory, direct `sha256:` digest syntax, and the + `read_only` input-file exclusion. +4. Authorize persistent retention before source resolution or generation lookup. +5. Canonically serialize the effective descriptor with the default + `retention: "session"` and calculate its digest. +6. Extend the existing new-session transition with pending/ready workspace + binding states. Do not add another response or session identity. +7. Reject the descriptor on every reused session path before hydration. On a + workspace-bound continuation, derive source/access/retention/cwd/harness from + stored state and reject a supplied harness mismatch through existing session + rules. +8. Preserve idempotency: the owning initial request performs one binding; a + duplicate receives the same response/session result and cannot claim another + generation or attachment. +9. Map validation, authorization, reuse, and binding failures into bounded UHP + errors without internal paths or secret/catalog details. +10. Add public provenance only after attachment is ready. Retrieval/replay uses + stored provenance rather than resolving it again. + +Proof includes valid descriptors for both source kinds; repository counts +`0`, `1`, `128`, and `129`; multiple repository entries; default retention; +authorized and unauthorized persistence; invalid unions, fields, digests, +paths, and destinations; workspace injection on both continuation mechanisms; +idempotent duplicates; harness mismatch; exact response-schema fixtures for +both provenance variants; and a byte-for-byte stock trace for requests without +the descriptor. + +### Phase 3: Add the immutable generation store and existing-workspace attachment seam + +**Outcome:** one generic runner seam claims, publishes, reuses, and attaches a +verified generation inside the existing hydrate lifecycle. + +Primary surfaces are existing runner/session hydrate code in `runner/server.py`, +its current gateway transport, checkpoint/session persistence, startup +reconciliation, and existing cleanup scheduling. Work: -1. Define the request and public provenance shapes through the upstream review. - An accepted UHP release owns standard compatibility and schema evolution; - otherwise the downstream fork release owns the extension shape. -2. Parse `metadata.workspace` only on an initial request. Leave unrelated - metadata unchanged. -3. Apply generic metadata byte/depth bounds before response or session - allocation. -4. Strictly validate required and optional fields, unknown-field rejection, - anonymous public HTTPS policy, and normalized relative - `working_directory` rules. -5. Extend session resolution to report whether it created or reused a session. - Store the canonical descriptor only after proving the session is new. Use one - canonical serialization and digest so idempotent replay cannot create a - second binding. -6. Reject workspace metadata for every reused session selected by either - `previous_response_id` or `metadata.session_id`, before Git, hydration, - runner, or provider work. -7. For a reused workspace-bound session, derive the harness from stored state and - reject a caller-supplied mismatch before hydration. Leave reused unbound - sessions on the ordinary route. -8. Map validation and reuse failures to stable UHP errors with the offending - parameter named. Never expose exceptions or internal paths. -9. When accepted upstream, update the specification, machine schema, reference - implementation, conformance suite, changelog, and docs in the same change - set. Otherwise update the fork implementation, extension tests, changelog, - and docs together without altering UHP conformance definitions. - -Acceptance examples: - -| Case | Observable result | -|---|---| -| Valid URL only | Accepted; checkout root becomes the effective working directory | -| Valid URL, ref, and nested directory | Accepted; values reach the single workspace hook | -| Missing URL, unknown field, array, or non-string field | 400 before materialization | -| Absolute or parent-traversing `working_directory` | 400 before materialization | -| Continuation containing `metadata.workspace` | 409 before materialization | -| Existing `metadata.session_id` plus workspace object | 409 before materialization | -| Workspace-bound session with a different `harness_id` | 409 before hydration | -| No workspace object | Same status, response, and runner path as upstream baseline | - -Tests assert the HTTP/UHP contract and absence of materializer/provider activity, -not helper calls or field-copy plumbing. - -### Phase 4: Land runner and materializer seams in the selected ownership path - -**Outcome:** HarnessRouter can prepare a workspace after session resolution and -before provider execution while preserving its ordinary path. - -Primary areas: `gateway/app.py`, `runner/server.py`, their existing transport, -the materializer package, focused integration tests, and operator docs. These -land upstream when accepted and otherwise remain an explicit fork patch. +1. Add one internal `resolve -> claim/reuse -> materialize -> verify -> publish -> + attach` pipeline selected by the closed source union. It is not a public API or + plugin registry. +2. Replace only the fresh empty-root initialization point for workspace-backed + sessions. The designated path remains the existing session workspace. +3. Persist bare-mirror refresh/snapshot state, generation claims, staging, + publication, session leases/references, and attachment-ready transitions using + the runner's current durable state and recovery ordering. +4. Build and verify the canonical source-visible manifest: normalized relative + path, type, mode, size/content identity, link target where applicable, declared + repository ownership, and optional normalized Git-history declaration. +5. Implement access-specific attachment and verify read-only alias resistance or + editable inode independence before marking the session ready. +6. Add a per-session writable control root outside source content and redirect + every stock workspace-internal mutable path there: `.harness`, HOME, + conversation state, plugins, skills, MCP/configuration, credentials, scratch, + produced-file indexes, and checkpoint control. No mutable runtime path may + resolve into an immutable generation. +7. Use harness-supported external instruction inputs or a non-shadowing + session-only mount for generated instructions. Reject the implementation if + either supported harness requires overwriting, overlaying, or copying up a + source path. +8. Reject UHP input files for `read_only`; for `editable`, apply them to the + private copy at the normal pre-turn point and establish the initial + produced-file baseline afterward. +9. Add access-specific checkpoint/hydrate behavior. A read-only checkpoint + excludes the generation mount and source bytes, persists only session-local + mutable state plus exact attachment evidence, and reattaches the same + generation before restore. An editable checkpoint carries the private + workspace copy and never the mirror or generation backing store. +10. Reuse existing user/sandbox isolation, process groups, timeouts, + cancellation, TTL, deletion, and cleanup. Source workers inherit + cancellation and must stop all descendants before terminal acknowledgement. +11. Reconcile abandoned mirror refreshes, immutable commit snapshots, staging, + incomplete publication, ready generations, pending attachments, leases, + references, private copies, control roots, and cleanup after restart. +12. Verify continuations against the stored generation epoch/manifest, durable + reference, and attachment evidence; never resolve or materialize source on + continuation. +13. Expose separate bounded counters/events for ref resolution, remote pack + acquisition, mirror singleflight, generation build/publication/hit, + read-only zero-copy attachment/checkpoint, editable copy/checkpoint, + reference, cancellation, quarantine, and cleanup. Do not log request + credentials, registry locations, internal paths, or uncontrolled tool + output. + +Proof uses deterministic fake Git and OCI builders to show one mirror refresh +and one publication under concurrent misses, independent waiter cancellation, +two read-only sessions directly attaching the same verified generation with +zero second clone/fetch/materialization/copy, and two editable sessions with +independent inodes and mutations. Real Codex and OMP probes prove generated +instructions and all mutable harness assets live outside source. A read-only +checkpoint contains no generation bytes. Continuation restores and validates +minimal binding/control metadata, reattaches the same protected generation, +then restores remaining session-local harness state; checkpoint creation writes +no root Git commit. Editable checkpoint/continuation preserves its private +mutations. Restart at every persisted transition and cleanup are idempotent. +Session/runtime state remains isolated even when immutable bytes are shared. A +stock session still creates and uses its normal root Git workspace. + +### Phase 4: Implement repository composition with mandatory depth-2 acquisition + +**Outcome:** repository mode deterministically builds one generation from one or +more repositories while retaining bounded recent Git history. Work: -1. Extend the existing gateway-to-runner turn envelope with an optional canonical - first-turn workspace descriptor and an optional hydrated binding for a - continuation. Do not add a public endpoint. -2. Add a first-hydrate mode that performs existing isolation, wipe, and ownership - setup but leaves an empty session root with checkout absent. It must not run - stock Git initialization, write `.gitignore`, apply input files, or create - harness state before publication. -3. Define one runner-owned materializer interface with two operations: - `materialize(first_turn_descriptor, checkout_target, limits, cancellation)` - and `cleanup(binding)`. Use one configured implementation; do not add plugin - discovery, registration, or hook chaining. -4. Preserve the four explicit root values in the binding. Validate the execution - working directory without following links; keep control as a sibling of - checkout. -5. After materialization, finish the pending-reservation/publication transition, - create control, then atomically commit the binding and public provenance. -6. On continuation, hydrate and verify the stored roots and identity. Do not - resolve, fetch, checkout, or validate caller workspace metadata again. -7. Spawn the harness in the execution working directory. Anchor durable HOME, - scratch, skills, and CLI state in control. Scope input files, produced-file - Git diffing, file APIs, and artifacts to checkout. Checkpoint the session root - only after removing ephemeral credential state. Do not initialize an outer - repository or overwrite the source repository's `.gitignore`. -8. Make first-turn transition idempotent. Same-key replay returns the owning - response and binding. A competing request cannot materialize or bind a second - checkout for the same session. -9. Preserve streaming, cancellation, terminal-state, idempotency, artifact, and - provider-error ordering. Workspace failure terminates before provider traffic. -10. Implement typed materializer request, result, and error records. Results - include internal roots plus safe public provenance; errors contain neither - secrets nor uncontrolled Git output. -11. Canonicalize and authorize the HTTPS URL before Git. Use one acquisition path - for DNS, redirects, IP policy, ref advertisement, and fetch. -12. Resolve omitted ref through the advertised symbolic default. Resolve full - branch/tag refs or unambiguous shorthand from advertised SHA-1 refs. Reject - raw object IDs, other namespaces, SHA-256 repositories, missing refs, and - ambiguous shorthand. Peel annotated tags and require a commit. -13. Create restrictive random staging as a sibling of the absent checkout target. - Fetch only the advertised ref needed for the resolved commit, disable helper - execution, and check out detached with isolated Git configuration. -14. Reject submodule entries and LFS-managed content. Validate confinement, - bytes, inodes, `HEAD`, and optional execution directory. -15. Atomically publish validated staging to checkout. Never write control or - derive a host path from URL, ref, working directory, response ID, or other - caller text. -16. Remove staging on every error, timeout, cancellation, and recovery sweep. - Bound concurrency with one server-owned semaphore and emit stable staged - errors with capped private stderr. -17. Document the runner/materializer contract and configuration in the owning - HarnessRouter changelog and operator docs; include it in UHP documentation - only when accepted as standard behavior. - -Behavior-focused proof: - -- a probe harness sees a committed repository file on its first instruction; -- `working_directory` becomes process cwd while HOME and runner state remain - outside checkout; -- repository `.harness` paths and `.gitignore` cannot collide with control state; -- exact commit remains fixed if the branch later advances; -- omitted ref selects the advertised default branch; -- raw object IDs, SHA-256 repositories, escaping paths, forbidden destinations, - protocols, credentials, hooks, filters, submodules, and LFS content fail; -- byte, inode, time, output, redirect, and concurrency limits apply during work; -- materializer failure produces no provider request, agent process, binding, or - published checkout; -- cancellation kills Git descendants and removes staging; -- restart and continuation restore checkout mutations and control state; -- two sessions at the same commit receive distinct writable roots; and -- ordinary requests exercise the unchanged upstream sequence. - -Use controlled origins and resolvers for hostile network cases and a stable -public fixture repository for built-image proof. Tests observe files, commit -identity, isolation, errors, and process termination rather than mock argument -forwarding. - -### Phase 5: Complete continuation and cleanup lifecycle - -**Outcome:** checkout lifecycle follows the existing session lifecycle with -bounded ephemeral storage and crash-safe recovery in the selected ownership -path. +1. Validate each caller URL under the fixed deployment egress policy. Callers may + not supply credentials, proxy configuration, Git config, or transport + options. Re-authorize redirects and resolved addresses; isolate Git config and + disable interactive helpers, hooks, filters, alternate protocols, submodules, + and LFS hydration. +2. Resolve an omitted ref through the advertised symbolic default. Resolve an + explicit advertised branch or tag, peel annotated tags as required, and + record the exact commit before fetch. Reject ambiguous, missing, unsupported, + or non-commit targets. +3. Maintain one operator-only bare shallow acquisition mirror per canonical URL. + Serialize every write for that repository through the same mirror and + singleflight concurrent refreshes for the same normalized ref request. +4. On a mirror miss, fetch exactly depth 2 with `--depth=2` into a private + bounded refresh area, verify it, and atomically import its pack/object and + shallow-boundary state. Fetch only the selected advertised branch/tag path. + Do not retry with a larger depth, `--unshallow`, full clone, arbitrary + object-ID fetch, another ref, or another source mode. +5. Verify the fetched tip/peeled commit exactly equals the commit observed during + resolution. A ref movement race fails the refresh rather than caching or + binding different bytes. +6. Publish an immutable commit snapshot inside the mirror cache. A cache hit for + the exact canonical URL, commit, depth, and policy performs no clone or remote + pack transfer. No session can access the mirror path or a writable mirror file + descriptor. +7. Export the selected cached commit into generation staging as a self-contained + repository with no alternates or writable link to the mirror. Preserve + `.git/shallow` and sufficient normalized administrative state for recent + offline `git log`, parent inspection, and diff. +8. For a merge tip, preserve both fetched parent edges at depth 2 and validate + the shallow boundary rather than flattening the merge. +9. Reject a server that cannot satisfy the bounded shallow fetch. Return a coded + source error and do not deepen, full-clone, strip history, or fall back to OCI. +10. Check out the exact detached commit at each declared destination. Reject + submodule gitlinks and LFS pointer-backed content rather than fetching them. +11. Enforce destination non-overlap before network work, then build all + repositories into one staging tree. No repository may create paths outside + its destination or add undeclared root files. +12. Validate each repository's `HEAD`, index/worktree equality at publication, + shallow metadata, closed refs/config, object reachability for the retained + depth, file modes, links, bytes, inodes, and absence of credentials/remotes + that would cause later network use. +13. Compute exact generation identity from canonical URL identity, resolved + commit, destination, fixed depth `2`, acquisition-policy revision, and layout + policy. Preserve requested URL/ref separately as provenance. +14. Clean incomplete mirror refresh, repository staging, and descendants on + error, timeout, cancellation, lost claim, or restart without invalidating a + previously verified commit snapshot or referenced generation. + +Behavior proof covers one repository at a top-level destination; several sibling/nested-path +repositories; the same URL at different refs and destinations; omitted default, +branch, lightweight tag, annotated tag, and moving-ref rejection; pairwise +non-overlap; depth exactly 2; `.git/shallow`; two-entry recent offline history; +merge-tip parents and diff semantics; detached exact commit; no network during +attached `git log`; submodule/LFS rejection; unsupported shallow server with no +fallback; cancellation; restart cleanup; and exact provenance. Concurrent cold +requests produce one advertised-ref refresh, one remote pack fetch, one verified +mirror snapshot, and one generation publication. After resolution confirms the +same exact identity, a second read-only request performs zero clone, zero pack +transfer, zero checkout/materialization, and zero tree copy, attaches the same +immutable generation bytes, and retains isolated session/runtime state. + +The release notes must state plainly that depth 2 reduces transferred history, +not the checked-out working-tree bytes. Large repositories still require the OCI +snapshot source and its release gate. + +### Phase 5: Implement mandatory OCI workspace snapshot materialization + +**Outcome:** the same generation pipeline safely restores an operator-cataloged, +digest-pinned workspace snapshot with no Git fallback. + +Deployment configuration owns a bounded snapshot catalog. Each `snapshot_name` +maps to one operator-controlled registry/repository origin, credential reference, +TLS policy, allowed media types, and resource policy. The request and public +provenance never reveal those private values. This catalog is HarnessRouter +configuration; it is not `workspace.yaml`. Work: -1. Add the server-owned idle TTL configuration selected through review, with a - finite safe default. Callers cannot set or extend it. -2. Start or reset idle expiry only after a terminal turn is durably recorded. An - active materialization or harness turn is never removed by the sweeper. -3. On continuation, verify stored identity, contract owner and revision, and all - roots, then use the same working directory and mutations. Reset expiry only - after the turn becomes terminal. -4. On first-turn cancellation during materialization, terminate the process - group, remove staging, and leave no resumable binding. -5. On cancellation after publication, stop the harness through ordinary - HarnessRouter behavior and retain the checkout only until finite idle expiry, - allowing a permitted continuation to see prior mutations. -6. On expiry or existing-session deletion, atomically mark the binding - unavailable before confined idempotent cleanup. A late continuation fails - closed and cannot recreate checkout. -7. On startup, reconcile abandoned staging, pending reservations, - published-but-unbound roots, and cleanup-marked session roots. Do not add a - second database, durable queue, deletion ledger, or general storage collector. -8. If cleanup encounters a transient host error, keep binding unavailable, emit - an operator-visible error, and retry the same idempotent removal on the next - bounded sweep or startup. Never make checkout executable again. -9. Before activating an incompatible workspace contract, explicitly migrate - compatible bindings or drain and delete them. Unsupported revisions fail - closed; do not add aliases or reinterpret stored descriptors. -10. Add lifecycle behavior to UHP conformance only where accepted and - protocol-visible. Always cover implementation behavior with focused - HarnessRouter integration tests in the owning repository. - -Acceptance examples: - -| Scenario | Observable result | -|---|---| -| Turn 1 edits a file; turn 2 reads it | Turn 2 sees the edit in the same checkout | -| Turn 2 omits workspace metadata | Stored binding selects checkout and cwd | -| Turn 2 includes workspace metadata | Rejected before Git or provider activity | -| Checkout missing or identity mismatched | Continuation fails; no rematerialization | -| Git cancellation | Child processes exit and staging disappears | -| Harness cancellation | Response is cancelled; checkout follows finite idle expiry | -| Expiry races with continuation | Existing session serialization selects one winner; checkout is not used after cleanup begins | -| Cleanup runs twice or after restart | Same absent final state; no neighboring path changes | +1. Require a direct OCI image manifest digest. Reject tags, mutable references, + manifest indexes/lists, caller-selected repositories, and catalog/digest + mismatches. +2. Fetch the direct image manifest, config, workspace manifest, and referenced + layers only from the selected catalog entry. Implement bounded registry + authentication and exact-host redirect policy without exposing credentials to + the harness, session workspace, logs, response, or provenance. +3. Verify every descriptor digest and size before use. Require the declared + `workspace_manifest_digest` to identify the exact workspace manifest used to + validate the final tree. +4. Stream decompression and extraction inside the fixed v1 envelope: at most 64 + distributable tar/gzip/zstd layers; a 4 MiB image manifest; a 128 MiB + workspace manifest with at most 128 repository roots; 8 GiB total compressed + layer bytes; 32 GiB expanded source bytes; 500,000 entries; 4 GiB per regular + file; paths of at most 4096 UTF-8 bytes and 128 components; and 1 MiB per PAX + or extended header. For each layer and the aggregate artifact, + `expanded_bytes / max(compressed_bytes, 1)` must not exceed `100`. Enforce + these bounds plus inode, output, and wall-time limits during streaming. + Deployment configuration may lower but cannot raise them without a contract + revision. +5. Apply OCI layers in order with correct file and opaque-directory whiteout + semantics. Whiteouts are extraction instructions and must never appear in the + published workspace. Reject malformed whiteouts and type transitions not + representable by the workspace manifest. +6. Reject absolute paths, traversal, NULs, ambiguous separators, duplicate + conflicting entries, devices, FIFOs, sockets, unsafe sparse files, and other + unsupported types. Validate symlink and hardlink targets against the final + workspace root; reject escaping, dangling-required-target, forward-link, and + link-cycle cases outside the supported bounded model. +7. Validate final paths, types, modes, sizes, content digests, links, repository + roots, and destination non-overlap against the workspace manifest. Extra, + missing, or changed source-visible entries fail before publication. +8. Support tree-only repository roots and optional normalized offline Git + history. For a history-bearing root, require detached `HEAD`, exact + index/tree/worktree equality, closed object reachability and declared object + digest, bounded refs/config, and no remotes, credentials, alternates, hooks, + includes, worktrees, replace/graft state, or unsafe administrative files. + Tree-only roots must not contain undeclared `.git` state. +9. Return the same canonical manifest/generation envelope as repository mode. + Include direct image/workspace digests and snapshot name in identity and exact + provenance; exclude registry origin and credentials. +10. On any resolution, registry, digest, extraction, manifest, Git-history, + cancellation, or capacity failure, terminate descendants, remove staging, + and return the source-specific error. Never clone Git, select a different + digest, or use a stale generation as fallback. + +Proof uses a local authenticated registry and malicious fixtures for digest/media +mismatch, indexes, redirects, authentication, truncation, compression bombs, +layer limits, whiteouts and opaque whiteouts, traversal, path/type/link attacks, +devices, sparse files, cancellation, partial cleanup, and restart. Positive +fixtures cover a tree-only workspace, multiple declared repository roots, +normalized offline Git history, offline `git log`/`git blame`/historical diff, +read-only attachment, editable copy, concurrent publication, and cache reuse +with zero second-session registry or extraction work. + +OCI implementation and this proof are required before v1 release. A passing Git +path cannot waive or defer them. + +### Phase 6: Adapt produced-file collection for nested and multiple repositories + +**Outcome:** existing Files/artifact behavior reports turn-produced changes +across composed workspaces without inventing a second file API. -### Phase 6: Add downstream Codex, OMP, and provider wiring +Work: -**Outcome:** the fork supplies AllAgents deployment policy while consuming the -accepted upstream capability or its explicitly documented downstream workspace -patch unchanged. +1. Preserve the existing produced-file cursor semantics but move the outer + workspace baseline/index into the per-session control root. Never initialize, + commit, or mutate a bookkeeping `.git` directory inside an immutable + generation. +2. Register source-manifest repository roots and history mode when the attachment + becomes ready. The set is immutable for the session. +3. At each turn boundary, record the external outer baseline plus a cursor for + each declared repository root: Git `HEAD`/index/worktree state for + history-bearing roots and manifest/file identity for tree-only roots. +4. Collect the union of outer and nested additions, modifications, deletions, + renames, and mode changes relative to the turn baseline. Normalize to + workspace-relative paths, assign each path to the most specific declared + owner, deduplicate it, and preserve existing file size/count/type limits. +5. Never expose `.git` administrative files, generation-store paths, mount + internals, control state, provider credentials, harness assets, or checkpoint + internals as produced files. +6. Do not report immutable source baseline files merely because they arrived + during first-turn composition. Report only changes after the established + source/turn baseline. +7. In `read_only`, any attempted source mutation fails at the filesystem boundary + and produces no changed source entry. Collection reads the external baseline + without modifying source. +8. In `editable`, changes remain private to the session and are visible on + continuation and through the existing file and artifact APIs. +9. Preserve collection-before-checkpoint ordering. Read-only checkpointing skips + the generation mount; editable checkpointing includes only the private + workspace and session-local state. Cancellation cannot publish a partial + cursor or checkpoint. + +Proof covers changes at workspace root and in every nested repository; two +repositories changed in one turn; same filename under different destinations; +add/modify/delete/rename; tree-only OCI roots; history-bearing OCI roots; Git +shallow roots; paths outside repository destinations; ignored files under the +existing policy; read-only denial; editable continuation; cancellation during +collection; restart; bounds; and absence of `.git`, credentials, generation +paths, or duplicate records. + +### Phase 7: Wire Codex, OMP, and the existing provider gateway + +**Outcome:** both supported harnesses execute in the attached existing workspace +without gaining source or long-lived provider credentials. -Work in `allagentsdev/harnessrouter`: +Work: 1. Install exact pinned Codex and OMP releases and enable only required release backends with `HR_BACKENDS=codex,omp`. -2. Define stable custom harness IDs such as `allagents-codex` and - `allagents-omp`. Store instructions, tools, allowed models, and base harness - in downstream deployment configuration. -3. Configure two logical connections using the same external-gateway base URL - and API-key secret: Responses for Codex and OpenAI Chat Completions for OMP. - Give each harness policy one matching connection and no fallback. -4. Fail startup/readiness if either selected model and endpoint combination is - unsupported. Never switch transport or connection after an error. -5. Force `HR_SANDBOX_TRUST=broker`. Configure the local loopback broker so - self-host mode does not require a public broker URL. -6. Add readiness proof that mints a scoped turn credential, reaches the loopback - broker, and keeps the real external-gateway key gateway-side. -7. Reject caller-selected models outside the harness allowlist and any request - field that attempts to replace provider routing. -8. Give OMP ordinary session-local home/config and cwd. Generate its - credential-bearing `models.json` and `models.yml` only under a per-turn - ephemeral control subtree using the scoped token. Delete both before terminal - checkpointing and exclude their paths from checkpoints and public walkers. -9. Redact the external key and scoped tokens from logs, traces, stored responses, - session files, checkpoints, artifacts, and snapshots. - -Acceptance examples: - -- Codex completes a real Responses turn through the external gateway; -- OMP completes a real Chat Completions turn through the same external gateway; -- each harness starts inside the materialized execution working directory; -- unsupported model fails before provider traffic; -- invalid provider credentials return the ordinary provider failure without - selecting another connection; and -- callers cannot observe or override provider URL, key, or transport. - -### Phase 7: Build the one-container operator surface - -**Outcome:** an operator can start the AllAgents-maintained HarnessRouter image -with Docker Compose, one data volume, and no helper service. - -The checked-in Compose contract is equivalent to: - -```yaml -services: - harnessrouter: - image: ghcr.io/allagentsdev/harnessrouter:${HARNESSROUTER_IMAGE_TAG}@${HARNESSROUTER_IMAGE_DIGEST} - ports: - - "127.0.0.1:3000:3000" - env_file: - - .env - environment: - HR_BACKENDS: codex,omp - HR_SANDBOX_TRUST: broker - HR_WORKSPACE_TTL_SECONDS: ${HR_WORKSPACE_TTL_SECONDS:-3600} - volumes: - - harnessrouter-data:/data - restart: on-failure - -volumes: - harnessrouter-data: -``` - -The final environment key uses the upstream-approved name when accepted and the -documented fork name otherwise. The actual Compose file also carries -HarnessRouter's required caller authentication, secret, -health, and process settings. The operator supplies one external provider base -URL and key through two protocol-specific connection records; neither value is -baked into the image. - -Startup contract: - -1. Copy the example environment file and set non-default Console/caller - credentials, provider route, model allowlists, TTL/resource limits, immutable - image tag, and manifest digest. -2. Run `docker compose up -d`. -3. Readiness succeeds only after HarnessRouter, runner, Codex/OMP runtimes, - materializer, writable `/data`, custom harnesses, both provider connections, - and loopback credential broker pass startup checks. Owner-trust pass-through - fails readiness. -4. The UHP base remains HarnessRouter's existing - `http://127.0.0.1:3000/api/harness`; remote access requires operator-owned TLS - and network controls. -5. Restart with the same volume preserves unexpired sessions and private - checkouts. Reconciliation removes only abandoned or cleanup-marked data. - -Container requirements: - -- the materializer is installed in the same image and invoked locally; -- Git and certificate roots are pinned and present; -- service binds to loopback by default; -- `/data` is the only required durable mount; -- secrets are runtime inputs, not layers, labels, build arguments, or examples; -- image has standard SBOM and build provenance; and -- OCI labels record source revision, upstream tag and commit, downstream - revision, UHP release, and runtime releases. - -### Phase 8: Add direct Promptfoo smoke and E2E coverage - -**Outcome:** lockfile-pinned Promptfoo calls the built image directly over UHP -and proves user-visible workspace behavior. - -Suggested downstream area: `e2e/promptfoo/` in -`allagentsdev/harnessrouter`, containing only Promptfoo configuration, small -fixtures/assertions, and package metadata. - -Work: - -1. Configure Promptfoo's OpenAI Responses-compatible provider directly against - `/api/harness/v1/responses`, with the HarnessRouter API key in an environment - variable and `metadata.workspace` in the request. Do not place an HTTP adapter - or another repository between Promptfoo and HarnessRouter. -2. Use a stable public Git fixture with known commits and a task that succeeds - only when the harness starts in the materialized checkout. -3. Run the same first-turn scenario for `allagents-codex` and `allagents-omp`, - using each harness's allowed model and provider transport. -4. Add a two-turn scenario that mutates a uniquely named file on turn one and - reads or changes it on turn two through `previous_response_id`, without - resending workspace metadata. -5. Cover malformed metadata, forbidden URL resolution, missing ref, invalid - working directory, source limit, provider failure, cancellation during Git, - cancellation during harness execution, continuation after cleanup, - reused-session workspace injection, and cross-harness mismatch. -6. Assert the expected resolved commit in public provenance and absence of - internal paths and credentials. -7. Use canary external keys and scoped tokens; assert both are absent from - session files, checkpoints, artifacts, logs, reports, and data retained after - each turn. -8. Exercise the built image, not an in-process server. Store only sanitized - reports; never upload provider traffic, secret-bearing prompts, or volume - contents. - -Focused permanent tests protect schema boundaries, security invariants, -ordering, session transitions, and cleanup races. Do not add tests that merely -assert config keys, field copies, mocks, or source text. - -Release-blocking E2E matrix: - -| Harness | First turn | Continuation | Cancellation | Provider failure | -|---|---:|---:|---:|---:| -| Codex / Responses | required | required | required | required | -| OMP / Chat Completions | required | required | required | required | - -### Phase 9: Integrate the upstream baseline and release the downstream image - -**Outcome:** the fork consumes an exact upstream baseline, carries only the -workspace delta required by the recorded ownership decision, preserves ordinary -HarnessRouter behavior, and publishes a reproducible image. +2. Define stable downstream custom harnesses such as `allagents-codex` and + `allagents-omp` with explicit model allowlists. +3. Configure two logical connections to the same external OAuth-to-OpenAI- + compatible gateway: Responses for Codex and OpenAI Chat Completions for OMP. + Each harness has exactly its matching connection and no fallback. +4. Retain brokered sandbox credentials. The long-lived external gateway key + remains server-side; the harness receives only the existing short-lived, + scoped turn credential and loopback route. +5. Start each harness in the validated `working_directory` while keeping HOME, + skills, scratch, conversation, credential projection, and checkpoint control + in their current session-isolated locations. +6. Reject unsupported models and any attempt to place provider URL, key, + transport, route, registry data, or source credentials in the request. +7. Remove ephemeral OMP/Codex provider configuration before checkpoint and file + collection using existing broker lifecycle hooks. +8. Provider failure returns the existing normalized failure and does not change + source, generation, access, attachment, harness, connection, or protocol. + +Proof runs both harnesses against Git and OCI sources, at root and a nested +working directory, in read-only and editable modes where applicable. It verifies +that caller and provider credentials, registry credentials, broker tokens, and +private origins are absent from process output, session/checkpoint files, +produced files, artifacts, logs, and public metadata. An invalid provider +credential or unsupported model causes no route fallback. A stock request still +uses its original workspace path and provider behavior. + +### Phase 8: Exercise lifecycle, restart, cancellation, and cleanup + +**Outcome:** source generations and attachments follow the existing HarnessRouter +session lifecycle under failures and restarts. + +Work and proof: + +1. Cancel during Git advertisement/fetch/checkout, OCI manifest/blob transfer, + decompression/extraction, generation wait, editable copy, harness execution, + checkpoint, and produced-file collection. Reap descendants before terminal + acknowledgement and remove only the cancelled request's incomplete state. +2. Cancel one waiter on a shared generation build while another continues. If no + waiter remains, cancel the bounded builder. At most one complete publication + can survive. +3. Restart after every durable transition: pending descriptor, source resolved, + claim held, staging populated, generation published, session reference + created, read-only attached, editable copy started/completed, attachment ready, + turn active, checkpoint written, cleanup marked, and reference released. +4. Before readiness, reconcile incomplete staging and copies, publication + evidence, references, read-only mounts, editable ownership, session binding, + and cleanup marks. Never attach an uncertain generation or expose an editable + copy to another session. +5. Prove a continuation after restart restores the exact generation/access/cwd; + editable mutations persist, read-only remains immutable, and no source + endpoint is contacted. +6. Prove session expiry and explicit deletion first make the session unavailable, + then release the mount/private copy and generation reference exactly once. + Persistent retention survives ordinary session-idle cleanup until authorized + deletion. +7. A cleanup failure quarantines the path and keeps it unavailable/accounted. + Retrying cleanup is confined, no-follow, and idempotent. +8. Generation cleanup removes only unreferenced complete publications under + bounded operator policy. Active session references and authorized persistent + sessions prevent removal. + +### Phase 9: Add direct large-OCI Promptfoo E2E + +**Outcome:** the built image proves the consumer-visible contract, mandatory +large-workspace behavior, and second-session generation reuse. + +Promptfoo calls the built image directly at the existing UHP Responses endpoint +with a HarnessRouter caller API key. There is no adapter service or alternate +execution protocol. + +Release-blocking scenarios: + +1. **Large OCI fixture:** publish a deterministic workspace snapshot with at + least 2 GiB of expanded source bytes and 100,000 source-visible filesystem + entries. Falling below either floor fails the gate. Include multiple + repository roots, a late-path sentinel, a nested working directory, and + normalized offline history. The release record publishes actual compressed + and expanded bytes, file/inode count, layer count, expansion ratio, and + manifest digests so “large” is measured rather than asserted. +2. **First session:** start Codex read-only from the large snapshot, read the + sentinel, run recent offline Git history, and complete from the nested working + directory. Registry counters and runner metrics must show one bounded download, + extraction, verification, and atomic generation publication. +3. **Second session reuse:** start OMP read-only with the identical snapshot + identity but a different session and harness. It must report the same + generation/manifest identity, share verified generation bytes while retaining + isolated session state, and complete with zero additional registry manifest or + blob requests, zero extraction, and zero publication. This is the required + second-session cache-reuse proof. +4. **Editable reuse:** start an editable session from the same cached generation. + It performs no registry/extraction work, receives inode-independent private + bytes, mutates a file, continues without resending workspace metadata, and + leaves the read-only sessions and generation unchanged. +5. **Repository matrix:** run depth-2 Git scenarios for default ref, branch, tag, + merge tip, several repositories, nested working directory, recent history, + and produced files across destinations. Concurrent cold requests must record + one serialized/singleflight mirror refresh, one pack fetch, and one generation + publication. A second read-only request with the same resolved source identity + must record zero clone, pack transfer, checkout/materialization, and tree copy; + it attaches the same immutable generation bytes while its session, runtime, + conversation, outputs, and cleanup remain isolated. +6. **Access/lifecycle matrix:** cover read-only write denial, editable isolation, + session and authorized persistent retention, continuation, restart, expiry, + explicit deletion, cleanup retry, and missing/corrupt attachment with no + rematerialization. +7. **Failure matrix:** cover malformed descriptor, overlap, bad working directory, + unauthorized persistence, workspace input files with `read_only`, + shallow-fetch refusal, moving ref, submodule/LFS, wrong OCI digest, + workspace-manifest mismatch, extraction limit, path/link/type attack, + cancellation in both source modes, provider failure, and reused-session + descriptor injection. Assert there is no Git/OCI/provider fallback. +8. **Provider matrix:** complete Codex/Responses and OMP/Chat Completions through + the one external provider gateway; reject route/model override; scan retained + and public surfaces for caller, source, registry, broker, and provider secrets. +9. **Stock matrix:** replay baseline requests without workspace metadata and + compare status, stream ordering, checkpoint/files behavior, and provider route + with the characterization fixtures. + +Reports retain only sanitized request/result assertions, image/source digests, +resource measurements, and source/generation counters. They never retain +credentials, private registry origins, provider traffic, internal paths, or +session volume contents. + +### Phase 10: Publish a digest-pinned release + +**Outcome:** a clean operator can deploy the exact tested image and reproduce the +Git/OCI contract. Work: -1. Confirm the upstream decision. If accepted, verify that the - UEP/specification, schema, reference implementation, conformance, changelog, - and docs landed together. If declined or deferred, record that decision and - the exact downstream workspace commits. -2. Advance the fork in a standalone synchronization change to the exact upstream - release tag and commit selected as the baseline. Rebase or replay the - downstream workspace and deployment commits separately. -3. Run the complete selected UHP conformance suite against the built image. - Downstream scenarios supplement it; they do not replace or exclude upstream - cases. -4. Run upstream HarnessRouter integration coverage for gateway, runner, Codex, - OMP, sessions, streaming, cancellation, idempotency, files, artifacts, and - ordinary requests. -5. Run the direct Promptfoo matrix against the exact image candidate. -6. Review the fork diff against its recorded upstream baseline. It must contain - only the declared workspace extension when needed, deployment defaults, - custom harness definitions, provider wiring, Promptfoo scenarios, and image - publication changes. Reusable protocol or runner improvements continue to be - proposed upstream. -7. Tag source and image using `-allagents.`, publish - `ghcr.io/allagentsdev/harnessrouter:`, attach SBOM/provenance, and record - the manifest digest. The first release targets `linux/amd64`; additional - architectures are separate work. -8. Verify a clean Compose deployment using the digest, a fresh volume, both - harnesses, first turn, continuation, cancellation, expiry, restart, and - cleanup. -9. Publish release notes containing upstream issue/UEP links, UHP release, - upstream tag and commit, downstream revision and source commit, pinned - Codex/OMP and Promptfoo releases, image digest, known limitations, and - upgrade/rollback instructions. -10. For future updates, synchronize the new upstream baseline alone, rerun - conformance and built-image E2E, then replay or revise the declared - downstream workspace and deployment patches. Never mix baseline movement - with product behavior. - -Release gate: - -- the upstream ownership decision is recorded and all deliverables for the - selected path are complete; -- all focused tests pass; -- UHP conformance passes without downstream exclusions; -- built-image Codex and OMP first-turn/continuation E2E passes through the - external provider gateway; -- security failures and cancellation leave no descendant or staging directory; -- the external key is absent from harness environments, and it plus scoped - tokens are absent from persistent and public surfaces while both broker routes - succeed; -- expiry and cleanup make checkout unavailable and remove it idempotently; -- ordinary requests remain upstream-compatible; -- the fork diff contains only the declared downstream surface, including the - workspace extension when upstream did not accept it; -- digest, SBOM, provenance, pins, baseline, and downstream revision are - available; and -- Compose smoke succeeds from a fresh checkout and `/data` volume. - -## Error contract - -The selected workspace contract defines stable, stage-oriented detail codes -under HarnessRouter's UHP error shape. When accepted upstream, the UEP, -specification, schema where applicable, reference implementation, conformance -expectations, and docs agree on these observable categories. Otherwise the fork -implementation, extension tests, changelog, and docs agree without claiming UHP -standardization: - -| Condition | HTTP class | Retry guidance | -|---|---:|---| -| Invalid workspace JSON or path | 400 | Caller must change request | -| Workspace supplied for a reused session | 409 | Caller must omit workspace | -| URL, ref, or source feature rejected | 400 | Caller must change source | -| Forbidden DNS or redirect destination | 400 | Caller or operator must change source/network policy | -| Materialization exceeds fixed source limit | 413 | Caller must choose a smaller repository | -| Materializer concurrency unavailable | 503 | Retry with backoff inside caller deadline | -| Git/network timeout before binding | 504 | Retry through ordinary idempotency rules | -| Materialization cancelled | Existing UHP cancelled outcome | Do not retry under cancelled response ID | -| Bound checkout missing, expired, cleanup-started, or on an unsupported contract revision | 410 | Start a new session with a new workspace request | -| Provider unavailable or rejects credentials | Existing provider error | Repair external provider gateway; no route fallback | - -A failure before publication exposes no checkout identity or provenance. A -failure after binding may include already committed public provenance, but never -internal paths, uncontrolled Git stderr, private network topology, or provider -secrets. - -## Configuration ownership - -| Value | Owner | Caller-overridable? | -|---|---|---:| -| Repository URL, optional ref, optional working directory | Initial UHP request | yes, within schema and policy | -| Harness ID and allowed model | New-session request constrained by server definition; stored state on reuse | only among configured values on creation | -| External provider base URL and API key | Operator secret configuration | no | -| Codex/OMP provider transport | Downstream harness definition | no | -| Materializer implementation | HarnessRouter image/runtime configuration in the owning upstream or fork implementation | no | -| DNS, redirect, and Git restrictions | HarnessRouter safe defaults plus operator policy | no weakening by caller | -| Byte, inode, time, and concurrency limits | Operator within image-safe bounds | no | -| Idle checkout TTL | Operator within image-safe bounds | no | -| Session, checkout, control, and execution paths | Runner | no | -| Resolved commit | Materializer observation | no | -| Workspace contract owner and revision | HarnessRouter release and session binding | no | - -## Delivery sequence and ownership - -A practical sequence is: - -1. Protocol owner opens the upstream issue, drives the UEP when required, and - records the governance decision or documented timed deferral before - substantial implementation. -2. Fork maintainer pins the examined baseline, supply chain, image publication, - and baseline-plus-revision release convention in - `allagentsdev/harnessrouter`. -3. If upstream accepts ownership, upstream protocol and runner owners land the - coordinated specification, schema, parsing, session rules, root separation, - materialization, lifecycle, errors, conformance, changelog, and docs. -4. If upstream declines or defers ownership, fork owners land the same bounded - implementation with separate extension tests, changelog, and docs, and record - every downstream workspace commit against the upstream baseline. -5. Downstream harness owner adds Codex/OMP definitions and two logical provider - connections to the one external route. -6. Downstream container owner adds deployment defaults, readiness, `/data`, - Compose, OCI labels, SBOM, and provenance. -7. Downstream E2E owner adds lockfile-pinned direct Promptfoo scenarios and - built-image smoke. -8. Release owner proves the recorded upstream baseline and declared downstream - diff, runs UHP conformance plus extension E2E, then publishes the tagged - digest. - -Upstream runner code depends only on an accepted upstream contract. The fork may -implement after the upstream ownership decision and must keep downstream -extension coverage separate from UHP conformance. Materializer security and -runner lifecycle may proceed in parallel after the selected contract freezes. -Provider wiring, downstream container work, and Promptfoo scenario authoring may -then proceed against that contract. No phase changes the AllAgents CLI -repository. +1. Review the fork diff against its exact upstream tag/commit. The workspace + changes must be limited to request/session binding, the existing hydrate and + attachment seam, immutable generation storage, Git/OCI materialization, + produced-file adaptation, focused lifecycle/configuration/docs, custom + harness/provider wiring, E2E, and release automation. +2. Run the complete pinned upstream UHP conformance suite without exclusions and + the focused fork coverage against the image candidate. +3. Run the Phase 9 Promptfoo matrix against that exact candidate digest. +4. Build `linux/amd64` from pinned inputs, attach standard SBOM and provenance, + and publish + `ghcr.io/allagentsdev/harnessrouter:-allagents.`. +5. Read back and deploy by manifest digest, for example + `ghcr.io/allagentsdev/harnessrouter:v0.25.4-allagents.1@sha256:`. +6. Verify a fresh-volume deployment and a same-volume restart with both + harnesses, Git and large OCI, cache reuse, continuation, cancellation, + produced files, expiry, persistent deletion, generation cleanup, and stock + requests. +7. Record upstream tag/commit, downstream source commit, UHP release, Codex/OMP/ + Promptfoo versions, base and package pins, Git depth/policy revision, OCI + validation-policy revision, image digest, source fixture digests, SBOM, + provenance, and E2E report identities. +8. Block release on any missing OCI implementation/evidence, large-workspace + failure, second-session rebuild, writable read-only alias, editable inode + sharing, continuation rematerialization, source fallback, credential leak, + stock regression, or unpinned input. + +Only after the downstream digest and evidence are available may maintainers +prepare an upstream issue or proposal for the generic contract and runner seams. +That work cites measured Git/OCI behavior, cache reuse, security failures, and +stock compatibility. Upstream discussion, acceptance, UEP timing, and merge are +outside the release critical path; a later upstream implementation replaces the +fork delta only after equivalent behavior passes the same gates. + +## Failure contract + +Workspace failures use bounded stable detail codes under the existing UHP error +shape. Exact HTTP mapping follows existing HarnessRouter conventions, but these +observable distinctions must remain: + +| Condition | Required behavior | +|---|---| +| Invalid shape, field, digest, destination, or working directory | Reject before session source work | +| Unauthorized `persistent` retention | Reject before source lookup/network work | +| Workspace metadata on a reused session | Reject without changing the existing binding or TTL | +| Workspace input files with `read_only` access | Reject before source resolution; never overlay or copy up immutable source | +| Repository ref missing/ambiguous/moved | Fail repository resolution; no alternate ref/source | +| Server cannot satisfy depth-2 fetch | Fail repository acquisition; no deepen/full clone/OCI fallback | +| Submodule or LFS content | Fail repository validation; no helper execution | +| OCI catalog name/digest/media mismatch | Fail snapshot resolution; do not reveal catalog origin | +| Layer digest, whiteout, path, link, type, limit, or workspace-manifest failure | Fail extraction/validation; no publication or Git fallback | +| Generation capacity unavailable | Return bounded retryable capacity failure before unbounded work | +| Source timeout or cancellation | Stop descendants, detach waiter, clean incomplete private state | +| Attachment evidence missing/corrupt on continuation | Fail closed; no source access or replacement generation | +| Read-only write attempt | Filesystem denial; generation and sibling sessions unchanged | +| Provider failure | Existing provider error; source and route binding unchanged | +| Cleanup failure | Session/path remains unavailable and accounted for retry | + +Failures before attachment-ready expose no generation or source provenance. +Failures after attachment-ready may return already committed public provenance, +but never internal paths, registry origins, credentials, raw tool stderr, or +private network details. + +## Verification matrix + +| Gate | Observable evidence | +|---|---| +| Stock compatibility | Requests without `metadata.workspace` match pinned status, stream, hydrate/checkpoint, files, cancellation, and provider behavior | +| Request/session binding | Both source variants bind only on a new session; continuation omits metadata and reuses the exact binding | +| Multiple repositories | Pairwise non-overlapping destinations compose correctly; overlap and source escape fail before acquisition | +| Git depth policy | Advertisement/default/branch/tag resolve exactly; fetch uses depth 2; tip matches; `.git/shallow`, recent history, and merge parents work offline; unsupported shallow fetch has no fallback | +| OCI integrity | Direct manifest and workspace-manifest digests, fixed `100` per-layer and aggregate expansion-ratio ceilings, bounded layers, whiteouts, paths, links, types, tree manifest, and optional normalized Git history all verify | +| Mandatory large OCI | A fixture with at least 2 GiB expanded source and 100,000 source-visible entries completes for both harnesses from the digest-pinned image; no Git acquisition occurs | +| Generation publication | Concurrent identical Git or OCI identities singleflight to one acquisition and one publication; failed/partial mirror snapshots or generations never attach | +| Git mirror/generation reuse | One operator-only bare shallow mirror exists per canonical repository identity; a second resolved-identity hit has zero clone/pack transfer/checkout/tree copy and directly attaches the same immutable generation | +| OCI generation reuse | A second exact-digest request has zero registry request/extraction/tree copy/publication and directly attaches the same immutable generation | +| Read-only sharing | Every matching read-only session leases and binds the same verified generation bytes with zero copy; all write paths/aliases fail and session/runtime state stays isolated | +| Read-only state separation | Mutable harness/runtime assets live in the session control root; checkpoint excludes generation bytes, writes no source Git state, and continuation reattaches the exact generation before restoring session state | +| Editable isolation | Private copies share no mutable inode; one session's changes and checkpoint never alter generation or siblings | +| Working directory | Root and nested valid directories become process cwd; missing, file, traversal, and symlink escape fail before harness execution | +| Produced files | Existing API reports root and nested-repository changes once, excludes baselines/admin/control/secrets, survives continuation/restart, and respects bounds | +| Continuation | Exact access, retention, generation, provenance, cwd, harness, conversation, and editable mutations restore without source traffic | +| Cancellation | Git, OCI, build wait, copy, harness, checkpoint, and collection cancellation reap descendants and leave recoverable state | +| Restart | Every generation/attachment/cleanup transition reconciles before readiness; exact attachments resume or fail closed | +| Cleanup | Expiry/deletion is unavailable-first, confined, idempotent, reference-safe, and quarantine-preserving on error | +| Provider boundary | Codex Responses and OMP Chat Completions use one operator route with brokered credentials, no caller override, fallback, or retained secrets | +| Release | GHCR image, digest, SBOM, provenance, source/runtime pins, UHP conformance, and direct Promptfoo reports identify the same candidate | ## Definition of done -This work is done when: - -1. The upstream issue and required UEP have a recorded decision. On acceptance, - the UHP specification, schema, HarnessRouter reference implementation, - conformance suite, changelog, and docs define the same generic - `metadata.workspace` contract. On rejection or deferral, the fork - implementation, extension tests, changelog, and docs define it consistently - without claiming UHP standardization. -2. `allagentsdev/harnessrouter` remains the GitHub fork and its release diff from - the recorded upstream baseline contains only the declared workspace extension - when needed, AllAgents deployment defaults, custom harness definitions, - provider wiring, Promptfoo scenarios, and image publication. -3. An operator can deploy one digest-pinned - `ghcr.io/allagentsdev/harnessrouter:-allagents.` - container with Docker Compose and one `/data` volume. -4. Promptfoo can directly start either custom harness with a public HTTPS Git - repository, optional advertised ref, and optional safe `working_directory`. -5. The public response reports the securely resolved exact commit without - internal paths, while the harness runs inside a private editable checkout. -6. Continuation through either supported session-reference path verifies the - recorded workspace-contract owner and revision, then reuses the same stored - harness, mutations, checkout, control state, and execution directory without - new Git work. -7. Session, checkout, control, and execution roots remain separated; staging is - invisible; binding publication and startup recovery are crash-safe. -8. Git and harness cancellation leak no descendants or staging, and expiry makes - the binding unavailable before deterministic idempotent cleanup. -9. Codex uses Responses and OMP uses Chat Completions through two logical - connections to one external OAuth gateway, with no fallback. -10. The long-lived external key never enters the harness, and neither it nor - scoped broker credentials survive in persisted or public surfaces. -11. Unsafe or invalid sources, limits, reused-session injection, missing - checkout, and provider failures return the agreed bounded errors without - violating ordering or making unintended provider calls. -12. Direct built-image Promptfoo coverage passes for both harnesses, first turn, - continuation, cancellation, provider failure, security negatives, restart, - expiry, and cleanup. -13. Ordinary requests remain upstream-compatible and the complete selected UHP - conformance suite passes without downstream exclusions. -14. Source tag, OCI tag, digest, SBOM, provenance, exact upstream tag/commit, - downstream revision/source commit, runtime pins, and release notes are - published and mutually consistent. -15. No implementation code, CLI command, profile migration, or `workspace.yaml` - change is required in `allagentsdev/allagents`. - -Upstream rejection or deferral does not rename the product or block the -distribution. It changes ownership: the workspace patch remains visible in -`allagentsdev/harnessrouter`, its release notes identify it as a downstream -HarnessRouter extension, and any later conflicting UHP standard triggers a -clean migration. +1. `allagentsdev/harnessrouter` remains the existing fork and the release image is + `ghcr.io/allagentsdev/harnessrouter` with the established + `-allagents.` tag and a deployed manifest digest. +2. The stock behavior inventory is protected by characterization coverage, and + requests without workspace metadata remain unchanged. +3. The exact first-turn-only snake_case contract supports required `access`, + optional `retention`, required repositories or workspace-snapshot `source`, + and optional workspace-relative `working_directory`, with no nested version. +4. Session state binds exact source provenance, generation/manifest identity, + access, retention, working directory, harness, and attachment evidence. + Continuation omits the descriptor and reuses that exact attachment. +5. Repository mode supports multiple non-overlapping destinations, resolves + advertised default/branch/tag refs, fetches at depth 2, verifies the fetched + tip, preserves shallow recent history and merge semantics, and never silently + deepens or falls back. Submodules and LFS remain off. +6. OCI snapshot mode is implemented and release-tested with direct manifests, + workspace-manifest verification, bounded layer extraction, whiteouts, + path/link/type checks, optional normalized offline Git history, and no Git + fallback. Callers never provide or observe registry origins or credentials. +7. Repository acquisition uses one operator-only bare shallow mirror per canonical + repository identity, serialized singleflight refresh, immutable exact-commit + snapshots, and a separate immutable multi-repository generation. OCI uses the + exact digest-keyed generation cache. Identical normalized source identities + publish once; access/retention/cwd/harness/session do not fragment identity. +8. Every matching read-only session leases and directly binds the same verified + generation bytes with zero clone, source-byte transfer, materialization, or + tree copy. Mirror internals are never session-visible. Editable sessions + receive inode-independent private copies. +9. Existing hydrate, user/sandbox isolation, cancellation, TTL, restart, + deletion, files, artifacts, and cleanup own the complete session lifecycle. + Workspace-backed sessions relocate mutable harness/runtime assets to a + per-session control root outside source. Read-only requests reject workspace + input files and checkpoint no generation bytes; editable requests apply input + files only to the private copy and checkpoint only that copy plus + session-local state. No parallel workspace system remains. +10. Existing produced-file cursor semantics are adapted with runner-owned + external baselines for outer paths plus declared nested/multiple repository + and tree-only/history-bearing cursors, without writing bookkeeping Git state + into an immutable generation or adding a second Files API. +11. Direct Promptfoo E2E against the tested image proves Git and large OCI, + multiple repositories, nested working directory, continuation, read-only + enforcement, editable isolation, produced files, cancellation, restart, + cleanup, provider boundaries, and unchanged stock requests. +12. Required second-session proof covers both caches: Git performs zero clone, + pack transfer, checkout/materialization, or tree copy after exact identity + resolution; OCI performs zero registry request, extraction, publication, or + tree copy. Both read-only sessions bind the same immutable generation bytes + while session/runtime state remains isolated. Editable reuse has independent + inodes and mutations. +13. Codex and OMP use their fixed protocol adapters through the existing external + OAuth-to-OpenAI-compatible gateway with brokered short-lived credentials and + no fallback or caller override. +14. UHP remains the only northbound protocol. There is no AllAgents CLI work, + local profile synchronization, `workspace.yaml` change, provider-login + implementation, runtime-image contract, or benchmark-environment coupling. +15. The release is blocked unless UHP conformance, focused integration coverage, + large-OCI Promptfoo E2E, generation-reuse evidence, credential scans, SBOM, + provenance, and digest-pinned fresh/restart deployment all pass for the same + image. +16. Any later upstream proposal is based on this downstream evidence and remains + outside the release path; no upstream issue, UEP, acceptance, or wait period + blocks implementation or publication. From ff2facd8bf0454251a7787301f12611283029195 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 06:34:19 +1000 Subject: [PATCH 32/44] docs(architecture): mount source roots read-only --- .../0002-adopt-uhp-through-harnessrouter.md | 276 +++++--- ...0837-feat-coding-execution-gateway-plan.md | 669 ++++++++++-------- 2 files changed, 574 insertions(+), 371 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 25d799d4..6c0b0bbd 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -37,35 +37,35 @@ The fork must preserve the stock ownership boundary: | Existing HarnessRouter responsibility | Extension responsibility | |---|---| -| Allocate the session workspace and user/sandbox identity | Validate the first-turn workspace descriptor | +| Allocate the private, writable session workspace and user/sandbox identity | Validate the first-turn descriptor and reserve declared non-root source destinations | | Choose fresh materialization or checkpoint hydration | Resolve Git commits or exact OCI artifact identity | | Transport and restore checkpoints | Build or reuse one verified immutable generation | -| Start the harness in the session workspace | Attach that generation read-only or as a private editable copy | +| Start the harness in the private session workspace | Bind each generation repository root read-only, or populate an inode-independent editable copy | | Track sessions, TTL, cancellation, files, and cleanup | Persist source provenance and attachment identity with the session | -| Resume an existing session workspace | Verify and reuse the exact prior attachment without resolving source again | +| Resume an existing writable session workspace | Restore outer state and reattach the exact prior protected generation without resolving source again | For a new workspace-backed session, source initialization runs after authentication, request validation, idempotency, session resolution, and -allocation of HarnessRouter's fresh session workspace, but before provider work -or harness execution. It attaches source content at the runner-designated -workspace root; it does not allocate another workspace root or move lifecycle -ownership out of HarnessRouter. - -Stock HarnessRouter writes `.harness` state, generated instruction files, -plugins, skills, MCP configuration, HOME, and conversation state under the -workspace. That is incompatible with a shared read-only source mount. For -workspace-backed sessions, the fork creates a per-session writable control root -inside the existing session allocation but outside source content and redirects -all mutable harness/runtime state there. Generated instructions use a -harness-supported external instruction channel or non-shadowing session mount; -they never overwrite, overlay, or copy up a source path. If Codex or OMP cannot -honor that separation, the read-only release gate fails. - -For continuation or recovery, workspace-aware checkpoint hydration first -restores and validates the minimal control metadata needed for attachment, -acquires and mounts the exact protected generation, then restores the remaining -session-local mutable state around it. The source initializer is not invoked. -Git refs are not resolved again, OCI is not fetched again, and a newer +allocation of HarnessRouter's fresh private session workspace, but before +provider work or harness execution. The workspace root remains private and +writable for both access modes. The initializer places source only at the +declared non-root repository destinations; it does not allocate another +workspace root or move lifecycle ownership out of HarnessRouter. + +For `read_only`, the fork creates empty destination directories in the session +workspace and attaches the corresponding immutable-generation repository roots +with per-session, namespace-confined, read-only bind mounts. `.harness`, HOME, +generated instructions, plugins, skills, MCP configuration, inputs, outputs, +scratch, conversation state, and other session data continue to use ordinary +writable paths in the private workspace, provided they are outside mounted +source destinations. There is no requirement to relocate all mutable state to a +separate control root. + +For continuation or recovery, workspace-aware checkpoint hydration unmounts +any stale attachment, restores the writable outer workspace without traversing +or restoring source destinations, validates empty non-link mountpoints, and +then reattaches the exact protected generation. The source initializer is not +invoked. Git refs are not resolved again, OCI is not fetched again, and a newer generation is not substituted. Workspace initialization reuses HarnessRouter's cancellation, process @@ -124,7 +124,7 @@ OCI snapshot mode: "image_manifest_digest": "sha256:…", "workspace_manifest_digest": "sha256:…" }, - "working_directory": "packages/compiler" + "working_directory": "repo/packages/compiler" } } } @@ -145,10 +145,18 @@ A repository source has exactly `kind: "repositories"` and `repositories`. The a |---|---:|---| | `url` | yes | Canonical public HTTPS Git URL. No userinfo, query, fragment, local path, or alternate transport. | | `ref` | no | Advertised full ref or unambiguous branch/tag shorthand. Omission uses the advertised remote default. The resolved commit, not the ref spelling, is authoritative. | -| `destination` | yes | Non-root, workspace-relative POSIX directory. Destinations must be unique, pairwise non-overlapping, and disjoint from runner-owned control paths. | +| `destination` | yes | Non-root, workspace-relative POSIX directory. Destinations must be unique, pairwise non-overlapping, and outside reserved runner paths. | Depth is not caller-selectable. The repository acquisition policy defaults every entry to Git depth `2`; that effective depth is returned as provenance and participates in generation identity. +Every source composition occupies 1 to 128 declared repository roots. Whether +the roots come from repository request entries or a verified OCI workspace +manifest, their destinations are non-root, unique, and pairwise +non-overlapping. A monorepo therefore uses a destination such as `repo`, with a +working directory such as `repo/packages/compiler`; source at destination `.` +is invalid. Ancestor directories may be created as empty mount scaffolding, but +must contain no source files. + A snapshot source has exactly: | Field | Required | Contract | @@ -158,17 +166,23 @@ A snapshot source has exactly: | `image_manifest_digest` | yes | Direct `sha256:` digest of the accepted OCI image manifest. Mutable tags and indexes are not accepted as source identity. | | `workspace_manifest_digest` | yes | `sha256:` digest of the canonical workspace manifest expected from that artifact. | +The verified canonical workspace manifest declares 1 to 128 repository roots +and their destinations under the same non-root and non-overlap rules. The +request cannot override those destinations. + The OCI catalog is HarnessRouter deployment configuration owned by the operator. It maps `snapshot_name` to a fixed registry repository, allowed media types, trust policy, and server-side registry credential reference. A caller never supplies a registry origin, repository, tag, header, or credential. -`working_directory` is interpreted only after the verified tree exists. It must resolve, without symlink escape, to a real directory inside the workspace. Absolute paths, empty components, `.` or `..` components, platform-specific separators, and reserved control paths are invalid. +`working_directory` is interpreted only after the verified tree is attached or copied and input files are placed. It must resolve, without symlink escape, to a real directory inside the workspace; it may be within a declared source root or elsewhere in the writable outer workspace. Absolute paths, empty components, `.` or `..` components, platform-specific separators, and reserved runner paths are invalid. Unknown keys are rejected at every level. The descriptor cannot contain credentials, headers, host paths, environment variables, commands, runtime images, Docker settings, materializer selection, resource limits, provider routes, or caller-selected TTLs. Request size, string length, array length, nesting, and validation work are bounded before source access. -UHP input files are workspace mutations. They are accepted only for `editable` -workspaces, after the private copy exists and before the initial produced-file -baseline is sealed. A `read_only` first turn containing workspace input files -fails before source resolution. Harness assets, HOME, credentials, scratch, and -checkpoint control remain in runner-owned session paths outside immutable source -content in both modes. +UHP input files remain normal session-workspace mutations in both access modes. +They are placed before the initial produced-file baseline is sealed. In +`read_only` mode they may target writable outer-workspace paths, but any input +whose path is a declared source destination or lies beneath one fails rather +than overlaying, copying up, or modifying the mounted generation. Generated +instructions and other HarnessRouter assets follow the same boundary: they may +be written outside source destinations. In `editable` mode inputs may overlay +the private source copies before baseline. A first turn may omit `metadata.workspace`; stock behavior then remains unchanged. A session created without workspace metadata cannot add it later. Any reused session selected through `previous_response_id` or HarnessRouter's existing session-recovery metadata must omit `metadata.workspace`, even if the repeated object is byte-for-byte identical. @@ -220,7 +234,7 @@ Snapshot response: "workspace": { "access": "read_only", "retention": "session", - "working_directory": "packages/compiler", + "working_directory": "repo/packages/compiler", "effective_descriptor_digest": "sha256:…", "generation_id": "sha256:…", "workspace_manifest_digest": "sha256:…", @@ -231,7 +245,7 @@ Snapshot response: "workspace_manifest_digest": "sha256:…", "repositories": [ { - "destination": ".", + "destination": "repo", "resolved_commit": "0123456789abcdef0123456789abcdef01234567", "object_set_digest": "sha256:…" } @@ -258,24 +272,74 @@ The response fields are exact: Repository provenance contains `kind: "repositories"` and the request-order `repositories` array. Each entry contains normalized `url`, `destination`, exact `resolved_commit`, effective `depth`, and `requested_ref` only when the request supplied `ref`. Branch or tag movement does not change stored provenance for an existing session. -Snapshot provenance contains `kind: "workspace_snapshot"`, `snapshot_name`, exact `image_manifest_digest`, exact `workspace_manifest_digest`, and a manifest-order `repositories` array. Each declared root contains `destination`; a history-bearing root additionally contains `resolved_commit` and `object_set_digest`. Tree-only roots contain neither. Snapshot provenance never exposes a registry origin, repository, credential reference, redirect, or physical path. +Snapshot provenance contains `kind: "workspace_snapshot"`, `snapshot_name`, exact `image_manifest_digest`, exact `workspace_manifest_digest`, and a manifest-order `repositories` array. Each declared root contains its non-root `destination`; a history-bearing root additionally contains `resolved_commit` and `object_set_digest`. Tree-only roots contain neither. Snapshot provenance never exposes a registry origin, repository, credential reference, redirect, backing path, or physical mount path. Failures before attachment reaches `ready` omit workspace metadata. Failures after `ready` return the complete committed object. Internal generation keys, policy versions, authorization scope, mount paths, attachment IDs, pins, leases, reservations, and other sessions' state remain private. ## Immutable generations and attachment behavior -Both source modes produce the same versioned canonical workspace manifest. It enumerates source-visible directories, regular files, and symbolic links in logical path order with normalized mode, size, content digest, or link target. HarnessRouter independently walks staging without following links, recomputes the canonical bytes, and requires the supplied and computed manifest digests to match before publication. - -The runner computes a private generation key from every input that can change source bytes, filesystem semantics, or sharing authorization. For Git this includes the normalized repository URLs, resolved commits, destinations, shallow depth (`2` in v1), acquisition-policy revision, and materializer contract revision. For OCI it includes the catalog identity, exact image-manifest and workspace-manifest digests, trust-policy revision, and materializer contract revision. Access, retention, working directory, harness, session, and physical paths are excluded because they do not change the generation's immutable bytes. A later depth or acquisition-policy change therefore cannot reuse an incompatible Git generation. - -Identical normalized source identity publishes exactly one live verified generation. The cache has two levels: an operator-only bare Git mirror/object cache per canonical repository URL for bounded acquisition, followed by an immutable multi-repository generation keyed by canonical URLs, exact resolved commits, destinations, depth, acquisition-policy revision, and materializer contract revision. Refreshes of one bare cache are serialized, and in-flight acquisition and generation misses singleflight by generation key. Publication is crash-safe: partial or failed staging is never attachable, and garbage collection cannot remove a generation while a build waiter, provisional pin, durable session reference, or attachment lease protects it. +Both source modes produce the same versioned canonical workspace manifest. It +declares 1 to 128 non-root, pairwise non-overlapping repository roots and +enumerates the source-visible directories, regular files, and symbolic links +beneath them in logical path order with normalized mode, size, content digest, +or link target. HarnessRouter independently walks staging without following +links, recomputes the canonical bytes, and requires the supplied and computed +manifest digests to match before publication. + +The runner computes a private generation key from every input that can change +source bytes, filesystem semantics, or sharing authorization. For Git this +includes normalized repository URLs, resolved commits, destinations, shallow +depth (`2` in v1), acquisition-policy revision, and materializer contract +revision. For OCI it includes catalog identity, exact image-manifest and +workspace-manifest digests, trust-policy revision, and materializer contract +revision. Access, retention, working directory, harness, session, and physical +paths are excluded because they do not change the generation's immutable bytes. +A later depth or acquisition-policy change therefore cannot reuse an +incompatible Git generation. + +Identical normalized source identity publishes exactly one live verified +generation. The cache has two levels: one operator-only bare Git mirror/object +cache per canonical repository URL for bounded acquisition, followed by an +immutable multi-repository generation keyed by canonical URLs, exact resolved +commits, destinations, depth, acquisition-policy revision, and materializer +contract revision. Refreshes of one bare cache are serialized, and in-flight +acquisition and generation misses singleflight by generation key. Publication +is crash-safe: partial or failed staging is never attachable, and garbage +collection cannot remove a generation while a build waiter, provisional pin, +durable session reference, or attachment lease protects it. Attachment depends on `access`: -- Every `read_only` Git request for the same normalized source identity leases and attaches the same cached immutable shallow-generation bytes; it never clones or copies that repository again. Every `read_only` OCI request for the same snapshot and generation identity likewise leases and attaches the exact digest-keyed generation. These sessions retain separate user and sandbox identity, checkpoint state, home, scratch space, control state, logs, outputs, and response state. Filesystem enforcement makes each attachment read-only; there is no copy-up path, and neither the generation backing store nor the writable acquisition cache is exposed to a session. -- `editable` sessions reuse the same acquisition cache and pinned verified generation as input, then receive a private, quota-bounded, inode-independent writable copy. No mutable inode may be shared with the generation, Git object cache, or another session. Later turns and checkpoints operate on that private copy. - -An attachment binds the normalized descriptor, exact generation key and epoch, access, retention, working directory, selected harness, provenance, and manifest digest to the HarnessRouter session. Continuation reuses that exact attachment. It never re-resolves source, changes access or retention, selects another generation with the same public ID, or rebuilds missing state. +- Every `read_only` Git request for the same normalized source identity, and + every equivalent OCI request, leases the same immutable generation. For each + declared root the runner creates an empty destination in the private writable + session workspace and bind-mounts the matching generation directory there + read-only. Matching sessions therefore see the same generation inodes and + cached source bytes while retaining separate outer-workspace state, user and + sandbox identity, HOME, scratch, logs, outputs, checkpoints, and response + state. +- Each bind mount is kernel-enforced read-only, namespace-confined, `nodev`, + and `nosuid`, while preserving repository execute bits required by tools. + Neither a writable alias nor a copy-up path is visible to the session, and + the generation backing store and writable acquisition cache remain + inaccessible. Any mount or remount failure fails closed. +- `editable` sessions reuse the same acquisition cache and pinned verified + generation as input, then receive an inode-independent, quota-bounded private + writable copy at each declared destination. No mutable inode may be shared + with the generation, Git object cache, or another session. + +Bind mounts are required rather than symlinks. A symlink neither enforces +read-only access nor confines traversal to the workspace; it exposes a backing +path, can escape workspace containment, and gives cwd and file tools surprising +path behavior. The mounted roots instead appear as ordinary directories at the +declared workspace-relative destinations. + +An attachment binds the normalized descriptor, exact generation key and epoch, +access, retention, working directory, selected harness, root-to-destination +attachment manifest, provenance, and workspace-manifest digest to the +HarnessRouter session. Continuation reuses that exact attachment. It never +re-resolves source, changes access or retention, selects another generation +with the same public ID, or rebuilds missing state. ## Git acquisition and integrity @@ -289,7 +353,7 @@ Git initialization must satisfy all of these requirements: - Keep one server-owned bare shallow Git mirror/object cache per canonical repository URL behind the generation builder so repeated acquisition can reuse fetched objects. The cache is mutable operator-only runner infrastructure, never a session attachment. Refreshes are serialized, and it is inaccessible to harness users, credentials, hooks, and workspace writes. Publication selects only the resolved ref's bounded object graph into the immutable generation; unrelated cached refs and objects are never exposed. The generation contains its own normalized shallow repository state, so later cache updates cannot change it. - Preserve the generation's normalized `.git/shallow` metadata and the acquired recent history so offline commands such as `git log` and recent diffs work within the fetched boundary. Remove credential-bearing remotes, hooks, worktree links, alternates, replace and graft state, locks, reflogs, and transient fetch state. Verify detached `HEAD`, shallow boundary, index-to-tree equality, included object integrity, and source-visible content against the recorded commit and acquisition policy. - Apply finite time, transferred-byte, inode, file-count, process, descendant, and concurrency limits across all repositories. Shallow depth reduces history transfer; it does not solve large working-tree transfer or materialization, which is why OCI snapshots remain mandatory. -- Stage every repository beneath its declared destination and reject overlaps, undeclared files, cross-root links, traversal, or reserved-path collisions. Publish the complete multi-repository generation atomically or publish nothing. +- Stage every repository beneath its declared non-root destination and reject overlaps, undeclared files, source files in destination ancestors, cross-root links, traversal, or reserved-path collisions. Publish the complete multi-repository generation atomically or publish nothing. - Record normalized URL, optional requested ref, exact resolved commit, destination, and effective depth for every repository. A failure in any repository fails the whole source; partial repository sets are never attached. ## OCI acquisition and integrity @@ -298,10 +362,11 @@ OCI snapshot support is mandatory in v1 and release-blocking. Snapshot acquisiti - Resolve `snapshot_name` only through the operator-owned catalog. Fetch only the direct image manifest named by `image_manifest_digest`; do not follow mutable tags, accept an index in its place, change registry authority on redirect, or expose catalog registry details to the caller. - Verify the image manifest digest, media type, descriptor sizes, every selected layer digest and size, the catalog-defined workspace-manifest media type, the workspace-manifest blob digest, and the recomputed source-visible manifest digest. +- Fetch and verify the canonical workspace manifest before requesting any layer. Validate its effective repository-root map and every input, generated-asset, reserved-path, and restored-outer-state collision before any layer request or outer workspace content write. - Enforce the v1 envelope before and during extraction: at most 64 distributable tar/gzip/zstd layers; a 4 MiB image manifest; a 128 MiB workspace manifest with at most 128 repository roots; 8 GiB total compressed layer bytes; 32 GiB expanded source bytes; 500,000 entries; 4 GiB per regular file; paths of at most 4096 UTF-8 bytes and 128 components; and 1 MiB per PAX or extended header. For each layer and for the aggregate artifact, expanded bytes divided by `max(compressed_bytes, 1)` must not exceed `100`. Cumulative-size and expansion-ratio checks apply while streaming, not only after extraction. Operators may configure lower limits, never higher ones without a contract revision. - Apply layers in order with OCI whiteout and opaque-directory semantics. Whiteouts are metadata operations, not source-visible files. Reject malformed, duplicate, conflicting, or out-of-root whiteouts. -- Before writing each entry, validate its normalized relative path, type, declared size, mode, and link target. Reject absolute paths, traversal, NULs, escaping hard links or symbolic links, devices, sockets, FIFOs, sparse-file tricks, unsupported types, and entries that collide with runner-owned paths. Extraction uses rooted, no-follow operations and cannot write through a previously extracted link. -- Require the canonical workspace manifest to declare every source-visible entry and repository root. Undeclared output, missing entries, type changes, digest mismatches, and paths outside declared roots fail closed. +- Before writing each entry, validate its normalized relative path, type, declared size, mode, and link target. Every hard link or symbolic link must remain within its owning declared repository root; links into another source root or the writable outer workspace fail closed. Reject absolute paths, traversal, NULs, escaping links, devices, sockets, FIFOs, sparse-file tricks, unsupported types, and entries that collide with runner-owned paths. Extraction uses rooted, no-follow operations and cannot write through a previously extracted link. +- Require the canonical workspace manifest to declare every source-visible entry and 1 to 128 non-root, pairwise non-overlapping repository roots. Undeclared output, missing entries, type changes, digest mismatches, source at destination `.`, source files in destination ancestors, and paths outside declared roots fail closed. A workspace snapshot may contain normalized offline Git history for any declared repository root. A history-bearing root records `resolved_commit` and `object_set_digest`; a tree-only root records neither. History-bearing roots must have detached `HEAD` at the recorded commit, an index equal to that tree, the complete required object closure matching `object_set_digest`, and no dirty, staged, untracked, unreachable, or extra source-visible state. They must contain no remote, credential helper, config include, hook, worktree link, alternate, shallow, replace, graft, reflog, transient fetch state, or credential-bearing configuration. OCI restore never contacts Git, and failure of snapshot or embedded Git verification never falls back to cloning. @@ -310,48 +375,62 @@ The direct image digest is part of OCI identity even when two artifacts have the ## Produced files, checkpoints, and continuation HarnessRouter's existing Files API and produced-file collection remain the only -public file surface. The fork adapts the existing root-Git-oriented bookkeeping -rather than introducing a second Files API or parallel change tracker. It stores -workspace-backed cursors and indexes under the per-session control root; it does -not initialize or mutate a bookkeeping `.git` directory inside an immutable -generation. - -Bookkeeping must understand the declared repository roots and source mode: - -- Git repository roots compare editable state with their recorded resolved - commits. -- History-bearing snapshot roots use their verified commit and object-set - records. -- Tree-only snapshot roots compare with the canonical workspace manifest. -- Workspace paths outside declared repository roots use an external - runner-owned baseline in the control root. -- Git control data, generation metadata, credentials, checkpoint control state, - and runner-owned paths are never reported as produced files. - -The adaptation must represent additions, modifications, deletions, renames, and -mode changes across multiple nested repository roots without assuming or -writing a single root `.git` directory. `read_only` attachments cannot produce -source mutations. Editable produced-file state and nested repository state are -covered by the session's private quota and lifecycle. - -Checkpoint behavior is access-specific. A `read_only` checkpoint excludes the -generation mount and all source bytes; it persists session-local mutable state -separately from the exact generation-key, epoch, manifest, durable reference, -and attachment evidence. Hydration validates that minimal binding/control -metadata, reattaches the same protected generation, then restores the remaining -harness state before the turn. An `editable` checkpoint contains the private -workspace copy and its nested repository state, never the bare acquisition -cache or immutable generation backing store. Checkpoint creation must not run a -root Git commit against a read-only attachment. +public file surface. The fork adapts the stock root-workspace Git/bookkeeping +path rather than introducing a second Files API or parallel change tracker. The +outer session workspace stays writable, but root bookkeeping explicitly +excludes every declared source destination and must not traverse its mount. +Bookkeeping state remains in normal runner-owned session paths; it never +initializes or mutates a `.git` directory inside an immutable generation. + +Bookkeeping understands the declared repository roots and source mode: + +- Writable outer-workspace paths retain stock-like root bookkeeping, excluding + all source destinations. +- Nested repository collectors compare editable Git roots with their recorded + resolved commits, history-bearing editable snapshot roots with their verified + commit and object-set records, and tree-only editable roots with the canonical + workspace manifest. +- Read-only roots cannot change and are never traversed by root Git, + produced-file scans, cleanup walks, or archive creation. +- Git control data, generation metadata, credentials, attachment evidence, and + runner-owned checkpoint state are never reported as produced files. + +The adaptation represents additions, modifications, deletions, renames, and +mode changes in the writable outer workspace and across multiple editable +nested repository roots without assuming one root `.git` directory. A +`read_only` attachment cannot produce source mutations, but files created +outside mounted roots are collected normally. Editable produced-file state and +nested repository state remain covered by the session's private quota and +lifecycle. + +Checkpoint behavior is access-specific. A `read_only` checkpoint archives the +writable outer workspace while explicitly excluding every mount destination +and all source bytes. It stores the exact generation key and epoch, root +attachment manifest, workspace manifest, durable reference, and mount evidence +separately. Archive and file operations never follow or cross a source mount. +An `editable` checkpoint includes the inode-independent private source copies +and nested repository state, but never the bare acquisition cache or immutable +generation backing store. A continuation supplies the existing predecessor/session reference and omits -`metadata.workspace`. HarnessRouter performs the access-specific hydration, -verifies the stored attachment evidence, and starts the stored harness in the -stored working directory. Edits from prior editable turns remain visible. A -changed ref, source digest, working directory, access, retention, or harness +`metadata.workspace`. HarnessRouter first unmounts any existing source +attachments, hydrates the writable outer state, validates that every declared +destination is an empty real directory rather than a link, and only then +reattaches the exact protected generation read-only. It verifies the stored +attachment evidence and starts the stored harness in the stored working +directory. Editable hydration restores its private copies instead. Edits from +prior editable turns and files written outside read-only roots remain visible. +A changed ref, source digest, working directory, access, retention, or harness requires a new session. -If attachment, generation, private-copy, checkpoint, or provenance evidence is expired, missing, corrupt, or inconsistent, continuation fails closed. It does not clone, repull, restore from OCI again, substitute another generation, or silently start a fresh session. +Attachments are unmounted before hydration, deletion, workspace cleanup, or +retrying cleanup. Tar, Files API traversal, recursive cleanup, and root Git +operations must stay on the writable outer filesystem and never cross a mount. +If unmount, attachment, generation, private-copy, checkpoint, or provenance +evidence is expired, missing, busy, corrupt, or inconsistent, continuation or +cleanup fails closed and the allocation remains accounted for. Continuation +does not clone, repull, restore from OCI again, substitute another generation, +or silently start a fresh session. ## Provider authentication and harness configuration @@ -371,7 +450,7 @@ The implementation fails closed without changing source mode, source identity, a |---|---| | Malformed, oversized, too-deep, or unknown workspace field | Reject before source access and without mutating an existing session. | | Workspace metadata on a continuation or reused session | Reject without changing the attachment, checkpoint, or TTL. | -| Workspace input files with `read_only` access | Reject before source resolution; never overlay or copy up immutable source. | +| Input file targets a declared `read_only` source destination | Reject without overlay, copy-up, or source mutation; inputs outside mounted roots remain allowed. | | Unauthorized `persistent` retention | Fail before source resolution; do not downgrade to `session`. | | Invalid Git URL, ref, destination, network target, redirect, or feature | Fail the response, cancel bounded source work, and remove staging; do not start a harness or provider call. | | Any repository in a multi-repository source fails | Fail the complete source; never attach a partial set. | @@ -381,9 +460,10 @@ The implementation fails closed without changing source mode, source identity, a | Materializer timeout, crash, cancellation, or live descendant | Terminate and reap the complete process tree before cleanup and terminal acknowledgement. | | Working directory missing, not a directory, or escaping by traversal/link | Fail before attachment and harness execution. | | Crash during publication or attachment commit | Recover to either a complete verified attachment or no attachment; never expose partial staging. | +| Mountpoint is non-empty, is a link, or a bind/remount operation fails | Fail closed before harness execution; never expose a writable source alias or partial attachment. | | Missing or corrupt bound state on continuation | Fail as non-resumable; never rematerialize or substitute. | | External provider authentication or execution failure | Return the normalized UHP failure; do not switch endpoint, protocol, credential, or harness. | -| Cleanup failure | Quarantine and continue accounting for the allocation; retry the same idempotent cleanup path. | +| Cleanup or unmount failure | Quarantine and continue accounting for the allocation; never traverse the mount, and retry the same idempotent unmount-then-cleanup path. | Promptfoo treats every non-success as an evaluation error. It does not convert a workspace failure to an empty success, source fallback, or implicit retry. @@ -398,7 +478,7 @@ The public image remains `ghcr.io/allagentsdev/harnessrouter`. Tags identify the Release verification must exercise both Codex and OMP through the configured external provider gateway. In addition, v1 cannot release without: 1. an end-to-end OCI test using at least 2 GiB of expanded source bytes and 100,000 source-visible filesystem entries that fetches by direct manifest digest, applies layers and whiteouts, verifies the workspace manifest and any offline Git history, starts a harness in `working_directory`, and continues the same session successfully; and -2. a cache-reuse proof showing that identical Git and OCI source identities publish once, concurrent cache misses singleflight, every concurrent or later `read_only` task/session leases the same immutable generation bytes without cloning or copying, read-only checkpoints contain no generation bytes or source Git writes, mutable harness state remains per-session outside source, `editable` sessions derive inode-independent copies from the pinned generation, unrelated bare-cache refs are not exposed, and continuation reattaches the exact generation without reacquisition. +2. a cache-reuse proof showing that identical Git and OCI source identities publish once, concurrent cache misses singleflight, every concurrent or later `read_only` task/session bind-mounts the same immutable generation inodes at its declared destinations without cloning or copying, the outer workspace remains private and writable, input and produced files outside source roots work normally, source-targeting inputs fail, mount failures fail closed, read-only checkpoints and Files/root-Git/cleanup traversal contain no generation bytes or source Git writes, `editable` sessions derive inode-independent copies from the pinned generation, unrelated bare-cache refs are not exposed, and continuation restores outer state before reattaching the exact generation without reacquisition. These are release gates, not deferred performance tests. Git and OCI failure-path coverage must also prove that no partial generation or source-mode fallback becomes visible. @@ -411,6 +491,7 @@ Downstream implementation and release do not wait on upstream work. Once downstr | Build a new execution gateway | Duplicates HarnessRouter's UHP, sessions, workspace lifecycle, streaming, cancellation, files, artifacts, and harness supervision. | | Put a workspace service in front of HarnessRouter | Splits source and session ownership and cannot safely participate in checkpoint hydration, continuation, or produced-file bookkeeping. | | Create a second checkout root inside each session | Competes with the runner-owned workspace, duplicates cleanup and quota state, and makes files and checkpoints ambiguous. | +| Attach shared source with symlinks | Symlinks do not enforce read-only access, expose backing paths, can escape workspace containment, and behave inconsistently for cwd and file tools; namespace-confined read-only bind mounts present ordinary destination directories and fail closed. | | Ship Git first and defer OCI | Fails the minimum large-repository use case and makes release viability depend on repeated acquisition. | | Treat OCI as a runtime or benchmark image | Mixes source provenance with tools, services, verifier assumptions, and execution policy. | | Wait for upstream before implementation | Makes delivery depend on a project we do not maintain and delays the evidence needed for a useful upstream proposal. | @@ -422,7 +503,7 @@ Downstream implementation and release do not wait on upstream work. Once downstr ## Deliberate v1 limits -V1 supports public HTTPS Git repositories acquired at fixed depth `2`, multiple pairwise non-overlapping destinations, advertised branch/tag/default refs, exact resolved-commit and effective-depth provenance, server-owned bare acquisition caches, operator-catalogued OCI snapshots selected by direct digests, optional normalized offline Git history, `read_only` and `editable` attachments, bounded `session` retention, authorized `persistent` retention, and a workspace-relative working directory. +V1 supports public HTTPS Git repositories acquired at fixed depth `2`, source placed under 1 to 128 pairwise non-overlapping non-root destinations, advertised branch/tag/default refs, exact resolved-commit and effective-depth provenance, one server-owned bare acquisition cache per canonical URL, operator-catalogued OCI snapshots selected by direct digests, optional normalized offline Git history, writable private session roots with `read_only` repository bind mounts or private `editable` copies, bounded `session` retention, authorized `persistent` retention, and a workspace-relative working directory. V1 does not include caller-supplied registry origins or credentials, mutable OCI tags, OCI indexes as source identity, transparent Git/OCI fallback, caller-selected runtime images or benchmark environments, arbitrary materializer commands, private-network Git origins, caller-selected TTLs, session branching, access or retention changes on continuation, or public multi-tenant authorization. It does not add an AllAgents CLI command or change project workspace configuration. @@ -430,10 +511,17 @@ Only Codex and OMP are required and release-validated. Other HarnessRouter backe ## Consequences -HarnessRouter remains the sole execution, workspace, and session control plane. The fork gains deterministic first-turn source initialization without adding a new northbound API, process supervisor, checkpoint system, file service, or workspace lifecycle. - -Mandatory OCI support and generation accounting make v1 more substantial than a Git clone hook, but they make the minimum large-repository use case viable. Shared immutable generations avoid repeated acquisition for `read_only` sessions; private inode-independent copies preserve isolation for `editable` sessions. - +HarnessRouter remains the sole execution, workspace, and session control plane. +The fork gains deterministic first-turn source initialization without adding a +new northbound API, process supervisor, checkpoint system, file service, or +workspace lifecycle. Each session keeps its private writable root; only +declared source roots participate in generation sharing. + +Mandatory OCI support and generation accounting make v1 more substantial than +a Git clone hook, but they make the minimum large-repository use case viable. +Read-only bind mounts let matching sessions reuse the same protected generation +inodes without making harness state or outputs read-only. Private +inode-independent copies preserve isolation for `editable` sessions. The operator assumes finite capacity management for staging, generations, editable copies, persistent sessions, tombstones, and quarantined deletion failures. Protected or uncertain state is never advertised as free capacity. Provider credential lifecycle remains outside HarnessRouter. The distribution depends on the external OAuth-to-OpenAI-compatible gateway, while the harness sees only brokered short-lived credentials. @@ -444,7 +532,7 @@ Revisit this decision if: - UHP or upstream HarnessRouter adopts an equivalent workspace-source contract; - HarnessRouter changes its fresh/checkpoint workspace lifecycle so the extension point no longer preserves one authoritative session workspace; -- the host cannot enforce immutable shared generations and inode-independent editable copies; +- the host cannot enforce namespace-confined read-only bind mounts for shared generations and inode-independent editable copies; - large-repository OCI materialization or cache reuse cannot meet finite release limits; - continuation and attachment recovery cannot fail closed without source reacquisition; - source acquisition requires a stronger isolation boundary; diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 0b33705b..c4abcd89 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -16,38 +16,50 @@ The pinned HarnessRouter baseline already owns the session workspace lifecycle. The fork must extend that lifecycle rather than introduce another workspace abstraction: -1. Session identity implicitly selects the session workspace; callers do not - currently describe a source workspace. -2. Fresh hydration creates the session workspace as an empty root Git - repository. +1. Session identity implicitly selects one private, writable session workspace; + callers do not currently describe source roots. +2. Fresh hydration creates that workspace as an empty root Git repository. 3. Continuation restores the session checkpoint selected by the existing response/session identity. -4. Attached files, `.harness` state, generated root instructions, plugins, - skills, MCP configuration, HOME, and conversation state are materialized - under the hydrated workspace before the harness turn starts. -5. Produced-file collection uses a cursor over the root Git repository. -6. Stock checkpointing mutates that root Git repository and archives the entire - workspace; stock hydration clears the workspace before restoring the archive. +4. Attached UHP input files, `.harness` state, generated root instructions, + plugins, skills, MCP configuration, HOME, conversation state, outputs, and + produced/checkpoint bookkeeping are written under the hydrated workspace + before or during a harness turn. +5. Produced-file collection uses a cursor over the root Git repository and may + assume that every descendant is part of that repository. +6. Stock checkpointing mutates the root Git repository, then archives the entire + workspace; stock hydration clears the workspace before restoring that archive. + Stock file walking, root Git operations, archive creation, restore, and + cleanup have no mount-boundary exclusions. 7. No stock UHP request field names a Git source, an OCI source, or a reusable immutable generation. -These are the starting facts and the integration constraints. The implementation -keeps the existing session allocation, hydrate/checkpoint cycle, file and -artifact APIs, user/sandbox isolation, cancellation, TTL, cleanup, and harness -supervision. It adds first-turn source composition at the existing hydration -boundary, stores the resulting attachment in the existing session state, and -adapts the existing produced-file cursor for nested repositories. It does not -create a second workspace, second session database, second Files API, external +Phase 1 must characterize the exact paths and ordering of every stock write, +root Git command, filesystem walk, archive/restore step, and pre-turn asset +materialization rather than assuming the summary above is exhaustive. Those +observations define the smallest adaptation seam: the outer session workspace +remains private and writable, while only declared source destinations become +access-specific attachments. + +The implementation keeps the existing session allocation, hydrate/checkpoint +cycle, file and artifact APIs, user/sandbox isolation, cancellation, TTL, +cleanup, and harness supervision. It adds first-turn source composition at the +existing hydration boundary, stores the resulting attachment manifest in the +existing session state, and adapts root Git, checkpointing, files, restore, and +cleanup so they never traverse a declared source mount. It does not create a +second workspace, second session database, second Files API, external materializer service, or parallel lifecycle. - ## Goal Extend `allagentsdev/harnessrouter` so a new UHP session can compose its existing -HarnessRouter workspace from either multiple Git repositories or a mandatory OCI -workspace snapshot. Bind the verified immutable generation and its access mode -to the session before the first harness turn. Continuations omit the descriptor -and recover the exact attachment through the access-specific workspace-aware -checkpoint path. +private, writable HarnessRouter workspace from either multiple Git repositories +or a mandatory OCI workspace snapshot. Bind each verified source root and its +access mode to the session before the first harness turn. For `read_only`, mount +the immutable generation's repository roots read-only at their declared +non-root destinations; keep `.harness`, HOME, generated instructions, inputs, +outputs, and all other outer workspace state writable. Continuations omit the +descriptor, restore the writable outer checkpoint, and recover the exact source +attachments through the access-specific workspace-aware checkpoint path. OCI workspace snapshots are a release-blocking v1 source, not a later optimization. Large repositories are part of the minimum deliverable. The Git @@ -81,11 +93,13 @@ Keep all other boundaries unchanged: and workspace-manifest digests. - One immutable generation store shared by Git and OCI sources. - Reuse of a verified immutable generation across sessions. -- Shared immutable bytes for `read_only` sessions and inode-independent private - copies for `editable` sessions. +- Shared immutable generation inodes for `read_only` source roots and + inode-independent private writable copies for `editable` source roots. +- A private writable outer session workspace for both access modes, with + generated assets and allowed UHP input files outside source destinations. - Existing session continuation, checkpoint, cancellation, TTL, deletion, - restart reconciliation, files, artifacts, and produced-file behavior adapted - to the attachment. + restart reconciliation, files, artifacts, root Git, and produced-file behavior + adapted to exclude declared source mounts. - Authorized persistent retention through the existing session lifecycle. - Exact source provenance and bounded, coded failures. - Direct Promptfoo coverage against the built image, including a large OCI @@ -175,30 +189,42 @@ Rules: succeeds and before source resolution, network traffic, or generation claims. 3. `working_directory` and every repository `destination` are normalized workspace-relative POSIX paths. The effective working directory must be a real - directory inside the final attached workspace without traversal or link - escape. -4. Repository destinations must be unique and pairwise non-overlapping: no two - destinations may be equal, and neither may be an ancestor of another. -5. The repository array contains 1 to 128 entries. Each URL, optional ref, and - destination is bounded before network or filesystem work. Lower runtime - capacity fails with the coded capacity error; it does not change schema - validity. -6. `source` is a closed discriminated union. Unknown fields and mixed Git/OCI + directory inside the final workspace without traversal or link escape. +4. Every source destination is non-root: `.` and any spelling that normalizes to + the workspace root are invalid. Destinations are unique and pairwise + non-overlapping: no two may be equal, and neither may be an ancestor of + another. A monorepo therefore uses a destination such as `repo`, and + `working_directory` may be `repo` or `repo/packages/api`. +5. Repository mode contains 1 to 128 entries. An OCI workspace manifest likewise + declares 1 to 128 repository roots, each with a non-root, pairwise + non-overlapping destination. Snapshot source files may exist only beneath + those roots; there is no snapshot source at destination `.` and no undeclared + root source file. Ancestor directories needed to reach a destination are + mount scaffolding only and contain no source files. +6. Each URL, optional ref, destination, snapshot field, and manifest root is + bounded before network or filesystem work. Lower runtime capacity fails with + the coded capacity error; it does not change schema validity. +7. `source` is a closed discriminated union. Unknown fields and mixed Git/OCI fields fail validation. -7. `snapshot_name` selects an operator-owned HarnessRouter deployment catalog +8. `snapshot_name` selects an operator-owned HarnessRouter deployment catalog entry. The request supplies only the two `sha256:` digests; it never supplies a registry, repository, credential, certificate, or mirror. -8. A continuation selected through `previous_response_id` or the existing +9. A continuation selected through `previous_response_id` or the existing session recovery mechanism omits `metadata.workspace`. Supplying it on a reused session fails before hydrate, source access, generation lookup, or provider traffic, even when it is identical to the stored value. -9. A session created without workspace metadata remains a stock session and - cannot add workspace metadata later. -10. UHP input files are workspace mutations. Reject them on a `read_only` - first turn before source resolution. For `editable`, apply them only after - the private copy exists and before the initial produced-file baseline. - Harness assets and runner control state remain outside source content. -11. Workspace fields cannot contain commands, environment variables, resource +10. A session created without workspace metadata remains a stock session and + cannot add workspace metadata later. +11. UHP input files target the writable outer workspace by default. For + `read_only`, reject an input whose normalized path is equal to or below an + effective source destination; inputs elsewhere remain valid. Repository-mode + destinations are known during request validation. Snapshot destinations are + validated after the image/workspace manifests establish the verified root + map, but before layer acquisition, outer workspace writes, generation + attachment, or harness execution. For `editable`, inputs may overlay the + private source copies. In both modes, reject paths that collide with mount + scaffolding, reserved runner paths, or generated assets. +12. Workspace fields cannot contain commands, environment variables, resource limits, provider settings, or harness settings. A workspace snapshot request is therefore: @@ -228,11 +254,12 @@ record. The binding contains: - the canonical effective descriptor and its digest; - the source kind and exact resolved source identity; -- the generation key, immutable publication/epoch identity, and verified tree - manifest digest; +- the generation key, immutable publication/epoch identity, verified tree + manifest digest, and immutable declared source-root map; - `access`, effective `retention`, and normalized `working_directory`; -- the attachment method and evidence needed to prove the restored session still - refers to that exact generation; +- the attachment manifest and evidence for every destination, including + generation root identity, mount/copy method, filesystem identity, and mount + protection needed to prove a restored session refers to the exact generation; - exact public provenance; - existing session expiry/deletion state; and - the selected harness/provider binding already owned by the session. @@ -266,49 +293,83 @@ The first-turn sequence is: idempotency ownership. 2. Resolve whether the request creates or reuses a session. 3. If new and workspace-backed, validate and authorize the closed workspace - descriptor, then store a pending binding in the existing session transition. -4. Run fresh hydration only far enough to allocate the existing session - workspace and isolation identity. Skip stock empty-root Git initialization - for workspace-backed sessions; do not allocate another checkout root. -5. Create a per-session writable control root in that allocation but outside - source content. Redirect `.harness` state, HOME, conversation data, plugins, - skills, MCP/configuration, credentials, scratch, and produced/checkpoint - bookkeeping to it. -6. Resolve the immutable source identity, claim or reuse the generation, and - attach it at the existing workspace path with the requested access mode. -7. Reject workspace input files for `read_only`. For `editable`, apply them to - the private copy before the initial produced-file baseline. Supply generated - instructions through a harness-supported external instruction path or a - non-shadowing session mount; never write, overlay, or copy up a source path. -8. Establish runner-owned outer and nested produced-file cursors, validate the - effective working directory, and prove a `read_only` attachment has no - writable alias. -9. Atomically mark the existing session attachment ready, then continue through - ordinary provider selection and harness execution. -10. Collect files/artifacts and checkpoint through the access-specific path: - `read_only` stores only session-local mutable state plus exact attachment - evidence and excludes generation bytes; `editable` checkpoints its private - workspace copy. Stream events, set terminal state, and schedule cleanup - through existing HarnessRouter paths. + descriptor, request-declared repository destinations, working-directory + syntax, and input paths that can be decided without source access. Store a + pending binding in the existing session transition. +4. Allocate the existing private, empty, writable session workspace and isolation + identity without running fresh-hydration writes, root Git, bookkeeping, or + asset materialization. +5. Resolve the exact source plan. For repositories this resolves exact commits; + for OCI it verifies the image and workspace manifests sufficiently to obtain + the authoritative 1–128 root map before fetching/extracting layers. Validate + all non-root, non-overlap, input, generated-asset, reserved-path, and restored + outer-state collisions against that effective root map before any outer + workspace content write or generation claim/materialization. +6. Run fresh hydration and materialize root Git/bookkeeping, `.harness`, HOME, + conversation data, generated instructions, plugins, skills, + MCP/configuration, credentials, scratch, and other stock mutable assets in + their normal private outer locations. Configure every root Git command and + filesystem walk to exclude the immutable destination set and never cross + mount boundaries. Apply allowed outer UHP inputs at the stock pre-turn point. + Create empty, non-link mountpoint directories and mount-ancestor scaffolding + only after collision validation. +7. Claim or reuse the generation. For `read_only`, take a lease and attach each + immutable generation repository root to its destination with a per-session + read-only bind mount. For `editable`, create each inode-independent private + writable source copy at its destination. +8. For `editable`, apply source-targeting inputs to the private copies. For + `read_only`, source-targeting inputs have already failed. Verify every + destination, protection flag, source identity, absence of writable aliases, + and editable inode independence before recording attachment-ready. +9. Establish the outer produced-file cursor without traversing source + destinations and the access-specific per-root collectors. Validate the + effective working directory after attachments are complete. +10. Atomically persist the attachment manifest/evidence and mark the existing + session ready, then continue through ordinary provider selection and harness + execution. +11. Collect files/artifacts without mount traversal, then checkpoint the writable + outer workspace while explicitly excluding every source destination and all + source bytes. Persist attachment manifest/evidence separately. For + `editable`, nested source collectors and the editable-source checkpoint path + preserve each private root without allowing the outer archive to traverse it. + Stream events, set terminal state, and schedule cleanup through existing + HarnessRouter paths. A continuation does not parse or resolve a source. Workspace-aware hydration -first restores and validates only the control metadata needed for attachment: -generation key/epoch, manifest, durable reference, and attachment evidence. It -then acquires and mounts that exact protected generation, and only afterward -restores the remaining session-local harness state around the source mount. A -missing, expired, corrupt, wrong-generation, or unsupported attachment fails -closed. Hydration must not reacquire Git, contact an OCI registry, use another -cached generation, archive or restore shared generation bytes, or start with an -empty workspace. +uses this exact order: + +1. recover and validate the minimal stored binding, durable generation reference, + source-root map, access mode, and attachment manifest/evidence without + resolving Git or OCI; +2. ensure stale session mounts are unmounted, then clear/hydrate the outer + workspace using no-follow, no-cross-mount operations; +3. restore the writable outer checkpoint, which contains no source bytes; +4. validate that every declared destination is an empty non-link directory, that + its ancestors contain only allowed outer state/scaffolding, and that no input, + generated asset, root Git entry, or restored path collides with an attachment; +5. reacquire the exact recorded generation lease and, for `read_only`, bind each + exact recorded generation root read-only at its destination; for `editable`, + restore the exact private source-root checkpoint at its destination; +6. verify mount flags, filesystem/generation identity, source manifest, no + writable alias, and editable ownership/inode independence; and +7. only then restore or activate remaining runtime state, rebuild external + cursors, validate `working_directory`, and start the harness. + +A missing, expired, corrupt, wrong-generation, writable, partially mounted, or +unsupported attachment fails closed. Hydration must not reacquire Git, contact +an OCI registry, select another cached generation, archive or restore shared +generation bytes, or start with an empty source root. Any partial continuation +attachment is unmounted before failure cleanup. `retention: "session"` follows existing finite session TTL and deletion. Authorized `persistent` retention pins the existing session and its generation reference until explicit deletion or operator policy permits removal; it does not create a second retention scheduler. Polling and response replay do not -extend retention. Cleanup makes the session unavailable before releasing its -attachment, private copy, or generation reference and remains idempotent across -restart. - +extend retention. Cleanup makes the session unavailable, unmounts every source +destination, verifies that no mount remains, then hydrates/deletes outer state, +removes editable copies, and releases the generation reference. The same +unmount-before-hydrate/delete rule applies to cancellation, retry, and restart +reconciliation and remains idempotent. ## Generation and attachment design A generation is a verified immutable source artifact, not a runnable workspace @@ -391,8 +452,9 @@ remote pack fetch as a hit. registry manifest/blob request, extraction, tree copy, or second publication. Both record only a new session lease/reference. - Every `read_only` session whose normalized source identity matches a cached - generation attaches that same immutable generation. It cannot choose to - reclone, re-extract, rematerialize, or copy cached source bytes. + generation bind-mounts the same immutable generation repository roots at its + declared destinations. It cannot choose to reclone, re-extract, rematerialize, + hard-link, symlink, or copy cached source bytes. - Failed or cancelled builds remove staging after descendants stop. A partially refreshed mirror, commit snapshot, or generation is quarantined and never attached. @@ -403,28 +465,40 @@ remote pack fetch as a hit. ### Access-specific attachment -- `read_only`: take a lease and bind the cached immutable generation directly at - the existing session workspace through a read-only filesystem view. A second - matching session performs zero source-tree copy and shares the same verified - generation inodes/bytes, while the writable control root, HOME, checkpoint, - conversation, produced records, harness runtime, and lifecycle state remain - isolated. Workspace input files are absent. After control-root assets and - external instructions are ready, verify that root, nested paths, symlink - paths, bind aliases, and alternate descriptors cannot write or copy up into - source. -- `editable`: take a lease, then create a private, inode-independent tree from - the verified generation inside the existing session workspace allocation. - Reflink/copy is allowed only when it yields independent inodes and writes - cannot alter the generation or another session. Hard-linked mutable files are - forbidden. +- `read_only`: take a lease, then create one per-session bind mount from each + immutable generation repository root to its declared non-root destination in + the existing private writable workspace. Remount or create the bind with a + kernel-enforced read-only view plus `nodev` and `nosuid`, while preserving + repository execute bits required by tools. Mount setup is namespace-confined + to the session, exposes no generation backing path or writable file + descriptor, has no writable alias or overlay/copy-up path, and fails closed if + any protection or identity check fails. Matching sessions see the same source + filesystem identities/inodes, while `.harness`, HOME, inputs, outputs, + generated instructions, root Git/bookkeeping, checkpoint, conversation, + harness runtime, and lifecycle state remain private and writable outside the + destinations. +- Symlinks are explicitly rejected as an attachment mechanism. They do not + enforce read-only access, can escape workspace containment, disclose backing + paths, make cwd and tool path behavior surprising, and do not provide a + trustworthy mount boundary for root Git, archives, Files, or cleanup. +- `editable`: take a lease, then create an inode-independent private writable + copy of each verified generation repository root at its declared destination. + Reflink/copy is allowed only when later writes cannot alter the generation or + another session. Hard-linked mutable files and writable aliases are forbidden. - Both modes preserve nested repository administrative state allowed by the - source manifest. The attachment never exposes the bare Git mirror or moves - harness HOME, credentials, scratch, or checkpoint control into repository - content. -- Continuation reuses the exact lease and attachment. An editable continuation - sees its mutations; a read-only continuation sees the same immutable - generation and its own session-local state. - + source manifest. Only declared source destinations contain source bytes; + ancestors are empty scaffolding apart from independently valid outer state. + The attachment never exposes the bare Git mirror or moves harness HOME, + credentials, scratch, generated assets, outputs, or checkpoint control into + repository content. +- Root Git, tar/archive, Files, hydrate, and cleanup receive the immutable + destination set and use no-follow, no-cross-mount traversal. They explicitly + exclude the destination paths rather than relying on the mounts being + read-only. +- Continuation reuses the exact lease and attachment map. An editable + continuation sees its private mutations; a read-only continuation restores + writable outer state first, then reattaches the same immutable generation + roots and its own session-local state. ## Implementation phases Each phase ends with observable proof. Source inspection or mock-forwarding @@ -445,14 +519,17 @@ Work: `allagentsdev/harnessrouter` as the existing fork. 3. Pin base image, OS packages, Git and OCI libraries/tools, Codex, OMP, Promptfoo, lockfiles, and CI actions. Build the unchanged image first. -4. Add focused characterization scenarios for fresh empty-root Git hydration, - continuation checkpoint restore, attached-file/harness-asset ordering, - root-Git produced-file cursor behavior, cancellation, TTL cleanup, restart, - and requests with arbitrary metadata. +4. Instrument focused characterization scenarios for fresh empty-root Git + hydration, every pre-turn workspace write, attached-file/harness-asset + ordering, root-Git commands and excludes, produced-file walks, checkpoint Git + mutation, archive creation/restoration, continuation clearing, cancellation, + TTL cleanup, deletion, and restart. Record path, ordering, symlink policy, and + whether each operation crosses a filesystem mount. 5. Capture stock UHP request/stream/error behavior for requests without `metadata.workspace`; these traces become compatibility fixtures. 6. Record the precise gateway/session/hydrate/runner/checkpoint/files call path - in code comments or tests where the fork seam lands. Do not add a generic + and the concrete root Git, tar/archive, file-walk, and cleanup entry points + that must receive source-destination exclusions. Do not add a generic extension framework. Exit proof: @@ -477,8 +554,11 @@ Work: 2. Apply metadata byte, nesting, list-count, and string-length bounds before session allocation or source work. 3. Strictly validate the closed union, snake_case names, access, retention, - destinations, working directory, direct `sha256:` digest syntax, and the - `read_only` input-file exclusion. + repository-mode destinations, working-directory syntax, direct `sha256:` + digest syntax, request-decidable input/asset/reserved-path collisions, and + the rule that `read_only` inputs may target only paths outside effective + source destinations. Snapshot-root validation occurs at verified-plan + resolution because the request does not carry those destinations. 4. Authorize persistent retention before source resolution or generation lookup. 5. Canonically serialize the effective descriptor with the default `retention: "session"` and calculate its digest. @@ -496,85 +576,100 @@ Work: 10. Add public provenance only after attachment is ready. Retrieval/replay uses stored provenance rather than resolving it again. -Proof includes valid descriptors for both source kinds; repository counts -`0`, `1`, `128`, and `129`; multiple repository entries; default retention; -authorized and unauthorized persistence; invalid unions, fields, digests, -paths, and destinations; workspace injection on both continuation mechanisms; -idempotent duplicates; harness mismatch; exact response-schema fixtures for -both provenance variants; and a byte-for-byte stock trace for requests without -the descriptor. +Proof includes valid descriptors for both source kinds; repository request +counts `0`, `1`, `128`, and `129`; multiple repository entries; rejection of +request-declared destination `.`, normalized aliases, equality, ancestor +overlap, and request-decidable input/asset/reserved-path collisions; acceptance +of read-only outer inputs and rejection of read-only repository-source inputs; +default retention; authorized and unauthorized persistence; invalid unions, +fields, digests, paths, and destinations; workspace injection on both +continuation mechanisms; idempotent duplicates; harness mismatch; exact +response-schema fixtures for both provenance variants; and a byte-for-byte +stock trace for requests without the descriptor. Snapshot root counts, +destinations, files outside roots, and snapshot-input collisions are proved in +the OCI phase after verified workspace-manifest resolution. -### Phase 3: Add the immutable generation store and existing-workspace attachment seam +### Phase 3: Add generation attachment and mount-aware workspace lifecycle -**Outcome:** one generic runner seam claims, publishes, reuses, and attaches a -verified generation inside the existing hydrate lifecycle. +**Outcome:** one runner seam publishes immutable generations and attaches only +their declared source roots inside the existing private writable workspace. Primary surfaces are existing runner/session hydrate code in `runner/server.py`, -its current gateway transport, checkpoint/session persistence, startup -reconciliation, and existing cleanup scheduling. +the gateway transport, root Git initialization and cursor code, checkpoint +archive/restore, Files walking, startup reconciliation, and cleanup scheduling. Work: 1. Add one internal `resolve -> claim/reuse -> materialize -> verify -> publish -> - attach` pipeline selected by the closed source union. It is not a public API or - plugin registry. -2. Replace only the fresh empty-root initialization point for workspace-backed - sessions. The designated path remains the existing session workspace. -3. Persist bare-mirror refresh/snapshot state, generation claims, staging, - publication, session leases/references, and attachment-ready transitions using - the runner's current durable state and recovery ordering. -4. Build and verify the canonical source-visible manifest: normalized relative - path, type, mode, size/content identity, link target where applicable, declared - repository ownership, and optional normalized Git-history declaration. -5. Implement access-specific attachment and verify read-only alias resistance or - editable inode independence before marking the session ready. -6. Add a per-session writable control root outside source content and redirect - every stock workspace-internal mutable path there: `.harness`, HOME, - conversation state, plugins, skills, MCP/configuration, credentials, scratch, - produced-file indexes, and checkpoint control. No mutable runtime path may - resolve into an immutable generation. -7. Use harness-supported external instruction inputs or a non-shadowing - session-only mount for generated instructions. Reject the implementation if - either supported harness requires overwriting, overlaying, or copying up a - source path. -8. Reject UHP input files for `read_only`; for `editable`, apply them to the - private copy at the normal pre-turn point and establish the initial - produced-file baseline afterward. -9. Add access-specific checkpoint/hydrate behavior. A read-only checkpoint - excludes the generation mount and source bytes, persists only session-local - mutable state plus exact attachment evidence, and reattaches the same - generation before restore. An editable checkpoint carries the private - workspace copy and never the mirror or generation backing store. -10. Reuse existing user/sandbox isolation, process groups, timeouts, + attach` pipeline selected by the closed source union. It is not a public API + or plugin registry. +2. Preserve fresh allocation of the existing private writable workspace and its + stock-like root Git/bookkeeping. Thread one immutable destination set through + root Git, produced-file walks, checkpoint tar/archive, hydrate clearing, + deletion, and cleanup. Each operation must use explicit path excludes plus + no-follow/no-cross-mount traversal; a read-only mount is not itself an + adequate traversal guard. +3. Persist mirror refresh/snapshot state, generation claims, staging, + publication, session references, attachment manifests, per-root mount + evidence, editable-copy identity, and attachment-ready transitions using the + runner's current durable state and recovery ordering. +4. Build and verify the canonical source manifest: 1–128 non-root pairwise + non-overlapping destinations; normalized relative path, type, mode, + size/content identity, link target, repository ownership, immutable + generation-root identity, and optional normalized Git-history declaration. +5. Reject mountpoints or ancestors that collide with restored outer files, + symlinks, root Git state, UHP inputs, generated assets, or another root. Create + only empty non-link destination directories and required ancestor scaffolding. +6. For `read_only`, create namespace-confined per-session bind mounts from exact + generation repository roots. Enforce read-only, `nodev`, and `nosuid`, preserve + executable file bits, retain no writable backing descriptor/alias, and fail + closed on any mount or remount error. Never use symlinks or overlay copy-up. +7. For `editable`, create per-root inode-independent private writable copies. + Apply source-targeting inputs only after copies exist. Apply allowed outer + inputs and generated assets in the writable workspace before final baselines. +8. Adapt checkpointing so the outer archive explicitly excludes every source + destination and contains no source bytes. Store attachment manifest/evidence + separately. Checkpoint editable roots through per-root collectors; never let + the outer tar traverse them. Root Git may retain stock-like bookkeeping for + outer paths but must exclude all source destinations from index, commit, + status, diff, and cleanup. +9. Implement the exact continuation order specified above: validate binding, + unmount stale roots, hydrate/restore outer state, validate empty non-link + mountpoints and collisions, reattach exact read-only roots or restore exact + editable roots, verify identities/protection, rebuild cursors, then start the + harness. No source endpoint or replacement generation is allowed. +10. Unmount all source destinations before hydrate, archive restore, deletion, or + cleanup. Partial mount sets unwind in reverse order. Mount absence, + attachment evidence, references, private copies, and cleanup marks reconcile + idempotently after restart; uncertainty quarantines and fails closed. +11. Reuse existing user/sandbox isolation, process groups, timeouts, cancellation, TTL, deletion, and cleanup. Source workers inherit - cancellation and must stop all descendants before terminal acknowledgement. -11. Reconcile abandoned mirror refreshes, immutable commit snapshots, staging, - incomplete publication, ready generations, pending attachments, leases, - references, private copies, control roots, and cleanup after restart. -12. Verify continuations against the stored generation epoch/manifest, durable - reference, and attachment evidence; never resolve or materialize source on - continuation. -13. Expose separate bounded counters/events for ref resolution, remote pack - acquisition, mirror singleflight, generation build/publication/hit, - read-only zero-copy attachment/checkpoint, editable copy/checkpoint, - reference, cancellation, quarantine, and cleanup. Do not log request - credentials, registry locations, internal paths, or uncontrolled tool - output. - -Proof uses deterministic fake Git and OCI builders to show one mirror refresh -and one publication under concurrent misses, independent waiter cancellation, -two read-only sessions directly attaching the same verified generation with -zero second clone/fetch/materialization/copy, and two editable sessions with -independent inodes and mutations. Real Codex and OMP probes prove generated -instructions and all mutable harness assets live outside source. A read-only -checkpoint contains no generation bytes. Continuation restores and validates -minimal binding/control metadata, reattaches the same protected generation, -then restores remaining session-local harness state; checkpoint creation writes -no root Git commit. Editable checkpoint/continuation preserves its private -mutations. Restart at every persisted transition and cleanup are idempotent. -Session/runtime state remains isolated even when immutable bytes are shared. A -stock session still creates and uses its normal root Git workspace. - + cancellation and stop descendants before terminal acknowledgement. +12. Expose separate bounded counters/events for ref resolution, remote pack + acquisition, generation build/publication/hit, per-root mount/unmount, + mount verification/failure, no-cross-mount exclusions, outer checkpoint, + editable copy/checkpoint, continuation reconciliation, quarantine, and + cleanup. Do not log credentials, origins, backing paths, or raw tool output. + +Proof mounts deterministic Git and OCI generations through the real runner. Two +read-only sessions have different writable workspace roots and independently +writable outer inputs, generated assets, `.harness`, HOME, and outputs, but +`stat`/filesystem evidence shows their matching source paths bind the same +generation inodes with no clone, extraction, materialization, hard link, +symlink, or tree copy. Writes by path, cwd, rename, link, descriptor, and +alternate alias fail with the filesystem's read-only error and leave the +generation and sibling unchanged. Two editable sessions have independent source +inodes and mutations. + +Checkpoint proof mutates outer state, creates a source sentinel larger than the +archive, and shows the archive contains the outer mutation and attachment +manifest but neither sentinel nor any source byte/path. Instrumented root Git, +tar/archive, Files, hydrate, and cleanup walkers observe zero entries below mount +destinations. Continuation first restores that outer mutation, validates empty +mountpoints, then reattaches the exact recorded roots and preserves executable +bits. Restart at every persisted mount/unmount/checkpoint transition is +idempotent; failed mounts expose no partial source. A stock session still uses +its unchanged root Git/checkpoint path. ### Phase 4: Implement repository composition with mandatory depth-2 acquisition **Outcome:** repository mode deterministically builds one generation from one or @@ -614,11 +709,12 @@ Work: the shallow boundary rather than flattening the merge. 9. Reject a server that cannot satisfy the bounded shallow fetch. Return a coded source error and do not deepen, full-clone, strip history, or fall back to OCI. -10. Check out the exact detached commit at each declared destination. Reject - submodule gitlinks and LFS pointer-backed content rather than fetching them. -11. Enforce destination non-overlap before network work, then build all - repositories into one staging tree. No repository may create paths outside - its destination or add undeclared root files. +10. Check out the exact detached commit under each declared non-root destination. + Reject destination `.`, overlap, submodule gitlinks, and LFS pointer-backed + content rather than fetching them. +11. Enforce destination shape and non-overlap before network work, then build all + repositories into one staging tree. No repository may create source outside + its destination or add undeclared root files; ancestors are scaffolding only. 12. Validate each repository's `HEAD`, index/worktree equality at publication, shallow metadata, closed refs/config, object reachability for the retained depth, file modes, links, bytes, inodes, and absence of credentials/remotes @@ -630,18 +726,19 @@ Work: error, timeout, cancellation, lost claim, or restart without invalidating a previously verified commit snapshot or referenced generation. -Behavior proof covers one repository at a top-level destination; several sibling/nested-path -repositories; the same URL at different refs and destinations; omitted default, -branch, lightweight tag, annotated tag, and moving-ref rejection; pairwise -non-overlap; depth exactly 2; `.git/shallow`; two-entry recent offline history; -merge-tip parents and diff semantics; detached exact commit; no network during -attached `git log`; submodule/LFS rejection; unsupported shallow server with no -fallback; cancellation; restart cleanup; and exact provenance. Concurrent cold -requests produce one advertised-ref refresh, one remote pack fetch, one verified -mirror snapshot, and one generation publication. After resolution confirms the -same exact identity, a second read-only request performs zero clone, zero pack -transfer, zero checkout/materialization, and zero tree copy, attaches the same -immutable generation bytes, and retains isolated session/runtime state. +Behavior proof covers rejection of root destination `.`; one repository at a +non-root top-level destination; several sibling/nested-path destinations; the +same URL at different refs and destinations; omitted default, branch, +lightweight tag, annotated tag, and moving-ref rejection; pairwise non-overlap; +depth exactly 2; `.git/shallow`; two-entry recent offline history; merge-tip +parents and diff semantics; detached exact commit; no network during attached +`git log`; submodule/LFS rejection; unsupported shallow server with no fallback; +cancellation; restart cleanup; and exact provenance. Concurrent cold requests +produce one advertised-ref refresh, one remote pack fetch, one verified mirror +snapshot, and one generation publication. After resolution confirms the same +exact identity, a second read-only request performs zero clone, pack transfer, +checkout/materialization, or tree copy and bind-mounts the same immutable +repository-root inodes while its writable outer state remains isolated. The release notes must state plainly that depth 2 reduces transferred history, not the checked-out working-tree bytes. Large repositories still require the OCI @@ -663,19 +760,22 @@ Work: 1. Require a direct OCI image manifest digest. Reject tags, mutable references, manifest indexes/lists, caller-selected repositories, and catalog/digest mismatches. -2. Fetch the direct image manifest, config, workspace manifest, and referenced - layers only from the selected catalog entry. Implement bounded registry - authentication and exact-host redirect policy without exposing credentials to - the harness, session workspace, logs, response, or provenance. -3. Verify every descriptor digest and size before use. Require the declared - `workspace_manifest_digest` to identify the exact workspace manifest used to - validate the final tree. -4. Stream decompression and extraction inside the fixed v1 envelope: at most 64 - distributable tar/gzip/zstd layers; a 4 MiB image manifest; a 128 MiB - workspace manifest with at most 128 repository roots; 8 GiB total compressed - layer bytes; 32 GiB expanded source bytes; 500,000 entries; 4 GiB per regular - file; paths of at most 4096 UTF-8 bytes and 128 components; and 1 MiB per PAX - or extended header. For each layer and the aggregate artifact, +2. Fetch the direct image manifest, config, and workspace manifest only from the + selected catalog entry. Implement bounded registry authentication and + exact-host redirect policy without exposing credentials to the harness, + session workspace, logs, response, or provenance. Do not request layers yet. +3. Verify every descriptor digest and size. Require the declared + `workspace_manifest_digest` to identify the exact workspace manifest. Before + layer acquisition, validate its 1–128 roots, non-root and non-overlap rules, + absence of source files outside roots or in ancestor scaffolding, and every + UHP input/generated-asset/reserved-path collision against the authoritative + root map. +4. Fetch referenced layers and stream decompression/extraction inside the fixed + v1 envelope: at most 64 distributable tar/gzip/zstd layers; a 4 MiB image + manifest; a 128 MiB workspace manifest; 8 GiB total compressed layer bytes; + 32 GiB expanded source bytes; 500,000 entries; 4 GiB per regular file; paths + of at most 4096 UTF-8 bytes and 128 components; and 1 MiB per PAX or extended + header. For each layer and the aggregate artifact, `expanded_bytes / max(compressed_bytes, 1)` must not exceed `100`. Enforce these bounds plus inode, output, and wall-time limits during streaming. Deployment configuration may lower but cannot raise them without a contract @@ -686,9 +786,10 @@ Work: representable by the workspace manifest. 6. Reject absolute paths, traversal, NULs, ambiguous separators, duplicate conflicting entries, devices, FIFOs, sockets, unsafe sparse files, and other - unsupported types. Validate symlink and hardlink targets against the final - workspace root; reject escaping, dangling-required-target, forward-link, and - link-cycle cases outside the supported bounded model. + unsupported types. Validate every symlink and hardlink target against its + owning declared repository root; reject links into another root or the + writable outer workspace, plus escaping, dangling-required-target, + forward-link, and link-cycle cases outside the supported bounded model. 7. Validate final paths, types, modes, sizes, content digests, links, repository roots, and destination non-overlap against the workspace manifest. Extra, missing, or changed source-visible entries fail before publication. @@ -706,14 +807,18 @@ Work: and return the source-specific error. Never clone Git, select a different digest, or use a stale generation as fallback. -Proof uses a local authenticated registry and malicious fixtures for digest/media -mismatch, indexes, redirects, authentication, truncation, compression bombs, -layer limits, whiteouts and opaque whiteouts, traversal, path/type/link attacks, -devices, sparse files, cancellation, partial cleanup, and restart. Positive -fixtures cover a tree-only workspace, multiple declared repository roots, -normalized offline Git history, offline `git log`/`git blame`/historical diff, -read-only attachment, editable copy, concurrent publication, and cache reuse -with zero second-session registry or extraction work. +Proof uses a local authenticated registry and malicious fixtures for +digest/media mismatch, indexes, redirects, authentication, truncation, +compression bombs, layer limits, whiteouts and opaque whiteouts, traversal, +path/type/link attacks, devices, sparse files, cancellation, partial cleanup, +and restart. Root-map proof covers 0, 1, 128, and 129 roots; destination `.`; +equal/ancestor overlaps; source files outside roots or in ancestor scaffolding; +and a `read_only` input collision rejected after workspace-manifest verification +but before any layer request or outer workspace write. Positive fixtures cover a +tree-only workspace, multiple declared repository roots, normalized offline Git +history, offline `git log`/`git blame`/historical diff, read-only attachment, +editable copy, concurrent publication, and cache reuse with zero second-session +registry or extraction work. OCI implementation and this proof are required before v1 release. A passing Git path cannot waive or defer them. @@ -725,34 +830,37 @@ across composed workspaces without inventing a second file API. Work: -1. Preserve the existing produced-file cursor semantics but move the outer - workspace baseline/index into the per-session control root. Never initialize, - commit, or mutate a bookkeeping `.git` directory inside an immutable - generation. -2. Register source-manifest repository roots and history mode when the attachment - becomes ready. The set is immutable for the session. -3. At each turn boundary, record the external outer baseline plus a cursor for - each declared repository root: Git `HEAD`/index/worktree state for +1. Preserve the existing root Git cursor in the writable outer workspace. + Register every source destination in root Git's internal excludes and pass the + immutable destination set to every root Git command so it never traverses a + mounted or copied source root. +2. Register source-manifest repository roots and history mode when the + attachment becomes ready. The set is immutable for the session. +3. At each turn boundary, record the outer root-Git cursor plus a cursor for each + editable declared repository root: Git `HEAD`/index/worktree state for history-bearing roots and manifest/file identity for tree-only roots. -4. Collect the union of outer and nested additions, modifications, deletions, - renames, and mode changes relative to the turn baseline. Normalize to - workspace-relative paths, assign each path to the most specific declared + Read-only roots require no change cursor because the filesystem prevents + mutation. +4. Collect the union of writable outer-workspace and editable-root additions, + modifications, deletions, renames, and mode changes relative to the turn + baseline. Normalize paths, assign each path to the most specific declared owner, deduplicate it, and preserve existing file size/count/type limits. 5. Never expose `.git` administrative files, generation-store paths, mount - internals, control state, provider credentials, harness assets, or checkpoint - internals as produced files. -6. Do not report immutable source baseline files merely because they arrived - during first-turn composition. Report only changes after the established - source/turn baseline. -7. In `read_only`, any attempted source mutation fails at the filesystem boundary - and produces no changed source entry. Collection reads the external baseline - without modifying source. -8. In `editable`, changes remain private to the session and are visible on - continuation and through the existing file and artifact APIs. -9. Preserve collection-before-checkpoint ordering. Read-only checkpointing skips - the generation mount; editable checkpointing includes only the private - workspace and session-local state. Cancellation cannot publish a partial - cursor or checkpoint. + internals, credentials, harness assets, or checkpoint internals as produced + files. +6. Do not report immutable or initial editable source files merely because they + arrived during first-turn composition. Report only changes after the + established source/turn baseline. +7. In `read_only`, any attempted source mutation fails at the filesystem + boundary, but additions and changes elsewhere in the writable workspace are + collected normally. No collector crosses a source mount. +8. In `editable`, source changes remain private to the session and are visible + on continuation and through the existing file and artifact APIs. +9. Preserve collection-before-checkpoint ordering. The outer checkpoint archive + explicitly excludes every source destination. Read-only sources are + reattached from the generation; editable sources use their separate private + source-root checkpoint path. Cancellation cannot publish a partial cursor or + checkpoint. Proof covers changes at workspace root and in every nested repository; two repositories changed in one turn; same filename under different destinations; @@ -872,16 +980,21 @@ Release-blocking scenarios: must record zero clone, pack transfer, checkout/materialization, and tree copy; it attaches the same immutable generation bytes while its session, runtime, conversation, outputs, and cleanup remain isolated. -6. **Access/lifecycle matrix:** cover read-only write denial, editable isolation, - session and authorized persistent retention, continuation, restart, expiry, - explicit deletion, cleanup retry, and missing/corrupt attachment with no +6. **Access/lifecycle matrix:** prove that read-only source writes fail while + root-level generated assets, an allowed UHP input, and an agent-created output + remain writable and are checkpointed. Prove root Git, Files, archive, hydrate, + and cleanup do not traverse source mounts. Cover editable isolation, session + and authorized persistent retention, continuation, restart, expiry, explicit + deletion, cleanup retry, and missing/corrupt attachment with no rematerialization. -7. **Failure matrix:** cover malformed descriptor, overlap, bad working directory, - unauthorized persistence, workspace input files with `read_only`, - shallow-fetch refusal, moving ref, submodule/LFS, wrong OCI digest, - workspace-manifest mismatch, extraction limit, path/link/type attack, - cancellation in both source modes, provider failure, and reused-session - descriptor injection. Assert there is no Git/OCI/provider fallback. +7. **Failure matrix:** cover malformed descriptor, root or overlapping + destination, bad working directory, unauthorized persistence, a `read_only` + input equal to or beneath a source destination, mountpoint/input/asset + collision, symlink attachment attempt, mount/remount failure, shallow-fetch + refusal, moving ref, submodule/LFS, wrong OCI digest, workspace-manifest + mismatch, extraction limit, path/link/type attack, cancellation in both + source modes, provider failure, and reused-session descriptor injection. + Assert there is no Git/OCI/provider fallback. 8. **Provider matrix:** complete Codex/Responses and OMP/Chat Completions through the one external provider gateway; reject route/model override; scan retained and public surfaces for caller, source, registry, broker, and provider secrets. @@ -945,7 +1058,7 @@ observable distinctions must remain: | Invalid shape, field, digest, destination, or working directory | Reject before session source work | | Unauthorized `persistent` retention | Reject before source lookup/network work | | Workspace metadata on a reused session | Reject without changing the existing binding or TTL | -| Workspace input files with `read_only` access | Reject before source resolution; never overlay or copy up immutable source | +| `read_only` input equal to or beneath a source destination | Repository mode rejects before network work; snapshot mode rejects after verified root-map resolution but before layers, outer writes, attachment, or harness execution; outer inputs remain valid | | Repository ref missing/ambiguous/moved | Fail repository resolution; no alternate ref/source | | Server cannot satisfy depth-2 fetch | Fail repository acquisition; no deepen/full clone/OCI fallback | | Submodule or LFS content | Fail repository validation; no helper execution | @@ -976,11 +1089,12 @@ private network details. | Generation publication | Concurrent identical Git or OCI identities singleflight to one acquisition and one publication; failed/partial mirror snapshots or generations never attach | | Git mirror/generation reuse | One operator-only bare shallow mirror exists per canonical repository identity; a second resolved-identity hit has zero clone/pack transfer/checkout/tree copy and directly attaches the same immutable generation | | OCI generation reuse | A second exact-digest request has zero registry request/extraction/tree copy/publication and directly attaches the same immutable generation | -| Read-only sharing | Every matching read-only session leases and binds the same verified generation bytes with zero copy; all write paths/aliases fail and session/runtime state stays isolated | -| Read-only state separation | Mutable harness/runtime assets live in the session control root; checkpoint excludes generation bytes, writes no source Git state, and continuation reattaches the exact generation before restoring session state | -| Editable isolation | Private copies share no mutable inode; one session's changes and checkpoint never alter generation or siblings | -| Working directory | Root and nested valid directories become process cwd; missing, file, traversal, and symlink escape fail before harness execution | -| Produced files | Existing API reports root and nested-repository changes once, excludes baselines/admin/control/secrets, survives continuation/restart, and respects bounds | +| Read-only sharing | Every matching session bind-mounts the same verified generation roots with zero copy; source writes and aliases fail while outer workspace paths remain private and writable | +| Mount boundary | Destinations are non-root directories, symlinks are never attachments, mount flags/identity verify, and root Git, Files, archive, hydrate, and cleanup never cross a source mount | +| Read-only checkpoint | The outer archive contains allowed inputs/outputs but no source bytes; continuation restores outer state, validates empty mountpoints, then reattaches the exact generation | +| Editable isolation | Private source copies share no mutable inode; one session's changes and source-root checkpoint never alter the generation or siblings | +| Working directory | Workspace root and valid directories inside or outside source roots become cwd; missing, file, traversal, and symlink escape fail before harness execution | +| Produced files | Existing API reports writable outer and editable nested-root changes once, excludes source baselines/admin/mount/secrets, survives continuation/restart, and respects bounds | | Continuation | Exact access, retention, generation, provenance, cwd, harness, conversation, and editable mutations restore without source traffic | | Cancellation | Git, OCI, build wait, copy, harness, checkpoint, and collection cancellation reap descendants and leave recoverable state | | Restart | Every generation/attachment/cleanup transition reconciles before readiness; exact attachments resume or fail closed | @@ -1018,17 +1132,18 @@ private network details. generation bytes with zero clone, source-byte transfer, materialization, or tree copy. Mirror internals are never session-visible. Editable sessions receive inode-independent private copies. -9. Existing hydrate, user/sandbox isolation, cancellation, TTL, restart, - deletion, files, artifacts, and cleanup own the complete session lifecycle. - Workspace-backed sessions relocate mutable harness/runtime assets to a - per-session control root outside source. Read-only requests reject workspace - input files and checkpoint no generation bytes; editable requests apply input - files only to the private copy and checkpoint only that copy plus - session-local state. No parallel workspace system remains. -10. Existing produced-file cursor semantics are adapted with runner-owned - external baselines for outer paths plus declared nested/multiple repository - and tree-only/history-bearing cursors, without writing bookkeeping Git state - into an immutable generation or adding a second Files API. +9. Existing hydrate, root Git, user/sandbox isolation, cancellation, TTL, + restart, deletion, files, artifacts, and cleanup own the complete session + lifecycle. The private outer workspace remains writable in both access modes. + Read-only inputs are allowed outside source destinations; source-targeting + inputs fail. Outer checkpoints exclude source destinations and source bytes; + editable roots use a separate private source-root checkpoint path. No + parallel workspace system remains. +10. Existing produced-file cursor semantics retain root Git for writable outer + paths, explicitly exclude every source destination, and add declared + editable nested/multiple repository and tree-only/history-bearing cursors + without traversing mounts, writing bookkeeping state into an immutable + generation, or adding a second Files API. 11. Direct Promptfoo E2E against the tested image proves Git and large OCI, multiple repositories, nested working directory, continuation, read-only enforcement, editable isolation, produced files, cancellation, restart, From 136022c74e8f53b9e24ea897497cbe5553a568ec Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 07:22:35 +1000 Subject: [PATCH 33/44] docs(architecture): define composable gateway sources --- .../0002-adopt-uhp-through-harnessrouter.md | 674 +++--- ...0837-feat-coding-execution-gateway-plan.md | 2025 ++++++++--------- 2 files changed, 1242 insertions(+), 1457 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 6c0b0bbd..e11eeef8 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -1,83 +1,67 @@ -# ADR 0002: Initialize HarnessRouter session workspaces from Git or OCI +# ADR 0002: Adopt UHP through AllAgents Gateway with composable workspace sources - Status: Accepted - Date: 2026-09-21 -- Updated: 2026-09-27 +- Updated: 2026-09-28 ## Context -Promptfoo needs a remote coding-harness endpoint that can prepare large repositories before the first turn, preserve their state across continuations, and expose exact source provenance through the Unified Harness Protocol (UHP). +Promptfoo needs a remote coding-harness endpoint that can prepare large source trees before the first turn, preserve session state across continuations, and expose exact source provenance through the Unified Harness Protocol (UHP). -HarnessRouter already owns the execution-plane lifecycle we need. Its runner materializes an existing per-session workspace either as a fresh workspace or by hydrating a checkpoint, and HarnessRouter already owns session identity, user and sandbox isolation, checkpoint transport, TTL, cancellation, files, artifacts, harness processes, and cleanup. The missing capability is narrower: on a first turn, initialize that runner-owned workspace from declared Git repositories or a verified OCI workspace snapshot before the harness starts. +Upstream [`HarnessRouter/harnessrouter`](https://github.com/HarnessRouter/harnessrouter) already provides the execution-plane lifecycle we need: session identity, user and sandbox isolation, a private per-session workspace, checkpoint transport, TTL, cancellation, Files and artifact surfaces, harness supervision, and cleanup. The missing capability is first-turn initialization of that workspace from one or more independently identified Git or OCI source trees. -The examined baseline is UHP [`2026-09-12`](https://github.com/HarnessRouter/harnessrouter/tree/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/versions/2026-09-12) at HarnessRouter commit [`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), released as [`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4). UHP reserves `metadata` for additive extensions, but stock HarnessRouter gives arbitrary metadata no workspace-initialization semantics. The extension is therefore downstream HarnessRouter behavior until UHP governance standardizes an equivalent contract. +The examined upstream baseline is UHP [`2026-09-12`](https://github.com/HarnessRouter/harnessrouter/tree/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/versions/2026-09-12) at HarnessRouter commit [`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), released as [`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4). UHP reserves `metadata` for additive extensions, but stock HarnessRouter gives arbitrary metadata no workspace-initialization semantics. -Large repositories are part of the minimum useful product, not a later optimization. Repository mode uses bounded shallow history to reduce Git history transfer, but that does not reduce a large working tree's transfer or extraction cost. OCI workspace snapshots and verified immutable-generation reuse solve that case and are therefore mandatory, release-blocking v1 capabilities alongside Git acquisition. +Large source trees are a minimum product requirement. Git depth `2` bounds history transfer but does not reduce working-tree transfer or extraction cost. OCI transport, immutable component caching, and cache reuse are therefore mandatory v1 capabilities rather than later optimizations. ## Decision -We will keep [`allagentsdev/harnessrouter`](https://github.com/allagentsdev/harnessrouter) as the existing fork of [`HarnessRouter/harnessrouter`](https://github.com/HarnessRouter/harnessrouter), preserving its name, fork relationship, and history. We will extend HarnessRouter's existing session workspace initialization path in that fork. We will not create a parallel checkout service, a second workspace abstraction, a replacement repository, or a new protocol. +We will ship a downstream product named **AllAgents Gateway** in [`allagentsdev/allagents-gateway`](https://github.com/allagentsdev/allagents-gateway), distributed as `ghcr.io/allagentsdev/allagents-gateway` and deployed under the service name `allagents-gateway`. AllAgents Gateway is derived from upstream HarnessRouter, but it is not named HarnessRouter and must not imply that its downstream workspace extension is standard upstream behavior. -UHP remains the only northbound protocol. A first turn may add `metadata.workspace`; requests that omit it retain stock HarnessRouter behavior unchanged. The extension initializes the same workspace that HarnessRouter would otherwise create fresh. Continuations omit the extension and use HarnessRouter's existing checkpoint hydration and session lifecycle to recover the exact bound attachment. +UHP remains the only northbound protocol. A first turn MAY add `metadata.workspace`. Requests without `metadata.workspace` MUST retain stock upstream UHP and HarnessRouter behavior. A continuation MUST omit the descriptor and recover the exact bound workspace state through the existing session and checkpoint lifecycle. -V1 supports both of these source modes: +A workspace descriptor contains a closed, ordered `sources` array of 1 to 128 independent entries. Every entry materializes exactly one source tree at one required, pairwise non-overlapping, non-root `destination`. The array MAY contain multiple Git entries, multiple OCI entries, or any mixture in any order. A monorepo is one source tree and therefore one entry. -- one or more Git repositories placed at declared, pairwise non-overlapping destinations; and -- an operator-catalogued OCI workspace snapshot selected by direct immutable digests. +An OCI image is a source-tree transport and cache unit. It is never the runtime workspace, a runtime or benchmark image, a verifier, a caller-selected container, or a bundle of multiple workspace roots. Its source manifest describes paths relative to its one tree; only the request assigns that tree a destination. There is no Git fallback for an OCI failure and no OCI fallback for a Git failure. -OCI is a source artifact for the workspace. It is not the HarnessRouter runtime image, a benchmark environment, a verifier, or a caller-selected container. There is no Git fallback for an OCI failure and no OCI fallback for a Git failure. +All sources support the same `read_only` and `editable` access modes. `read_only` exposes immutable cached component roots through namespace-confined read-only bind mounts. `editable` gives the session inode-independent private writable copies or reflinks. An editable OCI source is fully writable and is the expected mode for bug-fix evaluations. Cached component generations remain immutable in both modes. -Implementation in the fork starts immediately. Opening an upstream issue, writing a UHP proposal, or waiting for an upstream decision is not an implementation or release gate. After the downstream implementation and release evidence prove the capability, we may propose the generic contract upstream. Until accepted upstream, releases must label `metadata.workspace` as a documented `allagentsdev/harnessrouter` extension rather than standard UHP behavior. +Every component publishes a canonical baseline source manifest. Workspace-backed evaluation and produced-file collection compare final filesystem manifests with those baselines; Git state is not an evaluation authority. Root Git, nested Git, commits, index state, and Git rename detection MUST NOT determine correctness for a workspace-backed session. -The supported harnesses remain **Codex** and **OMP**. Provider traffic continues through the separately operated OAuth-to-OpenAI-compatible gateway using HarnessRouter's brokered credentials. This decision adds no AllAgents CLI integration and changes no local project workspace configuration. +Implementation and release do not wait for an upstream issue or UHP proposal. After downstream evidence proves the capability, maintainers MAY propose the generic contract upstream. Until upstream accepts an equivalent contract, releases MUST identify `metadata.workspace` as an AllAgents Gateway extension. -## Existing lifecycle and extension point +The supported harnesses remain **Codex** and **OMP**. Provider traffic continues through the separately operated OAuth-to-OpenAI-compatible gateway using brokered credentials. This decision adds no AllAgents CLI integration and changes no local project workspace configuration. + +## Upstream and downstream boundary + +AllAgents Gateway MUST preserve the upstream remote, the recorded fork point, upstream history needed for attribution, the upstream MIT license and copyright notices, and downstream modification notices. The repository MUST keep `https://github.com/HarnessRouter/harnessrouter` as its upstream source of record. Upstream changes are reviewed and selectively integrated; the downstream repository is not a blind mirror, and incompatible upstream changes are not accepted merely to track the latest commit. + +The downstream patch extends upstream fresh-workspace initialization, checkpoint hydration, source provenance, component storage, attachment, and produced-file bookkeeping. It does not create a second workspace service, replace UHP, or move session lifecycle ownership away from the inherited runner. + +When upstream accepts equivalent behavior, AllAgents Gateway SHOULD remove superseded downstream code and migrate cleanly. It MUST NOT retain compatibility aliases, deprecated descriptor shapes, or conflicting schema variants. -The fork must preserve the stock ownership boundary: +## Existing lifecycle and extension point -| Existing HarnessRouter responsibility | Extension responsibility | +| Inherited execution responsibility | AllAgents Gateway extension responsibility | |---|---| -| Allocate the private, writable session workspace and user/sandbox identity | Validate the first-turn descriptor and reserve declared non-root source destinations | -| Choose fresh materialization or checkpoint hydration | Resolve Git commits or exact OCI artifact identity | -| Transport and restore checkpoints | Build or reuse one verified immutable generation | -| Start the harness in the private session workspace | Bind each generation repository root read-only, or populate an inode-independent editable copy | -| Track sessions, TTL, cancellation, files, and cleanup | Persist source provenance and attachment identity with the session | -| Resume an existing writable session workspace | Restore outer state and reattach the exact prior protected generation without resolving source again | - -For a new workspace-backed session, source initialization runs after -authentication, request validation, idempotency, session resolution, and -allocation of HarnessRouter's fresh private session workspace, but before -provider work or harness execution. The workspace root remains private and -writable for both access modes. The initializer places source only at the -declared non-root repository destinations; it does not allocate another -workspace root or move lifecycle ownership out of HarnessRouter. - -For `read_only`, the fork creates empty destination directories in the session -workspace and attaches the corresponding immutable-generation repository roots -with per-session, namespace-confined, read-only bind mounts. `.harness`, HOME, -generated instructions, plugins, skills, MCP configuration, inputs, outputs, -scratch, conversation state, and other session data continue to use ordinary -writable paths in the private workspace, provided they are outside mounted -source destinations. There is no requirement to relocate all mutable state to a -separate control root. - -For continuation or recovery, workspace-aware checkpoint hydration unmounts -any stale attachment, restores the writable outer workspace without traversing -or restoring source destinations, validates empty non-link mountpoints, and -then reattaches the exact protected generation. The source initializer is not -invoked. Git refs are not resolved again, OCI is not fetched again, and a newer -generation is not substituted. - -Workspace initialization reuses HarnessRouter's cancellation, process -containment, user isolation, quota, TTL, checkpoint transport, deletion, and -crash-recovery machinery. Source staging and immutable generations are internal -runner resources subordinate to that lifecycle, not caller-visible workspaces. +| Allocate the private writable session workspace and user/sandbox identity | Validate the first-turn descriptor and reserve every declared destination | +| Choose fresh materialization or checkpoint hydration | Resolve and acquire each exact source component | +| Transport and restore checkpoints | Cache components independently and compose them atomically | +| Start the harness in the private session workspace | Attach immutable roots or create private editable roots before execution | +| Track sessions, TTL, cancellation, files, and cleanup | Persist component provenance, baseline manifests, composition identity, and attachment evidence | +| Resume an existing writable session workspace | Restore outer and editable-root state and recover exact component bindings without source resolution | + +For a new workspace-backed session, initialization runs after authentication, bounded request validation, idempotency handling, session resolution, and allocation of the private workspace, but before provider work or harness execution. The workspace root remains private and writable for both access modes. Sources occupy only their declared non-root destinations. + +Destination ownership MUST be decidable from the request before DNS, Git, registry, or other source access. The gateway MUST reject root destinations, overlapping destinations, reserved paths, and every known input, generated asset, restored outer path, or runner-owned path that collides at or below a destination. Ancestor directories MAY be created as empty scaffolding, but MUST NOT contain a file or link that prevents safe attachment. Inputs and assets outside source roots remain allowed. + +The outer workspace stores `.harness`, HOME, generated instructions, plugins, skills, MCP configuration, inputs, outputs, scratch, conversation state, and other session data outside source destinations. The extension does not allocate a second workspace root. ## Request contract A workspace-backed first turn uses the normal UHP `POST /v1/responses` endpoint. `metadata.workspace` is first-turn-only and has no nested schema version. -Repository mode: +This editable request composes two Git trees and one OCI tree. The OCI source is writable after attachment and can be modified by a bug-fix evaluation: ```json { @@ -88,109 +72,79 @@ Repository mode: "workspace": { "access": "editable", "retention": "session", - "source": { - "kind": "repositories", - "repositories": [ - { - "url": "https://github.com/acme/api.git", - "ref": "refs/heads/main", - "destination": "services/api" - }, - { - "url": "https://github.com/acme/web.git", - "destination": "services/web" - } - ] - }, + "sources": [ + { + "kind": "git", + "url": "https://github.com/acme/api.git", + "ref": "refs/heads/main", + "destination": "services/api" + }, + { + "kind": "oci", + "snapshot_name": "compiler-tree", + "image_manifest_digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111", + "source_manifest_digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222", + "destination": "vendor/compiler" + }, + { + "kind": "git", + "url": "https://github.com/acme/web.git", + "destination": "services/web" + } + ], "working_directory": "services/api/packages/server" } } } ``` -OCI snapshot mode: - -```json -{ - "model": "gpt-5.4", - "input": "Implement the requested change.", - "metadata": { - "harness_id": "chrn_…", - "workspace": { - "access": "read_only", - "source": { - "kind": "workspace_snapshot", - "snapshot_name": "large-monorepo", - "image_manifest_digest": "sha256:…", - "workspace_manifest_digest": "sha256:…" - }, - "working_directory": "repo/packages/compiler" - } - } -} -``` - `metadata.workspace` has exactly these fields: | Field | Required | Contract | |---|---:|---| -| `access` | yes | `read_only` or `editable`. | +| `access` | yes | `read_only` or `editable`; it applies identically to every entry. | | `retention` | no | `session` by default, or `persistent` when deployment policy authorizes it. | -| `source` | yes | Exactly one `repositories` or `workspace_snapshot` object as defined below. | +| `sources` | yes | Closed ordered array containing 1 to 128 Git or OCI entries. | | `working_directory` | no | Workspace-relative POSIX directory. Omission means the workspace root. | -A repository source has exactly `kind: "repositories"` and `repositories`. The array contains 1 to 128 entries. Each entry has exactly: +A Git entry has exactly these fields: | Field | Required | Contract | |---|---:|---| -| `url` | yes | Canonical public HTTPS Git URL. No userinfo, query, fragment, local path, or alternate transport. | -| `ref` | no | Advertised full ref or unambiguous branch/tag shorthand. Omission uses the advertised remote default. The resolved commit, not the ref spelling, is authoritative. | -| `destination` | yes | Non-root, workspace-relative POSIX directory. Destinations must be unique, pairwise non-overlapping, and outside reserved runner paths. | +| `kind` | yes | `git`. | +| `url` | yes | Canonical public HTTPS Git URL. Userinfo, query, fragment, local path, and alternate transports are forbidden. | +| `ref` | no | Advertised full ref or unambiguous branch/tag shorthand. Omission selects the advertised remote default. The resolved commit is authoritative. | +| `destination` | yes | Non-root workspace-relative POSIX directory, pairwise non-overlapping with every other destination. | -Depth is not caller-selectable. The repository acquisition policy defaults every entry to Git depth `2`; that effective depth is returned as provenance and participates in generation identity. +Git depth is not caller-selectable. Every Git entry uses effective depth `2`, which is returned in provenance and participates in component identity. -Every source composition occupies 1 to 128 declared repository roots. Whether -the roots come from repository request entries or a verified OCI workspace -manifest, their destinations are non-root, unique, and pairwise -non-overlapping. A monorepo therefore uses a destination such as `repo`, with a -working directory such as `repo/packages/compiler`; source at destination `.` -is invalid. Ancestor directories may be created as empty mount scaffolding, but -must contain no source files. - -A snapshot source has exactly: +An OCI entry has exactly these fields: | Field | Required | Contract | |---|---:|---| -| `kind` | yes | `workspace_snapshot`. | -| `snapshot_name` | yes | Selects an operator-owned catalog entry. It is not a registry repository or URL. | -| `image_manifest_digest` | yes | Direct `sha256:` digest of the accepted OCI image manifest. Mutable tags and indexes are not accepted as source identity. | -| `workspace_manifest_digest` | yes | `sha256:` digest of the canonical workspace manifest expected from that artifact. | +| `kind` | yes | `oci`. | +| `snapshot_name` | yes | Operator-catalog key; it is not a registry repository or URL. | +| `image_manifest_digest` | yes | Direct `sha256:` digest of the accepted OCI image manifest. Tags and indexes are not source identity. | +| `source_manifest_digest` | yes | `sha256:` digest of the canonical manifest for the image's one relative source tree. | +| `destination` | yes | Non-root workspace-relative POSIX directory, pairwise non-overlapping with every other destination. | + +The OCI catalog is operator-owned AllAgents Gateway configuration. It maps `snapshot_name` to a fixed registry repository, allowed media types, trust policy, and server-side credential reference. A caller never supplies a registry origin, repository, tag, header, redirect policy, or credential. -The verified canonical workspace manifest declares 1 to 128 repository roots -and their destinations under the same non-root and non-overlap rules. The -request cannot override those destinations. +The order of `sources` is semantic and retained in provenance and composition identity. It does not define overlay precedence: destinations cannot overlap, and one source cannot mask another. Unknown keys MUST be rejected at every level. No deprecated spelling or alternate shape is accepted. -The OCI catalog is HarnessRouter deployment configuration owned by the operator. It maps `snapshot_name` to a fixed registry repository, allowed media types, trust policy, and server-side registry credential reference. A caller never supplies a registry origin, repository, tag, header, or credential. +`working_directory` is interpreted only after all trees are attached or copied and allowed inputs are placed. It MUST resolve, without symlink escape, to a real directory inside the workspace. It MAY be inside a source tree or in the writable outer workspace. Absolute paths, empty components, `.` or `..` components, platform-specific separators, and reserved runner paths are invalid. -`working_directory` is interpreted only after the verified tree is attached or copied and input files are placed. It must resolve, without symlink escape, to a real directory inside the workspace; it may be within a declared source root or elsewhere in the writable outer workspace. Absolute paths, empty components, `.` or `..` components, platform-specific separators, and reserved runner paths are invalid. +The descriptor cannot contain credentials, headers, host paths, environment variables, commands, runtime images, Docker settings, materializer selection, resource limits, provider routes, or caller-selected TTLs. Request size, string length, array length, nesting, and validation work MUST be bounded before source access. -Unknown keys are rejected at every level. The descriptor cannot contain credentials, headers, host paths, environment variables, commands, runtime images, Docker settings, materializer selection, resource limits, provider routes, or caller-selected TTLs. Request size, string length, array length, nesting, and validation work are bounded before source access. -UHP input files remain normal session-workspace mutations in both access modes. -They are placed before the initial produced-file baseline is sealed. In -`read_only` mode they may target writable outer-workspace paths, but any input -whose path is a declared source destination or lies beneath one fails rather -than overlaying, copying up, or modifying the mounted generation. Generated -instructions and other HarnessRouter assets follow the same boundary: they may -be written outside source destinations. In `editable` mode inputs may overlay -the private source copies before baseline. +UHP input files and generated assets remain ordinary outer-workspace mutations. For either access mode, any such path at or below a declared destination MUST fail during pre-network validation rather than overlaying, copying up, or modifying a source tree. Paths outside all source destinations remain allowed and are included in the sealed outer-workspace baseline. -A first turn may omit `metadata.workspace`; stock behavior then remains unchanged. A session created without workspace metadata cannot add it later. Any reused session selected through `previous_response_id` or HarnessRouter's existing session-recovery metadata must omit `metadata.workspace`, even if the repeated object is byte-for-byte identical. +A first turn MAY omit `metadata.workspace`; stock behavior then remains unchanged. A session created without workspace metadata cannot add it later. Any reused session selected through `previous_response_id` or existing recovery metadata MUST omit `metadata.workspace`, even if a repeated descriptor would be byte-for-byte identical. ## Response and provenance contract -After attachment reaches `ready`, terminal events, response retrieval, replay, and later terminal failures expose the same committed `metadata.workspace` object. All public fields use snake case. +After attachment reaches `ready`, terminal events, response retrieval, replay, and later terminal failures expose the same committed `metadata.workspace` response object. All public fields use snake case. -Repository response: +The response for the mixed request above is shaped as follows: ```json { @@ -199,59 +153,38 @@ Repository response: "access": "editable", "retention": "session", "working_directory": "services/api/packages/server", - "effective_descriptor_digest": "sha256:…", - "generation_id": "sha256:…", - "workspace_manifest_digest": "sha256:…", - "provenance": { - "kind": "repositories", - "repositories": [ - { - "url": "https://github.com/acme/api.git", - "requested_ref": "refs/heads/main", - "resolved_commit": "0123456789abcdef0123456789abcdef01234567", - "destination": "services/api", - "depth": 2 - }, - { - "url": "https://github.com/acme/web.git", - "resolved_commit": "89abcdef0123456789abcdef0123456789abcdef", - "destination": "services/web", - "depth": 2 - } - ] - }, - "expires_at": "2026-09-28T00:00:00Z" - } - } -} -``` - -Snapshot response: - -```json -{ - "metadata": { - "workspace": { - "access": "read_only", - "retention": "session", - "working_directory": "repo/packages/compiler", - "effective_descriptor_digest": "sha256:…", - "generation_id": "sha256:…", - "workspace_manifest_digest": "sha256:…", - "provenance": { - "kind": "workspace_snapshot", - "snapshot_name": "large-monorepo", - "image_manifest_digest": "sha256:…", - "workspace_manifest_digest": "sha256:…", - "repositories": [ - { - "destination": "repo", - "resolved_commit": "0123456789abcdef0123456789abcdef01234567", - "object_set_digest": "sha256:…" - } - ] - }, - "expires_at": "2026-09-28T00:00:00Z" + "effective_descriptor_digest": "sha256:3333333333333333333333333333333333333333333333333333333333333333", + "composition_id": "sha256:4444444444444444444444444444444444444444444444444444444444444444", + "sources": [ + { + "kind": "git", + "url": "https://github.com/acme/api.git", + "requested_ref": "refs/heads/main", + "resolved_commit": "0123456789abcdef0123456789abcdef01234567", + "depth": 2, + "destination": "services/api", + "component_id": "sha256:5555555555555555555555555555555555555555555555555555555555555555", + "source_manifest_digest": "sha256:6666666666666666666666666666666666666666666666666666666666666666" + }, + { + "kind": "oci", + "snapshot_name": "compiler-tree", + "image_manifest_digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111", + "source_manifest_digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222", + "destination": "vendor/compiler", + "component_id": "sha256:7777777777777777777777777777777777777777777777777777777777777777" + }, + { + "kind": "git", + "url": "https://github.com/acme/web.git", + "resolved_commit": "89abcdef0123456789abcdef0123456789abcdef", + "depth": 2, + "destination": "services/web", + "component_id": "sha256:8888888888888888888888888888888888888888888888888888888888888888", + "source_manifest_digest": "sha256:9999999999999999999999999999999999999999999999999999999999999999" + } + ], + "expires_at": "2026-09-29T00:00:00Z" } } } @@ -262,279 +195,224 @@ The response fields are exact: | Field | Contract | |---|---| | `access` | Effective immutable access mode. | -| `retention` | Effective retention after authorization. It is never silently downgraded. | -| `working_directory` | Effective workspace-relative directory; the root is represented as `.`. | +| `retention` | Effective retention after authorization; it is never silently downgraded. | +| `working_directory` | Effective workspace-relative directory; the workspace root is represented as `.`. | | `effective_descriptor_digest` | Digest of the normalized first-turn descriptor, including applied defaults. | -| `generation_id` | Public content identifier for the verified immutable generation. It is not an authorization token or cache lookup key. | -| `workspace_manifest_digest` | Digest of the verified canonical source-visible manifest. | -| `provenance` | One of the exact source-mode objects below. | -| `expires_at` | Effective expiry timestamp for `session` retention, or `null` for authorized `persistent` retention. | - -Repository provenance contains `kind: "repositories"` and the request-order `repositories` array. Each entry contains normalized `url`, `destination`, exact `resolved_commit`, effective `depth`, and `requested_ref` only when the request supplied `ref`. Branch or tag movement does not change stored provenance for an existing session. - -Snapshot provenance contains `kind: "workspace_snapshot"`, `snapshot_name`, exact `image_manifest_digest`, exact `workspace_manifest_digest`, and a manifest-order `repositories` array. Each declared root contains its non-root `destination`; a history-bearing root additionally contains `resolved_commit` and `object_set_digest`. Tree-only roots contain neither. Snapshot provenance never exposes a registry origin, repository, credential reference, redirect, backing path, or physical mount path. - -Failures before attachment reaches `ready` omit workspace metadata. Failures after `ready` return the complete committed object. Internal generation keys, policy versions, authorization scope, mount paths, attachment IDs, pins, leases, reservations, and other sessions' state remain private. - -## Immutable generations and attachment behavior - -Both source modes produce the same versioned canonical workspace manifest. It -declares 1 to 128 non-root, pairwise non-overlapping repository roots and -enumerates the source-visible directories, regular files, and symbolic links -beneath them in logical path order with normalized mode, size, content digest, -or link target. HarnessRouter independently walks staging without following -links, recomputes the canonical bytes, and requires the supplied and computed -manifest digests to match before publication. - -The runner computes a private generation key from every input that can change -source bytes, filesystem semantics, or sharing authorization. For Git this -includes normalized repository URLs, resolved commits, destinations, shallow -depth (`2` in v1), acquisition-policy revision, and materializer contract -revision. For OCI it includes catalog identity, exact image-manifest and -workspace-manifest digests, trust-policy revision, and materializer contract -revision. Access, retention, working directory, harness, session, and physical -paths are excluded because they do not change the generation's immutable bytes. -A later depth or acquisition-policy change therefore cannot reuse an -incompatible Git generation. - -Identical normalized source identity publishes exactly one live verified -generation. The cache has two levels: one operator-only bare Git mirror/object -cache per canonical repository URL for bounded acquisition, followed by an -immutable multi-repository generation keyed by canonical URLs, exact resolved -commits, destinations, depth, acquisition-policy revision, and materializer -contract revision. Refreshes of one bare cache are serialized, and in-flight -acquisition and generation misses singleflight by generation key. Publication -is crash-safe: partial or failed staging is never attachable, and garbage -collection cannot remove a generation while a build waiter, provisional pin, -durable session reference, or attachment lease protects it. - -Attachment depends on `access`: - -- Every `read_only` Git request for the same normalized source identity, and - every equivalent OCI request, leases the same immutable generation. For each - declared root the runner creates an empty destination in the private writable - session workspace and bind-mounts the matching generation directory there - read-only. Matching sessions therefore see the same generation inodes and - cached source bytes while retaining separate outer-workspace state, user and - sandbox identity, HOME, scratch, logs, outputs, checkpoints, and response - state. -- Each bind mount is kernel-enforced read-only, namespace-confined, `nodev`, - and `nosuid`, while preserving repository execute bits required by tools. - Neither a writable alias nor a copy-up path is visible to the session, and - the generation backing store and writable acquisition cache remain - inaccessible. Any mount or remount failure fails closed. -- `editable` sessions reuse the same acquisition cache and pinned verified - generation as input, then receive an inode-independent, quota-bounded private - writable copy at each declared destination. No mutable inode may be shared - with the generation, Git object cache, or another session. - -Bind mounts are required rather than symlinks. A symlink neither enforces -read-only access nor confines traversal to the workspace; it exposes a backing -path, can escape workspace containment, and gives cwd and file tools surprising -path behavior. The mounted roots instead appear as ordinary directories at the -declared workspace-relative destinations. - -An attachment binds the normalized descriptor, exact generation key and epoch, -access, retention, working directory, selected harness, root-to-destination -attachment manifest, provenance, and workspace-manifest digest to the -HarnessRouter session. Continuation reuses that exact attachment. It never -re-resolves source, changes access or retention, selects another generation -with the same public ID, or rebuilds missing state. +| `composition_id` | Public identity of the ordered composition, derived from each exact `component_id` and its destination. It is not an authorization token or cache lookup key. | +| `sources` | Request-order closed provenance entries for every component. | +| `expires_at` | Effective expiry for `session` retention, or `null` for authorized `persistent` retention. | + +A Git provenance entry contains normalized `url`, exact `resolved_commit`, effective `depth`, `destination`, `component_id`, `source_manifest_digest`, and `requested_ref` only when the request supplied `ref`. Branch or tag movement does not change stored provenance for an existing session. + +An OCI provenance entry contains `snapshot_name`, exact `image_manifest_digest`, exact `source_manifest_digest`, `destination`, and `component_id`. It never exposes registry origin, repository, credential reference, redirect, backing path, layer-cache key, or physical mount path. OCI provenance has no implied Git commit and does not require Git metadata. + +Failures before attachment reaches `ready` omit workspace response metadata. Failures after `ready` return the complete committed object. Private generation keys, policy versions, authorization scope, mount paths, attachment IDs, pins, leases, reservations, and other sessions' state are never exposed. + +## Canonical manifests, identities, and caches + +Each verified component publishes a versioned canonical source manifest keyed by normalized relative POSIX path. Every entry records its type, regular-file content digest, executable or normalized mode semantics, and symbolic-link target where applicable. Canonical ordering, encoding, path normalization, directory treatment, and supported file types are part of the manifest contract. The gateway walks staging without following links, recomputes canonical bytes, and records their digest before publication. + +For Git, the gateway derives the canonical source manifest from the verified checkout. For OCI, the gateway MUST fetch and digest-verify `source_manifest_digest` before requesting any layer. That manifest describes only paths relative to the entry's single source root. The gateway validates all declared paths, types, sizes, modes, and links before layer requests, then requires extracted contents and recomputed canonical bytes to match it exactly before publication. + +A private component key includes every input that can change component bytes, filesystem semantics, or sharing authorization. A Git key includes canonical URL, exact resolved commit, depth `2`, acquisition-policy revision, and materializer-contract revision. An OCI key includes catalog identity, exact image-manifest digest, exact source-manifest digest, trust-policy revision, and materializer-contract revision. Destination, array position, access, retention, working directory, harness, session, and physical paths are excluded from component identity because they do not change component bytes. + +The component cache is independent for every entry. Git uses an operator-only bare mirror/object cache per canonical URL for bounded acquisition, followed by an immutable verified component generation. OCI uses verified blob/layer caches followed by an immutable verified component generation. Refreshes of a mutable acquisition cache are serialized. Concurrent misses singleflight by exact component key, not by the complete request. + +A composition does not merge source bytes into another cached generation. Its identity commits to the ordered sequence of exact component identities and destinations. The gateway MAY acquire or reuse independent components concurrently, but it MUST reserve every component and attach or copy the full composition atomically. All destinations become visible to the session or none do. A failure or cancellation of one entry rolls back provisional mounts, copies, leases, and pins for the complete composition. + +Publication is crash-safe. Partial staging is never attachable. Garbage collection MUST NOT remove an acquisition object or immutable component while a builder, waiter, provisional composition pin, durable session reference, or attachment lease protects it. A later acquisition, trust, or materializer policy revision cannot reuse an incompatible component. + +## Attachment behavior + +Attachment is uniform across Git and OCI: + +- For `read_only`, the gateway creates empty destination directories in the private writable outer workspace and bind-mounts each immutable component root there. Matching sessions MAY share cached source bytes and immutable inodes while retaining separate outer state, user and sandbox identity, HOME, scratch, logs, outputs, checkpoints, and response state. +- Every source bind mount MUST be kernel-enforced read-only, namespace-confined, `nodev`, and `nosuid`, while preserving required execute bits. The session MUST have no writable alias or copy-up path to the generation or acquisition cache. A mount or remount failure fails closed. +- For `editable`, the gateway materializes a quota-bounded, inode-independent private writable copy or reflink of every pinned component at its destination. No mutable inode may be shared with a cached generation, acquisition cache, or another session. Git and OCI entries receive identical write semantics. + +Bind mounts are required for shared read-only roots rather than symlinks. A symlink does not enforce read-only access, exposes a backing path, can escape workspace containment, and gives cwd and file tools surprising behavior. Mounted roots appear as ordinary directories at their declared destinations. + +An attachment binds the normalized descriptor, ordered exact component keys and epochs, composition identity, access, retention, working directory, selected harness, destination map, provenance, canonical source-manifest digests, and evaluation baselines to the session. Continuation MUST reuse that exact attachment. It never re-resolves a Git ref, repulls OCI, changes access or retention, or substitutes another component with the same public identifier. ## Git acquisition and integrity -Repository content is untrusted. Repository mode has a fixed v1 acquisition policy: `depth = 2` for every repository. This is Git's depth semantics—the requested tip plus bounded reachable history—not a guarantee of exactly two total commits when the tip is a merge. +Git content is untrusted. Every Git entry uses depth `2`: the selected tip plus bounded reachable history, not necessarily exactly two total commits when the tip is a merge. + +Git initialization MUST satisfy all of these requirements: -Git initialization must satisfy all of these requirements: +- Parse and authorize every URL before DNS or process launch. Accept only canonical public HTTPS origins. Revalidate every redirect; reject private, loopback, link-local, reserved, metadata-service, and otherwise disallowed addresses; pin approved addresses against DNS rebinding. +- Run Git without a shell, with a sanitized environment and isolated configuration. Disable interactive credentials, inherited proxies, hooks, checkout filters, Git LFS hydration, submodule recursion, alternates, and non-HTTPS helpers and protocols. +- Resolve only an advertised branch, tag, or remote default to one exact commit. Fetch that selection with `--depth=2`, then verify the fetched tip equals the resolved commit. Caller text MUST NOT be substituted into fetch or checkout commands. If the remote cannot satisfy the bounded shallow fetch, fail rather than deepen, unshallow, or clone fully. +- Keep the bare shallow acquisition cache mutable, operator-only, and inaccessible to harness users, credentials, workspace writes, and attachments. Publish only the selected bounded object graph into the immutable component; unrelated cached refs and objects MUST NOT be exposed. +- The published component MAY preserve normalized `.git/shallow` metadata and recent history for agent convenience. Remove credential-bearing remotes, hooks, worktree links, alternates, replace and graft state, locks, reflogs, transient fetch state, and unsafe configuration. Verify detached `HEAD`, shallow boundary, index-to-tree equality, included-object integrity, and source content against the resolved commit and canonical source manifest. +- Enforce finite time, transferred-byte, expanded-byte, inode, file-count, process, descendant, and concurrency limits independently for each Git entry and for the complete request. +- Reject undeclared output, traversal, unsafe links, reserved-path collisions, and any path that escapes its owning source root. A failed Git entry fails the complete composition; no partial set is attached. +- Record normalized URL, optional requested ref, exact resolved commit, effective depth, component identity, source-manifest digest, and destination. -- Parse and authorize every URL before DNS or process launch. Only canonical public HTTPS origins are accepted. Revalidate every redirect; reject private, loopback, link-local, reserved, metadata-service, and otherwise disallowed addresses; pin approved addresses against DNS rebinding. -- Run Git without a shell, with a sanitized environment and isolated configuration. Disable interactive credentials, inherited proxy configuration, hooks, checkout filters, Git LFS hydration, submodule recursion, alternates, and non-HTTPS helpers and protocols. -- Resolve only an advertised branch, advertised tag, or advertised remote default to one exact commit before acquisition. Fetch that selected ref with `--depth=2`, then verify the fetched tip equals the previously resolved commit. Fetch and checkout commands must not substitute caller text for the resolved selection. If the remote cannot satisfy the bounded shallow fetch, fail; never silently deepen, unshallow, or fall back to a full clone. -- Keep one server-owned bare shallow Git mirror/object cache per canonical repository URL behind the generation builder so repeated acquisition can reuse fetched objects. The cache is mutable operator-only runner infrastructure, never a session attachment. Refreshes are serialized, and it is inaccessible to harness users, credentials, hooks, and workspace writes. Publication selects only the resolved ref's bounded object graph into the immutable generation; unrelated cached refs and objects are never exposed. The generation contains its own normalized shallow repository state, so later cache updates cannot change it. -- Preserve the generation's normalized `.git/shallow` metadata and the acquired recent history so offline commands such as `git log` and recent diffs work within the fetched boundary. Remove credential-bearing remotes, hooks, worktree links, alternates, replace and graft state, locks, reflogs, and transient fetch state. Verify detached `HEAD`, shallow boundary, index-to-tree equality, included object integrity, and source-visible content against the recorded commit and acquisition policy. -- Apply finite time, transferred-byte, inode, file-count, process, descendant, and concurrency limits across all repositories. Shallow depth reduces history transfer; it does not solve large working-tree transfer or materialization, which is why OCI snapshots remain mandatory. -- Stage every repository beneath its declared non-root destination and reject overlaps, undeclared files, source files in destination ancestors, cross-root links, traversal, or reserved-path collisions. Publish the complete multi-repository generation atomically or publish nothing. -- Record normalized URL, optional requested ref, exact resolved commit, destination, and effective depth for every repository. A failure in any repository fails the whole source; partial repository sets are never attached. +Git metadata is acquisition evidence and MAY be useful to an agent. It is never the evaluator's diff engine. Its absence or mutation cannot change the baseline or final-tree comparison used to judge a workspace-backed evaluation. ## OCI acquisition and integrity -OCI snapshot support is mandatory in v1 and release-blocking. Snapshot acquisition must satisfy all of these requirements: +OCI support is mandatory and release-blocking. Each OCI entry transports exactly one relative source tree and MUST satisfy all of these requirements: -- Resolve `snapshot_name` only through the operator-owned catalog. Fetch only the direct image manifest named by `image_manifest_digest`; do not follow mutable tags, accept an index in its place, change registry authority on redirect, or expose catalog registry details to the caller. -- Verify the image manifest digest, media type, descriptor sizes, every selected layer digest and size, the catalog-defined workspace-manifest media type, the workspace-manifest blob digest, and the recomputed source-visible manifest digest. -- Fetch and verify the canonical workspace manifest before requesting any layer. Validate its effective repository-root map and every input, generated-asset, reserved-path, and restored-outer-state collision before any layer request or outer workspace content write. -- Enforce the v1 envelope before and during extraction: at most 64 distributable tar/gzip/zstd layers; a 4 MiB image manifest; a 128 MiB workspace manifest with at most 128 repository roots; 8 GiB total compressed layer bytes; 32 GiB expanded source bytes; 500,000 entries; 4 GiB per regular file; paths of at most 4096 UTF-8 bytes and 128 components; and 1 MiB per PAX or extended header. For each layer and for the aggregate artifact, expanded bytes divided by `max(compressed_bytes, 1)` must not exceed `100`. Cumulative-size and expansion-ratio checks apply while streaming, not only after extraction. Operators may configure lower limits, never higher ones without a contract revision. +- Resolve `snapshot_name` only through the operator catalog. Fetch only the direct image manifest named by `image_manifest_digest`; do not follow a mutable tag, accept an index in its place, change registry authority on redirect, or expose registry details to the caller. +- Verify the image-manifest digest, media type, descriptor sizes, every selected layer digest and size, the catalog-defined source-manifest media type, the source-manifest blob digest, and the recomputed canonical source-manifest digest. +- Fetch and verify the canonical source manifest before any layer. Validate all relative paths and types plus every request-known input, asset, reserved-path, and restored-outer-state collision before any layer request or outer-workspace content write. - Apply layers in order with OCI whiteout and opaque-directory semantics. Whiteouts are metadata operations, not source-visible files. Reject malformed, duplicate, conflicting, or out-of-root whiteouts. -- Before writing each entry, validate its normalized relative path, type, declared size, mode, and link target. Every hard link or symbolic link must remain within its owning declared repository root; links into another source root or the writable outer workspace fail closed. Reject absolute paths, traversal, NULs, escaping links, devices, sockets, FIFOs, sparse-file tricks, unsupported types, and entries that collide with runner-owned paths. Extraction uses rooted, no-follow operations and cannot write through a previously extracted link. -- Require the canonical workspace manifest to declare every source-visible entry and 1 to 128 non-root, pairwise non-overlapping repository roots. Undeclared output, missing entries, type changes, digest mismatches, source at destination `.`, source files in destination ancestors, and paths outside declared roots fail closed. - -A workspace snapshot may contain normalized offline Git history for any declared repository root. A history-bearing root records `resolved_commit` and `object_set_digest`; a tree-only root records neither. History-bearing roots must have detached `HEAD` at the recorded commit, an index equal to that tree, the complete required object closure matching `object_set_digest`, and no dirty, staged, untracked, unreachable, or extra source-visible state. They must contain no remote, credential helper, config include, hook, worktree link, alternate, shallow, replace, graft, reflog, transient fetch state, or credential-bearing configuration. OCI restore never contacts Git, and failure of snapshot or embedded Git verification never falls back to cloning. - -The direct image digest is part of OCI identity even when two artifacts have the same source-visible tree. Repacking layers or offline Git objects creates a different generation identity. Semantic verification proves an artifact's contents; it does not silently deduplicate distinct artifacts. - -## Produced files, checkpoints, and continuation - -HarnessRouter's existing Files API and produced-file collection remain the only -public file surface. The fork adapts the stock root-workspace Git/bookkeeping -path rather than introducing a second Files API or parallel change tracker. The -outer session workspace stays writable, but root bookkeeping explicitly -excludes every declared source destination and must not traverse its mount. -Bookkeeping state remains in normal runner-owned session paths; it never -initializes or mutates a `.git` directory inside an immutable generation. - -Bookkeeping understands the declared repository roots and source mode: - -- Writable outer-workspace paths retain stock-like root bookkeeping, excluding - all source destinations. -- Nested repository collectors compare editable Git roots with their recorded - resolved commits, history-bearing editable snapshot roots with their verified - commit and object-set records, and tree-only editable roots with the canonical - workspace manifest. -- Read-only roots cannot change and are never traversed by root Git, - produced-file scans, cleanup walks, or archive creation. -- Git control data, generation metadata, credentials, attachment evidence, and - runner-owned checkpoint state are never reported as produced files. - -The adaptation represents additions, modifications, deletions, renames, and -mode changes in the writable outer workspace and across multiple editable -nested repository roots without assuming one root `.git` directory. A -`read_only` attachment cannot produce source mutations, but files created -outside mounted roots are collected normally. Editable produced-file state and -nested repository state remain covered by the session's private quota and -lifecycle. - -Checkpoint behavior is access-specific. A `read_only` checkpoint archives the -writable outer workspace while explicitly excluding every mount destination -and all source bytes. It stores the exact generation key and epoch, root -attachment manifest, workspace manifest, durable reference, and mount evidence -separately. Archive and file operations never follow or cross a source mount. -An `editable` checkpoint includes the inode-independent private source copies -and nested repository state, but never the bare acquisition cache or immutable -generation backing store. - -A continuation supplies the existing predecessor/session reference and omits -`metadata.workspace`. HarnessRouter first unmounts any existing source -attachments, hydrates the writable outer state, validates that every declared -destination is an empty real directory rather than a link, and only then -reattaches the exact protected generation read-only. It verifies the stored -attachment evidence and starts the stored harness in the stored working -directory. Editable hydration restores its private copies instead. Edits from -prior editable turns and files written outside read-only roots remain visible. -A changed ref, source digest, working directory, access, retention, or harness -requires a new session. - -Attachments are unmounted before hydration, deletion, workspace cleanup, or -retrying cleanup. Tar, Files API traversal, recursive cleanup, and root Git -operations must stay on the writable outer filesystem and never cross a mount. -If unmount, attachment, generation, private-copy, checkpoint, or provenance -evidence is expired, missing, busy, corrupt, or inconsistent, continuation or -cleanup fails closed and the allocation remains accounted for. Continuation -does not clone, repull, restore from OCI again, substitute another generation, -or silently start a fresh session. +- Before writing an entry, validate its normalized relative path, type, declared size, mode, and link target. Every hard link and symbolic link MUST remain within that OCI entry's one source root. Links into another source, the writable outer workspace, or an absolute path fail closed. Extraction uses rooted no-follow operations and cannot write through a previously extracted link. +- Reject traversal, NULs, devices, sockets, FIFOs, sparse-file tricks, unsupported types, undeclared entries, missing entries, type changes, digest mismatches, and writes outside the one declared tree. +- Publish only after the extracted tree exactly matches the canonical source manifest. Failure MUST NOT fall back to Git or another catalog entry. + +The v1 envelope permits at most 64 distributable tar/gzip/zstd layers, a 4 MiB image manifest, a 128 MiB source manifest, 8 GiB compressed layer bytes, 32 GiB expanded source bytes, 500,000 entries, 4 GiB per regular file, paths of at most 4096 UTF-8 bytes and 128 components, and 1 MiB per PAX or extended header for one OCI entry. Expanded bytes divided by `max(compressed_bytes, 1)` MUST NOT exceed `100` for each layer and for the entry. Checks apply while streaming. + +Finite time, bytes, entries, inodes, layers, processes, descendants, and concurrency are also bounded across the complete request, including mixed Git and OCI compositions. Per-entry success does not bypass request-aggregate limits. Operators MAY configure lower limits and MUST NOT raise contractual maxima without a contract revision. + +An OCI tree need not contain `.git` or any other Git metadata. If such metadata is present, it is untrusted source content and optional agent convenience only; it conveys no implicit commit identity and is not used for evaluation diffs. Repacking layers produces a different OCI component identity even if canonical source-tree bytes are equal, because the exact image-manifest digest remains part of identity. + +## Produced files and evaluation diffs + +The inherited Files API and produced-file collection remain the only public file surface. AllAgents Gateway adapts that surface rather than adding another change API. + +Before harness execution, the gateway seals: + +1. the published canonical baseline source manifest for every attached component; and +2. a canonical baseline manifest of the writable outer workspace after allowed inputs and assets are placed, excluding every source destination and runner-owned state. + +At collection time, the gateway computes final manifests for every editable source root and the writable outer workspace using the same canonical rules. A read-only source root remains equal to its pinned immutable baseline by construction; collection MUST NOT traverse through its mount into backing storage. The gateway compares baseline and final maps by normalized path and reports additions, modifications, deletions, executable or normalized-mode changes, symbolic-link target changes, and binary content changes. Content digests, rather than text decoding, determine equality. + +Rename inference is OPTIONAL presentation metadata. Final-tree equality is authoritative: two sessions with the same included final path/type/mode/link/content maps are equivalent even if one report infers a rename and another reports delete-plus-add. + +For workspace-backed sessions, the manifest comparison is the only correctness source. The collector MUST NOT consult root Git, nested Git, commits, index state, worktree status, staged state, or Git rename detection. It MUST NOT initialize or mutate a root `.git` repository. Stock root-Git behavior MAY remain only on requests that omit `metadata.workspace`. + +The evaluation projection excludes `.git` trees, runner-owned state, credentials, caches, acquisition and generation metadata, attachment evidence, checkpoint metadata, and other internal control paths from reported changes. Those exclusions apply equally to Git and OCI roots and to outer-workspace bookkeeping. They do not weaken acquisition integrity checks. + +A `read_only` attachment cannot produce source mutations, but files created or changed outside mounted roots are collected through the outer manifest. Editable Git and editable OCI changes are collected identically. All final state remains subject to the session's private quota and lifecycle. + +## Checkpoints and continuation + +Checkpointing is mount-aware and preserves outer state and editable roots separately. + +A `read_only` checkpoint archives the writable outer workspace while excluding every source destination and all source bytes. It stores exact ordered component keys and epochs, composition identity, destination map, canonical baseline digests, durable references, and mount evidence as protected checkpoint metadata. Archive, Files, cleanup, and manifest operations MUST NOT follow or cross a source mount. + +An `editable` checkpoint stores the writable outer workspace and each inode-independent private source root as separate logical checkpoint members, together with their canonical baselines and attachment evidence. It never stores the bare Git cache, OCI blob cache, immutable component backing store, credentials, or transient acquisition state. Separation prevents a restored outer archive from overwriting, omitting, or aliasing an editable root. + +A continuation supplies the existing predecessor or session reference and omits `metadata.workspace`. The gateway first removes stale attachments, restores the writable outer state, and validates all destination mountpoints. For `read_only`, it then reattaches the exact protected components atomically. For `editable`, it restores each preserved private writable root at its exact destination. It verifies stored provenance, baselines, composition identity, and attachment evidence before starting the stored harness in the stored working directory. + +Edits from prior editable turns and files written outside read-only roots remain visible. A changed ref, source digest, destination, source order, working directory, access, retention, or harness requires a new session. Continuation never clones, repulls, rematerializes from OCI, substitutes a newer component, or silently starts fresh. + +Attachments are unmounted before hydration, deletion, workspace cleanup, or cleanup retry. If an unmount, component, private copy, checkpoint member, baseline, provenance record, pin, or lease is missing, busy, corrupt, expired, or inconsistent, continuation or cleanup fails closed and the allocation remains accounted for. ## Provider authentication and harness configuration -Provider authentication remains proxy-only. Each deployment configures one external OAuth-to-OpenAI-compatible gateway base URL and API key server-side. HarnessRouter represents that endpoint with two protocol-specific logical connections using the same secret: Responses for Codex and OpenAI Chat Completions for OMP. Each harness policy contains exactly its matching connection, with no fallback. The UHP caller cannot supply or override the endpoint, key, transport, or route. +Provider authentication remains proxy-only. Each deployment configures one external OAuth-to-OpenAI-compatible gateway base URL and API key server-side. AllAgents Gateway represents that endpoint with two protocol-specific logical connections using the same secret: Responses for Codex and OpenAI Chat Completions for OMP. Each harness policy contains exactly its matching connection, with no fallback. A UHP caller cannot supply or override the endpoint, key, transport, or route. -The external OAuth gateway owns login, token persistence, refresh, repair, provider API compatibility, and provider authorization. HarnessRouter does not implement provider login, import local credentials, mount developer credential files, or coordinate provider token refresh. HarnessRouter's caller API key authenticates the UHP caller only and is never reused as a provider credential. +The external OAuth gateway owns login, token persistence, refresh, repair, provider API compatibility, and provider authorization. AllAgents Gateway does not implement provider login, import local credentials, mount developer credential files, or coordinate provider token refresh. Its caller API key authenticates the UHP caller only and is never reused as a provider credential. -The deployment uses HarnessRouter's brokered sandbox mode, not owner-trust credential pass-through. The broker exchanges the long-lived external-gateway key server-side and gives each harness only a short-lived, session-scoped credential plus the loopback broker URL. The long-lived key never enters the harness process environment, session workspace, checkpoint, file output, artifact, log, response, or source provenance. +The deployment uses inherited brokered sandbox mode, not owner-trust credential pass-through. The broker exchanges the long-lived external-gateway key server-side and gives each harness only a short-lived, session-scoped credential plus a loopback broker URL. The long-lived key MUST NOT enter the harness environment, session workspace, checkpoint, file output, artifact, log, response, or source provenance. -Codex uses the gateway's OpenAI Responses-compatible surface. OMP uses the same gateway's OpenAI Chat Completions-compatible surface. V1 custom harnesses are Codex and ordinary session-local OMP only. OMP starts from container/session configuration; it does not import AllAgents profiles, host profiles, or developer state. +Codex uses the gateway's OpenAI Responses-compatible surface. OMP uses its OpenAI Chat Completions-compatible surface. V1 custom harnesses are Codex and ordinary session-local OMP only. OMP starts from container and session configuration; it does not import AllAgents profiles, host profiles, or developer state. ## Failure behavior -The implementation fails closed without changing source mode, source identity, access, retention, harness, model route, or provider protocol as a recovery shortcut. +The implementation fails closed without changing source identity, source kind, access, retention, harness, model route, or provider protocol as a recovery shortcut. -| Failure | Behavior | +| Failure | Required behavior | |---|---| -| Malformed, oversized, too-deep, or unknown workspace field | Reject before source access and without mutating an existing session. | -| Workspace metadata on a continuation or reused session | Reject without changing the attachment, checkpoint, or TTL. | -| Input file targets a declared `read_only` source destination | Reject without overlay, copy-up, or source mutation; inputs outside mounted roots remain allowed. | +| Malformed, oversized, too-deep, unknown, or deprecated workspace field | Reject before source access and without mutating an existing session. | +| Workspace metadata on a continuation or reused session | Reject without changing attachment, checkpoint, or TTL. | +| Invalid, root, overlapping, or reserved destination | Reject the complete descriptor before network access. | +| Input, generated asset, or restored outer path collides at or below a destination | Reject before source access or outer-workspace mutation. | | Unauthorized `persistent` retention | Fail before source resolution; do not downgrade to `session`. | -| Invalid Git URL, ref, destination, network target, redirect, or feature | Fail the response, cancel bounded source work, and remove staging; do not start a harness or provider call. | -| Any repository in a multi-repository source fails | Fail the complete source; never attach a partial set. | -| Unknown snapshot, digest mismatch, mutable reference, disallowed registry transition, or unsupported media type | Fail OCI acquisition; do not try Git or another snapshot. | -| Layer limit, extraction violation, whiteout error, unsafe path/link/type, or manifest mismatch | Terminate extraction, quarantine or remove staging, and publish nothing. | +| Invalid Git URL, ref, network target, redirect, or feature | Fail the complete composition, cancel bounded work, and remove staging. | +| Unknown OCI catalog entry, digest mismatch, mutable reference, disallowed registry transition, or unsupported media type | Fail that component and therefore the complete composition; do not try Git or another artifact. | +| Layer limit, extraction violation, whiteout error, unsafe path/link/type, or source-manifest mismatch | Terminate extraction, quarantine or remove staging, and publish nothing. | +| Any source fails or aggregate capacity is exceeded | Roll back the complete composition; never attach a partial set. | | Resource or concurrency capacity unavailable | Return a coded retryable capacity failure before unbounded acquisition. | | Materializer timeout, crash, cancellation, or live descendant | Terminate and reap the complete process tree before cleanup and terminal acknowledgement. | -| Working directory missing, not a directory, or escaping by traversal/link | Fail before attachment and harness execution. | -| Crash during publication or attachment commit | Recover to either a complete verified attachment or no attachment; never expose partial staging. | -| Mountpoint is non-empty, is a link, or a bind/remount operation fails | Fail closed before harness execution; never expose a writable source alias or partial attachment. | -| Missing or corrupt bound state on continuation | Fail as non-resumable; never rematerialize or substitute. | +| Working directory missing, not a directory, or escaping through traversal or a link | Fail before attachment commit and harness execution. | +| Crash during component publication or composition commit | Recover to either a complete verified composition or no attachment. | +| Mountpoint non-empty or linked, or bind/remount/copy failure | Fail closed before harness execution; expose no partial or writable alias. | +| Missing or corrupt bound state on continuation | Fail as non-resumable; never reacquire or substitute. | | External provider authentication or execution failure | Return the normalized UHP failure; do not switch endpoint, protocol, credential, or harness. | -| Cleanup or unmount failure | Quarantine and continue accounting for the allocation; never traverse the mount, and retry the same idempotent unmount-then-cleanup path. | - -Promptfoo treats every non-success as an evaluation error. It does not convert a workspace failure to an empty success, source fallback, or implicit retry. +| Cleanup or unmount failure | Quarantine and continue accounting for the allocation; retry the same idempotent unmount-then-cleanup path. | -## Fork, deployment, and release boundary +Promptfoo treats every non-success as an evaluation error. It does not convert a workspace failure to an empty success, a source fallback, or an implicit retry. -`allagentsdev/harnessrouter` remains the implementation and distribution repository and remains a GitHub fork of `HarnessRouter/harnessrouter`. The downstream patch extends the fresh-workspace initialization point, source provenance, generation storage, and existing produced-file bookkeeping while leaving UHP requests without `metadata.workspace` on stock paths. +## Distribution and release boundary -The source initializers and generation manager ship inside the HarnessRouter image; they are not another network service. The supported deployment remains one HarnessRouter container with durable `/data`, loopback binding by default, and `HR_BACKENDS=codex,omp`. +`allagentsdev/allagents-gateway` is the implementation and distribution repository. The source initializer, component caches, composition manager, manifest collector, and checkpoint adaptations ship in the AllAgents Gateway image; they are not another network service. -The public image remains `ghcr.io/allagentsdev/harnessrouter`. Tags identify the upstream HarnessRouter baseline plus the downstream revision; deployments pin the resulting image manifest digest. Releases produce standard SBOM and build-provenance attestations and run the upstream UHP conformance suite against the built image. +The supported deployment is one `allagents-gateway` service using `ghcr.io/allagentsdev/allagents-gateway`, durable `/data`, loopback binding by default, and the inherited Codex and OMP backend configuration. Deployments pin an exact image-manifest digest. Release tags identify both the upstream HarnessRouter baseline and downstream revision. Releases produce SBOM and build-provenance attestations and run the upstream UHP conformance suite against the built image. -Release verification must exercise both Codex and OMP through the configured external provider gateway. In addition, v1 cannot release without: +A v1 release is complete only when all of the following are demonstrated: -1. an end-to-end OCI test using at least 2 GiB of expanded source bytes and 100,000 source-visible filesystem entries that fetches by direct manifest digest, applies layers and whiteouts, verifies the workspace manifest and any offline Git history, starts a harness in `working_directory`, and continues the same session successfully; and -2. a cache-reuse proof showing that identical Git and OCI source identities publish once, concurrent cache misses singleflight, every concurrent or later `read_only` task/session bind-mounts the same immutable generation inodes at its declared destinations without cloning or copying, the outer workspace remains private and writable, input and produced files outside source roots work normally, source-targeting inputs fail, mount failures fail closed, read-only checkpoints and Files/root-Git/cleanup traversal contain no generation bytes or source Git writes, `editable` sessions derive inode-independent copies from the pinned generation, unrelated bare-cache refs are not exposed, and continuation restores outer state before reattaching the exact generation without reacquisition. +1. Codex and OMP each complete and continue a session through the configured external provider gateway. +2. A mixed composition with multiple Git and multiple OCI entries resolves independent provenance, preserves request order, rejects every overlap before network access, and becomes visible atomically. +3. An OCI entry with at least 2 GiB of expanded source and 100,000 source-visible entries is fetched by direct image digest; its source manifest is verified before layers, layers and whiteouts are applied, the one relative tree is verified, and a harness starts inside it. +4. Identical Git and OCI component identities publish once and singleflight independently, while compositions reuse those components without constructing a request-wide cached tree. +5. Concurrent and later `read_only` sessions bind the same protected component inodes at their destinations, keep writable outer workspaces private, and never include mounted bytes in checkpoint or cleanup archives. +6. `editable` Git and OCI sessions receive inode-independent writable roots. An editable OCI bug-fix run modifies source, continues from a checkpoint, and reports the exact final filesystem change without Git metadata. +7. Manifest comparison proves add, modify, delete, executable/mode, symlink-target, and binary changes across multiple editable roots and the outer workspace. Altering Git commits, indexes, staged state, or rename detection cannot change the evaluated final-tree result. +8. Input and asset paths outside roots work normally; collisions at or below a destination fail before network access. OCI links escaping their owning root and mount failures fail closed. +9. Cancellation, failure, crash recovery, and aggregate-limit tests prove that no partial composition, stale pin, source-kind fallback, or unaccounted allocation becomes visible. +10. A request without `metadata.workspace` passes stock UHP behavior and conformance without invoking workspace acquisition or manifest-based workspace evaluation. -These are release gates, not deferred performance tests. Git and OCI failure-path coverage must also prove that no partial generation or source-mode fallback becomes visible. - -Downstream implementation and release do not wait on upstream work. Once downstream evidence exists, maintainers may propose the generic capability upstream. If upstream accepts an equivalent contract, the fork should remove the superseded patch and migrate cleanly; it must not retain conflicting aliases or claim downstream conformance before acceptance. +These are release gates, not deferred performance tests. ## Alternatives rejected | Alternative | Why rejected | |---|---| -| Build a new execution gateway | Duplicates HarnessRouter's UHP, sessions, workspace lifecycle, streaming, cancellation, files, artifacts, and harness supervision. | -| Put a workspace service in front of HarnessRouter | Splits source and session ownership and cannot safely participate in checkpoint hydration, continuation, or produced-file bookkeeping. | -| Create a second checkout root inside each session | Competes with the runner-owned workspace, duplicates cleanup and quota state, and makes files and checkpoints ambiguous. | -| Attach shared source with symlinks | Symlinks do not enforce read-only access, expose backing paths, can escape workspace containment, and behave inconsistently for cwd and file tools; namespace-confined read-only bind mounts present ordinary destination directories and fail closed. | -| Ship Git first and defer OCI | Fails the minimum large-repository use case and makes release viability depend on repeated acquisition. | +| Keep the downstream product and image named HarnessRouter | Obscures which behavior is upstream and falsely suggests downstream extension conformance. | +| Build a new execution gateway from scratch | Duplicates UHP, sessions, workspace lifecycle, streaming, cancellation, files, artifacts, and harness supervision already inherited from HarnessRouter. | +| Put a workspace service in front of the gateway | Splits source and session ownership and cannot safely participate in checkpoint hydration, continuation, or produced-file collection. | +| Keep a mutually exclusive single source object | Prevents first-class mixed Git and OCI workspaces and couples unrelated source lifecycles and cache misses. | +| Let one OCI image declare multiple destination roots | Hides destination ownership until after network access, makes one artifact a multi-root workspace bundle, and prevents independent component caching. | +| Build one cached generation for the complete composition | Rebuilds and duplicates unchanged components whenever one entry or destination changes and singleflights at the wrong granularity. | +| Put source at the workspace root | Collides with runner-owned state, inputs, checkpoints, and the private writable outer workspace. | +| Attach shared source with symlinks | Does not enforce read-only access, exposes backing paths, permits containment escape, and gives cwd and file tools surprising behavior. | +| Share cached inodes with editable sessions | Lets one session mutate another session or the immutable cache and makes checkpoints non-isolating. | +| Use Git as the evaluation diff engine | Fails for OCI trees without Git, mishandles multiple roots and outer files, and lets mutable commits or index state redefine correctness. | +| Ship Git first and defer OCI | Fails the minimum large-source use case and makes release viability depend on repeated working-tree acquisition. | | Treat OCI as a runtime or benchmark image | Mixes source provenance with tools, services, verifier assumptions, and execution policy. | -| Wait for upstream before implementation | Makes delivery depend on a project we do not maintain and delays the evidence needed for a useful upstream proposal. | -| Add a general plugin or materializer framework | V1 has two explicit source modes and no demonstrated need for caller-selectable plugins. | -| Put source instructions in the prompt or a model tool | Makes acquisition model-dependent, non-deterministic, too late to set the initial directory, and unsafe for provenance. | -| Upload every source file through UHP | Pushes acquisition to callers and loses authoritative Git and OCI identity, history, links, and modes. | -| Make the AllAgents CLI the remote control plane | Couples local developer configuration to an independently deployed HarnessRouter service. | -| Add a new Files API for initialized workspaces | Duplicates HarnessRouter behavior instead of adapting its existing produced-file bookkeeping. | +| Wait for upstream before implementation | Makes delivery depend on a project we do not maintain and delays evidence needed for an upstream proposal. | +| Add a general plugin or materializer framework | V1 has two explicit entry kinds and no demonstrated need for caller-selectable plugins. | +| Put acquisition instructions in the prompt or a model tool | Makes acquisition model-dependent, non-deterministic, too late to set the initial directory, and unsafe for provenance. | +| Upload every source file through UHP | Pushes acquisition to callers and loses authoritative Git and OCI identity, links, modes, and cache reuse. | +| Make the AllAgents CLI the remote control plane | Couples local developer configuration to an independently deployed service. | +| Add a second Files or changes API | Duplicates inherited behavior instead of adapting it around canonical filesystem manifests. | ## Deliberate v1 limits -V1 supports public HTTPS Git repositories acquired at fixed depth `2`, source placed under 1 to 128 pairwise non-overlapping non-root destinations, advertised branch/tag/default refs, exact resolved-commit and effective-depth provenance, one server-owned bare acquisition cache per canonical URL, operator-catalogued OCI snapshots selected by direct digests, optional normalized offline Git history, writable private session roots with `read_only` repository bind mounts or private `editable` copies, bounded `session` retention, authorized `persistent` retention, and a workspace-relative working directory. +V1 supports 1 to 128 ordered Git or OCI source entries at pairwise non-overlapping non-root destinations; public HTTPS Git at fixed depth `2`; advertised branch, tag, or default refs; operator-catalogued OCI images selected by direct digests; one relative source tree per OCI image; independent component caches; atomic composition; canonical source and outer-workspace manifests; private writable outer workspaces; read-only bind mounts or private editable copies; `session` and authorized `persistent` retention; and a workspace-relative working directory. -V1 does not include caller-supplied registry origins or credentials, mutable OCI tags, OCI indexes as source identity, transparent Git/OCI fallback, caller-selected runtime images or benchmark environments, arbitrary materializer commands, private-network Git origins, caller-selected TTLs, session branching, access or retention changes on continuation, or public multi-tenant authorization. It does not add an AllAgents CLI command or change project workspace configuration. +V1 excludes caller-supplied registry origins or credentials, mutable OCI tags, OCI indexes as source identity, OCI multi-root bundles, transparent source-kind fallback, caller-selected runtime images or benchmark environments, arbitrary materializer commands, private-network Git origins, caller-selected Git depth or TTL, session branching, access or retention changes on continuation, compatibility aliases, deprecated schemas, and public multi-tenant authorization. It adds no AllAgents CLI command and changes no local project workspace configuration. -Only Codex and OMP are required and release-validated. Other HarnessRouter backends, local-profile import, host-profile projection, provider-route override, automatic provider fallback, scoring, datasets, assertions, and evaluation-task orchestration are outside this decision. +Only Codex and OMP are required and release-validated. Other inherited backends, local-profile import, host-profile projection, provider-route override, automatic provider fallback, scoring, datasets, assertions, and evaluation-task orchestration are outside this decision. ## Consequences -HarnessRouter remains the sole execution, workspace, and session control plane. -The fork gains deterministic first-turn source initialization without adding a -new northbound API, process supervisor, checkpoint system, file service, or -workspace lifecycle. Each session keeps its private writable root; only -declared source roots participate in generation sharing. +AllAgents Gateway remains one execution, workspace, and session control plane while explicitly preserving its derivation from upstream HarnessRouter. UHP clients that do not request workspace initialization retain stock compatibility. The downstream product, repository, image, and service have one unambiguous identity. + +Independent component caching makes mixed-source composition efficient and lets repeated requests reuse unchanged Git or OCI trees. Atomic composition and pre-network destination ownership add reservation and rollback complexity, but prevent partial or network-dependent workspace layouts. + +Mandatory OCI support makes v1 more substantial than a Git clone hook, but it makes large-source evaluations viable. Read-only bind mounts share protected bytes without making the outer workspace read-only. Private writable copies give Git and OCI identical editable behavior, including OCI-backed bug-fix evaluations. + +Canonical filesystem manifests add scanning and digest cost, but establish one source-kind-independent definition of final state. Evaluations no longer depend on whether Git metadata exists, whether an agent modified an index or commit, or whether rename inference agrees. -Mandatory OCI support and generation accounting make v1 more substantial than -a Git clone hook, but they make the minimum large-repository use case viable. -Read-only bind mounts let matching sessions reuse the same protected generation -inodes without making harness state or outputs read-only. Private -inode-independent copies preserve isolation for `editable` sessions. -The operator assumes finite capacity management for staging, generations, editable copies, persistent sessions, tombstones, and quarantined deletion failures. Protected or uncertain state is never advertised as free capacity. +Checkpoint metadata and storage become root-aware: immutable roots are referenced, editable roots are preserved separately, and outer state remains independent. The operator assumes finite capacity management for acquisition caches, staging, immutable components, editable copies, persistent sessions, tombstones, and quarantined deletion failures. Protected or uncertain state is never advertised as free capacity. -Provider credential lifecycle remains outside HarnessRouter. The distribution depends on the external OAuth-to-OpenAI-compatible gateway, while the harness sees only brokered short-lived credentials. +Provider credential lifecycle remains outside AllAgents Gateway. The distribution depends on the external OAuth-to-OpenAI-compatible gateway, while each harness sees only a brokered short-lived credential. ## Reconsider when Revisit this decision if: -- UHP or upstream HarnessRouter adopts an equivalent workspace-source contract; -- HarnessRouter changes its fresh/checkpoint workspace lifecycle so the extension point no longer preserves one authoritative session workspace; -- the host cannot enforce namespace-confined read-only bind mounts for shared generations and inode-independent editable copies; -- large-repository OCI materialization or cache reuse cannot meet finite release limits; -- continuation and attachment recovery cannot fail closed without source reacquisition; +- UHP or upstream HarnessRouter adopts an equivalent ordered multi-source contract; +- upstream workspace lifecycle changes so the extension point no longer preserves one authoritative session workspace; +- the host cannot enforce namespace-confined read-only bind mounts and inode-independent editable copies; +- large OCI materialization, canonical manifest comparison, or component cache reuse cannot meet finite release limits; +- continuation cannot fail closed without source reacquisition; - source acquisition requires a stronger isolation boundary; - public multi-tenancy or caller-owned private-source credentials become requirements; - Codex or OMP can no longer use the external provider gateway's required compatible surface; or diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index c4abcd89..1731420b 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,134 +1,125 @@ --- -title: "HarnessRouter Workspace Composition - Implementation Plan" +title: "AllAgents Gateway Workspace Composition - Implementation Plan" date: 2026-09-18 -updated: 2026-09-27 +updated: 2026-09-28 type: feat artifact_contract: ce-unified-plan/v1 artifact_readiness: implementation-ready execution: code --- -# HarnessRouter Workspace Composition - Implementation Plan - -## Stock behavior inventory - -The pinned HarnessRouter baseline already owns the session workspace lifecycle. -The fork must extend that lifecycle rather than introduce another workspace -abstraction: - -1. Session identity implicitly selects one private, writable session workspace; - callers do not currently describe source roots. -2. Fresh hydration creates that workspace as an empty root Git repository. -3. Continuation restores the session checkpoint selected by the existing - response/session identity. -4. Attached UHP input files, `.harness` state, generated root instructions, - plugins, skills, MCP configuration, HOME, conversation state, outputs, and - produced/checkpoint bookkeeping are written under the hydrated workspace - before or during a harness turn. -5. Produced-file collection uses a cursor over the root Git repository and may - assume that every descendant is part of that repository. -6. Stock checkpointing mutates the root Git repository, then archives the entire - workspace; stock hydration clears the workspace before restoring that archive. - Stock file walking, root Git operations, archive creation, restore, and - cleanup have no mount-boundary exclusions. -7. No stock UHP request field names a Git source, an OCI source, or a reusable - immutable generation. - -Phase 1 must characterize the exact paths and ordering of every stock write, -root Git command, filesystem walk, archive/restore step, and pre-turn asset -materialization rather than assuming the summary above is exhaustive. Those -observations define the smallest adaptation seam: the outer session workspace -remains private and writable, while only declared source destinations become -access-specific attachments. - -The implementation keeps the existing session allocation, hydrate/checkpoint -cycle, file and artifact APIs, user/sandbox isolation, cancellation, TTL, -cleanup, and harness supervision. It adds first-turn source composition at the -existing hydration boundary, stores the resulting attachment manifest in the -existing session state, and adapts root Git, checkpointing, files, restore, and -cleanup so they never traverse a declared source mount. It does not create a -second workspace, second session database, second Files API, external -materializer service, or parallel lifecycle. +# AllAgents Gateway Workspace Composition - Implementation Plan + ## Goal -Extend `allagentsdev/harnessrouter` so a new UHP session can compose its existing -private, writable HarnessRouter workspace from either multiple Git repositories -or a mandatory OCI workspace snapshot. Bind each verified source root and its -access mode to the session before the first harness turn. For `read_only`, mount -the immutable generation's repository roots read-only at their declared -non-root destinations; keep `.harness`, HOME, generated instructions, inputs, -outputs, and all other outer workspace state writable. Continuations omit the -descriptor, restore the writable outer checkpoint, and recover the exact source -attachments through the access-specific workspace-aware checkpoint path. - -OCI workspace snapshots are a release-blocking v1 source, not a later -optimization. Large repositories are part of the minimum deliverable. The Git -path limits history transfer with a fixed shallow fetch, but it still transfers -and checks out every working-tree byte; it therefore does not replace OCI for -large workspaces. - -Keep all other boundaries unchanged: - -- UHP is the only northbound execution protocol. -- Requests without `metadata.workspace` take the stock path without new source, - generation, attachment, or response semantics. -- Codex and OMP are the supported harnesses. -- Both use the existing separately operated OAuth-to-OpenAI-compatible provider - gateway through server-owned, brokered credentials. -- `allagentsdev/harnessrouter` remains the existing GitHub fork and - `ghcr.io/allagentsdev/harnessrouter` remains the image name. -- There is no AllAgents CLI implementation, profile import, local gateway - command, or `workspace.yaml` change. +Create AllAgents Gateway as the downstream distribution derived from +`HarnessRouter/harnessrouter`. A first-turn UHP request MAY declare an ordered +set of independent Git and OCI source trees. The gateway MUST acquire and cache +each component independently, then compose every requested tree at its declared +non-root destination in the existing private session workspace. A composition +MUST become visible atomically: either every source root is ready at the exact +requested identity or none is visible. + +The public downstream names are: + +- repository: `allagentsdev/allagents-gateway`; +- image: `ghcr.io/allagentsdev/allagents-gateway`; +- service: `allagents-gateway`; and +- product: **AllAgents Gateway**. + +The fork MUST retain `HarnessRouter/harnessrouter` as its upstream remote, the +recorded fork point, the upstream license, and required attribution. Upstream +changes MUST be taken selectively and reviewed against the downstream contract; +upstream acceptance is not a release dependency. + +Requests without `metadata.workspace` MUST remain on the stock UHP path. Their +root-Git hydration, produced-file behavior, checkpointing, continuation, +provider routing, cancellation, retention, and cleanup MUST remain compatible +with the pinned upstream baseline. The new component, composition, manifest, +and workspace-aware lifecycle apply only when `metadata.workspace` is present +on a new session. + +## Stock behavior and adaptation boundary + +The pinned upstream baseline owns session identity, one private writable +workspace per session, hydrate/checkpoint, UHP inputs, generated instructions, +`.harness`, HOME, skills, plugins, MCP configuration, conversation state, +Files/artifacts, cancellation, TTL, deletion, restart reconciliation, harness +supervision, and provider brokering. Stock fresh hydration initializes an empty +root Git repository. Stock produced-file collection and checkpointing use that +root repository and may traverse the whole workspace. + +Phase 1 MUST characterize the exact upstream call paths and ordering rather than +assuming that summary is exhaustive. The workspace-backed path MUST reuse the +same session and runner lifecycle but replace root/nested Git correctness with +filesystem-manifest correctness. For a workspace-backed session: + +- the outer workspace remains private and writable; +- source destination ownership and identities are immutable for the life of the + session, while `editable` tree contents may change; +- root Git, nested Git commits, indexes, status, diff, and rename detection MUST + NOT determine produced changes, evaluation results, or checkpoint correctness; +- Git metadata MAY be present as acquisition data and agent convenience only; +- OCI components need not contain Git metadata; +- Files collection and final evaluation MUST compare canonical filesystem + manifests for every source root and the writable outer workspace; +- checkpoint, restore, Files walks, and cleanup MUST understand source + boundaries and MUST NOT accidentally traverse a read-only mount; and +- no second workspace service, session database, Files API, scheduler, or + external materializer service is introduced. ## Product and ownership boundary ### In scope -- A strict first-turn-only `metadata.workspace` HarnessRouter extension. -- Repository composition from one or more caller-declared repositories at - pairwise non-overlapping workspace-relative destinations. -- Fixed depth-2 Git acquisition with exact resolved-commit provenance and useful - recent offline history. -- Operator-cataloged OCI workspace snapshots selected by direct image-manifest - and workspace-manifest digests. -- One immutable generation store shared by Git and OCI sources. -- Reuse of a verified immutable generation across sessions. -- Shared immutable generation inodes for `read_only` source roots and - inode-independent private writable copies for `editable` source roots. -- A private writable outer session workspace for both access modes, with - generated assets and allowed UHP input files outside source destinations. -- Existing session continuation, checkpoint, cancellation, TTL, deletion, - restart reconciliation, files, artifacts, root Git, and produced-file behavior - adapted to exclude declared source mounts. -- Authorized persistent retention through the existing session lifecycle. -- Exact source provenance and bounded, coded failures. -- Direct Promptfoo coverage against the built image, including a large OCI - workspace and second-session cache reuse. -- Digest-pinned publication to GHCR with SBOM and build provenance. +- A strict first-turn-only `metadata.workspace` extension to UHP. +- A closed ordered `sources` array containing 1 to 128 independent Git, OCI, or + mixed source entries. +- Exactly one source tree and one required non-root `destination` per entry. A + monorepo is one tree, not an implicit bundle of roots. +- Canonical public HTTPS Git acquisition with exact commit resolution and a + mandatory depth-2 fetch policy. +- Operator-cataloged OCI source-tree transport selected by a direct image + manifest digest and a canonical source manifest digest. +- Independent component cache and singleflight, followed by atomic composition. +- Identical `read_only` and `editable` attachment semantics for Git and OCI. +- Canonical baseline source manifests for all components and a canonical + baseline for the writable outer workspace. +- Manifest-based add, modify, delete, mode, executable, symlink, and binary + change detection for workspace-backed sessions. +- Mount-aware checkpoint and continuation, including separate preservation of + editable roots. +- Per-source and request-aggregate resource limits. +- Codex and OMP through the existing separately operated + OAuth-to-OpenAI-compatible provider gateway and brokered credentials. +- Direct Promptfoo proof against the built image, including multiple OCI + components, a mixed Git/OCI composition, editable OCI mutation, and cache + reuse. +- Digest-pinned GHCR publication with SBOM and build provenance. ### Explicit non-goals -- A separate workspace service, workspace database, scheduler, Files API, or - task protocol. - Caller-selected runtime/container images or benchmark environments. An OCI - workspace snapshot is source content only. -- Caller-provided registry origins, registry credentials, headers, proxy - settings, or source commands. -- Arbitrary materializer plugins, hook discovery, or a public generation API. -- Silent Git fallback for an OCI failure, silent OCI fallback for a Git failure, - or silent deepening/full-clone fallback for a bounded Git failure. -- Submodule initialization, Git LFS hydration, checkout filters, or repository - hook execution. -- Provider login, refresh, or repair in HarnessRouter; the external provider - gateway retains that responsibility. -- A caller-selected provider route, API key, transport, or fallback chain. -- Upstream acceptance as a release condition. Upstreaming is considered only - after downstream release evidence exists. + image is only a transport and cache unit for one source tree; it is never the + runtime workspace and never a multi-root workspace bundle. +- Caller-provided registry origins, credentials, headers, certificates, mirrors, + proxy settings, Git configuration, source commands, or materializer hooks. +- A public component-cache or composition API. +- Silent Git-to-OCI, OCI-to-Git, ref, digest, mirror, deepening, full-clone, or + provider fallback. +- Git submodule initialization, Git LFS hydration, checkout filters, or hook + execution. +- Requiring a Git repository at workspace root or inside an OCI tree. +- Provider login, refresh, or repair in AllAgents Gateway. +- An AllAgents CLI, profile import, local gateway command, or `workspace.yaml` + change. +- Upstream acceptance as a release condition. ## Request contract -`metadata.workspace` is accepted only when the request creates a new session. It -uses snake_case and has no nested schema version: +`metadata.workspace` MUST be accepted only while creating a new session. The +schema uses snake_case, has no nested schema version, and is closed at every +object boundary. ```json { @@ -137,20 +128,21 @@ uses snake_case and has no nested schema version: "workspace": { "access": "editable", "retention": "session", - "source": { - "kind": "repositories", - "repositories": [ - { - "url": "https://github.com/example/service.git", - "ref": "refs/heads/main", - "destination": "service" - }, - { - "url": "https://github.com/example/shared.git", - "destination": "libraries/shared" - } - ] - }, + "sources": [ + { + "kind": "git", + "url": "https://github.com/example/service.git", + "ref": "refs/heads/main", + "destination": "service" + }, + { + "kind": "oci", + "snapshot_name": "shared-release", + "image_manifest_digest": "sha256:...", + "source_manifest_digest": "sha256:...", + "destination": "libraries/shared" + } + ], "working_directory": "service" } } @@ -163,1007 +155,922 @@ The exact shape is: metadata.workspace = { access: "read_only" | "editable", retention?: "session" | "persistent", - source: + sources: Array< | { - kind: "repositories", - repositories: Array<{ - url: string, - ref?: string, - destination: string - }> + kind: "git", + url: string, + ref?: string, + destination: string } | { - kind: "workspace_snapshot", + kind: "oci", snapshot_name: string, image_manifest_digest: string, - workspace_manifest_digest: string - }, + source_manifest_digest: string, + destination: string + } + >, working_directory?: string } ``` -Rules: - -1. `access` and `source` are required. `retention` defaults to `session`. -2. `persistent` is accepted only after existing caller/session authorization - succeeds and before source resolution, network traffic, or generation claims. -3. `working_directory` and every repository `destination` are normalized - workspace-relative POSIX paths. The effective working directory must be a real - directory inside the final workspace without traversal or link escape. -4. Every source destination is non-root: `.` and any spelling that normalizes to - the workspace root are invalid. Destinations are unique and pairwise - non-overlapping: no two may be equal, and neither may be an ancestor of - another. A monorepo therefore uses a destination such as `repo`, and - `working_directory` may be `repo` or `repo/packages/api`. -5. Repository mode contains 1 to 128 entries. An OCI workspace manifest likewise - declares 1 to 128 repository roots, each with a non-root, pairwise - non-overlapping destination. Snapshot source files may exist only beneath - those roots; there is no snapshot source at destination `.` and no undeclared - root source file. Ancestor directories needed to reach a destination are - mount scaffolding only and contain no source files. -6. Each URL, optional ref, destination, snapshot field, and manifest root is - bounded before network or filesystem work. Lower runtime capacity fails with - the coded capacity error; it does not change schema validity. -7. `source` is a closed discriminated union. Unknown fields and mixed Git/OCI - fields fail validation. -8. `snapshot_name` selects an operator-owned HarnessRouter deployment catalog - entry. The request supplies only the two `sha256:` digests; it never supplies - a registry, repository, credential, certificate, or mirror. -9. A continuation selected through `previous_response_id` or the existing - session recovery mechanism omits `metadata.workspace`. Supplying it on a - reused session fails before hydrate, source access, generation lookup, or - provider traffic, even when it is identical to the stored value. -10. A session created without workspace metadata remains a stock session and - cannot add workspace metadata later. -11. UHP input files target the writable outer workspace by default. For - `read_only`, reject an input whose normalized path is equal to or below an - effective source destination; inputs elsewhere remain valid. Repository-mode - destinations are known during request validation. Snapshot destinations are - validated after the image/workspace manifests establish the verified root - map, but before layer acquisition, outer workspace writes, generation - attachment, or harness execution. For `editable`, inputs may overlay the - private source copies. In both modes, reject paths that collide with mount - scaffolding, reserved runner paths, or generated assets. -12. Workspace fields cannot contain commands, environment variables, resource - limits, provider settings, or harness settings. - -A workspace snapshot request is therefore: - -```json -{ - "metadata": { - "harness_id": "allagents-omp", - "workspace": { - "access": "read_only", - "source": { - "kind": "workspace_snapshot", - "snapshot_name": "monorepo-release", - "image_manifest_digest": "sha256:...", - "workspace_manifest_digest": "sha256:..." - }, - "working_directory": "packages/api" - } - } -} -``` +The obsolete singular `source`, `kind: "repositories"`, `repositories`, +`kind: "workspace_snapshot"`, and `workspace_manifest_digest` fields MUST NOT be +accepted as aliases. There is no compatibility schema. + +### Validation rules + +1. `access` and `sources` are REQUIRED. `retention` defaults to `session`. + `sources` MUST contain 1 to 128 entries and preserves request order. +2. Every entry MUST materialize exactly one tree at its own required + `destination`. Multiple Git entries, multiple OCI entries, duplicate + component identities at different destinations, and mixed Git/OCI entries + are valid. +3. Every destination and supplied `working_directory` MUST be a relative POSIX + path with no empty, `.`, `..`, ambiguous, or platform-specific component and + no link escape. A destination that normalizes to the workspace root is + invalid. An omitted `working_directory` means the root, represented as `.` + only in the response. +4. Destinations MUST be pairwise non-overlapping after normalization: no two are + equal and neither is an ancestor of another. Their ownership is determined + entirely from the request; OCI content MUST NOT add or move a root. +5. UHP input paths, generated assets, reserved runner paths, credential paths, + cache paths, attachment state, and checkpoint state MUST be normalized and + checked against every destination before source resolution, network traffic, + cache lookup, session workspace writes, or component claims. Any path equal + to or below a source destination MUST be rejected for both access modes. + Inputs and generated assets outside all source destinations remain valid in + the writable outer workspace. +6. Destination ancestor scaffolding MAY coexist only with independently valid + outer directories. A file, symlink, generated asset, input, or reserved path + at an ancestor that prevents safe directory scaffolding MUST fail before + network work. +7. `working_directory` syntax is validated before network work. After atomic + composition it MUST resolve, without link escape, to a real directory either + in the outer workspace or within exactly one source root. +8. A Git `url` MUST be canonical public HTTPS under the deployment egress policy. + User information, query strings, fragments, credentials, alternate + transports, ambiguous encodings, and caller transport options are forbidden. + `ref`, when present, is bounded and resolves only through advertised branch + or tag semantics. +9. An OCI entry MUST provide an operator-catalog `snapshot_name`, a direct + `sha256:` image manifest digest, and a canonical `sha256:` + `source_manifest_digest`. Tags, indexes/lists, mutable references, and caller + registry coordinates are forbidden. +10. Unknown fields, wrong-kind fields, mixed fields within one entry, invalid + digests, and obsolete schema fields MUST fail closed. +11. Metadata bytes, nesting, strings, path lengths, source count, and input count + MUST be bounded during parsing. Persistent retention MUST be authorized + before cache lookup or network work. +12. A continuation selected by `previous_response_id` or the existing recovery + mechanism MUST omit `metadata.workspace`. Supplying it on a reused session + MUST fail before hydrate, component lookup, provider traffic, or TTL changes, + even if it equals the stored descriptor. A stock session cannot become a + workspace-backed session later. +13. Workspace fields MUST NOT contain commands, environment variables, resource + limits, provider settings, model settings, registry settings, or harness + settings. + +Because every source declares its destination, all destination ownership, +source-source overlap, and input/asset collision decisions are request-decidable +and MUST complete before any network request. OCI source-manifest validation is +content validation, not destination discovery. + +## Source and request limits + +Bounds MUST be enforced while streaming, before allocation whenever the size is +known, and both per source and across the whole request. Deployment policy MAY +lower a bound but MUST NOT raise these v1 ceilings without a contract revision. +A lower runtime capacity is a coded capacity failure, not a schema change. + +| Resource | Per source ceiling | Request aggregate ceiling | +|---|---:|---:| +| Expanded source bytes | 32 GiB | 64 GiB | +| Source-visible entries | 500,000 | 1,000,000 | +| Compressed Git pack or OCI layer bytes | 8 GiB | 16 GiB | +| Regular file size | 4 GiB | 4 GiB | +| Path | 4096 UTF-8 bytes / 128 components | same per path | +| Acquisition and materialization wall time | bounded by operator policy | bounded by session policy | + +Each OCI source additionally permits at most 64 distributable tar/gzip/zstd +layers, a 4 MiB image manifest, a 128 MiB canonical source manifest, and 1 MiB +per PAX or extended header. For each layer, each OCI source, and the request +aggregate, `expanded_bytes / max(compressed_bytes, 1)` MUST NOT exceed `100`. +Git object, checkout, inode, output, and filesystem quotas MUST feed the same +per-source and aggregate accounting rather than becoming unbounded exceptions. + +## Canonical manifests and change semantics + +### Component baseline source manifest + +Every published Git or OCI component MUST have a canonical baseline source +manifest. It is keyed by normalized path relative to that component root and +contains, for every included filesystem entry: + +- normalized relative path and entry type; +- regular-file content digest and size; +- executable bit and the platform-normalized mode semantics needed to reproduce + observable permissions; +- symlink target bytes after canonical encoding; and +- any bounded hardlink representation required by the extraction policy. + +Ordering and serialization MUST be canonical. Directory and link semantics MUST +make type changes observable. Binary files use content digests exactly like text +files; no text decoding or line diff is required for correctness. The component +manifest digest is part of the immutable component publication. + +For Git, the runner computes the manifest from the verified detached depth-2 +checkout. For OCI, the canonical manifest named by `source_manifest_digest` MUST +be fetched and verified before any layer request. Its paths are relative to the +single requested destination. It MUST declare the final types, modes, sizes, +content digests, and links that layer application is expected to produce. Final +extraction MUST exactly match it before publication. + +Git administrative state MAY be acquired and retained for agent convenience, +but it is never a change baseline. OCI trees MAY omit it entirely. A source +manifest MAY verify declared Git administrative files when present; the change +collector MUST exclude every `.git` entry from reported changes. + +### Outer workspace baseline + +For a workspace-backed session, fresh hydration MUST create the writable outer +workspace and apply allowed UHP inputs, generated instructions, harness assets, +and other initial session material outside source destinations. Immediately +before the first harness process starts, the runner MUST publish a canonical +outer baseline manifest using the same path/type/content/mode/link model. It +MUST exclude source destinations and runner-owned state. + +The session binding MUST preserve the exact component baseline manifests and the +outer baseline across continuation. A continuation MUST NOT silently regenerate +a baseline from already-mutated content. + +### Final-tree comparison + +At every required collection/evaluation boundary, the runner MUST walk the +writable outer workspace without crossing a source mount and MUST walk every +source root through its declared ownership boundary. It MUST compare each final +manifest to the corresponding persisted baseline and report the union of: + +- additions; +- content modifications, including binary changes; +- deletions; +- executable or other observable mode changes; and +- symlink additions, removals, retargeting, and type transitions. + +Rename inference is OPTIONAL. Delete-plus-add is correct; final-tree equality is +authoritative. Root Git, nested Git, commits, index state, ignored-file rules, +and Git rename detection MUST NOT be correctness sources. A Git command MAY be +available to the agent, but it MUST NOT change evaluator results. + +The collector MUST normalize each reported path into workspace-relative form, +assign it to exactly one owner (outer workspace or one source destination), and +deduplicate it. It MUST exclude `.git`, runner-owned state, credentials, caches, +attachment evidence, checkpoint metadata, harness-private ephemeral state, and +the component store. Existing count, size, and artifact limits still apply to +reported outputs. Initial source and outer baseline entries MUST NOT be reported +merely because composition or hydration created them. + +`read_only` source roots are compared as an integrity check and MUST remain equal +to their component baselines. `editable` Git and OCI roots are compared with the +same algorithm and MUST report mutations identically. Bug-fix evaluations MUST +request `access: "editable"`. + +## Identity, cache, and generation design + +A **component** is one verified immutable source tree. A **composition** is an +ordered mapping of exact component identities to normalized destinations. A +**session attachment** applies one composition under an access mode to one +private outer workspace. None is a runtime image or a second session. + +### Component identities + +A Git component identity MUST include the canonical URL, exact resolved commit, +fixed depth `2`, Git acquisition-policy revision, materializer revision, and +canonical baseline source-manifest digest. The request destination, ref spelling, +access, session, harness, provider, retention, and working directory MUST NOT +fragment the component cache. + +An OCI component identity MUST include the operator catalog identity, +`snapshot_name`, direct `image_manifest_digest`, canonical +`source_manifest_digest`, OCI validation-policy revision, materializer revision, +and verified baseline source-manifest digest. Registry origin and credentials +MUST remain private. Destination and session concerns MUST NOT fragment the +component cache. + +The canonical composition identity MUST reference, in request order, each exact +component publication identity paired with its normalized destination, plus the +layout/materializer contract revision. Reordering entries therefore changes the +composition identity even when component bytes are the same. The composition +record contains references and evidence, not another copy of component bytes. -## Session binding and public provenance +### Git acquisition cache -The fork stores workspace binding fields in the existing session/checkpoint -record. The binding contains: - -- the canonical effective descriptor and its digest; -- the source kind and exact resolved source identity; -- the generation key, immutable publication/epoch identity, verified tree - manifest digest, and immutable declared source-root map; -- `access`, effective `retention`, and normalized `working_directory`; -- the attachment manifest and evidence for every destination, including - generation root identity, mount/copy method, filesystem identity, and mount - protection needed to prove a restored session refers to the exact generation; -- exact public provenance; -- existing session expiry/deletion state; and -- the selected harness/provider binding already owned by the session. - -Public response `metadata.workspace` has exactly `access`, `retention`, -`working_directory`, `effective_descriptor_digest`, `generation_id`, -`workspace_manifest_digest`, `provenance`, and `expires_at`. -`working_directory` is always present and uses `.` for the workspace root. -`expires_at` is the effective timestamp for `session` retention and `null` only -for authorized `persistent` retention. - -Repository `provenance` has `kind: "repositories"` and a request-order -`repositories` array. Each entry has normalized `url`, `destination`, exact -`resolved_commit`, effective `depth`, and `requested_ref` only when the request -supplied a ref. Snapshot `provenance` has `kind: "workspace_snapshot"`, -`snapshot_name`, exact `image_manifest_digest`, exact -`workspace_manifest_digest`, and a manifest-order `repositories` array. Each -snapshot root has `destination`; a history-bearing root also has -`resolved_commit` and `object_set_digest`, while a tree-only root has neither. - -Acquisition-policy revisions, catalog origins, mirrors, credentials, host and -mount paths, attachment IDs, leases, and internal generation keys are not -public. A continuation and response replay return the same committed object; -they never report a newly resolved ref or substituted generation. - -## Existing lifecycle integration - -The first-turn sequence is: - -1. Authenticate and validate the stock UHP envelope and establish existing - idempotency ownership. -2. Resolve whether the request creates or reuses a session. -3. If new and workspace-backed, validate and authorize the closed workspace - descriptor, request-declared repository destinations, working-directory - syntax, and input paths that can be decided without source access. Store a - pending binding in the existing session transition. -4. Allocate the existing private, empty, writable session workspace and isolation - identity without running fresh-hydration writes, root Git, bookkeeping, or - asset materialization. -5. Resolve the exact source plan. For repositories this resolves exact commits; - for OCI it verifies the image and workspace manifests sufficiently to obtain - the authoritative 1–128 root map before fetching/extracting layers. Validate - all non-root, non-overlap, input, generated-asset, reserved-path, and restored - outer-state collisions against that effective root map before any outer - workspace content write or generation claim/materialization. -6. Run fresh hydration and materialize root Git/bookkeeping, `.harness`, HOME, - conversation data, generated instructions, plugins, skills, - MCP/configuration, credentials, scratch, and other stock mutable assets in - their normal private outer locations. Configure every root Git command and - filesystem walk to exclude the immutable destination set and never cross - mount boundaries. Apply allowed outer UHP inputs at the stock pre-turn point. - Create empty, non-link mountpoint directories and mount-ancestor scaffolding - only after collision validation. -7. Claim or reuse the generation. For `read_only`, take a lease and attach each - immutable generation repository root to its destination with a per-session - read-only bind mount. For `editable`, create each inode-independent private - writable source copy at its destination. -8. For `editable`, apply source-targeting inputs to the private copies. For - `read_only`, source-targeting inputs have already failed. Verify every - destination, protection flag, source identity, absence of writable aliases, - and editable inode independence before recording attachment-ready. -9. Establish the outer produced-file cursor without traversing source - destinations and the access-specific per-root collectors. Validate the - effective working directory after attachments are complete. -10. Atomically persist the attachment manifest/evidence and mark the existing - session ready, then continue through ordinary provider selection and harness - execution. -11. Collect files/artifacts without mount traversal, then checkpoint the writable - outer workspace while explicitly excluding every source destination and all - source bytes. Persist attachment manifest/evidence separately. For - `editable`, nested source collectors and the editable-source checkpoint path - preserve each private root without allowing the outer archive to traverse it. - Stream events, set terminal state, and schedule cleanup through existing - HarnessRouter paths. - -A continuation does not parse or resolve a source. Workspace-aware hydration -uses this exact order: - -1. recover and validate the minimal stored binding, durable generation reference, - source-root map, access mode, and attachment manifest/evidence without - resolving Git or OCI; -2. ensure stale session mounts are unmounted, then clear/hydrate the outer - workspace using no-follow, no-cross-mount operations; -3. restore the writable outer checkpoint, which contains no source bytes; -4. validate that every declared destination is an empty non-link directory, that - its ancestors contain only allowed outer state/scaffolding, and that no input, - generated asset, root Git entry, or restored path collides with an attachment; -5. reacquire the exact recorded generation lease and, for `read_only`, bind each - exact recorded generation root read-only at its destination; for `editable`, - restore the exact private source-root checkpoint at its destination; -6. verify mount flags, filesystem/generation identity, source manifest, no - writable alias, and editable ownership/inode independence; and -7. only then restore or activate remaining runtime state, rebuild external - cursors, validate `working_directory`, and start the harness. - -A missing, expired, corrupt, wrong-generation, writable, partially mounted, or -unsupported attachment fails closed. Hydration must not reacquire Git, contact -an OCI registry, select another cached generation, archive or restore shared -generation bytes, or start with an empty source root. Any partial continuation -attachment is unmounted before failure cleanup. - -`retention: "session"` follows existing finite session TTL and deletion. -Authorized `persistent` retention pins the existing session and its generation -reference until explicit deletion or operator policy permits removal; it does -not create a second retention scheduler. Polling and response replay do not -extend retention. Cleanup makes the session unavailable, unmounts every source -destination, verifies that no mount remains, then hydrates/deletes outer state, -removes editable copies, and releases the generation reference. The same -unmount-before-hydrate/delete rule applies to cancellation, retry, and restart -reconciliation and remains idempotent. -## Generation and attachment design - -A generation is a verified immutable source artifact, not a runnable workspace -or session. It is stored outside session allocations under runner-owned `/data` -state and can be attached only through the existing hydrate path. - -### Identity - -The canonical generation key includes only immutable source and layout inputs: - -- source kind; -- for every repository in request order: canonical normalized URL, exact - resolved commit, and normalized destination; or the selected OCI catalog - identity plus both direct digests; -- the normalized source-layout/workspace-manifest schema revision; -- materializer contract revision; -- Git fetch depth (`2`) and Git acquisition-policy revision for repository - sources; -- OCI extraction and validation-policy revision for snapshot sources; and -- any operator acquisition-policy identity that can change resulting bytes. - -The key excludes session ID, response ID, harness, provider, access, retention, -working directory, and caller display data. Those values affect attachment or -execution, not generation bytes. Different request ref spellings that resolve to -the same canonical repositories, commits, destinations, depth, and policy reuse -the same generation. Exact provenance retains the original requested values even -when the immutable generation key is shared. +The runner MUST maintain one operator-only bare shallow acquisition mirror per +canonical Git URL. Every write to that mirror MUST be serialized; concurrent +resolution/fetch for the same normalized request MUST singleflight. On a miss, +the worker MUST resolve the advertised default, branch, or tag to an exact +commit, fetch exactly depth 2, verify the fetched tip and shallow boundary, and +atomically import the bounded result. It MUST NOT deepen, unshallow, full-clone, +fetch an arbitrary object ID, choose another ref, or fall back to OCI. + +An immutable self-contained component checkout MUST be exported without +alternates or writable links to the mirror. It MUST preserve enough normalized +`.git` data for recent offline `git log`, parent inspection, blame where the +shallow history permits, and diff. A merge tip MUST retain both fetched parent +edges when the server supplies them at depth 2. Submodule gitlinks and LFS +pointer-backed content MUST fail rather than invoke helpers. + +A hit for the same exact component identity MAY advertise a mutable ref to prove +that it still resolves to that commit, but MUST perform zero pack acquisition, +checkout, tree copy, baseline recomputation, or publication. Separate counters +MUST distinguish advertisement from source-byte transfer. + +### OCI component cache + +The deployment owns a bounded catalog. Each `snapshot_name` maps to one +operator-controlled registry/repository origin, credential reference, TLS and +redirect policy, allowed media types, and resource policy. Those values MUST NOT +appear in the request, session workspace, logs, public provenance, or harness +environment. + +The worker MUST require a direct image manifest digest and reject tags, +indexes/lists, mutable references, and catalog mismatches. It MUST fetch and +verify the image manifest, config, and canonical source manifest before any +layer request. It MUST validate the source manifest's relative paths, types, +sizes, modes, content digests, links, declared layer requirements, and per-source +and aggregate limits before downloading layers. + +Layers MUST be streamed, digest-checked, and applied in order with correct file +and opaque-directory whiteout semantics. Whiteouts are instructions and MUST +NOT appear in the publication. Extraction MUST reject absolute paths, traversal, +NULs, ambiguous separators, conflicting duplicates, devices, FIFOs, sockets, +unsafe sparse files, unsupported types, and unbounded metadata. Symlinks and +hardlinks MUST remain within their owning source root; links to the outer +workspace or another source are invalid. The completed tree MUST exactly match +the canonical source manifest before atomic publication. + +An exact OCI component-cache hit MUST perform zero registry manifest/config/ +source-manifest/layer requests, extraction, tree copy, baseline recomputation, or +publication. An OCI component is always one tree; an image that encodes multiple +workspace roots or files outside that tree's relative manifest MUST fail. + +### Singleflight, publication, and reuse + +Each component identity MUST have its own durable singleflight claim and random +private staging directory. Independent components MAY acquire concurrently +within request and operator limits. A waiter cancellation MUST detach only that +waiter while another live request still needs the build. When no waiter remains, +the bounded builder MAY be cancelled. Failed, timed-out, cancelled, partial, or +unverified staging MUST never become attachable. + +After streamed accounting and full manifest verification, publication MUST use +an atomic rename and record ownership, policy revisions, manifest digest, +publication epoch, and completeness. Published component bytes and manifests +MUST be immutable. Startup reconciliation MUST quarantine uncertain state. A +lease/reference MUST protect a component from cleanup; cleanup MUST remove only +complete unreferenced publications and MUST NOT invalidate an attached session. + +A composition resolver MUST wait for every independently claimed component, +verify the ordered identities and destinations, and commit one immutable +composition record. One component failure MUST roll back request-local staging +and references without invalidating successful shared components needed by +other sessions. No partial composition can be attached. + +## Access and atomic attachment + +Both acquisition kinds MUST implement the same access behavior. + +- `read_only`: lease each immutable component and bind-mount its root at the + declared destination with kernel-enforced read-only, `nodev`, and `nosuid` + semantics while preserving required execute bits. The session MUST receive no + writable backing descriptor, alias, overlay/copy-up path, mirror path, or + component-store path. Symlinks are forbidden as an attachment mechanism. +- `editable`: lease each immutable component and create an inode-independent + private writable copy or safe reflink at the declared destination. A later + write MUST NOT mutate the cache or any sibling session. Hard-linked mutable + files and writable aliases are forbidden. + +The runner MUST construct fresh and continued workspace-backed sessions in a +private, non-runnable staging workspace or private mount namespace. It MUST +prepare the outer state, all mountpoint scaffolding, every read-only mount and +editable copy, manifest evidence, and the validated working directory there. It +MUST expose no Files, checkpoint, provider, or harness consumer until every +entry verifies. A single atomic workspace-path publication or namespace handoff, +paired with the session ready transition, MUST make all roots visible together. +Any failure MUST unwind mounts in reverse order, remove private copies/staging, +release request-local references, and leave no runnable or externally visible +partial workspace. + +Matching read-only sessions MUST bind the same immutable component bytes. +Editable sessions MUST start from those same publications without reacquisition +but have independent inodes. The outer workspace remains private and writable +in both modes. +## Session binding and public provenance -### Git acquisition cache +The existing session/checkpoint record MUST store: + +- the canonical effective descriptor and digest; +- the ordered resolved source plan; +- each exact component identity, publication epoch, and baseline source-manifest + digest; +- the composition identity and ordered destination ownership map; +- `access`, effective `retention`, normalized `working_directory`, and selected + harness/provider binding; +- the outer baseline manifest identity; +- attachment evidence per destination, including mount/copy method, filesystem + identity, read-only protection or editable inode independence; +- checkpoint identities for writable outer state and each editable source root; +- exact sanitized public provenance; and +- existing expiry, deletion, and lifecycle state. + +The public `metadata.workspace` response MUST contain exactly `access`, +`retention`, `working_directory`, `effective_descriptor_digest`, +`composition_id`, ordered `sources`, and `expires_at`. `working_directory` is +always present and uses `.` for workspace root. `expires_at` is `null` only for +authorized persistent retention. + +Each public Git source entry contains `kind: "git"`, normalized public `url`, +`destination`, exact `resolved_commit`, `depth: 2`, `component_id`, +`source_manifest_digest`, and `requested_ref` only when supplied. Each public +OCI entry contains `kind: "oci"`, `snapshot_name`, `destination`, exact +`image_manifest_digest`, exact `source_manifest_digest`, and `component_id`. +Private component keys, catalog origins, mirrors, credentials, host paths, mount +IDs, leases, policy identities, and attachment paths MUST NOT be public. Replay +and continuation return the stored committed object and MUST NOT re-resolve +mutable references. + +## Lifecycle + +### New workspace-backed session + +1. Authenticate and validate the stock UHP envelope; establish existing + idempotency ownership and whether the request creates or reuses a session. +2. For a new workspace-backed session, parse and strictly validate the closed + descriptor, authorize retention, normalize the ordered 1–128 entries, and + reject all destination overlap and destination/input/asset/reserved-path + collisions before network, cache lookup, workspace writes, or component + claims. +3. Persist a pending session binding containing only the validated canonical + request. Allocate an inaccessible staging workspace under the existing + session/isolation lifecycle. +4. Resolve exact components. Git resolves advertised refs and depth-2 commits. + OCI verifies the image and canonical source manifest before layer requests. + Enforce per-source and aggregate limits as facts become known. +5. Claim/reuse each component independently. Acquire, materialize, verify, and + atomically publish misses; retain leases for hits. Wait for all entries and + commit the ordered composition identity. On any failure, attach none. +6. Fresh-hydrate the private outer staging workspace. Apply allowed inputs and + generated assets only outside destinations. Create empty, non-link + destination directories and safe ancestor scaffolding after rechecking the + prevalidated ownership map. +7. Attach every component using the requested access mode. Verify exact + identities, read-only flags/no writable aliases, or editable ownership/inode + independence. No consumer can observe the workspace during this step. +8. Validate the effective working directory. Publish the canonical outer + baseline after all initial outer assets are present, retain every canonical + component baseline, and persist attachment/checkpoint evidence. +9. Atomically publish the complete workspace and transition the existing session + binding to ready. Only then select the provider and start the harness. +10. At collection, compare final outer and source manifests to their baselines. + Checkpoint the writable outer tree without crossing source destinations; + checkpoint each editable source separately. Read-only source bytes are never + archived. Complete the existing stream/session transition and schedule + cleanup. + +Duplicate initial requests MUST share the existing idempotent result and MUST +NOT claim a second composition or attachment. + +### Continuation + +A continuation MUST NOT parse or resolve sources or contact Git/OCI endpoints. +It MUST: + +1. recover and validate the stored descriptor digest, exact component and + composition identities, destination map, access, manifests, attachment + evidence, checkpoints, and lifecycle state; +2. ensure stale mounts are unmounted, then restore the writable outer checkpoint + into an inaccessible staging workspace using no-follow/no-cross-mount + operations; +3. restore each editable root from its separate checkpoint, or reacquire the + exact recorded immutable lease and prepare the exact read-only bind mount; +4. validate destination scaffolding, collisions, component/baseline identities, + editable ownership, and read-only protection; +5. restore the original outer and component baselines without recomputing them + from mutated session content; +6. validate the working directory and atomically publish the complete workspace; + and +7. only then activate provider credentials, Files collection, and the harness. + +A missing, expired, corrupt, wrong-generation, writable, partially restored, or +unsupported attachment MUST fail closed. Continuation MUST NOT select another +cached component, reacquire source bytes, publish empty roots, or discard +editable mutations. + +### Checkpoint, retention, and cleanup + +The outer checkpoint MUST exclude every source destination and all source bytes. +Each editable Git or OCI root MUST have a separate private checkpoint and retain +its baseline identity. Read-only roots are reattached from immutable component +publications. Checkpointing and restore MUST be no-follow, mount-aware, bounded, +and cancellation-safe. + +`retention: "session"` follows existing finite TTL and deletion. Authorized +`persistent` retention pins the existing session and required references until +explicit deletion or applicable operator policy; it does not create another +scheduler. Polling and replay MUST NOT extend retention. + +Cleanup MUST first make the session unavailable, stop descendants, unmount every +source destination, verify mount absence, remove editable copies and outer +state, and release composition/component references exactly once. Hydrate, +restore, delete, cancellation, and restart reconciliation MUST follow the same +unmount-before-traversal rule. An uncertain or failed cleanup MUST quarantine the +path, keep it unavailable and accounted, and permit confined idempotent retry. -Repository sources use two server-owned cache levels inside runner-owned -`/data`; neither is a session workspace: - -1. One operator-only bare shallow acquisition mirror exists per canonical - repository URL. Only the runner's acquisition worker can write it. Every - refresh of that repository is serialized through the same mirror, and - concurrent refreshes of the same normalized ref request use one in-flight - operation. Depth and acquisition-policy revisions belong to immutable commit - snapshots and generation identity, not the mutable mirror key. -2. Verified depth-2 commit snapshots from that mirror feed immutable multi- - repository generations keyed by canonical URLs, exact commits, destinations, - depth, policy, and materializer contract revision. A generation is the only - cache object that can be attached to a session. - -The bare mirror stores acquired objects and shallow-boundary metadata so a -second generation needing the same commit does not clone or transfer its pack -again. Updates import a verified bounded fetch atomically; they never mutate a -published commit snapshot, silently deepen it, or make a partially refreshed -mirror eligible for generation construction. The generation builder exports a -self-contained repository with no alternates and no writable link to the mirror. -Mirror paths, file descriptors, credentials, refs, and writable internals are -never mounted into or disclosed to a session. - -Resolving a mutable advertised ref may contact the origin to determine its -current exact commit. Once resolution yields an identity already present in the -mirror and generation store, there is no source pack acquisition, checkout, -tree materialization, or publication. Separate counters distinguish ref -advertisement from source-byte acquisition so cache-reuse proof cannot count a -remote pack fetch as a hit. - -### Publication and reuse - -- Build into a random private sibling staging directory. -- Persist one singleflight claim per bare-mirror refresh and one per generation - key. Concurrent misses perform at most one remote pack acquisition and one - immutable generation publication. Other requests wait independently, and one - waiter's cancellation does not cancel work still needed by another live - waiter. -- Stream validation and accounting during acquisition. Verify the final manifest - before publication. -- Atomically rename verified staging into an immutable publication and record its - complete metadata in runner state. -- Treat bare commit snapshots and published generations as immutable. Startup - verifies recorded ownership, publication completeness, shallow boundary, - policy revision, and manifest evidence before readiness. -- A repository generation hit may refresh mutable ref advertisement, but performs - no clone, pack fetch, checkout, tree copy, or second publication after the - exact normalized source identity matches. An OCI digest-keyed hit performs no - registry manifest/blob request, extraction, tree copy, or second publication. - Both record only a new session lease/reference. -- Every `read_only` session whose normalized source identity matches a cached - generation bind-mounts the same immutable generation repository roots at its - declared destinations. It cannot choose to reclone, re-extract, rematerialize, - hard-link, symlink, or copy cached source bytes. -- Failed or cancelled builds remove staging after descendants stop. A partially - refreshed mirror, commit snapshot, or generation is quarantined and never - attached. -- Existing leases/session references protect a generation from cleanup. Existing - cleanup scheduling removes only complete, unreferenced generations under - bounded operator policy. Mirror/object-cache cleanup is separately confined - and cannot invalidate a referenced generation. - -### Access-specific attachment - -- `read_only`: take a lease, then create one per-session bind mount from each - immutable generation repository root to its declared non-root destination in - the existing private writable workspace. Remount or create the bind with a - kernel-enforced read-only view plus `nodev` and `nosuid`, while preserving - repository execute bits required by tools. Mount setup is namespace-confined - to the session, exposes no generation backing path or writable file - descriptor, has no writable alias or overlay/copy-up path, and fails closed if - any protection or identity check fails. Matching sessions see the same source - filesystem identities/inodes, while `.harness`, HOME, inputs, outputs, - generated instructions, root Git/bookkeeping, checkpoint, conversation, - harness runtime, and lifecycle state remain private and writable outside the - destinations. -- Symlinks are explicitly rejected as an attachment mechanism. They do not - enforce read-only access, can escape workspace containment, disclose backing - paths, make cwd and tool path behavior surprising, and do not provide a - trustworthy mount boundary for root Git, archives, Files, or cleanup. -- `editable`: take a lease, then create an inode-independent private writable - copy of each verified generation repository root at its declared destination. - Reflink/copy is allowed only when later writes cannot alter the generation or - another session. Hard-linked mutable files and writable aliases are forbidden. -- Both modes preserve nested repository administrative state allowed by the - source manifest. Only declared source destinations contain source bytes; - ancestors are empty scaffolding apart from independently valid outer state. - The attachment never exposes the bare Git mirror or moves harness HOME, - credentials, scratch, generated assets, outputs, or checkpoint control into - repository content. -- Root Git, tar/archive, Files, hydrate, and cleanup receive the immutable - destination set and use no-follow, no-cross-mount traversal. They explicitly - exclude the destination paths rather than relying on the mounts being - read-only. -- Continuation reuses the exact lease and attachment map. An editable - continuation sees its private mutations; a read-only continuation restores - writable outer state first, then reattaches the same immutable generation - roots and its own session-local state. ## Implementation phases -Each phase ends with observable proof. Source inspection or mock-forwarding -assertions are not sufficient. +Every phase ends in observable behavior through the real boundary. Source-text +inspection and mock forwarding are not sufficient proof. -### Phase 1: Pin the baseline and record stock behavior +### Phase 1: Rename the downstream and pin the upstream baseline -**Outcome:** the unchanged baseline is reproducible and the fork has regression -proof for the lifecycle being extended. +**Outcome:** AllAgents Gateway has stable distribution identity and a recorded, +reproducible upstream relationship. Work: -1. Pin the initial examined baseline to HarnessRouter `v0.25.4`, commit - `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`, and UHP `2026-09-12`. Before - implementation, record the exact release baseline actually selected; move it - only in a standalone synchronization change. -2. Preserve `HarnessRouter/harnessrouter` as the upstream remote and - `allagentsdev/harnessrouter` as the existing fork. -3. Pin base image, OS packages, Git and OCI libraries/tools, Codex, OMP, - Promptfoo, lockfiles, and CI actions. Build the unchanged image first. -4. Instrument focused characterization scenarios for fresh empty-root Git - hydration, every pre-turn workspace write, attached-file/harness-asset - ordering, root-Git commands and excludes, produced-file walks, checkpoint Git - mutation, archive creation/restoration, continuation clearing, cancellation, - TTL cleanup, deletion, and restart. Record path, ordering, symlink policy, and - whether each operation crosses a filesystem mount. -5. Capture stock UHP request/stream/error behavior for requests without - `metadata.workspace`; these traces become compatibility fixtures. -6. Record the precise gateway/session/hydrate/runner/checkpoint/files call path - and the concrete root Git, tar/archive, file-walk, and cleanup entry points - that must receive source-destination exclusions. Do not add a generic - extension framework. +1. Record HarnessRouter `v0.25.4`, commit + `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`, and UHP `2026-09-12` as the + initially examined fork point. The implementation PR MUST record the exact + selected baseline and move it only in a standalone synchronization change. +2. Rename downstream repository/package references from + `allagentsdev/harnessrouter` to `allagentsdev/allagents-gateway`, the image to + `ghcr.io/allagentsdev/allagents-gateway`, the service to `allagents-gateway`, + and user-facing product text to AllAgents Gateway. +3. Preserve `HarnessRouter/harnessrouter` as upstream, the upstream license and + notices, copyright/attribution, commit history where available, and a durable + fork-point record. Configure origin as `allagentsdev/allagents-gateway`. +4. Define selective upstream intake: inspect each upstream diff, preserve the + downstream schema/security/lifecycle contract, and run stock plus downstream + gates before accepting it. Do not mirror upstream blindly. +5. Pin base image, OS packages, Git and OCI libraries/tools, Codex, OMP, + Promptfoo, lockfiles, and CI actions. Build the unchanged renamed baseline. +6. Characterize fresh root-Git hydration, pre-turn writes, Files collection, + checkpoint Git mutation/archive/restore, continuation clearing, + cancellation, TTL, deletion, cleanup, restart, and provider routing. +7. Save stock UHP traces for requests without `metadata.workspace`. Identify the + exact branch point where workspace-backed requests stop using root-Git change + collection while stock requests remain unchanged. Exit proof: -- a clean checkout builds the unchanged pinned image; -- characterization runs demonstrate all seven inventory facts; and -- a stock request completes through each supported existing route with no - workspace-specific state. +- repository, image, service, and product surfaces use only the new downstream + names while attribution and the upstream remote/fork point remain intact; +- a clean checkout builds the pinned image; and +- stock requests complete with byte-for-byte compatible status/stream fixtures + and no workspace-specific state. -### Phase 2: Add request validation and immutable session binding +### Phase 2: Implement the ordered closed request schema -**Outcome:** the gateway accepts the exact first-turn descriptor and binds it to -the existing session transition without changing stock requests. - -Primary surfaces are the existing UHP request handling/session resolution in -`gateway/app.py`, the existing gateway-to-runner turn envelope, existing session -persistence, focused integration tests, changelog, and extension documentation. +**Outcome:** request validation decides ownership and collisions before source or +session side effects. Work: -1. Parse only `metadata.workspace`; keep unrelated metadata behavior unchanged. -2. Apply metadata byte, nesting, list-count, and string-length bounds before - session allocation or source work. -3. Strictly validate the closed union, snake_case names, access, retention, - repository-mode destinations, working-directory syntax, direct `sha256:` - digest syntax, request-decidable input/asset/reserved-path collisions, and - the rule that `read_only` inputs may target only paths outside effective - source destinations. Snapshot-root validation occurs at verified-plan - resolution because the request does not carry those destinations. -4. Authorize persistent retention before source resolution or generation lookup. -5. Canonically serialize the effective descriptor with the default - `retention: "session"` and calculate its digest. -6. Extend the existing new-session transition with pending/ready workspace - binding states. Do not add another response or session identity. -7. Reject the descriptor on every reused session path before hydration. On a - workspace-bound continuation, derive source/access/retention/cwd/harness from - stored state and reject a supplied harness mismatch through existing session - rules. -8. Preserve idempotency: the owning initial request performs one binding; a - duplicate receives the same response/session result and cannot claim another - generation or attachment. -9. Map validation, authorization, reuse, and binding failures into bounded UHP - errors without internal paths or secret/catalog details. -10. Add public provenance only after attachment is ready. Retrieval/replay uses - stored provenance rather than resolving it again. - -Proof includes valid descriptors for both source kinds; repository request -counts `0`, `1`, `128`, and `129`; multiple repository entries; rejection of -request-declared destination `.`, normalized aliases, equality, ancestor -overlap, and request-decidable input/asset/reserved-path collisions; acceptance -of read-only outer inputs and rejection of read-only repository-source inputs; -default retention; authorized and unauthorized persistence; invalid unions, -fields, digests, paths, and destinations; workspace injection on both -continuation mechanisms; idempotent duplicates; harness mismatch; exact -response-schema fixtures for both provenance variants; and a byte-for-byte -stock trace for requests without the descriptor. Snapshot root counts, -destinations, files outside roots, and snapshot-input collisions are proved in -the OCI phase after verified workspace-manifest resolution. - -### Phase 3: Add generation attachment and mount-aware workspace lifecycle - -**Outcome:** one runner seam publishes immutable generations and attaches only -their declared source roots inside the existing private writable workspace. - -Primary surfaces are existing runner/session hydrate code in `runner/server.py`, -the gateway transport, root Git initialization and cursor code, checkpoint -archive/restore, Files walking, startup reconciliation, and cleanup scheduling. +1. Parse only `metadata.workspace` and reject obsolete schema names. +2. Apply metadata byte, nesting, list, string, digest, URL, and path bounds during + parsing. +3. Validate the closed `sources` array for counts `1..128`, entry-kind fields, + canonical Git HTTPS URLs, OCI direct digests, required destinations, global + pairwise non-overlap, and working-directory syntax. +4. Normalize UHP inputs, generated assets, reserved paths, credentials, caches, + attachment/checkpoint locations, and source destinations in one request-level + ownership validator. Reject every collision at or below a destination and + unsafe ancestor before network or cache work, regardless of access mode. +5. Authorize persistent retention before source lookup. Canonically serialize + the descriptor with defaults and calculate its digest. +6. Extend the existing new-session transition with pending and ready workspace + binding states. Reject metadata on every reuse/continuation path before + hydration. +7. Preserve initial-request idempotency and bounded UHP error details. Commit + public provenance only after atomic attachment readiness. + +Exit proof covers zero, one, 128, and 129 sources; all-Git, all-OCI, and mixed +arrays; repeated component identity at different destinations; ordering; +unknown/obsolete/wrong-kind fields; normalized root/equal/ancestor overlap; +input/asset/reserved collisions; outer inputs; URL/digest/path bounds; +retention authorization; both continuation mechanisms; idempotent duplicates; +harness mismatch; and an unchanged stock trace. Network and component counters +MUST remain zero for every request-decidable rejection. + +### Phase 3: Add canonical manifests and manifest-based collection + +**Outcome:** workspace-backed correctness depends only on canonical filesystem +state, never Git state. Work: -1. Add one internal `resolve -> claim/reuse -> materialize -> verify -> publish -> - attach` pipeline selected by the closed source union. It is not a public API - or plugin registry. -2. Preserve fresh allocation of the existing private writable workspace and its - stock-like root Git/bookkeeping. Thread one immutable destination set through - root Git, produced-file walks, checkpoint tar/archive, hydrate clearing, - deletion, and cleanup. Each operation must use explicit path excludes plus - no-follow/no-cross-mount traversal; a read-only mount is not itself an - adequate traversal guard. -3. Persist mirror refresh/snapshot state, generation claims, staging, - publication, session references, attachment manifests, per-root mount - evidence, editable-copy identity, and attachment-ready transitions using the - runner's current durable state and recovery ordering. -4. Build and verify the canonical source manifest: 1–128 non-root pairwise - non-overlapping destinations; normalized relative path, type, mode, - size/content identity, link target, repository ownership, immutable - generation-root identity, and optional normalized Git-history declaration. -5. Reject mountpoints or ancestors that collide with restored outer files, - symlinks, root Git state, UHP inputs, generated assets, or another root. Create - only empty non-link destination directories and required ancestor scaffolding. -6. For `read_only`, create namespace-confined per-session bind mounts from exact - generation repository roots. Enforce read-only, `nodev`, and `nosuid`, preserve - executable file bits, retain no writable backing descriptor/alias, and fail - closed on any mount or remount error. Never use symlinks or overlay copy-up. -7. For `editable`, create per-root inode-independent private writable copies. - Apply source-targeting inputs only after copies exist. Apply allowed outer - inputs and generated assets in the writable workspace before final baselines. -8. Adapt checkpointing so the outer archive explicitly excludes every source - destination and contains no source bytes. Store attachment manifest/evidence - separately. Checkpoint editable roots through per-root collectors; never let - the outer tar traverse them. Root Git may retain stock-like bookkeeping for - outer paths but must exclude all source destinations from index, commit, - status, diff, and cleanup. -9. Implement the exact continuation order specified above: validate binding, - unmount stale roots, hydrate/restore outer state, validate empty non-link - mountpoints and collisions, reattach exact read-only roots or restore exact - editable roots, verify identities/protection, rebuild cursors, then start the - harness. No source endpoint or replacement generation is allowed. -10. Unmount all source destinations before hydrate, archive restore, deletion, or - cleanup. Partial mount sets unwind in reverse order. Mount absence, - attachment evidence, references, private copies, and cleanup marks reconcile - idempotently after restart; uncertainty quarantines and fails closed. -11. Reuse existing user/sandbox isolation, process groups, timeouts, - cancellation, TTL, deletion, and cleanup. Source workers inherit - cancellation and stop descendants before terminal acknowledgement. -12. Expose separate bounded counters/events for ref resolution, remote pack - acquisition, generation build/publication/hit, per-root mount/unmount, - mount verification/failure, no-cross-mount exclusions, outer checkpoint, - editable copy/checkpoint, continuation reconciliation, quarantine, and - cleanup. Do not log credentials, origins, backing paths, or raw tool output. - -Proof mounts deterministic Git and OCI generations through the real runner. Two -read-only sessions have different writable workspace roots and independently -writable outer inputs, generated assets, `.harness`, HOME, and outputs, but -`stat`/filesystem evidence shows their matching source paths bind the same -generation inodes with no clone, extraction, materialization, hard link, -symlink, or tree copy. Writes by path, cwd, rename, link, descriptor, and -alternate alias fail with the filesystem's read-only error and leave the -generation and sibling unchanged. Two editable sessions have independent source -inodes and mutations. - -Checkpoint proof mutates outer state, creates a source sentinel larger than the -archive, and shows the archive contains the outer mutation and attachment -manifest but neither sentinel nor any source byte/path. Instrumented root Git, -tar/archive, Files, hydrate, and cleanup walkers observe zero entries below mount -destinations. Continuation first restores that outer mutation, validates empty -mountpoints, then reattaches the exact recorded roots and preserves executable -bits. Restart at every persisted mount/unmount/checkpoint transition is -idempotent; failed mounts expose no partial source. A stock session still uses -its unchanged root Git/checkpoint path. -### Phase 4: Implement repository composition with mandatory depth-2 acquisition - -**Outcome:** repository mode deterministically builds one generation from one or -more repositories while retaining bounded recent Git history. +1. Implement canonical streaming manifest creation and comparison for regular + files, binary bytes, directories, executable/mode semantics, symlinks, and + supported hardlinks. +2. Define deterministic ordering/serialization and content-digest algorithms. + Enforce no-follow traversal, ownership boundaries, source and aggregate + limits, and cancellation. +3. Publish one immutable baseline manifest with every component. Create and + persist the outer baseline only after initial outer inputs/assets exist and + before the first harness process. +4. Route workspace-backed Files/evaluation collection through final-manifest + comparison for the outer workspace and every source. Remove root/nested Git + commits, indexes, status/diff, ignore behavior, and rename detection from the + correctness path. Leave the stock collector untouched for requests without + workspace metadata. +5. Normalize ownership and exclude `.git`, runner state, credentials, caches, + attachment evidence, checkpoint metadata, harness-private ephemeral state, + and component-store paths. +6. Persist baseline identities across checkpoint/continuation and fail closed on + missing or mismatched baseline evidence. + +Exit proof mutates outer, Git, and Git-free OCI trees and observes identical +add/modify/delete/mode/symlink/binary results. It changes Git index, commits, +ignore files, and rename heuristics without changing final-tree results. It +proves delete-plus-add is accepted as a rename representation, initial trees are +not reported, exclusions never leak, read-only trees remain equal, and a stock +request still uses unchanged root-Git behavior. + +### Phase 4: Add independent component storage and atomic composition + +**Outcome:** components singleflight and cache independently, while consumers see +all requested roots or none. Work: -1. Validate each caller URL under the fixed deployment egress policy. Callers may - not supply credentials, proxy configuration, Git config, or transport - options. Re-authorize redirects and resolved addresses; isolate Git config and - disable interactive helpers, hooks, filters, alternate protocols, submodules, - and LFS hydration. -2. Resolve an omitted ref through the advertised symbolic default. Resolve an - explicit advertised branch or tag, peel annotated tags as required, and - record the exact commit before fetch. Reject ambiguous, missing, unsupported, - or non-commit targets. -3. Maintain one operator-only bare shallow acquisition mirror per canonical URL. - Serialize every write for that repository through the same mirror and - singleflight concurrent refreshes for the same normalized ref request. -4. On a mirror miss, fetch exactly depth 2 with `--depth=2` into a private - bounded refresh area, verify it, and atomically import its pack/object and - shallow-boundary state. Fetch only the selected advertised branch/tag path. - Do not retry with a larger depth, `--unshallow`, full clone, arbitrary - object-ID fetch, another ref, or another source mode. -5. Verify the fetched tip/peeled commit exactly equals the commit observed during - resolution. A ref movement race fails the refresh rather than caching or - binding different bytes. -6. Publish an immutable commit snapshot inside the mirror cache. A cache hit for - the exact canonical URL, commit, depth, and policy performs no clone or remote - pack transfer. No session can access the mirror path or a writable mirror file - descriptor. -7. Export the selected cached commit into generation staging as a self-contained - repository with no alternates or writable link to the mirror. Preserve - `.git/shallow` and sufficient normalized administrative state for recent - offline `git log`, parent inspection, and diff. -8. For a merge tip, preserve both fetched parent edges at depth 2 and validate - the shallow boundary rather than flattening the merge. -9. Reject a server that cannot satisfy the bounded shallow fetch. Return a coded - source error and do not deepen, full-clone, strip history, or fall back to OCI. -10. Check out the exact detached commit under each declared non-root destination. - Reject destination `.`, overlap, submodule gitlinks, and LFS pointer-backed - content rather than fetching them. -11. Enforce destination shape and non-overlap before network work, then build all - repositories into one staging tree. No repository may create source outside - its destination or add undeclared root files; ancestors are scaffolding only. -12. Validate each repository's `HEAD`, index/worktree equality at publication, - shallow metadata, closed refs/config, object reachability for the retained - depth, file modes, links, bytes, inodes, and absence of credentials/remotes - that would cause later network use. -13. Compute exact generation identity from canonical URL identity, resolved - commit, destination, fixed depth `2`, acquisition-policy revision, and layout - policy. Preserve requested URL/ref separately as provenance. -14. Clean incomplete mirror refresh, repository staging, and descendants on - error, timeout, cancellation, lost claim, or restart without invalidating a - previously verified commit snapshot or referenced generation. - -Behavior proof covers rejection of root destination `.`; one repository at a -non-root top-level destination; several sibling/nested-path destinations; the -same URL at different refs and destinations; omitted default, branch, -lightweight tag, annotated tag, and moving-ref rejection; pairwise non-overlap; -depth exactly 2; `.git/shallow`; two-entry recent offline history; merge-tip -parents and diff semantics; detached exact commit; no network during attached -`git log`; submodule/LFS rejection; unsupported shallow server with no fallback; -cancellation; restart cleanup; and exact provenance. Concurrent cold requests -produce one advertised-ref refresh, one remote pack fetch, one verified mirror -snapshot, and one generation publication. After resolution confirms the same -exact identity, a second read-only request performs zero clone, pack transfer, -checkout/materialization, or tree copy and bind-mounts the same immutable -repository-root inodes while its writable outer state remains isolated. - -The release notes must state plainly that depth 2 reduces transferred history, -not the checked-out working-tree bytes. Large repositories still require the OCI -snapshot source and its release gate. - -### Phase 5: Implement mandatory OCI workspace snapshot materialization - -**Outcome:** the same generation pipeline safely restores an operator-cataloged, -digest-pinned workspace snapshot with no Git fallback. - -Deployment configuration owns a bounded snapshot catalog. Each `snapshot_name` -maps to one operator-controlled registry/repository origin, credential reference, -TLS policy, allowed media types, and resource policy. The request and public -provenance never reveal those private values. This catalog is HarnessRouter -configuration; it is not `workspace.yaml`. +1. Add durable component claim, staging, verification, atomic publication, + lease, quarantine, and cleanup states under runner-owned `/data`. +2. Key components without destination/session concerns. Build composition + identity from the ordered exact component identities and destinations; store + references rather than copied source bytes. +3. Resolve and claim independent entries concurrently within bounded worker, + network, disk, and aggregate request limits. Detach cancelled waiters without + cancelling a component still needed elsewhere. +4. Construct the outer workspace and all source attachments in an inaccessible + staging path or private mount namespace. Add one atomic publish/handoff plus + ready transition. Reverse-unwind every partial mount/copy/reference on error. +5. Implement identical read-only bind and editable copy/reflink behavior for Git + and OCI. Verify mount flags, no writable aliases, component identity, and + editable inode independence. +6. Thread the immutable destination map through Files, archive, hydrate, restore, + delete, and cleanup with explicit excludes and no-cross-mount traversal. +7. Add bounded metrics for component resolution/acquisition/hit/publication, + composition commit/hit/rollback, mount/copy, manifest creation/comparison, + checkpoint, reconciliation, quarantine, and cleanup without paths or secrets. + +Exit proof races identical and partially overlapping compositions. Each exact +component publishes at most once; a warm Git/cold OCI request reuses Git while +building only OCI; a warm OCI/cold Git request does the inverse. Ordered +composition IDs change with order/destination while component IDs stay stable. +Injected failure in the last of several roots exposes no source or runnable +workspace. Two read-only sessions share component inodes but not outer state; +two editable sessions share no mutable inode. + +### Phase 5: Implement depth-2 Git components + +**Outcome:** each Git entry produces one verified immutable component with useful +bounded recent history. Work: -1. Require a direct OCI image manifest digest. Reject tags, mutable references, - manifest indexes/lists, caller-selected repositories, and catalog/digest - mismatches. -2. Fetch the direct image manifest, config, and workspace manifest only from the - selected catalog entry. Implement bounded registry authentication and - exact-host redirect policy without exposing credentials to the harness, - session workspace, logs, response, or provenance. Do not request layers yet. -3. Verify every descriptor digest and size. Require the declared - `workspace_manifest_digest` to identify the exact workspace manifest. Before - layer acquisition, validate its 1–128 roots, non-root and non-overlap rules, - absence of source files outside roots or in ancestor scaffolding, and every - UHP input/generated-asset/reserved-path collision against the authoritative - root map. -4. Fetch referenced layers and stream decompression/extraction inside the fixed - v1 envelope: at most 64 distributable tar/gzip/zstd layers; a 4 MiB image - manifest; a 128 MiB workspace manifest; 8 GiB total compressed layer bytes; - 32 GiB expanded source bytes; 500,000 entries; 4 GiB per regular file; paths - of at most 4096 UTF-8 bytes and 128 components; and 1 MiB per PAX or extended - header. For each layer and the aggregate artifact, - `expanded_bytes / max(compressed_bytes, 1)` must not exceed `100`. Enforce - these bounds plus inode, output, and wall-time limits during streaming. - Deployment configuration may lower but cannot raise them without a contract - revision. -5. Apply OCI layers in order with correct file and opaque-directory whiteout - semantics. Whiteouts are extraction instructions and must never appear in the - published workspace. Reject malformed whiteouts and type transitions not - representable by the workspace manifest. -6. Reject absolute paths, traversal, NULs, ambiguous separators, duplicate - conflicting entries, devices, FIFOs, sockets, unsafe sparse files, and other - unsupported types. Validate every symlink and hardlink target against its - owning declared repository root; reject links into another root or the - writable outer workspace, plus escaping, dangling-required-target, - forward-link, and link-cycle cases outside the supported bounded model. -7. Validate final paths, types, modes, sizes, content digests, links, repository - roots, and destination non-overlap against the workspace manifest. Extra, - missing, or changed source-visible entries fail before publication. -8. Support tree-only repository roots and optional normalized offline Git - history. For a history-bearing root, require detached `HEAD`, exact - index/tree/worktree equality, closed object reachability and declared object - digest, bounded refs/config, and no remotes, credentials, alternates, hooks, - includes, worktrees, replace/graft state, or unsafe administrative files. - Tree-only roots must not contain undeclared `.git` state. -9. Return the same canonical manifest/generation envelope as repository mode. - Include direct image/workspace digests and snapshot name in identity and exact - provenance; exclude registry origin and credentials. -10. On any resolution, registry, digest, extraction, manifest, Git-history, - cancellation, or capacity failure, terminate descendants, remove staging, - and return the source-specific error. Never clone Git, select a different - digest, or use a stale generation as fallback. - -Proof uses a local authenticated registry and malicious fixtures for -digest/media mismatch, indexes, redirects, authentication, truncation, -compression bombs, layer limits, whiteouts and opaque whiteouts, traversal, -path/type/link attacks, devices, sparse files, cancellation, partial cleanup, -and restart. Root-map proof covers 0, 1, 128, and 129 roots; destination `.`; -equal/ancestor overlaps; source files outside roots or in ancestor scaffolding; -and a `read_only` input collision rejected after workspace-manifest verification -but before any layer request or outer workspace write. Positive fixtures cover a -tree-only workspace, multiple declared repository roots, normalized offline Git -history, offline `git log`/`git blame`/historical diff, read-only attachment, -editable copy, concurrent publication, and cache reuse with zero second-session -registry or extraction work. - -OCI implementation and this proof are required before v1 release. A passing Git -path cannot waive or defer them. - -### Phase 6: Adapt produced-file collection for nested and multiple repositories - -**Outcome:** existing Files/artifact behavior reports turn-produced changes -across composed workspaces without inventing a second file API. +1. Enforce canonical public HTTPS and the fixed egress/redirect/address policy. + Isolate Git config and disable interactive credentials, hooks, filters, + alternates, alternate protocols, submodules, and LFS hydration. +2. Resolve omitted ref through advertised symbolic default; resolve advertised + branches and lightweight/annotated tags to exact commits. Reject ambiguous, + missing, unsupported, non-commit, or moved targets. +3. Serialize one operator-only bare shallow mirror per canonical URL. Fetch the + selected advertised path with exactly `--depth=2` into bounded private state, + verify the expected commit and shallow boundary, then atomically import. +4. Export a self-contained detached checkout with normalized bounded `.git` + metadata and no writable mirror link. Preserve merge parents when available + within depth 2. Reject gitlinks and LFS pointer-backed content. +5. Compute/verify the canonical component baseline, enforce per-source and + aggregate checkout bounds, and publish independently of destination. +6. Ensure exact identity hits transfer no pack and perform no checkout, tree + copy, baseline recomputation, or publication after optional ref confirmation. + +Exit proof covers default, branch, lightweight/annotated tag, moved ref, same URL +at different refs, depth exactly 2, `.git/shallow`, recent offline log/blame/diff, +merge parents, detached exact commit, malicious redirects, unsupported shallow +server, submodule/LFS rejection, cancellation, restart, and exact provenance. +Concurrent cold requests perform one mirror refresh and component publication; +an exact hit records zero source-byte work. Release notes state that depth 2 +limits history, not working-tree bytes. + +### Phase 6: Implement OCI source-tree components + +**Outcome:** each OCI entry produces exactly one verified tree component without +Git fallback or implicit workspace roots. Work: -1. Preserve the existing root Git cursor in the writable outer workspace. - Register every source destination in root Git's internal excludes and pass the - immutable destination set to every root Git command so it never traverses a - mounted or copied source root. -2. Register source-manifest repository roots and history mode when the - attachment becomes ready. The set is immutable for the session. -3. At each turn boundary, record the outer root-Git cursor plus a cursor for each - editable declared repository root: Git `HEAD`/index/worktree state for - history-bearing roots and manifest/file identity for tree-only roots. - Read-only roots require no change cursor because the filesystem prevents - mutation. -4. Collect the union of writable outer-workspace and editable-root additions, - modifications, deletions, renames, and mode changes relative to the turn - baseline. Normalize paths, assign each path to the most specific declared - owner, deduplicate it, and preserve existing file size/count/type limits. -5. Never expose `.git` administrative files, generation-store paths, mount - internals, credentials, harness assets, or checkpoint internals as produced - files. -6. Do not report immutable or initial editable source files merely because they - arrived during first-turn composition. Report only changes after the - established source/turn baseline. -7. In `read_only`, any attempted source mutation fails at the filesystem - boundary, but additions and changes elsewhere in the writable workspace are - collected normally. No collector crosses a source mount. -8. In `editable`, source changes remain private to the session and are visible - on continuation and through the existing file and artifact APIs. -9. Preserve collection-before-checkpoint ordering. The outer checkpoint archive - explicitly excludes every source destination. Read-only sources are - reattached from the generation; editable sources use their separate private - source-root checkpoint path. Cancellation cannot publish a partial cursor or - checkpoint. - -Proof covers changes at workspace root and in every nested repository; two -repositories changed in one turn; same filename under different destinations; -add/modify/delete/rename; tree-only OCI roots; history-bearing OCI roots; Git -shallow roots; paths outside repository destinations; ignored files under the -existing policy; read-only denial; editable continuation; cancellation during -collection; restart; bounds; and absence of `.git`, credentials, generation -paths, or duplicate records. +1. Implement the bounded operator catalog and direct image-manifest resolution. + Keep registry/repository origin, credentials, TLS configuration, and redirects + server-side. +2. Fetch and verify the image manifest, config, and exact canonical source + manifest before layers. Validate relative ownership, all expected final + entries, links, modes, sizes, digests, and known per-source/aggregate limits. +3. Stream bounded layers, verify descriptors, apply whiteouts, and enforce path, + type, link, sparse-file, compression-ratio, inode, byte, output, cancellation, + and time limits. +4. Confine links to the one owning component root and reject multi-root or outer + workspace content. Match the final tree exactly to the canonical source + manifest before publication. +5. Support Git-free trees and optionally normalized bounded Git administrative + data for agent convenience. Neither form changes manifest-based evaluation. +6. Ensure exact identity hits make zero registry requests, extraction, tree copy, + baseline recomputation, or publication. + +Exit proof uses an authenticated local registry and malicious fixtures for +catalog/digest/media mismatch, indexes, redirects, authentication, truncation, +compression bombs, limits, whiteouts, traversal, path/type/link attacks, +devices, sparse files, cancellation, partial cleanup, and restart. Positive +fixtures cover Git-free and history-bearing trees, read-only and editable +attachment, two independent OCI entries, exact-hit reuse, and source-manifest +rejection before the first layer request. + +### Phase 7: Make checkpoint and continuation composition-aware + +**Outcome:** complete compositions survive turns and restarts without source +traffic or baseline loss. -### Phase 7: Wire Codex, OMP, and the existing provider gateway +Work: -**Outcome:** both supported harnesses execute in the attached existing workspace -without gaining source or long-lived provider credentials. +1. Archive the writable outer workspace with every destination excluded and + no-follow/no-cross-mount enforcement. Store no source bytes in the outer + checkpoint. +2. Preserve every editable Git or OCI root in an independent private checkpoint; + preserve immutable component references for read-only roots. Retain original + baseline identities separately from mutable final state. +3. Implement the staged continuation order: validate binding/evidence, unmount + stale roots, restore outer state, restore editable roots or exact read-only + leases, verify all roots, restore baselines, validate cwd, then atomically + publish. +4. Reconcile every durable transition after restart. Missing/corrupt evidence, + publication, checkpoint, mount, or baseline fails closed without network or + replacement component selection. +5. Make cancellation, expiry, explicit deletion, persistent retention, cleanup, + quarantine, and reference release composition-aware and idempotent. + +Exit proof checkpoints mixed compositions after mutating outer, editable Git, +and editable OCI paths. Continuation preserves every mutation and baseline, +makes zero source requests, and reports the same final-tree changes. A read-only +continuation rebinds exact component inodes. Restart and cancellation are +injected at each claim, publication, composition, mount/copy, baseline, +checkpoint, handoff, and cleanup transition. No partial workspace becomes ready. + +### Phase 8: Wire harnesses and provider boundary + +**Outcome:** Codex and OMP run in the atomically composed workspace without +source or long-lived provider credentials. Work: -1. Install exact pinned Codex and OMP releases and enable only required release - backends with `HR_BACKENDS=codex,omp`. -2. Define stable downstream custom harnesses such as `allagents-codex` and - `allagents-omp` with explicit model allowlists. -3. Configure two logical connections to the same external OAuth-to-OpenAI- - compatible gateway: Responses for Codex and OpenAI Chat Completions for OMP. - Each harness has exactly its matching connection and no fallback. -4. Retain brokered sandbox credentials. The long-lived external gateway key - remains server-side; the harness receives only the existing short-lived, - scoped turn credential and loopback route. -5. Start each harness in the validated `working_directory` while keeping HOME, - skills, scratch, conversation, credential projection, and checkpoint control - in their current session-isolated locations. -6. Reject unsupported models and any attempt to place provider URL, key, - transport, route, registry data, or source credentials in the request. -7. Remove ephemeral OMP/Codex provider configuration before checkpoint and file - collection using existing broker lifecycle hooks. -8. Provider failure returns the existing normalized failure and does not change - source, generation, access, attachment, harness, connection, or protocol. - -Proof runs both harnesses against Git and OCI sources, at root and a nested -working directory, in read-only and editable modes where applicable. It verifies -that caller and provider credentials, registry credentials, broker tokens, and -private origins are absent from process output, session/checkpoint files, -produced files, artifacts, logs, and public metadata. An invalid provider -credential or unsupported model causes no route fallback. A stock request still -uses its original workspace path and provider behavior. - -### Phase 8: Exercise lifecycle, restart, cancellation, and cleanup - -**Outcome:** source generations and attachments follow the existing HarnessRouter -session lifecycle under failures and restarts. - -Work and proof: - -1. Cancel during Git advertisement/fetch/checkout, OCI manifest/blob transfer, - decompression/extraction, generation wait, editable copy, harness execution, - checkpoint, and produced-file collection. Reap descendants before terminal - acknowledgement and remove only the cancelled request's incomplete state. -2. Cancel one waiter on a shared generation build while another continues. If no - waiter remains, cancel the bounded builder. At most one complete publication - can survive. -3. Restart after every durable transition: pending descriptor, source resolved, - claim held, staging populated, generation published, session reference - created, read-only attached, editable copy started/completed, attachment ready, - turn active, checkpoint written, cleanup marked, and reference released. -4. Before readiness, reconcile incomplete staging and copies, publication - evidence, references, read-only mounts, editable ownership, session binding, - and cleanup marks. Never attach an uncertain generation or expose an editable - copy to another session. -5. Prove a continuation after restart restores the exact generation/access/cwd; - editable mutations persist, read-only remains immutable, and no source - endpoint is contacted. -6. Prove session expiry and explicit deletion first make the session unavailable, - then release the mount/private copy and generation reference exactly once. - Persistent retention survives ordinary session-idle cleanup until authorized - deletion. -7. A cleanup failure quarantines the path and keeps it unavailable/accounted. - Retrying cleanup is confined, no-follow, and idempotent. -8. Generation cleanup removes only unreferenced complete publications under - bounded operator policy. Active session references and authorized persistent - sessions prevent removal. - -### Phase 9: Add direct large-OCI Promptfoo E2E - -**Outcome:** the built image proves the consumer-visible contract, mandatory -large-workspace behavior, and second-session generation reuse. - -Promptfoo calls the built image directly at the existing UHP Responses endpoint -with a HarnessRouter caller API key. There is no adapter service or alternate -execution protocol. +1. Pin Codex and OMP and enable only required backends with + `HR_BACKENDS=codex,omp` unless the renamed downstream configuration surface + adopts an equivalent key in the same change. +2. Define stable `allagents-codex` and `allagents-omp` harnesses with explicit + model allowlists. +3. Use the same external provider gateway through one Responses connection for + Codex and one Chat Completions connection for OMP, with no fallback. +4. Retain server-side long-lived credentials and existing short-lived scoped + turn credentials/loopback route. Remove ephemeral provider configuration + before checkpoint and collection. +5. Start each harness only after atomic attachment and from the validated working + directory. Keep HOME, skills, scratch, conversation, credentials, generated + assets, and checkpoint control in private outer locations. +6. Reject caller provider, route, model, transport, source credential, and + registry overrides. Provider failure MUST NOT alter composition binding. + +Exit proof runs both harnesses against all-Git, all-OCI, and mixed compositions +at outer and nested working directories in both access modes. It verifies that +caller, Git, registry, broker, and provider secrets are absent from output, +checkpoints, reported files, artifacts, logs, and public metadata. Unsupported +models and bad credentials produce no route fallback. Stock requests retain +their original provider path. + +### Phase 9: Run direct release-blocking E2E + +**Outcome:** the built image proves the consumer-visible component, composition, +manifest, lifecycle, and compatibility contracts. + +Promptfoo MUST call the built image directly at the existing UHP Responses +endpoint with an AllAgents Gateway caller API key. There is no adapter service or +alternate execution protocol. Release-blocking scenarios: -1. **Large OCI fixture:** publish a deterministic workspace snapshot with at - least 2 GiB of expanded source bytes and 100,000 source-visible filesystem - entries. Falling below either floor fails the gate. Include multiple - repository roots, a late-path sentinel, a nested working directory, and - normalized offline history. The release record publishes actual compressed - and expanded bytes, file/inode count, layer count, expansion ratio, and - manifest digests so “large” is measured rather than asserted. -2. **First session:** start Codex read-only from the large snapshot, read the - sentinel, run recent offline Git history, and complete from the nested working - directory. Registry counters and runner metrics must show one bounded download, - extraction, verification, and atomic generation publication. -3. **Second session reuse:** start OMP read-only with the identical snapshot - identity but a different session and harness. It must report the same - generation/manifest identity, share verified generation bytes while retaining - isolated session state, and complete with zero additional registry manifest or - blob requests, zero extraction, and zero publication. This is the required - second-session cache-reuse proof. -4. **Editable reuse:** start an editable session from the same cached generation. - It performs no registry/extraction work, receives inode-independent private - bytes, mutates a file, continues without resending workspace metadata, and - leaves the read-only sessions and generation unchanged. -5. **Repository matrix:** run depth-2 Git scenarios for default ref, branch, tag, - merge tip, several repositories, nested working directory, recent history, - and produced files across destinations. Concurrent cold requests must record - one serialized/singleflight mirror refresh, one pack fetch, and one generation - publication. A second read-only request with the same resolved source identity - must record zero clone, pack transfer, checkout/materialization, and tree copy; - it attaches the same immutable generation bytes while its session, runtime, - conversation, outputs, and cleanup remain isolated. -6. **Access/lifecycle matrix:** prove that read-only source writes fail while - root-level generated assets, an allowed UHP input, and an agent-created output - remain writable and are checkpointed. Prove root Git, Files, archive, hydrate, - and cleanup do not traverse source mounts. Cover editable isolation, session - and authorized persistent retention, continuation, restart, expiry, explicit - deletion, cleanup retry, and missing/corrupt attachment with no - rematerialization. -7. **Failure matrix:** cover malformed descriptor, root or overlapping - destination, bad working directory, unauthorized persistence, a `read_only` - input equal to or beneath a source destination, mountpoint/input/asset - collision, symlink attachment attempt, mount/remount failure, shallow-fetch - refusal, moving ref, submodule/LFS, wrong OCI digest, workspace-manifest - mismatch, extraction limit, path/link/type attack, cancellation in both - source modes, provider failure, and reused-session descriptor injection. - Assert there is no Git/OCI/provider fallback. -8. **Provider matrix:** complete Codex/Responses and OMP/Chat Completions through - the one external provider gateway; reject route/model override; scan retained - and public surfaces for caller, source, registry, broker, and provider secrets. -9. **Stock matrix:** replay baseline requests without workspace metadata and - compare status, stream ordering, checkpoint/files behavior, and provider route - with the characterization fixtures. - -Reports retain only sanitized request/result assertions, image/source digests, -resource measurements, and source/generation counters. They never retain -credentials, private registry origins, provider traffic, internal paths, or -session volume contents. - -### Phase 10: Publish a digest-pinned release - -**Outcome:** a clean operator can deploy the exact tested image and reproduce the -Git/OCI contract. +1. **Large OCI component:** publish a deterministic source tree with at least + 2 GiB expanded bytes and 100,000 source-visible entries, a late-path sentinel, + a nested working directory, and optional normalized offline history. Record + actual compressed/expanded bytes, entry count, layers, ratio, and digests. +2. **Multiple OCI:** compose at least two independent OCI entries at sibling + destinations. Codex MUST read both; a second session MUST reuse both with zero + registry, extraction, copy, baseline, or publication work. +3. **Mixed source:** compose at least two depth-2 Git and two OCI entries. OMP + MUST read all four, work from a nested directory, and report ordered + provenance and composition identity. Reject an overlap before network access. + Warm only one component, then prove the cold components build independently + and atomic visibility waits for every entry. +4. **Editable OCI:** run a bug-fix evaluation with `access: "editable"`, mutate + an OCI file, add a binary, delete another path, change executable mode, and + retarget a symlink. Manifest comparison MUST report the exact final state; + continuation MUST preserve it; cache and sibling sessions MUST remain + unchanged. +5. **Editable mixed composition:** mutate Git, OCI, and outer paths in one turn. + Prove ownership, deduplication, exclusion, binary/mode/link semantics, and + final-tree equality without Git status/index/commit dependence. +6. **Git matrix:** cover default/branch/tag/merge, depth 2, several entries, + repeated identity at distinct destinations, recent offline history, cache + singleflight, and zero source-byte work on exact hit. +7. **Atomicity and access:** delay/fail the last component and observe no partial + Files/harness/workspace visibility. Prove read-only mutation denial, shared + immutable inodes, editable independent inodes, writable outer assets, and + pre-network input/asset collision rejection for Git and OCI destinations. +8. **Lifecycle:** cover continuation, restart, cancellation during each + acquisition kind and composition wait, expiry, authorized persistence, + explicit deletion, cleanup retry, corrupt evidence, and no rematerialization. +9. **Failure/security:** cover malformed/obsolete descriptors, 0/129 sources, + root/overlap paths, per-source and aggregate limits, unsafe links/types, + digest mismatch, depth refusal, provider failure, reused-session injection, + and absence of Git/OCI/provider fallback or secret leakage. +10. **Stock compatibility:** replay requests without workspace metadata and + compare status, stream ordering, root-Git Files/checkpoint behavior, + continuation, cancellation, and provider route with Phase 1 fixtures. + +Reports MUST retain only sanitized assertions, image/source digests, measured +resource values, and component/composition counters. They MUST NOT retain +credentials, private origins, provider traffic, internal paths, or volume +contents. + +### Phase 10: Publish the digest-pinned AllAgents Gateway release + +**Outcome:** a clean operator can deploy the exact tested downstream image and +reproduce the Git/OCI/mixed contract. Work: -1. Review the fork diff against its exact upstream tag/commit. The workspace - changes must be limited to request/session binding, the existing hydrate and - attachment seam, immutable generation storage, Git/OCI materialization, - produced-file adaptation, focused lifecycle/configuration/docs, custom - harness/provider wiring, E2E, and release automation. -2. Run the complete pinned upstream UHP conformance suite without exclusions and - the focused fork coverage against the image candidate. -3. Run the Phase 9 Promptfoo matrix against that exact candidate digest. +1. Review the downstream diff from the recorded upstream fork point, including + license/attribution, renamed distribution surfaces, selective upstream + changes, request/session binding, component caches, atomic composition, + manifests, lifecycle, provider wiring, E2E, and release automation. +2. Run pinned upstream UHP conformance without exclusions and all focused + downstream gates against one image candidate. +3. Run Phase 9 against that exact candidate digest. 4. Build `linux/amd64` from pinned inputs, attach standard SBOM and provenance, and publish - `ghcr.io/allagentsdev/harnessrouter:-allagents.`. + `ghcr.io/allagentsdev/allagents-gateway:-allagents.`. 5. Read back and deploy by manifest digest, for example - `ghcr.io/allagentsdev/harnessrouter:v0.25.4-allagents.1@sha256:`. -6. Verify a fresh-volume deployment and a same-volume restart with both - harnesses, Git and large OCI, cache reuse, continuation, cancellation, - produced files, expiry, persistent deletion, generation cleanup, and stock - requests. -7. Record upstream tag/commit, downstream source commit, UHP release, Codex/OMP/ - Promptfoo versions, base and package pins, Git depth/policy revision, OCI - validation-policy revision, image digest, source fixture digests, SBOM, - provenance, and E2E report identities. -8. Block release on any missing OCI implementation/evidence, large-workspace - failure, second-session rebuild, writable read-only alias, editable inode - sharing, continuation rematerialization, source fallback, credential leak, - stock regression, or unpinned input. - -Only after the downstream digest and evidence are available may maintainers -prepare an upstream issue or proposal for the generic contract and runner seams. -That work cites measured Git/OCI behavior, cache reuse, security failures, and -stock compatibility. Upstream discussion, acceptance, UEP timing, and merge are -outside the release critical path; a later upstream implementation replaces the -fork delta only after equivalent behavior passes the same gates. + `ghcr.io/allagentsdev/allagents-gateway:v0.25.4-allagents.1@sha256:`, + under service name `allagents-gateway`. +6. Verify fresh-volume and same-volume restart with both harnesses, Git, multiple + OCI, mixed composition, cache reuse, editable mutation, continuation, + cancellation, cleanup, manifests, and stock requests. +7. Record upstream tag/commit and fork point, downstream commit, license/notices, + UHP and harness versions, base/package pins, Git/OCI/manifest/materializer + policy revisions, image digest, fixtures, SBOM, provenance, and E2E reports. +8. Block release on any old downstream name in a public distribution surface, + missing attribution, obsolete accepted schema, partial composition exposure, + cache mutation, incorrect editable isolation, Git-based workspace evaluation, + continuation source traffic, credential leak, stock regression, or unpinned + input. + +Only after downstream evidence exists MAY maintainers propose generic seams +upstream. Upstream issue, UEP, acceptance, merge, and release timing remain +outside the downstream critical path. ## Failure contract -Workspace failures use bounded stable detail codes under the existing UHP error -shape. Exact HTTP mapping follows existing HarnessRouter conventions, but these -observable distinctions must remain: +Workspace failures MUST use bounded stable detail codes under the existing UHP +error shape. Exact HTTP mapping follows pinned upstream conventions. -| Condition | Required behavior | +| Condition | Required behavior and timing | |---|---| -| Invalid shape, field, digest, destination, or working directory | Reject before session source work | -| Unauthorized `persistent` retention | Reject before source lookup/network work | -| Workspace metadata on a reused session | Reject without changing the existing binding or TTL | -| `read_only` input equal to or beneath a source destination | Repository mode rejects before network work; snapshot mode rejects after verified root-map resolution but before layers, outer writes, attachment, or harness execution; outer inputs remain valid | -| Repository ref missing/ambiguous/moved | Fail repository resolution; no alternate ref/source | -| Server cannot satisfy depth-2 fetch | Fail repository acquisition; no deepen/full clone/OCI fallback | -| Submodule or LFS content | Fail repository validation; no helper execution | -| OCI catalog name/digest/media mismatch | Fail snapshot resolution; do not reveal catalog origin | -| Layer digest, whiteout, path, link, type, limit, or workspace-manifest failure | Fail extraction/validation; no publication or Git fallback | -| Generation capacity unavailable | Return bounded retryable capacity failure before unbounded work | -| Source timeout or cancellation | Stop descendants, detach waiter, clean incomplete private state | -| Attachment evidence missing/corrupt on continuation | Fail closed; no source access or replacement generation | -| Read-only write attempt | Filesystem denial; generation and sibling sessions unchanged | -| Provider failure | Existing provider error; source and route binding unchanged | -| Cleanup failure | Session/path remains unavailable and accounted for retry | - -Failures before attachment-ready expose no generation or source provenance. -Failures after attachment-ready may return already committed public provenance, -but never internal paths, registry origins, credentials, raw tool stderr, or -private network details. +| Invalid/obsolete shape, count, field, URL, digest, destination, overlap, or cwd syntax | Reject before cache lookup, network, session workspace write, or component claim | +| Input/generated/reserved-path collision at or below any destination | Reject before network for Git and OCI and for both access modes; outer paths remain valid | +| Unauthorized `persistent` retention | Reject before cache lookup or network | +| Workspace metadata on reused session | Reject without changing binding, checkpoint, or TTL | +| Git ref missing, ambiguous, non-commit, or moved | Fail that component; no alternate ref or source | +| Server cannot satisfy depth-2 Git fetch | Fail that component; no deepen, full clone, history stripping, or OCI fallback | +| Submodule or LFS-backed content | Fail Git validation; no helper execution | +| OCI catalog, manifest, media, or digest mismatch | Fail that component without revealing catalog origin | +| Invalid canonical OCI source manifest | Fail before any layer request | +| OCI layer digest, whiteout, path, link, type, sparse, ratio, or final-manifest mismatch | Fail extraction; no publication or Git fallback | +| Per-source limit exceeded | Stop that component, clean staging, and fail the composition | +| Request aggregate limit/capacity exceeded | Cancel/detach request-local work, preserve valid shared work, attach none | +| One component fails after siblings succeed | Publish no composition attachment; release request-local references; retain only independently valid shared cache entries | +| Cancellation/timeout during component or composition work | Stop unneeded descendants, detach waiter, clean private staging, expose no partial roots | +| Mount/copy/atomic handoff failure | Reverse-unwind all roots; session never becomes ready | +| Missing/corrupt attachment, checkpoint, or baseline on continuation | Fail closed; no source access, replacement component, or empty root | +| Read-only mutation attempt or changed final manifest | Filesystem denial/integrity failure; cache and siblings unchanged | +| Manifest collection limit or unsafe traversal | Fail collection/checkpoint; do not publish an incomplete result/baseline | +| Provider failure | Return existing provider error; source/composition/route binding remains unchanged | +| Cleanup failure | Keep session/path unavailable, quarantined, and accounted for idempotent retry | + +Failures before attachment-ready MUST expose no component/composition provenance. +Every failure after readiness MUST return the complete stored sanitized +`metadata.workspace` object while excluding internal paths, registry origins, +credentials, raw tool stderr, and private network details. ## Verification matrix | Gate | Observable evidence | |---|---| -| Stock compatibility | Requests without `metadata.workspace` match pinned status, stream, hydrate/checkpoint, files, cancellation, and provider behavior | -| Request/session binding | Both source variants bind only on a new session; continuation omits metadata and reuses the exact binding | -| Multiple repositories | Pairwise non-overlapping destinations compose correctly; overlap and source escape fail before acquisition | -| Git depth policy | Advertisement/default/branch/tag resolve exactly; fetch uses depth 2; tip matches; `.git/shallow`, recent history, and merge parents work offline; unsupported shallow fetch has no fallback | -| OCI integrity | Direct manifest and workspace-manifest digests, fixed `100` per-layer and aggregate expansion-ratio ceilings, bounded layers, whiteouts, paths, links, types, tree manifest, and optional normalized Git history all verify | -| Mandatory large OCI | A fixture with at least 2 GiB expanded source and 100,000 source-visible entries completes for both harnesses from the digest-pinned image; no Git acquisition occurs | -| Generation publication | Concurrent identical Git or OCI identities singleflight to one acquisition and one publication; failed/partial mirror snapshots or generations never attach | -| Git mirror/generation reuse | One operator-only bare shallow mirror exists per canonical repository identity; a second resolved-identity hit has zero clone/pack transfer/checkout/tree copy and directly attaches the same immutable generation | -| OCI generation reuse | A second exact-digest request has zero registry request/extraction/tree copy/publication and directly attaches the same immutable generation | -| Read-only sharing | Every matching session bind-mounts the same verified generation roots with zero copy; source writes and aliases fail while outer workspace paths remain private and writable | -| Mount boundary | Destinations are non-root directories, symlinks are never attachments, mount flags/identity verify, and root Git, Files, archive, hydrate, and cleanup never cross a source mount | -| Read-only checkpoint | The outer archive contains allowed inputs/outputs but no source bytes; continuation restores outer state, validates empty mountpoints, then reattaches the exact generation | -| Editable isolation | Private source copies share no mutable inode; one session's changes and source-root checkpoint never alter the generation or siblings | -| Working directory | Workspace root and valid directories inside or outside source roots become cwd; missing, file, traversal, and symlink escape fail before harness execution | -| Produced files | Existing API reports writable outer and editable nested-root changes once, excludes source baselines/admin/mount/secrets, survives continuation/restart, and respects bounds | -| Continuation | Exact access, retention, generation, provenance, cwd, harness, conversation, and editable mutations restore without source traffic | -| Cancellation | Git, OCI, build wait, copy, harness, checkpoint, and collection cancellation reap descendants and leave recoverable state | -| Restart | Every generation/attachment/cleanup transition reconciles before readiness; exact attachments resume or fail closed | -| Cleanup | Expiry/deletion is unavailable-first, confined, idempotent, reference-safe, and quarantine-preserving on error | -| Provider boundary | Codex Responses and OMP Chat Completions use one operator route with brokered credentials, no caller override, fallback, or retained secrets | -| Release | GHCR image, digest, SBOM, provenance, source/runtime pins, UHP conformance, and direct Promptfoo reports identify the same candidate | +| Naming/distribution | Repository is `allagentsdev/allagents-gateway`, image is `ghcr.io/allagentsdev/allagents-gateway`, service is `allagents-gateway`, and public product text says AllAgents Gateway | +| Upstream relationship | `HarnessRouter/harnessrouter`, fork point, license, notices, attribution, and selective-intake procedure are recorded and intact | +| Stock compatibility | Requests without `metadata.workspace` match pinned root-Git hydrate/checkpoint/Files, stream, continuation, cancellation, and provider behavior | +| Closed ordered schema | `sources` accepts 1..128 Git/OCI/mixed entries; obsolete singular/bundle forms and unknown fields fail | +| Pre-network ownership | Root/equal/ancestor destinations and all input/asset/reserved collisions fail with zero source/cache/component activity | +| Component semantics | Every entry owns exactly one non-root tree; OCI never supplies a runtime or multi-root workspace | +| Git depth policy | Default/branch/tag resolve exactly; depth is 2; shallow recent history and merge parents work offline; refusal has no fallback | +| OCI integrity | Direct image and canonical source-manifest digests, pre-layer manifest validation, bounded layers, whiteouts, paths, links, types, and exact final tree verify | +| Per-source/aggregate limits | Bytes, entries, ratio, path, file, inode, time, and output limits fail at the correct component or composition boundary | +| Independent singleflight | Identical components publish once even across different compositions; unrelated entries proceed independently | +| Composition identity | Ordered exact component identities plus destinations determine the ID without copying bytes | +| Atomic composition | Late failure/cancellation exposes no subset to Files, checkpoint, provider, or harness; success reveals all roots together | +| Git cache reuse | Exact hit has zero pack acquisition, checkout, tree copy, manifest recomputation, or publication | +| OCI cache reuse | Exact hit has zero registry request, extraction, tree copy, manifest recomputation, or publication | +| Read-only sharing | Matching sessions bind the same immutable component inodes with enforced flags/no aliases; outer paths remain private and writable | +| Editable isolation | Git and OCI copies/reflinks share no mutable inode; mutations survive continuation and cannot affect cache/siblings | +| Canonical baselines | Every component and outer workspace have persisted canonical path/type/digest/mode/link manifests | +| Manifest final diff | Add/modify/delete/mode/symlink/binary changes across outer, Git, and Git-free OCI roots reflect final-tree equality independent of Git state | +| Exclusions | `.git`, credentials, caches, runner state, attachment evidence, checkpoint metadata, and component paths never appear as reported changes | +| Checkpoint | Outer archive contains no source bytes; editable roots persist separately; read-only roots reattach by exact identity | +| Continuation | Exact composition/access/cwd/provenance/baselines and editable mutations restore with zero source traffic or fallback | +| Cancellation/restart | Every claim/publication/composition/attachment/checkpoint/cleanup transition reconciles or fails closed | +| Cleanup | Expiry/deletion is unavailable-first, mount-aware, confined, idempotent, reference-safe, and quarantine-preserving | +| Multiple OCI E2E | Two independent OCI entries compose, execute, and reuse independently in a second session | +| Mixed E2E | Git and OCI compose atomically; partial cache warmth builds only missing components; both harnesses consume the result | +| Editable OCI E2E | Bug-fix evaluation mutates OCI content and manifest comparison reports exact persistent final changes | +| Provider boundary | Codex Responses and OMP Chat Completions use fixed brokered routes with no caller override, fallback, or retained secrets | +| Release | Candidate digest, GHCR image, deployed service, SBOM, provenance, pins, UHP conformance, and Promptfoo report all identify the same build | ## Definition of done -1. `allagentsdev/harnessrouter` remains the existing fork and the release image is - `ghcr.io/allagentsdev/harnessrouter` with the established - `-allagents.` tag and a deployed manifest digest. -2. The stock behavior inventory is protected by characterization coverage, and - requests without workspace metadata remain unchanged. -3. The exact first-turn-only snake_case contract supports required `access`, - optional `retention`, required repositories or workspace-snapshot `source`, - and optional workspace-relative `working_directory`, with no nested version. -4. Session state binds exact source provenance, generation/manifest identity, - access, retention, working directory, harness, and attachment evidence. - Continuation omits the descriptor and reuses that exact attachment. -5. Repository mode supports multiple non-overlapping destinations, resolves - advertised default/branch/tag refs, fetches at depth 2, verifies the fetched - tip, preserves shallow recent history and merge semantics, and never silently - deepens or falls back. Submodules and LFS remain off. -6. OCI snapshot mode is implemented and release-tested with direct manifests, - workspace-manifest verification, bounded layer extraction, whiteouts, - path/link/type checks, optional normalized offline Git history, and no Git - fallback. Callers never provide or observe registry origins or credentials. -7. Repository acquisition uses one operator-only bare shallow mirror per canonical - repository identity, serialized singleflight refresh, immutable exact-commit - snapshots, and a separate immutable multi-repository generation. OCI uses the - exact digest-keyed generation cache. Identical normalized source identities - publish once; access/retention/cwd/harness/session do not fragment identity. -8. Every matching read-only session leases and directly binds the same verified - generation bytes with zero clone, source-byte transfer, materialization, or - tree copy. Mirror internals are never session-visible. Editable sessions - receive inode-independent private copies. -9. Existing hydrate, root Git, user/sandbox isolation, cancellation, TTL, - restart, deletion, files, artifacts, and cleanup own the complete session - lifecycle. The private outer workspace remains writable in both access modes. - Read-only inputs are allowed outside source destinations; source-targeting - inputs fail. Outer checkpoints exclude source destinations and source bytes; - editable roots use a separate private source-root checkpoint path. No - parallel workspace system remains. -10. Existing produced-file cursor semantics retain root Git for writable outer - paths, explicitly exclude every source destination, and add declared - editable nested/multiple repository and tree-only/history-bearing cursors - without traversing mounts, writing bookkeeping state into an immutable - generation, or adding a second Files API. -11. Direct Promptfoo E2E against the tested image proves Git and large OCI, - multiple repositories, nested working directory, continuation, read-only - enforcement, editable isolation, produced files, cancellation, restart, - cleanup, provider boundaries, and unchanged stock requests. -12. Required second-session proof covers both caches: Git performs zero clone, - pack transfer, checkout/materialization, or tree copy after exact identity - resolution; OCI performs zero registry request, extraction, publication, or - tree copy. Both read-only sessions bind the same immutable generation bytes - while session/runtime state remains isolated. Editable reuse has independent - inodes and mutations. -13. Codex and OMP use their fixed protocol adapters through the existing external - OAuth-to-OpenAI-compatible gateway with brokered short-lived credentials and - no fallback or caller override. -14. UHP remains the only northbound protocol. There is no AllAgents CLI work, - local profile synchronization, `workspace.yaml` change, provider-login - implementation, runtime-image contract, or benchmark-environment coupling. -15. The release is blocked unless UHP conformance, focused integration coverage, - large-OCI Promptfoo E2E, generation-reuse evidence, credential scans, SBOM, - provenance, and digest-pinned fresh/restart deployment all pass for the same - image. -16. Any later upstream proposal is based on this downstream evidence and remains - outside the release path; no upstream issue, UEP, acceptance, or wait period - blocks implementation or publication. +1. All public downstream repository, image, service, and product names use + AllAgents Gateway. `HarnessRouter/harnessrouter` remains the attributed + upstream with its recorded fork point, license, notices, and selective-intake + process. +2. Requests without `metadata.workspace` remain stock-compatible and are the + only requests permitted to use stock root-Git change collection and + checkpoint correctness. +3. The first-turn-only closed contract requires `access` and an ordered + `sources` array of 1–128 independent `git`/`oci` entries, permits mixed and + repeated identities at distinct destinations, and accepts no obsolete alias. +4. Every source has one required non-root pairwise-non-overlapping destination. + All source ownership and input/asset/reserved collisions fail before network, + cache lookup, claims, or workspace writes. +5. Git components use canonical public HTTPS, exact advertised commit resolution, + mandatory depth 2, useful bounded offline history, no submodule/LFS helpers, + and no deepening/full-clone/source fallback. +6. OCI components use an operator catalog, direct image manifest digest, direct + canonical source manifest digest, pre-layer relative-tree validation, bounded + secure extraction, exact final-tree verification, and no Git fallback. Each + OCI image is one source-tree transport/cache unit, never a runtime or bundle. +7. Per-source and aggregate bounds are enforced for all acquisition kinds and at + composition time. +8. Components cache and singleflight independently. The composition identity + references ordered exact component identities and destinations. A composition + is atomically visible only after every component and attachment verifies. +9. `read_only` Git and OCI roots bind immutable cached bytes. `editable` Git and + OCI roots use inode-independent private writable copies/reflinks. Bug-fix + evaluation uses editable mode. Cached publications never mutate. +10. Every component publishes a canonical baseline keyed by normalized relative + path with type, content digest, executable/mode semantics, and symlink target. + Every workspace-backed session persists an equivalent outer baseline. +11. Final collection/evaluation compares source and outer filesystem manifests, + detects add/modify/delete/mode/symlink/binary changes, treats final-tree + equality as authoritative, and does not rely on root/nested Git state or + rename detection. +12. `.git`, runner-owned state, credentials, caches, attachment evidence, and + checkpoint metadata are excluded from reported changes. OCI roots work with + no Git metadata. +13. The private outer workspace remains writable. Inputs/assets outside source + destinations work; paths at or below destinations fail before network. +14. Checkpoint and continuation are mount-aware. Outer archives exclude all + source bytes, editable roots persist separately, read-only roots reattach by + exact identity, baselines remain stable, and continuation performs no source + traffic. +15. Direct Promptfoo against the tested image proves multiple OCI, mixed Git/OCI, + partial cache warmth, component and composition singleflight, atomic + visibility, editable OCI mutation/evaluation, manifest final diffs, both + harnesses, lifecycle failures, security boundaries, and stock compatibility. +16. A second exact-identity session proves zero Git source-byte work and zero OCI + registry/extraction work while reusing immutable publications; editable + reuse proves independent mutation. +17. Codex and OMP use fixed adapters through the existing external provider + gateway with brokered short-lived credentials, no caller override, and no + fallback. +18. The same digest-pinned `ghcr.io/allagentsdev/allagents-gateway` candidate + passes UHP conformance, focused integration, large/multiple/mixed Promptfoo + E2E, restart deployment, secret scans, SBOM, and provenance gates under + service name `allagents-gateway`. +19. No upstream issue, UEP, acceptance, merge, or release blocks downstream + implementation or publication. From a528dde1080be9d1883733abd30336726c8f9f7a Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 08:11:09 +1000 Subject: [PATCH 34/44] docs(architecture): make gateway plan handoff ready --- .../0002-adopt-uhp-through-harnessrouter.md | 482 ++--- ...0837-feat-coding-execution-gateway-plan.md | 1691 +++++++---------- 2 files changed, 852 insertions(+), 1321 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index e11eeef8..8bd03cb5 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -6,414 +6,252 @@ ## Context -Promptfoo needs a remote coding-harness endpoint that can prepare large source trees before the first turn, preserve session state across continuations, and expose exact source provenance through the Unified Harness Protocol (UHP). +Promptfoo needs a remote coding-harness endpoint that can prepare large source trees before the first turn, preserve session state across continuations, and expose exact source provenance and filesystem changes through the Unified Harness Protocol (UHP). -Upstream [`HarnessRouter/harnessrouter`](https://github.com/HarnessRouter/harnessrouter) already provides the execution-plane lifecycle we need: session identity, user and sandbox isolation, a private per-session workspace, checkpoint transport, TTL, cancellation, Files and artifact surfaces, harness supervision, and cleanup. The missing capability is first-turn initialization of that workspace from one or more independently identified Git or OCI source trees. +The pinned baseline is HarnessRouter commit [`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), release [`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4), and UHP version [`2026-09-12`](https://github.com/HarnessRouter/harnessrouter/tree/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/versions/2026-09-12). That source is Apache-2.0 licensed and includes a `NOTICE`; a downstream repository MUST preserve the license, `NOTICE`, attribution, and history. -The examined upstream baseline is UHP [`2026-09-12`](https://github.com/HarnessRouter/harnessrouter/tree/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/versions/2026-09-12) at HarnessRouter commit [`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), released as [`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4). UHP reserves `metadata` for additive extensions, but stock HarnessRouter gives arbitrary metadata no workspace-initialization semantics. +Stock HarnessRouter already owns session identity, user and sandbox isolation, the private workspace, checkpoints, cancellation, Files and artifacts, harness supervision, cleanup, live-workspace cache reaping, and explicit deletion of durable sessions. A fresh stock workspace initializes a root Git repository. Git supplies the produced-file listing cursor: checkpoint creation commits and then archives the directory, while hydration restores the archive. Git is not a durable-session expiry mechanism. -Large source trees are a minimum product requirement. Git depth `2` bounds history transfer but does not reduce working-tree transfer or extraction cost. OCI transport, immutable component caching, and cache reuse are therefore mandatory v1 capabilities rather than later optimizations. +The existing extension seams are sufficient: -## Decision - -We will ship a downstream product named **AllAgents Gateway** in [`allagentsdev/allagents-gateway`](https://github.com/allagentsdev/allagents-gateway), distributed as `ghcr.io/allagentsdev/allagents-gateway` and deployed under the service name `allagents-gateway`. AllAgents Gateway is derived from upstream HarnessRouter, but it is not named HarnessRouter and must not imply that its downstream workspace extension is standard upstream behavior. - -UHP remains the only northbound protocol. A first turn MAY add `metadata.workspace`. Requests without `metadata.workspace` MUST retain stock upstream UHP and HarnessRouter behavior. A continuation MUST omit the descriptor and recover the exact bound workspace state through the existing session and checkpoint lifecycle. - -A workspace descriptor contains a closed, ordered `sources` array of 1 to 128 independent entries. Every entry materializes exactly one source tree at one required, pairwise non-overlapping, non-root `destination`. The array MAY contain multiple Git entries, multiple OCI entries, or any mixture in any order. A monorepo is one source tree and therefore one entry. - -An OCI image is a source-tree transport and cache unit. It is never the runtime workspace, a runtime or benchmark image, a verifier, a caller-selected container, or a bundle of multiple workspace roots. Its source manifest describes paths relative to its one tree; only the request assigns that tree a destination. There is no Git fallback for an OCI failure and no OCI fallback for a Git failure. - -All sources support the same `read_only` and `editable` access modes. `read_only` exposes immutable cached component roots through namespace-confined read-only bind mounts. `editable` gives the session inode-independent private writable copies or reflinks. An editable OCI source is fully writable and is the expected mode for bug-fix evaluations. Cached component generations remain immutable in both modes. - -Every component publishes a canonical baseline source manifest. Workspace-backed evaluation and produced-file collection compare final filesystem manifests with those baselines; Git state is not an evaluation authority. Root Git, nested Git, commits, index state, and Git rename detection MUST NOT determine correctness for a workspace-backed session. - -Implementation and release do not wait for an upstream issue or UHP proposal. After downstream evidence proves the capability, maintainers MAY propose the generic contract upstream. Until upstream accepts an equivalent contract, releases MUST identify `metadata.workspace` as an AllAgents Gateway extension. +- runner `_produced_list` lists produced paths from a cursor and `_produced_ack` advances that cursor; +- gateway `_collect_produced` durably captures listed files before acknowledging them; +- `BACKING.workspace` exposes either `RunnerWorkspaceFiles` or `CheckpointWorkspaceFiles`; and +- the `HarnessSession` vertex owns session identity, while checkpoint, artifact, and control records remain separate. -The supported harnesses remain **Codex** and **OMP**. Provider traffic continues through the separately operated OAuth-to-OpenAI-compatible gateway using brokered credentials. This decision adds no AllAgents CLI integration and changes no local project workspace configuration. +The missing capability is first-turn initialization from one or more independently identified Git or OCI source trees. Git depth `2` bounds history but not working-tree transfer, so OCI transport and independent immutable component reuse are required in v1. -## Upstream and downstream boundary - -AllAgents Gateway MUST preserve the upstream remote, the recorded fork point, upstream history needed for attribution, the upstream MIT license and copyright notices, and downstream modification notices. The repository MUST keep `https://github.com/HarnessRouter/harnessrouter` as its upstream source of record. Upstream changes are reviewed and selectively integrated; the downstream repository is not a blind mirror, and incompatible upstream changes are not accepted merely to track the latest commit. - -The downstream patch extends upstream fresh-workspace initialization, checkpoint hydration, source provenance, component storage, attachment, and produced-file bookkeeping. It does not create a second workspace service, replace UHP, or move session lifecycle ownership away from the inherited runner. - -When upstream accepts equivalent behavior, AllAgents Gateway SHOULD remove superseded downstream code and migrate cleanly. It MUST NOT retain compatibility aliases, deprecated descriptor shapes, or conflicting schema variants. - -## Existing lifecycle and extension point +## Decision -| Inherited execution responsibility | AllAgents Gateway extension responsibility | -|---|---| -| Allocate the private writable session workspace and user/sandbox identity | Validate the first-turn descriptor and reserve every declared destination | -| Choose fresh materialization or checkpoint hydration | Resolve and acquire each exact source component | -| Transport and restore checkpoints | Cache components independently and compose them atomically | -| Start the harness in the private session workspace | Attach immutable roots or create private editable roots before execution | -| Track sessions, TTL, cancellation, files, and cleanup | Persist component provenance, baseline manifests, composition identity, and attachment evidence | -| Resume an existing writable session workspace | Restore outer and editable-root state and recover exact component bindings without source resolution | +We will ship a downstream product named **AllAgents Gateway**, derived from the pinned HarnessRouter baseline and distributed as `ghcr.io/allagentsdev/allagents-gateway`. UHP remains the only northbound protocol. `metadata.workspace` is an explicitly downstream first-turn extension; requests that omit it retain pinned stock behavior. -For a new workspace-backed session, initialization runs after authentication, bounded request validation, idempotency handling, session resolution, and allocation of the private workspace, but before provider work or harness execution. The workspace root remains private and writable for both access modes. Sources occupy only their declared non-root destinations. +A descriptor contains one ordered `sources` array with 1 to 128 entries. Each entry materializes one tree at a pairwise non-overlapping, non-root destination. Git and OCI entries MAY be mixed in any order. One OCI image represents one source tree, not a runtime image, benchmark image, verifier, or multi-root bundle. A source-kind failure never falls back to the other kind. -Destination ownership MUST be decidable from the request before DNS, Git, registry, or other source access. The gateway MUST reject root destinations, overlapping destinations, reserved paths, and every known input, generated asset, restored outer path, or runner-owned path that collides at or below a destination. Ancestor directories MAY be created as empty scaffolding, but MUST NOT contain a file or link that prevents safe attachment. Inputs and assets outside source roots remain allowed. +The gateway adds workspace binding and a `pending` to `ready` transition to the existing session lifecycle. It MUST resolve, verify, and materialize every root and validate the working directory before making any source visible to the harness or starting provider work. “Atomic” means application visibility after all roots verify; it does not require an atomic filesystem rename or namespace handoff. Failure before `ready` exposes no partial workspace. -The outer workspace stores `.harness`, HOME, generated instructions, plugins, skills, MCP configuration, inputs, outputs, scratch, conversation state, and other session data outside source destinations. The extension does not allocate a second workspace root. +Implementation and release do not wait for an upstream issue or UHP proposal. Until upstream accepts an equivalent contract, releases MUST identify this behavior as an AllAgents Gateway extension. ## Request contract -A workspace-backed first turn uses the normal UHP `POST /v1/responses` endpoint. `metadata.workspace` is first-turn-only and has no nested schema version. - -This editable request composes two Git trees and one OCI tree. The OCI source is writable after attachment and can be modified by a bug-fix evaluation: +A workspace-backed first turn uses the normal `POST /v1/responses` endpoint. `metadata.workspace` is a closed object with no nested schema version: ```json { - "model": "gpt-5.4", - "input": "Implement the requested change.", - "metadata": { - "harness_id": "chrn_…", - "workspace": { - "access": "editable", - "retention": "session", - "sources": [ - { - "kind": "git", - "url": "https://github.com/acme/api.git", - "ref": "refs/heads/main", - "destination": "services/api" - }, - { - "kind": "oci", - "snapshot_name": "compiler-tree", - "image_manifest_digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111", - "source_manifest_digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222", - "destination": "vendor/compiler" - }, - { - "kind": "git", - "url": "https://github.com/acme/web.git", - "destination": "services/web" - } - ], - "working_directory": "services/api/packages/server" + "access": "editable", + "sources": [ + { + "kind": "git", + "url": "https://github.com/acme/api.git", + "ref": "refs/heads/main", + "destination": "services/api" + }, + { + "kind": "oci", + "snapshot_name": "compiler-tree", + "image_manifest_digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111", + "source_manifest_digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222", + "destination": "vendor/compiler" } - } + ], + "working_directory": "services/api/packages/server" } ``` -`metadata.workspace` has exactly these fields: - -| Field | Required | Contract | -|---|---:|---| -| `access` | yes | `read_only` or `editable`; it applies identically to every entry. | -| `retention` | no | `session` by default, or `persistent` when deployment policy authorizes it. | -| `sources` | yes | Closed ordered array containing 1 to 128 Git or OCI entries. | -| `working_directory` | no | Workspace-relative POSIX directory. Omission means the workspace root. | - -A Git entry has exactly these fields: - | Field | Required | Contract | |---|---:|---| -| `kind` | yes | `git`. | -| `url` | yes | Canonical public HTTPS Git URL. Userinfo, query, fragment, local path, and alternate transports are forbidden. | -| `ref` | no | Advertised full ref or unambiguous branch/tag shorthand. Omission selects the advertised remote default. The resolved commit is authoritative. | -| `destination` | yes | Non-root workspace-relative POSIX directory, pairwise non-overlapping with every other destination. | - -Git depth is not caller-selectable. Every Git entry uses effective depth `2`, which is returned in provenance and participates in component identity. +| `access` | yes | `read_only` or `editable`, applied to every source. | +| `sources` | yes | Ordered closed array of 1 to 128 Git or OCI entries. | +| `working_directory` | no | Workspace-relative POSIX directory; omission means `.`. | -An OCI entry has exactly these fields: +A Git entry contains only `kind: "git"`, canonical public HTTPS `url`, optional `ref`, and `destination`. Userinfo, query, fragment, local paths, alternate transports, and private or otherwise disallowed network targets are forbidden. `ref` is an advertised full ref or unambiguous branch/tag shorthand; omission selects the advertised default. Resolution produces one exact commit. Depth is always `2` and is not caller-selectable. -| Field | Required | Contract | -|---|---:|---| -| `kind` | yes | `oci`. | -| `snapshot_name` | yes | Operator-catalog key; it is not a registry repository or URL. | -| `image_manifest_digest` | yes | Direct `sha256:` digest of the accepted OCI image manifest. Tags and indexes are not source identity. | -| `source_manifest_digest` | yes | `sha256:` digest of the canonical manifest for the image's one relative source tree. | -| `destination` | yes | Non-root workspace-relative POSIX directory, pairwise non-overlapping with every other destination. | +An OCI entry contains only `kind: "oci"`, `snapshot_name`, exact `image_manifest_digest`, exact `source_manifest_digest`, and `destination`. `snapshot_name` resolves through an operator-owned catalog to a fixed registry repository, catalog-entry identity, allowed media types, trust policy, and server-side credential reference. Callers cannot supply registry origins, repositories, tags, indexes, headers, redirects, or credentials. -The OCI catalog is operator-owned AllAgents Gateway configuration. It maps `snapshot_name` to a fixed registry repository, allowed media types, trust policy, and server-side credential reference. A caller never supplies a registry origin, repository, tag, header, redirect policy, or credential. +Every destination and `working_directory` is an NFC-normalized relative POSIX path with no empty, `.`, `..`, absolute, platform-specific, or reserved component. Destinations MUST be non-root and pairwise non-overlapping. Destination ownership, reserved-path conflicts, and collisions with known inputs or generated assets MUST be rejected before DNS, Git, registry, or other source access. Ancestor directories may be empty scaffolding only. After materialization, `working_directory` MUST resolve without symlink escape to a real directory. -The order of `sources` is semantic and retained in provenance and composition identity. It does not define overlay precedence: destinations cannot overlap, and one source cannot mask another. Unknown keys MUST be rejected at every level. No deprecated spelling or alternate shape is accepted. +The order of `sources` is semantic but does not establish overlay precedence. Unknown keys are rejected at every level. The descriptor cannot contain credentials, headers, host paths, commands, environment variables, runtime images, materializer selection, resource limits, provider routes, or expiry controls. Request size, string length, nesting, and validation work are bounded before source access. -`working_directory` is interpreted only after all trees are attached or copied and allowed inputs are placed. It MUST resolve, without symlink escape, to a real directory inside the workspace. It MAY be inside a source tree or in the writable outer workspace. Absolute paths, empty components, `.` or `..` components, platform-specific separators, and reserved runner paths are invalid. - -The descriptor cannot contain credentials, headers, host paths, environment variables, commands, runtime images, Docker settings, materializer selection, resource limits, provider routes, or caller-selected TTLs. Request size, string length, array length, nesting, and validation work MUST be bounded before source access. - -UHP input files and generated assets remain ordinary outer-workspace mutations. For either access mode, any such path at or below a declared destination MUST fail during pre-network validation rather than overlaying, copying up, or modifying a source tree. Paths outside all source destinations remain allowed and are included in the sealed outer-workspace baseline. - -A first turn MAY omit `metadata.workspace`; stock behavior then remains unchanged. A session created without workspace metadata cannot add it later. Any reused session selected through `previous_response_id` or existing recovery metadata MUST omit `metadata.workspace`, even if a repeated descriptor would be byte-for-byte identical. +A continuation selected by `previous_response_id` or other inherited recovery state MUST omit `metadata.workspace`. A stock session cannot acquire a workspace binding later, and an existing workspace-backed session cannot replace or repeat its descriptor. ## Response and provenance contract -After attachment reaches `ready`, terminal events, response retrieval, replay, and later terminal failures expose the same committed `metadata.workspace` response object. All public fields use snake case. - -The response for the mixed request above is shaped as follows: +After the binding reaches `ready`, terminal events, response retrieval, replay, and later terminal failures expose the same sanitized `metadata.workspace` object: ```json { - "metadata": { - "workspace": { - "access": "editable", - "retention": "session", - "working_directory": "services/api/packages/server", - "effective_descriptor_digest": "sha256:3333333333333333333333333333333333333333333333333333333333333333", - "composition_id": "sha256:4444444444444444444444444444444444444444444444444444444444444444", - "sources": [ - { - "kind": "git", - "url": "https://github.com/acme/api.git", - "requested_ref": "refs/heads/main", - "resolved_commit": "0123456789abcdef0123456789abcdef01234567", - "depth": 2, - "destination": "services/api", - "component_id": "sha256:5555555555555555555555555555555555555555555555555555555555555555", - "source_manifest_digest": "sha256:6666666666666666666666666666666666666666666666666666666666666666" - }, - { - "kind": "oci", - "snapshot_name": "compiler-tree", - "image_manifest_digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111", - "source_manifest_digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222", - "destination": "vendor/compiler", - "component_id": "sha256:7777777777777777777777777777777777777777777777777777777777777777" - }, - { - "kind": "git", - "url": "https://github.com/acme/web.git", - "resolved_commit": "89abcdef0123456789abcdef0123456789abcdef", - "depth": 2, - "destination": "services/web", - "component_id": "sha256:8888888888888888888888888888888888888888888888888888888888888888", - "source_manifest_digest": "sha256:9999999999999999999999999999999999999999999999999999999999999999" - } - ], - "expires_at": "2026-09-29T00:00:00Z" + "access": "editable", + "working_directory": "services/api/packages/server", + "sources": [ + { + "kind": "git", + "url": "https://github.com/acme/api.git", + "requested_ref": "refs/heads/main", + "resolved_commit": "0123456789abcdef0123456789abcdef01234567", + "depth": 2, + "destination": "services/api", + "source_manifest_digest": "sha256:6666666666666666666666666666666666666666666666666666666666666666" + }, + { + "kind": "oci", + "snapshot_name": "compiler-tree", + "image_manifest_digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111", + "source_manifest_digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222", + "destination": "vendor/compiler" } - } + ], + "expires_at": "2026-09-29T00:00:00Z" } ``` -The response fields are exact: - -| Field | Contract | -|---|---| -| `access` | Effective immutable access mode. | -| `retention` | Effective retention after authorization; it is never silently downgraded. | -| `working_directory` | Effective workspace-relative directory; the workspace root is represented as `.`. | -| `effective_descriptor_digest` | Digest of the normalized first-turn descriptor, including applied defaults. | -| `composition_id` | Public identity of the ordered composition, derived from each exact `component_id` and its destination. It is not an authorization token or cache lookup key. | -| `sources` | Request-order closed provenance entries for every component. | -| `expires_at` | Effective expiry for `session` retention, or `null` for authorized `persistent` retention. | - -A Git provenance entry contains normalized `url`, exact `resolved_commit`, effective `depth`, `destination`, `component_id`, `source_manifest_digest`, and `requested_ref` only when the request supplied `ref`. Branch or tag movement does not change stored provenance for an existing session. +`working_directory` is always present and uses `.` for the workspace root. Git provenance contains normalized `url`, optional `requested_ref`, exact `resolved_commit`, `depth: 2`, `destination`, and the verified source-manifest digest. OCI provenance contains the catalog key, exact image and source-manifest digests, and `destination`. Registry details, credentials, private cache keys, backing paths, and live attachment details are never public. -An OCI provenance entry contains `snapshot_name`, exact `image_manifest_digest`, exact `source_manifest_digest`, `destination`, and `component_id`. It never exposes registry origin, repository, credential reference, redirect, backing path, layer-cache key, or physical mount path. OCI provenance has no implied Git commit and does not require Git metadata. +Every workspace-backed session receives one operator-configured finite expiry at creation. `expires_at` is always a timestamp; polling, replay, and continuation do not extend it. Explicit deletion remains supported. Failures before `ready` omit workspace metadata; failures after `ready` return the stored sanitized object. -Failures before attachment reaches `ready` omit workspace response metadata. Failures after `ready` return the complete committed object. Private generation keys, policy versions, authorization scope, mount paths, attachment IDs, pins, leases, reservations, and other sessions' state are never exposed. +## Canonical source manifest v1 -## Canonical manifests, identities, and caches +The source-manifest media type is `application/vnd.allagents.source-manifest.v1+json`. Its exact bytes are the RFC 8785 JSON Canonicalization Scheme representation of: -Each verified component publishes a versioned canonical source manifest keyed by normalized relative POSIX path. Every entry records its type, regular-file content digest, executable or normalized mode semantics, and symbolic-link target where applicable. Canonical ordering, encoding, path normalization, directory treatment, and supported file types are part of the manifest contract. The gateway walks staging without following links, recomputes canonical bytes, and records their digest before publication. - -For Git, the gateway derives the canonical source manifest from the verified checkout. For OCI, the gateway MUST fetch and digest-verify `source_manifest_digest` before requesting any layer. That manifest describes only paths relative to the entry's single source root. The gateway validates all declared paths, types, sizes, modes, and links before layer requests, then requires extracted contents and recomputed canonical bytes to match it exactly before publication. - -A private component key includes every input that can change component bytes, filesystem semantics, or sharing authorization. A Git key includes canonical URL, exact resolved commit, depth `2`, acquisition-policy revision, and materializer-contract revision. An OCI key includes catalog identity, exact image-manifest digest, exact source-manifest digest, trust-policy revision, and materializer-contract revision. Destination, array position, access, retention, working directory, harness, session, and physical paths are excluded from component identity because they do not change component bytes. - -The component cache is independent for every entry. Git uses an operator-only bare mirror/object cache per canonical URL for bounded acquisition, followed by an immutable verified component generation. OCI uses verified blob/layer caches followed by an immutable verified component generation. Refreshes of a mutable acquisition cache are serialized. Concurrent misses singleflight by exact component key, not by the complete request. - -A composition does not merge source bytes into another cached generation. Its identity commits to the ordered sequence of exact component identities and destinations. The gateway MAY acquire or reuse independent components concurrently, but it MUST reserve every component and attach or copy the full composition atomically. All destinations become visible to the session or none do. A failure or cancellation of one entry rolls back provisional mounts, copies, leases, and pins for the complete composition. - -Publication is crash-safe. Partial staging is never attachable. Garbage collection MUST NOT remove an acquisition object or immutable component while a builder, waiter, provisional composition pin, durable session reference, or attachment lease protects it. A later acquisition, trust, or materializer policy revision cannot reuse an incompatible component. - -## Attachment behavior - -Attachment is uniform across Git and OCI: - -- For `read_only`, the gateway creates empty destination directories in the private writable outer workspace and bind-mounts each immutable component root there. Matching sessions MAY share cached source bytes and immutable inodes while retaining separate outer state, user and sandbox identity, HOME, scratch, logs, outputs, checkpoints, and response state. -- Every source bind mount MUST be kernel-enforced read-only, namespace-confined, `nodev`, and `nosuid`, while preserving required execute bits. The session MUST have no writable alias or copy-up path to the generation or acquisition cache. A mount or remount failure fails closed. -- For `editable`, the gateway materializes a quota-bounded, inode-independent private writable copy or reflink of every pinned component at its destination. No mutable inode may be shared with a cached generation, acquisition cache, or another session. Git and OCI entries receive identical write semantics. - -Bind mounts are required for shared read-only roots rather than symlinks. A symlink does not enforce read-only access, exposes a backing path, can escape workspace containment, and gives cwd and file tools surprising behavior. Mounted roots appear as ordinary directories at their declared destinations. - -An attachment binds the normalized descriptor, ordered exact component keys and epochs, composition identity, access, retention, working directory, selected harness, destination map, provenance, canonical source-manifest digests, and evaluation baselines to the session. Continuation MUST reuse that exact attachment. It never re-resolves a Git ref, repulls OCI, changes access or retention, or substitutes another component with the same public identifier. - -## Git acquisition and integrity - -Git content is untrusted. Every Git entry uses depth `2`: the selected tip plus bounded reachable history, not necessarily exactly two total commits when the tip is a merge. - -Git initialization MUST satisfy all of these requirements: - -- Parse and authorize every URL before DNS or process launch. Accept only canonical public HTTPS origins. Revalidate every redirect; reject private, loopback, link-local, reserved, metadata-service, and otherwise disallowed addresses; pin approved addresses against DNS rebinding. -- Run Git without a shell, with a sanitized environment and isolated configuration. Disable interactive credentials, inherited proxies, hooks, checkout filters, Git LFS hydration, submodule recursion, alternates, and non-HTTPS helpers and protocols. -- Resolve only an advertised branch, tag, or remote default to one exact commit. Fetch that selection with `--depth=2`, then verify the fetched tip equals the resolved commit. Caller text MUST NOT be substituted into fetch or checkout commands. If the remote cannot satisfy the bounded shallow fetch, fail rather than deepen, unshallow, or clone fully. -- Keep the bare shallow acquisition cache mutable, operator-only, and inaccessible to harness users, credentials, workspace writes, and attachments. Publish only the selected bounded object graph into the immutable component; unrelated cached refs and objects MUST NOT be exposed. -- The published component MAY preserve normalized `.git/shallow` metadata and recent history for agent convenience. Remove credential-bearing remotes, hooks, worktree links, alternates, replace and graft state, locks, reflogs, transient fetch state, and unsafe configuration. Verify detached `HEAD`, shallow boundary, index-to-tree equality, included-object integrity, and source content against the resolved commit and canonical source manifest. -- Enforce finite time, transferred-byte, expanded-byte, inode, file-count, process, descendant, and concurrency limits independently for each Git entry and for the complete request. -- Reject undeclared output, traversal, unsafe links, reserved-path collisions, and any path that escapes its owning source root. A failed Git entry fails the complete composition; no partial set is attached. -- Record normalized URL, optional requested ref, exact resolved commit, effective depth, component identity, source-manifest digest, and destination. +```json +{"version":1,"entries":[]} +``` -Git metadata is acquisition evidence and MAY be useful to an agent. It is never the evaluator's diff engine. Its absence or mutation cannot change the baseline or final-tree comparison used to judge a workspace-backed evaluation. +`entries` is sorted by the UTF-8 bytes of each NFC-normalized relative POSIX `path`. The root is omitted. Duplicate paths, non-UTF-8 or non-NFC names, empty, `.` or `..` path components, type conflicts, and unsupported file types fail validation. The digest exposed as `source_manifest_digest` is `sha256:` followed by the lowercase hexadecimal SHA-256 of the canonical bytes. -## OCI acquisition and integrity +Entries have exactly one of these forms: -OCI support is mandatory and release-blocking. Each OCI entry transports exactly one relative source tree and MUST satisfy all of these requirements: +```json +{"path":"src","type":"directory"} +{"path":"src/main.ts","type":"file","size":123,"sha256":"sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef","executable":false} +{"path":"bin/tool","type":"symlink","target":"../src/tool"} +``` -- Resolve `snapshot_name` only through the operator catalog. Fetch only the direct image manifest named by `image_manifest_digest`; do not follow a mutable tag, accept an index in its place, change registry authority on redirect, or expose registry details to the caller. -- Verify the image-manifest digest, media type, descriptor sizes, every selected layer digest and size, the catalog-defined source-manifest media type, the source-manifest blob digest, and the recomputed canonical source-manifest digest. -- Fetch and verify the canonical source manifest before any layer. Validate all relative paths and types plus every request-known input, asset, reserved-path, and restored-outer-state collision before any layer request or outer-workspace content write. -- Apply layers in order with OCI whiteout and opaque-directory semantics. Whiteouts are metadata operations, not source-visible files. Reject malformed, duplicate, conflicting, or out-of-root whiteouts. -- Before writing an entry, validate its normalized relative path, type, declared size, mode, and link target. Every hard link and symbolic link MUST remain within that OCI entry's one source root. Links into another source, the writable outer workspace, or an absolute path fail closed. Extraction uses rooted no-follow operations and cannot write through a previously extracted link. -- Reject traversal, NULs, devices, sockets, FIFOs, sparse-file tricks, unsupported types, undeclared entries, missing entries, type changes, digest mismatches, and writes outside the one declared tree. -- Publish only after the extracted tree exactly matches the canonical source manifest. Failure MUST NOT fall back to Git or another catalog entry. +A file `sha256` is `sha256:` followed by the lowercase hexadecimal SHA-256 of +its content. `executable: false` represents normalized mode `0644`; `true` +represents `0755`. A symlink target MUST be UTF-8, NFC, relative, and confined +to its owning source root when resolved from the link's parent. Empty +directories are represented. Safe in-root OCI hardlinks may be materialized as +ordinary file entries and are not a manifest type. `.git` content MAY be +integrity-verified in a source manifest but is excluded from change reporting. -The v1 envelope permits at most 64 distributable tar/gzip/zstd layers, a 4 MiB image manifest, a 128 MiB source manifest, 8 GiB compressed layer bytes, 32 GiB expanded source bytes, 500,000 entries, 4 GiB per regular file, paths of at most 4096 UTF-8 bytes and 128 components, and 1 MiB per PAX or extended header for one OCI entry. Expanded bytes divided by `max(compressed_bytes, 1)` MUST NOT exceed `100` for each layer and for the entry. Checks apply while streaming. +The gateway walks without following links and recomputes the canonical manifest before publication. For OCI, it fetches and digest-verifies the source manifest before any layer, validates its paths and limits, applies ordinary OCI image/layer/whiteout semantics, and requires the extracted tree to reproduce the declared canonical manifest exactly. -Finite time, bytes, entries, inodes, layers, processes, descendants, and concurrency are also bounded across the complete request, including mixed Git and OCI compositions. Per-entry success does not bypass request-aggregate limits. Operators MAY configure lower limits and MUST NOT raise contractual maxima without a contract revision. +## Resolution, caching, and visibility -An OCI tree need not contain `.git` or any other Git metadata. If such metadata is present, it is untrusted source content and optional agent convenience only; it conveys no implicit commit identity and is not used for evaluation diffs. Repacking layers produces a different OCI component identity even if canonical source-tree bytes are equal, because the exact image-manifest digest remains part of identity. +A private Git component key is exactly the canonical URL, exact resolved commit, depth `2`, and one cache-schema revision. A private OCI component key is exactly the catalog-entry identity, exact image-manifest digest, exact source-manifest digest, and one cache-schema revision. Recomputing a baseline digest proves publication integrity; it is not a cache-key input. -## Produced files and evaluation diffs +Components cache independently and exact-key misses singleflight independently. Git retains the accepted operator-only acquisition mirror per canonical URL, then publishes an immutable verified generation. OCI MAY use standard registry-client, image, and layer caches; this decision does not require a separate gateway-managed blob-cache lifecycle. There is no request-wide composition cache, record, or public identity. -The inherited Files API and produced-file collection remain the only public file surface. AllAgents Gateway adapts that surface rather than adding another change API. +The session binding stores the private descriptor digest, ordered resolved source plan, exact private component keys, public provenance, access, working directory, canonical baselines and acknowledged manifest cursors, durable component and checkpoint references, and `expires_at`. It MUST NOT persist live mount IDs, filesystem identities, attachment flags, publication generations, or inode evidence. Live read-only protection, destination ownership, and writable-copy isolation are revalidated on every attach. -Before harness execution, the gateway seals: +The ordered plan remains `pending` until every component is verified, each destination is safely materialized, and the working directory is valid. One transition to `ready` makes the plan visible to application code. A failure rolls back provisional work and leaves no visible partial plan. -1. the published canonical baseline source manifest for every attached component; and -2. a canonical baseline manifest of the writable outer workspace after allowed inputs and assets are placed, excluding every source destination and runner-owned state. +## Access and source integrity -At collection time, the gateway computes final manifests for every editable source root and the writable outer workspace using the same canonical rules. A read-only source root remains equal to its pinned immutable baseline by construction; collection MUST NOT traverse through its mount into backing storage. The gateway compares baseline and final maps by normalized path and reports additions, modifications, deletions, executable or normalized-mode changes, symbolic-link target changes, and binary content changes. Content digests, rather than text decoding, determine equality. +For `read_only`, each immutable component root is exposed at its destination through a namespace-confined read-only bind mount with `nodev` and `nosuid`, without a writable alias or copy-up path. For `editable`, each destination is a quota-bounded, inode-independent private reflink or copy. Git and OCI receive identical write semantics. The outer workspace remains private and writable in both modes. -Rename inference is OPTIONAL presentation metadata. Final-tree equality is authoritative: two sessions with the same included final path/type/mode/link/content maps are equivalent even if one report infers a rename and another reports delete-plus-add. +Git acquisition resolves only advertised refs, fetches the selected commit at depth `2`, and verifies the fetched tip, bounded object graph, checkout, and source manifest. Commands run without a shell in a sanitized, isolated configuration; credentials, inherited proxies, hooks, filters, LFS hydration, submodule recursion, alternates, and non-HTTPS helpers are disabled. The published tree may retain safe shallow `.git` metadata and bounded recent history for agent convenience, but removes credential-bearing remotes and unsafe or transient state. Git metadata never defines evaluation correctness. -For workspace-backed sessions, the manifest comparison is the only correctness source. The collector MUST NOT consult root Git, nested Git, commits, index state, worktree status, staged state, or Git rename detection. It MUST NOT initialize or mutate a root `.git` repository. Stock root-Git behavior MAY remain only on requests that omit `metadata.workspace`. +OCI acquisition uses the catalog-selected direct image manifest and the declared source manifest. It verifies descriptor media types, sizes, and digests; applies layers in order with standard whiteout and opaque-directory behavior; and extracts with rooted no-follow operations. Traversal, out-of-root links, devices, sockets, FIFOs, sparse-file tricks, undeclared or missing entries, unsupported types, and digest or type mismatches fail closed. OCI sources need not contain Git metadata. -The evaluation projection excludes `.git` trees, runner-owned state, credentials, caches, acquisition and generation metadata, attachment evidence, checkpoint metadata, and other internal control paths from reported changes. Those exclusions apply equally to Git and OCI roots and to outer-workspace bookkeeping. They do not weaken acquisition integrity checks. +The v1 maximum for one OCI entry is 64 tar/gzip/zstd layers, a 4 MiB image manifest, a 128 MiB source manifest, 8 GiB compressed layers, 32 GiB expanded source, 500,000 entries, 4 GiB per regular file, 4096 UTF-8 bytes and 128 components per path, and 1 MiB per PAX or extended header. Expanded bytes divided by `max(compressed bytes, 1)` MUST NOT exceed 100 per layer or entry. Time, bytes, inodes, processes, descendants, and concurrency are also bounded per entry and per request. Operators MAY lower but not raise these contractual maxima without a contract revision. -A `read_only` attachment cannot produce source mutations, but files created or changed outside mounted roots are collected through the outer manifest. Editable Git and editable OCI changes are collected identically. All final state remains subject to the session's private quota and lifecycle. +## Produced files and evaluation changes -## Checkpoints and continuation +The inherited UHP Files/artifact surface remains the only public file surface. Workspace-backed collection replaces only the stock root-Git listing/cursor implementation behind the existing seams: -Checkpointing is mount-aware and preserves outer state and editable roots separately. +1. `_produced_list` compares the last acknowledged canonical manifest cursor for the outer workspace and every editable root with their final manifests. Read-only roots are not walked; their unchanged state is trusted only from immutable component identity plus freshly verified mount protection. +2. `_collect_produced` captures every added or modified regular file through the existing gateway file-artifact path. +3. `_collect_produced` then captures one server-generated `workspace-changes-.json` artifact with media type `application/vnd.allagents.workspace-changes.v1+json`. +4. Only after all file artifacts and the change artifact are durable does `_produced_ack` advance all cursor manifests. A retry before acknowledgement reproduces the same logical change set. -A `read_only` checkpoint archives the writable outer workspace while excluding every source destination and all source bytes. It stores exact ordered component keys and epochs, composition identity, destination map, canonical baseline digests, durable references, and mount evidence as protected checkpoint metadata. Archive, Files, cleanup, and manifest operations MUST NOT follow or cross a source mount. +The change artifact is RFC 8785 canonical JSON: -An `editable` checkpoint stores the writable outer workspace and each inode-independent private source root as separate logical checkpoint members, together with their canonical baselines and attachment evidence. It never stores the bare Git cache, OCI blob cache, immutable component backing store, credentials, or transient acquisition state. Separation prevents a restored outer archive from overwriting, omitting, or aliasing an editable root. +```json +{ + "version": 1, + "entries": [ + { + "path": "services/api/src/main.ts", + "operation": "modify", + "before": {"type": "file", "sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "size": 100, "executable": false}, + "after": {"type": "file", "sha256": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "size": 120, "executable": false}, + "file_id": "file_…" + } + ] +} +``` -A continuation supplies the existing predecessor or session reference and omits `metadata.workspace`. The gateway first removes stale attachments, restores the writable outer state, and validates all destination mountpoints. For `read_only`, it then reattaches the exact protected components atomically. For `editable`, it restores each preserved private writable root at its exact destination. It verifies stored provenance, baselines, composition identity, and attachment evidence before starting the stored harness in the stored working directory. +Entries sort by the UTF-8 bytes of NFC-normalized workspace-relative `path`. `operation` is `add`, `modify`, or `delete`. `add` has `after`; `delete` has `before`; `modify` has both. State objects use `{type:"directory"}`, `{type:"file",sha256,size,executable}`, or `{type:"symlink",target}`. `file_id` is required exactly when `after.type` is `file` and refers to the captured regular-file artifact. Mode or symlink-target changes are `modify`; a rename is `delete` plus `add`. An empty collection still emits the change artifact with an empty `entries` array. -Edits from prior editable turns and files written outside read-only roots remain visible. A changed ref, source digest, destination, source order, working directory, access, retention, or harness requires a new session. Continuation never clones, repulls, rematerializes from OCI, substitutes a newer component, or silently starts fresh. +Runner-owned state, credentials, caches, control metadata, checkpoint metadata, and every `.git` tree are excluded from reporting. Promptfoo consumes change artifacts in response order and applies them to its previous reconstructed state. It does not inspect gateway Git state or require a new endpoint. -Attachments are unmounted before hydration, deletion, workspace cleanup, or cleanup retry. If an unmount, component, private copy, checkpoint member, baseline, provenance record, pin, or lease is missing, busy, corrupt, expired, or inconsistent, continuation or cleanup fails closed and the allocation remains accounted for. +## Checkpoints, continuation, expiry, and deletion -## Provider authentication and harness configuration +Workspace-backed checkpointing keeps the inherited archive/hydration lifecycle but is mount-aware. Archive, Files, cleanup, and manifest operations MUST NOT follow or cross read-only mounts. Checkpoints preserve the writable outer workspace and editable-root bytes, plus durable references to the ordered binding and baselines; they never archive immutable cache roots or acquisition state. -Provider authentication remains proxy-only. Each deployment configures one external OAuth-to-OpenAI-compatible gateway base URL and API key server-side. AllAgents Gateway represents that endpoint with two protocol-specific logical connections using the same secret: Responses for Codex and OpenAI Chat Completions for OMP. Each harness policy contains exactly its matching connection, with no fallback. A UHP caller cannot supply or override the endpoint, key, transport, or route. +Continuation restores the inherited checkpoint archive, recovers the exact stored source plan without resolving refs or pulling OCI again, verifies durable component and baseline references, revalidates destinations and live attachment protection, attaches read-only roots or restores editable roots, validates the working directory, and only then starts the stored harness. Expired, missing, or corrupt bound state fails closed rather than substituting a newer source or silently starting fresh. -The external OAuth gateway owns login, token persistence, refresh, repair, provider API compatibility, and provider authorization. AllAgents Gateway does not implement provider login, import local credentials, mount developer credential files, or coordinate provider token refresh. Its caller API key authenticates the UHP caller only and is never reused as a provider credential. +Live-workspace cache reaping and durable session deletion remain distinct operations. The gateway rejects continuation at or after `expires_at` and performs deletion through the inherited explicit durable-session deletion path. Attachments are removed before hydration, deletion, workspace cleanup, or cleanup retry. Uncertain or busy state remains accounted for until cleanup succeeds. -The deployment uses inherited brokered sandbox mode, not owner-trust credential pass-through. The broker exchanges the long-lived external-gateway key server-side and gives each harness only a short-lived, session-scoped credential plus a loopback broker URL. The long-lived key MUST NOT enter the harness environment, session workspace, checkpoint, file output, artifact, log, response, or source provenance. +## Failure contract -Codex uses the gateway's OpenAI Responses-compatible surface. OMP uses its OpenAI Chat Completions-compatible surface. V1 custom harnesses are Codex and ordinary session-local OMP only. OMP starts from container and session configuration; it does not import AllAgents profiles, host profiles, or developer state. +Workspace failures use these stable detail codes in the existing UHP error +shape. `retryable` states whether the same operation may succeed without +changing the request. Pre-`ready` failures omit workspace metadata; post-`ready` +workspace failures return the stored sanitized metadata except inherited +`session_expired`, which retains the pinned UHP response unchanged. -## Failure behavior +| Detail code | HTTP | Retryable | Timing and condition | +|---|---:|:---:|---| +| `workspace_invalid_request` | 400 | no | Pre-`ready`: malformed/unknown fields, invalid paths or refs, continuation metadata, or invalid working-directory syntax. | +| `workspace_path_collision` | 409 | no | Pre-`ready`, before source access: overlapping/reserved destinations or input/asset collisions. | +| `workspace_source_unknown` | 404 | no | Pre-`ready`: unknown catalog entry or missing, ambiguous, or unsupported Git ref identity. | +| `workspace_source_invalid` | 422 | no | Pre-`ready`: digest, manifest, layer, path, link, type, checkout, or extracted-content verification failure. | +| `workspace_acquisition_unavailable` | 503 | yes | Pre-`ready`: bounded transient Git, registry, DNS, transport, or upstream service failure. | +| `workspace_contract_limit_exceeded` | 413 | no | Pre-`ready`: request, component, expansion, file, path, process, time, or aggregate contractual maximum exceeded. | +| `workspace_capacity_exceeded` | 503 | yes | Pre-`ready`: operator storage, inode, mount, worker, or concurrency capacity unavailable. | +| `workspace_attachment_failed` | 500 | yes | Initial attachment before `ready` or live reattachment after `ready`: bind/remount, copy/reflink, destination, protection, baseline, or working-directory validation failure. | +| `session_expired` | 404 | no | Inherited UHP response when continuation targets a session at or after `expires_at`; omit workspace metadata and do not restore or extend expiry. | +| `workspace_restore_invalid` | 500 | no | Post-`ready`: binding, component, checkpoint, baseline, or cursor evidence is missing, corrupt, or inconsistent. | +| `workspace_collection_failed` | 500 | yes | Post-`ready`: manifest comparison, file/change-artifact capture, or cursor acknowledgement cannot complete. The cursor is not advanced. | +| `workspace_checkpoint_failed` | 500 | yes | Post-`ready`: mount-aware archive creation or durable checkpoint persistence fails after collection. | -The implementation fails closed without changing source identity, source kind, access, retention, harness, model route, or provider protocol as a recovery shortcut. +Cancellation is not a workspace error. The inherited cancel endpoints remain +idempotent and successful, and a cancelled task terminates with +`status: "cancelled"`, never failed. A cancellation before `ready` omits +workspace metadata; one after `ready` retains the stored sanitized metadata. -| Failure | Required behavior | -|---|---| -| Malformed, oversized, too-deep, unknown, or deprecated workspace field | Reject before source access and without mutating an existing session. | -| Workspace metadata on a continuation or reused session | Reject without changing attachment, checkpoint, or TTL. | -| Invalid, root, overlapping, or reserved destination | Reject the complete descriptor before network access. | -| Input, generated asset, or restored outer path collides at or below a destination | Reject before source access or outer-workspace mutation. | -| Unauthorized `persistent` retention | Fail before source resolution; do not downgrade to `session`. | -| Invalid Git URL, ref, network target, redirect, or feature | Fail the complete composition, cancel bounded work, and remove staging. | -| Unknown OCI catalog entry, digest mismatch, mutable reference, disallowed registry transition, or unsupported media type | Fail that component and therefore the complete composition; do not try Git or another artifact. | -| Layer limit, extraction violation, whiteout error, unsafe path/link/type, or source-manifest mismatch | Terminate extraction, quarantine or remove staging, and publish nothing. | -| Any source fails or aggregate capacity is exceeded | Roll back the complete composition; never attach a partial set. | -| Resource or concurrency capacity unavailable | Return a coded retryable capacity failure before unbounded acquisition. | -| Materializer timeout, crash, cancellation, or live descendant | Terminate and reap the complete process tree before cleanup and terminal acknowledgement. | -| Working directory missing, not a directory, or escaping through traversal or a link | Fail before attachment commit and harness execution. | -| Crash during component publication or composition commit | Recover to either a complete verified composition or no attachment. | -| Mountpoint non-empty or linked, or bind/remount/copy failure | Fail closed before harness execution; expose no partial or writable alias. | -| Missing or corrupt bound state on continuation | Fail as non-resumable; never reacquire or substitute. | -| External provider authentication or execution failure | Return the normalized UHP failure; do not switch endpoint, protocol, credential, or harness. | -| Cleanup or unmount failure | Quarantine and continue accounting for the allocation; retry the same idempotent unmount-then-cleanup path. | - -Promptfoo treats every non-success as an evaluation error. It does not convert a workspace failure to an empty success, a source fallback, or an implicit retry. +Provider and harness failures that are not workspace failures retain their exact inherited UHP codes. Implementations MUST NOT reuse an inherited code for a workspace condition unless status, retryability, and semantics are identical. No failure may change source kind, source identity, access, working directory, harness, or provider route as a recovery shortcut. ## Distribution and release boundary -`allagentsdev/allagents-gateway` is the implementation and distribution repository. The source initializer, component caches, composition manager, manifest collector, and checkpoint adaptations ship in the AllAgents Gateway image; they are not another network service. - -The supported deployment is one `allagents-gateway` service using `ghcr.io/allagentsdev/allagents-gateway`, durable `/data`, loopback binding by default, and the inherited Codex and OMP backend configuration. Deployments pin an exact image-manifest digest. Release tags identify both the upstream HarnessRouter baseline and downstream revision. Releases produce SBOM and build-provenance attestations and run the upstream UHP conformance suite against the built image. - -A v1 release is complete only when all of the following are demonstrated: +`allagentsdev/allagents-gateway` does not exist at the time of this decision. Before implementation, an organization repository administrator must rename or bootstrap the current `allagentsdev/harnessrouter` repository from the pinned commit, preserve Apache-2.0, `NOTICE`, attribution, and history, add `HarnessRouter/harnessrouter` as the upstream remote, and create a writable implementation branch. -1. Codex and OMP each complete and continue a session through the configured external provider gateway. -2. A mixed composition with multiple Git and multiple OCI entries resolves independent provenance, preserves request order, rejects every overlap before network access, and becomes visible atomically. -3. An OCI entry with at least 2 GiB of expanded source and 100,000 source-visible entries is fetched by direct image digest; its source manifest is verified before layers, layers and whiteouts are applied, the one relative tree is verified, and a harness starts inside it. -4. Identical Git and OCI component identities publish once and singleflight independently, while compositions reuse those components without constructing a request-wide cached tree. -5. Concurrent and later `read_only` sessions bind the same protected component inodes at their destinations, keep writable outer workspaces private, and never include mounted bytes in checkpoint or cleanup archives. -6. `editable` Git and OCI sessions receive inode-independent writable roots. An editable OCI bug-fix run modifies source, continues from a checkpoint, and reports the exact final filesystem change without Git metadata. -7. Manifest comparison proves add, modify, delete, executable/mode, symlink-target, and binary changes across multiple editable roots and the outer workspace. Altering Git commits, indexes, staged state, or rename detection cannot change the evaluated final-tree result. -8. Input and asset paths outside roots work normally; collisions at or below a destination fail before network access. OCI links escaping their owning root and mount failures fail closed. -9. Cancellation, failure, crash recovery, and aggregate-limit tests prove that no partial composition, stale pin, source-kind fallback, or unaccounted allocation becomes visible. -10. A request without `metadata.workspace` passes stock UHP behavior and conformance without invoking workspace acquisition or manifest-based workspace evaluation. +Implementation also requires checked-in local builders for an authenticated OCI registry, catalog, and source fixtures, plus a Linux environment capable of namespace-confined bind mounts and reflink-or-copy isolation. The implementation agent can deliver a pull request using those fixtures and mocked provider transport without production secrets. Provider gateway URL/key, UHP caller credentials, GHCR rights, protected settings, and final publication/deployment are operator-owned inputs supplied through the repository's secret mechanism. -These are release gates, not deferred performance tests. +Workspace code only gates inherited harness start until the binding is `ready` and the working directory is valid; it does not redesign Codex, OMP, or provider authentication. Release validation runs once against the published image read back and pinned by digest. Completion requires stock UHP compatibility, Git/OCI/mixed workspace flows, read-only and editable isolation, canonical change artifacts, continuation, cancellation, expiry/deletion, cache reuse, fresh-volume and same-volume restart, both supported harnesses, UHP conformance, security scans, SBOM, and build provenance. -## Alternatives rejected +## Rejected alternatives | Alternative | Why rejected | |---|---| -| Keep the downstream product and image named HarnessRouter | Obscures which behavior is upstream and falsely suggests downstream extension conformance. | -| Build a new execution gateway from scratch | Duplicates UHP, sessions, workspace lifecycle, streaming, cancellation, files, artifacts, and harness supervision already inherited from HarnessRouter. | -| Put a workspace service in front of the gateway | Splits source and session ownership and cannot safely participate in checkpoint hydration, continuation, or produced-file collection. | -| Keep a mutually exclusive single source object | Prevents first-class mixed Git and OCI workspaces and couples unrelated source lifecycles and cache misses. | -| Let one OCI image declare multiple destination roots | Hides destination ownership until after network access, makes one artifact a multi-root workspace bundle, and prevents independent component caching. | -| Build one cached generation for the complete composition | Rebuilds and duplicates unchanged components whenever one entry or destination changes and singleflights at the wrong granularity. | -| Put source at the workspace root | Collides with runner-owned state, inputs, checkpoints, and the private writable outer workspace. | -| Attach shared source with symlinks | Does not enforce read-only access, exposes backing paths, permits containment escape, and gives cwd and file tools surprising behavior. | -| Share cached inodes with editable sessions | Lets one session mutate another session or the immutable cache and makes checkpoints non-isolating. | -| Use Git as the evaluation diff engine | Fails for OCI trees without Git, mishandles multiple roots and outer files, and lets mutable commits or index state redefine correctness. | -| Ship Git first and defer OCI | Fails the minimum large-source use case and makes release viability depend on repeated working-tree acquisition. | -| Treat OCI as a runtime or benchmark image | Mixes source provenance with tools, services, verifier assumptions, and execution policy. | -| Wait for upstream before implementation | Makes delivery depend on a project we do not maintain and delays evidence needed for an upstream proposal. | -| Add a general plugin or materializer framework | V1 has two explicit entry kinds and no demonstrated need for caller-selectable plugins. | -| Put acquisition instructions in the prompt or a model tool | Makes acquisition model-dependent, non-deterministic, too late to set the initial directory, and unsafe for provenance. | -| Upload every source file through UHP | Pushes acquisition to callers and loses authoritative Git and OCI identity, links, modes, and cache reuse. | -| Make the AllAgents CLI the remote control plane | Couples local developer configuration to an independently deployed service. | -| Add a second Files or changes API | Duplicates inherited behavior instead of adapting it around canonical filesystem manifests. | - -## Deliberate v1 limits - -V1 supports 1 to 128 ordered Git or OCI source entries at pairwise non-overlapping non-root destinations; public HTTPS Git at fixed depth `2`; advertised branch, tag, or default refs; operator-catalogued OCI images selected by direct digests; one relative source tree per OCI image; independent component caches; atomic composition; canonical source and outer-workspace manifests; private writable outer workspaces; read-only bind mounts or private editable copies; `session` and authorized `persistent` retention; and a workspace-relative working directory. - -V1 excludes caller-supplied registry origins or credentials, mutable OCI tags, OCI indexes as source identity, OCI multi-root bundles, transparent source-kind fallback, caller-selected runtime images or benchmark environments, arbitrary materializer commands, private-network Git origins, caller-selected Git depth or TTL, session branching, access or retention changes on continuation, compatibility aliases, deprecated schemas, and public multi-tenant authorization. It adds no AllAgents CLI command and changes no local project workspace configuration. - -Only Codex and OMP are required and release-validated. Other inherited backends, local-profile import, host-profile projection, provider-route override, automatic provider fallback, scoring, datasets, assertions, and evaluation-task orchestration are outside this decision. +| Build a new execution service or a separate workspace service | Duplicates UHP/session behavior or splits ownership away from checkpoint, continuation, collection, and deletion. | +| Use one source object, one multi-root OCI bundle, or one request-wide cached tree | Prevents independent mixed-source identity and reuse and hides destination ownership. | +| Put a source at workspace root | Collides with runner-owned state, inputs, checkpoints, and the writable outer workspace. | +| Use symlinks for shared read-only roots or shared inodes for editable roots | Does not enforce isolation and can expose or mutate backing storage. | +| Use Git as the evaluation diff engine | Cannot represent Git-free OCI roots, multiple roots, outer files, or immutable final-tree semantics. | +| Add a second changes endpoint | Duplicates the inherited Files/artifact surface and bypasses its capture-before-ack behavior. | +| Add a general materializer plugin framework | V1 has two explicit source kinds and no caller-selectable implementation need. | +| Wait for upstream acceptance | Delays downstream evidence and makes delivery depend on a project we do not control. | ## Consequences -AllAgents Gateway remains one execution, workspace, and session control plane while explicitly preserving its derivation from upstream HarnessRouter. UHP clients that do not request workspace initialization retain stock compatibility. The downstream product, repository, image, and service have one unambiguous identity. - -Independent component caching makes mixed-source composition efficient and lets repeated requests reuse unchanged Git or OCI trees. Atomic composition and pre-network destination ownership add reservation and rollback complexity, but prevent partial or network-dependent workspace layouts. - -Mandatory OCI support makes v1 more substantial than a Git clone hook, but it makes large-source evaluations viable. Read-only bind mounts share protected bytes without making the outer workspace read-only. Private writable copies give Git and OCI identical editable behavior, including OCI-backed bug-fix evaluations. - -Canonical filesystem manifests add scanning and digest cost, but establish one source-kind-independent definition of final state. Evaluations no longer depend on whether Git metadata exists, whether an agent modified an index or commit, or whether rename inference agrees. +AllAgents Gateway remains one execution, workspace, and session control plane. Independent component caching permits reuse without a global composition object. Visibility gating and mount-aware lifecycle work add implementation complexity but prevent partial workspaces, mutable shared state, and checkpoint traversal into caches. -Checkpoint metadata and storage become root-aware: immutable roots are referenced, editable roots are preserved separately, and outer state remains independent. The operator assumes finite capacity management for acquisition caches, staging, immutable components, editable copies, persistent sessions, tombstones, and quarantined deletion failures. Protected or uncertain state is never advertised as free capacity. +Canonical manifests and change artifacts add bounded filesystem scanning and hashing, but give Git and OCI one evaluator-visible definition of state. Promptfoo can reconstruct results entirely from ordered UHP artifacts, regardless of Git metadata or index state. -Provider credential lifecycle remains outside AllAgents Gateway. The distribution depends on the external OAuth-to-OpenAI-compatible gateway, while each harness sees only a brokered short-lived credential. +Mandatory OCI support and Linux mount/copy capabilities make v1 substantial, but they satisfy the large-source requirement while preserving safe shallow Git history for agent convenience and Git-free OCI operation. ## Reconsider when -Revisit this decision if: - -- UHP or upstream HarnessRouter adopts an equivalent ordered multi-source contract; -- upstream workspace lifecycle changes so the extension point no longer preserves one authoritative session workspace; -- the host cannot enforce namespace-confined read-only bind mounts and inode-independent editable copies; -- large OCI materialization, canonical manifest comparison, or component cache reuse cannot meet finite release limits; -- continuation cannot fail closed without source reacquisition; -- source acquisition requires a stronger isolation boundary; -- public multi-tenancy or caller-owned private-source credentials become requirements; -- Codex or OMP can no longer use the external provider gateway's required compatible surface; or -- another UHP implementation offers a materially smaller and more stable integration surface. +Revisit this decision if upstream adopts an equivalent ordered multi-source contract, its lifecycle removes these extension seams, the deployment platform cannot enforce the required isolation, bounded OCI materialization or manifest collection cannot meet release limits, continuation cannot fail closed without reacquisition, or another UHP implementation offers a materially smaller and equally stable integration surface. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 1731420b..d497f363 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -12,114 +12,143 @@ execution: code ## Goal -Create AllAgents Gateway as the downstream distribution derived from -`HarnessRouter/harnessrouter`. A first-turn UHP request MAY declare an ordered -set of independent Git and OCI source trees. The gateway MUST acquire and cache -each component independently, then compose every requested tree at its declared -non-root destination in the existing private session workspace. A composition -MUST become visible atomically: either every source root is ready at the exact -requested identity or none is visible. - -The public downstream names are: - -- repository: `allagentsdev/allagents-gateway`; -- image: `ghcr.io/allagentsdev/allagents-gateway`; -- service: `allagents-gateway`; and -- product: **AllAgents Gateway**. - -The fork MUST retain `HarnessRouter/harnessrouter` as its upstream remote, the -recorded fork point, the upstream license, and required attribution. Upstream -changes MUST be taken selectively and reviewed against the downstream contract; -upstream acceptance is not a release dependency. - -Requests without `metadata.workspace` MUST remain on the stock UHP path. Their -root-Git hydration, produced-file behavior, checkpointing, continuation, -provider routing, cancellation, retention, and cleanup MUST remain compatible -with the pinned upstream baseline. The new component, composition, manifest, -and workspace-aware lifecycle apply only when `metadata.workspace` is present -on a new session. - -## Stock behavior and adaptation boundary - -The pinned upstream baseline owns session identity, one private writable -workspace per session, hydrate/checkpoint, UHP inputs, generated instructions, -`.harness`, HOME, skills, plugins, MCP configuration, conversation state, -Files/artifacts, cancellation, TTL, deletion, restart reconciliation, harness -supervision, and provider brokering. Stock fresh hydration initializes an empty -root Git repository. Stock produced-file collection and checkpointing use that -root repository and may traverse the whole workspace. - -Phase 1 MUST characterize the exact upstream call paths and ordering rather than -assuming that summary is exhaustive. The workspace-backed path MUST reuse the -same session and runner lifecycle but replace root/nested Git correctness with -filesystem-manifest correctness. For a workspace-backed session: - -- the outer workspace remains private and writable; -- source destination ownership and identities are immutable for the life of the - session, while `editable` tree contents may change; -- root Git, nested Git commits, indexes, status, diff, and rename detection MUST - NOT determine produced changes, evaluation results, or checkpoint correctness; -- Git metadata MAY be present as acquisition data and agent convenience only; -- OCI components need not contain Git metadata; -- Files collection and final evaluation MUST compare canonical filesystem - manifests for every source root and the writable outer workspace; -- checkpoint, restore, Files walks, and cleanup MUST understand source - boundaries and MUST NOT accidentally traverse a read-only mount; and -- no second workspace service, session database, Files API, scheduler, or - external materializer service is introduced. - -## Product and ownership boundary +Create AllAgents Gateway as a downstream distribution of +`HarnessRouter/harnessrouter`. On the first request of a new UHP session, a +caller MAY declare an ordered set of independent Git and OCI source trees. The +gateway acquires and caches each component independently and places each tree at +its requested non-root destination in the existing private session workspace. +The application exposes the workspace to Files, checkpoint, provider, or harness +consumers only after every root verifies. + +Requests without `metadata.workspace` remain on the pinned stock path. Workspace +support introduces no second session database, Files API, scheduler, execution +protocol, global composition cache, or materializer service. + +## Implementation-handoff prerequisites + +### Target bootstrap: operator-owned and required before code work + +`allagentsdev/allagents-gateway` does not exist yet. An organization repository +administrator, not the implementation agent, MUST complete this bootstrap: + +1. Rename/bootstrap the current `allagentsdev/harnessrouter` repository as + `allagentsdev/allagents-gateway` from pinned HarnessRouter commit + `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`. +2. Preserve the complete available history, Apache-2.0 `LICENSE`, `NOTICE`, + copyright and attribution. HarnessRouter is Apache-2.0 with a NOTICE; it is + not MIT. +3. Set `origin` to the renamed writable repository and retain + `HarnessRouter/harnessrouter` as `upstream`. Record the pinned commit as the + downstream fork point. Upstream acceptance is not a dependency. +4. Create writable branch `feat/workspace-composition` at that commit and grant + the implementation agent normal pull-request rights to the branch. +5. Keep the downstream distribution names fixed: + - repository: `allagentsdev/allagents-gateway`; + - image: `ghcr.io/allagentsdev/allagents-gateway`; + - service: `allagents-gateway`; and + - product: **AllAgents Gateway**. + +The implementation environment also requires these checked-in local test +capabilities, with no external credentials: + +- `oci-auth-registry`: an authenticated local OCI registry fixture; +- `oci-catalog-builder`: a builder for the server-owned catalog fixture; and +- `oci-source-fixture-builder`: a builder for deterministic image manifests, + source manifests, layers, whiteouts, and malicious extraction fixtures. + +The Linux integration runner MUST support read-only bind mounts and either safe +reflinks or inode-independent private copies. If reflinks are unavailable, the +implementation MUST use the private-copy path; it MUST NOT hard-link mutable +files. These are implementation prerequisites, not caller-visible options. + +### Secrets and actions outside the implementation agent + +The implementation agent can deliver the complete implementation PR using local +fixtures and mocked provider transport. The following remain operator-owned: + +- the external provider gateway URL and key; +- a real UHP caller credential; +- GHCR publish rights; +- protected repository, registry, network, and deployment settings; and +- final image publication and deployment. + +Release gates that require these values run only after an operator supplies them +through the repository's existing secret mechanism. The plan MUST NOT put +secrets in source, fixtures, build arguments, logs, artifacts, session metadata, +or harness environments. + +## Pinned stock behavior and exact adaptation seams + +The baseline owns UHP request/stream handling, authentication, idempotency, +provider brokering, harness supervision, one private workspace per session, +inputs and generated assets, checkpoint/artifact/control records, cancellation, +and deletion. + +The implementation MUST begin by confirming these pinned behaviors at the +named seams, then adapt them rather than adding parallel machinery: + +- Fresh stock hydration initializes Git at the workspace root. +- Git powers only the stock produced-file listing and cursor. It is not the + workspace storage or restore mechanism. +- Stock checkpointing commits and then tars the directory; hydration restores + the tar. +- Runner `_produced_list` returns produced files and `_produced_ack` advances the + produced cursor. +- Gateway `_collect_produced` captures artifacts before calling the ACK path. +- `BACKING.workspace` exposes live files through `RunnerWorkspaceFiles` and + checkpoint files through `CheckpointWorkspaceFiles`. +- The durable graph uses the `HarnessSession` vertex plus separate checkpoint, + artifact, and control records. Workspace state extends those seams; it does + not collapse them into a new session store. +- Stock reaps cached live workspaces and supports explicit deletion of durable + sessions. It does not provide a durable-session TTL. + +For workspace-backed sessions, `_produced_list` and `_produced_ack` remain the +runner protocol, `_collect_produced` remains the capture-before-ACK gateway +boundary, and `BACKING.workspace` remains the Files abstraction. Only the +workspace-backed produced-file implementation changes from root-Git cursoring +to canonical manifest cursoring. Stock requests retain root-Git listing and all +other pinned behavior. + +Provider routing remains inherited. Workspace code only delays harness start +until the workspace binding is ready and the working directory has verified. +There is no provider-specific workspace implementation phase. + +## Scope and invariants ### In scope -- A strict first-turn-only `metadata.workspace` extension to UHP. -- A closed ordered `sources` array containing 1 to 128 independent Git, OCI, or - mixed source entries. -- Exactly one source tree and one required non-root `destination` per entry. A - monorepo is one tree, not an implicit bundle of roots. -- Canonical public HTTPS Git acquisition with exact commit resolution and a - mandatory depth-2 fetch policy. -- Operator-cataloged OCI source-tree transport selected by a direct image - manifest digest and a canonical source manifest digest. -- Independent component cache and singleflight, followed by atomic composition. +- A strict, closed, first-turn-only `metadata.workspace` request extension. +- An ordered array of 1 to 128 Git, OCI, or mixed source entries. +- Exactly one tree and one required non-root destination per source. +- Canonical public HTTPS Git acquisition at an exact resolved commit with fixed + depth 2 and safe bounded `.git` metadata for agent convenience. +- Operator-cataloged OCI source trees selected by exact image-manifest and + source-manifest digests, including Git-free trees. +- Per-component cache/singleflight followed by one session-local resolved plan. - Identical `read_only` and `editable` attachment semantics for Git and OCI. -- Canonical baseline source manifests for all components and a canonical - baseline for the writable outer workspace. -- Manifest-based add, modify, delete, mode, executable, symlink, and binary - change detection for workspace-backed sessions. -- Mount-aware checkpoint and continuation, including separate preservation of - editable roots. -- Per-source and request-aggregate resource limits. -- Codex and OMP through the existing separately operated - OAuth-to-OpenAI-compatible provider gateway and brokered credentials. -- Direct Promptfoo proof against the built image, including multiple OCI - components, a mixed Git/OCI composition, editable OCI mutation, and cache - reuse. -- Digest-pinned GHCR publication with SBOM and build provenance. - -### Explicit non-goals - -- Caller-selected runtime/container images or benchmark environments. An OCI - image is only a transport and cache unit for one source tree; it is never the - runtime workspace and never a multi-root workspace bundle. -- Caller-provided registry origins, credentials, headers, certificates, mirrors, - proxy settings, Git configuration, source commands, or materializer hooks. -- A public component-cache or composition API. -- Silent Git-to-OCI, OCI-to-Git, ref, digest, mirror, deepening, full-clone, or - provider fallback. -- Git submodule initialization, Git LFS hydration, checkout filters, or hook +- Canonical manifests, cursor-based change projection, mount-aware checkpoint + and continuation, finite expiry, and explicit deletion. +- Existing Codex and OMP paths through the existing provider broker. + +### Non-goals + +- Caller-selected runtime images, registry origins, credentials, mirrors, + transport settings, commands, materializer hooks, provider routes, models, or + resource limits. +- A public component-cache or composition API, a global composition record, + cache, identifier, or compatibility aliases for obsolete workspace schemas. +- Git-to-OCI, OCI-to-Git, ref, digest, registry, provider, deepening, full-clone, + or history fallback. +- Git submodule initialization, LFS hydration, checkout filters, or hook execution. -- Requiring a Git repository at workspace root or inside an OCI tree. -- Provider login, refresh, or repair in AllAgents Gateway. -- An AllAgents CLI, profile import, local gateway command, or `workspace.yaml` - change. -- Upstream acceptance as a release condition. +- Requiring Git at workspace root or in OCI trees. +- A second Files endpoint or an AllAgents CLI/profile format change. -## Request contract +## Public request and response contract -`metadata.workspace` MUST be accepted only while creating a new session. The -schema uses snake_case, has no nested schema version, and is closed at every -object boundary. +`metadata.workspace` is accepted only on the first request that creates a new +session. All objects are closed, use snake_case, and have no nested version. ```json { @@ -127,7 +156,6 @@ object boundary. "harness_id": "allagents-codex", "workspace": { "access": "editable", - "retention": "session", "sources": [ { "kind": "git", @@ -149,12 +177,9 @@ object boundary. } ``` -The exact shape is: - ```text metadata.workspace = { access: "read_only" | "editable", - retention?: "session" | "persistent", sources: Array< | { kind: "git", @@ -165,8 +190,8 @@ metadata.workspace = { | { kind: "oci", snapshot_name: string, - image_manifest_digest: string, - source_manifest_digest: string, + image_manifest_digest: "sha256:<64 lowercase hex>", + source_manifest_digest: "sha256:<64 lowercase hex>", destination: string } >, @@ -174,903 +199,571 @@ metadata.workspace = { } ``` -The obsolete singular `source`, `kind: "repositories"`, `repositories`, -`kind: "workspace_snapshot"`, and `workspace_manifest_digest` fields MUST NOT be -accepted as aliases. There is no compatibility schema. - -### Validation rules - -1. `access` and `sources` are REQUIRED. `retention` defaults to `session`. - `sources` MUST contain 1 to 128 entries and preserves request order. -2. Every entry MUST materialize exactly one tree at its own required - `destination`. Multiple Git entries, multiple OCI entries, duplicate - component identities at different destinations, and mixed Git/OCI entries - are valid. -3. Every destination and supplied `working_directory` MUST be a relative POSIX - path with no empty, `.`, `..`, ambiguous, or platform-specific component and - no link escape. A destination that normalizes to the workspace root is - invalid. An omitted `working_directory` means the root, represented as `.` - only in the response. -4. Destinations MUST be pairwise non-overlapping after normalization: no two are - equal and neither is an ancestor of another. Their ownership is determined - entirely from the request; OCI content MUST NOT add or move a root. -5. UHP input paths, generated assets, reserved runner paths, credential paths, - cache paths, attachment state, and checkpoint state MUST be normalized and - checked against every destination before source resolution, network traffic, - cache lookup, session workspace writes, or component claims. Any path equal - to or below a source destination MUST be rejected for both access modes. - Inputs and generated assets outside all source destinations remain valid in - the writable outer workspace. -6. Destination ancestor scaffolding MAY coexist only with independently valid - outer directories. A file, symlink, generated asset, input, or reserved path - at an ancestor that prevents safe directory scaffolding MUST fail before - network work. -7. `working_directory` syntax is validated before network work. After atomic - composition it MUST resolve, without link escape, to a real directory either - in the outer workspace or within exactly one source root. -8. A Git `url` MUST be canonical public HTTPS under the deployment egress policy. - User information, query strings, fragments, credentials, alternate - transports, ambiguous encodings, and caller transport options are forbidden. - `ref`, when present, is bounded and resolves only through advertised branch - or tag semantics. -9. An OCI entry MUST provide an operator-catalog `snapshot_name`, a direct - `sha256:` image manifest digest, and a canonical `sha256:` - `source_manifest_digest`. Tags, indexes/lists, mutable references, and caller - registry coordinates are forbidden. -10. Unknown fields, wrong-kind fields, mixed fields within one entry, invalid - digests, and obsolete schema fields MUST fail closed. -11. Metadata bytes, nesting, strings, path lengths, source count, and input count - MUST be bounded during parsing. Persistent retention MUST be authorized - before cache lookup or network work. -12. A continuation selected by `previous_response_id` or the existing recovery - mechanism MUST omit `metadata.workspace`. Supplying it on a reused session - MUST fail before hydrate, component lookup, provider traffic, or TTL changes, - even if it equals the stored descriptor. A stock session cannot become a - workspace-backed session later. -13. Workspace fields MUST NOT contain commands, environment variables, resource - limits, provider settings, model settings, registry settings, or harness - settings. - -Because every source declares its destination, all destination ownership, -source-source overlap, and input/asset collision decisions are request-decidable -and MUST complete before any network request. OCI source-manifest validation is -content validation, not destination discovery. - -## Source and request limits - -Bounds MUST be enforced while streaming, before allocation whenever the size is -known, and both per source and across the whole request. Deployment policy MAY -lower a bound but MUST NOT raise these v1 ceilings without a contract revision. -A lower runtime capacity is a coded capacity failure, not a schema change. - -| Resource | Per source ceiling | Request aggregate ceiling | +There is no request `retention` field and no `persistent` mode. The obsolete +singular `source`, `kind: "repositories"`, `repositories`, +`kind: "workspace_snapshot"`, and `workspace_manifest_digest` forms are +rejected, not aliased. + +Every workspace-backed session receives the same operator-configured finite, +non-extendable expiry at creation. Replay, polling, turns, attachment, and +continuation MUST NOT move it. Explicit deletion remains available. + +After the ready transition, responses return this exact sanitized shape: + +```text +metadata.workspace = { + access: "read_only" | "editable", + working_directory: string, + expires_at: RFC3339 timestamp, + sources: Array< + | { + kind: "git", + url: string, + destination: string, + resolved_commit: string, + depth: 2, + source_manifest_digest: "sha256:<64 lowercase hex>", + requested_ref?: string + } + | { + kind: "oci", + snapshot_name: string, + destination: string, + image_manifest_digest: "sha256:<64 lowercase hex>", + source_manifest_digest: "sha256:<64 lowercase hex>" + } + > +} +``` + +`working_directory` is always present and uses `.` for the outer root. There is +no public `retention`, `effective_descriptor_digest`, `composition_id`, or +per-source `component_id`. Catalog coordinates, private cache keys, mirrors, +host paths, leases, mounts, credentials, and policy identifiers are also +private. Replays and continuations return the stored ready metadata and never +re-resolve mutable Git refs. + +### Validation and ownership + +Validation MUST complete every request-decidable check before source network +traffic, cache lookup, component claim, workspace write, or expiry mutation: + +1. Require `access` and `sources`; accept 1 through 128 sources in request order. +2. Require one non-root relative POSIX `destination` per source. Reject empty, + `.`, `..`, non-NFC, ambiguous, platform-specific, overlong, or link-escaping + components. +3. Reject equal or ancestor/descendant destinations. Repeated component + identities at different non-overlapping destinations are valid. +4. Normalize UHP inputs, generated assets, runner-reserved paths, credential and + control paths, and every destination through one ownership validator. Reject + a file, symlink, input, asset, reserved path, or unsafe ancestor at or below a + destination in either access mode. Outer paths remain writable and valid. +5. Validate optional `working_directory` syntax before network work. After all + attachments, require it to resolve without link escape to one real directory. +6. Accept only canonical public HTTPS Git URLs allowed by deployment egress + policy. Reject userinfo, query, fragment, ambiguous encodings, alternate + transports, and caller Git options. A ref resolves only through advertised + default, branch, or tag semantics. +7. For OCI require a catalog `snapshot_name` and direct SHA-256 image/source + manifest digests. Reject tags, indexes/lists, caller registry coordinates, + and mutable references. +8. Bound metadata bytes, nesting, strings, paths, source count, and input count + while parsing. Reject unknown and wrong-kind fields. +9. Reject `metadata.workspace` on every replay, reused session, and continuation + before hydrate, cache lookup, provider traffic, or any lifecycle mutation. + +### Resource ceilings + +Deployment policy MAY lower but MUST NOT raise these v1 ceilings without a +contract revision: + +| Resource | Per source | Request aggregate | |---|---:|---:| -| Expanded source bytes | 32 GiB | 64 GiB | +| Expanded bytes | 32 GiB | 64 GiB | | Source-visible entries | 500,000 | 1,000,000 | | Compressed Git pack or OCI layer bytes | 8 GiB | 16 GiB | -| Regular file size | 4 GiB | 4 GiB | +| Regular-file bytes | 4 GiB | 4 GiB per file | | Path | 4096 UTF-8 bytes / 128 components | same per path | -| Acquisition and materialization wall time | bounded by operator policy | bounded by session policy | - -Each OCI source additionally permits at most 64 distributable tar/gzip/zstd -layers, a 4 MiB image manifest, a 128 MiB canonical source manifest, and 1 MiB -per PAX or extended header. For each layer, each OCI source, and the request -aggregate, `expanded_bytes / max(compressed_bytes, 1)` MUST NOT exceed `100`. -Git object, checkout, inode, output, and filesystem quotas MUST feed the same -per-source and aggregate accounting rather than becoming unbounded exceptions. - -## Canonical manifests and change semantics - -### Component baseline source manifest - -Every published Git or OCI component MUST have a canonical baseline source -manifest. It is keyed by normalized path relative to that component root and -contains, for every included filesystem entry: - -- normalized relative path and entry type; -- regular-file content digest and size; -- executable bit and the platform-normalized mode semantics needed to reproduce - observable permissions; -- symlink target bytes after canonical encoding; and -- any bounded hardlink representation required by the extraction policy. - -Ordering and serialization MUST be canonical. Directory and link semantics MUST -make type changes observable. Binary files use content digests exactly like text -files; no text decoding or line diff is required for correctness. The component -manifest digest is part of the immutable component publication. - -For Git, the runner computes the manifest from the verified detached depth-2 -checkout. For OCI, the canonical manifest named by `source_manifest_digest` MUST -be fetched and verified before any layer request. Its paths are relative to the -single requested destination. It MUST declare the final types, modes, sizes, -content digests, and links that layer application is expected to produce. Final -extraction MUST exactly match it before publication. - -Git administrative state MAY be acquired and retained for agent convenience, -but it is never a change baseline. OCI trees MAY omit it entirely. A source -manifest MAY verify declared Git administrative files when present; the change -collector MUST exclude every `.git` entry from reported changes. - -### Outer workspace baseline - -For a workspace-backed session, fresh hydration MUST create the writable outer -workspace and apply allowed UHP inputs, generated instructions, harness assets, -and other initial session material outside source destinations. Immediately -before the first harness process starts, the runner MUST publish a canonical -outer baseline manifest using the same path/type/content/mode/link model. It -MUST exclude source destinations and runner-owned state. - -The session binding MUST preserve the exact component baseline manifests and the -outer baseline across continuation. A continuation MUST NOT silently regenerate -a baseline from already-mutated content. - -### Final-tree comparison - -At every required collection/evaluation boundary, the runner MUST walk the -writable outer workspace without crossing a source mount and MUST walk every -source root through its declared ownership boundary. It MUST compare each final -manifest to the corresponding persisted baseline and report the union of: - -- additions; -- content modifications, including binary changes; -- deletions; -- executable or other observable mode changes; and -- symlink additions, removals, retargeting, and type transitions. - -Rename inference is OPTIONAL. Delete-plus-add is correct; final-tree equality is -authoritative. Root Git, nested Git, commits, index state, ignored-file rules, -and Git rename detection MUST NOT be correctness sources. A Git command MAY be -available to the agent, but it MUST NOT change evaluator results. - -The collector MUST normalize each reported path into workspace-relative form, -assign it to exactly one owner (outer workspace or one source destination), and -deduplicate it. It MUST exclude `.git`, runner-owned state, credentials, caches, -attachment evidence, checkpoint metadata, harness-private ephemeral state, and -the component store. Existing count, size, and artifact limits still apply to -reported outputs. Initial source and outer baseline entries MUST NOT be reported -merely because composition or hydration created them. - -`read_only` source roots are compared as an integrity check and MUST remain equal -to their component baselines. `editable` Git and OCI roots are compared with the -same algorithm and MUST report mutations identically. Bug-fix evaluations MUST -request `access: "editable"`. - -## Identity, cache, and generation design - -A **component** is one verified immutable source tree. A **composition** is an -ordered mapping of exact component identities to normalized destinations. A -**session attachment** applies one composition under an access mode to one -private outer workspace. None is a runtime image or a second session. - -### Component identities - -A Git component identity MUST include the canonical URL, exact resolved commit, -fixed depth `2`, Git acquisition-policy revision, materializer revision, and -canonical baseline source-manifest digest. The request destination, ref spelling, -access, session, harness, provider, retention, and working directory MUST NOT -fragment the component cache. - -An OCI component identity MUST include the operator catalog identity, -`snapshot_name`, direct `image_manifest_digest`, canonical -`source_manifest_digest`, OCI validation-policy revision, materializer revision, -and verified baseline source-manifest digest. Registry origin and credentials -MUST remain private. Destination and session concerns MUST NOT fragment the -component cache. - -The canonical composition identity MUST reference, in request order, each exact -component publication identity paired with its normalized destination, plus the -layout/materializer contract revision. Reordering entries therefore changes the -composition identity even when component bytes are the same. The composition -record contains references and evidence, not another copy of component bytes. - -### Git acquisition cache - -The runner MUST maintain one operator-only bare shallow acquisition mirror per -canonical Git URL. Every write to that mirror MUST be serialized; concurrent -resolution/fetch for the same normalized request MUST singleflight. On a miss, -the worker MUST resolve the advertised default, branch, or tag to an exact -commit, fetch exactly depth 2, verify the fetched tip and shallow boundary, and -atomically import the bounded result. It MUST NOT deepen, unshallow, full-clone, -fetch an arbitrary object ID, choose another ref, or fall back to OCI. - -An immutable self-contained component checkout MUST be exported without -alternates or writable links to the mirror. It MUST preserve enough normalized -`.git` data for recent offline `git log`, parent inspection, blame where the -shallow history permits, and diff. A merge tip MUST retain both fetched parent -edges when the server supplies them at depth 2. Submodule gitlinks and LFS -pointer-backed content MUST fail rather than invoke helpers. - -A hit for the same exact component identity MAY advertise a mutable ref to prove -that it still resolves to that commit, but MUST perform zero pack acquisition, -checkout, tree copy, baseline recomputation, or publication. Separate counters -MUST distinguish advertisement from source-byte transfer. - -### OCI component cache - -The deployment owns a bounded catalog. Each `snapshot_name` maps to one -operator-controlled registry/repository origin, credential reference, TLS and -redirect policy, allowed media types, and resource policy. Those values MUST NOT -appear in the request, session workspace, logs, public provenance, or harness -environment. - -The worker MUST require a direct image manifest digest and reject tags, -indexes/lists, mutable references, and catalog mismatches. It MUST fetch and -verify the image manifest, config, and canonical source manifest before any -layer request. It MUST validate the source manifest's relative paths, types, -sizes, modes, content digests, links, declared layer requirements, and per-source -and aggregate limits before downloading layers. - -Layers MUST be streamed, digest-checked, and applied in order with correct file -and opaque-directory whiteout semantics. Whiteouts are instructions and MUST -NOT appear in the publication. Extraction MUST reject absolute paths, traversal, -NULs, ambiguous separators, conflicting duplicates, devices, FIFOs, sockets, -unsafe sparse files, unsupported types, and unbounded metadata. Symlinks and -hardlinks MUST remain within their owning source root; links to the outer -workspace or another source are invalid. The completed tree MUST exactly match -the canonical source manifest before atomic publication. - -An exact OCI component-cache hit MUST perform zero registry manifest/config/ -source-manifest/layer requests, extraction, tree copy, baseline recomputation, or -publication. An OCI component is always one tree; an image that encodes multiple -workspace roots or files outside that tree's relative manifest MUST fail. - -### Singleflight, publication, and reuse - -Each component identity MUST have its own durable singleflight claim and random -private staging directory. Independent components MAY acquire concurrently -within request and operator limits. A waiter cancellation MUST detach only that -waiter while another live request still needs the build. When no waiter remains, -the bounded builder MAY be cancelled. Failed, timed-out, cancelled, partial, or -unverified staging MUST never become attachable. - -After streamed accounting and full manifest verification, publication MUST use -an atomic rename and record ownership, policy revisions, manifest digest, -publication epoch, and completeness. Published component bytes and manifests -MUST be immutable. Startup reconciliation MUST quarantine uncertain state. A -lease/reference MUST protect a component from cleanup; cleanup MUST remove only -complete unreferenced publications and MUST NOT invalidate an attached session. - -A composition resolver MUST wait for every independently claimed component, -verify the ordered identities and destinations, and commit one immutable -composition record. One component failure MUST roll back request-local staging -and references without invalidating successful shared components needed by -other sessions. No partial composition can be attached. - -## Access and atomic attachment - -Both acquisition kinds MUST implement the same access behavior. - -- `read_only`: lease each immutable component and bind-mount its root at the - declared destination with kernel-enforced read-only, `nodev`, and `nosuid` - semantics while preserving required execute bits. The session MUST receive no - writable backing descriptor, alias, overlay/copy-up path, mirror path, or - component-store path. Symlinks are forbidden as an attachment mechanism. -- `editable`: lease each immutable component and create an inode-independent - private writable copy or safe reflink at the declared destination. A later - write MUST NOT mutate the cache or any sibling session. Hard-linked mutable - files and writable aliases are forbidden. - -The runner MUST construct fresh and continued workspace-backed sessions in a -private, non-runnable staging workspace or private mount namespace. It MUST -prepare the outer state, all mountpoint scaffolding, every read-only mount and -editable copy, manifest evidence, and the validated working directory there. It -MUST expose no Files, checkpoint, provider, or harness consumer until every -entry verifies. A single atomic workspace-path publication or namespace handoff, -paired with the session ready transition, MUST make all roots visible together. -Any failure MUST unwind mounts in reverse order, remove private copies/staging, -release request-local references, and leave no runnable or externally visible -partial workspace. - -Matching read-only sessions MUST bind the same immutable component bytes. -Editable sessions MUST start from those same publications without reacquisition -but have independent inodes. The outer workspace remains private and writable -in both modes. - -## Session binding and public provenance - -The existing session/checkpoint record MUST store: - -- the canonical effective descriptor and digest; -- the ordered resolved source plan; -- each exact component identity, publication epoch, and baseline source-manifest - digest; -- the composition identity and ordered destination ownership map; -- `access`, effective `retention`, normalized `working_directory`, and selected - harness/provider binding; -- the outer baseline manifest identity; -- attachment evidence per destination, including mount/copy method, filesystem - identity, read-only protection or editable inode independence; -- checkpoint identities for writable outer state and each editable source root; -- exact sanitized public provenance; and -- existing expiry, deletion, and lifecycle state. - -The public `metadata.workspace` response MUST contain exactly `access`, -`retention`, `working_directory`, `effective_descriptor_digest`, -`composition_id`, ordered `sources`, and `expires_at`. `working_directory` is -always present and uses `.` for workspace root. `expires_at` is `null` only for -authorized persistent retention. - -Each public Git source entry contains `kind: "git"`, normalized public `url`, -`destination`, exact `resolved_commit`, `depth: 2`, `component_id`, -`source_manifest_digest`, and `requested_ref` only when supplied. Each public -OCI entry contains `kind: "oci"`, `snapshot_name`, `destination`, exact -`image_manifest_digest`, exact `source_manifest_digest`, and `component_id`. -Private component keys, catalog origins, mirrors, credentials, host paths, mount -IDs, leases, policy identities, and attachment paths MUST NOT be public. Replay -and continuation return the stored committed object and MUST NOT re-resolve -mutable references. +| Acquisition/materialization time | bounded operator policy | bounded session policy | + +Each OCI source permits at most 64 distributable tar/gzip/zstd layers, a 4 MiB +image manifest, a 128 MiB source manifest, and 1 MiB per PAX/extended header. +For every layer, source, and request, `expanded_bytes / max(compressed_bytes, 1)` +MUST NOT exceed 100. Git objects, checkout, filesystem entries, output, and +checkpoint accounting feed the same per-source and aggregate enforcement. + +## Canonical source-manifest v1 + +The pinned media type is +`application/vnd.allagents.source-manifest.v1+json`. + +The exact bytes are the RFC 8785 JSON Canonicalization Scheme encoding of: + +```text +{ + version: 1, + entries: Array< + | {path: string, type: "directory"} + | { + path: string, + type: "file", + size: integer, + sha256: "sha256:<64 lowercase hex>", + executable: boolean + } + | {path: string, type: "symlink", target: string} + > +} +``` + +The manifest digest is SHA-256 over those exact canonical bytes. Entries sort by +the UTF-8 bytes of their NFC-normalized relative POSIX `path`. The root entry is +omitted; empty directories are represented. Duplicate paths, non-UTF-8 or +non-NFC names, empty/`.`/`..` components, type conflicts, and unsupported types +fail validation. Regular files normalize to 0644 or 0755 according to +`executable`; all other mode bits are outside this schema. File SHA-256 is over +exact content bytes. + +A symlink target is a UTF-8 NFC string whose resolution from the symlink's parent +stays within the owning root. Absolute, escaping, malformed, or cyclic targets +that cannot be safely materialized fail. Safe in-root OCI hardlinks MAY be +materialized as ordinary files and are not a manifest type. `.git` entries MAY +be covered by source integrity manifests, but `.git` is always excluded from +public change reporting. + +Git acquisition computes this manifest from the verified detached depth-2 tree. +OCI fetches and validates the named source manifest before requesting any layer, +applies standard OCI image/layer/whiteout semantics, and requires the extracted +final tree to match exactly. + +## Public change projection on the existing Files/artifact surface + +No endpoint is added. Workspace-backed collection replaces only root-Git +produced-file listing behind the existing runner/gateway seams. + +At ready time, persist canonical manifest cursors for the writable outer tree +(excluding source destinations) and each editable source root. At every terminal +collection: + +1. `_produced_list` compares the last acknowledged cursor with fresh final + manifests for the outer and editable roots. Traversal is no-follow, bounded, + owner-aware, and does not cross mounts. +2. Read-only roots are not walked. They are trusted from immutable component + identity plus freshly revalidated read-only mount evidence. +3. The runner projects add, modify, and delete operations. Content, type, + executable-mode, and symlink-target changes are `modify`; rename is + `delete` plus `add`. Git status, commits, indexes, ignore rules, and rename + inference do not affect the result. +4. Gateway `_collect_produced` captures every added/modified regular file through + the existing artifact path. It then captures one server-generated artifact + named `workspace-changes-.json`. +5. Only after all file artifacts and the change artifact are durable does the + gateway call `_produced_ack`. ACK persists the new outer/editable cursor + manifests. A retry before ACK reproduces the same logical changes. + +The change artifact media type is +`application/vnd.allagents.workspace-changes.v1+json`. Its exact bytes are RFC +8785 canonical JSON: + +```text +{ + version: 1, + entries: Array<{ + path: string, + operation: "add" | "modify" | "delete", + before?: + | {type: "directory"} + | {type: "file", size: integer, sha256: "sha256:<64 lowercase hex>", executable: boolean} + | {type: "symlink", target: string}, + after?: + | {type: "directory"} + | {type: "file", size: integer, sha256: "sha256:<64 lowercase hex>", executable: boolean} + | {type: "symlink", target: string}, + file_id?: string + }> +} +``` + +Entries sort by UTF-8 bytes of NFC-normalized workspace-relative `path`. +`before` is absent for `add`; `after` is absent for `delete`. `file_id` is +required exactly when an added or modified `after` value is a regular file and +references its already-durable existing-path artifact; it is otherwise absent. +The artifact excludes `.git`, runner/control state, credentials, caches, +checkpoint metadata, and component-store paths. Promptfoo consumes ordered +change artifacts to reconstruct final state. + +## Private identity, cache, and publication + +A component is one verified immutable source tree. There is no durable or cached +composition object. The session binding contains the ordered resolved source +plan and becomes visible in one downstream `ready` transition after every root +verifies. + +Private component keys MUST be computable before materialization: + +- Git key: canonical URL + exact resolved commit + depth `2` + one + cache-schema revision. +- OCI key: catalog entry identity + exact image-manifest digest + exact + source-manifest digest + one cache-schema revision. + +Destination, ref spelling, access, session, harness, provider, working directory, +and expiry do not fragment component keys. The recomputed canonical baseline +digest is evidence required to publish or reuse a component; it is not a cache +key input. Do not add separate acquisition-policy, materializer, publication, +epoch, or baseline-digest dimensions to the key. + +### Git acquisition + +Maintain one operator-only bare shallow acquisition mirror per canonical URL and +serialize its writes. Resolve the advertised default, branch, or lightweight or +annotated tag to an exact commit, fetch with fixed depth 2, verify tip and shallow +boundary, and export a self-contained detached checkout with no alternates or +writable mirror links. Preserve safe bounded `.git` metadata sufficient for +recent offline log, parent inspection, blame where shallow history permits, and +diff. Preserve available merge parents within depth 2. + +Disable interactive credentials, hooks, filters, alternates, alternate +protocols, submodules, and LFS hydration. Reject gitlinks and LFS pointer-backed +content. Never deepen, unshallow, full-clone, fetch arbitrary object IDs, choose +another ref, or fall back to OCI. + +### OCI acquisition + +The bounded server-owned catalog maps `snapshot_name` to registry/repository +origin, credential reference, TLS/redirect policy, allowed media types, and +resource policy. These remain private. Require the direct image-manifest digest, +verify manifest and config, fetch and validate the canonical source manifest +before any layer, then stream and digest-check layers in order. + +Use standard OCI layer caching supplied by the selected library/client where +useful. Do not create a separate gateway-managed OCI blob-cache lifecycle. +Apply standard file and opaque-directory whiteouts. Reject absolute/traversing +paths, NULs, ambiguous separators, duplicate/type conflicts, devices, FIFOs, +sockets, unsafe sparse files, unsupported types, escaping links, and unbounded +metadata. OCI sources MAY be Git-free. There is no Git fallback and one OCI +image always represents one tree. + +### Component singleflight and publication + +Singleflight independently by private component key. Each miss uses private +staging and streamed accounting; canceled waiters detach without canceling work +still needed by another live waiter. Publish only after exact manifest +verification. Published component bytes and evidence are immutable; failed or +uncertain staging is unavailable and cleaned or quarantined. Existing live +workspace cache reaping may delete only complete, unreferenced publications and +must not invalidate an attached session. + +## Access, binding, and application-level visibility + +- `read_only`: bind the immutable component root at its destination with + kernel-enforced read-only, `nodev`, and `nosuid` behavior while preserving + required execute bits. Expose no writable alias, backing descriptor, mirror, + or component-store path. +- `editable`: create an inode-independent private reflink or copy at the + destination. No write may mutate the cache or a sibling session. + +Prepare the outer workspace, destination scaffolding, every bind/copy, baseline, +and working directory while the `HarnessSession` workspace binding is pending. +Files, checkpoint, provider, and harness consumers already gate on application +state; extend that gate to require workspace `ready`. After all roots verify, +persist the resolved plan and transition once to ready. Atomicity means +application visibility after this transition. It does not require a special +filesystem rename, mount-namespace handoff, global composition transaction, or +composition identifier. + +The durable workspace binding stores only: + +- the validated descriptor and its private digest; +- pending/ready/failed state and fixed `expires_at`; +- the ordered resolved source plan and sanitized public metadata; +- immutable component and baseline-manifest references; +- the outer/editable acknowledged cursor manifests; and +- outer/editable checkpoint references. + +Checkpoint, artifact, and control data remain in their existing separate +records. Do not persist live mount IDs, filesystem identities, verified flags, +publication epochs, inode evidence, or other process-local attachment facts. +On every initial attach, continuation, restart, and live-workspace cache attach, +revalidate component identity, ownership, mount target, read-only flags/no +writable aliases, or editable inode independence before ready. ## Lifecycle -### New workspace-backed session - -1. Authenticate and validate the stock UHP envelope; establish existing - idempotency ownership and whether the request creates or reuses a session. -2. For a new workspace-backed session, parse and strictly validate the closed - descriptor, authorize retention, normalize the ordered 1–128 entries, and - reject all destination overlap and destination/input/asset/reserved-path - collisions before network, cache lookup, workspace writes, or component - claims. -3. Persist a pending session binding containing only the validated canonical - request. Allocate an inaccessible staging workspace under the existing - session/isolation lifecycle. -4. Resolve exact components. Git resolves advertised refs and depth-2 commits. - OCI verifies the image and canonical source manifest before layer requests. - Enforce per-source and aggregate limits as facts become known. -5. Claim/reuse each component independently. Acquire, materialize, verify, and - atomically publish misses; retain leases for hits. Wait for all entries and - commit the ordered composition identity. On any failure, attach none. -6. Fresh-hydrate the private outer staging workspace. Apply allowed inputs and - generated assets only outside destinations. Create empty, non-link - destination directories and safe ancestor scaffolding after rechecking the - prevalidated ownership map. -7. Attach every component using the requested access mode. Verify exact - identities, read-only flags/no writable aliases, or editable ownership/inode - independence. No consumer can observe the workspace during this step. -8. Validate the effective working directory. Publish the canonical outer - baseline after all initial outer assets are present, retain every canonical - component baseline, and persist attachment/checkpoint evidence. -9. Atomically publish the complete workspace and transition the existing session - binding to ready. Only then select the provider and start the harness. -10. At collection, compare final outer and source manifests to their baselines. - Checkpoint the writable outer tree without crossing source destinations; - checkpoint each editable source separately. Read-only source bytes are never - archived. Complete the existing stream/session transition and schedule - cleanup. - -Duplicate initial requests MUST share the existing idempotent result and MUST -NOT claim a second composition or attachment. - -### Continuation - -A continuation MUST NOT parse or resolve sources or contact Git/OCI endpoints. -It MUST: - -1. recover and validate the stored descriptor digest, exact component and - composition identities, destination map, access, manifests, attachment - evidence, checkpoints, and lifecycle state; -2. ensure stale mounts are unmounted, then restore the writable outer checkpoint - into an inaccessible staging workspace using no-follow/no-cross-mount - operations; -3. restore each editable root from its separate checkpoint, or reacquire the - exact recorded immutable lease and prepare the exact read-only bind mount; -4. validate destination scaffolding, collisions, component/baseline identities, - editable ownership, and read-only protection; -5. restore the original outer and component baselines without recomputing them - from mutated session content; -6. validate the working directory and atomically publish the complete workspace; - and -7. only then activate provider credentials, Files collection, and the harness. - -A missing, expired, corrupt, wrong-generation, writable, partially restored, or -unsupported attachment MUST fail closed. Continuation MUST NOT select another -cached component, reacquire source bytes, publish empty roots, or discard -editable mutations. - -### Checkpoint, retention, and cleanup - -The outer checkpoint MUST exclude every source destination and all source bytes. -Each editable Git or OCI root MUST have a separate private checkpoint and retain -its baseline identity. Read-only roots are reattached from immutable component -publications. Checkpointing and restore MUST be no-follow, mount-aware, bounded, -and cancellation-safe. - -`retention: "session"` follows existing finite TTL and deletion. Authorized -`persistent` retention pins the existing session and required references until -explicit deletion or applicable operator policy; it does not create another -scheduler. Polling and replay MUST NOT extend retention. - -Cleanup MUST first make the session unavailable, stop descendants, unmount every -source destination, verify mount absence, remove editable copies and outer -state, and release composition/component references exactly once. Hydrate, -restore, delete, cancellation, and restart reconciliation MUST follow the same -unmount-before-traversal rule. An uncertain or failed cleanup MUST quarantine the -path, keep it unavailable and accounted, and permit confined idempotent retry. - -## Implementation phases - -Every phase ends in observable behavior through the real boundary. Source-text -inspection and mock forwarding are not sufficient proof. - -### Phase 1: Rename the downstream and pin the upstream baseline - -**Outcome:** AllAgents Gateway has stable distribution identity and a recorded, -reproducible upstream relationship. - -Work: - -1. Record HarnessRouter `v0.25.4`, commit - `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`, and UHP `2026-09-12` as the - initially examined fork point. The implementation PR MUST record the exact - selected baseline and move it only in a standalone synchronization change. -2. Rename downstream repository/package references from - `allagentsdev/harnessrouter` to `allagentsdev/allagents-gateway`, the image to - `ghcr.io/allagentsdev/allagents-gateway`, the service to `allagents-gateway`, - and user-facing product text to AllAgents Gateway. -3. Preserve `HarnessRouter/harnessrouter` as upstream, the upstream license and - notices, copyright/attribution, commit history where available, and a durable - fork-point record. Configure origin as `allagentsdev/allagents-gateway`. -4. Define selective upstream intake: inspect each upstream diff, preserve the - downstream schema/security/lifecycle contract, and run stock plus downstream - gates before accepting it. Do not mirror upstream blindly. -5. Pin base image, OS packages, Git and OCI libraries/tools, Codex, OMP, - Promptfoo, lockfiles, and CI actions. Build the unchanged renamed baseline. -6. Characterize fresh root-Git hydration, pre-turn writes, Files collection, - checkpoint Git mutation/archive/restore, continuation clearing, - cancellation, TTL, deletion, cleanup, restart, and provider routing. -7. Save stock UHP traces for requests without `metadata.workspace`. Identify the - exact branch point where workspace-backed requests stop using root-Git change - collection while stock requests remain unchanged. - -Exit proof: - -- repository, image, service, and product surfaces use only the new downstream - names while attribution and the upstream remote/fork point remain intact; -- a clean checkout builds the pinned image; and -- stock requests complete with byte-for-byte compatible status/stream fixtures - and no workspace-specific state. - -### Phase 2: Implement the ordered closed request schema - -**Outcome:** request validation decides ownership and collisions before source or -session side effects. - -Work: - -1. Parse only `metadata.workspace` and reject obsolete schema names. -2. Apply metadata byte, nesting, list, string, digest, URL, and path bounds during - parsing. -3. Validate the closed `sources` array for counts `1..128`, entry-kind fields, - canonical Git HTTPS URLs, OCI direct digests, required destinations, global - pairwise non-overlap, and working-directory syntax. -4. Normalize UHP inputs, generated assets, reserved paths, credentials, caches, - attachment/checkpoint locations, and source destinations in one request-level - ownership validator. Reject every collision at or below a destination and - unsafe ancestor before network or cache work, regardless of access mode. -5. Authorize persistent retention before source lookup. Canonically serialize - the descriptor with defaults and calculate its digest. -6. Extend the existing new-session transition with pending and ready workspace - binding states. Reject metadata on every reuse/continuation path before - hydration. -7. Preserve initial-request idempotency and bounded UHP error details. Commit - public provenance only after atomic attachment readiness. - -Exit proof covers zero, one, 128, and 129 sources; all-Git, all-OCI, and mixed -arrays; repeated component identity at different destinations; ordering; -unknown/obsolete/wrong-kind fields; normalized root/equal/ancestor overlap; -input/asset/reserved collisions; outer inputs; URL/digest/path bounds; -retention authorization; both continuation mechanisms; idempotent duplicates; -harness mismatch; and an unchanged stock trace. Network and component counters -MUST remain zero for every request-decidable rejection. - -### Phase 3: Add canonical manifests and manifest-based collection - -**Outcome:** workspace-backed correctness depends only on canonical filesystem -state, never Git state. - -Work: - -1. Implement canonical streaming manifest creation and comparison for regular - files, binary bytes, directories, executable/mode semantics, symlinks, and - supported hardlinks. -2. Define deterministic ordering/serialization and content-digest algorithms. - Enforce no-follow traversal, ownership boundaries, source and aggregate - limits, and cancellation. -3. Publish one immutable baseline manifest with every component. Create and - persist the outer baseline only after initial outer inputs/assets exist and - before the first harness process. -4. Route workspace-backed Files/evaluation collection through final-manifest - comparison for the outer workspace and every source. Remove root/nested Git - commits, indexes, status/diff, ignore behavior, and rename detection from the - correctness path. Leave the stock collector untouched for requests without - workspace metadata. -5. Normalize ownership and exclude `.git`, runner state, credentials, caches, - attachment evidence, checkpoint metadata, harness-private ephemeral state, - and component-store paths. -6. Persist baseline identities across checkpoint/continuation and fail closed on - missing or mismatched baseline evidence. - -Exit proof mutates outer, Git, and Git-free OCI trees and observes identical -add/modify/delete/mode/symlink/binary results. It changes Git index, commits, -ignore files, and rename heuristics without changing final-tree results. It -proves delete-plus-add is accepted as a rename representation, initial trees are -not reported, exclusions never leak, read-only trees remain equal, and a stock -request still uses unchanged root-Git behavior. - -### Phase 4: Add independent component storage and atomic composition - -**Outcome:** components singleflight and cache independently, while consumers see -all requested roots or none. - -Work: - -1. Add durable component claim, staging, verification, atomic publication, - lease, quarantine, and cleanup states under runner-owned `/data`. -2. Key components without destination/session concerns. Build composition - identity from the ordered exact component identities and destinations; store - references rather than copied source bytes. -3. Resolve and claim independent entries concurrently within bounded worker, - network, disk, and aggregate request limits. Detach cancelled waiters without - cancelling a component still needed elsewhere. -4. Construct the outer workspace and all source attachments in an inaccessible - staging path or private mount namespace. Add one atomic publish/handoff plus - ready transition. Reverse-unwind every partial mount/copy/reference on error. -5. Implement identical read-only bind and editable copy/reflink behavior for Git - and OCI. Verify mount flags, no writable aliases, component identity, and - editable inode independence. -6. Thread the immutable destination map through Files, archive, hydrate, restore, - delete, and cleanup with explicit excludes and no-cross-mount traversal. -7. Add bounded metrics for component resolution/acquisition/hit/publication, - composition commit/hit/rollback, mount/copy, manifest creation/comparison, - checkpoint, reconciliation, quarantine, and cleanup without paths or secrets. - -Exit proof races identical and partially overlapping compositions. Each exact -component publishes at most once; a warm Git/cold OCI request reuses Git while -building only OCI; a warm OCI/cold Git request does the inverse. Ordered -composition IDs change with order/destination while component IDs stay stable. -Injected failure in the last of several roots exposes no source or runnable -workspace. Two read-only sessions share component inodes but not outer state; -two editable sessions share no mutable inode. - -### Phase 5: Implement depth-2 Git components - -**Outcome:** each Git entry produces one verified immutable component with useful -bounded recent history. - -Work: - -1. Enforce canonical public HTTPS and the fixed egress/redirect/address policy. - Isolate Git config and disable interactive credentials, hooks, filters, - alternates, alternate protocols, submodules, and LFS hydration. -2. Resolve omitted ref through advertised symbolic default; resolve advertised - branches and lightweight/annotated tags to exact commits. Reject ambiguous, - missing, unsupported, non-commit, or moved targets. -3. Serialize one operator-only bare shallow mirror per canonical URL. Fetch the - selected advertised path with exactly `--depth=2` into bounded private state, - verify the expected commit and shallow boundary, then atomically import. -4. Export a self-contained detached checkout with normalized bounded `.git` - metadata and no writable mirror link. Preserve merge parents when available - within depth 2. Reject gitlinks and LFS pointer-backed content. -5. Compute/verify the canonical component baseline, enforce per-source and - aggregate checkout bounds, and publish independently of destination. -6. Ensure exact identity hits transfer no pack and perform no checkout, tree - copy, baseline recomputation, or publication after optional ref confirmation. - -Exit proof covers default, branch, lightweight/annotated tag, moved ref, same URL -at different refs, depth exactly 2, `.git/shallow`, recent offline log/blame/diff, -merge parents, detached exact commit, malicious redirects, unsupported shallow -server, submodule/LFS rejection, cancellation, restart, and exact provenance. -Concurrent cold requests perform one mirror refresh and component publication; -an exact hit records zero source-byte work. Release notes state that depth 2 -limits history, not working-tree bytes. - -### Phase 6: Implement OCI source-tree components - -**Outcome:** each OCI entry produces exactly one verified tree component without -Git fallback or implicit workspace roots. - -Work: - -1. Implement the bounded operator catalog and direct image-manifest resolution. - Keep registry/repository origin, credentials, TLS configuration, and redirects - server-side. -2. Fetch and verify the image manifest, config, and exact canonical source - manifest before layers. Validate relative ownership, all expected final - entries, links, modes, sizes, digests, and known per-source/aggregate limits. -3. Stream bounded layers, verify descriptors, apply whiteouts, and enforce path, - type, link, sparse-file, compression-ratio, inode, byte, output, cancellation, - and time limits. -4. Confine links to the one owning component root and reject multi-root or outer - workspace content. Match the final tree exactly to the canonical source - manifest before publication. -5. Support Git-free trees and optionally normalized bounded Git administrative - data for agent convenience. Neither form changes manifest-based evaluation. -6. Ensure exact identity hits make zero registry requests, extraction, tree copy, - baseline recomputation, or publication. - -Exit proof uses an authenticated local registry and malicious fixtures for -catalog/digest/media mismatch, indexes, redirects, authentication, truncation, -compression bombs, limits, whiteouts, traversal, path/type/link attacks, -devices, sparse files, cancellation, partial cleanup, and restart. Positive -fixtures cover Git-free and history-bearing trees, read-only and editable -attachment, two independent OCI entries, exact-hit reuse, and source-manifest -rejection before the first layer request. - -### Phase 7: Make checkpoint and continuation composition-aware - -**Outcome:** complete compositions survive turns and restarts without source -traffic or baseline loss. - -Work: - -1. Archive the writable outer workspace with every destination excluded and - no-follow/no-cross-mount enforcement. Store no source bytes in the outer - checkpoint. -2. Preserve every editable Git or OCI root in an independent private checkpoint; - preserve immutable component references for read-only roots. Retain original - baseline identities separately from mutable final state. -3. Implement the staged continuation order: validate binding/evidence, unmount - stale roots, restore outer state, restore editable roots or exact read-only - leases, verify all roots, restore baselines, validate cwd, then atomically - publish. -4. Reconcile every durable transition after restart. Missing/corrupt evidence, - publication, checkpoint, mount, or baseline fails closed without network or - replacement component selection. -5. Make cancellation, expiry, explicit deletion, persistent retention, cleanup, - quarantine, and reference release composition-aware and idempotent. - -Exit proof checkpoints mixed compositions after mutating outer, editable Git, -and editable OCI paths. Continuation preserves every mutation and baseline, -makes zero source requests, and reports the same final-tree changes. A read-only -continuation rebinds exact component inodes. Restart and cancellation are -injected at each claim, publication, composition, mount/copy, baseline, -checkpoint, handoff, and cleanup transition. No partial workspace becomes ready. - -### Phase 8: Wire harnesses and provider boundary - -**Outcome:** Codex and OMP run in the atomically composed workspace without -source or long-lived provider credentials. - -Work: - -1. Pin Codex and OMP and enable only required backends with - `HR_BACKENDS=codex,omp` unless the renamed downstream configuration surface - adopts an equivalent key in the same change. -2. Define stable `allagents-codex` and `allagents-omp` harnesses with explicit - model allowlists. -3. Use the same external provider gateway through one Responses connection for - Codex and one Chat Completions connection for OMP, with no fallback. -4. Retain server-side long-lived credentials and existing short-lived scoped - turn credentials/loopback route. Remove ephemeral provider configuration - before checkpoint and collection. -5. Start each harness only after atomic attachment and from the validated working - directory. Keep HOME, skills, scratch, conversation, credentials, generated - assets, and checkpoint control in private outer locations. -6. Reject caller provider, route, model, transport, source credential, and - registry overrides. Provider failure MUST NOT alter composition binding. - -Exit proof runs both harnesses against all-Git, all-OCI, and mixed compositions -at outer and nested working directories in both access modes. It verifies that -caller, Git, registry, broker, and provider secrets are absent from output, -checkpoints, reported files, artifacts, logs, and public metadata. Unsupported -models and bad credentials produce no route fallback. Stock requests retain -their original provider path. - -### Phase 9: Run direct release-blocking E2E - -**Outcome:** the built image proves the consumer-visible component, composition, -manifest, lifecycle, and compatibility contracts. - -Promptfoo MUST call the built image directly at the existing UHP Responses -endpoint with an AllAgents Gateway caller API key. There is no adapter service or -alternate execution protocol. - -Release-blocking scenarios: - -1. **Large OCI component:** publish a deterministic source tree with at least - 2 GiB expanded bytes and 100,000 source-visible entries, a late-path sentinel, - a nested working directory, and optional normalized offline history. Record - actual compressed/expanded bytes, entry count, layers, ratio, and digests. -2. **Multiple OCI:** compose at least two independent OCI entries at sibling - destinations. Codex MUST read both; a second session MUST reuse both with zero - registry, extraction, copy, baseline, or publication work. -3. **Mixed source:** compose at least two depth-2 Git and two OCI entries. OMP - MUST read all four, work from a nested directory, and report ordered - provenance and composition identity. Reject an overlap before network access. - Warm only one component, then prove the cold components build independently - and atomic visibility waits for every entry. -4. **Editable OCI:** run a bug-fix evaluation with `access: "editable"`, mutate - an OCI file, add a binary, delete another path, change executable mode, and - retarget a symlink. Manifest comparison MUST report the exact final state; - continuation MUST preserve it; cache and sibling sessions MUST remain - unchanged. -5. **Editable mixed composition:** mutate Git, OCI, and outer paths in one turn. - Prove ownership, deduplication, exclusion, binary/mode/link semantics, and - final-tree equality without Git status/index/commit dependence. -6. **Git matrix:** cover default/branch/tag/merge, depth 2, several entries, - repeated identity at distinct destinations, recent offline history, cache - singleflight, and zero source-byte work on exact hit. -7. **Atomicity and access:** delay/fail the last component and observe no partial - Files/harness/workspace visibility. Prove read-only mutation denial, shared - immutable inodes, editable independent inodes, writable outer assets, and - pre-network input/asset collision rejection for Git and OCI destinations. -8. **Lifecycle:** cover continuation, restart, cancellation during each - acquisition kind and composition wait, expiry, authorized persistence, - explicit deletion, cleanup retry, corrupt evidence, and no rematerialization. -9. **Failure/security:** cover malformed/obsolete descriptors, 0/129 sources, - root/overlap paths, per-source and aggregate limits, unsafe links/types, - digest mismatch, depth refusal, provider failure, reused-session injection, - and absence of Git/OCI/provider fallback or secret leakage. -10. **Stock compatibility:** replay requests without workspace metadata and - compare status, stream ordering, root-Git Files/checkpoint behavior, - continuation, cancellation, and provider route with Phase 1 fixtures. - -Reports MUST retain only sanitized assertions, image/source digests, measured -resource values, and component/composition counters. They MUST NOT retain -credentials, private origins, provider traffic, internal paths, or volume -contents. - -### Phase 10: Publish the digest-pinned AllAgents Gateway release - -**Outcome:** a clean operator can deploy the exact tested downstream image and -reproduce the Git/OCI/mixed contract. - -Work: - -1. Review the downstream diff from the recorded upstream fork point, including - license/attribution, renamed distribution surfaces, selective upstream - changes, request/session binding, component caches, atomic composition, - manifests, lifecycle, provider wiring, E2E, and release automation. -2. Run pinned upstream UHP conformance without exclusions and all focused - downstream gates against one image candidate. -3. Run Phase 9 against that exact candidate digest. -4. Build `linux/amd64` from pinned inputs, attach standard SBOM and provenance, - and publish +### New session + +1. Use stock authentication, UHP validation, idempotency, and new-versus-reused + session selection. +2. Parse the closed workspace request and complete request-decidable ownership, + collision, bounds, and continuation checks. +3. Assign the fixed expiry and persist a pending binding on `HarnessSession`. +4. Resolve exact Git/OCI identities and private keys; independently claim, reuse, + or build each component under per-source and aggregate limits. +5. Hydrate the writable outer workspace through the existing path. Apply inputs + and generated assets only outside source destinations. +6. Attach every source according to `access`; recompute/verify baselines and live + protection. Any failure leaves the binding non-ready and exposes no partial + workspace. +7. Validate `working_directory`, persist ordered resolved plan, public metadata, + baselines, and initial manifest cursors; transition the binding once to ready. +8. Only then start the inherited harness/provider path. +9. On terminal collection, use manifest projection and capture-before-ACK. Then + checkpoint and complete the existing turn/session transition. + +An idempotent duplicate shares the same pending or ready binding and does not +create another attachment plan. + +### Checkpoint and continuation + +The existing stock sequence commits and tars a normal workspace. For a +workspace-backed session, retain that lifecycle while making archive boundaries +explicit: + +- tar the writable outer workspace without crossing any source destination; +- checkpoint each editable Git/OCI root separately; +- store only immutable component references for read-only roots; and +- preserve original baselines and acknowledged cursor manifests separately from + mutable final content. + +Continuation never resolves a ref or contacts Git/OCI. Restore the outer tar and +editable-root checkpoints, reattach recorded read-only components, revalidate +all live attachment protections, restore baseline/cursor references, validate +cwd, and only then mark the live workspace ready. Missing, expired, corrupt, or +mismatched binding, baseline, component, or checkpoint evidence fails closed. +Do not select a replacement component, rematerialize from a source, expose an +empty root, or discard editable mutations. + +### Expiry, deletion, restart, and cleanup + +The fixed workspace expiry applies to every workspace-backed session and is +never extended. Expiry and explicit deletion first make the session unavailable, +stop descendants, unmount source roots, verify mount absence, remove editable +and outer state, release references once, and delete durable binding/checkpoint +state through existing records. Cleanup is mount-aware, confined, idempotent, +and unavailable-first. Uncertain paths remain unavailable, accounted, and +quarantined for retry. + +Restart reconciliation resumes or fails each pending durable transition without +trusting process-local attachment facts. Stock live-workspace cache reaping +remains distinct from durable session expiry and deletion. + +## Stable workspace failure contract + +Workspace failures use the existing bounded UHP error envelope with the exact +detail codes below. `retryable` describes retrying the same logical operation +after its stated cause is corrected; it is not permission to extend expiry or +change a binding. Base UHP authentication, envelope, provider, and transport +errors retain stock codes only where their meaning is exact. + +| Detail code | Condition | HTTP | Retryable | Required behavior | +|---|---|---:|:---:|---| +| `workspace_invalid_request` | Closed-schema, count, field, URL, digest syntax, path, cwd, first-turn, or reused-session violation | 400 | no | Fail before cache, network, workspace write, claim, or lifecycle mutation. | +| `workspace_path_collision` | Equal/overlapping destinations or input/generated/reserved/ancestor collision | 409 | no | Fail before cache or network; report only sanitized conflicting workspace-relative fields. | +| `workspace_source_unknown` | Unknown OCI catalog entry or missing/ambiguous/unsupported Git ref identity | 404 | no | Fail that source with no alternate ref, catalog entry, or source kind. | +| `workspace_source_invalid` | Moved/non-commit Git target, depth-2 refusal, gitlink/LFS content, OCI media/digest/source-manifest/layer/final-tree failure, or unsafe source content | 422 | no | Publish no failed component and perform no fallback; invalid OCI source manifest fails before layer requests. | +| `workspace_acquisition_unavailable` | Timeout, DNS, registry/Git service, or other transient source transport failure | 503 | yes | Detach request-local work, preserve independently valid shared components, and expose no partial workspace. | +| `workspace_contract_limit_exceeded` | A fixed v1 per-source or aggregate count/byte/path/ratio/time/output ceiling is exceeded | 413 | no | Stop bounded work, clean/quarantine staging, and expose no partial workspace. | +| `workspace_capacity_exceeded` | Operator concurrency, disk, inode, mount, or lower policy capacity is temporarily unavailable | 503 | yes | Admit no partial binding; capacity policy must not masquerade as a schema limit. | +| `workspace_attachment_failed` | Initial attachment or later live reattachment fails bind/copy, ownership, protection, baseline, cwd, or ready validation | 500 | yes | Keep an initial binding non-ready or fail a post-ready reattachment; reverse/unmount request-local state and never run the harness. | +| `session_expired` | Continuation targets a workspace-backed session at or after fixed `expires_at` | 404 | no | Preserve the pinned UHP `session_expired` response, omit workspace metadata, and do not restore, reacquire, or extend expiry. | +| `workspace_restore_invalid` | Required binding, component, checkpoint, baseline, or cursor evidence is missing, corrupt, or mismatched | 500 | no | Fail closed with no source traffic, replacement selection, or empty-root recovery. | +| `workspace_collection_failed` | Manifest traversal/comparison, file/change-artifact capture, or ACK persistence fails | 500 | yes | Do not ACK or publish an incomplete change set; retry reproduces the same logical changes. | +| `workspace_checkpoint_failed` | Mount-aware archive creation or durable checkpoint persistence fails after collection | 500 | yes | Preserve the acknowledged collection state, publish no invalid checkpoint, and retry checkpoint persistence without rerunning the harness. | + +Failures before the first ready transition omit `metadata.workspace`; workspace +failures after ready return the stored sanitized `metadata.workspace`. The +inherited `session_expired` response is the sole exception and remains unchanged. +No case exposes private keys, catalog origins, host paths, mounts, credentials, +raw tool stderr, or network details. + +Cancellation is not a workspace error. Preserve the inherited idempotent `2xx` +cancel endpoints and terminal `status: "cancelled"` rather than returning an +error code. Cancellation before ready omits workspace metadata; cancellation +after ready returns the stored sanitized metadata. Stop unneeded descendants, +detach shared waiters safely, and leave no partial ready state. + +## Implementation phases and exit proofs + +Every phase changes the real named seam and ends with observable focused proof. +Mocks may isolate external provider transport, but source-text assertions and +mock forwarding are not proof. + +### Phase 1: Bootstrap baseline and characterize stock seams + +**Work** + +- After the admin bootstrap, verify target/ref, Apache-2.0/NOTICE/history, + `origin`, `upstream`, and fork-point record. +- Rename downstream distribution surfaces without changing attributed upstream + material. +- Trace and record `_produced_list`, `_produced_ack`, `_collect_produced`, + `BACKING.workspace`, `RunnerWorkspaceFiles`, `CheckpointWorkspaceFiles`, + `HarnessSession`, and separate checkpoint/artifact/control records. +- Capture stock traces proving fresh root Git, Git produced cursoring, + checkpoint commit-then-tar, tar hydration, live-workspace cache reaping, + durable explicit deletion, continuation, cancellation, and provider routing. + +**Exit proof:** a request without `metadata.workspace` matches pinned status, +stream, Files, checkpoint/restore, continuation, cancellation, deletion, and +provider traces; the renamed checkout preserves license/NOTICE/history and the +exact fork point. + +### Phase 2: Add the closed request and session binding + +**Work** + +- Implement parsing, all pre-network ownership/collision checks, fixed expiry, + private descriptor digest, and pending/ready/failed binding states on the + existing session seam. +- Persist only the durable fields listed above and return only the pinned public + shape after ready. +- Reject workspace metadata on every reuse/continuation path before hydration. + +**Exit proof:** 1 and 128 Git/OCI/mixed entries pass; 0/129, obsolete fields, +unknown fields, root/overlap/collision, malformed identities, and reused-session +injection return their exact coded errors with zero source/cache activity. Stock +traces remain unchanged. + +### Phase 3: Implement canonical manifests and produced projection + +**Work** + +- Implement source-manifest v1 canonicalization and secure bounded traversal. +- Replace workspace-backed root-Git listing behind `_produced_list` with + outer/editable cursor comparison; keep stock implementation unchanged. +- Extend `_collect_produced` to capture changed regular files and the canonical + change artifact before `_produced_ack`; advance cursors only in ACK. +- Route live/checkpoint reads through the existing `BACKING.workspace` classes. + +**Exit proof:** synthetic outer/editable-root fixtures produce exact +add/modify/delete/type/mode/symlink/binary artifacts independent of Git state; a +failure before ACK retries identically; and the canonical change-artifact parser +reconstructs final state from ordered artifacts. + +### Phase 4: Add component cache, access modes, and ready gating + +**Work** + +- Implement the two exact private cache keys, independent singleflight, private + staging, immutable publication evidence, references, reconciliation, and + quarantine. +- Attach read-only bind mounts or editable reflink/private copies into the + existing private workspace and revalidate live protection on every attach. +- Persist the ordered plan on the session binding and expose it only through the + single ready transition. Add no composition cache, record, ID, or filesystem + handoff protocol. + +**Exit proof:** overlapping concurrent requests publish each missing component +once; partial cache warmth builds only misses; last-root failure exposes no +Files/checkpoint/provider/harness view; read-only sessions share immutable bytes +without source traversal during collection and require fresh mount evidence; +editable sessions cannot mutate cache/sibling content. Restart discards or +reconciles uncertain publications without persisted mount/inode facts. + +### Phase 5: Implement depth-2 Git acquisition + +**Work** + +- Implement canonical HTTPS validation, advertised ref resolution, the per-URL + serialized mirror, fixed depth-2 fetch, detached self-contained export, + bounded safe `.git`, and source-manifest publication evidence. +- Enforce egress/redirect policy, limits, disabled helpers, and no fallback. + +**Exit proof:** default/branch/lightweight-tag/annotated-tag/merge cases resolve +to exact commits; recent shallow offline log/blame/diff works within depth 2; +moved refs, unsupported shallow servers, malicious redirects, gitlinks, LFS, +limits, cancellation, and restart fail with exact codes. An exact key hit does +no pack acquisition or materialization. + +### Phase 6: Implement OCI source-tree acquisition + +**Work** + +- Use `oci-auth-registry`, `oci-catalog-builder`, and + `oci-source-fixture-builder` for catalog resolution, direct digest fetch, + pre-layer canonical manifest validation, standard layer/whiteout extraction, + limits, secure links/types, and exact final verification. +- Use standard client/library layer caching only; add no managed blob-cache + subsystem. Preserve Git-free operation. + +**Exit proof:** authenticated positive fixtures cover multiple OCI roots, +read-only/editable access, whiteouts, empty directories, safe hardlinks, and +exact reuse. An editable Git-free OCI mutation produces the exact canonical +change artifacts. Malicious fixtures cover catalog/media/digest mismatch, +manifest rejection before layers, traversal, links, types, sparse/compression +bombs, limits, cancellation, partial cleanup, and restart. Exact component hits +require no registry fetch or extraction. + +### Phase 7: Make checkpoint, continuation, expiry, and deletion mount-aware + +**Work** + +- Adapt stock commit/tar and hydrate through existing checkpoint seams: outer tar + excludes all source roots; editable roots checkpoint separately; read-only + roots retain immutable references. +- Restore without source traffic, revalidate live attachment protection, restore + baselines/cursors, and gate cwd/harness on ready. +- Implement fixed non-extendable expiry, explicit deletion, cancellation, + restart reconciliation, reference release, and unavailable-first cleanup. + +**Exit proof:** mixed outer/editable Git/editable OCI mutations survive turns and +restart and retain their original comparison cursors; read-only roots reattach +by exact identity; polling and continuation do not move `expires_at`; corrupt or +expired evidence fails closed; cancellation and cleanup never traverse a live +mount or expose a partial session. + +### Phase 8: Produce the implementation handoff and run one release flow + +The implementation agent completes the PR with local fixtures, mocked provider +transport, focused phase proofs, pinned dependencies, and release automation. +There is no separate provider phase: tests assert that both inherited harness +paths start only after ready/cwd and that workspace code does not change broker, +credential, model, or route behavior. + +After review, the operator performs one release flow: + +1. Build one `linux/amd64` candidate from pinned inputs, attach SBOM and build + provenance, and publish it to `ghcr.io/allagentsdev/allagents-gateway:-allagents.`. -5. Read back and deploy by manifest digest, for example - `ghcr.io/allagentsdev/allagents-gateway:v0.25.4-allagents.1@sha256:`, - under service name `allagents-gateway`. -6. Verify fresh-volume and same-volume restart with both harnesses, Git, multiple - OCI, mixed composition, cache reuse, editable mutation, continuation, - cancellation, cleanup, manifests, and stock requests. -7. Record upstream tag/commit and fork point, downstream commit, license/notices, - UHP and harness versions, base/package pins, Git/OCI/manifest/materializer - policy revisions, image digest, fixtures, SBOM, provenance, and E2E reports. -8. Block release on any old downstream name in a public distribution surface, - missing attribution, obsolete accepted schema, partial composition exposure, - cache mutation, incorrect editable isolation, Git-based workspace evaluation, - continuation source traffic, credential leak, stock regression, or unpinned - input. - -Only after downstream evidence exists MAY maintainers propose generic seams -upstream. Upstream issue, UEP, acceptance, merge, and release timing remain -outside the downstream critical path. - -## Failure contract - -Workspace failures MUST use bounded stable detail codes under the existing UHP -error shape. Exact HTTP mapping follows pinned upstream conventions. - -| Condition | Required behavior and timing | -|---|---| -| Invalid/obsolete shape, count, field, URL, digest, destination, overlap, or cwd syntax | Reject before cache lookup, network, session workspace write, or component claim | -| Input/generated/reserved-path collision at or below any destination | Reject before network for Git and OCI and for both access modes; outer paths remain valid | -| Unauthorized `persistent` retention | Reject before cache lookup or network | -| Workspace metadata on reused session | Reject without changing binding, checkpoint, or TTL | -| Git ref missing, ambiguous, non-commit, or moved | Fail that component; no alternate ref or source | -| Server cannot satisfy depth-2 Git fetch | Fail that component; no deepen, full clone, history stripping, or OCI fallback | -| Submodule or LFS-backed content | Fail Git validation; no helper execution | -| OCI catalog, manifest, media, or digest mismatch | Fail that component without revealing catalog origin | -| Invalid canonical OCI source manifest | Fail before any layer request | -| OCI layer digest, whiteout, path, link, type, sparse, ratio, or final-manifest mismatch | Fail extraction; no publication or Git fallback | -| Per-source limit exceeded | Stop that component, clean staging, and fail the composition | -| Request aggregate limit/capacity exceeded | Cancel/detach request-local work, preserve valid shared work, attach none | -| One component fails after siblings succeed | Publish no composition attachment; release request-local references; retain only independently valid shared cache entries | -| Cancellation/timeout during component or composition work | Stop unneeded descendants, detach waiter, clean private staging, expose no partial roots | -| Mount/copy/atomic handoff failure | Reverse-unwind all roots; session never becomes ready | -| Missing/corrupt attachment, checkpoint, or baseline on continuation | Fail closed; no source access, replacement component, or empty root | -| Read-only mutation attempt or changed final manifest | Filesystem denial/integrity failure; cache and siblings unchanged | -| Manifest collection limit or unsafe traversal | Fail collection/checkpoint; do not publish an incomplete result/baseline | -| Provider failure | Return existing provider error; source/composition/route binding remains unchanged | -| Cleanup failure | Keep session/path unavailable, quarantined, and accounted for idempotent retry | - -Failures before attachment-ready MUST expose no component/composition provenance. -Every failure after readiness MUST return the complete stored sanitized -`metadata.workspace` object while excluding internal paths, registry origins, -credentials, raw tool stderr, and private network details. - -## Verification matrix - -| Gate | Observable evidence | -|---|---| -| Naming/distribution | Repository is `allagentsdev/allagents-gateway`, image is `ghcr.io/allagentsdev/allagents-gateway`, service is `allagents-gateway`, and public product text says AllAgents Gateway | -| Upstream relationship | `HarnessRouter/harnessrouter`, fork point, license, notices, attribution, and selective-intake procedure are recorded and intact | -| Stock compatibility | Requests without `metadata.workspace` match pinned root-Git hydrate/checkpoint/Files, stream, continuation, cancellation, and provider behavior | -| Closed ordered schema | `sources` accepts 1..128 Git/OCI/mixed entries; obsolete singular/bundle forms and unknown fields fail | -| Pre-network ownership | Root/equal/ancestor destinations and all input/asset/reserved collisions fail with zero source/cache/component activity | -| Component semantics | Every entry owns exactly one non-root tree; OCI never supplies a runtime or multi-root workspace | -| Git depth policy | Default/branch/tag resolve exactly; depth is 2; shallow recent history and merge parents work offline; refusal has no fallback | -| OCI integrity | Direct image and canonical source-manifest digests, pre-layer manifest validation, bounded layers, whiteouts, paths, links, types, and exact final tree verify | -| Per-source/aggregate limits | Bytes, entries, ratio, path, file, inode, time, and output limits fail at the correct component or composition boundary | -| Independent singleflight | Identical components publish once even across different compositions; unrelated entries proceed independently | -| Composition identity | Ordered exact component identities plus destinations determine the ID without copying bytes | -| Atomic composition | Late failure/cancellation exposes no subset to Files, checkpoint, provider, or harness; success reveals all roots together | -| Git cache reuse | Exact hit has zero pack acquisition, checkout, tree copy, manifest recomputation, or publication | -| OCI cache reuse | Exact hit has zero registry request, extraction, tree copy, manifest recomputation, or publication | -| Read-only sharing | Matching sessions bind the same immutable component inodes with enforced flags/no aliases; outer paths remain private and writable | -| Editable isolation | Git and OCI copies/reflinks share no mutable inode; mutations survive continuation and cannot affect cache/siblings | -| Canonical baselines | Every component and outer workspace have persisted canonical path/type/digest/mode/link manifests | -| Manifest final diff | Add/modify/delete/mode/symlink/binary changes across outer, Git, and Git-free OCI roots reflect final-tree equality independent of Git state | -| Exclusions | `.git`, credentials, caches, runner state, attachment evidence, checkpoint metadata, and component paths never appear as reported changes | -| Checkpoint | Outer archive contains no source bytes; editable roots persist separately; read-only roots reattach by exact identity | -| Continuation | Exact composition/access/cwd/provenance/baselines and editable mutations restore with zero source traffic or fallback | -| Cancellation/restart | Every claim/publication/composition/attachment/checkpoint/cleanup transition reconciles or fails closed | -| Cleanup | Expiry/deletion is unavailable-first, mount-aware, confined, idempotent, reference-safe, and quarantine-preserving | -| Multiple OCI E2E | Two independent OCI entries compose, execute, and reuse independently in a second session | -| Mixed E2E | Git and OCI compose atomically; partial cache warmth builds only missing components; both harnesses consume the result | -| Editable OCI E2E | Bug-fix evaluation mutates OCI content and manifest comparison reports exact persistent final changes | -| Provider boundary | Codex Responses and OMP Chat Completions use fixed brokered routes with no caller override, fallback, or retained secrets | -| Release | Candidate digest, GHCR image, deployed service, SBOM, provenance, pins, UHP conformance, and Promptfoo report all identify the same build | - -## Definition of done - -1. All public downstream repository, image, service, and product names use - AllAgents Gateway. `HarnessRouter/harnessrouter` remains the attributed - upstream with its recorded fork point, license, notices, and selective-intake - process. -2. Requests without `metadata.workspace` remain stock-compatible and are the - only requests permitted to use stock root-Git change collection and - checkpoint correctness. -3. The first-turn-only closed contract requires `access` and an ordered - `sources` array of 1–128 independent `git`/`oci` entries, permits mixed and - repeated identities at distinct destinations, and accepts no obsolete alias. -4. Every source has one required non-root pairwise-non-overlapping destination. - All source ownership and input/asset/reserved collisions fail before network, - cache lookup, claims, or workspace writes. -5. Git components use canonical public HTTPS, exact advertised commit resolution, - mandatory depth 2, useful bounded offline history, no submodule/LFS helpers, - and no deepening/full-clone/source fallback. -6. OCI components use an operator catalog, direct image manifest digest, direct - canonical source manifest digest, pre-layer relative-tree validation, bounded - secure extraction, exact final-tree verification, and no Git fallback. Each - OCI image is one source-tree transport/cache unit, never a runtime or bundle. -7. Per-source and aggregate bounds are enforced for all acquisition kinds and at - composition time. -8. Components cache and singleflight independently. The composition identity - references ordered exact component identities and destinations. A composition - is atomically visible only after every component and attachment verifies. -9. `read_only` Git and OCI roots bind immutable cached bytes. `editable` Git and - OCI roots use inode-independent private writable copies/reflinks. Bug-fix - evaluation uses editable mode. Cached publications never mutate. -10. Every component publishes a canonical baseline keyed by normalized relative - path with type, content digest, executable/mode semantics, and symlink target. - Every workspace-backed session persists an equivalent outer baseline. -11. Final collection/evaluation compares source and outer filesystem manifests, - detects add/modify/delete/mode/symlink/binary changes, treats final-tree - equality as authoritative, and does not rely on root/nested Git state or - rename detection. -12. `.git`, runner-owned state, credentials, caches, attachment evidence, and - checkpoint metadata are excluded from reported changes. OCI roots work with - no Git metadata. -13. The private outer workspace remains writable. Inputs/assets outside source - destinations work; paths at or below destinations fail before network. -14. Checkpoint and continuation are mount-aware. Outer archives exclude all - source bytes, editable roots persist separately, read-only roots reattach by - exact identity, baselines remain stable, and continuation performs no source - traffic. -15. Direct Promptfoo against the tested image proves multiple OCI, mixed Git/OCI, - partial cache warmth, component and composition singleflight, atomic - visibility, editable OCI mutation/evaluation, manifest final diffs, both - harnesses, lifecycle failures, security boundaries, and stock compatibility. -16. A second exact-identity session proves zero Git source-byte work and zero OCI - registry/extraction work while reusing immutable publications; editable - reuse proves independent mutation. -17. Codex and OMP use fixed adapters through the existing external provider - gateway with brokered short-lived credentials, no caller override, and no - fallback. -18. The same digest-pinned `ghcr.io/allagentsdev/allagents-gateway` candidate - passes UHP conformance, focused integration, large/multiple/mixed Promptfoo - E2E, restart deployment, secret scans, SBOM, and provenance gates under - service name `allagents-gateway`. -19. No upstream issue, UEP, acceptance, merge, or release blocks downstream - implementation or publication. +2. Read the image back and record its registry manifest digest. Deploy and test + only `ghcr.io/allagentsdev/allagents-gateway@sha256:`. +3. Run the full release matrix once against that published/read-back digest: + stock UHP compatibility; Codex and OMP; all-Git, multiple-OCI, and mixed + ordered plans; partial cache warmth and singleflight; editable Git/OCI/outer + mutation and change-artifact reconstruction; read-only enforcement; limits, + failures, cancellation, expiry, deletion, cleanup, and secret scans. +4. Include fresh-volume and same-volume restart cases in that same matrix, + covering pending acquisition/publication, ready attachment, checkpoint, + continuation, cache reuse, quarantine, and cleanup reconciliation. +5. Record the upstream/fork/downstream commits, dependency and harness pins, + fixture digests, published image digest, SBOM, provenance, and one E2E report. + +Do not run a duplicate full matrix against a pre-publication build and then again +after publication. Focused local phase proofs protect implementation; the one +release matrix proves the exact distributed digest. + +## Completion checklist + +### Implementation agent + +- PR targets bootstrapped `allagentsdev/allagents-gateway` branch + `feat/workspace-composition` at the pinned fork point and preserves + Apache-2.0/NOTICE/history/upstream. +- Stock seams remain the integration points, and stock requests retain their + pinned behavior. +- Request/response, source-manifest, change-artifact, cache-key, binding, + lifecycle, and coded-error contracts match this plan exactly. +- Local OCI fixtures and Linux capability checks prove Git/OCI/mixed access, + manifest cursoring, checkpoint/restore, restart reconciliation, and cleanup. +- No composition record/ID, retention/persistent field, durable live attachment + facts, managed OCI blob-cache lifecycle, provider fork, or compatibility shim + remains. + +### Operator + +- Supply provider/UHP/GHCR/deployment secrets only through the existing secret + mechanism and retain protected settings. +- Review and merge the implementation PR, publish/read back one candidate, + deploy by digest, and run the single full release/restart matrix. +- Accept release only when the digest, SBOM, provenance, deployment, and E2E + report identify the same image and no secret or private workspace metadata is + exposed. From 6661567cf82962439382aab5e4f3c4aab34ec6a6 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 08:18:48 +1000 Subject: [PATCH 35/44] docs(architecture): rename existing gateway fork --- .../0002-adopt-uhp-through-harnessrouter.md | 2 +- ...0837-feat-coding-execution-gateway-plan.md | 41 +++++++++++-------- 2 files changed, 24 insertions(+), 19 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 8bd03cb5..432ff307 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -225,7 +225,7 @@ Provider and harness failures that are not workspace failures retain their exact ## Distribution and release boundary -`allagentsdev/allagents-gateway` does not exist at the time of this decision. Before implementation, an organization repository administrator must rename or bootstrap the current `allagentsdev/harnessrouter` repository from the pinned commit, preserve Apache-2.0, `NOTICE`, attribution, and history, add `HarnessRouter/harnessrouter` as the upstream remote, and create a writable implementation branch. +`allagentsdev/harnessrouter` is the existing GitHub fork of `HarnessRouter/harnessrouter`; creating a second repository would discard the fork identity and create two downstream sources of truth. Before implementation, an organization repository administrator MUST rename that repository in place to `allagentsdev/allagents-gateway`. The rename preserves repository identity, fork-network relationship, history, settings, and GitHub redirects. After the rename, verify the upstream parent and Apache-2.0 `LICENSE`/`NOTICE`, keep the pinned upstream commit as the characterized fork point, and create `feat/workspace-composition` from downstream `main` commit `fbcb73132423c8c4575113fc8943c6a6280a4746`, which contains that baseline plus the existing downstream commits. Implementation also requires checked-in local builders for an authenticated OCI registry, catalog, and source fixtures, plus a Linux environment capable of namespace-confined bind mounts and reflink-or-copy isolation. The implementation agent can deliver a pull request using those fixtures and mocked provider transport without production secrets. Provider gateway URL/key, UHP caller credentials, GHCR rights, protected settings, and final publication/deployment are operator-owned inputs supplied through the repository's secret mechanism. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index d497f363..add66403 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -26,22 +26,26 @@ protocol, global composition cache, or materializer service. ## Implementation-handoff prerequisites -### Target bootstrap: operator-owned and required before code work - -`allagentsdev/allagents-gateway` does not exist yet. An organization repository -administrator, not the implementation agent, MUST complete this bootstrap: - -1. Rename/bootstrap the current `allagentsdev/harnessrouter` repository as - `allagentsdev/allagents-gateway` from pinned HarnessRouter commit - `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`. -2. Preserve the complete available history, Apache-2.0 `LICENSE`, `NOTICE`, - copyright and attribution. HarnessRouter is Apache-2.0 with a NOTICE; it is - not MIT. -3. Set `origin` to the renamed writable repository and retain - `HarnessRouter/harnessrouter` as `upstream`. Record the pinned commit as the - downstream fork point. Upstream acceptance is not a dependency. -4. Create writable branch `feat/workspace-composition` at that commit and grant - the implementation agent normal pull-request rights to the branch. +### Target rename: operator-owned and required before code work + +`allagentsdev/harnessrouter` already exists and GitHub reports it as a fork of +`HarnessRouter/harnessrouter`. Do not create or bootstrap a second repository. +An organization repository administrator, not the implementation agent, MUST: + +1. Rename `allagentsdev/harnessrouter` in place to + `allagentsdev/allagents-gateway`. Preserve its repository identity, fork + network, complete history, settings, redirects, Apache-2.0 `LICENSE`, + `NOTICE`, copyright, and attribution. +2. Verify GitHub still identifies `HarnessRouter/harnessrouter` as the parent. + Set `origin` to the renamed writable repository and retain that parent as + `upstream`. Record upstream commit + `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` as the characterized baseline; + upstream acceptance is not a dependency. +3. Create writable branch `feat/workspace-composition` from downstream `main` + commit `fbcb73132423c8c4575113fc8943c6a6280a4746`. That commit is the pinned + upstream baseline plus the existing three downstream commits; implementation + MUST NOT discard them by branching directly from the older upstream commit. +4. Grant the implementation agent normal pull-request rights to that branch. 5. Keep the downstream distribution names fixed: - repository: `allagentsdev/allagents-gateway`; - image: `ghcr.io/allagentsdev/allagents-gateway`; @@ -745,8 +749,9 @@ release matrix proves the exact distributed digest. ### Implementation agent -- PR targets bootstrapped `allagentsdev/allagents-gateway` branch - `feat/workspace-composition` at the pinned fork point and preserves +- PR targets renamed `allagentsdev/allagents-gateway` branch + `feat/workspace-composition` at downstream commit `fbcb73132423c8c4575113fc8943c6a6280a4746` + and preserves the fork network, existing downstream commits, Apache-2.0/NOTICE/history/upstream. - Stock seams remain the integration points, and stock requests retain their pinned behavior. From 4356becc61e159c3f71cff5b2dfcc2c4e171ed19 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 08:38:24 +1000 Subject: [PATCH 36/44] docs(architecture): record gateway repository setup --- .../0002-adopt-uhp-through-harnessrouter.md | 2 +- ...0837-feat-coding-execution-gateway-plan.md | 44 ++++++++++--------- 2 files changed, 25 insertions(+), 21 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 432ff307..1335f344 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -225,7 +225,7 @@ Provider and harness failures that are not workspace failures retain their exact ## Distribution and release boundary -`allagentsdev/harnessrouter` is the existing GitHub fork of `HarnessRouter/harnessrouter`; creating a second repository would discard the fork identity and create two downstream sources of truth. Before implementation, an organization repository administrator MUST rename that repository in place to `allagentsdev/allagents-gateway`. The rename preserves repository identity, fork-network relationship, history, settings, and GitHub redirects. After the rename, verify the upstream parent and Apache-2.0 `LICENSE`/`NOTICE`, keep the pinned upstream commit as the characterized fork point, and create `feat/workspace-composition` from downstream `main` commit `fbcb73132423c8c4575113fc8943c6a6280a4746`, which contains that baseline plus the existing downstream commits. +On 2026-09-27, `allagentsdev/harnessrouter` was renamed in place to `allagentsdev/allagents-gateway`. GitHub preserves the repository identity, fork-network relationship, history, settings, and redirects, and still reports `HarnessRouter/harnessrouter` as the parent. The Apache-2.0 `LICENSE`/`NOTICE` and the pinned upstream commit remain the characterized baseline. Writable branch `feat/workspace-composition` exists at downstream `main` commit `fbcb73132423c8c4575113fc8943c6a6280a4746`, which contains that baseline plus the existing downstream commits. A second repository MUST NOT be created. Implementation also requires checked-in local builders for an authenticated OCI registry, catalog, and source fixtures, plus a Linux environment capable of namespace-confined bind mounts and reflink-or-copy isolation. The implementation agent can deliver a pull request using those fixtures and mocked provider transport without production secrets. Provider gateway URL/key, UHP caller credentials, GHCR rights, protected settings, and final publication/deployment are operator-owned inputs supplied through the repository's secret mechanism. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index add66403..131d7cc0 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -26,31 +26,35 @@ protocol, global composition cache, or materializer service. ## Implementation-handoff prerequisites -### Target rename: operator-owned and required before code work +### Target repository: operator setup complete -`allagentsdev/harnessrouter` already exists and GitHub reports it as a fork of -`HarnessRouter/harnessrouter`. Do not create or bootstrap a second repository. -An organization repository administrator, not the implementation agent, MUST: +On 2026-09-27, an organization repository administrator completed the +repository setup: -1. Rename `allagentsdev/harnessrouter` in place to - `allagentsdev/allagents-gateway`. Preserve its repository identity, fork +1. Renamed the existing `allagentsdev/harnessrouter` GitHub fork in place to + `allagentsdev/allagents-gateway`, preserving repository identity, fork network, complete history, settings, redirects, Apache-2.0 `LICENSE`, `NOTICE`, copyright, and attribution. -2. Verify GitHub still identifies `HarnessRouter/harnessrouter` as the parent. - Set `origin` to the renamed writable repository and retain that parent as - `upstream`. Record upstream commit - `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` as the characterized baseline; - upstream acceptance is not a dependency. -3. Create writable branch `feat/workspace-composition` from downstream `main` +2. Verified that GitHub still identifies `HarnessRouter/harnessrouter` as the + parent. Upstream commit `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` + remains the characterized baseline; upstream acceptance is not a dependency. +3. Created writable branch `feat/workspace-composition` from downstream `main` commit `fbcb73132423c8c4575113fc8943c6a6280a4746`. That commit is the pinned - upstream baseline plus the existing three downstream commits; implementation - MUST NOT discard them by branching directly from the older upstream commit. -4. Grant the implementation agent normal pull-request rights to that branch. -5. Keep the downstream distribution names fixed: - - repository: `allagentsdev/allagents-gateway`; - - image: `ghcr.io/allagentsdev/allagents-gateway`; - - service: `allagents-gateway`; and - - product: **AllAgents Gateway**. + upstream baseline plus the existing three downstream commits. +4. Updated the repository description, homepage, topics, and merge policy for + the AllAgents Gateway identity. + +The implementation clone MUST use the renamed repository as `origin` and retain +`HarnessRouter/harnessrouter` as `upstream`. It MUST start from the existing +`feat/workspace-composition` branch rather than discard downstream work by +branching from the older upstream commit. Do not create a second repository. + +Keep the downstream distribution names fixed: + +- repository: `allagentsdev/allagents-gateway`; +- image: `ghcr.io/allagentsdev/allagents-gateway`; +- service: `allagents-gateway`; and +- product: **AllAgents Gateway**. The implementation environment also requires these checked-in local test capabilities, with no external credentials: From c9c8e746e875c011dc3cfb6707c0f7a257abbe52 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 09:09:22 +1000 Subject: [PATCH 37/44] docs(architecture): preserve full Git history by default --- .../0002-adopt-uhp-through-harnessrouter.md | 13 ++- ...0837-feat-coding-execution-gateway-plan.md | 94 +++++++++++++------ 2 files changed, 69 insertions(+), 38 deletions(-) diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index 1335f344..ce3789fc 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -19,7 +19,7 @@ The existing extension seams are sufficient: - `BACKING.workspace` exposes either `RunnerWorkspaceFiles` or `CheckpointWorkspaceFiles`; and - the `HarnessSession` vertex owns session identity, while checkpoint, artifact, and control records remain separate. -The missing capability is first-turn initialization from one or more independently identified Git or OCI source trees. Git depth `2` bounds history but not working-tree transfer, so OCI transport and independent immutable component reuse are required in v1. +The missing capability is first-turn initialization from one or more independently identified Git or OCI source trees. Git sources preserve the complete ancestry reachable from the selected commit by default so agents can perform regression analysis; callers MAY request bounded shallow history explicitly. Full Git history can still be large, so OCI transport and independent immutable component reuse remain required in v1. ## Decision @@ -63,7 +63,7 @@ A workspace-backed first turn uses the normal `POST /v1/responses` endpoint. `me | `sources` | yes | Ordered closed array of 1 to 128 Git or OCI entries. | | `working_directory` | no | Workspace-relative POSIX directory; omission means `.`. | -A Git entry contains only `kind: "git"`, canonical public HTTPS `url`, optional `ref`, and `destination`. Userinfo, query, fragment, local paths, alternate transports, and private or otherwise disallowed network targets are forbidden. `ref` is an advertised full ref or unambiguous branch/tag shorthand; omission selects the advertised default. Resolution produces one exact commit. Depth is always `2` and is not caller-selectable. +A Git entry contains only `kind: "git"`, canonical public HTTPS `url`, optional `ref`, optional `depth`, and `destination`. Userinfo, query, fragment, local paths, alternate transports, and private or otherwise disallowed network targets are forbidden. `ref` is an advertised full ref or unambiguous branch/tag shorthand; omission selects the advertised default. Resolution produces one exact commit. Omitted `depth` means the complete ancestry reachable from that commit; a supplied `depth` is an integer from 1 through 1,000,000 and requests exactly that shallow history. Unrelated refs and tags are not fetched merely to satisfy full history. An OCI entry contains only `kind: "oci"`, `snapshot_name`, exact `image_manifest_digest`, exact `source_manifest_digest`, and `destination`. `snapshot_name` resolves through an operator-owned catalog to a fixed registry repository, catalog-entry identity, allowed media types, trust policy, and server-side credential reference. Callers cannot supply registry origins, repositories, tags, indexes, headers, redirects, or credentials. @@ -87,7 +87,6 @@ After the binding reaches `ready`, terminal events, response retrieval, replay, "url": "https://github.com/acme/api.git", "requested_ref": "refs/heads/main", "resolved_commit": "0123456789abcdef0123456789abcdef01234567", - "depth": 2, "destination": "services/api", "source_manifest_digest": "sha256:6666666666666666666666666666666666666666666666666666666666666666" }, @@ -103,7 +102,7 @@ After the binding reaches `ready`, terminal events, response retrieval, replay, } ``` -`working_directory` is always present and uses `.` for the workspace root. Git provenance contains normalized `url`, optional `requested_ref`, exact `resolved_commit`, `depth: 2`, `destination`, and the verified source-manifest digest. OCI provenance contains the catalog key, exact image and source-manifest digests, and `destination`. Registry details, credentials, private cache keys, backing paths, and live attachment details are never public. +`working_directory` is always present and uses `.` for the workspace root. Git provenance contains normalized `url`, optional `requested_ref`, exact `resolved_commit`, optional `depth`, `destination`, and the verified source-manifest digest. Omitted `depth` explicitly means full reachable ancestry; a present value records the requested shallow depth. OCI provenance contains the catalog key, exact image and source-manifest digests, and `destination`. Registry details, credentials, private cache keys, backing paths, and live attachment details are never public. Every workspace-backed session receives one operator-configured finite expiry at creation. `expires_at` is always a timestamp; polling, replay, and continuation do not extend it. Explicit deletion remains supported. Failures before `ready` omit workspace metadata; failures after `ready` return the stored sanitized object. @@ -137,7 +136,7 @@ The gateway walks without following links and recomputes the canonical manifest ## Resolution, caching, and visibility -A private Git component key is exactly the canonical URL, exact resolved commit, depth `2`, and one cache-schema revision. A private OCI component key is exactly the catalog-entry identity, exact image-manifest digest, exact source-manifest digest, and one cache-schema revision. Recomputing a baseline digest proves publication integrity; it is not a cache-key input. +A private Git component key is exactly the canonical URL, exact resolved commit, history selector (`full` or exact requested depth), and one cache-schema revision. A private OCI component key is exactly the catalog-entry identity, exact image-manifest digest, exact source-manifest digest, and one cache-schema revision. Recomputing a baseline digest proves publication integrity; it is not a cache-key input. Components cache independently and exact-key misses singleflight independently. Git retains the accepted operator-only acquisition mirror per canonical URL, then publishes an immutable verified generation. OCI MAY use standard registry-client, image, and layer caches; this decision does not require a separate gateway-managed blob-cache lifecycle. There is no request-wide composition cache, record, or public identity. @@ -149,7 +148,7 @@ The ordered plan remains `pending` until every component is verified, each desti For `read_only`, each immutable component root is exposed at its destination through a namespace-confined read-only bind mount with `nodev` and `nosuid`, without a writable alias or copy-up path. For `editable`, each destination is a quota-bounded, inode-independent private reflink or copy. Git and OCI receive identical write semantics. The outer workspace remains private and writable in both modes. -Git acquisition resolves only advertised refs, fetches the selected commit at depth `2`, and verifies the fetched tip, bounded object graph, checkout, and source manifest. Commands run without a shell in a sanitized, isolated configuration; credentials, inherited proxies, hooks, filters, LFS hydration, submodule recursion, alternates, and non-HTTPS helpers are disabled. The published tree may retain safe shallow `.git` metadata and bounded recent history for agent convenience, but removes credential-bearing remotes and unsafe or transient state. Git metadata never defines evaluation correctness. +Git acquisition resolves only advertised refs, fetches either the complete ancestry reachable from the selected commit or the exact requested shallow depth, and verifies the fetched tip, bounded object graph, checkout, and source manifest. Full history does not imply unrelated branches or tags. Commands run without a shell in a sanitized, isolated configuration; credentials, inherited proxies, hooks, filters, LFS hydration, submodule recursion, alternates, and non-HTTPS helpers are disabled. The published tree retains safe self-contained `.git` metadata for the selected history while removing credential-bearing remotes and unsafe or transient state. Git metadata never defines evaluation correctness. Every fetch remains subject to contractual byte, object, time, and process limits; exceeding one fails rather than silently reducing history. OCI acquisition uses the catalog-selected direct image manifest and the declared source manifest. It verifies descriptor media types, sizes, and digests; applies layers in order with standard whiteout and opaque-directory behavior; and extracts with rooted no-follow operations. Traversal, out-of-root links, devices, sockets, FIFOs, sparse-file tricks, undeclared or missing entries, unsupported types, and digest or type mismatches fail closed. OCI sources need not contain Git metadata. @@ -250,7 +249,7 @@ AllAgents Gateway remains one execution, workspace, and session control plane. I Canonical manifests and change artifacts add bounded filesystem scanning and hashing, but give Git and OCI one evaluator-visible definition of state. Promptfoo can reconstruct results entirely from ordered UHP artifacts, regardless of Git metadata or index state. -Mandatory OCI support and Linux mount/copy capabilities make v1 substantial, but they satisfy the large-source requirement while preserving safe shallow Git history for agent convenience and Git-free OCI operation. +Mandatory OCI support and Linux mount/copy capabilities make v1 substantial, but they satisfy the large-source requirement while preserving full Git ancestry by default, explicit shallow acquisition when requested, and Git-free OCI operation. ## Reconsider when diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 131d7cc0..f64f1f83 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -129,8 +129,9 @@ There is no provider-specific workspace implementation phase. - A strict, closed, first-turn-only `metadata.workspace` request extension. - An ordered array of 1 to 128 Git, OCI, or mixed source entries. - Exactly one tree and one required non-root destination per source. -- Canonical public HTTPS Git acquisition at an exact resolved commit with fixed - depth 2 and safe bounded `.git` metadata for agent convenience. +- Canonical public HTTPS Git acquisition at an exact resolved commit with full + reachable ancestry by default, optional explicit shallow depth, and safe + self-contained `.git` metadata for regression analysis. - Operator-cataloged OCI source trees selected by exact image-manifest and source-manifest digests, including Git-free trees. - Per-component cache/singleflight followed by one session-local resolved plan. @@ -146,8 +147,9 @@ There is no provider-specific workspace implementation phase. resource limits. - A public component-cache or composition API, a global composition record, cache, identifier, or compatibility aliases for obsolete workspace schemas. -- Git-to-OCI, OCI-to-Git, ref, digest, registry, provider, deepening, full-clone, - or history fallback. +- Git-to-OCI, OCI-to-Git, ref, digest, registry, provider, or history-mode + fallback. A failed full fetch never silently becomes shallow, and a failed + shallow fetch never deepens. - Git submodule initialization, LFS hydration, checkout filters, or hook execution. - Requiring Git at workspace root or in OCI trees. @@ -193,6 +195,7 @@ metadata.workspace = { kind: "git", url: string, ref?: string, + depth?: integer, destination: string } | { @@ -207,6 +210,11 @@ metadata.workspace = { } ``` +For a Git source, omitted `depth` means the complete ancestry reachable from the +resolved commit. A supplied `depth` is an integer from 1 through 1,000,000 and +requests exactly that shallow boundary. Full history does not fetch unrelated +refs or tags merely for completeness. + There is no request `retention` field and no `persistent` mode. The obsolete singular `source`, `kind: "repositories"`, `repositories`, `kind: "workspace_snapshot"`, and `workspace_manifest_digest` forms are @@ -229,7 +237,7 @@ metadata.workspace = { url: string, destination: string, resolved_commit: string, - depth: 2, + depth?: integer, source_manifest_digest: "sha256:<64 lowercase hex>", requested_ref?: string } @@ -244,6 +252,9 @@ metadata.workspace = { } ``` +Git response provenance mirrors the selected history: omitted `depth` means full +reachable ancestry, while a present value is the exact requested shallow depth. + `working_directory` is always present and uses `.` for the outer root. There is no public `retention`, `effective_descriptor_digest`, `composition_id`, or per-source `component_id`. Catalog coordinates, private cache keys, mirrors, @@ -271,7 +282,9 @@ traffic, cache lookup, component claim, workspace write, or expiry mutation: 6. Accept only canonical public HTTPS Git URLs allowed by deployment egress policy. Reject userinfo, query, fragment, ambiguous encodings, alternate transports, and caller Git options. A ref resolves only through advertised - default, branch, or tag semantics. + default, branch, or tag semantics. Omitted `depth` means complete ancestry + reachable from the resolved commit; a supplied `depth` MUST be an integer + from 1 through 1,000,000. 7. For OCI require a catalog `snapshot_name` and direct SHA-256 image/source manifest digests. Reject tags, indexes/lists, caller registry coordinates, and mutable references. @@ -290,6 +303,7 @@ contract revision: | Expanded bytes | 32 GiB | 64 GiB | | Source-visible entries | 500,000 | 1,000,000 | | Compressed Git pack or OCI layer bytes | 8 GiB | 16 GiB | +| Reachable Git objects | 5,000,000 | 10,000,000 | | Regular-file bytes | 4 GiB | 4 GiB per file | | Path | 4096 UTF-8 bytes / 128 components | same per path | | Acquisition/materialization time | bounded operator policy | bounded session policy | @@ -339,7 +353,8 @@ materialized as ordinary files and are not a manifest type. `.git` entries MAY be covered by source integrity manifests, but `.git` is always excluded from public change reporting. -Git acquisition computes this manifest from the verified detached depth-2 tree. +Git acquisition computes this manifest from the verified detached tree with the +selected full or shallow history. OCI fetches and validates the named source manifest before requesting any layer, applies standard OCI image/layer/whiteout semantics, and requires the extracted final tree to match exactly. @@ -409,8 +424,8 @@ verifies. Private component keys MUST be computable before materialization: -- Git key: canonical URL + exact resolved commit + depth `2` + one - cache-schema revision. +- Git key: canonical URL + exact resolved commit + history selector (`full` or + exact requested depth) + one cache-schema revision. - OCI key: catalog entry identity + exact image-manifest digest + exact source-manifest digest + one cache-schema revision. @@ -422,18 +437,24 @@ epoch, or baseline-digest dimensions to the key. ### Git acquisition -Maintain one operator-only bare shallow acquisition mirror per canonical URL and +Maintain one operator-only bare acquisition mirror per canonical URL and serialize its writes. Resolve the advertised default, branch, or lightweight or -annotated tag to an exact commit, fetch with fixed depth 2, verify tip and shallow -boundary, and export a self-contained detached checkout with no alternates or -writable mirror links. Preserve safe bounded `.git` metadata sufficient for -recent offline log, parent inspection, blame where shallow history permits, and -diff. Preserve available merge parents within depth 2. +annotated tag to an exact commit. With omitted `depth`, fetch the complete +ancestry reachable from that commit without fetching unrelated refs merely for +completeness. With supplied `depth`, fetch exactly that shallow ancestry. +Verify the tip and requested history boundary, then export a self-contained +detached checkout with no alternates or writable mirror links. Preserve safe +`.git` metadata for offline log, parent inspection, blame, and diff within the +selected history. Editable private copies additionally support bisect; read-only +mounts do not promise Git operations that mutate the worktree or repository. Disable interactive credentials, hooks, filters, alternates, alternate protocols, submodules, and LFS hydration. Reject gitlinks and LFS pointer-backed -content. Never deepen, unshallow, full-clone, fetch arbitrary object IDs, choose -another ref, or fall back to OCI. +content. Never change the requested history mode, fetch arbitrary object IDs, +choose another ref, fetch unrelated refs as a completeness shortcut, or fall +back to OCI. Contractual pack, expanded-byte, object, time, process, and output +limits apply to full and shallow acquisition; exceeding one fails without +publishing a component. ### OCI acquisition @@ -564,10 +585,10 @@ errors retain stock codes only where their meaning is exact. | Detail code | Condition | HTTP | Retryable | Required behavior | |---|---|---:|:---:|---| -| `workspace_invalid_request` | Closed-schema, count, field, URL, digest syntax, path, cwd, first-turn, or reused-session violation | 400 | no | Fail before cache, network, workspace write, claim, or lifecycle mutation. | +| `workspace_invalid_request` | Closed-schema, count, field, URL, depth syntax/range, digest syntax, path, cwd, first-turn, or reused-session violation | 400 | no | Fail before cache, network, workspace write, claim, or lifecycle mutation. | | `workspace_path_collision` | Equal/overlapping destinations or input/generated/reserved/ancestor collision | 409 | no | Fail before cache or network; report only sanitized conflicting workspace-relative fields. | | `workspace_source_unknown` | Unknown OCI catalog entry or missing/ambiguous/unsupported Git ref identity | 404 | no | Fail that source with no alternate ref, catalog entry, or source kind. | -| `workspace_source_invalid` | Moved/non-commit Git target, depth-2 refusal, gitlink/LFS content, OCI media/digest/source-manifest/layer/final-tree failure, or unsafe source content | 422 | no | Publish no failed component and perform no fallback; invalid OCI source manifest fails before layer requests. | +| `workspace_source_invalid` | Moved/non-commit Git target, requested-history refusal, gitlink/LFS content, OCI media/digest/source-manifest/layer/final-tree failure, or unsafe source content | 422 | no | Publish no failed component and perform no ref, history-mode, or source-kind fallback; invalid OCI source manifest fails before layer requests. | | `workspace_acquisition_unavailable` | Timeout, DNS, registry/Git service, or other transient source transport failure | 503 | yes | Detach request-local work, preserve independently valid shared components, and expose no partial workspace. | | `workspace_contract_limit_exceeded` | A fixed v1 per-source or aggregate count/byte/path/ratio/time/output ceiling is exceeded | 413 | no | Stop bounded work, clean/quarantine staging, and expose no partial workspace. | | `workspace_capacity_exceeded` | Operator concurrency, disk, inode, mount, or lower policy capacity is temporarily unavailable | 503 | yes | Admit no partial binding; capacity policy must not masquerade as a schema limit. | @@ -626,10 +647,13 @@ exact fork point. shape after ready. - Reject workspace metadata on every reuse/continuation path before hydration. -**Exit proof:** 1 and 128 Git/OCI/mixed entries pass; 0/129, obsolete fields, -unknown fields, root/overlap/collision, malformed identities, and reused-session -injection return their exact coded errors with zero source/cache activity. Stock -traces remain unchanged. +**Exit proof:** omitted depth and depths 1 and 1,000,000 parse and persist; depth +0, 1,000,001, fractional, and wrong-type values return +`400 workspace_invalid_request` with zero source/cache activity. One and 128 +Git/OCI/mixed entries pass; 0/129, obsolete fields, unknown fields, +root/overlap/collision, malformed identities, and reused-session injection +return their exact coded errors with zero source/cache activity. Stock traces +remain unchanged. ### Phase 3: Implement canonical manifests and produced projection @@ -667,20 +691,28 @@ without source traversal during collection and require fresh mount evidence; editable sessions cannot mutate cache/sibling content. Restart discards or reconciles uncertain publications without persisted mount/inode facts. -### Phase 5: Implement depth-2 Git acquisition +### Phase 5: Implement full-by-default Git acquisition **Work** - Implement canonical HTTPS validation, advertised ref resolution, the per-URL - serialized mirror, fixed depth-2 fetch, detached self-contained export, - bounded safe `.git`, and source-manifest publication evidence. -- Enforce egress/redirect policy, limits, disabled helpers, and no fallback. + serialized mirror, full reachable ancestry by default, exact optional shallow + depth, detached self-contained export, safe `.git`, and source-manifest + publication evidence. +- Enforce egress/redirect policy, limits, disabled helpers, and no ref, + history-mode, or source-kind fallback. **Exit proof:** default/branch/lightweight-tag/annotated-tag/merge cases resolve -to exact commits; recent shallow offline log/blame/diff works within depth 2; -moved refs, unsupported shallow servers, malicious redirects, gitlinks, LFS, -limits, cancellation, and restart fail with exact codes. An exact key hit does -no pack acquisition or materialization. +to exact commits. Omitted depth provides offline log/blame/diff across complete +fetched ancestry in both access modes and bisect in editable mode. A fixture +with unrelated branches and tags proves they are neither requested for the +selected full ancestry nor exposed in the published component. Depths 1 and 2 +expose exactly their shallow boundaries, publish distinct components, and each +reuses only its exact-depth component on repetition. Different ref spellings +resolving to one commit and the same history selector reuse one component; full, +depth 1, and depth 2 remain distinct. Moved refs, unsupported history requests, +malicious redirects, gitlinks, LFS, limits, cancellation, and restart fail with +exact codes. An exact key hit performs no pack acquisition or materialization. ### Phase 6: Implement OCI source-tree acquisition From 7ba2d02175bf4eef2ec00172ab988b160ede2c6d Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 11:31:04 +1000 Subject: [PATCH 38/44] docs(architecture): separate workspace preparation from execution --- .../0002-adopt-uhp-through-harnessrouter.md | 390 +++-- ...0837-feat-coding-execution-gateway-plan.md | 1328 ++++++++--------- .../allagents-gateway-snapshot-boundary.md | 367 +++++ .../e2b-execution-gateway-patterns.md | 6 +- .../harbor-repository-materialization.md | 62 +- .../source-credential-broker-precedents.md | 45 +- .../research/workspace-contract-incumbents.md | 63 +- 7 files changed, 1274 insertions(+), 987 deletions(-) create mode 100644 docs/research/allagents-gateway-snapshot-boundary.md diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md index ce3789fc..996d7203 100644 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md @@ -1,4 +1,4 @@ -# ADR 0002: Adopt UHP through AllAgents Gateway with composable workspace sources +# ADR 0002: Adopt UHP through AllAgents Gateway with prepared workspace snapshots - Status: Accepted - Date: 2026-09-21 @@ -6,117 +6,115 @@ ## Context -Promptfoo needs a remote coding-harness endpoint that can prepare large source trees before the first turn, preserve session state across continuations, and expose exact source provenance and filesystem changes through the Unified Harness Protocol (UHP). +Promptfoo needs a remote coding-harness endpoint that can start from large reproducible workspaces, preserve several complete Git histories, continue a session without source drift, and return exact filesystem changes. -The pinned baseline is HarnessRouter commit [`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), release [`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4), and UHP version [`2026-09-12`](https://github.com/HarnessRouter/harnessrouter/tree/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/versions/2026-09-12). That source is Apache-2.0 licensed and includes a `NOTICE`; a downstream repository MUST preserve the license, `NOTICE`, attribution, and history. +The accepted HarnessRouter baseline is commit [`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), release [`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4), and UHP version [`2026-09-12`](https://github.com/HarnessRouter/harnessrouter/tree/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/versions/2026-09-12). The current implementation point inspected for this revision is commit [`8f7868ccb2c97d1f611acf11e7cad0357a43064e`](https://github.com/HarnessRouter/harnessrouter/commit/8f7868ccb2c97d1f611acf11e7cad0357a43064e). The relevant workspace behavior is unchanged between those points. -Stock HarnessRouter already owns session identity, user and sandbox isolation, the private workspace, checkpoints, cancellation, Files and artifacts, harness supervision, cleanup, live-workspace cache reaping, and explicit deletion of durable sessions. A fresh stock workspace initializes a root Git repository. Git supplies the produced-file listing cursor: checkpoint creation commits and then archives the directory, while hydration restores the archive. Git is not a durable-session expiry mechanism. +Stock HarnessRouter already owns UHP, authentication, session and response identity, harness supervision, one private session filesystem, checkpoint/hydrate, Files and artifacts, cancellation, and deletion. Its root Git repository is an internal produced-file journal, not a model that the agent may edit only one repository. Repositories are ordinary content inside the session filesystem. -The existing extension seams are sufficient: +Stock behavior is insufficient for prepared multi-repository workspaces: -- runner `_produced_list` lists produced paths from a cursor and `_produced_ack` advances that cursor; -- gateway `_collect_produced` durably captures listed files before acknowledging them; -- `BACKING.workspace` exposes either `RunnerWorkspaceFiles` or `CheckpointWorkspaceFiles`; and -- the `HarnessSession` vertex owns session identity, while checkpoint, artifact, and control records remain separate. +- there is no supported public immutable-snapshot import contract; +- `_git_ensure` creates or mutates `.git` at the workspace root; +- root Git cannot report exact descendant changes across embedded repositories and stock collection omits deletions; +- stock checkpoints rearchive the full workspace and are session-keyed rather than a cross-session immutable snapshot cache; and +- `BACKING.workspace` reads or writes one file and is not an acquisition seam. -The missing capability is first-turn initialization from one or more independently identified Git or OCI source trees. Git sources preserve the complete ancestry reachable from the selected commit by default so agents can perform regression analysis; callers MAY request bounded shallow history explicitly. Full Git history can still be large, so OCI transport and independent immutable component reuse remain required in v1. +The previous version of this ADR placed Git resolution, OCI acquisition, multi-source composition, source credentials, caching, attachment, execution, and collection inside AllAgents Gateway. That crosses two trust and lifecycle boundaries. Mutable source preparation belongs before execution. HarnessRouter should receive one already-published immutable filesystem, not a product-specific source plan. + +The supporting evidence is in [Prebuilt immutable workspace snapshots at the HarnessRouter boundary](../research/allagents-gateway-snapshot-boundary.md). ## Decision -We will ship a downstream product named **AllAgents Gateway**, derived from the pinned HarnessRouter baseline and distributed as `ghcr.io/allagentsdev/allagents-gateway`. UHP remains the only northbound protocol. `metadata.workspace` is an explicitly downstream first-turn extension; requests that omit it retain pinned stock behavior. +We will use two components in two source repositories: -A descriptor contains one ordered `sources` array with 1 to 128 entries. Each entry materializes one tree at a pairwise non-overlapping, non-root destination. Git and OCI entries MAY be mixed in any order. One OCI image represents one source tree, not a runtime image, benchmark image, verifier, or multi-root bundle. A source-kind failure never falls back to the other kind. +1. **AllAgents Workspace Builder** in `allagentsdev/allagents-workspace-builder` owns Git and OCI acquisition, credentials, multi-repository composition, source policy, provenance, canonical baseline creation, and immutable snapshot publication. +2. **AllAgents Gateway** in `allagentsdev/allagents-gateway` remains a stock-derived HarnessRouter distribution. It accepts one exact snapshot descriptor, authorizes it, initializes a private writable session tree, journals filesystem changes without root Git, and otherwise retains HarnessRouter's UHP/session lifecycle. -The gateway adds workspace binding and a `pending` to `ready` transition to the existing session lifecycle. It MUST resolve, verify, and materialize every root and validate the working directory before making any source visible to the harness or starting provider work. “Atomic” means application visibility after all roots verify; it does not require an atomic filesystem rename or namespace handoff. Failure before `ready` exposes no partial workspace. +This separation is a source and trust boundary, not a requirement to deploy two always-on services. The builder SHOULD begin as a CLI/library usable from CI or a job worker. It MAY gain an asynchronous service wrapper when workload or latency requires one. Published OCI artifacts are the only execution handoff. -Implementation and release do not wait for an upstream issue or UHP proposal. Until upstream accepts an equivalent contract, releases MUST identify this behavior as an AllAgents Gateway extension. +UHP remains the only northbound execution protocol. Snapshot execution is an AllAgents vendor extension, not a claim that UHP 2026-09-12 standardizes workspace snapshots. Requests without the extension retain characterized stock behavior. -## Request contract +AllAgents will propose upstream-neutral immutable workspace initialization, Git-independent journaling, and recoverable terminal finalization seams to HarnessRouter. Downstream implementation may proceed while the proposal is reviewed. Product-specific Git/OCI composition and the AllAgents snapshot format remain outside HarnessRouter. -A workspace-backed first turn uses the normal `POST /v1/responses` endpoint. `metadata.workspace` is a closed object with no nested schema version: +## Preparation boundary -```json -{ - "access": "editable", - "sources": [ - { - "kind": "git", - "url": "https://github.com/acme/api.git", - "ref": "refs/heads/main", - "destination": "services/api" - }, - { - "kind": "oci", - "snapshot_name": "compiler-tree", - "image_manifest_digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111", - "source_manifest_digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222", - "destination": "vendor/compiler" - } - ], - "working_directory": "services/api/packages/server" -} -``` +The builder accepts an AllAgents-owned build request that MAY contain several Git and OCI inputs, non-overlapping destinations, and one default working directory. That build API is not a UHP request and is not accepted by AllAgents Gateway. -| Field | Required | Contract | -|---|---:|---| -| `access` | yes | `read_only` or `editable`, applied to every source. | -| `sources` | yes | Ordered closed array of 1 to 128 Git or OCI entries. | -| `working_directory` | no | Workspace-relative POSIX directory; omission means `.`. | +The builder MUST: -A Git entry contains only `kind: "git"`, canonical public HTTPS `url`, optional `ref`, optional `depth`, and `destination`. Userinfo, query, fragment, local paths, alternate transports, and private or otherwise disallowed network targets are forbidden. `ref` is an advertised full ref or unambiguous branch/tag shorthand; omission selects the advertised default. Resolution produces one exact commit. Omitted `depth` means the complete ancestry reachable from that commit; a supplied `depth` is an integer from 1 through 1,000,000 and requests exactly that shallow history. Unrelated refs and tags are not fetched merely to satisfy full history. +- resolve every mutable Git ref to an exact commit before publication; +- preserve complete ancestry reachable from the selected commit by default, with shallow history only when explicitly requested; +- keep source credentials, Git helpers, mirrors, registry coordinates, and acquisition network policy out of the artifact and agent environment; +- compose all inputs into one staging tree without overlapping destinations or reserved-path collisions; +- remove acquisition-only state, credential-bearing remotes, unsafe alternates, transient Git locks, devices, sockets, FIFOs, capabilities, ACLs, xattrs, and special permission bits; +- validate self-contained `.git` repositories semantically while keeping their bytes inside the snapshot; +- generate a canonical visible-tree manifest that excludes every `.git` tree and runner-owned paths; +- generate digest-covered provenance that records ordered source identity, destination, requested selector, resolved immutable identity, history completeness, builder version, and policy version; +- publish layers, config, provenance, and the tree manifest completely before returning a direct OCI image-manifest descriptor; and +- never return a tag or multi-platform index as the execution identity. -An OCI entry contains only `kind: "oci"`, `snapshot_name`, exact `image_manifest_digest`, exact `source_manifest_digest`, and `destination`. `snapshot_name` resolves through an operator-owned catalog to a fixed registry repository, catalog-entry identity, allowed media types, trust policy, and server-side credential reference. Callers cannot supply registry origins, repositories, tags, indexes, headers, redirects, or credentials. +Preparation failure occurs before UHP session creation. The gateway never retries a failed build, chooses another ref, changes history depth, or falls back between Git and OCI. -Every destination and `working_directory` is an NFC-normalized relative POSIX path with no empty, `.`, `..`, absolute, platform-specific, or reserved component. Destinations MUST be non-root and pairwise non-overlapping. Destination ownership, reserved-path conflicts, and collisions with known inputs or generated assets MUST be rejected before DNS, Git, registry, or other source access. Ancestor directories may be empty scaffolding only. After materialization, `working_directory` MUST resolve without symlink escape to a real directory. +## Snapshot artifact contract -The order of `sources` is semantic but does not establish overlay precedence. Unknown keys are rejected at every level. The descriptor cannot contain credentials, headers, host paths, commands, environment variables, runtime images, materializer selection, resource limits, provider routes, or expiry controls. Request size, string length, nesting, and validation work are bounded before source access. +A workspace snapshot is one OCI image manifest with: -A continuation selected by `previous_response_id` or other inherited recovery state MUST omit `metadata.workspace`. A stock session cannot acquire a workspace binding later, and an existing workspace-backed session cannot replace or repeat its descriptor. +- `artifactType: application/vnd.allagents.workspace-snapshot.v1`; +- config media type `application/vnd.allagents.workspace-snapshot.config.v1+json`; +- provenance media type `application/vnd.allagents.workspace-provenance.v1+json`; +- canonical visible-tree media type `application/vnd.allagents.workspace-manifest.v1+json`; +- only gzip filesystem layers with media type `application/vnd.oci.image.layer.v1.tar+gzip`; and +- ordered OCI filesystem changesets applied to an empty directory. -## Response and provenance contract +The closed digest-covered config contains `version: 1`, default relative working directory, workspace-manifest descriptor, provenance descriptor, builder/policy identity, `reserved_paths_schema: 1`, and `available_until`. The builder emits deterministic gzip with fixed headers and rejects uncompressed, zstd, and nondistributable layer media types in V1. -After the binding reaches `ready`, terminal events, response retrieval, replay, and later terminal failures expose the same sanitized `metadata.workspace` object: +Reserved-path schema 1 contains exactly the root `.harness` path and every descendant. The builder rejects any entry or link alias at that location. Harness instruction files such as root `AGENTS.md` are visible snapshot content, not reserved paths: snapshot mode MUST preserve existing content, merge any runner-managed block deterministically, and establish the initial visible-tree cursor only after that merge and UHP input application. -```json +Provenance is RFC 8785 canonical JSON. Its closed schema is: + +```text { - "access": "editable", - "working_directory": "services/api/packages/server", - "sources": [ - { - "kind": "git", - "url": "https://github.com/acme/api.git", - "requested_ref": "refs/heads/main", - "resolved_commit": "0123456789abcdef0123456789abcdef01234567", - "destination": "services/api", - "source_manifest_digest": "sha256:6666666666666666666666666666666666666666666666666666666666666666" - }, - { - "kind": "oci", - "snapshot_name": "compiler-tree", - "image_manifest_digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111", - "source_manifest_digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222", - "destination": "vendor/compiler" - } - ], - "expires_at": "2026-09-29T00:00:00Z" + version: 1, + sources: Array< + | {kind: "git", url: string, requested_ref?: string, resolved_commit: string, + history: {mode: "full"} | {mode: "shallow", depth: integer}, destination: string} + | {kind: "oci", source_name: string, + descriptor: {media_type: string, digest: string, size: integer}, destination: string} + > } ``` -`working_directory` is always present and uses `.` for the workspace root. Git provenance contains normalized `url`, optional `requested_ref`, exact `resolved_commit`, optional `depth`, `destination`, and the verified source-manifest digest. Omitted `depth` explicitly means full reachable ancestry; a present value records the requested shallow depth. OCI provenance contains the catalog key, exact image and source-manifest digests, and `destination`. Registry details, credentials, private cache keys, backing paths, and live attachment details are never public. +Source order is build order. The builder and policy identity remain in config. Provenance contains no credential, private registry coordinate, header, host path, or helper state. + +The builder returns one versioned result: -Every workspace-backed session receives one operator-configured finite expiry at creation. `expires_at` is always a timestamp; polling, replay, and continuation do not extend it. Explicit deletion remains supported. Failures before `ready` omit workspace metadata; failures after `ready` return the stored sanitized object. +```json +{ + "version": 1, + "descriptor": { + "media_type": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", + "size": 123456 + }, + "workspace_manifest_digest": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "provenance_digest": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "working_directory": "services/api", + "available_until": "2026-10-29T00:00:00Z" +} +``` + +Only `descriptor` is forwarded to the gateway. `available_until` is also digest-covered by config and is backed by an operator-enforced registry retention lease. The gateway admits a new session only when the remaining lease covers initialization deadline plus maximum session TTL plus safety margin. -## Canonical source manifest v1 +The direct manifest digest transitively binds config, provenance, the visible-tree manifest, layers, all `.git` bytes, and the retention deadline. OCI referrers MAY add signatures or attestations, but required provenance MUST remain digest-covered by the admitted manifest because an OCI `subject` association is weak and can change independently. -The source-manifest media type is `application/vnd.allagents.source-manifest.v1+json`. Its exact bytes are the RFC 8785 JSON Canonicalization Scheme representation of: +The canonical visible-tree manifest uses media type `application/vnd.allagents.workspace-manifest.v1+json` and RFC 8785 canonical JSON: ```json {"version":1,"entries":[]} ``` -`entries` is sorted by the UTF-8 bytes of each NFC-normalized relative POSIX `path`. The root is omitted. Duplicate paths, non-UTF-8 or non-NFC names, empty, `.` or `..` path components, type conflicts, and unsupported file types fail validation. The digest exposed as `source_manifest_digest` is `sha256:` followed by the lowercase hexadecimal SHA-256 of the canonical bytes. - -Entries have exactly one of these forms: +Entries sort by the UTF-8 bytes of their NFC-normalized relative POSIX path. The root is omitted; empty directories are represented. A path has exactly one of these states: ```json {"path":"src","type":"directory"} @@ -124,46 +122,100 @@ Entries have exactly one of these forms: {"path":"bin/tool","type":"symlink","target":"../src/tool"} ``` -A file `sha256` is `sha256:` followed by the lowercase hexadecimal SHA-256 of -its content. `executable: false` represents normalized mode `0644`; `true` -represents `0755`. A symlink target MUST be UTF-8, NFC, relative, and confined -to its owning source root when resolved from the link's parent. Empty -directories are represented. Safe in-root OCI hardlinks may be materialized as -ordinary file entries and are not a manifest type. `.git` content MAY be -integrity-verified in a source manifest but is excluded from change reporting. +Traversal never follows links. File modes normalize to `0644` or `0755`; directories normalize to `0755`; ownership is runtime-assigned rather than artifact-controlled. Symlink targets MUST be UTF-8, NFC, relative, and confined when resolved from the link parent. Safe in-root OCI hardlinks MAY be materialized as ordinary files. Duplicate, non-UTF-8, non-NFC, absolute, traversing, escaping, unsupported, or conflicting entries fail publication or admission. -The gateway walks without following links and recomputes the canonical manifest before publication. For OCI, it fetches and digest-verifies the source manifest before any layer, validates its paths and limits, applies ordinary OCI image/layer/whiteout semantics, and requires the extracted tree to reproduce the declared canonical manifest exactly. +Every `.git` tree and reserved `.harness` tree is excluded from the visible-tree manifest and public changes. The complete materialized namespace is still scanned independently for forbidden `.harness` entries before cache publication and ready. Layer digests bind `.git` bytes, and builder provenance records semantic Git verification. Evaluation correctness never depends on a Git index, status, commit, ignore rule, or rename heuristic. -## Resolution, caching, and visibility +## Gateway request contract -A private Git component key is exactly the canonical URL, exact resolved commit, history selector (`full` or exact requested depth), and one cache-schema revision. A private OCI component key is exactly the catalog-entry identity, exact image-manifest digest, exact source-manifest digest, and one cache-schema revision. Recomputing a baseline digest proves publication integrity; it is not a cache-key input. +A snapshot-backed first turn uses `POST /v1/responses` with a closed, versioned vendor extension: -Components cache independently and exact-key misses singleflight independently. Git retains the accepted operator-only acquisition mirror per canonical URL, then publishes an immutable verified generation. OCI MAY use standard registry-client, image, and layer caches; this decision does not require a separate gateway-managed blob-cache lifecycle. There is no request-wide composition cache, record, or public identity. +```json +{ + "input": "Make the requested change.", + "metadata": { + "allagents_workspace_snapshot": { + "version": 1, + "descriptor": { + "media_type": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", + "size": 123456 + } + } + } +} +``` -The session binding stores the private descriptor digest, ordered resolved source plan, exact private component keys, public provenance, access, working directory, canonical baselines and acknowledged manifest cursors, durable component and checkpoint references, and `expires_at`. It MUST NOT persist live mount IDs, filesystem identities, attachment flags, publication generations, or inode evidence. Live read-only protection, destination ownership, and writable-copy isolation are revalidated on every attach. +Only `version` and `descriptor` are accepted. Unknown keys fail. The caller cannot provide a registry, repository, tag, index, credential, header, redirect policy, host path, source list, materializer, working directory, resource limit, provider route, or expiry. -The ordered plan remains `pending` until every component is verified, each destination is safely materialized, and the working directory is valid. One transition to `ready` makes the plan visible to application code. A failure rolls back provisional work and leaves no visible partial plan. +V1 maps each authenticated product domain to exactly one trusted snapshot repository and one authorization catalog; zero or multiple repository mappings are a deployment error. The catalog entry binds the exact descriptor and digest-covered `available_until`. Before cache use or registry traffic, the gateway resolves one entry and authorizes `(principal, domain, repository_id, catalog_entry_id, media_type, digest, size)` against current policy. A digest proves identity, not authorization. Every cache hit reauthorizes the tuple; policy revision does not fragment the byte-cache key. -## Access and source integrity +The extension is accepted only on the request that creates a new session. Continuations selected by `previous_response_id` MUST omit it. A stock session cannot acquire a snapshot later, and a bound session cannot repeat or replace its descriptor. -For `read_only`, each immutable component root is exposed at its destination through a namespace-confined read-only bind mount with `nodev` and `nosuid`, without a writable alias or copy-up path. For `editable`, each destination is a quota-bounded, inode-independent private reflink or copy. Git and OCI receive identical write semantics. The outer workspace remains private and writable in both modes. +After initialization reaches `ready`, terminal events, response retrieval, replay, and later terminal failures expose the same sanitized metadata: -Git acquisition resolves only advertised refs, fetches either the complete ancestry reachable from the selected commit or the exact requested shallow depth, and verifies the fetched tip, bounded object graph, checkout, and source manifest. Full history does not imply unrelated branches or tags. Commands run without a shell in a sanitized, isolated configuration; credentials, inherited proxies, hooks, filters, LFS hydration, submodule recursion, alternates, and non-HTTPS helpers are disabled. The published tree retains safe self-contained `.git` metadata for the selected history while removing credential-bearing remotes and unsafe or transient state. Git metadata never defines evaluation correctness. Every fetch remains subject to contractual byte, object, time, and process limits; exceeding one fails rather than silently reducing history. +```json +{ + "version": 1, + "descriptor": { + "media_type": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", + "size": 123456 + }, + "snapshot_manifest_digest": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "ready_manifest_digest": "sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "provenance_digest": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "initialization_changes_file_id": "file_...", + "working_directory": "services/api", + "available_until": "2026-10-29T00:00:00Z", + "expires_at": "2026-09-29T00:00:00Z" +} +``` + +Repository coordinates, catalog entries, policy revisions, credentials, private cache keys, host paths, local clone mechanisms, and builder job identifiers are never public. Full source provenance is retrieved from the digest-covered snapshot provenance; UHP returns its immutable digest rather than copying a product-specific source schema into HarnessRouter. + +## Binding and initialization lifecycle + +A snapshot session has durable `pending`, `ready`, `failed`, and `deleting` states. Before acquisition, the gateway persists the exact descriptor, internal immutable `repository_id`, `catalog_entry_id` and version, admitted signature-policy/evidence identity, authorization audit reference, initialization deadline, and one provisional reference record keyed by `(binding_id, snapshot_key)`. It never persists credentials, public registry coordinates, local cache paths, inode identities, live attachment flags, or process-local locks. + +Initialization is all-or-nothing: + +1. Validate the request and first-turn rule without network or workspace writes. +2. Resolve exactly one repository/catalog entry and authorize the full tuple against current policy. +3. Persist the `pending` binding and provisional reference. +4. Fetch only from the bound repository by digest; verify response size and digest before use. +5. Verify config, provenance, retention deadline, tree manifest, gzip layer descriptors, signatures when policy requires them, and every referenced size/digest/media type. Reject expired or insufficient remaining retention with `workspace_snapshot_retention_invalid`. +6. Reject unknown reserved-path schemas. During layer application and final scan, reject every entry, whiteout, hardlink, symlink alias, or type transition at `.harness` or below. +7. Apply OCI changesets in private staging with rooted no-follow operations and bounded bytes, entries, paths, metadata, processes, descendants, and time. +8. Recompute the canonical snapshot manifest and a private full-tree seal that includes `.git`; require the declared snapshot digest and persist the seal as cache evidence. +9. Publish or reuse one immutable unpacked cache generation keyed by descriptor plus materializer-schema revision. +10. Reauthorize current policy, then create an inode-independent private session tree by same-filesystem reflink clone when supported or full private copy otherwise. Hardlinks to cache content are forbidden. +11. Apply UHP inputs and deterministic runner instruction/control preparation. Compare the snapshot manifest to the resulting ready tree, durably capture `workspace-initialization-changes-.json` plus changed regular-file artifacts, and store the ready manifest as initial cursor. +12. Validate the digest-covered working directory, activate the reference, and atomically transition to `ready` with the snapshot/ready digests and initialization artifact; start finite session TTL and only then provider or harness work. -OCI acquisition uses the catalog-selected direct image manifest and the declared source manifest. It verifies descriptor media types, sizes, and digests; applies layers in order with standard whiteout and opaque-directory behavior; and extracts with rooted no-follow operations. Traversal, out-of-root links, devices, sockets, FIFOs, sparse-file tricks, undeclared or missing entries, unsupported types, and digest or type mismatches fail closed. OCI sources need not contain Git metadata. +Credentialed Git and OCI clients authorize scheme, host, port, and resolved IP before every connection; reject loopback, link-local, private, Unix-socket, rebinding, or other disallowed targets; bound and reauthorize every redirect; never forward authorization, cookies, or client certificates across origins; require TLS; and use scoped short-lived credentials. -The v1 maximum for one OCI entry is 64 tar/gzip/zstd layers, a 4 MiB image manifest, a 128 MiB source manifest, 8 GiB compressed layers, 32 GiB expanded source, 500,000 entries, 4 GiB per regular file, 4096 UTF-8 bytes and 128 components per path, and 1 MiB per PAX or extended header. Expanded bytes divided by `max(compressed bytes, 1)` MUST NOT exceed 100 per layer or entry. Time, bytes, inodes, processes, descendants, and concurrency are also bounded per entry and per request. Operators MAY lower but not raise these contractual maxima without a contract revision. +The initialization deadline is separate from session TTL. Deadline exhaustion is a retryable availability failure, not a request-size failure. Every pre-ready failure is terminal for that binding: it transitions to `failed`, releases the provisional reference idempotently, and leaves no runnable tree. `retryable: true` means a caller may create a new session with the same descriptor; an idempotent duplicate of the failed request returns the same failure. Restart reconciliation may resume a still-running `pending` attempt internally. + +Exact-key cache misses singleflight. Cache roots and generations are owned by a gateway/materializer identity no harness UID can assume, are non-writable and non-searchable from the runner namespace, and are cloned only through trusted directory file descriptors with no-follow operations under an eviction/clone lease. The gateway verifies the full-tree seal immediately before and after cloning; unexpected mutation quarantines the generation and fails closed. + +Staging and quarantine have global byte, inode, and age bounds. Complete unreferenced cache generations are evictable under high and low watermarks. Durable reference records, not counters, make claim/release and crash reconciliation idempotent. Cache attachment eligibility is reauthorized on every use. + +V1 intentionally materializes a normal private directory. Reflink is preferred and full copy is required as the correctness fallback. OverlayFS and base-plus-delta checkpoints are deferred until measurements justify the larger mount-aware lifecycle change. + +## Resource and extraction limits + +The v1 format maximum is 64 layers, a 4 MiB OCI manifest, a 4 MiB config, a 128 MiB workspace manifest, 8 GiB total compressed layers, 64 GiB expanded workspace bytes, 1,000,000 visible entries, 4 GiB per regular file, 4096 UTF-8 bytes and 128 components per path, and 1 MiB per PAX or extended header. Expanded bytes divided by `max(compressed bytes, 1)` MUST NOT exceed 100 for each layer and for the artifact as a whole. Operators MAY lower but not raise these maxima without a contract revision. + +Extraction rejects absolute or traversing paths, ambiguous separators, NULs, escaping links, devices, sockets, FIFOs, unsupported sparse files, unbounded metadata, unknown compression, duplicate/type conflicts, undeclared final content, and digest mismatches. Runtime byte and inode quotas apply to staging, cache, private workspaces, outputs, and checkpoints. Materialized roots run `nodev` and `nosuid` where the deployment filesystem supports mount flags; normalized content contains no device nodes, set-ID bits, file capabilities, or executable metadata outside the manifest contract. ## Produced files and evaluation changes -The inherited UHP Files/artifact surface remains the only public file surface. Workspace-backed collection replaces only the stock root-Git listing/cursor implementation behind the existing seams: +Snapshot-backed sessions replace only the stock root-Git journal behind the existing produced-list, capture, and acknowledge seams. Requests without the snapshot extension retain stock root-Git behavior. -1. `_produced_list` compares the last acknowledged canonical manifest cursor for the outer workspace and every editable root with their final manifests. Read-only roots are not walked; their unchanged state is trusted only from immutable component identity plus freshly verified mount protection. -2. `_collect_produced` captures every added or modified regular file through the existing gateway file-artifact path. -3. `_collect_produced` then captures one server-generated `workspace-changes-.json` artifact with media type `application/vnd.allagents.workspace-changes.v1+json`. -4. Only after all file artifacts and the change artifact are durable does `_produced_ack` advance all cursor manifests. A retry before acknowledgement reproduces the same logical change set. +The gateway stores the canonical no-follow manifest cursor in protected blob storage as the sole durable journal authority. It excludes every `.git` and `.harness` tree. The runner receives the explicit acknowledged cursor, compares it with the final visible tree, and returns add, modify, and delete plus the next cursor/token. Content, type, executable mode, and symlink-target changes are modifications; rename is delete plus add. -The change artifact is RFC 8785 canonical JSON: +The gateway captures every added or modified regular file through the existing Files/artifact path and creates exactly one RFC 8785 canonical artifact named `workspace-changes-.json` with media type `application/vnd.allagents.workspace-changes.v1+json`: ```json { @@ -172,85 +224,123 @@ The change artifact is RFC 8785 canonical JSON: { "path": "services/api/src/main.ts", "operation": "modify", - "before": {"type": "file", "sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "size": 100, "executable": false}, - "after": {"type": "file", "sha256": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "size": 120, "executable": false}, - "file_id": "file_…" + "before": {"type":"file","sha256":"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","size":100,"executable":false}, + "after": {"type":"file","sha256":"sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb","size":120,"executable":false}, + "file_id": "file_..." } ] } ``` -Entries sort by the UTF-8 bytes of NFC-normalized workspace-relative `path`. `operation` is `add`, `modify`, or `delete`. `add` has `after`; `delete` has `before`; `modify` has both. State objects use `{type:"directory"}`, `{type:"file",sha256,size,executable}`, or `{type:"symlink",target}`. `file_id` is required exactly when `after.type` is `file` and refers to the captured regular-file artifact. Mode or symlink-target changes are `modify`; a rename is `delete` plus `add`. An empty collection still emits the change artifact with an empty `entries` array. +Entries sort by normalized path bytes. `add` has only `after`; `delete` has only `before`; `modify` has both. `file_id` exists exactly when `after.type` is `file` and refers to the already-durable file artifact. Empty turns emit an empty change artifact. + +Collection, checkpoint, cursor advancement, and terminal response finalization form one recoverable response transaction under an exclusive workspace mutation lease. The lease blocks Files writes, new turns, cleanup, and every other session writer after all descendants stop: -Runner-owned state, credentials, caches, control metadata, checkpoint metadata, and every `.git` tree are excluded from reporting. Promptfoo consumes change artifacts in response order and applies them to its previous reconstructed state. It does not inspect gateway Git state or require a new endpoint. +1. send the gateway-owned acknowledged cursor blob to the runner and compute the next manifest; +2. create deterministic artifact identities keyed by `response_id`; +3. open changed regular files with rooted no-follow operations, hash while streaming, and require type, size, and digest to equal each `after` state; +4. durably store regular-file artifacts and the change artifact; +5. create and durably store the exact snapshot-mode checkpoint, then verify its visible manifest equals the next cursor; +6. durably store the next cursor manifest; +7. call idempotent `/produced/ack` with base digest, next digest, and collection token; and +8. atomically compare-and-set the gateway's cursor/checkpoint/artifact pointers and terminal response to committed. -## Checkpoints, continuation, expiry, and deletion +The gateway cursor blob is the sole durable journal authority. Runner ACK is a recoverable confirmation barrier, not independent cursor state; every retry supplies the base cursor explicitly. A crash before step 8 leaves the previous gateway cursor authoritative even if ACK ran, so retry reproduces the same logical transaction. Terminal response visibility occurs only after ACK and the final compare-and-set. -Workspace-backed checkpointing keeps the inherited archive/hydration lifecycle but is mount-aware. Archive, Files, cleanup, and manifest operations MUST NOT follow or cross read-only mounts. Checkpoints preserve the writable outer workspace and editable-root bytes, plus durable references to the ordered binding and baselines; they never archive immutable cache roots or acquisition state. +A retry reuses the same immutable artifacts and never reruns the harness. The exclusive lease plus streamed hash/checkpoint verification prevents a file artifact, change manifest, and checkpoint from describing different trees. -Continuation restores the inherited checkpoint archive, recovers the exact stored source plan without resolving refs or pulling OCI again, verifies durable component and baseline references, revalidates destinations and live attachment protection, attaches read-only roots or restores editable roots, validates the working directory, and only then starts the stored harness. Expired, missing, or corrupt bound state fails closed rather than substituting a newer source or silently starting fresh. +Promptfoo reconstructs the final visible tree by starting from the builder's exact snapshot, applying the durable initialization-change artifact, then applying ordered per-response change artifacts. The ready manifest digest proves that first transition. Change artifacts are deterministic deltas; they do not pretend to contain baseline bytes or `.git` history. -Live-workspace cache reaping and durable session deletion remain distinct operations. The gateway rejects continuation at or after `expires_at` and performs deletion through the inherited explicit durable-session deletion path. Attachments are removed before hydration, deletion, workspace cleanup, or cleanup retry. Uncertain or busy state remains accounted for until cleanup succeeds. +## Checkpoint, continuation, expiry, and deletion + +Snapshot-backed V1 checkpoints the complete normal private workspace, including self-contained `.git` histories. It excludes only `.harness/tmp/**`, `.harness/home/.codex/auth.json`, `.harness/home/.omp/agent/auth.json`, `.harness/home/.omp/agent/models.json`, and `.harness/home/.omp/agent/models.yml`; every other `.harness` resume path remains. Stock broad dependency exclusions MUST NOT remove declared snapshot/session content. Any exclusion change requires a checkpoint-schema revision and compatible-reader gate. + +A continuation restores the exact durable checkpoint and cursor, verifies the stored binding, working directory, quotas, and ownership, and only then starts the stored harness. It does not accept another descriptor or resolve mutable sources. Missing or corrupt binding, checkpoint, cursor, or snapshot identity fails closed rather than starting empty. An evicted unpacked cache MAY be repopulated only from the same authorized direct manifest digest; the session's checkpoint remains the source of mutable state. + +The finite session TTL starts only after `ready`. Polling, replay, turns, and continuation do not extend `expires_at`. Expiry and explicit deletion make the session unavailable first, stop descendants, remove the private workspace, delete durable checkpoint/cursor state through existing records, and release each durable snapshot reference exactly once. Busy or uncertain state remains unavailable, accounted, and queued for idempotent reconciliation. ## Failure contract -Workspace failures use these stable detail codes in the existing UHP error -shape. `retryable` states whether the same operation may succeed without -changing the request. Pre-`ready` failures omit workspace metadata; post-`ready` -workspace failures return the stored sanitized metadata except inherited -`session_expired`, which retains the pinned UHP response unchanged. +Workspace failures use stable detail codes in the existing UHP error shape. For pre-`ready` initialization failures, `retryable` means a new request/binding with the same descriptor may succeed; it never revives a failed binding. For post-`ready` collection/checkpoint failures, retry resumes the same `WorkspaceTurnCommit` and MUST NOT rerun the harness. Pre-`ready` failures omit snapshot metadata. Post-`ready` workspace failures return stored sanitized metadata except inherited `session_expired`. -| Detail code | HTTP | Retryable | Timing and condition | +| Detail code | HTTP | Retryable | Condition | |---|---:|:---:|---| -| `workspace_invalid_request` | 400 | no | Pre-`ready`: malformed/unknown fields, invalid paths or refs, continuation metadata, or invalid working-directory syntax. | -| `workspace_path_collision` | 409 | no | Pre-`ready`, before source access: overlapping/reserved destinations or input/asset collisions. | -| `workspace_source_unknown` | 404 | no | Pre-`ready`: unknown catalog entry or missing, ambiguous, or unsupported Git ref identity. | -| `workspace_source_invalid` | 422 | no | Pre-`ready`: digest, manifest, layer, path, link, type, checkout, or extracted-content verification failure. | -| `workspace_acquisition_unavailable` | 503 | yes | Pre-`ready`: bounded transient Git, registry, DNS, transport, or upstream service failure. | -| `workspace_contract_limit_exceeded` | 413 | no | Pre-`ready`: request, component, expansion, file, path, process, time, or aggregate contractual maximum exceeded. | -| `workspace_capacity_exceeded` | 503 | yes | Pre-`ready`: operator storage, inode, mount, worker, or concurrency capacity unavailable. | -| `workspace_attachment_failed` | 500 | yes | Initial attachment before `ready` or live reattachment after `ready`: bind/remount, copy/reflink, destination, protection, baseline, or working-directory validation failure. | -| `session_expired` | 404 | no | Inherited UHP response when continuation targets a session at or after `expires_at`; omit workspace metadata and do not restore or extend expiry. | -| `workspace_restore_invalid` | 500 | no | Post-`ready`: binding, component, checkpoint, baseline, or cursor evidence is missing, corrupt, or inconsistent. | -| `workspace_collection_failed` | 500 | yes | Post-`ready`: manifest comparison, file/change-artifact capture, or cursor acknowledgement cannot complete. The cursor is not advanced. | -| `workspace_checkpoint_failed` | 500 | yes | Post-`ready`: mount-aware archive creation or durable checkpoint persistence fails after collection. | - -Cancellation is not a workspace error. The inherited cancel endpoints remain -idempotent and successful, and a cancelled task terminates with -`status: "cancelled"`, never failed. A cancellation before `ready` omits -workspace metadata; one after `ready` retains the stored sanitized metadata. - -Provider and harness failures that are not workspace failures retain their exact inherited UHP codes. Implementations MUST NOT reuse an inherited code for a workspace condition unless status, retryability, and semantics are identical. No failure may change source kind, source identity, access, working directory, harness, or provider route as a recovery shortcut. +| `workspace_snapshot_invalid_request` | 400 | no | Malformed/unknown fields, unsupported version/media type, bad digest/size, continuation injection, or other request-decidable violation. | +| `workspace_snapshot_unknown` | 404 | no | Exact descriptor is absent from the caller's authorized catalog view; hides whether it exists for another principal. | +| `workspace_snapshot_invalid` | 422 | no | Manifest, config, provenance, layer, path, link, type, working directory, final-tree, or signature verification failure. | +| `workspace_snapshot_retention_invalid` | 422 | no | Digest-covered/catalog retention is expired or cannot cover initialization deadline plus maximum session TTL plus safety margin; caller must publish a new snapshot. | +| `workspace_snapshot_unavailable` | 503 | yes | Bounded transient registry, DNS, transport, initialization-deadline failure, or an authorized manifest missing before its promised `available_until`. | +| `workspace_contract_limit_exceeded` | 413 | no | Fixed format count, byte, path, ratio, metadata, or file maximum exceeded. Timeouts do not use this code. | +| `workspace_capacity_exceeded` | 503 | yes | Operator disk, inode, worker, quota, or concurrency capacity unavailable. | +| `workspace_initialization_failed` | 500 | yes | Reflink/copy, staging publication, private-tree, baseline, or ready transition fails after valid artifact admission. | +| `session_expired` | 404 | no | Inherited UHP response when continuation targets a session at or after `expires_at`. | +| `workspace_restore_invalid` | 500 | no | Durable binding, checkpoint, cursor, or identity is missing, corrupt, or inconsistent. | +| `workspace_collection_failed` | 500 | yes | Manifest comparison or artifact persistence cannot complete; cursor and checkpoint do not advance. | +| `workspace_checkpoint_failed` | 500 | yes | Exact private-workspace checkpoint persistence fails; terminal response is not finalized. | + +Cancellation is not a workspace error. Inherited idempotent cancellation and terminal `status: "cancelled"` remain. Cancellation before `ready` omits snapshot metadata; after `ready` it retains stored sanitized metadata. + +## Upstream boundary + +The upstream proposal is tracked in [HarnessRouter issue #304](https://github.com/HarnessRouter/harnessrouter/issues/304). It is an internal implementation seam, not a UHP Enhancement Proposal: + +1. an optional operator-configured `WorkspaceInitializer` invoked once after an empty private session root exists and before input or harness start; +2. a pluggable `WorkspaceJournal` behind `/produced` and `/produced/ack`, with the gateway cursor as durable authority and current root Git as the default; and +3. a recoverable terminal-finalization seam that orders artifacts, journal ACK, checkpoint/cursor publication, and terminal response visibility. + +The initializer receives one opaque immutable descriptor, verifies and materializes it atomically, and returns verified identity, relative working directory, initializer schema, and journal mode. It returns no source plan, credential, registry URL, host path, cache path, or mount identity. Requests without an initializer extension remain on the stock path. + +Focused upstream tests must prove initialization-before-input, idempotent retry, descriptor replacement rejection, explicit cursor handoff, exact add/modify/delete journal behavior, capture-before-ACK, checkpoint/cursor durability before terminal visibility, continuation, fail-closed restore, and unchanged stock behavior. + +Until upstream ships equivalent initializer, journal, and finalization seams, `allagents-gateway` carries the smallest downstream patch and maps every remaining delta explicitly. When upstream ships all required seams, forked implementations are deleted. The gateway then becomes a thin stock-derived distribution or, if external backend loading is supported, no source fork at all. `allagents-workspace-builder` remains unchanged. ## Distribution and release boundary -On 2026-09-27, `allagentsdev/harnessrouter` was renamed in place to `allagentsdev/allagents-gateway`. GitHub preserves the repository identity, fork-network relationship, history, settings, and redirects, and still reports `HarnessRouter/harnessrouter` as the parent. The Apache-2.0 `LICENSE`/`NOTICE` and the pinned upstream commit remain the characterized baseline. Writable branch `feat/workspace-composition` exists at downstream `main` commit `fbcb73132423c8c4575113fc8943c6a6280a4746`, which contains that baseline plus the existing downstream commits. A second repository MUST NOT be created. +`allagentsdev/allagents-gateway` preserves the HarnessRouter fork network, full Git history, Apache-2.0 `LICENSE`, `NOTICE`, attribution, and upstream remote. The existing `feat/workspace-composition` branch MUST be replaced before implementation by a snapshot-specific branch based on current downstream `main`; obsolete runtime-composition code or aliases are not retained. + +`allagentsdev/allagents-workspace-builder` is a separate AllAgents-owned repository with its own release cadence, threat model, credentials, and artifact-format compatibility tests. The cross-repository compatibility contract is the versioned OCI snapshot artifact, not source-level imports or a private RPC schema. -Implementation also requires checked-in local builders for an authenticated OCI registry, catalog, and source fixtures, plus a Linux environment capable of namespace-confined bind mounts and reflink-or-copy isolation. The implementation agent can deliver a pull request using those fixtures and mocked provider transport without production secrets. Provider gateway URL/key, UHP caller credentials, GHCR rights, protected settings, and final publication/deployment are operator-owned inputs supplied through the repository's secret mechanism. +Before release, the target production filesystem is probed for same-filesystem reflink behavior and hard byte/inode quotas. Full-copy fallback is exercised even where reflink succeeds. The candidate supports only architectures explicitly built, preflighted, and tested; V1 MAY declare `linux/amd64` only. -Workspace code only gates inherited harness start until the binding is `ready` and the working directory is valid; it does not redesign Codex, OMP, or provider authentication. Release validation runs once against the published image read back and pinned by digest. Completion requires stock UHP compatibility, Git/OCI/mixed workspace flows, read-only and editable isolation, canonical change artifacts, continuation, cancellation, expiry/deletion, cache reuse, fresh-volume and same-volume restart, both supported harnesses, UHP conformance, security scans, SBOM, and build provenance. +Capability `allagents_workspace_snapshot_v1` remains disabled until every request-serving gateway and runner understands artifact, binding, journal, checkpoint, and finalization schema V1. Snapshot requests and bound continuations carry an internal minimum-reader version and route only to compatible replicas; incompatible replicas reject before hydrate. Rollback retains compatible readers until all snapshot sessions are deleted, or first makes those sessions unavailable and drains them before old code serves traffic. + +Release proof runs against the exact published image digests and includes: + +- stock UHP compatibility and both supported harnesses; +- snapshot cache miss/hit, attempted cache-path access from a session UID, concurrent singleflight, private-write isolation, limits, and malicious OCI fixtures; +- exact add/modify/delete artifacts, streamed artifact/hash binding, checkpoint/continuation, cancellation, expiry, deletion, and crash reconciliation; +- fresh-volume and same-volume restart during pending initialization, collection, journal ACK, checkpoint, ready execution, and deletion; +- an N-1-to-candidate upgrade plus proven compatible routing and rollback-or-drain fencing; +- repeated race-sensitive restart, cancellation, reference-release, turn-deletion, and cleanup cases against the exact digest; +- SBOM, build provenance, secret scan, and correlation-safe telemetry that never emits credentials, private registry paths, or workspace contents; and +- an upstream-intake record linking the proposed hook issue, maintainer decision, downstream delta, and removal trigger. ## Rejected alternatives | Alternative | Why rejected | |---|---| -| Build a new execution service or a separate workspace service | Duplicates UHP/session behavior or splits ownership away from checkpoint, continuation, collection, and deletion. | -| Use one source object, one multi-root OCI bundle, or one request-wide cached tree | Prevents independent mixed-source identity and reuse and hides destination ownership. | -| Put a source at workspace root | Collides with runner-owned state, inputs, checkpoints, and the writable outer workspace. | -| Use symlinks for shared read-only roots or shared inodes for editable roots | Does not enforce isolation and can expose or mutate backing storage. | -| Use Git as the evaluation diff engine | Cannot represent Git-free OCI roots, multiple roots, outer files, or immutable final-tree semantics. | -| Add a second changes endpoint | Duplicates the inherited Files/artifact surface and bypasses its capture-before-ack behavior. | -| Add a general materializer plugin framework | V1 has two explicit source kinds and no caller-selectable implementation need. | -| Wait for upstream acceptance | Delays downstream evidence and makes delivery depend on a project we do not control. | +| Resolve Git and compose OCI sources inside AllAgents Gateway | Mixes product source credentials and policy with execution, enlarges the permanent fork, and makes failures part of task startup. | +| Keep preparation and execution in one source repository | Couples release and trust boundaries and makes returning to stock a source-tree surgery. | +| Upload a tar through stock `/hydrate` | Internal checkpoint route with no public OCI identity, authorization, provenance, cache, or exact multi-repository journal. | +| Make the agent clone repositories | Runs acquisition after harness start, exposes credentials/network policy to agent code, and cannot establish a trusted pre-turn baseline. | +| Use stock root Git as evaluator | Mutates a root repository, reports embedded repositories coarsely, and omits deletions. | +| Use read-only shared trees, hardlinks, or symlinks | Harnesses need a writable workspace; these mechanisms risk cache or sibling mutation. | +| Adopt OverlayFS and delta checkpoints in V1 | Expands hydrate, checkpoint, archive, Files, deletion, mount, and crash recovery before measured need. | +| Put provenance only in OCI referrers | Referrer associations are weak and do not contribute to admitted manifest identity. | +| Accept tags or indexes at execution | Selection can change independently of the request; execution requires one direct manifest. | +| Add a second public changes endpoint | Duplicates UHP Files/artifacts and bypasses capture-before-ack. | +| Upstream the AllAgents source descriptor | Git/OCI composition is product policy, not HarnessRouter execution infrastructure. | ## Consequences -AllAgents Gateway remains one execution, workspace, and session control plane. Independent component caching permits reuse without a global composition object. Visibility gating and mount-aware lifecycle work add implementation complexity but prevent partial workspaces, mutable shared state, and checkpoint traversal into caches. +The gateway becomes materially smaller: no Git client, source credential broker, per-component destination planner, or runtime composition state. Task startup sees one authorized immutable artifact and one private filesystem. + +The builder takes on explicit distributed-system obligations: asynchronous build status when needed, complete-before-return publication, artifact retention, provenance, garbage collection, and a compatibility matrix with gateway snapshot versions. -Canonical manifests and change artifacts add bounded filesystem scanning and hashing, but give Git and OCI one evaluator-visible definition of state. Promptfoo can reconstruct results entirely from ordered UHP artifacts, regardless of Git metadata or index state. +V1 pays the I/O/storage cost of a private reflink/copy and full exact checkpoint. That cost is deliberate. It preserves ordinary-directory semantics and minimizes the downstream/upstream patch. Metrics determine whether a later ADR adopts OverlayFS or base-plus-delta checkpoints. -Mandatory OCI support and Linux mount/copy capabilities make v1 substantial, but they satisfy the large-source requirement while preserving full Git ancestry by default, explicit shallow acquisition when requested, and Git-free OCI operation. +Promptfoo must retain or fetch the admitted snapshot baseline, then apply the initialization delta and ordered response deltas to reconstruct complete final state. These UHP artifacts do not replace baseline storage. ## Reconsider when -Revisit this decision if upstream adopts an equivalent ordered multi-source contract, its lifecycle removes these extension seams, the deployment platform cannot enforce the required isolation, bounded OCI materialization or manifest collection cannot meet release limits, continuation cannot fail closed without reacquisition, or another UHP implementation offers a materially smaller and equally stable integration surface. +Revisit this decision if measurements show private clone or full checkpoint costs violate release SLOs; the production filesystem cannot provide correct reflink or bounded full-copy behavior; upstream rejects the required initialization, journal, and finalization seams and fork cost exceeds a standalone executor; the snapshot format cannot preserve required Git workflows safely; or a later UHP version standardizes an equivalent immutable workspace contract. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index f64f1f83..a04aa32c 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,5 +1,5 @@ --- -title: "AllAgents Gateway Workspace Composition - Implementation Plan" +title: "Prepared Workspace Snapshots - Cross-Repository Implementation Plan" date: 2026-09-18 updated: 2026-09-28 type: feat @@ -8,803 +8,681 @@ artifact_readiness: implementation-ready execution: code --- -# AllAgents Gateway Workspace Composition - Implementation Plan +# Prepared Workspace Snapshots - Cross-Repository Implementation Plan ## Goal -Create AllAgents Gateway as a downstream distribution of -`HarnessRouter/harnessrouter`. On the first request of a new UHP session, a -caller MAY declare an ordered set of independent Git and OCI source trees. The -gateway acquires and caches each component independently and places each tree at -its requested non-root destination in the existing private session workspace. -The application exposes the workspace to Files, checkpoint, provider, or harness -consumers only after every root verifies. - -Requests without `metadata.workspace` remain on the pinned stock path. Workspace -support introduces no second session database, Files API, scheduler, execution -protocol, global composition cache, or materializer service. - -## Implementation-handoff prerequisites - -### Target repository: operator setup complete - -On 2026-09-27, an organization repository administrator completed the -repository setup: - -1. Renamed the existing `allagentsdev/harnessrouter` GitHub fork in place to - `allagentsdev/allagents-gateway`, preserving repository identity, fork - network, complete history, settings, redirects, Apache-2.0 `LICENSE`, - `NOTICE`, copyright, and attribution. -2. Verified that GitHub still identifies `HarnessRouter/harnessrouter` as the - parent. Upstream commit `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` - remains the characterized baseline; upstream acceptance is not a dependency. -3. Created writable branch `feat/workspace-composition` from downstream `main` - commit `fbcb73132423c8c4575113fc8943c6a6280a4746`. That commit is the pinned - upstream baseline plus the existing three downstream commits. -4. Updated the repository description, homepage, topics, and merge policy for - the AllAgents Gateway identity. - -The implementation clone MUST use the renamed repository as `origin` and retain -`HarnessRouter/harnessrouter` as `upstream`. It MUST start from the existing -`feat/workspace-composition` branch rather than discard downstream work by -branching from the older upstream commit. Do not create a second repository. - -Keep the downstream distribution names fixed: - -- repository: `allagentsdev/allagents-gateway`; -- image: `ghcr.io/allagentsdev/allagents-gateway`; -- service: `allagents-gateway`; and -- product: **AllAgents Gateway**. - -The implementation environment also requires these checked-in local test -capabilities, with no external credentials: - -- `oci-auth-registry`: an authenticated local OCI registry fixture; -- `oci-catalog-builder`: a builder for the server-owned catalog fixture; and -- `oci-source-fixture-builder`: a builder for deterministic image manifests, - source manifests, layers, whiteouts, and malicious extraction fixtures. - -The Linux integration runner MUST support read-only bind mounts and either safe -reflinks or inode-independent private copies. If reflinks are unavailable, the -implementation MUST use the private-copy path; it MUST NOT hard-link mutable -files. These are implementation prerequisites, not caller-visible options. - -### Secrets and actions outside the implementation agent - -The implementation agent can deliver the complete implementation PR using local -fixtures and mocked provider transport. The following remain operator-owned: - -- the external provider gateway URL and key; -- a real UHP caller credential; -- GHCR publish rights; -- protected repository, registry, network, and deployment settings; and -- final image publication and deployment. - -Release gates that require these values run only after an operator supplies them -through the repository's existing secret mechanism. The plan MUST NOT put -secrets in source, fixtures, build arguments, logs, artifacts, session metadata, -or harness environments. - -## Pinned stock behavior and exact adaptation seams - -The baseline owns UHP request/stream handling, authentication, idempotency, -provider brokering, harness supervision, one private workspace per session, -inputs and generated assets, checkpoint/artifact/control records, cancellation, -and deletion. - -The implementation MUST begin by confirming these pinned behaviors at the -named seams, then adapt them rather than adding parallel machinery: - -- Fresh stock hydration initializes Git at the workspace root. -- Git powers only the stock produced-file listing and cursor. It is not the - workspace storage or restore mechanism. -- Stock checkpointing commits and then tars the directory; hydration restores - the tar. -- Runner `_produced_list` returns produced files and `_produced_ack` advances the - produced cursor. -- Gateway `_collect_produced` captures artifacts before calling the ACK path. -- `BACKING.workspace` exposes live files through `RunnerWorkspaceFiles` and - checkpoint files through `CheckpointWorkspaceFiles`. -- The durable graph uses the `HarnessSession` vertex plus separate checkpoint, - artifact, and control records. Workspace state extends those seams; it does - not collapse them into a new session store. -- Stock reaps cached live workspaces and supports explicit deletion of durable - sessions. It does not provide a durable-session TTL. - -For workspace-backed sessions, `_produced_list` and `_produced_ack` remain the -runner protocol, `_collect_produced` remains the capture-before-ACK gateway -boundary, and `BACKING.workspace` remains the Files abstraction. Only the -workspace-backed produced-file implementation changes from root-Git cursoring -to canonical manifest cursoring. Stock requests retain root-Git listing and all -other pinned behavior. - -Provider routing remains inherited. Workspace code only delays harness start -until the workspace binding is ready and the working directory has verified. -There is no provider-specific workspace implementation phase. - -## Scope and invariants - -### In scope - -- A strict, closed, first-turn-only `metadata.workspace` request extension. -- An ordered array of 1 to 128 Git, OCI, or mixed source entries. -- Exactly one tree and one required non-root destination per source. -- Canonical public HTTPS Git acquisition at an exact resolved commit with full - reachable ancestry by default, optional explicit shallow depth, and safe - self-contained `.git` metadata for regression analysis. -- Operator-cataloged OCI source trees selected by exact image-manifest and - source-manifest digests, including Git-free trees. -- Per-component cache/singleflight followed by one session-local resolved plan. -- Identical `read_only` and `editable` attachment semantics for Git and OCI. -- Canonical manifests, cursor-based change projection, mount-aware checkpoint - and continuation, finite expiry, and explicit deletion. -- Existing Codex and OMP paths through the existing provider broker. - -### Non-goals - -- Caller-selected runtime images, registry origins, credentials, mirrors, - transport settings, commands, materializer hooks, provider routes, models, or - resource limits. -- A public component-cache or composition API, a global composition record, - cache, identifier, or compatibility aliases for obsolete workspace schemas. -- Git-to-OCI, OCI-to-Git, ref, digest, registry, provider, or history-mode - fallback. A failed full fetch never silently becomes shallow, and a failed - shallow fetch never deepens. -- Git submodule initialization, LFS hydration, checkout filters, or hook - execution. -- Requiring Git at workspace root or in OCI trees. -- A second Files endpoint or an AllAgents CLI/profile format change. - -## Public request and response contract - -`metadata.workspace` is accepted only on the first request that creates a new -session. All objects are closed, use snake_case, and have no nested version. +Deliver reproducible large coding workspaces through two narrow components: + +1. **AllAgents Workspace Builder** prepares Git/OCI inputs, preserves required Git histories, publishes one immutable OCI workspace snapshot, and returns its direct manifest descriptor. +2. **AllAgents Gateway** consumes only that exact descriptor, authorizes and materializes a private writable session directory, runs the existing HarnessRouter/UHP lifecycle, and returns exact filesystem deltas. + +The execution gateway MUST NOT resolve Git refs, receive source credentials, compose repository arrays, or call the builder during task execution. The cross-repository contract is the versioned OCI artifact. A new snapshot session is runnable only after descriptor authorization, artifact verification, private materialization, working-directory validation, and baseline-journal creation complete. + +Requests without the vendor snapshot extension remain on characterized stock HarnessRouter behavior. + +## Architecture decision + +The authoritative decision is [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md). The evidence is [Prebuilt immutable workspace snapshots at the HarnessRouter boundary](../research/allagents-gateway-snapshot-boundary.md). + +Two repositories are intentional: + +| Repository | Owns | Must not own | +|---|---|---| +| `allagentsdev/allagents-workspace-builder` | Git/OCI acquisition, credentials, source policy, composition, provenance, canonical baseline, OCI publication, build retention | UHP, harness/provider routing, session continuation, produced-file ACK, response lifecycle | +| `allagentsdev/allagents-gateway` | UHP, descriptor admission, authorization, immutable cache, private session tree, manifest journal, Files/artifacts, checkpoint, continuation, cancellation, deletion | Git refs, source credentials, caller-selected registries, repository arrays, composition jobs | + +This is not a mandate for two always-on services. V1 builder delivery is a CLI/library suitable for CI or a job worker. A service wrapper is optional and must preserve the same artifact contract. + +The upstream change is tracked in [HarnessRouter issue #304](https://github.com/HarnessRouter/harnessrouter/issues/304). The proposal covers an immutable workspace initializer, an explicit-cursor non-Git journal, and recoverable terminal finalization; it does not upstream AllAgents source semantics or require a UHP change. + +## Repository and branch prerequisites + +### AllAgents Gateway + +`allagentsdev/allagents-gateway` already exists as the renamed HarnessRouter fork. It preserves the fork relationship, full history, Apache-2.0 `LICENSE`, `NOTICE`, attribution, settings, and redirects. `HarnessRouter/harnessrouter` remains the upstream parent. + +Before implementation: + +1. Fetch current downstream `main` and upstream. +2. Record the accepted characterized baseline `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` and current inspected point `8f7868ccb2c97d1f611acf11e7cad0357a43064e`. +3. Create `feat/workspace-snapshots` from current downstream `main`. +4. Retire the unused `feat/workspace-composition` branch after confirming it contains no unique implementation. Do not carry runtime-composition scaffolding, aliases, or dead request fields. +5. Keep `origin` on `allagentsdev/allagents-gateway` and `upstream` on `HarnessRouter/harnessrouter`. + +### AllAgents Workspace Builder + +Create public repository `allagentsdev/allagents-workspace-builder` with Apache-2.0 licensing, protected `main`, required checks, immutable release tags, GHCR package access, and a security policy. Start implementation on `feat/workspace-snapshot-v1`. + +The repository MUST produce one pinned CLI binary and one reusable package from the same code. Prefer a small static implementation suitable for untrusted archive handling; pin the language toolchain and every dependency in the initial bootstrap. The CLI surface is: + +```text +allagents-workspace-builder build \ + --context /run/secrets/build-context.v1.json \ + --spec workspace-build.v1.json \ + --output descriptor.json +``` + +A later queue/API wrapper invokes the same package. It does not define a second artifact format or source-resolution path. + +### Shared fixtures, not shared source + +Each repository owns local focused fixtures. Cross-repository E2E uses published fixture snapshots pinned by digest. Do not share code through Git submodules, relative checkouts, unpublished packages, or branch references. If schema generation is added, publish a versioned schema artifact and check generated outputs in each consumer. + +Required fixture classes: + +- one root Git repository with complete merge history; +- two independent nested Git repositories; +- one Git-free OCI tree; +- composition with whiteouts, empty directories, symlinks, executable files, and safe hardlinks; +- malformed manifests, configs, layers, links, paths, types, sparse files, compression bombs, metadata bombs, and digest mismatches; +- source trees colliding with versioned runner-reserved paths; and +- a snapshot large enough to exercise cache watermarks, quota rejection, reflink, and full-copy fallback. + +## Secrets and operator-owned inputs + +No implementation or fixture requires production credentials. Local authenticated registries, local Git servers, and mocked provider transport prove behavior. + +Operator-owned values remain outside source and tests: + +- source and snapshot registry credentials; +- Git credentials and network allowlists; +- snapshot authorization-catalog entries; +- UHP caller credentials; +- provider gateway URL/key; +- GHCR publish rights; and +- production deployment, object-store, quota, and telemetry configuration. + +Secrets MUST enter through each repository's secret mechanism. They MUST NOT appear in build specs, OCI config/provenance, layers, logs, errors, response metadata, checkpoints, SBOMs, build arguments, or harness environments. + +All credentialed Git and OCI fetchers share one minimum transport policy: authorize scheme, host, port, and every resolved IP before connection; reject loopback, link-local, private, Unix-socket, rebinding, and other non-allowlisted targets; cap and reauthorize redirects; never forward authorization, cookies, or client certificates across origins; require TLS; use repository-scoped short-lived credentials; and redact redirect/auth diagnostics. + +## Characterized HarnessRouter seams + +Implementation starts from these existing seams rather than a parallel executor: + +- gateway request/session resolution and `CreateResponseBody.metadata` extension handling; +- gateway `_resp_execute` and `_hydrate`, where initialization must gate input and harness start; +- runner `_ws`, `/hydrate`, `/checkpoint`, and `/turn`; +- runner `_produced_list`, `_produced_ack`, and the internal `/produced` routes; +- gateway `_collect_produced`, which captures artifacts before ACK; +- `BACKING.workspace`, `RunnerWorkspaceFiles`, and `CheckpointWorkspaceFiles`; +- durable `HarnessSession` plus separate checkpoint, artifact, response, and control records; +- existing cancellation, explicit session deletion, live-workspace reaping, graph/blob backing, provider routing, SSE translation, and harness adapters. + +The red characterization MUST prove current limitations before code changes: + +1. a root repository is mutated by `_git_ensure`; +2. an embedded repository's internal file changes are not emitted exactly by root Git; +3. a deletion is absent from stock produced-file artifacts; +4. `/hydrate` accepts only the internal checkpoint shape and is not a supported OCI import; and +5. a request without the vendor extension retains current UHP behavior. + +## Workspace Builder contract + +### Build request v1 + +The builder accepts a closed versioned build spec plus a trusted out-of-band `BuildContext`. This is an AllAgents preparation contract, not UHP. `BuildContext` contains authenticated `principal`, `product_domain`, and operator-selected `policy_id`; it is supplied by the embedding job/CLI environment, never the build spec. V1 CLI execution is operator-trusted. A later service wrapper MUST authenticate its caller and map it to the same `BuildContext` before invoking the package. ```json { - "metadata": { - "harness_id": "allagents-codex", - "workspace": { - "access": "editable", - "sources": [ - { - "kind": "git", - "url": "https://github.com/example/service.git", - "ref": "refs/heads/main", - "destination": "service" - }, - { - "kind": "oci", - "snapshot_name": "shared-release", - "image_manifest_digest": "sha256:...", - "source_manifest_digest": "sha256:...", - "destination": "libraries/shared" - } - ], - "working_directory": "service" + "version": 1, + "working_directory": "services/api", + "sources": [ + { + "kind": "git", + "url": "https://github.com/example/api.git", + "ref": "refs/heads/main", + "history": {"mode": "full"}, + "destination": "services/api" + }, + { + "kind": "oci", + "source_name": "compiler-tree", + "descriptor": { + "media_type": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111", + "size": 123456 + }, + "destination": "vendor/compiler" } - } + ] } ``` +Rules: + +- `version` is exactly `1`; all objects are closed. +- `sources` contains 1 through 128 ordered entries. +- `destination` is NFC-normalized relative POSIX. `.` is allowed only when it is the sole source. Otherwise destinations are non-root and pairwise non-overlapping. +- `working_directory` is relative to the final snapshot root, defaults to `.`, and must resolve without symlink escape to a real directory. +- Git URLs are canonical HTTPS identities. Userinfo, query, fragment, local paths, alternate transports, and caller Git options are rejected. +- Omitted Git `ref` selects the advertised default. A supplied ref resolves only through advertised full-ref or unambiguous branch/tag semantics. +- Omitted `history` means `{"mode":"full"}`. Shallow mode is `{"mode":"shallow","depth":N}` with integer `N` from 1 through 1,000,000. Full means complete ancestry reachable from the selected commit, not unrelated refs/tags. +- `source_name` selects an operator configuration for registry origin, credentials, TLS, redirects, media types, and trust. Callers do not supply these. +- Unknown kinds, commands, environment, credentials, host paths, runtime images, provider routes, resource-limit overrides, and tags/indexes fail before network access. +- Source and destination authorization is evaluated against `BuildContext` before credentials or network are used. + +### Git preparation + +For each Git source: + +1. resolve allowed credentials and egress policy from `BuildContext`, outside the request; +2. run Git without a shell under sanitized environment/config; +3. disable interactive prompts, hooks, filters, credential persistence, submodule recursion, LFS hydration, alternate helpers, inherited proxies except explicit policy, and file/local transports; enforce the shared per-connection DNS/redirect/credential policy; +4. resolve the advertised selector to one exact commit; +5. fetch complete reachable ancestry or the exact requested shallow boundary under byte/object/time/process/output limits; +6. export a detached self-contained repository with no alternates, transient locks, credential-bearing remotes, or acquisition-only state; +7. reject gitlinks, unhydrated LFS pointers when policy requires real content, unsafe symlinks, and reserved-path collisions; +8. verify offline `log`, parent traversal, blame/diff across the selected history, and bisect prerequisites for full history; and +9. record requested selector, exact commit, history mode, normalized URL identity, destination, and verification policy in provenance. + +A source failure never chooses a different ref, deepens or shallows history, fetches an arbitrary object ID, or falls back to OCI. + +### OCI input preparation + +OCI source inputs are direct descriptors admitted by `source_name` and `BuildContext`. Apply the shared per-connection transport policy plus descriptor, distribution, and layer rules equivalent to the final snapshot. Tags, indexes, caller registries, ambiguous media types, traversal, links, devices, FIFOs, sockets, capabilities, ACLs, xattrs, set-ID bits, unsupported sparse files, and unbounded metadata fail closed. + +OCI inputs may be Git-free. The builder never invents Git metadata. + +### Composition and reserved paths + +Build in private staging. Validate all destination ownership and reserved-path conflicts before network access. Apply sources in request order only for deterministic construction; overlap is invalid, so order never grants overwrite precedence. + +Reserved-path schema 1 is the root `.harness` path and every descendant, with path comparison after UTF-8/NFC/POSIX normalization. Builder and gateway share golden accept/reject fixtures but independent implementations. Reject an entry, whiteout, hardlink, symlink path or target alias, or type transition that occupies `.harness` or a descendant. Unknown schema versions fail before extraction. Root instruction files such as `AGENTS.md` are not reserved: snapshot mode preserves their content and merges the runner-managed block before the initial cursor is established. + +After composition: + +- normalize ownership, timestamps where format policy requires them, modes, and metadata; +- walk without following links; +- validate the default working directory; +- produce the canonical visible-tree manifest excluding `.git` and `.harness`; +- produce digest-covered provenance; +- construct deterministic OCI layers/config/manifest; and +- publish blobs/config/layers before the direct image manifest. + +A failed or canceled publication never returns a result. Staging is cleaned or quarantined under bounded age/bytes/inodes. Config includes `available_until`, backed by an operator-enforced registry retention lease. Automatic garbage collection MUST NOT remove the manifest or referenced blobs before that timestamp. The lease duration covers expected scheduling delay plus gateway initialization deadline, maximum session TTL, and safety margin. + +## Snapshot artifact v1 + +The final artifact contract is fixed by ADR 0002: + +- OCI image manifest media type `application/vnd.oci.image.manifest.v1+json`; +- `artifactType` `application/vnd.allagents.workspace-snapshot.v1`; +- config media type `application/vnd.allagents.workspace-snapshot.config.v1+json`; +- provenance media type `application/vnd.allagents.workspace-provenance.v1+json`; +- canonical visible-tree media type `application/vnd.allagents.workspace-manifest.v1+json`; +- direct execution descriptor `{media_type,digest,size}`; and +- deterministic gzip layers only, media type `application/vnd.oci.image.layer.v1.tar+gzip`, applied in order to an empty root. V1 rejects uncompressed, zstd, and nondistributable layers. + +Config contains exactly: + ```text -metadata.workspace = { - access: "read_only" | "editable", - sources: Array< - | { - kind: "git", - url: string, - ref?: string, - depth?: integer, - destination: string - } - | { - kind: "oci", - snapshot_name: string, - image_manifest_digest: "sha256:<64 lowercase hex>", - source_manifest_digest: "sha256:<64 lowercase hex>", - destination: string - } - >, - working_directory?: string +{ + version: 1, + working_directory: string, + workspace_manifest: OCI descriptor, + provenance: OCI descriptor, + builder: {name: string, version: string, policy: string}, + reserved_paths_schema: 1, + available_until: RFC3339 timestamp } ``` -For a Git source, omitted `depth` means the complete ancestry reachable from the -resolved commit. A supplied `depth` is an integer from 1 through 1,000,000 and -requests exactly that shallow boundary. Full history does not fetch unrelated -refs or tags merely for completeness. +Required provenance is RFC 8785 canonical JSON with media type `application/vnd.allagents.workspace-provenance.v1+json`. It is a closed `{version:1,sources:[...]}` object. A Git record contains only `kind`, normalized `url`, optional `requested_ref`, exact `resolved_commit`, closed full-or-shallow `history`, and `destination`. An OCI record contains only `kind`, public logical `source_name`, exact direct `descriptor`, and `destination`. Source order is build order. Credentials, registry coordinates, headers, helpers, and host paths are forbidden. Builder and policy identity remain in config. -There is no request `retention` field and no `persistent` mode. The obsolete -singular `source`, `kind: "repositories"`, `repositories`, -`kind: "workspace_snapshot"`, and `workspace_manifest_digest` forms are -rejected, not aliased. +The canonical visible-tree entry forms, sorting, hashing, link confinement, mode normalization, `.git` exclusion, and runner-path exclusion match ADR 0002 exactly. Builder and gateway MUST use independent implementations against the same golden corpus; a shared implementation would hide interoperability bugs. -Every workspace-backed session receives the same operator-configured finite, -non-extendable expiry at creation. Replay, polling, turns, attachment, and -continuation MUST NOT move it. Explicit deletion remains available. +The builder outputs: -After the ready transition, responses return this exact sanitized shape: +```json +{ + "version": 1, + "descriptor": { + "media_type": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:...", + "size": 123456 + }, + "workspace_manifest_digest": "sha256:...", + "provenance_digest": "sha256:...", + "working_directory": "services/api", + "available_until": "2026-10-29T00:00:00Z" +} +``` + +This versioned builder result contains no credentials or private registry transport details. The caller supplies only nested `descriptor` to the gateway and retains the complete result as its baseline/provenance/retention record. + +## Gateway public contract + +### Request + +Only the first request creating a new session may contain: ```text -metadata.workspace = { - access: "read_only" | "editable", - working_directory: string, - expires_at: RFC3339 timestamp, - sources: Array< - | { - kind: "git", - url: string, - destination: string, - resolved_commit: string, - depth?: integer, - source_manifest_digest: "sha256:<64 lowercase hex>", - requested_ref?: string - } - | { - kind: "oci", - snapshot_name: string, - destination: string, - image_manifest_digest: "sha256:<64 lowercase hex>", - source_manifest_digest: "sha256:<64 lowercase hex>" - } - > +metadata.allagents_workspace_snapshot = { + version: 1, + descriptor: { + media_type: "application/vnd.oci.image.manifest.v1+json", + digest: "sha256:<64 lowercase hex>", + size: positive integer + } } ``` -Git response provenance mirrors the selected history: omitted `depth` means full -reachable ancestry, while a present value is the exact requested shallow depth. - -`working_directory` is always present and uses `.` for the outer root. There is -no public `retention`, `effective_descriptor_digest`, `composition_id`, or -per-source `component_id`. Catalog coordinates, private cache keys, mirrors, -host paths, leases, mounts, credentials, and policy identifiers are also -private. Replays and continuations return the stored ready metadata and never -re-resolve mutable Git refs. - -### Validation and ownership - -Validation MUST complete every request-decidable check before source network -traffic, cache lookup, component claim, workspace write, or expiry mutation: - -1. Require `access` and `sources`; accept 1 through 128 sources in request order. -2. Require one non-root relative POSIX `destination` per source. Reject empty, - `.`, `..`, non-NFC, ambiguous, platform-specific, overlong, or link-escaping - components. -3. Reject equal or ancestor/descendant destinations. Repeated component - identities at different non-overlapping destinations are valid. -4. Normalize UHP inputs, generated assets, runner-reserved paths, credential and - control paths, and every destination through one ownership validator. Reject - a file, symlink, input, asset, reserved path, or unsafe ancestor at or below a - destination in either access mode. Outer paths remain writable and valid. -5. Validate optional `working_directory` syntax before network work. After all - attachments, require it to resolve without link escape to one real directory. -6. Accept only canonical public HTTPS Git URLs allowed by deployment egress - policy. Reject userinfo, query, fragment, ambiguous encodings, alternate - transports, and caller Git options. A ref resolves only through advertised - default, branch, or tag semantics. Omitted `depth` means complete ancestry - reachable from the resolved commit; a supplied `depth` MUST be an integer - from 1 through 1,000,000. -7. For OCI require a catalog `snapshot_name` and direct SHA-256 image/source - manifest digests. Reject tags, indexes/lists, caller registry coordinates, - and mutable references. -8. Bound metadata bytes, nesting, strings, paths, source count, and input count - while parsing. Reject unknown and wrong-kind fields. -9. Reject `metadata.workspace` on every replay, reused session, and continuation - before hydrate, cache lookup, provider traffic, or any lifecycle mutation. - -### Resource ceilings - -Deployment policy MAY lower but MUST NOT raise these v1 ceilings without a -contract revision: - -| Resource | Per source | Request aggregate | -|---|---:|---:| -| Expanded bytes | 32 GiB | 64 GiB | -| Source-visible entries | 500,000 | 1,000,000 | -| Compressed Git pack or OCI layer bytes | 8 GiB | 16 GiB | -| Reachable Git objects | 5,000,000 | 10,000,000 | -| Regular-file bytes | 4 GiB | 4 GiB per file | -| Path | 4096 UTF-8 bytes / 128 components | same per path | -| Acquisition/materialization time | bounded operator policy | bounded session policy | - -Each OCI source permits at most 64 distributable tar/gzip/zstd layers, a 4 MiB -image manifest, a 128 MiB source manifest, and 1 MiB per PAX/extended header. -For every layer, source, and request, `expanded_bytes / max(compressed_bytes, 1)` -MUST NOT exceed 100. Git objects, checkout, filesystem entries, output, and -checkpoint accounting feed the same per-source and aggregate enforcement. - -## Canonical source-manifest v1 - -The pinned media type is -`application/vnd.allagents.source-manifest.v1+json`. - -The exact bytes are the RFC 8785 JSON Canonicalization Scheme encoding of: +All objects are closed. No alias for the old `metadata.workspace`, `sources`, `access`, `snapshot_name`, `image_manifest_digest`, or `source_manifest_digest` shape is retained. + +Request-decidable validation finishes before catalog, cache, registry, workspace, or lifecycle mutation. Continuation/replay/reused-session injection fails before hydrate. The media type is exact. Digest is SHA-256 lowercase. Size is bounded by manifest maximum. + +### Authorization + +V1 server configuration maps each authenticated product domain to exactly one trusted snapshot repository and one catalog. Zero or multiple mappings fail deployment preflight. Catalog lookup returns exactly one immutable `catalog_entry_id` and version binding the descriptor and digest-covered `available_until`; ambiguity fails before registry traffic. + +Authorization key is the exact `(principal, domain, repository_id, catalog_entry_id, media_type, digest, size)` tuple plus current policy. Authorization occurs before fetch, cache attachment, restart completion, and exact-digest reacquisition. Unauthorized and unknown both return `404 workspace_snapshot_unknown`. A cache hit is never authorization. + +### Response + +After `ready`, return: ```text -{ +metadata.allagents_workspace_snapshot = { version: 1, - entries: Array< - | {path: string, type: "directory"} - | { - path: string, - type: "file", - size: integer, - sha256: "sha256:<64 lowercase hex>", - executable: boolean - } - | {path: string, type: "symlink", target: string} - > + descriptor: {media_type, digest, size}, + snapshot_manifest_digest: "sha256:<64 lowercase hex>", + ready_manifest_digest: "sha256:<64 lowercase hex>", + provenance_digest: "sha256:<64 lowercase hex>", + initialization_changes_file_id: string, + working_directory: string, + available_until: RFC3339 timestamp, + expires_at: RFC3339 timestamp } ``` -The manifest digest is SHA-256 over those exact canonical bytes. Entries sort by -the UTF-8 bytes of their NFC-normalized relative POSIX `path`. The root entry is -omitted; empty directories are represented. Duplicate paths, non-UTF-8 or -non-NFC names, empty/`.`/`..` components, type conflicts, and unsupported types -fail validation. Regular files normalize to 0644 or 0755 according to -`executable`; all other mode bits are outside this schema. File SHA-256 is over -exact content bytes. - -A symlink target is a UTF-8 NFC string whose resolution from the symlink's parent -stays within the owning root. Absolute, escaping, malformed, or cyclic targets -that cannot be safely materialized fail. Safe in-root OCI hardlinks MAY be -materialized as ordinary files and are not a manifest type. `.git` entries MAY -be covered by source integrity manifests, but `.git` is always excluded from -public change reporting. - -Git acquisition computes this manifest from the verified detached tree with the -selected full or shallow history. -OCI fetches and validates the named source manifest before requesting any layer, -applies standard OCI image/layer/whiteout semantics, and requires the extracted -final tree to match exactly. - -## Public change projection on the existing Files/artifact surface - -No endpoint is added. Workspace-backed collection replaces only root-Git -produced-file listing behind the existing runner/gateway seams. - -At ready time, persist canonical manifest cursors for the writable outer tree -(excluding source destinations) and each editable source root. At every terminal -collection: - -1. `_produced_list` compares the last acknowledged cursor with fresh final - manifests for the outer and editable roots. Traversal is no-follow, bounded, - owner-aware, and does not cross mounts. -2. Read-only roots are not walked. They are trusted from immutable component - identity plus freshly revalidated read-only mount evidence. -3. The runner projects add, modify, and delete operations. Content, type, - executable-mode, and symlink-target changes are `modify`; rename is - `delete` plus `add`. Git status, commits, indexes, ignore rules, and rename - inference do not affect the result. -4. Gateway `_collect_produced` captures every added/modified regular file through - the existing artifact path. It then captures one server-generated artifact - named `workspace-changes-.json`. -5. Only after all file artifacts and the change artifact are durable does the - gateway call `_produced_ack`. ACK persists the new outer/editable cursor - manifests. A retry before ACK reproduces the same logical changes. - -The change artifact media type is -`application/vnd.allagents.workspace-changes.v1+json`. Its exact bytes are RFC -8785 canonical JSON: +Terminal streaming, retrieval, replay, and later workspace failures use the stored sanitized object. Pre-ready failures omit it. Catalog/repository names, policy revisions, credentials, private cache keys, local paths, reference IDs, builder jobs, and live filesystem facts remain private. + +Advertise vendor capability `allagents_workspace_snapshot_v1: true`. It means request schema V1 and snapshot artifact/config/provenance/manifest V1 are all supported. Absence or false means unsupported. Do not modify UHP conformance claims. + +## Gateway durable model + +### Snapshot binding + +Extend `HarnessSession` with one logical binding reference. Store large manifests in blob storage, not graph properties. ```text -{ - version: 1, - entries: Array<{ - path: string, - operation: "add" | "modify" | "delete", - before?: - | {type: "directory"} - | {type: "file", size: integer, sha256: "sha256:<64 lowercase hex>", executable: boolean} - | {type: "symlink", target: string}, - after?: - | {type: "directory"} - | {type: "file", size: integer, sha256: "sha256:<64 lowercase hex>", executable: boolean} - | {type: "symlink", target: string}, - file_id?: string - }> +WorkspaceSnapshotBinding = { + binding_id, + state: pending | ready | failed | deleting, + descriptor: {media_type, digest, size}, + snapshot_key, + repository_id, + catalog_entry_id, + catalog_entry_version, + signature_policy_id, + signature_evidence_digest, + authorization_audit_ref, + minimum_reader_version, + initializer_schema, + journal_schema, + initialization_deadline, + ready_at?, + expires_at?, + available_until?, + snapshot_manifest_digest?, + ready_manifest_digest?, + provenance_digest?, + initialization_changes_file_id?, + working_directory?, + cursor_blob_digest?, + failure_code? } ``` -Entries sort by UTF-8 bytes of NFC-normalized workspace-relative `path`. -`before` is absent for `add`; `after` is absent for `delete`. `file_id` is -required exactly when an added or modified `after` value is a regular file and -references its already-durable existing-path artifact; it is otherwise absent. -The artifact excludes `.git`, runner/control state, credentials, caches, -checkpoint metadata, and component-store paths. Promptfoo consumes ordered -change artifacts to reconstruct final state. - -## Private identity, cache, and publication - -A component is one verified immutable source tree. There is no durable or cached -composition object. The session binding contains the ordered resolved source -plan and becomes visible in one downstream `ready` transition after every root -verifies. - -Private component keys MUST be computable before materialization: - -- Git key: canonical URL + exact resolved commit + history selector (`full` or - exact requested depth) + one cache-schema revision. -- OCI key: catalog entry identity + exact image-manifest digest + exact - source-manifest digest + one cache-schema revision. - -Destination, ref spelling, access, session, harness, provider, working directory, -and expiry do not fragment component keys. The recomputed canonical baseline -digest is evidence required to publish or reuse a component; it is not a cache -key input. Do not add separate acquisition-policy, materializer, publication, -epoch, or baseline-digest dimensions to the key. - -### Git acquisition - -Maintain one operator-only bare acquisition mirror per canonical URL and -serialize its writes. Resolve the advertised default, branch, or lightweight or -annotated tag to an exact commit. With omitted `depth`, fetch the complete -ancestry reachable from that commit without fetching unrelated refs merely for -completeness. With supplied `depth`, fetch exactly that shallow ancestry. -Verify the tip and requested history boundary, then export a self-contained -detached checkout with no alternates or writable mirror links. Preserve safe -`.git` metadata for offline log, parent inspection, blame, and diff within the -selected history. Editable private copies additionally support bisect; read-only -mounts do not promise Git operations that mutate the worktree or repository. - -Disable interactive credentials, hooks, filters, alternates, alternate -protocols, submodules, and LFS hydration. Reject gitlinks and LFS pointer-backed -content. Never change the requested history mode, fetch arbitrary object IDs, -choose another ref, fetch unrelated refs as a completeness shortcut, or fall -back to OCI. Contractual pack, expanded-byte, object, time, process, and output -limits apply to full and shallow acquisition; exceeding one fails without -publishing a component. - -### OCI acquisition - -The bounded server-owned catalog maps `snapshot_name` to registry/repository -origin, credential reference, TLS/redirect policy, allowed media types, and -resource policy. These remain private. Require the direct image-manifest digest, -verify manifest and config, fetch and validate the canonical source manifest -before any layer, then stream and digest-check layers in order. - -Use standard OCI layer caching supplied by the selected library/client where -useful. Do not create a separate gateway-managed OCI blob-cache lifecycle. -Apply standard file and opaque-directory whiteouts. Reject absolute/traversing -paths, NULs, ambiguous separators, duplicate/type conflicts, devices, FIFOs, -sockets, unsafe sparse files, unsupported types, escaping links, and unbounded -metadata. OCI sources MAY be Git-free. There is no Git fallback and one OCI -image always represents one tree. - -### Component singleflight and publication - -Singleflight independently by private component key. Each miss uses private -staging and streamed accounting; canceled waiters detach without canceling work -still needed by another live waiter. Publish only after exact manifest -verification. Published component bytes and evidence are immutable; failed or -uncertain staging is unavailable and cleaned or quarantined. Existing live -workspace cache reaping may delete only complete, unreferenced publications and -must not invalidate an attached session. - -## Access, binding, and application-level visibility - -- `read_only`: bind the immutable component root at its destination with - kernel-enforced read-only, `nodev`, and `nosuid` behavior while preserving - required execute bits. Expose no writable alias, backing descriptor, mirror, - or component-store path. -- `editable`: create an inode-independent private reflink or copy at the - destination. No write may mutate the cache or a sibling session. - -Prepare the outer workspace, destination scaffolding, every bind/copy, baseline, -and working directory while the `HarnessSession` workspace binding is pending. -Files, checkpoint, provider, and harness consumers already gate on application -state; extend that gate to require workspace `ready`. After all roots verify, -persist the resolved plan and transition once to ready. Atomicity means -application visibility after this transition. It does not require a special -filesystem rename, mount-namespace handoff, global composition transaction, or -composition identifier. - -The durable workspace binding stores only: - -- the validated descriptor and its private digest; -- pending/ready/failed state and fixed `expires_at`; -- the ordered resolved source plan and sanitized public metadata; -- immutable component and baseline-manifest references; -- the outer/editable acknowledged cursor manifests; and -- outer/editable checkpoint references. - -Checkpoint, artifact, and control data remain in their existing separate -records. Do not persist live mount IDs, filesystem identities, verified flags, -publication epochs, inode evidence, or other process-local attachment facts. -On every initial attach, continuation, restart, and live-workspace cache attach, -revalidate component identity, ownership, mount target, read-only flags/no -writable aliases, or editable inode independence before ready. - -## Lifecycle - -### New session - -1. Use stock authentication, UHP validation, idempotency, and new-versus-reused - session selection. -2. Parse the closed workspace request and complete request-decidable ownership, - collision, bounds, and continuation checks. -3. Assign the fixed expiry and persist a pending binding on `HarnessSession`. -4. Resolve exact Git/OCI identities and private keys; independently claim, reuse, - or build each component under per-source and aggregate limits. -5. Hydrate the writable outer workspace through the existing path. Apply inputs - and generated assets only outside source destinations. -6. Attach every source according to `access`; recompute/verify baselines and live - protection. Any failure leaves the binding non-ready and exposes no partial - workspace. -7. Validate `working_directory`, persist ordered resolved plan, public metadata, - baselines, and initial manifest cursors; transition the binding once to ready. -8. Only then start the inherited harness/provider path. -9. On terminal collection, use manifest projection and capture-before-ACK. Then - checkpoint and complete the existing turn/session transition. - -An idempotent duplicate shares the same pending or ready binding and does not -create another attachment plan. - -### Checkpoint and continuation - -The existing stock sequence commits and tars a normal workspace. For a -workspace-backed session, retain that lifecycle while making archive boundaries -explicit: - -- tar the writable outer workspace without crossing any source destination; -- checkpoint each editable Git/OCI root separately; -- store only immutable component references for read-only roots; and -- preserve original baselines and acknowledged cursor manifests separately from - mutable final content. - -Continuation never resolves a ref or contacts Git/OCI. Restore the outer tar and -editable-root checkpoints, reattach recorded read-only components, revalidate -all live attachment protections, restore baseline/cursor references, validate -cwd, and only then mark the live workspace ready. Missing, expired, corrupt, or -mismatched binding, baseline, component, or checkpoint evidence fails closed. -Do not select a replacement component, rematerialize from a source, expose an -empty root, or discard editable mutations. - -### Expiry, deletion, restart, and cleanup - -The fixed workspace expiry applies to every workspace-backed session and is -never extended. Expiry and explicit deletion first make the session unavailable, -stop descendants, unmount source roots, verify mount absence, remove editable -and outer state, release references once, and delete durable binding/checkpoint -state through existing records. Cleanup is mount-aware, confined, idempotent, -and unavailable-first. Uncertain paths remain unavailable, accounted, and -quarantined for retry. - -Restart reconciliation resumes or fails each pending durable transition without -trusting process-local attachment facts. Stock live-workspace cache reaping -remains distinct from durable session expiry and deletion. - -## Stable workspace failure contract - -Workspace failures use the existing bounded UHP error envelope with the exact -detail codes below. `retryable` describes retrying the same logical operation -after its stated cause is corrected; it is not permission to extend expiry or -change a binding. Base UHP authentication, envelope, provider, and transport -errors retain stock codes only where their meaning is exact. +`expires_at` is absent until the `ready` transition and is computed from `ready_at`. Polling and continuation never change it. + +`WorkspaceSnapshotReference` is a durable record keyed by `(binding_id, snapshot_key)` with `provisional | active | released`. Creation, activation, and release use compare-and-set. A still-running pending attempt retains its provisional claim across internal restart reconciliation. Every pre-ready response failure makes the binding terminal `failed` and releases the claim; a caller retry creates a new binding. Counters are derived, never authoritative. + +Do not persist local cache paths, inode/device identities, reflink facts, attachment flags, staging paths, lock owners, registry credentials, or process IDs. + +### Turn finalization record + +Add a recoverable record keyed by `response_id`: + +```text +WorkspaceTurnCommit = { + response_id, + state: preparing | durable | acknowledged | committed | aborted, + base_cursor_digest, + next_cursor_digest?, + change_artifact_id?, + file_artifact_ids: [], + checkpoint_digest?, + collection_token?, + collection_fingerprint? +``` + +All artifact IDs and blob keys are deterministic from response identity plus content identity. Retry verifies and reuses existing durable objects. `acknowledged` means the runner accepted the explicit base/next cursor token, while the gateway cursor remains authoritative. `committed` is reached only when response artifacts, checkpoint, cursor pointers, and terminal state publish together under the existing response/session control lease. + +## Snapshot admission, cache, and private materialization + +### Admission + +Fetch only from the binding's persisted `repository_id` by direct digest under the shared per-connection transport policy. Never search another repository or follow caller-selected origins or tags. Verify declared size and response digest before parsing. Reject indexes/lists. Verify manifest/config/provenance/tree/layer media types, descriptor sizes, digests, supported `reserved_paths_schema`, and catalog/config `available_until` equality before use. + +If retention is expired or cannot cover initialization deadline plus maximum session TTL plus safety margin, return nonretryable `422 workspace_snapshot_retention_invalid`. If an authorized manifest is missing before `available_until`, return `503 workspace_snapshot_unavailable` and alert on the broken retention lease. Never search another repository. + +Apply only gzip OCI layer changesets in order, including whiteouts and opaque directories, through rooted no-follow operations; do not shell out to `tar`. Reject every layer or final-tree path/link/whiteout/type transition at `.harness` or below, including content omitted from the public manifest. Normalize or reject ownership/mode/xattr/ACL/capability metadata. Compute the public snapshot manifest plus a private full-tree seal covering `.git` before publication. + +### Immutable cache + +Cache key: + +```text +sha256(canonical({descriptor, materializer_schema})) +``` + +Destination, principal, session, harness, provider, expiry, policy revision, and working directory do not fragment the verified byte cache. Authorization remains separate. + +An exact-key miss singleflights. Publication sequence is staging -> full verification -> immutable generation -> complete marker -> available. Cache roots and generations are owned by a gateway/materializer identity no harness/session UID can assume, are non-writable, and are non-searchable from the runner namespace. Failed or uncertain staging is unavailable and quarantined. + +Clone/reflink reads use trusted directory file descriptors, rooted no-follow operations, and an eviction/clone lease. Verify the private full-tree seal immediately before and after cloning. Any unexpected metadata/content mutation quarantines the generation and fails closed. + +Global controls are mandatory: maximum cache/staging bytes and inodes; high/low watermarks; eviction only among complete unreferenced generations; staging/quarantine age and size; bounded waiters/builds; metrics; and reconciliation from durable reference records. A malicious session test MUST prove a guessed cache path cannot be searched, read, or mutated. + +### Private writable tree + +Every snapshot session gets an ordinary private writable directory: + +1. prefer same-filesystem per-file reflink with independently created directory entries/inodes; +2. fall back to full private copy; +3. never hardlink regular files to cache, expose a writable cache alias, or symlink the workspace to cache; +4. walk no-follow and revalidate source generation before cloning; +5. apply hard runtime byte/inode quotas; and +6. validate the digest-covered working directory beneath the private root. + +Probe production filesystem behavior before implementation is considered deployable. The integration suite forces both reflink and copy paths. OverlayFS, bind-mounted source roots, per-source read-only/editable modes, and delta checkpoints are out of V1. + +### Ready transition + +New-session sequence: + +1. run stock auth, UHP validation, idempotency, and session selection; +2. parse/validate the closed extension and compatible minimum-reader routing; +3. resolve one repository/catalog entry and authorize the exact tuple; +4. persist pending binding, private authorization subject, and provisional reference before fetch; +5. claim/reuse/build immutable generation; +6. reauthorize current policy; +7. materialize and full-seal-check the private tree; +8. apply UHP inputs and deterministic runner instruction/control preparation; +9. compare snapshot manifest with the ready tree; durably capture changed regular files plus `workspace-initialization-changes-.json`; +10. store the ready manifest as authoritative initial cursor and validate working directory; +11. activate reference and transition binding to `ready` with snapshot/ready digests, initialization artifact, `ready_at`, and `expires_at`; and +12. start the stored harness/provider path. + +The initialization artifact uses `application/vnd.allagents.workspace-changes.v1+json` and the same canonical schema as turn deltas. It is emitted even when empty. Promptfoo applies it to builder snapshot bytes before response deltas. No Files, checkpoint, provider, harness, or response success path observes the workspace before step 11. A pre-ready failure is terminal for that binding, releases provisional state once, and leaves no runnable tree. + +## Manifest journal and change artifacts + +Snapshot sessions select `manifest-v1`; stock sessions keep root Git. The gateway-owned cursor blob is the sole durable journal authority. Gateway `/produced` sends the explicit base cursor blob/digest to the runner; the runner returns changes, next canonical cursor, and a collection token without mutating durable state. `/produced/ack` idempotently confirms `(base,next,token)` but cannot supersede the gateway cursor. + +Terminal collection holds one exclusive workspace mutation lease. It stops and proves absence of descendants and blocks Files writes, new turns, cancellation cleanup, deletion cleanup, and every other writer until finalization: + +1. send/load the acknowledged base cursor; +2. compute the final canonical manifest under output byte/inode/time limits; +3. derive add/modify/delete operations; +4. open every added/modified regular file with rooted `openat`/no-follow semantics, hash while streaming, and require type/size/digest to equal its `after` state; +5. durably store file artifacts and one canonical `workspace-changes-.json`; +6. create the exact snapshot-mode checkpoint and verify its visible manifest equals the next cursor; +7. durably store the next cursor; +8. call idempotent ACK with base, next, and token; and +9. compare-and-set artifacts, checkpoint, authoritative cursor, and terminal response to committed. + +Change artifact media type is `application/vnd.allagents.workspace-changes.v1+json`. Schema and ordering match ADR 0002. `file_id` exists exactly for added/modified regular files after streamed bytes match their manifest state. + +`WorkspaceTurnCommit` resumes crashes without rerunning the harness. A crash before step 9 leaves the old gateway cursor authoritative even when ACK ran; retry supplies the old base and reproduces the transaction. Terminal response visibility occurs only at step 9. + +Promptfoo applies ordered deltas to the exact builder baseline. Test reconstruction against the baseline snapshot; do not assert that deltas alone contain the original tree. + +## Checkpoint, continuation, cancellation, and deletion + +### Exact snapshot-mode checkpoint + +Implement this exact snapshot-mode checkpoint before integrating turn finalization: + +- skip `_git_ensure` and any root-Git commit; +- archive the complete normal private directory, including nested/root `.git` histories; +- reject any snapshot-origin `.harness` content before ready; +- exclude from checkpoints only `.harness/tmp/**`, `.harness/home/.codex/auth.json`, `.harness/home/.omp/agent/auth.json`, `.harness/home/.omp/agent/models.json`, and `.harness/home/.omp/agent/models.yml`; +- retain every other `.harness` path required for conversation, skills, plugins, and resume; +- stream through bounded disk rather than memory; +- store checksum/size; and +- reconstruct the checkpoint's visible manifest and require equality with the next cursor before publishing its pointer. + +Matching uses NFC-normalized POSIX paths after no-follow traversal. Directories named `node_modules`, `.venv`, `venv`, or other stock scratch names remain checkpointed when declared snapshot or session content. Any future exclusion changes require a checkpoint-schema revision and mixed-version reader gate. + +### Continuation + +A continuation: + +1. resolves the original session and enforces current principal/session ownership; +2. rejects supplied snapshot metadata; +3. rejects at or after `expires_at` before restore; +4. restores the exact checkpoint and cursor under quotas; +5. verifies binding, checkpoint checksum, journal schema, reserved-path schema, and working directory; +6. reacquires an evicted immutable base only by the same authorized direct digest when needed for integrity evidence, never to replace mutable checkpoint state; and +7. starts the stored harness only after the restored workspace is ready. + +Missing/corrupt binding, checkpoint, cursor, or identity fails closed. Do not start empty, substitute a newer descriptor, call the builder, or resolve a source. + +### Cancellation + +Preserve stock idempotent cancel endpoints and terminal `status: "cancelled"`. Stop descendants before final collection/checkpoint. Cancellation before ready releases provisional initialization and omits snapshot metadata. Cancellation after ready runs the same recoverable finalization transaction and retains stored metadata. + +### Expiry and deletion + +TTL begins at ready. Expiry/delete first atomically makes the session unavailable and stops descendants, then handles `WorkspaceTurnCommit` deterministically: + +| Turn state | Deletion action | +|---|---| +| `preparing` | Mark `aborted`, discard only unreferenced staging objects, retain the old checkpoint/cursor as authority, and finalize the response through stock cancellation/deletion semantics. | +| `durable` or `acknowledged` | Finish idempotent ACK/CAS to `committed` from durable evidence; do not rerun the harness. The terminal response may remain retrievable while the session is unavailable. | +| `committed` | Retain response/artifact records per stock policy and proceed with session cleanup. | +| `aborted` | Reconcile/discard unreferenced staging and proceed. | + +Only after turn state resolves does deletion remove the private tree with rooted no-follow operations, delete checkpoint/cursor state under retention policy, release `(binding_id,snapshot_key)` once, and tombstone/delete the binding. A turn record is removed only after response state and every referenced/unreferenced blob are reconciled. + +Busy or uncertain state stays unavailable, counted, and queued for retry. Restart reconciliation resumes pending initialization/finalization/deletion from durable records and never trusts process-local filesystem facts. + +## Stable failure contract | Detail code | Condition | HTTP | Retryable | Required behavior | |---|---|---:|:---:|---| -| `workspace_invalid_request` | Closed-schema, count, field, URL, depth syntax/range, digest syntax, path, cwd, first-turn, or reused-session violation | 400 | no | Fail before cache, network, workspace write, claim, or lifecycle mutation. | -| `workspace_path_collision` | Equal/overlapping destinations or input/generated/reserved/ancestor collision | 409 | no | Fail before cache or network; report only sanitized conflicting workspace-relative fields. | -| `workspace_source_unknown` | Unknown OCI catalog entry or missing/ambiguous/unsupported Git ref identity | 404 | no | Fail that source with no alternate ref, catalog entry, or source kind. | -| `workspace_source_invalid` | Moved/non-commit Git target, requested-history refusal, gitlink/LFS content, OCI media/digest/source-manifest/layer/final-tree failure, or unsafe source content | 422 | no | Publish no failed component and perform no ref, history-mode, or source-kind fallback; invalid OCI source manifest fails before layer requests. | -| `workspace_acquisition_unavailable` | Timeout, DNS, registry/Git service, or other transient source transport failure | 503 | yes | Detach request-local work, preserve independently valid shared components, and expose no partial workspace. | -| `workspace_contract_limit_exceeded` | A fixed v1 per-source or aggregate count/byte/path/ratio/time/output ceiling is exceeded | 413 | no | Stop bounded work, clean/quarantine staging, and expose no partial workspace. | -| `workspace_capacity_exceeded` | Operator concurrency, disk, inode, mount, or lower policy capacity is temporarily unavailable | 503 | yes | Admit no partial binding; capacity policy must not masquerade as a schema limit. | -| `workspace_attachment_failed` | Initial attachment or later live reattachment fails bind/copy, ownership, protection, baseline, cwd, or ready validation | 500 | yes | Keep an initial binding non-ready or fail a post-ready reattachment; reverse/unmount request-local state and never run the harness. | -| `session_expired` | Continuation targets a workspace-backed session at or after fixed `expires_at` | 404 | no | Preserve the pinned UHP `session_expired` response, omit workspace metadata, and do not restore, reacquire, or extend expiry. | -| `workspace_restore_invalid` | Required binding, component, checkpoint, baseline, or cursor evidence is missing, corrupt, or mismatched | 500 | no | Fail closed with no source traffic, replacement selection, or empty-root recovery. | -| `workspace_collection_failed` | Manifest traversal/comparison, file/change-artifact capture, or ACK persistence fails | 500 | yes | Do not ACK or publish an incomplete change set; retry reproduces the same logical changes. | -| `workspace_checkpoint_failed` | Mount-aware archive creation or durable checkpoint persistence fails after collection | 500 | yes | Preserve the acknowledged collection state, publish no invalid checkpoint, and retry checkpoint persistence without rerunning the harness. | - -Failures before the first ready transition omit `metadata.workspace`; workspace -failures after ready return the stored sanitized `metadata.workspace`. The -inherited `session_expired` response is the sole exception and remains unchanged. -No case exposes private keys, catalog origins, host paths, mounts, credentials, -raw tool stderr, or network details. - -Cancellation is not a workspace error. Preserve the inherited idempotent `2xx` -cancel endpoints and terminal `status: "cancelled"` rather than returning an -error code. Cancellation before ready omits workspace metadata; cancellation -after ready returns the stored sanitized metadata. Stop unneeded descendants, -detach shared waiters safely, and leave no partial ready state. +| `workspace_snapshot_invalid_request` | Closed-schema, version, media type, digest, size, first-turn, or continuation violation | 400 | no | Fail before catalog/cache/registry/workspace/lifecycle mutation. | +| `workspace_snapshot_unknown` | Exact descriptor absent from caller's authorized catalog view | 404 | no | Hide cross-principal existence; do not consult cache as authorization. | +| `workspace_snapshot_invalid` | Manifest/config/provenance/layer/path/link/type/signature/final-tree/cwd verification failure | 422 | no | Publish no generation and expose no partial workspace. | +| `workspace_snapshot_retention_invalid` | Catalog/config retention expired or cannot cover initialization deadline + maximum session TTL + safety margin | 422 | no | Fail before cache attachment; caller must publish a new snapshot descriptor. | +| `workspace_snapshot_unavailable` | Registry/DNS/transport, initialization deadline, or authorized manifest missing before `available_until` | 503 | yes | Fail the binding, release its provisional claim, and allow a new request with the same descriptor; never search another repository. | +| `workspace_contract_limit_exceeded` | Fixed format count/byte/path/ratio/metadata/file maximum | 413 | no | Stop bounded work; elapsed time never uses this code. | +| `workspace_capacity_exceeded` | Lower operator disk/inode/quota/worker/concurrency capacity | 503 | yes | Admit no partial generation or private tree. | +| `workspace_initialization_failed` | Verified artifact cannot publish/clone/baseline/transition because of internal failure | 500 | yes | Mark binding failed, clean/quarantine, and release provisional reference safely. | +| `session_expired` | Continuation at or after ready-based expiry | 404 | no | Preserve inherited UHP shape; do not restore or extend TTL. | +| `workspace_restore_invalid` | Binding/checkpoint/cursor/identity missing, corrupt, or inconsistent | 500 | no | Fail closed without empty-root or replacement recovery. | +| `workspace_collection_failed` | Manifest or artifact persistence cannot complete | 500 | yes | Keep previous cursor/checkpoint authoritative; resume transaction. | +| `workspace_checkpoint_failed` | Exact checkpoint cannot persist | 500 | yes | Do not finalize terminal response or advance cursor; resume without rerunning harness. | + +Pre-ready failures omit snapshot metadata. Post-ready workspace failures return stored sanitized metadata except inherited `session_expired`. Provider/harness failures retain stock codes. Errors and logs never expose credentials, registry paths, private catalog names, local paths, source content, or raw tool stderr. + +For pre-ready retryable failures, retry means a new request/binding with the same descriptor; an idempotent duplicate returns the original failed response. For post-ready collection/checkpoint failures, retry resumes the same `WorkspaceTurnCommit` and never reruns the harness. + +## Observability and audit + +Use correlation-safe identifiers: request/response/session ID, binding ID, redacted descriptor prefix, snapshot-key prefix, cache outcome, state transition, duration, bounded byte/inode counters, and stable error code. Never log full private URLs, headers, credentials, provenance content, source filenames, file bytes, host paths, or complete digests where organizational policy treats them as sensitive. + +Required metrics: + +- authorization allow/deny by stable reason; +- pending/ready/failed/deleting transitions and duration; +- registry bytes/time and verification failures; +- cache hit/miss/singleflight wait/build/evict/quarantine; +- reflink/copy selection, bytes, inodes, and duration; +- workspace quota utilization; +- manifest walk/change counts and duration; +- turn-finalization resume/failure stage; +- checkpoint bytes/time/failure; +- reference claim/activation/release/reconciliation; and +- cleanup backlog age and capacity impact. + +Audit records identify the exact descriptor, catalog/policy decision reference, builder/provenance digests, gateway/upstream commit, and published image digest without secrets. ## Implementation phases and exit proofs -Every phase changes the real named seam and ends with observable focused proof. -Mocks may isolate external provider transport, but source-text assertions and -mock forwarding are not proof. +Every phase changes the named real seam and ends with observable proof. Project-wide suites run only after focused phase work. Source-text assertions and mock forwarding are not proof. -### Phase 1: Bootstrap baseline and characterize stock seams +### Phase 0: Bootstrap repositories and characterize stock -**Work** +**Workspace Builder work** + +- Create repository, license/security/CI/release skeleton, pinned toolchain, CLI/library boundary, local registry/Git fixtures, and deterministic golden corpus. +- Record artifact media types and version ownership. -- After the admin bootstrap, verify target/ref, Apache-2.0/NOTICE/history, - `origin`, `upstream`, and fork-point record. -- Rename downstream distribution surfaces without changing attributed upstream - material. -- Trace and record `_produced_list`, `_produced_ack`, `_collect_produced`, - `BACKING.workspace`, `RunnerWorkspaceFiles`, `CheckpointWorkspaceFiles`, - `HarnessSession`, and separate checkpoint/artifact/control records. -- Capture stock traces proving fresh root Git, Git produced cursoring, - checkpoint commit-then-tar, tar hydration, live-workspace cache reaping, - durable explicit deletion, continuation, cancellation, and provider routing. +**Gateway work** -**Exit proof:** a request without `metadata.workspace` matches pinned status, -stream, Files, checkpoint/restore, continuation, cancellation, deletion, and -provider traces; the renamed checkout preserves license/NOTICE/history and the -exact fork point. +- Create `feat/workspace-snapshots`, preserve fork history/license/NOTICE, add upstream remote, and record fork point. +- Capture the red stock limitations and unchanged stock UHP trace. +- Probe the target deployment filesystem for reflink correctness, quota support, path/mode semantics, and full-copy fallback before deeper implementation. +- Link [upstream issue #304](https://github.com/HarnessRouter/harnessrouter/issues/304) in the downstream delta record. -### Phase 2: Add the closed request and session binding +**Exit proof:** both repos have protected reproducible builds; red fixtures prove root-Git mutation/nested-repo/deletion limitations; stock no-extension traces are recorded; production filesystem capability is known rather than assumed. + +### Phase 1: Build and publish snapshot v1 **Work** -- Implement parsing, all pre-network ownership/collision checks, fixed expiry, - private descriptor digest, and pending/ready/failed binding states on the - existing session seam. -- Persist only the durable fields listed above and return only the pinned public - shape after ready. -- Reject workspace metadata on every reuse/continuation path before hydration. +- Implement closed build spec plus trusted `BuildContext` and pre-network path/ownership authorization. +- Implement full-by-default Git and exact optional shallow acquisition under the shared per-connection fetch policy. +- Implement digest-pinned OCI input admission and secure extraction. +- Implement deterministic composition, exact reserved-path schema, independent canonical manifest/provenance schemas, canonical gzip OCI layers/config/manifest, complete-before-return publication, and digest-covered retention lease. +- Add malicious fixtures and bounded cancellation/restart cleanup. -**Exit proof:** omitted depth and depths 1 and 1,000,000 parse and persist; depth -0, 1,000,001, fractional, and wrong-type values return -`400 workspace_invalid_request` with zero source/cache activity. One and 128 -Git/OCI/mixed entries pass; 0/129, obsolete fields, unknown fields, -root/overlap/collision, malformed identities, and reused-session injection -return their exact coded errors with zero source/cache activity. Stock traces -remain unchanged. +**Exit proof:** root single-repo, multi-repo, Git-free, full-history, shallow, whiteout, empty-dir, link, executable, and repeated deterministic builds publish expected descriptors. Offline Git operations work at promised history depth. Every malicious fixture fails before descriptor return. Pulling by returned digest reconstructs the exact canonical tree and provenance with no secret material. -### Phase 3: Implement canonical manifests and produced projection +### Phase 2: Add gateway contract, authorization, and durable binding **Work** -- Implement source-manifest v1 canonicalization and secure bounded traversal. -- Replace workspace-backed root-Git listing behind `_produced_list` with - outer/editable cursor comparison; keep stock implementation unchanged. -- Extend `_collect_produced` to capture changed regular files and the canonical - change artifact before `_produced_ack`; advance cursors only in ACK. -- Route live/checkpoint reads through the existing `BACKING.workspace` classes. +- Parse vendor extension in create-response path; reject every obsolete source schema and continuation injection. +- Add literal capability `allagents_workspace_snapshot_v1`, sanitized response metadata, private repository/catalog/signature binding fields, reference records, pending transition, ready-based TTL, minimum-reader routing, and restart reconciliation. +- Implement unique repository/catalog resolution, descriptor authorization, and cache-hit reauthorization. -**Exit proof:** synthetic outer/editable-root fixtures produce exact -add/modify/delete/type/mode/symlink/binary artifacts independent of Git state; a -failure before ACK retries identically; and the canonical change-artifact parser -reconstructs final state from ordered artifacts. +**Exit proof:** malformed/unknown fields fail `400` with zero cache/network/write activity; unknown/unauthorized both fail `404`; a valid descriptor persists pending identity before fetch; duplicate idempotency shares one binding; continuation replacement fails before hydrate; stock requests remain trace-equivalent. -### Phase 4: Add component cache, access modes, and ready gating +### Phase 3: Implement admission, immutable cache, and private tree **Work** -- Implement the two exact private cache keys, independent singleflight, private - staging, immutable publication evidence, references, reconciliation, and - quarantine. -- Attach read-only bind mounts or editable reflink/private copies into the - existing private workspace and revalidate live protection on every attach. -- Persist the ordered plan on the session binding and expose it only through the - single ready transition. Add no composition cache, record, ID, or filesystem - handoff protocol. +- Fetch from the persisted repository and verify all descriptor/media/signature/provenance/tree/layer/retention identities. +- Apply layers securely; independently reject `.harness` aliases and recompute public manifest plus private full-tree seal. +- Implement inaccessible immutable cache ownership, FD-based clone lease, seal checks, singleflight, watermarks, eviction, quarantine, and reconciliation. +- Implement private reflink clone and forced full-copy fallback; apply inputs/instruction merge, capture the initialization delta, validate cwd, and transition ready only after ready-manifest cursor storage. -**Exit proof:** overlapping concurrent requests publish each missing component -once; partial cache warmth builds only misses; last-root failure exposes no -Files/checkpoint/provider/harness view; read-only sessions share immutable bytes -without source traversal during collection and require fresh mount evidence; -editable sessions cannot mutate cache/sibling content. Restart discards or -reconciles uncertain publications without persisted mount/inode facts. +**Exit proof:** exact descriptor miss publishes once under concurrency; hits reauthorize; expired/short retention returns `422`, while a promised-but-missing manifest returns `503` without repository search; initialization delta reconstructs the ready tree from builder baseline; failed final verification exposes no Files/provider/harness state; session UIDs cannot search/read/mutate guessed cache paths; malicious reserved paths fail independently; both reflink and copy pass. -### Phase 5: Implement full-by-default Git acquisition +### Phase 4: Add exact checkpoint, manifest journal, and recoverable finalization **Work** -- Implement canonical HTTPS validation, advertised ref resolution, the per-URL - serialized mirror, full reachable ancestry by default, exact optional shallow - depth, detached self-contained export, safe `.git`, and source-manifest - publication evidence. -- Enforce egress/redirect policy, limits, disabled helpers, and no ref, - history-mode, or source-kind fallback. - -**Exit proof:** default/branch/lightweight-tag/annotated-tag/merge cases resolve -to exact commits. Omitted depth provides offline log/blame/diff across complete -fetched ancestry in both access modes and bisect in editable mode. A fixture -with unrelated branches and tags proves they are neither requested for the -selected full ancestry nor exposed in the published component. Depths 1 and 2 -expose exactly their shallow boundaries, publish distinct components, and each -reuses only its exact-depth component on repetition. Different ref spellings -resolving to one commit and the same history selector reuse one component; full, -depth 1, and depth 2 remain distinct. Moved refs, unsupported history requests, -malicious redirects, gitlinks, LFS, limits, cancellation, and restart fail with -exact codes. An exact key hit performs no pack acquisition or materialization. - -### Phase 6: Implement OCI source-tree acquisition +- Implement snapshot-mode checkpoint/hydrate first: no root Git, exact `.harness` policy, complete Git history, checksum, and visible-manifest verification. +- Add `manifest-v1` behind produced routes with explicit gateway-to-runner cursor input while retaining stock root Git. +- Implement streamed file/hash binding and exact add/modify/delete/type/mode/link behavior under the exclusive mutation lease. +- Add `WorkspaceTurnCommit` and integrate artifacts, exact checkpoint, next cursor, ACK, CAS, deletion states, and terminal visibility. + +**Exit proof:** independent fixtures produce exact operations regardless of Git state; `.git` and `.harness` never appear; injected failure at every checkpoint/ACK/CAS/deletion stage resumes to one logical artifact/checkpoint/cursor without rerunning; baseline plus deltas reconstructs final state; raced writers cannot make artifact bytes, manifest, and checkpoint disagree. + +### Phase 5: Complete continuation and lifecycle **Work** -- Use `oci-auth-registry`, `oci-catalog-builder`, and - `oci-source-fixture-builder` for catalog resolution, direct digest fetch, - pre-layer canonical manifest validation, standard layer/whiteout extraction, - limits, secure links/types, and exact final verification. -- Use standard client/library layer caching only; add no managed blob-cache - subsystem. Preserve Git-free operation. +- Implement continuation, cancellation, ready-based expiry, state-specific explicit deletion, reference release, and unavailable-first reconciliation. +- Exercise `RunnerWorkspaceFiles` and `CheckpointWorkspaceFiles` against live and restored snapshot sessions. +- Add mixed-version admission fencing and internal minimum-reader routing. -**Exit proof:** authenticated positive fixtures cover multiple OCI roots, -read-only/editable access, whiteouts, empty directories, safe hardlinks, and -exact reuse. An editable Git-free OCI mutation produces the exact canonical -change artifacts. Malicious fixtures cover catalog/media/digest mismatch, -manifest rejection before layers, traversal, links, types, sparse/compression -bombs, limits, cancellation, partial cleanup, and restart. Exact component hits -require no registry fetch or extraction. +**Exit proof:** mutations and Git history survive turns/restart; corrupt evidence fails closed; cancellation produces one terminal state; polling/continuation do not move expiry; every turn-commit deletion state reconciles without orphaning/publishing inconsistent data; concurrent expiry/delete/restart releases one reference; incompatible replicas reject before hydrate. -### Phase 7: Make checkpoint, continuation, expiry, and deletion mount-aware +### Phase 6: Cross-repository and Promptfoo integration **Work** -- Adapt stock commit/tar and hydrate through existing checkpoint seams: outer tar - excludes all source roots; editable roots checkpoint separately; read-only - roots retain immutable references. -- Restore without source traffic, revalidate live attachment protection, restore - baselines/cursors, and gate cwd/harness on ready. -- Implement fixed non-extendable expiry, explicit deletion, cancellation, - restart reconciliation, reference release, and unavailable-first cleanup. - -**Exit proof:** mixed outer/editable Git/editable OCI mutations survive turns and -restart and retain their original comparison cursors; read-only roots reattach -by exact identity; polling and continuation do not move `expires_at`; corrupt or -expired evidence fails closed; cancellation and cleanup never traverse a live -mount or expose a partial session. - -### Phase 8: Produce the implementation handoff and run one release flow - -The implementation agent completes the PR with local fixtures, mocked provider -transport, focused phase proofs, pinned dependencies, and release automation. -There is no separate provider phase: tests assert that both inherited harness -paths start only after ready/cwd and that workspace code does not change broker, -credential, model, or route behavior. - -After review, the operator performs one release flow: - -1. Build one `linux/amd64` candidate from pinned inputs, attach SBOM and build - provenance, and publish it to - `ghcr.io/allagentsdev/allagents-gateway:-allagents.`. -2. Read the image back and record its registry manifest digest. Deploy and test - only `ghcr.io/allagentsdev/allagents-gateway@sha256:`. -3. Run the full release matrix once against that published/read-back digest: - stock UHP compatibility; Codex and OMP; all-Git, multiple-OCI, and mixed - ordered plans; partial cache warmth and singleflight; editable Git/OCI/outer - mutation and change-artifact reconstruction; read-only enforcement; limits, - failures, cancellation, expiry, deletion, cleanup, and secret scans. -4. Include fresh-volume and same-volume restart cases in that same matrix, - covering pending acquisition/publication, ready attachment, checkpoint, - continuation, cache reuse, quarantine, and cleanup reconciliation. -5. Record the upstream/fork/downstream commits, dependency and harness pins, - fixture digests, published image digest, SBOM, provenance, and one E2E report. - -Do not run a duplicate full matrix against a pre-publication build and then again -after publication. Focused local phase proofs protect implementation; the one -release matrix proves the exact distributed digest. +- Build fixture snapshots with the published builder binary, push to an authenticated local registry, authorize exact descriptors, and run Codex and OMP through the built gateway. +- Implement Promptfoo flow: retain builder snapshot/provenance, invoke UHP with descriptor, apply initialization delta, consume ordered response changes, and reconstruct final tree. +- Prove no task-time call to builder and no Git/source credentials in gateway or harness. + +**Exit proof:** one snapshot runs with both harnesses; cache miss/hit have identical semantics; builder baseline + initialization delta equals `ready_manifest_digest`; subsequent deltas reconstruct the final private tree; continuation performs no mutable source resolution. + +### Phase 7: Upstream the generic seam + +**Work after maintainer agreement on issue #304** + +- Split changes into the smallest accepted upstream reviews: initializer/lifecycle seam, explicit-cursor manifest journal, recoverable terminal-finalization seam, and focused tests/docs as maintainers direct. +- Keep stock implementations default and avoid AllAgents names/source schemas in core. +- Maintain one downstream delta map from every patch to upstream issue/PR/release/removal condition. + +**Exit proof:** upstream tests prove initialization order, idempotency, replacement rejection, explicit cursor handoff, exact journal, capture-before-ACK, checkpoint/cursor durability before terminal visibility, continuation, restore failure, and unchanged stock behavior. Downstream adapter builds against all accepted seams without aliases; unaccepted seams remain explicit fork deltas. + +If maintainers reject or materially reshape the proposal, update ADR 0002 before introducing a different fork architecture. Do not push product-specific source preparation into HarnessRouter as a shortcut. + +### Phase 8: Review and release exact digests + +Run final architecture/security review before the green E2E. Resolve important findings, then: + +1. publish builder and gateway candidates with SBOM and build provenance; +2. read both back and record exact registry manifest digests; +3. deploy/test only those digests on the preflighted production-equivalent filesystem; +4. keep `allagents_workspace_snapshot_v1` disabled until every serving gateway/runner is compatible, then prove version-aware routing rejects old replicas before hydrate; +5. run stock UHP conformance and snapshot E2E with both supported harnesses; +6. run N-1-to-candidate upgrade and prove compatible-reader rollback or enforce unavailable-first drain before old code serves snapshot sessions; +7. repeat race-sensitive singleflight, cache-path attack, writer race, ACK/CAS, cancellation, crash, turn deletion, reference release, eviction, and cleanup cases against the exact digest; +8. run fresh/same-volume restarts at pending initialization, ready, active turn, durable/acknowledged finalization, checkpoint, and every deletion state; +9. run secret, artifact-content, and telemetry scans; and +10. record upstream issue/PR status, downstream delta, fixture digests, builder/gateway digests, SBOMs, provenance, compatibility versions, and E2E report. + +Do not prove a local build and assume the published image is equivalent. Do not support an architecture that was not built, preflighted, and tested; V1 MAY declare `linux/amd64` only. ## Completion checklist -### Implementation agent - -- PR targets renamed `allagentsdev/allagents-gateway` branch - `feat/workspace-composition` at downstream commit `fbcb73132423c8c4575113fc8943c6a6280a4746` - and preserves the fork network, existing downstream commits, - Apache-2.0/NOTICE/history/upstream. -- Stock seams remain the integration points, and stock requests retain their - pinned behavior. -- Request/response, source-manifest, change-artifact, cache-key, binding, - lifecycle, and coded-error contracts match this plan exactly. -- Local OCI fixtures and Linux capability checks prove Git/OCI/mixed access, - manifest cursoring, checkpoint/restore, restart reconciliation, and cleanup. -- No composition record/ID, retention/persistent field, durable live attachment - facts, managed OCI blob-cache lifecycle, provider fork, or compatibility shim - remains. +### AllAgents Workspace Builder + +- Repository exists with protected main, pinned toolchain/dependencies, license, security policy, and reproducible releases. +- Build spec v1, full/shallow Git, OCI inputs, secure composition, provenance, canonical manifest, and OCI publication match this plan. +- Versioned builder result and digest-covered `available_until` match registry retention; delayed execution inside the admitted window succeeds. +- Credentials and private transport configuration never enter artifact/log/output. +- Independent provenance/manifest implementations, canonical gzip, golden fixtures, malicious fixtures, and published readback reproduce identity. + +### AllAgents Gateway + +- Fork relationship, history, `LICENSE`, `NOTICE`, upstream remote, and stock behavior remain intact. +- Runtime source composition and obsolete schemas/branches are absent. +- Vendor request/response, authorization, binding/reference state, cache, private clone, ready gating, journal, turn commit, checkpoint, continuation, cancellation, expiry, and deletion match ADR 0002. +- Cache hits reauthorize; private repository/catalog/signature subject persists; durable references are idempotent; TTL starts at ready; expired/short retention returns `422`, deadlines/promised-retention misses return `503`, and invalid cwd returns `422 workspace_snapshot_invalid`. +- Snapshot baseline plus initialization delta equals ready manifest; exact response changes include deletion and exclude `.git`/`.harness`; streamed bytes equal `after` state before ACK. +- Response visibility cannot outrun journal ACK, exact checkpoint, or authoritative cursor durability. +- Reflink/copy, inaccessible cache, reserved paths, quota, mixed-version fencing, restart, rollback-or-drain, turn deletion, and race repetitions pass against the exact published digest. + +### Promptfoo/integrator + +- Preparation and execution are separate steps. +- Builder result and snapshot baseline are retained. +- UHP request carries only the direct descriptor extension. +- Initialization delta is applied before ordered response deltas; neither is treated as a self-contained snapshot. +- Continuation omits snapshot metadata and reuses the original session. + +### Upstream/migration + +- [HarnessRouter issue #304](https://github.com/HarnessRouter/harnessrouter/issues/304) has a recorded maintainer decision. +- Downstream code is isolated behind initializer, explicit-cursor journal, and recoverable-finalization interfaces compatible with the proposal. +- Accepted upstream patches contain no AllAgents source model. +- Only an upstream release containing every required seam triggers deletion of corresponding forked implementations; remaining deltas stay explicit. ### Operator -- Supply provider/UHP/GHCR/deployment secrets only through the existing secret - mechanism and retain protected settings. -- Review and merge the implementation PR, publish/read back one candidate, - deploy by digest, and run the single full release/restart matrix. -- Accept release only when the digest, SBOM, provenance, deployment, and E2E - report identify the same image and no secret or private workspace metadata is - exposed. +- Supply secrets through protected mechanisms only. +- Maintain snapshot authorization catalog, registry retention, cache/quota policy, and production filesystem capability. +- Publish and deploy by digest. +- Accept release only when builder/gateway digests, SBOMs, provenance, compatibility record, upstream delta, and E2E evidence identify the same tested artifacts. diff --git a/docs/research/allagents-gateway-snapshot-boundary.md b/docs/research/allagents-gateway-snapshot-boundary.md new file mode 100644 index 00000000..6e5d6cc8 --- /dev/null +++ b/docs/research/allagents-gateway-snapshot-boundary.md @@ -0,0 +1,367 @@ +# Prebuilt immutable workspace snapshots at the HarnessRouter boundary + +## Decision + +AllAgents Gateway should consume **one prebuilt immutable workspace snapshot**, while a separate preparation plane owns Git resolution, OCI acquisition, credentials, multi-repository composition, source policy, provenance generation, and snapshot publication. + +The execution boundary should be one direct, digest-pinned OCI image-manifest descriptor. The gateway should never receive Git URLs, refs, per-repository destinations, source credentials, tags, indexes, or caller-selected registry locations. It should authorize the descriptor against one operator-configured snapshot repository, materialize a private writable session tree, establish an exact filesystem-manifest cursor, and then enter HarnessRouter's existing turn lifecycle. + +This should be implemented in **two source repositories**: + +1. `allagents-workspace-builder`: product-specific Git/OCI/multi-repository preparation and immutable snapshot publication. +2. `allagents-gateway`: the stock-derived execution distribution, carrying only a generic immutable-snapshot initializer, explicit-cursor Git-independent journal, and recoverable turn finalizer until equivalent seams land upstream. + +This recommendation reverses the current runtime-composition boundary in [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) and its [implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). + +## Evidence points + +The accepted baseline is HarnessRouter commit [`5f82db1`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3). The current source point inspected here is [`8f7868c`](https://github.com/HarnessRouter/harnessrouter/commit/8f7868ccb2c97d1f611acf11e7cad0357a43064e), seven commits later in the [comparison](https://github.com/HarnessRouter/harnessrouter/compare/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3...8f7868ccb2c97d1f611acf11e7cad0357a43064e). Release [`v0.25.6`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.6) points to `fbcb731`; `8f7868c` is a later commit. HarnessRouter code claims below therefore use pinned `8f7868c` links, with the accepted baseline cited separately. + +The inspected baseline and current source retain the same relevant workspace architecture: a per-session directory, internal checkpoint hydrate/tar routes, root-Git produced-file cursoring, gateway capture-before-ack, and one-file `BACKING.workspace` access. Compare the baseline runner's [`/hydrate`, `/checkpoint`, `_produced_list`, and `_produced_ack`](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L6968-L7176) with the current implementations cited below. + +## Three meanings of workspace + +These terms must not be collapsed: + +| Term | Meaning | Owner | +| --- | --- | --- | +| HarnessRouter product **Workspace** | Security/product-integration boundary containing API keys, configured agents, sessions, and returned files | HarnessRouter control plane | +| UHP/session filesystem workspace | Working directory and file namespace shared by the responses in one session | HarnessRouter session/runner | +| Repositories in the filesystem | Ordinary directory trees that may contain independent `.git` metadata | Snapshot preparation and tools operating in the session | + +HarnessRouter's product documentation defines a Workspace as “the boundary for one product integration” and says API keys, configured agents, sessions, and files live in it; it also tells integrating products to retain their own user, tenant, session, response, and artifact records ([Workspace docs](https://www.harnessrouter.ai/docs/workspace)). That product object is not the runner's `/workspace` directory. + +UHP defines a session as a chain of responses sharing conversational context and a working directory, and a container as the session's file namespace ([UHP architecture](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/architecture.md#L67-L91)). Continuation through `previous_response_id` must use the same session, working directory, files, and configured harness ([UHP sessions](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/sessions.md#L7-L31)). + +HarnessRouter does not model repositories as UHP objects. Its Community Edition README promises native filesystem, shell, and Git workflows with separate session workspaces, and says self-hosted sessions use separate workspaces and operating-system users rather than separate containers ([pinned README](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/README.md#L290-L329)). A repository is content inside the session filesystem, not another HarnessRouter Workspace or session. + +## Stock HarnessRouter behavior + +### Session directory and turn sequence + +The runner derives a workspace from the session identifier. Hosted per-session sandboxes use `WORKSPACE_ROOT` itself; the shared self-hosted runner uses a sanitized per-session subdirectory and optionally a per-session UID write wall ([`WORKSPACE_ROOT`, `_ws`, and isolation invariants](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L78-L171)). + +The gateway's turn sequence is hydrate, refuse execution if an existing checkpoint could not be restored, then launch the runner turn; the runner writes attached input files immediately before the harness starts ([`_resp_execute`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L6816-L6888), [runner `/turn`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L7286-L7335)). Snapshot initialization must gate execution at this boundary. + +### Checkpoint and continuation + +The internal runner `POST /hydrate` spools a gzip tar body to disk, wipes the session directory only after the full body arrives, runs `tar xzf` into the directory, and then calls `_git_ensure`. Empty input creates a fresh workspace ([runner `/hydrate`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L6982-L7062)). `GET /checkpoint` calls `git add -A`, creates an allow-empty commit, and tars the whole directory subject to `CHECKPOINT_EXCLUDE` ([runner `/checkpoint`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L7065-L7110)). `.git` is not excluded; Git history travels in the tarball ([checkpoint exclusions and `_git_ensure`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L452-L631)). + +The gateway streams the runner checkpoint to `sessions/{sid}/workspace.tgz`, stores its SHA only after the blob write succeeds, and restores that blob before a later turn. A transient blob or runner failure does not silently become an empty workspace ([checkpoint/hydrate relays](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L1497-L1615), [`_hydrate` and `_checkpoint`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L2074-L2195)). `HarnessSession` graph state is the durable, replica-independent session record ([`_vertex_upsert` and `_vertex_get`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L2203-L2225)). + +This machinery supports continuation after a snapshot has been initialized, but it is not a public snapshot-import protocol. The runner route consumes HarnessRouter's own checkpoint shape, has no OCI descriptor, size, digest, provenance, or authorization contract, and directly extracts an internally supplied tar. It must not be exposed as an untrusted northbound upload. + +### Produced-file journal + +`_git_ensure` creates or reuses a Git repository at the session root and overwrites the root `.gitignore`. The repository has one HarnessRouter reader, `/produced`; the directory tar, not Git, is durable storage ([`_git_ensure`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L596-L631)). + +`_produced_list` combines `git diff --name-status refs/hr/collected` and `git status --porcelain -uall`. `_produced_ack` stages and commits the current root tree, then advances `refs/hr/collected`. `_produced_keep` deliberately drops deleted paths and runtime noise ([collection cursor and produced routes](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L7111-L7205)). The gateway fetches each listed regular file through `/file`, stores it as an artifact, and calls `/produced/ack` only after capture; if acknowledgement fails, the cursor remains behind and the next collection can retry ([`_collect_produced`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L6659-L6734)). Capture-before-ack is worth preserving. + +`BACKING.workspace` is not a snapshot hook. Its protocol reads or writes one path. `RunnerWorkspaceFiles` proxies a live runner file, while `CheckpointWorkspaceFiles` rewrites one member in a stored checkpoint tar ([`WorkspaceFiles`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/backing.py#L76-L99), [implementations](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/backing.py#L389-L535)). It should remain the application/files seam, not be stretched into acquisition. + +## Stock capability verdict + +| Required property | Stock result | Why | +| --- | --- | --- | +| Public import of one prepared large snapshot | **No** | UHP file input is inline bytes or an uploaded file; public session/file endpoints expose artifacts and archives, not session-root initialization ([UHP Files](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/files.md), [official session/file endpoints](https://www.harnessrouter.ai/docs/sessions-and-files)). `/hydrate` is internal. | +| Preserve several nested `.git` histories as bytes | **Conditionally yes after unsupported injection** | The full-directory tar includes `.git`, so nested repositories survive checkpoint/hydrate. A `.git` at workspace root is commandeered by `_git_ensure`, which writes `.gitignore`, stages, commits, and adds `refs/hr/collected`. | +| Exact changes across several repositories | **No** | Root Git sees an embedded repository as a repository boundary/gitlink, not recursively tracked files, and stock deliberately discards deletions. | +| Checkpoint and continuation | **Yes after initialization, with qualifications** | The tar and UHP session machinery preserve the session. Stock re-archives the entire snapshot each checkpoint and excludes dependency/scratch names such as `node_modules`, `.venv`, and `venv`, so arbitrary prepared content is not an exact round trip without snapshot-specific policy. | +| Reusable immutable snapshot cache | **No** | The warm probe verifies only that one live runner still holds one session's exact `ws_sha`; durable blobs are session-keyed. There is no descriptor-keyed cross-session cache ([`_ws_blob`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L1497-L1504), [`_hydrate`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L2074-L2158)). | + +### Why root Git cannot evaluate multiple repositories + +Git defines a submodule as one repository embedded inside another, with independent history; the superproject records only a gitlink containing the expected commit ([`gitsubmodules`](https://git-scm.com/docs/gitsubmodules/2.52.0)). Git also recognizes an old-form submodule whose working directory contains an embedded `.git` directory. When `git add` encounters an embedded repository without `git submodule add`, it warns because the outer index records an embedded repository rather than ordinary descendant files ([`git-add`](https://git-scm.com/docs/git-add/2.54.0#Documentation/git-add.txt---no-warn-embedded-repo)). + +The superproject's short status reports only that a nested repository's HEAD changed (`M`), it has modified content (`m`), or it has untracked content (`?`); modified and untracked files inside the nested repository cannot be added through `git add` in the superproject ([`git-status`](https://git-scm.com/docs/git-status/2.53.0#_short_format)). `git diff` defaults to ignoring submodules; even `--ignore-submodules=none` reports whether a submodule is dirty or at another HEAD, not a canonical workspace-wide per-file add/modify/delete manifest ([`git-diff`](https://git-scm.com/docs/git-diff/2.55.0#Documentation/git-diff.txt---ignore-submodulesnoneuntrackeddirtyall)). + +A prepared workspace may contain multiple self-contained `.git` histories as data, but HarnessRouter's stock root-Git cursor cannot be the exact change-reporting engine. If a source repository occupies the workspace root, HarnessRouter mutates that repository to operate its cursor. + +## Recommended snapshot artifact + +The preparation plane should publish one OCI artifact and return one direct descriptor: + +```json +{ + "media_type": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:<64 lowercase hex>", + "size": 123456 +} +``` + +An OCI descriptor's required `mediaType`, `digest`, and `size` provide type, content identity, and a pre-processing length check. Consumers should verify size and digest before expensive processing, and OCI requires SHA-256 verification support ([OCI Image Specification 1.1.1 descriptor](https://github.com/opencontainers/image-spec/blob/v1.1.1/descriptor.md)). The Distribution Specification permits retrieving a manifest by digest and says clients should verify that a digest-addressed response matches the requested digest ([OCI Distribution Specification 1.1.1](https://github.com/opencontainers/distribution-spec/blob/v1.1.1/spec.md#pulling-manifests)). The execution request rejects tags and image indexes so platform or tag selection cannot change the admitted bytes. + +Use an OCI image manifest as an artifact container with: + +- custom `artifactType` `application/vnd.allagents.workspace-snapshot.v1`; +- a custom config media type containing the snapshot-format version, final workspace-manifest descriptor, default working directory, preparation implementation/policy identity, and ordered source provenance; +- ordered layer descriptors that the AllAgents snapshot format defines as standard OCI filesystem changesets applied to an empty directory. + +OCI permits non-container content to be packaged with an image manifest and permits an unknown/custom config media type to represent arbitrary artifact metadata ([artifact guidance](https://github.com/opencontainers/image-spec/blob/v1.1.1/artifacts-guidance.md), [manifest config rules](https://github.com/opencontainers/image-spec/blob/v1.1.1/manifest.md#image-manifest-property-descriptions)). OCI layers are changesets, not tarballs to concatenate or blindly extract: consumers must apply ordered additions, modifications, whiteouts, opaque-directory behavior, and replacement semantics ([OCI layer specification](https://github.com/opencontainers/image-spec/blob/v1.1.1/layer.md#applying-changesets)). + +The digest identifies the serialized manifest and everything it references; it is not human-auditable source provenance or a canonical final-tree identity. Put the composition record in the digest-covered config, including each repository's normalized identity, requested selector, resolved commit, history completeness, destination, preparation implementation, and the final tree-manifest digest. Keep the exact manifest descriptor as execution identity and expose the final tree digest separately as baseline identity. + +Do not rely on an OCI `subject` relationship alone for provenance or signatures. The OCI manifest specification calls `subject` a weak association, and referrers may be added independently after the subject artifact exists ([OCI manifest `subject`](https://github.com/opencontainers/image-spec/blob/v1.1.1/manifest.md#image-manifest-property-descriptions)). Referrer signatures and attestations are useful additional evidence, but source provenance required to interpret the workspace must be digest-covered by the admitted manifest. + +To preserve multiple Git histories, the preparation plane packages self-contained `.git` directories as snapshot content after removing credentials, unsafe alternates, transient locks, and acquisition-only state. HarnessRouter neither interprets nor rewrites those repositories. A Git-free repository tree is equally valid. + +## Preparation plane versus execution plane + +```mermaid +flowchart LR + C[Product or benchmark adapter] -->|Git/OCI/multi-repo request| P[Workspace preparation API] + P -->|resolve, compose, verify| S[Immutable staging tree] + S -->|publish config + layers + manifest| R[OCI registry] + P -->|exact manifest descriptor| C + C -->|UHP task + descriptor| G[AllAgents Gateway] + G -->|authorize and initialize| I[Generic snapshot backend] + I -->|pull/cache by digest| R + I -->|private reflink/copy| W[Session workspace] + G --> H[Existing HarnessRouter turn lifecycle] + H -->|manifest changes + artifacts| C +``` + +Preparation should be asynchronous when expensive: submit preparation, poll or receive completion, then invoke UHP with the returned descriptor. HarnessRouter does not call back into the product-specific preparation API during a task. This keeps preparation availability and Git credentials out of the execution critical path after publication. + +| Concern | Compose Git/OCI inside HarnessRouter | Prebuild one snapshot, then execute | +| --- | --- | --- | +| Northbound request | Product-specific source list | One immutable descriptor | +| Runtime credentials and egress | Git and registry credentials, DNS, ref resolution, redirects, helpers, and source policy | Snapshot-registry read only | +| HarnessRouter changes | Source schema, resolvers, Git/OCI clients, per-component caches, destination rules, mount/copy lifecycle, provenance, expiry/reconciliation | Generic initializer, descriptor binding, journal, optional cwd | +| Cache identity | Per-component keys plus composition state | One final manifest key; OCI blob/layer reuse remains available underneath | +| First-task latency | Acquisition and composition happen on task start | Moved to preparation; task start is pull/cache/materialize | +| Failure surface | Partial source resolution and attachment must reconcile with session state | Preparation fails before UHP; execution sees only published immutable artifacts | +| Upstreamability | Low: Git/OCI/source policy is AllAgents product logic | High: immutable workspace initialization is execution-runtime plumbing | +| Cost | No separate preparation API, but a large permanent fork | Separate contract; snapshots need retention and garbage collection | + +A one-snapshot execution cache cannot independently swap one component at runtime. That reuse stays in preparation and the registry: a builder may reuse Git mirrors, resolved trees, and unchanged OCI blobs/layers while publishing a new final manifest. The executor treats the result as one atomic filesystem. + +## Immutable base plus private writable session + +“Immutable snapshot” describes the reusable baseline, not the agent's live workspace. The harness needs a private writable view. + +| Mechanism | Assessment | +| --- | --- | +| Per-file reflink tree clone from an immutable unpacked cache | **Recommended first choice.** Linux `FICLONE` shares physical data copy-on-write, requires source and destination on the same filesystem, and keeps later writes private ([`ioctl_ficlone(2)`](https://man7.org/linux/man-pages/man2/ioctl_ficlonerange.2.html)). Walk without following links, create new directory entries/inodes, reflink regular files, and never hardlink mutable files. | +| Full private copy | **Required fallback.** Portable and leaves a normal directory, at startup I/O and space cost. | +| OverlayFS with immutable lower and per-session upper/work directories | **Defer.** OverlayFS supports a non-writable lower, writable upper, whiteouts, and copy-up ([Linux v6.17 OverlayFS documentation](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/Documentation/filesystems/overlayfs.rst?h=v6.17)). It expands checkpoint, hydrate, archive, deletion, mount, and crash-cleanup work. | +| Read-only bind mount of the whole snapshot | **Reject for editable sessions.** Mounts require privilege and namespace care; HarnessRouter writes input, instruction, and `.harness` state into the workspace ([`mount(2)`](https://man7.org/linux/man-pages/man2/mount.2.html), [`mount_namespaces(7)`](https://man7.org/linux/man-pages/man7/mount_namespaces.7.html)). | +| Shared writable tree, hardlinks, or symlink to cache | **Reject.** A session could mutate cache or sibling state. | + +A private reflink/copy leaves checkpointing and `BACKING.workspace` operating on an ordinary directory. It is the smallest correct boundary. It means stock full-tar checkpoints re-archive the baseline on cold continuation; measure that cost before adopting OverlayFS or base-plus-delta checkpoints. The runner already streams large checkpoints through disk rather than buffering them in memory, so this is primarily I/O/storage cost, not a reason to redesign UHP. + +The unpacked cache key is the verified manifest descriptor plus one materializer/schema revision. Publication is immutable and complete before readers claim it. The cache never contains a live session's writable tree. + +## Minimal northbound extension + +UHP permits additional request/response metadata and recommends vendor prefixes; extensions must not redefine specified fields or add a required field ([UHP schema extension points](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/schema.md#L47-L58)). HarnessRouter models create-response metadata as an open dictionary, and its OpenAPI schema permits additional metadata properties ([`CreateResponseBody`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L7524-L7539), [pinned OpenAPI schema](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/schema/uhp-2026-09-12.openapi.yaml#L1041-L1082)). + +Use a first-turn-only vendor extension: + +```json +{ + "input": "Make the requested change.", + "metadata": { + "allagents_workspace_snapshot": { + "version": 1, + "descriptor": { + "media_type": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:...", + "size": 123456 + } + } + } +} +``` + +The registry, repository, credentials, redirects, and trust policy are server configuration. The caller cannot select them. The digest-covered snapshot config supplies and binds the default working directory and provenance. Continuations omit this metadata; repeating or replacing the descriptor on a continuation fails before hydrate or materialization. + +Advertise support as an additional vendor capability, treated as false when absent, consistent with UHP discovery's extensible named-boolean capability model ([UHP capability discovery](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/lifecycle.md#L29-L83)). Do not claim the extension is part of UHP 2026-09-12. + +## Minimal upstream-neutral hook and durable state + +The upstream proposal should not mention Git, repository arrays, source credentials, composition, catalog entries, or AllAgents provenance fields. It needs one optional immutable-workspace initializer and one optional non-Git journal mode. + +Conceptual interface: + +```text +WorkspaceInitializer.initialize( + principal, + session_id, + immutable_descriptor, + empty_session_root +) -> InitializedWorkspace + +InitializedWorkspace = { + verified_descriptor, + tree_digest, + working_directory, + initializer_schema, + journal_mode +} +``` + +Required semantics: + +- descriptor immutable and verified before content is consumed; +- initializer configured by the operator, never selected by caller; +- destination is an empty private session root; +- success all-or-nothing before input files or harness start; +- retries for the same session/descriptor idempotent; +- failure leaves no runnable partial tree; +- returned working directory relative, confined, and verified as a directory; +- hook returns no credentials, registry URL, host/cache path, mount identity, or product source plan. + +The durable `HarnessSession` extension contains only: + +```text +workspace_snapshot = { + state: pending | ready | failed, + descriptor: { media_type, digest, size }, + verified_tree_digest, + working_directory, + initializer_schema, + journal_cursor_blob_digest +} +``` + +Keep existing `ws_sha`, checkpoint blob, response/artifact records, and control leases separate. The durable binding also needs a private immutable repository/catalog/signature-policy selector so restart can reauthorize and refetch the original tuple; the public descriptor alone is insufficient. Do not persist cache paths, mount IDs, inode numbers, attachment flags, registry credentials, or preparation-service records. + +For exact reporting, the gateway-owned cursor blob is the sole durable authority. The gateway supplies its explicit base cursor to the runner's manifest journal; the runner returns the next cursor and an idempotent collection token without owning durable cursor state. A recoverable turn commit captures artifacts and an exact checkpoint, verifies streamed file hashes and checkpoint state against the next manifest, acknowledges the token, then atomically publishes gateway cursor/checkpoint/artifact pointers and terminal visibility. This preserves capture-before-ack without a hidden runner/gateway cursor split. + +## Smallest required HarnessRouter changes + +### Gateway + +1. Parse and bind the extension in the existing create-response path. Validate first-turn-only use after session resolution but before hydration; bind the exact descriptor plus private repository/catalog authorization subject to `HarnessSession`. +2. Gate execution around `_resp_execute`/`_hydrate`. A new snapshot session performs empty hydrate/wipe, initialization, input/control preparation, captures a canonical initialization delta from builder snapshot to ready tree, stores the ready cursor, and transitions `pending -> ready` before `/turn`. Continuation restores checkpoint and cursor. +3. Return the verified descriptor, snapshot and ready manifest digests, provenance digest, initialization-change artifact, working directory, retention deadline, and expiry after ready; replay uses stored values. +4. Extend `_collect_produced`, not public Files endpoints. Supply the ready/acknowledged base cursor to the runner, hash changed files while capturing, verify the exact checkpoint, then acknowledge and publish terminal state through a recoverable turn commit. +5. Release initializer/cache references on session deletion and reconcile every preparing/durable/acknowledged/committed/aborted turn state idempotently. + +### Runner + +1. Add a trusted internal initialization route or equivalent hydrate mode that invokes the configured initializer. Do not reuse raw checkpoint `/hydrate` as a public OCI importer. +2. Do not call `_git_ensure` for snapshot-backed sessions. `/hydrate`, `/checkpoint`, `/produced`, and `/produced/ack` need a session journal/checkpoint mode. +3. Use an explicit-cursor manifest journal behind `/produced` and `/produced/ack`; keep root Git for stock sessions. +4. Allow a validated relative working directory beneath `_ws(identifier)` while retaining the session UID ([runner `/turn`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L7286-L7335)). +5. Make snapshot-mode checkpoint exclusions exact and versioned. Stock dependency exclusions cannot silently remove declared files. + +### What remains stock + +Keep UHP versioning/authentication, idempotency, session and response identity, provider routing, harness adapters and supervision, SSE translation, cancellation, graph/blob backing, warm `ws_sha` probe, checkpoint relays, file/artifact endpoints, `BACKING.workspace`, capture-before-ack ordering, explicit session deletion, and ordinary-session root-Git behavior. + +With a normal private reflink/copy, checkpoint/hydrate can remain full-directory tar operations after two snapshot-mode adjustments: skip root Git and use an exact compatible exclusion policy. OverlayFS would expand changes to mount-aware hydrate, checkpoint, archive, file access, deletion, reaping, and crash recovery; it is not the minimum. + +## Two repositories versus one + +Two repositories are the better boundary. + +**Why two wins:** + +- Preparation owns product policy, source credentials, Git behavior, OCI composition, provenance, asynchronous jobs, and snapshot publication. None is HarnessRouter execution logic. +- It can scale, release, and cache independently from latency-sensitive execution. +- The gateway fork becomes narrow enough to propose upstream without asking HarnessRouter to adopt AllAgents source semantics. +- Migration to stock becomes possible only after upstream exposes initializer, explicit-cursor journal, and recoverable finalization seams; `allagents-gateway` can then become a thin distribution or disappear, while preparation remains unchanged. +- The OCI descriptor is a stable cross-repository contract and release boundary. + +**Costs:** two components require contract versioning, availability/retention ownership, integration tests, and a compatibility matrix. Preparation must publish fully before returning a descriptor; execution must fail clearly if a retained digest disappears. These obligations are smaller and better isolated than permanent Git/OCI product-policy code in a HarnessRouter fork. + +A monorepo with two binaries would reduce atomic-edit friction, but would keep source-product lifecycle coupled to the upstream-derived repository and make returning to stock a source-tree surgery. Given the explicit upstream-migration goal, separate repositories are preferable. + +## Upstream proposal + +Propose three generic changes to HarnessRouter: + +1. **Optional immutable workspace initialization hook** + - registered by operator configuration; + - invoked once after a fresh session root is established and before inputs/harness start; + - receives an immutable descriptor and returns verified identity, relative cwd, and journal mode; + - persists a minimal pending/ready binding on `HarnessSession`; + - advertises one optional capability; + - leaves requests without the extension on the stock path. + +2. **Pluggable explicit-cursor journal behind `/produced` and `/produced/ack`** + - existing root Git remains default; + - gateway supplies the authoritative base cursor; + - a generic manifest journal supports non-Git and multi-repository workspaces; + - capture-before-ack remains in the gateway; + - hook has no AllAgents artifact schema or source model. + +3. **Recoverable terminal-finalization seam** + - orders changed-file artifacts, change artifact, exact checkpoint, next cursor, journal ACK, and terminal response; + - exposes terminal state only after durable checkpoint/cursor evidence and ACK; + - resumes crashes without rerunning the harness. + +Focused tests should prove initialization before inputs/harness, idempotent duplicate first requests, continuation with the same binding, descriptor-replacement rejection, explicit cursor handoff, restore failure that does not run or overwrite a checkpoint, exact add/modify/delete behavior excluding `.git`, checkpoint/cursor durability before terminal visibility, and unchanged stock behavior. UHP itself need not change because vendor-prefixed metadata is already an extension point. The proposal is tracked in [HarnessRouter issue #304](https://github.com/HarnessRouter/harnessrouter/issues/304). + +## Migration from fork to stock + +1. Implement preparation and the versioned OCI snapshot format in `allagents-workspace-builder`. +2. In the existing fork, isolate execution changes behind `WorkspaceInitializer`, `WorkspaceJournal`, and `WorkspaceTurnFinalizer`; keep Git/OCI composition outside. +3. Submit generic initializer, explicit-cursor journal, and recoverable-finalization seams upstream with stock defaults and no AllAgents source model. +4. While review is pending, ship the same interfaces in `allagents-gateway`; keep the OCI backend and metadata adapter separate from copied HarnessRouter logic. +5. When upstream contains equivalent seams, rebase onto that release and delete only superseded fork implementations rather than preserving aliases. +6. Make `allagents-gateway` consume stock HarnessRouter plus backend packaging once every required seam is upstream. If upstream supports external backend loading, stop maintaining a source fork. +7. Retain cross-version tests for stock UHP, first-turn initialization, exact changes, finalization, continuation, and descriptor-keyed cache reuse before dropping the fork. + +## Rejected alternatives + +| Alternative | Reason | +| --- | --- | +| Keep Git/OCI/multi-repository composition inside AllAgents Gateway | Couples product source policy and credentials to HarnessRouter lifecycle and produces the largest, least upstreamable patch. | +| Upload a prepared tar through stock `/hydrate` | Internal checkpoint route without OCI identity/provenance/import policy; still invokes root Git and has no cross-session cache or exact nested-repository changes. | +| Make the agent clone repositories | Acquisition occurs after harness start, exposes credentials/network policy to agent code, and cannot establish a verified pre-turn baseline. | +| Use one repository at workspace root and nested repos below it | HarnessRouter mutates the root repository, while Git reports nested repositories only coarsely and stock omits deletions. | +| Use a shared writable unpacked snapshot or hardlinks | A session can mutate cache or sibling state. | +| Adopt OverlayFS immediately | Expands every filesystem lifecycle seam before a measured need. Start with reflink/private copy. | +| Keep preparation and execution in one source repository | Easier atomic edits, but undermines independent ownership and migration from a fork to stock. | +| Put required provenance only in OCI referrers | `subject` is a weak association and referrers do not contribute to admitted snapshot manifest digest. | +| Accept tags or indexes at execution | Selection can change independently of request; execution receives one direct manifest descriptor. | + +## Conclusion + +Stock HarnessRouter can continue a private tree once it is inside its checkpoint lifecycle, and its tar format generally carries nested `.git` bytes. It cannot, as a supported stock product, import a prepared immutable large snapshot with untouched multiple repository histories, exact workspace-wide changes including deletions, reusable descriptor-keyed cache, and a public immutable provenance contract. + +The smallest robust change is a generic first-turn snapshot initializer, an explicit-cursor Git-independent journal, and recoverable terminal finalization, feeding an ordinary private writable session directory while retaining the rest of HarnessRouter. Put every mutable source and composition concern in a separate preparation repository, publish one OCI descriptor, and use that descriptor as the execution-plane source identity. + +## Primary sources + +### HarnessRouter and UHP + +- [HarnessRouter Workspace documentation](https://www.harnessrouter.ai/docs/workspace) +- [HarnessRouter sessions and files documentation](https://www.harnessrouter.ai/docs/sessions-and-files) +- [HarnessRouter run-task boundary](https://www.harnessrouter.ai/docs/run-a-task) +- [HarnessRouter release v0.25.6](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.6) +- [HarnessRouter commit `8f7868c`](https://github.com/HarnessRouter/harnessrouter/commit/8f7868ccb2c97d1f611acf11e7cad0357a43064e) +- [Pinned runner source](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py) +- [Pinned gateway source](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py) +- [Pinned backing abstractions](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/backing.py) +- [UHP 2026-09-12 architecture](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/architecture.md) +- [UHP 2026-09-12 lifecycle](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/lifecycle.md) +- [UHP 2026-09-12 sessions](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/sessions.md) +- [UHP 2026-09-12 files](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/files.md) +- [UHP 2026-09-12 schema extension rules](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/schema.md) + +### OCI + +- [OCI Image Specification 1.1.1 descriptors](https://github.com/opencontainers/image-spec/blob/v1.1.1/descriptor.md) +- [OCI Image Specification 1.1.1 manifests](https://github.com/opencontainers/image-spec/blob/v1.1.1/manifest.md) +- [OCI Image Specification 1.1.1 layers](https://github.com/opencontainers/image-spec/blob/v1.1.1/layer.md) +- [OCI Image Specification 1.1.1 artifact guidance](https://github.com/opencontainers/image-spec/blob/v1.1.1/artifacts-guidance.md) +- [OCI Distribution Specification 1.1.1](https://github.com/opencontainers/distribution-spec/blob/v1.1.1/spec.md) + +### Git + +- [Git submodule model 2.52.0](https://git-scm.com/docs/gitsubmodules/2.52.0) +- [Git add embedded-repository behavior 2.54.0](https://git-scm.com/docs/git-add/2.54.0#Documentation/git-add.txt---no-warn-embedded-repo) +- [Git status submodule behavior 2.53.0](https://git-scm.com/docs/git-status/2.53.0#_short_format) +- [Git diff submodule behavior 2.55.0](https://git-scm.com/docs/git-diff/2.55.0#Documentation/git-diff.txt---ignore-submodulesnoneuntrackeddirtyall) + +### Linux filesystem behavior + +- [Linux `FICLONE`/`FICLONERANGE`](https://man7.org/linux/man-pages/man2/ioctl_ficlonerange.2.html) +- [Linux v6.17 OverlayFS documentation](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/Documentation/filesystems/overlayfs.rst?h=v6.17) +- [Linux `mount(2)` bind/read-only behavior](https://man7.org/linux/man-pages/man2/mount.2.html) +- [Linux mount namespaces](https://man7.org/linux/man-pages/man7/mount_namespaces.7.html) +- [Linux recursive mount attributes](https://man7.org/linux/man-pages/man2/mount_setattr.2.html) diff --git a/docs/research/e2b-execution-gateway-patterns.md b/docs/research/e2b-execution-gateway-patterns.md index 22df48cb..6f18306a 100644 --- a/docs/research/e2b-execution-gateway-patterns.md +++ b/docs/research/e2b-execution-gateway-patterns.md @@ -128,7 +128,7 @@ The runtime and dashboard repositories use Apache-2.0 ([runtime license](https:/ **Reject E2B as a replacement for the planned UHP/HarnessRouter gateway. Trial it later only as a stronger sandbox runtime beneath the runner if hostile-code isolation becomes a product requirement.** -The current AllAgents decision is accepted but not implemented: [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) selects UHP `2026-09-12` through a pinned HarnessRouter CE fork, and the [implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md) assigns the wire protocol, caller authentication, normalized streaming, cancellation, idempotency, conversation continuity, harness execution, usage, and artifacts to HarnessRouter. AllAgents owns deterministic Git/OCI materialization and source provenance. Repository inspection found no gateway, fork, materializer, or deployment implementation yet. +The current AllAgents decision is accepted but not implemented: [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) selects UHP `2026-09-12` through a pinned HarnessRouter CE fork, and the [implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md) assigns the wire protocol, caller authentication, normalized streaming, cancellation, idempotency, conversation continuity, harness execution, usage, and artifacts to HarnessRouter. A separate AllAgents Workspace Builder owns Git/OCI preparation and publishes one immutable snapshot before execution. E2B does not implement that contract. It creates an isolated machine and exposes low-level process, filesystem, network, and lifecycle APIs. Its official Codex and Pi integrations leave command construction, harness credentials, event parsing, continuation, and result extraction in caller code. Replacing HarnessRouter with E2B would therefore recreate the custom gateway, session, event-normalization, harness-adapter, and artifact layers that ADR 0002 rejected. @@ -141,13 +141,13 @@ No. The systems overlap at the execution-workspace layer but own different abstr | Northbound contract | UHP request, ordered events, cancellation, idempotency, continuation, files, usage, artifacts, and normalized errors | Sandbox REST API plus process/filesystem APIs; no agent-neutral request/event/result protocol | | Harness execution | HarnessRouter selects and runs configured Codex or Pi targets | Caller starts an agent-specific command inside a sandbox and interprets its output | | Session identity | `previous_response_id` binds conversation, writable workspace, harness target, auth binding, and provenance | Sandbox ID binds machine state; agent thread/session identity remains harness- and caller-specific | -| Workspace acquisition | Server-authoritative logical source catalog; exact Git commits or immutable OCI digests; pre-agent checkpoint; returned provenance | Caller uploads files, clones Git, or starts from a template/snapshot; no agent-run source-provenance contract | +| Workspace acquisition | Separate builder resolves sources and publishes one direct OCI snapshot; gateway authorizes, verifies, privately materializes, and returns immutable manifest/provenance identities before agent start | Caller uploads files, clones Git, or starts from a template/snapshot; no agent-run source-provenance contract | | Isolation | Private per-session UID/workspace; explicitly not a hostile-code sandbox | One Firecracker microVM and guest kernel per sandbox | | Credentials | Separate caller, source, and provider trust domains; native OAuth owner-trust mode or explicit brokered proxy | Core sandbox auth plus caller-supplied agent/Git credentials; managed egress secrets and workload identity are not fully present in standard Embed | | Results | UHP output, usage, artifacts, produced-file collection, and identical provenance across stream/retrieval/replay paths | Guest files, command streams, VM snapshots, and templates; the caller defines an agent result or artifact manifest | | Deployment | Planned pinned private HarnessRouter image with durable sessions and a narrow AllAgents hook | Multi-service KVM stack with API, proxies, orchestrator, guest daemon, three datastores, and template/snapshot storage | -E2B could occupy the runner's future sandbox-runtime slot. HarnessRouter and the AllAgents materializer would still remain above it. +E2B could occupy the runner's future sandbox-runtime slot. HarnessRouter and the separate AllAgents Workspace Builder would still remain above and before it respectively. ### Is it completely self-hosted? diff --git a/docs/research/harbor-repository-materialization.md b/docs/research/harbor-repository-materialization.md index f6caba95..c579cc42 100644 --- a/docs/research/harbor-repository-materialization.md +++ b/docs/research/harbor-repository-materialization.md @@ -2,21 +2,13 @@ ## Decision -Use Harbor's content-addressed package cache, sparse Git reads, staged -publication, and prebuilt-environment model as inputs to the AllAgents workspace -materializer. Keep source selection and provenance in the AllAgents contract -rather than adopting Harbor's task-owned workspace model. - -Harbor does not expose a first-class, general-purpose “repositories in a -workspace” layer. It first downloads a Harbor **task package**. The task then -defines an execution environment with a Dockerfile, Compose file, or prebuilt -image. Acquisition of the repository the agent edits may be baked into an image, -cloned by a Dockerfile, copied as task content, or prepared by the task author. - -AllAgents keeps repository and workspace provenance explicit in the initial -workspace descriptor and response metadata. The materializer accepts only -declared Git repositories and named digest-pinned OCI workspace snapshots; -custom materializers are outside the version-one contract. +Use Harbor's content-addressed package cache, sparse Git reads, staged publication, and prebuilt-environment model in **AllAgents Workspace Builder**. Publish one complete immutable workspace snapshot before invoking AllAgents Gateway. + +Harbor does not expose a first-class general-purpose “repositories in a workspace” layer. It first downloads a Harbor task package. The task then defines an execution environment with a Dockerfile, Compose file, or prebuilt image. Acquisition of the repository the agent edits may be baked into an image, cloned by a Dockerfile, copied as task content, or prepared by the task author. + +AllAgents keeps source selection and provenance explicit in the builder contract. The gateway receives only a direct digest-pinned OCI workspace-snapshot descriptor. Git URLs, mutable refs, source credentials, destinations, and custom preparation behavior do not cross into HarnessRouter. + +This replaces the earlier recommendation to invoke a materializer inside HarnessRouter. The primary-source observations below remain valid; the current boundary is defined by [snapshot-boundary research](./allagents-gateway-snapshot-boundary.md) and [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md). ## What Harbor fetches @@ -143,33 +135,19 @@ through to the other after admission. ## Recommended boundary -The HarnessRouter runner invokes the AllAgents materializer before provider -selection: - -1. HarnessRouter validates generic metadata bounds, creates the session, and - enters the durable materialization state. -2. The materializer validates the workspace descriptor and configured repository - or snapshot identities before source network access. -3. The materializer resolves phase-scoped source credentials without exposing - them to the coding agent or later evidence collection. -4. The materializer populates a fixed staging directory on the publication - filesystem, or pulls and unpacks a digest-pinned workspace snapshot there. -5. The materializer verifies commits, paths, limits, content, the expected - manifest digest, and the standard workspace manifest; snapshot-attested claims - remain distinct from independently verified identities. -6. The materializer stops acquisition processes, removes credentials, helpers, - and mounts, and returns only the validated credential-free staging tree plus - bounded provenance. -7. The runner independently validates staging, publishes it, creates checkpoint - and collection baselines, applies UHP input files, and only then launches the - coding agent. Project or user `setup` shell commands are not run. - -Operator-selected builders and additional source variants require a new decision -for trust, configuration, credential, provenance, and isolation boundaries. - -The practical conclusion is narrow: Harbor is strong evidence for content- -addressed input bundles and staged publication. It is not evidence for making -repository acquisition opaque or task-defined in the AllAgents public contract. +AllAgents Workspace Builder runs before UHP execution: + +1. The builder validates the versioned source plan, destination ownership, configured repository/snapshot identities, and principal authorization before source network access. +2. It resolves phase-scoped source credentials without exposing them to the eventual coding agent or gateway. +3. It populates a private staging tree from exact Git commits and digest-pinned OCI inputs. +4. It verifies commits, history completeness, paths, limits, links, content, the canonical workspace manifest, and digest-covered provenance. +5. It stops acquisition processes and removes credentials, helpers, unsafe Git state, and mounts. +6. It publishes config, provenance, layers, and blobs completely before publishing and returning one direct OCI image-manifest descriptor. +7. AllAgents Gateway authorizes that descriptor, independently verifies/materializes it into a private session tree, establishes checkpoint and collection baselines, applies UHP inputs, and only then launches the coding agent. + +Project or user `setup` shell commands are not run as trusted preparation. Additional source kinds or caller-selected builders require a new decision for trust, credential, provenance, and isolation boundaries. + +The practical conclusion is narrow: Harbor is strong evidence for content-addressed input bundles, isolated staging, and prebuilt publication. It is not evidence for resolving repositories inside the execution gateway or making preparation task-authored and opaque. ## Primary sources diff --git a/docs/research/source-credential-broker-precedents.md b/docs/research/source-credential-broker-precedents.md index 08fbe1ce..8a46c52b 100644 --- a/docs/research/source-credential-broker-precedents.md +++ b/docs/research/source-credential-broker-precedents.md @@ -2,18 +2,11 @@ ## Decision -The initial trusted-network deployment uses deployment-supplied source -credentials referenced from the project `workspace.yaml` as `${ENV_VAR}` values. -The AllAgents materializer receives only the configured credential variables in -its allowlisted child environment. HarnessRouter removes every -materializer-only variable from agent child environments regardless of its name. - -Git credentials are exposed only to the acquisition process through a -short-lived, materializer-owned credential helper or registry-auth channel. -The materializer uses hermetic Git and registry configuration, removes temporary -auth state before returning, and emits no secret in logs, provenance, checkpoints, -or response metadata. It never consults arbitrary ambient credential helpers and -never falls through to a different credential identity after a failure. +The initial trusted-network deployment supplies source credentials only to AllAgents Workspace Builder. Build specifications contain source identities and policy-selected references, never literal credentials. The builder receives only allowlisted credential variables or secret-channel handles in its acquisition process. + +Git credentials are exposed only through a short-lived builder-owned credential helper or registry-auth channel. The builder uses hermetic Git and registry configuration, removes temporary auth state before publication, and emits no secret in logs, provenance, snapshots, descriptors, or response metadata. It never consults arbitrary ambient credential helpers and never falls through to a different credential identity after failure. + +AllAgents Gateway receives no source credentials or Git configuration. It has separate read-only credentials for trusted snapshot repositories, and those credentials never enter the session or harness environment. Git credential helpers, GitHub App installation tokens, and BuildKit secret mounts establish the process- and phase-boundary precedents. The deployment does @@ -174,26 +167,14 @@ secret trustworthy. ### Initial trusted-network deployment -1. Store only `${ENV_VAR}` references in the project workspace configuration; - reject literal credentials and caller-supplied credential identifiers. -2. Supply secret values through the deployment environment and validate required - names during materializer preflight without contacting sources. -3. Pass only the referenced, allowlisted names to the materializer child. Remove - the complete allowlist from every coding-agent child independent of - secret-looking name patterns. -4. Select one configured credential identity before acquisition. Authentication, - authorization, rate-limit, or service failure terminates acquisition and - never falls through to another identity or source mode. -5. Give the credential only to the dedicated acquisition subprocess through a - temporary helper or registry-auth channel. Invoke helpers directly without a - shell and bound their input, output, stderr, and lifetime. -6. Use isolated Git/registry configuration. Prevent credentials from entering - remote URLs, Git config, generated CLI config, workspace files, nested - repositories, checkpoints, logs, provenance, or response metadata. -7. Remove helper files, auth configuration, and the credential-bearing process - before returning the validated staging tree to HarnessRouter. -8. Verify containment with a deliberately non-secret-looking environment name, - because name-based secret filters are not the security boundary. +1. Store only policy-recognized secret references in builder deployment configuration; reject literal credentials and caller-supplied credential identities in build specifications. +2. Supply secret values through the builder's protected environment or secret channel and validate required names during preflight without contacting sources. +3. Pass only the selected allowlisted values to the acquisition child; do not expose them to snapshot packaging, the gateway, or coding-agent processes. +4. Select one configured credential identity before acquisition. Authentication, authorization, rate-limit, or service failure terminates acquisition and never falls through to another identity or source mode. +5. Give the credential only to the dedicated acquisition subprocess through a temporary helper or registry-auth channel. Invoke helpers directly without a shell and bound input, output, stderr, and lifetime. +6. Use isolated Git/registry configuration. Prevent credentials from entering remote URLs, Git config, generated CLI config, workspace files, nested repositories, OCI config/layers, logs, provenance, or descriptors. +7. Remove helper files, auth configuration, and credential-bearing processes before publishing the immutable snapshot. +8. Verify containment with a deliberately non-secret-looking environment name, because name-based secret filters are not the security boundary. ### Remote or multi-tenant deployment diff --git a/docs/research/workspace-contract-incumbents.md b/docs/research/workspace-contract-incumbents.md index 0ed7587a..134a26ee 100644 --- a/docs/research/workspace-contract-incumbents.md +++ b/docs/research/workspace-contract-incumbents.md @@ -2,43 +2,35 @@ ## Decision -AllAgents should **not** replace its execution-gateway workspace descriptor with Harbor, Devfile, Dev Containers, E2B, Daytona, GitHub Codespaces, or Gitpod. No examined contract standardizes the same boundary: caller-selected Git or OCI source, deterministic materialization, attachment before the harness starts, a logical working directory, and immutable resolved provenance returned only after attachment is committed. +No examined incumbent replaces the complete AllAgents preparation and execution stack. The useful separation is now: -The closest portable **source-layout precedent** is Devfile 2.3's `projects` model. The closest field-level operational API is Daytona's Git clone operation. GitHub Codespaces and Gitpod Classic are stronger examples of products that bind source acquisition to workspace lifecycle, but both are provider-specific. None is a compatible normative replacement. +1. **Northbound execution:** UHP remains the sole request/response protocol. +2. **Source layout:** an AllAgents Workspace Builder contract owns Git/OCI inputs, destinations, history selection, credentials, and composition before execution. +3. **Immutable handoff:** one direct OCI workspace-snapshot descriptor crosses into execution. +4. **Runtime:** AllAgents Gateway initializes a private session tree and retains HarnessRouter's UHP/session lifecycle. -The recommended contract stack is therefore: +Devfile 2.3 remains the closest portable source-layout precedent. Daytona, Codespaces, and Gitpod remain provider-specific operational precedents. Harbor's prebuilt environments and content-addressed packages are stronger evidence for the new build-then-execute boundary than for runtime composition. -1. **Northbound execution protocol:** UHP remains the sole request/response protocol. -2. **Workspace/source descriptor:** retain `metadata["allagents.workspace"]` version 1 as an AllAgents-owned extension. -3. **Runtime sandbox:** keep provider APIs behind the gateway; Harbor ASP is a promising future runtime seam, not a workspace descriptor. -4. **Immutable artifacts:** use Git commit identity and OCI Image Specification descriptors/manifests as the normative identities, while retaining the AllAgents workspace manifest and attachment result as the binding provenance record. - -Devfile should be cited as design precedent for repository URL/revision/destination concepts, not claimed as an implemented profile or conformance target. +This note's incumbent comparisons remain useful. Its earlier recommendation to send caller-selected Git/OCI sources directly to the gateway is superseded by [Prebuilt immutable workspace snapshots at the HarnessRouter boundary](./allagents-gateway-snapshot-boundary.md) and [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md). ## The four contracts are different | Layer | AllAgents boundary | Best established precedent | Assessment | | --- | --- | --- | --- | -| Northbound execution | UHP requests, events, results, cancellation, and continuation | UHP | Already selected. A workspace standard should not displace the execution protocol. | -| Workspace/source descriptor | `{url, ref?, destination}` or an OCI snapshot, plus `workingDirectory` | Devfile `projects` is the closest portable schema; Codespaces/Gitpod are product precedents | No incumbent covers AllAgents' complete semantics. Keep the extension. | -| Runtime sandbox | Process, filesystem, network, and lifecycle implementation behind the gateway | Harbor ASP, E2B, Daytona, Dev Containers | These contracts start at or after sandbox provisioning. They can inform or implement the southbound seam without becoming the northbound source contract. | -| Immutable artifacts | Resolved Git commit or OCI manifest digest, canonical workspace manifest, committed attachment metadata | Git object identity and OCI Image Specification 1.1.1 | Adopt the artifact standards directly. No workspace incumbent supplies the complete result record. | +| Northbound execution | UHP requests, events, results, cancellation, continuation, files, and artifacts | UHP | A workspace standard should not displace the execution protocol. | +| Preparation | Builder-owned Git/OCI source plan and deterministic composition | Devfile `projects`, Daytona clone operations, Harbor prebuilds | No incumbent covers the complete source/history/provenance contract; keep it outside HarnessRouter. | +| Immutable handoff | One direct OCI workspace-snapshot descriptor with digest-covered manifest/provenance | OCI Image Specification 1.1.1 | Adopt OCI identity directly and reject tags/indexes at execution. | +| Runtime sandbox | Private writable session tree behind AllAgents Gateway | HarnessRouter, Harbor ASP, E2B, Daytona | Runtime providers stay southbound and do not become the source contract. | -This separation matters. Choosing one product contract across all four layers would either expose provider operations northbound or weaken the source and provenance guarantees already specified in [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) and the [execution-gateway implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). +This separation avoids exposing provider operations northbound or source credentials to the execution plane. -## Current AllAgents contract to preserve +## Historical contract evaluated -The current design has a small request surface and a comparatively strong result contract: +The comparisons below originally evaluated a runtime source descriptor with repository URLs, refs, destinations, OCI snapshots, logical working directory, pre-agent materialization, and immutable returned provenance. -- A repository source has a canonical public HTTPS `url`, optional `ref`, and required unique non-root relative `destination`. A request may contain multiple repositories. -- Omitted `ref` means the remote symbolic HEAD; a supplied ref is resolved fail-closed to a full commit. The result preserves both requested and resolved identities. -- An OCI workspace source is selected by immutable image/workspace-manifest digests from an operator-owned snapshot catalog. -- `workingDirectory` is a logical union: `workspaceRoot`, or `workspacePath` with a relative path that must be a directory in the verified workspace manifest. It is not a host path or provider mount path. -- Source credentials and policy are server-owned and limited to acquisition. They are not request fields or agent environment variables. -- Materialization and attachment complete before initial UHP files or the provider process are admitted. Public provenance is emitted only after the gateway has committed a `ready` attachment and the runner has acknowledged or reserved it. -- Continuations reuse the exact descriptor, attachment, access mode, retention, working directory, harness, and authorization context rather than accepting a new source request. +Those properties remain requirements, but ownership changed: source selection, ref resolution, credentials, destination composition, and full-history verification now belong to AllAgents Workspace Builder. AllAgents Gateway receives one already-published direct manifest descriptor, verifies its digest-covered working directory and provenance identities, and never receives the source plan. -The comparison below treats those properties as requirements rather than matching field names alone. +The comparison below evaluates semantic coverage rather than current field placement. ## Incumbent comparison @@ -142,19 +134,20 @@ For Git, a full commit object ID is the resolved source identity. The request st Adopt the following rule for future changes: -- **Normative:** UHP northbound; AllAgents workspace extension for acquisition and attachment; Git commit identity and OCI Image Specification 1.1.1 for immutable artifacts. -- **Benchmark compatibility:** ingest Harbor task packages and SWE-bench/Hugging Face records through adapters. Preserve Harbor's task/environment/verifier split and its preference for prebuilt OCI environments; map benchmark source identities into the canonical AllAgents descriptor. -- **Source-layout precedent:** use Devfile 2.3 `projects` semantics when adding or naming direct repository fields. Document intentional divergence, especially for URL shape, revision behavior, and destination paths. -- **Runtime precedent:** evaluate Harbor ASP as a southbound execute/filesystem adapter when its draft stabilizes. E2B and Daytona remain provider adapters. Dev Containers may define an optional environment-building layer after source acquisition. -- **Do not conflate:** Harbor's benchmark repository with the target application source; a runnable environment image with a workspace source snapshot, even when the snapshot preserves Git history; or a vendor sandbox/codespace object with the northbound contract. Do not adopt Devfile's default-on-missing revision behavior, ZIP-without-digest source, or runtime credential exposure. +- **Normative execution:** UHP northbound and the AllAgents vendor snapshot descriptor only on a first turn. +- **Normative preparation:** the builder owns source-layout fields informed by Devfile/Daytona, but claims no conformance to either. +- **Normative handoff:** direct OCI manifest identity plus digest-covered canonical workspace manifest and provenance. +- **Benchmark compatibility:** ingest Harbor task packages and SWE-bench/Hugging Face records through preparation adapters, preserving their task/environment/verifier separation. +- **Runtime precedent:** evaluate Harbor ASP, E2B, Daytona, or Dev Containers only as southbound sandbox/runtime layers. +- **Do not conflate:** benchmark repository with target source; runtime image with workspace snapshot; snapshot digest with authorization; or provider sandbox identity with the UHP session. -This is deliberately a layered answer rather than a claim that AllAgents has invented a universal workspace standard. The narrow extension exists because the examined standards stop either before source acquisition or before committed, immutable provenance. +This remains a layered answer rather than a claim that AllAgents invented a universal workspace standard. The builder contract exists because source-layout standards stop before the required immutable provenance. The gateway extension remains narrow because execution receives only the published result. ## Existing research status -- [Harbor repository materialization](./harbor-repository-materialization.md) correctly identifies Harbor's task-repository cloning, package cache, staged publication, and prebuilt-environment model. Its statement that Harbor lacks a first-class arbitrary target-repository layer remains accurate, but should not be read as saying Harbor lacks workspace materialization. Its descriptions of AllAgents selecting configured repository names or overriding a declared repository ref are stale: current Git mode accepts caller-supplied canonical public HTTPS URLs; only OCI snapshots use the operator-owned catalog. -- [E2B execution-gateway patterns](./e2b-execution-gateway-patterns.md) remains correct that E2B is a runtime provider rather than a replacement northbound protocol. Its description of a server-authoritative logical source catalog is stale for Git sources and remains applicable only to the OCI snapshot catalog. -- The general AI research wiki has relevant Harbor and benchmark-provenance coverage but no dedicated Devfile, Dev Containers, E2B, Daytona, or Codespaces contract comparison. Its primary-source links informed source discovery; its prose is not a normative input here. -- The private AllAgents research wiki contains one execution-gateway comparison based on the older A2A-era decision baseline. That coverage is now historical because ADR 0002 selects UHP. No private synthesis or conclusion is reproduced in this public note. +- [Harbor repository materialization](./harbor-repository-materialization.md) supplies the content-addressed-cache, staging, and prebuilt-publication precedents now adopted by the builder. +- [E2B execution-gateway patterns](./e2b-execution-gateway-patterns.md) remains correct that E2B is a runtime provider rather than a replacement northbound protocol. +- [Prebuilt immutable workspace snapshots at the HarnessRouter boundary](./allagents-gateway-snapshot-boundary.md) is the current boundary analysis and supersedes runtime-composition conclusions in earlier notes. +- General and private research wikis were discovery inputs only; public primary sources and ADR 0002 are normative for this decision. -ADR 0002 and the implementation plan now codify this layered result: the canonical JSON request keeps `url`, `ref`, `destination`, and logical `workingDirectory`; local `workspace.yaml` replaces `source` plus `repo` with `url` while retaining `path`; OCI requests use explicit `snapshotName`, `imageManifestDigest`, and `workspaceManifestDigest`; workspace-manifest version 2 lets each snapshot root be tree-only or carry normalized offline Git history without a configured Git remote; and benchmark task/environment ingestion remains a separate future adapter boundary. The two older public research notes still need their stale pre-cutover descriptions corrected when they are next maintained; any private-wiki refresh remains a separate private edit. +ADR 0002 and the implementation plan now codify a builder-to-gateway OCI artifact boundary. Direct Git URLs, refs, destinations, history policy, and source credentials are builder inputs; the gateway accepts one authorized direct descriptor and returns its verified manifest/provenance identities. From 479a43819b38c500a1a334581bbdd7216547a228 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 13:25:32 +1000 Subject: [PATCH 39/44] docs(architecture): define one-shot agent gateway --- .../0002-adopt-uhp-through-harnessrouter.md | 346 ----- ...use-allagents-gateway-for-one-shot-runs.md | 381 +++++ ...0837-feat-coding-execution-gateway-plan.md | 1308 +++++++++++------ .../allagents-gateway-snapshot-boundary.md | 367 ----- .../e2b-execution-gateway-patterns.md | 100 +- .../harbor-repository-materialization.md | 76 +- .../one-shot-coding-agent-gateway-boundary.md | 335 +++++ .../source-credential-broker-precedents.md | 113 +- .../research/workspace-contract-incumbents.md | 136 +- 9 files changed, 1826 insertions(+), 1336 deletions(-) delete mode 100644 docs/decisions/0002-adopt-uhp-through-harnessrouter.md create mode 100644 docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md delete mode 100644 docs/research/allagents-gateway-snapshot-boundary.md create mode 100644 docs/research/one-shot-coding-agent-gateway-boundary.md diff --git a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md b/docs/decisions/0002-adopt-uhp-through-harnessrouter.md deleted file mode 100644 index 996d7203..00000000 --- a/docs/decisions/0002-adopt-uhp-through-harnessrouter.md +++ /dev/null @@ -1,346 +0,0 @@ -# ADR 0002: Adopt UHP through AllAgents Gateway with prepared workspace snapshots - -- Status: Accepted -- Date: 2026-09-21 -- Updated: 2026-09-28 - -## Context - -Promptfoo needs a remote coding-harness endpoint that can start from large reproducible workspaces, preserve several complete Git histories, continue a session without source drift, and return exact filesystem changes. - -The accepted HarnessRouter baseline is commit [`5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3), release [`v0.25.4`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.4), and UHP version [`2026-09-12`](https://github.com/HarnessRouter/harnessrouter/tree/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/protocol/versions/2026-09-12). The current implementation point inspected for this revision is commit [`8f7868ccb2c97d1f611acf11e7cad0357a43064e`](https://github.com/HarnessRouter/harnessrouter/commit/8f7868ccb2c97d1f611acf11e7cad0357a43064e). The relevant workspace behavior is unchanged between those points. - -Stock HarnessRouter already owns UHP, authentication, session and response identity, harness supervision, one private session filesystem, checkpoint/hydrate, Files and artifacts, cancellation, and deletion. Its root Git repository is an internal produced-file journal, not a model that the agent may edit only one repository. Repositories are ordinary content inside the session filesystem. - -Stock behavior is insufficient for prepared multi-repository workspaces: - -- there is no supported public immutable-snapshot import contract; -- `_git_ensure` creates or mutates `.git` at the workspace root; -- root Git cannot report exact descendant changes across embedded repositories and stock collection omits deletions; -- stock checkpoints rearchive the full workspace and are session-keyed rather than a cross-session immutable snapshot cache; and -- `BACKING.workspace` reads or writes one file and is not an acquisition seam. - -The previous version of this ADR placed Git resolution, OCI acquisition, multi-source composition, source credentials, caching, attachment, execution, and collection inside AllAgents Gateway. That crosses two trust and lifecycle boundaries. Mutable source preparation belongs before execution. HarnessRouter should receive one already-published immutable filesystem, not a product-specific source plan. - -The supporting evidence is in [Prebuilt immutable workspace snapshots at the HarnessRouter boundary](../research/allagents-gateway-snapshot-boundary.md). - -## Decision - -We will use two components in two source repositories: - -1. **AllAgents Workspace Builder** in `allagentsdev/allagents-workspace-builder` owns Git and OCI acquisition, credentials, multi-repository composition, source policy, provenance, canonical baseline creation, and immutable snapshot publication. -2. **AllAgents Gateway** in `allagentsdev/allagents-gateway` remains a stock-derived HarnessRouter distribution. It accepts one exact snapshot descriptor, authorizes it, initializes a private writable session tree, journals filesystem changes without root Git, and otherwise retains HarnessRouter's UHP/session lifecycle. - -This separation is a source and trust boundary, not a requirement to deploy two always-on services. The builder SHOULD begin as a CLI/library usable from CI or a job worker. It MAY gain an asynchronous service wrapper when workload or latency requires one. Published OCI artifacts are the only execution handoff. - -UHP remains the only northbound execution protocol. Snapshot execution is an AllAgents vendor extension, not a claim that UHP 2026-09-12 standardizes workspace snapshots. Requests without the extension retain characterized stock behavior. - -AllAgents will propose upstream-neutral immutable workspace initialization, Git-independent journaling, and recoverable terminal finalization seams to HarnessRouter. Downstream implementation may proceed while the proposal is reviewed. Product-specific Git/OCI composition and the AllAgents snapshot format remain outside HarnessRouter. - -## Preparation boundary - -The builder accepts an AllAgents-owned build request that MAY contain several Git and OCI inputs, non-overlapping destinations, and one default working directory. That build API is not a UHP request and is not accepted by AllAgents Gateway. - -The builder MUST: - -- resolve every mutable Git ref to an exact commit before publication; -- preserve complete ancestry reachable from the selected commit by default, with shallow history only when explicitly requested; -- keep source credentials, Git helpers, mirrors, registry coordinates, and acquisition network policy out of the artifact and agent environment; -- compose all inputs into one staging tree without overlapping destinations or reserved-path collisions; -- remove acquisition-only state, credential-bearing remotes, unsafe alternates, transient Git locks, devices, sockets, FIFOs, capabilities, ACLs, xattrs, and special permission bits; -- validate self-contained `.git` repositories semantically while keeping their bytes inside the snapshot; -- generate a canonical visible-tree manifest that excludes every `.git` tree and runner-owned paths; -- generate digest-covered provenance that records ordered source identity, destination, requested selector, resolved immutable identity, history completeness, builder version, and policy version; -- publish layers, config, provenance, and the tree manifest completely before returning a direct OCI image-manifest descriptor; and -- never return a tag or multi-platform index as the execution identity. - -Preparation failure occurs before UHP session creation. The gateway never retries a failed build, chooses another ref, changes history depth, or falls back between Git and OCI. - -## Snapshot artifact contract - -A workspace snapshot is one OCI image manifest with: - -- `artifactType: application/vnd.allagents.workspace-snapshot.v1`; -- config media type `application/vnd.allagents.workspace-snapshot.config.v1+json`; -- provenance media type `application/vnd.allagents.workspace-provenance.v1+json`; -- canonical visible-tree media type `application/vnd.allagents.workspace-manifest.v1+json`; -- only gzip filesystem layers with media type `application/vnd.oci.image.layer.v1.tar+gzip`; and -- ordered OCI filesystem changesets applied to an empty directory. - -The closed digest-covered config contains `version: 1`, default relative working directory, workspace-manifest descriptor, provenance descriptor, builder/policy identity, `reserved_paths_schema: 1`, and `available_until`. The builder emits deterministic gzip with fixed headers and rejects uncompressed, zstd, and nondistributable layer media types in V1. - -Reserved-path schema 1 contains exactly the root `.harness` path and every descendant. The builder rejects any entry or link alias at that location. Harness instruction files such as root `AGENTS.md` are visible snapshot content, not reserved paths: snapshot mode MUST preserve existing content, merge any runner-managed block deterministically, and establish the initial visible-tree cursor only after that merge and UHP input application. - -Provenance is RFC 8785 canonical JSON. Its closed schema is: - -```text -{ - version: 1, - sources: Array< - | {kind: "git", url: string, requested_ref?: string, resolved_commit: string, - history: {mode: "full"} | {mode: "shallow", depth: integer}, destination: string} - | {kind: "oci", source_name: string, - descriptor: {media_type: string, digest: string, size: integer}, destination: string} - > -} -``` - -Source order is build order. The builder and policy identity remain in config. Provenance contains no credential, private registry coordinate, header, host path, or helper state. - -The builder returns one versioned result: - -```json -{ - "version": 1, - "descriptor": { - "media_type": "application/vnd.oci.image.manifest.v1+json", - "digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", - "size": 123456 - }, - "workspace_manifest_digest": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "provenance_digest": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", - "working_directory": "services/api", - "available_until": "2026-10-29T00:00:00Z" -} -``` - -Only `descriptor` is forwarded to the gateway. `available_until` is also digest-covered by config and is backed by an operator-enforced registry retention lease. The gateway admits a new session only when the remaining lease covers initialization deadline plus maximum session TTL plus safety margin. - -The direct manifest digest transitively binds config, provenance, the visible-tree manifest, layers, all `.git` bytes, and the retention deadline. OCI referrers MAY add signatures or attestations, but required provenance MUST remain digest-covered by the admitted manifest because an OCI `subject` association is weak and can change independently. - -The canonical visible-tree manifest uses media type `application/vnd.allagents.workspace-manifest.v1+json` and RFC 8785 canonical JSON: - -```json -{"version":1,"entries":[]} -``` - -Entries sort by the UTF-8 bytes of their NFC-normalized relative POSIX path. The root is omitted; empty directories are represented. A path has exactly one of these states: - -```json -{"path":"src","type":"directory"} -{"path":"src/main.ts","type":"file","size":123,"sha256":"sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef","executable":false} -{"path":"bin/tool","type":"symlink","target":"../src/tool"} -``` - -Traversal never follows links. File modes normalize to `0644` or `0755`; directories normalize to `0755`; ownership is runtime-assigned rather than artifact-controlled. Symlink targets MUST be UTF-8, NFC, relative, and confined when resolved from the link parent. Safe in-root OCI hardlinks MAY be materialized as ordinary files. Duplicate, non-UTF-8, non-NFC, absolute, traversing, escaping, unsupported, or conflicting entries fail publication or admission. - -Every `.git` tree and reserved `.harness` tree is excluded from the visible-tree manifest and public changes. The complete materialized namespace is still scanned independently for forbidden `.harness` entries before cache publication and ready. Layer digests bind `.git` bytes, and builder provenance records semantic Git verification. Evaluation correctness never depends on a Git index, status, commit, ignore rule, or rename heuristic. - -## Gateway request contract - -A snapshot-backed first turn uses `POST /v1/responses` with a closed, versioned vendor extension: - -```json -{ - "input": "Make the requested change.", - "metadata": { - "allagents_workspace_snapshot": { - "version": 1, - "descriptor": { - "media_type": "application/vnd.oci.image.manifest.v1+json", - "digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", - "size": 123456 - } - } - } -} -``` - -Only `version` and `descriptor` are accepted. Unknown keys fail. The caller cannot provide a registry, repository, tag, index, credential, header, redirect policy, host path, source list, materializer, working directory, resource limit, provider route, or expiry. - -V1 maps each authenticated product domain to exactly one trusted snapshot repository and one authorization catalog; zero or multiple repository mappings are a deployment error. The catalog entry binds the exact descriptor and digest-covered `available_until`. Before cache use or registry traffic, the gateway resolves one entry and authorizes `(principal, domain, repository_id, catalog_entry_id, media_type, digest, size)` against current policy. A digest proves identity, not authorization. Every cache hit reauthorizes the tuple; policy revision does not fragment the byte-cache key. - -The extension is accepted only on the request that creates a new session. Continuations selected by `previous_response_id` MUST omit it. A stock session cannot acquire a snapshot later, and a bound session cannot repeat or replace its descriptor. - -After initialization reaches `ready`, terminal events, response retrieval, replay, and later terminal failures expose the same sanitized metadata: - -```json -{ - "version": 1, - "descriptor": { - "media_type": "application/vnd.oci.image.manifest.v1+json", - "digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", - "size": 123456 - }, - "snapshot_manifest_digest": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "ready_manifest_digest": "sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", - "provenance_digest": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", - "initialization_changes_file_id": "file_...", - "working_directory": "services/api", - "available_until": "2026-10-29T00:00:00Z", - "expires_at": "2026-09-29T00:00:00Z" -} -``` - -Repository coordinates, catalog entries, policy revisions, credentials, private cache keys, host paths, local clone mechanisms, and builder job identifiers are never public. Full source provenance is retrieved from the digest-covered snapshot provenance; UHP returns its immutable digest rather than copying a product-specific source schema into HarnessRouter. - -## Binding and initialization lifecycle - -A snapshot session has durable `pending`, `ready`, `failed`, and `deleting` states. Before acquisition, the gateway persists the exact descriptor, internal immutable `repository_id`, `catalog_entry_id` and version, admitted signature-policy/evidence identity, authorization audit reference, initialization deadline, and one provisional reference record keyed by `(binding_id, snapshot_key)`. It never persists credentials, public registry coordinates, local cache paths, inode identities, live attachment flags, or process-local locks. - -Initialization is all-or-nothing: - -1. Validate the request and first-turn rule without network or workspace writes. -2. Resolve exactly one repository/catalog entry and authorize the full tuple against current policy. -3. Persist the `pending` binding and provisional reference. -4. Fetch only from the bound repository by digest; verify response size and digest before use. -5. Verify config, provenance, retention deadline, tree manifest, gzip layer descriptors, signatures when policy requires them, and every referenced size/digest/media type. Reject expired or insufficient remaining retention with `workspace_snapshot_retention_invalid`. -6. Reject unknown reserved-path schemas. During layer application and final scan, reject every entry, whiteout, hardlink, symlink alias, or type transition at `.harness` or below. -7. Apply OCI changesets in private staging with rooted no-follow operations and bounded bytes, entries, paths, metadata, processes, descendants, and time. -8. Recompute the canonical snapshot manifest and a private full-tree seal that includes `.git`; require the declared snapshot digest and persist the seal as cache evidence. -9. Publish or reuse one immutable unpacked cache generation keyed by descriptor plus materializer-schema revision. -10. Reauthorize current policy, then create an inode-independent private session tree by same-filesystem reflink clone when supported or full private copy otherwise. Hardlinks to cache content are forbidden. -11. Apply UHP inputs and deterministic runner instruction/control preparation. Compare the snapshot manifest to the resulting ready tree, durably capture `workspace-initialization-changes-.json` plus changed regular-file artifacts, and store the ready manifest as initial cursor. -12. Validate the digest-covered working directory, activate the reference, and atomically transition to `ready` with the snapshot/ready digests and initialization artifact; start finite session TTL and only then provider or harness work. - -Credentialed Git and OCI clients authorize scheme, host, port, and resolved IP before every connection; reject loopback, link-local, private, Unix-socket, rebinding, or other disallowed targets; bound and reauthorize every redirect; never forward authorization, cookies, or client certificates across origins; require TLS; and use scoped short-lived credentials. - -The initialization deadline is separate from session TTL. Deadline exhaustion is a retryable availability failure, not a request-size failure. Every pre-ready failure is terminal for that binding: it transitions to `failed`, releases the provisional reference idempotently, and leaves no runnable tree. `retryable: true` means a caller may create a new session with the same descriptor; an idempotent duplicate of the failed request returns the same failure. Restart reconciliation may resume a still-running `pending` attempt internally. - -Exact-key cache misses singleflight. Cache roots and generations are owned by a gateway/materializer identity no harness UID can assume, are non-writable and non-searchable from the runner namespace, and are cloned only through trusted directory file descriptors with no-follow operations under an eviction/clone lease. The gateway verifies the full-tree seal immediately before and after cloning; unexpected mutation quarantines the generation and fails closed. - -Staging and quarantine have global byte, inode, and age bounds. Complete unreferenced cache generations are evictable under high and low watermarks. Durable reference records, not counters, make claim/release and crash reconciliation idempotent. Cache attachment eligibility is reauthorized on every use. - -V1 intentionally materializes a normal private directory. Reflink is preferred and full copy is required as the correctness fallback. OverlayFS and base-plus-delta checkpoints are deferred until measurements justify the larger mount-aware lifecycle change. - -## Resource and extraction limits - -The v1 format maximum is 64 layers, a 4 MiB OCI manifest, a 4 MiB config, a 128 MiB workspace manifest, 8 GiB total compressed layers, 64 GiB expanded workspace bytes, 1,000,000 visible entries, 4 GiB per regular file, 4096 UTF-8 bytes and 128 components per path, and 1 MiB per PAX or extended header. Expanded bytes divided by `max(compressed bytes, 1)` MUST NOT exceed 100 for each layer and for the artifact as a whole. Operators MAY lower but not raise these maxima without a contract revision. - -Extraction rejects absolute or traversing paths, ambiguous separators, NULs, escaping links, devices, sockets, FIFOs, unsupported sparse files, unbounded metadata, unknown compression, duplicate/type conflicts, undeclared final content, and digest mismatches. Runtime byte and inode quotas apply to staging, cache, private workspaces, outputs, and checkpoints. Materialized roots run `nodev` and `nosuid` where the deployment filesystem supports mount flags; normalized content contains no device nodes, set-ID bits, file capabilities, or executable metadata outside the manifest contract. - -## Produced files and evaluation changes - -Snapshot-backed sessions replace only the stock root-Git journal behind the existing produced-list, capture, and acknowledge seams. Requests without the snapshot extension retain stock root-Git behavior. - -The gateway stores the canonical no-follow manifest cursor in protected blob storage as the sole durable journal authority. It excludes every `.git` and `.harness` tree. The runner receives the explicit acknowledged cursor, compares it with the final visible tree, and returns add, modify, and delete plus the next cursor/token. Content, type, executable mode, and symlink-target changes are modifications; rename is delete plus add. - -The gateway captures every added or modified regular file through the existing Files/artifact path and creates exactly one RFC 8785 canonical artifact named `workspace-changes-.json` with media type `application/vnd.allagents.workspace-changes.v1+json`: - -```json -{ - "version": 1, - "entries": [ - { - "path": "services/api/src/main.ts", - "operation": "modify", - "before": {"type":"file","sha256":"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","size":100,"executable":false}, - "after": {"type":"file","sha256":"sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb","size":120,"executable":false}, - "file_id": "file_..." - } - ] -} -``` - -Entries sort by normalized path bytes. `add` has only `after`; `delete` has only `before`; `modify` has both. `file_id` exists exactly when `after.type` is `file` and refers to the already-durable file artifact. Empty turns emit an empty change artifact. - -Collection, checkpoint, cursor advancement, and terminal response finalization form one recoverable response transaction under an exclusive workspace mutation lease. The lease blocks Files writes, new turns, cleanup, and every other session writer after all descendants stop: - -1. send the gateway-owned acknowledged cursor blob to the runner and compute the next manifest; -2. create deterministic artifact identities keyed by `response_id`; -3. open changed regular files with rooted no-follow operations, hash while streaming, and require type, size, and digest to equal each `after` state; -4. durably store regular-file artifacts and the change artifact; -5. create and durably store the exact snapshot-mode checkpoint, then verify its visible manifest equals the next cursor; -6. durably store the next cursor manifest; -7. call idempotent `/produced/ack` with base digest, next digest, and collection token; and -8. atomically compare-and-set the gateway's cursor/checkpoint/artifact pointers and terminal response to committed. - -The gateway cursor blob is the sole durable journal authority. Runner ACK is a recoverable confirmation barrier, not independent cursor state; every retry supplies the base cursor explicitly. A crash before step 8 leaves the previous gateway cursor authoritative even if ACK ran, so retry reproduces the same logical transaction. Terminal response visibility occurs only after ACK and the final compare-and-set. - -A retry reuses the same immutable artifacts and never reruns the harness. The exclusive lease plus streamed hash/checkpoint verification prevents a file artifact, change manifest, and checkpoint from describing different trees. - -Promptfoo reconstructs the final visible tree by starting from the builder's exact snapshot, applying the durable initialization-change artifact, then applying ordered per-response change artifacts. The ready manifest digest proves that first transition. Change artifacts are deterministic deltas; they do not pretend to contain baseline bytes or `.git` history. - -## Checkpoint, continuation, expiry, and deletion - -Snapshot-backed V1 checkpoints the complete normal private workspace, including self-contained `.git` histories. It excludes only `.harness/tmp/**`, `.harness/home/.codex/auth.json`, `.harness/home/.omp/agent/auth.json`, `.harness/home/.omp/agent/models.json`, and `.harness/home/.omp/agent/models.yml`; every other `.harness` resume path remains. Stock broad dependency exclusions MUST NOT remove declared snapshot/session content. Any exclusion change requires a checkpoint-schema revision and compatible-reader gate. - -A continuation restores the exact durable checkpoint and cursor, verifies the stored binding, working directory, quotas, and ownership, and only then starts the stored harness. It does not accept another descriptor or resolve mutable sources. Missing or corrupt binding, checkpoint, cursor, or snapshot identity fails closed rather than starting empty. An evicted unpacked cache MAY be repopulated only from the same authorized direct manifest digest; the session's checkpoint remains the source of mutable state. - -The finite session TTL starts only after `ready`. Polling, replay, turns, and continuation do not extend `expires_at`. Expiry and explicit deletion make the session unavailable first, stop descendants, remove the private workspace, delete durable checkpoint/cursor state through existing records, and release each durable snapshot reference exactly once. Busy or uncertain state remains unavailable, accounted, and queued for idempotent reconciliation. - -## Failure contract - -Workspace failures use stable detail codes in the existing UHP error shape. For pre-`ready` initialization failures, `retryable` means a new request/binding with the same descriptor may succeed; it never revives a failed binding. For post-`ready` collection/checkpoint failures, retry resumes the same `WorkspaceTurnCommit` and MUST NOT rerun the harness. Pre-`ready` failures omit snapshot metadata. Post-`ready` workspace failures return stored sanitized metadata except inherited `session_expired`. - -| Detail code | HTTP | Retryable | Condition | -|---|---:|:---:|---| -| `workspace_snapshot_invalid_request` | 400 | no | Malformed/unknown fields, unsupported version/media type, bad digest/size, continuation injection, or other request-decidable violation. | -| `workspace_snapshot_unknown` | 404 | no | Exact descriptor is absent from the caller's authorized catalog view; hides whether it exists for another principal. | -| `workspace_snapshot_invalid` | 422 | no | Manifest, config, provenance, layer, path, link, type, working directory, final-tree, or signature verification failure. | -| `workspace_snapshot_retention_invalid` | 422 | no | Digest-covered/catalog retention is expired or cannot cover initialization deadline plus maximum session TTL plus safety margin; caller must publish a new snapshot. | -| `workspace_snapshot_unavailable` | 503 | yes | Bounded transient registry, DNS, transport, initialization-deadline failure, or an authorized manifest missing before its promised `available_until`. | -| `workspace_contract_limit_exceeded` | 413 | no | Fixed format count, byte, path, ratio, metadata, or file maximum exceeded. Timeouts do not use this code. | -| `workspace_capacity_exceeded` | 503 | yes | Operator disk, inode, worker, quota, or concurrency capacity unavailable. | -| `workspace_initialization_failed` | 500 | yes | Reflink/copy, staging publication, private-tree, baseline, or ready transition fails after valid artifact admission. | -| `session_expired` | 404 | no | Inherited UHP response when continuation targets a session at or after `expires_at`. | -| `workspace_restore_invalid` | 500 | no | Durable binding, checkpoint, cursor, or identity is missing, corrupt, or inconsistent. | -| `workspace_collection_failed` | 500 | yes | Manifest comparison or artifact persistence cannot complete; cursor and checkpoint do not advance. | -| `workspace_checkpoint_failed` | 500 | yes | Exact private-workspace checkpoint persistence fails; terminal response is not finalized. | - -Cancellation is not a workspace error. Inherited idempotent cancellation and terminal `status: "cancelled"` remain. Cancellation before `ready` omits snapshot metadata; after `ready` it retains stored sanitized metadata. - -## Upstream boundary - -The upstream proposal is tracked in [HarnessRouter issue #304](https://github.com/HarnessRouter/harnessrouter/issues/304). It is an internal implementation seam, not a UHP Enhancement Proposal: - -1. an optional operator-configured `WorkspaceInitializer` invoked once after an empty private session root exists and before input or harness start; -2. a pluggable `WorkspaceJournal` behind `/produced` and `/produced/ack`, with the gateway cursor as durable authority and current root Git as the default; and -3. a recoverable terminal-finalization seam that orders artifacts, journal ACK, checkpoint/cursor publication, and terminal response visibility. - -The initializer receives one opaque immutable descriptor, verifies and materializes it atomically, and returns verified identity, relative working directory, initializer schema, and journal mode. It returns no source plan, credential, registry URL, host path, cache path, or mount identity. Requests without an initializer extension remain on the stock path. - -Focused upstream tests must prove initialization-before-input, idempotent retry, descriptor replacement rejection, explicit cursor handoff, exact add/modify/delete journal behavior, capture-before-ACK, checkpoint/cursor durability before terminal visibility, continuation, fail-closed restore, and unchanged stock behavior. - -Until upstream ships equivalent initializer, journal, and finalization seams, `allagents-gateway` carries the smallest downstream patch and maps every remaining delta explicitly. When upstream ships all required seams, forked implementations are deleted. The gateway then becomes a thin stock-derived distribution or, if external backend loading is supported, no source fork at all. `allagents-workspace-builder` remains unchanged. - -## Distribution and release boundary - -`allagentsdev/allagents-gateway` preserves the HarnessRouter fork network, full Git history, Apache-2.0 `LICENSE`, `NOTICE`, attribution, and upstream remote. The existing `feat/workspace-composition` branch MUST be replaced before implementation by a snapshot-specific branch based on current downstream `main`; obsolete runtime-composition code or aliases are not retained. - -`allagentsdev/allagents-workspace-builder` is a separate AllAgents-owned repository with its own release cadence, threat model, credentials, and artifact-format compatibility tests. The cross-repository compatibility contract is the versioned OCI snapshot artifact, not source-level imports or a private RPC schema. - -Before release, the target production filesystem is probed for same-filesystem reflink behavior and hard byte/inode quotas. Full-copy fallback is exercised even where reflink succeeds. The candidate supports only architectures explicitly built, preflighted, and tested; V1 MAY declare `linux/amd64` only. - -Capability `allagents_workspace_snapshot_v1` remains disabled until every request-serving gateway and runner understands artifact, binding, journal, checkpoint, and finalization schema V1. Snapshot requests and bound continuations carry an internal minimum-reader version and route only to compatible replicas; incompatible replicas reject before hydrate. Rollback retains compatible readers until all snapshot sessions are deleted, or first makes those sessions unavailable and drains them before old code serves traffic. - -Release proof runs against the exact published image digests and includes: - -- stock UHP compatibility and both supported harnesses; -- snapshot cache miss/hit, attempted cache-path access from a session UID, concurrent singleflight, private-write isolation, limits, and malicious OCI fixtures; -- exact add/modify/delete artifacts, streamed artifact/hash binding, checkpoint/continuation, cancellation, expiry, deletion, and crash reconciliation; -- fresh-volume and same-volume restart during pending initialization, collection, journal ACK, checkpoint, ready execution, and deletion; -- an N-1-to-candidate upgrade plus proven compatible routing and rollback-or-drain fencing; -- repeated race-sensitive restart, cancellation, reference-release, turn-deletion, and cleanup cases against the exact digest; -- SBOM, build provenance, secret scan, and correlation-safe telemetry that never emits credentials, private registry paths, or workspace contents; and -- an upstream-intake record linking the proposed hook issue, maintainer decision, downstream delta, and removal trigger. - -## Rejected alternatives - -| Alternative | Why rejected | -|---|---| -| Resolve Git and compose OCI sources inside AllAgents Gateway | Mixes product source credentials and policy with execution, enlarges the permanent fork, and makes failures part of task startup. | -| Keep preparation and execution in one source repository | Couples release and trust boundaries and makes returning to stock a source-tree surgery. | -| Upload a tar through stock `/hydrate` | Internal checkpoint route with no public OCI identity, authorization, provenance, cache, or exact multi-repository journal. | -| Make the agent clone repositories | Runs acquisition after harness start, exposes credentials/network policy to agent code, and cannot establish a trusted pre-turn baseline. | -| Use stock root Git as evaluator | Mutates a root repository, reports embedded repositories coarsely, and omits deletions. | -| Use read-only shared trees, hardlinks, or symlinks | Harnesses need a writable workspace; these mechanisms risk cache or sibling mutation. | -| Adopt OverlayFS and delta checkpoints in V1 | Expands hydrate, checkpoint, archive, Files, deletion, mount, and crash recovery before measured need. | -| Put provenance only in OCI referrers | Referrer associations are weak and do not contribute to admitted manifest identity. | -| Accept tags or indexes at execution | Selection can change independently of the request; execution requires one direct manifest. | -| Add a second public changes endpoint | Duplicates UHP Files/artifacts and bypasses capture-before-ack. | -| Upstream the AllAgents source descriptor | Git/OCI composition is product policy, not HarnessRouter execution infrastructure. | - -## Consequences - -The gateway becomes materially smaller: no Git client, source credential broker, per-component destination planner, or runtime composition state. Task startup sees one authorized immutable artifact and one private filesystem. - -The builder takes on explicit distributed-system obligations: asynchronous build status when needed, complete-before-return publication, artifact retention, provenance, garbage collection, and a compatibility matrix with gateway snapshot versions. - -V1 pays the I/O/storage cost of a private reflink/copy and full exact checkpoint. That cost is deliberate. It preserves ordinary-directory semantics and minimizes the downstream/upstream patch. Metrics determine whether a later ADR adopts OverlayFS or base-plus-delta checkpoints. - -Promptfoo must retain or fetch the admitted snapshot baseline, then apply the initialization delta and ordered response deltas to reconstruct complete final state. These UHP artifacts do not replace baseline storage. - -## Reconsider when - -Revisit this decision if measurements show private clone or full checkpoint costs violate release SLOs; the production filesystem cannot provide correct reflink or bounded full-copy behavior; upstream rejects the required initialization, journal, and finalization seams and fork cost exceeds a standalone executor; the snapshot format cannot preserve required Git workflows safely; or a later UHP version standardizes an equivalent immutable workspace contract. diff --git a/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md b/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md new file mode 100644 index 00000000..31b28dc2 --- /dev/null +++ b/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md @@ -0,0 +1,381 @@ +# ADR 0002: Use a one-shot AllAgents Gateway with Promptfoo as the first caller + +- Status: Accepted +- Date: 2026-09-21 +- Updated: 2026-09-28 + +## At a glance + +A Promptfoo case asks OMP to fix a bug in a workspace composed from writable application Git commit `8c4f…` at `app/` and a read-only OCI-hosted fixture at `fixtures/`, with `app/` as the working directory. It selects the authorized `node22` runtime profile and `package-install` agent network policy, then requests post-run evidence from a hidden immutable check bundle: confined executable `verify`, literal arguments `["--json", "reports/test.json"]`, bounded output file `reports/test.json`, and the `integration-checks` post-run network policy. The provider packages local content and submits one `AgentRunRequest v1`. A worker resolves the profile to an immutable image and creates a fresh `RunSandbox`; the runner attaches authorized immutable source generations and invokes OMP once. After OMP exits, the runner terminates its process tree, destroys its network namespace and credentials, preserves the runtime and workspace, and runs the check in a separately brokered post-run network namespace. Only after evidence persistence, service stop, unmount, private-root deletion, and lease release succeed does the gateway return `status: "completed"` with direct and runtime provenance, bounded agent output, usage, trajectory, ordered raw command evidence, and the report's `ArtifactReference`. The provider downloads that artifact through the authenticated result-artifact endpoint, verifies its length and digest, and gives the evidence to Promptfoo's code grader; the gateway itself never decides pass or reward. + +The gateway's product is one-shot execution output and raw evidence, not a behavioral judgment or workspace diff. No caller continues the coding session, restores a checkpoint, or reconstructs the final tree from change artifacts. The previous HarnessRouter/UHP session and snapshot architecture therefore solves lifecycle and transport problems that one-shot runs do not have. + +## Context + +AllAgents needs a general remote gateway for one-shot coding-agent runs, similar in purpose to HarnessRouter but deliberately smaller in lifecycle. Promptfoo is its first caller and remains the authoring and experiment layer for evaluations: it owns test cases, variable and provider matrices, prompt rendering, repetitions, grading, assertions, and reports. Other callers may omit post-run evidence collection and consume the agent output directly. + +A run has one rendered instruction, one composed workspace, one selected agent, and optional post-run evidence collection. Its mutable state exists only for that run. It is not a reusable coding session. Promptfoo maps each evaluation trial and repetition to a distinct run and applies its own code or LLM grader to the returned evidence. Retries must not accidentally share state or run the agent more than once. + +The earlier version of this decision treated remote agent execution as a UHP session served by a HarnessRouter-derived gateway. It introduced a separate workspace-builder repository, published OCI workspace snapshots, added snapshot import and filesystem journaling to HarnessRouter, checkpointed workspaces for continuation, and exported generic workspace-change artifacts. Those capabilities are unnecessary for one-shot runs and create protocol, fork, storage, and recovery obligations unrelated to the result callers need. + +Supporting evidence and comparisons are recorded in [One-shot coding-agent gateway boundary](../research/one-shot-coding-agent-gateway-boundary.md). + +## Decision + +We will build one general one-shot AllAgents Gateway. The gateway API, reusable worker and runner packages, Promptfoo provider, and Codex and OMP adapters will live in one source repository: `allagentsdev/allagents-gateway`. + +The public operation is `AgentRunRequest v1` and `AgentRunResult v1`. Both are closed, versioned JSON contracts: unknown fields are rejected, identifiers are explicit, and incompatible changes require a new version. The contract borrows useful UHP conventions—stable IDs, idempotency, cancellation, structured errors, and bounded structured telemetry—but V1 does not implement UHP or expose reusable sessions. + +HarnessRouter issue [#304](https://github.com/HarnessRouter/harnessrouter/issues/304) is superseded by this decision. The two-repository gateway/workspace-builder snapshot design is also superseded and is not an implementation contract for the AllAgents Gateway. +The `allagentsdev/allagents-gateway` repository name is retained for the new standalone service; none of the former HarnessRouter fork, snapshot, or workspace-builder contract survives. + +## Main flow + +1. A caller defines an ordered multi-source workspace, selects an authorized logical runtime profile, and renders one instruction. For evaluations, Promptfoo first expands its cases, variables, providers, and repetitions and assigns every repetition a distinct run identity. +2. The Promptfoo provider deterministically packages local source and optional check content, reserves uploads with client-generated `upload_id` values, uploads immutable bundles, and configures the raw evidence its graders need. It calls the AllAgents Gateway with artifact references and digests; remote JSON never contains a caller host path. +3. The gateway authenticates and validates the closed request, then performs tenant-scoped duplicate lookup before mutable admission. An exact duplicate returns the existing run without re-admission. For a genuinely new identity, the gateway applies source, model, network, and resource policy, resolves `runtime_profile_id` to an authorized immutable `profile_digest`, and durably pins that revision; rejected new requests create no public run or job. +4. A worker accepts only the pinned profile revision and creates one fresh ephemeral sandbox from its immutable image and sandbox policy; unavailable or mismatched profile content fails the run. For Git and OCI, the gateway maps the authenticated caller plus canonical URL or repository to exactly one internal source policy and credential route before credentials or network access; zero or ambiguous matches reject the run. The runner resolves sources, rechecks policy, and attaches operator-owned immutable generations. `read_only` sources mount directly read-only; `writable` sources receive private CoW/reflink clones or full-copy fallback. +5. The selected Codex or OMP adapter runs the agent once under the agent network phase. The runner captures bounded structured final output, usage, timing, and trajectory. +6. After the agent exits, the runner terminates the agent process tree, destroys its network namespace and flows, and removes model and source credentials while preserving the sandbox, pinned runtime, final workspace access modes, and operator-declared task services. Only after that transition does it inject the immutable check bundle when requested, create the separately authorized post-run network namespace, and execute declared runtime or bundle executables with literal arguments. +7. The worker durably seals each raw post-run observation in request order, persists requested file artifacts, then stops post-run processes and task services, unmounts every trial mount, deletes the private trial root, and releases every cache lease. These operations are idempotent and must all succeed before any terminal result becomes visible. +8. The gateway then publishes `completed`, `cancelled`, or `infrastructure_error`. Cleanup failure forces `infrastructure_error` with partial evidence; reconciliation completes cleanup before publication. Promptfoo retrieves referenced artifacts, verifies their declared size and digest, and exposes the evidence to code or LLM graders, which alone decide behavioral outcomes. + +There is no checkpoint, continuation, reusable coding session, or implicit rerun in this flow. + +## Ownership + +### Promptfoo + +Promptfoo owns evaluation intent and aggregation: + +- case and dataset authoring; +- variables, provider and agent matrices, and repetition counts; +- rendering the complete instruction supplied to a run; +- defining post-run evidence collection and applying code or LLM graders to agent output, command results, and file artifacts; and +- experiment reports, pass/fail decisions, rewards, assertions, and comparisons across runs. + +Promptfoo does not own the remote sandbox, materialize host paths remotely, inspect gateway credentials, or infer success from changed files. + +### AllAgents Promptfoo provider + +The provider is the boundary adapter between Promptfoo and the AllAgents Gateway API. It assigns run and idempotency identity, converts Promptfoo configuration into `AgentRunRequest v1`, packages any check bundle, uploads and content-addresses local inputs before submission, waits for or cancels the run, and exposes `AgentRunResult v1` output, command results, and file artifacts to Promptfoo's code or LLM graders. Those graders alone assign pass, fail, or reward. + +Packaging is intentionally client-side. A local repository path, check-bundle path, fixture path, or requested output destination on the caller host is never meaningful to a remote worker and must not appear in the remote request. + +### AllAgents Gateway, worker, and runner + +The gateway API is the durable public control plane. It owns tenant authentication, request validation, duplicate lookup, admission, idempotency, cancellation, worker dispatch, artifact authorization, and result retrieval. A worker owns one admitted run's process and sandbox lifecycle. Inside that worker, the reusable runner package owns workspace composition, isolation, resource enforcement, agent supervision, the agent-to-evidence phase transition, ordered durable evidence capture, cleanup, and terminalization. Codex and OMP adapters translate the common run contract into each agent's invocation and normalize output, usage, and trajectory data; they do not define separate public execution contracts. + +Operator configuration owns credentials, source authorization, model routing, runtime profiles, fixed runtime tools, sandbox policies, phase-specific network policies, task-service definitions, resource ceilings, retention, and deployment policy. None of those secrets or policy documents are caller-controlled fields. + +### Post-run evidence + +The `post_run` field is required but nullable: `null` requests no extra evidence, while a nonnull value requests evidence collection rather than verification. It may identify an immutable check bundle, commands with individual timeouts, bounded workspace-relative `output_files`, and a separately named post-run network policy. Each command selects either an operator-allowed executable already present in the task runtime or an executable at a confined path inside the optional check bundle, then supplies literal arguments. A bundle executable requires `bundle`; runtime executables and output-only collection do not. Bundle bytes and a populated bundle mount are absent throughout agent execution; at most an empty, runner-owned reserved mountpoint exists. The runner first terminates every agent-owned descendant, destroys the agent network namespace and flows, and removes model and source credentials. Only then may it materialize the bundle into that mountpoint read-only and make it visible to post-run commands. Those commands use the same pinned runtime and final workspace with each source's declared access mode preserved. This lets checks use build tools such as `npm`, build the project, and exercise its actual runtime. Operator-declared task services may remain running or be restarted for checks; arbitrary agent-owned background processes never cross the phase boundary. Checks that need to modify a source must declare it `writable` or place build and test output in a separate writable path. Nonzero exits, timeouts, stdout, stderr, and generated files are observations. The gateway records them without interpreting task success. Promptfoo or another caller owns any code or LLM grader that turns those observations into pass, fail, reward, or commentary. + +## Public request boundary + +`AgentRunRequest v1` is defined by the normative closed schema `schemas/agent-run-request.v1.schema.json`. Its logical shape is: + +```text +{ + schema_version: "agent_run_request.v1", + identity: { + run_id: UUIDv7, + idempotency_key: string + }, + instruction: string, + workspace: { + working_directory: relative path or ".", + sources: array of 1..128 + | { + kind: "git", + url: canonical HTTPS URL, + ref: string, + history: { mode: "full" } | + { mode: "shallow", depth: positive integer }, + access: "read_only" | "writable", + destination: relative path or "." + } + | { + kind: "oci", + repository: canonical OCI repository, + descriptor: { + media_type: "application/vnd.oci.image.manifest.v1+json", + digest: SHA-256 digest, + size_bytes: positive integer + }, + access: "read_only" | "writable", + destination: relative path or "." + } + | { + kind: "uploaded_bundle", + bundle: BundleReference, + access: "read_only" | "writable", + destination: relative path or "." + } + }, + agent: + | { kind: "codex", model: logical ID, reasoning_effort: "low" | "medium" | "high" } + | { kind: "omp", model: logical ID }, + runtime_profile_id: logical ID, + post_run: null | { + bundle?: BundleReference, + network_policy_id: logical ID, + commands: array of 0..32 { + command_id: logical ID, + executable: + | { kind: "runtime", name: logical ID } + | { kind: "bundle", path: relative path }, + args: array of literal strings, + timeout_ms: positive integer + }, + output_files: array of 0..128 { + name: logical ID, + path: relative path, + media_type: IANA media type, + max_bytes: positive integer + } + }, + agent_network_policy_id: logical ID, + limits: { + total_timeout_ms: positive integer, + acquisition_timeout_ms: positive integer, + agent_timeout_ms: positive integer, + post_run_timeout_ms: positive integer, + max_workspace_bytes: positive integer, + max_workspace_files: positive integer, + max_agent_output_bytes: positive integer, + max_artifact_bytes: positive integer, + max_trace_bytes: positive integer, + max_command_output_bytes: positive integer + } +} + +BundleReference = { + artifact_id: UUIDv7, + media_type: "application/vnd.allagents.gateway-bundle.v1.tar+gzip", + digest: SHA-256 digest, + size_bytes: positive integer +} +``` + +Source order is authoritative. Normalized destinations are pairwise non-overlapping; `.` is allowed only as the sole destination. `working_directory` must resolve without symlink escape to a directory in the composed tree. Every source requires `access: read_only | writable`. + +Git carries a canonical URL, requested ref, history, access mode, and destination. The runner resolves the ref once to an exact commit and reports it in provenance. OCI carries a canonical repository because the exact direct image-manifest descriptor has no location. Uploaded content uses the tenant-authorized `BundleReference`. + +For Git, the gateway maps `(authenticated tenant, canonical URL)` to exactly one internal source policy and credential route. For OCI it maps `(authenticated tenant, canonical repository)` the same way. Bundle `artifact_id` is resolved through tenant-scoped artifact authorization. Zero or multiple matches reject before credential resolution, DNS, registry, Git, artifact reads, or mutable run admission. The caller never names a credential, internal route, mirror, token realm, or policy record. + +V1 rejects all Git and OCI redirects. The exact internal OCI route pins its permitted bearer-token realm; OCI manifests and blobs must be fetched through that repository route, and foreign blob or descriptor URLs are rejected. Credentials are never forwarded to a target selected by remote content. + +`runtime_profile_id`, agent model, `agent_network_policy_id`, and `post_run.network_policy_id` are independently authorized logical IDs. Admission resolves the runtime ID to an immutable `profile_digest` and persists that exact revision with its runtime image digest, fixed read-only runtime `PATH` and tool implementations, sandbox policy, and operator task-service implementations. Dispatch may resolve only the pinned revision and rejects unavailable or mismatched content. The caller cannot submit an image, executable path, service definition, raw network destination, policy body, credential, environment map, shell string, host path, grading rule, patch request, or workspace-persistence option. + +`post_run` is required and is either `null` or the closed evidence specification. At least one of `commands` or `output_files` is nonempty. `bundle` is present if and only if a bundle executable is requested. Runtime executable names resolve through the profile's fixed read-only tool mapping, never an agent-modifiable `PATH`; bundle paths remain confined beneath the hidden immutable bundle. Arguments are literal. There is no shell, interpolation, globbing, working-directory override, caller environment, or per-command network override. + +All ten limits are required and caller-lowerable beneath operator and runtime-profile ceilings. `max_agent_output_bytes` bounds the agent's structured final response. `max_artifact_bytes` is the aggregate logical-byte budget for result artifacts, excluding input bundles and source caches. Admission requires the sum of `output_files[].max_bytes` to fit this budget. Actual collected files charge first in request order, followed by promoted agent output, command stdout/stderr in command order, and trajectory; content deduplication gives no accounting discount. Optional promotion that cannot fit falls back to bounded inline truncated data or an explicit omitted observation, never unbounded storage. + +## Public result boundary + +`AgentRunResult v1` is defined by the normative closed schema `schemas/agent-run-result.v1.schema.json`. Its common logical shape is: + +```text +{ + schema_version: "agent_run_result.v1", + identity: { + run_id: UUIDv7, + idempotency_key: string + }, + status: "completed" | "cancelled" | "infrastructure_error", + post_run_mode: "none" | "requested", + effective_limits: { the ten required request limit fields }, + workspace_provenance: DirectWorkspaceProvenance | null, + runtime_provenance: RuntimeProvenance | null, + agent: AgentResult | null, + post_run: PostRunEvidence | null, + usage: Usage, + timing: Timing, + trajectory: InlineTrajectory | ArtifactTrajectory | null, + error: RunError | null +} + +DirectWorkspaceProvenance = { + kind: "direct", + working_directory: relative path or ".", + sources: ordered array of DirectSourceProvenance +} + +RuntimeProvenance = { + runtime_profile_id: logical ID, + profile_digest: SHA-256 digest, + image_digest: SHA-256 digest, + sandbox_policy_version: string, + tools: array of { + name: logical ID, + version: string, + implementation_digest: SHA-256 digest + }, + services: array of { + name: logical ID, + version: string, + implementation_digest: SHA-256 digest, + image_digest: SHA-256 digest | null + } +} + +AgentResult = { + kind: "codex" | "omp", + model: logical ID, + adapter_version: string, + termination: AgentTermination, + exit_code: integer | null, + final_output: CapturedText | null +} + +CapturedText = + | { + storage: "inline", + encoding: "utf-8", + text: string, + digest: SHA-256 digest, + size_bytes: integer >= 0, + truncated: boolean + } + | { + storage: "artifact", + encoding: "utf-8", + artifact: ArtifactReference, + truncated: boolean + } + +ArtifactReference = { + artifact_id: UUIDv7, + media_type: IANA media type, + digest: SHA-256 digest, + size_bytes: integer >= 0, + expires_at: UTC RFC 3339 timestamp +} + +PostRunEvidence = { + commands: array in request order of PostRunCommandObservation, + output_files: array in request order of OutputFileObservation +} +``` + +`completed`, `cancelled`, and `infrastructure_error` describe only gateway lifecycle. The gateway never returns behavioral pass, fail, or reward. `completed` requires nonnull workspace and runtime provenance, agent result with nonnull final output, and trajectory, requires `error: null`, and requires nonnull post-run evidence exactly when `post_run_mode` is `requested`. Cancelled and infrastructure-error results may carry partial or null provenance, agent, and trajectory fields and require a structured error. Runtime provenance records what actually ran without exposing policy bodies, host paths, credentials, private endpoints, or mutable runtime aliases. + +Post-run evidence always preserves request order. Command observations are discriminated as `completed`, `timed_out`, `not_run`, or `unavailable`; output-file observations are `collected`, `missing`, `limit_exceeded`, `not_regular_file`, `not_run`, or `unavailable`. Each observation is sealed durably as it becomes terminal. For cancellation or infrastructure failure before the post-run skeleton is durable, `post_run` may be null. Once it is durable, `post_run` is nonnull and contains the full requested vectors: durable observations followed by explicit `not_run` or `unavailable` entries. Already durable evidence is never discarded or reordered. +A `completed` command observation records `command_id`, exit or signal termination, duration, and bounded stdout and stderr; `timed_out` records its signal, duration, and bounded streams. `not_run` and `unavailable` record a structured reason. Every output-file observation retains its requested name and uses its discriminant to distinguish a collected `ArtifactReference` from missing, over-limit, non-regular, not-run, and unavailable outcomes. + + +A nonzero command exit, configured command timeout, missing file, non-regular file, or requested-file limit breach is raw evidence in a completed run. It is not an infrastructure or behavioral failure. Referenced-bundle failure, approved-executable resolution or launch failure, sandbox failure, inability to persist promised evidence, or cleanup failure is `infrastructure_error`. + +Direct provenance contains the ordered canonical Git, OCI, and uploaded-bundle identities, access modes, materializer versions, destinations, and resolved digests. `AgentRunResult v1` has no imported-task or alternate-backend provenance variant. + +V1 does not return a generic workspace diff, reconstructed or modified workspace, checkpoint, patch, or change artifact. Post-run mutations are permitted only in sources declared `writable` or in separate writable build paths; they are discarded with the sandbox and are not agent output or a persisted caller artifact. Every direct run removes its mutable workspace and releases cache leases before terminal result visibility. + +## Bundle and result-artifact API + +Every bundle reservation, uploaded bundle, run, status, cancellation target, result artifact, and result is owned by one authenticated tenant. Lookups use `(authenticated tenant, object identity)` and cross-tenant access receives the same non-enumerating denial as an unknown object. UUIDs and digests are identities, never authorization. + +`POST /v1/bundles` accepts the closed reservation `{schema_version: "bundle_reservation_request.v1", upload_id: UUIDv7, media_type: "application/vnd.allagents.gateway-bundle.v1.tar+gzip", digest: SHA-256 digest, size_bytes: positive integer}`. `(tenant, upload_id)` is idempotent: an exact retry returns the original immutable reservation, while reuse with different canonical content returns conflict. `PUT /v1/bundles/{artifact_id}` is tenant-authorized, accepts exactly the reserved byte count, verifies the streamed digest, and marks the bundle usable only after both checks pass. + +`GET /v1/artifacts/{artifact_id}` is an authenticated, tenant- and run-authorized download of immutable result bytes. For an authorized live reference, `Content-Type`, `Content-Length`, and `Digest` match the `ArtifactReference`; after `expires_at` it returns `410 Gone`. Unknown, cross-tenant, wrong-run, and otherwise unauthorized identities use one non-enumerating denial. Artifact expiry and retention cover every artifact-backed agent output, command stream, requested file, and trajectory. + +The Promptfoo provider dereferences every artifact it exposes to a grader through this endpoint, verifies those response headers, streams exactly `size_bytes`, computes the declared digest, and rejects expired, short, long, wrong-media-type, or mismatched content. Unverified artifact bytes never enter grading. + +## Isolation and trust guarantees + +An authorized `runtime_profile_id` resolves during admission to a persisted immutable `profile_digest`; the worker executes only that pinned revision and rejects digest mismatch or unavailability. The revision fixes the runtime image digest, read-only runtime `PATH`, sandbox policy, tool implementation closures, and task-service implementations. `RuntimeProvenance` reports the logical ID and profile digest, image and sandbox identities, each tool's implementation digest, and each service's implementation digest plus container image digest where applicable. These are canonical nonsecret identities: tool digests cover the executable and immutable dependency closure, service implementation digests cover the canonical launch implementation, and secret arguments and host paths are never returned. + +Every direct execution uses the profile's `RunSandbox` security baseline: a non-root process under an unprivileged UID/GID mapping; an empty capability set with `no_new_privs`; private PID, mount, IPC, UTS, and network namespaces; a read-only runtime root; and only the declared workspace/build mounts writable. Host `/proc` and `/sys` surfaces, cgroup control files, device nodes, host sockets, container APIs, and control-plane mounts are masked or absent. A versioned explicit seccomp allowlist applies to agent and post-run processes. Cgroup v2 enforcement of PID count, memory, CPU, and I/O is mandatory in addition to wall-time, workspace, file-count, and output limits. A backend that cannot establish this baseline must reject the run rather than weaken isolation. + +Each direct run receives a newly created sandbox and composed workspace. No mutable filesystem, process, home directory, or agent session is reused between runs. Concurrent runs may share only operator-owned immutable source generations; they never share writable workspace state, home state, process state, or post-run output. + +The gateway caches each admitted source as an immutable materialized generation. A Git generation key binds the canonical source origin, resolved exact commit, requested history mode and depth, and materializer-schema version. An OCI generation key binds the canonical repository, exact direct descriptor, and materializer-schema version. An uploaded bundle generation key binds its artifact digest and materializer-schema version. Authorization, internal policy identity, and credential route are deliberately separate from the content key. Before every attachment, including every cache hit, the gateway remaps the authenticated caller and canonical origin or repository and rechecks authorization for the exact resolved identity; bundle hits recheck artifact authorization. A cache key proves identity, not authorization, and a cache hit never bypasses current policy. + +Exact-key misses, fetches, and unpacks are singleflighted so concurrent runs do not refetch or rematerialize the same large source. Complete generations are immutable and operator-owned. A run holds an eviction lease from attachment through cleanup. Cache roots and unrelated generations are inaccessible to the agent; a generation is visible only through its declared trial mount. + +A `read_only` source mounts the cached generation directly and read-only into every requesting trial. The filesystem rejects writes. Git operations that are valid against a read-only repository run with optional locks disabled; any Git operation requiring a lock or write fails rather than mutating shared state. A `writable` source receives a private reflink or other copy-on-write clone, with a full private copy as the correctness fallback. Hardlinks and any other shared writable alias to cached content are forbidden. + +Agent and post-run commands see the same per-source access modes. A check that compiles in-tree, installs dependencies into a source, creates a database there, or otherwise writes beneath a source must mark it `writable`; alternatively it must direct build and test output to a separate writable path. Read-only fixtures and repositories remain read-only throughout both phases. + +The gateway control plane remains outside worker, agent, and post-run command authority. Within a worker, the runner enforces process, time, CPU, memory, storage, output, and phase-specific network limits and can terminate the complete agent-owned descendant process tree. Gateway records, runner control files, result storage, and cleanup authority are inaccessible to the agent and evidence commands. + +Source, registry, model, optional check-bundle, and artifact credentials are resolved from operator-controlled configuration. They are absent from caller JSON, workspace contents, logs, trajectories, and returned evidence. Source acquisition and model authentication occur through gateway- and runner-owned mechanisms that do not disclose credentials to workspace commands. A referenced check bundle is authorized before the run, but its bytes and populated mount are absent from the agent sandbox; only an empty, runner-owned, non-writable reserved mountpoint may exist. Evidence commands receive no acquisition or model credentials. + +Network access is phase-specific, namespace-separated, brokered, and deny-by-default. Source acquisition follows the resolved internal source route. Agent execution runs in its own fresh network namespace under `agent_network_policy_id`; post-run evidence runs in a different fresh network namespace under `post_run.network_policy_id`. The default-drop rule covers external interfaces and loopback. A policy can expose only operator-declared broker endpoints for the phase, including individually named task-service endpoints; it cannot implicitly expose the worker host or another phase. The caller cannot submit hosts, ports, CIDRs, headers, proxies, service definitions, credentials, or policy bodies. + +The phase transition is ordered and fail-closed. The runner first terminates every agent-owned descendant, destroys the agent network namespace and all agent flows, removes model and source credentials from the runtime, environment, and runner-managed auth material, and verifies those steps. Only then may it materialize the authorized bundle into the reserved mountpoint read-only, expose it to post-run commands, and create the post-run network namespace with a newly resolved endpoint set. A task service may remain supervised across the transition, but it is unreachable unless the pinned runtime profile and active phase policy both authorize its broker endpoint. Namespace teardown, rather than a best-effort firewall rewrite, prevents an agent-opened loopback or sidecar connection from surviving. + +The runner keeps the same pinned runtime and final workspace with declared source access modes alive across this transition. Runtime executables resolve only through the pinned tool mapping; bundle executables resolve only beneath the newly populated bundle root; arguments are literal and never interpreted by a shell. Commands may build in writable paths, write test artifacts, and exercise authorized task services, but cannot depend on arbitrary agent-owned background processes or mutate `read_only` sources. The runner records raw outcomes and only requested bounded paths. Referenced-bundle acquisition failure or runner failure is `infrastructure_error`; a command's own nonzero exit or timeout remains completed-run evidence. + +Cleanup is mandatory, idempotent, and part of terminalization. After persisting available evidence, the worker stops every remaining evidence process and operator task service, unmounts every trial mount, deletes the private trial root, and releases every cache lease. The gateway does not expose a terminal result until all cleanup steps succeed. If any cleanup step initially fails, the run's eventual status is `infrastructure_error`, not `completed`; its available evidence is marked partial, and reconciliation finishes cleanup without rerunning the agent before the terminal error becomes visible. + +## V1 execution scope + +V1 supports only direct execution. The AllAgents Gateway admits and tracks the run; a worker and runner own ordered workspace composition, sandboxing, Codex or OMP execution, the credential-stripping phase transition, optional post-run bundle injection, runtime or bundle evidence commands in the same pinned environment, raw result construction, cleanup, and terminalization. + +Harbor and Terminal-Bench imports are deferred entirely beyond V1. Any future adapter requires a separate ADR and a new request/result schema version that defines its ownership, provenance, isolation, and evidence semantics. V1 makes no Harbor source, backend, provenance, or result-mapping promise. + +## Idempotency, retries, and cancellation + +`run_id` identifies the logical run and `idempotency_key` protects submission. After authentication, closed-schema validation, and canonical request hashing—but before mutable source admission, capacity allocation, artifact consumption, or creation of a public run—the gateway looks up both identities under the authenticated tenant. + +- An exact tenant-scoped duplicate returns the same run handle or terminal result without re-running current admission or starting new work, including after client timeout, policy change, or worker restart. +- Reusing either identity for a different canonical request is a conflict and starts nothing. +- A genuinely new request first acquires an internal tenant-scoped identity claim so concurrent duplicates converge. The gateway then performs every policy, bundle, source, limit, runtime-profile, and capacity admission check. A rejected new request releases that claim and leaves no public run or job record. +- Only successful admission atomically commits the immutable canonical request and public run before dispatch. Recovery can finish dispatch from that record but cannot admit or start a second agent. +- Promptfoo repetitions use distinct identities even when every evaluation input is otherwise identical. + +Transport retries and result polling are safe because they do not rerun the agent. The gateway may reconcile or resume its durable orchestration around a running request, but no worker or runner automatically starts the agent a second time after an attempt has begun. An infrastructure failure is terminal for that run. An intentional rerun requires a new run identity and therefore a fresh workspace; the caller, not the gateway, decides whether to schedule it. + +Cancellation is an idempotent request against `run_id`. Before execution it prevents agent start; during execution it terminates the full descendant process tree; during post-run collection it terminates evidence commands. Once cancellation wins the terminal-state race, the worker preserves bounded partial evidence and performs the same process/service stop, unmount, private-root deletion, and cache-lease release. `cancelled` becomes visible only after cleanup succeeds; any cleanup failure changes the eventual status to `infrastructure_error` after reconciliation finishes cleanup. If a terminal result was published first, later cancellation returns that unchanged result. Cancellation never preserves a workspace for continuation. + +Failures in admission, materialization, sandbox control, agent launch or supervision, referenced check-bundle acquisition, evidence-command supervision, artifact storage, cleanup, and result commitment use stable structured error codes. A `retryable` flag tells the caller whether a new run might succeed; it never authorizes the gateway or worker to silently rerun the agent. Cleanup failure produces `infrastructure_error` with partial evidence after reconciliation completes cleanup. The gateway never converts infrastructure failure or command evidence into pass, fail, or reward. + +## Rejected alternatives + +### UHP through HarnessRouter + +UHP and HarnessRouter are designed around reusable harness sessions: response and session identity, continuation, checkpoint and hydrate, produced-file collection, and session deletion. The AllAgents Gateway is similar in purpose but V1 needs one isolated run, captured agent output, and optional raw post-run evidence. Adopting the session protocol would force the gateway to define which state survives, how continuation interacts with caller retries, and how checkpoints and produced files become run results even though none are required. + +A HarnessRouter fork would also make AllAgents carry upstream integration seams and release work for workspace initialization, journaling, and terminal finalization. The former issue #304 proposal pursued those seams for the superseded snapshot design. Borrowing its sound protocol habits is useful; retaining its execution architecture is not. + +We therefore reject UHP conformance, a HarnessRouter distribution or fork, checkpoints, continuation, and reusable coding sessions for this system. + +### Two repositories and published workspace snapshots + +Separating workspace construction into a builder repository and passing published OCI snapshots into a session gateway optimized an imagined reuse boundary. In a one-shot run, the worker already owns acquisition, isolated materialization, execution, optional evidence collection, and destruction as one lifecycle. Splitting that lifecycle adds artifact publication, leases, cross-repository versioning, cache authorization, and recovery states without improving the run result. + +OCI remains a valid immutable workspace input, not the mandatory handoff between two AllAgents systems. One repository keeps the provider, public contract, lifecycle, and agent adapters versioned and tested together. + +### Generic workspace diffs as product output + +A generic diff says which bytes changed, not what the agent reported or what checks observed. It is ambiguous around generated files, ignored files, nested repositories, modes, links, and task-specific equivalence; it can also expose unnecessary source content. Making diffs authoritative would require baseline retention and reconstruction machinery that a one-shot result does not need. + +For Promptfoo, code or LLM graders consume agent output, raw command results, and requested file artifacts and emit the task-specific outcome and reward outside the gateway. Other callers consume the same uninterpreted evidence. No caller receives modified workspace state in V1. Any future patch or workspace export requires a separately approved, explicit artifact contract rather than an extension of the core result by convention. + +## Consequences + +Positive consequences: + +- the architecture supports a general one-shot coding-agent gateway while matching Promptfoo's unit of work: one independently graded run; +- a reader can locate authoring and grading in Promptfoo, public lifecycle in the gateway, and execution and raw evidence collection in a worker and runner; +- every run starts clean and cannot inherit a prior coding session; +- concurrent runs can reuse large immutable Git, OCI, and bundle generations without sharing mutable trial state or refetching exact sources; +- idempotent network retries cannot duplicate agent work; +- gateway lifecycle remains separate from caller-owned behavioral grading; +- credentials and operator policy stay on the operator side of a small closed boundary; +- Codex and OMP share isolation, output, evidence, and lifecycle semantics without pretending their CLIs are identical. + +Costs and constraints: + +- workers and the runner must implement strong sandbox, process-tree, credential, network, artifact, and cleanup controls rather than inheriting them from a session service; +- there is deliberately no interactive continuation, modified-workspace export, or post-run workspace recovery; +- local inputs and check bundles incur a packaging and upload step before remote execution; +- immutable source generations require bounded cache storage, eviction leases, singleflight recovery, authorization on every attachment, and materializer-schema migrations; +- callers must choose source access deliberately; agents and checks that write in-tree require `writable`, while `read_only` sources require a separate writable build path; +- check bundles and evidence commands are security-sensitive executable inputs and require authorization, immutability, isolation, and resource bounds; +- closed contracts require explicit versioning when new workspace, agent, post-run evidence, or artifact capabilities are introduced; and +- debugging and grading rely on bounded agent output, trajectory, command results, and file artifacts because the mutable workspace is destroyed. + +We will reconsider this decision if the product requires reusable interactive coding sessions rather than one-shot runs, or if direct execution can no longer provide the required isolation guarantees. A need for more Promptfoo matrices, grader types, benchmark adapters, or raw evidence does not by itself justify UHP sessions, HarnessRouter, snapshot publication, gateway-owned grading, or generic workspace diffs. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index a04aa32c..7aaa1608 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,5 +1,5 @@ --- -title: "Prepared Workspace Snapshots - Cross-Repository Implementation Plan" +title: "One-Shot Coding-Agent Gateway - Implementation Plan" date: 2026-09-18 updated: 2026-09-28 type: feat @@ -8,681 +8,1041 @@ artifact_readiness: implementation-ready execution: code --- -# Prepared Workspace Snapshots - Cross-Repository Implementation Plan +# One-Shot Coding-Agent Gateway - Implementation Plan -## Goal +## Goal capsule -Deliver reproducible large coding workspaces through two narrow components: +- **Objective:** Run one Codex or OMP coding-agent attempt in a fresh composed workspace and return bounded output, usage, bounded trajectory, provenance, and optional raw post-run evidence with authenticated artifact retrieval. +- **Means:** Build one repository and service, `allagentsdev/allagents-gateway`, containing the Promptfoo provider, gateway API, worker/runner, contracts, workspace composition, post-run collector, and agent adapters. +- **First caller:** Promptfoo is the evaluation layer. It owns rendered prompts, workspace-source JSON, matrices, repetitions, JavaScript/LLM graders, pass/fail decisions, scores, and reports. +- **Stop conditions:** Do not build UHP, a HarnessRouter fork, reusable sessions, continuation, checkpoints, composed-workspace/snapshot caches, generic diffs, change artifacts, or a second repository. -1. **AllAgents Workspace Builder** prepares Git/OCI inputs, preserves required Git histories, publishes one immutable OCI workspace snapshot, and returns its direct manifest descriptor. -2. **AllAgents Gateway** consumes only that exact descriptor, authorizes and materializes a private writable session directory, runs the existing HarnessRouter/UHP lifecycle, and returns exact filesystem deltas. +The authoritative decision is [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). Supporting evidence is in [One-shot coding-agent gateway boundary](../research/one-shot-coding-agent-gateway-boundary.md). -The execution gateway MUST NOT resolve Git refs, receive source credentials, compose repository arrays, or call the builder during task execution. The cross-repository contract is the versioned OCI artifact. A new snapshot session is runnable only after descriptor authorization, artifact verification, private materialization, working-directory validation, and baseline-journal creation complete. +## Product contract -Requests without the vendor snapshot extension remain on characterized stock HarnessRouter behavior. +`allagentsdev/allagents-gateway` is a general one-shot coding-agent gateway. A run receives one rendered instruction and an ordered list of authorized Git, OCI, or uploaded-bundle sources, each declared `read_only` or `writable`. The worker resolves exact immutable source identities, reauthorizes and leases complete source-cache generations, mounts read-only sources directly, and makes private reflink/CoW clones for writable sources. It composes those non-overlapping destinations in one fresh sandbox, invokes one selected agent exactly once, stops and reaps the agent-owned process tree, seals the agent output/usage/trajectory record, and retains the same sandbox, runtime image, and final writable workspace for optional evidence collection. -## Architecture decision +A request may add `post_run` evidence collection. Only after the agent has stopped, the worker strips model/source credentials, destroys the agent network namespace and flows, creates a distinct default-drop check namespace, conditionally mounts an immutable hidden bundle for bundle executables, and executes structured commands in the same retained sandbox and final workspace. Commands select either an operator-approved executable from the runtime profile's fixed read-only `PATH` or a confined executable from that optional bundle, plus literal arguments. All external and service access is denied by default and exists only through endpoints brokered for the named post-run policy; direct loopback, host, and sidecar sockets stay unreachable. Commands may compile, test, start short-lived children, and create build output. The gateway returns raw command observations and requested files as bounded inline values or authenticated tenant/run-bound artifact references. -The authoritative decision is [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md). The evidence is [Prebuilt immutable workspace snapshots at the HarnessRouter boundary](../research/allagents-gateway-snapshot-boundary.md). +The gateway never decides whether behavior passes, fails, or deserves a reward. The Promptfoo provider exposes agent and post-run evidence to Promptfoo JavaScript or LLM graders, which own those decisions. -Two repositories are intentional: +The lifecycle status is only `completed`, `cancelled`, or `infrastructure_error`. Agent launch/crash/timeout, requested post-run bundle/sandbox/executable-resolution/command-launch failure, artifact-store failure, result-finalization failure, or cleanup failure is infrastructure failure. Every run destroys its private clones, workspace mounts, any injected bundle, and scratch storage and releases cache leases before its terminal result is published. -| Repository | Owns | Must not own | -|---|---|---| -| `allagentsdev/allagents-workspace-builder` | Git/OCI acquisition, credentials, source policy, composition, provenance, canonical baseline, OCI publication, build retention | UHP, harness/provider routing, session continuation, produced-file ACK, response lifecycle | -| `allagentsdev/allagents-gateway` | UHP, descriptor admission, authorization, immutable cache, private session tree, manifest journal, Files/artifacts, checkpoint, continuation, cancellation, deletion | Git refs, source credentials, caller-selected registries, repository arrays, composition jobs | +V1 supports **direct mode** only, owned completely by this repository. -This is not a mandate for two always-on services. V1 builder delivery is a CLI/library suitable for CI or a job worker. A service wrapper is optional and must preserve the same artifact contract. +## Scope -The upstream change is tracked in [HarnessRouter issue #304](https://github.com/HarnessRouter/harnessrouter/issues/304). The proposal covers an immutable workspace initializer, an explicit-cursor non-Git journal, and recoverable terminal finalization; it does not upstream AllAgents source semantics or require a UHP change. +### Included -## Repository and branch prerequisites +- One TypeScript repository with a Promptfoo provider, gateway API, worker/runner, contracts, Codex adapter, OMP adapter, post-run collector, and Linux isolation backend. +- Closed `AgentRunRequest v1`, `AgentRunResult v1`, status-envelope, bundle-upload, and post-run JSON Schemas. +- Promptfoo local and remote provider modes using the same gateway API and request/result contract. +- Promptfoo-authored ordered multi-source workspace JSON. +- Deterministic packaging of provider-local source paths and optional post-run bundle paths before gateway submission. +- Bounded composition from one through 128 authorized Git, OCI, and uploaded-bundle sources with required `read_only` or `writable` access. +- An immutable exact-source cache with authorization on every use, exact-key singleflight, complete generations, leases, watermarks, and startup reconciliation. +- A fresh private sandbox, process hierarchy, mount tree, and writable source clone for every run. +- One agent invocation with idempotency, cancellation, deadlines, cleanup, bounded final output, usage, timing, and bounded ATIF trajectory. +- Optional ordered post-run commands and requested-file collection in the same runtime and final workspace after agent stop and credential stripping. +- Operator-managed task services with separate lifecycle ownership from agent-created processes. +- Resolved provenance for every source in request order. -### AllAgents Gateway +### Excluded -`allagentsdev/allagents-gateway` already exists as the renamed HarnessRouter fork. It preserves the fork relationship, full history, Apache-2.0 `LICENSE`, `NOTICE`, attribution, settings, and redirects. `HarnessRouter/harnessrouter` remains the upstream parent. +- Pass/fail, reward, rubric, or grading fields and decisions in the gateway contract. +- UHP endpoints, objects, conformance, or compatibility. +- HarnessRouter code, forking, routing, or provider abstractions. +- Long-lived or reusable coding sessions, additional turns, continuation, resume, replay, or checkpoints. +- Reusable composed workspaces, mutable source caches, unkeyed Git clones, prepared snapshots, or OCI workspace publication. Immutable exact-source generations are required only as specified below. +- A workspace-builder service or repository. +- Generic before/after diffs, patch output, or a core modified-workspace artifact. +- Persistence of a non-evaluation run's modified workspace in V1. That requires a separately approved artifact contract. +- Caller-provided credentials, environment variables, shell commands, container images, raw network rules, policy documents, model endpoints, or proxies. +- Automatic agent or post-run retry. Promptfoo repetitions are distinct runs; transport retry reuses one idempotency identity. -Before implementation: +## Fixed implementation choices -1. Fetch current downstream `main` and upstream. -2. Record the accepted characterized baseline `5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3` and current inspected point `8f7868ccb2c97d1f611acf11e7cad0357a43064e`. -3. Create `feat/workspace-snapshots` from current downstream `main`. -4. Retire the unused `feat/workspace-composition` branch after confirming it contains no unique implementation. Do not carry runtime-composition scaffolding, aliases, or dead request fields. -5. Keep `origin` on `allagentsdev/allagents-gateway` and `upstream` on `HarnessRouter/harnessrouter`. +Implementation must not choose another framework partway through delivery. -### AllAgents Workspace Builder +- **Repository and service:** `allagentsdev/allagents-gateway`, Apache-2.0, protected `main`, release tags, lockfile, generated schemas checked in. +- **Runtime:** Bun 1.4 for development, tests, packaging, and worker execution; strict TypeScript; ESM; Node 22-compatible Promptfoo provider output. +- **HTTP:** Fastify 5 with strict Ajv validation against checked-in schemas. External JSON is not translated through a second hand-maintained DTO shape. +- **Persistence:** PostgreSQL stores tenant-owned upload reservations, runs, idempotency claims, state, deadlines, artifact budgets, and cache leases. An S3-compatible store holds immutable tenant/run-bound input bundles and result artifacts for agent output, command output, requested files, and trajectories. Local mode uses SQLite plus a private local artifact directory behind the same interfaces. +- **Source cache:** one V1 worker pool is one cache domain: all worker processes share an operator-owned cache root on the same reflink/CoW-capable filesystem and coordinate exact keys through PostgreSQL plus atomic filesystem publication. This guarantees one materialization for concurrent exact-key requests in the pool. Additional independent cache domains are explicit deployments and may materialize separately. The cache has a versioned materializer schema, ready markers, leases, byte/inode watermarks, exact-key singleflight, and reconciliation; it never caches a composed workspace. +- **Queue:** PostgreSQL-backed leased jobs in V1. Lease recovery may resume safe control-plane work but may not invoke an ambiguously started agent again. +- **Runtime profiles:** required `runtime_profile_id` selects a caller-authorized operator profile. Admission resolves and persists one immutable profile revision plus its `profile_digest`, runtime `image_digest`, sandbox-policy version, read-only tool implementations, phase-specific broker identities, cgroup-v2 ceilings, and service implementations/images. Dispatch uses only that pinned revision and fails before agent invocation if it is unavailable or any digest differs. The result returns sufficient immutable, nonsecret provenance to identify the exact runtime/tool/service implementations without exposing paths, launch arguments, or policy contents. +- **Isolation:** production `RunSandbox` uses an unprivileged UID/GID mapping and non-root process identity, empty Linux capabilities, `no_new_privs`, private PID/mount/IPC/UTS/network namespaces, a read-only root filesystem, minimal bounded tmpfs mounts, masked `/proc` and no exposed `/sys`, cgroup filesystem, host devices, Docker/container sockets, worker sockets, or writable control-plane mounts. A versioned explicit seccomp allowlist denies by default. Worker-owned cgroup v2 controllers mandatorily cap pids, memory, CPU, and IO for service, agent, and each check group; sandbox processes cannot modify membership or limits. The retained mount/runtime stays alive through post-run, but phase process and network namespaces are distinct. Rootless development is allowed; production is incomplete until this backend passes breakout, namespace, fork-bomb, OOM, CPU, IO, device/socket, mount, and syscall conformance. +- **Phase networking:** agent and post-run checks use different short-lived network namespaces with default-drop nftables (or an equivalent kernel enforcement point) covering external, sidecar, and loopback traffic. They receive only operator-brokered endpoints authorized by their phase policy; no direct host/sidecar socket is reachable. The worker destroys the agent namespace, conntrack/flows, and broker handles before creating the check namespace. Post-run-only services are therefore unreachable to the agent even on localhost. +- **Validation:** JSON Schema is authoritative at public and subprocess boundaries. TypeScript types are generated from it. Every owned object uses `additionalProperties: false`; unions use `oneOf` with a required discriminator. +- **Identity and time:** UUIDv7 run and artifact IDs; UTC RFC 3339 timestamps; monotonic internal durations; lowercase SHA-256 digests; byte sizes; millisecond limits. +- **Trace:** ATIF v1 is the only V1 public trajectory format. Vendor and pin its exact schema; adapters convert native events instead of creating another event model. -Create public repository `allagentsdev/allagents-workspace-builder` with Apache-2.0 licensing, protected `main`, required checks, immutable release tags, GHCR package access, and a security policy. Start implementation on `feat/workspace-snapshot-v1`. +## Actors and ownership -The repository MUST produce one pinned CLI binary and one reusable package from the same code. Prefer a small static implementation suitable for untrusted archive handling; pin the language toolchain and every dependency in the initial bootstrap. The CLI surface is: +| Actor | Owns | Must not own or receive | +|---|---|---| +| Promptfoo | Datasets, variables, prompt rendering, ordered source JSON, matrices, repetitions, JS/LLM grading, pass/fail, scores, reports | Source/model credentials, gateway policy bodies, a live workspace | +| `@allagents/promptfoo-provider` | Stable Promptfoo invocation/upload identity, local path packaging, idempotent upload/submission, polling/cancel mapping, authenticated artifact download/verification, result/evidence mapping | Agent execution, grading decisions, automatic run retry | +| Gateway API | Authentication, closed-schema validation, tenant-scoped object access, duplicate-before-admission idempotency, admission, state reads, cancellation, deterministic caller/source-address mapping, bundle ownership, model/network/runtime logical-name authorization, immutable result-artifact delivery, worker dispatch | Host paths, caller secrets, source credential selectors, policy bodies, workspace mutation | +| Worker/runner | State machine, deadlines, composition sequencing, sandbox/profile allocation, one agent invocation, process/network ownership boundaries, bounded output/artifact allocation, durable post-run sequencing, result finalization, cleanup | Prompt rendering, matrix expansion, grading | +| Workspace composer | Ordered attachment of cached immutable generations at non-overlapping destinations, access-mode enforcement, source-by-source provenance | Overwrite precedence, mutable shared trees, composed-workspace caches | +| Source cache | Exact-key materialization singleflight, immutable complete generations, leases, watermarks, eviction, reconciliation | Authorization decisions, writable agent aliases, visibility inside sandboxes except leased read-only mounts | +| `RunSandbox` | Pinned profile runtime, unprivileged private mounts/namespaces, seccomp and cgroup-v2 enforcement, distinct default-drop service/agent/check network boundaries, stop/kill boundaries | Evaluation semantics | +| Task service manager | Operator-declared sidecar startup/readiness/restart/stop in the service cgroup | Agent-created daemons, caller-defined service commands, grading | +| Codex adapter | Pinned Codex invocation, native-event normalization, final output, usage | Acquisition, post-run collection, retries | +| OMP adapter | Pinned OMP invocation, native-event normalization, final output, usage | Acquisition, post-run collection, retries | +| Post-run collector | Late bundle acquisition/verification/read-only mount after agent teardown, pinned-profile runtime and bundle-executable resolution, literal-argument execution in the retained sandbox/workspace, bounded output capture, requested-file promotion | Model/source credentials, grading, arbitrary external egress | +| Artifact store | Immutable tenant-owned input bundles and tenant/run-bound result artifacts for agent output, command output, requested files, and trajectories | Mutable run state, prepared workspaces, policy decisions | + +## Repository layout ```text -allagents-workspace-builder build \ - --context /run/secrets/build-context.v1.json \ - --spec workspace-build.v1.json \ - --output descriptor.json +allagents-gateway/ + apps/gateway/ # Fastify admission, status, upload, cancellation API + apps/worker/ # leased-job consumer and runner entrypoint + packages/contracts/ # schemas, generated TS types, canonicalization + packages/runner/ # state machine and direct-mode coordinator + packages/promptfoo-provider/ # local/remote provider and deterministic packager + packages/workspace/ # ordered source attachment and access modes + packages/source-cache/ # exact-key immutable generations/leases/eviction + packages/sandbox-linux/ # retained runtime, cgroups, mounts, network policy + packages/services/ # operator-declared task sidecar lifecycle + packages/adapter-codex/ # pinned Codex adapter + packages/adapter-omp/ # pinned OMP adapter + packages/post-run/ # hidden checks, output capture, file collection + packages/trace-atif/ # native events to bounded ATIF v1 + schemas/ # generated public schemas, checked in + tests/fixtures/ # local Git/OCI/bundle/agent/check fixtures ``` -A later queue/API wrapper invokes the same package. It does not define a second artifact format or source-resolution path. - -### Shared fixtures, not shared source +The runner depends only on `RunStore`, `ArtifactStore`, `SourceCache`, `WorkspaceComposer`, `RunSandbox`, `TaskServiceManager`, `AgentAdapter`, and `PostRunCollector`. It must not import Fastify, Promptfoo, Codex, or OMP types. The gateway API never accesses a run directory; the worker never interprets Promptfoo evaluation metadata. -Each repository owns local focused fixtures. Cross-repository E2E uses published fixture snapshots pinned by digest. Do not share code through Git submodules, relative checkouts, unpublished packages, or branch references. If schema generation is added, publish a versioned schema artifact and check generated outputs in each consumer. +## Public contract -Required fixture classes: +### Shared scalar and path rules -- one root Git repository with complete merge history; -- two independent nested Git repositories; -- one Git-free OCI tree; -- composition with whiteouts, empty directories, symlinks, executable files, and safe hardlinks; -- malformed manifests, configs, layers, links, paths, types, sparse files, compression bombs, metadata bombs, and digest mismatches; -- source trees colliding with versioned runner-reserved paths; and -- a snapshot large enough to exercise cache watermarks, quota rejection, reflink, and full-copy fallback. +- All owned objects are closed. Unknown fields fail before a run record or directory exists. +- Strings are UTF-8, contain no NUL, and obey their byte bounds. +- Logical IDs match `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`. +- Digests match `^sha256:[0-9a-f]{64}$`. +- Relative paths are NFC-normalized POSIX paths, are not absolute, and contain neither empty, `.` nor `..` segments. `destination` and `working_directory` may be exactly `.` where explicitly allowed. +- Byte sizes and millisecond limits are safe positive JSON integers. Counts may be zero where stated. +- Timestamps are UTC RFC 3339 with `Z`. +- A bundle or artifact reference is accepted only when declared digest, declared size, stored size, and streamed digest all agree. -## Secrets and operator-owned inputs +### `AgentRunRequest v1` -No implementation or fixture requires production credentials. Local authenticated registries, local Git servers, and mocked provider transport prove behavior. +The authoritative file is `schemas/agent-run-request.v1.schema.json`. Its exact logical shape is: -Operator-owned values remain outside source and tests: - -- source and snapshot registry credentials; -- Git credentials and network allowlists; -- snapshot authorization-catalog entries; -- UHP caller credentials; -- provider gateway URL/key; -- GHCR publish rights; and -- production deployment, object-store, quota, and telemetry configuration. - -Secrets MUST enter through each repository's secret mechanism. They MUST NOT appear in build specs, OCI config/provenance, layers, logs, errors, response metadata, checkpoints, SBOMs, build arguments, or harness environments. +```text +{ + schema_version: "agent_run_request.v1", + identity: { + run_id: UUIDv7, + idempotency_key: string, 16..128 characters + }, + instruction: string, 1..1_000_000 UTF-8 bytes, + workspace: { + working_directory: relative path or ".", + sources: array of 1..128 GitSource | OciSource | UploadedBundleSource + }, + agent: CodexAgent | OmpAgent, + runtime_profile_id: logical ID, + post_run: null | PostRunSpec, + agent_network_policy_id: logical ID, + limits: { + total_timeout_ms: positive integer, + acquisition_timeout_ms: positive integer, + agent_timeout_ms: positive integer, + post_run_timeout_ms: positive integer, + max_workspace_bytes: positive integer, + max_workspace_files: positive integer, + max_agent_output_bytes: positive integer, + max_artifact_bytes: positive integer, + max_trace_bytes: positive integer, + max_command_output_bytes: positive integer + } +} +``` -All credentialed Git and OCI fetchers share one minimum transport policy: authorize scheme, host, port, and every resolved IP before connection; reject loopback, link-local, private, Unix-socket, rebinding, and other non-allowlisted targets; cap and reauthorize redirects; never forward authorization, cookies, or client certificates across origins; require TLS; use repository-scoped short-lived credentials; and redact redirect/auth diagnostics. +`post_run` is required and is either `null` or the closed specification below. A general run with `null` still returns a completed agent result. Promptfoo uses post-run evidence only when its graders need it. -## Characterized HarnessRouter seams +`runtime_profile_id` is required. At admission it resolves to one caller-authorized immutable operator profile revision and canonical `profile_digest`, including the runtime image, sandbox policy, fixed read-only executable `PATH` and tool implementations, phase-specific broker identities, cgroup ceilings, and task-service implementations. The persisted run pins that revision rather than resolving the logical ID again at dispatch. Caller JSON never contains an image, executable path, service command, implementation digest, or sandbox policy body. -Implementation starts from these existing seams rather than a parallel executor: +`max_agent_output_bytes`, `max_trace_bytes`, and `max_command_output_bytes` are independent capture bounds. `max_artifact_bytes` is the aggregate logical byte budget for content-addressed **result** artifacts in one run; input bundles and cache generations are excluded. Admission requires `sum(post_run.output_files[].max_bytes) <= max_artifact_bytes` and reserves that sum. Finalization charges actual collected-file bytes in request order, then releases unused reservation and spends the deterministic remainder on agent output, command stdout/stderr in command/stream order, then trajectory. Every promoted reference charges its full `size_bytes` even when storage deduplicates the digest. Exhausted optional promotion falls back to explicitly bounded inline/truncated or omitted evidence; it is not infrastructure failure and never permits unbounded persistence. -- gateway request/session resolution and `CreateResponseBody.metadata` extension handling; -- gateway `_resp_execute` and `_hydrate`, where initialization must gate input and harness start; -- runner `_ws`, `/hydrate`, `/checkpoint`, and `/turn`; -- runner `_produced_list`, `_produced_ack`, and the internal `/produced` routes; -- gateway `_collect_produced`, which captures artifacts before ACK; -- `BACKING.workspace`, `RunnerWorkspaceFiles`, and `CheckpointWorkspaceFiles`; -- durable `HarnessSession` plus separate checkpoint, artifact, response, and control records; -- existing cancellation, explicit session deletion, live-workspace reaping, graph/blob backing, provider routing, SSE translation, and harness adapters. -The red characterization MUST prove current limitations before code changes: +The three closed source variants are: -1. a root repository is mutated by `_git_ensure`; -2. an embedded repository's internal file changes are not emitted exactly by root Git; -3. a deletion is absent from stock produced-file artifacts; -4. `/hydrate` accepts only the internal checkpoint shape and is not a supported OCI import; and -5. a request without the vendor extension retains current UHP behavior. +```text +GitSource = { + kind: "git", + url: canonical HTTPS URL, 1..2_048 UTF-8 bytes, + ref: string, 1..1_024 UTF-8 bytes, + history: { mode: "full" } | + { mode: "shallow", depth: integer >= 1 and <= 1_000_000 }, + access: "read_only" | "writable", + destination: relative path or "." +} -## Workspace Builder contract +OciSource = { + kind: "oci", + repository: canonical OCI repository, 1..2_048 UTF-8 bytes, + descriptor: { + media_type: "application/vnd.oci.image.manifest.v1+json", + digest: SHA-256 digest, + size_bytes: positive integer + }, + access: "read_only" | "writable", + destination: relative path or "." +} -### Build request v1 +UploadedBundleSource = { + kind: "uploaded_bundle", + bundle: BundleReference, + access: "read_only" | "writable", + destination: relative path or "." +} -The builder accepts a closed versioned build spec plus a trusted out-of-band `BuildContext`. This is an AllAgents preparation contract, not UHP. `BuildContext` contains authenticated `principal`, `product_domain`, and operator-selected `policy_id`; it is supplied by the embedding job/CLI environment, never the build spec. V1 CLI execution is operator-trusted. A later service wrapper MUST authenticate its caller and map it to the same `BuildContext` before invoking the package. +BundleReference = { + artifact_id: UUIDv7, + media_type: "application/vnd.allagents.gateway-bundle.v1.tar+gzip", + digest: SHA-256 digest, + size_bytes: positive integer +} -```json -{ - "version": 1, - "working_directory": "services/api", - "sources": [ - { - "kind": "git", - "url": "https://github.com/example/api.git", - "ref": "refs/heads/main", - "history": {"mode": "full"}, - "destination": "services/api" - }, - { - "kind": "oci", - "source_name": "compiler-tree", - "descriptor": { - "media_type": "application/vnd.oci.image.manifest.v1+json", - "digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111", - "size": 123456 - }, - "destination": "vendor/compiler" - } - ] +ArtifactReference = { + artifact_id: UUIDv7, + media_type: nonempty IANA media type, <= 127 bytes, + digest: SHA-256 digest, + size_bytes: integer >= 0, + expires_at: timestamp } ``` -Rules: +Source order is authoritative for acquisition logs and returned provenance. Destinations are normalized before comparison and must be pairwise non-overlapping: no destination may equal, contain, or be contained by another. `.` is allowed only when it is the sole source destination. The final `working_directory` must resolve within the composed tree, without symlink escape, to a directory. Destination overlap and working-directory syntax are rejected before credentials, network, artifact reads, or run-directory creation. Ordering never grants overwrite precedence. -- `version` is exactly `1`; all objects are closed. -- `sources` contains 1 through 128 ordered entries. -- `destination` is NFC-normalized relative POSIX. `.` is allowed only when it is the sole source. Otherwise destinations are non-root and pairwise non-overlapping. -- `working_directory` is relative to the final snapshot root, defaults to `.`, and must resolve without symlink escape to a real directory. -- Git URLs are canonical HTTPS identities. Userinfo, query, fragment, local paths, alternate transports, and caller Git options are rejected. -- Omitted Git `ref` selects the advertised default. A supplied ref resolves only through advertised full-ref or unambiguous branch/tag semantics. -- Omitted `history` means `{"mode":"full"}`. Shallow mode is `{"mode":"shallow","depth":N}` with integer `N` from 1 through 1,000,000. Full means complete ancestry reachable from the selected commit, not unrelated refs/tags. -- `source_name` selects an operator configuration for registry origin, credentials, TLS, redirects, media types, and trust. Callers do not supply these. -- Unknown kinds, commands, environment, credentials, host paths, runtime images, provider routes, resource-limit overrides, and tags/indexes fail before network access. -- Source and destination authorization is evaluated against `BuildContext` before credentials or network are used. +Canonical source addresses have one accepted serialization. A Git URL is HTTPS with lowercase IDNA host, omitted default port, no userinfo/query/fragment, normalized percent-encoding, and a nonempty absolute repository path. An OCI repository is lowercase `registry[:nondefault-port]/name[/name...]` under OCI Distribution name rules, with no scheme, tag, digest, userinfo, query, or fragment. The gateway parses and reserializes either form and rejects the request if bytes differ; it never silently normalizes an alias before policy lookup or cache-key construction. -### Git preparation +Git `ref` may be a full advertised ref, an unambiguous advertised branch/tag selector, or an exact commit permitted by source policy. It is resolved once to an exact commit. `full` retains ancestry reachable from that commit; `shallow` retains the requested bounded depth. The worker never substitutes a default branch, another ref, another commit, or another history mode. -For each Git source: +The closed agent variants are: -1. resolve allowed credentials and egress policy from `BuildContext`, outside the request; -2. run Git without a shell under sanitized environment/config; -3. disable interactive prompts, hooks, filters, credential persistence, submodule recursion, LFS hydration, alternate helpers, inherited proxies except explicit policy, and file/local transports; enforce the shared per-connection DNS/redirect/credential policy; -4. resolve the advertised selector to one exact commit; -5. fetch complete reachable ancestry or the exact requested shallow boundary under byte/object/time/process/output limits; -6. export a detached self-contained repository with no alternates, transient locks, credential-bearing remotes, or acquisition-only state; -7. reject gitlinks, unhydrated LFS pointers when policy requires real content, unsafe symlinks, and reserved-path collisions; -8. verify offline `log`, parent traversal, blame/diff across the selected history, and bisect prerequisites for full history; and -9. record requested selector, exact commit, history mode, normalized URL identity, destination, and verification policy in provenance. +```text +CodexAgent = { + kind: "codex", + model: logical ID, + reasoning_effort: "low" | "medium" | "high" +} -A source failure never chooses a different ref, deepens or shallows history, fetches an arbitrary object ID, or falls back to OCI. +OmpAgent = { + kind: "omp", + model: logical ID +} +``` -### OCI input preparation +`model`, `runtime_profile_id`, `agent_network_policy_id`, and `post_run.network_policy_id` are logical names resolved under authenticated operator configuration. Git `url` and OCI `repository` are canonical source addresses, never credential or policy selectors. Before credentials or network access, the gateway deterministically maps `(authenticated caller identity, source kind, canonical URL/repository)` to exactly one internal authorization/transport/credential route; zero or multiple matches return `source_policy_mapping_failed`. Uploaded bundles are authorized by the authenticated caller's access to `artifact_id`. The caller can never select an internal source identity, credential route, runtime image, tool path, sandbox policy, or service command. Runtime, model, and both phase-network policy IDs are authorized independently. Limits are caller-lowerable: callers send explicit values no greater than the selected profile/configured ceilings. The gateway rejects a value over a ceiling instead of silently changing it. Provider defaults are applied before request construction. -OCI source inputs are direct descriptors admitted by `source_name` and `BuildContext`. Apply the shared per-connection transport policy plus descriptor, distribution, and layer rules equivalent to the final snapshot. Tags, indexes, caller registries, ambiguous media types, traversal, links, devices, FIFOs, sockets, capabilities, ACLs, xattrs, set-ID bits, unsupported sparse files, and unbounded metadata fail closed. -OCI inputs may be Git-free. The builder never invents Git metadata. +The request can never contain a host path, credential, environment map, shell string, raw network destination, proxy, runtime image, policy body, retry count, output directory, grading rubric, pass/fail expectation, reward, patch request, or modified-workspace persistence option. -### Composition and reserved paths +### `PostRunSpec v1` -Build in private staging. Validate all destination ownership and reserved-path conflicts before network access. Apply sources in request order only for deterministic construction; overlap is invalid, so order never grants overwrite precedence. +```text +PostRunSpec = { + bundle?: BundleReference, + network_policy_id: logical ID, + commands: array of 0..32 PostRunCommand, + output_files: array of 0..128 OutputFileRequest +} -Reserved-path schema 1 is the root `.harness` path and every descendant, with path comparison after UTF-8/NFC/POSIX normalization. Builder and gateway share golden accept/reject fixtures but independent implementations. Reject an entry, whiteout, hardlink, symlink path or target alias, or type transition that occupies `.harness` or a descendant. Unknown schema versions fail before extraction. Root instruction files such as `AGENTS.md` are not reserved: snapshot mode preserves their content and merges the runner-managed block before the initial cursor is established. +PostRunCommand = { + command_id: logical ID, + executable: RuntimeExecutable | BundleExecutable, + args: array of 0..63 strings, each 0..4_096 UTF-8 bytes, + timeout_ms: positive integer +} -After composition: +RuntimeExecutable = { + kind: "runtime", + name: string matching ^[A-Za-z0-9][A-Za-z0-9._+-]{0,127}$ +} -- normalize ownership, timestamps where format policy requires them, modes, and metadata; -- walk without following links; -- validate the default working directory; -- produce the canonical visible-tree manifest excluding `.git` and `.harness`; -- produce digest-covered provenance; -- construct deterministic OCI layers/config/manifest; and -- publish blobs/config/layers before the direct image manifest. +BundleExecutable = { + kind: "bundle", + path: relative path +} -A failed or canceled publication never returns a result. Staging is cleaned or quarantined under bounded age/bytes/inodes. Config includes `available_until`, backed by an operator-enforced registry retention lease. Automatic garbage collection MUST NOT remove the manifest or referenced blobs before that timestamp. The lease duration covers expected scheduling delay plus gateway initialization deadline, maximum session TTL, and safety margin. +OutputFileRequest = { + name: logical ID, + path: relative path, + media_type: nonempty IANA media type, <= 127 bytes, + max_bytes: positive integer +} +``` -## Snapshot artifact v1 +At least one of `commands` or `output_files` is nonempty. Command IDs and output-file names are unique. `bundle` is present if and only if at least one command uses `BundleExecutable`; output-only collection and runtime-only commands omit it. A runtime executable name resolves only through the operator-approved fixed `PATH` of read-only runtime directories; it contains no slash, does not search the workspace, and must resolve to a regular executable beneath an approved directory. A bundle executable path resolves beneath the immutable hidden bundle without symlink escape. `args` are literal. No shell parsing, interpolation, globbing, caller environment, working-directory override, or other executable lookup is allowed. Each `timeout_ms` must be no greater than both `limits.post_run_timeout_ms` and the remaining total deadline. Each output-file path is relative to the final workspace root and is collected after all commands settle. `max_bytes` must not exceed `limits.max_artifact_bytes`. -The final artifact contract is fixed by ADR 0002: +Commands run sequentially in request order in the same retained sandbox, runtime image, and final workspace used by the agent, with current directory set to `workspace.working_directory`. A nonzero exit is recorded and does not stop later commands. A per-command timeout kills and reaps that command's process group, records a `timed_out` observation, and continues if the post-run and total deadlines permit. Failure to validate/extract/inject a requested bundle, resolve an approved executable, enforce the post-run sandbox/network policy, or launch a command is infrastructure failure and aborts remaining collection. -- OCI image manifest media type `application/vnd.oci.image.manifest.v1+json`; -- `artifactType` `application/vnd.allagents.workspace-snapshot.v1`; -- config media type `application/vnd.allagents.workspace-snapshot.config.v1+json`; -- provenance media type `application/vnd.allagents.workspace-provenance.v1+json`; -- canonical visible-tree media type `application/vnd.allagents.workspace-manifest.v1+json`; -- direct execution descriptor `{media_type,digest,size}`; and -- deterministic gzip layers only, media type `application/vnd.oci.image.layer.v1.tar+gzip`, applied in order to an empty root. V1 rejects uncompressed, zstd, and nondistributable layers. +### `AgentRunResult v1` -Config contains exactly: +The authoritative file is `schemas/agent-run-result.v1.schema.json`. Its exact common shape is: ```text { - version: 1, - working_directory: string, - workspace_manifest: OCI descriptor, - provenance: OCI descriptor, - builder: {name: string, version: string, policy: string}, - reserved_paths_schema: 1, - available_until: RFC3339 timestamp + schema_version: "agent_run_result.v1", + identity: { + run_id: UUIDv7, + idempotency_key: string + }, + status: "completed" | "cancelled" | "infrastructure_error", + post_run_mode: "none" | "requested", + effective_limits: { same ten required fields as request.limits }, + workspace_provenance: DirectWorkspaceProvenance | null, + runtime_provenance: RuntimeProvenance | null, + agent: AgentResult | null, + post_run: PostRunEvidence | null, + usage: Usage, + timing: Timing, + trajectory: InlineTrajectory | ArtifactTrajectory | null, + error: RunError | null } ``` -Required provenance is RFC 8785 canonical JSON with media type `application/vnd.allagents.workspace-provenance.v1+json`. It is a closed `{version:1,sources:[...]}` object. A Git record contains only `kind`, normalized `url`, optional `requested_ref`, exact `resolved_commit`, closed full-or-shallow `history`, and `destination`. An OCI record contains only `kind`, public logical `source_name`, exact direct `descriptor`, and `destination`. Source order is build order. Credentials, registry coordinates, headers, helpers, and host paths are forbidden. Builder and policy identity remain in config. +The schema encodes these variants: -The canonical visible-tree entry forms, sorting, hashing, link confinement, mode normalization, `.git` exclusion, and runner-path exclusion match ADR 0002 exactly. Builder and gateway MUST use independent implementations against the same golden corpus; a shared implementation would hide interoperability bugs. +- `completed` + `post_run_mode: "none"`: non-null workspace/runtime provenance, `agent.termination: "completed"`, non-null bounded `CapturedText` final output (empty inline text is valid), usage, and trajectory; `post_run: null`; `error: null`. +- `completed` + `post_run_mode: "requested"`: the same completed agent fields plus non-null `PostRunEvidence`; every command observation is `completed` or `timed_out`, and every file observation is `collected`, `missing`, `limit_exceeded`, or `not_regular_file`; `error: null`. +- `cancelled`: `error.code: "run_cancelled"` and any already durable provenance/agent/usage/trajectory. If post-run initialization was durably recorded, `post_run` is a full request-order observation vector containing sealed observations plus `not_run` entries for work prevented by cancellation; otherwise it is null. +- `infrastructure_error`: a non-cancellation error plus any already durable provenance, agent, usage, trajectory, and `PostRunEvidence`. If post-run initialization was durable, the evidence vector contains sealed observations, `unavailable` for the item whose evidence/launch/collection failed, and `not_run` for later work; otherwise it is null. Cleanup failure preserves the fully sealed vector and can never publish `completed`. -The builder outputs: +A completed run always exposes bounded agent final output, usage, and a valid trajectory even if no post-run evidence was requested or no files changed. Nonzero exits, timed-out commands, missing files, oversized files, and non-regular paths are behavioral observations and do not change lifecycle status. Any `unavailable` observation implies `infrastructure_error`; `not_run` appears only on cancelled or infrastructure-error results. Mandatory result-finalization or cleanup failure downgrades the terminal candidate to `infrastructure_error`, retaining already durable evidence. The result remains invisible until cleanup succeeds and cache leases are released. -```json -{ - "version": 1, - "descriptor": { - "media_type": "application/vnd.oci.image.manifest.v1+json", - "digest": "sha256:...", - "size": 123456 - }, - "workspace_manifest_digest": "sha256:...", - "provenance_digest": "sha256:...", - "working_directory": "services/api", - "available_until": "2026-10-29T00:00:00Z" +Closed result components are: + +```text +DirectWorkspaceProvenance = { + kind: "direct", + working_directory: string, + sources: array in request order of + GitSourceProvenance | OciSourceProvenance | UploadedSourceProvenance } -``` -This versioned builder result contains no credentials or private registry transport details. The caller supplies only nested `descriptor` to the gateway and retains the complete result as its baseline/provenance/retention record. +GitSourceProvenance = { + kind: "git", + url: canonical HTTPS URL, + requested_ref: string, + resolved_commit: 40 lowercase hexadecimal characters, + history: { mode: "full" } | + { mode: "shallow", depth: positive integer }, + access: "read_only" | "writable", + materializer_schema: 1, + destination: string, + tree_digest: SHA-256 digest +} -## Gateway public contract +OciSourceProvenance = { + kind: "oci", + repository: canonical OCI repository, + requested_descriptor: OciSource.descriptor, + resolved_manifest_digest: SHA-256 digest, + rootfs_digest: SHA-256 digest, + access: "read_only" | "writable", + materializer_schema: 1, + destination: string +} -### Request +UploadedSourceProvenance = { + kind: "uploaded_bundle", + bundle: BundleReference, + rootfs_digest: SHA-256 digest, + access: "read_only" | "writable", + materializer_schema: 1, + destination: string +} -Only the first request creating a new session may contain: -```text -metadata.allagents_workspace_snapshot = { - version: 1, - descriptor: { - media_type: "application/vnd.oci.image.manifest.v1+json", - digest: "sha256:<64 lowercase hex>", - size: positive integer +RuntimeProvenance = { + runtime_profile_id: logical ID, + profile_digest: SHA-256 digest, + image_digest: SHA-256 digest, + sandbox_policy_version: string, + tools: array of 0..128 { + name: logical ID, + version: string, + implementation_digest: SHA-256 digest + }, + services: array of 0..32 { + name: logical ID, + version: string, + implementation_digest: SHA-256 digest, + image_digest: SHA-256 digest | null } } -``` - -All objects are closed. No alias for the old `metadata.workspace`, `sources`, `access`, `snapshot_name`, `image_manifest_digest`, or `source_manifest_digest` shape is retained. - -Request-decidable validation finishes before catalog, cache, registry, workspace, or lifecycle mutation. Continuation/replay/reused-session injection fails before hydrate. The media type is exact. Digest is SHA-256 lowercase. Size is bounded by manifest maximum. -### Authorization +`profile_digest` identifies the canonical immutable nonsecret resolved-profile manifest. A tool `implementation_digest` commits to its executable bytes and immutable runtime dependency closure. A service `implementation_digest` commits to its credential-free canonical launch manifest and executable closure; `image_digest` is non-null exactly for a containerized service and names its immutable image. The `tools` and `services` arrays enumerate the entire pinned revision with unique names sorted by name. Operator profiles never put credentials or secret values in commands, arguments, environment, or these manifests; brokers supply secrets out of process. Versions are descriptive, while digests are the implementation identities. -V1 server configuration maps each authenticated product domain to exactly one trusted snapshot repository and one catalog. Zero or multiple mappings fail deployment preflight. Catalog lookup returns exactly one immutable `catalog_entry_id` and version binding the descriptor and digest-covered `available_until`; ambiguity fails before registry traffic. +AgentResult = { + kind: "codex" | "omp", + model: logical ID, + adapter_version: string, + termination: "completed" | "cancelled" | "timed_out" | "failed", + exit_code: integer | null, + final_output: CapturedText | null +} -Authorization key is the exact `(principal, domain, repository_id, catalog_entry_id, media_type, digest, size)` tuple plus current policy. Authorization occurs before fetch, cache attachment, restart completion, and exact-digest reacquisition. Unauthorized and unknown both return `404 workspace_snapshot_unknown`. A cache hit is never authorization. +CapturedText = + { storage: "inline", encoding: "utf-8", text: string, + digest: SHA-256 digest, size_bytes: integer >= 0, truncated: boolean } | + { storage: "artifact", encoding: "utf-8", + artifact: ArtifactReference, truncated: boolean } +``` -### Response +Agent output capture retains at most `max_agent_output_bytes` of valid UTF-8 and sets `truncated: true` if additional bytes existed. At most 64 KiB is inline. Larger retained text uses a result artifact when its deterministic artifact-budget turn fits; otherwise it returns the first at-most-64-KiB UTF-8 prefix inline with `truncated: true`. Digests and sizes describe returned bytes. The provider verifies and decodes inline text or downloads/verifies the artifact before giving Promptfoo the final string; truncation stays visible in metadata. -After `ready`, return: +`PostRunEvidence` contains raw observations only: ```text -metadata.allagents_workspace_snapshot = { - version: 1, - descriptor: {media_type, digest, size}, - snapshot_manifest_digest: "sha256:<64 lowercase hex>", - ready_manifest_digest: "sha256:<64 lowercase hex>", - provenance_digest: "sha256:<64 lowercase hex>", - initialization_changes_file_id: string, - working_directory: string, - available_until: RFC3339 timestamp, - expires_at: RFC3339 timestamp +PostRunEvidence = { + commands: one PostRunCommandObservation per requested command, in order, + output_files: one OutputFileObservation per requested file, in order } -``` -Terminal streaming, retrieval, replay, and later workspace failures use the stored sanitized object. Pre-ready failures omit it. Catalog/repository names, policy revisions, credentials, private cache keys, local paths, reference IDs, builder jobs, and live filesystem facts remain private. +PostRunCommandObservation = + { status: "completed", command_id, executable, args, + exit_code: integer | null, signal: integer | null, + duration_ms: integer >= 0, stdout: CapturedOutput, stderr: CapturedOutput } | + { status: "timed_out", command_id, executable, args, + signal: integer | null, duration_ms: integer >= 0, + stdout: CapturedOutput, stderr: CapturedOutput } | + { status: "not_run", command_id, executable, args, + reason: "cancelled" | "prior_infrastructure_error" | "deadline" } | + { status: "unavailable", command_id, executable, args, + reason: "cancelled" | "deadline" | "launch_failed" | "sandbox_failed" | + "evidence_persistence_failed", + duration_ms: integer >= 0 | null, + stdout: CapturedOutput | null, stderr: CapturedOutput | null } + +CapturedOutput = + { storage: "inline", encoding: "utf-8", text: string, + digest: SHA-256 digest, size_bytes: integer >= 0, truncated: boolean } | + { storage: "artifact", artifact: ArtifactReference, truncated: boolean } | + { storage: "omitted", digest: SHA-256 digest, + size_bytes: integer >= 0, truncated: true, + reason: "artifact_budget_exhausted" } + +OutputFileObservation = + { name: logical ID, path: string, status: "collected", + media_type: string, digest: SHA-256 digest, size_bytes: integer >= 0, + artifact: ArtifactReference } | + { name: logical ID, path: string, status: "missing" } | + { name: logical ID, path: string, status: "limit_exceeded", + observed_size_bytes: integer >= 0 } | + { name: logical ID, path: string, status: "not_regular_file" } | + { name: logical ID, path: string, status: "not_run", + reason: "cancelled" | "prior_infrastructure_error" | "deadline" } | + { name: logical ID, path: string, status: "unavailable", + reason: "cancelled" | "deadline" | "unsafe_path" | + "changed_during_read" | "read_failed" | + "artifact_persistence_failed" } +``` -Advertise vendor capability `allagents_workspace_snapshot_v1: true`. It means request schema V1 and snapshot artifact/config/provenance/manifest V1 are all supported. Absence or false means unsupported. Do not modify UHP conformance claims. +The worker durably seals each `completed` or `timed_out` command observation before starting the next command and each file observation before collecting the next file. Post-run initialization durably records the ordered item skeleton. On cancellation or infrastructure failure it seals the in-flight item as `unavailable` with any safely captured bounded streams and fills all untouched items as `not_run`, so a published non-null `PostRunEvidence` always has exactly one entry per request item. -## Gateway durable model +For a completed command, exit code and signal are not both null; nonzero exit is ordinary evidence. A timed-out command records partial bounded streams after its cgroup is reaped. Each stream captures at most `max_command_output_bytes`. Valid UTF-8 at or below 64 KiB is inline; larger or binary bytes use an artifact if budget remains. Without budget, valid UTF-8 falls back to a truncated inline prefix and binary bytes use `omitted`; budget exhaustion is never infrastructure failure. The result echoes validated executable/args, never a resolved host path. -### Snapshot binding +Requested files are read after commands settle without following the final path as a symlink. Missing, over-limit, and non-regular paths are completed observations. Unsafe races/read failures become `unavailable` and infrastructure error. Because requested-file maxima were reserved at admission, any valid collected file fits the artifact byte budget; artifact-store failure marks that item unavailable while retaining prior observations. -Extend `HarnessSession` with one logical binding reference. Store large manifests in blob storage, not graph properties. ```text -WorkspaceSnapshotBinding = { - binding_id, - state: pending | ready | failed | deleting, - descriptor: {media_type, digest, size}, - snapshot_key, - repository_id, - catalog_entry_id, - catalog_entry_version, - signature_policy_id, - signature_evidence_digest, - authorization_audit_ref, - minimum_reader_version, - initializer_schema, - journal_schema, - initialization_deadline, - ready_at?, - expires_at?, - available_until?, - snapshot_manifest_digest?, - ready_manifest_digest?, - provenance_digest?, - initialization_changes_file_id?, - working_directory?, - cursor_blob_digest?, - failure_code? +Usage = { + input_tokens: integer >= 0 | null, + cached_input_tokens: integer >= 0 | null, + output_tokens: integer >= 0 | null, + reasoning_tokens: integer >= 0 | null, + tool_calls: integer >= 0, + estimated_cost_usd: finite number >= 0 | null, + source: "adapter_reported" | "partially_reported" | "unavailable" } -``` -`expires_at` is absent until the `ready` transition and is computed from `ready_at`. Polling and continuation never change it. +Timing = { + accepted_at: timestamp, + started_at: timestamp | null, + agent_started_at: timestamp | null, + agent_stopped_at: timestamp | null, + post_run_started_at: timestamp | null, + completed_at: timestamp, + queue_ms: integer >= 0, + acquisition_ms: integer >= 0, + agent_ms: integer >= 0, + post_run_ms: integer >= 0, + cleanup_ms: integer >= 0, + total_ms: integer >= 0 +} -`WorkspaceSnapshotReference` is a durable record keyed by `(binding_id, snapshot_key)` with `provisional | active | released`. Creation, activation, and release use compare-and-set. A still-running pending attempt retains its provisional claim across internal restart reconciliation. Every pre-ready response failure makes the binding terminal `failed` and releases the claim; a caller retry creates a new binding. Counters are derived, never authoritative. +InlineTrajectory = { + storage: "inline", + format: "atif-v1", + digest: SHA-256 digest, + size_bytes: integer >= 0, + truncated: boolean, + document: ATIF-v1 document +} -Do not persist local cache paths, inode/device identities, reflink facts, attachment flags, staging paths, lock owners, registry credentials, or process IDs. +ArtifactTrajectory = { + storage: "artifact", + format: "atif-v1", + digest: SHA-256 digest, + size_bytes: integer >= 0, + truncated: boolean, + artifact: ArtifactReference +} -### Turn finalization record +RunError = { + code: StableErrorCode, + message: sanitized string, 1..1_024 UTF-8 bytes, + retryable: boolean, + phase: "admission" | "queue" | "acquisition" | "agent" | + "sealing" | "post_run" | "finalization" | "cleanup" | + "cancellation" +} +``` -Add a recoverable record keyed by `response_id`: +ATIF documents are schema-validated before publication. One byte-counting recorder owns `max_trace_bytes`; at the limit it emits one valid truncation marker and refuses further payload without corrupting the document. Up to 256 KiB is inline. A larger document uses an artifact at the final artifact-budget priority. If that promotion does not fit, the recorder emits a valid at-most-256-KiB inline ATIF document ending in the same truncation marker. Artifact-budget exhaustion is not infrastructure failure, and completed runs never omit the trajectory. + +## Stable errors and consequences + +| Code | API/result consequence | `retryable` | +|---|---|---:| +| `invalid_request` | HTTP 400; no run record, fetch, or directory | false | +| `unauthenticated` | HTTP 401; no run record | false | +| `forbidden` | HTTP 403; authenticated caller lacks policy permission; no public run | false | +| `object_not_found` | non-enumerating HTTP 404 for absent or cross-tenant run/upload/result artifact | false | +| `source_policy_mapping_failed` | HTTP 403; no credentials resolved, network request, or public run | false | +| `runtime_profile_forbidden` | HTTP 403; profile not authorized for caller; no public run | false | +| `post_run_executable_forbidden` | HTTP 403; runtime name absent from selected profile; no public run | false | +| `bundle_integrity_failed` | HTTP 422; reservation unusable; no bundle published | false | +| `upload_id_conflict` | HTTP 409; original tenant/upload reservation unchanged | false | +| `idempotency_conflict` | HTTP 409; original run unchanged | false | +| `admission_limit_exceeded` | HTTP 422; no public run | false | +| `queue_unavailable` | `infrastructure_error`; no agent invocation | true | +| `runtime_profile_admission_failed` | HTTP 503; authorized immutable revision cannot be resolved and verified; no public run | true | +| `runtime_profile_unavailable` | `infrastructure_error`; pinned revision or implementation unavailable at dispatch; no agent invocation | true | +| `runtime_profile_integrity_failed` | `infrastructure_error`; pinned profile/runtime/tool/service digest mismatch; no agent invocation | false | +| `workspace_acquisition_failed` | `infrastructure_error`; no agent invocation | false | +| `source_authorization_revoked` | `infrastructure_error`; cached generation is not attached; no agent invocation | false | +| `source_cache_capacity_exceeded` | `infrastructure_error`; no unsafe eviction; no agent invocation | true | +| `source_cache_integrity_failed` | `infrastructure_error`; generation quarantined; no agent invocation | false | +| `workspace_integrity_failed` | `infrastructure_error`; offending source quarantined for operators | false | +| `workspace_limit_exceeded` | `infrastructure_error`; partial tree removed | false | +| `agent_start_failed` | `infrastructure_error`; no post-run collection | false | +| `agent_failed` | `infrastructure_error`; no post-run collection | false | +| `agent_timed_out` | `infrastructure_error`; process tree killed; no post-run collection | false | +| `agent_invocation_ambiguous` | `infrastructure_error`; recovery refuses a second invocation | false | +| `agent_seal_failed` | `infrastructure_error`; no post-run collection | false | +| `post_run_bundle_failed` | `infrastructure_error`; requested bundle could not be safely injected; ordered unavailable/not-run evidence retained | false | +| `post_run_executable_resolution_failed` | `infrastructure_error`; an approved runtime or bundle executable could not be resolved safely; partial evidence retained | false | +| `post_run_sandbox_failed` | `infrastructure_error`; ordered unavailable/not-run evidence retained | false | +| `task_service_failed` | `infrastructure_error`; ordered unavailable/not-run evidence retained | true | +| `post_run_command_launch_failed` | `infrastructure_error`; failed command unavailable, later work not-run, earlier evidence retained | false | +| `post_run_collection_failed` | `infrastructure_error`; failed file unavailable, later files not-run, earlier evidence retained | false | +| `trace_finalization_failed` | `infrastructure_error`; other durable evidence retained | false | +| `result_finalization_failed` | `infrastructure_error`; no completed result published | false | +| `artifact_integrity_failed` | HTTP 502 on result-artifact download; provider reports infrastructure failure | true | +| `artifact_expired` | HTTP 410 on an expired result artifact | false | +| `cleanup_failed` | terminal candidate becomes `infrastructure_error`; durable evidence retained; publication waits for successful reconciled cleanup and lease release | true | +| `run_timed_out` | `infrastructure_error`; active processes killed and cleaned | false | +| `run_cancelled` | `cancelled`; active processes killed and cleaned | false | + +`retryable: true` means only that an explicit new run may succeed after transient operator recovery; it never permits the worker or provider to rerun an agent, and retrying the same idempotency identity returns the same terminal run. + +A command's nonzero exit or per-command timeout is deliberately absent from this table: both are observations in `PostRunCommandObservation`. Public errors never contain credentials, policy bodies, private endpoints, headers, host paths, resolved executable paths, raw model transport, or command stderr. Operator-only diagnostics use `run_id` correlation. + +## Gateway API, uploads, and idempotency ```text -WorkspaceTurnCommit = { - response_id, - state: preparing | durable | acknowledged | committed | aborted, - base_cursor_digest, - next_cursor_digest?, - change_artifact_id?, - file_artifact_ids: [], - checkpoint_digest?, - collection_token?, - collection_fingerprint? +POST /v1/bundles # idempotent authenticated upload reservation +PUT /v1/bundles/{artifact_id} # idempotent exact-byte upload +POST /v1/runs # submit AgentRunRequest v1 +GET /v1/runs/{run_id} # current state or terminal AgentRunResult v1 +POST /v1/runs/{run_id}/cancel +GET /v1/artifacts/{artifact_id} # immutable result-artifact bytes ``` -All artifact IDs and blob keys are deterministic from response identity plus content identity. Retry verifies and reuses existing durable objects. `acknowledged` means the runner accepted the explicit base/next cursor token, while the gateway cursor remains authoritative. `committed` is reached only when response artifacts, checkpoint, cursor pointers, and terminal state publish together under the existing response/session control lease. - -## Snapshot admission, cache, and private materialization +Every upload reservation, stored input bundle, run, result, and result artifact persists a non-public `owner_tenant`. Run/result artifacts also persist `run_id` and evidence purpose. Every duplicate-submit lookup, upload PUT, run GET, cancel, and artifact GET resolves the tuple `(authenticated_tenant, object_id)`; an absent or other-tenant object returns the same non-enumerating HTTP 404. UUIDv7 is never authorization. Cancellation and artifact access require the same tenant/run authorization as result polling. -### Admission +`POST /v1/bundles` accepts the closed body below. `(owner_tenant, upload_id)` is unique. A first reservation returns HTTP 201 and its `BundleReference`; an exact retry with identical media type/digest/size returns HTTP 200 and the same reference; any changed metadata returns `upload_id_conflict` without mutating the original. The provider persists `upload_id` before the request and reuses it after response loss. -Fetch only from the binding's persisted `repository_id` by direct digest under the shared per-connection transport policy. Never search another repository or follow caller-selected origins or tags. Verify declared size and response digest before parsing. Reject indexes/lists. Verify manifest/config/provenance/tree/layer media types, descriptor sizes, digests, supported `reserved_paths_schema`, and catalog/config `available_until` equality before use. - -If retention is expired or cannot cover initialization deadline plus maximum session TTL plus safety margin, return nonretryable `422 workspace_snapshot_retention_invalid`. If an authorized manifest is missing before `available_until`, return `503 workspace_snapshot_unavailable` and alert on the broken retention lease. Never search another repository. - -Apply only gzip OCI layer changesets in order, including whiteouts and opaque directories, through rooted no-follow operations; do not shell out to `tar`. Reject every layer or final-tree path/link/whiteout/type transition at `.harness` or below, including content omitted from the public manifest. Normalize or reject ownership/mode/xattr/ACL/capability metadata. Compute the public snapshot manifest plus a private full-tree seal covering `.git` before publication. +```text +{ + schema_version: "bundle_reservation_request.v1", + upload_id: UUIDv7, + media_type: "application/vnd.allagents.gateway-bundle.v1.tar+gzip", + digest: SHA-256 digest, + size_bytes: positive integer +} +``` -### Immutable cache +`PUT /v1/bundles/{artifact_id}` accepts exactly the reserved size, verifies the streamed digest, and atomically marks the reservation usable. Repeating the same completed upload returns success without rewriting bytes; failed or different bytes can never mutate it. Reservations expire tenant-scoped. Input/source/post-run bundles are not result artifacts and are never readable through `/v1/artifacts`. -Cache key: +Submission, polling, and cancellation return the same closed status envelope: ```text -sha256(canonical({descriptor, materializer_schema})) +{ + schema_version: "agent_run_status.v1", + run_id: UUIDv7, + state: "received" | "queued" | "acquiring_workspace" | + "preparing_run" | "running_agent" | "stopping_agent" | + "sealing_agent_result" | "preparing_post_run" | + "running_post_run" | "finalizing" | "cleaning" | + "cancelling" | "failing" | "stopping_processes" | + "completed" | "cancelled" | "infrastructure_error", + terminal: boolean, + result: AgentRunResult | null +} ``` -Destination, principal, session, harness, provider, expiry, policy revision, and working directory do not fragment the verified byte cache. Authorization remains separate. +For nonterminal states, `terminal` is `false` and `result` is null; terminal states require a matching result. A new admitted run returns HTTP 202. An exact duplicate or poll of a terminal run returns HTTP 200. Accepted cancellation returns HTTP 202; a terminal run returns HTTP 200 unchanged. -An exact-key miss singleflights. Publication sequence is staging -> full verification -> immutable generation -> complete marker -> available. Cache roots and generations are owned by a gateway/materializer identity no harness/session UID can assume, are non-writable, and are non-searchable from the runner namespace. Failed or uncertain staging is unavailable and quarantined. +`GET /v1/artifacts/{artifact_id}` serves only unexpired immutable **result** artifacts linked to a run visible to the authenticated tenant. A successful response streams exact bytes without redirect and includes `Content-Type`, `Content-Length`, `Digest: sha-256=...`, immutable `ETag`, and `Cache-Control: private, immutable`; these encode the `ArtifactReference` media type, size, and digest. Other-tenant, unknown, input-bundle, and detached artifacts are indistinguishable HTTP 404. An authorized reference past `expires_at` returns HTTP 410; its tenant/run metadata tombstone remains through result retention even after bytes are deleted asynchronously. The service re-verifies stored size/digest while streaming and returns `artifact_integrity_failed` instead of corrupt bytes. The provider downloads every referenced final output, command stream, requested file, or trajectory through this endpoint and verifies headers and bytes before use. -Clone/reflink reads use trusted directory file descriptors, rooted no-follow operations, and an eviction/clone lease. Verify the private full-tree seal immediately before and after cloning. Any unexpected metadata/content mutation quarantines the generation and fails closed. +Run submission order is normative: -Global controls are mandatory: maximum cache/staging bytes and inodes; high/low watermarks; eviction only among complete unreferenced generations; staging/quarantine age and size; bounded waiters/builds; metrics; and reconciliation from durable reference records. A malicious session test MUST prove a guessed cache path cannot be searched, read, or mutated. +1. Authenticate the caller; validate the closed schema; canonicalize and hash the complete request. +2. Look up both `(owner_tenant, run_id)` and `(owner_tenant, idempotency_key)` before mutable admission. If both identify the same request digest/run, return that existing status without rechecking current source/profile/network policy or bundle mutability. Any identity/digest mismatch returns `idempotency_conflict`. Other tenants' identities are outside this lookup. +3. For a genuinely new identity, acquire an internal tenant/identity submission claim. Concurrent exact submissions wait on that claim; it is not a public run and has bounded crash recovery. +4. Under the claim, perform all mutable admission: deterministic source-route authorization, uploaded-bundle ownership/usability, model/runtime-profile/network-policy authorization, runtime executable allowlist, limit ceilings, destination/path rules, and aggregate requested-file artifact reservation. Resolve the authorized `runtime_profile_id` once to an immutable revision and verify its canonical `profile_digest`, runtime image digest, tool/service implementation digests, and any service image digests. An admission failure returns its HTTP error and creates no public run or job. +5. On successful admission, one transaction writes the tenant-owned public run in `received`, pins canonical request/digest plus the complete immutable resolved-profile revision and digests, absolute deadlines, and artifact reservation, enqueues one job, then completes the claim. Exact retries now take step 2. -### Private writable tree +The provider may retry upload reservation, byte upload, run submit, poll, cancel, and artifact download with the same identities. It never creates a replacement run for transport loss. A Promptfoo repetition creates a new run. Duplicate submission does not re-admit mutable policy, but worker execution still reauthorizes each source before resolution and cache attachment. Worker recovery may repeat safe acquisition or cleanup; if durable state cannot prove the agent was never invoked, it records infrastructure error instead of invoking again. -Every snapshot session gets an ordinary private writable directory: +## State machine -1. prefer same-filesystem per-file reflink with independently created directory entries/inodes; -2. fall back to full private copy; -3. never hardlink regular files to cache, expose a writable cache alias, or symlink the workspace to cache; -4. walk no-follow and revalidate source generation before cloning; -5. apply hard runtime byte/inode quotas; and -6. validate the digest-covered working directory beneath the private root. +Persist every transition with a monotonic sequence and compare-and-swap expected state: -Probe production filesystem behavior before implementation is considered deployable. The integration suite forces both reflink and copy paths. OverlayFS, bind-mounted source roots, per-source read-only/editable modes, and delta checkpoints are out of V1. +```text +received -> queued -> acquiring_workspace -> preparing_run + -> running_agent -> stopping_agent -> sealing_agent_result + -> preparing_post_run -> running_post_run + -> finalizing -> cleaning -> completed + +sealing_agent_result -> finalizing # post_run_mode = none +any nonterminal state -> cancelling -> stopping_processes -> cleaning -> cancelled +any nonterminal state -> failing -> stopping_processes -> cleaning -> infrastructure_error +cleaning -> cleaning # failed attempt downgrades candidate to cleanup_failed; retry/reconcile +``` -### Ready transition +Rules: -New-session sequence: +1. Authentication, schema/canonical hashing, tenant-scoped duplicate/conflict lookup, and all mutable policy/resource admission complete before `received`; admission failure creates no public state-machine record. Successful admission atomically creates the tenant-owned run and one queued job. +2. Queue work is leased. Lease expiry may repeat authorization, cache attachment, or cleanup only before the durable agent-invocation marker. +3. Entering `running_agent` atomically records the sole permitted invocation number, `1`. +4. `stopping_agent` gracefully terminates, waits a bounded grace period, kills the agent cgroup, closes the model proxy and agent egress, and confirms no agent-owned process remains. It does not kill operator-owned task services in the separate service cgroup. +5. `sealing_agent_result` finalizes and durably records bounded `CapturedText`, usage, and trajectory, closes agent-owned descriptors, removes adapter home/model/source credentials, and changes process ownership from agent to post-run. The final workspace remains in the same sandbox and keeps its declared source permissions. +6. If post-run work was requested, `preparing_post_run` durably writes the full ordered `PostRunEvidence` skeleton, applies the independently authorized post-run network policy, and—only after the agent teardown and credential-removal invariants hold—acquires, verifies, extracts, and mounts requested bundle bytes at the reserved late-mount point. It then resolves executables from the pinned profile revision and starts or health-checks its declared task services. +7. `running_post_run` executes commands and collects files in order, durably sealing each observation before advancing. Cancellation or infrastructure failure fills the current/later entries with `unavailable`/`not_run`. It is reachable only after agent stop and result-seal confirmation. +8. With `post_run: null`, the run skips from sealing to finalization. +9. Cancellation is first-writer-wins against terminalization. Accepted cancellation yields `cancelled` only if cleanup completes without error; a cleanup fault overrides it with `cleanup_failed`/`infrastructure_error`. Late cancellation returns the already published terminal result. +10. The total deadline dominates phase deadlines. Total expiry is infrastructure error; a command's own timeout is a completed observation if cleanup completes within the remaining total deadline. +11. `stopping_processes` revokes remaining network/model access and kills agent, check, service, acquisition, helper, and descendant cgroups before cleanup. +12. `cleaning` unmounts read-only generations, deletes private writable clones and any injected bundle/scratch/root, stops task services, and releases source-cache leases only after all mounts are gone. Any cleanup-step failure durably changes the terminal candidate—even a completed or cancelled candidate—to `infrastructure_error` with `error.code: "cleanup_failed"` while preserving already sealed evidence. The run remains nonterminal in `cleaning`; bounded retries and startup reconciliation continue, and leases needed for safety remain held until the associated mounts are gone. +13. Terminal states are immutable. The API exposes `AgentRunResult` only after process/cgroup absence, per-run storage deletion, unmount, task-service stop, and cache-lease release are all verified. Only then does it finalize `cleanup_ms`/`completed_at` and atomically publish the result. Cleanup failure can therefore never leak a `completed` result; after reconciliation succeeds it publishes `infrastructure_error` with the durable evidence. -1. run stock auth, UHP validation, idempotency, and session selection; -2. parse/validate the closed extension and compatible minimum-reader routing; -3. resolve one repository/catalog entry and authorize the exact tuple; -4. persist pending binding, private authorization subject, and provisional reference before fetch; -5. claim/reuse/build immutable generation; -6. reauthorize current policy; -7. materialize and full-seal-check the private tree; -8. apply UHP inputs and deterministic runner instruction/control preparation; -9. compare snapshot manifest with the ready tree; durably capture changed regular files plus `workspace-initialization-changes-.json`; -10. store the ready manifest as authoritative initial cursor and validate working directory; -11. activate reference and transition binding to `ready` with snapshot/ready digests, initialization artifact, `ready_at`, and `expires_at`; and -12. start the stored harness/provider path. +## Main direct-mode flow -The initialization artifact uses `application/vnd.allagents.workspace-changes.v1+json` and the same canonical schema as turn deltas. It is emitted even when empty. Promptfoo applies it to builder snapshot bytes before response deltas. No Files, checkpoint, provider, harness, or response success path observes the workspace before step 11. A pre-ready failure is terminal for that binding, releases provisional state once, and leaves no runnable tree. +1. **Author.** Promptfoo defines the rendered instruction, ordered sources/access, agent and required `runtime_profile_id` matrix values, repetition, optional raw post-run evidence, separate agent/post-run network policy names, and caller-lowerable limits. +2. **Package local paths.** The provider deterministically packages local sources and an optional post-run bundle, persists one `upload_id` per package, idempotently reserves/uploads immutable bytes, and substitutes `BundleReference` objects. Runtime-only/output-only post-run configurations upload no bundle; gateway JSON never contains a host path. +3. **Submit.** The gateway follows the normative ordering above: authenticate/schema/hash, tenant-scoped existing identity lookup, then an internal new-identity claim, mutable source/bundle/model/runtime/network/limit/artifact-budget admission, and only then one tenant-owned public run/job. Exact duplicates bypass mutable re-admission. +4. **Dispatch.** The worker leases the job, loads only the profile revision pinned at admission, and re-verifies its `profile_digest` plus runtime/tool/service implementation digests. It creates a mode-0700 run root and retained mount/runtime with the profile's unprivileged identity, read-only root, namespace/seccomp/cgroup limits, separate service/agent/check process and network boundaries, private source-clone locations, adapter home, output, scratch, and an **empty** worker-reserved late-mount point. It neither acquires nor mounts post-run bundle bytes before agent teardown; no cross-run writable or control-plane mount exists. +5. **Resolve and compose.** In request order, reauthorize each source, resolve its exact cache key, lease/build one complete immutable generation, then attach it. Mount `read_only` generations directly read-only. For `writable`, create a private reflink/CoW clone or bounded full copy. Validate the working directory and record source plus runtime provenance. +6. **Prepare services and agent.** Start only task services declared by the pinned immutable profile revision, using their verified implementation and image digests, each behind phase-specific broker endpoints; then invoke the agent in its own cgroup/process/network namespace. Default-drop policy exposes only `agent_network_policy_id` endpoints and the credential-free model proxy; post-run-only services and direct loopback/sidecar sockets are unreachable. +7. **Run once.** The adapter invokes its pinned CLI directly without a shell. It captures at most `max_agent_output_bytes` into `CapturedText`, normalizes usage, and writes bounded ATIF. There is no worker retry. +8. **Stop and seal the agent result.** Stop/reap the agent cgroup, destroy the agent network namespace/flows/broker handles, finalize bounded output/usage/ATIF, and remove model/source credentials and agent home. Keep the mount/runtime image, final workspace, source attachments, and operator service cgroup alive. +9. **Optionally collect evidence.** After the agent cgroup/network namespace/flows are gone and credentials/home removed, durably initialize the ordered evidence skeleton and create a distinct default-drop check network namespace for `post_run.network_policy_id`. Only now acquire and verify requested bundle bytes, extract them safely, and mount them read-only at the previously empty reserved point. Health-check profile-declared services, resolve runtime tools from the pinned revision's `PATH`, then execute/collect in the same final workspace, sealing each observation before advancing. No grading occurs. +10. **Allocate artifacts and seal evidence.** Charge actual requested files against their admission reservation, then deterministically promote agent output, command streams, and trajectory. Use bounded inline/truncated/omitted fallbacks when optional promotion does not fit. Persist artifacts with owner tenant/run/purpose/expiry and the pending outcome; do not create a public result/timing completion yet. +11. **Destroy and publish.** Kill check/service cgroups and namespaces, remove any bundle/scratch, unmount sources, delete clones/root, release cache leases, and verify absence. A fault downgrades the pending outcome to `cleanup_failed` while retaining sealed evidence and hiding the result through reconciliation. After verified cleanup, finalize timing/runtime provenance and atomically publish. +12. **Report.** The provider maps verified inline or downloaded `CapturedText` and usage to Promptfoo, dereferences/verifies every result artifact, and exposes the complete run/evidence under `metadata.allagents_run`. Promptfoo graders alone decide outcomes. Cancellation, infrastructure, artifact expiry, or artifact integrity errors remain typed provider errors. -## Manifest journal and change artifacts +## Workspace composition and immutable source cache -Snapshot sessions select `manifest-v1`; stock sessions keep root Git. The gateway-owned cursor blob is the sole durable journal authority. Gateway `/produced` sends the explicit base cursor blob/digest to the runner; the runner returns changes, next canonical cursor, and a collection token without mutating durable state. `/produced/ack` idempotently confirms `(base,next,token)` but cannot supersede the gateway cursor. +### Admission and common bounds -Terminal collection holds one exclusive workspace mutation lease. It stops and proves absence of descendants and blocks Files writes, new turns, cancellation cleanup, deletion cleanup, and every other writer until finalization: +Validate source count, required access mode, destinations, destination relationships, bundle ownership, and working-directory syntax before any source side effect. For each Git/OCI source, deterministically resolve the authenticated caller identity plus canonical URL/repository to exactly one internal authorization/transport/credential route; reject zero or ambiguous matches before credentials or network access. Reauthorize that mapping before ref/descriptor acquisition and again before attaching any cache generation, including a local hit. Reauthorize uploaded bundles through the caller's artifact-store access before cache attachment. A cached generation proves content identity, never current caller authorization. -1. send/load the acknowledged base cursor; -2. compute the final canonical manifest under output byte/inode/time limits; -3. derive add/modify/delete operations; -4. open every added/modified regular file with rooted `openat`/no-follow semantics, hash while streaming, and require type/size/digest to equal its `after` state; -5. durably store file artifacts and one canonical `workspace-changes-.json`; -6. create the exact snapshot-mode checkpoint and verify its visible manifest equals the next cursor; -7. durably store the next cursor; -8. call idempotent ACK with base, next, and token; and -9. compare-and-set artifacts, checkpoint, authoritative cursor, and terminal response to committed. +For every network connection, re-authorize the full canonical path-specific Git URL or OCI repository route, TLS name, port, and each resolved IP. V1 rejects every HTTP redirect for Git manifests/refs, OCI manifests/blobs, and OCI authentication; it never broadens a route to origin scope. Enforce TLS and reject loopback, link-local, private, metadata-service, Unix-socket, and non-allowlisted targets unless the exact operator route owns that destination. -Change artifact media type is `application/vnd.allagents.workspace-changes.v1+json`. Schema and ordering match ADR 0002. `file_id` exists exactly for added/modified regular files after streamed bytes match their manifest state. +Enforce aggregate workspace byte/file limits across attached source generations and private clones. Materialization also caps path length, component count, archive entries, headers, compression ratio, subprocesses, output, and acquisition time. Reject absolute paths, `..`, NUL, duplicate normalized paths, case-fold collisions, devices, sockets, FIFOs, set-ID bits, capabilities, unsafe hardlinks, and escaping symlinks. Preserve only regular files, directories, confined symlinks, executable mode, and deterministic ownership. -`WorkspaceTurnCommit` resumes crashes without rerunning the harness. A crash before step 9 leaves the old gateway cursor authoritative even when ACK ran; retry supplies the old base and reproduces the transaction. Terminal response visibility occurs only at step 9. +### Cache keys and generations -Promptfoo applies ordered deltas to the exact builder baseline. Test reconstruction against the baseline snapshot; do not assert that deltas alone contain the original tree. +The source cache stores immutable, self-contained source generations, never a composed workspace. Key the canonical request below by SHA-256: -## Checkpoint, continuation, cancellation, and deletion +```text +GitCacheKey = { + materializer_schema: 1, + kind: "git", + canonical_url: string, + resolved_commit: 40 lowercase hexadecimal characters, + history: { mode: "full" } | + { mode: "shallow", depth: positive integer } +} -### Exact snapshot-mode checkpoint +OciCacheKey = { + materializer_schema: 1, + kind: "oci", + canonical_repository: string, + descriptor: exact direct OCI descriptor +} -Implement this exact snapshot-mode checkpoint before integrating turn finalization: +UploadedCacheKey = { + materializer_schema: 1, + kind: "uploaded_bundle", + artifact_digest: SHA-256 digest, + artifact_size_bytes: positive integer +} +``` -- skip `_git_ensure` and any root-Git commit; -- archive the complete normal private directory, including nested/root `.git` histories; -- reject any snapshot-origin `.harness` content before ready; -- exclude from checkpoints only `.harness/tmp/**`, `.harness/home/.codex/auth.json`, `.harness/home/.omp/agent/auth.json`, `.harness/home/.omp/agent/models.json`, and `.harness/home/.omp/agent/models.yml`; -- retain every other `.harness` path required for conversation, skills, plugins, and resume; -- stream through bounded disk rather than memory; -- store checksum/size; and -- reconstruct the checkpoint's visible manifest and require equality with the next cursor before publishing its pointer. +The access mode and destination are not cache-key fields: both affect attachment, not immutable bytes. A schema/version change creates a new key; no compatibility fallback is allowed. -Matching uses NFC-normalized POSIX paths after no-follow traversal. Directories named `node_modules`, `.venv`, `venv`, or other stock scratch names remain checkpointed when declared snapshot or session content. Any future exclusion changes require a checkpoint-schema revision and mixed-version reader gate. +Each ready directory contains one closed manifest and no credentials or transport metadata: -### Continuation +```text +{ + schema_version: "source_cache_generation.v1", + generation_id: UUIDv7, + cache_key_digest: SHA-256 digest, + materializer_schema: 1, + kind: "git" | "oci" | "uploaded_bundle", + content_digest: SHA-256 digest, + size_bytes: integer >= 0, + inode_count: integer >= 1, + created_at: timestamp +} +``` -A continuation: +The durable lease row is exactly `{generation_id, run_id, worker_id, expires_at}` with a unique `(generation_id, run_id)` key. The worker renews `expires_at`; expiry is a reconciliation alarm, never by itself proof that eviction is safe. Only a provably terminal/absent run permits stale-lease repair. -1. resolves the original session and enforces current principal/session ownership; -2. rejects supplied snapshot metadata; -3. rejects at or after `expires_at` before restore; -4. restores the exact checkpoint and cursor under quotas; -5. verifies binding, checkpoint checksum, journal schema, reserved-path schema, and working directory; -6. reacquires an evicted immutable base only by the same authorized direct digest when needed for integrity evidence, never to replace mutable checkpoint state; and -7. starts the stored harness only after the restored workspace is ready. +For an exact-key miss, one worker owns a singleflight materialization and all concurrent callers wait under their own acquisition deadlines. The owner creates a cache-owned private staging directory, fetches/unpacks and validates within bounds, removes acquisition credentials/transient locks, computes its content digest/bytes/inodes, writes the closed generation manifest and ready marker inside staging, makes the generation immutable to worker and sandbox identities, fsyncs data/directories, then atomically renames staging to the key's final path. Waiters lease only a final-path generation whose ready manifest validates. Failure or cancellation removes/quarantines staging and returns the same infrastructure cause to current waiters; a partial generation is never a hit. -Missing/corrupt binding, checkpoint, cursor, or identity fails closed. Do not start empty, substitute a newer descriptor, call the builder, or resolve a source. +Every attachment acquires a durable lease `(generation_id, run_id, worker_id, expires_at)` after reauthorization and before mount/clone. The worker renews it through agent and post-run execution. Cleanup releases it only after mounts/clones are gone. Eviction never selects a leased generation. -### Cancellation +Track byte and inode high/low watermarks. Crossing either high watermark evicts least-recently-leased unleased generations until both low watermarks are satisfied. Admission fails with `source_cache_capacity_exceeded` when sufficient safe space cannot be made. Startup and periodic reconciliation remove incomplete staging after its bounded grace period, validate ready manifests against directory metadata, repair only leases whose run is provably terminal/absent, quarantine corrupt generations, and resume watermark eviction. Reconciliation never guesses that a live lease is stale from age alone. -Preserve stock idempotent cancel endpoints and terminal `status: "cancelled"`. Stop descendants before final collection/checkpoint. Cancellation before ready releases provisional initialization and omits snapshot metadata. Cancellation after ready runs the same recoverable finalization transaction and retains stored metadata. +Cache directories are owned by a dedicated host identity. The sandbox cannot browse the cache root. It sees only an explicitly leased generation through a read-only bind mount, or a private clone located inside its run root. No cache file is ever exposed through a shared writable alias. -### Expiry and deletion +### Attachment by access mode -TTL begins at ready. Expiry/delete first atomically makes the session unavailable and stops descendants, then handles `WorkspaceTurnCommit` deterministically: +- `read_only`: bind-mount the leased generation at its destination with `ro,nosuid,nodev`; executable-file handling follows the fixed runtime policy. The agent and post-run checks see the same read-only destination. Filesystem writes and Git writes fail. +- `writable`: create a per-run reflink/CoW clone of the entire generation, including required Git history, inside the private run root. If reflink/CoW is unavailable, make a bounded full copy. Never hardlink cache files and never share a writable upper layer or clone across runs. Mount the clone writable at its destination. +- Build/test output that targets a `read_only` destination must be redirected to a writable source, scratch, or another writable destination. The gateway does not silently promote access. -| Turn state | Deletion action | -|---|---| -| `preparing` | Mark `aborted`, discard only unreferenced staging objects, retain the old checkpoint/cursor as authority, and finalize the response through stock cancellation/deletion semantics. | -| `durable` or `acknowledged` | Finish idempotent ACK/CAS to `committed` from durable evidence; do not rerun the harness. The terminal response may remain retrievable while the session is unavailable. | -| `committed` | Retain response/artifact records per stock policy and proceed with session cleanup. | -| `aborted` | Reconcile/discard unreferenced staging and proceed. | +### Git source -Only after turn state resolves does deletion remove the private tree with rooted no-follow operations, delete checkpoint/cursor state under retention policy, release `(binding_id,snapshot_key)` once, and tombstone/delete the binding. A turn record is removed only after response state and every referenced/unreferenced blob are reconciled. +- Accept canonical HTTPS URL and the defined ref/history fields only. +- Map authenticated caller identity plus canonical URL to exactly one internal transport/credential route in the worker control process; the request never names that route. +- Resolve the requested selector to an exact commit, derive `GitCacheKey`, reauthorize, then hit or singleflight the generation. +- On a miss, invoke pinned Git without a shell under sanitized config. Disable prompts, hooks, filters, credential persistence, alternates, submodules, LFS smudge, local/file transports, inherited config, and inherited proxies. Fetch complete reachable ancestry for `full` or exactly the requested bounded depth for `shallow`. +- Configure Git transport to reject HTTP redirects. Credentials are scoped to the exact matched canonical URL path and are never sent to another path, origin, helper, or submodule. +- Remove credential-bearing remote state and transient locks; verify the self-contained generation and checked-out commit before publish. +- When any Git source is attached `read_only`, set `GIT_OPTIONAL_LOCKS=0` for agent and post-run processes. Read-only Git commands must work without optional lock writes; mutating Git and filesystem commands must fail. +- Record requested ref, resolved commit, history, access, materializer schema, destination, and canonical tree digest. -Busy or uncertain state stays unavailable, counted, and queued for retry. Restart reconciliation resumes pending initialization/finalization/deletion from durable records and never trusts process-local filesystem facts. +### OCI source -## Stable failure contract +- Accept a canonical repository plus exact manifest descriptor. Map authenticated caller identity plus the full repository path to exactly one internal registry/credential/TLS/media-policy route; the request never names that route. +- The matched route pins the one registry API origin and optional OCI bearer-token realm origin/path. Credentials may be sent only to that registry and the pinned realm, with audience/scope restricted to the exact repository. Any unpinned authentication challenge, redirect, foreign layer URL, descriptor URL, alternate blob host, or cross-repository mount is rejected in V1. +- Reauthorize, derive the exact `OciCacheKey`, then hit or singleflight. On a miss, fetch the exact manifest and all blobs through the authorized repository transport, verifying media type, size, digest, config, and each layer while streaming. +- V1 accepts gzip OCI image layers only. Apply whiteouts in order into private cache staging under common path/type limits. +- Never resolve tags/indexes, follow redirects/foreign URLs, or publish the composed result. +- Record requested descriptor, resolved manifest/rootfs digests, access, materializer schema, and destination. -| Detail code | Condition | HTTP | Retryable | Required behavior | -|---|---|---:|:---:|---| -| `workspace_snapshot_invalid_request` | Closed-schema, version, media type, digest, size, first-turn, or continuation violation | 400 | no | Fail before catalog/cache/registry/workspace/lifecycle mutation. | -| `workspace_snapshot_unknown` | Exact descriptor absent from caller's authorized catalog view | 404 | no | Hide cross-principal existence; do not consult cache as authorization. | -| `workspace_snapshot_invalid` | Manifest/config/provenance/layer/path/link/type/signature/final-tree/cwd verification failure | 422 | no | Publish no generation and expose no partial workspace. | -| `workspace_snapshot_retention_invalid` | Catalog/config retention expired or cannot cover initialization deadline + maximum session TTL + safety margin | 422 | no | Fail before cache attachment; caller must publish a new snapshot descriptor. | -| `workspace_snapshot_unavailable` | Registry/DNS/transport, initialization deadline, or authorized manifest missing before `available_until` | 503 | yes | Fail the binding, release its provisional claim, and allow a new request with the same descriptor; never search another repository. | -| `workspace_contract_limit_exceeded` | Fixed format count/byte/path/ratio/metadata/file maximum | 413 | no | Stop bounded work; elapsed time never uses this code. | -| `workspace_capacity_exceeded` | Lower operator disk/inode/quota/worker/concurrency capacity | 503 | yes | Admit no partial generation or private tree. | -| `workspace_initialization_failed` | Verified artifact cannot publish/clone/baseline/transition because of internal failure | 500 | yes | Mark binding failed, clean/quarantine, and release provisional reference safely. | -| `session_expired` | Continuation at or after ready-based expiry | 404 | no | Preserve inherited UHP shape; do not restore or extend TTL. | -| `workspace_restore_invalid` | Binding/checkpoint/cursor/identity missing, corrupt, or inconsistent | 500 | no | Fail closed without empty-root or replacement recovery. | -| `workspace_collection_failed` | Manifest or artifact persistence cannot complete | 500 | yes | Keep previous cursor/checkpoint authoritative; resume transaction. | -| `workspace_checkpoint_failed` | Exact checkpoint cannot persist | 500 | yes | Do not finalize terminal response or advance cursor; resume without rerunning harness. | +### Uploaded-bundle source -Pre-ready failures omit snapshot metadata. Post-ready workspace failures return stored sanitized metadata except inherited `session_expired`. Provider/harness failures retain stock codes. Errors and logs never expose credentials, registry paths, private catalog names, local paths, source content, or raw tool stderr. +- The provider writes deterministic tar+gzip: lexical NFC POSIX paths, normalized uid/gid/mtime, explicit directories, preserved executable bits, no host-specific metadata, and one canonical gzip header. +- The artifact store verifies upload bytes before admission. The cache verifies them again on an exact bundle-key miss before publishing the immutable generation. +- An optional post-run bundle uses the same archive safety rules but is not acquired, extracted, or mounted by the worker until after agent cgroup/network teardown and credential/home removal. It exists only when a bundle executable requires it and is never a workspace source/cache generation. +- Record exact bundle reference, extracted rootfs digest, access, materializer schema, and destination. -For pre-ready retryable failures, retry means a new request/binding with the same descriptor; an idempotent duplicate returns the original failed response. For post-ready collection/checkpoint failures, retry resumes the same `WorkspaceTurnCommit` and never reruns the harness. +## Promptfoo provider -## Observability and audit +The provider is the first client of the general gateway. Its authoring configuration is closed but is not `AgentRunRequest`: -Use correlation-safe identifiers: request/response/session ID, binding ID, redacted descriptor prefix, snapshot-key prefix, cache outcome, state transition, duration, bounded byte/inode counters, and stable error code. Never log full private URLs, headers, credentials, provenance content, source filenames, file bytes, host paths, or complete digests where organizational policy treats them as sensitive. +```text +LocalMode = { + mode: "local", + state_directory: host path, + workspace: { + working_directory: string, + sources: [GitSource | OciSource | LocalBundleSource | UploadedBundleSource] + }, + post_run: null | { + bundle_path?: host path, + network_policy_id: logical ID, + commands: PostRunCommand[], + output_files: OutputFileRequest[] + }, + request_defaults: { agent, runtime_profile_id, agent_network_policy_id, limits } +} -Required metrics: +RemoteMode = { + mode: "remote", + base_url: HTTPS URL, + workspace: same authoring union, + post_run: same authoring union, + request_defaults: { agent, runtime_profile_id, agent_network_policy_id, limits } +} -- authorization allow/deny by stable reason; -- pending/ready/failed/deleting transitions and duration; -- registry bytes/time and verification failures; -- cache hit/miss/singleflight wait/build/evict/quarantine; -- reflink/copy selection, bytes, inodes, and duration; -- workspace quota utilization; -- manifest walk/change counts and duration; -- turn-finalization resume/failure stage; -- checkpoint bytes/time/failure; -- reference claim/activation/release/reconciliation; and -- cleanup backlog age and capacity impact. +LocalBundleSource = { + kind: "local_bundle", + path: host path, + access: "read_only" | "writable", + destination: relative path or "." +} +``` -Audit records identify the exact descriptor, catalog/policy decision reference, builder/provenance digests, gateway/upstream commit, and published image digest without secrets. +The provider enforces the same conditional invariant as the gateway schema: `bundle_path` is required exactly when any command has `executable.kind: "bundle"` and forbidden otherwise. Thus output-only collection and commands such as `{executable: {kind: "runtime", name: "npm"}, args: ["test"]}` require no bundle or upload. -## Implementation phases and exit proofs +Local mode starts an ephemeral loopback gateway and worker using the same API handlers, runner, schemas, and states with SQLite/local artifacts. It does not bypass admission or call an adapter directly. Remote mode uses authenticated HTTPS. Promptfoo's secret mechanism supplies gateway authentication outside provider config and request JSON. -Every phase changes the named real seam and ends with observable proof. Project-wide suites run only after focused phase work. Source-text assertions and mock forwarding are not proof. +For each `callApi(renderedPrompt, context)` the provider: -### Phase 0: Bootstrap repositories and characterize stock +1. derives stable Promptfoo evaluation/case/repetition identity, allocates and persists one UUIDv7 `run_id` plus idempotency key before side effects; +2. deterministically packages each local source/optional post-run bundle, allocates and persists one UUIDv7 `upload_id` per package, then idempotently reserves/uploads and substitutes bundle references without changing order/destinations; +3. builds and locally schema-validates the complete request, including `runtime_profile_id`, all ten limits, and requested-file artifact reservation; +4. submits, polls the tenant-owned run, and propagates Promptfoo abort to cancellation using the same identity; +5. waits for the cleanup-backed terminal result; +6. for every `ArtifactReference`, calls authenticated `GET /v1/artifacts/{artifact_id}` and verifies content type, length, digest, and expiry before use; +7. on `completed`, decodes verified `CapturedText` (or inline text), maps usage, retains truncation metadata, and attaches the complete result as `metadata.allagents_run`; +8. exposes all ordered command/file observations plus verified bytes to JS/LLM graders without interpreting them; +9. on `cancelled` or `infrastructure_error`, raises a typed provider error carrying the result and its durable partial evidence, so infrastructure is excluded from behavioral rates. -**Workspace Builder work** +Promptfoo owns matrices, repetitions, pass/fail, scores, rewards, and grader prompts; the provider does not duplicate or infer them. Export small accessors for `metadata.allagents_run`, but no gateway-specific grader or default pass rule. A configuration with `post_run: null` can use Promptfoo-native graders against agent output. -- Create repository, license/security/CI/release skeleton, pinned toolchain, CLI/library boundary, local registry/Git fixtures, and deterministic golden corpus. -- Record artifact media types and version ownership. +## Agent adapters -**Gateway work** +Both adapters implement: -- Create `feat/workspace-snapshots`, preserve fork history/license/NOTICE, add upstream remote, and record fork point. -- Capture the red stock limitations and unchanged stock UHP trace. -- Probe the target deployment filesystem for reflink correctness, quota support, path/mode semantics, and full-copy fallback before deeper implementation. -- Link [upstream issue #304](https://github.com/HarnessRouter/harnessrouter/issues/304) in the downstream delta record. +```text +AgentAdapter.run({ + instruction, + workspacePath, + modelProxy, + limits, + outputSink, + traceSink, + abortSignal +}) -> { termination, exitCode, Usage } +``` -**Exit proof:** both repos have protected reproducible builds; red fixtures prove root-Git mutation/nested-repo/deletion limitations; stock no-extension traces are recorded; production filesystem capability is known rather than assumed. +Requirements: -### Phase 1: Build and publish snapshot v1 +- Pin and verify the CLI version at worker startup. +- Invoke an argument vector directly, never a shell or caller-provided flags. +- Use a fresh adapter home/config directory in run scratch. +- Disable interactive approval, login, self-update, unshipped plugins/extensions, inherited config, and persistent history. +- Accept only the request's working directory, logical model, limits, rendered instruction, and pinned runtime-profile mapping. +- Route model traffic through a worker-owned agent-phase broker. The CLI receives no upstream credential, cookie, client certificate, provider endpoint, or policy body. +- Stream normalized final text through the runner-owned `outputSink` enforcing `max_agent_output_bytes`; never accumulate or return an unbounded string. Normalize native events to ATIF incrementally. Report usage conservatively; unknown fields remain null. +- A clean agent completion is completed even if no files changed. Startup failure, crash, protocol loss, invalid native event stream, or timeout is infrastructure failure. +- On abort, stop descendants and wait for sandbox confirmation. -**Work** +Conformance fixtures prove each adapter receives the instruction once, sees exact source destinations, mutates only its private workspace, streams bounded final output/usage/valid ATIF, honors logical model/runtime routing, leaks no canary secret, and is never invoked twice during recovery. -- Implement closed build spec plus trusted `BuildContext` and pre-network path/ownership authorization. -- Implement full-by-default Git and exact optional shallow acquisition under the shared per-connection fetch policy. -- Implement digest-pinned OCI input admission and secure extraction. -- Implement deterministic composition, exact reserved-path schema, independent canonical manifest/provenance schemas, canonical gzip OCI layers/config/manifest, complete-before-return publication, and digest-covered retention lease. -- Add malicious fixtures and bounded cancellation/restart cleanup. +## Post-run isolation, services, and raw evidence -**Exit proof:** root single-repo, multi-repo, Git-free, full-history, shallow, whiteout, empty-dir, link, executable, and repeated deterministic builds publish expected descriptors. Offline Git operations work at promised history depth. Every malicious fixture fails before descriptor return. Pulling by returned digest reconstructs the exact canonical tree and provenance with no secret material. +Post-run commands may build and run integration tests, so they operate in the same retained sandbox/runtime image and final workspace that the agent used. The workspace is not copied or overlaid by default. The agent result is sealed as output/usage/trajectory plus a process-ownership boundary; post-run filesystem changes remain ephemeral and are destroyed at cleanup. -### Phase 2: Add gateway contract, authorization, and durable binding +1. The retained runtime uses the pinned immutable profile revision's unprivileged identity, read-only root, explicit seccomp allowlist, private mount/process/IPC/UTS boundaries, and mandatory cgroup-v2 pids/memory/CPU/IO limits. Operator services, the agent, and each post-run command have distinct worker-owned cgroups; sandbox processes cannot change membership or controllers. +2. The pinned profile revision—not caller JSON—declares credential-free service launch manifests/images, implementation/image digests, readiness, restart policy, broker identities/ports, phase visibility, and resource limits. Services run in worker-owned namespaces/cgroups and are never directly addressable from agent/check loopback. +3. Run the agent in its own default-drop network namespace. nftables or an equivalent kernel layer filters loopback as well as external traffic; the namespace exposes only agent-policy broker endpoints. A post-run-only service has no route or broker endpoint in this namespace. +4. After adapter completion, close model/source brokers, remove adapter secrets/home, stop/reap the agent cgroup, destroy its network namespace plus conntrack/flows/broker handles, and confirm no agent process or socket remains. Only then durably seal bounded output, usage, and ATIF while retaining the mount/runtime/workspace and service cgroup. +5. If `bundle` is present, only after step 4 acquire its bytes from the input artifact store, reauthorize and verify digest/size, safely extract them, and mount the result read-only at the empty worker-reserved path that was absent from the agent mount namespace. Resolve every bundle executable without symlink escape. No bundle executable means no bundle bytes are acquired or mounted. +6. Create a fresh check network namespace with default-drop filtering over loopback/external traffic. Resolve `post_run.network_policy_id` to explicit operator-brokered external/service endpoints; no agent-phase flow or direct sidecar socket is reused. +7. Health-check services declared by the pinned profile revision through the same phase broker and restart them under its immutable policy if allowed. Failure is infrastructure error. Agent-owned daemons are never retained/restarted, and undeclared/post-run-disallowed services are unreachable. +8. Resolve runtime names only through the pinned revision's fixed approved read-only `PATH`, never the workspace or inherited `PATH`. Execute selected executable/literal args in a fresh check cgroup/process group and the check network namespace, with bounded scratch/fixed nonsecret environment. Source access modes remain unchanged. +9. Capture bounded stdout/stderr, reap the command cgroup, then durably seal its `completed` or `timed_out` observation before starting the next command. Launch/sandbox/evidence failure seals `unavailable`; cancellation/deadline/prior failure fills untouched commands `not_run`. +10. After commands settle, collect requested files in order without following final symlinks and durably seal every observation before advancing. Unsafe/read/artifact failure seals `unavailable`; remaining files become `not_run`. Admission-reserved maxima guarantee collected-file budget. +11. Deterministically promote optional result artifacts, stop services/check namespace, and destroy the run workspace. Post-run build output persists only when explicitly requested within bounds. -**Work** +The boundary does not calculate a generic diff, publish a snapshot, preserve a workspace, or create a user-visible change artifact. Post-run observations are evidence, not a judgment. -- Parse vendor extension in create-response path; reject every obsolete source schema and continuation injection. -- Add literal capability `allagents_workspace_snapshot_v1`, sanitized response metadata, private repository/catalog/signature binding fields, reference records, pending transition, ready-based TTL, minimum-reader routing, and restart reconciliation. -- Implement unique repository/catalog resolution, descriptor authorization, and cache-hit reauthorization. +## Secrets and operator-policy boundary -**Exit proof:** malformed/unknown fields fail `400` with zero cache/network/write activity; unknown/unauthorized both fail `404`; a valid descriptor persists pending identity before fetch; duplicate idempotency shares one binding; continuation replacement fails before hydrate; stock requests remain trace-equivalent. +Credentials and operator policy exist only in gateway/worker control processes and dedicated brokers. -### Phase 3: Implement admission, immutable cache, and private tree +- Git/OCI credentials are resolved after authorization and passed through broker sockets or inherited descriptors not mounted into agent/post-run namespaces. +- Model credentials terminate in the model proxy. Agents see only a non-secret per-run local socket and logical model mapping. +- Gateway API credentials authenticate provider transport and never enter run storage. +- After agent exit, the worker closes model/source brokers, destroys the agent cgroup/network namespace/flows, and removes adapter credential/config mounts before acquiring optional bundle bytes or resolving any post-run executable. +- Agent and post-run commands receive separate fixed environment allowlists containing locale, deterministic home/temp paths, and required non-secret switches. They do not inherit the worker environment. +- Post-run commands receive no model/source credential socket or agent home. Their fresh network namespace has default-drop loopback/external rules and only independently authorized operator-brokered endpoints; no direct sidecar or host socket is exposed. +- Operator configuration owns caller/source-route mappings (including pinned OCI token realms), credentials, model routes, runtime profiles (image/sandbox/cgroups/tools/services), phase network brokers, sidecar definitions, resource ceilings, artifact retention, and secrets. Caller JSON contains only canonical source addresses, uploaded artifact IDs, approved runtime names/relative bundle paths, literal arguments, and authorized logical model/runtime/network IDs—never a credential identity, raw path, image, service command, socket, or policy body. +- Structured logs are redacted at ingestion. Seeded-canary tests fail on appearance in request/result JSON, cache generations, ATIF, command output, requested files, subprocess environment, errors, or artifacts. -**Work** +`agent_network_policy_id` and `post_run.network_policy_id` are authorized separately per tenant and realized in separate default-drop namespaces. Allowed external or service access exists only through phase-specific operator brokers. Agent namespace/flows are destroyed before check namespace creation; post-run-only and undeclared service endpoints are never reachable by the agent, including through loopback. -- Fetch from the persisted repository and verify all descriptor/media/signature/provenance/tree/layer/retention identities. -- Apply layers securely; independently reject `.harness` aliases and recompute public manifest plus private full-tree seal. -- Implement inaccessible immutable cache ownership, FD-based clone lease, seal checks, singleflight, watermarks, eviction, quarantine, and reconciliation. -- Implement private reflink clone and forced full-copy fallback; apply inputs/instruction merge, capture the initialization delta, validate cwd, and transition ready only after ready-manifest cursor storage. +## Cancellation, deadlines, recovery, and cleanup -**Exit proof:** exact descriptor miss publishes once under concurrency; hits reauthorize; expired/short retention returns `422`, while a promised-but-missing manifest returns `503` without repository search; initialization delta reconstructs the ready tree from builder baseline; failed final verification exposes no Files/provider/harness state; session UIDs cannot search/read/mutate guessed cache paths; malicious reserved paths fail independently; both reflink and copy pass. +- One root abort controller fans into authorization/materialization, adapter, trace/artifact writers, service manager, post-run collector, and sandbox operations. +- Cancellation is idempotent. Repeated cancellation returns the same state; a terminal run is unchanged. +- Total and phase deadlines are persisted as absolute times so worker restart cannot reset them. +- Agent timeout revokes model/network access and kills only the agent cgroup before run cleanup; operator service/check cgroups have separate ownership. +- Post-run never starts after cancellation/agent infrastructure failure/result-seal failure. Cancellation or deadline during post-run stops the current check cgroup, seals available bounded streams, fills the full evidence vector with `not_run`/`unavailable`, then cleans up. +- A command timeout is a `timed_out` observation and later commands may continue. Exhausting the phase/total deadline produces infrastructure error with the sealed partial vector. +- Neither agent nor post-run command is retried. Exact-key materialization is singleflight, not agent retry. Safe network reads may retry only before side effects and under the original deadline. +- Cleanup runs after success, failure, cancellation, process crash recovery, and worker shutdown. It is idempotent and keyed by run ID. It stops check/service cgroups, removes any injected bundle and private clones/mounts/root, then releases cache leases. +- A cleanup-step failure atomically replaces the pending terminal outcome with `cleanup_failed`/`infrastructure_error`, keeps already durable evidence, and leaves the status envelope nonterminal with `result: null`. Cleanup retry/reconciliation continues; only verified resource absence and lease release allow publication of that infrastructure-error result. +- Startup reconciliation finds nonterminal run records, private roots, cache staging, and leases. It completes safe staging/cleanup; if agent invocation may have begun, it records infrastructure error instead of invoking again. It never publishes a result while cleanup residue or a run lease remains. +- Run/result records and immutable cache generations follow operator retention. Input bundles have separate non-downloadable retention. Every result artifact stores owner tenant/run/purpose plus `expires_at`, remains immutable until expiry, then returns 410 to its authorized owner and is deleted asynchronously. Per-run workspaces, clones, optional bundles, and scratch are destroyed before result visibility. -### Phase 4: Add exact checkpoint, manifest journal, and recoverable finalization -**Work** +## Delivery phases and observable exit proofs -- Implement snapshot-mode checkpoint/hydrate first: no root Git, exact `.harness` policy, complete Git history, checksum, and visible-manifest verification. -- Add `manifest-v1` behind produced routes with explicit gateway-to-runner cursor input while retaining stock root Git. -- Implement streamed file/hash binding and exact add/modify/delete/type/mode/link behavior under the exclusive mutation lease. -- Add `WorkspaceTurnCommit` and integrate artifacts, exact checkpoint, next cursor, ACK, CAS, deletion states, and terminal visibility. +### Phase 0 — Bootstrap the one repository -**Exit proof:** independent fixtures produce exact operations regardless of Git state; `.git` and `.harness` never appear; injected failure at every checkpoint/ACK/CAS/deletion stage resumes to one logical artifact/checkpoint/cursor without rerunning; baseline plus deltas reconstructs final state; raced writers cannot make artifact bytes, manifest, and checkpoint disagree. +Create the Bun workspace, package boundaries, pinned toolchain, CI, license, schema-generation command, gateway/worker process entrypoints, PostgreSQL/SQLite migrations, artifact/cache interfaces, and health/readiness endpoints. Add dependency-boundary checks that keep Promptfoo, Fastify, and adapters out of `packages/runner`. -### Phase 5: Complete continuation and lifecycle +**Exit proof:** a clean checkout installs from the lockfile, generates schemas/types, applies and rolls back migrations on disposable state, starts gateway and worker, reports healthy/readiness with artifact/database/cache dependencies, and shuts down cleanly. No run submission, agent, cache materialization, or end-to-end claim belongs to Phase 0. -**Work** +### Phase 1 — Freeze contracts, API, persistence, and state -- Implement continuation, cancellation, ready-based expiry, state-specific explicit deletion, reference release, and unavailable-first reconciliation. -- Exercise `RunnerWorkspaceFiles` and `CheckpointWorkspaceFiles` against live and restored snapshot sessions. -- Add mixed-version admission fencing and internal minimum-reader routing. +Implement exact schemas/generated types, canonical hashing, tenant-owned upload/run/result-artifact persistence, idempotent bundle reserve/upload, artifact download, status/cancel, internal submission claims, duplicate-before-admission ordering, mutable admission, state CAS, errors, deadlines, cancellation, immutable runtime-profile revision pinning, ten limits/artifact reservation, direct provenance, bounded `CapturedText`, and full partial `PostRunEvidence` unions. Add golden documents for every union/status/nullability/limit/provenance/error. -**Exit proof:** mutations and Git history survive turns/restart; corrupt evidence fails closed; cancellation produces one terminal state; polling/continuation do not move expiry; every turn-commit deletion state reconciles without orphaning/publishing inconsistent data; concurrent expiry/delete/restart releases one reference; incompatible replicas reject before hydrate. +**Exit proofs:** -### Phase 6: Cross-repository and Promptfoo integration +- unknown fields, omitted runtime profile/source access, host paths, policy bodies, shell strings, overlap, noncanonical sources, caller credential/image/service controls, malformed executable unions, incorrect bundle conditional, and excessive/reservation limits fail before any public run; +- exact duplicate lookup returns an existing run after source/profile policy changes without mutable re-admission; conflict rejects; a new identity that fails admission leaves no public run/job, while 50 concurrent valid identical submits create one; +- every run/upload/result artifact stores owner tenant; same-tenant GET/cancel/download succeeds, while another tenant using the same run/artifact/upload UUID receives the same non-enumerating 404 as an unknown object and cannot cancel/read/write; +- bundle reservation retry with one `upload_id` returns the same reference, changed metadata conflicts, exact PUT replay succeeds, and cross-tenant PUT fails; +- artifact GET streams immutable bytes with matching media type/length/digest/expiry, returns 410 after authorized expiry, rejects input bundles/detached artifacts, and refuses corrupt stored bytes; +- golden results prove ten effective limits, `DirectWorkspaceProvenance`, revision-pinned `RuntimeProvenance`, bounded `CapturedText`, and completed/cancelled/infrastructure variants with null or full ordered partial evidence vectors; +- cancellation races terminalization deterministically; terminal results are immutable; public errors/logs omit seeded secrets and nonsecret resolved paths. -**Work** +### Phase 2 — Cache, package, and compose bounded sources -- Build fixture snapshots with the published builder binary, push to an authenticated local registry, authorize exact descriptors, and run Codex and OMP through the built gateway. -- Implement Promptfoo flow: retain builder snapshot/provenance, invoke UHP with descriptor, apply initialization delta, consume ordered response changes, and reconstruct final tree. -- Prove no task-time call to builder and no Git/source credentials in gateway or harness. +Implement deterministic packaging, streamed uploads, exact Git/OCI/bundle cache keys, cache-owned staging, exact-key singleflight, atomic complete generations, authorization on hits, durable leases, reflink/full-copy attachment, read-only mounts, watermarks, eviction, reconciliation, private root allocation, aggregate bounds, ordered provenance, working-directory validation, and cleanup. -**Exit proof:** one snapshot runs with both harnesses; cache miss/hit have identical semantics; builder baseline + initialization delta equals `ready_manifest_digest`; subsequent deltas reconstruct the final private tree; continuation performs no mutable source resolution. +**Exit proofs:** -### Phase 7: Upstream the generic seam +- two packagings of one tree are byte-identical; submitted JSON contains a bundle reference/access mode and no host path; +- three sources land at declared non-overlapping destinations and provenance preserves request order, canonical Git URL/OCI repository, access, exact resolved identity, and materializer schema; +- overlap, omitted access, destination aliasing, or `.` with multiple sources fails before credential/network/cache attachment; +- Git keys use canonical URL plus exact commit/history; OCI keys use canonical repository plus exact descriptor; bundle keys use exact digest/size; schema/key changes never fall back to another generation; +- Git/OCI reject every redirect; path-scoped credentials never reach another path/origin; OCI accepts only its pinned token realm/exact repository and rejects foreign blob/descriptor URLs, cross-repository mounts, and unpinned auth challenges; +- 50 concurrent cold runs for one exact large Git or OCI source perform one fetch/materialization, publish one complete generation, and acquire 50 leases; +- a cache hit after caller authorization is revoked or remapped is denied before mount even though identical bytes exist locally, and a different tenant cannot obtain access merely by guessing the canonical source address; +- `read_only` mounts share one immutable generation, accept read-only Git commands under `GIT_OPTIONAL_LOCKS=0`, and reject filesystem/Git writes; +- two `writable` runs receive distinct reflink/CoW clones (or bounded full copies), can mutate independently, and cannot change each other or the cache; inode/link checks prove no hardlinked writable alias; +- malicious traversal/links/devices, digest mismatch, gzip bomb, aggregate limits, and corrupt/incomplete generations fail before agent start; +- eviction skips a generation while any run lease exists, removes it only after final lease release, and converges from high to low byte/inode watermarks; +- restart reconciliation removes stale staging, preserves live leases, quarantines corrupt generations, and never exposes a partial generation; +- run cleanup removes mounts/clones/root and releases leases while leaving authorized immutable generations reusable. -**Work after maintainer agreement on issue #304** +### Phase 3 — Run Codex and OMP exactly once -- Split changes into the smallest accepted upstream reviews: initializer/lifecycle seam, explicit-cursor manifest journal, recoverable terminal-finalization seam, and focused tests/docs as maintainers direct. -- Keep stock implementations default and avoid AllAgents names/source schemas in core. -- Maintain one downstream delta map from every patch to upstream issue/PR/release/removal condition. +Implement admission-time immutable runtime-profile revision/digest pinning, dispatch-time re-verification, exact `RuntimeProvenance`, the production `RunSandbox` invariants, cgroup-v2 controllers, default-drop agent networking/brokers, model proxy, retained mount/runtime, Codex/OMP adapters, bounded `CapturedText`/ATIF/usage, stop/reap/network-destruction boundary, and durable invocation marker. -**Exit proof:** upstream tests prove initialization order, idempotency, replacement rejection, explicit cursor handoff, exact journal, capture-before-ACK, checkpoint/cursor durability before terminal visibility, continuation, restore failure, and unchanged stock behavior. Downstream adapter builds against all accepted seams without aliases; unaccepted seams remain explicit fork deltas. +**Exit proofs:** -If maintainers reject or materially reshape the proposal, update ADR 0002 before introducing a different fork architecture. Do not push product-specific source preparation into HarnessRouter as a shortcut. +- the full proof moved from Phase 0 now runs: a clean checkout starts disposable services, admits one no-post-run Git fixture, materializes/cache-leases it, invokes each adapter once, returns bounded output/usage/ATIF/source+runtime provenance, removes every run resource/lease, and retains only the authorized immutable source generation; +- a profile ID resolves once at admission to a persisted immutable revision. Dispatch uses only that revision and verifies the expected `profile_digest`, runtime `image_digest`, sandbox-policy version, every tool/service `implementation_digest`, containerized service `image_digest`, cgroup ceiling, and broker identity; changed/missing/mismatched revisions fail before agent invocation, and returned provenance proves exact implementations without paths, commands, policy bodies, or secrets; +- output below/at/above `max_agent_output_bytes` yields verified inline/artifact/truncated `CapturedText` without unbounded memory or persistence; +- non-root UID/GID mapping, empty capabilities, `no_new_privs`, private namespaces, read-only root, bounded tmpfs, masked proc/sys/cgroup/devices/host sockets, explicit seccomp, and no writable control-plane mount pass escape/mount/device/socket/syscall probes; +- fork-bomb, memory/OOM, CPU, and IO fixtures remain within cgroup-v2 limits and cannot affect a sibling run or worker; +- agent namespace default-drop covers loopback/external traffic: allowed broker/model fixtures work, undeclared and post-run-only services fail, and teardown removes all agent processes, sockets, flows, and namespace handles; +- crash/timeout/cancel produce infrastructure/cancelled results only after cleanup; crash around the invocation marker never starts a second agent; canary credentials/policy stay absent. -### Phase 8: Review and release exact digests +### Phase 4 — Seal agent output and collect evidence in the retained sandbox -Run final architecture/security review before the green E2E. Resolve important findings, then: +Implement durable evidence skeleton/observation sealing, an empty reserved late-mount point plus post-agent bundle acquisition/mount, pinned-profile tool/service resolution, distinct default-drop check network namespace/brokers, ordered commands/files, partial cancellation/error vectors, aggregate artifact allocation/promotion, artifact metadata/expiry, cleanup downgrade, and publication gate. -1. publish builder and gateway candidates with SBOM and build provenance; -2. read both back and record exact registry manifest digests; -3. deploy/test only those digests on the preflighted production-equivalent filesystem; -4. keep `allagents_workspace_snapshot_v1` disabled until every serving gateway/runner is compatible, then prove version-aware routing rejects old replicas before hydrate; -5. run stock UHP conformance and snapshot E2E with both supported harnesses; -6. run N-1-to-candidate upgrade and prove compatible-reader rollback or enforce unavailable-first drain before old code serves snapshot sessions; -7. repeat race-sensitive singleflight, cache-path attack, writer race, ACK/CAS, cancellation, crash, turn deletion, reference release, eviction, and cleanup cases against the exact digest; -8. run fresh/same-volume restarts at pending initialization, ready, active turn, durable/acknowledged finalization, checkpoint, and every deletion state; -9. run secret, artifact-content, and telemetry scans; and -10. record upstream issue/PR status, downstream delta, fixture digests, builder/gateway digests, SBOMs, provenance, compatibility versions, and E2E report. +**Exit proofs:** -Do not prove a local build and assume the published image is equivalent. Do not support an architecture that was not built, preflighted, and tested; V1 MAY declare `linux/amd64` only. +- output-only and runtime-only requests omit a bundle. Before agent teardown even a requested bundle's bytes are neither acquired nor mounted and its reserved point is empty; afterward profile `npm` with literal `["test"]` runs in the exact final workspace, while a bundle executable triggers the late verified read-only confined mount; +- each command/file seals before the next. Cancellation between commands returns prior evidence plus ordered `not_run`; cancellation during command 2 seals it `unavailable` with safe bounded partial streams and later items `not_run`; command-2 launch failure returns command 1, command 2 `unavailable`, command 3/files `not_run`; file read/artifact failure similarly preserves earlier observations; +- exit zero/nonzero/timed-out commands are lifecycle completed with `completed|timed_out` observations; a timed-out cgroup is gone before the next command; +- read-only sources stay read-only; writable sources hold build output across commands; requested files produce collected/missing/limit-exceeded/not-regular states; +- the check namespace exists only after agent namespace/flows are destroyed. Default-drop includes loopback; post-run-only brokered sidecar works for checks but was unreachable to the agent; undeclared/direct sidecar and external endpoints fail; +- model/source credentials and agent home are absent and the agent cgroup/network/flows are destroyed before any optional bundle-byte acquisition, extraction, mount, or executable resolution; +- admission rejects `sum(output_files.max_bytes)` over budget. Actual files charge first, then agent output, command stdout/stderr order, then trajectory; exact-boundary fixtures prove accounting/no dedup discount and deterministic inline-truncated/omitted fallback without infrastructure error; +- downloaded artifact bytes match owner/run/purpose/media/digest/size/expiry; hidden input bundles never appear as downloadable result artifacts; +- runtime/bundle/service/sandbox/launch/unsafe collection failures return infrastructure error with the exact partial vector; cleanup failure preserves it, withholds all result visibility through reconciliation, and never publishes completed; +- large output stays bounded and ATIF fallback remains valid with one truncation marker. -## Completion checklist +### Phase 5 — Complete Promptfoo local and remote flows -### AllAgents Workspace Builder +Implement provider modes, crash-safe run/upload identities, multi-source/runtime-profile mapping, idempotent packaging/upload, polling/cancel, artifact download/verification, bounded output/usage/metadata mapping, partial-evidence accessors, and native matrix/repetition/JS/LLM examples. -- Repository exists with protected main, pinned toolchain/dependencies, license, security policy, and reproducible releases. -- Build spec v1, full/shallow Git, OCI inputs, secure composition, provenance, canonical manifest, and OCI publication match this plan. -- Versioned builder result and digest-covered `available_until` match registry retention; delayed execution inside the admitted window succeeds. -- Credentials and private transport configuration never enter artifact/log/output. -- Independent provenance/manifest implementations, canonical gzip, golden fixtures, malicious fixtures, and published readback reproduce identity. +**Exit proofs:** -### AllAgents Gateway +- one case runs local/remote with schema-equivalent results; Git/OCI/local bundles preserve order/access without host paths; +- provider maps runtime plus phase-network logical IDs and never embeds an image, PATH, service, policy, or credential; +- lost reservation/submit/download responses replay `upload_id`/run/artifact identity without duplicate upload/run/agent; +- inline and artifact-backed agent output, command streams, files, and trajectories are byte/digest/media verified; expiry/corruption becomes typed infrastructure, never a grader input; +- cancelled/infrastructure errors carry their result/partial evidence to diagnostics but are excluded from behavioral rates; +- a Codex/OMP matrix with two repetitions creates four private runs sharing authorized immutable generations; JS/LLM graders alone decide outcomes from verified raw evidence; +- cross-tenant run/cancel/artifact attempts fail identically in local and remote modes; no host path, credential, policy body, process, mount, clone, or run directory survives. -- Fork relationship, history, `LICENSE`, `NOTICE`, upstream remote, and stock behavior remain intact. -- Runtime source composition and obsolete schemas/branches are absent. -- Vendor request/response, authorization, binding/reference state, cache, private clone, ready gating, journal, turn commit, checkpoint, continuation, cancellation, expiry, and deletion match ADR 0002. -- Cache hits reauthorize; private repository/catalog/signature subject persists; durable references are idempotent; TTL starts at ready; expired/short retention returns `422`, deadlines/promised-retention misses return `503`, and invalid cwd returns `422 workspace_snapshot_invalid`. -- Snapshot baseline plus initialization delta equals ready manifest; exact response changes include deletion and exclude `.git`/`.harness`; streamed bytes equal `after` state before ACK. -- Response visibility cannot outrun journal ACK, exact checkpoint, or authoritative cursor durability. -- Reflink/copy, inaccessible cache, reserved paths, quota, mixed-version fencing, restart, rollback-or-drain, turn deletion, and race repetitions pass against the exact published digest. +## Deferred integrations -### Promptfoo/integrator +Harbor, Terminal-Bench, and SWE-bench integration are outside this V1 implementation plan. V1 has no Harbor source variant, request, backend, package, result variant, provenance variant, sandbox delegation, or patch exporter. Future work requires a separate ADR and closed schema extension defining its own request plus result/provenance mapping into raw nongrading evidence without changing direct-mode semantics. -- Preparation and execution are separate steps. -- Builder result and snapshot baseline are retained. -- UHP request carries only the direct descriptor extension. -- Initialization delta is applied before ordered response deltas; neither is treated as a self-contained snapshot. -- Continuation omits snapshot metadata and reuses the original session. +## Focused release E2E -### Upstream/migration +The release gate runs the built gateway, worker, packaged Promptfoo provider, real local Git/OCI/object-store fixtures, source cache, and pinned agent fixtures/CLIs. For each scenario record request/result JSON, state sequence, invocation count, source fetch/materialization count, generation/lease state, resolved provenance, per-run cleanup, and artifact digests. -- [HarnessRouter issue #304](https://github.com/HarnessRouter/harnessrouter/issues/304) has a recorded maintainer decision. -- Downstream code is isolated behind initializer, explicit-cursor journal, and recoverable-finalization interfaces compatible with the proposal. -- Accepted upstream patches contain no AllAgents source model. -- Only an upstream release containing every required seam triggers deletion of corresponding forked implementations; remaining deltas stay explicit. +| Scenario | Expected proof | +|---|---| +| Multi-source Codex/OMP direct runs | exact access/destinations; one invocation; source+runtime provenance; cleanup | +| Runtime-profile revision matrix | admission pins `profile_digest`; dispatch re-verifies exact runtime/tool/service implementation and image digests; no caller paths/commands | +| Runtime `npm test`, no bundle | approved profile tool, literal args, same final workspace, no bundle upload/mount | +| Bundle executable | bundle required iff used; worker does not acquire/mount bytes before agent teardown; later verified confined immutable mount | +| 50 cold exact-source runs | one materialization, 50 leases, isolated writable clones/shared read-only generation | +| Redirect/OCI credential attacks | all redirects/foreign URLs/unpinned realms rejected; credentials stay exact path/repository scoped | +| Read-only/writable/eviction | writes fail on shared RO; writable clones isolate; live lease blocks eviction | +| Sandbox breakout suite | non-root/no capabilities/no_new_privs/private namespaces/RO root/seccomp/hidden host resources | +| Fork/OOM/CPU/IO attacks | cgroup-v2 ceilings contain run without worker/sibling impact | +| Phase network isolation | default-drop loopback/external; agent cannot reach post-run service; check uses broker only | +| Nonzero/timed-out commands | lifecycle completed; exact completed/timed_out evidence; timed-out cgroup reaped | +| Cancel during second command | first observation retained; in-flight command unavailable with safe partial streams; untouched commands/files ordered not_run | +| Launch/file collection failure | prior observations retained; failed item unavailable; later items not_run; infrastructure_error | +| Artifact-budget boundary | files charge first, then agent/streams/trajectory; deterministic truncated/omitted fallback | +| Bounded agent output | inline/artifact/truncated `CapturedText` at below/exact/above cap; no unbounded state | +| Result artifact download | tenant/run-authorized immutable bytes verify media/size/digest; expiry 410; corrupt bytes refused | +| Cross-tenant UUID replay | run GET/cancel, upload PUT, artifact GET all match unknown-object 404 with no effect | +| Upload response loss | same `upload_id` returns one reservation/reference and exact PUT replay | +| Duplicate run after policy change | same result without re-admission; conflicting body rejects; new admission failure leaves no run | +| Promptfoo local/remote matrix | persistent identities, verified artifact dereference, four private runs, grader-only outcomes | +| Crash/cleanup fault | no second agent; result hidden through cleanup; final infra result retains durable evidence | +| Seeded canaries | absent from environments/cache/public artifacts/errors/logs | + +CI may retain sanitized JSON, immutable cache fixtures, and referenced evidence, but never a live run workspace, writable clone, generic diff, grading decision, reward, or modified-workspace artifact. -### Operator +## Completion checklist -- Supply secrets through protected mechanisms only. -- Maintain snapshot authorization catalog, registry retention, cache/quota policy, and production filesystem capability. -- Publish and deploy by digest. -- Accept release only when builder/gateway digests, SBOMs, provenance, compatibility record, upstream delta, and E2E evidence identify the same tested artifacts. +- [ ] `allagentsdev/allagents-gateway` is the only new repository and contains provider, gateway API, worker/runner, contracts, source cache/composer, service manager, post-run collector, and both agent adapters. +- [ ] Runtime, HTTP, persistence, queue, cache, isolation, schema, identity, and ATIF choices match this plan; no implementation framework remains undecided. +- [ ] Closed schemas cover `runtime_profile_id`, ten limits, bounded `CapturedText`, aggregate artifact budget, `PostRunEvidence`, direct workspace and revision-pinned runtime provenance, upload idempotency, artifact references, and every result nullability/invariant. +- [ ] Promptfoo is the first caller and sole evaluation/grading layer; gateway JSON has only completed/cancelled/infrastructure_error and no pass/fail/reward. +- [ ] Authentication/schema/hash and tenant-scoped duplicate/conflict lookup precede mutable admission; exact duplicates bypass re-admission; new admission failures create no public run/job. +- [ ] Every upload/run/result artifact persists owner tenant; duplicate submit, GET, cancel, upload PUT, and artifact GET resolve tenant+object with indistinguishable unknown/cross-tenant 404 behavior. +- [ ] Bundle reserve uses persisted client `upload_id`; exact reserve/upload replay is idempotent and changed metadata conflicts. +- [ ] Authenticated artifact GET serves only tenant/run-bound unexpired result bytes with immutable media/size/digest headers; input bundles are not exposed, owner expiry is 410, and corruption is refused. +- [ ] `max_artifact_bytes` reserves requested-file maxima and charges actual files, agent output, ordered command streams, then trajectory; no dedup discount; budget exhaustion uses bounded truncated/omitted evidence rather than infrastructure failure. +- [ ] Local paths are packaged before submission; gateway JSON has no host path. +- [ ] Git canonical URL/ref/history, OCI repository/exact descriptor, bundles, access, schema, destination, and resolved identities return ordered provenance. +- [ ] Git/OCI use exact path-specific caller routes, reject all redirects, pin OCI token realm/repository scope, and reject foreign descriptor/blob URLs; cache hits reauthorize. +- [ ] Exact cache keys singleflight into complete immutable generations with durable leases, byte/inode watermarks, safe eviction/reconciliation, direct RO mounts, and private CoW/full-copy writable clones without hardlinks. +- [ ] Admission persists one immutable runtime-profile revision and `profile_digest`; dispatch uses only that revision and re-verifies runtime image, sandbox, broker, tool/service implementation, and containerized-service image identities before agent invocation; `RuntimeProvenance` proves them without secret args/paths. +- [ ] `RunSandbox` enforces unprivileged non-root UID/GID, empty capabilities, no_new_privs, private namespaces, RO root, bounded tmpfs, masked host resources/sockets, explicit seccomp allowlist, mandatory cgroup-v2 pids/memory/CPU/IO, and no writable control-plane mounts. +- [ ] Agent/check network namespaces are distinct and default-drop including loopback; all external/service access is phase-brokered; agent flows/namespaces die before post-run, so post-run-only services are never agent-reachable. +- [ ] Codex/OMP invoke once and stream at most `max_agent_output_bytes`; completed results expose inline/artifact/truncated `CapturedText`, usage, and valid bounded ATIF. +- [ ] Optional bundle is present iff a bundle executable is requested; dispatch creates only an empty reserved late-mount point and does not acquire or mount bundle bytes until agent cgroup/network/flows and credentials/home are gone. Runtime tools come only from the pinned profile revision's PATH; args are literal; commands use the exact final workspace. +- [ ] Post-run initializes one full ordered evidence skeleton, seals each command/file before advancing, and represents completed/timed_out/not_run/unavailable plus all file states. +- [ ] Cancelled/infrastructure results retain durable partial evidence; behavioral nonzero/timeout/missing/limit/non-regular observations do not become infrastructure. +- [ ] Provider persists run/upload identities, downloads/verifies all artifact-backed output/evidence/trajectory, exposes partial evidence diagnostically, and leaves grading to Promptfoo. +- [ ] Cleanup removes every process/namespace/flow/service/mount/clone/root and releases leases before result visibility; cleanup failure forces infrastructure_error while retaining evidence through reconciliation. +- [ ] Credentials/policy remain outside caller JSON, run environments, cache, evidence, errors, artifacts, and logs. +- [ ] No UHP, HarnessRouter fork, reusable session, continuation, checkpoint, composed-workspace/snapshot publication cache, generic core change artifact, or second repository remains. diff --git a/docs/research/allagents-gateway-snapshot-boundary.md b/docs/research/allagents-gateway-snapshot-boundary.md deleted file mode 100644 index 6e5d6cc8..00000000 --- a/docs/research/allagents-gateway-snapshot-boundary.md +++ /dev/null @@ -1,367 +0,0 @@ -# Prebuilt immutable workspace snapshots at the HarnessRouter boundary - -## Decision - -AllAgents Gateway should consume **one prebuilt immutable workspace snapshot**, while a separate preparation plane owns Git resolution, OCI acquisition, credentials, multi-repository composition, source policy, provenance generation, and snapshot publication. - -The execution boundary should be one direct, digest-pinned OCI image-manifest descriptor. The gateway should never receive Git URLs, refs, per-repository destinations, source credentials, tags, indexes, or caller-selected registry locations. It should authorize the descriptor against one operator-configured snapshot repository, materialize a private writable session tree, establish an exact filesystem-manifest cursor, and then enter HarnessRouter's existing turn lifecycle. - -This should be implemented in **two source repositories**: - -1. `allagents-workspace-builder`: product-specific Git/OCI/multi-repository preparation and immutable snapshot publication. -2. `allagents-gateway`: the stock-derived execution distribution, carrying only a generic immutable-snapshot initializer, explicit-cursor Git-independent journal, and recoverable turn finalizer until equivalent seams land upstream. - -This recommendation reverses the current runtime-composition boundary in [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) and its [implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). - -## Evidence points - -The accepted baseline is HarnessRouter commit [`5f82db1`](https://github.com/HarnessRouter/harnessrouter/commit/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3). The current source point inspected here is [`8f7868c`](https://github.com/HarnessRouter/harnessrouter/commit/8f7868ccb2c97d1f611acf11e7cad0357a43064e), seven commits later in the [comparison](https://github.com/HarnessRouter/harnessrouter/compare/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3...8f7868ccb2c97d1f611acf11e7cad0357a43064e). Release [`v0.25.6`](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.6) points to `fbcb731`; `8f7868c` is a later commit. HarnessRouter code claims below therefore use pinned `8f7868c` links, with the accepted baseline cited separately. - -The inspected baseline and current source retain the same relevant workspace architecture: a per-session directory, internal checkpoint hydrate/tar routes, root-Git produced-file cursoring, gateway capture-before-ack, and one-file `BACKING.workspace` access. Compare the baseline runner's [`/hydrate`, `/checkpoint`, `_produced_list`, and `_produced_ack`](https://github.com/HarnessRouter/harnessrouter/blob/5f82db1d1f13ea25b8ed0893c38b5b7d2e3e57e3/runner/server.py#L6968-L7176) with the current implementations cited below. - -## Three meanings of workspace - -These terms must not be collapsed: - -| Term | Meaning | Owner | -| --- | --- | --- | -| HarnessRouter product **Workspace** | Security/product-integration boundary containing API keys, configured agents, sessions, and returned files | HarnessRouter control plane | -| UHP/session filesystem workspace | Working directory and file namespace shared by the responses in one session | HarnessRouter session/runner | -| Repositories in the filesystem | Ordinary directory trees that may contain independent `.git` metadata | Snapshot preparation and tools operating in the session | - -HarnessRouter's product documentation defines a Workspace as “the boundary for one product integration” and says API keys, configured agents, sessions, and files live in it; it also tells integrating products to retain their own user, tenant, session, response, and artifact records ([Workspace docs](https://www.harnessrouter.ai/docs/workspace)). That product object is not the runner's `/workspace` directory. - -UHP defines a session as a chain of responses sharing conversational context and a working directory, and a container as the session's file namespace ([UHP architecture](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/architecture.md#L67-L91)). Continuation through `previous_response_id` must use the same session, working directory, files, and configured harness ([UHP sessions](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/sessions.md#L7-L31)). - -HarnessRouter does not model repositories as UHP objects. Its Community Edition README promises native filesystem, shell, and Git workflows with separate session workspaces, and says self-hosted sessions use separate workspaces and operating-system users rather than separate containers ([pinned README](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/README.md#L290-L329)). A repository is content inside the session filesystem, not another HarnessRouter Workspace or session. - -## Stock HarnessRouter behavior - -### Session directory and turn sequence - -The runner derives a workspace from the session identifier. Hosted per-session sandboxes use `WORKSPACE_ROOT` itself; the shared self-hosted runner uses a sanitized per-session subdirectory and optionally a per-session UID write wall ([`WORKSPACE_ROOT`, `_ws`, and isolation invariants](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L78-L171)). - -The gateway's turn sequence is hydrate, refuse execution if an existing checkpoint could not be restored, then launch the runner turn; the runner writes attached input files immediately before the harness starts ([`_resp_execute`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L6816-L6888), [runner `/turn`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L7286-L7335)). Snapshot initialization must gate execution at this boundary. - -### Checkpoint and continuation - -The internal runner `POST /hydrate` spools a gzip tar body to disk, wipes the session directory only after the full body arrives, runs `tar xzf` into the directory, and then calls `_git_ensure`. Empty input creates a fresh workspace ([runner `/hydrate`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L6982-L7062)). `GET /checkpoint` calls `git add -A`, creates an allow-empty commit, and tars the whole directory subject to `CHECKPOINT_EXCLUDE` ([runner `/checkpoint`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L7065-L7110)). `.git` is not excluded; Git history travels in the tarball ([checkpoint exclusions and `_git_ensure`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L452-L631)). - -The gateway streams the runner checkpoint to `sessions/{sid}/workspace.tgz`, stores its SHA only after the blob write succeeds, and restores that blob before a later turn. A transient blob or runner failure does not silently become an empty workspace ([checkpoint/hydrate relays](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L1497-L1615), [`_hydrate` and `_checkpoint`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L2074-L2195)). `HarnessSession` graph state is the durable, replica-independent session record ([`_vertex_upsert` and `_vertex_get`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L2203-L2225)). - -This machinery supports continuation after a snapshot has been initialized, but it is not a public snapshot-import protocol. The runner route consumes HarnessRouter's own checkpoint shape, has no OCI descriptor, size, digest, provenance, or authorization contract, and directly extracts an internally supplied tar. It must not be exposed as an untrusted northbound upload. - -### Produced-file journal - -`_git_ensure` creates or reuses a Git repository at the session root and overwrites the root `.gitignore`. The repository has one HarnessRouter reader, `/produced`; the directory tar, not Git, is durable storage ([`_git_ensure`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L596-L631)). - -`_produced_list` combines `git diff --name-status refs/hr/collected` and `git status --porcelain -uall`. `_produced_ack` stages and commits the current root tree, then advances `refs/hr/collected`. `_produced_keep` deliberately drops deleted paths and runtime noise ([collection cursor and produced routes](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L7111-L7205)). The gateway fetches each listed regular file through `/file`, stores it as an artifact, and calls `/produced/ack` only after capture; if acknowledgement fails, the cursor remains behind and the next collection can retry ([`_collect_produced`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L6659-L6734)). Capture-before-ack is worth preserving. - -`BACKING.workspace` is not a snapshot hook. Its protocol reads or writes one path. `RunnerWorkspaceFiles` proxies a live runner file, while `CheckpointWorkspaceFiles` rewrites one member in a stored checkpoint tar ([`WorkspaceFiles`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/backing.py#L76-L99), [implementations](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/backing.py#L389-L535)). It should remain the application/files seam, not be stretched into acquisition. - -## Stock capability verdict - -| Required property | Stock result | Why | -| --- | --- | --- | -| Public import of one prepared large snapshot | **No** | UHP file input is inline bytes or an uploaded file; public session/file endpoints expose artifacts and archives, not session-root initialization ([UHP Files](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/files.md), [official session/file endpoints](https://www.harnessrouter.ai/docs/sessions-and-files)). `/hydrate` is internal. | -| Preserve several nested `.git` histories as bytes | **Conditionally yes after unsupported injection** | The full-directory tar includes `.git`, so nested repositories survive checkpoint/hydrate. A `.git` at workspace root is commandeered by `_git_ensure`, which writes `.gitignore`, stages, commits, and adds `refs/hr/collected`. | -| Exact changes across several repositories | **No** | Root Git sees an embedded repository as a repository boundary/gitlink, not recursively tracked files, and stock deliberately discards deletions. | -| Checkpoint and continuation | **Yes after initialization, with qualifications** | The tar and UHP session machinery preserve the session. Stock re-archives the entire snapshot each checkpoint and excludes dependency/scratch names such as `node_modules`, `.venv`, and `venv`, so arbitrary prepared content is not an exact round trip without snapshot-specific policy. | -| Reusable immutable snapshot cache | **No** | The warm probe verifies only that one live runner still holds one session's exact `ws_sha`; durable blobs are session-keyed. There is no descriptor-keyed cross-session cache ([`_ws_blob`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L1497-L1504), [`_hydrate`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L2074-L2158)). | - -### Why root Git cannot evaluate multiple repositories - -Git defines a submodule as one repository embedded inside another, with independent history; the superproject records only a gitlink containing the expected commit ([`gitsubmodules`](https://git-scm.com/docs/gitsubmodules/2.52.0)). Git also recognizes an old-form submodule whose working directory contains an embedded `.git` directory. When `git add` encounters an embedded repository without `git submodule add`, it warns because the outer index records an embedded repository rather than ordinary descendant files ([`git-add`](https://git-scm.com/docs/git-add/2.54.0#Documentation/git-add.txt---no-warn-embedded-repo)). - -The superproject's short status reports only that a nested repository's HEAD changed (`M`), it has modified content (`m`), or it has untracked content (`?`); modified and untracked files inside the nested repository cannot be added through `git add` in the superproject ([`git-status`](https://git-scm.com/docs/git-status/2.53.0#_short_format)). `git diff` defaults to ignoring submodules; even `--ignore-submodules=none` reports whether a submodule is dirty or at another HEAD, not a canonical workspace-wide per-file add/modify/delete manifest ([`git-diff`](https://git-scm.com/docs/git-diff/2.55.0#Documentation/git-diff.txt---ignore-submodulesnoneuntrackeddirtyall)). - -A prepared workspace may contain multiple self-contained `.git` histories as data, but HarnessRouter's stock root-Git cursor cannot be the exact change-reporting engine. If a source repository occupies the workspace root, HarnessRouter mutates that repository to operate its cursor. - -## Recommended snapshot artifact - -The preparation plane should publish one OCI artifact and return one direct descriptor: - -```json -{ - "media_type": "application/vnd.oci.image.manifest.v1+json", - "digest": "sha256:<64 lowercase hex>", - "size": 123456 -} -``` - -An OCI descriptor's required `mediaType`, `digest`, and `size` provide type, content identity, and a pre-processing length check. Consumers should verify size and digest before expensive processing, and OCI requires SHA-256 verification support ([OCI Image Specification 1.1.1 descriptor](https://github.com/opencontainers/image-spec/blob/v1.1.1/descriptor.md)). The Distribution Specification permits retrieving a manifest by digest and says clients should verify that a digest-addressed response matches the requested digest ([OCI Distribution Specification 1.1.1](https://github.com/opencontainers/distribution-spec/blob/v1.1.1/spec.md#pulling-manifests)). The execution request rejects tags and image indexes so platform or tag selection cannot change the admitted bytes. - -Use an OCI image manifest as an artifact container with: - -- custom `artifactType` `application/vnd.allagents.workspace-snapshot.v1`; -- a custom config media type containing the snapshot-format version, final workspace-manifest descriptor, default working directory, preparation implementation/policy identity, and ordered source provenance; -- ordered layer descriptors that the AllAgents snapshot format defines as standard OCI filesystem changesets applied to an empty directory. - -OCI permits non-container content to be packaged with an image manifest and permits an unknown/custom config media type to represent arbitrary artifact metadata ([artifact guidance](https://github.com/opencontainers/image-spec/blob/v1.1.1/artifacts-guidance.md), [manifest config rules](https://github.com/opencontainers/image-spec/blob/v1.1.1/manifest.md#image-manifest-property-descriptions)). OCI layers are changesets, not tarballs to concatenate or blindly extract: consumers must apply ordered additions, modifications, whiteouts, opaque-directory behavior, and replacement semantics ([OCI layer specification](https://github.com/opencontainers/image-spec/blob/v1.1.1/layer.md#applying-changesets)). - -The digest identifies the serialized manifest and everything it references; it is not human-auditable source provenance or a canonical final-tree identity. Put the composition record in the digest-covered config, including each repository's normalized identity, requested selector, resolved commit, history completeness, destination, preparation implementation, and the final tree-manifest digest. Keep the exact manifest descriptor as execution identity and expose the final tree digest separately as baseline identity. - -Do not rely on an OCI `subject` relationship alone for provenance or signatures. The OCI manifest specification calls `subject` a weak association, and referrers may be added independently after the subject artifact exists ([OCI manifest `subject`](https://github.com/opencontainers/image-spec/blob/v1.1.1/manifest.md#image-manifest-property-descriptions)). Referrer signatures and attestations are useful additional evidence, but source provenance required to interpret the workspace must be digest-covered by the admitted manifest. - -To preserve multiple Git histories, the preparation plane packages self-contained `.git` directories as snapshot content after removing credentials, unsafe alternates, transient locks, and acquisition-only state. HarnessRouter neither interprets nor rewrites those repositories. A Git-free repository tree is equally valid. - -## Preparation plane versus execution plane - -```mermaid -flowchart LR - C[Product or benchmark adapter] -->|Git/OCI/multi-repo request| P[Workspace preparation API] - P -->|resolve, compose, verify| S[Immutable staging tree] - S -->|publish config + layers + manifest| R[OCI registry] - P -->|exact manifest descriptor| C - C -->|UHP task + descriptor| G[AllAgents Gateway] - G -->|authorize and initialize| I[Generic snapshot backend] - I -->|pull/cache by digest| R - I -->|private reflink/copy| W[Session workspace] - G --> H[Existing HarnessRouter turn lifecycle] - H -->|manifest changes + artifacts| C -``` - -Preparation should be asynchronous when expensive: submit preparation, poll or receive completion, then invoke UHP with the returned descriptor. HarnessRouter does not call back into the product-specific preparation API during a task. This keeps preparation availability and Git credentials out of the execution critical path after publication. - -| Concern | Compose Git/OCI inside HarnessRouter | Prebuild one snapshot, then execute | -| --- | --- | --- | -| Northbound request | Product-specific source list | One immutable descriptor | -| Runtime credentials and egress | Git and registry credentials, DNS, ref resolution, redirects, helpers, and source policy | Snapshot-registry read only | -| HarnessRouter changes | Source schema, resolvers, Git/OCI clients, per-component caches, destination rules, mount/copy lifecycle, provenance, expiry/reconciliation | Generic initializer, descriptor binding, journal, optional cwd | -| Cache identity | Per-component keys plus composition state | One final manifest key; OCI blob/layer reuse remains available underneath | -| First-task latency | Acquisition and composition happen on task start | Moved to preparation; task start is pull/cache/materialize | -| Failure surface | Partial source resolution and attachment must reconcile with session state | Preparation fails before UHP; execution sees only published immutable artifacts | -| Upstreamability | Low: Git/OCI/source policy is AllAgents product logic | High: immutable workspace initialization is execution-runtime plumbing | -| Cost | No separate preparation API, but a large permanent fork | Separate contract; snapshots need retention and garbage collection | - -A one-snapshot execution cache cannot independently swap one component at runtime. That reuse stays in preparation and the registry: a builder may reuse Git mirrors, resolved trees, and unchanged OCI blobs/layers while publishing a new final manifest. The executor treats the result as one atomic filesystem. - -## Immutable base plus private writable session - -“Immutable snapshot” describes the reusable baseline, not the agent's live workspace. The harness needs a private writable view. - -| Mechanism | Assessment | -| --- | --- | -| Per-file reflink tree clone from an immutable unpacked cache | **Recommended first choice.** Linux `FICLONE` shares physical data copy-on-write, requires source and destination on the same filesystem, and keeps later writes private ([`ioctl_ficlone(2)`](https://man7.org/linux/man-pages/man2/ioctl_ficlonerange.2.html)). Walk without following links, create new directory entries/inodes, reflink regular files, and never hardlink mutable files. | -| Full private copy | **Required fallback.** Portable and leaves a normal directory, at startup I/O and space cost. | -| OverlayFS with immutable lower and per-session upper/work directories | **Defer.** OverlayFS supports a non-writable lower, writable upper, whiteouts, and copy-up ([Linux v6.17 OverlayFS documentation](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/Documentation/filesystems/overlayfs.rst?h=v6.17)). It expands checkpoint, hydrate, archive, deletion, mount, and crash-cleanup work. | -| Read-only bind mount of the whole snapshot | **Reject for editable sessions.** Mounts require privilege and namespace care; HarnessRouter writes input, instruction, and `.harness` state into the workspace ([`mount(2)`](https://man7.org/linux/man-pages/man2/mount.2.html), [`mount_namespaces(7)`](https://man7.org/linux/man-pages/man7/mount_namespaces.7.html)). | -| Shared writable tree, hardlinks, or symlink to cache | **Reject.** A session could mutate cache or sibling state. | - -A private reflink/copy leaves checkpointing and `BACKING.workspace` operating on an ordinary directory. It is the smallest correct boundary. It means stock full-tar checkpoints re-archive the baseline on cold continuation; measure that cost before adopting OverlayFS or base-plus-delta checkpoints. The runner already streams large checkpoints through disk rather than buffering them in memory, so this is primarily I/O/storage cost, not a reason to redesign UHP. - -The unpacked cache key is the verified manifest descriptor plus one materializer/schema revision. Publication is immutable and complete before readers claim it. The cache never contains a live session's writable tree. - -## Minimal northbound extension - -UHP permits additional request/response metadata and recommends vendor prefixes; extensions must not redefine specified fields or add a required field ([UHP schema extension points](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/schema.md#L47-L58)). HarnessRouter models create-response metadata as an open dictionary, and its OpenAPI schema permits additional metadata properties ([`CreateResponseBody`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L7524-L7539), [pinned OpenAPI schema](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/schema/uhp-2026-09-12.openapi.yaml#L1041-L1082)). - -Use a first-turn-only vendor extension: - -```json -{ - "input": "Make the requested change.", - "metadata": { - "allagents_workspace_snapshot": { - "version": 1, - "descriptor": { - "media_type": "application/vnd.oci.image.manifest.v1+json", - "digest": "sha256:...", - "size": 123456 - } - } - } -} -``` - -The registry, repository, credentials, redirects, and trust policy are server configuration. The caller cannot select them. The digest-covered snapshot config supplies and binds the default working directory and provenance. Continuations omit this metadata; repeating or replacing the descriptor on a continuation fails before hydrate or materialization. - -Advertise support as an additional vendor capability, treated as false when absent, consistent with UHP discovery's extensible named-boolean capability model ([UHP capability discovery](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/lifecycle.md#L29-L83)). Do not claim the extension is part of UHP 2026-09-12. - -## Minimal upstream-neutral hook and durable state - -The upstream proposal should not mention Git, repository arrays, source credentials, composition, catalog entries, or AllAgents provenance fields. It needs one optional immutable-workspace initializer and one optional non-Git journal mode. - -Conceptual interface: - -```text -WorkspaceInitializer.initialize( - principal, - session_id, - immutable_descriptor, - empty_session_root -) -> InitializedWorkspace - -InitializedWorkspace = { - verified_descriptor, - tree_digest, - working_directory, - initializer_schema, - journal_mode -} -``` - -Required semantics: - -- descriptor immutable and verified before content is consumed; -- initializer configured by the operator, never selected by caller; -- destination is an empty private session root; -- success all-or-nothing before input files or harness start; -- retries for the same session/descriptor idempotent; -- failure leaves no runnable partial tree; -- returned working directory relative, confined, and verified as a directory; -- hook returns no credentials, registry URL, host/cache path, mount identity, or product source plan. - -The durable `HarnessSession` extension contains only: - -```text -workspace_snapshot = { - state: pending | ready | failed, - descriptor: { media_type, digest, size }, - verified_tree_digest, - working_directory, - initializer_schema, - journal_cursor_blob_digest -} -``` - -Keep existing `ws_sha`, checkpoint blob, response/artifact records, and control leases separate. The durable binding also needs a private immutable repository/catalog/signature-policy selector so restart can reauthorize and refetch the original tuple; the public descriptor alone is insufficient. Do not persist cache paths, mount IDs, inode numbers, attachment flags, registry credentials, or preparation-service records. - -For exact reporting, the gateway-owned cursor blob is the sole durable authority. The gateway supplies its explicit base cursor to the runner's manifest journal; the runner returns the next cursor and an idempotent collection token without owning durable cursor state. A recoverable turn commit captures artifacts and an exact checkpoint, verifies streamed file hashes and checkpoint state against the next manifest, acknowledges the token, then atomically publishes gateway cursor/checkpoint/artifact pointers and terminal visibility. This preserves capture-before-ack without a hidden runner/gateway cursor split. - -## Smallest required HarnessRouter changes - -### Gateway - -1. Parse and bind the extension in the existing create-response path. Validate first-turn-only use after session resolution but before hydration; bind the exact descriptor plus private repository/catalog authorization subject to `HarnessSession`. -2. Gate execution around `_resp_execute`/`_hydrate`. A new snapshot session performs empty hydrate/wipe, initialization, input/control preparation, captures a canonical initialization delta from builder snapshot to ready tree, stores the ready cursor, and transitions `pending -> ready` before `/turn`. Continuation restores checkpoint and cursor. -3. Return the verified descriptor, snapshot and ready manifest digests, provenance digest, initialization-change artifact, working directory, retention deadline, and expiry after ready; replay uses stored values. -4. Extend `_collect_produced`, not public Files endpoints. Supply the ready/acknowledged base cursor to the runner, hash changed files while capturing, verify the exact checkpoint, then acknowledge and publish terminal state through a recoverable turn commit. -5. Release initializer/cache references on session deletion and reconcile every preparing/durable/acknowledged/committed/aborted turn state idempotently. - -### Runner - -1. Add a trusted internal initialization route or equivalent hydrate mode that invokes the configured initializer. Do not reuse raw checkpoint `/hydrate` as a public OCI importer. -2. Do not call `_git_ensure` for snapshot-backed sessions. `/hydrate`, `/checkpoint`, `/produced`, and `/produced/ack` need a session journal/checkpoint mode. -3. Use an explicit-cursor manifest journal behind `/produced` and `/produced/ack`; keep root Git for stock sessions. -4. Allow a validated relative working directory beneath `_ws(identifier)` while retaining the session UID ([runner `/turn`](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L7286-L7335)). -5. Make snapshot-mode checkpoint exclusions exact and versioned. Stock dependency exclusions cannot silently remove declared files. - -### What remains stock - -Keep UHP versioning/authentication, idempotency, session and response identity, provider routing, harness adapters and supervision, SSE translation, cancellation, graph/blob backing, warm `ws_sha` probe, checkpoint relays, file/artifact endpoints, `BACKING.workspace`, capture-before-ack ordering, explicit session deletion, and ordinary-session root-Git behavior. - -With a normal private reflink/copy, checkpoint/hydrate can remain full-directory tar operations after two snapshot-mode adjustments: skip root Git and use an exact compatible exclusion policy. OverlayFS would expand changes to mount-aware hydrate, checkpoint, archive, file access, deletion, reaping, and crash recovery; it is not the minimum. - -## Two repositories versus one - -Two repositories are the better boundary. - -**Why two wins:** - -- Preparation owns product policy, source credentials, Git behavior, OCI composition, provenance, asynchronous jobs, and snapshot publication. None is HarnessRouter execution logic. -- It can scale, release, and cache independently from latency-sensitive execution. -- The gateway fork becomes narrow enough to propose upstream without asking HarnessRouter to adopt AllAgents source semantics. -- Migration to stock becomes possible only after upstream exposes initializer, explicit-cursor journal, and recoverable finalization seams; `allagents-gateway` can then become a thin distribution or disappear, while preparation remains unchanged. -- The OCI descriptor is a stable cross-repository contract and release boundary. - -**Costs:** two components require contract versioning, availability/retention ownership, integration tests, and a compatibility matrix. Preparation must publish fully before returning a descriptor; execution must fail clearly if a retained digest disappears. These obligations are smaller and better isolated than permanent Git/OCI product-policy code in a HarnessRouter fork. - -A monorepo with two binaries would reduce atomic-edit friction, but would keep source-product lifecycle coupled to the upstream-derived repository and make returning to stock a source-tree surgery. Given the explicit upstream-migration goal, separate repositories are preferable. - -## Upstream proposal - -Propose three generic changes to HarnessRouter: - -1. **Optional immutable workspace initialization hook** - - registered by operator configuration; - - invoked once after a fresh session root is established and before inputs/harness start; - - receives an immutable descriptor and returns verified identity, relative cwd, and journal mode; - - persists a minimal pending/ready binding on `HarnessSession`; - - advertises one optional capability; - - leaves requests without the extension on the stock path. - -2. **Pluggable explicit-cursor journal behind `/produced` and `/produced/ack`** - - existing root Git remains default; - - gateway supplies the authoritative base cursor; - - a generic manifest journal supports non-Git and multi-repository workspaces; - - capture-before-ack remains in the gateway; - - hook has no AllAgents artifact schema or source model. - -3. **Recoverable terminal-finalization seam** - - orders changed-file artifacts, change artifact, exact checkpoint, next cursor, journal ACK, and terminal response; - - exposes terminal state only after durable checkpoint/cursor evidence and ACK; - - resumes crashes without rerunning the harness. - -Focused tests should prove initialization before inputs/harness, idempotent duplicate first requests, continuation with the same binding, descriptor-replacement rejection, explicit cursor handoff, restore failure that does not run or overwrite a checkpoint, exact add/modify/delete behavior excluding `.git`, checkpoint/cursor durability before terminal visibility, and unchanged stock behavior. UHP itself need not change because vendor-prefixed metadata is already an extension point. The proposal is tracked in [HarnessRouter issue #304](https://github.com/HarnessRouter/harnessrouter/issues/304). - -## Migration from fork to stock - -1. Implement preparation and the versioned OCI snapshot format in `allagents-workspace-builder`. -2. In the existing fork, isolate execution changes behind `WorkspaceInitializer`, `WorkspaceJournal`, and `WorkspaceTurnFinalizer`; keep Git/OCI composition outside. -3. Submit generic initializer, explicit-cursor journal, and recoverable-finalization seams upstream with stock defaults and no AllAgents source model. -4. While review is pending, ship the same interfaces in `allagents-gateway`; keep the OCI backend and metadata adapter separate from copied HarnessRouter logic. -5. When upstream contains equivalent seams, rebase onto that release and delete only superseded fork implementations rather than preserving aliases. -6. Make `allagents-gateway` consume stock HarnessRouter plus backend packaging once every required seam is upstream. If upstream supports external backend loading, stop maintaining a source fork. -7. Retain cross-version tests for stock UHP, first-turn initialization, exact changes, finalization, continuation, and descriptor-keyed cache reuse before dropping the fork. - -## Rejected alternatives - -| Alternative | Reason | -| --- | --- | -| Keep Git/OCI/multi-repository composition inside AllAgents Gateway | Couples product source policy and credentials to HarnessRouter lifecycle and produces the largest, least upstreamable patch. | -| Upload a prepared tar through stock `/hydrate` | Internal checkpoint route without OCI identity/provenance/import policy; still invokes root Git and has no cross-session cache or exact nested-repository changes. | -| Make the agent clone repositories | Acquisition occurs after harness start, exposes credentials/network policy to agent code, and cannot establish a verified pre-turn baseline. | -| Use one repository at workspace root and nested repos below it | HarnessRouter mutates the root repository, while Git reports nested repositories only coarsely and stock omits deletions. | -| Use a shared writable unpacked snapshot or hardlinks | A session can mutate cache or sibling state. | -| Adopt OverlayFS immediately | Expands every filesystem lifecycle seam before a measured need. Start with reflink/private copy. | -| Keep preparation and execution in one source repository | Easier atomic edits, but undermines independent ownership and migration from a fork to stock. | -| Put required provenance only in OCI referrers | `subject` is a weak association and referrers do not contribute to admitted snapshot manifest digest. | -| Accept tags or indexes at execution | Selection can change independently of request; execution receives one direct manifest descriptor. | - -## Conclusion - -Stock HarnessRouter can continue a private tree once it is inside its checkpoint lifecycle, and its tar format generally carries nested `.git` bytes. It cannot, as a supported stock product, import a prepared immutable large snapshot with untouched multiple repository histories, exact workspace-wide changes including deletions, reusable descriptor-keyed cache, and a public immutable provenance contract. - -The smallest robust change is a generic first-turn snapshot initializer, an explicit-cursor Git-independent journal, and recoverable terminal finalization, feeding an ordinary private writable session directory while retaining the rest of HarnessRouter. Put every mutable source and composition concern in a separate preparation repository, publish one OCI descriptor, and use that descriptor as the execution-plane source identity. - -## Primary sources - -### HarnessRouter and UHP - -- [HarnessRouter Workspace documentation](https://www.harnessrouter.ai/docs/workspace) -- [HarnessRouter sessions and files documentation](https://www.harnessrouter.ai/docs/sessions-and-files) -- [HarnessRouter run-task boundary](https://www.harnessrouter.ai/docs/run-a-task) -- [HarnessRouter release v0.25.6](https://github.com/HarnessRouter/harnessrouter/releases/tag/v0.25.6) -- [HarnessRouter commit `8f7868c`](https://github.com/HarnessRouter/harnessrouter/commit/8f7868ccb2c97d1f611acf11e7cad0357a43064e) -- [Pinned runner source](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py) -- [Pinned gateway source](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py) -- [Pinned backing abstractions](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/backing.py) -- [UHP 2026-09-12 architecture](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/architecture.md) -- [UHP 2026-09-12 lifecycle](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/lifecycle.md) -- [UHP 2026-09-12 sessions](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/sessions.md) -- [UHP 2026-09-12 files](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/files.md) -- [UHP 2026-09-12 schema extension rules](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/schema.md) - -### OCI - -- [OCI Image Specification 1.1.1 descriptors](https://github.com/opencontainers/image-spec/blob/v1.1.1/descriptor.md) -- [OCI Image Specification 1.1.1 manifests](https://github.com/opencontainers/image-spec/blob/v1.1.1/manifest.md) -- [OCI Image Specification 1.1.1 layers](https://github.com/opencontainers/image-spec/blob/v1.1.1/layer.md) -- [OCI Image Specification 1.1.1 artifact guidance](https://github.com/opencontainers/image-spec/blob/v1.1.1/artifacts-guidance.md) -- [OCI Distribution Specification 1.1.1](https://github.com/opencontainers/distribution-spec/blob/v1.1.1/spec.md) - -### Git - -- [Git submodule model 2.52.0](https://git-scm.com/docs/gitsubmodules/2.52.0) -- [Git add embedded-repository behavior 2.54.0](https://git-scm.com/docs/git-add/2.54.0#Documentation/git-add.txt---no-warn-embedded-repo) -- [Git status submodule behavior 2.53.0](https://git-scm.com/docs/git-status/2.53.0#_short_format) -- [Git diff submodule behavior 2.55.0](https://git-scm.com/docs/git-diff/2.55.0#Documentation/git-diff.txt---ignore-submodulesnoneuntrackeddirtyall) - -### Linux filesystem behavior - -- [Linux `FICLONE`/`FICLONERANGE`](https://man7.org/linux/man-pages/man2/ioctl_ficlonerange.2.html) -- [Linux v6.17 OverlayFS documentation](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/Documentation/filesystems/overlayfs.rst?h=v6.17) -- [Linux `mount(2)` bind/read-only behavior](https://man7.org/linux/man-pages/man2/mount.2.html) -- [Linux mount namespaces](https://man7.org/linux/man-pages/man7/mount_namespaces.7.html) -- [Linux recursive mount attributes](https://man7.org/linux/man-pages/man2/mount_setattr.2.html) diff --git a/docs/research/e2b-execution-gateway-patterns.md b/docs/research/e2b-execution-gateway-patterns.md index 6f18306a..85b420c5 100644 --- a/docs/research/e2b-execution-gateway-patterns.md +++ b/docs/research/e2b-execution-gateway-patterns.md @@ -126,28 +126,37 @@ The runtime and dashboard repositories use Apache-2.0 ([runtime license](https:/ ### Verdict -**Reject E2B as a replacement for the planned UHP/HarnessRouter gateway. Trial it later only as a stronger sandbox runtime beneath the runner if hostile-code isolation becomes a product requirement.** - -The current AllAgents decision is accepted but not implemented: [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md) selects UHP `2026-09-12` through a pinned HarnessRouter CE fork, and the [implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md) assigns the wire protocol, caller authentication, normalized streaming, cancellation, idempotency, conversation continuity, harness execution, usage, and artifacts to HarnessRouter. A separate AllAgents Workspace Builder owns Git/OCI preparation and publishes one immutable snapshot before execution. - -E2B does not implement that contract. It creates an isolated machine and exposes low-level process, filesystem, network, and lifecycle APIs. Its official Codex and Pi integrations leave command construction, harness credentials, event parsing, continuation, and result extraction in caller code. Replacing HarnessRouter with E2B would therefore recreate the custom gateway, session, event-normalization, harness-adapter, and artifact layers that ADR 0002 rejected. - -### Does E2B do the same thing? - -No. The systems overlap at the execution-workspace layer but own different abstractions. - -| Concern | Planned AllAgents gateway | E2B | +**Do not add E2B to the default gateway backend.** +`allagentsdev/allagents-gateway` can use local disposable containers or +processes, a policy-bound immutable source cache, direct read-only mounts, +private CoW/full-copy writable views, and required operator-authorized immutable +runtime profiles. E2B remains an optional worker backend only if a later +requirement needs its microVM boundary or managed sandbox lifecycle. + +This conclusion follows the boundary in +[One-shot coding-agent gateway boundary](./one-shot-coding-agent-gateway-boundary.md) +and [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). + +E2B still does not own the gateway abstraction. It creates a sandbox and exposes +process, filesystem, network, and lifecycle APIs. The AllAgents gateway owns the +normative `AgentRun v1` contract, source materialization, runtime-profile +resolution, agent launch, phase-separated isolation, lifecycle fencing, bounded +raw evidence, authenticated artifact service, and cleanup. Promptfoo is its +first caller and solely owns evaluation and grading. + +| Concern | One-shot coding-agent gateway | E2B | | --- | --- | --- | -| Northbound contract | UHP request, ordered events, cancellation, idempotency, continuation, files, usage, artifacts, and normalized errors | Sandbox REST API plus process/filesystem APIs; no agent-neutral request/event/result protocol | -| Harness execution | HarnessRouter selects and runs configured Codex or Pi targets | Caller starts an agent-specific command inside a sandbox and interprets its output | -| Session identity | `previous_response_id` binds conversation, writable workspace, harness target, auth binding, and provenance | Sandbox ID binds machine state; agent thread/session identity remains harness- and caller-specific | -| Workspace acquisition | Separate builder resolves sources and publishes one direct OCI snapshot; gateway authorizes, verifies, privately materializes, and returns immutable manifest/provenance identities before agent start | Caller uploads files, clones Git, or starts from a template/snapshot; no agent-run source-provenance contract | -| Isolation | Private per-session UID/workspace; explicitly not a hostile-code sandbox | One Firecracker microVM and guest kernel per sandbox | -| Credentials | Separate caller, source, and provider trust domains; native OAuth owner-trust mode or explicit brokered proxy | Core sandbox auth plus caller-supplied agent/Git credentials; managed egress secrets and workload identity are not fully present in standard Embed | -| Results | UHP output, usage, artifacts, produced-file collection, and identical provenance across stream/retrieval/replay paths | Guest files, command streams, VM snapshots, and templates; the caller defines an agent result or artifact manifest | -| Deployment | Planned pinned private HarnessRouter image with durable sessions and a narrow AllAgents hook | Multi-service KVM stack with API, proxies, orchestrator, guest daemon, three datastores, and template/snapshot storage | - -E2B could occupy the runner's future sandbox-runtime slot. HarnessRouter and the separate AllAgents Workspace Builder would still remain above and before it respectively. +| Caller policy | Promptfoo is the first caller and owns evaluation policy | No experiment matrix or assertion owner | +| AgentRun | One public request, fresh internal trial, one agent attempt, then verified deletion | Sandbox lifecycle and guest APIs | +| Sources | Reauthorize exact cached generation; direct read-only mount or private CoW/full copy | Caller uploads or acquires content | +| Runtime | Required authorized `runtime_profile_id`; provenance pins profile/image, tool/service implementation, and applicable service-image digests | Caller selects/builds template | +| Checks | Agent process/cgroup and netns gone, descendants absent, credentials removed, then optional bundle bytes materialized and structured commands run in the retained runtime/final workspace | Caller-defined commands | +| Network | Phase-separated default-drop namespaces; named agent/post-run policy | Caller-defined sandbox networking | +| Evidence | Bounded `CapturedText` plus full request-order complete/partial `PostRunEvidence` | Caller-defined files and command output | +| Artifacts | Expiring tenant/run-authorized references; provider verifies streamed size/digest | Caller-defined download/storage | +| Lifecycle | Seal evidence, clean trial and release leases, then publish `completed`, `cancelled`, or `infrastructure_error`; cleanup failure retains partial evidence | Caller-defined | +| Judgment | Promptfoo code or LLM graders | Caller-defined | +| Adoption | Default local worker backend | Optional backend after a concrete isolation need | ### Is it completely self-hosted? @@ -161,26 +170,29 @@ The answer depends on the operating standard: The [enterprise page](https://e2b.dev/enterprise) markets Embed as an available self-hosting pattern. The [Embed source guide](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/README.md) sets the narrower operational boundary. For architecture decisions, use the source guide's single-node/evaluation limit. -### No E2B-derived changes for version one - -Version one should adopt none of E2B's runtime patterns. The accepted gateway already has a coherent private owner-trust scope, and E2B addresses requirements that version one explicitly excludes: hostile-code isolation, public multi-tenancy, VM suspension and forking, multi-node placement, and microVM startup optimization. - -Do not add an E2B dependency, a sandbox-provider interface, a guest daemon, egress-policy machinery, runtime telemetry infrastructure, or any other future-runtime seam to version one. The runner boundary is already a sufficient future integration point. Building an abstraction before a second runtime and a measured requirement exist would increase the initial implementation and verification burden without satisfying an acceptance criterion. - -Existing requirements for private runner routes, credential containment, crash-consistent materialization, immutable source identity, release pinning, and provenance remain necessary on their own merits. They are not E2B adoption. - -If a reconsideration trigger is reached later, E2B provides a useful checklist for that separate design: place stronger isolation beneath the runner; separate public and private control routes; bind egress and ingress policy to the execution identity; keep environment identity distinct from source provenance; keep long-lived credentials outside the guest; reclaim orphaned runtime resources after crashes; correlate runtime telemetry with UHP identifiers without creating another agent protocol; and pin the sandbox SDK, API, guest daemon, kernel, and runtime as one tested compatibility set. - -### Patterns to defer or reject - -- **Defer Firecracker, memory snapshots, forking, lazy restore, COW root filesystems, placement, and multi-node scheduling.** They solve hostile multi-tenancy, recovery, or startup-cost problems that v1 does not claim. Adopt them only after a requirement or measurement justifies their operational weight. -- **Reject E2B's API as the AllAgents northbound contract.** It would discard UHP conformance and make callers own harness-specific behavior. -- **Reject caller-side repository acquisition and inline long-lived credentials.** E2B's convenience examples conflict with the server-authoritative source catalog, hermetic materializer, and credential-containment requirements. -- **Reject open internet by default for a future sandboxed mode.** Outbound access should be an explicit target policy. -- **Reject live VM snapshots as source provenance.** They are useful recovery artifacts, not reproducible evidence of which repositories and commits an agent received. - -### Reconsideration conditions - -Evaluate E2B as a runner backend when AllAgents must execute mutually untrusted tenant code, support public multi-tenancy, preserve in-flight processes across suspension, fork live workspaces, or meet measured sandbox-start targets that process/UID isolation cannot satisfy. The trial must keep UHP and AllAgents provenance above E2B, pin an SDK/runtime compatibility pair, prove private-route containment and default-deny egress, and exercise crash recovery on the exact self-hosted deployment shape. - -E2B is not the ADR's “second independent UHP implementation” reconsideration trigger because it does not implement UHP. It becomes relevant when the isolation requirement changes, not because it duplicates the current gateway. +### Adopt the cache pattern without E2B + +Do not add an E2B dependency, generic sandbox-provider interface, guest daemon, +memory snapshots, forking, or multi-node placement. Do adopt the evidence-backed +mechanism directly: content-address exact immutable source generations, share +read-only layers, and give writable trials private CoW overlays/clones with a +full-copy fallback. E2B's template build cache and overlay-backed sandbox starts +are precedent for that mechanism, not a reason to adopt its platform. +The local disposable gateway worker remains the default backend. + +If mutually untrusted tenant code or measured sandbox-start requirements later +outgrow that boundary, trial E2B behind the normative +[`AgentRun v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#public-contract) +contract. It must preserve required authorized runtime profiles and their +profile/image, tool/service implementation, and applicable service-image +digests; fresh workspaces and declared source modes; verified agent +process/netns teardown before hidden-bundle +materialization, phase-separated default-drop networking, bounded +complete/partial evidence, tenant/run-bound artifact access, and cleanup/lease +release before terminal-result publication. E2B remains only an execution +backend, never another caller/result protocol or grading layer. + +Useful patterns to carry forward without adopting E2B are narrow guest APIs, +explicit lifecycle states, immutable environment identity, credential +containment, and orphan cleanup. Live VM snapshots remain recovery artifacts, +not proof that a coding task passed. diff --git a/docs/research/harbor-repository-materialization.md b/docs/research/harbor-repository-materialization.md index c579cc42..3fce3881 100644 --- a/docs/research/harbor-repository-materialization.md +++ b/docs/research/harbor-repository-materialization.md @@ -1,14 +1,25 @@ # Harbor repository materialization lessons -## Decision - -Use Harbor's content-addressed package cache, sparse Git reads, staged publication, and prebuilt-environment model in **AllAgents Workspace Builder**. Publish one complete immutable workspace snapshot before invoking AllAgents Gateway. - -Harbor does not expose a first-class general-purpose “repositories in a workspace” layer. It first downloads a Harbor task package. The task then defines an execution environment with a Dockerfile, Compose file, or prebuilt image. Acquisition of the repository the agent edits may be baked into an image, cloned by a Dockerfile, copied as task content, or prepared by the task author. - -AllAgents keeps source selection and provenance explicit in the builder contract. The gateway receives only a direct digest-pinned OCI workspace-snapshot descriptor. Git URLs, mutable refs, source credentials, destinations, and custom preparation behavior do not cross into HarnessRouter. - -This replaces the earlier recommendation to invoke a materializer inside HarnessRouter. The primary-source observations below remain valid; the current boundary is defined by [snapshot-boundary research](./allagents-gateway-snapshot-boundary.md) and [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md). +## Current conclusion + +Treat this note as future-adapter evidence only. Harbor, Terminal-Bench, and +SWE-bench integration are not part of the current v1 contract or plan. Harbor +can still inform a later design because it packages an instruction, +environment, workdir, test script, and reward artifact around one disposable +task, but it must not mediate ordinary Promptfoo Git/OCI runs or become a core +source kind. + +Any future integration requires its own ADR and closed adapter request/result +mapping. That decision must define provenance, bounded complete/partial +evidence, artifact authorization/expiry, cancellation, cleanup-gated +publication, and how Promptfoo consumes raw observations. It must not reuse or +extend the current v1 request/result schemas without an explicit future contract +decision. Harbor may own its sandbox and checks, but the gateway must +not convert Harbor scores into pass/fail or reward. + +The current boundary is defined by +[One-shot coding-agent gateway boundary](./one-shot-coding-agent-gateway-boundary.md) +and [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). ## What Harbor fetches @@ -65,7 +76,11 @@ an upstream `mswebench/...:pr-...` base image that already contains the reposito `/home/{repo_name}`. Its Dockerfile creates `/workspace/{repo_name}` as a symlink and sets that as `WORKDIR`; Harbor itself never clones that application repository. -## Lessons for the AllAgents workspace materializer +## Superseded materializer lessons (historical) + +The sections below preserve conclusions from the rejected builder/snapshot +architecture. They are evidence about Harbor's implementation, not current +recommendations for the evaluation-only runner. ### Adopt @@ -133,21 +148,32 @@ through to the other after admission. correct files while obscuring which repositories, commits, generator, and setup produced them. -## Recommended boundary - -AllAgents Workspace Builder runs before UHP execution: - -1. The builder validates the versioned source plan, destination ownership, configured repository/snapshot identities, and principal authorization before source network access. -2. It resolves phase-scoped source credentials without exposing them to the eventual coding agent or gateway. -3. It populates a private staging tree from exact Git commits and digest-pinned OCI inputs. -4. It verifies commits, history completeness, paths, limits, links, content, the canonical workspace manifest, and digest-covered provenance. -5. It stops acquisition processes and removes credentials, helpers, unsafe Git state, and mounts. -6. It publishes config, provenance, layers, and blobs completely before publishing and returning one direct OCI image-manifest descriptor. -7. AllAgents Gateway authorizes that descriptor, independently verifies/materializes it into a private session tree, establishes checkpoint and collection baselines, applies UHP inputs, and only then launches the coding agent. - -Project or user `setup` shell commands are not run as trusted preparation. Additional source kinds or caller-selected builders require a new decision for trust, credential, provenance, and isolation boundaries. - -The practical conclusion is narrow: Harbor is strong evidence for content-addressed input bundles, isolated staging, and prebuilt publication. It is not evidence for resolving repositories inside the execution gateway or making preparation task-authored and opaque. +## Future-adapter questions + +Before implementing any Harbor, Terminal-Bench, or SWE-bench adapter: + +1. Write a dedicated ADR and closed adapter request/result mapping. Do not add a + Harbor source kind or field to the current `AgentRun v1` schemas. +2. Decide and specify who owns the sandbox, agent invocation, checks, + cancellation, cleanup, and terminal publication. Do not imply direct-mode + isolation governed a Harbor-owned sandbox. +3. Define authoritative registry/task/version/environment provenance and, where + a runtime profile exists, immutable profile/image/tool/service implementation + digests. +4. Preserve bounded raw complete/partial agent and check evidence without + translating Harbor scores into gateway pass/fail or reward. +5. Define authenticated expiring artifact access and consumer digest/size + verification. +6. Prove cleanup before terminal result visibility and define the + infrastructure-error behavior for Harbor outages and cleanup failure. +7. Keep direct Promptfoo runs independent of Harbor and let Promptfoo assertions + or graders make every behavioral judgment. + +The practical conclusion is narrow: Harbor is useful primary-source evidence +for content-addressed task packages, isolated tasks, and colocated checks. None +of Harbor, Terminal-Bench, or SWE-bench is an approved current adapter, control +plane, provenance variant, or result contract. They do not justify UHP, +HarnessRouter, or reusable sessions. ## Primary sources diff --git a/docs/research/one-shot-coding-agent-gateway-boundary.md b/docs/research/one-shot-coding-agent-gateway-boundary.md new file mode 100644 index 00000000..5531d792 --- /dev/null +++ b/docs/research/one-shot-coding-agent-gateway-boundary.md @@ -0,0 +1,335 @@ +# One-shot coding-agent gateway boundary + +## Decision + +Use **Promptfoo as the first caller of +`allagentsdev/allagents-gateway`**, a general one-shot coding-agent gateway. +Promptfoo owns prompts, provider and model variants, test matrices, repetition, +assertions, metrics, and result presentation. Promptfoo authoring may declare +multiple Git, OCI, or provider-local workspace sources; the provider packages +local content as an uploaded bundle before submitting `AgentRunRequest v1`. + +The same repository contains the API, disposable trial worker, agent adapters, +source materializers, artifact service, and Promptfoo provider. One public +`AgentRun` request owns one fresh internal trial: + +1. authorize every source, reuse or populate exact Git/OCI cache generations, + and materialize the requested read-only or writable views; +2. authorize the required immutable runtime profile and run Codex or OMP in its + isolated runtime; +3. after the agent reaches a terminal state, tear down its process/cgroup and + network namespace, verify descendant absence, and remove credentials; only + then materialize any authorized hidden bundle and run configured structured + post-run commands in the retained runtime/final workspace with declared + source modes preserved; +4. durably seal bounded agent output plus complete or partial raw command/file + observations; +5. complete workspace/runtime cleanup and release cache/artifact leases; and +6. only then publish `AgentRunResult v1`. + +Source credentials, source policy, credential selection, network policies, and +runtime profiles are operator configuration, never caller-supplied policy +bodies. Harbor and Terminal-Bench integration is outside the current v1 and +requires a future ADR plus closed adapter request/result mapping. HarnessRouter +and UHP are unnecessary: +the gateway has no reusable sessions, checkpoint/continuation contracts, or +generic produced-file service. Any exact patch production belongs only in a +future benchmark adapter whose upstream evaluator requires it. + +This replaces the former immutable-snapshot/HarnessRouter recommendation at +this path. See +[ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md) for the current +decision record. + +## Why Promptfoo is the control plane + +Promptfoo already owns the evaluation-shaped abstractions: + +- its configuration expands prompts, providers, and test variables into a matrix and applies + per-test assertions ([configuration guide](https://www.promptfoo.dev/docs/configuration/guide/)); +- its assertion layer supports deterministic JavaScript/Python checks, structured-output checks, + weighted scores, cost/latency limits, and model-graded rubrics + ([assertions and metrics](https://www.promptfoo.dev/docs/configuration/expected-outputs/)); +- its coding-agent guidance recommends repeated runs for stochastic agents, disposable + workspaces for write-capable tests, and checking files after the run when final text is not + sufficient evidence + ([Evaluate Coding Agents](https://www.promptfoo.dev/docs/guides/evaluate-coding-agents/)); and +- a custom JavaScript or TypeScript provider needs only `id()` and `callApi()`, and may return + structured `output`, `error`, usage, cost, and arbitrary metadata + ([custom JavaScript provider](https://www.promptfoo.dev/docs/providers/custom-api/)). + +Promptfoo's stock Codex provider is useful when evaluating text, traces, or +operations in an already prepared directory. It creates an ephemeral thread by +default and accepts an explicit working directory and sandbox policy +([OpenAI Codex SDK provider](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/)). +It does not by itself materialize several Git/OCI sources, create a pristine +remote trial, optionally verify the resulting filesystem, or guarantee cleanup. +The repository's custom provider is therefore a thin Promptfoo-to-gateway +adapter; the general gateway supplies materialization, one-shot agent execution, +same-runtime post-run checks, raw evidence, and cleanup while Promptfoo retains +all grading and reward policy. + +## AgentRun contract + +The normative wire contracts are +[`AgentRunRequest v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#agentrunrequest-v1), +[`PostRunSpec v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#postrunspec-v1), +and +[`AgentRunResult v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#agentrunresult-v1). +Their checked-in JSON Schemas are authoritative for implementation. This +research note deliberately does not publish a second pseudo-wire schema. + +The request contract is closed. At a logical level it carries request identity, +the instruction, ordered workspace sources and working directory, an agent +selection, a required `runtime_profile_id`, an agent network-policy name, +`post_run` as either `null` or the closed post-run specification, and all +required limits including `max_agent_output_bytes`. Admission resolves and +persists one authorized immutable runtime-profile revision and its canonical +`profile_digest`; dispatch uses only that revision and re-verifies the profile, +runtime image, tool/service implementation, and containerized-service image +digests. Callers do not submit an image, environment map, tool path, service +command, implementation digest, or mutable runtime configuration. + +Git sources carry a canonical HTTPS URL, ref, history policy, destination, and +access mode. OCI sources carry a canonical repository fetch location, direct +descriptor, destination, and access mode; a digest alone has no fetch location. +Uploaded bundles carry an authorized `BundleReference`. For Git and OCI, the +authenticated caller plus canonical source must match exactly one +operator-configured policy/credential route; zero or ambiguous matches reject +before network access. Caller JSON carries no credential or route selector. +Source acquisition denies redirects. Any OCI bearer-token realm is pinned by +the matched operator route rather than trusted from an arbitrary challenge. + +`PostRunSpec v1` uses an optional authorized bundle reference, one named +post-run network policy, ordered structured executable variants with literal +arguments, and ordered output-file requests. It permits no shell parsing, +caller environment, command-specific working directory, or command-specific +network policy. Runtime-only commands and output-only collection need no bundle. + +`AgentRunResult v1` has status `completed`, `cancelled`, or +`infrastructure_error`; direct workspace provenance when available; immutable +runtime provenance; bounded agent output, usage, timing, trajectory, raw +post-run evidence, and typed error data. `RuntimeProvenance` records +`runtime_profile_id`, `profile_digest`, `image_digest`, sandbox-policy version, +each tool's name/version/`implementation_digest`, and each service's +name/version/`implementation_digest` plus container image digest when applicable. + +`AgentResult.final_output` is a bounded `CapturedText` union. Its inline form +records UTF-8 text, digest, byte size, and truncation; its artifact form records +an `ArtifactReference` and truncation. `PostRunEvidence` keeps complete +request-order command and output-file observation vectors. Command observations +distinguish `completed`, `timed_out`, `not_run`, and `unavailable`; file +observations distinguish collected, missing, limit-exceeded, non-regular, +not-run, and unavailable states. Each completed observation is sealed durably, +so cancellation and infrastructure errors can return raw partial evidence +instead of discarding it. Promptfoo alone interprets that evidence as behavioral +pass/fail or reward. + +An `ArtifactReference` includes `artifact_id`, media type, digest, byte size, and +expiry. +Status, cancellation, terminal-result, and artifact-dereference operations +authenticate the caller and enforce tenant/run ownership. The provider obtains +referenced bytes through the authenticated artifact endpoint before expiry and +verifies streamed size and digest; an opaque artifact ID is never treated as +self-authenticating evidence. + +Post-run commands execute only after the agent reaches a terminal state and its +process/cgroup and network namespace are torn down, descendant absence is +verified, and source/model credentials are removed. Hidden-bundle bytes do not +exist in the run filesystem before that boundary. The worker then materializes +any authorized bundle. Commands run in the retained runtime and final workspace +with the exact dependencies, services, filesystem state, and declared source +modes produced by setup and the agent. + +Network access is phase-separated and default-drop: private acquisition, +task-service, agent, and post-run namespaces expose only the destinations +authorized for that phase. Post-run external egress is denied by default; its +named policy may expose only declared localhost or sidecars. A nonzero or timed +out command remains a raw observation, while failure to create/materialize the +workspace, enforce isolation, or launch a command is infrastructure failure. + +Terminal visibility is cleanup-gated. The gateway first assembles and seals all +available evidence, then cleans the workspace/runtime and releases cache and +artifact leases, and only afterward publishes `AgentRunResult v1`. Cleanup or +lease-release failure returns `infrastructure_error` with the partial evidence +collected so far; it never publishes a misleading completed result. + +Keep the API, disposable trial worker, agent adapters, source materializers, +post-run command executor, artifact service, and first Promptfoo provider +together in `allagentsdev/allagents-gateway`. This is a general one-shot +coding-agent boundary, not an evaluation-specific service or session platform. + +## Policy-bound immutable source cache + +The gateway owns a shared cache of verified, immutable source generations so +concurrent trials do not refetch or recopy large inputs. A cache lookup never +bypasses policy: canonicalize the source and repeat the authenticated-caller +route match on every request, including hits. Credentials are used only to +populate a missing generation and are not stored in cached content. + +Cache identity is an exact Git commit/object generation or an OCI repository +plus direct manifest descriptor and the materializer/cache format version. The +descriptor supplies content identity; the repository supplies fetch and policy +context. Populate misses in private staging, verify identity and limits, remove +acquisition-only state, then publish the generation atomically and read-only. A +mutable Git ref or OCI tag may be an input to resolution but never a cache +identity. + +Materialize each declared source according to access: + +- **read-only:** mount the policy-admitted cached generation directly into every + concurrent trial; +- **writable:** create a private reflink/copy-on-write clone; if the filesystem + cannot clone, make a full private copy. + +No trial may write the cache or another trial's view. Agent files, hidden check +bundles, post-run mutations, credentials, processes, and service state remain +trial-private. Cleanup removes those trial views but not the immutable cache +generation. + +This adopts the useful cache mechanism, not E2B itself. E2B hashes template +steps, caches reusable immutable layers, and overlays writes for new sandboxes +([template mechanics](https://docs.e2b.dev/template/how-it-works.md), +[cache semantics](https://docs.e2b.dev/template/caching.md)); Linux reflinks +provide the local copy-on-write primitive with same-filesystem constraints +([`FICLONE`](https://man7.org/linux/man-pages/man2/ioctl_ficlonerange.2.html)). + +## Authenticated private precedent + +The authenticated WiseTechGlobal example +[`exercises/coding-agent-harness`](https://github.com/WiseTechGlobal/ai-evals-examples/tree/main/exercises/coding-agent-harness) +is direct evidence for this boundary (the repository is private and the links require access): + +- [`lib/coding-agent.mjs`](https://github.com/WiseTechGlobal/ai-evals-examples/blob/main/exercises/coding-agent-harness/lib/coding-agent.mjs) + creates a unique temporary root with `mkdtempSync`, recursively copies the fixture into a fresh + workspace, launches the SDK agent in a constrained Docker container, and removes the complete + temporary root in `finally`; +- after the agent exits, checks run with the final workspace mounted into the + check environment and networking disabled; +- [`lib/coding-agent.mjs`](https://github.com/WiseTechGlobal/ai-evals-examples/blob/main/exercises/coding-agent-harness/lib/coding-agent.mjs) + collects fixture-test exits and hidden-check JSON from the workspace, proving + that filesystem checks must execute where the final files and task runtime are + available; +- the example uses separate check containers. The gateway should instead keep + checks in the same trial runtime and final workspace so installed dependencies + and declared services remain available without changing declared source modes; + hidden checks are injected only after the agent and its descendants stop and + credentials are removed; +- the example currently converts observations into pass/fail/reward inside its + provider. The gateway boundary should stop one step earlier: assemble raw + exits, output, and requested artifacts, complete cleanup, then return the + terminal result for Promptfoo code or LLM graders to judge; +- the trial path contains no Git initialization, diff, commit, or produced-file + cursor—the authoritative object is the final filesystem state; and +- [`promptfooconfig.yaml`](https://github.com/WiseTechGlobal/ai-evals-examples/blob/main/exercises/coding-agent-harness/promptfooconfig.yaml) + leaves repetition, prompt variants, task rows, assertions, tracing, timeout, + and concurrency to Promptfoo. + +The example currently runs a Copilot SDK agent in-process with Promptfoo rather +than calling a general gateway. Its reusable evidence is the disposable trial +lifecycle and the requirement that checks execute beside the final workspace. +Keep all judgment in Promptfoo. + +## Harbor and Terminal-Bench are future adapter research + +Harbor packages an instruction, environment, and test script as a self-contained +task. A trial starts the environment, runs the agent, then runs the test script +in that environment; the script writes a numeric or structured reward under +`/logs/verifier/` +([task overview](https://docs.harborframework.com/core-concepts/tasks/overview), +[task tutorial](https://docs.harborframework.com/tutorials/create-a-task)). + +Harbor and Terminal-Bench are not part of the current `AgentRun v1` contract or +implementation plan. Any future integration requires its own ADR and closed +adapter request/result mapping, including provenance, evidence, artifact, +cancellation, and cleanup semantics. It must remain optional, must not become a +core source kind, and must not move pass/fail or reward into the gateway. + +## SWE-bench is future adapter research + +SWE-bench's official evaluator consumes a prediction record containing +`instance_id`, model identity, and `model_patch`; it creates a Docker +environment, applies that patch, runs tests, and writes per-instance reports and +logs +([evaluation guide](https://www.swebench.com/SWE-bench/guides/evaluation/), +[harness reference](https://www.swebench.com/SWE-bench/reference/harness/)). +That is useful evidence that any future SWE-bench adapter owns its exact patch +transport. + +SWE-bench is outside the current v1 contract and implementation plan. A future +adapter requires its own ADR and closed extension, including base-commit, +filename, diff-format, size, and evaluator compatibility rules. Core +`AgentRun v1` exposes no generic diff, patch, modified workspace, or change +artifact. + +## Why stock HarnessRouter evidence does not change the decision + +The superseded research established several stock behaviors that remain factually useful: + +- UHP continuation binds a response chain to the same session, working directory, files, and + harness ([UHP sessions](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/sessions.md#L7-L31)); +- HarnessRouter derives per-session directories and has a durable hydrate/checkpoint lifecycle + ([runner workspace isolation](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L78-L171), + [runner hydrate/checkpoint routes](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L6982-L7110)); and +- its produced-file collector is a root-Git cursor with gateway-side artifact capture and + acknowledgement + ([runner produced routes](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L7111-L7205), + [gateway collector](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L2428-L2524)). + +Those are valuable product-session behaviors, but they solve a different +problem. A Promptfoo coding eval needs one isolated attempt plus raw final-state +evidence; Promptfoo code or LLM graders decide pass/fail and reward. The general +gateway can run optional named post-run checks in the same trial runtime and +final workspace, preserving declared source access modes, after stopping +agent-owned descendants, removing credentials, and injecting hidden checks. It +owns no evaluation verdict. + +Adopting UHP, forking HarnessRouter, or proposing upstream +snapshot/journal/finalization seams would add session, checkpoint, and artifact +semantics that this boundary does not use. + +## Recommendation + +Implement the smallest complete loop: + +```mermaid +flowchart LR + P[Promptfoo JSON matrix] --> Q[Promptfoo provider] + Q --> G[AgentRun API] + G --> K[Map caller + canonical source to one policy route] + K --> M[RO mount or private CoW / full copy] + M --> W[Disposable trial + authorized runtime profile] + W --> A[Codex or OMP] + A --> T[Teardown agent process/netns + remove credentials] + T --> C{Post-run commands?} + C -->|yes| H[Only now materialize authorized bundle] + H --> E[Same runtime + declared source modes] + E --> R[Raw exits + output + artifacts] + C -->|no| O[Agent output + evidence] + R --> Z[Assemble terminal evidence] + O --> Z + Z --> D[Cleanup + release leases] + D --> V[Return terminal AgentRunResult to provider] + V --> X[Dereference authorized artifacts + verify digest] + X --> J[Promptfoo code / LLM graders] +``` + +Promptfoo is the first caller and sole evaluation owner. +`allagentsdev/allagents-gateway` owns the general one-shot `AgentRun v1` API, +policy-rechecked immutable source cache, read-only mounts, private CoW/full-copy +writable views, operator-authorized immutable runtime profiles with profile and +image digests plus tool/service implementation digests, disposable trial workers, +agent adapters, phase-separated default-drop networking, lifecycle fencing, optional +same-runtime post-run commands, bounded structured agent output, raw +complete/partial evidence, and cleanup. Result and artifact access +is tenant/run authorized; the provider dereferences expiring artifacts and +verifies size and digest. + +The gateway assembles and seals evidence, completes cleanup and lease release, +and only then returns `completed`, `cancelled`, or `infrastructure_error`; +cleanup failure produces `infrastructure_error` with partial evidence. It does +not own pass/fail or reward, and no mutable trial state is shared. Source +credentials and policy/credential routes stay out-of-band. Harbor, +Terminal-Bench, and SWE-bench are future work requiring dedicated ADRs and +closed extensions. No reusable-session layer, generic diff service, checkpoint +architecture, UHP endpoint, or HarnessRouter change is warranted. diff --git a/docs/research/source-credential-broker-precedents.md b/docs/research/source-credential-broker-precedents.md index 8a46c52b..dba530f4 100644 --- a/docs/research/source-credential-broker-precedents.md +++ b/docs/research/source-credential-broker-precedents.md @@ -1,18 +1,45 @@ # Source credential broker precedents -## Decision - -The initial trusted-network deployment supplies source credentials only to AllAgents Workspace Builder. Build specifications contain source identities and policy-selected references, never literal credentials. The builder receives only allowlisted credential variables or secret-channel handles in its acquisition process. - -Git credentials are exposed only through a short-lived builder-owned credential helper or registry-auth channel. The builder uses hermetic Git and registry configuration, removes temporary auth state before publication, and emits no secret in logs, provenance, snapshots, descriptors, or response metadata. It never consults arbitrary ambient credential helpers and never falls through to a different credential identity after failure. - -AllAgents Gateway receives no source credentials or Git configuration. It has separate read-only credentials for trusted snapshot repositories, and those credentials never enter the session or harness environment. +## Current conclusion + +`allagentsdev/allagents-gateway` materializes sources declared by normative +[`AgentRunRequest v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#agentrunrequest-v1). +Git supplies a canonical URL and ref; OCI supplies a canonical repository plus +direct descriptor because a digest alone has no fetch location. Caller JSON +never carries a credential, policy-route name, or credential selector. + +The authenticated caller plus canonical Git URL or OCI repository must match +exactly one operator-configured policy/credential route; zero or ambiguous +matches reject before network access. The same match runs on every cache lookup. +Private acquisition denies redirects and permits only the matched route; any OCI +bearer-token realm is operator-pinned rather than accepted from an arbitrary +challenge. A miss may use the matched credential, while a hit needs no source +credential. Cached generations contain neither credentials nor mutable state. + +The required `runtime_profile_id` is independently authorized at admission, +which persists one immutable revision and `profile_digest`; dispatch uses only +that revision and re-verifies its profile/image, tool/service implementation, +and applicable service-image digests. `RuntimeProvenance` returns those exact +digests plus the logical ID, versions, and sandbox-policy version. Source +acquisition, task-service, agent, and post-run network namespaces are +phase-separated and default-drop. The agent +uses a run-scoped, credential-free local model proxy; model credentials never +enter its workspace. After any terminal state, the worker tears down the agent +process/cgroup and network namespace, verifies descendants are absent, and +removes acquisition/model material. Hidden-bundle bytes are not materialized +until that boundary has passed. Post-run commands preserve declared source modes +and receive no source/model credentials. +Status, cancellation, and result access enforce tenant/run +ownership; bundle upload enforces tenant ownership; expiring result-artifact +downloads enforce tenant/run ownership and are verified for size and digest. No +credential may enter bundles, raw evidence, logs, or retained artifacts. Git credential helpers, GitHub App installation tokens, and BuildKit secret -mounts establish the process- and phase-boundary precedents. The deployment does -not require a standalone network credential broker. Central token minting, -delivery leases, remote workers, and multi-tenant credential policy require a -separate decision if the deployment boundary changes. +mounts establish the phase-boundary precedents below. A standalone network +credential broker is unnecessary for the initial deployment. The current +boundary is defined by +[One-shot coding-agent gateway boundary](./one-shot-coding-agent-gateway-boundary.md) +and [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). ## Precedents @@ -163,26 +190,44 @@ mount/socket/environment before agent execution. Like BuildKit, this delivery mechanism does not mint credentials and does not make code with access to the secret trustworthy. -## Recommendation for AllAgents - -### Initial trusted-network deployment - -1. Store only policy-recognized secret references in builder deployment configuration; reject literal credentials and caller-supplied credential identities in build specifications. -2. Supply secret values through the builder's protected environment or secret channel and validate required names during preflight without contacting sources. -3. Pass only the selected allowlisted values to the acquisition child; do not expose them to snapshot packaging, the gateway, or coding-agent processes. -4. Select one configured credential identity before acquisition. Authentication, authorization, rate-limit, or service failure terminates acquisition and never falls through to another identity or source mode. -5. Give the credential only to the dedicated acquisition subprocess through a temporary helper or registry-auth channel. Invoke helpers directly without a shell and bound input, output, stderr, and lifetime. -6. Use isolated Git/registry configuration. Prevent credentials from entering remote URLs, Git config, generated CLI config, workspace files, nested repositories, OCI config/layers, logs, provenance, or descriptors. -7. Remove helper files, auth configuration, and credential-bearing processes before publishing the immutable snapshot. -8. Verify containment with a deliberately non-secret-looking environment name, because name-based secret filters are not the security boundary. - -### Remote or multi-tenant deployment - -A future deployment may require a central token minter, authenticated single-use -delivery leases, entitlement generations, revocation reconciliation, worker -identity, fencing, and a snapshot-delivery protocol. Those mechanisms require a -separate decision when remote workers or tenant isolation become product -requirements. - -The resulting rule is: **source credentials exist only during the materializer's -acquisition phase and never enter the coding-agent environment.** +## Recommendation for evaluations + +1. Use the normative `AgentRunRequest v1`, `PostRunSpec v1`, and + `AgentRunResult v1` contracts rather than a second credential-specific shape. +2. Configure source matchers, redirect policy, OCI auth realms, credential + mapping, runtime profiles, network policies, and secret handles out-of-band. +3. Combine authenticated caller identity with canonical Git URL or OCI + repository. Require exactly one source route; reject zero or ambiguous + matches before any network access. +4. Deny source redirects and pin any OCI bearer-token realm through the matched + operator route. Repeat source authorization on every immutable-cache lookup. +5. Give the matched source credential only to bounded private acquisition on a + miss; remove it before publishing the verified immutable generation. Never + store credentials or mutable trial state in the shared cache. +6. Resolve and persist an authorized immutable runtime-profile revision and + `profile_digest` at admission. Dispatch uses only that revision, re-verifies + profile/image, tool/service implementation, and applicable service-image + digests, and returns them in `RuntimeProvenance`. Give the agent model access + only through its run-scoped local proxy; never expose model credentials in + the process, environment, workspace, or artifacts. +7. Keep acquisition, task-service, agent, and post-run networks phase-separated + and default-drop. Apply only the named policy authorized for each phase. +8. Tear down the agent process/cgroup and network namespace, verify descendants + are absent, and remove credentials before any hidden-bundle bytes are + materialized. Only then authorize/materialize the optional `BundleReference` + and run structured commands with declared source modes intact. +9. Seal bounded complete/partial raw evidence as observations finish. Promptfoo + alone owns pass/fail and reward. +10. Destroy the workspace, agent home, containers, temporary credential + material, and network namespaces and release leases before publishing + `AgentRunResult v1`. Cleanup failure is `infrastructure_error` with sealed + partial evidence. +11. Tenant/run-authorize status, cancellation, result, and result-artifact + operations; tenant-authorize bundle upload. Dereference expiring result + artifacts through the authenticated endpoint and verify streamed size and + digest. + +A central token minter, delivery lease, or generic credential-broker protocol is +out of scope until remote multi-tenant workers create a concrete need. The rule +for the current gateway is: **credentials exist only in the phase that consumes +them and never enter raw post-run evidence or durable trial state.** diff --git a/docs/research/workspace-contract-incumbents.md b/docs/research/workspace-contract-incumbents.md index 134a26ee..0073aa25 100644 --- a/docs/research/workspace-contract-incumbents.md +++ b/docs/research/workspace-contract-incumbents.md @@ -1,36 +1,43 @@ # Workspace contract incumbents -## Decision - -No examined incumbent replaces the complete AllAgents preparation and execution stack. The useful separation is now: - -1. **Northbound execution:** UHP remains the sole request/response protocol. -2. **Source layout:** an AllAgents Workspace Builder contract owns Git/OCI inputs, destinations, history selection, credentials, and composition before execution. -3. **Immutable handoff:** one direct OCI workspace-snapshot descriptor crosses into execution. -4. **Runtime:** AllAgents Gateway initializes a private session tree and retains HarnessRouter's UHP/session lifecycle. - -Devfile 2.3 remains the closest portable source-layout precedent. Daytona, Codespaces, and Gitpod remain provider-specific operational precedents. Harbor's prebuilt environments and content-addressed packages are stronger evidence for the new build-then-execute boundary than for runtime composition. - -This note's incumbent comparisons remain useful. Its earlier recommendation to send caller-selected Git/OCI sources directly to the gateway is superseded by [Prebuilt immutable workspace snapshots at the HarnessRouter boundary](./allagents-gateway-snapshot-boundary.md) and [ADR 0002](../decisions/0002-adopt-uhp-through-harnessrouter.md). - -## The four contracts are different - -| Layer | AllAgents boundary | Best established precedent | Assessment | -| --- | --- | --- | --- | -| Northbound execution | UHP requests, events, results, cancellation, continuation, files, and artifacts | UHP | A workspace standard should not displace the execution protocol. | -| Preparation | Builder-owned Git/OCI source plan and deterministic composition | Devfile `projects`, Daytona clone operations, Harbor prebuilds | No incumbent covers the complete source/history/provenance contract; keep it outside HarnessRouter. | -| Immutable handoff | One direct OCI workspace-snapshot descriptor with digest-covered manifest/provenance | OCI Image Specification 1.1.1 | Adopt OCI identity directly and reject tags/indexes at execution. | -| Runtime sandbox | Private writable session tree behind AllAgents Gateway | HarnessRouter, Harbor ASP, E2B, Daytona | Runtime providers stay southbound and do not become the source contract. | - -This separation avoids exposing provider operations northbound or source credentials to the execution plane. +## Current conclusion + +No examined incumbent replaces the chosen boundary. Promptfoo is the first +caller and sole evaluation owner, while `allagentsdev/allagents-gateway` owns the +general one-shot `AgentRun v1` API, policy-bound immutable source cache, +operator-authorized immutable runtime profiles with profile/image and tool/service +implementation digests, disposable trial workers, agent adapters, +phase-separated default-drop networking, bounded raw evidence, +cleanup, and first Promptfoo provider. Promptfoo JSON may describe several +workspace sources; policy/credential routes and runtime profiles remain operator +configuration. + +Each `AgentRun` creates one fresh workspace and runs Codex or OMP under the +required `runtime_profile_id`. After the agent terminates, the worker tears down +its process/cgroup and network namespace, verifies descendants are absent, and +removes credentials. Hidden-bundle bytes are materialized only after that +boundary; structured post-run commands then use the retained runtime/final +workspace with declared source modes intact. The gateway durably seals complete +or partial observations, destroys trial state, releases leases, and only then +publishes `AgentRunResult v1`. Cleanup failure is `infrastructure_error` with +partial evidence. Tenant/run-authorized artifact retrieval is time-bounded and +the provider verifies byte size and digest. Promptfoo alone decides pass/fail +and reward. Harbor, Terminal-Bench, and SWE-bench require future ADRs and closed +adapter extensions; none is part of current v1. No incumbent justifies UHP, +HarnessRouter, or reusable execution sessions. + +The current boundary is defined by +[One-shot coding-agent gateway boundary](./one-shot-coding-agent-gateway-boundary.md) +and [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). ## Historical contract evaluated -The comparisons below originally evaluated a runtime source descriptor with repository URLs, refs, destinations, OCI snapshots, logical working directory, pre-agent materialization, and immutable returned provenance. - -Those properties remain requirements, but ownership changed: source selection, ref resolution, credentials, destination composition, and full-history verification now belong to AllAgents Workspace Builder. AllAgents Gateway receives one already-published direct manifest descriptor, verifies its digest-covered working directory and provenance identities, and never receives the source plan. - -The comparison below evaluates semantic coverage rather than current field placement. +The comparisons below originally evaluated a product execution stack with UHP, +a separate source builder, an immutable OCI handoff, and a long-lived gateway. +That architecture is rejected for the chosen evaluation-only scope. Statements +below that prescribe source descriptors, snapshot manifests, builder ownership, +or gateway behavior are retained as historical comparison, not current +recommendations. Their primary-source descriptions of incumbents remain useful. ## Incumbent comparison @@ -130,24 +137,61 @@ For OCI snapshots, the [OCI Image Specification 1.1.1 descriptor](https://github For Git, a full commit object ID is the resolved source identity. The request still needs the original ref because a branch/tag name and its resolved commit answer different audit questions. OCI snapshot provenance instead retains the verified `snapshotName` and `imageManifestDigest`; each history-bearing root adds only its destination, resolved commit, and object-set digest, with no repository URL or requested ref. Neither Git nor OCI defines when a runner has successfully attached that content, so `effectiveDescriptorDigest`, `generationId`, `sourceIdentity`, `workingDirectory`, and `workspaceManifestDigest` must remain AllAgents result fields. -## Recommendation and adoption rule - -Adopt the following rule for future changes: - -- **Normative execution:** UHP northbound and the AllAgents vendor snapshot descriptor only on a first turn. -- **Normative preparation:** the builder owns source-layout fields informed by Devfile/Daytona, but claims no conformance to either. -- **Normative handoff:** direct OCI manifest identity plus digest-covered canonical workspace manifest and provenance. -- **Benchmark compatibility:** ingest Harbor task packages and SWE-bench/Hugging Face records through preparation adapters, preserving their task/environment/verifier separation. -- **Runtime precedent:** evaluate Harbor ASP, E2B, Daytona, or Dev Containers only as southbound sandbox/runtime layers. -- **Do not conflate:** benchmark repository with target source; runtime image with workspace snapshot; snapshot digest with authorization; or provider sandbox identity with the UHP session. - -This remains a layered answer rather than a claim that AllAgents invented a universal workspace standard. The builder contract exists because source-layout standards stop before the required immutable provenance. The gateway extension remains narrow because execution receives only the published result. +## Current recommendation + +Adopt the following rule for future evaluation work: + +- **First caller:** Promptfoo owns prompt/provider matrices, repeats, assertions, + code/LLM grading, rewards, metrics, and result presentation. +- **Invocation contract:** use the normative + [`AgentRunRequest v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#agentrunrequest-v1), + [`PostRunSpec v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#postrunspec-v1), + and + [`AgentRunResult v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#agentrunresult-v1); + this research note defines no alternate wire shape. +- **Runtime boundary:** admission resolves and persists one authorized immutable + runtime-profile revision and `profile_digest`; dispatch uses only that revision + and re-verifies its profile/image, tool/service implementation, and applicable + service-image digests. `RuntimeProvenance` returns those identities with + `runtime_profile_id`, versions, and sandbox-policy version. +- **Source route:** canonical Git URL or OCI repository plus authenticated + caller must match exactly one operator policy/credential route; reject zero or + ambiguous matches before network access. Deny redirects and pin any OCI auth + realm through the matched route. +- **Gateway ownership:** `allagentsdev/allagents-gateway` contains the general + one-shot API, disposable trial worker, materializers, agent adapters, lifecycle + fencing, artifact service, post-run execution, raw evidence, cleanup, and + Promptfoo provider. +- **Cache boundary:** repeat source authorization on every hit; publish only + verified exact generations. Mount read-only generations directly; give + writable sources a private reflink/CoW clone or full-copy fallback. +- **Post-run boundary:** tear down the agent process/cgroup and network + namespace, verify descendants are absent, and remove credentials before hidden + bundle bytes exist in the run filesystem. Only then materialize an authorized + optional bundle and run structured commands in the final workspace with + declared source modes intact. Phase-separated namespaces default-drop traffic; + post-run policy may allow only declared localhost/sidecars. +- **Result boundary:** bound agent final output as `CapturedText`; preserve full + request-order complete/partial `PostRunEvidence`; seal evidence, clean the + workspace, and release leases before publishing `completed`, `cancelled`, or + `infrastructure_error`. Cleanup failure retains partial evidence. +- **Artifact boundary:** status, cancel, result, and artifact access are + tenant/run authorized. Expiring artifact references are dereferenced through + the authenticated endpoint and verified for streamed size and digest. +- **Judgment boundary:** Promptfoo alone assigns behavioral pass/fail/reward. +- **Benchmark compatibility:** Harbor, Terminal-Bench, and SWE-bench are future + work requiring dedicated ADRs and closed adapter extensions. None changes the + current v1 schemas. +- **Runtime precedent:** evaluate E2B, Daytona, Dev Containers, or Harbor ASP + only if a concrete isolation or imported-task requirement needs them. ## Existing research status -- [Harbor repository materialization](./harbor-repository-materialization.md) supplies the content-addressed-cache, staging, and prebuilt-publication precedents now adopted by the builder. -- [E2B execution-gateway patterns](./e2b-execution-gateway-patterns.md) remains correct that E2B is a runtime provider rather than a replacement northbound protocol. -- [Prebuilt immutable workspace snapshots at the HarnessRouter boundary](./allagents-gateway-snapshot-boundary.md) is the current boundary analysis and supersedes runtime-composition conclusions in earlier notes. -- General and private research wikis were discovery inputs only; public primary sources and ADR 0002 are normative for this decision. - -ADR 0002 and the implementation plan now codify a builder-to-gateway OCI artifact boundary. Direct Git URLs, refs, destinations, history policy, and source credentials are builder inputs; the gateway accepts one authorized direct descriptor and returns its verified manifest/provenance identities. +- [One-shot coding-agent gateway boundary](./one-shot-coding-agent-gateway-boundary.md) + is the current boundary analysis. +- [Harbor repository materialization](./harbor-repository-materialization.md) + remains useful evidence for task packages and separate verifiers. +- [E2B execution-gateway patterns](./e2b-execution-gateway-patterns.md) remains + useful sandbox evidence, but E2B is not required by the default runner. +- General and private research wikis were discovery inputs only; cited primary + sources and ADR 0002 carry the decision. From be84bcf98a448b070e37d54fdc1ec6cd0c90b5ad Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 13:42:49 +1000 Subject: [PATCH 40/44] docs(architecture): simplify gateway ADR --- ...use-allagents-gateway-for-one-shot-runs.md | 477 ++++++------------ 1 file changed, 164 insertions(+), 313 deletions(-) diff --git a/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md b/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md index 31b28dc2..73633cf6 100644 --- a/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md +++ b/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md @@ -4,378 +4,229 @@ - Date: 2026-09-21 - Updated: 2026-09-28 -## At a glance - -A Promptfoo case asks OMP to fix a bug in a workspace composed from writable application Git commit `8c4f…` at `app/` and a read-only OCI-hosted fixture at `fixtures/`, with `app/` as the working directory. It selects the authorized `node22` runtime profile and `package-install` agent network policy, then requests post-run evidence from a hidden immutable check bundle: confined executable `verify`, literal arguments `["--json", "reports/test.json"]`, bounded output file `reports/test.json`, and the `integration-checks` post-run network policy. The provider packages local content and submits one `AgentRunRequest v1`. A worker resolves the profile to an immutable image and creates a fresh `RunSandbox`; the runner attaches authorized immutable source generations and invokes OMP once. After OMP exits, the runner terminates its process tree, destroys its network namespace and credentials, preserves the runtime and workspace, and runs the check in a separately brokered post-run network namespace. Only after evidence persistence, service stop, unmount, private-root deletion, and lease release succeed does the gateway return `status: "completed"` with direct and runtime provenance, bounded agent output, usage, trajectory, ordered raw command evidence, and the report's `ArtifactReference`. The provider downloads that artifact through the authenticated result-artifact endpoint, verifies its length and digest, and gives the evidence to Promptfoo's code grader; the gateway itself never decides pass or reward. +## Decision -The gateway's product is one-shot execution output and raw evidence, not a behavioral judgment or workspace diff. No caller continues the coding session, restores a checkpoint, or reconstructs the final tree from change artifacts. The previous HarnessRouter/UHP session and snapshot architecture therefore solves lifecycle and transport problems that one-shot runs do not have. +We will build `allagentsdev/allagents-gateway` as a general service for one-shot coding-agent runs. The API, worker, runner, Promptfoo provider, and Codex and OMP adapters will live in that repository. -## Context +Promptfoo will be the first caller. Promptfoo owns evaluation cases, prompt and model matrices, repetitions, grading, pass/fail decisions, rewards, and reports. The gateway runs an agent and returns raw evidence. It does not grade the result. -AllAgents needs a general remote gateway for one-shot coding-agent runs, similar in purpose to HarnessRouter but deliberately smaller in lifecycle. Promptfoo is its first caller and remains the authoring and experiment layer for evaluations: it owns test cases, variable and provider matrices, prompt rendering, repetitions, grading, assertions, and reports. Other callers may omit post-run evidence collection and consume the agent output directly. +Each request creates a fresh workspace, runs one agent once, optionally collects evidence, and destroys the workspace. V1 has no reusable session, continuation, checkpoint, generic workspace diff, or implicit rerun. -A run has one rendered instruction, one composed workspace, one selected agent, and optional post-run evidence collection. Its mutable state exists only for that run. It is not a reusable coding session. Promptfoo maps each evaluation trial and repetition to a distinct run and applies its own code or LLM grader to the returned evidence. Retries must not accidentally share state or run the agent more than once. +The public contracts are `AgentRunRequest v1` and `AgentRunResult v1`. They are closed, versioned JSON schemas. Unknown fields are rejected, and incompatible changes require a new version. The detailed fields belong in the gateway schemas and are recorded in the [implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). -The earlier version of this decision treated remote agent execution as a UHP session served by a HarnessRouter-derived gateway. It introduced a separate workspace-builder repository, published OCI workspace snapshots, added snapshot import and filesystem journaling to HarnessRouter, checkpointed workspaces for continuation, and exported generic workspace-change artifacts. Those capabilities are unnecessary for one-shot runs and create protocol, fork, storage, and recovery obligations unrelated to the result callers need. +This decision supersedes the earlier two-repository snapshot design and HarnessRouter issue [#304](https://github.com/HarnessRouter/harnessrouter/issues/304). We keep the `allagentsdev/allagents-gateway` name, but none of the earlier HarnessRouter fork, UHP session, workspace-builder, or snapshot contracts remain. -Supporting evidence and comparisons are recorded in [One-shot coding-agent gateway boundary](../research/one-shot-coding-agent-gateway-boundary.md). +## Why -## Decision +AllAgents needs remote, isolated coding-agent execution. It does not currently need long-lived interactive sessions. -We will build one general one-shot AllAgents Gateway. The gateway API, reusable worker and runner packages, Promptfoo provider, and Codex and OMP adapters will live in one source repository: `allagentsdev/allagents-gateway`. +The earlier design used HarnessRouter sessions, published workspace snapshots, checkpoints, filesystem journals, and generic change artifacts. Those features solve continuation and state-transfer problems. A one-shot run only needs a clean workspace, one agent attempt, optional evidence from the final state, and reliable cleanup. Keeping the session design would add protocol, storage, recovery, and fork-maintenance work without improving that result. -The public operation is `AgentRunRequest v1` and `AgentRunResult v1`. Both are closed, versioned JSON contracts: unknown fields are rejected, identifiers are explicit, and incompatible changes require a new version. The contract borrows useful UHP conventions—stable IDs, idempotency, cancellation, structured errors, and bounded structured telemetry—but V1 does not implement UHP or expose reusable sessions. - -HarnessRouter issue [#304](https://github.com/HarnessRouter/harnessrouter/issues/304) is superseded by this decision. The two-repository gateway/workspace-builder snapshot design is also superseded and is not an implementation contract for the AllAgents Gateway. -The `allagentsdev/allagents-gateway` repository name is retained for the new standalone service; none of the former HarnessRouter fork, snapshot, or workspace-builder contract survives. +Supporting research is in [One-shot coding-agent gateway boundary](../research/one-shot-coding-agent-gateway-boundary.md). ## Main flow -1. A caller defines an ordered multi-source workspace, selects an authorized logical runtime profile, and renders one instruction. For evaluations, Promptfoo first expands its cases, variables, providers, and repetitions and assigns every repetition a distinct run identity. -2. The Promptfoo provider deterministically packages local source and optional check content, reserves uploads with client-generated `upload_id` values, uploads immutable bundles, and configures the raw evidence its graders need. It calls the AllAgents Gateway with artifact references and digests; remote JSON never contains a caller host path. -3. The gateway authenticates and validates the closed request, then performs tenant-scoped duplicate lookup before mutable admission. An exact duplicate returns the existing run without re-admission. For a genuinely new identity, the gateway applies source, model, network, and resource policy, resolves `runtime_profile_id` to an authorized immutable `profile_digest`, and durably pins that revision; rejected new requests create no public run or job. -4. A worker accepts only the pinned profile revision and creates one fresh ephemeral sandbox from its immutable image and sandbox policy; unavailable or mismatched profile content fails the run. For Git and OCI, the gateway maps the authenticated caller plus canonical URL or repository to exactly one internal source policy and credential route before credentials or network access; zero or ambiguous matches reject the run. The runner resolves sources, rechecks policy, and attaches operator-owned immutable generations. `read_only` sources mount directly read-only; `writable` sources receive private CoW/reflink clones or full-copy fallback. -5. The selected Codex or OMP adapter runs the agent once under the agent network phase. The runner captures bounded structured final output, usage, timing, and trajectory. -6. After the agent exits, the runner terminates the agent process tree, destroys its network namespace and flows, and removes model and source credentials while preserving the sandbox, pinned runtime, final workspace access modes, and operator-declared task services. Only after that transition does it inject the immutable check bundle when requested, create the separately authorized post-run network namespace, and execute declared runtime or bundle executables with literal arguments. -7. The worker durably seals each raw post-run observation in request order, persists requested file artifacts, then stops post-run processes and task services, unmounts every trial mount, deletes the private trial root, and releases every cache lease. These operations are idempotent and must all succeed before any terminal result becomes visible. -8. The gateway then publishes `completed`, `cancelled`, or `infrastructure_error`. Cleanup failure forces `infrastructure_error` with partial evidence; reconciliation completes cleanup before publication. Promptfoo retrieves referenced artifacts, verifies their declared size and digest, and exposes the evidence to code or LLM graders, which alone decide behavioral outcomes. - -There is no checkpoint, continuation, reusable coding session, or implicit rerun in this flow. - -## Ownership - -### Promptfoo - -Promptfoo owns evaluation intent and aggregation: - -- case and dataset authoring; -- variables, provider and agent matrices, and repetition counts; -- rendering the complete instruction supplied to a run; -- defining post-run evidence collection and applying code or LLM graders to agent output, command results, and file artifacts; and -- experiment reports, pass/fail decisions, rewards, assertions, and comparisons across runs. - -Promptfoo does not own the remote sandbox, materialize host paths remotely, inspect gateway credentials, or infer success from changed files. - -### AllAgents Promptfoo provider - -The provider is the boundary adapter between Promptfoo and the AllAgents Gateway API. It assigns run and idempotency identity, converts Promptfoo configuration into `AgentRunRequest v1`, packages any check bundle, uploads and content-addresses local inputs before submission, waits for or cancels the run, and exposes `AgentRunResult v1` output, command results, and file artifacts to Promptfoo's code or LLM graders. Those graders alone assign pass, fail, or reward. - -Packaging is intentionally client-side. A local repository path, check-bundle path, fixture path, or requested output destination on the caller host is never meaningful to a remote worker and must not appear in the remote request. - -### AllAgents Gateway, worker, and runner - -The gateway API is the durable public control plane. It owns tenant authentication, request validation, duplicate lookup, admission, idempotency, cancellation, worker dispatch, artifact authorization, and result retrieval. A worker owns one admitted run's process and sandbox lifecycle. Inside that worker, the reusable runner package owns workspace composition, isolation, resource enforcement, agent supervision, the agent-to-evidence phase transition, ordered durable evidence capture, cleanup, and terminalization. Codex and OMP adapters translate the common run contract into each agent's invocation and normalize output, usage, and trajectory data; they do not define separate public execution contracts. - -Operator configuration owns credentials, source authorization, model routing, runtime profiles, fixed runtime tools, sandbox policies, phase-specific network policies, task-service definitions, resource ceilings, retention, and deployment policy. None of those secrets or policy documents are caller-controlled fields. - -### Post-run evidence - -The `post_run` field is required but nullable: `null` requests no extra evidence, while a nonnull value requests evidence collection rather than verification. It may identify an immutable check bundle, commands with individual timeouts, bounded workspace-relative `output_files`, and a separately named post-run network policy. Each command selects either an operator-allowed executable already present in the task runtime or an executable at a confined path inside the optional check bundle, then supplies literal arguments. A bundle executable requires `bundle`; runtime executables and output-only collection do not. Bundle bytes and a populated bundle mount are absent throughout agent execution; at most an empty, runner-owned reserved mountpoint exists. The runner first terminates every agent-owned descendant, destroys the agent network namespace and flows, and removes model and source credentials. Only then may it materialize the bundle into that mountpoint read-only and make it visible to post-run commands. Those commands use the same pinned runtime and final workspace with each source's declared access mode preserved. This lets checks use build tools such as `npm`, build the project, and exercise its actual runtime. Operator-declared task services may remain running or be restarted for checks; arbitrary agent-owned background processes never cross the phase boundary. Checks that need to modify a source must declare it `writable` or place build and test output in a separate writable path. Nonzero exits, timeouts, stdout, stderr, and generated files are observations. The gateway records them without interpreting task success. Promptfoo or another caller owns any code or LLM grader that turns those observations into pass, fail, reward, or commentary. - -## Public request boundary - -`AgentRunRequest v1` is defined by the normative closed schema `schemas/agent-run-request.v1.schema.json`. Its logical shape is: - -```text -{ - schema_version: "agent_run_request.v1", - identity: { - run_id: UUIDv7, - idempotency_key: string - }, - instruction: string, - workspace: { - working_directory: relative path or ".", - sources: array of 1..128 - | { - kind: "git", - url: canonical HTTPS URL, - ref: string, - history: { mode: "full" } | - { mode: "shallow", depth: positive integer }, - access: "read_only" | "writable", - destination: relative path or "." - } - | { - kind: "oci", - repository: canonical OCI repository, - descriptor: { - media_type: "application/vnd.oci.image.manifest.v1+json", - digest: SHA-256 digest, - size_bytes: positive integer - }, - access: "read_only" | "writable", - destination: relative path or "." - } - | { - kind: "uploaded_bundle", - bundle: BundleReference, - access: "read_only" | "writable", - destination: relative path or "." - } - }, - agent: - | { kind: "codex", model: logical ID, reasoning_effort: "low" | "medium" | "high" } - | { kind: "omp", model: logical ID }, - runtime_profile_id: logical ID, - post_run: null | { - bundle?: BundleReference, - network_policy_id: logical ID, - commands: array of 0..32 { - command_id: logical ID, - executable: - | { kind: "runtime", name: logical ID } - | { kind: "bundle", path: relative path }, - args: array of literal strings, - timeout_ms: positive integer - }, - output_files: array of 0..128 { - name: logical ID, - path: relative path, - media_type: IANA media type, - max_bytes: positive integer - } - }, - agent_network_policy_id: logical ID, - limits: { - total_timeout_ms: positive integer, - acquisition_timeout_ms: positive integer, - agent_timeout_ms: positive integer, - post_run_timeout_ms: positive integer, - max_workspace_bytes: positive integer, - max_workspace_files: positive integer, - max_agent_output_bytes: positive integer, - max_artifact_bytes: positive integer, - max_trace_bytes: positive integer, - max_command_output_bytes: positive integer - } -} - -BundleReference = { - artifact_id: UUIDv7, - media_type: "application/vnd.allagents.gateway-bundle.v1.tar+gzip", - digest: SHA-256 digest, - size_bytes: positive integer -} +1. The caller provides one instruction, an ordered list of workspace sources, an agent, an authorized runtime profile, network-policy names, resource limits, and optional post-run evidence requests. Promptfoo gives every evaluation repetition a distinct run identity. +2. The Promptfoo provider packages local sources and optional check content into immutable uploads. Remote requests contain artifact references and digests, never paths on the caller's machine. +3. The gateway authenticates the caller, validates the request, and checks for an existing run with the same identity. It then authorizes a new request and pins the exact runtime profile revision that the worker must use. +4. The worker creates a fresh sandbox. It authorizes and materializes each source at its requested destination. Read-only sources stay read-only. Writable sources receive private copies. +5. The selected Codex or OMP adapter starts the agent once. The runner records bounded final output, usage, timing, and trajectory data. +6. After the agent exits, the runner stops every agent-owned process, destroys the agent network environment, and removes model and source credentials. If the request asks for post-run evidence, the runner then exposes the hidden check bundle, creates a separate post-run network environment, and runs the declared commands in the same retained runtime and final workspace. +7. The worker stores each command result and requested file observation in request order. It then stops remaining processes and services, unmounts the workspace, deletes private files, and releases source-cache leases. +8. Only after cleanup succeeds does the gateway publish `completed`, `cancelled`, or `infrastructure_error`. Promptfoo downloads and verifies referenced artifacts, then applies its own graders. + +```mermaid +flowchart LR + P[Caller or Promptfoo] --> G[Gateway validates and admits] + G --> W[Worker composes a fresh workspace] + W --> A[Codex or OMP runs once] + A --> E{Post-run evidence requested?} + E -->|No| S[Seal agent result] + E -->|Yes| C[Stop agent, remove credentials, run checks] + S --> K[Clean workspace and release leases] + C --> K + K --> R[Publish terminal result] + R --> P ``` -Source order is authoritative. Normalized destinations are pairwise non-overlapping; `.` is allowed only as the sole destination. `working_directory` must resolve without symlink escape to a directory in the composed tree. Every source requires `access: read_only | writable`. - -Git carries a canonical URL, requested ref, history, access mode, and destination. The runner resolves the ref once to an exact commit and reports it in provenance. OCI carries a canonical repository because the exact direct image-manifest descriptor has no location. Uploaded content uses the tenant-authorized `BundleReference`. - -For Git, the gateway maps `(authenticated tenant, canonical URL)` to exactly one internal source policy and credential route. For OCI it maps `(authenticated tenant, canonical repository)` the same way. Bundle `artifact_id` is resolved through tenant-scoped artifact authorization. Zero or multiple matches reject before credential resolution, DNS, registry, Git, artifact reads, or mutable run admission. The caller never names a credential, internal route, mirror, token realm, or policy record. - -V1 rejects all Git and OCI redirects. The exact internal OCI route pins its permitted bearer-token realm; OCI manifests and blobs must be fetched through that repository route, and foreign blob or descriptor URLs are rejected. Credentials are never forwarded to a target selected by remote content. - -`runtime_profile_id`, agent model, `agent_network_policy_id`, and `post_run.network_policy_id` are independently authorized logical IDs. Admission resolves the runtime ID to an immutable `profile_digest` and persists that exact revision with its runtime image digest, fixed read-only runtime `PATH` and tool implementations, sandbox policy, and operator task-service implementations. Dispatch may resolve only the pinned revision and rejects unavailable or mismatched content. The caller cannot submit an image, executable path, service definition, raw network destination, policy body, credential, environment map, shell string, host path, grading rule, patch request, or workspace-persistence option. - -`post_run` is required and is either `null` or the closed evidence specification. At least one of `commands` or `output_files` is nonempty. `bundle` is present if and only if a bundle executable is requested. Runtime executable names resolve through the profile's fixed read-only tool mapping, never an agent-modifiable `PATH`; bundle paths remain confined beneath the hidden immutable bundle. Arguments are literal. There is no shell, interpolation, globbing, working-directory override, caller environment, or per-command network override. - -All ten limits are required and caller-lowerable beneath operator and runtime-profile ceilings. `max_agent_output_bytes` bounds the agent's structured final response. `max_artifact_bytes` is the aggregate logical-byte budget for result artifacts, excluding input bundles and source caches. Admission requires the sum of `output_files[].max_bytes` to fit this budget. Actual collected files charge first in request order, followed by promoted agent output, command stdout/stderr in command order, and trajectory; content deduplication gives no accounting discount. Optional promotion that cannot fit falls back to bounded inline truncated data or an explicit omitted observation, never unbounded storage. - -## Public result boundary - -`AgentRunResult v1` is defined by the normative closed schema `schemas/agent-run-result.v1.schema.json`. Its common logical shape is: - -```text -{ - schema_version: "agent_run_result.v1", - identity: { - run_id: UUIDv7, - idempotency_key: string - }, - status: "completed" | "cancelled" | "infrastructure_error", - post_run_mode: "none" | "requested", - effective_limits: { the ten required request limit fields }, - workspace_provenance: DirectWorkspaceProvenance | null, - runtime_provenance: RuntimeProvenance | null, - agent: AgentResult | null, - post_run: PostRunEvidence | null, - usage: Usage, - timing: Timing, - trajectory: InlineTrajectory | ArtifactTrajectory | null, - error: RunError | null -} - -DirectWorkspaceProvenance = { - kind: "direct", - working_directory: relative path or ".", - sources: ordered array of DirectSourceProvenance -} - -RuntimeProvenance = { - runtime_profile_id: logical ID, - profile_digest: SHA-256 digest, - image_digest: SHA-256 digest, - sandbox_policy_version: string, - tools: array of { - name: logical ID, - version: string, - implementation_digest: SHA-256 digest - }, - services: array of { - name: logical ID, - version: string, - implementation_digest: SHA-256 digest, - image_digest: SHA-256 digest | null - } -} - -AgentResult = { - kind: "codex" | "omp", - model: logical ID, - adapter_version: string, - termination: AgentTermination, - exit_code: integer | null, - final_output: CapturedText | null -} - -CapturedText = - | { - storage: "inline", - encoding: "utf-8", - text: string, - digest: SHA-256 digest, - size_bytes: integer >= 0, - truncated: boolean - } - | { - storage: "artifact", - encoding: "utf-8", - artifact: ArtifactReference, - truncated: boolean - } - -ArtifactReference = { - artifact_id: UUIDv7, - media_type: IANA media type, - digest: SHA-256 digest, - size_bytes: integer >= 0, - expires_at: UTC RFC 3339 timestamp -} - -PostRunEvidence = { - commands: array in request order of PostRunCommandObservation, - output_files: array in request order of OutputFileObservation -} -``` +## Responsibilities + +| Component | Owns | Does not own | +|---|---|---| +| Promptfoo | Evaluation cases, prompt rendering, model matrices, repetitions, requested evidence, graders, pass/fail, rewards, and reports | The remote sandbox or gateway lifecycle | +| Promptfoo provider | Stable run and retry identities, local packaging, immutable uploads, submission, cancellation, polling, artifact verification, and exposing evidence to Promptfoo | Grading or remote execution | +| Gateway | Authentication, validation, authorization, duplicate-request handling, cancellation, dispatch, artifact access, and result retrieval | Behavioral pass/fail or reward | +| Worker and runner | One run's sandbox, workspace composition, limits, agent supervision, evidence collection, and cleanup | Evaluation policy | +| Operator configuration | Credentials, source access rules, model routing, runtime profiles, sandbox and network policies, task services, resource ceilings, and retention | Caller-controlled request data | + +Callers select authorized logical names. They cannot submit credentials or policy definitions. + +## Request boundary + +A request includes: + +- `run_id` and `idempotency_key`; +- one rendered instruction; +- an ordered workspace and working directory; +- Codex or OMP plus a logical model name; +- a required `runtime_profile_id`; +- an agent network-policy name; +- required time, workspace, output, trace, command-output, and artifact limits; and +- `post_run`, which is either `null` or a closed evidence request. -`completed`, `cancelled`, and `infrastructure_error` describe only gateway lifecycle. The gateway never returns behavioral pass, fail, or reward. `completed` requires nonnull workspace and runtime provenance, agent result with nonnull final output, and trajectory, requires `error: null`, and requires nonnull post-run evidence exactly when `post_run_mode` is `requested`. Cancelled and infrastructure-error results may carry partial or null provenance, agent, and trajectory fields and require a structured error. Runtime provenance records what actually ran without exposing policy bodies, host paths, credentials, private endpoints, or mutable runtime aliases. +A workspace contains between 1 and 128 sources. Every source declares a non-overlapping destination and `access: read_only | writable`. -Post-run evidence always preserves request order. Command observations are discriminated as `completed`, `timed_out`, `not_run`, or `unavailable`; output-file observations are `collected`, `missing`, `limit_exceeded`, `not_regular_file`, `not_run`, or `unavailable`. Each observation is sealed durably as it becomes terminal. For cancellation or infrastructure failure before the post-run skeleton is durable, `post_run` may be null. Once it is durable, `post_run` is nonnull and contains the full requested vectors: durable observations followed by explicit `not_run` or `unavailable` entries. Already durable evidence is never discarded or reordered. -A `completed` command observation records `command_id`, exit or signal termination, duration, and bounded stdout and stderr; `timed_out` records its signal, duration, and bounded streams. `not_run` and `unavailable` record a structured reason. Every output-file observation retains its requested name and uses its discriminant to distinguish a collected `ArtifactReference` from missing, over-limit, non-regular, not-run, and unavailable outcomes. +| Source | Caller provides | Gateway resolves | +|---|---|---| +| Git | Canonical HTTPS repository URL, ref, and full or bounded shallow history | One exact commit | +| OCI | Canonical repository and exact image-manifest descriptor | That exact manifest and materialized filesystem | +| Uploaded bundle | Authorized immutable bundle reference | The declared artifact digest | +The caller cannot submit host paths, credentials, internal credential routes, runtime images, executable paths, service definitions, shell commands, raw network destinations, policy bodies, grading rules, patch requests, or workspace-persistence options. -A nonzero command exit, configured command timeout, missing file, non-regular file, or requested-file limit breach is raw evidence in a completed run. It is not an infrastructure or behavioral failure. Referenced-bundle failure, approved-executable resolution or launch failure, sandbox failure, inability to persist promised evidence, or cleanup failure is `infrastructure_error`. +## Source authorization and caching -Direct provenance contains the ordered canonical Git, OCI, and uploaded-bundle identities, access modes, materializer versions, destinations, and resolved digests. `AgentRunResult v1` has no imported-task or alternate-backend provenance variant. +For Git, the gateway maps the authenticated tenant and canonical repository URL to exactly one internal access and credential route. OCI uses the authenticated tenant and canonical repository in the same way. Zero matches or more than one match reject the request before credentials or network access. -V1 does not return a generic workspace diff, reconstructed or modified workspace, checkpoint, patch, or change artifact. Post-run mutations are permitted only in sources declared `writable` or in separate writable build paths; they are discarded with the sandbox and are not agent output or a persisted caller artifact. Every direct run removes its mutable workspace and releases cache leases before terminal result visibility. +V1 rejects Git and OCI redirects. An OCI route fixes the allowed token service. The worker fetches manifests and blobs through that route and rejects foreign blob locations. Remote content cannot redirect credentials to another target. -## Bundle and result-artifact API +The gateway caches immutable source generations: -Every bundle reservation, uploaded bundle, run, status, cancellation target, result artifact, and result is owned by one authenticated tenant. Lookups use `(authenticated tenant, object identity)` and cross-tenant access receives the same non-enumerating denial as an unknown object. UUIDs and digests are identities, never authorization. +- Git: canonical repository, resolved commit, history rule, and materializer version; +- OCI: canonical repository, exact descriptor, and materializer version; and +- uploaded bundle: artifact digest and materializer version. -`POST /v1/bundles` accepts the closed reservation `{schema_version: "bundle_reservation_request.v1", upload_id: UUIDv7, media_type: "application/vnd.allagents.gateway-bundle.v1.tar+gzip", digest: SHA-256 digest, size_bytes: positive integer}`. `(tenant, upload_id)` is idempotent: an exact retry returns the original immutable reservation, while reuse with different canonical content returns conflict. `PUT /v1/bundles/{artifact_id}` is tenant-authorized, accepts exactly the reserved byte count, verifies the streamed digest, and marks the bundle usable only after both checks pass. +Authorization is not part of the cache key. The gateway checks current authorization before every attachment, including cache hits. -`GET /v1/artifacts/{artifact_id}` is an authenticated, tenant- and run-authorized download of immutable result bytes. For an authorized live reference, `Content-Type`, `Content-Length`, and `Digest` match the `ArtifactReference`; after `expires_at` it returns `410 Gone`. Unknown, cross-tenant, wrong-run, and otherwise unauthorized identities use one non-enumerating denial. Artifact expiry and retention cover every artifact-backed agent output, command stream, requested file, and trajectory. +Concurrent requests for the same missing generation share one fetch and materialization. Complete generations are immutable. A run holds an eviction lease until cleanup finishes. -The Promptfoo provider dereferences every artifact it exposes to a grader through this endpoint, verifies those response headers, streams exactly `size_bytes`, computes the declared digest, and rejects expired, short, long, wrong-media-type, or mismatched content. Unverified artifact bytes never enter grading. +Read-only sources mount the cached generation directly and read-only. Writable sources receive a private filesystem copy-on-write clone when supported, with a full private copy as the fallback. The gateway never uses hardlinks or another shared writable alias. -## Isolation and trust guarantees +## Runtime and isolation -An authorized `runtime_profile_id` resolves during admission to a persisted immutable `profile_digest`; the worker executes only that pinned revision and rejects digest mismatch or unavailability. The revision fixes the runtime image digest, read-only runtime `PATH`, sandbox policy, tool implementation closures, and task-service implementations. `RuntimeProvenance` reports the logical ID and profile digest, image and sandbox identities, each tool's implementation digest, and each service's implementation digest plus container image digest where applicable. These are canonical nonsecret identities: tool digests cover the executable and immutable dependency closure, service implementation digests cover the canonical launch implementation, and secret arguments and host paths are never returned. +`runtime_profile_id` selects an operator-defined profile that the caller is allowed to use. Before dispatch, the gateway resolves it to one immutable `profile_digest` and stores that exact revision with the run. The revision fixes the runtime image, sandbox policy, approved tools, task-service implementations, network brokers, and resource ceilings. The worker refuses to run if the pinned content is missing or its digest changed. -Every direct execution uses the profile's `RunSandbox` security baseline: a non-root process under an unprivileged UID/GID mapping; an empty capability set with `no_new_privs`; private PID, mount, IPC, UTS, and network namespaces; a read-only runtime root; and only the declared workspace/build mounts writable. Host `/proc` and `/sys` surfaces, cgroup control files, device nodes, host sockets, container APIs, and control-plane mounts are masked or absent. A versioned explicit seccomp allowlist applies to agent and post-run processes. Cgroup v2 enforcement of PID count, memory, CPU, and I/O is mandatory in addition to wall-time, workspace, file-count, and output limits. A backend that cannot establish this baseline must reject the run rather than weaken isolation. +`AgentRunResult v1` reports the profile and immutable runtime, tool, and service identities that actually ran. It does not expose secret arguments, credentials, private endpoints, or host paths. -Each direct run receives a newly created sandbox and composed workspace. No mutable filesystem, process, home directory, or agent session is reused between runs. Concurrent runs may share only operator-owned immutable source generations; they never share writable workspace state, home state, process state, or post-run output. +Every run uses a new Linux sandbox with: -The gateway caches each admitted source as an immutable materialized generation. A Git generation key binds the canonical source origin, resolved exact commit, requested history mode and depth, and materializer-schema version. An OCI generation key binds the canonical repository, exact direct descriptor, and materializer-schema version. An uploaded bundle generation key binds its artifact digest and materializer-schema version. Authorization, internal policy identity, and credential route are deliberately separate from the content key. Before every attachment, including every cache hit, the gateway remaps the authenticated caller and canonical origin or repository and rechecks authorization for the exact resolved identity; bundle hits recheck artifact authorization. A cache key proves identity, not authorization, and a cache hit never bypasses current policy. +- an unprivileged, non-root user and no Linux capabilities; +- `no_new_privs` and an explicit system-call allowlist; +- private process, mount, inter-process communication, hostname, and network environments; +- a read-only runtime filesystem with only declared workspace and build locations writable; +- no host devices, container socket, control-plane mount, or writable cgroup control; and +- enforced process-count, memory, CPU, disk I/O, time, workspace, file-count, and output limits. -Exact-key misses, fetches, and unpacks are singleflighted so concurrent runs do not refetch or rematerialize the same large source. Complete generations are immutable and operator-owned. A run holds an eviction lease from attachment through cleanup. Cache roots and unrelated generations are inaccessible to the agent; a generation is visible only through its declared trial mount. +If the worker cannot establish this baseline, it rejects the run rather than weakening isolation. -A `read_only` source mounts the cached generation directly and read-only into every requesting trial. The filesystem rejects writes. Git operations that are valid against a read-only repository run with optional locks disabled; any Git operation requiring a lock or write fails rather than mutating shared state. A `writable` source receives a private reflink or other copy-on-write clone, with a full private copy as the correctness fallback. Hardlinks and any other shared writable alias to cached content are forbidden. +The agent and post-run commands use separate, deny-by-default network environments. Both external and local traffic must go through operator-managed endpoints allowed for that phase. The worker destroys the agent network environment and its active connections before creating the post-run environment. This prevents the agent from reaching checks or services that are available only after it exits. -Agent and post-run commands see the same per-source access modes. A check that compiles in-tree, installs dependencies into a source, creates a database there, or otherwise writes beneath a source must mark it `writable`; alternatively it must direct build and test output to a separate writable path. Read-only fixtures and repositories remain read-only throughout both phases. +Credentials never enter caller JSON, workspace files, logs, trajectories, or returned evidence. Source and model access use worker-owned mechanisms. Evidence commands receive no source or model credentials. -The gateway control plane remains outside worker, agent, and post-run command authority. Within a worker, the runner enforces process, time, CPU, memory, storage, output, and phase-specific network limits and can terminate the complete agent-owned descendant process tree. Gateway records, runner control files, result storage, and cleanup authority are inaccessible to the agent and evidence commands. +## Post-run evidence -Source, registry, model, optional check-bundle, and artifact credentials are resolved from operator-controlled configuration. They are absent from caller JSON, workspace contents, logs, trajectories, and returned evidence. Source acquisition and model authentication occur through gateway- and runner-owned mechanisms that do not disclose credentials to workspace commands. A referenced check bundle is authorized before the run, but its bytes and populated mount are absent from the agent sandbox; only an empty, runner-owned, non-writable reserved mountpoint may exist. Evidence commands receive no acquisition or model credentials. +`post_run: null` means the caller wants only the agent result. A non-null value asks the gateway to collect raw evidence; it does not ask the gateway to judge the run. -Network access is phase-specific, namespace-separated, brokered, and deny-by-default. Source acquisition follows the resolved internal source route. Agent execution runs in its own fresh network namespace under `agent_network_policy_id`; post-run evidence runs in a different fresh network namespace under `post_run.network_policy_id`. The default-drop rule covers external interfaces and loopback. A policy can expose only operator-declared broker endpoints for the phase, including individually named task-service endpoints; it cannot implicitly expose the worker host or another phase. The caller cannot submit hosts, ports, CIDRs, headers, proxies, service definitions, credentials, or policy bodies. +A post-run request may include: -The phase transition is ordered and fail-closed. The runner first terminates every agent-owned descendant, destroys the agent network namespace and all agent flows, removes model and source credentials from the runtime, environment, and runner-managed auth material, and verifies those steps. Only then may it materialize the authorized bundle into the reserved mountpoint read-only, expose it to post-run commands, and create the post-run network namespace with a newly resolved endpoint set. A task service may remain supervised across the transition, but it is unreachable unless the pinned runtime profile and active phase policy both authorize its broker endpoint. Namespace teardown, rather than a best-effort firewall rewrite, prevents an agent-opened loopback or sidecar connection from surviving. +- an immutable hidden check bundle; +- up to 32 ordered commands; +- up to 128 bounded workspace-relative output files; and +- a separately authorized post-run network policy. -The runner keeps the same pinned runtime and final workspace with declared source access modes alive across this transition. Runtime executables resolve only through the pinned tool mapping; bundle executables resolve only beneath the newly populated bundle root; arguments are literal and never interpreted by a shell. Commands may build in writable paths, write test artifacts, and exercise authorized task services, but cannot depend on arbitrary agent-owned background processes or mutate `read_only` sources. The runner records raw outcomes and only requested bounded paths. Referenced-bundle acquisition failure or runner failure is `infrastructure_error`; a command's own nonzero exit or timeout remains completed-run evidence. +Each command selects either an approved tool from the pinned runtime profile or an executable inside the hidden bundle. Arguments are literal strings. The gateway does not invoke a shell, expand variables or globs, accept a command-specific working directory, or accept a command-specific network policy. -Cleanup is mandatory, idempotent, and part of terminalization. After persisting available evidence, the worker stops every remaining evidence process and operator task service, unmounts every trial mount, deletes the private trial root, and releases every cache lease. The gateway does not expose a terminal result until all cleanup steps succeed. If any cleanup step initially fails, the run's eventual status is `infrastructure_error`, not `completed`; its available evidence is marked partial, and reconciliation finishes cleanup without rerunning the agent before the terminal error becomes visible. +The hidden bundle is absent while the agent runs. The worker may create an empty reserved mount point, but it does not fetch or mount the bundle until the agent and its descendants are gone, the agent network environment is destroyed, and credentials are removed. -## V1 execution scope +Evidence commands run in the same pinned runtime and final workspace so they can use installed dependencies and declared task services. Source access remains unchanged: a command cannot write to a `read_only` source. -V1 supports only direct execution. The AllAgents Gateway admits and tracks the run; a worker and runner own ordered workspace composition, sandboxing, Codex or OMP execution, the credential-stripping phase transition, optional post-run bundle injection, runtime or bundle evidence commands in the same pinned environment, raw result construction, cleanup, and terminalization. +A command's nonzero exit or configured timeout is raw evidence in a `completed` run. A failure to create the sandbox, mount the bundle, resolve or start an approved executable, store promised evidence, or clean up is an `infrastructure_error`. -Harbor and Terminal-Bench imports are deferred entirely beyond V1. Any future adapter requires a separate ADR and a new request/result schema version that defines its ownership, provenance, isolation, and evidence semantics. V1 makes no Harbor source, backend, provenance, or result-mapping promise. +The worker records one observation for every requested command and file, in request order. Cancellation or infrastructure failure preserves observations already stored and marks later work as not run or unavailable. It never silently drops or reorders partial evidence. -## Idempotency, retries, and cancellation +## Results and artifacts -`run_id` identifies the logical run and `idempotency_key` protects submission. After authentication, closed-schema validation, and canonical request hashing—but before mutable source admission, capacity allocation, artifact consumption, or creation of a public run—the gateway looks up both identities under the authenticated tenant. +`AgentRunResult v1` reports gateway lifecycle, not task quality: -- An exact tenant-scoped duplicate returns the same run handle or terminal result without re-running current admission or starting new work, including after client timeout, policy change, or worker restart. -- Reusing either identity for a different canonical request is a conflict and starts nothing. -- A genuinely new request first acquires an internal tenant-scoped identity claim so concurrent duplicates converge. The gateway then performs every policy, bundle, source, limit, runtime-profile, and capacity admission check. A rejected new request releases that claim and leaves no public run or job record. -- Only successful admission atomically commits the immutable canonical request and public run before dispatch. Recovery can finish dispatch from that record but cannot admit or start a second agent. -- Promptfoo repetitions use distinct identities even when every evaluation input is otherwise identical. +| Status | Meaning | +|---|---| +| `completed` | The agent ran and requested evidence collection finished. A command may still have failed or timed out. | +| `cancelled` | Cancellation won before the terminal result was published. | +| `infrastructure_error` | The gateway could not safely complete the run lifecycle. | -Transport retries and result polling are safe because they do not rerun the agent. The gateway may reconcile or resume its durable orchestration around a running request, but no worker or runner automatically starts the agent a second time after an attempt has begun. An infrastructure failure is terminal for that run. An intentional rerun requires a new run identity and therefore a fresh workspace; the caller, not the gateway, decides whether to schedule it. +A completed result includes source and runtime provenance, bounded agent output, usage, timing, and a valid bounded trajectory. It includes post-run observations when requested. The gateway never returns behavioral pass/fail or reward. -Cancellation is an idempotent request against `run_id`. Before execution it prevents agent start; during execution it terminates the full descendant process tree; during post-run collection it terminates evidence commands. Once cancellation wins the terminal-state race, the worker preserves bounded partial evidence and performs the same process/service stop, unmount, private-root deletion, and cache-lease release. `cancelled` becomes visible only after cleanup succeeds; any cleanup failure changes the eventual status to `infrastructure_error` after reconciliation finishes cleanup. If a terminal result was published first, later cancellation returns that unchanged result. Cancellation never preserves a workspace for continuation. +### Where transcripts and traces come from -Failures in admission, materialization, sandbox control, agent launch or supervision, referenced check-bundle acquisition, evidence-command supervision, artifact storage, cleanup, and result commitment use stable structured error codes. A `retryable` flag tells the caller whether a new run might succeed; it never authorizes the gateway or worker to silently rerun the agent. Cleanup failure produces `infrastructure_error` with partial evidence after reconciliation completes cleanup. The gateway never converts infrastructure failure or command evidence into pass, fail, or reward. +HarnessRouter is no longer in the trace path. While Codex or OMP runs, its adapter reads the structured events emitted by that pinned agent CLI and converts them incrementally to ATIF v1, the public trajectory format for V1. The runner stops recording at `max_trace_bytes` and adds a valid truncation marker rather than producing an invalid or unbounded document. -## Rejected alternatives +```mermaid +flowchart LR + A[Codex or OMP native events] --> N[Adapter normalizes to ATIF v1] + N --> B[Runner applies trace byte limit] + B --> I{Fits inline?} + I -->|Yes| R[AgentRunResult trajectory] + I -->|No| O[Result artifact reference] + O --> V[Provider downloads and verifies] + V --> P[Promptfoo] + R --> P +``` + +The agent's final answer is captured separately as bounded final output. V1 returns a trajectory for one run; it does not preserve a HarnessRouter-style session transcript or enough state to resume the conversation. The native vendor event stream is normalized rather than exposed as a second public trace format. + +All captured data is bounded. The request reserves enough artifact capacity for the declared output-file maxima. Actual output files use that capacity first, followed by agent output, command output in request order, and trajectory data. If optional artifact storage is exhausted, the gateway returns bounded inline, truncated, or explicitly omitted data rather than storing an unbounded value. + +Every upload, run, cancellation target, result, and result artifact belongs to one authenticated tenant. Object lookup always includes that tenant. A cross-tenant identifier is treated like an unknown identifier; UUIDs and digests are not authorization. + +Result artifact references include media type, digest, byte size, and expiry. The authenticated artifact endpoint serves only unexpired result artifacts belonging to the caller's run. The Promptfoo provider verifies the returned media type, length, and digest before passing bytes to a grader. Input and hidden-check bundles are never exposed through the result-artifact endpoint. -### UHP through HarnessRouter +## Retries, cancellation, and cleanup -UHP and HarnessRouter are designed around reusable harness sessions: response and session identity, continuation, checkpoint and hydrate, produced-file collection, and session deletion. The AllAgents Gateway is similar in purpose but V1 needs one isolated run, captured agent output, and optional raw post-run evidence. Adopting the session protocol would force the gateway to define which state survives, how continuation interacts with caller retries, and how checkpoints and produced files become run results even though none are required. +After authentication and schema validation, the gateway checks `run_id` and `idempotency_key` before source admission or resource allocation. -A HarnessRouter fork would also make AllAgents carry upstream integration seams and release work for workspace initialization, journaling, and terminal finalization. The former issue #304 proposal pursued those seams for the superseded snapshot design. Borrowing its sound protocol habits is useful; retaining its execution architecture is not. +- An exact duplicate returns the existing run without starting another agent. +- Reusing either identity for a different request returns a conflict and starts nothing. +- A new request is admitted before a public run is created. A rejected request leaves no public run or job. +- Once the worker may have started the agent, recovery never starts it again. If the worker cannot prove that the agent was never invoked, the run ends with `infrastructure_error`. -We therefore reject UHP conformance, a HarnessRouter distribution or fork, checkpoints, continuation, and reusable coding sessions for this system. +Retries of uploads, submission, polling, cancellation, and artifact downloads reuse the same identities. Promptfoo repetitions use new identities because they are intentional new attempts. -### Two repositories and published workspace snapshots +Cancellation stops the current agent or evidence process and preserves bounded partial evidence. The worker then performs the same cleanup as any other run. `cancelled` becomes visible only after cleanup succeeds. A cleanup failure changes the eventual result to `infrastructure_error`. -Separating workspace construction into a builder repository and passing published OCI snapshots into a session gateway optimized an imagined reuse boundary. In a one-shot run, the worker already owns acquisition, isolated materialization, execution, optional evidence collection, and destruction as one lifecycle. Splitting that lifecycle adds artifact publication, leases, cross-repository versioning, cache authorization, and recovery states without improving the run result. +Cleanup is part of terminalization. Before publishing any terminal result, the worker must stop all remaining processes and services, remove mounts and private files, and release cache leases. Cleanup can be retried safely, but the gateway cannot publish a successful result while cleanup remains incomplete. -OCI remains a valid immutable workspace input, not the mandatory handoff between two AllAgents systems. One repository keeps the provider, public contract, lifecycle, and agent adapters versioned and tested together. +## V1 scope -### Generic workspace diffs as product output +V1 supports direct Codex and OMP execution only. -A generic diff says which bytes changed, not what the agent reported or what checks observed. It is ambiguous around generated files, ignored files, nested repositories, modes, links, and task-specific equivalence; it can also expose unnecessary source content. Making diffs authoritative would require baseline retention and reconstruction machinery that a one-shot result does not need. +It does not include: -For Promptfoo, code or LLM graders consume agent output, raw command results, and requested file artifacts and emit the task-specific outcome and reward outside the gateway. Other callers consume the same uninterpreted evidence. No caller receives modified workspace state in V1. Any future patch or workspace export requires a separately approved, explicit artifact contract rather than an extension of the core result by convention. +- UHP or a HarnessRouter fork; +- reusable or interactive sessions; +- continuation, checkpoints, or workspace recovery; +- generic diffs, patches, or modified-workspace export; +- Harbor, Terminal-Bench, or SWE-bench adapters; or +- gateway-owned grading. + +A future benchmark adapter or workspace-export feature requires a separate decision and a new closed contract. OCI remains a supported workspace input; it is not a mandatory snapshot handoff between two AllAgents services. ## Consequences -Positive consequences: - -- the architecture supports a general one-shot coding-agent gateway while matching Promptfoo's unit of work: one independently graded run; -- a reader can locate authoring and grading in Promptfoo, public lifecycle in the gateway, and execution and raw evidence collection in a worker and runner; -- every run starts clean and cannot inherit a prior coding session; -- concurrent runs can reuse large immutable Git, OCI, and bundle generations without sharing mutable trial state or refetching exact sources; -- idempotent network retries cannot duplicate agent work; -- gateway lifecycle remains separate from caller-owned behavioral grading; -- credentials and operator policy stay on the operator side of a small closed boundary; -- Codex and OMP share isolation, output, evidence, and lifecycle semantics without pretending their CLIs are identical. - -Costs and constraints: - -- workers and the runner must implement strong sandbox, process-tree, credential, network, artifact, and cleanup controls rather than inheriting them from a session service; -- there is deliberately no interactive continuation, modified-workspace export, or post-run workspace recovery; -- local inputs and check bundles incur a packaging and upload step before remote execution; -- immutable source generations require bounded cache storage, eviction leases, singleflight recovery, authorization on every attachment, and materializer-schema migrations; -- callers must choose source access deliberately; agents and checks that write in-tree require `writable`, while `read_only` sources require a separate writable build path; -- check bundles and evidence commands are security-sensitive executable inputs and require authorization, immutability, isolation, and resource bounds; -- closed contracts require explicit versioning when new workspace, agent, post-run evidence, or artifact capabilities are introduced; and -- debugging and grading rely on bounded agent output, trajectory, command results, and file artifacts because the mutable workspace is destroyed. - -We will reconsider this decision if the product requires reusable interactive coding sessions rather than one-shot runs, or if direct execution can no longer provide the required isolation guarantees. A need for more Promptfoo matrices, grader types, benchmark adapters, or raw evidence does not by itself justify UHP sessions, HarnessRouter, snapshot publication, gateway-owned grading, or generic workspace diffs. +Benefits: + +- each run starts clean and cannot inherit another run's mutable state; +- network retries cannot duplicate agent work; +- concurrent runs can reuse exact immutable sources without sharing writable state; +- Promptfoo grading stays separate from gateway lifecycle; +- Codex and OMP share one isolation and result contract; and +- credentials and operator policy remain outside the caller-controlled request. + +Costs: + +- AllAgents must build and operate strong sandbox, network, credential, artifact, and cleanup controls; +- local inputs and hidden checks must be packaged and uploaded; +- immutable source caching requires storage limits, leases, safe eviction, recovery, and authorization on every attachment; +- callers must choose `read_only` or `writable` correctly; and +- debugging relies on bounded output, trajectory, command results, and requested files because the workspace is destroyed. + +We will reconsider this decision if the product needs reusable interactive coding sessions or if direct execution cannot provide the required isolation. More Promptfoo matrices, graders, benchmark formats, or raw evidence do not by themselves justify sessions, snapshots, generic diffs, or gateway-owned grading. From 5710ab2404f2cd90e6cb4b0e4b477481a8ae984a Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 13:55:57 +1000 Subject: [PATCH 41/44] docs(research): remove discarded E2B analysis --- .../e2b-execution-gateway-patterns.md | 198 ------------------ .../one-shot-coding-agent-gateway-boundary.md | 8 +- .../research/workspace-contract-incumbents.md | 10 +- 3 files changed, 4 insertions(+), 212 deletions(-) delete mode 100644 docs/research/e2b-execution-gateway-patterns.md diff --git a/docs/research/e2b-execution-gateway-patterns.md b/docs/research/e2b-execution-gateway-patterns.md deleted file mode 100644 index 85b420c5..00000000 --- a/docs/research/e2b-execution-gateway-patterns.md +++ /dev/null @@ -1,198 +0,0 @@ -# E2B execution gateway patterns - -## Scope and evidence date - -This note describes E2B's public product and source as of **2026-09-22**. Source links to code pin runtime commit [`9dd5b72`](https://github.com/e2b-dev/runtime/tree/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8) (2026-09-22) and SDK commit [`ccaf9fc`](https://github.com/e2b-dev/E2B/tree/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b) (2026-09-18). Links to unversioned official product documentation were accessed 2026-09-22. - -## Product boundary - -E2B exposes **sandbox infrastructure**, not an agent-neutral execution protocol. Its public contract creates and controls Linux sandboxes, runs commands and PTYs, reads and writes files, exposes guest ports, and manages templates, snapshots, and volumes. The official [coding-agent guide](https://docs.e2b.dev/use-cases/coding-agents.md) requires the application to install its chosen agent in a template and extract the resulting diff or files. The [Codex integration](https://docs.e2b.dev/agents/codex.md) likewise creates a sandbox, passes a Codex credential, clones a repository, invokes `codex exec`, interprets Codex-specific output, retrieves a diff, and kills the sandbox in caller code. The [public OpenAPI document](https://docs.e2b.dev/openapi-public.yaml) describes sandbox infrastructure resources rather than a common agent run/session/event/result model. - -That makes E2B usable as a sandbox provider beneath an execution gateway. Repository acquisition, harness selection, credential policy, prompt construction, reconnect/retry behavior, normalized events, terminal outcomes, and result provenance remain responsibilities above E2B. - -## Runtime architecture - -The open [`e2b-dev/runtime`](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/README.md) repository describes itself as the complete backend used by E2B Cloud, Enterprise, and Embed. Its [architecture specification](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/ARCHITECTURE.md) defines these boundaries: - -- **API/control plane:** authenticates callers, enforces quotas, places sandboxes, and records durable and ephemeral state. -- **Orchestrator:** runs on each KVM host and owns Firecracker, cgroups, network namespaces, veth/tap devices, NBD-backed root filesystems, snapshots, caches, and template builds. -- **Client proxy/data plane:** routes sandbox traffic directly to the selected orchestrator; guest traffic does not pass through the control-plane API. -- **`envd`:** runs inside each microVM and exposes authenticated process, PTY, filesystem, watcher, upload/download, signal, and port APIs. Public proxying rejects internal init, upgrade, freeze, and thaw routes. -- **State stores:** PostgreSQL holds durable metadata; Redis holds running-sandbox and routing state; ClickHouse holds events, metrics, and optionally logs; object storage holds template and paused-sandbox artifacts. - -Each sandbox is one Firecracker microVM with its own guest kernel. The Firecracker process gets its own cgroup and network namespace; the host supplies a COW root filesystem, memory restore, and network policy. This is a stronger guest boundary than a shared-kernel container, while still trusting a privileged, root-running host orchestrator and the host kernel/KVM boundary ([architecture](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/ARCHITECTURE.md), [security](https://e2b.dev/security)). - -## Lifecycle, templates, and persistence - -A template is a pre-booted snapshot of memory, filesystem, and machine state. Template builds execute layered phases, hash step inputs, cache reusable layers, and produce immutable artifacts. Sandbox creation restores that snapshot, lazily faults memory through `userfaultfd`, and overlays root filesystem writes. The result is fast creation without making the container image or install recipe the live runtime boundary ([architecture](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/ARCHITECTURE.md), [template mechanics](https://docs.e2b.dev/template/how-it-works.md), [cache semantics](https://docs.e2b.dev/template/caching.md)). - -E2B distinguishes three related artifacts: - -- **Declarative template:** reproducible start state built from a recipe; the preferred durable baseline. -- **Paused sandbox:** preserves filesystem, memory, and processes until explicitly killed in managed E2B ([persistence](https://docs.e2b.dev/sandbox/persistence.md)). -- **Live snapshot/fork:** captures a running sandbox as a reusable checkpoint; active PTY, WebSocket, and command streams disconnect and must be re-established ([snapshots](https://docs.e2b.dev/sandbox/snapshots.md)). - -Template and paused-sandbox storage use the same broad artifact shape: memory, rootfs, VM state, metadata, and memory/rootfs index files. Persistent volumes are a separate, private-beta resource, and their content path is served by a separate `belt` API rather than the main control-plane API ([volumes](https://docs.e2b.dev/volumes.md), [architecture](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/ARCHITECTURE.md)). E2B's file API transfers and manipulates guest files, but it does not define an agent-run artifact manifest or source/workspace provenance model ([filesystem upload](https://docs.e2b.dev/filesystem/upload.md)). - -## SDK and API shape - -The JavaScript and Python SDKs expose sandbox lifecycle, commands/process streaming, PTYs, filesystem operations, networking, templates, snapshots, volumes, and secrets. Callers can target another deployment through `E2B_API_URL`, `E2B_SANDBOX_URL`, an API key, or an explicit client/domain ([connection configuration](https://github.com/e2b-dev/E2B/blob/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b/packages/js-sdk/src/connectionConfig.ts), [custom client](https://docs.e2b.dev/client.md)). This is a clean provider seam, but it is a sandbox API seam rather than an agent/harness abstraction. - -Version compatibility needs active management. SDK release [`e2b@2.51.0`](https://github.com/e2b-dev/E2B/releases/tag/e2b%402.51.0) (2026-09-18) moved create/connect behavior onto v2 endpoints. The SDK [changelog](https://github.com/e2b-dev/E2B/blob/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b/packages/js-sdk/CHANGELOG.md) also warns that an older self-hosted/BYOC control plane can silently ignore a newer resume option. A provider integration therefore needs an explicit tested SDK/control-plane compatibility range. - -## Deployment and self-hosting boundary - -“Self-hosted E2B” currently describes more than one materially different operating model: - -| Mode | Where it runs | Who operates it | Current boundary | -| --- | --- | --- | --- | -| E2B Cloud | E2B account | E2B | Managed control and data planes. | -| BYOC | Customer AWS or GCP account/VPC | E2B | E2B provisions, monitors, and upgrades it; E2B Cloud remains the management/control plane. Official docs explicitly say this is managed deployment, not self-hosting ([BYOC](https://docs.e2b.dev/byoc.md), [security](https://e2b.dev/security)). | -| E2B Embed | One operator-owned KVM machine | Operator | The whole functional stack and sandboxes run locally; available as Compose, one-GCE-instance Terraform, or a single-node Kubernetes StatefulSet ([Embed](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/README.md)). | -| Private Cloud | Customer environment | Customer/E2B contract not yet public | Listed as “in development”; intended to keep both planes within the customer boundary ([enterprise](https://e2b.dev/enterprise)). | - -E2B is therefore self-runnable today as a complete **single-node functional stack**, but the public package is not a complete, supported **production multi-node self-operated distribution**. The Embed repository calls Compose, GCP Terraform, and Kubernetes “single-machine evaluation packages, not deployment patterns.” The current enterprise page calls Embed available and suitable for self-hosting/embedding while separately listing Private Cloud as in development. Both statements matter: availability does not establish HA, production operations, or air-gap support. - -### E2B Embed operational facts - -The [Compose guide](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/compose/README.md) requires Linux, KVM, `/dev/net/tun`, cgroup v2, NBD, 4 KiB pages, hugepages, Docker Engine 27+, Compose 2.24+, about 12 GiB RAM, and 20 GiB free disk. It supports x86-64 and arm64, with newer arm64 kernel requirements. It is not a rootless or container-only Firecracker deployment. - -The [reference](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/docs/REFERENCE.md) shows that Embed includes local PostgreSQL, Redis, ClickHouse, Vector, dashboard/API, client proxy, template builder, orchestrator, and local template/build storage. Logs remain in local ClickHouse for seven days. Running sandboxes end when the orchestrator stops; the launcher now sweeps sandbox cgroups on termination and after a crash. - -Durability depends on packaging: - -- Compose named volumes and local storage survive ordinary stack shutdown, but running VMs do not. -- The [Kubernetes package](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/kubernetes/README.md) is a privileged, host-network/host-PID StatefulSet pinned to one labelled KVM node. It uses node-local `hostPath`; sandboxes end on pod deletion/restart, and the guide calls for a dedicated node. -- The [GCP Terraform package](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/terraform/gcp/README.md) creates a managed instance group of one with no persistent data disk. Instance replacement loses databases and built templates. - -The default installation is not turnkey air-gapped. Initial Compose setup fetches images from Docker Hub and Google Artifact Registry and binaries from Google Storage/GitHub; template builds normally pull a base image. Published binaries are pinned and accompanied by SHA-256 files, but the public runtime repository is a read-only mirror of E2B's internal source-of-truth monorepo ([release process](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/RELEASING.md), [Embed reference](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/docs/REFERENCE.md)). No official offline-mirroring deployment procedure was found in the current public documentation. - -### Managed-feature parity - -The open runtime is substantial, but standard Embed is not configuration-equivalent to E2B Cloud/BYOC: - -- Embed's [Compose definition](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/compose/compose.yaml) does not configure the separate secret-store backend and explicitly disables volume-content token support. The API returns an error when the secret backend/feature is unavailable ([secret handler](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/packages/api/internal/handlers/secrets.go)). -- The SDK documents SOCKS5 egress proxying as Cloud/BYOC functionality that an open-source runtime deployment rejects ([SDK source](https://github.com/e2b-dev/E2B/blob/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b/packages/js-sdk/src/sandbox/sandboxApi.ts)). -- Workload-identity definitions can cross the open API/orchestrator contract, but the open [orchestrator protocol](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/packages/orchestrator/orchestrator.proto) explicitly says it does not mint, sign, or deliver the credential. - -These are feature-boundary facts, not evidence that the single-node runtime is a stub: it can build templates and run real Firecracker sandboxes through the same SDK surface. - -## Security and networking boundary - -The Firecracker/KVM boundary is complemented by per-sandbox cgroups, namespaces, NBD devices, NAT, nftables, and token-authenticated `envd`. Public sandbox ingress can require an access token. Those controls do not remove operator obligations around the privileged host and management network. - -Embed deliberately exposes a low-level local stack. Its [README](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/README.md) says 13 ports bind all interfaces. Only API, dashboard, and client proxy ports 3000–3002 are for trusted clients; the other ten must be firewalled. In particular, orchestrator gRPC on port 5008 is unauthenticated and grants full orchestrator control. Embed does not supply wildcard DNS or TLS. Its plain-HTTP dashboard configuration cannot mark the team-key cookie `Secure` ([reference](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/docs/REFERENCE.md)). - -Outbound internet access defaults on. E2B supports IP/CIDR allow and deny lists plus HTTP Host/TLS SNI domain allowlists, but its [network documentation](https://docs.e2b.dev/network/internet-access.md) records important limits: domain filtering sees Host only on HTTP/80 and SNI only on TLS/443; it does not cover UDP/QUIC or arbitrary ports; allow wins over deny; shared CDN/IP use weakens domain isolation; and a blocked TCP connection can appear established until application data is attempted. E2B itself says a domain allowlist is a routing control, not a strict security boundary on shared infrastructure. - -## Operations and observability - -Runtime services emit OpenTelemetry. Orchestrators publish lifecycle events and host statistics; ClickHouse stores metrics/events and optionally logs. Production architecture supports centralized observability, while Embed intentionally uses local Vector → ClickHouse with a fixed seven-day retention and no Loki or LaunchDarkly dependency ([architecture](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/docs/ARCHITECTURE.md), [Embed reference](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/docs/REFERENCE.md)). - -A 2025 [self-hosting report](https://github.com/e2b-dev/runtime/issues/1421) documented orphaned Firecracker processes/veth state after an orchestrator crash, cache-locality concerns, and difficult Kubernetes resource accounting. At the time, E2B recommended replacing the node. Current runtime code adds startup reclaim and single-instance locking, and Embed's launcher sweeps cgroups before restart. The history is still useful: privileged node reconciliation, cache placement, and capacity accounting are production concerns, not incidental packaging details. - -## Patterns supported by the evidence - -Patterns that can be evaluated independently of an E2B adoption decision: - -1. **Separate lifecycle control from sandbox traffic.** Keep placement, quotas, and durable state in the control plane; route high-volume process/file/port traffic directly through a data-plane proxy. -2. **Hide host mechanics behind a node-local orchestrator.** The gateway should not understand NBD, Firecracker, cgroups, namespaces, or snapshot files. -3. **Use an explicit lifecycle state machine.** Running, pausing, paused, resuming, snapshotting, forking, and killed states need durable identities and well-defined terminal behavior. -4. **Separate reproducible templates from live checkpoints.** A build recipe and a memory-preserving snapshot answer different provenance and recovery questions. -5. **Make cold-start optimizations content-addressed.** Hash steps and inputs, share immutable layers, prefetch likely pages, and use COW overlays rather than copying a workspace/rootfs on every start. -6. **Keep the guest API narrow.** Authenticated process/PTY and filesystem operations are a useful runtime primitive; private init/upgrade/checkpoint routes should not share the public proxy path. -7. **Treat networking as a first-class per-run contract.** Record ingress authentication and egress policy with the run. E2B's documented domain-filter limits show why policy claims must match enforcement layers. -8. **Design crash reconciliation with the runtime.** Startup reclaim, idempotent teardown, single-instance locks, and explicit cache/storage recovery belong in the node contract before multi-node production use. -9. **Expose runtime telemetry without making it the agent protocol.** Lifecycle events, resource metrics, logs, and trace context should correlate with a gateway run ID, while normalized agent events remain above the sandbox provider. -10. **Pin a provider compatibility matrix.** SDK, API, guest daemon, kernel, Firecracker, and template versions change on different cadences; deployment provenance needs immutable versions and checksums. - -Patterns that should remain above any E2B provider adapter are harness/provider routing, credentials and source authorization, workspace provenance, event normalization, completion taxonomy, result/artifact manifests, retries/idempotency, and cross-provider conformance. - -## Licensing and current activity - -The runtime and dashboard repositories use Apache-2.0 ([runtime license](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/LICENSE), [dashboard license](https://github.com/e2b-dev/dashboard/blob/main/LICENSE)). The JavaScript and Python SDK package licenses are MIT ([JS SDK](https://github.com/e2b-dev/E2B/blob/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b/packages/js-sdk/LICENSE)). Runtime `main` had changes on 2026-09-22, runtime release [`2026.30`](https://github.com/e2b-dev/runtime/releases/tag/2026.30) was published 2026-09-10, and SDK release [`e2b@2.51.0`](https://github.com/e2b-dev/E2B/releases/tag/e2b%402.51.0) was published 2026-09-18. Embed itself is recent and still explicitly framed as evaluation packaging in source. - -## Primary sources - -- [Runtime repository and architecture](https://github.com/e2b-dev/runtime/tree/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8) — source dated 2026-09-22. -- [SDK repository](https://github.com/e2b-dev/E2B/tree/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b) — source dated 2026-09-18. -- [E2B Embed](https://github.com/e2b-dev/runtime/tree/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed) — source dated 2026-09-22. -- [Official E2B documentation index](https://docs.e2b.dev/llms.txt) — accessed 2026-09-22. -- [Enterprise deployment options](https://e2b.dev/enterprise), [BYOC](https://docs.e2b.dev/byoc.md), and [security](https://e2b.dev/security) — accessed 2026-09-22. -- [Runtime release 2026.30](https://github.com/e2b-dev/runtime/releases/tag/2026.30) — published 2026-09-10. -- [SDK release 2.51.0](https://github.com/e2b-dev/E2B/releases/tag/e2b%402.51.0) — published 2026-09-18. - -## AllAgents comparison and decision - -### Verdict - -**Do not add E2B to the default gateway backend.** -`allagentsdev/allagents-gateway` can use local disposable containers or -processes, a policy-bound immutable source cache, direct read-only mounts, -private CoW/full-copy writable views, and required operator-authorized immutable -runtime profiles. E2B remains an optional worker backend only if a later -requirement needs its microVM boundary or managed sandbox lifecycle. - -This conclusion follows the boundary in -[One-shot coding-agent gateway boundary](./one-shot-coding-agent-gateway-boundary.md) -and [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). - -E2B still does not own the gateway abstraction. It creates a sandbox and exposes -process, filesystem, network, and lifecycle APIs. The AllAgents gateway owns the -normative `AgentRun v1` contract, source materialization, runtime-profile -resolution, agent launch, phase-separated isolation, lifecycle fencing, bounded -raw evidence, authenticated artifact service, and cleanup. Promptfoo is its -first caller and solely owns evaluation and grading. - -| Concern | One-shot coding-agent gateway | E2B | -| --- | --- | --- | -| Caller policy | Promptfoo is the first caller and owns evaluation policy | No experiment matrix or assertion owner | -| AgentRun | One public request, fresh internal trial, one agent attempt, then verified deletion | Sandbox lifecycle and guest APIs | -| Sources | Reauthorize exact cached generation; direct read-only mount or private CoW/full copy | Caller uploads or acquires content | -| Runtime | Required authorized `runtime_profile_id`; provenance pins profile/image, tool/service implementation, and applicable service-image digests | Caller selects/builds template | -| Checks | Agent process/cgroup and netns gone, descendants absent, credentials removed, then optional bundle bytes materialized and structured commands run in the retained runtime/final workspace | Caller-defined commands | -| Network | Phase-separated default-drop namespaces; named agent/post-run policy | Caller-defined sandbox networking | -| Evidence | Bounded `CapturedText` plus full request-order complete/partial `PostRunEvidence` | Caller-defined files and command output | -| Artifacts | Expiring tenant/run-authorized references; provider verifies streamed size/digest | Caller-defined download/storage | -| Lifecycle | Seal evidence, clean trial and release leases, then publish `completed`, `cancelled`, or `infrastructure_error`; cleanup failure retains partial evidence | Caller-defined | -| Judgment | Promptfoo code or LLM graders | Caller-defined | -| Adoption | Default local worker backend | Optional backend after a concrete isolation need | - -### Is it completely self-hosted? - -The answer depends on the operating standard: - -- **Yes for a functional single-node deployment.** Apache-2.0 E2B Embed runs the API, dashboard, proxies, PostgreSQL, Redis, ClickHouse, Vector, template builder, Firecracker orchestrator, storage, and sandboxes on an operator-owned KVM host. It generates its own team API key and stores its runtime data locally. -- **No for a turnkey production-equivalent distribution.** The source guides call every Embed shape a “single-machine evaluation package, not a deployment pattern.” Compose, the one-node Kubernetes StatefulSet, and the one-instance GCP module do not establish HA, multi-node recovery, or production scaling. -- **No for managed-feature parity.** Standard Embed does not configure every managed backend. Current gaps include the separate secret store, volume-content service/token path, managed SOCKS5 egress, and credential delivery for workload identity. -- **No for turnkey air-gapped installation.** Installation fetches pinned public images and binaries from Docker Hub, Google Artifact Registry/Storage, and GitHub; no current public offline-mirroring guide was found. -- **BYOC is not self-hosting.** E2B provisions, monitors, and operates BYOC in the customer's AWS or GCP account, while E2B Cloud remains part of its management plane. E2B's fully private production option is listed as in development. - -The [enterprise page](https://e2b.dev/enterprise) markets Embed as an available self-hosting pattern. The [Embed source guide](https://github.com/e2b-dev/runtime/blob/9dd5b727318831ebbdd84cc9f51b25c5f3af96c8/embed/README.md) sets the narrower operational boundary. For architecture decisions, use the source guide's single-node/evaluation limit. - -### Adopt the cache pattern without E2B - -Do not add an E2B dependency, generic sandbox-provider interface, guest daemon, -memory snapshots, forking, or multi-node placement. Do adopt the evidence-backed -mechanism directly: content-address exact immutable source generations, share -read-only layers, and give writable trials private CoW overlays/clones with a -full-copy fallback. E2B's template build cache and overlay-backed sandbox starts -are precedent for that mechanism, not a reason to adopt its platform. -The local disposable gateway worker remains the default backend. - -If mutually untrusted tenant code or measured sandbox-start requirements later -outgrow that boundary, trial E2B behind the normative -[`AgentRun v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#public-contract) -contract. It must preserve required authorized runtime profiles and their -profile/image, tool/service implementation, and applicable service-image -digests; fresh workspaces and declared source modes; verified agent -process/netns teardown before hidden-bundle -materialization, phase-separated default-drop networking, bounded -complete/partial evidence, tenant/run-bound artifact access, and cleanup/lease -release before terminal-result publication. E2B remains only an execution -backend, never another caller/result protocol or grading layer. - -Useful patterns to carry forward without adopting E2B are narrow guest APIs, -explicit lifecycle states, immutable environment identity, credential -containment, and orphan cleanup. Live VM snapshots remain recovery artifacts, -not proof that a coding task passed. diff --git a/docs/research/one-shot-coding-agent-gateway-boundary.md b/docs/research/one-shot-coding-agent-gateway-boundary.md index 5531d792..ab2114b6 100644 --- a/docs/research/one-shot-coding-agent-gateway-boundary.md +++ b/docs/research/one-shot-coding-agent-gateway-boundary.md @@ -187,12 +187,8 @@ bundles, post-run mutations, credentials, processes, and service state remain trial-private. Cleanup removes those trial views but not the immutable cache generation. -This adopts the useful cache mechanism, not E2B itself. E2B hashes template -steps, caches reusable immutable layers, and overlays writes for new sandboxes -([template mechanics](https://docs.e2b.dev/template/how-it-works.md), -[cache semantics](https://docs.e2b.dev/template/caching.md)); Linux reflinks -provide the local copy-on-write primitive with same-filesystem constraints -([`FICLONE`](https://man7.org/linux/man-pages/man2/ioctl_ficlonerange.2.html)). +Linux reflinks provide the local copy-on-write primitive, with same-filesystem +constraints ([`FICLONE`](https://man7.org/linux/man-pages/man2/ioctl_ficlonerange.2.html)). ## Authenticated private precedent diff --git a/docs/research/workspace-contract-incumbents.md b/docs/research/workspace-contract-incumbents.md index 0073aa25..5d860e90 100644 --- a/docs/research/workspace-contract-incumbents.md +++ b/docs/research/workspace-contract-incumbents.md @@ -113,13 +113,11 @@ This is the most credible portable standard for a possible future **development- Primary evidence: pinned [normative specification](https://github.com/devcontainers/spec/blob/c95ffeed1d059abfe9ffbe79762dc2fa4e7c2421/docs/specs/devcontainer-reference.md), [JSON Schema](https://github.com/devcontainers/spec/blob/c95ffeed1d059abfe9ffbe79762dc2fa4e7c2421/schemas/devContainer.base.schema.json), [field and lifecycle reference](https://github.com/devcontainers/spec/blob/c95ffeed1d059abfe9ffbe79762dc2fa4e7c2421/docs/specs/devcontainerjson-reference.md), [supporting tools](https://github.com/devcontainers/spec/blob/c95ffeed1d059abfe9ffbe79762dc2fa4e7c2421/docs/specs/supporting-tools.md), and [contribution process](https://github.com/devcontainers/spec/blob/c95ffeed1d059abfe9ffbe79762dc2fa4e7c2421/CONTRIBUTING.md). -### E2B and Daytona: runtime providers with clone operations - -E2B creates a sandbox from a template and exposes filesystem, process, pause/resume, snapshot, and Git operations. Its sandbox-creation schema has template, timeout, network, metadata, environment, MCP, IAM, and volume fields, but no repository source. Git clone is a runtime SDK operation with URL/path/branch/depth and inline credentials. E2B warns that credentials stored in the sandbox are readable by the agent. E2B is consequently a possible runtime backend, not an agent-neutral execution or workspace contract. +### Daytona: runtime provider with clone operations Daytona is the closest field-level operational match: its Git clone operation accepts `url`, `path`, optional branch or commit, credentials, depth, and an insecure-TLS option. But this is an imperative operation against an already-created Daytona sandbox. It does not standardize multi-source declaration, strict canonicalization, immutable result provenance, or committed attachment timing. Its per-operation credentials and optional TLS bypass also conflict with the AllAgents trust boundary. Older Daytona workspace models coupled repository metadata, devcontainer/build configuration, and provider workspace state, illustrating the portability cost of adopting a vendor workspace object. -Primary evidence: pinned E2B [OpenAPI schema](https://github.com/e2b-dev/E2B/blob/ccaf9fc0ffe6ac39c7ec786af7608ab1de19467b/spec/openapi.yml), [sandbox SDK](https://docs.e2b.dev/sdk-reference/js-sdk/v2.51.0/sandbox), [template definition](https://docs.e2b.dev/template/defining-template), [Git integration](https://docs.e2b.dev/sandbox/git-integration), Daytona [Git operations](https://www.daytona.io/docs/en/git-operations), and pinned Daytona [workspace](https://github.com/daytonaio/daytona/blob/dfb50e8a31e9a93b31181113d7b44b657cf27168/pkg/models/workspace.go), [repository](https://github.com/daytonaio/daytona/blob/dfb50e8a31e9a93b31181113d7b44b657cf27168/pkg/apiclient/model_git_repository.go), and [workspace-creation](https://github.com/daytonaio/daytona/blob/dfb50e8a31e9a93b31181113d7b44b657cf27168/pkg/apiclient/model_create_workspace_dto.go) models. +Primary evidence: Daytona [Git operations](https://www.daytona.io/docs/en/git-operations), and pinned Daytona [workspace](https://github.com/daytonaio/daytona/blob/dfb50e8a31e9a93b31181113d7b44b657cf27168/pkg/models/workspace.go), [repository](https://github.com/daytonaio/daytona/blob/dfb50e8a31e9a93b31181113d7b44b657cf27168/pkg/apiclient/model_git_repository.go), and [workspace-creation](https://github.com/daytonaio/daytona/blob/dfb50e8a31e9a93b31181113d7b44b657cf27168/pkg/apiclient/model_create_workspace_from_git_repository.go) models. ### GitHub Codespaces and Gitpod Classic: lifecycle precedents, not portable standards @@ -182,8 +180,6 @@ Adopt the following rule for future evaluation work: - **Benchmark compatibility:** Harbor, Terminal-Bench, and SWE-bench are future work requiring dedicated ADRs and closed adapter extensions. None changes the current v1 schemas. -- **Runtime precedent:** evaluate E2B, Daytona, Dev Containers, or Harbor ASP - only if a concrete isolation or imported-task requirement needs them. ## Existing research status @@ -191,7 +187,5 @@ Adopt the following rule for future evaluation work: is the current boundary analysis. - [Harbor repository materialization](./harbor-repository-materialization.md) remains useful evidence for task packages and separate verifiers. -- [E2B execution-gateway patterns](./e2b-execution-gateway-patterns.md) remains - useful sandbox evidence, but E2B is not required by the default runner. - General and private research wikis were discovery inputs only; cited primary sources and ADR 0002 carry the decision. From e7b36cb69034a4d6d12084e962a1471295072333 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 14:01:27 +1000 Subject: [PATCH 42/44] docs(architecture): remove discarded session framing --- ...use-allagents-gateway-for-one-shot-runs.md | 9 ++-- ...0837-feat-coding-execution-gateway-plan.md | 6 +-- .../harbor-repository-materialization.md | 4 +- .../one-shot-coding-agent-gateway-boundary.md | 42 ++++--------------- .../research/workspace-contract-incumbents.md | 10 ++--- 5 files changed, 20 insertions(+), 51 deletions(-) diff --git a/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md b/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md index 73633cf6..d3fbea79 100644 --- a/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md +++ b/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md @@ -14,13 +14,13 @@ Each request creates a fresh workspace, runs one agent once, optionally collects The public contracts are `AgentRunRequest v1` and `AgentRunResult v1`. They are closed, versioned JSON schemas. Unknown fields are rejected, and incompatible changes require a new version. The detailed fields belong in the gateway schemas and are recorded in the [implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). -This decision supersedes the earlier two-repository snapshot design and HarnessRouter issue [#304](https://github.com/HarnessRouter/harnessrouter/issues/304). We keep the `allagentsdev/allagents-gateway` name, but none of the earlier HarnessRouter fork, UHP session, workspace-builder, or snapshot contracts remain. +This decision supersedes the earlier two-repository snapshot design. We keep the `allagentsdev/allagents-gateway` name, but the workspace-builder and snapshot contracts do not remain. ## Why AllAgents needs remote, isolated coding-agent execution. It does not currently need long-lived interactive sessions. -The earlier design used HarnessRouter sessions, published workspace snapshots, checkpoints, filesystem journals, and generic change artifacts. Those features solve continuation and state-transfer problems. A one-shot run only needs a clean workspace, one agent attempt, optional evidence from the final state, and reliable cleanup. Keeping the session design would add protocol, storage, recovery, and fork-maintenance work without improving that result. +The earlier design used reusable sessions, published workspace snapshots, checkpoints, filesystem journals, and generic change artifacts. Those features solve continuation and state-transfer problems. A one-shot run only needs a clean workspace, one agent attempt, optional evidence from the final state, and reliable cleanup. Keeping the session design would add protocol, storage, and recovery work without improving that result. Supporting research is in [One-shot coding-agent gateway boundary](../research/one-shot-coding-agent-gateway-boundary.md). @@ -158,7 +158,7 @@ A completed result includes source and runtime provenance, bounded agent output, ### Where transcripts and traces come from -HarnessRouter is no longer in the trace path. While Codex or OMP runs, its adapter reads the structured events emitted by that pinned agent CLI and converts them incrementally to ATIF v1, the public trajectory format for V1. The runner stops recording at `max_trace_bytes` and adds a valid truncation marker rather than producing an invalid or unbounded document. +While Codex or OMP runs, its adapter reads the structured events emitted by that pinned agent CLI and converts them incrementally to ATIF v1, the public trajectory format for V1. The runner stops recording at `max_trace_bytes` and adds a valid truncation marker rather than producing an invalid or unbounded document. ```mermaid flowchart LR @@ -172,7 +172,7 @@ flowchart LR R --> P ``` -The agent's final answer is captured separately as bounded final output. V1 returns a trajectory for one run; it does not preserve a HarnessRouter-style session transcript or enough state to resume the conversation. The native vendor event stream is normalized rather than exposed as a second public trace format. +The agent's final answer is captured separately as bounded final output. V1 returns a trajectory for one run; it does not preserve a resumable session transcript or enough state to continue the conversation. The native vendor event stream is normalized rather than exposed as a second public trace format. All captured data is bounded. The request reserves enough artifact capacity for the declared output-file maxima. Actual output files use that capacity first, followed by agent output, command output in request order, and trajectory data. If optional artifact storage is exhausted, the gateway returns bounded inline, truncated, or explicitly omitted data rather than storing an unbounded value. @@ -201,7 +201,6 @@ V1 supports direct Codex and OMP execution only. It does not include: -- UHP or a HarnessRouter fork; - reusable or interactive sessions; - continuation, checkpoints, or workspace recovery; - generic diffs, patches, or modified-workspace export; diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 7aaa1608..a1452a80 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -15,7 +15,7 @@ execution: code - **Objective:** Run one Codex or OMP coding-agent attempt in a fresh composed workspace and return bounded output, usage, bounded trajectory, provenance, and optional raw post-run evidence with authenticated artifact retrieval. - **Means:** Build one repository and service, `allagentsdev/allagents-gateway`, containing the Promptfoo provider, gateway API, worker/runner, contracts, workspace composition, post-run collector, and agent adapters. - **First caller:** Promptfoo is the evaluation layer. It owns rendered prompts, workspace-source JSON, matrices, repetitions, JavaScript/LLM graders, pass/fail decisions, scores, and reports. -- **Stop conditions:** Do not build UHP, a HarnessRouter fork, reusable sessions, continuation, checkpoints, composed-workspace/snapshot caches, generic diffs, change artifacts, or a second repository. +- **Stop conditions:** Do not build reusable sessions, continuation, checkpoints, composed-workspace or snapshot caches, generic diffs, change artifacts, or a second repository. The authoritative decision is [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). Supporting evidence is in [One-shot coding-agent gateway boundary](../research/one-shot-coding-agent-gateway-boundary.md). @@ -51,8 +51,6 @@ V1 supports **direct mode** only, owned completely by this repository. ### Excluded - Pass/fail, reward, rubric, or grading fields and decisions in the gateway contract. -- UHP endpoints, objects, conformance, or compatibility. -- HarnessRouter code, forking, routing, or provider abstractions. - Long-lived or reusable coding sessions, additional turns, continuation, resume, replay, or checkpoints. - Reusable composed workspaces, mutable source caches, unkeyed Git clones, prepared snapshots, or OCI workspace publication. Immutable exact-source generations are required only as specified below. - A workspace-builder service or repository. @@ -1045,4 +1043,4 @@ CI may retain sanitized JSON, immutable cache fixtures, and referenced evidence, - [ ] Provider persists run/upload identities, downloads/verifies all artifact-backed output/evidence/trajectory, exposes partial evidence diagnostically, and leaves grading to Promptfoo. - [ ] Cleanup removes every process/namespace/flow/service/mount/clone/root and releases leases before result visibility; cleanup failure forces infrastructure_error while retaining evidence through reconciliation. - [ ] Credentials/policy remain outside caller JSON, run environments, cache, evidence, errors, artifacts, and logs. -- [ ] No UHP, HarnessRouter fork, reusable session, continuation, checkpoint, composed-workspace/snapshot publication cache, generic core change artifact, or second repository remains. +- [ ] No reusable session, continuation, checkpoint, composed-workspace or snapshot publication cache, generic core change artifact, or second repository remains. diff --git a/docs/research/harbor-repository-materialization.md b/docs/research/harbor-repository-materialization.md index 3fce3881..b9fe13f8 100644 --- a/docs/research/harbor-repository-materialization.md +++ b/docs/research/harbor-repository-materialization.md @@ -172,8 +172,8 @@ Before implementing any Harbor, Terminal-Bench, or SWE-bench adapter: The practical conclusion is narrow: Harbor is useful primary-source evidence for content-addressed task packages, isolated tasks, and colocated checks. None of Harbor, Terminal-Bench, or SWE-bench is an approved current adapter, control -plane, provenance variant, or result contract. They do not justify UHP, -HarnessRouter, or reusable sessions. +plane, provenance variant, or result contract. They do not justify reusable +sessions. ## Primary sources diff --git a/docs/research/one-shot-coding-agent-gateway-boundary.md b/docs/research/one-shot-coding-agent-gateway-boundary.md index ab2114b6..fc022e0f 100644 --- a/docs/research/one-shot-coding-agent-gateway-boundary.md +++ b/docs/research/one-shot-coding-agent-gateway-boundary.md @@ -30,16 +30,13 @@ source materializers, artifact service, and Promptfoo provider. One public Source credentials, source policy, credential selection, network policies, and runtime profiles are operator configuration, never caller-supplied policy bodies. Harbor and Terminal-Bench integration is outside the current v1 and -requires a future ADR plus closed adapter request/result mapping. HarnessRouter -and UHP are unnecessary: -the gateway has no reusable sessions, checkpoint/continuation contracts, or -generic produced-file service. Any exact patch production belongs only in a -future benchmark adapter whose upstream evaluator requires it. +requires a future ADR plus closed adapter request/result mapping. The gateway +has no reusable sessions, checkpoint or continuation contract, or generic +produced-file service. Any exact patch production belongs only in a future +benchmark adapter whose upstream evaluator requires it. -This replaces the former immutable-snapshot/HarnessRouter recommendation at -this path. See -[ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md) for the current -decision record. +See [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md) +for the current decision record. ## Why Promptfoo is the control plane @@ -258,31 +255,6 @@ filename, diff-format, size, and evaluator compatibility rules. Core `AgentRun v1` exposes no generic diff, patch, modified workspace, or change artifact. -## Why stock HarnessRouter evidence does not change the decision - -The superseded research established several stock behaviors that remain factually useful: - -- UHP continuation binds a response chain to the same session, working directory, files, and - harness ([UHP sessions](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/protocol/versions/2026-09-12/sessions.md#L7-L31)); -- HarnessRouter derives per-session directories and has a durable hydrate/checkpoint lifecycle - ([runner workspace isolation](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L78-L171), - [runner hydrate/checkpoint routes](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L6982-L7110)); and -- its produced-file collector is a root-Git cursor with gateway-side artifact capture and - acknowledgement - ([runner produced routes](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/runner/server.py#L7111-L7205), - [gateway collector](https://github.com/HarnessRouter/harnessrouter/blob/8f7868ccb2c97d1f611acf11e7cad0357a43064e/gateway/app.py#L2428-L2524)). - -Those are valuable product-session behaviors, but they solve a different -problem. A Promptfoo coding eval needs one isolated attempt plus raw final-state -evidence; Promptfoo code or LLM graders decide pass/fail and reward. The general -gateway can run optional named post-run checks in the same trial runtime and -final workspace, preserving declared source access modes, after stopping -agent-owned descendants, removing credentials, and injecting hidden checks. It -owns no evaluation verdict. - -Adopting UHP, forking HarnessRouter, or proposing upstream -snapshot/journal/finalization seams would add session, checkpoint, and artifact -semantics that this boundary does not use. ## Recommendation @@ -328,4 +300,4 @@ not own pass/fail or reward, and no mutable trial state is shared. Source credentials and policy/credential routes stay out-of-band. Harbor, Terminal-Bench, and SWE-bench are future work requiring dedicated ADRs and closed extensions. No reusable-session layer, generic diff service, checkpoint -architecture, UHP endpoint, or HarnessRouter change is warranted. +architecture or session protocol is warranted. diff --git a/docs/research/workspace-contract-incumbents.md b/docs/research/workspace-contract-incumbents.md index 5d860e90..b07e5e90 100644 --- a/docs/research/workspace-contract-incumbents.md +++ b/docs/research/workspace-contract-incumbents.md @@ -23,8 +23,8 @@ publishes `AgentRunResult v1`. Cleanup failure is `infrastructure_error` with partial evidence. Tenant/run-authorized artifact retrieval is time-bounded and the provider verifies byte size and digest. Promptfoo alone decides pass/fail and reward. Harbor, Terminal-Bench, and SWE-bench require future ADRs and closed -adapter extensions; none is part of current v1. No incumbent justifies UHP, -HarnessRouter, or reusable execution sessions. +adapter extensions; none is part of current v1, and no incumbent justifies +reusable execution sessions. The current boundary is defined by [One-shot coding-agent gateway boundary](./one-shot-coding-agent-gateway-boundary.md) @@ -32,8 +32,8 @@ and [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). ## Historical contract evaluated -The comparisons below originally evaluated a product execution stack with UHP, -a separate source builder, an immutable OCI handoff, and a long-lived gateway. +The comparisons below originally evaluated a product execution stack with a +separate source builder, an immutable OCI handoff, and a long-lived gateway. That architecture is rejected for the chosen evaluation-only scope. Statements below that prescribe source descriptors, snapshot manifests, builder ownership, or gateway behavior are retained as historical comparison, not current @@ -88,7 +88,7 @@ It is not a safe wholesale replacement: - ZIP sources have no required content digest. There is no OCI workspace-source variant. - Devfile has no standard resolved-commit result, canonical source-visible manifest, generation identity, or attachment-commit acknowledgement. - Runtime implementations own credential behavior. The DevWorkspace Operator, for example, may expose configured Git credentials to workspace containers; that is weaker than acquisition-only credentials. -- Devfile lifecycle events and component `sourceMapping` configure a development environment. They do not define the UHP timing rule that source is attached before the harness starts and metadata appears only after attachment commit. +- Devfile lifecycle events and component `sourceMapping` configure a development environment. They do not define the AllAgents timing rule that source is attached before the agent starts and metadata appears only after attachment completes. AllAgents should cite and follow Devfile's vocabulary where it fits, while preserving stricter semantics: From 100b993a7b1579b4f1b817cbb9cec29adc2233a2 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 15:27:20 +1000 Subject: [PATCH 43/44] docs(architecture): clarify multi-turn evaluation boundary --- ...use-allagents-gateway-for-one-shot-runs.md | 2 ++ ...0837-feat-coding-execution-gateway-plan.md | 11 +++++---- .../one-shot-coding-agent-gateway-boundary.md | 24 +++++++++++++++++++ 3 files changed, 33 insertions(+), 4 deletions(-) diff --git a/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md b/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md index d3fbea79..0b9787f7 100644 --- a/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md +++ b/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md @@ -12,6 +12,8 @@ Promptfoo will be the first caller. Promptfoo owns evaluation cases, prompt and Each request creates a fresh workspace, runs one agent once, optionally collects evidence, and destroys the workspace. V1 has no reusable session, continuation, checkpoint, generic workspace diff, or implicit rerun. +Promptfoo may render earlier conversational turns into that one instruction. This is transcript replay into a new run, not continuation: previous workspace mutations, tool state, and agent state do not survive. Promptfoo's stateful target mode is outside V1 because it sends only the newest turn and requires the target to own a reusable session. + The public contracts are `AgentRunRequest v1` and `AgentRunResult v1`. They are closed, versioned JSON schemas. Unknown fields are rejected, and incompatible changes require a new version. The detailed fields belong in the gateway schemas and are recorded in the [implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). This decision supersedes the earlier two-repository snapshot design. We keep the `allagentsdev/allagents-gateway` name, but the workspace-builder and snapshot contracts do not remain. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index a1452a80..4768054b 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -51,7 +51,7 @@ V1 supports **direct mode** only, owned completely by this repository. ### Excluded - Pass/fail, reward, rubric, or grading fields and decisions in the gateway contract. -- Long-lived or reusable coding sessions, additional turns, continuation, resume, replay, or checkpoints. +- Long-lived or reusable coding sessions, additional target turns, continuation, resume, session-event replay, or checkpoints. - Reusable composed workspaces, mutable source caches, unkeyed Git clones, prepared snapshots, or OCI workspace publication. Immutable exact-source generations are required only as specified below. - A workspace-builder service or repository. - Generic before/after diffs, patch output, or a core modified-workspace artifact. @@ -804,9 +804,11 @@ The provider enforces the same conditional invariant as the gateway schema: `bun Local mode starts an ephemeral loopback gateway and worker using the same API handlers, runner, schemas, and states with SQLite/local artifacts. It does not bypass admission or call an adapter directly. Remote mode uses authenticated HTTPS. Promptfoo's secret mechanism supplies gateway authentication outside provider config and request JSON. +The provider treats `renderedPrompt` as opaque input. Promptfoo may include a complete `_conversation` transcript in that value; every `callApi` still creates a distinct run with a fresh workspace and no retained agent or tool state. The provider does not return a reusable `sessionId`, enable backend thread pooling, or claim support for Promptfoo simulated-user `stateful: true`, which sends only the newest turn and requires the target to retain its own session. Stateful interactive coding evaluation requires a separate session contract. + For each `callApi(renderedPrompt, context)` the provider: -1. derives stable Promptfoo evaluation/case/repetition identity, allocates and persists one UUIDv7 `run_id` plus idempotency key before side effects; +1. derives a stable Promptfoo evaluation/case/repetition conversation key and atomically claims the next durable provider-call record containing an ordinal, request digest, UUIDv7 `run_id`, and idempotency key before side effects. Re-entry into an unfinished call with the same digest reuses that record; a completed call advances the ordinal even when the next rendered prompt is byte-identical; 2. deterministically packages each local source/optional post-run bundle, allocates and persists one UUIDv7 `upload_id` per package, then idempotently reserves/uploads and substitutes bundle references without changing order/destinations; 3. builds and locally schema-validates the complete request, including `runtime_profile_id`, all ten limits, and requested-file artifact reservation; 4. submits, polls the tenant-owned run, and propagates Promptfoo abort to cancellation using the same identity; @@ -981,6 +983,7 @@ Implement provider modes, crash-safe run/upload identities, multi-source/runtime - inline and artifact-backed agent output, command streams, files, and trajectories are byte/digest/media verified; expiry/corruption becomes typed infrastructure, never a grader input; - cancelled/infrastructure errors carry their result/partial evidence to diagnostics but are excluded from behavioral rates; - a Codex/OMP matrix with two repetitions creates four private runs sharing authorized immutable generations; JS/LLM graders alone decide outcomes from verified raw evidence; +- a two-turn full-history Promptfoo conversation receives distinct durable provider-call ordinals, run IDs, and fresh workspaces; losing and retrying turn 2's response reuses only turn 2's unfinished identity, while the provider returns no reusable session identity and never pools Codex/OMP state across calls; - cross-tenant run/cancel/artifact attempts fail identically in local and remote modes; no host path, credential, policy body, process, mount, clone, or run directory survives. ## Deferred integrations @@ -1040,7 +1043,7 @@ CI may retain sanitized JSON, immutable cache fixtures, and referenced evidence, - [ ] Optional bundle is present iff a bundle executable is requested; dispatch creates only an empty reserved late-mount point and does not acquire or mount bundle bytes until agent cgroup/network/flows and credentials/home are gone. Runtime tools come only from the pinned profile revision's PATH; args are literal; commands use the exact final workspace. - [ ] Post-run initializes one full ordered evidence skeleton, seals each command/file before advancing, and represents completed/timed_out/not_run/unavailable plus all file states. - [ ] Cancelled/infrastructure results retain durable partial evidence; behavioral nonzero/timeout/missing/limit/non-regular observations do not become infrastructure. -- [ ] Provider persists run/upload identities, downloads/verifies all artifact-backed output/evidence/trajectory, exposes partial evidence diagnostically, and leaves grading to Promptfoo. +- [ ] Provider persists per-conversation call ordinals plus run/upload identities, reuses only an unfinished matching call, downloads/verifies all artifact-backed output/evidence/trajectory, exposes partial evidence diagnostically, and leaves grading to Promptfoo. - [ ] Cleanup removes every process/namespace/flow/service/mount/clone/root and releases leases before result visibility; cleanup failure forces infrastructure_error while retaining evidence through reconciliation. - [ ] Credentials/policy remain outside caller JSON, run environments, cache, evidence, errors, artifacts, and logs. -- [ ] No reusable session, continuation, checkpoint, composed-workspace or snapshot publication cache, generic core change artifact, or second repository remains. +- [ ] No reusable session, provider session identity, backend thread pooling, continuation, checkpoint, composed-workspace or snapshot publication cache, generic core change artifact, or second repository remains; full-history Promptfoo conversation rendering still creates a fresh run per turn. diff --git a/docs/research/one-shot-coding-agent-gateway-boundary.md b/docs/research/one-shot-coding-agent-gateway-boundary.md index fc022e0f..40cf7b6e 100644 --- a/docs/research/one-shot-coding-agent-gateway-boundary.md +++ b/docs/research/one-shot-coding-agent-gateway-boundary.md @@ -66,6 +66,30 @@ adapter; the general gateway supplies materialization, one-shot agent execution, same-runtime post-run checks, raw evidence, and cleanup while Promptfoo retains all grading and reward policy. +## Multi-turn and sandboxed-code boundaries + +Promptfoo's simulated-user provider has two different transport modes. Its default +resends the complete transcript on each turn. With `stateful: true`, Promptfoo +sends only the newest user message after the target returns a session ID and +expects that target to retain its own history +([simulated-user provider](https://www.promptfoo.dev/docs/providers/simulated-user/)). +The gateway can accept a fully rendered transcript as one instruction, but every +provider call still creates a fresh run and workspace. That can test textual +conversation continuity; it cannot test a coding conversation that depends on +files, processes, tools, or services from an earlier turn. The provider therefore +must not return a reusable session ID or pool native Codex/OMP sessions in V1. + +Promptfoo's +[sandboxed-code guide](https://www.promptfoo.dev/docs/guides/sandboxed-code-evals/) +does not put Promptfoo or its provider inside a sandbox. Its `type: python` +assertion runs trusted user code, and that assertion explicitly calls Epicbox to +launch the generated code snippet in a one-time Docker container. This is useful +for grading code returned as text. It does not prepare a repository, isolate a +write-capable coding agent, preserve the agent's final workspace for hidden +checks, or provide the source, credential, artifact, and cleanup contracts needed +here. A larger custom assertion could rebuild those responsibilities, but that +would be another implementation of the gateway rather than a Promptfoo feature. + ## AgentRun contract The normative wire contracts are From c00ebc7c38c47a681aad080f9735f4ba9860571b Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 28 Sep 2026 18:25:27 +1000 Subject: [PATCH 44/44] docs(architecture): use Promptfoo-native agent execution --- ...use-allagents-gateway-for-one-shot-runs.md | 233 ---- ...02-use-promptfoo-native-agent-execution.md | 277 +++++ ...0837-feat-coding-execution-gateway-plan.md | 1049 ----------------- ...-feat-promptfoo-coding-agent-evals-plan.md | 549 +++++++++ .../harbor-repository-materialization.md | 66 +- .../one-shot-coding-agent-gateway-boundary.md | 173 ++- .../promptfoo-native-agent-workspaces.md | 496 ++++++++ .../source-credential-broker-precedents.md | 77 +- .../research/workspace-contract-incumbents.md | 130 +- 9 files changed, 1516 insertions(+), 1534 deletions(-) delete mode 100644 docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md create mode 100644 docs/decisions/0002-use-promptfoo-native-agent-execution.md delete mode 100644 docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md create mode 100644 docs/plans/2026-09-18-0837-feat-promptfoo-coding-agent-evals-plan.md create mode 100644 docs/research/promptfoo-native-agent-workspaces.md diff --git a/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md b/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md deleted file mode 100644 index 0b9787f7..00000000 --- a/docs/decisions/0002-use-allagents-gateway-for-one-shot-runs.md +++ /dev/null @@ -1,233 +0,0 @@ -# ADR 0002: Use a one-shot AllAgents Gateway with Promptfoo as the first caller - -- Status: Accepted -- Date: 2026-09-21 -- Updated: 2026-09-28 - -## Decision - -We will build `allagentsdev/allagents-gateway` as a general service for one-shot coding-agent runs. The API, worker, runner, Promptfoo provider, and Codex and OMP adapters will live in that repository. - -Promptfoo will be the first caller. Promptfoo owns evaluation cases, prompt and model matrices, repetitions, grading, pass/fail decisions, rewards, and reports. The gateway runs an agent and returns raw evidence. It does not grade the result. - -Each request creates a fresh workspace, runs one agent once, optionally collects evidence, and destroys the workspace. V1 has no reusable session, continuation, checkpoint, generic workspace diff, or implicit rerun. - -Promptfoo may render earlier conversational turns into that one instruction. This is transcript replay into a new run, not continuation: previous workspace mutations, tool state, and agent state do not survive. Promptfoo's stateful target mode is outside V1 because it sends only the newest turn and requires the target to own a reusable session. - -The public contracts are `AgentRunRequest v1` and `AgentRunResult v1`. They are closed, versioned JSON schemas. Unknown fields are rejected, and incompatible changes require a new version. The detailed fields belong in the gateway schemas and are recorded in the [implementation plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). - -This decision supersedes the earlier two-repository snapshot design. We keep the `allagentsdev/allagents-gateway` name, but the workspace-builder and snapshot contracts do not remain. - -## Why - -AllAgents needs remote, isolated coding-agent execution. It does not currently need long-lived interactive sessions. - -The earlier design used reusable sessions, published workspace snapshots, checkpoints, filesystem journals, and generic change artifacts. Those features solve continuation and state-transfer problems. A one-shot run only needs a clean workspace, one agent attempt, optional evidence from the final state, and reliable cleanup. Keeping the session design would add protocol, storage, and recovery work without improving that result. - -Supporting research is in [One-shot coding-agent gateway boundary](../research/one-shot-coding-agent-gateway-boundary.md). - -## Main flow - -1. The caller provides one instruction, an ordered list of workspace sources, an agent, an authorized runtime profile, network-policy names, resource limits, and optional post-run evidence requests. Promptfoo gives every evaluation repetition a distinct run identity. -2. The Promptfoo provider packages local sources and optional check content into immutable uploads. Remote requests contain artifact references and digests, never paths on the caller's machine. -3. The gateway authenticates the caller, validates the request, and checks for an existing run with the same identity. It then authorizes a new request and pins the exact runtime profile revision that the worker must use. -4. The worker creates a fresh sandbox. It authorizes and materializes each source at its requested destination. Read-only sources stay read-only. Writable sources receive private copies. -5. The selected Codex or OMP adapter starts the agent once. The runner records bounded final output, usage, timing, and trajectory data. -6. After the agent exits, the runner stops every agent-owned process, destroys the agent network environment, and removes model and source credentials. If the request asks for post-run evidence, the runner then exposes the hidden check bundle, creates a separate post-run network environment, and runs the declared commands in the same retained runtime and final workspace. -7. The worker stores each command result and requested file observation in request order. It then stops remaining processes and services, unmounts the workspace, deletes private files, and releases source-cache leases. -8. Only after cleanup succeeds does the gateway publish `completed`, `cancelled`, or `infrastructure_error`. Promptfoo downloads and verifies referenced artifacts, then applies its own graders. - -```mermaid -flowchart LR - P[Caller or Promptfoo] --> G[Gateway validates and admits] - G --> W[Worker composes a fresh workspace] - W --> A[Codex or OMP runs once] - A --> E{Post-run evidence requested?} - E -->|No| S[Seal agent result] - E -->|Yes| C[Stop agent, remove credentials, run checks] - S --> K[Clean workspace and release leases] - C --> K - K --> R[Publish terminal result] - R --> P -``` - -## Responsibilities - -| Component | Owns | Does not own | -|---|---|---| -| Promptfoo | Evaluation cases, prompt rendering, model matrices, repetitions, requested evidence, graders, pass/fail, rewards, and reports | The remote sandbox or gateway lifecycle | -| Promptfoo provider | Stable run and retry identities, local packaging, immutable uploads, submission, cancellation, polling, artifact verification, and exposing evidence to Promptfoo | Grading or remote execution | -| Gateway | Authentication, validation, authorization, duplicate-request handling, cancellation, dispatch, artifact access, and result retrieval | Behavioral pass/fail or reward | -| Worker and runner | One run's sandbox, workspace composition, limits, agent supervision, evidence collection, and cleanup | Evaluation policy | -| Operator configuration | Credentials, source access rules, model routing, runtime profiles, sandbox and network policies, task services, resource ceilings, and retention | Caller-controlled request data | - -Callers select authorized logical names. They cannot submit credentials or policy definitions. - -## Request boundary - -A request includes: - -- `run_id` and `idempotency_key`; -- one rendered instruction; -- an ordered workspace and working directory; -- Codex or OMP plus a logical model name; -- a required `runtime_profile_id`; -- an agent network-policy name; -- required time, workspace, output, trace, command-output, and artifact limits; and -- `post_run`, which is either `null` or a closed evidence request. - -A workspace contains between 1 and 128 sources. Every source declares a non-overlapping destination and `access: read_only | writable`. - -| Source | Caller provides | Gateway resolves | -|---|---|---| -| Git | Canonical HTTPS repository URL, ref, and full or bounded shallow history | One exact commit | -| OCI | Canonical repository and exact image-manifest descriptor | That exact manifest and materialized filesystem | -| Uploaded bundle | Authorized immutable bundle reference | The declared artifact digest | - -The caller cannot submit host paths, credentials, internal credential routes, runtime images, executable paths, service definitions, shell commands, raw network destinations, policy bodies, grading rules, patch requests, or workspace-persistence options. - -## Source authorization and caching - -For Git, the gateway maps the authenticated tenant and canonical repository URL to exactly one internal access and credential route. OCI uses the authenticated tenant and canonical repository in the same way. Zero matches or more than one match reject the request before credentials or network access. - -V1 rejects Git and OCI redirects. An OCI route fixes the allowed token service. The worker fetches manifests and blobs through that route and rejects foreign blob locations. Remote content cannot redirect credentials to another target. - -The gateway caches immutable source generations: - -- Git: canonical repository, resolved commit, history rule, and materializer version; -- OCI: canonical repository, exact descriptor, and materializer version; and -- uploaded bundle: artifact digest and materializer version. - -Authorization is not part of the cache key. The gateway checks current authorization before every attachment, including cache hits. - -Concurrent requests for the same missing generation share one fetch and materialization. Complete generations are immutable. A run holds an eviction lease until cleanup finishes. - -Read-only sources mount the cached generation directly and read-only. Writable sources receive a private filesystem copy-on-write clone when supported, with a full private copy as the fallback. The gateway never uses hardlinks or another shared writable alias. - -## Runtime and isolation - -`runtime_profile_id` selects an operator-defined profile that the caller is allowed to use. Before dispatch, the gateway resolves it to one immutable `profile_digest` and stores that exact revision with the run. The revision fixes the runtime image, sandbox policy, approved tools, task-service implementations, network brokers, and resource ceilings. The worker refuses to run if the pinned content is missing or its digest changed. - -`AgentRunResult v1` reports the profile and immutable runtime, tool, and service identities that actually ran. It does not expose secret arguments, credentials, private endpoints, or host paths. - -Every run uses a new Linux sandbox with: - -- an unprivileged, non-root user and no Linux capabilities; -- `no_new_privs` and an explicit system-call allowlist; -- private process, mount, inter-process communication, hostname, and network environments; -- a read-only runtime filesystem with only declared workspace and build locations writable; -- no host devices, container socket, control-plane mount, or writable cgroup control; and -- enforced process-count, memory, CPU, disk I/O, time, workspace, file-count, and output limits. - -If the worker cannot establish this baseline, it rejects the run rather than weakening isolation. - -The agent and post-run commands use separate, deny-by-default network environments. Both external and local traffic must go through operator-managed endpoints allowed for that phase. The worker destroys the agent network environment and its active connections before creating the post-run environment. This prevents the agent from reaching checks or services that are available only after it exits. - -Credentials never enter caller JSON, workspace files, logs, trajectories, or returned evidence. Source and model access use worker-owned mechanisms. Evidence commands receive no source or model credentials. - -## Post-run evidence - -`post_run: null` means the caller wants only the agent result. A non-null value asks the gateway to collect raw evidence; it does not ask the gateway to judge the run. - -A post-run request may include: - -- an immutable hidden check bundle; -- up to 32 ordered commands; -- up to 128 bounded workspace-relative output files; and -- a separately authorized post-run network policy. - -Each command selects either an approved tool from the pinned runtime profile or an executable inside the hidden bundle. Arguments are literal strings. The gateway does not invoke a shell, expand variables or globs, accept a command-specific working directory, or accept a command-specific network policy. - -The hidden bundle is absent while the agent runs. The worker may create an empty reserved mount point, but it does not fetch or mount the bundle until the agent and its descendants are gone, the agent network environment is destroyed, and credentials are removed. - -Evidence commands run in the same pinned runtime and final workspace so they can use installed dependencies and declared task services. Source access remains unchanged: a command cannot write to a `read_only` source. - -A command's nonzero exit or configured timeout is raw evidence in a `completed` run. A failure to create the sandbox, mount the bundle, resolve or start an approved executable, store promised evidence, or clean up is an `infrastructure_error`. - -The worker records one observation for every requested command and file, in request order. Cancellation or infrastructure failure preserves observations already stored and marks later work as not run or unavailable. It never silently drops or reorders partial evidence. - -## Results and artifacts - -`AgentRunResult v1` reports gateway lifecycle, not task quality: - -| Status | Meaning | -|---|---| -| `completed` | The agent ran and requested evidence collection finished. A command may still have failed or timed out. | -| `cancelled` | Cancellation won before the terminal result was published. | -| `infrastructure_error` | The gateway could not safely complete the run lifecycle. | - -A completed result includes source and runtime provenance, bounded agent output, usage, timing, and a valid bounded trajectory. It includes post-run observations when requested. The gateway never returns behavioral pass/fail or reward. - -### Where transcripts and traces come from - -While Codex or OMP runs, its adapter reads the structured events emitted by that pinned agent CLI and converts them incrementally to ATIF v1, the public trajectory format for V1. The runner stops recording at `max_trace_bytes` and adds a valid truncation marker rather than producing an invalid or unbounded document. - -```mermaid -flowchart LR - A[Codex or OMP native events] --> N[Adapter normalizes to ATIF v1] - N --> B[Runner applies trace byte limit] - B --> I{Fits inline?} - I -->|Yes| R[AgentRunResult trajectory] - I -->|No| O[Result artifact reference] - O --> V[Provider downloads and verifies] - V --> P[Promptfoo] - R --> P -``` - -The agent's final answer is captured separately as bounded final output. V1 returns a trajectory for one run; it does not preserve a resumable session transcript or enough state to continue the conversation. The native vendor event stream is normalized rather than exposed as a second public trace format. - -All captured data is bounded. The request reserves enough artifact capacity for the declared output-file maxima. Actual output files use that capacity first, followed by agent output, command output in request order, and trajectory data. If optional artifact storage is exhausted, the gateway returns bounded inline, truncated, or explicitly omitted data rather than storing an unbounded value. - -Every upload, run, cancellation target, result, and result artifact belongs to one authenticated tenant. Object lookup always includes that tenant. A cross-tenant identifier is treated like an unknown identifier; UUIDs and digests are not authorization. - -Result artifact references include media type, digest, byte size, and expiry. The authenticated artifact endpoint serves only unexpired result artifacts belonging to the caller's run. The Promptfoo provider verifies the returned media type, length, and digest before passing bytes to a grader. Input and hidden-check bundles are never exposed through the result-artifact endpoint. - -## Retries, cancellation, and cleanup - -After authentication and schema validation, the gateway checks `run_id` and `idempotency_key` before source admission or resource allocation. - -- An exact duplicate returns the existing run without starting another agent. -- Reusing either identity for a different request returns a conflict and starts nothing. -- A new request is admitted before a public run is created. A rejected request leaves no public run or job. -- Once the worker may have started the agent, recovery never starts it again. If the worker cannot prove that the agent was never invoked, the run ends with `infrastructure_error`. - -Retries of uploads, submission, polling, cancellation, and artifact downloads reuse the same identities. Promptfoo repetitions use new identities because they are intentional new attempts. - -Cancellation stops the current agent or evidence process and preserves bounded partial evidence. The worker then performs the same cleanup as any other run. `cancelled` becomes visible only after cleanup succeeds. A cleanup failure changes the eventual result to `infrastructure_error`. - -Cleanup is part of terminalization. Before publishing any terminal result, the worker must stop all remaining processes and services, remove mounts and private files, and release cache leases. Cleanup can be retried safely, but the gateway cannot publish a successful result while cleanup remains incomplete. - -## V1 scope - -V1 supports direct Codex and OMP execution only. - -It does not include: - -- reusable or interactive sessions; -- continuation, checkpoints, or workspace recovery; -- generic diffs, patches, or modified-workspace export; -- Harbor, Terminal-Bench, or SWE-bench adapters; or -- gateway-owned grading. - -A future benchmark adapter or workspace-export feature requires a separate decision and a new closed contract. OCI remains a supported workspace input; it is not a mandatory snapshot handoff between two AllAgents services. - -## Consequences - -Benefits: - -- each run starts clean and cannot inherit another run's mutable state; -- network retries cannot duplicate agent work; -- concurrent runs can reuse exact immutable sources without sharing writable state; -- Promptfoo grading stays separate from gateway lifecycle; -- Codex and OMP share one isolation and result contract; and -- credentials and operator policy remain outside the caller-controlled request. - -Costs: - -- AllAgents must build and operate strong sandbox, network, credential, artifact, and cleanup controls; -- local inputs and hidden checks must be packaged and uploaded; -- immutable source caching requires storage limits, leases, safe eviction, recovery, and authorization on every attachment; -- callers must choose `read_only` or `writable` correctly; and -- debugging relies on bounded output, trajectory, command results, and requested files because the workspace is destroyed. - -We will reconsider this decision if the product needs reusable interactive coding sessions or if direct execution cannot provide the required isolation. More Promptfoo matrices, graders, benchmark formats, or raw evidence do not by themselves justify sessions, snapshots, generic diffs, or gateway-owned grading. diff --git a/docs/decisions/0002-use-promptfoo-native-agent-execution.md b/docs/decisions/0002-use-promptfoo-native-agent-execution.md new file mode 100644 index 00000000..5d5d3584 --- /dev/null +++ b/docs/decisions/0002-use-promptfoo-native-agent-execution.md @@ -0,0 +1,277 @@ +# ADR 0002: Use Promptfoo native agent providers with managed disposable workspaces + +- Status: Accepted +- Date: 2026-09-21 +- Updated: 2026-09-28 + +## Decision + +Coding-agent evaluations will run directly through Promptfoo's built-in agent +providers inside one disposable local or CI job. V1 will not introduce an +AllAgents execution gateway, a custom Promptfoo provider, or a separate runner +service. + +Both supported providers receive the same fixed job-private working directory: + +```yaml +working_dir: ./.eval/workspace +``` + +A Promptfoo lifecycle extension owns the mutable directory around every +evaluation row: + +1. job bootstrap resolves and materializes exact source inputs into immutable, + job-private seeds; +2. `beforeEach` removes any previous workspace and creates a private copy of the + selected seed at `.eval/workspace`; +3. Promptfoo invokes the selected built-in agent provider in that directory; +4. deterministic JavaScript assertions inspect the final filesystem and run + declared checks before teardown; +5. `afterEach` records bounded diagnostic metadata and removes the workspace; +6. `afterAll` removes remaining job-private evaluation state. + +Setup fails closed in `beforeEach`: any reset or copy error throws before the +provider runs. Promptfoo currently catches and logs `afterEach` failures, so an +`afterEach` exception alone cannot fail the evaluation command. The extension +records a failure sentinel, and the surrounding job wrapper checks that sentinel +and workspace absence before accepting the run. `beforeEach` always deletes the +fixed workspace before copying a seed, even when the previous cleanup appeared +successful. + +Promptfoo runs these rows serially and without its response cache: + +```yaml +evaluateOptions: + maxConcurrency: 1 + cache: false +``` + +Serial execution makes one fixed working directory unambiguous. Parallelism, if +needed later, is job-level: each disposable job receives its own `.eval` root. + +Promptfoo remains the evaluation system of record. It owns prompts, provider and +model matrices, repetitions, assertions, scores, pass/fail decisions, traces, +and reports. The workspace extension owns only setup, reset, bounded diagnostic +collection, and cleanup. + +## Why + +Promptfoo already implements the agent-facing behavior the earlier gateway +design planned to recreate: + +- the built-in Claude Agent SDK and Codex SDK providers both accept + `working_dir`, resolved relative to the configuration file; +- Promptfoo documents extension hooks for `beforeAll`, `beforeEach`, + `afterEach`, and `afterAll`; +- Promptfoo's own write-capable Claude example combines an extension-managed + workspace with `maxConcurrency: 1`; +- external JavaScript assertions can inspect the final workspace and return + structured grading results; +- built-in providers return final output, usage, provider metadata, and session + identifiers where supported; and +- Promptfoo emits and ingests OpenTelemetry traces and projects recognized tool + spans into `trajectory:*` assertions. + +The proposed gateway added an HTTP API, custom provider, queue, database, +artifact service, source cache, worker state machine, custom agent adapters, +ATIF conversion, sandbox implementation, cancellation protocol, and recovery +semantics. None of those components is necessary for a trusted, single-tenant +evaluation that already runs inside a disposable job. + +Supporting evidence and provider-specific limits are recorded in +[Promptfoo native agent workspaces](../research/promptfoo-native-agent-workspaces.md). +The implementation sequence is in the +[Promptfoo coding-agent evaluation plan](../plans/2026-09-18-0837-feat-promptfoo-coding-agent-evals-plan.md). + +## Execution boundary + +The disposable job is the outer lifecycle and isolation boundary. It may be a +CI job, rootless container, or VM. The job: + +- starts without mutable state from another evaluation job; +- receives only the source and model credentials required for that job; +- materializes exact source revisions before starting Promptfoo; +- runs Promptfoo and all assertions; +- exports the requested Promptfoo results, traces, and bounded diagnostics; and +- destroys the complete job filesystem and process tree when finished. + +Promptfoo provider sandboxes constrain agent operations but are not a substitute +for a hostile multi-tenant execution boundary. Write-capable or adversarial +evaluations must run in a disposable container or VM rather than directly on a +developer workstation. + +Source acquisition occurs before agent execution. Acquisition credentials must +not be copied into `.eval/seeds`, `.eval/workspace`, result metadata, or trace +attributes. Job bootstrap removes them from the environment before Promptfoo +starts whenever the source transport permits that separation. + +## Promptfoo configuration + +The initial provider matrix uses Promptfoo's providers directly: + +```yaml +providers: + - id: anthropic:claude-agent-sdk + config: + working_dir: ./.eval/workspace + append_allowed_tools: [Write, Edit, MultiEdit, Bash] + permission_mode: acceptEdits + persist_session: false + sandbox: + enabled: true + failIfUnavailable: true + + - id: openai:codex-sdk + config: + working_dir: ./.eval/workspace + sandbox_mode: workspace-write + approval_policy: never + enable_streaming: true + persist_threads: false + +extensions: + - file://extensions/workspace.cjs:workspaceLifecycle + +evaluateOptions: + maxConcurrency: 1 + cache: false + +tracing: + enabled: true + otlp: + http: {} +``` + +Provider-specific permissions remain explicit. A common working directory does +not imply identical tool or sandbox behavior. + +Codex `enable_streaming` is enabled because Promptfoo uses its SDK events to +emit provider-level command, file, search, MCP, and turn spans. Deep native +tracing is optional, not the default: it can expose additional payloads and, for +Codex, disables thread persistence. + +## Workspace contract + +`.eval/seeds` contains resolved inputs for the current job, published through a +read-only mount or owned by a bootstrap identity that the unprivileged +Promptfoo/agent user cannot modify. `.eval/workspace` is always disposable and +writable. `.eval/artifacts` may contain explicitly selected bounded diagnostics. + +The source manifest records the requested identity and resolved immutable +identity for each seed. A mutable Git ref may be an input to resolution but is +never the recorded resolved identity. OCI input, if used, records the verified +manifest digest. + +The workspace copy must not use hardlinks or any writable alias back to a seed. +A reflink or another copy-on-write primitive is acceptable only when later +writes cannot mutate the seed. A full recursive copy is the portable fallback. + +Every row starts from the same selected seed state. Workspace mutations, +installed dependencies, generated files, and provider session state must not +cross row boundaries. Cross-row provider thread persistence is disabled in V1. +Agent-started background services are unsupported because the extension cannot +guarantee process-tree cleanup between rows; disposable job teardown is the +process cleanup boundary. + +## Assertions and evidence + +Behavioral success is decided by Promptfoo assertions, not lifecycle hooks. +Rows that inspect the live workspace use deterministic assertions only. +Promptfoo may defer a row's complete assertion set when model-graded assertions +are present; a later `beforeEach` could then replace the shared workspace before +the earlier filesystem assertion executes. + +For a deterministic-only row, the filesystem assertion runs after the provider +returns and before `afterEach` removes the workspace. It may: + +- verify required files and contents; +- execute bounded commands without shell interpolation; +- check exit status, timeout, and selected output; +- inspect Promptfoo provider metadata; and +- inspect OpenTelemetry trace data or use built-in `trajectory:*` assertions. + +If model grading is required, the deterministic phase first serializes all +needed facts to a unique row artifact and a separate evaluation grades that +immutable evidence. It must not read the shared live workspace later. + +`afterEach` may add diagnostic metadata or named scores that Promptfoo permits +hooks to mutate, but it cannot override `success`, `score`, or +`response.output`. It therefore must not contain the authoritative grader. + +V1 stores Promptfoo's native result and OpenTelemetry trace exports. It does not +convert provider events to ATIF. Promptfoo's normalized trajectory view is +sufficient for tool-use, argument, sequence, step-count, and goal assertions. +An ATIF adapter may be added later at an explicit interoperability boundary; it +must not fabricate reasoning, messages, or tool results absent from provider +telemetry. + +Hidden checks are not promised by this design. Keeping a verifier outside +`working_dir` does not prove that a shell-capable agent in the same job cannot +read it. A requirement for secret verifier bytes needs a separate isolation +decision. + +## Scope + +V1 includes: + +- direct Promptfoo execution through its Claude Agent SDK and Codex SDK + providers; +- one fixed extension-managed workspace; +- serial, uncached evaluation rows; +- exact source staging and private per-row copies; +- deterministic filesystem and command assertions; +- Promptfoo-native output, metadata, usage, and OpenTelemetry traces; and +- disposable job-level cleanup. + +V1 excludes: + +- a network execution API or shared remote service; +- a custom Promptfoo provider; +- a durable run database, queue, or artifact service; +- shared mutable workspaces or cross-row provider sessions; +- gateway-owned agent adapters or grading; +- ATIF normalization; +- hostile multi-tenant isolation; +- secret post-run verifier injection; and +- resumable runs, checkpoints, or workspace recovery. + +## Consequences + +Benefits: + +- the implementation is a small Promptfoo configuration, lifecycle extension, + source-staging helper, and deterministic assertion module; +- Claude and Codex provider behavior stays aligned with Promptfoo releases; +- Promptfoo's result, trace, assertion, repetition, and report machinery remains + authoritative; +- source seeds can be reused within a job without sharing mutable workspaces; +- the fixed working directory keeps provider configuration static; and +- deleting the gateway removes a second protocol and telemetry model. + +Costs and limits: + +- V1 is a trusted single-tenant job design, not a hosted execution service; +- rows run serially inside a job; +- live-workspace assertions cannot be mixed with deferred model grading; +- provider tool, transcript, and trace coverage differs; +- `afterEach` failures need a wrapper-visible sentinel because Promptfoo logs + them rather than converting a passing row to an error; +- a process crash can bypass extension cleanup, so disposable job teardown is + required; +- Promptfoo's trace is observability data, not a lossless replayable transcript; + and +- strong credential brokering, hidden verifiers, and tenant isolation remain + unsolved because they are outside the selected scope. + +## Reconsider when + +Introduce a separate execution service only when a concrete requirement needs +one of the boundaries the native design does not provide: mutually untrusted +tenants, remote API callers, centrally enforced network policy, source/model +credential brokering, secret verifier injection, durable cancellation and +recovery, retention beyond the disposable job, or shared scheduling across +machines. + +Need for more providers, matrices, repetitions, deterministic assertions, +workspace copies, or Promptfoo trajectory checks does not by itself justify a +gateway. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md deleted file mode 100644 index 4768054b..00000000 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ /dev/null @@ -1,1049 +0,0 @@ ---- -title: "One-Shot Coding-Agent Gateway - Implementation Plan" -date: 2026-09-18 -updated: 2026-09-28 -type: feat -artifact_contract: ce-unified-plan/v1 -artifact_readiness: implementation-ready -execution: code ---- - -# One-Shot Coding-Agent Gateway - Implementation Plan - -## Goal capsule - -- **Objective:** Run one Codex or OMP coding-agent attempt in a fresh composed workspace and return bounded output, usage, bounded trajectory, provenance, and optional raw post-run evidence with authenticated artifact retrieval. -- **Means:** Build one repository and service, `allagentsdev/allagents-gateway`, containing the Promptfoo provider, gateway API, worker/runner, contracts, workspace composition, post-run collector, and agent adapters. -- **First caller:** Promptfoo is the evaluation layer. It owns rendered prompts, workspace-source JSON, matrices, repetitions, JavaScript/LLM graders, pass/fail decisions, scores, and reports. -- **Stop conditions:** Do not build reusable sessions, continuation, checkpoints, composed-workspace or snapshot caches, generic diffs, change artifacts, or a second repository. - -The authoritative decision is [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). Supporting evidence is in [One-shot coding-agent gateway boundary](../research/one-shot-coding-agent-gateway-boundary.md). - -## Product contract - -`allagentsdev/allagents-gateway` is a general one-shot coding-agent gateway. A run receives one rendered instruction and an ordered list of authorized Git, OCI, or uploaded-bundle sources, each declared `read_only` or `writable`. The worker resolves exact immutable source identities, reauthorizes and leases complete source-cache generations, mounts read-only sources directly, and makes private reflink/CoW clones for writable sources. It composes those non-overlapping destinations in one fresh sandbox, invokes one selected agent exactly once, stops and reaps the agent-owned process tree, seals the agent output/usage/trajectory record, and retains the same sandbox, runtime image, and final writable workspace for optional evidence collection. - -A request may add `post_run` evidence collection. Only after the agent has stopped, the worker strips model/source credentials, destroys the agent network namespace and flows, creates a distinct default-drop check namespace, conditionally mounts an immutable hidden bundle for bundle executables, and executes structured commands in the same retained sandbox and final workspace. Commands select either an operator-approved executable from the runtime profile's fixed read-only `PATH` or a confined executable from that optional bundle, plus literal arguments. All external and service access is denied by default and exists only through endpoints brokered for the named post-run policy; direct loopback, host, and sidecar sockets stay unreachable. Commands may compile, test, start short-lived children, and create build output. The gateway returns raw command observations and requested files as bounded inline values or authenticated tenant/run-bound artifact references. - -The gateway never decides whether behavior passes, fails, or deserves a reward. The Promptfoo provider exposes agent and post-run evidence to Promptfoo JavaScript or LLM graders, which own those decisions. - -The lifecycle status is only `completed`, `cancelled`, or `infrastructure_error`. Agent launch/crash/timeout, requested post-run bundle/sandbox/executable-resolution/command-launch failure, artifact-store failure, result-finalization failure, or cleanup failure is infrastructure failure. Every run destroys its private clones, workspace mounts, any injected bundle, and scratch storage and releases cache leases before its terminal result is published. - -V1 supports **direct mode** only, owned completely by this repository. - -## Scope - -### Included - -- One TypeScript repository with a Promptfoo provider, gateway API, worker/runner, contracts, Codex adapter, OMP adapter, post-run collector, and Linux isolation backend. -- Closed `AgentRunRequest v1`, `AgentRunResult v1`, status-envelope, bundle-upload, and post-run JSON Schemas. -- Promptfoo local and remote provider modes using the same gateway API and request/result contract. -- Promptfoo-authored ordered multi-source workspace JSON. -- Deterministic packaging of provider-local source paths and optional post-run bundle paths before gateway submission. -- Bounded composition from one through 128 authorized Git, OCI, and uploaded-bundle sources with required `read_only` or `writable` access. -- An immutable exact-source cache with authorization on every use, exact-key singleflight, complete generations, leases, watermarks, and startup reconciliation. -- A fresh private sandbox, process hierarchy, mount tree, and writable source clone for every run. -- One agent invocation with idempotency, cancellation, deadlines, cleanup, bounded final output, usage, timing, and bounded ATIF trajectory. -- Optional ordered post-run commands and requested-file collection in the same runtime and final workspace after agent stop and credential stripping. -- Operator-managed task services with separate lifecycle ownership from agent-created processes. -- Resolved provenance for every source in request order. - -### Excluded - -- Pass/fail, reward, rubric, or grading fields and decisions in the gateway contract. -- Long-lived or reusable coding sessions, additional target turns, continuation, resume, session-event replay, or checkpoints. -- Reusable composed workspaces, mutable source caches, unkeyed Git clones, prepared snapshots, or OCI workspace publication. Immutable exact-source generations are required only as specified below. -- A workspace-builder service or repository. -- Generic before/after diffs, patch output, or a core modified-workspace artifact. -- Persistence of a non-evaluation run's modified workspace in V1. That requires a separately approved artifact contract. -- Caller-provided credentials, environment variables, shell commands, container images, raw network rules, policy documents, model endpoints, or proxies. -- Automatic agent or post-run retry. Promptfoo repetitions are distinct runs; transport retry reuses one idempotency identity. - -## Fixed implementation choices - -Implementation must not choose another framework partway through delivery. - -- **Repository and service:** `allagentsdev/allagents-gateway`, Apache-2.0, protected `main`, release tags, lockfile, generated schemas checked in. -- **Runtime:** Bun 1.4 for development, tests, packaging, and worker execution; strict TypeScript; ESM; Node 22-compatible Promptfoo provider output. -- **HTTP:** Fastify 5 with strict Ajv validation against checked-in schemas. External JSON is not translated through a second hand-maintained DTO shape. -- **Persistence:** PostgreSQL stores tenant-owned upload reservations, runs, idempotency claims, state, deadlines, artifact budgets, and cache leases. An S3-compatible store holds immutable tenant/run-bound input bundles and result artifacts for agent output, command output, requested files, and trajectories. Local mode uses SQLite plus a private local artifact directory behind the same interfaces. -- **Source cache:** one V1 worker pool is one cache domain: all worker processes share an operator-owned cache root on the same reflink/CoW-capable filesystem and coordinate exact keys through PostgreSQL plus atomic filesystem publication. This guarantees one materialization for concurrent exact-key requests in the pool. Additional independent cache domains are explicit deployments and may materialize separately. The cache has a versioned materializer schema, ready markers, leases, byte/inode watermarks, exact-key singleflight, and reconciliation; it never caches a composed workspace. -- **Queue:** PostgreSQL-backed leased jobs in V1. Lease recovery may resume safe control-plane work but may not invoke an ambiguously started agent again. -- **Runtime profiles:** required `runtime_profile_id` selects a caller-authorized operator profile. Admission resolves and persists one immutable profile revision plus its `profile_digest`, runtime `image_digest`, sandbox-policy version, read-only tool implementations, phase-specific broker identities, cgroup-v2 ceilings, and service implementations/images. Dispatch uses only that pinned revision and fails before agent invocation if it is unavailable or any digest differs. The result returns sufficient immutable, nonsecret provenance to identify the exact runtime/tool/service implementations without exposing paths, launch arguments, or policy contents. -- **Isolation:** production `RunSandbox` uses an unprivileged UID/GID mapping and non-root process identity, empty Linux capabilities, `no_new_privs`, private PID/mount/IPC/UTS/network namespaces, a read-only root filesystem, minimal bounded tmpfs mounts, masked `/proc` and no exposed `/sys`, cgroup filesystem, host devices, Docker/container sockets, worker sockets, or writable control-plane mounts. A versioned explicit seccomp allowlist denies by default. Worker-owned cgroup v2 controllers mandatorily cap pids, memory, CPU, and IO for service, agent, and each check group; sandbox processes cannot modify membership or limits. The retained mount/runtime stays alive through post-run, but phase process and network namespaces are distinct. Rootless development is allowed; production is incomplete until this backend passes breakout, namespace, fork-bomb, OOM, CPU, IO, device/socket, mount, and syscall conformance. -- **Phase networking:** agent and post-run checks use different short-lived network namespaces with default-drop nftables (or an equivalent kernel enforcement point) covering external, sidecar, and loopback traffic. They receive only operator-brokered endpoints authorized by their phase policy; no direct host/sidecar socket is reachable. The worker destroys the agent namespace, conntrack/flows, and broker handles before creating the check namespace. Post-run-only services are therefore unreachable to the agent even on localhost. -- **Validation:** JSON Schema is authoritative at public and subprocess boundaries. TypeScript types are generated from it. Every owned object uses `additionalProperties: false`; unions use `oneOf` with a required discriminator. -- **Identity and time:** UUIDv7 run and artifact IDs; UTC RFC 3339 timestamps; monotonic internal durations; lowercase SHA-256 digests; byte sizes; millisecond limits. -- **Trace:** ATIF v1 is the only V1 public trajectory format. Vendor and pin its exact schema; adapters convert native events instead of creating another event model. - -## Actors and ownership - -| Actor | Owns | Must not own or receive | -|---|---|---| -| Promptfoo | Datasets, variables, prompt rendering, ordered source JSON, matrices, repetitions, JS/LLM grading, pass/fail, scores, reports | Source/model credentials, gateway policy bodies, a live workspace | -| `@allagents/promptfoo-provider` | Stable Promptfoo invocation/upload identity, local path packaging, idempotent upload/submission, polling/cancel mapping, authenticated artifact download/verification, result/evidence mapping | Agent execution, grading decisions, automatic run retry | -| Gateway API | Authentication, closed-schema validation, tenant-scoped object access, duplicate-before-admission idempotency, admission, state reads, cancellation, deterministic caller/source-address mapping, bundle ownership, model/network/runtime logical-name authorization, immutable result-artifact delivery, worker dispatch | Host paths, caller secrets, source credential selectors, policy bodies, workspace mutation | -| Worker/runner | State machine, deadlines, composition sequencing, sandbox/profile allocation, one agent invocation, process/network ownership boundaries, bounded output/artifact allocation, durable post-run sequencing, result finalization, cleanup | Prompt rendering, matrix expansion, grading | -| Workspace composer | Ordered attachment of cached immutable generations at non-overlapping destinations, access-mode enforcement, source-by-source provenance | Overwrite precedence, mutable shared trees, composed-workspace caches | -| Source cache | Exact-key materialization singleflight, immutable complete generations, leases, watermarks, eviction, reconciliation | Authorization decisions, writable agent aliases, visibility inside sandboxes except leased read-only mounts | -| `RunSandbox` | Pinned profile runtime, unprivileged private mounts/namespaces, seccomp and cgroup-v2 enforcement, distinct default-drop service/agent/check network boundaries, stop/kill boundaries | Evaluation semantics | -| Task service manager | Operator-declared sidecar startup/readiness/restart/stop in the service cgroup | Agent-created daemons, caller-defined service commands, grading | -| Codex adapter | Pinned Codex invocation, native-event normalization, final output, usage | Acquisition, post-run collection, retries | -| OMP adapter | Pinned OMP invocation, native-event normalization, final output, usage | Acquisition, post-run collection, retries | -| Post-run collector | Late bundle acquisition/verification/read-only mount after agent teardown, pinned-profile runtime and bundle-executable resolution, literal-argument execution in the retained sandbox/workspace, bounded output capture, requested-file promotion | Model/source credentials, grading, arbitrary external egress | -| Artifact store | Immutable tenant-owned input bundles and tenant/run-bound result artifacts for agent output, command output, requested files, and trajectories | Mutable run state, prepared workspaces, policy decisions | - -## Repository layout - -```text -allagents-gateway/ - apps/gateway/ # Fastify admission, status, upload, cancellation API - apps/worker/ # leased-job consumer and runner entrypoint - packages/contracts/ # schemas, generated TS types, canonicalization - packages/runner/ # state machine and direct-mode coordinator - packages/promptfoo-provider/ # local/remote provider and deterministic packager - packages/workspace/ # ordered source attachment and access modes - packages/source-cache/ # exact-key immutable generations/leases/eviction - packages/sandbox-linux/ # retained runtime, cgroups, mounts, network policy - packages/services/ # operator-declared task sidecar lifecycle - packages/adapter-codex/ # pinned Codex adapter - packages/adapter-omp/ # pinned OMP adapter - packages/post-run/ # hidden checks, output capture, file collection - packages/trace-atif/ # native events to bounded ATIF v1 - schemas/ # generated public schemas, checked in - tests/fixtures/ # local Git/OCI/bundle/agent/check fixtures -``` - -The runner depends only on `RunStore`, `ArtifactStore`, `SourceCache`, `WorkspaceComposer`, `RunSandbox`, `TaskServiceManager`, `AgentAdapter`, and `PostRunCollector`. It must not import Fastify, Promptfoo, Codex, or OMP types. The gateway API never accesses a run directory; the worker never interprets Promptfoo evaluation metadata. - -## Public contract - -### Shared scalar and path rules - -- All owned objects are closed. Unknown fields fail before a run record or directory exists. -- Strings are UTF-8, contain no NUL, and obey their byte bounds. -- Logical IDs match `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`. -- Digests match `^sha256:[0-9a-f]{64}$`. -- Relative paths are NFC-normalized POSIX paths, are not absolute, and contain neither empty, `.` nor `..` segments. `destination` and `working_directory` may be exactly `.` where explicitly allowed. -- Byte sizes and millisecond limits are safe positive JSON integers. Counts may be zero where stated. -- Timestamps are UTC RFC 3339 with `Z`. -- A bundle or artifact reference is accepted only when declared digest, declared size, stored size, and streamed digest all agree. - -### `AgentRunRequest v1` - -The authoritative file is `schemas/agent-run-request.v1.schema.json`. Its exact logical shape is: - -```text -{ - schema_version: "agent_run_request.v1", - identity: { - run_id: UUIDv7, - idempotency_key: string, 16..128 characters - }, - instruction: string, 1..1_000_000 UTF-8 bytes, - workspace: { - working_directory: relative path or ".", - sources: array of 1..128 GitSource | OciSource | UploadedBundleSource - }, - agent: CodexAgent | OmpAgent, - runtime_profile_id: logical ID, - post_run: null | PostRunSpec, - agent_network_policy_id: logical ID, - limits: { - total_timeout_ms: positive integer, - acquisition_timeout_ms: positive integer, - agent_timeout_ms: positive integer, - post_run_timeout_ms: positive integer, - max_workspace_bytes: positive integer, - max_workspace_files: positive integer, - max_agent_output_bytes: positive integer, - max_artifact_bytes: positive integer, - max_trace_bytes: positive integer, - max_command_output_bytes: positive integer - } -} -``` - -`post_run` is required and is either `null` or the closed specification below. A general run with `null` still returns a completed agent result. Promptfoo uses post-run evidence only when its graders need it. - -`runtime_profile_id` is required. At admission it resolves to one caller-authorized immutable operator profile revision and canonical `profile_digest`, including the runtime image, sandbox policy, fixed read-only executable `PATH` and tool implementations, phase-specific broker identities, cgroup ceilings, and task-service implementations. The persisted run pins that revision rather than resolving the logical ID again at dispatch. Caller JSON never contains an image, executable path, service command, implementation digest, or sandbox policy body. - -`max_agent_output_bytes`, `max_trace_bytes`, and `max_command_output_bytes` are independent capture bounds. `max_artifact_bytes` is the aggregate logical byte budget for content-addressed **result** artifacts in one run; input bundles and cache generations are excluded. Admission requires `sum(post_run.output_files[].max_bytes) <= max_artifact_bytes` and reserves that sum. Finalization charges actual collected-file bytes in request order, then releases unused reservation and spends the deterministic remainder on agent output, command stdout/stderr in command/stream order, then trajectory. Every promoted reference charges its full `size_bytes` even when storage deduplicates the digest. Exhausted optional promotion falls back to explicitly bounded inline/truncated or omitted evidence; it is not infrastructure failure and never permits unbounded persistence. - - -The three closed source variants are: - -```text -GitSource = { - kind: "git", - url: canonical HTTPS URL, 1..2_048 UTF-8 bytes, - ref: string, 1..1_024 UTF-8 bytes, - history: { mode: "full" } | - { mode: "shallow", depth: integer >= 1 and <= 1_000_000 }, - access: "read_only" | "writable", - destination: relative path or "." -} - -OciSource = { - kind: "oci", - repository: canonical OCI repository, 1..2_048 UTF-8 bytes, - descriptor: { - media_type: "application/vnd.oci.image.manifest.v1+json", - digest: SHA-256 digest, - size_bytes: positive integer - }, - access: "read_only" | "writable", - destination: relative path or "." -} - -UploadedBundleSource = { - kind: "uploaded_bundle", - bundle: BundleReference, - access: "read_only" | "writable", - destination: relative path or "." -} - -BundleReference = { - artifact_id: UUIDv7, - media_type: "application/vnd.allagents.gateway-bundle.v1.tar+gzip", - digest: SHA-256 digest, - size_bytes: positive integer -} - -ArtifactReference = { - artifact_id: UUIDv7, - media_type: nonempty IANA media type, <= 127 bytes, - digest: SHA-256 digest, - size_bytes: integer >= 0, - expires_at: timestamp -} -``` - -Source order is authoritative for acquisition logs and returned provenance. Destinations are normalized before comparison and must be pairwise non-overlapping: no destination may equal, contain, or be contained by another. `.` is allowed only when it is the sole source destination. The final `working_directory` must resolve within the composed tree, without symlink escape, to a directory. Destination overlap and working-directory syntax are rejected before credentials, network, artifact reads, or run-directory creation. Ordering never grants overwrite precedence. - -Canonical source addresses have one accepted serialization. A Git URL is HTTPS with lowercase IDNA host, omitted default port, no userinfo/query/fragment, normalized percent-encoding, and a nonempty absolute repository path. An OCI repository is lowercase `registry[:nondefault-port]/name[/name...]` under OCI Distribution name rules, with no scheme, tag, digest, userinfo, query, or fragment. The gateway parses and reserializes either form and rejects the request if bytes differ; it never silently normalizes an alias before policy lookup or cache-key construction. - -Git `ref` may be a full advertised ref, an unambiguous advertised branch/tag selector, or an exact commit permitted by source policy. It is resolved once to an exact commit. `full` retains ancestry reachable from that commit; `shallow` retains the requested bounded depth. The worker never substitutes a default branch, another ref, another commit, or another history mode. - -The closed agent variants are: - -```text -CodexAgent = { - kind: "codex", - model: logical ID, - reasoning_effort: "low" | "medium" | "high" -} - -OmpAgent = { - kind: "omp", - model: logical ID -} -``` - -`model`, `runtime_profile_id`, `agent_network_policy_id`, and `post_run.network_policy_id` are logical names resolved under authenticated operator configuration. Git `url` and OCI `repository` are canonical source addresses, never credential or policy selectors. Before credentials or network access, the gateway deterministically maps `(authenticated caller identity, source kind, canonical URL/repository)` to exactly one internal authorization/transport/credential route; zero or multiple matches return `source_policy_mapping_failed`. Uploaded bundles are authorized by the authenticated caller's access to `artifact_id`. The caller can never select an internal source identity, credential route, runtime image, tool path, sandbox policy, or service command. Runtime, model, and both phase-network policy IDs are authorized independently. Limits are caller-lowerable: callers send explicit values no greater than the selected profile/configured ceilings. The gateway rejects a value over a ceiling instead of silently changing it. Provider defaults are applied before request construction. - - -The request can never contain a host path, credential, environment map, shell string, raw network destination, proxy, runtime image, policy body, retry count, output directory, grading rubric, pass/fail expectation, reward, patch request, or modified-workspace persistence option. - -### `PostRunSpec v1` - -```text -PostRunSpec = { - bundle?: BundleReference, - network_policy_id: logical ID, - commands: array of 0..32 PostRunCommand, - output_files: array of 0..128 OutputFileRequest -} - -PostRunCommand = { - command_id: logical ID, - executable: RuntimeExecutable | BundleExecutable, - args: array of 0..63 strings, each 0..4_096 UTF-8 bytes, - timeout_ms: positive integer -} - -RuntimeExecutable = { - kind: "runtime", - name: string matching ^[A-Za-z0-9][A-Za-z0-9._+-]{0,127}$ -} - -BundleExecutable = { - kind: "bundle", - path: relative path -} - -OutputFileRequest = { - name: logical ID, - path: relative path, - media_type: nonempty IANA media type, <= 127 bytes, - max_bytes: positive integer -} -``` - -At least one of `commands` or `output_files` is nonempty. Command IDs and output-file names are unique. `bundle` is present if and only if at least one command uses `BundleExecutable`; output-only collection and runtime-only commands omit it. A runtime executable name resolves only through the operator-approved fixed `PATH` of read-only runtime directories; it contains no slash, does not search the workspace, and must resolve to a regular executable beneath an approved directory. A bundle executable path resolves beneath the immutable hidden bundle without symlink escape. `args` are literal. No shell parsing, interpolation, globbing, caller environment, working-directory override, or other executable lookup is allowed. Each `timeout_ms` must be no greater than both `limits.post_run_timeout_ms` and the remaining total deadline. Each output-file path is relative to the final workspace root and is collected after all commands settle. `max_bytes` must not exceed `limits.max_artifact_bytes`. - -Commands run sequentially in request order in the same retained sandbox, runtime image, and final workspace used by the agent, with current directory set to `workspace.working_directory`. A nonzero exit is recorded and does not stop later commands. A per-command timeout kills and reaps that command's process group, records a `timed_out` observation, and continues if the post-run and total deadlines permit. Failure to validate/extract/inject a requested bundle, resolve an approved executable, enforce the post-run sandbox/network policy, or launch a command is infrastructure failure and aborts remaining collection. - -### `AgentRunResult v1` - -The authoritative file is `schemas/agent-run-result.v1.schema.json`. Its exact common shape is: - -```text -{ - schema_version: "agent_run_result.v1", - identity: { - run_id: UUIDv7, - idempotency_key: string - }, - status: "completed" | "cancelled" | "infrastructure_error", - post_run_mode: "none" | "requested", - effective_limits: { same ten required fields as request.limits }, - workspace_provenance: DirectWorkspaceProvenance | null, - runtime_provenance: RuntimeProvenance | null, - agent: AgentResult | null, - post_run: PostRunEvidence | null, - usage: Usage, - timing: Timing, - trajectory: InlineTrajectory | ArtifactTrajectory | null, - error: RunError | null -} -``` - -The schema encodes these variants: - -- `completed` + `post_run_mode: "none"`: non-null workspace/runtime provenance, `agent.termination: "completed"`, non-null bounded `CapturedText` final output (empty inline text is valid), usage, and trajectory; `post_run: null`; `error: null`. -- `completed` + `post_run_mode: "requested"`: the same completed agent fields plus non-null `PostRunEvidence`; every command observation is `completed` or `timed_out`, and every file observation is `collected`, `missing`, `limit_exceeded`, or `not_regular_file`; `error: null`. -- `cancelled`: `error.code: "run_cancelled"` and any already durable provenance/agent/usage/trajectory. If post-run initialization was durably recorded, `post_run` is a full request-order observation vector containing sealed observations plus `not_run` entries for work prevented by cancellation; otherwise it is null. -- `infrastructure_error`: a non-cancellation error plus any already durable provenance, agent, usage, trajectory, and `PostRunEvidence`. If post-run initialization was durable, the evidence vector contains sealed observations, `unavailable` for the item whose evidence/launch/collection failed, and `not_run` for later work; otherwise it is null. Cleanup failure preserves the fully sealed vector and can never publish `completed`. - -A completed run always exposes bounded agent final output, usage, and a valid trajectory even if no post-run evidence was requested or no files changed. Nonzero exits, timed-out commands, missing files, oversized files, and non-regular paths are behavioral observations and do not change lifecycle status. Any `unavailable` observation implies `infrastructure_error`; `not_run` appears only on cancelled or infrastructure-error results. Mandatory result-finalization or cleanup failure downgrades the terminal candidate to `infrastructure_error`, retaining already durable evidence. The result remains invisible until cleanup succeeds and cache leases are released. - -Closed result components are: - -```text -DirectWorkspaceProvenance = { - kind: "direct", - working_directory: string, - sources: array in request order of - GitSourceProvenance | OciSourceProvenance | UploadedSourceProvenance -} - -GitSourceProvenance = { - kind: "git", - url: canonical HTTPS URL, - requested_ref: string, - resolved_commit: 40 lowercase hexadecimal characters, - history: { mode: "full" } | - { mode: "shallow", depth: positive integer }, - access: "read_only" | "writable", - materializer_schema: 1, - destination: string, - tree_digest: SHA-256 digest -} - -OciSourceProvenance = { - kind: "oci", - repository: canonical OCI repository, - requested_descriptor: OciSource.descriptor, - resolved_manifest_digest: SHA-256 digest, - rootfs_digest: SHA-256 digest, - access: "read_only" | "writable", - materializer_schema: 1, - destination: string -} - -UploadedSourceProvenance = { - kind: "uploaded_bundle", - bundle: BundleReference, - rootfs_digest: SHA-256 digest, - access: "read_only" | "writable", - materializer_schema: 1, - destination: string -} - - -RuntimeProvenance = { - runtime_profile_id: logical ID, - profile_digest: SHA-256 digest, - image_digest: SHA-256 digest, - sandbox_policy_version: string, - tools: array of 0..128 { - name: logical ID, - version: string, - implementation_digest: SHA-256 digest - }, - services: array of 0..32 { - name: logical ID, - version: string, - implementation_digest: SHA-256 digest, - image_digest: SHA-256 digest | null - } -} - -`profile_digest` identifies the canonical immutable nonsecret resolved-profile manifest. A tool `implementation_digest` commits to its executable bytes and immutable runtime dependency closure. A service `implementation_digest` commits to its credential-free canonical launch manifest and executable closure; `image_digest` is non-null exactly for a containerized service and names its immutable image. The `tools` and `services` arrays enumerate the entire pinned revision with unique names sorted by name. Operator profiles never put credentials or secret values in commands, arguments, environment, or these manifests; brokers supply secrets out of process. Versions are descriptive, while digests are the implementation identities. - -AgentResult = { - kind: "codex" | "omp", - model: logical ID, - adapter_version: string, - termination: "completed" | "cancelled" | "timed_out" | "failed", - exit_code: integer | null, - final_output: CapturedText | null -} - -CapturedText = - { storage: "inline", encoding: "utf-8", text: string, - digest: SHA-256 digest, size_bytes: integer >= 0, truncated: boolean } | - { storage: "artifact", encoding: "utf-8", - artifact: ArtifactReference, truncated: boolean } -``` - -Agent output capture retains at most `max_agent_output_bytes` of valid UTF-8 and sets `truncated: true` if additional bytes existed. At most 64 KiB is inline. Larger retained text uses a result artifact when its deterministic artifact-budget turn fits; otherwise it returns the first at-most-64-KiB UTF-8 prefix inline with `truncated: true`. Digests and sizes describe returned bytes. The provider verifies and decodes inline text or downloads/verifies the artifact before giving Promptfoo the final string; truncation stays visible in metadata. - -`PostRunEvidence` contains raw observations only: - -```text -PostRunEvidence = { - commands: one PostRunCommandObservation per requested command, in order, - output_files: one OutputFileObservation per requested file, in order -} - -PostRunCommandObservation = - { status: "completed", command_id, executable, args, - exit_code: integer | null, signal: integer | null, - duration_ms: integer >= 0, stdout: CapturedOutput, stderr: CapturedOutput } | - { status: "timed_out", command_id, executable, args, - signal: integer | null, duration_ms: integer >= 0, - stdout: CapturedOutput, stderr: CapturedOutput } | - { status: "not_run", command_id, executable, args, - reason: "cancelled" | "prior_infrastructure_error" | "deadline" } | - { status: "unavailable", command_id, executable, args, - reason: "cancelled" | "deadline" | "launch_failed" | "sandbox_failed" | - "evidence_persistence_failed", - duration_ms: integer >= 0 | null, - stdout: CapturedOutput | null, stderr: CapturedOutput | null } - -CapturedOutput = - { storage: "inline", encoding: "utf-8", text: string, - digest: SHA-256 digest, size_bytes: integer >= 0, truncated: boolean } | - { storage: "artifact", artifact: ArtifactReference, truncated: boolean } | - { storage: "omitted", digest: SHA-256 digest, - size_bytes: integer >= 0, truncated: true, - reason: "artifact_budget_exhausted" } - -OutputFileObservation = - { name: logical ID, path: string, status: "collected", - media_type: string, digest: SHA-256 digest, size_bytes: integer >= 0, - artifact: ArtifactReference } | - { name: logical ID, path: string, status: "missing" } | - { name: logical ID, path: string, status: "limit_exceeded", - observed_size_bytes: integer >= 0 } | - { name: logical ID, path: string, status: "not_regular_file" } | - { name: logical ID, path: string, status: "not_run", - reason: "cancelled" | "prior_infrastructure_error" | "deadline" } | - { name: logical ID, path: string, status: "unavailable", - reason: "cancelled" | "deadline" | "unsafe_path" | - "changed_during_read" | "read_failed" | - "artifact_persistence_failed" } -``` - -The worker durably seals each `completed` or `timed_out` command observation before starting the next command and each file observation before collecting the next file. Post-run initialization durably records the ordered item skeleton. On cancellation or infrastructure failure it seals the in-flight item as `unavailable` with any safely captured bounded streams and fills all untouched items as `not_run`, so a published non-null `PostRunEvidence` always has exactly one entry per request item. - -For a completed command, exit code and signal are not both null; nonzero exit is ordinary evidence. A timed-out command records partial bounded streams after its cgroup is reaped. Each stream captures at most `max_command_output_bytes`. Valid UTF-8 at or below 64 KiB is inline; larger or binary bytes use an artifact if budget remains. Without budget, valid UTF-8 falls back to a truncated inline prefix and binary bytes use `omitted`; budget exhaustion is never infrastructure failure. The result echoes validated executable/args, never a resolved host path. - -Requested files are read after commands settle without following the final path as a symlink. Missing, over-limit, and non-regular paths are completed observations. Unsafe races/read failures become `unavailable` and infrastructure error. Because requested-file maxima were reserved at admission, any valid collected file fits the artifact byte budget; artifact-store failure marks that item unavailable while retaining prior observations. - - -```text -Usage = { - input_tokens: integer >= 0 | null, - cached_input_tokens: integer >= 0 | null, - output_tokens: integer >= 0 | null, - reasoning_tokens: integer >= 0 | null, - tool_calls: integer >= 0, - estimated_cost_usd: finite number >= 0 | null, - source: "adapter_reported" | "partially_reported" | "unavailable" -} - -Timing = { - accepted_at: timestamp, - started_at: timestamp | null, - agent_started_at: timestamp | null, - agent_stopped_at: timestamp | null, - post_run_started_at: timestamp | null, - completed_at: timestamp, - queue_ms: integer >= 0, - acquisition_ms: integer >= 0, - agent_ms: integer >= 0, - post_run_ms: integer >= 0, - cleanup_ms: integer >= 0, - total_ms: integer >= 0 -} - -InlineTrajectory = { - storage: "inline", - format: "atif-v1", - digest: SHA-256 digest, - size_bytes: integer >= 0, - truncated: boolean, - document: ATIF-v1 document -} - -ArtifactTrajectory = { - storage: "artifact", - format: "atif-v1", - digest: SHA-256 digest, - size_bytes: integer >= 0, - truncated: boolean, - artifact: ArtifactReference -} - -RunError = { - code: StableErrorCode, - message: sanitized string, 1..1_024 UTF-8 bytes, - retryable: boolean, - phase: "admission" | "queue" | "acquisition" | "agent" | - "sealing" | "post_run" | "finalization" | "cleanup" | - "cancellation" -} -``` - -ATIF documents are schema-validated before publication. One byte-counting recorder owns `max_trace_bytes`; at the limit it emits one valid truncation marker and refuses further payload without corrupting the document. Up to 256 KiB is inline. A larger document uses an artifact at the final artifact-budget priority. If that promotion does not fit, the recorder emits a valid at-most-256-KiB inline ATIF document ending in the same truncation marker. Artifact-budget exhaustion is not infrastructure failure, and completed runs never omit the trajectory. - -## Stable errors and consequences - -| Code | API/result consequence | `retryable` | -|---|---|---:| -| `invalid_request` | HTTP 400; no run record, fetch, or directory | false | -| `unauthenticated` | HTTP 401; no run record | false | -| `forbidden` | HTTP 403; authenticated caller lacks policy permission; no public run | false | -| `object_not_found` | non-enumerating HTTP 404 for absent or cross-tenant run/upload/result artifact | false | -| `source_policy_mapping_failed` | HTTP 403; no credentials resolved, network request, or public run | false | -| `runtime_profile_forbidden` | HTTP 403; profile not authorized for caller; no public run | false | -| `post_run_executable_forbidden` | HTTP 403; runtime name absent from selected profile; no public run | false | -| `bundle_integrity_failed` | HTTP 422; reservation unusable; no bundle published | false | -| `upload_id_conflict` | HTTP 409; original tenant/upload reservation unchanged | false | -| `idempotency_conflict` | HTTP 409; original run unchanged | false | -| `admission_limit_exceeded` | HTTP 422; no public run | false | -| `queue_unavailable` | `infrastructure_error`; no agent invocation | true | -| `runtime_profile_admission_failed` | HTTP 503; authorized immutable revision cannot be resolved and verified; no public run | true | -| `runtime_profile_unavailable` | `infrastructure_error`; pinned revision or implementation unavailable at dispatch; no agent invocation | true | -| `runtime_profile_integrity_failed` | `infrastructure_error`; pinned profile/runtime/tool/service digest mismatch; no agent invocation | false | -| `workspace_acquisition_failed` | `infrastructure_error`; no agent invocation | false | -| `source_authorization_revoked` | `infrastructure_error`; cached generation is not attached; no agent invocation | false | -| `source_cache_capacity_exceeded` | `infrastructure_error`; no unsafe eviction; no agent invocation | true | -| `source_cache_integrity_failed` | `infrastructure_error`; generation quarantined; no agent invocation | false | -| `workspace_integrity_failed` | `infrastructure_error`; offending source quarantined for operators | false | -| `workspace_limit_exceeded` | `infrastructure_error`; partial tree removed | false | -| `agent_start_failed` | `infrastructure_error`; no post-run collection | false | -| `agent_failed` | `infrastructure_error`; no post-run collection | false | -| `agent_timed_out` | `infrastructure_error`; process tree killed; no post-run collection | false | -| `agent_invocation_ambiguous` | `infrastructure_error`; recovery refuses a second invocation | false | -| `agent_seal_failed` | `infrastructure_error`; no post-run collection | false | -| `post_run_bundle_failed` | `infrastructure_error`; requested bundle could not be safely injected; ordered unavailable/not-run evidence retained | false | -| `post_run_executable_resolution_failed` | `infrastructure_error`; an approved runtime or bundle executable could not be resolved safely; partial evidence retained | false | -| `post_run_sandbox_failed` | `infrastructure_error`; ordered unavailable/not-run evidence retained | false | -| `task_service_failed` | `infrastructure_error`; ordered unavailable/not-run evidence retained | true | -| `post_run_command_launch_failed` | `infrastructure_error`; failed command unavailable, later work not-run, earlier evidence retained | false | -| `post_run_collection_failed` | `infrastructure_error`; failed file unavailable, later files not-run, earlier evidence retained | false | -| `trace_finalization_failed` | `infrastructure_error`; other durable evidence retained | false | -| `result_finalization_failed` | `infrastructure_error`; no completed result published | false | -| `artifact_integrity_failed` | HTTP 502 on result-artifact download; provider reports infrastructure failure | true | -| `artifact_expired` | HTTP 410 on an expired result artifact | false | -| `cleanup_failed` | terminal candidate becomes `infrastructure_error`; durable evidence retained; publication waits for successful reconciled cleanup and lease release | true | -| `run_timed_out` | `infrastructure_error`; active processes killed and cleaned | false | -| `run_cancelled` | `cancelled`; active processes killed and cleaned | false | - -`retryable: true` means only that an explicit new run may succeed after transient operator recovery; it never permits the worker or provider to rerun an agent, and retrying the same idempotency identity returns the same terminal run. - -A command's nonzero exit or per-command timeout is deliberately absent from this table: both are observations in `PostRunCommandObservation`. Public errors never contain credentials, policy bodies, private endpoints, headers, host paths, resolved executable paths, raw model transport, or command stderr. Operator-only diagnostics use `run_id` correlation. - -## Gateway API, uploads, and idempotency - -```text -POST /v1/bundles # idempotent authenticated upload reservation -PUT /v1/bundles/{artifact_id} # idempotent exact-byte upload -POST /v1/runs # submit AgentRunRequest v1 -GET /v1/runs/{run_id} # current state or terminal AgentRunResult v1 -POST /v1/runs/{run_id}/cancel -GET /v1/artifacts/{artifact_id} # immutable result-artifact bytes -``` - -Every upload reservation, stored input bundle, run, result, and result artifact persists a non-public `owner_tenant`. Run/result artifacts also persist `run_id` and evidence purpose. Every duplicate-submit lookup, upload PUT, run GET, cancel, and artifact GET resolves the tuple `(authenticated_tenant, object_id)`; an absent or other-tenant object returns the same non-enumerating HTTP 404. UUIDv7 is never authorization. Cancellation and artifact access require the same tenant/run authorization as result polling. - -`POST /v1/bundles` accepts the closed body below. `(owner_tenant, upload_id)` is unique. A first reservation returns HTTP 201 and its `BundleReference`; an exact retry with identical media type/digest/size returns HTTP 200 and the same reference; any changed metadata returns `upload_id_conflict` without mutating the original. The provider persists `upload_id` before the request and reuses it after response loss. - -```text -{ - schema_version: "bundle_reservation_request.v1", - upload_id: UUIDv7, - media_type: "application/vnd.allagents.gateway-bundle.v1.tar+gzip", - digest: SHA-256 digest, - size_bytes: positive integer -} -``` - -`PUT /v1/bundles/{artifact_id}` accepts exactly the reserved size, verifies the streamed digest, and atomically marks the reservation usable. Repeating the same completed upload returns success without rewriting bytes; failed or different bytes can never mutate it. Reservations expire tenant-scoped. Input/source/post-run bundles are not result artifacts and are never readable through `/v1/artifacts`. - -Submission, polling, and cancellation return the same closed status envelope: - -```text -{ - schema_version: "agent_run_status.v1", - run_id: UUIDv7, - state: "received" | "queued" | "acquiring_workspace" | - "preparing_run" | "running_agent" | "stopping_agent" | - "sealing_agent_result" | "preparing_post_run" | - "running_post_run" | "finalizing" | "cleaning" | - "cancelling" | "failing" | "stopping_processes" | - "completed" | "cancelled" | "infrastructure_error", - terminal: boolean, - result: AgentRunResult | null -} -``` - -For nonterminal states, `terminal` is `false` and `result` is null; terminal states require a matching result. A new admitted run returns HTTP 202. An exact duplicate or poll of a terminal run returns HTTP 200. Accepted cancellation returns HTTP 202; a terminal run returns HTTP 200 unchanged. - -`GET /v1/artifacts/{artifact_id}` serves only unexpired immutable **result** artifacts linked to a run visible to the authenticated tenant. A successful response streams exact bytes without redirect and includes `Content-Type`, `Content-Length`, `Digest: sha-256=...`, immutable `ETag`, and `Cache-Control: private, immutable`; these encode the `ArtifactReference` media type, size, and digest. Other-tenant, unknown, input-bundle, and detached artifacts are indistinguishable HTTP 404. An authorized reference past `expires_at` returns HTTP 410; its tenant/run metadata tombstone remains through result retention even after bytes are deleted asynchronously. The service re-verifies stored size/digest while streaming and returns `artifact_integrity_failed` instead of corrupt bytes. The provider downloads every referenced final output, command stream, requested file, or trajectory through this endpoint and verifies headers and bytes before use. - -Run submission order is normative: - -1. Authenticate the caller; validate the closed schema; canonicalize and hash the complete request. -2. Look up both `(owner_tenant, run_id)` and `(owner_tenant, idempotency_key)` before mutable admission. If both identify the same request digest/run, return that existing status without rechecking current source/profile/network policy or bundle mutability. Any identity/digest mismatch returns `idempotency_conflict`. Other tenants' identities are outside this lookup. -3. For a genuinely new identity, acquire an internal tenant/identity submission claim. Concurrent exact submissions wait on that claim; it is not a public run and has bounded crash recovery. -4. Under the claim, perform all mutable admission: deterministic source-route authorization, uploaded-bundle ownership/usability, model/runtime-profile/network-policy authorization, runtime executable allowlist, limit ceilings, destination/path rules, and aggregate requested-file artifact reservation. Resolve the authorized `runtime_profile_id` once to an immutable revision and verify its canonical `profile_digest`, runtime image digest, tool/service implementation digests, and any service image digests. An admission failure returns its HTTP error and creates no public run or job. -5. On successful admission, one transaction writes the tenant-owned public run in `received`, pins canonical request/digest plus the complete immutable resolved-profile revision and digests, absolute deadlines, and artifact reservation, enqueues one job, then completes the claim. Exact retries now take step 2. - -The provider may retry upload reservation, byte upload, run submit, poll, cancel, and artifact download with the same identities. It never creates a replacement run for transport loss. A Promptfoo repetition creates a new run. Duplicate submission does not re-admit mutable policy, but worker execution still reauthorizes each source before resolution and cache attachment. Worker recovery may repeat safe acquisition or cleanup; if durable state cannot prove the agent was never invoked, it records infrastructure error instead of invoking again. - -## State machine - -Persist every transition with a monotonic sequence and compare-and-swap expected state: - -```text -received -> queued -> acquiring_workspace -> preparing_run - -> running_agent -> stopping_agent -> sealing_agent_result - -> preparing_post_run -> running_post_run - -> finalizing -> cleaning -> completed - -sealing_agent_result -> finalizing # post_run_mode = none -any nonterminal state -> cancelling -> stopping_processes -> cleaning -> cancelled -any nonterminal state -> failing -> stopping_processes -> cleaning -> infrastructure_error -cleaning -> cleaning # failed attempt downgrades candidate to cleanup_failed; retry/reconcile -``` - -Rules: - -1. Authentication, schema/canonical hashing, tenant-scoped duplicate/conflict lookup, and all mutable policy/resource admission complete before `received`; admission failure creates no public state-machine record. Successful admission atomically creates the tenant-owned run and one queued job. -2. Queue work is leased. Lease expiry may repeat authorization, cache attachment, or cleanup only before the durable agent-invocation marker. -3. Entering `running_agent` atomically records the sole permitted invocation number, `1`. -4. `stopping_agent` gracefully terminates, waits a bounded grace period, kills the agent cgroup, closes the model proxy and agent egress, and confirms no agent-owned process remains. It does not kill operator-owned task services in the separate service cgroup. -5. `sealing_agent_result` finalizes and durably records bounded `CapturedText`, usage, and trajectory, closes agent-owned descriptors, removes adapter home/model/source credentials, and changes process ownership from agent to post-run. The final workspace remains in the same sandbox and keeps its declared source permissions. -6. If post-run work was requested, `preparing_post_run` durably writes the full ordered `PostRunEvidence` skeleton, applies the independently authorized post-run network policy, and—only after the agent teardown and credential-removal invariants hold—acquires, verifies, extracts, and mounts requested bundle bytes at the reserved late-mount point. It then resolves executables from the pinned profile revision and starts or health-checks its declared task services. -7. `running_post_run` executes commands and collects files in order, durably sealing each observation before advancing. Cancellation or infrastructure failure fills the current/later entries with `unavailable`/`not_run`. It is reachable only after agent stop and result-seal confirmation. -8. With `post_run: null`, the run skips from sealing to finalization. -9. Cancellation is first-writer-wins against terminalization. Accepted cancellation yields `cancelled` only if cleanup completes without error; a cleanup fault overrides it with `cleanup_failed`/`infrastructure_error`. Late cancellation returns the already published terminal result. -10. The total deadline dominates phase deadlines. Total expiry is infrastructure error; a command's own timeout is a completed observation if cleanup completes within the remaining total deadline. -11. `stopping_processes` revokes remaining network/model access and kills agent, check, service, acquisition, helper, and descendant cgroups before cleanup. -12. `cleaning` unmounts read-only generations, deletes private writable clones and any injected bundle/scratch/root, stops task services, and releases source-cache leases only after all mounts are gone. Any cleanup-step failure durably changes the terminal candidate—even a completed or cancelled candidate—to `infrastructure_error` with `error.code: "cleanup_failed"` while preserving already sealed evidence. The run remains nonterminal in `cleaning`; bounded retries and startup reconciliation continue, and leases needed for safety remain held until the associated mounts are gone. -13. Terminal states are immutable. The API exposes `AgentRunResult` only after process/cgroup absence, per-run storage deletion, unmount, task-service stop, and cache-lease release are all verified. Only then does it finalize `cleanup_ms`/`completed_at` and atomically publish the result. Cleanup failure can therefore never leak a `completed` result; after reconciliation succeeds it publishes `infrastructure_error` with the durable evidence. - -## Main direct-mode flow - -1. **Author.** Promptfoo defines the rendered instruction, ordered sources/access, agent and required `runtime_profile_id` matrix values, repetition, optional raw post-run evidence, separate agent/post-run network policy names, and caller-lowerable limits. -2. **Package local paths.** The provider deterministically packages local sources and an optional post-run bundle, persists one `upload_id` per package, idempotently reserves/uploads immutable bytes, and substitutes `BundleReference` objects. Runtime-only/output-only post-run configurations upload no bundle; gateway JSON never contains a host path. -3. **Submit.** The gateway follows the normative ordering above: authenticate/schema/hash, tenant-scoped existing identity lookup, then an internal new-identity claim, mutable source/bundle/model/runtime/network/limit/artifact-budget admission, and only then one tenant-owned public run/job. Exact duplicates bypass mutable re-admission. -4. **Dispatch.** The worker leases the job, loads only the profile revision pinned at admission, and re-verifies its `profile_digest` plus runtime/tool/service implementation digests. It creates a mode-0700 run root and retained mount/runtime with the profile's unprivileged identity, read-only root, namespace/seccomp/cgroup limits, separate service/agent/check process and network boundaries, private source-clone locations, adapter home, output, scratch, and an **empty** worker-reserved late-mount point. It neither acquires nor mounts post-run bundle bytes before agent teardown; no cross-run writable or control-plane mount exists. -5. **Resolve and compose.** In request order, reauthorize each source, resolve its exact cache key, lease/build one complete immutable generation, then attach it. Mount `read_only` generations directly read-only. For `writable`, create a private reflink/CoW clone or bounded full copy. Validate the working directory and record source plus runtime provenance. -6. **Prepare services and agent.** Start only task services declared by the pinned immutable profile revision, using their verified implementation and image digests, each behind phase-specific broker endpoints; then invoke the agent in its own cgroup/process/network namespace. Default-drop policy exposes only `agent_network_policy_id` endpoints and the credential-free model proxy; post-run-only services and direct loopback/sidecar sockets are unreachable. -7. **Run once.** The adapter invokes its pinned CLI directly without a shell. It captures at most `max_agent_output_bytes` into `CapturedText`, normalizes usage, and writes bounded ATIF. There is no worker retry. -8. **Stop and seal the agent result.** Stop/reap the agent cgroup, destroy the agent network namespace/flows/broker handles, finalize bounded output/usage/ATIF, and remove model/source credentials and agent home. Keep the mount/runtime image, final workspace, source attachments, and operator service cgroup alive. -9. **Optionally collect evidence.** After the agent cgroup/network namespace/flows are gone and credentials/home removed, durably initialize the ordered evidence skeleton and create a distinct default-drop check network namespace for `post_run.network_policy_id`. Only now acquire and verify requested bundle bytes, extract them safely, and mount them read-only at the previously empty reserved point. Health-check profile-declared services, resolve runtime tools from the pinned revision's `PATH`, then execute/collect in the same final workspace, sealing each observation before advancing. No grading occurs. -10. **Allocate artifacts and seal evidence.** Charge actual requested files against their admission reservation, then deterministically promote agent output, command streams, and trajectory. Use bounded inline/truncated/omitted fallbacks when optional promotion does not fit. Persist artifacts with owner tenant/run/purpose/expiry and the pending outcome; do not create a public result/timing completion yet. -11. **Destroy and publish.** Kill check/service cgroups and namespaces, remove any bundle/scratch, unmount sources, delete clones/root, release cache leases, and verify absence. A fault downgrades the pending outcome to `cleanup_failed` while retaining sealed evidence and hiding the result through reconciliation. After verified cleanup, finalize timing/runtime provenance and atomically publish. -12. **Report.** The provider maps verified inline or downloaded `CapturedText` and usage to Promptfoo, dereferences/verifies every result artifact, and exposes the complete run/evidence under `metadata.allagents_run`. Promptfoo graders alone decide outcomes. Cancellation, infrastructure, artifact expiry, or artifact integrity errors remain typed provider errors. - -## Workspace composition and immutable source cache - -### Admission and common bounds - -Validate source count, required access mode, destinations, destination relationships, bundle ownership, and working-directory syntax before any source side effect. For each Git/OCI source, deterministically resolve the authenticated caller identity plus canonical URL/repository to exactly one internal authorization/transport/credential route; reject zero or ambiguous matches before credentials or network access. Reauthorize that mapping before ref/descriptor acquisition and again before attaching any cache generation, including a local hit. Reauthorize uploaded bundles through the caller's artifact-store access before cache attachment. A cached generation proves content identity, never current caller authorization. - -For every network connection, re-authorize the full canonical path-specific Git URL or OCI repository route, TLS name, port, and each resolved IP. V1 rejects every HTTP redirect for Git manifests/refs, OCI manifests/blobs, and OCI authentication; it never broadens a route to origin scope. Enforce TLS and reject loopback, link-local, private, metadata-service, Unix-socket, and non-allowlisted targets unless the exact operator route owns that destination. - -Enforce aggregate workspace byte/file limits across attached source generations and private clones. Materialization also caps path length, component count, archive entries, headers, compression ratio, subprocesses, output, and acquisition time. Reject absolute paths, `..`, NUL, duplicate normalized paths, case-fold collisions, devices, sockets, FIFOs, set-ID bits, capabilities, unsafe hardlinks, and escaping symlinks. Preserve only regular files, directories, confined symlinks, executable mode, and deterministic ownership. - -### Cache keys and generations - -The source cache stores immutable, self-contained source generations, never a composed workspace. Key the canonical request below by SHA-256: - -```text -GitCacheKey = { - materializer_schema: 1, - kind: "git", - canonical_url: string, - resolved_commit: 40 lowercase hexadecimal characters, - history: { mode: "full" } | - { mode: "shallow", depth: positive integer } -} - -OciCacheKey = { - materializer_schema: 1, - kind: "oci", - canonical_repository: string, - descriptor: exact direct OCI descriptor -} - -UploadedCacheKey = { - materializer_schema: 1, - kind: "uploaded_bundle", - artifact_digest: SHA-256 digest, - artifact_size_bytes: positive integer -} -``` - -The access mode and destination are not cache-key fields: both affect attachment, not immutable bytes. A schema/version change creates a new key; no compatibility fallback is allowed. - -Each ready directory contains one closed manifest and no credentials or transport metadata: - -```text -{ - schema_version: "source_cache_generation.v1", - generation_id: UUIDv7, - cache_key_digest: SHA-256 digest, - materializer_schema: 1, - kind: "git" | "oci" | "uploaded_bundle", - content_digest: SHA-256 digest, - size_bytes: integer >= 0, - inode_count: integer >= 1, - created_at: timestamp -} -``` - -The durable lease row is exactly `{generation_id, run_id, worker_id, expires_at}` with a unique `(generation_id, run_id)` key. The worker renews `expires_at`; expiry is a reconciliation alarm, never by itself proof that eviction is safe. Only a provably terminal/absent run permits stale-lease repair. - -For an exact-key miss, one worker owns a singleflight materialization and all concurrent callers wait under their own acquisition deadlines. The owner creates a cache-owned private staging directory, fetches/unpacks and validates within bounds, removes acquisition credentials/transient locks, computes its content digest/bytes/inodes, writes the closed generation manifest and ready marker inside staging, makes the generation immutable to worker and sandbox identities, fsyncs data/directories, then atomically renames staging to the key's final path. Waiters lease only a final-path generation whose ready manifest validates. Failure or cancellation removes/quarantines staging and returns the same infrastructure cause to current waiters; a partial generation is never a hit. - -Every attachment acquires a durable lease `(generation_id, run_id, worker_id, expires_at)` after reauthorization and before mount/clone. The worker renews it through agent and post-run execution. Cleanup releases it only after mounts/clones are gone. Eviction never selects a leased generation. - -Track byte and inode high/low watermarks. Crossing either high watermark evicts least-recently-leased unleased generations until both low watermarks are satisfied. Admission fails with `source_cache_capacity_exceeded` when sufficient safe space cannot be made. Startup and periodic reconciliation remove incomplete staging after its bounded grace period, validate ready manifests against directory metadata, repair only leases whose run is provably terminal/absent, quarantine corrupt generations, and resume watermark eviction. Reconciliation never guesses that a live lease is stale from age alone. - -Cache directories are owned by a dedicated host identity. The sandbox cannot browse the cache root. It sees only an explicitly leased generation through a read-only bind mount, or a private clone located inside its run root. No cache file is ever exposed through a shared writable alias. - -### Attachment by access mode - -- `read_only`: bind-mount the leased generation at its destination with `ro,nosuid,nodev`; executable-file handling follows the fixed runtime policy. The agent and post-run checks see the same read-only destination. Filesystem writes and Git writes fail. -- `writable`: create a per-run reflink/CoW clone of the entire generation, including required Git history, inside the private run root. If reflink/CoW is unavailable, make a bounded full copy. Never hardlink cache files and never share a writable upper layer or clone across runs. Mount the clone writable at its destination. -- Build/test output that targets a `read_only` destination must be redirected to a writable source, scratch, or another writable destination. The gateway does not silently promote access. - -### Git source - -- Accept canonical HTTPS URL and the defined ref/history fields only. -- Map authenticated caller identity plus canonical URL to exactly one internal transport/credential route in the worker control process; the request never names that route. -- Resolve the requested selector to an exact commit, derive `GitCacheKey`, reauthorize, then hit or singleflight the generation. -- On a miss, invoke pinned Git without a shell under sanitized config. Disable prompts, hooks, filters, credential persistence, alternates, submodules, LFS smudge, local/file transports, inherited config, and inherited proxies. Fetch complete reachable ancestry for `full` or exactly the requested bounded depth for `shallow`. -- Configure Git transport to reject HTTP redirects. Credentials are scoped to the exact matched canonical URL path and are never sent to another path, origin, helper, or submodule. -- Remove credential-bearing remote state and transient locks; verify the self-contained generation and checked-out commit before publish. -- When any Git source is attached `read_only`, set `GIT_OPTIONAL_LOCKS=0` for agent and post-run processes. Read-only Git commands must work without optional lock writes; mutating Git and filesystem commands must fail. -- Record requested ref, resolved commit, history, access, materializer schema, destination, and canonical tree digest. - -### OCI source - -- Accept a canonical repository plus exact manifest descriptor. Map authenticated caller identity plus the full repository path to exactly one internal registry/credential/TLS/media-policy route; the request never names that route. -- The matched route pins the one registry API origin and optional OCI bearer-token realm origin/path. Credentials may be sent only to that registry and the pinned realm, with audience/scope restricted to the exact repository. Any unpinned authentication challenge, redirect, foreign layer URL, descriptor URL, alternate blob host, or cross-repository mount is rejected in V1. -- Reauthorize, derive the exact `OciCacheKey`, then hit or singleflight. On a miss, fetch the exact manifest and all blobs through the authorized repository transport, verifying media type, size, digest, config, and each layer while streaming. -- V1 accepts gzip OCI image layers only. Apply whiteouts in order into private cache staging under common path/type limits. -- Never resolve tags/indexes, follow redirects/foreign URLs, or publish the composed result. -- Record requested descriptor, resolved manifest/rootfs digests, access, materializer schema, and destination. - -### Uploaded-bundle source - -- The provider writes deterministic tar+gzip: lexical NFC POSIX paths, normalized uid/gid/mtime, explicit directories, preserved executable bits, no host-specific metadata, and one canonical gzip header. -- The artifact store verifies upload bytes before admission. The cache verifies them again on an exact bundle-key miss before publishing the immutable generation. -- An optional post-run bundle uses the same archive safety rules but is not acquired, extracted, or mounted by the worker until after agent cgroup/network teardown and credential/home removal. It exists only when a bundle executable requires it and is never a workspace source/cache generation. -- Record exact bundle reference, extracted rootfs digest, access, materializer schema, and destination. - -## Promptfoo provider - -The provider is the first client of the general gateway. Its authoring configuration is closed but is not `AgentRunRequest`: - -```text -LocalMode = { - mode: "local", - state_directory: host path, - workspace: { - working_directory: string, - sources: [GitSource | OciSource | LocalBundleSource | UploadedBundleSource] - }, - post_run: null | { - bundle_path?: host path, - network_policy_id: logical ID, - commands: PostRunCommand[], - output_files: OutputFileRequest[] - }, - request_defaults: { agent, runtime_profile_id, agent_network_policy_id, limits } -} - -RemoteMode = { - mode: "remote", - base_url: HTTPS URL, - workspace: same authoring union, - post_run: same authoring union, - request_defaults: { agent, runtime_profile_id, agent_network_policy_id, limits } -} - -LocalBundleSource = { - kind: "local_bundle", - path: host path, - access: "read_only" | "writable", - destination: relative path or "." -} -``` - -The provider enforces the same conditional invariant as the gateway schema: `bundle_path` is required exactly when any command has `executable.kind: "bundle"` and forbidden otherwise. Thus output-only collection and commands such as `{executable: {kind: "runtime", name: "npm"}, args: ["test"]}` require no bundle or upload. - -Local mode starts an ephemeral loopback gateway and worker using the same API handlers, runner, schemas, and states with SQLite/local artifacts. It does not bypass admission or call an adapter directly. Remote mode uses authenticated HTTPS. Promptfoo's secret mechanism supplies gateway authentication outside provider config and request JSON. - -The provider treats `renderedPrompt` as opaque input. Promptfoo may include a complete `_conversation` transcript in that value; every `callApi` still creates a distinct run with a fresh workspace and no retained agent or tool state. The provider does not return a reusable `sessionId`, enable backend thread pooling, or claim support for Promptfoo simulated-user `stateful: true`, which sends only the newest turn and requires the target to retain its own session. Stateful interactive coding evaluation requires a separate session contract. - -For each `callApi(renderedPrompt, context)` the provider: - -1. derives a stable Promptfoo evaluation/case/repetition conversation key and atomically claims the next durable provider-call record containing an ordinal, request digest, UUIDv7 `run_id`, and idempotency key before side effects. Re-entry into an unfinished call with the same digest reuses that record; a completed call advances the ordinal even when the next rendered prompt is byte-identical; -2. deterministically packages each local source/optional post-run bundle, allocates and persists one UUIDv7 `upload_id` per package, then idempotently reserves/uploads and substitutes bundle references without changing order/destinations; -3. builds and locally schema-validates the complete request, including `runtime_profile_id`, all ten limits, and requested-file artifact reservation; -4. submits, polls the tenant-owned run, and propagates Promptfoo abort to cancellation using the same identity; -5. waits for the cleanup-backed terminal result; -6. for every `ArtifactReference`, calls authenticated `GET /v1/artifacts/{artifact_id}` and verifies content type, length, digest, and expiry before use; -7. on `completed`, decodes verified `CapturedText` (or inline text), maps usage, retains truncation metadata, and attaches the complete result as `metadata.allagents_run`; -8. exposes all ordered command/file observations plus verified bytes to JS/LLM graders without interpreting them; -9. on `cancelled` or `infrastructure_error`, raises a typed provider error carrying the result and its durable partial evidence, so infrastructure is excluded from behavioral rates. - -Promptfoo owns matrices, repetitions, pass/fail, scores, rewards, and grader prompts; the provider does not duplicate or infer them. Export small accessors for `metadata.allagents_run`, but no gateway-specific grader or default pass rule. A configuration with `post_run: null` can use Promptfoo-native graders against agent output. - -## Agent adapters - -Both adapters implement: - -```text -AgentAdapter.run({ - instruction, - workspacePath, - modelProxy, - limits, - outputSink, - traceSink, - abortSignal -}) -> { termination, exitCode, Usage } -``` - -Requirements: - -- Pin and verify the CLI version at worker startup. -- Invoke an argument vector directly, never a shell or caller-provided flags. -- Use a fresh adapter home/config directory in run scratch. -- Disable interactive approval, login, self-update, unshipped plugins/extensions, inherited config, and persistent history. -- Accept only the request's working directory, logical model, limits, rendered instruction, and pinned runtime-profile mapping. -- Route model traffic through a worker-owned agent-phase broker. The CLI receives no upstream credential, cookie, client certificate, provider endpoint, or policy body. -- Stream normalized final text through the runner-owned `outputSink` enforcing `max_agent_output_bytes`; never accumulate or return an unbounded string. Normalize native events to ATIF incrementally. Report usage conservatively; unknown fields remain null. -- A clean agent completion is completed even if no files changed. Startup failure, crash, protocol loss, invalid native event stream, or timeout is infrastructure failure. -- On abort, stop descendants and wait for sandbox confirmation. - -Conformance fixtures prove each adapter receives the instruction once, sees exact source destinations, mutates only its private workspace, streams bounded final output/usage/valid ATIF, honors logical model/runtime routing, leaks no canary secret, and is never invoked twice during recovery. - -## Post-run isolation, services, and raw evidence - -Post-run commands may build and run integration tests, so they operate in the same retained sandbox/runtime image and final workspace that the agent used. The workspace is not copied or overlaid by default. The agent result is sealed as output/usage/trajectory plus a process-ownership boundary; post-run filesystem changes remain ephemeral and are destroyed at cleanup. - -1. The retained runtime uses the pinned immutable profile revision's unprivileged identity, read-only root, explicit seccomp allowlist, private mount/process/IPC/UTS boundaries, and mandatory cgroup-v2 pids/memory/CPU/IO limits. Operator services, the agent, and each post-run command have distinct worker-owned cgroups; sandbox processes cannot change membership or controllers. -2. The pinned profile revision—not caller JSON—declares credential-free service launch manifests/images, implementation/image digests, readiness, restart policy, broker identities/ports, phase visibility, and resource limits. Services run in worker-owned namespaces/cgroups and are never directly addressable from agent/check loopback. -3. Run the agent in its own default-drop network namespace. nftables or an equivalent kernel layer filters loopback as well as external traffic; the namespace exposes only agent-policy broker endpoints. A post-run-only service has no route or broker endpoint in this namespace. -4. After adapter completion, close model/source brokers, remove adapter secrets/home, stop/reap the agent cgroup, destroy its network namespace plus conntrack/flows/broker handles, and confirm no agent process or socket remains. Only then durably seal bounded output, usage, and ATIF while retaining the mount/runtime/workspace and service cgroup. -5. If `bundle` is present, only after step 4 acquire its bytes from the input artifact store, reauthorize and verify digest/size, safely extract them, and mount the result read-only at the empty worker-reserved path that was absent from the agent mount namespace. Resolve every bundle executable without symlink escape. No bundle executable means no bundle bytes are acquired or mounted. -6. Create a fresh check network namespace with default-drop filtering over loopback/external traffic. Resolve `post_run.network_policy_id` to explicit operator-brokered external/service endpoints; no agent-phase flow or direct sidecar socket is reused. -7. Health-check services declared by the pinned profile revision through the same phase broker and restart them under its immutable policy if allowed. Failure is infrastructure error. Agent-owned daemons are never retained/restarted, and undeclared/post-run-disallowed services are unreachable. -8. Resolve runtime names only through the pinned revision's fixed approved read-only `PATH`, never the workspace or inherited `PATH`. Execute selected executable/literal args in a fresh check cgroup/process group and the check network namespace, with bounded scratch/fixed nonsecret environment. Source access modes remain unchanged. -9. Capture bounded stdout/stderr, reap the command cgroup, then durably seal its `completed` or `timed_out` observation before starting the next command. Launch/sandbox/evidence failure seals `unavailable`; cancellation/deadline/prior failure fills untouched commands `not_run`. -10. After commands settle, collect requested files in order without following final symlinks and durably seal every observation before advancing. Unsafe/read/artifact failure seals `unavailable`; remaining files become `not_run`. Admission-reserved maxima guarantee collected-file budget. -11. Deterministically promote optional result artifacts, stop services/check namespace, and destroy the run workspace. Post-run build output persists only when explicitly requested within bounds. - -The boundary does not calculate a generic diff, publish a snapshot, preserve a workspace, or create a user-visible change artifact. Post-run observations are evidence, not a judgment. - -## Secrets and operator-policy boundary - -Credentials and operator policy exist only in gateway/worker control processes and dedicated brokers. - -- Git/OCI credentials are resolved after authorization and passed through broker sockets or inherited descriptors not mounted into agent/post-run namespaces. -- Model credentials terminate in the model proxy. Agents see only a non-secret per-run local socket and logical model mapping. -- Gateway API credentials authenticate provider transport and never enter run storage. -- After agent exit, the worker closes model/source brokers, destroys the agent cgroup/network namespace/flows, and removes adapter credential/config mounts before acquiring optional bundle bytes or resolving any post-run executable. -- Agent and post-run commands receive separate fixed environment allowlists containing locale, deterministic home/temp paths, and required non-secret switches. They do not inherit the worker environment. -- Post-run commands receive no model/source credential socket or agent home. Their fresh network namespace has default-drop loopback/external rules and only independently authorized operator-brokered endpoints; no direct sidecar or host socket is exposed. -- Operator configuration owns caller/source-route mappings (including pinned OCI token realms), credentials, model routes, runtime profiles (image/sandbox/cgroups/tools/services), phase network brokers, sidecar definitions, resource ceilings, artifact retention, and secrets. Caller JSON contains only canonical source addresses, uploaded artifact IDs, approved runtime names/relative bundle paths, literal arguments, and authorized logical model/runtime/network IDs—never a credential identity, raw path, image, service command, socket, or policy body. -- Structured logs are redacted at ingestion. Seeded-canary tests fail on appearance in request/result JSON, cache generations, ATIF, command output, requested files, subprocess environment, errors, or artifacts. - -`agent_network_policy_id` and `post_run.network_policy_id` are authorized separately per tenant and realized in separate default-drop namespaces. Allowed external or service access exists only through phase-specific operator brokers. Agent namespace/flows are destroyed before check namespace creation; post-run-only and undeclared service endpoints are never reachable by the agent, including through loopback. - -## Cancellation, deadlines, recovery, and cleanup - -- One root abort controller fans into authorization/materialization, adapter, trace/artifact writers, service manager, post-run collector, and sandbox operations. -- Cancellation is idempotent. Repeated cancellation returns the same state; a terminal run is unchanged. -- Total and phase deadlines are persisted as absolute times so worker restart cannot reset them. -- Agent timeout revokes model/network access and kills only the agent cgroup before run cleanup; operator service/check cgroups have separate ownership. -- Post-run never starts after cancellation/agent infrastructure failure/result-seal failure. Cancellation or deadline during post-run stops the current check cgroup, seals available bounded streams, fills the full evidence vector with `not_run`/`unavailable`, then cleans up. -- A command timeout is a `timed_out` observation and later commands may continue. Exhausting the phase/total deadline produces infrastructure error with the sealed partial vector. -- Neither agent nor post-run command is retried. Exact-key materialization is singleflight, not agent retry. Safe network reads may retry only before side effects and under the original deadline. -- Cleanup runs after success, failure, cancellation, process crash recovery, and worker shutdown. It is idempotent and keyed by run ID. It stops check/service cgroups, removes any injected bundle and private clones/mounts/root, then releases cache leases. -- A cleanup-step failure atomically replaces the pending terminal outcome with `cleanup_failed`/`infrastructure_error`, keeps already durable evidence, and leaves the status envelope nonterminal with `result: null`. Cleanup retry/reconciliation continues; only verified resource absence and lease release allow publication of that infrastructure-error result. -- Startup reconciliation finds nonterminal run records, private roots, cache staging, and leases. It completes safe staging/cleanup; if agent invocation may have begun, it records infrastructure error instead of invoking again. It never publishes a result while cleanup residue or a run lease remains. -- Run/result records and immutable cache generations follow operator retention. Input bundles have separate non-downloadable retention. Every result artifact stores owner tenant/run/purpose plus `expires_at`, remains immutable until expiry, then returns 410 to its authorized owner and is deleted asynchronously. Per-run workspaces, clones, optional bundles, and scratch are destroyed before result visibility. - - -## Delivery phases and observable exit proofs - -### Phase 0 — Bootstrap the one repository - -Create the Bun workspace, package boundaries, pinned toolchain, CI, license, schema-generation command, gateway/worker process entrypoints, PostgreSQL/SQLite migrations, artifact/cache interfaces, and health/readiness endpoints. Add dependency-boundary checks that keep Promptfoo, Fastify, and adapters out of `packages/runner`. - -**Exit proof:** a clean checkout installs from the lockfile, generates schemas/types, applies and rolls back migrations on disposable state, starts gateway and worker, reports healthy/readiness with artifact/database/cache dependencies, and shuts down cleanly. No run submission, agent, cache materialization, or end-to-end claim belongs to Phase 0. - -### Phase 1 — Freeze contracts, API, persistence, and state - -Implement exact schemas/generated types, canonical hashing, tenant-owned upload/run/result-artifact persistence, idempotent bundle reserve/upload, artifact download, status/cancel, internal submission claims, duplicate-before-admission ordering, mutable admission, state CAS, errors, deadlines, cancellation, immutable runtime-profile revision pinning, ten limits/artifact reservation, direct provenance, bounded `CapturedText`, and full partial `PostRunEvidence` unions. Add golden documents for every union/status/nullability/limit/provenance/error. - -**Exit proofs:** - -- unknown fields, omitted runtime profile/source access, host paths, policy bodies, shell strings, overlap, noncanonical sources, caller credential/image/service controls, malformed executable unions, incorrect bundle conditional, and excessive/reservation limits fail before any public run; -- exact duplicate lookup returns an existing run after source/profile policy changes without mutable re-admission; conflict rejects; a new identity that fails admission leaves no public run/job, while 50 concurrent valid identical submits create one; -- every run/upload/result artifact stores owner tenant; same-tenant GET/cancel/download succeeds, while another tenant using the same run/artifact/upload UUID receives the same non-enumerating 404 as an unknown object and cannot cancel/read/write; -- bundle reservation retry with one `upload_id` returns the same reference, changed metadata conflicts, exact PUT replay succeeds, and cross-tenant PUT fails; -- artifact GET streams immutable bytes with matching media type/length/digest/expiry, returns 410 after authorized expiry, rejects input bundles/detached artifacts, and refuses corrupt stored bytes; -- golden results prove ten effective limits, `DirectWorkspaceProvenance`, revision-pinned `RuntimeProvenance`, bounded `CapturedText`, and completed/cancelled/infrastructure variants with null or full ordered partial evidence vectors; -- cancellation races terminalization deterministically; terminal results are immutable; public errors/logs omit seeded secrets and nonsecret resolved paths. - -### Phase 2 — Cache, package, and compose bounded sources - -Implement deterministic packaging, streamed uploads, exact Git/OCI/bundle cache keys, cache-owned staging, exact-key singleflight, atomic complete generations, authorization on hits, durable leases, reflink/full-copy attachment, read-only mounts, watermarks, eviction, reconciliation, private root allocation, aggregate bounds, ordered provenance, working-directory validation, and cleanup. - -**Exit proofs:** - -- two packagings of one tree are byte-identical; submitted JSON contains a bundle reference/access mode and no host path; -- three sources land at declared non-overlapping destinations and provenance preserves request order, canonical Git URL/OCI repository, access, exact resolved identity, and materializer schema; -- overlap, omitted access, destination aliasing, or `.` with multiple sources fails before credential/network/cache attachment; -- Git keys use canonical URL plus exact commit/history; OCI keys use canonical repository plus exact descriptor; bundle keys use exact digest/size; schema/key changes never fall back to another generation; -- Git/OCI reject every redirect; path-scoped credentials never reach another path/origin; OCI accepts only its pinned token realm/exact repository and rejects foreign blob/descriptor URLs, cross-repository mounts, and unpinned auth challenges; -- 50 concurrent cold runs for one exact large Git or OCI source perform one fetch/materialization, publish one complete generation, and acquire 50 leases; -- a cache hit after caller authorization is revoked or remapped is denied before mount even though identical bytes exist locally, and a different tenant cannot obtain access merely by guessing the canonical source address; -- `read_only` mounts share one immutable generation, accept read-only Git commands under `GIT_OPTIONAL_LOCKS=0`, and reject filesystem/Git writes; -- two `writable` runs receive distinct reflink/CoW clones (or bounded full copies), can mutate independently, and cannot change each other or the cache; inode/link checks prove no hardlinked writable alias; -- malicious traversal/links/devices, digest mismatch, gzip bomb, aggregate limits, and corrupt/incomplete generations fail before agent start; -- eviction skips a generation while any run lease exists, removes it only after final lease release, and converges from high to low byte/inode watermarks; -- restart reconciliation removes stale staging, preserves live leases, quarantines corrupt generations, and never exposes a partial generation; -- run cleanup removes mounts/clones/root and releases leases while leaving authorized immutable generations reusable. - -### Phase 3 — Run Codex and OMP exactly once - -Implement admission-time immutable runtime-profile revision/digest pinning, dispatch-time re-verification, exact `RuntimeProvenance`, the production `RunSandbox` invariants, cgroup-v2 controllers, default-drop agent networking/brokers, model proxy, retained mount/runtime, Codex/OMP adapters, bounded `CapturedText`/ATIF/usage, stop/reap/network-destruction boundary, and durable invocation marker. - -**Exit proofs:** - -- the full proof moved from Phase 0 now runs: a clean checkout starts disposable services, admits one no-post-run Git fixture, materializes/cache-leases it, invokes each adapter once, returns bounded output/usage/ATIF/source+runtime provenance, removes every run resource/lease, and retains only the authorized immutable source generation; -- a profile ID resolves once at admission to a persisted immutable revision. Dispatch uses only that revision and verifies the expected `profile_digest`, runtime `image_digest`, sandbox-policy version, every tool/service `implementation_digest`, containerized service `image_digest`, cgroup ceiling, and broker identity; changed/missing/mismatched revisions fail before agent invocation, and returned provenance proves exact implementations without paths, commands, policy bodies, or secrets; -- output below/at/above `max_agent_output_bytes` yields verified inline/artifact/truncated `CapturedText` without unbounded memory or persistence; -- non-root UID/GID mapping, empty capabilities, `no_new_privs`, private namespaces, read-only root, bounded tmpfs, masked proc/sys/cgroup/devices/host sockets, explicit seccomp, and no writable control-plane mount pass escape/mount/device/socket/syscall probes; -- fork-bomb, memory/OOM, CPU, and IO fixtures remain within cgroup-v2 limits and cannot affect a sibling run or worker; -- agent namespace default-drop covers loopback/external traffic: allowed broker/model fixtures work, undeclared and post-run-only services fail, and teardown removes all agent processes, sockets, flows, and namespace handles; -- crash/timeout/cancel produce infrastructure/cancelled results only after cleanup; crash around the invocation marker never starts a second agent; canary credentials/policy stay absent. - -### Phase 4 — Seal agent output and collect evidence in the retained sandbox - -Implement durable evidence skeleton/observation sealing, an empty reserved late-mount point plus post-agent bundle acquisition/mount, pinned-profile tool/service resolution, distinct default-drop check network namespace/brokers, ordered commands/files, partial cancellation/error vectors, aggregate artifact allocation/promotion, artifact metadata/expiry, cleanup downgrade, and publication gate. - -**Exit proofs:** - -- output-only and runtime-only requests omit a bundle. Before agent teardown even a requested bundle's bytes are neither acquired nor mounted and its reserved point is empty; afterward profile `npm` with literal `["test"]` runs in the exact final workspace, while a bundle executable triggers the late verified read-only confined mount; -- each command/file seals before the next. Cancellation between commands returns prior evidence plus ordered `not_run`; cancellation during command 2 seals it `unavailable` with safe bounded partial streams and later items `not_run`; command-2 launch failure returns command 1, command 2 `unavailable`, command 3/files `not_run`; file read/artifact failure similarly preserves earlier observations; -- exit zero/nonzero/timed-out commands are lifecycle completed with `completed|timed_out` observations; a timed-out cgroup is gone before the next command; -- read-only sources stay read-only; writable sources hold build output across commands; requested files produce collected/missing/limit-exceeded/not-regular states; -- the check namespace exists only after agent namespace/flows are destroyed. Default-drop includes loopback; post-run-only brokered sidecar works for checks but was unreachable to the agent; undeclared/direct sidecar and external endpoints fail; -- model/source credentials and agent home are absent and the agent cgroup/network/flows are destroyed before any optional bundle-byte acquisition, extraction, mount, or executable resolution; -- admission rejects `sum(output_files.max_bytes)` over budget. Actual files charge first, then agent output, command stdout/stderr order, then trajectory; exact-boundary fixtures prove accounting/no dedup discount and deterministic inline-truncated/omitted fallback without infrastructure error; -- downloaded artifact bytes match owner/run/purpose/media/digest/size/expiry; hidden input bundles never appear as downloadable result artifacts; -- runtime/bundle/service/sandbox/launch/unsafe collection failures return infrastructure error with the exact partial vector; cleanup failure preserves it, withholds all result visibility through reconciliation, and never publishes completed; -- large output stays bounded and ATIF fallback remains valid with one truncation marker. - -### Phase 5 — Complete Promptfoo local and remote flows - -Implement provider modes, crash-safe run/upload identities, multi-source/runtime-profile mapping, idempotent packaging/upload, polling/cancel, artifact download/verification, bounded output/usage/metadata mapping, partial-evidence accessors, and native matrix/repetition/JS/LLM examples. - -**Exit proofs:** - -- one case runs local/remote with schema-equivalent results; Git/OCI/local bundles preserve order/access without host paths; -- provider maps runtime plus phase-network logical IDs and never embeds an image, PATH, service, policy, or credential; -- lost reservation/submit/download responses replay `upload_id`/run/artifact identity without duplicate upload/run/agent; -- inline and artifact-backed agent output, command streams, files, and trajectories are byte/digest/media verified; expiry/corruption becomes typed infrastructure, never a grader input; -- cancelled/infrastructure errors carry their result/partial evidence to diagnostics but are excluded from behavioral rates; -- a Codex/OMP matrix with two repetitions creates four private runs sharing authorized immutable generations; JS/LLM graders alone decide outcomes from verified raw evidence; -- a two-turn full-history Promptfoo conversation receives distinct durable provider-call ordinals, run IDs, and fresh workspaces; losing and retrying turn 2's response reuses only turn 2's unfinished identity, while the provider returns no reusable session identity and never pools Codex/OMP state across calls; -- cross-tenant run/cancel/artifact attempts fail identically in local and remote modes; no host path, credential, policy body, process, mount, clone, or run directory survives. - -## Deferred integrations - -Harbor, Terminal-Bench, and SWE-bench integration are outside this V1 implementation plan. V1 has no Harbor source variant, request, backend, package, result variant, provenance variant, sandbox delegation, or patch exporter. Future work requires a separate ADR and closed schema extension defining its own request plus result/provenance mapping into raw nongrading evidence without changing direct-mode semantics. - -## Focused release E2E - -The release gate runs the built gateway, worker, packaged Promptfoo provider, real local Git/OCI/object-store fixtures, source cache, and pinned agent fixtures/CLIs. For each scenario record request/result JSON, state sequence, invocation count, source fetch/materialization count, generation/lease state, resolved provenance, per-run cleanup, and artifact digests. - -| Scenario | Expected proof | -|---|---| -| Multi-source Codex/OMP direct runs | exact access/destinations; one invocation; source+runtime provenance; cleanup | -| Runtime-profile revision matrix | admission pins `profile_digest`; dispatch re-verifies exact runtime/tool/service implementation and image digests; no caller paths/commands | -| Runtime `npm test`, no bundle | approved profile tool, literal args, same final workspace, no bundle upload/mount | -| Bundle executable | bundle required iff used; worker does not acquire/mount bytes before agent teardown; later verified confined immutable mount | -| 50 cold exact-source runs | one materialization, 50 leases, isolated writable clones/shared read-only generation | -| Redirect/OCI credential attacks | all redirects/foreign URLs/unpinned realms rejected; credentials stay exact path/repository scoped | -| Read-only/writable/eviction | writes fail on shared RO; writable clones isolate; live lease blocks eviction | -| Sandbox breakout suite | non-root/no capabilities/no_new_privs/private namespaces/RO root/seccomp/hidden host resources | -| Fork/OOM/CPU/IO attacks | cgroup-v2 ceilings contain run without worker/sibling impact | -| Phase network isolation | default-drop loopback/external; agent cannot reach post-run service; check uses broker only | -| Nonzero/timed-out commands | lifecycle completed; exact completed/timed_out evidence; timed-out cgroup reaped | -| Cancel during second command | first observation retained; in-flight command unavailable with safe partial streams; untouched commands/files ordered not_run | -| Launch/file collection failure | prior observations retained; failed item unavailable; later items not_run; infrastructure_error | -| Artifact-budget boundary | files charge first, then agent/streams/trajectory; deterministic truncated/omitted fallback | -| Bounded agent output | inline/artifact/truncated `CapturedText` at below/exact/above cap; no unbounded state | -| Result artifact download | tenant/run-authorized immutable bytes verify media/size/digest; expiry 410; corrupt bytes refused | -| Cross-tenant UUID replay | run GET/cancel, upload PUT, artifact GET all match unknown-object 404 with no effect | -| Upload response loss | same `upload_id` returns one reservation/reference and exact PUT replay | -| Duplicate run after policy change | same result without re-admission; conflicting body rejects; new admission failure leaves no run | -| Promptfoo local/remote matrix | persistent identities, verified artifact dereference, four private runs, grader-only outcomes | -| Crash/cleanup fault | no second agent; result hidden through cleanup; final infra result retains durable evidence | -| Seeded canaries | absent from environments/cache/public artifacts/errors/logs | - -CI may retain sanitized JSON, immutable cache fixtures, and referenced evidence, but never a live run workspace, writable clone, generic diff, grading decision, reward, or modified-workspace artifact. - -## Completion checklist - -- [ ] `allagentsdev/allagents-gateway` is the only new repository and contains provider, gateway API, worker/runner, contracts, source cache/composer, service manager, post-run collector, and both agent adapters. -- [ ] Runtime, HTTP, persistence, queue, cache, isolation, schema, identity, and ATIF choices match this plan; no implementation framework remains undecided. -- [ ] Closed schemas cover `runtime_profile_id`, ten limits, bounded `CapturedText`, aggregate artifact budget, `PostRunEvidence`, direct workspace and revision-pinned runtime provenance, upload idempotency, artifact references, and every result nullability/invariant. -- [ ] Promptfoo is the first caller and sole evaluation/grading layer; gateway JSON has only completed/cancelled/infrastructure_error and no pass/fail/reward. -- [ ] Authentication/schema/hash and tenant-scoped duplicate/conflict lookup precede mutable admission; exact duplicates bypass re-admission; new admission failures create no public run/job. -- [ ] Every upload/run/result artifact persists owner tenant; duplicate submit, GET, cancel, upload PUT, and artifact GET resolve tenant+object with indistinguishable unknown/cross-tenant 404 behavior. -- [ ] Bundle reserve uses persisted client `upload_id`; exact reserve/upload replay is idempotent and changed metadata conflicts. -- [ ] Authenticated artifact GET serves only tenant/run-bound unexpired result bytes with immutable media/size/digest headers; input bundles are not exposed, owner expiry is 410, and corruption is refused. -- [ ] `max_artifact_bytes` reserves requested-file maxima and charges actual files, agent output, ordered command streams, then trajectory; no dedup discount; budget exhaustion uses bounded truncated/omitted evidence rather than infrastructure failure. -- [ ] Local paths are packaged before submission; gateway JSON has no host path. -- [ ] Git canonical URL/ref/history, OCI repository/exact descriptor, bundles, access, schema, destination, and resolved identities return ordered provenance. -- [ ] Git/OCI use exact path-specific caller routes, reject all redirects, pin OCI token realm/repository scope, and reject foreign descriptor/blob URLs; cache hits reauthorize. -- [ ] Exact cache keys singleflight into complete immutable generations with durable leases, byte/inode watermarks, safe eviction/reconciliation, direct RO mounts, and private CoW/full-copy writable clones without hardlinks. -- [ ] Admission persists one immutable runtime-profile revision and `profile_digest`; dispatch uses only that revision and re-verifies runtime image, sandbox, broker, tool/service implementation, and containerized-service image identities before agent invocation; `RuntimeProvenance` proves them without secret args/paths. -- [ ] `RunSandbox` enforces unprivileged non-root UID/GID, empty capabilities, no_new_privs, private namespaces, RO root, bounded tmpfs, masked host resources/sockets, explicit seccomp allowlist, mandatory cgroup-v2 pids/memory/CPU/IO, and no writable control-plane mounts. -- [ ] Agent/check network namespaces are distinct and default-drop including loopback; all external/service access is phase-brokered; agent flows/namespaces die before post-run, so post-run-only services are never agent-reachable. -- [ ] Codex/OMP invoke once and stream at most `max_agent_output_bytes`; completed results expose inline/artifact/truncated `CapturedText`, usage, and valid bounded ATIF. -- [ ] Optional bundle is present iff a bundle executable is requested; dispatch creates only an empty reserved late-mount point and does not acquire or mount bundle bytes until agent cgroup/network/flows and credentials/home are gone. Runtime tools come only from the pinned profile revision's PATH; args are literal; commands use the exact final workspace. -- [ ] Post-run initializes one full ordered evidence skeleton, seals each command/file before advancing, and represents completed/timed_out/not_run/unavailable plus all file states. -- [ ] Cancelled/infrastructure results retain durable partial evidence; behavioral nonzero/timeout/missing/limit/non-regular observations do not become infrastructure. -- [ ] Provider persists per-conversation call ordinals plus run/upload identities, reuses only an unfinished matching call, downloads/verifies all artifact-backed output/evidence/trajectory, exposes partial evidence diagnostically, and leaves grading to Promptfoo. -- [ ] Cleanup removes every process/namespace/flow/service/mount/clone/root and releases leases before result visibility; cleanup failure forces infrastructure_error while retaining evidence through reconciliation. -- [ ] Credentials/policy remain outside caller JSON, run environments, cache, evidence, errors, artifacts, and logs. -- [ ] No reusable session, provider session identity, backend thread pooling, continuation, checkpoint, composed-workspace or snapshot publication cache, generic core change artifact, or second repository remains; full-history Promptfoo conversation rendering still creates a fresh run per turn. diff --git a/docs/plans/2026-09-18-0837-feat-promptfoo-coding-agent-evals-plan.md b/docs/plans/2026-09-18-0837-feat-promptfoo-coding-agent-evals-plan.md new file mode 100644 index 00000000..c87e6ff9 --- /dev/null +++ b/docs/plans/2026-09-18-0837-feat-promptfoo-coding-agent-evals-plan.md @@ -0,0 +1,549 @@ +--- +title: "Promptfoo Coding-Agent Evaluations - Implementation Plan" +date: 2026-09-18 +updated: 2026-09-28 +type: feat +artifact_contract: ce-unified-plan/v1 +artifact_readiness: implementation-ready +execution: code +--- + +# Promptfoo Coding-Agent Evaluations - Implementation Plan + +## Goal capsule + +- **Objective:** Evaluate write-capable Claude and Codex agents against fresh, + reproducible workspaces and grade their final filesystem state. +- **Means:** Run Promptfoo's built-in agent providers in one disposable job, + using a lifecycle extension to copy an immutable seed into + `.eval/workspace` before every row and remove it afterward. +- **Evaluation owner:** Promptfoo owns prompts, provider matrices, repetitions, + assertions, scores, pass/fail decisions, OpenTelemetry traces, and reports. +- **Stop conditions:** Do not build an execution gateway, custom Promptfoo + provider, agent adapter, ATIF converter, durable run service, or reusable + session layer. + +The authoritative decision is +[ADR 0002](../decisions/0002-use-promptfoo-native-agent-execution.md). +Primary-source findings are in +[Promptfoo native agent workspaces](../research/promptfoo-native-agent-workspaces.md). +The earlier +[one-shot gateway research](../research/one-shot-coding-agent-gateway-boundary.md) +is retained as analysis of the rejected hosted-service alternative. + +## Product contract + +One evaluation job owns one private `.eval` root. Job bootstrap materializes +exact source inputs once as read-only seeds. Promptfoo then runs every expanded +test/provider/repetition row serially: + +```mermaid +flowchart LR + B[Job bootstrap] --> S[Resolve exact source seeds] + S --> H[Promptfoo beforeEach] + H --> C[Private copy to .eval/workspace] + C --> P[Built-in Claude or Codex provider] + P --> A[Promptfoo assertions inspect final workspace and trace] + A --> T[Promptfoo afterEach captures bounded diagnostics] + T --> D[Delete workspace] + D --> N{More rows?} + N -->|yes| H + N -->|no| J[Export results and destroy job] +``` + +The fixed provider working directory is: + +```text +/.eval/workspace +``` + +The extension is lifecycle glue, not a provider. It never calls a model, +interprets agent output, assigns a reward, or replaces Promptfoo's provider +metadata and tracing. + +## Scope + +### Included + +- Promptfoo configuration for `anthropic:claude-agent-sdk` and + `openai:codex-sdk`. +- One source-staging command that creates immutable job-private seeds from + declared local, Git, or OCI inputs. +- One JavaScript lifecycle extension implementing `beforeAll`, `beforeEach`, + `afterEach`, and `afterAll`. +- One fixed `.eval/workspace` configured on every write-capable provider. +- Serial execution with Promptfoo response caching disabled. +- Deterministic JavaScript assertions over files, commands, provider metadata, + and trace data. +- Promptfoo's built-in OpenTelemetry receiver and trajectory assertions. +- Result, trace, source-provenance, and bounded diagnostic artifacts retained by + the surrounding job. +- Local and CI/VPS execution inside the same disposable container or VM shape. + +### Excluded + +- `allagentsdev/allagents-gateway` or another network execution service. +- A custom Promptfoo provider or local/remote provider mode. +- Queues, databases, idempotency APIs, artifact download APIs, leases, or + cancellation protocols. +- Custom Claude, Codex, or OMP adapters. +- ATIF normalization or a second public trajectory format. +- Cross-row session or thread persistence. +- Concurrent rows sharing one workspace. +- Hidden verifier bytes that must be inaccessible to a shell-capable agent. +- Hostile multi-tenant isolation or caller-specific authorization policy. +- Checkpoints, resumable runs, workspace recovery, or generic patch export. + +## Repository layout + +Add the evaluation implementation to this repository: + +```text +evals/coding-agent/ + promptfooconfig.yaml + sources.yaml + extensions/ + workspace.cjs + assertions/ + workspace.cjs + scripts/ + stage-sources.ts + fixtures/ + ... + .eval/ # ignored, job-private runtime state + seeds/ + workspace/ + artifacts/ +``` + +The configuration, extension, assertions, and source catalog are checked in. +`.eval` is always generated and ignored. + +## Fixed Promptfoo configuration + +Start from this shape: + +```yaml +description: AllAgents coding-agent evaluations + +providers: + - id: anthropic:claude-agent-sdk + config: + working_dir: ./.eval/workspace + append_allowed_tools: [Write, Edit, MultiEdit, Bash] + permission_mode: acceptEdits + persist_session: false + sandbox: + enabled: true + failIfUnavailable: true + + - id: openai:codex-sdk + config: + working_dir: ./.eval/workspace + sandbox_mode: workspace-write + approval_policy: never + enable_streaming: true + persist_threads: false + +extensions: + - file://extensions/workspace.cjs:workspaceLifecycle + +evaluateOptions: + maxConcurrency: 1 + cache: false + +tracing: + enabled: true + otlp: + http: {} + +defaultTest: + assert: + - type: javascript + value: file://assertions/workspace.cjs +``` + +Provider settings remain provider-specific: + +- Claude receives only the explicit tools required by a case. Use + `acceptEdits` for unattended edits, set `persist_session: false`, and require + its sandbox with `failIfUnavailable: true`; do not use permission bypass by + default. +- Codex uses `workspace-write`, `approval_policy: never`, explicit + `persist_threads: false`, and its minimal process environment. + `enable_streaming` supplies provider-level operation and turn spans. +- Do not enable provider thread or session persistence. Every row is a new + attempt. +- Do not enable deep tracing globally. Add it to a focused configuration only + when native SDK spans answer a specific question and their additional data + exposure is acceptable. + +The normal evaluation command uses `--no-cache` as defense in depth even though +the checked-in configuration sets `evaluateOptions.cache: false`. + +## Case contract + +Every test row supplies: + +```yaml +vars: + case_id: safe-stable-id + source_id: staged-source-id + task: rendered agent instruction + check: + command: [bun, test] + timeout_ms: 120000 + expected_exit: 0 +``` + +`case_id` and `source_id` are identifiers, not paths. Restrict them to a short +ASCII identifier grammar such as `^[a-z0-9][a-z0-9._-]*$`. The extension maps +`source_id` beneath its own resolved `.eval/seeds` directory and rejects +unknown identifiers, symlinks escaping the seed, and any resolved path outside +the evaluation root. + +The extension assigns each expanded provider/test/repetition row a monotonic +`workspace_row_id` such as `000001-safe-stable-id` during `beforeEach` and +returns it in the test variables. Authors do not supply this value. It is the +artifact and receipt key, so repeated cases and provider matrices cannot +overwrite one another. + +Checks are closed data consumed by the checked-in assertion module. A command +is an executable plus literal argument vector; it is never a shell string. +Cases cannot supply an environment map, arbitrary assertion module, working +directory, or cleanup command. + +Promptfoo `metadata` is descriptive report data, not the execution control +plane. Repository URLs, refs, workdir paths, Git-cache settings, skill-copy +commands, and verifier commands belong in the checked-in source/workspace +catalog. A suite may select a catalog entry with `defaultTest.vars.source_id`; +the extension consumes that validated identifier. Keep suite metadata for +source links, experiment tags, and other annotations. + +## Source staging + +`scripts/stage-sources.ts` runs before Promptfoo. It reads `sources.yaml` and +materializes only source IDs used by the selected evaluation: + +- local sources are copied from an explicitly allowed repository-relative + path; +- Git sources resolve a requested ref to a full commit and check out that exact + commit; +- OCI sources, when present, resolve and verify an exact manifest digest before + extracting the declared filesystem content. + +Each staged seed contains a provenance record with: + +- source ID and kind; +- requested source and ref, when applicable; +- resolved Git commit or OCI manifest digest; +- materializer version; and +- a deterministic digest of the staged tree or source descriptor. + +Staging uses private temporary directories and publishes a seed only after +materialization and verification succeed. Before Promptfoo starts, bootstrap +exposes published seeds through a read-only mount or transfers them to an +identity the unprivileged Promptfoo/agent user cannot modify. Mode bits owned by +that same user are not an immutability boundary. No credentials, Git +credential-helper responses, registry tokens, or temporary acquisition files +enter the seed. + +Seeds are reusable only inside the current disposable job. V1 does not define a +cross-job cache, eviction protocol, or shared authorization boundary. + +For a composed workspace, the source catalog defines non-overlapping +destinations and staging produces one complete seed tree. Composition happens +once during bootstrap rather than during every Promptfoo row. + +## Workspace lifecycle extension + +Export one extension function: + +```javascript +module.exports.workspaceLifecycle = async function workspaceLifecycle(hookName, context) { + // beforeAll | beforeEach | afterEach | afterAll +}; +``` + +Resolve all paths from CommonJS `__dirname`, not `process.cwd()`. + +### `beforeAll` + +- Refuse to run when `.eval` or the selected seeds resolve outside the + evaluation directory. +- Remove a stale `.eval/workspace` from an interrupted local run. +- Verify that every referenced source ID has a complete staged seed and + provenance record. +- Create a fresh bounded artifacts directory for this evaluation. + +### `beforeEach` + +- Validate `context.test.vars.case_id` and `source_id`. +- Allocate the next unique `workspace_row_id` and add it to the returned test + variables. +- Remove `.eval/workspace` unconditionally. +- Copy the selected read-only seed to a private temporary sibling. +- Never hardlink files or create another writable alias to seed content. +- Use a reflink/copy-on-write copy only when writes cannot reach the seed; use a + full recursive copy otherwise. +- Make the private copy writable, then atomically rename it to + `.eval/workspace`. +- Return the modified context so `workspace_row_id` reaches assertions and + `afterEach`. + +Any setup failure throws and prevents the provider call. + +### `afterEach` + +For deterministic-only rows, Promptfoo's normal path invokes `afterEach` after +provider execution and assertions. The hook: + +- records the row ID, case ID, source provenance digest, and bounded workspace + status in `context.result.metadata`; +- optionally copies explicitly allowlisted diagnostic files to + `.eval/artifacts//`; +- removes `.eval/workspace` in `finally`; +- writes the row's success receipt only after diagnostics and cleanup finish; + and +- writes `.eval/hook-failure.json` before throwing if diagnostic collection or + cleanup fails. + +Promptfoo currently catches and logs `afterEach` exceptions. Throwing is still +useful for logs, but it does not make the CLI exit nonzero. The job wrapper must +reject any run with a hook-failure sentinel, a surviving workspace, duplicate +row IDs, or a missing receipt for any Promptfoo result row. + +The hook must not attempt to rewrite `success`, `score`, or `response.output`; +Promptfoo does not persist such overrides from `afterEach`. + +### `afterAll` + +- Verify that `.eval/workspace` is absent. +- Write a compact source/artifact manifest for the surrounding job. +- Remove remaining temporary directories. +- Leave only explicitly retained seeds or artifacts required by job export. + +The job wrapper performs the authoritative post-Promptfoo sentinel, workspace, +receipt, and manifest checks. Disposable job teardown remains the final cleanup +boundary if Promptfoo or the extension process crashes before hooks complete. + +## Deterministic workspace assertion + +`assertions/workspace.cjs` receives `output` and Promptfoo's assertion context. +It resolves the same fixed workspace path independently of test-controlled +strings. + +For each declared check it: + +1. validates the closed check object; +2. resolves the executable from the disposable job's fixed `PATH`; +3. starts the executable directly with a literal argument vector and + `cwd = .eval/workspace`; +4. enforces the per-check timeout and terminates the spawned process tree; +5. captures bounded stdout and stderr; +6. compares the actual exit status with `expected_exit`; and +7. returns a `GradingResult` with a factual reason and named scores. + +Additional file assertions read only declared workspace-relative paths, reject +absolute paths and `..`, reject symlink escape, and bound bytes read. + +The assertion is authoritative for filesystem behavior because a +deterministic-only row executes it before `afterEach` removes the workspace. The +hook may retain its summary for diagnostics but does not re-grade it. + +Do not combine a live-workspace assertion with a model-graded assertion in the +same evaluation row. Promptfoo can defer the complete assertion set for grouped +model grading, allowing a later `beforeEach` to replace the shared workspace +first. When semantic grading is required, the deterministic evaluation must +serialize all required facts to a unique row artifact; a separate +evaluation grades that artifact without reading `.eval/workspace`. + +Tests for the assertion cover: + +- passing and failing exit statuses; +- timeout and process termination; +- missing executable and launch failure; +- stdout/stderr truncation; +- missing, non-regular, oversized, and symlink-escaping files; and +- an assertion observing an agent mutation before teardown. + +## Traces and transcripts + +Promptfoo OpenTelemetry is the only V1 trace model. + +- Claude's built-in provider emits an `invoke_agent` span, per-turn markers, + and completed tool spans; detailed tool calls also remain in + `response.metadata.toolCalls`. +- Codex uses `enable_streaming: true` to emit provider-level turn, command, + file, search, MCP, response, and reasoning-item spans where the SDK exposes + them. +- Built-in `trajectory:*` assertions check tool use, arguments, sequence, step + count, and goal success. +- JavaScript assertions may inspect `context.trace` when a built-in assertion + is insufficient. + +Retain the Promptfoo result export and trace JSON as job artifacts. These are +observability and evaluation records, not guaranteed lossless transcripts. +Provider coverage differs, subagent text may be summarized or omitted, and +native reasoning may be unavailable. + +Do not add ATIF in V1. Add a converter only when a named downstream consumer +requires ATIF, and preserve explicit missing fields rather than inventing +content absent from Promptfoo/provider telemetry. + +Trace retention and redaction need explicit job settings. Promptfoo's OTLP +receiver redaction does not filter all spans emitted by built-in providers +before local storage. The disposable trace store must contain no test variables +or custom attributes with credentials, and the job must delete local trace +storage after exporting approved artifacts. + +## Isolation and security boundary + +Run write-capable evaluations inside a disposable rootless container or VM with: + +- source bootstrap separated from the unprivileged Promptfoo/agent user; +- `.eval/seeds` mounted read-only or owned by a non-agent identity; +- no host workspace mounted writable; +- no container runtime socket or host device access; +- a private writable evaluation workspace and artifacts directory; +- explicit CPU, memory, process, disk, and wall-time limits; +- network disabled unless the provider call requires an allowlisted endpoint; + and +- only scoped credentials required by source staging and model access. + +Source acquisition should finish before Promptfoo starts so acquisition +credentials can be removed. Model credentials remain provider concerns and must +not be copied into the workspace or test variables. + +This is a trusted single-tenant evaluation design. It does not safely expose a +remote endpoint to arbitrary callers. It also does not guarantee that verifier +files elsewhere in the same job are hidden from an agent with shell access. + +Agent-started background processes may outlive one tool call depending on the +provider runtime. V1 cases must not depend on persistent background services, +and disposable job teardown is the guaranteed process cleanup boundary. A +requirement for process-perfect row isolation changes the design to one +disposable sandbox per row. + +## Failure semantics + +- Source staging failure stops the job before Promptfoo. +- `beforeAll` or `beforeEach` failure prevents affected provider execution. +- Provider launch, timeout, or SDK failure remains a Promptfoo error row. +- A deterministic check returning the wrong exit status is a behavioral + assertion failure, not an infrastructure error. +- Check launch failure, timeout, invalid configuration, or unsafe path is an + assertion error with its exact cause. +- `afterEach` collection or cleanup failure writes a failure sentinel; the job + wrapper fails the run even though Promptfoo itself only logs the hook error. +- A surviving workspace, missing source/artifact manifest, job export failure, + or outer cleanup failure fails the job. + +No automatic agent retry exists. Promptfoo repetitions are intentional new +attempts, each with a new workspace. Provider-internal transport retries remain +provider behavior and must be visible through its output or trace where +supported. + +## Delivery phases and proof + +### Phase 0 — Pin the native execution surface + +Add Promptfoo and the two optional SDK dependencies at tested versions. Add the +evaluation directory, `.eval` ignore rule, configuration, and one read-only +fixture case. + +**Proof:** Promptfoo validates the configuration and both providers resolve the +same absolute `.eval/workspace` from the config directory. + +### Phase 1 — Stage exact reusable seeds + +Implement the source catalog reader and staging command with local and exact +Git sources first. Add OCI materialization only when an initial case requires +it. Record source provenance and reject path escape, mutable published seeds, +credentials in output, and partial staging directories. + +**Proof:** stage the same exact source twice in one job, observe one immutable +seed identity, and verify a failed staging attempt publishes nothing. + +### Phase 2 — Reset one private workspace per row + +Implement the lifecycle extension. Exercise two serial rows against one seed: +the first mutates and adds files; the second must observe only seed content. +Force an `afterEach` failure and verify the next `beforeEach` still deletes the +stale workspace before copying. + +**Proof:** both rows start from the same seed digest, receive different writable +workspace instances, cannot mutate the seed, and leave no workspace after the +suite. + +### Phase 3 — Grade the final filesystem + +Implement closed command and file checks. Keep check programs outside the +workspace and treat them as trusted job code, not secret material. Add focused +tests for the behavioral and safety branches listed above. + +**Proof:** a fixture agent mutation is visible to the assertion, a correct +change passes, an incorrect change fails with exact command/file evidence, and +cleanup runs after either outcome. + +### Phase 4 — Capture native traces and metadata + +Enable Promptfoo tracing, Codex streaming spans, Claude tool metadata, and +trajectory assertions. Configure approved result and trace exports plus local +trace-store deletion. + +**Proof:** one Claude and one Codex run each show final output, usage when +reported, at least one provider/tool span for a tool-using case, deterministic +workspace evidence, and no ATIF artifact. + +### Phase 5 — Run in the disposable job + +Package the exact local and CI/VPS invocation in a rootless container or VM. +Apply resource, filesystem, environment, and network limits. Export results only +after Promptfoo and the extension finish, then destroy the job. + +**Proof:** two consecutive jobs cannot see one another's workspace, seeds, +provider sessions, processes, or local trace database; approved result +artifacts remain available to CI. + +## Focused release E2E + +Run the built evaluation path, not an isolated test helper: + +1. Build the disposable job image. +2. Stage an exact fixture repository revision. +3. Run Promptfoo with `maxConcurrency: 1`, caching disabled, and both built-in + providers. +4. Give each provider a task that must edit a file and run a command. +5. Let the external assertion run the repository's deterministic check. +6. Repeat the case and verify the second row starts pristine. +7. Inspect Promptfoo output, provider metadata, and the trace timeline. +8. Verify the seed is unchanged and `.eval/workspace` is absent. +9. Export approved results/traces and destroy the container. +10. Start another container and verify no mutable state or provider session is + present. + +Record the exact source revision, image digest, Promptfoo/provider versions, +commands, and observed results in the eventual pull request. + +## Completion checklist + +- [ ] Promptfoo invokes Claude and Codex through built-in providers only. +- [ ] Every provider uses `working_dir: ./.eval/workspace`. +- [ ] `maxConcurrency: 1` and response-cache disablement are checked in. +- [ ] Source staging records exact immutable identities and leaks no credential + material. +- [ ] Every row receives a fresh private copy and cannot mutate its seed. +- [ ] Setup failures stop provider execution; cleanup failures create a + wrapper-checked failure sentinel. +- [ ] Live-workspace assertions are deterministic-only and execute before + teardown; model grading, when needed, consumes separately persisted + evidence. +- [ ] Assertions execute literal argument vectors with bounded output and time. +- [ ] Promptfoo results and OpenTelemetry traces are retained as the native + evidence formats. +- [ ] No custom provider, gateway protocol, runner service, or ATIF conversion + remains. +- [ ] Write-capable E2E runs inside a disposable rootless container or VM. +- [ ] The focused release E2E proves row and job isolation, grading, tracing, + export, and cleanup. diff --git a/docs/research/harbor-repository-materialization.md b/docs/research/harbor-repository-materialization.md index b9fe13f8..7929c584 100644 --- a/docs/research/harbor-repository-materialization.md +++ b/docs/research/harbor-repository-materialization.md @@ -1,25 +1,23 @@ # Harbor repository materialization lessons -## Current conclusion - -Treat this note as future-adapter evidence only. Harbor, Terminal-Bench, and -SWE-bench integration are not part of the current v1 contract or plan. Harbor -can still inform a later design because it packages an instruction, -environment, workdir, test script, and reward artifact around one disposable -task, but it must not mediate ordinary Promptfoo Git/OCI runs or become a core -source kind. - -Any future integration requires its own ADR and closed adapter request/result -mapping. That decision must define provenance, bounded complete/partial -evidence, artifact authorization/expiry, cancellation, cleanup-gated -publication, and how Promptfoo consumes raw observations. It must not reuse or -extend the current v1 request/result schemas without an explicit future contract -decision. Harbor may own its sandbox and checks, but the gateway must -not convert Harbor scores into pass/fail or reward. +## Status + +This note remains future-adapter research. Harbor, Terminal-Bench, and +SWE-bench are not part of the native Promptfoo V1. + +Harbor is still useful evidence because it packages an instruction, +environment, workdir, verifier, and reward artifact around one disposable task. +The selected design does not put Harbor between Promptfoo and its built-in +Claude or Codex providers, and does not treat Harbor as a source kind. + +Any future Harbor integration needs its own decision covering task provenance, +workspace ownership, verifier visibility, result mapping, and which system owns +the sandbox. It must not silently reintroduce the rejected gateway contracts or +move behavioral judgment out of the selected evaluation owner. The current boundary is defined by -[One-shot coding-agent gateway boundary](./one-shot-coding-agent-gateway-boundary.md) -and [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). +[ADR 0002](../decisions/0002-use-promptfoo-native-agent-execution.md) and +[Promptfoo native agent workspaces](./promptfoo-native-agent-workspaces.md). ## What Harbor fetches @@ -38,10 +36,11 @@ is not a mechanism for assembling several application repositories into one agen workspace. Harbor also accepts an omitted commit or a mutable ref and resolves it to a -commit. AllAgents permits a caller to override a declared repository with a -branch, tag, or commit for developer convenience, but resolves and records the -full commit before agent execution. Reproducibility-sensitive callers use a full -commit; OCI snapshots remain digest-pinned at admission. +commit. The native source catalog may declare a branch, tag, or commit for +developer convenience, but job bootstrap resolves and records the full commit +before publishing the seed. Reproducibility-sensitive catalog entries use a full +commit; OCI entries resolve and record the verified manifest digest before seed +publication. ### Task packages from the package registry @@ -152,20 +151,17 @@ through to the other after admission. Before implementing any Harbor, Terminal-Bench, or SWE-bench adapter: -1. Write a dedicated ADR and closed adapter request/result mapping. Do not add a - Harbor source kind or field to the current `AgentRun v1` schemas. +1. Write a dedicated ADR and adapter mapping. Do not add a Harbor source kind to + the native workspace catalog implicitly. 2. Decide and specify who owns the sandbox, agent invocation, checks, - cancellation, cleanup, and terminal publication. Do not imply direct-mode - isolation governed a Harbor-owned sandbox. -3. Define authoritative registry/task/version/environment provenance and, where - a runtime profile exists, immutable profile/image/tool/service implementation - digests. -4. Preserve bounded raw complete/partial agent and check evidence without - translating Harbor scores into gateway pass/fail or reward. -5. Define authenticated expiring artifact access and consumer digest/size - verification. -6. Prove cleanup before terminal result visibility and define the - infrastructure-error behavior for Harbor outages and cleanup failure. + cancellation, cleanup, and result publication. +3. Define authoritative registry, task, version, environment, and agent + provenance. +4. Preserve bounded raw agent and check evidence without translating Harbor + scores into an unrelated pass/fail shape. +5. Define artifact ownership, retention, integrity, and consumer access when + evidence must outlive the disposable job. +6. Prove cleanup and define behavior for Harbor outages and cleanup failure. 7. Keep direct Promptfoo runs independent of Harbor and let Promptfoo assertions or graders make every behavioral judgment. diff --git a/docs/research/one-shot-coding-agent-gateway-boundary.md b/docs/research/one-shot-coding-agent-gateway-boundary.md index 40cf7b6e..aa60d1f0 100644 --- a/docs/research/one-shot-coding-agent-gateway-boundary.md +++ b/docs/research/one-shot-coding-agent-gateway-boundary.md @@ -1,42 +1,34 @@ # One-shot coding-agent gateway boundary -## Decision - -Use **Promptfoo as the first caller of -`allagentsdev/allagents-gateway`**, a general one-shot coding-agent gateway. -Promptfoo owns prompts, provider and model variants, test matrices, repetition, -assertions, metrics, and result presentation. Promptfoo authoring may declare -multiple Git, OCI, or provider-local workspace sources; the provider packages -local content as an uploaded bundle before submitting `AgentRunRequest v1`. - -The same repository contains the API, disposable trial worker, agent adapters, -source materializers, artifact service, and Promptfoo provider. One public -`AgentRun` request owns one fresh internal trial: - -1. authorize every source, reuse or populate exact Git/OCI cache generations, - and materialize the requested read-only or writable views; -2. authorize the required immutable runtime profile and run Codex or OMP in its - isolated runtime; -3. after the agent reaches a terminal state, tear down its process/cgroup and - network namespace, verify descendant absence, and remove credentials; only - then materialize any authorized hidden bundle and run configured structured - post-run commands in the retained runtime/final workspace with declared - source modes preserved; -4. durably seal bounded agent output plus complete or partial raw command/file - observations; -5. complete workspace/runtime cleanup and release cache/artifact leases; and -6. only then publish `AgentRunResult v1`. - -Source credentials, source policy, credential selection, network policies, and -runtime profiles are operator configuration, never caller-supplied policy -bodies. Harbor and Terminal-Bench integration is outside the current v1 and -requires a future ADR plus closed adapter request/result mapping. The gateway -has no reusable sessions, checkpoint or continuation contract, or generic -produced-file service. Any exact patch production belongs only in a future -benchmark adapter whose upstream evaluator requires it. - -See [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md) -for the current decision record. +## Status + +This note records the hosted one-shot gateway alternative that was considered +and rejected for the initial trusted local/CI evaluation scope. Its source, +sandbox, credential, and artifact analysis remains useful if AllAgents later +needs a remote multi-tenant execution service. + +The current decision uses Promptfoo's built-in Claude and Codex providers with +an extension-managed disposable workspace. See +[ADR 0002](../decisions/0002-use-promptfoo-native-agent-execution.md) and +[Promptfoo native agent workspaces](./promptfoo-native-agent-workspaces.md). + +## Historical decision + +The prior proposal placed a general one-shot `AgentRun` API between Promptfoo +and coding agents. It assigned source materialization, immutable caching, +runtime profiles, agent adapters, post-run evidence, artifacts, and cleanup to +`allagentsdev/allagents-gateway`. + +That boundary is not part of V1. Promptfoo now invokes its built-in agent +providers directly inside one disposable job. A lifecycle extension resets +`.eval/workspace` for every serial row, deterministic assertions inspect the +final filesystem, and Promptfoo retains its native result and OpenTelemetry +trace formats. + +The gateway alternative remains relevant only if later requirements demand +remote callers, hostile tenant isolation, credential brokering, hidden +verifiers, durable cancellation/recovery, or retention independent of a +disposable job. ## Why Promptfoo is the control plane @@ -55,16 +47,12 @@ Promptfoo already owns the evaluation-shaped abstractions: structured `output`, `error`, usage, cost, and arbitrary metadata ([custom JavaScript provider](https://www.promptfoo.dev/docs/providers/custom-api/)). -Promptfoo's stock Codex provider is useful when evaluating text, traces, or -operations in an already prepared directory. It creates an ephemeral thread by -default and accepts an explicit working directory and sandbox policy -([OpenAI Codex SDK provider](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/)). -It does not by itself materialize several Git/OCI sources, create a pristine -remote trial, optionally verify the resulting filesystem, or guarantee cleanup. -The repository's custom provider is therefore a thin Promptfoo-to-gateway -adapter; the general gateway supplies materialization, one-shot agent execution, -same-runtime post-run checks, raw evidence, and cleanup while Promptfoo retains -all grading and reward policy. +Promptfoo's stock Codex and Claude Agent SDK providers accept an explicit +working directory and expose provider-native output, usage, metadata, and +tracing. They do not materialize pristine source trees themselves. The selected +design supplies that missing lifecycle with job bootstrap plus Promptfoo +`beforeEach` and `afterEach` hooks; it does not require a custom provider or +gateway. ## Multi-turn and sandboxed-code boundaries @@ -73,32 +61,30 @@ resends the complete transcript on each turn. With `stateful: true`, Promptfoo sends only the newest user message after the target returns a session ID and expects that target to retain its own history ([simulated-user provider](https://www.promptfoo.dev/docs/providers/simulated-user/)). -The gateway can accept a fully rendered transcript as one instruction, but every -provider call still creates a fresh run and workspace. That can test textual -conversation continuity; it cannot test a coding conversation that depends on -files, processes, tools, or services from an earlier turn. The provider therefore -must not return a reusable session ID or pool native Codex/OMP sessions in V1. +The native V1 deliberately disables cross-row session persistence. Provider +turns that occur inside one Promptfoo row share that row's private workspace, +but the next provider/test/repetition row starts from a new seed copy. A later +multi-turn evaluation that intentionally preserves filesystem state needs an +explicit case-level lifecycle rather than accidental thread pooling. Promptfoo's [sandboxed-code guide](https://www.promptfoo.dev/docs/guides/sandboxed-code-evals/) does not put Promptfoo or its provider inside a sandbox. Its `type: python` -assertion runs trusted user code, and that assertion explicitly calls Epicbox to -launch the generated code snippet in a one-time Docker container. This is useful -for grading code returned as text. It does not prepare a repository, isolate a -write-capable coding agent, preserve the agent's final workspace for hidden -checks, or provide the source, credential, artifact, and cleanup contracts needed -here. A larger custom assertion could rebuild those responsibilities, but that -would be another implementation of the gateway rather than a Promptfoo feature. - -## AgentRun contract - -The normative wire contracts are -[`AgentRunRequest v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#agentrunrequest-v1), -[`PostRunSpec v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#postrunspec-v1), -and -[`AgentRunResult v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#agentrunresult-v1). -Their checked-in JSON Schemas are authoritative for implementation. This -research note deliberately does not publish a second pseudo-wire schema. +assertion runs trusted user code, and that assertion explicitly calls Epicbox +to launch generated code in a one-time Docker container. The current design +therefore uses a disposable outer container or VM for write-capable agent +execution and uses Promptfoo JavaScript assertions only for trusted +deterministic checks against the retained final workspace. + +## Historical `AgentRun` contract + +The rejected gateway proposal defined closed `AgentRunRequest`, +`PostRunSpec`, and `AgentRunResult` wire contracts. They are not current +implementation contracts and the superseded implementation plan has been +replaced by the +[Promptfoo coding-agent evaluation plan](../plans/2026-09-18-0837-feat-promptfoo-coding-agent-evals-plan.md). +The details below are retained only to document what a future hosted execution +service would need to decide. The request contract is closed. At a logical level it carries request identity, the instruction, ordered workspace sources and working directory, an agent @@ -256,11 +242,10 @@ in that environment; the script writes a numeric or structured reward under ([task overview](https://docs.harborframework.com/core-concepts/tasks/overview), [task tutorial](https://docs.harborframework.com/tutorials/create-a-task)). -Harbor and Terminal-Bench are not part of the current `AgentRun v1` contract or -implementation plan. Any future integration requires its own ADR and closed -adapter request/result mapping, including provenance, evidence, artifact, -cancellation, and cleanup semantics. It must remain optional, must not become a -core source kind, and must not move pass/fail or reward into the gateway. +Harbor and Terminal-Bench are outside the native Promptfoo V1. A future +integration should define its own adapter boundary and provenance rather than +revive the rejected gateway implicitly. Promptfoo remains the owner of +behavioral pass/fail and reward. ## SWE-bench is future adapter research @@ -273,14 +258,13 @@ logs That is useful evidence that any future SWE-bench adapter owns its exact patch transport. -SWE-bench is outside the current v1 contract and implementation plan. A future -adapter requires its own ADR and closed extension, including base-commit, -filename, diff-format, size, and evaluator compatibility rules. Core -`AgentRun v1` exposes no generic diff, patch, modified workspace, or change -artifact. +SWE-bench is outside the native Promptfoo V1. A future adapter must define its +base-commit, filename, diff-format, size, and evaluator compatibility rules. +The current workspace extension does not expose a generic diff, patch, +modified-workspace, or change-artifact API. -## Recommendation +## Historical recommendation Implement the smallest complete loop: @@ -306,22 +290,13 @@ flowchart LR X --> J[Promptfoo code / LLM graders] ``` -Promptfoo is the first caller and sole evaluation owner. -`allagentsdev/allagents-gateway` owns the general one-shot `AgentRun v1` API, -policy-rechecked immutable source cache, read-only mounts, private CoW/full-copy -writable views, operator-authorized immutable runtime profiles with profile and -image digests plus tool/service implementation digests, disposable trial workers, -agent adapters, phase-separated default-drop networking, lifecycle fencing, optional -same-runtime post-run commands, bounded structured agent output, raw -complete/partial evidence, and cleanup. Result and artifact access -is tenant/run authorized; the provider dereferences expiring artifacts and -verifies size and digest. - -The gateway assembles and seals evidence, completes cleanup and lease release, -and only then returns `completed`, `cancelled`, or `infrastructure_error`; -cleanup failure produces `infrastructure_error` with partial evidence. It does -not own pass/fail or reward, and no mutable trial state is shared. Source -credentials and policy/credential routes stay out-of-band. Harbor, -Terminal-Bench, and SWE-bench are future work requiring dedicated ADRs and -closed extensions. No reusable-session layer, generic diff service, checkpoint -architecture or session protocol is warranted. +The diagram above summarizes the rejected hosted-service recommendation. It +would be appropriate only if AllAgents needed a remote execution product with +tenant authorization, strong sandboxing, policy-bound source and model +credentials, cleanup-gated results, and durable artifacts. + +For the selected trusted local/CI scope, those controls would duplicate the +disposable job and Promptfoo's built-in providers while adding a second API, +provider, adapter, trace, and persistence stack. The current design therefore +keeps the reusable insight—fresh private workspaces and post-agent filesystem +checks—but implements it with Promptfoo lifecycle hooks and assertions. diff --git a/docs/research/promptfoo-native-agent-workspaces.md b/docs/research/promptfoo-native-agent-workspaces.md new file mode 100644 index 00000000..80e09392 --- /dev/null +++ b/docs/research/promptfoo-native-agent-workspaces.md @@ -0,0 +1,496 @@ +# Promptfoo-native coding-agent workspaces + +## Conclusion + +For a **trusted, single-host local or CI evaluation**, Promptfoo's built-in Claude +Agent SDK or OpenAI Codex SDK provider can replace the proposed execution gateway's +basic evaluation loop: + +1. a Promptfoo `beforeEach` extension deletes and re-copies a fixture into the fixed + `./.eval/workspace` path; +2. `evaluateOptions.maxConcurrency: 1` prevents two cases from sharing that path; +3. the provider receives `config.working_dir: ./.eval/workspace` and runs one agent; +4. a deterministic `javascript` assertion reads the final filesystem; +5. `afterEach` records bounded observations if needed and removes the workspace; and +6. `afterAll` retries cleanup of remaining job-private state. + +That is enough when the source fixture is already local, the CI runner is the +security boundary, the checks are trusted, and raw Promptfoo results plus +OpenTelemetry traces are sufficient. It is composition, not a built-in disposable +workspace feature: the reset/materialization and filesystem verifier are glue code. +Promptfoo explicitly recommends serial execution, extension hooks, wrapper scripts, +Git, or containers for side-effecting Claude runs, and its official advanced example +uses `maxConcurrency: 1` plus an extension reset +([side-effect guidance](https://www.promptfoo.dev/docs/providers/claude-agent-sdk/#managing-side-effects), +[pinned example config](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/examples/claude-agent-sdk/advanced/promptfooconfig.yaml#L8-L29), +[pinned reset hook](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/examples/claude-agent-sdk/advanced/hooks.js#L58-L101)). + +It does **not** replace the gateway's stronger contracts: authenticated and +provenance-preserving Git/OCI/bundle composition, isolation from an untrusted agent, +phase-separated credentials and networking, resource/process-tree enforcement, +hidden post-run checks, bounded and durable evidence, normalized ATIF trajectories, +idempotency/cancellation, or an OMP adapter. Those requirements can justify an +external runner even when there is no remote multi-tenant service. + +## Evidence labels and source snapshot + +- **Documented** means Promptfoo's public documentation promises the behavior. +- **Source-observed** means the current implementation does it, but the public docs do + not define it as a stable contract. +- **Proposed glue** means code this evaluation repository must own; it is not supplied + by Promptfoo or either agent SDK provider. + +Source was inspected at Promptfoo commit +[`712a506de6412ca6879fe8dba1319569ea820cbf`](https://github.com/promptfoo/promptfoo/tree/712a506de6412ca6879fe8dba1319569ea820cbf). +All source links below are pinned to that revision. Public documentation links are +unversioned and describe the current docs as inspected on 2026-09-28. + +## Direct answer: can the fixed-workspace loop work? + +Yes, with the following boundaries. + +| Step | Status | Exact behavior and boundary | +|---|---|---| +| Serialize cases | **Documented** | `evaluateOptions.maxConcurrency: 1` sets the maximum concurrent requests to one; the CLI equivalent is `--max-concurrency 1`. `tests[].options.runSerially: true` is also supported, but global concurrency one is simpler when every case shares one path ([configuration reference](https://www.promptfoo.dev/docs/configuration/reference/#config), [test-case reference](https://www.promptfoo.dev/docs/configuration/reference/#test-case)). | +| Reset before a case | **Documented hook, proposed reset** | Root `extensions` supports `beforeEach`, whose context is `{ test }`, before each individual evaluation. Promptfoo provides the callback point, not copy/clone/reset logic ([extension hooks](https://www.promptfoo.dev/docs/configuration/reference/#extension-hooks)). | +| Point the agent at the path | **Documented** | Both providers accept `config.working_dir`; relative values resolve from the config file's directory ([Claude working directory](https://www.promptfoo.dev/docs/providers/claude-agent-sdk/#with-working-directory), [Codex working directory](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/#with-working-directory)). The shared resolver in source confirms config-relative resolution ([`resolveAgenticWorkingDir`](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/providers/agentic-utils.ts#L74-L90)). | +| Run a write-capable agent | **Documented** | Codex uses `sandbox_mode: workspace-write` by default. Claude requires explicit write/edit tools and a permission mode such as `acceptEdits`; a configured directory is read-only by default ([Codex sandbox modes](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/#sandbox-modes), [Claude tools and permissions](https://www.promptfoo.dev/docs/providers/claude-agent-sdk/#tools-and-permissions)). | +| Inspect the final filesystem | **Documented assertion API; source-observed ordering** | An external `javascript` assertion is trusted Node code and receives `output` plus `context`, so it can read files or invoke a fixed verifier ([JavaScript assertions](https://www.promptfoo.dev/docs/configuration/expected-outputs/javascript/#external-script)). Current source invokes `beforeEach`, then `runEvalInternal`, and only afterward invokes `afterEach` and persists the row ([provider/evaluation order](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluator.ts#L3567-L3593), [post-row hook order](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluator.ts#L3595-L3653)). Thus a normal deterministic assertion sees the provider's final filesystem. | +| Reset for the next case | **Proposed glue** | `afterEach` may archive bounded evidence and then remove the workspace; the next `beforeEach` removes it again before copying the seed. Cleanup failures are only logged by the current evaluator and do not automatically turn a passing row into an error ([caught `afterEach` failure](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluator.ts#L3614-L3650)). Therefore `beforeEach` must throw on a failed reset, `afterAll` must retry cleanup, and the outer job should verify teardown when cleanup is a correctness requirement. | +| Prevent a cache hit from skipping the run | **Documented and source-observed** | Set `evaluateOptions.cache: false` or use `--no-cache`. This disables the scoped cache used by agentic providers; the Claude provider otherwise fingerprints the working directory and can return a prior response ([caching configuration](https://www.promptfoo.dev/docs/configuration/caching/), [evaluation cache scope](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluate.ts#L361-L364), [agentic cache initialization](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/providers/agentic-utils.ts#L203-L273)). A cached response cannot recreate cached filesystem mutations. | + +### Important grading-order caveat + +The fixed shared path is safe for the deterministic filesystem assertion shown below. +Do not assume it remains safe if the same test also contains a model-graded assertion. +At concurrency one, the current evaluator can group model-graded assertions by provider: +it performs several target calls and defers each row's complete assertion set before +flushing grading ([grouping predicate](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluator.ts#L519-L537), +[grouped execution](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluator.ts#L3912-L4007), +[deferred assertion call](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluator.ts#L1449-L1469)). +A later `beforeEach` can therefore replace the workspace before the earlier row's +filesystem assertion executes. + +Use one of these clean boundaries: + +- keep filesystem-observing rows deterministic-only; +- have the deterministic assertion capture all needed facts and grade later from those + persisted facts in a separate evaluation; or +- use a wrapper/custom provider that returns a per-run evidence snapshot with the agent + response. + +A nonzero `evaluateOptions.timeoutMs` currently disables grouped grading, but that is +an implementation detail rather than a documented workspace-lifecycle guarantee; it +should not be the foundation of evidence correctness +([grouping condition](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluator.ts#L4939-L4960)). + +## Minimal verified composition + +The following shape is verified against the configuration reference, both provider +implementations, and Promptfoo's official Claude reset example at the pinned revision. +It intentionally uses one prompt, one provider, deterministic-only assertions, global +serialization, and disabled response caching. + +Expected repository layout: + +```text +promptfooconfig.yaml +fixtures/base/ # immutable or otherwise protected seed +.eval/workspace/ # disposable; ignored by version control +promptfoo/workspace-hooks.cjs +promptfoo/assert-workspace.cjs +``` + +### `promptfooconfig.yaml` + +```yaml +# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json + +description: Native fixed-workspace coding-agent evaluation + +prompts: + - | + Implement the requested change. Write the exact value "{{ expected }}" to result.txt. + +providers: + - id: openai:codex-sdk + config: + working_dir: ./.eval/workspace + skip_git_repo_check: true # remove when fixtures contain a Git repository + sandbox_mode: workspace-write + approval_policy: never + network_access_enabled: false + web_search_mode: disabled + enable_streaming: true + persist_threads: false + +extensions: + - file://./promptfoo/workspace-hooks.cjs:extensionHook + +evaluateOptions: + maxConcurrency: 1 + cache: false + +tracing: + enabled: true + otlp: + http: {} + +outputPath: ./.eval/results.json + +tests: + - description: writes the requested result + vars: + expected: READY + assert: + - type: javascript + value: file://./promptfoo/assert-workspace.cjs +``` + +`skip_git_repo_check: true` is necessary only for a non-Git fixture; Codex otherwise +requires the working directory or a parent to be a Git repository. `approval_policy: +never` is the documented unattended-CI recommendation, and network, search, and full +process-environment inheritance are separate settings from the filesystem sandbox +([Codex parameters and caveats](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/#supported-parameters), +[Codex sandbox modes](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/#sandbox-modes)). + +For Claude, replace only the provider block: + +```yaml +providers: + - id: anthropic:claude-agent-sdk + config: + working_dir: ./.eval/workspace + append_allowed_tools: [Write, Edit, MultiEdit, Bash] + permission_mode: acceptEdits + persist_session: false + sandbox: + enabled: true + failIfUnavailable: true +``` + +Claude's `sandbox.enabled` is not implied by `working_dir` or `permission_mode`. +`failIfUnavailable` defaults to true when the sandbox is enabled, but spelling it out +makes the CI requirement visible. Network domains, local binding, Unix sockets, and +credential masking have their own `sandbox.network.*` and +`sandbox.credentials.envVars` keys +([Claude sandbox configuration](https://www.promptfoo.dev/docs/providers/claude-agent-sdk/#sandbox-configuration)). + +`deep_tracing` is intentionally omitted from the baseline. Root tracing plus Claude's +provider spans, or Codex `enable_streaming`, supplies the normal evaluation trajectory +with less payload exposure. Opt into deep tracing only when SDK/CLI-internal spans are +required: Codex deep tracing disables `persist_threads`, `thread_id`, and +`thread_pool_size`, and native CLI spans can contain payloads outside Promptfoo's +stream-event sanitizer +([Codex deep tracing](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/#deep-tracing)). + +### `promptfoo/workspace-hooks.cjs` — proposed glue + +```js +const fs = require('node:fs/promises'); +const path = require('node:path'); + +const root = path.resolve(__dirname, '..'); +const seed = path.join(root, 'fixtures', 'base'); +const workspace = path.join(root, '.eval', 'workspace'); + +async function materializeFreshWorkspace() { + await fs.rm(workspace, { recursive: true, force: true }); + await fs.mkdir(path.dirname(workspace), { recursive: true }); + await fs.cp(seed, workspace, { recursive: true, errorOnExist: true }); +} + +module.exports = async function extensionHook(hookName, context) { + if (hookName === 'beforeEach') { + // Do not catch this error: a failed reset must prevent the agent call. + await materializeFreshWorkspace(); + } else if (hookName === 'afterEach' || hookName === 'afterAll') { + // Assertions have completed before afterEach on the normal deterministic path. + await fs.rm(workspace, { recursive: true, force: true }); + } + return context; +}; +``` + +This materializes a local directory tree; it does not resolve a Git ref, pull OCI +content, validate a bundle digest, enforce read-only subtrees, or record provenance. +A CI checkout step or wrapper must prepare `fixtures/base`. The seed must not be +agent-writable. If checks need the final tree after the entire evaluation, copy selected +evidence to a case-specific artifact directory in `afterEach` before deleting the +workspace. + +### `promptfoo/assert-workspace.cjs` — proposed deterministic grader + +```js +const fs = require('node:fs/promises'); +const path = require('node:path'); + +const workspace = path.resolve(__dirname, '..', '.eval', 'workspace'); + +module.exports = async function assertWorkspace(_output, context) { + try { + const actual = await fs.readFile(path.join(workspace, 'result.txt'), 'utf8'); + const expected = String(context.vars.expected); + const pass = actual.trim() === expected; + return { + pass, + score: pass ? 1 : 0, + reason: pass + ? 'result.txt has the expected content' + : `result.txt was ${JSON.stringify(actual.trim())}`, + }; + } catch (error) { + return { pass: false, score: 0, reason: `result.txt unavailable: ${error.message}` }; + } +}; +``` + +For executable checks, this module can use `execFile`/`spawn` with a fixed executable +and literal argument array. That remains trusted assertion code running as the +Promptfoo process. It is not the gateway's separately sandboxed, credential-stripped, +structured `post_run` phase. + +## Lifecycle hooks: exact names and timing + +Two unrelated hook systems must not be conflated. + +### Promptfoo evaluation extensions + +Root `extensions` accepts JavaScript or Python functions. The exact lifecycle names +are `beforeAll`, `beforeEach`, `afterEach`, and `afterAll` +([configuration reference](https://www.promptfoo.dev/docs/configuration/reference/#available-hooks)). + +| Hook | Documented context | Relevant timing | Mutation contract | +|---|---|---|---| +| `beforeAll` | `{ suite }` | Once before the evaluation | May return selected mutated suite fields. | +| `beforeEach` | `{ test }` | Before one expanded evaluation step invokes the provider | Returning `{ test }` replaces the test context; a thrown reset error propagates. Current source calls it immediately before `runEvalInternal` ([source](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluator.ts#L3567-L3593)). | +| `afterEach` | `{ test, result }` | After the row is graded, before persistence in the normal non-deferred path | Only `result.namedScores`, `result.metadata`, and `result.response.metadata` are persisted; it cannot override `success`, `score`, or `response.output` ([mutation reference](https://www.promptfoo.dev/docs/configuration/reference/#aftereach)). | +| `afterAll` | `{ results, prompts, suite, evalId, config }` | Once after all rows | Side effects only; its return value is not persisted. | + +A path naming a custom function, such as +`file://./workspace-hooks.cjs:extensionHook`, receives every event as +`(hookName, context)`. A path whose function name is exactly one lifecycle name runs +only for that event and is called as `(context, { hookName })` +([implementation rules](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluatorHelpers.ts#L774-L816)). +Returned mutable fields are shallow-merged, not deep-merged +([extension mutation docs](https://www.promptfoo.dev/docs/configuration/reference/#extension-hooks)). + +`beforeEach` is invoked for each expanded `RunEvalOptions`, not merely once for the +original YAML object (**source-observed**). With multiple prompts, providers, or +repeats, the same logical YAML test can therefore be reset several times. The scheduler +runs `options.runSerially` steps first and other steps through a concurrency-limited +loop; global concurrency one makes both phases single-file +([scheduler source](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluator.ts#L4048-L4111), +[matrix partition](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/evaluator.ts#L4924-L4951)). + +### Claude Agent SDK provider hooks + +`providers[].config.hooks` is a different, SDK-native interception system for events +such as `PreToolUse` and `PostToolUse`. Promptfoo preserves SDK input/return shapes; +these callbacks are programmatic-only and must be defined in a JS/TS provider file, +not expressed as YAML functions. For example, `PostToolUse` can return +`updatedToolOutput` before the model sees a tool result +([Claude provider hooks](https://www.promptfoo.dev/docs/providers/claude-agent-sdk/#hooks)). +These hooks are useful inside a run, but they do not reset the shared workspace between +Promptfoo cases. + +## Provider comparison + +| Capability | Claude Agent SDK (`anthropic:claude-agent-sdk`) | OpenAI Codex SDK (`openai:codex-sdk`) | +|---|---|---| +| Working directory | `working_dir`; omitted means a provider-created temporary directory that is removed after the call. An explicit directory persists and is read-only by default ([docs](https://www.promptfoo.dev/docs/providers/claude-agent-sdk/#quick-start), [cleanup source](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/providers/claude-agent-sdk.ts#L1801-L1824)). | `working_dir`; omitted means the current directory. It must be in a Git repo unless `skip_git_repo_check: true` ([docs](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/#quick-start)). | +| Write enablement | Add `Write`, `Edit`, `MultiEdit`, and optionally `Bash` via `append_allowed_tools`/`custom_allowed_tools`/`tools`; set `permission_mode`. | `sandbox_mode: workspace-write` is default; set `approval_policy: never` for unattended evaluation. | +| Filesystem isolation | Claude SDK `sandbox.enabled`; `failIfUnavailable`, network/socket policy, exclusions, and credential masking are explicit. | `sandbox_mode` controls filesystem access only; network/search, environment inheritance, and approvals are separate. | +| Extra paths | `additional_directories` | `additional_directories` | +| Fresh session default | Auto-generated SDK session; `persist_session` defaults true on disk, while `continue`/`resume` are opt-in. A new Promptfoo call is not by itself a fresh filesystem. | New ephemeral thread for each case by default. `persist_threads`, `thread_id`, and `thread_pool_size` opt into reuse; `deep_tracing` ignores all three and makes a fresh SDK client/thread ([thread docs](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/#thread-management), [deep-trace caveat](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/#deep-tracing)). | +| Provider response metadata | `metadata.toolCalls`, `skillCalls`, `numTurns`, `durationMs`, `durationApiMs`, `modelUsage`, permission denials, terminal reason, structured output, and surfaced assistant errors when present ([response construction](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/providers/claude-agent-sdk.ts#L2267-L2339)). | General Codex items are not copied into stable `metadata`; current `metadata` is skill-detection data when present. `output` is final text, `sessionId` is the thread, token usage/cost are separate, and `raw` serializes the SDK turn ([response construction](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/providers/openai/codex-sdk.ts#L2419-L2449)). | +| Tool/trajectory visibility | Completed tool calls are available directly in `metadata.toolCalls`; provider tracing also emits tool and turn spans. | Set `enable_streaming: true` for command, file-change, MCP, search, reasoning, message, and turn spans; set `deep_tracing: true` for CLI-native spans. | +| Full transcript | No stable full-transcript result. `raw` is the terminal SDK result; `metadata.toolCalls` carries tool I/O. Subagent text/thinking is omitted by default and requires `forward_subagent_text: true`, with documented redaction behavior ([tool tracking](https://www.promptfoo.dev/docs/providers/claude-agent-sdk/#tool-call-tracking)). | No stable normalized transcript result. Final text is `output`; streaming-mode `raw` includes provider-specific `items`, sanitized `reasoningTexts`, and `conversationMessages`, while traces carry operation events ([stream-result source](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/providers/openai/codex-sdk.ts#L1544-L1561)). | + +Neither provider materializes an application repository into an explicit directory. +Claude's automatic temporary directory is empty and deleted, and Codex defaults to the +current directory. An explicit `working_dir` means "operate here," not "make this path +fresh." The provider validates/accesses the directory after the evaluation extension +has prepared it +([Claude validation and `cwd`](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/providers/claude-agent-sdk.ts#L1801-L1844), +[Codex resolution and validation](https://github.com/promptfoo/promptfoo/blob/712a506de6412ca6879fe8dba1319569ea820cbf/src/providers/openai/codex-sdk.ts#L2222-L2273)). + +## Deterministic filesystem grading and result metadata + +A JavaScript assertion can return a boolean, number, or `{ pass, score, reason, +namedScores?, componentResults? }`; its `context` includes the prompt, vars, test, +provider, complete `providerResponse`, response metadata shortcut, and trace data when +enabled +([JavaScript assertion context](https://www.promptfoo.dev/docs/configuration/expected-outputs/javascript/#using-test-context)). +This supports deterministic checks of: + +- exact file content, mode, or absence; +- a parsed manifest or compiler output; +- a fixed executable's exit code/stdout/stderr; and +- Claude's `context.providerResponse.metadata.toolCalls`. + +Promptfoo can export complete evaluation data to JSON or one row per line to JSONL +([output formats](https://www.promptfoo.dev/docs/configuration/outputs/)). An +`afterEach` hook can add structured observations to `result.metadata`, add numeric +metrics to `result.namedScores`, or add provider-level details to +`result.response.metadata`; it cannot retroactively change the grade +([afterEach contract](https://www.promptfoo.dev/docs/configuration/reference/#aftereach)). + +Limitations of this native pattern: + +1. Assertions run with the Promptfoo process's authority, not a separate restricted + checker identity. +2. There is no built-in hidden-check-bundle timing boundary. Keeping the assertion + outside `working_dir` is not a guarantee that a shell-capable agent cannot read it. +3. There is no built-in structured post-run command/result schema, byte limit, file + promotion, digest, or artifact retention contract. +4. A fixed workspace has only one live final state. A later `beforeEach` overwrites it; + evidence that must survive must be copied or serialized per case. +5. `afterEach` cleanup is best-effort in the current evaluator because its exception + is caught. The next `beforeEach` and final `afterAll` should retry and throw, and the + surrounding job should independently verify cleanup when it is a release condition. +6. Provider and evaluation timeouts stop the call, but native composition does not + establish the gateway's process-tree/cgroup cleanup guarantee for arbitrary daemons + the agent launched. + +## OpenTelemetry, trajectory assertions, and transcript limits + +Root `tracing.enabled: true` creates a distinct trace per test-case execution and puts +`traceId` plus `evaluationId` on each result row. Promptfoo's built-in OTLP receiver is +configured under `tracing.otlp.http`; traces can be inspected in the UI, fetched through +`GET /api/traces/:traceId` or `GET /api/traces/evaluation/:evaluationId`, or exported as +JSON +([tracing overview](https://www.promptfoo.dev/docs/tracing/#built-in-provider-instrumentation), +[result-row linkage and API](https://www.promptfoo.dev/docs/tracing/#trace-linkage-on-result-rows), +[JSON export](https://www.promptfoo.dev/docs/tracing/#exporting-traces)). +JavaScript assertions receive trace spans as `context.trace`, and built-in +`trajectory:tool-used`, `trajectory:tool-args-match`, `trajectory:tool-sequence`, +`trajectory:step-count`, and `trajectory:goal-success` assertions consume normalized +span information +([traced-workflow assertions](https://www.promptfoo.dev/docs/tracing/#4-assert-on-traced-workflows)). + +Provider differences matter: + +- Claude emits an `invoke_agent` span, `gen_ai.turn N` spans, and a child span for each + completed tool call. `deep_tracing: true` asks the SDK subprocess to export native + model/tool/subagent spans to the receiver + ([Claude tracing](https://www.promptfoo.dev/docs/providers/claude-agent-sdk/#tracing)). +- Codex needs `enable_streaming: true` for Promptfoo to turn SDK events into command, + file-change, MCP, search, reasoning, message, and turn spans. `deep_tracing: true` + additionally injects OTEL context into the Codex CLI + ([Codex tracing](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/#tracing-and-observability)). +- Codex streaming still returns only after the turn completes; it is event aggregation, + not live partial-token delivery to assertions + ([streaming behavior](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/#streaming)). + +These traces are useful trajectory evidence, but they are not the proposed gateway's +bounded, normalized ATIF v1 artifact. No ATIF exporter or stable cross-provider full +transcript contract was found in the inspected Promptfoo docs/source (**source-review +finding, not a documented guarantee of absence**). The supported export is Promptfoo's +OpenTelemetry-shaped trace JSON. Provider `raw` payloads remain SDK-specific, and +neither provider documents them as a complete, size-bounded transcript schema. + +Treat trace redaction as defense in depth. Promptfoo warns that Codex CLI-native spans +created by `deep_tracing` are outside Promptfoo's stream-event sanitizer, and the OTLP +receiver's `redactAttributes` does not filter in-process built-in provider spans before +local storage +([Codex deep-trace warning](https://www.promptfoo.dev/docs/providers/openai-codex-sdk/#deep-tracing), +[trace redaction scope](https://www.promptfoo.dev/docs/tracing/#configuration-reference)). + +## Native composition versus the proposed gateway + +The accepted ADR now selects this native composition and explicitly rejects a gateway +for V1. It makes the disposable job the outer isolation/lifecycle boundary and assigns +exact source staging plus bounded diagnostics to job-owned glue +([native-execution decision](../decisions/0002-use-promptfoo-native-agent-execution.md#decision), +[evaluation plan contract](../plans/2026-09-18-0837-feat-promptfoo-coding-agent-evals-plan.md#product-contract)). +The table below compares that decision with the stronger responsibilities of the +earlier proposed gateway so the point at which a future external runner becomes +justified remains explicit. + +| Gateway responsibility | Native Promptfoo + fixed local path | Assessment for trusted local/CI | +|---|---|---| +| Prompt/test/provider matrices, repeats, grading, reports | Native Promptfoo strength | **Replace gateway portion; Promptfoo already owns this.** | +| Run Claude or Codex in an existing checkout | Built-in SDK providers with `working_dir` | **Sufficient.** | +| Fresh workspace per case | Extension/wrapper deletes and copies a fixture | **Sufficient as repo-owned glue**, provided reset failure aborts and the seed is protected. | +| Deterministic final-filesystem checks | Trusted JS assertion reads files or runs a fixed verifier | **Sufficient** for deterministic-only rows and trusted checks. | +| Basic tool/step evidence | Claude metadata plus OTLP traces; Codex streamed and optionally deep OTLP traces | **Usually sufficient**, if Promptfoo JSON/trace JSON is the accepted evidence format. | +| Exact Git/OCI/bundle acquisition and provenance | Not provided by these providers or extensions | Use CI checkout/container tooling for a simple local case; retain an external materializer when exact multi-source provenance is a requirement. | +| Read-only and writable source views in one composed tree | No first-class source/access-mode model | External runner required when access modes are security properties rather than fixture convention. | +| Agent isolation, resource ceilings, process-tree reaping | Partial provider-specific sandbox controls; no gateway-equivalent lifecycle contract | External sandbox/runner required for untrusted code or strict CPU/memory/PID/IO cleanup. | +| Separate agent/check credentials and network namespaces | Assertions share the host process/runtime; no hidden late bundle | External runner required for secret tests, phase separation, or policy-enforced egress. | +| Bounded output, requested files, digests, immutable artifacts, retention | Promptfoo outputs/traces, plus arbitrary glue | External evidence/artifact service required when these are contractual. | +| Stable normalized ATIF trajectory | OTLP spans and provider-specific raw/metadata | External normalization required if ATIF is mandatory. | +| Idempotency, cancellation, queue recovery, durable terminal states | Local process semantics only | External control plane required if ambiguous/retried infrastructure execution matters. | +| OMP execution and cross-agent parity | No built-in OMP provider among the two evaluated here | Custom provider or external adapter required. | + +## Application to the PR 679 Promptfoo experiment + +The authenticated +[`framework-parity/promptfoo/pr-679`](https://github.com/EntityProcess/wtg-ai-prompts-experiment/tree/main/framework-parity/promptfoo/pr-679) +experiment currently uses top-level Promptfoo `metadata` as an execution +configuration channel: + +- the + [`with-agentrules` suite](https://github.com/EntityProcess/wtg-ai-prompts-experiment/blob/main/framework-parity/promptfoo/pr-679/with-agentrules.suite.yaml) + puts repository URLs, revisions, a workdir, Git-cache configuration, and a + skills-config path under `metadata`; +- `setup_environment_extension.ts` reads `suite.metadata.environment`, creates a + timestamped workspace, and publishes its path through process environment + variables; and +- `skills_extension.ts` reads `suite.metadata.skills`, while the PI provider + consumes the resulting workspace and manifest through `workdirEnv` and + `environmentManifestEnv`. + +That works, but Promptfoo documents top-level `metadata` as arbitrary data stored +with the eval config, not as a workspace lifecycle schema. The fixed-workspace +composition provides a cleaner replacement: + +1. Move repository and exact-revision declarations to the checked-in source + catalog owned by the evaluation harness. +2. Let the job launcher stage the CargoWise seed before Promptfoo starts. +3. Put `source_id` or a workspace-profile ID in suite `defaultTest.vars`; the + `beforeEach` extension copies that seed to `./.eval/workspace`. +4. Configure a built-in agent with + `working_dir: ./.eval/workspace`. If the PI provider remains, give it the same + ordinary `working_dir` config field instead of discovering a path through + suite metadata and environment-variable indirection. +5. Treat the with-skill and without-skill variants as distinct staged workspace + profiles. Skill materialization belongs in source/profile setup, or in the + built-in provider's documented skill configuration when that provider owns + skill loading. +6. Keep Promptfoo `metadata` descriptive only: source PR, source eval, experiment + tags, and other report annotations. + +The shared PR 679 cases currently model-grade the agent's textual review and do +not inspect a mutable final filesystem, so Promptfoo's deferred-grading behavior +does not invalidate them. If those cases later add live-workspace assertions, +the deterministic phase must persist row evidence before a separate model- +grading evaluation, as described above. + +## Narrow recommendation + +Adopt the native composition first for evaluations that meet **all** of these +conditions: + +- one trusted local/CI runner owns the workspace and credentials; +- inputs are already checked out or can be copied from a protected local fixture; +- one shared `./.eval/workspace` is acceptable with `maxConcurrency: 1`; +- tests can use deterministic filesystem assertions without same-row deferred + model grading; +- provider-specific Claude/Codex metadata plus Promptfoo OTLP trace JSON is adequate; +- caching is disabled so every row actually executes; and +- the provider sandbox plus the CI/container boundary is an acceptable risk boundary. + +Under those conditions, a gateway adds little evaluation value. Keep the composition +small: one fail-closed `beforeEach` materializer, one deterministic assertion module, +one built-in provider, and optional tracing. Do not recreate an API, job queue, upload +protocol, or artifact store around a local run. + +Retain or introduce an external gateway/runner only when at least one concrete +requirement crosses that boundary: untrusted agent execution; multi-source Git/OCI +composition with verified provenance; read-only mount enforcement; hidden checks; +credential/network phase separation; strict cgroup/process cleanup; durable bounded +artifacts; idempotent cancellation/recovery; mandatory ATIF; OMP support; or execution +on a machine other than the Promptfoo process. Remote multi-tenancy is one reason for +those contracts, not a prerequisite for them. diff --git a/docs/research/source-credential-broker-precedents.md b/docs/research/source-credential-broker-precedents.md index dba530f4..88f65533 100644 --- a/docs/research/source-credential-broker-precedents.md +++ b/docs/research/source-credential-broker-precedents.md @@ -1,45 +1,28 @@ # Source credential broker precedents -## Current conclusion - -`allagentsdev/allagents-gateway` materializes sources declared by normative -[`AgentRunRequest v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#agentrunrequest-v1). -Git supplies a canonical URL and ref; OCI supplies a canonical repository plus -direct descriptor because a digest alone has no fetch location. Caller JSON -never carries a credential, policy-route name, or credential selector. - -The authenticated caller plus canonical Git URL or OCI repository must match -exactly one operator-configured policy/credential route; zero or ambiguous -matches reject before network access. The same match runs on every cache lookup. -Private acquisition denies redirects and permits only the matched route; any OCI -bearer-token realm is operator-pinned rather than accepted from an arbitrary -challenge. A miss may use the matched credential, while a hit needs no source -credential. Cached generations contain neither credentials nor mutable state. - -The required `runtime_profile_id` is independently authorized at admission, -which persists one immutable revision and `profile_digest`; dispatch uses only -that revision and re-verifies its profile/image, tool/service implementation, -and applicable service-image digests. `RuntimeProvenance` returns those exact -digests plus the logical ID, versions, and sandbox-policy version. Source -acquisition, task-service, agent, and post-run network namespaces are -phase-separated and default-drop. The agent -uses a run-scoped, credential-free local model proxy; model credentials never -enter its workspace. After any terminal state, the worker tears down the agent -process/cgroup and network namespace, verifies descendants are absent, and -removes acquisition/model material. Hidden-bundle bytes are not materialized -until that boundary has passed. Post-run commands preserve declared source modes -and receive no source/model credentials. -Status, cancellation, and result access enforce tenant/run -ownership; bundle upload enforces tenant ownership; expiring result-artifact -downloads enforce tenant/run ownership and are verified for size and digest. No -credential may enter bundles, raw evidence, logs, or retained artifacts. - -Git credential helpers, GitHub App installation tokens, and BuildKit secret -mounts establish the phase-boundary precedents below. A standalone network -credential broker is unnecessary for the initial deployment. The current -boundary is defined by -[One-shot coding-agent gateway boundary](./one-shot-coding-agent-gateway-boundary.md) -and [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). +## Status + +This note records credential and policy precedents for the rejected hosted +gateway design. Its primary-source findings remain relevant if AllAgents later +accepts remote callers or must broker source and model credentials across a +tenant boundary. + +The current trusted local/CI design has no gateway credential broker. Disposable +job bootstrap resolves exact sources and materializes job-private seeds before +Promptfoo starts. Acquisition credentials must not enter the seed, mutable +workspace, test variables, result metadata, or trace attributes, and should be +removed from the environment before agent execution whenever the source +transport permits it. + +Promptfoo then invokes its built-in Claude or Codex provider in +`.eval/workspace`. The outer disposable container or VM is the selected +isolation boundary; this is not equivalent to the operator-authorized, +phase-separated credential and network controls described below. + +See [ADR 0002](../decisions/0002-use-promptfoo-native-agent-execution.md) and +[Promptfoo native agent workspaces](./promptfoo-native-agent-workspaces.md) for +the current decision. The detailed broker design below is historical +future-service research, not a V1 implementation contract. ## Precedents @@ -190,7 +173,10 @@ mount/socket/environment before agent execution. Like BuildKit, this delivery mechanism does not mint credentials and does not make code with access to the secret trustworthy. -## Recommendation for evaluations +## Historical gateway recommendation + +If a future hosted execution service needs the stronger boundary studied here, +the prior recommendation was: 1. Use the normative `AgentRunRequest v1`, `PostRunSpec v1`, and `AgentRunResult v1` contracts rather than a second credential-specific shape. @@ -227,7 +213,8 @@ secret trustworthy. artifacts through the authenticated endpoint and verify streamed size and digest. -A central token minter, delivery lease, or generic credential-broker protocol is -out of scope until remote multi-tenant workers create a concrete need. The rule -for the current gateway is: **credentials exist only in the phase that consumes -them and never enter raw post-run evidence or durable trial state.** +A central token minter, delivery lease, or generic credential-broker protocol +remains out of scope until remote or otherwise untrusted workers create a +concrete need. The future-service rule is: **credentials exist only in the +phase that consumes them and never enter raw post-run evidence or durable trial +state.** diff --git a/docs/research/workspace-contract-incumbents.md b/docs/research/workspace-contract-incumbents.md index b07e5e90..e27e5774 100644 --- a/docs/research/workspace-contract-incumbents.md +++ b/docs/research/workspace-contract-incumbents.md @@ -2,33 +2,32 @@ ## Current conclusion -No examined incumbent replaces the chosen boundary. Promptfoo is the first -caller and sole evaluation owner, while `allagentsdev/allagents-gateway` owns the -general one-shot `AgentRun v1` API, policy-bound immutable source cache, -operator-authorized immutable runtime profiles with profile/image and tool/service -implementation digests, disposable trial workers, agent adapters, -phase-separated default-drop networking, bounded raw evidence, -cleanup, and first Promptfoo provider. Promptfoo JSON may describe several -workspace sources; policy/credential routes and runtime profiles remain operator -configuration. - -Each `AgentRun` creates one fresh workspace and runs Codex or OMP under the -required `runtime_profile_id`. After the agent terminates, the worker tears down -its process/cgroup and network namespace, verifies descendants are absent, and -removes credentials. Hidden-bundle bytes are materialized only after that -boundary; structured post-run commands then use the retained runtime/final -workspace with declared source modes intact. The gateway durably seals complete -or partial observations, destroys trial state, releases leases, and only then -publishes `AgentRunResult v1`. Cleanup failure is `infrastructure_error` with -partial evidence. Tenant/run-authorized artifact retrieval is time-bounded and -the provider verifies byte size and digest. Promptfoo alone decides pass/fail -and reward. Harbor, Terminal-Bench, and SWE-bench require future ADRs and closed -adapter extensions; none is part of current v1, and no incumbent justifies -reusable execution sessions. - -The current boundary is defined by -[One-shot coding-agent gateway boundary](./one-shot-coding-agent-gateway-boundary.md) -and [ADR 0002](../decisions/0002-use-allagents-gateway-for-one-shot-runs.md). +No examined incumbent is needed between Promptfoo and the coding agents for the +initial trusted local/CI scope. Promptfoo's built-in Claude Agent SDK and Codex +SDK providers already accept a prepared `working_dir`, while Promptfoo lifecycle +hooks can reset that directory around each serial evaluation row. + +The selected boundary is one disposable job: + +- job bootstrap resolves exact sources and creates read-only seeds; +- `beforeEach` privately copies the selected seed to `.eval/workspace`; +- the built-in provider runs there; +- Promptfoo assertions inspect the final filesystem and native trace; and +- `afterEach` removes the workspace before the next row. + +Promptfoo owns matrices, repetitions, assertions, pass/fail, rewards, metrics, +OpenTelemetry traces, and result presentation. The extension owns only +workspace setup, bounded diagnostics, reset, and cleanup. A container or VM owns +the outer process and filesystem isolation boundary. + +The gateway, source-builder, snapshot, and long-lived session contracts +evaluated below are rejected for V1. Their incumbent comparisons remain useful +if a future remote multi-tenant execution service needs stronger authorization, +credential, artifact, cancellation, or recovery boundaries. + +The current decision is +[ADR 0002](../decisions/0002-use-promptfoo-native-agent-execution.md), supported +by [Promptfoo native agent workspaces](./promptfoo-native-agent-workspaces.md). ## Historical contract evaluated @@ -137,55 +136,40 @@ For Git, a full commit object ID is the resolved source identity. The request st ## Current recommendation -Adopt the following rule for future evaluation work: - -- **First caller:** Promptfoo owns prompt/provider matrices, repeats, assertions, - code/LLM grading, rewards, metrics, and result presentation. -- **Invocation contract:** use the normative - [`AgentRunRequest v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#agentrunrequest-v1), - [`PostRunSpec v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#postrunspec-v1), - and - [`AgentRunResult v1`](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md#agentrunresult-v1); - this research note defines no alternate wire shape. -- **Runtime boundary:** admission resolves and persists one authorized immutable - runtime-profile revision and `profile_digest`; dispatch uses only that revision - and re-verifies its profile/image, tool/service implementation, and applicable - service-image digests. `RuntimeProvenance` returns those identities with - `runtime_profile_id`, versions, and sandbox-policy version. -- **Source route:** canonical Git URL or OCI repository plus authenticated - caller must match exactly one operator policy/credential route; reject zero or - ambiguous matches before network access. Deny redirects and pin any OCI auth - realm through the matched route. -- **Gateway ownership:** `allagentsdev/allagents-gateway` contains the general - one-shot API, disposable trial worker, materializers, agent adapters, lifecycle - fencing, artifact service, post-run execution, raw evidence, cleanup, and - Promptfoo provider. -- **Cache boundary:** repeat source authorization on every hit; publish only - verified exact generations. Mount read-only generations directly; give - writable sources a private reflink/CoW clone or full-copy fallback. -- **Post-run boundary:** tear down the agent process/cgroup and network - namespace, verify descendants are absent, and remove credentials before hidden - bundle bytes exist in the run filesystem. Only then materialize an authorized - optional bundle and run structured commands in the final workspace with - declared source modes intact. Phase-separated namespaces default-drop traffic; - post-run policy may allow only declared localhost/sidecars. -- **Result boundary:** bound agent final output as `CapturedText`; preserve full - request-order complete/partial `PostRunEvidence`; seal evidence, clean the - workspace, and release leases before publishing `completed`, `cancelled`, or - `infrastructure_error`. Cleanup failure retains partial evidence. -- **Artifact boundary:** status, cancel, result, and artifact access are - tenant/run authorized. Expiring artifact references are dereferenced through - the authenticated endpoint and verified for streamed size and digest. -- **Judgment boundary:** Promptfoo alone assigns behavioral pass/fail/reward. -- **Benchmark compatibility:** Harbor, Terminal-Bench, and SWE-bench are future - work requiring dedicated ADRs and closed adapter extensions. None changes the - current v1 schemas. +Adopt the native Promptfoo boundary for initial evaluation work: + +- **Evaluation owner:** Promptfoo owns prompt/provider matrices, repeats, + assertions, grading, rewards, metrics, traces, and result presentation. +- **Agent invocation:** use Promptfoo's built-in Claude Agent SDK and Codex SDK + providers rather than a custom provider or AllAgents adapter. +- **Working directory:** configure every write-capable provider with + `working_dir: ./.eval/workspace`. +- **Workspace lifecycle:** stage exact read-only seeds once per disposable job; + `beforeEach` creates a private writable copy and `afterEach` removes it. +- **Concurrency:** set `evaluateOptions.maxConcurrency: 1`; parallelize only by + running independent disposable jobs with separate `.eval` roots. +- **Caching:** disable Promptfoo response caching so a cached response cannot + bypass the agent and leave filesystem assertions grading an unrelated + workspace. +- **Judgment:** run deterministic command and file assertions after the provider + returns and before workspace teardown. +- **Trace:** retain Promptfoo OpenTelemetry data and use its built-in + `trajectory:*` assertions. Do not add ATIF without a named interoperability + consumer. +- **Isolation:** use a disposable rootless container or VM for write-capable + evals. Provider sandboxes are not a hostile tenant boundary. +- **Future service:** reconsider a gateway only for remote callers, mutually + untrusted tenants, centrally enforced network policy, credential brokering, + secret verifier injection, durable cancellation/recovery, or cross-machine + scheduling and retention. ## Existing research status +- [Promptfoo native agent workspaces](./promptfoo-native-agent-workspaces.md) is + the current execution research. - [One-shot coding-agent gateway boundary](./one-shot-coding-agent-gateway-boundary.md) - is the current boundary analysis. + records the rejected hosted-service alternative. - [Harbor repository materialization](./harbor-repository-materialization.md) remains useful evidence for task packages and separate verifiers. -- General and private research wikis were discovery inputs only; cited primary - sources and ADR 0002 carry the decision. +- The incumbent descriptions above remain primary-source evidence; their + historical gateway prescriptions are not current implementation requirements.