Add WebFetch access with short-lived device delegation

.github/workflows/ci.yml

@@ -10,6 +10,21 @@ permissions:
 jobs:
   test:
     runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres:17-alpine
+        env:
+          POSTGRES_PASSWORD: wanctl-ci
+          POSTGRES_DB: wanctl_test
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 5s
+          --health-timeout 5s
+          --health-retries 10
+    env:
+      WANCTL_TEST_POSTGRES: postgres://postgres:wanctl-ci@127.0.0.1:5432/wanctl_test?sslmode=disable
     steps:
       - name: Check out repository
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7

README.md

@@ -13,6 +13,7 @@ relay, while each device keeps final authority through a local approval policy.
 - One Go binary for relay, portal, agent, controller, and MCP roles.
 - Proxy-agnostic HTTP long-poll transport that works through ordinary reverse proxies; WebSocket remains optional.
 - CLI and MCP surfaces designed for scripted and AI-agent-driven control.
+- Optional [WebFetch access](docs/webfetch.md) for URL-reading web AIs, with owner-approved, short-lived device delegation and existing device policy.
 
 ## Quick start
 
@@ -86,6 +87,10 @@ database of its own; it authenticates users and scopes calls to the relay's
 Postgres-backed admin API. See [Architecture](docs/architecture.md) for the
 trust model, transports, sharing rules, and component map.
 
+When enabled, the WebFetch component is itself a trusted controller endpoint
+and sees its own commands/results. The broker's other controller-to-device
+sessions remain opaque; see the [adapter trust boundary](docs/webfetch.md).
+
 ## Build
 
 wanctl requires the Go release named by the `go` directive in `go.mod` (currently 1.26.6); the Go tool downloads it automatically.

docs/architecture.md

@@ -46,6 +46,14 @@ transport + pgstore + admin + dist), `agent`, `client`, `server` (shell+files),
 `policy` (rules+approver), `console` (transport-neutral approval queue),
 `portal`, `eventlog`, `sessionauth` (relay-issued capability grants).
 
+The opt-in [WebFetch adapter](webfetch.md) adds URL-only AI clients. Its owner
+approval and short-lived device scopes live in the existing portal/token store;
+the adapter is an ordinary E2E controller with a distinct identity per grant.
+Unlike the opaque broker, this explicitly enabled controller component sees
+the plaintext it sends and receives. Device-local action policy remains the
+authority. Delegated metadata is preserved across admission and both carriers;
+namespace-only legacy paths refuse delegated credentials.
+
 ## Transports
 
 Two carriers speak the **same** TLS + framed protocol:

docs/architecture.zh.md

@@ -36,6 +36,11 @@ controller (you/agent) ──┐                            ┌── device (wa
 `server`（shell + 文件）、`policy`（规则 + 审批者）、`console`（与传输无关的审批队列）、
 `portal`、`eventlog`、`sessionauth`（relay 签发的能力授予）。
 
+可选的 [WebFetch 接入层](webfetch.zh.md) 支持只会读取 URL 的网页 AI。申请、审批、
+设备范围、过期和吊销使用现有门户与 Token 存储；适配器以每份授权独立的控制端身份连接设备。
+这个主动启用的控制端组件能看到自己收发的指令和结果，普通 broker 仍然只转发加密字节。
+操作权限仍由设备决定；委托凭证不会在旧的 namespace-only 接口中被降级为账号全权凭证。
+
 ## 传输层
 
 两种载体说的是**同一套** TLS + 分帧协议：

docs/environment.md

@@ -21,6 +21,9 @@ Variables marked "conditional" are required only for the feature described.
 | `WANCTL_MCP_LOCAL_ROOT` | MCP stdio | No | process working directory | Only local tree `wanctl_push` and `wanctl_pull` may access. The wanctl config directory is always excluded. |
 | `WANCTL_MCP_ALLOWED_ORIGINS` | MCP HTTP | No | none | Comma-separated browser Origin allowlist. Requests with an Origin are denied unless listed; programmatic clients normally send none. |
 | `WANCTL_MCP_ALLOW_UNSAFE_TRUST_SERVER` | MCP | No | `0` | Set to `1` only to restore model-callable device TOFU pinning. Default is fail-closed because the model cannot distinguish an independently verified fingerprint from one supplied by a hostile relay. |
+| `WANCTL_WEBFETCH_SEED` | relay/controller adapter | No | disabled | Secret hex seed, at least 32 decoded bytes, enabling `/webfetch`; requires PostgreSQL. See [WebFetch](webfetch.md). |
+| `WANCTL_WEBFETCH_PORTAL_ORIGIN` | WebFetch | Conditional | none | Canonical HTTPS portal origin for authenticated owner approvals. |
+| `WANCTL_WEBFETCH_RELAY_URL` | WebFetch | No | `WANCTL_PUBLIC_ORIGIN` | Internal controller-to-relay origin; HTTPS or loopback HTTP. |
 | `RELAY_ADMIN_URL` | portal | Yes | none | Internal relay base URL used for the portal's admin proxy, such as `http://relay:8080`. |
 | `WANCTL_GITHUB_CLIENT_ID` | portal | Conditional | none | Enables GitHub OAuth login. Mutually exclusive with `PORTAL_USER_HEADER`. |
 | `WANCTL_GITHUB_CLIENT_SECRET` | portal | Conditional | none | OAuth App client secret; required when the client ID is set. |

docs/environment.zh.md

@@ -20,6 +20,9 @@
 | `WANCTL_MCP_LOCAL_ROOT` | MCP stdio | 否 | 进程工作目录 | `wanctl_push` 和 `wanctl_pull` 唯一可以访问的本地目录树。wanctl 配置目录永远被排除在外。 |
 | `WANCTL_MCP_ALLOWED_ORIGINS` | MCP HTTP | 否 | 无 | 逗号分隔的浏览器 Origin 白名单。带 Origin 的请求不在名单里就拒绝；程序化的客户端通常一个都不带。 |
 | `WANCTL_MCP_ALLOW_UNSAFE_TRUST_SERVER` | MCP | 否 | `0` | 只有想恢复「模型可调用的设备 TOFU 钉扎」时才设成 `1`。默认是失败即关闭，因为模型分不清一个独立验证过的指纹和一个由敌意 relay 递过来的指纹。 |
+| `WANCTL_WEBFETCH_SEED` | relay 内的控制端适配器 | 否 | 关闭 | 至少 32 字节的十六进制秘密种子，启用 `/webfetch`；必须连接 PostgreSQL。见 [WebFetch](webfetch.zh.md)。 |
+| `WANCTL_WEBFETCH_PORTAL_ORIGIN` | WebFetch | 视情况 | 无 | 设备主人审批所用的门户 HTTPS origin。 |
+| `WANCTL_WEBFETCH_RELAY_URL` | WebFetch | 否 | `WANCTL_PUBLIC_ORIGIN` | 适配器连接 relay 的 origin；仅接受 HTTPS 或回环 HTTP。 |
 | `RELAY_ADMIN_URL` | portal | 是 | 无 | 门户的管理代理所用的 relay 内网基址，比如 `http://relay:8080`。 |
 | `WANCTL_GITHUB_CLIENT_ID` | portal | 视情况 | 无 | 启用 GitHub OAuth 登录。与 `PORTAL_USER_HEADER` 互斥。 |
 | `WANCTL_GITHUB_CLIENT_SECRET` | portal | 视情况 | 无 | OAuth App 的 client secret；设了 client ID 就必需。 |

docs/releases/v0.9.0.md

@@ -0,0 +1,15 @@
+# v0.9.0
+
+Adds optional WebFetch access for URL-reading web AI clients, with owner-approved
+device-scoped delegation and existing wanctl device policy. See [WebFetch](../webfetch.md)
+for configuration, limits and compatibility, and [acceptance](../webfetch-acceptance.md)
+for the Qwen browser validation.
+
+Deploy relay and portal with database migration 009, then upgrade the controlled
+agents before using delegated sessions. Self-host Compose forwards the optional
+WebFetch seed and uses the configured portal origin and local relay endpoint.
+Existing CLI/MCP credentials retain their behavior.
+
+Rollback requires revoking every delegated token before starting a relay version
+that lacks scope-aware token resolution. The migration is additive; restore a
+database backup only as part of an explicitly planned data rollback.

docs/webfetch-acceptance.md

@@ -0,0 +1,48 @@
+# WebFetch acceptance — 2026-09-16
+
+The implementation was exercised through Qwen3.8-Max in its normal web chat UI,
+using a public HTTPS WebFetch endpoint and an isolated wanctl deployment. The
+owner portal was loopback-only with the test fixture's fixed identity; production
+GitHub OAuth was not reconfigured or used for this run. The relay used real
+PostgreSQL, and the device used wanctl mutual TLS and normal device policy.
+
+Observed sequence:
+
+1. Qwen read the entry page, followed its start URL, received `pending` and
+   returned the owner approval link. No device operation ran.
+2. The owner portal selected the single isolated device and approved 15 minutes.
+   The displayed device fingerprint matched the locally generated device identity;
+   the controller fingerprint was independently derived from the local adapter seed.
+3. Qwen's first command was refused because the controller was not paired.
+   The existing wanctl pairing page then trusted the verified controller.
+4. Qwen retried with a new request ID, and `printf wanctl-webfetch-ok` returned
+   the expected stdout and exit code 0.
+5. Qwen wrote and read a text file containing Chinese, newlines, and `& + % # ?`.
+   Both task results and the actual device file were **62 UTF-8 bytes**, SHA-256
+   `61476ad9436a5bc24c4060a5f7ae79177d700829ca2cdcf6900c757417c7acb6`.
+6. The owner revoked the delegation through the existing Access tokens page.
+   Qwen's attempt to read the previous result and submit a new command both
+   returned HTTP 403 with status `revoked`. Independent HTTP checks agreed.
+   The ledger retained four jobs (one unpaired refusal and three successes),
+   with no post-revocation job. The device log recorded only the three authorized
+   operations, each linked to its authenticated grant, credential and session.
+
+The device was never put into bypass mode. Its existing rules allowed only the
+test directory and the exact harmless command. No credentials or live browser
+tickets are included in this record.
+
+An initial browser attempt encountered a disconnected development SSH tunnel
+and correctly reported 502. After restoring the test ingress and adding reconnect
+supervision, the complete workflow above passed. That tunnel is a development
+fixture, not part of the WebFetch protocol.
+
+Automated validation includes real-PostgreSQL lifecycle/concurrency/retention
+tests, both transports and all four carrier combinations, device-scope and
+management rejection, credential-bound sessions, active revocation and expiry,
+late human approval after revocation, old-agent and upstream compatibility,
+bounded client cancellation, immutable request deduplication, and scoped audit.
+
+This establishes the observed Qwen workflow and tested enforcement paths. It is
+not a claim that every web AI can fetch arbitrary URLs or that arbitrary
+background side effects can be rolled back. See [WebFetch](webfetch.md) for
+limits, compatibility, deployment and rollback requirements.

docs/webfetch.md

@@ -0,0 +1,151 @@
+# WebFetch access
+
+WebFetch lets a web AI that can read URLs use wanctl without an MCP connector.
+It is an optional controller adapter hosted alongside the relay. The owner
+approves a short-lived delegation in the existing wanctl portal; device trust,
+rules, mode and per-request approvals still decide what runs.
+
+## Owner workflow
+
+1. Ask the AI to open `https://RELAY/webfetch`, then open its `start_url`.
+2. The AI returns an `approval_url`. Open it yourself, sign in to wanctl,
+   verify the controller and selected device identities, choose your devices
+   and a duration, and approve. Fetching this URL cannot approve a request.
+3. Ask the AI to reread its `status_url`. The approved response contains the
+   exact allowed device targets and tool parameters.
+4. On first use, the selected device may require ordinary controller pairing.
+   The AI must show that owner link, not approve it. Pairing does not change
+   the device's operation rules or enable bypass.
+5. Revoke the delegation from **Settings → Access tokens** when finished.
+
+Only owned devices with persistent IDs and recorded fingerprints can be selected
+in this initial version. Ordinary cross-account sharing is unchanged. Device
+renames do not change grants; device removal or certificate rotation invalidates
+them. A grant has device-use rights only, never console/management rights.
+
+The device's mode is authoritative: granting use of a bypass-mode device gives
+the client broad use of that device. WebFetch does not pretend that an exec
+permission can be separated from what an arbitrary shell command can do.
+
+## Operator setup
+
+Upgrade **both the relay and the controlled agents**. Older agents do not
+advertise delegated-session enforcement, so delegated dials fail closed.
+The existing portal must also be upgraded for the approval page.
+
+WebFetch is disabled unless `WANCTL_WEBFETCH_SEED` is set. Configure:
+
+| Variable | Meaning |
+| --- | --- |
+| `DATABASE_URL` | Existing wanctl PostgreSQL database; required for durable grants and request deduplication |
+| `WANCTL_WEBFETCH_SEED` | Secret hex seed, at least 32 decoded bytes; keep it in the operator's secret store |
+| `WANCTL_PUBLIC_ORIGIN` | Canonical public HTTPS relay origin used in AI-facing links |
+| `WANCTL_WEBFETCH_PORTAL_ORIGIN` | Canonical public HTTPS portal origin used in owner approval links |
+| `WANCTL_WEBFETCH_RELAY_URL` | Optional adapter-to-relay origin; defaults to the public relay origin. HTTPS or loopback HTTP only |
+
+The self-host Compose file forwards these settings: provide the seed through a
+protected environment file, and it reuses `PORTAL_PUBLIC_ORIGIN` plus the relay
+container's loopback endpoint. For example, run Compose with both
+`--env-file .env --env-file /secure/webfetch.env`. Keep using the same protected
+file on subsequent deployments so an omitted seed does not disable the adapter.
+
+Keep the seed stable while delegations are active. Domain-separated derivation
+produces a controller identity and a relay credential for each request. Browser
+pages receive a temporary browser ticket, never the reusable seed, private key,
+owner token, portal token or raw delegated relay credential. PostgreSQL stores
+credential hashes.
+
+The public `/webfetch` endpoint must be reachable by the web AI's fetch service.
+Its calls cannot depend on the owner's browser cookies. Owner approval remains
+on the authenticated portal and uses its existing CSRF protection.
+
+Exclude **both access logs and request-bearing error logs** for `/webfetch/` at
+your ingress: paths contain bearer tickets and query strings contain tool
+arguments. For example, use an ingress-specific redacted log format, or scoped
+`access_log off` and `error_log /var/log/nginx/webfetch.error.log crit` in nginx.
+Do not disable diagnostic logs globally. Application logs never record tickets
+or relay credentials. Responses use `no-store`, `no-referrer` and `noindex`;
+third-party fetch-provider retention is outside wanctl's control.
+
+The adapter is a trusted controller endpoint: it sees commands and returned
+data. Controller-to-device traffic retains wanctl's mutual TLS; this is not
+end-to-end encryption from the web model through an unreadable adapter.
+
+## GET tool protocol
+
+Default responses are static HTML with visible structured data. Add
+`format=json` for JSON. There is no JavaScript or streaming requirement.
+
+The approved manifest returns a `call_endpoint`; construct:
+
+```text
+GET CALL_ENDPOINT?rid=UNIQUE_REQUEST&tool=TOOL&target=CANONICAL_TARGET&...
+```
+
+URL-encode every parameter. The available tools are:
+
+| Tool | Parameters | Result |
+| --- | --- | --- |
+| `exec` | `command`, optional `cwd`, optional `timeout_seconds` | One-shot execution, exit code, bounded stdout/stderr |
+| `write_text` | `path`, `content` | wanctl file upload, byte count and SHA-256 |
+| `read_text` | `path` | wanctl file download, UTF-8 contents, byte count and SHA-256 |
+
+The response contains a `job_id` and `result_url`. Running jobs additionally
+return a fresh `next_url`; read that URL until `done`, `failed` or `unknown`.
+Execution is asynchronous in the adapter but uses normal synchronous, one-shot
+wanctl operations; it does not expose device-side persistent shells or detached
+async jobs to delegated clients.
+
+`rid` is scoped to the grant. Reusing it with identical parameters returns the
+same job; changing parameters returns 409. The durable ledger records the job
+before dispatch, so repeated fetches and an adapter restart never automatically
+repeat an operation. An interrupted call may have produced a side effect even
+without a result: `unknown` means the owner must inspect the device before
+deciding whether to try a new request. This is not a claim of exactly-once
+execution of arbitrary external effects.
+
+Limits: pending requests expire after 10 minutes; approved grants last 1–60
+minutes on up to 16 devices; each grant allows 64 jobs; calls allow 1–60 seconds including queue
+time (default 30); four operations run concurrently; URLs are capped at 8 KiB;
+writes at 2 KiB UTF-8; reads at 32 KiB; exec captures at most 16 KiB each of
+stdout and stderr and cancels on overflow. `HEAD` cannot create or execute tasks.
+
+Browser tickets have an immutable 70-minute envelope. Inactive grants and their
+task contents are removed after at least 24 hours; old browser URLs cannot
+recreate deleted grants. Existing account/device audit is retained separately.
+
+## Authorization and cancellation boundaries
+
+Delegated tokens are checked by the relay on discovery, canonical target
+resolution and both HTTP/WebSocket session paths. They cannot enroll devices,
+impersonate the agent side, touch another credential's session, change device
+management state, mint credentials or call ordinary account-management routes.
+
+Active connections have an exact expiry deadline and revalidate authorization
+every second (revocation propagation also includes store/network latency).
+The device rechecks the grant after a human operation approval, before executing
+or remembering a rule. A late approval cannot revive an expired/revoked grant.
+Result reads require a live grant as well.
+
+Closing a session cancels connected one-shot execution on upgraded agents. It
+does not undo completed writes or guarantee control of a deliberately detached
+background process created by an otherwise authorized command.
+
+**Before rolling a relay back to a version without this feature, revoke all
+`kind='delegated'` tokens.** Older namespace-only token resolvers do not understand
+the new constraints. New relays preserve delegated metadata through the upstream
+inspection API and refuse to downgrade the reserved `wfd_` token prefix through
+a legacy resolver.
+
+## Development acceptance
+
+Set `WANCTL_TEST_POSTGRES` to a disposable PostgreSQL instance to run the real
+grant lifecycle, migration, encrypted controller/agent and file-operation tests.
+CI provisions PostgreSQL and enables those tests by default.
+
+`go run ./tools/webfetch-demo` provides an isolated manual/browser fixture. It
+requires a private `--state-dir`, `--public-origin` and disposable PostgreSQL.
+Only expose its relay `/webfetch` routes. Its owner portal is **loopback-only**
+and intentionally supplies a fixed test identity; it is not a production login
+configuration and must never be proxied to the public Internet. The fixture's
+device uses normal policy with one test directory and one harmless command.