Skip to content

Make chat reply streaming actually reach the browser in real time - #84

Merged
obaid merged 2 commits into
mainfrom
feature/gateway-ws-streaming
Aug 24, 2026
Merged

obaid merged 2 commits into
mainfrom
feature/gateway-ws-streaming

Conversation

@obaid

@obaid obaid commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Stacked on #83. The streaming pipeline (gateway WS → provisiond relay → relay controller → Reverb → chat UI) already existed end-to-end but was silently broken in three places — found by live-probing a browser tab against the E2E droplet while instrumenting each hop:

  1. Queued delta broadcasts: ChatMessageStreamingEvent was ShouldBroadcast, so every delta waited on a Horizon worker that was busy running the chat job itself — deltas arrived seconds late or never, collapsing streaming into one final message. Now ShouldBroadcastNow, with a per-run sequence in the payload and a stale-batch guard in the chat page (synchronous publishes can still reorder adjacent HTTP calls).
  2. provisiond crash-on-restart: the relay closed its socket with reserved close codes (1001/1009/1012/1013) that WebSocket clients may not send — undici throws InvalidAccessError, killing the daemon (observed 4 crash loops in one afternoon, each triggered by a gateway restart). Client-legal 4xxx codes now; bundle v0.4.1, 37 tests green.
  3. SSR port squatting (found during testing, drive-by fix): config/inertia.php was hardcoded to the machine-shared default SSR port — a stray SSR server from another local repo rendered its pages into this app's responses. Now env-driven.

Live verification on the E2E droplet: a fresh browser tab received 13 streaming deltas with cumulative text growing in real time across a ~6s generation (monotonic sequences, final event closing the stream). Note for reviewers: first-token latency (~10-15s) is still dominated by the send path (queue pickup + SSH exec chain) — that's the next optimization target, not addressed here.

obaid added 2 commits August 24, 2026 12:49
Every change here was verified against the shipped openclaw@2026.7.1 npm
package (not the online docs, which describe a newer release) and
exercised end-to-end on a freshly provisioned DigitalOcean server.

Auth (runtime reads SQLite only — auth-profiles.json is dead upstream):
- Replace every auth-profiles.json write with per-agent
  `openclaw models --agent X auth paste-api-key` injection (env job,
  install script, update script); the legacy files were never read on
  2026.7.x and corrupted auth-presence heuristics
- ChatGPT OAuth polling/disconnect moved from jq-on-file to
  `models auth list/logout --json` (pairing detection was broken)
- Subscription guards: never inject an api_key profile over a ChatGPT
  (openai) or Claude (anthropic) subscription agent's slot

Claude subscription (BYO) support:
- ClaudeAuthService (setup-token over piped stdin, order pinning,
  sanitized errors), controller + routes, migration, connect UI, tier
  wiring, disconnect hooks on model change / switch-to-pay-per-use

Model routing:
- BYO Anthropic/OpenAI keys now route direct (anthropic/, openai/)
  instead of dead-ending at OpenRouter without a credential; memoized
  per-team key lookups; prompt-caching keys fixed for both route forms

Config management:
- ConfigPatchService agent add/update + Slack ops now go through the
  gateway config RPC (config.get hash -> config.patch with baseHash,
  3 attempts, file-edit fallback) — closes the clobber race and a
  Slack-token shell-injection hole; removal reordered RPC-before-file

Fleet updates:
- UpdateOpenClawVersionJob + hourly reconciliation via
  `openclaw update --tag --yes` (no fleet mechanism existed); fixed
  version parses that reinstalled OpenClaw on every agent update
  (output is "OpenClaw 2026.7.1 (hash)" — positional parses broke)

Retirements (verified safe on 2026.7.1):
- OperatorPairingPatch: loopback CLI + gateway token bypasses device
  pairing entirely since #95997 — our only access path
- GatewayClient's OpenClaw arm (both call sites are Hermes-gated);
  dead AgentBrowserProvider contract; _legacyUpdateAgent

Browser:
- Profiles switched from driver=existing-session to raw CDP attach-only
  (no npx chrome-devtools-mcp subprocess; regains per-tab WS, element
  screenshots, downloads; attachOnly is honored on this path)

Provisioning hardening:
- Installer runs with OPENCLAW_NO_PROMPT/NO_ONBOARD and fails loudly
- ProvisionDigitalOceanServerJob is retry-safe (reuses a surviving
  volume by name instead of 409ing — found live during a DO outage)
The full streaming pipeline (gateway WS -> provisiond relay -> relay
controller -> Reverb -> chat UI) already existed but was broken in three
places, verified live on a provisioned DO server:

- ChatMessageStreamingEvent implemented queued ShouldBroadcast: every
  delta broadcast waited on a Horizon worker behind the very chat job
  producing it, so deltas arrived seconds late or drowned entirely and
  the UI showed nothing until the finished reply. Now ShouldBroadcastNow
  (deltas are ephemeral and relay-batched at ~300ms), with a monotonic
  sequence in the payload and a stale-batch guard in the chat page.

- provisiond closed its gateway socket with reserved WebSocket close
  codes (1001/1009/1012/1013) that a client may not send — undici threw
  InvalidAccessError and crashed the entire daemon on every gateway
  restart (4 crash loops observed in one afternoon). Client-legal 4xxx
  codes now; bundle rebuilt as v0.4.1.

- config/inertia.php SSR settings are now env-driven: the default SSR
  port 13714 is shared machine-wide, and a stray SSR server from another
  repo silently rendered its pages into this app's responses.

Verified in a live browser: 13 delta events with cumulative text growing
in real time over a ~6s generation, ordered sequences, final event
closing the stream.
@obaid
obaid changed the base branch from feature/openclaw-upstream-alignment to main August 24, 2026 20:50
@obaid
obaid merged commit 2c35355 into main Aug 24, 2026
obaid added a commit that referenced this pull request Aug 31, 2026
…efaults

The completion/error events are what re-enable the chat composer — riding
the queued broadcasts lane left seconds of dead air after the final
streamed token (deltas already broadcast inline since #84). All chat
conversation events now ShouldBroadcastNow.

sqlite connections default to WAL journal mode with a 5s busy timeout so
concurrent Horizon workers and web requests wait for the write lock
instead of throwing 'database is locked' (local dev + Docker; prod MySQL
unaffected). Both env-overridable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant